Commit Graph

75 Commits

Author SHA1 Message Date
David Karlsson
b08ff81204 builder: fall back to defaultKeepStorage if keepStorage is unset for GC policy
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-12-11 11:02:49 +01:00
Rob Murray
af0b973595 Allow IPv4 and IPv6 host-gateway-ip addresses
Running a container with "--add-host blah:host-gateway" adds an /etc/hosts
entry for host "blah" and an address on the docker host - to give the
container a convenient way of reaching the host.

If no --host-gateway-ip option is supplied, the IPv4 address of the
default bridge is used - and that's been fine until now, it's a host
address we know will exist. But, in a container that's only connected
to IPv6-only networks, that doesn't work.

So:
- if the default bridge has an IPv6 address, create an additional
  /etc/hosts entry with that adddress
- allow two --host-gateway-ip options
  - at most one IPv4 and one IPv6 address
- in daemon.json, allow a JSON array value in --host-gateway-ips (plural)
  - for a single address, a JSON string is also allowed

For example:
  --host-gateway-ip 192.0.2.1 --host-gateway-ip 2001:db8::1111
And the daemon.json version would be:
  "host-gateway-ips": ["192.0.2.1", "2001:db8::1111"]
But, this is also still valid:
  "host-gateway-ip": "192.0.2.1"

Note that the /etc/hosts entries follow the usual rules. If IPv6 is
disabled in a container (by sysctl, or lack of kernel support), IPv6
addresses are not included in the file. In other cases, IPv4 and IPv6
addresses will both be included, whether or not the container currently
has network endpoints that support IPv4 or IPv6.

buildx has its own code to interpret the host-gateway-ip option. When
it's updated to understand two addresses, moby will need to pass it
both. For now, it passes an IPv4 address if there is one, else IPv6.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-11-26 11:47:31 +00:00
Tonis Tiigi
48b237f7af vendor: update buildkit to v0.18.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-11-22 13:59:09 +01:00
Paweł Gronowski
c0045476b8 all: Remove redundant units alias for go-units
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-07 15:55:16 +01:00
Sebastiaan van Stijn
d828b032a8 Merge pull request #48565 from crazy-max/build-split-history-db
build: create distinct history db for each store
2024-10-17 22:49:05 +02:00
Sebastiaan van Stijn
7fe04c142d vendor: github.com/moby/buildkit v0.17.0-rc1
full diff:

- https://github.com/moby/buildkit/compare/v0.16.0...v0.17.0-rc1
- 91a3fc4684...3f140a1299
- 9b4947da39...701f63a606

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-11 18:37:23 +02:00
CrazyMax
58f4e916a3 build: create distinct history db for each store
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-10-08 14:49:09 +02:00
Sebastiaan van Stijn
f39409f80d Merge pull request #48078 from vvoland/c8d-buildkit-tag-event
c8d/build: Log `image tag` event when image was built with Buildkit
2024-07-18 21:46:49 +02:00
Tonis Tiigi
1787c364e0 vendor: update buildkit to v0.15.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-07-08 20:35:36 -07:00
Sebastiaan van Stijn
d0aa3eaccf Migrate to github.com/containerd/platforms module
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.

The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-02 21:20:29 +02:00
Paweł Gronowski
1506bbcfe8 builder-next: Add ImageNamedByBuildkit callback
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.

Add a callback that is called by the exporter wrapper.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-02 12:34:52 +02:00
Paweł Gronowski
09eb6ec4f1 builder/exporter: Wrap exporter to hook the image export
Buildkit doesn't call the engine API when it builds an image without
tag. Wrap the exporter returned by the worker that calls a callback when
a new image is exported from buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:11 +02:00
Paweł Gronowski
0b5f7b9ff8 builder-next: Adjust NewGatewayFrontend invocation
b5c50afa882e2b34aba880fd5028615e2ef94e07 changed the signature of
NewGatewayFrontend to include a slice of allowed repositories.

Docker does not allow to specify this option, so don't place any
restrictions by passing an empty slice.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:18 +02:00
Jonathan A. Sternberg
fa03db1b82 builder: Update detect usage for new detect API from buildkit
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:17 +02:00
Paweł Gronowski
f5722da5e0 mobyexporter: Store temporary config descriptor
Temporarily store the produced config descriptor for the buildkit
history to work.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:09 +01:00
Paweł Gronowski
b04a2dad6b builder/controller: Adjust NewWorkerOpt call
8bfd280ab7
added a new argument that allows to specify different runtime.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:15 +01:00
Paweł Gronowski
bc6d88c09a cmd/dockerd: Fix overriding OTEL resource
e358792815
changed that field to a function and added an `OverrideResource`
function that allows to override it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:14 +01:00
Paweł Gronowski
f4a93b6993 vendor: github.com/moby/buildkit v0.12.5
full diff: https://github.com/moby/buildkit/compare/v0.12.4...v0.12.5

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-31 21:24:50 +01:00
Paul "TBBle" Hampson
ec041193f9 Windows doesn't have 'host' mode networking, so use 'auto'.
That means 'null', not that we can call builder-next on Windows. If and
when we do get builder-next going, this will need to be solved properly
in some way.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:30 +01:00
Tonis Tiigi
a99bb24ea8 builder-next: make sure worker platforms normalized for containerd
These platforms are filled by default from containerd
introspection API and may not be normalized. Initializing
wrong platform in here results in incorrect platform
for BUILDPLATFORM and TARGETPLATFORM build-args for
Dockerfile frontend (and probably other side effects).

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-10-30 23:14:34 -07:00
Sebastiaan van Stijn
919fe72f60 vendor: github.com/moby/buildkit v0.12.3-dev
update buildkit to the latest code in the v0.12 branch:

full diff: f94ed7cec3...6560bb937e

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-10-13 17:05:56 -07:00
Sebastiaan van Stijn
cff4f20c44 migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Bjorn Neergaard
2246297ae6 builder-next: adopt new wrapped types
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:

* containerd's LeaseManager and ContentStore have to be a
  (namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
  bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:48 -06:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
e9b09b49fd builder/builder-next: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:50 +02:00
Sebastiaan van Stijn
2f61620339 builder/builder-next: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:26:51 +02:00
Sebastiaan van Stijn
9d4abbbc30 builder/builder-next: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
CrazyMax
21e50b89c9 builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-21 13:21:38 +02:00
Kevin Alvarez
6d139e5e95 build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-18 21:17:29 +02:00
Djordje Lukic
a1e1038736 Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 15:10:01 +02:00
Tonis Tiigi
0ac3bf837b builder-next: disable mergeop and diffop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-16 08:23:22 -06:00
Tonis Tiigi
f8b468fda2 builder-next: enable more cache backends
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-07 10:32:40 +01:00
Djordje Lukic
bba77163ff c8d: Make build and buildx work
- Only use the image exporter in build if we don't use containerd
  Without this "docker build" fails with:

    Error response from daemon: exporter "image" could not be found

- let buildx know we support containerd snapshotter
- Pass the current snapshotter to the buildkit worker

  If buildkit uses a different snapshotter we can't list the images any
  more because we can't find the snapshot.

builder/builder-next: make ContainerdWorker a minimal wrapper

Note that this makes "Worker" a public field, so technically one could
overwrite it.

builder-next: reenable runc executor

Currently, without special CNI config the builder would
only create host network containers that is a security issue.

Using runc directly instead of shim is faster as well
as builder doesn’t need anything from shim. The overhead
of setting up network sandbox is much slower of course.

builder/builder-next: simplify options handling

Trying to simplify the logic;

- Use an early return if multiple outputs are provided
- Only construct the list of tags if we're using an image (or moby) exporter
- Combine some logic for snapshotter and non-snapshotter handling

Create a constant for the moby exporter

Pass a context when creating a router

The context has a 10 seconds timeout which should be more than enough to
get the answer from containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-07 10:01:47 +01:00
Tonis Tiigi
666334bd48 vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-02-01 23:29:10 -08:00
CrazyMax
a2aaf4cc83 vendor buildkit v0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-22 18:51:27 +01:00
Tonis Tiigi
5c01d06f72 builder: fix blobs releasing via leases after pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-12 20:27:16 -08:00
Brian Goff
94c07441c2 buildkit: Apply apparmor profile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 611eb6ffb3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-02 13:32:24 +01:00
Tibor Vass
de56a90929 vendor buildkit 6861f17f15364de0fe1fd1e6e8da07598a485123
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-14 03:57:36 +00:00
Tonis Tiigi
9b28939345 vendor: update buildkit to df35e9818
Update to new buildkit and fix upgrade bugs

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-15 09:44:41 -07:00
Tonis Tiigi
0cdf6ba9c8 vendor: update buildkit to ae7ff174
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-14 08:26:07 -07:00
Akihiro Suda
9a82a9a8ea vendor containerd, BuildKit, protobuf, grpc, and golang.org/x
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 10:25:20 +09:00
Tonis Tiigi
21dfcc730b builder-next: clear temp leases on startup
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-05 12:50:41 -08:00
Tonis Tiigi
f14c9d4df5 builder-next: track layers and graphdrivers with leases
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-05 10:14:33 -08:00
Tonis Tiigi
fe16d95dcd builder-next: update adapters to new buildkit interfaces
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-05 10:14:33 -08:00
Sebastiaan van Stijn
4addf7ab1c Merge pull request #39979 from tiborvass/fix-buildkit-prunegc-filter-config
daemon/config: fix filter type in BuildKit GC config
2019-10-18 22:44:48 +02:00
Derek McGowan
bc5484d2dd bump moby/buildkit f7042823e340d38d1746aa675b83d1aca431cee3
full diff: 588c73e1e4...f7042823e3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

fix daemon for changes in containerd registry configuration

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

Update buildernext and daemon for buildkit update

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-04 15:05:35 -07:00
Tibor Vass
fbdd437d29 daemon/config: fix filter type in BuildKit GC config
For backwards compatibility, the old incorrect object format for
builder.GC.Rule.Filter still works but is deprecated in favor of array of
strings akin to what needs to be passed on the CLI.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-10-03 00:09:58 +00:00
Kunal Kushwaha
8b7bbf180f builder entitlements configutation added.
buildkit supports entitlements like network-host and security-insecure.
this patch aims to make it configurable through daemon.json file.
by default network-host is enabled & secuirty-insecure is disabled.

Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
2019-09-26 16:59:44 +09:00
Tibor Vass
a1cdd4bfcc build: buildkit now honors daemon's DNS config
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-07-10 00:26:03 +00:00
Tonis Tiigi
07b3aac902 builder-next: userns remap support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-10 21:49:17 -07:00