Commit Graph

295 Commits

Author SHA1 Message Date
Derek McGowan
0aa8fe0bf9 Update to containerd v2.0.2, buildkit v0.19.0-rc2
Update buildkit version to commit which uses 2.0

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-15 14:09:30 +01:00
Djordje Lukic
8d5cf1db1e Use the roundtripper during build
The roundtripper is responsible for giving back the build context when
it comes from a tar directly. So we add it to the source manager of the
containerd worker.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-12-30 23:39:52 +01:00
Paweł Gronowski
ad6929339a Merge pull request #49147 from thaJeztah/fix_gc_default
builder: don't fall back to defaultKeepStorage when set to zero
2024-12-20 12:12:30 +00:00
Sebastiaan van Stijn
275bbcd300 builder: don't fall back to defaultKeepStorage when set to zero
commit b08ff81204 updated this code to
only produce an error if an invalid value was set by the user, and to
avoid errors on empty values.

However, the intent of this code was to allow `0` as a valid value for
cases where gc is to be handled through other properties / filters.

This patch only sets the default if no value was set by the user, but
doesn't set the default if a value is set by the user, but zero.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 21:23:09 +01:00
Paweł Gronowski
90fef061ec daemon/c8d: Force c8dimages alias for containerd/images
Change all github.com/containerd/containerd/images imports to be
imported as `c8dimages`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-19 18:39:43 +01:00
David Karlsson
b08ff81204 builder: fall back to defaultKeepStorage if keepStorage is unset for GC policy
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-12-11 11:02:49 +01:00
Rob Murray
af0b973595 Allow IPv4 and IPv6 host-gateway-ip addresses
Running a container with "--add-host blah:host-gateway" adds an /etc/hosts
entry for host "blah" and an address on the docker host - to give the
container a convenient way of reaching the host.

If no --host-gateway-ip option is supplied, the IPv4 address of the
default bridge is used - and that's been fine until now, it's a host
address we know will exist. But, in a container that's only connected
to IPv6-only networks, that doesn't work.

So:
- if the default bridge has an IPv6 address, create an additional
  /etc/hosts entry with that adddress
- allow two --host-gateway-ip options
  - at most one IPv4 and one IPv6 address
- in daemon.json, allow a JSON array value in --host-gateway-ips (plural)
  - for a single address, a JSON string is also allowed

For example:
  --host-gateway-ip 192.0.2.1 --host-gateway-ip 2001:db8::1111
And the daemon.json version would be:
  "host-gateway-ips": ["192.0.2.1", "2001:db8::1111"]
But, this is also still valid:
  "host-gateway-ip": "192.0.2.1"

Note that the /etc/hosts entries follow the usual rules. If IPv6 is
disabled in a container (by sysctl, or lack of kernel support), IPv6
addresses are not included in the file. In other cases, IPv4 and IPv6
addresses will both be included, whether or not the container currently
has network endpoints that support IPv4 or IPv6.

buildx has its own code to interpret the host-gateway-ip option. When
it's updated to understand two addresses, moby will need to pass it
both. For now, it passes an IPv4 address if there is one, else IPv6.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-11-26 11:47:31 +00:00
Tonis Tiigi
48b237f7af vendor: update buildkit to v0.18.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-11-22 13:59:09 +01:00
Sebastiaan van Stijn
b453aa65fa update go:build tags to use go1.22
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.

go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-12 14:02:09 +01:00
Sebastiaan van Stijn
222b3a9e5c Merge pull request #48834 from vvoland/gounits-alias
all: Remove redundant `units` alias for `go-units`
2024-11-07 19:37:19 +01:00
Paweł Gronowski
c0045476b8 all: Remove redundant units alias for go-units
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-07 15:55:16 +01:00
Sebastiaan van Stijn
6dc31171c5 builder/builder-next/exporter: patchImageConfig: fix shadowed variable (govet)
builder/builder-next/exporter/mobyexporter/writer.go:83:3: shadow: declaration of "dt" shadows declaration at line 42 (govet)
            dt, err := json.Marshal(cache.Data)
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-06 12:54:51 +01:00
Sebastiaan van Stijn
ee425a9773 builder/builder-next: fix "id" variable being shadowed (govet)
builder/builder-next/builder.go:435:3: shadow: declaration of "id" shadows declaration at line 294 (govet)
            id, ok := resp.ExporterResponse["containerimage.digest"]
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-06 12:54:50 +01:00
Sebastiaan van Stijn
741cc494cd builder-next: exporter: emptyImageConfig: use platform directly
The OCI image spec now has the platform as field, so we can just
set it directly now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-30 01:15:12 +01:00
Sebastiaan van Stijn
5c48736863 remove redundant alias for runtime-spec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 18:31:39 +02:00
Sebastiaan van Stijn
d828b032a8 Merge pull request #48565 from crazy-max/build-split-history-db
build: create distinct history db for each store
2024-10-17 22:49:05 +02:00
Sebastiaan van Stijn
805e2c67e2 builder/builder-next: Builder.Build: use network-mode consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 16:02:39 +02:00
Sebastiaan van Stijn
7fe04c142d vendor: github.com/moby/buildkit v0.17.0-rc1
full diff:

- https://github.com/moby/buildkit/compare/v0.16.0...v0.17.0-rc1
- 91a3fc4684...3f140a1299
- 9b4947da39...701f63a606

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-11 18:37:23 +02:00
CrazyMax
58f4e916a3 build: create distinct history db for each store
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-10-08 14:49:09 +02:00
Tonis Tiigi
6f37e5a168 vendor: update buildkit to v0.16.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-09-09 16:36:22 -07:00
Tonis Tiigi
c459986399 vendor: update buildkit to v0.15.2
Also brings in fix for moby/buildkit#5242

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-08-15 18:14:56 +03:00
Sebastiaan van Stijn
f39409f80d Merge pull request #48078 from vvoland/c8d-buildkit-tag-event
c8d/build: Log `image tag` event when image was built with Buildkit
2024-07-18 21:46:49 +02:00
Tonis Tiigi
1787c364e0 vendor: update buildkit to v0.15.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-07-08 20:35:36 -07:00
Sebastiaan van Stijn
d0aa3eaccf Migrate to github.com/containerd/platforms module
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.

The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-02 21:20:29 +02:00
Paweł Gronowski
1506bbcfe8 builder-next: Add ImageNamedByBuildkit callback
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.

Add a callback that is called by the exporter wrapper.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-02 12:34:52 +02:00
Paweł Gronowski
bce76d486e builder-next: Don't return error from exported callback
This is only a callback that notifies about event so there is no way to
react to the error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-02 12:34:51 +02:00
Sebastiaan van Stijn
bbced07f96 builder/builder-next: applySourcePolicies: remove redundant check and vars
- the check for `ok` was redundant as the line above it would return early
- the `t` and `ok` variable declaration was redundant, as all variables to
  the left of the `strings.Cut` were locally scoped variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 13:15:17 +02:00
Sebastiaan van Stijn
517fb0991e api/types/container: provide alias for github.com/docker/go-units.Ulimit
This type is included in various types used in the API, but comes from
a separate module. The go-units module may be moving to the moby org,
and it is yet to be decided if the Ulimit type is a good fit for that
module (which deals with more generic units, such as "size" and "duration"
otherwise).

This patch introduces an alias to help during the transition of this type
to it's new location. The alias makes sure that existing code continues
to work (at least for now), but we need to start updating such code after
this PR is merged.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 13:18:20 +02:00
Paweł Gronowski
09eb6ec4f1 builder/exporter: Wrap exporter to hook the image export
Buildkit doesn't call the engine API when it builds an image without
tag. Wrap the exporter returned by the worker that calls a callback when
a new image is exported from buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:11 +02:00
Paweł Gronowski
0e84482ef5 builder-next: Move exporter wrapper to exporter package
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:09 +02:00
Paweł Gronowski
34c3569768 Merge pull request #47985 from vvoland/bk-patchimageconfig-panic
builder/mobyexporter: Add missing nil check
2024-06-14 17:05:21 +02:00
Albin Kerouanton
1882da852e Merge pull request #47906 from akerouanton/libnet-add-otel-spans-v3
api, daemon, libnet: Create OTel spans at various places
2024-06-14 17:03:56 +02:00
Paweł Gronowski
642242a26b builder/mobyexporter: Add missing nil check
Add a nil check to handle a case where the image config JSON would
deserialize into a nil map.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-14 15:25:04 +02:00
Sebastiaan van Stijn
e4e40558ba builder/builder-next: gofmt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 21:50:12 +02:00
Albin Kerouanton
cec0d50361 libnet: add ctx to Sandbox.Destroy()
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 17:13:43 +02:00
Albin Kerouanton
9391052700 libnet: Add ctx to NewSandbox
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:52:48 +02:00
Albin Kerouanton
c5c1d133ef libnet/driverapi: Add ctx to Join
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:45:54 +02:00
Albin Kerouanton
8dcded102e libnet: add OTel spans to CreateEndpoint
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:45:31 +02:00
Sebastiaan van Stijn
b7d5a42168 Update go:build comments to go1.21
Match the minimum version that's specified on our vendor.mod.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 14:59:54 +02:00
Sebastiaan van Stijn
5343c7b451 remove internal/compatcontext and use context instead
This internal package was added in f6e44bc0e8
to preserve compatibility with go1.20 and older. At the time, our vendor.mod
still had go1.18 as minimum version requirement (see [1]), which got updated to go1.20
in 16063c7456, and go1.21 in f90b03ee5d

The version of BuildKit we use already started using context.WithoutCancel,
without a fallback, so we no longer can provide compatibility with older
versions of Go, which makes our compatiblity package redundant.

This patch removes the package, and updates our code to use stdlib's context
instead.

[1]: f6e44bc0e8/vendor.mod (L7)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 13:29:39 +02:00
Tonis Tiigi
18ff5ef537 vendor: update buildkit to v0.14.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-06-11 12:08:34 -07:00
Paweł Gronowski
0b5f7b9ff8 builder-next: Adjust NewGatewayFrontend invocation
b5c50afa882e2b34aba880fd5028615e2ef94e07 changed the signature of
NewGatewayFrontend to include a slice of allowed repositories.

Docker does not allow to specify this option, so don't place any
restrictions by passing an empty slice.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:18 +02:00
Jonathan A. Sternberg
fa03db1b82 builder: Update detect usage for new detect API from buildkit
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:17 +02:00
Sebastiaan van Stijn
86f7762d48 vendor: github.com/containerd/containerd v1.7.18
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.

This patch:

- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to prevent using the "log" package, which was also
  migrated to a separate module.

There are still some uses of the old package in (indirect) dependencies,
which should go away over time.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-05 09:21:00 +02:00
Sebastiaan van Stijn
cd1ed46d73 don't depend on containerd platform.Parse to return a typed error
We currently depend on the containerd platform-parsing to return typed
errdefs errors; the new containerd platforms module does not return such
errors, and documents that errors returned should not be used as sentinel
errors; c1438e911a/errors.go (L21-L30)

Let's type these errors ourselves, so that we don't depend on the error-types
returned by containerd, and consider that eny platform string that results in
an error is an invalid parameter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 10:18:29 +02:00
Sebastiaan van Stijn
ff05850e7e Merge pull request #47563 from vvoland/buildkit-runc-override
builder-next: Add env-var to override runc used by buildkit
2024-03-14 20:17:01 +01:00
Paweł Gronowski
10bdc7136c builder-next: Add env-var to override runc used by buildkit
Adds an experimental `DOCKER_BUILDKIT_RUNC_COMMAND` variable that allows
to specify different runc-compatible binary to be used by the buildkit's
runc executor.

This allows runtimes like sysbox be used for the containers spawned by
buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-14 17:35:26 +01:00
Tonis Tiigi
37545cc644 builder-next: fix missing lock in ensurelayer
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.

In that case the reference count got mixed up and on
release only one of these layers was actually released.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-03-06 23:11:32 -08:00
Paweł Gronowski
49b77753cb builder-next/export: Use leaseutil for descref lease
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 17:47:37 +01:00
Paweł Gronowski
c4fc6c3371 builder-next/executor: Replace removed network.Sample
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 12:02:12 +01:00