Commit Graph

51401 Commits

Author SHA1 Message Date
Brian Goff
2dc03f934f cmd/dockerd: refactor buildkit init in daemon startup
- Move the "Daemon has completed initialization" log to where it has
  actually completed initialization.
- Move buildkit init to its own function.
- Move the builder options to a separate struct, and change initBuildkit
  to return it instead of passing the router-options and manipulate it.

Co-authored-by: Brian Goff <cpuguy83@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-06 13:53:29 +01:00
Paweł Gronowski
02563f2805 Merge pull request #48798 from LaurentGoderre/implement-30449-image-mount
Implement mount from image
2025-02-05 18:04:22 +00:00
Sebastiaan van Stijn
779bdc4fab Merge pull request #49393 from vvoland/update-go
update to go1.23.6
2025-02-05 16:43:24 +01:00
Rob Murray
997a6e167e Merge pull request #49372 from robmry/remote_driver_skip_gw_alloc
Implement GwAllocChecker for the remote network driver
2025-02-05 11:30:33 +00:00
Paweł Gronowski
62a747f5e4 Merge pull request #49390 from thaJeztah/implement_unwrapper
implement missing "Unwrapper" interface on errors that implemented "Causer" interface
2025-02-05 09:49:26 +00:00
Paweł Gronowski
6ee81dc3eb Merge pull request #49262 from robmry/gw_mode_isolated
Add gateway mode "isolated"
2025-02-05 09:29:20 +00:00
Laurent Goderre
1126d477fd Add live-restore tests for mount image
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
2025-02-04 21:32:04 -05:00
Laurent Goderre
90aea3b85f Add image subpath mounting functionality
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
2025-02-04 21:32:04 -05:00
Sebastiaan van Stijn
63ea5dc10e Merge pull request #49388 from thaJeztah/client_unify_interfaces
client: deprecate CommonAPIClient interface, add HijackDialer, SwarmManagementAPIClient interfaces
2025-02-04 22:50:24 +01:00
Sebastiaan van Stijn
2277b35061 Merge pull request #49389 from thaJeztah/client_deprecate_ErrorConnectionFailed
client: deprecate ErrorConnectionFailed helper
2025-02-04 22:48:32 +01:00
Paweł Gronowski
126d4bc65b Merge pull request #49391 from thaJeztah/errdefs_godoc
errdefs: touch-up godoc for helpers
2025-02-04 19:41:15 +00:00
Rob Murray
634951457d Implement GwAllocChecker for the remote network driver
Commit 38e76eb (Ask network drivers if they'll use a gateway
address) added an optional driver interface "GwAllocChecker"
to give the driver a chance to say whether, given network
config options, it would use a gateway address if one was
reserved for it in IPAM.

So, implement support for that in the remote network driver.
The driver itself implements the interface, but only tries
to make an HTTP request to the driver plugin if the plugin
has reported support for it in response to an initial
capabilities request.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-02-04 19:35:20 +00:00
Paweł Gronowski
12756071f1 update to go1.23.6
This minor release include 1 security fix following the security policy:

- crypto/elliptic: timing sidechannel for P-256 on ppc64le

  Due to the usage of a variable time instruction in the assembly implementation
  of an internal function, a small number of bits of secret scalars are leaked on
  the ppc64le architecture. Due to the way this function is used, we do not
  believe this leakage is enough to allow recovery of the private key when P-256
  is used in any well known protocols.

This is CVE-2025-22866 and Go issue https://go.dev/issue/71383.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.23.6

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-04 20:27:27 +01:00
Sebastiaan van Stijn
294df1c447 volumes/service: OpErr: implement go1.13 unwrapper
This error implemented the Causer interface, but did not implement
the go1.13 unwrapper, which could prevent errors from being matched.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 18:10:51 +01:00
Sebastiaan van Stijn
db68c99d4b distribution: fallbackError, notFoundError implement go1.13 unwrapper
These errors implemented the Causer interface, but did not implement
the go1.13 unwrapper, which could prevent errors from being matched.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 18:08:35 +01:00
Sebastiaan van Stijn
b96b14d078 cluster/executor/container: exitError: implement go1.13 unwrapper
This error implemented the Causer interface, but did not implement
the go1.13 unwrapper, which could prevent errors from being matched.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 17:47:10 +01:00
Sebastiaan van Stijn
2997c0ddc0 client: deprecate CommonAPIClient interface
The CommonAPIClient was used to define all the stable interfaces,
and combined with the experimental ones through APIClient. In theory,
this would allow someone to make sure they only depended on non-experimental
methods or to implement an alternative client that only implements the
stable methods.

While there are users currently using this interface, all those uses
depend on the actual client implementation, not a custom one, so they
should be able to switch to use APIClient instead. In the meantime,
start with deprecating, but keeping the interface the same for now,
scheduling it to become an alias, and removed in a future release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 17:21:31 +01:00
Sebastiaan van Stijn
846b2272e6 client: deprecate ErrorConnectionFailed helper
This function was only used internally, and will be removed in the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 17:16:52 +01:00
Sebastiaan van Stijn
e93ff742e8 client: doRequest: use errors.As for error-detection
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 17:09:59 +01:00
Sebastiaan van Stijn
3725998e7d client: define interface for all Swarm-specific methods
Introduce a SwarmManagementAPIClient interface that captures
all swarm-specific methods on the API client.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 16:56:08 +01:00
Sebastiaan van Stijn
a57d737a86 client: define separate interface for HijackDialer
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 16:44:12 +01:00
Sebastiaan van Stijn
902c06fdf0 client: make setupHijackConn a regular function
pass the dialer as argument

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 16:39:44 +01:00
Sebastiaan van Stijn
e6dabfa3b1 client: move resetting mediaType for hijack to where applicable
The mediaType is only used in a single location; reset it in
that location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 16:39:44 +01:00
Sebastiaan van Stijn
b0e206b807 client: separate Dialer() implementation from public API
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 16:39:44 +01:00
Sebastiaan van Stijn
fcb924712b integration/internal: JobComplete: require shallower interface
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 16:12:06 +01:00
Sebastiaan van Stijn
0f04532956 errdefs: touch-up godoc for helpers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-04 15:50:07 +01:00
Sebastiaan van Stijn
e7da72a464 Merge pull request #49381 from thaJeztah/improve_validation
client: normalize and validate empty ID / name arguments to fail early
2025-02-04 15:49:23 +01:00
Paweł Gronowski
250378a715 Merge pull request #49362 from vvoland/pr-changelog-desc
github: Clarify release notes description
2025-02-04 09:16:13 +00:00
Paweł Gronowski
72b835151a Merge pull request #48720 from jsternberg/buildkit-gc-options
builder: wire up new gc types for buildkit prune functionality
2025-02-03 16:55:46 +00:00
Sebastiaan van Stijn
c8f1a41965 Merge pull request #49361 from vvoland/pr-validate-sync
gha/validate-pr: Also run when PR has new commits
2025-02-03 17:11:56 +01:00
Sebastiaan van Stijn
329b2a26f3 client: normalize and validate empty ID / name arguments to fail early
In situations where an empty ID was passed, the client would construct an
invalid API endpoint URL, which either resulted in the "not found" handler
being hit (resulting in a "page not found" error), or even the wrong endpoint
being hit if the client follows redirects.

For example, `/containers/<empty id>/json` (inspect) redirects to `/containers/json`
(docker ps))

Given that empty IDs should never be expected (especially if they're part of
the API URL path), we can validate these and return early.

Its worth noting that a few methods already had an error in place; those
methods were related to the situation mentioned above, where (e.g.) an
"inspect" would redirect to a "list" endpoint. The existing errors, for
convenience, mimicked a "not found" error; this patch changes such errors
to an "Invalid Parameter" instead, which is more correct, but it could be
a breaking change for some edge cases where users parsed the output;

    git grep 'objectNotFoundError{'
    client/config_inspect.go:        return swarm.Config{}, nil, objectNotFoundError{object: "config", id: id}
    client/container_inspect.go:     return container.InspectResponse{}, nil, objectNotFoundError{object: "container", id: containerID}
    client/container_inspect.go:     return container.InspectResponse{}, objectNotFoundError{object: "container", id: containerID}
    client/distribution_inspect.go:  return distributionInspect, objectNotFoundError{object: "distribution", id: imageRef}
    client/image_inspect.go:         return image.InspectResponse{}, nil, objectNotFoundError{object: "image", id: imageID}
    client/network_inspect.go:       return network.Inspect{}, nil, objectNotFoundError{object: "network", id: networkID}
    client/node_inspect.go:          return swarm.Node{}, nil, objectNotFoundError{object: "node", id: nodeID}
    client/plugin_inspect.go:        return nil, nil, objectNotFoundError{object: "plugin", id: name}
    client/secret_inspect.go:        return swarm.Secret{}, nil, objectNotFoundError{object: "secret", id: id}
    client/service_inspect.go:       return swarm.Service{}, nil, objectNotFoundError{object: "service", id: serviceID}
    client/task_inspect.go:          return swarm.Task{}, nil, objectNotFoundError{object: "task", id: taskID}
    client/volume_inspect.go:        return volume.Volume{}, nil, objectNotFoundError{object: "volume", id: volumeID}

Two such errors are still left, as "ID or name" would probably be confusing,
but perhaps we can use a more generic error to include those as well (e.g.
"invalid <object> reference: value is empty");

    client/distribution_inspect.go:  return distributionInspect, objectNotFoundError{object: "distribution", id: imageRef}
    client/image_inspect.go:         return image.InspectResponse{}, nil, objectNotFoundError{object: "image", id: imageID}

Before this patch:

    docker container start ""
    Error response from daemon: page not found
    Error: failed to start containers:

    docker container start " "
    Error response from daemon: No such container:
    Error: failed to start containers:

With this patch:

    docker container start ""
    invalid container name or ID: value is empty
    Error: failed to start containers:

    docker container start " "
    invalid container name or ID: value is empty
    Error: failed to start containers:

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-03 11:21:51 +01:00
Sebastiaan van Stijn
96ded2a1ba Merge pull request #48850 from JSchltggr/fix-sed-nix-dind
Fix concurrency issue in dind
2025-02-01 23:39:53 +01:00
Sebastiaan van Stijn
bae1ac820e Merge pull request #49375 from LaurentGoderre/fix-typo
Fix typo in API docs
2025-01-31 22:12:52 +01:00
Laurent Goderre
844797348e Implement image mount for the snapshotter
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
2025-01-31 10:34:27 -05:00
Laurent Goderre
8c58934106 Implement mount from image
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
2025-01-31 10:34:27 -05:00
Sebastiaan van Stijn
f88304adf0 Merge pull request #49373 from thaJeztah/client_improve_error_response_handling
client: improve handling of JSON error-responses with incorrect schema
2025-01-31 13:04:39 +01:00
Laurent Goderre
c935a4609d Fix typo in API docs
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
2025-01-30 16:57:43 -05:00
Rob Murray
1463c99461 Merge pull request #49364 from robmry/wait_for_bridge_forwarding
Before sending ARPs/NAs, check the bridge is ready
2025-01-30 21:50:01 +00:00
Sebastiaan van Stijn
30e75b8396 client: improve handling of JSON error-responses with incorrect schema
Before this patch, an API response that's valid JSON, but not the right
schema would be silently discarded by the CLI. For example, due to a bug
in Docker Desktop's API proxy, the "normal" (not JSON error) response
would be returned together with a non-200 status code when using an
unsupported API version;

    curl -s -w 'STATUS: %{http_code}\n' --unix-socket /var/run/docker.sock 'http://localhost/v1.99/version'
    {"Platform":{"Name":"Docker Desktop 4.38.0 (181016)"},"Version":"","ApiVersion":"","GitCommit":"","GoVersion":"","Os":"","Arch":""}
    STATUS: 400

Before this patch, this resulted in no output being shown;

    DOCKER_API_VERSION=1.99 docker version
    Client:
     Version:           27.5.1
     API version:       1.99 (downgraded from 1.47)
     Go version:        go1.22.11
     Git commit:        9f9e405
     Built:             Wed Jan 22 13:37:19 2025
     OS/Arch:           darwin/arm64
     Context:           desktop-linux
    Error response from daemon:

With this patch, an error is generated based on the status:

    DOCKER_API_VERSION=1.99 docker version
    Client:
     Version:           27.5.1
     API version:       1.99 (downgraded from 1.47)
     Go version:        go1.22.11
     Git commit:        9f9e405
     Built:             Wed Jan 22 13:37:19 2025
     OS/Arch:           darwin/arm64
     Context:           desktop-linux
    Error response from daemon: API returned a 400 (Bad Request) but provided no error-message

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-30 22:01:13 +01:00
Sebastiaan van Stijn
e82d90384e Merge pull request #49367 from thaJeztah/fix_getNetworkedContainer_errors
daemon: Daemon.getNetworkedContainer: fix errors for invalid network container
2025-01-30 21:40:33 +01:00
Rob Murray
9a6e96fd9c Before sending ARPs/NAs, check the bridge is ready
We don't enable STP on bridges belonging to bridge networks,
but bridge ports still need to transition from "disabled" to
"forwarding", after the veth device comes "up". Until then,
the bridge will just drop packets.

So, if a container's network is a veth device, and its other
end is slaved to a bridge - wait for the bridge port to
be "forwarding".

Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-01-30 20:40:01 +00:00
Sebastiaan van Stijn
a5db4280d8 Merge pull request #49368 from thaJeztah/rmlink_nofuzzy
daemon: Daemon.rmLink: don't fuzzy-match container when using ID
2025-01-30 19:04:44 +01:00
Sebastiaan van Stijn
8860e98d1a Merge pull request #49369 from thaJeztah/daemon_remove_wrappers
daemon: remove Daemon.children(), Daemon.parents() wrappers
2025-01-30 18:04:41 +01:00
Sebastiaan van Stijn
e7f5d73fa5 Merge pull request #49354 from thaJeztah/client_test_coverage
client: improve test-coverage for error-responses
2025-01-30 17:07:00 +01:00
Sebastiaan van Stijn
5b5ad4df2c Merge pull request #49352 from thaJeztah/api_version_prefix
client: WithVersion: strip v-prefix when setting API version
2025-01-30 17:04:31 +01:00
Jonathan A. Sternberg
8e529682af builder: wire up new gc types for buildkit prune functionality
This wires up the new gc types that buildkit exposes in version 0.17.
The previous flag, `KeepBytes`, was renamed to `ReservedBytes` and two
new options, `MaxUsed` and `MinFree` were added.

`MaxUsed` corresponds to the maximum amount of space that buildkit will
use for the build cache and `MinFree` amount of free disk space for the
system to prevent the cache from using that space. This allows greater
configuration of the cache storage usage when used in situations where
docker is not the only service on the system using disk space.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-01-30 09:54:12 -06:00
Sebastiaan van Stijn
52ea449c55 daemon: Daemon.restore: make legacy-link code conditional
Make it more clear that this loop is for legacy-links, and the timer is
only needed for that purpose.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-30 14:23:10 +01:00
Sebastiaan van Stijn
9a69161992 daemon: remove Daemon.children(), Daemon.parents() wrappers
Remove the wrappers to make it more explicit that these are related to
the legacy links feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-30 14:23:10 +01:00
Sebastiaan van Stijn
513fd86710 daemon: Daemon.getNetworkedContainer: fix errors for invalid network container
When failing to resolve the network container, a "not found" error should
not return a 404, but either a "invalid parameter" (400) or "system" (500)
error.

Given that this function is called on container start, not container create,
a 500 (internal server error) is more appropriate, because the API request
(start the container) is valid, but the state of the container isn't.

While working on this, I discovered that no validation happens during container
create; TODO's were added to look into that, but this may be partially
by design (allow a container to be created before the "donor" network
container is created).

Before this patch:

    docker container create --name hello --network=container:nosuchcontainer alpine
    docker container start hello
    Error response from daemon: No such container: nosuchcontainer
    Error: failed to start containers: hello

    # daemon logs:
    DEBU[2025-01-30T11:32:33.595636043Z] error response for POST request               error-response="No such container: nosuchcontainer" method=POST module=api request-url=/v1.47/containers/hello/start status=404 vars="map[name:hello version:1.47]"

    docker container create --name hello2 --network=container:hello2 alpine
    docker container start hello2
    Error response from daemon: cannot join own network
    Error: failed to start containers: hello2

    # daemon logs:
    DEBU[2025-01-30T11:33:19.545287551Z] FIXME: Got an API for which error does not match any expected type!!!  error="cannot join own network" error_type="*errors.errorString" module=api
    DEBU[2025-01-30T11:33:19.545346093Z] error response for POST request               error-response="cannot join own network" method=POST module=api request-url=/v1.47/containers/hello2/start status=500 vars="map[name:hello2 version:1.47]"
    DEBU[2025-01-30T11:33:19.545369968Z] FIXME: Got an API for which error does not match any expected type!!!  error="cannot join own network" error_type="*errors.errorString" module=api
    ERRO[2025-01-30T11:33:19.545375426Z] Handler for POST /v1.47/containers/hello2/start returned error: cannot join own network

With this patch:

    docker container create --name hello --network=container:nosuchcontainer alpine
    docker container start hello
    Error response from daemon: joining network of container: No such container: nosuchcontainer
    Error: failed to start containers: hello

    # daemon logs:
    DEBU[2025-01-30T11:35:50.406462760Z] error response for POST request               error-response="joining network of container: No such container: nosuchcontainer" method=POST module=api request-url=/v1.47/containers/hello/start status=500 vars="map[name:hello version:1.47]"
    ERRO[2025-01-30T11:35:50.406501468Z] Handler for POST /v1.47/containers/hello/start returned error: joining network of container: No such container: nosuchcontainer

    docker container create --name hello2 --network=container:hello2 alpine
    docker container start hello2
    Error response from daemon: cannot join own network namespace
    Error: failed to start containers: hello2

    # daemon logs:
    DEBU[2025-01-30T11:36:15.178475049Z] error response for POST request               error-response="cannot join own network" method=POST module=api request-url=/v1.47/containers/hello2/start status=500 vars="map[name:hello2 version:1.47]"
    ERRO[2025-01-30T11:36:15.178536507Z] Handler for POST /v1.47/containers/hello2/start returned error: cannot join own network

    docker run --name exitedcontainer alpine
    docker run --rm --network=container:exitedcontainer alpine
    docker: Error response from daemon: cannot join network namespace of a non running container: container exitedcontainer is exited.

    # daemon logs:
    DEBU[2025-01-30T12:54:28.040637429Z] error response for POST request               error-response="cannot join network namespace of a non running container: container exitedcontainer is exited" method=POST module=api request-url=/v1.47/containers/hello2/start status=409 vars="map[name:hello2 version:1.47]"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-30 14:18:33 +01:00
Sebastiaan van Stijn
51cb066ec8 daemon: Daemon.rmLink: don't fuzzy-match container when using ID
rmLink already looked up the parent container's ID, so we should not use
daemon.GetContainer to resolve the container, as that performs fuzzy
matching (name, ID-prefix, or ID).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-30 11:39:19 +01:00