Commit Graph

1492 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
e8c6e7ceb0 build: don't print warning when connection was terminated
A terminated connection is not an error on the daemon-side, and expected
if the client disconnects. This patch detects if the error is because of
a broken pipe, and skips the warning in that case.

Before this patch:

    WARN[2025-01-18T12:38:04.115298341Z] could not write error response: write unix /var/run/docker.sock->@: write: broken pipe

After this patch, no warning is logged. This patch also changes the log
format to use structured logs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-18 16:24:54 +01:00
Sebastiaan van Stijn
81a2acdd23 Merge pull request #48696 from thaJeztah/build_log_cancelled
build: log when build is cancelled
2025-01-18 16:20:38 +01:00
Sebastiaan van Stijn
c817ea2159 api/server/middleware: log before, not after the request
commit 1701bce9e0 updated the debug logs
to use structured logs, and to include more information in the logs, such
as the form-data used and the status-code, and combining them into a
single log.

However, for the status-code, we need to wait for the handler to do its
thing and (ahum) this was staring me right in the face, and crossed my
mind, but then I didn't act on it; the handler may take some time to
run, and produce logs, which now means that our nice "handling request"
log will be logged _after_ the request, which is obviously confusing.

This patch splits the log into two;

- a log entry when starting to handle the request
- a log entry if a non-200 status is returned (assuming 200 status
  codes are less interesting to log).

Before this patch:

    DEBU[2024-10-23T15:23:31.677184128Z] handling HEAD request                         method=HEAD module=api request-url=/_ping spanID=8180b03fa17f9783 status=200 traceID=a5dfa9b86445467889274145ad31bb9a vars="map[]"
    DEBU[2024-10-23T15:23:31.712833045Z] resolving                                     host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
    DEBU[2024-10-23T15:23:31.712883670Z] do request                                    host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:23:32.051728378Z] fetch response received                       host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:23:32 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:23:32.051830920Z] Unauthorized                                  header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
    DEBU[2024-10-23T15:23:32.051909045Z] do request                                    host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:23:32.544987920Z] fetch response received                       host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:23:32 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:23:32.545112212Z] Unauthorized                                  header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
    INFO[2024-10-23T15:23:32.545270087Z] trying next host                              error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
    DEBU[2024-10-23T15:23:32.550666962Z] handling POST request                         error-response="pull access denied for nosuchimage, repository does not exist or may require 'docker login'" method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=38286e48a07445ef status=404 traceID=3c01e6c2cf19cf82237fdd01c0294fb4 vars="map[version:1.47]"

With this patch applied:

    DEBU[2024-10-23T15:18:18.876346178Z] handling HEAD request                         method=HEAD module=api request-url=/_ping spanID=7fd5eb011140f546 traceID=80ffd75a39de78a1f51ffda89fc4f227 vars="map[]"
    DEBU[2024-10-23T15:18:18.878006428Z] handling POST request                         method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=40dea95727e38394 traceID=2f901d99f1cf2105e2614d6929d53d3b vars="map[version:1.47]"
    DEBU[2024-10-23T15:18:18.919686136Z] resolving                                     host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
    DEBU[2024-10-23T15:18:18.919748094Z] do request                                    host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:18:19.258132303Z] fetch response received                       host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:18:19 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:18:19.258219803Z] Unauthorized                                  header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
    DEBU[2024-10-23T15:18:19.258406886Z] do request                                    host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:18:19.755911762Z] fetch response received                       host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:18:19 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
    DEBU[2024-10-23T15:18:19.757501928Z] Unauthorized                                  header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
    INFO[2024-10-23T15:18:19.757689387Z] trying next host                              error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
    DEBU[2024-10-23T15:18:19.763826095Z] error response for POST request               error-response="pull access denied for nosuchimage, repository does not exist or may require 'docker login'" method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=40dea95727e38394 status=404 traceID=2f901d99f1cf2105e2614d6929d53d3b vars="map[version:1.47]"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-17 11:00:17 +01:00
Derek McGowan
0aa8fe0bf9 Update to containerd v2.0.2, buildkit v0.19.0-rc2
Update buildkit version to commit which uses 2.0

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-15 14:09:30 +01:00
Sebastiaan van Stijn
818a180fce pkg/ioutils: deprecate NopFlusher
Apart from being used internally for NewWriteFlusher, it's only used
in a single location outside of this package. Copy the implementation
where it's used, and mark it deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 17:56:34 +01:00
Paweł Gronowski
7e9a5064dc api: Remove unused imageStore and layerStore
Commit 8fb71ce208 moved access to these to
the image service directly, so they are no longer used in the router.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-19 14:50:50 +01:00
Sebastiaan van Stijn
01b98afeb6 Merge pull request #49054 from maggie44/apiversion
Accurately reflect the canonical casing of `API-Version` and `OS-Type` headers
2024-12-16 13:06:31 +01:00
Sebastiaan van Stijn
1e0477f6ec Merge pull request #49065 from thaJeztah/deprecate_nondistributable_artifacts
remove support for configuring pushing non-distributable artifacts and deprecate API fields and config
2024-12-12 21:39:34 +01:00
Paweł Gronowski
d8358ebc87 otel: Use non-noop tracer provider for grpc
Needed for Buildkit history

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 19:31:00 +01:00
Sebastiaan van Stijn
1932091e21 remove support for non-distributable artifacts and deprecate API fields and config
Non-distributable artifacts (foreign layers) were introduced in commit
05bd04350b to accommodate Windows images,
for which the EULA did not allow layers to be distributed through registries
other than those hosted by Microsoft. The concept of foreign / non-distributable
layers was adopted by the OCI distribution spec in [oci#233].

These restrictions were relaxed later to allow distributing these images
through non-public registries, for which a configuration was added in the
daemon in 67fdf574d5. In 2022, Microsoft updated
the EULA and [removed these restrictions altogether][1], and the OCI distribution
spec deprecated the  functionality in [oci#965].

In 2023, Microsoft [removed the use of foreign data layers][2] for their images,
making this functionality obsolete.

This patch:

- Deprecates the `--allow-nondistributable-artifacts` daemon flag and corresponding
  `allow-nondistributable-artifacts` field in `daemon.json`. Setting either
  option will no longer take an effect, but a deprecation warning log is added
  to raise awareness about the deprecation. This warning is planned to become
  an error in the next release.
- Deprecates the `RegistryConfig.AllowNondistributableArtifactsCIDRs` and
  `RegistryConfig.AllowNondistributableArtifactsHostnames` fields in the
  `GET /info` API response. For API version v1.48 and lower, the fields are
  still included in the response, but always `null`. In API version v1.49 and
  higher, the field will be omitted entirely.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsCIDRs`
  field.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsHostnames`
  field.
- Deprecates the `registry.ServiceOptions.AllowNondistributableArtifacts` field.

[oci#233]: https://github.com/opencontainers/image-spec/pull/233
[oci#965]: https://github.com/opencontainers/image-spec/pull/965
[1]: https://techcommunity.microsoft.com/blog/containers/announcing-windows-container-base-image-redistribution-rights-change/3645201
[2]: https://techcommunity.microsoft.com/blog/containers/announcing-removal-of-foreign-layers-from-windows-container-images/3846833

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-11 20:29:46 +01:00
Sebastiaan van Stijn
9da0e69608 api/server: Server.CreateMux: pass context and use structured logs
Pass the context that's used for logging, and add minimal handling of
context-cancellation. Also update logs to use structured fields.

Before this  patch:

    DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
    DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
    DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}

With this patch:

    DEBU[2024-12-08T15:33:50.408445543Z] Registering route                              method=POST path="/networks/{id:.*}/disconnect"
    DEBU[2024-12-08T15:33:50.408484335Z] Registering route                              method=POST path=/networks/prune
    DEBU[2024-12-08T15:33:50.408505251Z] Registering route                              method=DELETE path="/networks/{id:.*}"

Or in JSON format:

    {"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/connect","time":"2024-12-08T15:37:19.235209667Z"}
    {"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/disconnect","time":"2024-12-08T15:37:19.235243001Z"}
    {"level":"debug","method":"POST","msg":"Registering route","path":"/networks/prune","time":"2024-12-08T15:37:19.235290876Z"}

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-09 15:58:46 +01:00
Sebastiaan van Stijn
9c1ff095e8 cmd/dockerd: pass debug-router instead of constructing in CreateMux
Now that debug-routes are identical to regular routers, we can pass them
the same as those routers. With this, the daemon also logs those routes
as part of its startup (when in debug mode).

Before this patch, only non-debug endpoints would be logged:

    DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
    ...
    DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
    DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
    DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
    INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock

With this patch, debug endpoints are also logged:

    DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
    ...
    DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
    DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
    DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
    DEBU[2024-12-08T15:24:47.324486834Z] Registering GET, /debug/vars
    DEBU[2024-12-08T15:24:47.324506751Z] Registering GET, /debug/pprof/
    DEBU[2024-12-08T15:24:47.324532126Z] Registering GET, /debug/pprof/cmdline
    DEBU[2024-12-08T15:24:47.324549293Z] Registering GET, /debug/pprof/profile
    DEBU[2024-12-08T15:24:47.324564501Z] Registering GET, /debug/pprof/symbol
    DEBU[2024-12-08T15:24:47.324582043Z] Registering GET, /debug/pprof/trace
    DEBU[2024-12-08T15:24:47.324604751Z] Registering GET, /debug/pprof/{name}
    INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-09 15:55:26 +01:00
Sebastiaan van Stijn
27294df576 api/server: Server.CreateMux: also register API-version debug endpoints
The debug endpoints are currently only provided non-versioned (e.g. `/debug/vars`).
While this is convenient, we "officially" deprecated non-versioned endpoints
in the API.

This patch also registers the debug-endpoints under the API-versioned paths,
so that they can be used either without version ("latest") and versioned
paths.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-09 15:55:25 +01:00
Sebastiaan van Stijn
e2d2834be1 api/server: Server.CreateMux: register debug endpoints with correct methods
The debug handlers were created for GET methods, but were registered for
any method;

    curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
    ["dockerd","--debug"]
    curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
    ["dockerd","--debug"]
    curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
    ["dockerd","--debug"]

After this patch, they're only registered with the intended method, and a
404 is returned for incorrect ones;

    curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
    ["dockerd","--debug"]
    curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars
    {"message":"page not found"}
    curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars
    {"message":"page not found"}

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-09 15:55:25 +01:00
Sebastiaan van Stijn
2f5412de8d api/server: set /debug prefix as part of debug-router routes
Update the debug-router to include  the prefix, instead of adding it
when registering the routes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-09 15:55:22 +01:00
maggie44
76a5ca1d4d Accurately reflect the canonical casing of API-Version and OS-Type headers
Go automatically canonicalises HTTP headers, meaning the string `API-Version` passed as a header has always been returned as `Api-Version`. Similarly, `OSType` is returned as `Ostype`.

This commit updates the documentation to reflect this behaviour and modifies the codebase to ensure that input strings are aligned with their canonical output values.

Signed-off-by: maggie44 <64841595+maggie44@users.noreply.github.com>
2024-12-08 22:23:57 +00:00
Albin Kerouanton
5b752fab32 api: add Priority field to EndpointSettings
This new field is used by libnetwork to determine which endpoint
provides the default gateway for a container.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-11-28 17:06:26 +01:00
Sebastiaan van Stijn
968213c763 Merge pull request #48902 from thaJeztah/save_load_multiplat
client: support multiple platforms on save and load
2024-11-20 15:24:51 +01:00
Sebastiaan van Stijn
9c9eccfb23 client: support multiple platforms on save and load
We don't yet support this at the API level, so for now it returns
an error when trying to set multiple, but this makes sure that the
client types are already ready for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-19 15:56:28 +01:00
Paweł Gronowski
44ed3067ca c8d/container/inspect: Return ImageManifestDescriptor
`ImageManifestDescriptor` will contain an OCI descriptor of
platform-specific manifest of the image that was picked when creating
the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-19 13:55:56 +01:00
Sebastiaan van Stijn
02a2f64986 Merge pull request #48891 from thaJeztah/remove_logentries_migration
remove logentries check and migration code
2024-11-18 19:01:28 +01:00
Paweł Gronowski
d88ab0f3a2 c8d/image/inspect: Return Descriptor
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-18 13:00:44 +01:00
Sebastiaan van Stijn
c2c0046d11 remove logentries check and migration code
This code was added in 3b1d9f1a26 when the
logentries logging-driver was removed in v25.0.0. The logentries service
was already defunct, so unlikely to have any consumers, so let's remove
this code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-18 10:42:00 +01:00
Rob Murray
5ef5ca3adc EnableIPv4 will be in API 1.48, not 1.47
Numbering has changed since commit d4d8611164

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-11-15 18:21:25 +00:00
Sebastiaan van Stijn
0f0c5eeaca api/server/router/container: add missing go:build tag
This was introduced in 3f2e9da0100af2ceb3ef0d6431cb2b27dc3e1e49;

    make BIND_DIR=. shell
    make -C ./internal/gocompat/
    GO111MODULE=on go test -v
    # github.com/docker/docker/api/server/router/container
    ../../api/server/router/container/inspect.go:29:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    FAIL	gocompat [build failed]

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 12:53:20 +01:00
Sebastiaan van Stijn
cd8d2c56c3 Merge pull request #48861 from vvoland/c8d-list-index
c8d/list: Return `Descriptor`
2024-11-14 21:27:45 +01:00
Paweł Gronowski
b15cd283d8 c8d/list: Return Descriptor
While the endpoint returns a detailed information about its children, it
doesn't actually expose the descriptor of the root OCI index/manifest
list.
This commits adds the target description to the returned JSON.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-13 15:02:18 +01:00
Sebastiaan van Stijn
59f68d3f99 api/server/middleware: remove redundant capturing of loop vars (copyloopvar)
api/server/middleware/version_test.go:59:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
            tc := tc
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-12 14:02:18 +01:00
Sebastiaan van Stijn
c2e22d85b3 api/server/httputils: remove redundant capturing of loop vars (copyloopvar)
api/server/httputils/form_test.go:152:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
            tc := tc
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-12 14:02:11 +01:00
Sebastiaan van Stijn
b453aa65fa update go:build tags to use go1.22
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.

go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-12 14:02:09 +01:00
Sebastiaan van Stijn
cb966073ce container create: add warning for volumeDriver together with mounts
commit 77fe35b3b9 added a warning if the
"volumeDriver" option was used in combination with --mount.

That change put the warning in the CLI, which should not have to handle
this logic. Let's move it to the daemon as we have a warnings response
in the container create response.

The warning is gated by API version, so that older clients can continue
to produce the warning client-side.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-29 12:16:48 +01:00
Rob Murray
533e7d150a From 28.0.0, don't migrate per-endpoint sysctls.
Re-add checks that were removed until 28.0.0's API version was
known, in commit 17adc1478b

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-24 12:20:01 +01:00
Sebastiaan van Stijn
66f159dc24 build: log when build is cancelled
I noticed that the daemon logs, even in debug, did not give any
indication that the build was cancelled. This patch adds some logs
to help debugging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-18 23:22:32 +02:00
Brian Goff
e7f9f0640f Merge pull request #48677 from thaJeztah/debug_structured_logs
api/server/middleware: use structured logs for debug-logs
2024-10-18 11:02:11 -07:00
Sebastiaan van Stijn
ed44a05607 api: backend.ContainerExecResize: pass context and use uint32 for width, height
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 14:04:37 +02:00
Sebastiaan van Stijn
2b2aa69100 api: exec resize: improve errors for invalid width/height
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 14:04:37 +02:00
Sebastiaan van Stijn
8553d34593 api: backend.ContainerResize: pass context and use uint32 for width, height
Containerd accepts uints for these, so make the backend signature  align
with that, so that we don't have to cast values. Also pass the context
along.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 14:04:36 +02:00
Sebastiaan van Stijn
d19aa0c590 api: container resize: improve errors for invalid width/height
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 14:04:36 +02:00
Sebastiaan van Stijn
1dd9d2c780 api/server/httputils: add Uint32Value utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 14:04:32 +02:00
Sebastiaan van Stijn
039e164bf5 api/server/httputils: DecodePlatform: improve test-coverage
- Use fixtures for the JSON strings
- Add test-cases for invalid / malformed JSON
- Check error-message produced
- Add test for "happy path"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 10:47:10 +02:00
Sebastiaan van Stijn
1701bce9e0 api/server/middleware: use structured logs for debug-logs
Rewrite the debug-logs produced

- Use structured logs
- Combine into a single log per request, instead of separate log-entry
  for the "form-data".
- Include error-messages returned by the handler ("error-response" field)
- Include HTTP status-code returned ("status" field)
- Include the "vars" as a field; these are fields extracted from the URL
  and passed to the handler

Examples below are logs for:

    docker ps
    docker container inspect nosuchcontainer
    docker volume create --name foo

Before this change:

    DEBU[2024-10-16T10:59:40.484254465Z] Calling HEAD /_ping                           spanID=43d76043f8e30dbb traceID=04f980a33901f35ba33c3927d3bb4bbb
    DEBU[2024-10-16T10:59:40.485551840Z] Calling GET /v1.47/containers/json            spanID=b9979f2b36572a43 traceID=5c2167537df2dede6bdbab030f8350bc
    DEBU[2024-10-16T11:00:00.374864502Z] Calling HEAD /_ping                           spanID=d637e39684d56a16 traceID=efaed7838901dd6a597c5446ce3f83e2
    DEBU[2024-10-16T11:00:00.384198127Z] Calling GET /v1.47/containers/nosuchcontainer/json  spanID=f9cc4520b95d814b traceID=c15ae04ca248929d6e52474e711d48b0
    DEBU[2024-10-16T11:00:11.576426632Z] Calling HEAD /_ping                           spanID=2bc30d2be873a8e5 traceID=53ccc3d2af87aa5425421306906660a6
    DEBU[2024-10-16T11:00:11.588877966Z] Calling POST /v1.47/volumes/create            spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
    DEBU[2024-10-16T11:00:11.589198966Z] form data: {"Driver":"local","Name":"foo"}    spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
    DEBU[2024-10-16T11:00:11.594828216Z] using regular volume                          spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23

After this:

When using plain-text, we continue encoding the form-data to JSON, but
as it's now in a field, it'll be shown escaped;

    DEBU[2024-10-16T11:17:35.465777379Z] handling HEAD request                         method=HEAD module=api request-url=/_ping spanID=9b7ea0288b2b70c3 status=200 traceID=94ef9345624e92ac0263931fbe9e15db vars="map[]"
    DEBU[2024-10-16T11:17:35.468050171Z] handling GET request                          method=GET module=api request-url=/v1.47/containers/json spanID=04675edee7b5ec9d status=200 traceID=a9d81dcdbf2650fa6d794a7a856fb66b vars="map[version:1.47]"
    DEBU[2024-10-16T11:17:38.502289297Z] handling HEAD request                         method=HEAD module=api request-url=/_ping spanID=7c43a8dfd8fb5043 status=200 traceID=7a2a7c71cd421570e811474749a04ccd vars="map[]"
    DEBU[2024-10-16T11:17:38.504847506Z] handling GET request                          error-response="No such container: nosuchcontainer" method=GET module=api request-url=/v1.47/containers/nosuchcontainer/json spanID=ab721bbbe5cf8035 status=404 traceID=4a08dcd5054fc8090e3af8846beea10d vars="map[name:nosuchcontainer version:1.47]"
    DEBU[2024-10-16T11:17:40.788838340Z] handling HEAD request                         method=HEAD module=api request-url=/_ping spanID=2dbc18ba1334635b status=200 traceID=ea9af681d096dc4a2c2f2ed7338ea417 vars="map[]"
    DEBU[2024-10-16T11:17:40.790496465Z] handling POST request                         form-data="{\"Driver\":\"local\",\"Name\":\"foo\"}" method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"

The alternative to the above would be to unconditionally set it as-is,
but in that case it would use Go's formatting for `map[string]any`;

    DEBU[2024-10-16T11:27:54.937232805Z] handling POST request                         form-data="map[Driver:local Name:foo]" method=POST module=api request-url=/v1.47/volumes/create spanID=2d7985a900791bf6 status=200 traceID=33feab9fd5feba3b0f4b6ec5a6971a67 vars="map[version:1.47]"

Or to use some trickery to not quote this specific field, but that may limit the
output from being parsable;

    DEBU[2024-10-16T11:17:40.790496465Z] handling POST request                         form-data={"Driver":"local","Name":"foo"} method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"

When using `--log-format=json`, the form-data is kept as structured, becoming
part of the main JSON struct:

    {"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"166dc12eeeadf82b","status":200,"time":"2024-10-16T11:16:09.427380423Z","traceID":"7f4f2501eee3b15ae608481ba214bd56","vars":{}}
    {"level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/json","spanID":"bf95e2ce9eca41c2","status":200,"time":"2024-10-16T11:16:09.429077631Z","traceID":"041b26b30dacc240e8e3afc9c567195d","vars":{"version":"1.47"}}
    {"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"454953906c36ea6b","status":200,"time":"2024-10-16T11:16:13.455633008Z","traceID":"3ffc0a256d6ec1a56cd7f6bf1008e55d","vars":{}}
    {"error-response":"No such container: nosuchcontainer","level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/nosuchcontainer/json","spanID":"dcf0d42921928b29","status":404,"time":"2024-10-16T11:16:13.460309925Z","traceID":"fdfd2c89941c9c7a459bec7a05e46ef8","vars":{"name":"nosuchcontainer","version":"1.47"}}
    {"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"701dc623cf1b0253","status":200,"time":"2024-10-16T11:16:16.155730884Z","traceID":"786885a9f79cbfba99097eeb4145ca1e","vars":{}}
    {"form-data":{"Driver":"local","Name":"foo"},"level":"debug","method":"POST","module":"api","msg":"handling POST request","request-url":"/v1.47/volumes/create","spanID":"dc1429c1c636b30a","status":200,"time":"2024-10-16T11:16:16.162002426Z","traceID":"fc49ee4a7acafbbb8eb50ed34c434765","vars":{"version":"1.47"}}

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-16 14:47:53 +02:00
Sebastiaan van Stijn
3f2e9da010 api/server/router/container: move API adjustments to API
The daemon used to have various implementation to adjust the container-inspect
output for different API versions, which could return different go structs,
and because of that required a function with a `interface{}` output type.

Most of those adjustments have been removed, and we no longer need separate
types for backward compatibility with old API versions.

This patch;

- Removes the Daemon.ContainerInspectCurrent method
- Introduces a backend.ContainerInspectOptions struct
- Updates the Daemon.ContainerInspect method's signature to accept the above
- Moves API-version specific adjustments to api/server/router/container,
  similar to how such adjustments are made for other endpoints.

Note that we should probably change the backend's signature further,
and define separate types for the backend's inspect and the API's
inspect response. Considering that the Backend signatures should be
considered "internal", we can do that in a future change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-15 17:03:03 +02:00
Sebastiaan van Stijn
0c595fe781 api/server/router/container: fix inconsistent receiver name
It's good practice to use a consistent name; we couldn't use `r` as name,
as it's used for the request argument, and `s` honestly didn't make much
sense as name, so changing it to `c`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-15 16:43:16 +02:00
Sebastiaan van Stijn
7eb508e14c api: postImagesLoad: fix API version for platform
This option was added in f143f4ec51,
which changed the minimum API version for "save" but forgot to update
the version for "load".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-07 11:40:47 +02:00
Sebastiaan van Stijn
ff191c58f7 api: info: deprecate "Commit.Expected" fields
The `Commit` type was introduced in 2790ac68b3,
to assist triaging issues that were reported with an incorrect version of
runc or containerd. At the time, both `runc` and `containerd` were not yet
stable, and had to be built from a specific commit to guarantee compatibility.

We encountered various situations where unexpected (and incompatible) versions
of those binaries were packaged, resulting in hard to trace bug-reports.
For those situations, a "expected" version was set at compile time, to
indicate if the version installed was different from the expected version;

    docker info
    ...
    runc version: a592beb5bc4c4092b1b1bac971afed27687340c5 (expected: 69663f0bd4b60df09991c08812a60108003fa340)

Both `runc` and `containerd` are stable now, and docker 19.03 and up set the
expected version to the actual version since c65f0bd13c
and 23.0 did the same for the `init` binary b585c64e2b,
to prevent the CLI from reporting "unexpected version".

In short; the `Expected` fields no longer serves a real purpose.

In future, we can even consider deprecating the `ContainerdCommit`, `RuncCommit`
and `InitCommit` fields on the `/info` response (as we also include this
information as part of the components returned in `/version`), but those
can still be useful currently for situations where a user only provides
`docker info` output.

This patch starts with deprecating the `Expected` field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 19:40:12 +02:00
Paweł Gronowski
f143f4ec51 image/save&load: Support Platform parameter
Add `Platform` parameter that allows to select a specific platform to
save/load.

This is a breaking change to the Go client as it changes the signatures
of `ImageLoad` and `ImageSave`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 19:44:35 +02:00
Paweł Gronowski
ab075ecd10 image/history: Support Platform parameter
Add `Platform` parameter that allows to select a specific platform to
show the history for.

This is a breaking change to the Go client as it changes the signature
of `ImageHistory`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 19:44:32 +02:00
Nathan Baulch
59eba0ae13 Fix typos
Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
2024-09-06 21:53:09 +10:00
Sebastiaan van Stijn
0fd3a53c12 api/server/router: fix non-constant format string in call (govet)
api/server/router/container/container_routes.go:943:22: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
                                  ^
    api/server/router/image/image_routes.go:144:50: printf: non-constant format string in call to github.com/docker/docker/pkg/streamformatter.FormatStatus (govet)
                output.Write(streamformatter.FormatStatus("", id.String()))
                                                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-27 10:41:46 +02:00
Paweł Gronowski
050afe1e1a api/list: Expose manifests
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).

There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-07 13:48:54 +02:00