Add `Manifests` field to image inspect (`/images/{name}/json`) response.
This is the same as in `/images/json`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This wires up the new gc types that buildkit exposes in version 0.17.
The previous flag, `KeepBytes`, was renamed to `ReservedBytes` and two
new options, `MaxUsed` and `MinFree` were added.
`MaxUsed` corresponds to the maximum amount of space that buildkit will
use for the build cache and `MinFree` amount of free disk space for the
system to prevent the cache from using that space. This allows greater
configuration of the cache storage usage when used in situations where
docker is not the only service on the system using disk space.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
It was the only use of errdefs inside api/types, which is the package
that's imported by external users.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was the only use of github.com/pkg/errors inside api/types,
which is the package that's imported by external users.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A terminated connection is not an error on the daemon-side, and expected
if the client disconnects. This patch detects if the error is because of
a broken pipe, and skips the warning in that case.
Before this patch:
WARN[2025-01-18T12:38:04.115298341Z] could not write error response: write unix /var/run/docker.sock->@: write: broken pipe
After this patch, no warning is logged. This patch also changes the log
format to use structured logs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 1701bce9e0 updated the debug logs
to use structured logs, and to include more information in the logs, such
as the form-data used and the status-code, and combining them into a
single log.
However, for the status-code, we need to wait for the handler to do its
thing and (ahum) this was staring me right in the face, and crossed my
mind, but then I didn't act on it; the handler may take some time to
run, and produce logs, which now means that our nice "handling request"
log will be logged _after_ the request, which is obviously confusing.
This patch splits the log into two;
- a log entry when starting to handle the request
- a log entry if a non-200 status is returned (assuming 200 status
codes are less interesting to log).
Before this patch:
DEBU[2024-10-23T15:23:31.677184128Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=8180b03fa17f9783 status=200 traceID=a5dfa9b86445467889274145ad31bb9a vars="map[]"
DEBU[2024-10-23T15:23:31.712833045Z] resolving host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
DEBU[2024-10-23T15:23:31.712883670Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.051728378Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:23:32 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.051830920Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
DEBU[2024-10-23T15:23:32.051909045Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.544987920Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:23:32 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.545112212Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
INFO[2024-10-23T15:23:32.545270087Z] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
DEBU[2024-10-23T15:23:32.550666962Z] handling POST request error-response="pull access denied for nosuchimage, repository does not exist or may require 'docker login'" method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=38286e48a07445ef status=404 traceID=3c01e6c2cf19cf82237fdd01c0294fb4 vars="map[version:1.47]"
With this patch applied:
DEBU[2024-10-23T15:18:18.876346178Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=7fd5eb011140f546 traceID=80ffd75a39de78a1f51ffda89fc4f227 vars="map[]"
DEBU[2024-10-23T15:18:18.878006428Z] handling POST request method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=40dea95727e38394 traceID=2f901d99f1cf2105e2614d6929d53d3b vars="map[version:1.47]"
DEBU[2024-10-23T15:18:18.919686136Z] resolving host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
DEBU[2024-10-23T15:18:18.919748094Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.258132303Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:18:19 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.258219803Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
DEBU[2024-10-23T15:18:19.258406886Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.755911762Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:18:19 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.757501928Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
INFO[2024-10-23T15:18:19.757689387Z] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
DEBU[2024-10-23T15:18:19.763826095Z] error response for POST request error-response="pull access denied for nosuchimage, repository does not exist or may require 'docker login'" method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=40dea95727e38394 status=404 traceID=2f901d99f1cf2105e2614d6929d53d3b vars="map[version:1.47]"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The StatsResponse type was a compatibility-wrapper introduced in
d3379946ec to differentiate responses
for API < 1.21 and API >= 1.21. API versions lower than 1.24 are
deprecated, and we can merge these types again.
The Stats type was not used directly, but deprecating it, and
making it an alias for StatsResponse, which provides a superset
of its fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Apart from being used internally for NewWriteFlusher, it's only used
in a single location outside of this package. Copy the implementation
where it's used, and mark it deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 8fb71ce208 moved access to these to
the image service directly, so they are no longer used in the router.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- Updated the example section in `swagger.yaml` for the `DataPathAddr` parameter in `SwarmJoinRequest`.
- Updated corresponding example sections in `docs/api/v1.*`
Signed-off-by: Adam Lamers <adam.lamers@wmsdev.pl>
The netfilter module is now loaded on-demand, and no longer during daemon
startup, making these fields obsolete. These fields are now always `false`
and will be removed in the next relase.
This patch deprecates:
- the `BridgeNfIptables` field in `api/types/system.Info`
- the `BridgeNfIp6tables` field in `api/types/system.Info`
- the `BridgeNFCallIPTablesDisabled` field in `pkg/sysinfo.SysInfo`
- the `BridgeNFCallIP6TablesDisabled` field in `pkg/sysinfo.SysInfo`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Historically, the `bridge` network-driver would detect whether netfiltering
was enabled in the kernel or, if disabled, try to do a `modprobe` when
initializing the driver. This approach became problematic, as loading the
module was not always performed at startup depending on daemon configuration,
or the daemon may have failed to load the module. The `/info` response
would include a warning to inform the user that some functionality may not
be available;
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Starting with db25b0dcd0, detecting whether
netfiltering is enabled now [happens when needed][1], which was further improved
on to not depend on `modprobe` in 264c15bfc4 and
4740820716.
Because of the above, the `/info` output would now return warnings in any
situation where netfiltering was not enabled on the host before the daemon
started, which may be either _incorrect_ (i.e., the module may have been
loaded afterwards), or irrelevant, because netfiltering is not needed in
all situations.
This patch removes the warnings from the `/info` response,
[1]: 944e403502/libnetwork/drivers/bridge/setup_bridgenetfiltering.go (L16-L77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Non-distributable artifacts (foreign layers) were introduced in commit
05bd04350b to accommodate Windows images,
for which the EULA did not allow layers to be distributed through registries
other than those hosted by Microsoft. The concept of foreign / non-distributable
layers was adopted by the OCI distribution spec in [oci#233].
These restrictions were relaxed later to allow distributing these images
through non-public registries, for which a configuration was added in the
daemon in 67fdf574d5. In 2022, Microsoft updated
the EULA and [removed these restrictions altogether][1], and the OCI distribution
spec deprecated the functionality in [oci#965].
In 2023, Microsoft [removed the use of foreign data layers][2] for their images,
making this functionality obsolete.
This patch:
- Deprecates the `--allow-nondistributable-artifacts` daemon flag and corresponding
`allow-nondistributable-artifacts` field in `daemon.json`. Setting either
option will no longer take an effect, but a deprecation warning log is added
to raise awareness about the deprecation. This warning is planned to become
an error in the next release.
- Deprecates the `RegistryConfig.AllowNondistributableArtifactsCIDRs` and
`RegistryConfig.AllowNondistributableArtifactsHostnames` fields in the
`GET /info` API response. For API version v1.48 and lower, the fields are
still included in the response, but always `null`. In API version v1.49 and
higher, the field will be omitted entirely.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsCIDRs`
field.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsHostnames`
field.
- Deprecates the `registry.ServiceOptions.AllowNondistributableArtifacts` field.
[oci#233]: https://github.com/opencontainers/image-spec/pull/233
[oci#965]: https://github.com/opencontainers/image-spec/pull/965
[1]: https://techcommunity.microsoft.com/blog/containers/announcing-windows-container-base-image-redistribution-rights-change/3645201
[2]: https://techcommunity.microsoft.com/blog/containers/announcing-removal-of-foreign-layers-from-windows-container-images/3846833
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pass the context that's used for logging, and add minimal handling of
context-cancellation. Also update logs to use structured fields.
Before this patch:
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
With this patch:
DEBU[2024-12-08T15:33:50.408445543Z] Registering route method=POST path="/networks/{id:.*}/disconnect"
DEBU[2024-12-08T15:33:50.408484335Z] Registering route method=POST path=/networks/prune
DEBU[2024-12-08T15:33:50.408505251Z] Registering route method=DELETE path="/networks/{id:.*}"
Or in JSON format:
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/connect","time":"2024-12-08T15:37:19.235209667Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/disconnect","time":"2024-12-08T15:37:19.235243001Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/prune","time":"2024-12-08T15:37:19.235290876Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that debug-routes are identical to regular routers, we can pass them
the same as those routers. With this, the daemon also logs those routes
as part of its startup (when in debug mode).
Before this patch, only non-debug endpoints would be logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
With this patch, debug endpoints are also logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
DEBU[2024-12-08T15:24:47.324486834Z] Registering GET, /debug/vars
DEBU[2024-12-08T15:24:47.324506751Z] Registering GET, /debug/pprof/
DEBU[2024-12-08T15:24:47.324532126Z] Registering GET, /debug/pprof/cmdline
DEBU[2024-12-08T15:24:47.324549293Z] Registering GET, /debug/pprof/profile
DEBU[2024-12-08T15:24:47.324564501Z] Registering GET, /debug/pprof/symbol
DEBU[2024-12-08T15:24:47.324582043Z] Registering GET, /debug/pprof/trace
DEBU[2024-12-08T15:24:47.324604751Z] Registering GET, /debug/pprof/{name}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug endpoints are currently only provided non-versioned (e.g. `/debug/vars`).
While this is convenient, we "officially" deprecated non-versioned endpoints
in the API.
This patch also registers the debug-endpoints under the API-versioned paths,
so that they can be used either without version ("latest") and versioned
paths.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug handlers were created for GET methods, but were registered for
any method;
curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
After this patch, they're only registered with the intended method, and a
404 is returned for incorrect ones;
curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars
{"message":"page not found"}
curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars
{"message":"page not found"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go automatically canonicalises HTTP headers, meaning the string `API-Version` passed as a header has always been returned as `Api-Version`. Similarly, `OSType` is returned as `Ostype`.
This commit updates the documentation to reflect this behaviour and modifies the codebase to ensure that input strings are aligned with their canonical output values.
Signed-off-by: maggie44 <64841595+maggie44@users.noreply.github.com>
This field was introduced in 5b752fab32,
which added documentation in the API documentation (swagger), but did
not document the field in the API go types.
This patch adds documentation, using a variant of the description used
in swagger.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This new field is used by libnetwork to determine which endpoint
provides the default gateway for a container.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
There were a handful of direct checks against errors.Is that can be
translated to assert.ErrorIs without too much thought. Unfortunately
there are a load of other examples where ErrorIs probably makes sense
but would require testing whether this subtly breaks the test.
These transformations were done by hand.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
We don't yet support this at the API level, so for now it returns
an error when trying to set multiple, but this makes sure that the
client types are already ready for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`ImageManifestDescriptor` will contain an OCI descriptor of
platform-specific manifest of the image that was picked when creating
the container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This code was added in 3b1d9f1a26 when the
logentries logging-driver was removed in v25.0.0. The logentries service
was already defunct, so unlikely to have any consumers, so let's remove
this code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
