The netfilter module is now loaded on-demand, and no longer during daemon
startup, making these fields obsolete. These fields are now always `false`
and will be removed in the next relase.
This patch deprecates:
- the `BridgeNfIptables` field in `api/types/system.Info`
- the `BridgeNfIp6tables` field in `api/types/system.Info`
- the `BridgeNFCallIPTablesDisabled` field in `pkg/sysinfo.SysInfo`
- the `BridgeNFCallIP6TablesDisabled` field in `pkg/sysinfo.SysInfo`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Historically, the `bridge` network-driver would detect whether netfiltering
was enabled in the kernel or, if disabled, try to do a `modprobe` when
initializing the driver. This approach became problematic, as loading the
module was not always performed at startup depending on daemon configuration,
or the daemon may have failed to load the module. The `/info` response
would include a warning to inform the user that some functionality may not
be available;
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Starting with db25b0dcd0, detecting whether
netfiltering is enabled now [happens when needed][1], which was further improved
on to not depend on `modprobe` in 264c15bfc4 and
4740820716.
Because of the above, the `/info` output would now return warnings in any
situation where netfiltering was not enabled on the host before the daemon
started, which may be either _incorrect_ (i.e., the module may have been
loaded afterwards), or irrelevant, because netfiltering is not needed in
all situations.
This patch removes the warnings from the `/info` response,
[1]: 944e403502/libnetwork/drivers/bridge/setup_bridgenetfiltering.go (L16-L77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Non-distributable artifacts (foreign layers) were introduced in commit
05bd04350b to accommodate Windows images,
for which the EULA did not allow layers to be distributed through registries
other than those hosted by Microsoft. The concept of foreign / non-distributable
layers was adopted by the OCI distribution spec in [oci#233].
These restrictions were relaxed later to allow distributing these images
through non-public registries, for which a configuration was added in the
daemon in 67fdf574d5. In 2022, Microsoft updated
the EULA and [removed these restrictions altogether][1], and the OCI distribution
spec deprecated the functionality in [oci#965].
In 2023, Microsoft [removed the use of foreign data layers][2] for their images,
making this functionality obsolete.
This patch:
- Deprecates the `--allow-nondistributable-artifacts` daemon flag and corresponding
`allow-nondistributable-artifacts` field in `daemon.json`. Setting either
option will no longer take an effect, but a deprecation warning log is added
to raise awareness about the deprecation. This warning is planned to become
an error in the next release.
- Deprecates the `RegistryConfig.AllowNondistributableArtifactsCIDRs` and
`RegistryConfig.AllowNondistributableArtifactsHostnames` fields in the
`GET /info` API response. For API version v1.48 and lower, the fields are
still included in the response, but always `null`. In API version v1.49 and
higher, the field will be omitted entirely.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsCIDRs`
field.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsHostnames`
field.
- Deprecates the `registry.ServiceOptions.AllowNondistributableArtifacts` field.
[oci#233]: https://github.com/opencontainers/image-spec/pull/233
[oci#965]: https://github.com/opencontainers/image-spec/pull/965
[1]: https://techcommunity.microsoft.com/blog/containers/announcing-windows-container-base-image-redistribution-rights-change/3645201
[2]: https://techcommunity.microsoft.com/blog/containers/announcing-removal-of-foreign-layers-from-windows-container-images/3846833
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pass the context that's used for logging, and add minimal handling of
context-cancellation. Also update logs to use structured fields.
Before this patch:
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
With this patch:
DEBU[2024-12-08T15:33:50.408445543Z] Registering route method=POST path="/networks/{id:.*}/disconnect"
DEBU[2024-12-08T15:33:50.408484335Z] Registering route method=POST path=/networks/prune
DEBU[2024-12-08T15:33:50.408505251Z] Registering route method=DELETE path="/networks/{id:.*}"
Or in JSON format:
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/connect","time":"2024-12-08T15:37:19.235209667Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/disconnect","time":"2024-12-08T15:37:19.235243001Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/prune","time":"2024-12-08T15:37:19.235290876Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that debug-routes are identical to regular routers, we can pass them
the same as those routers. With this, the daemon also logs those routes
as part of its startup (when in debug mode).
Before this patch, only non-debug endpoints would be logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
With this patch, debug endpoints are also logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
DEBU[2024-12-08T15:24:47.324486834Z] Registering GET, /debug/vars
DEBU[2024-12-08T15:24:47.324506751Z] Registering GET, /debug/pprof/
DEBU[2024-12-08T15:24:47.324532126Z] Registering GET, /debug/pprof/cmdline
DEBU[2024-12-08T15:24:47.324549293Z] Registering GET, /debug/pprof/profile
DEBU[2024-12-08T15:24:47.324564501Z] Registering GET, /debug/pprof/symbol
DEBU[2024-12-08T15:24:47.324582043Z] Registering GET, /debug/pprof/trace
DEBU[2024-12-08T15:24:47.324604751Z] Registering GET, /debug/pprof/{name}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug endpoints are currently only provided non-versioned (e.g. `/debug/vars`).
While this is convenient, we "officially" deprecated non-versioned endpoints
in the API.
This patch also registers the debug-endpoints under the API-versioned paths,
so that they can be used either without version ("latest") and versioned
paths.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug handlers were created for GET methods, but were registered for
any method;
curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
After this patch, they're only registered with the intended method, and a
404 is returned for incorrect ones;
curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars
{"message":"page not found"}
curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars
{"message":"page not found"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go automatically canonicalises HTTP headers, meaning the string `API-Version` passed as a header has always been returned as `Api-Version`. Similarly, `OSType` is returned as `Ostype`.
This commit updates the documentation to reflect this behaviour and modifies the codebase to ensure that input strings are aligned with their canonical output values.
Signed-off-by: maggie44 <64841595+maggie44@users.noreply.github.com>
This field was introduced in 5b752fab32,
which added documentation in the API documentation (swagger), but did
not document the field in the API go types.
This patch adds documentation, using a variant of the description used
in swagger.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This new field is used by libnetwork to determine which endpoint
provides the default gateway for a container.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
There were a handful of direct checks against errors.Is that can be
translated to assert.ErrorIs without too much thought. Unfortunately
there are a load of other examples where ErrorIs probably makes sense
but would require testing whether this subtly breaks the test.
These transformations were done by hand.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
We don't yet support this at the API level, so for now it returns
an error when trying to set multiple, but this makes sure that the
client types are already ready for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`ImageManifestDescriptor` will contain an OCI descriptor of
platform-specific manifest of the image that was picked when creating
the container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This code was added in 3b1d9f1a26 when the
logentries logging-driver was removed in v25.0.0. The logentries service
was already defunct, so unlikely to have any consumers, so let's remove
this code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in 3f2e9da0100af2ceb3ef0d6431cb2b27dc3e1e49;
make BIND_DIR=. shell
make -C ./internal/gocompat/
GO111MODULE=on go test -v
# github.com/docker/docker/api/server/router/container
../../api/server/router/container/inspect.go:29:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL gocompat [build failed]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While the endpoint returns a detailed information about its children, it
doesn't actually expose the descriptor of the root OCI index/manifest
list.
This commits adds the target description to the returned JSON.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
api/server/middleware/version_test.go:59:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/container/hostconfig_test.go:94:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/filters/parse_test.go:543:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/network/endpoint_test.go:84:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
api/types/network/endpoint_test.go:166:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
api/types/network/ipam_test.go:127:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/server/httputils/form_test.go:152:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.
go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
commit 77fe35b3b9 added a warning if the
"volumeDriver" option was used in combination with --mount.
That change put the warning in the CLI, which should not have to handle
this logic. Let's move it to the daemon as we have a warnings response
in the container create response.
The warning is gated by API version, so that older clients can continue
to produce the warning client-side.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We already check if
- the key is set (otherwise default)
- a value is set (otherwise default and error)
This check can be simplified to check if they're equal (boolean cannot be both
true and false), or both false (boolean must be either true or false), although
the latter could be considered for a tri-state boolean (but we already do this
through the "not set" case).
We may need some additional checks, for example, currently it ignores invalid
values if the filter contains at least one valid one (e.g. ["true", "bananas"]).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use fixtures for the JSON strings
- Add test-cases for invalid / malformed JSON
- Check error-message produced
- Add test for "happy path"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rewrite the debug-logs produced
- Use structured logs
- Combine into a single log per request, instead of separate log-entry
for the "form-data".
- Include error-messages returned by the handler ("error-response" field)
- Include HTTP status-code returned ("status" field)
- Include the "vars" as a field; these are fields extracted from the URL
and passed to the handler
Examples below are logs for:
docker ps
docker container inspect nosuchcontainer
docker volume create --name foo
Before this change:
DEBU[2024-10-16T10:59:40.484254465Z] Calling HEAD /_ping spanID=43d76043f8e30dbb traceID=04f980a33901f35ba33c3927d3bb4bbb
DEBU[2024-10-16T10:59:40.485551840Z] Calling GET /v1.47/containers/json spanID=b9979f2b36572a43 traceID=5c2167537df2dede6bdbab030f8350bc
DEBU[2024-10-16T11:00:00.374864502Z] Calling HEAD /_ping spanID=d637e39684d56a16 traceID=efaed7838901dd6a597c5446ce3f83e2
DEBU[2024-10-16T11:00:00.384198127Z] Calling GET /v1.47/containers/nosuchcontainer/json spanID=f9cc4520b95d814b traceID=c15ae04ca248929d6e52474e711d48b0
DEBU[2024-10-16T11:00:11.576426632Z] Calling HEAD /_ping spanID=2bc30d2be873a8e5 traceID=53ccc3d2af87aa5425421306906660a6
DEBU[2024-10-16T11:00:11.588877966Z] Calling POST /v1.47/volumes/create spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
DEBU[2024-10-16T11:00:11.589198966Z] form data: {"Driver":"local","Name":"foo"} spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
DEBU[2024-10-16T11:00:11.594828216Z] using regular volume spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
After this:
When using plain-text, we continue encoding the form-data to JSON, but
as it's now in a field, it'll be shown escaped;
DEBU[2024-10-16T11:17:35.465777379Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=9b7ea0288b2b70c3 status=200 traceID=94ef9345624e92ac0263931fbe9e15db vars="map[]"
DEBU[2024-10-16T11:17:35.468050171Z] handling GET request method=GET module=api request-url=/v1.47/containers/json spanID=04675edee7b5ec9d status=200 traceID=a9d81dcdbf2650fa6d794a7a856fb66b vars="map[version:1.47]"
DEBU[2024-10-16T11:17:38.502289297Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=7c43a8dfd8fb5043 status=200 traceID=7a2a7c71cd421570e811474749a04ccd vars="map[]"
DEBU[2024-10-16T11:17:38.504847506Z] handling GET request error-response="No such container: nosuchcontainer" method=GET module=api request-url=/v1.47/containers/nosuchcontainer/json spanID=ab721bbbe5cf8035 status=404 traceID=4a08dcd5054fc8090e3af8846beea10d vars="map[name:nosuchcontainer version:1.47]"
DEBU[2024-10-16T11:17:40.788838340Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=2dbc18ba1334635b status=200 traceID=ea9af681d096dc4a2c2f2ed7338ea417 vars="map[]"
DEBU[2024-10-16T11:17:40.790496465Z] handling POST request form-data="{\"Driver\":\"local\",\"Name\":\"foo\"}" method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"
The alternative to the above would be to unconditionally set it as-is,
but in that case it would use Go's formatting for `map[string]any`;
DEBU[2024-10-16T11:27:54.937232805Z] handling POST request form-data="map[Driver:local Name:foo]" method=POST module=api request-url=/v1.47/volumes/create spanID=2d7985a900791bf6 status=200 traceID=33feab9fd5feba3b0f4b6ec5a6971a67 vars="map[version:1.47]"
Or to use some trickery to not quote this specific field, but that may limit the
output from being parsable;
DEBU[2024-10-16T11:17:40.790496465Z] handling POST request form-data={"Driver":"local","Name":"foo"} method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"
When using `--log-format=json`, the form-data is kept as structured, becoming
part of the main JSON struct:
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"166dc12eeeadf82b","status":200,"time":"2024-10-16T11:16:09.427380423Z","traceID":"7f4f2501eee3b15ae608481ba214bd56","vars":{}}
{"level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/json","spanID":"bf95e2ce9eca41c2","status":200,"time":"2024-10-16T11:16:09.429077631Z","traceID":"041b26b30dacc240e8e3afc9c567195d","vars":{"version":"1.47"}}
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"454953906c36ea6b","status":200,"time":"2024-10-16T11:16:13.455633008Z","traceID":"3ffc0a256d6ec1a56cd7f6bf1008e55d","vars":{}}
{"error-response":"No such container: nosuchcontainer","level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/nosuchcontainer/json","spanID":"dcf0d42921928b29","status":404,"time":"2024-10-16T11:16:13.460309925Z","traceID":"fdfd2c89941c9c7a459bec7a05e46ef8","vars":{"name":"nosuchcontainer","version":"1.47"}}
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"701dc623cf1b0253","status":200,"time":"2024-10-16T11:16:16.155730884Z","traceID":"786885a9f79cbfba99097eeb4145ca1e","vars":{}}
{"form-data":{"Driver":"local","Name":"foo"},"level":"debug","method":"POST","module":"api","msg":"handling POST request","request-url":"/v1.47/volumes/create","spanID":"dc1429c1c636b30a","status":200,"time":"2024-10-16T11:16:16.162002426Z","traceID":"fc49ee4a7acafbbb8eb50ed34c434765","vars":{"version":"1.47"}}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon used to have various implementation to adjust the container-inspect
output for different API versions, which could return different go structs,
and because of that required a function with a `interface{}` output type.
Most of those adjustments have been removed, and we no longer need separate
types for backward compatibility with old API versions.
This patch;
- Removes the Daemon.ContainerInspectCurrent method
- Introduces a backend.ContainerInspectOptions struct
- Updates the Daemon.ContainerInspect method's signature to accept the above
- Moves API-version specific adjustments to api/server/router/container,
similar to how such adjustments are made for other endpoints.
Note that we should probably change the backend's signature further,
and define separate types for the backend's inspect and the API's
inspect response. Considering that the Backend signatures should be
considered "internal", we can do that in a future change.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
