I noticed that the daemon logs, even in debug, did not give any
indication that the build was cancelled. This patch adds some logs
to help debugging.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use fixtures for the JSON strings
- Add test-cases for invalid / malformed JSON
- Check error-message produced
- Add test for "happy path"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rewrite the debug-logs produced
- Use structured logs
- Combine into a single log per request, instead of separate log-entry
for the "form-data".
- Include error-messages returned by the handler ("error-response" field)
- Include HTTP status-code returned ("status" field)
- Include the "vars" as a field; these are fields extracted from the URL
and passed to the handler
Examples below are logs for:
docker ps
docker container inspect nosuchcontainer
docker volume create --name foo
Before this change:
DEBU[2024-10-16T10:59:40.484254465Z] Calling HEAD /_ping spanID=43d76043f8e30dbb traceID=04f980a33901f35ba33c3927d3bb4bbb
DEBU[2024-10-16T10:59:40.485551840Z] Calling GET /v1.47/containers/json spanID=b9979f2b36572a43 traceID=5c2167537df2dede6bdbab030f8350bc
DEBU[2024-10-16T11:00:00.374864502Z] Calling HEAD /_ping spanID=d637e39684d56a16 traceID=efaed7838901dd6a597c5446ce3f83e2
DEBU[2024-10-16T11:00:00.384198127Z] Calling GET /v1.47/containers/nosuchcontainer/json spanID=f9cc4520b95d814b traceID=c15ae04ca248929d6e52474e711d48b0
DEBU[2024-10-16T11:00:11.576426632Z] Calling HEAD /_ping spanID=2bc30d2be873a8e5 traceID=53ccc3d2af87aa5425421306906660a6
DEBU[2024-10-16T11:00:11.588877966Z] Calling POST /v1.47/volumes/create spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
DEBU[2024-10-16T11:00:11.589198966Z] form data: {"Driver":"local","Name":"foo"} spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
DEBU[2024-10-16T11:00:11.594828216Z] using regular volume spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
After this:
When using plain-text, we continue encoding the form-data to JSON, but
as it's now in a field, it'll be shown escaped;
DEBU[2024-10-16T11:17:35.465777379Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=9b7ea0288b2b70c3 status=200 traceID=94ef9345624e92ac0263931fbe9e15db vars="map[]"
DEBU[2024-10-16T11:17:35.468050171Z] handling GET request method=GET module=api request-url=/v1.47/containers/json spanID=04675edee7b5ec9d status=200 traceID=a9d81dcdbf2650fa6d794a7a856fb66b vars="map[version:1.47]"
DEBU[2024-10-16T11:17:38.502289297Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=7c43a8dfd8fb5043 status=200 traceID=7a2a7c71cd421570e811474749a04ccd vars="map[]"
DEBU[2024-10-16T11:17:38.504847506Z] handling GET request error-response="No such container: nosuchcontainer" method=GET module=api request-url=/v1.47/containers/nosuchcontainer/json spanID=ab721bbbe5cf8035 status=404 traceID=4a08dcd5054fc8090e3af8846beea10d vars="map[name:nosuchcontainer version:1.47]"
DEBU[2024-10-16T11:17:40.788838340Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=2dbc18ba1334635b status=200 traceID=ea9af681d096dc4a2c2f2ed7338ea417 vars="map[]"
DEBU[2024-10-16T11:17:40.790496465Z] handling POST request form-data="{\"Driver\":\"local\",\"Name\":\"foo\"}" method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"
The alternative to the above would be to unconditionally set it as-is,
but in that case it would use Go's formatting for `map[string]any`;
DEBU[2024-10-16T11:27:54.937232805Z] handling POST request form-data="map[Driver:local Name:foo]" method=POST module=api request-url=/v1.47/volumes/create spanID=2d7985a900791bf6 status=200 traceID=33feab9fd5feba3b0f4b6ec5a6971a67 vars="map[version:1.47]"
Or to use some trickery to not quote this specific field, but that may limit the
output from being parsable;
DEBU[2024-10-16T11:17:40.790496465Z] handling POST request form-data={"Driver":"local","Name":"foo"} method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"
When using `--log-format=json`, the form-data is kept as structured, becoming
part of the main JSON struct:
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"166dc12eeeadf82b","status":200,"time":"2024-10-16T11:16:09.427380423Z","traceID":"7f4f2501eee3b15ae608481ba214bd56","vars":{}}
{"level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/json","spanID":"bf95e2ce9eca41c2","status":200,"time":"2024-10-16T11:16:09.429077631Z","traceID":"041b26b30dacc240e8e3afc9c567195d","vars":{"version":"1.47"}}
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"454953906c36ea6b","status":200,"time":"2024-10-16T11:16:13.455633008Z","traceID":"3ffc0a256d6ec1a56cd7f6bf1008e55d","vars":{}}
{"error-response":"No such container: nosuchcontainer","level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/nosuchcontainer/json","spanID":"dcf0d42921928b29","status":404,"time":"2024-10-16T11:16:13.460309925Z","traceID":"fdfd2c89941c9c7a459bec7a05e46ef8","vars":{"name":"nosuchcontainer","version":"1.47"}}
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"701dc623cf1b0253","status":200,"time":"2024-10-16T11:16:16.155730884Z","traceID":"786885a9f79cbfba99097eeb4145ca1e","vars":{}}
{"form-data":{"Driver":"local","Name":"foo"},"level":"debug","method":"POST","module":"api","msg":"handling POST request","request-url":"/v1.47/volumes/create","spanID":"dc1429c1c636b30a","status":200,"time":"2024-10-16T11:16:16.162002426Z","traceID":"fc49ee4a7acafbbb8eb50ed34c434765","vars":{"version":"1.47"}}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon used to have various implementation to adjust the container-inspect
output for different API versions, which could return different go structs,
and because of that required a function with a `interface{}` output type.
Most of those adjustments have been removed, and we no longer need separate
types for backward compatibility with old API versions.
This patch;
- Removes the Daemon.ContainerInspectCurrent method
- Introduces a backend.ContainerInspectOptions struct
- Updates the Daemon.ContainerInspect method's signature to accept the above
- Moves API-version specific adjustments to api/server/router/container,
similar to how such adjustments are made for other endpoints.
Note that we should probably change the backend's signature further,
and define separate types for the backend's inspect and the API's
inspect response. Considering that the Backend signatures should be
considered "internal", we can do that in a future change.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's good practice to use a consistent name; we couldn't use `r` as name,
as it's used for the request argument, and `s` honestly didn't make much
sense as name, so changing it to `c`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in f143f4ec51,
which changed the minimum API version for "save" but forgot to update
the version for "load".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `Commit` type was introduced in 2790ac68b3,
to assist triaging issues that were reported with an incorrect version of
runc or containerd. At the time, both `runc` and `containerd` were not yet
stable, and had to be built from a specific commit to guarantee compatibility.
We encountered various situations where unexpected (and incompatible) versions
of those binaries were packaged, resulting in hard to trace bug-reports.
For those situations, a "expected" version was set at compile time, to
indicate if the version installed was different from the expected version;
docker info
...
runc version: a592beb5bc4c4092b1b1bac971afed27687340c5 (expected: 69663f0bd4b60df09991c08812a60108003fa340)
Both `runc` and `containerd` are stable now, and docker 19.03 and up set the
expected version to the actual version since c65f0bd13c
and 23.0 did the same for the `init` binary b585c64e2b,
to prevent the CLI from reporting "unexpected version".
In short; the `Expected` fields no longer serves a real purpose.
In future, we can even consider deprecating the `ContainerdCommit`, `RuncCommit`
and `InitCommit` fields on the `/info` response (as we also include this
information as part of the components returned in `/version`), but those
can still be useful currently for situations where a user only provides
`docker info` output.
This patch starts with deprecating the `Expected` field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `Platform` parameter that allows to select a specific platform to
save/load.
This is a breaking change to the Go client as it changes the signatures
of `ImageLoad` and `ImageSave`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Platform` parameter that allows to select a specific platform to
show the history for.
This is a breaking change to the Go client as it changes the signature
of `ImageHistory`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).
There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit 0071832226 introduced
per-endpoint sysctls, and migration to them from the top-level
'--sysctl' option.
The migration was intended to be short-term, disabled in the
next major release and code was added to check for the next
API version. But now, the API version will be bumped in a
minor release - this breaking change needs to wait until the
next major release, and we don't yet know the API version
number for that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
[buildkit@29b4b1a537][1] applied changes to `buildkitd` to set the correct
defaults, which should be 16MB, but used the library defaults. Without that
change, builds using large Dockerfiles would fail with a `ResourceExhausted`
error;
=> [internal] load build definition from Dockerfile
=> => transferring dockerfile: 896.44kB
ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (44865299 vs. 16777216)
However those changes were applied to the `buildkitd` code, which is the
daemon when running BuildKit standalone (or in a container through the
`container` driver). When running a build with the BuildKit builder compiled
into the Docker Engine, that code is not used, so the BuildKit changes did
not fix the issue.
This patch applies the same changes as were made in [buildkit@29b4b1a537][1]
to the gRPC endpoint provided by the dockerd daemon.
[1]: 29b4b1a537
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove a special `Details` parameter from the `GetImage` options and
extract its behavior to a `ImageInspect` method as it was only used by
the `/images/{name}/json` endpoint (`docker image inspect`).
This makes it easier for the containerd image service to output an image
inspect output without having to use the same data structures as the
graphdrivers.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Configuring CORS headers was deprecated in docker 27.0 through
7ea9acc97f, which disabled them by default
with a temporary `DOCKERD_DEPRECATED_CORS_HEADER` env-var to allow using
the option.
This patch removes the feature altogether; the flag is kept for one more
release to allow printing a more informative error, but can be removed in
the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `Container` type to the containere package, rename
it to `Summary`, and deprecates the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `ImageInspect` and `RootFS` types to the image package,
and deprecates the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `GraphDriverData` type is shared between images and containers, and
putting it in either package would result in a circular import, so adding
a new package for this type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.
Add a callback that is called by the exporter wrapper.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The execution-driver was replaced with containerd since docker 1.11 (API
v1.23) in 9c4570a958, after which the value
was no longer set. The field was left in the type definition.
Commit 1fb1136fec removed its use from the
CLI and [docker/engine-api@39c7d7e] removed it from the API type, followed
by an update to the API docs in 3c6ef4c29d.
Changes to the API types were not pulled into the engine until v1.13, and
probably because of that gated it on API version < 1.25 instead of < 1.24
(see 6d98e344c7); setting a "not supported"
value for older versions.
Based on the above; this field was deprecated in API v1.23, and empty
since then. Given that the minimum API version supported by the engine
is not v1.24, we can safely remove it.
[docker/engine-api@39c7d7e]: 39c7d7ec19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type is included in various types used in the API, but comes from
a separate module. The go-units module may be moving to the moby org,
and it is yet to be decided if the Ulimit type is a good fit for that
module (which deals with more generic units, such as "size" and "duration"
otherwise).
This patch introduces an alias to help during the transition of this type
to it's new location. The alias makes sure that existing code continues
to work (at least for now), but we need to start updating such code after
this PR is merged.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function returns the default network to use for the daemon platform;
moving this to a location separate from runconfig, which is planned to
be dismantled and moved to the API.
While it might be convenient to move this utility inside api/types/container,
we don't want to advertise this function too widely, as the default returned
can ONLY be considered correct when ran on the daemon-side. An alternative
would be to introduce an argument (daemonPlatform), which isn't very convenient
to use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This will be used in the next commit to test that changes are propagated
to the containerd store.
It is also just generally useful for debugging purposes.
- docs/api: update version history
- daemon: add fillContainerdInfo utility
- api: update swagger file with new types
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The commit endpoint accepts a container.Config, but uses the decoder to
unmarshal the request. The decoder uses a CreateRequest, which is a superset,
and also contains HostConfig and network.NetworkConfig. Those structs are
discarded in the router, but decoder.DecodeConfig also performs validation,
so a request containing those additional fields would result in a validation
error.
We should rewrite this code to only unmarshal what's expected.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The health-check start interval added in API v1.44, and the start
interval option is ignored when creating a Swarm service using an older
API version. However, due to an oversight, the option is not ignored
when older API clients _update_ a Swarm service. Fix this oversight by
moving the adjustment code into the adjustForAPIVersion function used by
both the createService and updateService handler functions.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Includes two commits from Arash Deshmeh:
add exec option to API TmpfsOptions and the related volume functions
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
feature: daemon handles tmpfs mounts exec option
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
Updated by Drew Erny
Signed-off-by: Drew Erny <derny@mirantis.com>
