201 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
fe01a5fc30 api/server/router: fix non-constant format string in call (govet)
api/server/router/container/container_routes.go:943:22: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
                                  ^
    api/server/router/image/image_routes.go:144:50: printf: non-constant format string in call to github.com/docker/docker/pkg/streamformatter.FormatStatus (govet)
                output.Write(streamformatter.FormatStatus("", id.String()))
                                                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fd3a53c12)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Rob Murray
8f12906274 Migrate per-endpoint sysctls until 28.0.0
Commit 0071832226 introduced
per-endpoint sysctls, and migration to them from the top-level
'--sysctl' option.

The migration was intended to be short-term, disabled in the
next major release and code was added to check for the next
API version. But now, the API version will be bumped in a
minor release - this breaking change needs to wait until the
next major release, and we don't yet know the API version
number for that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 17adc1478b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:29:19 +02:00
Sebastiaan van Stijn
7b31435cf8 Migrate to github.com/containerd/platforms module
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.

The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d0aa3eaccf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:52:58 +02:00
Sebastiaan van Stijn
8e91b64e07 runconfig: deprecate DefaultDaemonNetworkMode, move to daemon/network
This function returns the default network to use for the daemon platform;
moving this to a location separate from runconfig, which is planned to
be dismantled and moved to the API.

While it might be convenient to move this utility inside api/types/container,
we don't want to advertise this function too widely, as the default returned
can ONLY be considered correct when ran on the daemon-side. An alternative
would be to introduce an argument (daemonPlatform), which isn't very convenient
to use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 17:32:56 +02:00
Sebastiaan van Stijn
b2441c7419 Merge pull request #45052 from cpuguy83/attach_fd_leak
Fix attach goroutine/fd leak when no I/O is ready
2024-06-17 13:16:08 +02:00
Sebastiaan van Stijn
4c7228663c api/server/router/container.postCommit: add TODO about use of CreateRequest
The commit endpoint accepts a container.Config, but uses the decoder to
unmarshal the request. The decoder uses a CreateRequest, which is a superset,
and also contains HostConfig and network.NetworkConfig. Those structs are
discarded in the router, but decoder.DecodeConfig also performs validation,
so a request containing those additional fields would result in a validation
error.

We should rewrite this code to only unmarshal what's expected.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:46 +02:00
Albin Kerouanton
8dcded102e libnet: add OTel spans to CreateEndpoint
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:45:31 +02:00
Sebastiaan van Stijn
47d7c9e31d api/types: move ContainerPathStat to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:20:45 +02:00
Sebastiaan van Stijn
db2f1acd5d api/types: move ContainersPruneReport to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:47 +02:00
Sebastiaan van Stijn
d91638e295 api/types: move ExecStartCheck to api/types/container
This moves the type to api/types/container and creates an alias for
exec attach; ContainerExecAttach currently uses the same type as
ContainerExecStart, but does not all the same options (and some
options cannot be used).

We need to split the actual types, but lets start with aliasing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Sebastiaan van Stijn
452e134001 api/types: move ExecStartOptions to api/types/backend
It's a type used by the backend, so moving it there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Sebastiaan van Stijn
cd76e3e7f8 api/types: move ExecConfig to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Sebastiaan van Stijn
75843620a6 api/server/router/container: minor nits in exec router
- remove intermediate variable
- format a "todo" comment as an actual todo ':)
- explicitly suppress some unhandled errors to keep linters happy

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 20:54:38 +02:00
cncal
ca0529f984 api/server: ContainerList returns container annotations
Allow clients (e.g. cri-dockerd) to fetch container annotations in
ContainerList api.

Signed-off-by: cncal <flycalvin@qq.com>
2024-06-07 09:50:11 +08:00
Albin Kerouanton
2ebf19129f Merge pull request #47686 from robmry/47639_per-interface-sysctls
Per-interface sysctls
2024-05-29 10:54:05 +02:00
Rob Murray
0071832226 Add per-endpoint sysctls to DriverOpts
Until now it's been possible to set per-interface sysctls using, for
example, '--sysctl net.ipv6.conf.eth0.accept_ra=2'. But, the index in
the interface name is allocated serially, and the numbering in a container
with more than one interface may change when a container is restarted.
The change to make it possible to connect a container to more than one
network when it's created increased the ambiguity.

This change adds label "com.docker.network.endpoint.sysctls" to the
DriverOpts in EndpointSettings. This option is explicitly associated
with the interface.

Settings in "--sysctl" for "eth0" are migrated to DriverOpts.

Because using "--sysctl" with any interface apart from "eth0" would have
unpredictable results, it is now an error to use any other interface name
in the top level "--sysctl" option. The error message includes a hint at
how to use the new per-interface setting.

The per-endpoint sysctl name has the interface name replaced by
"IFNAME". For example:
    net.ipv6.conf.eth0.accept_ra=2
becomes:
    net.ipv6.conf.IFNAME.accept_ra=2

The value of DriverOpts["com.docker.network.endpoint.sysctls"] is a
comma separated list.

Settings from '--sysctl' are applied by the runtime lib during task
creation. So, task creation fails if the endpoint does not exist.
Applying per-endpoint settings during interface configuration means the
endpoint can be created later, which paves the way for removal of the
SetKey OCI prestart hook.

Unlike other DriverOpts, the sysctl label itself is not driver-specific,
but each driver has a chance to check settings/values and raise an error
if a setting would cause it a problem - no such checks have been added
in this initial version. As a future extension, if required, it would be
possible for the driver to echo back valid/extended/modified settings to
libnetwork for it to apply to the interface. (At that point, the syntax
for the options could become driver specific to allow, for example, a
driver to create more than one interface).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-29 08:59:48 +01:00
Brian Goff
2d134c5abd Fix goroutine/fd leak when client disconnects
In cases where the client disconnects and there is nothing to read from
a stdio stream after that disconnect, the copy goroutines and file
descriptors are leaked because `io.Copy` is just blocked waiting for
data from the container's I/O stream.

This fix only applies to Linux.
Windows will need a separate fix.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-24 21:32:15 +00:00
Rob Murray
a35716f5b9 Factor out selection of endpoint for config migration
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-17 11:30:14 +01:00
Brian Goff
707ab48cbb Explicity write http headers on streaming endpoints
This works around issues with the otel http handler wrapper causing
multiple calls to `WriteHeader` when a `Flush` is called before `Write`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-16 18:00:02 +00:00
Albin Kerouanton
c4689034fd daemon: don't call NetworkMode.IsDefault()
Previous commit made this unnecessary.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-28 12:35:47 +01:00
Albin Kerouanton
4eed3dcdfe api: normalize the default NetworkMode
The NetworkMode "default" is now normalized into the value it
aliases ("bridge" on Linux and "nat" on Windows) by the
ContainerCreate endpoint, the legacy image builder, Swarm's
cluster executor and by the container restore codepath.

builder-next is left untouched as it already uses the normalized
value (ie. bridge).

Going forward, this will make maintenance easier as there's one
less NetworkMode to care about.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-28 12:34:23 +01:00
Albin Kerouanton
7c7e453255 Merge pull request #47474 from robmry/47441_mac_addr_config_migration
Don't create endpoint config for MAC addr config migration
2024-03-06 11:04:17 +01:00
Rob Murray
a580544d82 Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-29 17:02:19 +00:00
Paweł Gronowski
432390320e api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 11:37:30 +01:00
Sebastiaan van Stijn
8758d08bb4 api: remove handling of HostConfig on POST /containers/{id}/start (api < v1.24)
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
b3a0ff9944 api: remove POST /containers/{id}/copy endpoint (api < v1.23)
This endpoint was deprecated in API v1.20 (Docker Engine v1.8.0) in
commit db9cc91a9e, in favor of the
`PUT /containers/{id}/archive` and `HEAD /containers/{id}/archive`
endpoints, and disabled in API v1.24 (Docker Engine v1.12.0) through
commit 428328908d.

This patch removes the endpoint, and the associated `daemon.ContainerCopy`
method in the backend.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
83f790cccc api: POST /exec/{id}/start: remove support for API < v1.21
API v1.21 (Docker Engine v1.9.0) enforces the request to have a JSON
content-type on exec start (see 45dc57f229).
An exception was added in 0b5e628e14 to
make this check conditional (supporting API < 1.21).

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
d1974aa492 api: remove code for container stats on api < v1.21
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.20 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
dfdf2adf0c api: POST /containers/{id}/kill: remove handling for api < 1.20
API v1.20 and up produces an error when signalling / killing a non-running
container (see c92377e300). Older API versions
allowed this, and an exception was added in 621e3d8587.

API v1.23 and older are deprecated, so we can remove this handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:42 +01:00
Sebastiaan van Stijn
2970b320aa api: remove code for adjusting CPU shares (api < v1.19)
API versions before 1.19 allowed CpuShares that were greater than the maximum
or less than the minimum supported by the kernel, and relied on the kernel to
do the right thing.

Commit ed39fbeb2a introduced code to adjust the
CPU shares to be within the accepted range when using API version 1.18 or
lower.

API v1.23 and older are deprecated, so we can remove support for this
functionality.

Currently, there's no validation for CPU shares to be within an acceptable
range; a TODO was added to add validation for this option, and to use the
`linuxMinCPUShares` and `linuxMaxCPUShares` consts for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:33 +01:00
Sebastiaan van Stijn
1b1147e46b api: POST /commit: remove version-gate for "pause" (api < v1.16)
The "pause" flag was added in API v1.13 (Docker Engine v1.1.0), and is
enabled by default (see 17d870bed5).

API v1.23 and older are deprecated, so we can remove the version-gate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Paweł Gronowski
bfb810445c volumes: Implement subpath mount
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume
directory.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:10 +01:00
Sebastiaan van Stijn
484e6b784c api/types: move ContainerCreateConfig, ContainerRmConfig to api/types/backend
The `ContainerCreateConfig` and `ContainerRmConfig` structs are used for
options to be passed to the backend, and are not used in client code.

Thess struct currently is intended for internal use only (for example, the
`AdjustCPUShares` is an internal implementation details to adjust the container's
config when older API versions are used).

Somewhat ironically, the signature of the Backend has a nicer UX than that
of the client's `ContainerCreate` signature (which expects all options to
be passed as separate arguments), so we may want to update that signature
to be closer to what the backend is using, but that can be left as a future
exercise.

This patch moves the `ContainerCreateConfig` and `ContainerRmConfig` structs
to the backend package to prevent it being imported in the client, and to make
it more clear that this is part of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 16:41:36 +01:00
Albin Kerouanton
ee9f0ed895 api: Deprecate ContainerConfig.MacAddress
Having a sandbox/container-wide MacAddress field makes little sense
since a container can be connected to multiple networks at the same
time. This field is an artefact of old times where a container could be
connected to a single network only.

As we now have a way to specify per-endpoint mac address, this field is
now deprecated.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-25 22:55:59 +02:00
Albin Kerouanton
4f0cab3b74 api: ContainerCreate: re-organize BC conditions
- Merge BC conds for API < v1.42 together
- Merge BC conds for API < v1.44 together
- Re-order BC conds by API version
- Move pids-limit normalization after BC conds

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:18 +02:00
Albin Kerouanton
79589354df api: ContainerCreate: remove repetitive nil checks
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:17 +02:00
Albin Kerouanton
91eee33f62 api: ContainerCreate: return an error when config is nil
The same error is already returned by `(*Daemon).containerCreate()` but
since this function is also called by the cluster executor, the error
has to be duplicated.

Doing that allows to remove a nil check on container config in
`postContainersCreate`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:17 +02:00
Albin Kerouanton
a73dfe68d3 api: ContainerCreate: init hostConfig and networkingConfig when nil
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 15:40:40 +02:00
Albin Kerouanton
70158284b7 api: fix a nil check on the wrong var
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 09:40:18 +02:00
Sebastiaan van Stijn
ebef4efb88 api/types: move ContainerLogsOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:30:12 +02:00
Sebastiaan van Stijn
9670d9364d api/types: move ContainerListOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:24 +02:00
Sebastiaan van Stijn
cff4f20c44 migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Albin Kerouanton
5e15ed314b api: Improve error on ContainerCreate with multiple endpoints
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Albin Kerouanton
bbcd662532 api: Allow ContainerCreate to take several EndpointsConfig for >= 1.44
The API endpoint `/containers/create` accepts several EndpointsConfig
since v1.22 but the daemon would error out in such case. This check is
moved from the daemon to the api and is now applied only for API < 1.44,
effectively allowing the daemon to create containers connected to
several networks.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 10:07:29 +02:00
Sebastiaan van Stijn
d52f20048d api/server: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 18:00:19 +02:00
Brian Goff
2216d3ca8d Add health start interval
This adds an additional interval to be used by healthchecks during the
start period.
Typically when a container is just starting you want to check if it is
ready more quickly than a typical healthcheck might run. Without this
users have to balance between running healthchecks to frequently vs
taking a very long time to mark a container as healthy for the first
time.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:44:17 +00:00
Sebastiaan van Stijn
6592646632 api/server: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Akihiro Suda
5045a2de24 Support recursively read-only (RRO) mounts
`docker run -v /foo:/foo:ro` is now recursively read-only on kernel >= 5.12.

Automatically falls back to the legacy non-recursively read-only mount mode on kernel < 5.12.

Use `ro-non-recursive` to disable RRO.
Use `ro-force-recursive` or `rro` to explicitly enable RRO. (Fails on kernel < 5.12)

Fix issue 44978
Fix docker/for-linux issue 788

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-26 01:58:24 +09:00
Cory Snider
967c7bc5d3 api/server: allow empty body for POST /commit again
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-17 16:01:11 -04:00