The test suite is executed via "env -i" with an explicit allow-list of
environment variables, so DBUS_SESSION_BUS_ADDRESS (exported by
.integration-daemon-start in rootless-systemd mode) never reached the
test binaries, nor the daemons they spawn via internal/testutil/daemon
("sudo --preserve-env"). Without it, runc inside those daemons cannot
detect the systemd user manager ("busctl --user" fails with the
per-daemon XDG_RUNTIME_DIR), assumes a rootful systemd, and fails to
create the container scope on the system bus:
unable to start unit "docker-<id>.scope" ...: Permission denied
Also skip TestRunWithAlternativeContainerdShim in rootless-systemd
mode: the daemon only passes the SystemdCgroup option to the stock
runc runtime, so with an ad-hoc shim name runc falls back to the
cgroupfs driver and treats the systemd-style cgroupsPath as a literal
path, which cannot be created by a rootless daemon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Previously, the rootless-systemd CI mode failed with:
open /sys/fs/cgroup/user.slice/user-1000.slice/cgroup.controllers: no such file or directory
on every container start. The daemon auto-selects the systemd cgroup
driver (cgroup v2 + systemd detected), but the test harness launched
dockerd-rootless.sh via plain "sudo" with a fabricated XDG_RUNTIME_DIR,
so there was no systemd user session at all: no user-${uid}.slice, no
user@${uid}.service, and no user D-Bus socket for runc to talk to.
withRootless() (daemon/oci_linux.go) then failed reading the controllers
file at every container start.
Set up the environment the way it is documented for production rootless
installations instead:
- hack/make/.integration-daemon-start: when systemd is running, enable
lingering for unprivilegeduser (which starts user@${uid}.service),
use /run/user/${uid} as XDG_RUNTIME_DIR, and export
DBUS_SESSION_BUS_ADDRESS so that runc can find the systemd user
manager. The variable also propagates to the daemons spawned by the
test suite (sudo --preserve-env in internal/testutil/daemon), whose
XDG_RUNTIME_DIR points to a non-standard location.
- hack/dind-systemd: delegate the cpu/cpuset/io/memory/pids cgroup
controllers to unprivileged users, matching
https://docs.docker.com/engine/security/rootless/#limiting-resources
Fixes the "test (amd64, *, rootless-systemd)" CI failures tracked in
issue 44084.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
gotestsum enables color output for known CI environments by checking
whether CI is set first, then looking for a vendor-specific variable
(GITHUB_ACTIONS, GITLAB_CI, etc.).
GITHUB_ACTIONS was already forwarded into the test containers, but CI
was not, so the check was never reached and gotestsum fell back to
isatty, which returns no-color for the non-TTY pipes used inside Docker.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
set_repeat_timeout is called inside run_test_integration, which is
itself invoked once per iteration by the repeat loop (TEST_REPEAT
times). Each call multiplied TIMEOUT by TEST_REPEAT again, so with
TEST_REPEAT=10 and TIMEOUT=10m the progression was:
iter 1: 100m, iter 2: 1000m, ..., iter 8: 1000000000m
Go's time.ParseDuration overflows on 1000000000m (≈ 6×10²² ns >
int64 max), producing:
invalid value "1000000000m" for flag -test.timeout: parse error
The multiplier (TEST_REPEAT) and the scaled value (100m) are both
correct; the problem is repeated application. Fix by moving the single
set_repeat_timeout call out of run_test_integration and into
hack/make/test-integration, directly before the repeat loop. That way
TIMEOUT is scaled exactly once regardless of how many iterations the
loop runs.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Allows PR authors to stress-test specific integration tests for
flakiness even when the diff doesn't touch them.
Add a /flaky-check directive on its own line in the PR body:
/flaky-check=TestFoo,TestBar
The GHA integration-flaky job parses the directive from the PR body and
exports the names as FLAKY_EXTRA_TESTS, which the script appends to the
diff-detected set before running the stress loop.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Also run `test-integration-flaky` for changed existing tests.
Keep the old added-test detection as-is, and add modified-test detection
on top of it to avoid regressing the existing behavior.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The test was skipped because Container.run() returned stale container
state after 7bc56c5365 changed when the
short ID was added to Aliases. That behaviour is now stable and the
test passes against the current docker-py commit (059d371).
Signed-off-by: Rajnish Kumar <rajnishkumar448@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Split the diff from the grep in the bundle so a failed diff is no longer
swallowed and misreported as "no new tests".
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- slirp4netns/vpnkit is no longer needed as gvisor-tap-vsock is now embedded in RootlessKit.
slirp4netns/vpnkit is still used when installed.
- The `builtin` port driver can now correctly propagate the source IP, when
`userland-proxy` is disabled.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This option was added in 9672afa339 in 2016,
at which time tweaking GOGC showed a significant difference when building
the binary (`1m14.221s` vs `0m49.929s`).
Testing this Today doesn't show a real difference;
With GOGC=1000:
rm -rf bundles/
time hack/make.sh binary
real 0m15.303s
user 0m26.285s
sys 0m6.383s
real 0m4.931s
user 0m3.103s
sys 0m1.670s
real 0m5.693s
user 0m3.827s
sys 0m2.384s
Without GOGC set;
real 0m5.012s
user 0m3.689s
sys 0m1.549s
real 0m5.298s
user 0m3.997s
sys 0m1.695s
real 0m4.899s
user 0m3.579s
sys 0m1.423s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 081987b647 updated the Dockerfile
to disable go's telemetry in our dev / test-environment; as collecting
this data doesn't serve much purpose.
However, the configuration to disable telemetry is tied to the user's
home-directory (HOME); and disabling telemetry writs a config-file to the
user's home-directory (`~/.config/go/telemetry/mode`). While go provides
env-vars about the mode and location (`GOTELEMETRY` and `GOTELEMETRYDIR`),
those env-vars are read-only and cannot be used to either disable telemetry
or to set the location;
> Information about telemetry configuration is also available via read-only Go environment variables:
>
> go env GOTELEMETRY reports the telemetry mode.
> go env GOTELEMETRYDIR reports the directory holding telemetry configuration and data.
Some steps in our CI set up a different home-directory, which is not configured
to disable telemetry, which means that CI currently leaves behind a bunch of
files related to this;
make TEST_FILTER=TestPruneDontDeleteUsedDangling test-integration
tree -a bundles/test-integration/fake-HOME/
bundles/test-integration/fake-HOME/
└── .config
└── go
└── telemetry
├── local
│ ├── asm@go1.25.3-go1.25.3-linux-arm64-2025-10-21.v1.count
│ ├── compile@go1.25.3-go1.25.3-linux-arm64-2025-10-21.v1.count
│ ├── go@go1.25.3-go1.25.3-linux-arm64-2025-10-21.v1.count
│ ├── link@go1.25.3-go1.25.3-linux-arm64-2025-10-21.v1.count
│ ├── test2json@go1.25.3-go1.25.3-linux-arm64-2025-10-21.v1.count
│ ├── upload.token
│ └── weekends
└── upload
6 directories, 7 files
This patch disables go telemetry also for this home-directory to prevent
those files from being created, and to prevent go from producing the
telemetry.
With this patch, only the file to disable telemetry is produced:
make TEST_FILTER=TestPruneDontDeleteUsedDangling test-integration
tree -a bundles/test-integration/fake-HOME/
bundles/test-integration/fake-HOME/
└── .config
└── go
└── telemetry
└── mode
4 directories, 1 file
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- relates to 96b29f5a1f
- similar to 08e4e88482
The daemon currently provides support for API versions all the way back
to v1.24, which is the version of the API that shipped with docker 1.12.0
(released in 2016).
Such old versions of the client are rare, and supporting older API versions
has accumulated significant amounts of code to remain backward-compatible
(which is largely untested, and a "best-effort" at most).
This patch updates the minimum API version to v1.44, matching the minimum
version of the client, and matching the API version of docker v25.0, which
is the oldest supported version (through Mirantis MCR).
The intent is to start deprecating older API versions when daemons implementing
them reach EOL. This patch does not yet remove backward-compatibility code
for older API versions, and the DOCKER_MIN_API_VERSION environment variable
allows overriding the minimum version (to allow restoring the behavior from
before this patch), however, API versions below v1.44 should be considered
"best effort", and we may remove compatibility code to provide "degraded"
support.
With this patch the daemon defaults to API v1.44 as minimum:
docker version
Client:
Version: 28.5.0
API version: 1.51
Go version: go1.24.7
Git commit: 887030f
Built: Thu Oct 2 14:54:39 2025
OS/Arch: linux/arm64
Context: default
Server:
Engine:
Version: dev
API version: 1.52 (minimum version 1.44)
....
Trying to use an older version of the API produces an error:
DOCKER_API_VERSION=1.43 docker version
Client:
Version: 28.5.0
API version: 1.43 (downgraded from 1.51)
Go version: go1.24.7
Git commit: 887030f
Built: Thu Oct 2 14:54:39 2025
OS/Arch: linux/arm64
Context: default
Error response from daemon: client version 1.43 is too old. Minimum supported API version is 1.44, please upgrade your client to a newer version
To restore the previous minimum, users can start the daemon with the
DOCKER_MIN_API_VERSION environment variable set:
DOCKER_MIN_API_VERSION=1.24 dockerd
API 1.24 is the oldest supported API version;
docker version
Client:
Version: 28.5.0
API version: 1.24 (downgraded from 1.51)
Go version: go1.24.7
Git commit: 887030f
Built: Thu Oct 2 14:54:39 2025
OS/Arch: linux/arm64
Context: default
Server:
Engine:
Version: dev
API version: 1.52 (minimum version 1.24)
....
When using the `DOCKER_MIN_API_VERSION` with a version of the API that
is not supported, an error is produced when starting the daemon;
DOCKER_MIN_API_VERSION=1.23 dockerd --validate
invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.24: 1.23
DOCKER_MIN_API_VERSION=1.99 dockerd --validate
invalid DOCKER_MIN_API_VERSION: maximum supported API version is 1.52: 1.99
Specifying a malformed API version also produces the same error;
DOCKER_MIN_API_VERSION=hello dockerd --validate
invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.24: hello
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Complete the removal of the deprecated network structs by dropping the
remaining references in daemon code.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The daemon started by the test-integration script needs to run without
firewalld integration to make sure that daemons started by networking
tests will handle firewalld reload without any interference (i.e.
without another daemon racing against them to recreate the iptables
chains).
Most tests are already running their own daemons, but the few that don't
and need firewalld integration are updated to start their own.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The environment variables set by legacy links are not particularly
useful because you need to know the name of the linked container to use
them, or you need to scan all enviornment variables to find them.
Legacy links are deprecated / marked "legacy" since a long time, and we
want to replace them with non-legacy links. This will help make the
default bridge work like custom networks.
For now, stop setting these environment variables inside of linking
containers by default, but provide an escape hatch to allow users who
still rely on these to re-enable them.
The integration-cli tests `TestExecEnvLinksHost` and `TestLinksEnvs` are
removed as they need to run against a daemon with legacy links env vars
enabled, and a new integration test`TestLegacyLinksEnvVars` is added to
fill the gap. Similarly, the docker-py test `test_create_with_links` is
skipped.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
For nftables only, never enable IP forwarding on the host. Instead,
return an error on network creation if forwarding is not enabled,
required by a bridge network, and --ip-forward=true.
If IPv4 forwarding is not enabled when the daemon is started with
nftables enabled and other config at defaults, the daemon will
exit when it tries to create the default bridge.
Otherwise, network creation will fail with an error if IPv4/IPv6
forwarding is not enabled when a network is created with IPv4/IPv6.
It's the user's responsibility to configure and secure their host
when they run Docker with nftables.
Signed-off-by: Rob Murray <rob.murray@docker.com>
commit af0cdc36c7 marked these fields as
deprecated and to be removed in API v1.47 (which was targeted for v28.0).
We shipped v1.47 with the v27.2 release, but did not yet remove the erroneous
fields, so the version to deprecate was updated to v1.48 through
3df03d8e66
This patch removes fields that are not part of the image by replacing the
type with the Config struct from the docker image-spec.
curl -s --unix-socket /var/run/docker.sock http://localhost/v1.50/images/alpine/json | jq .Config
{
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
]
}
curl -s --unix-socket /var/run/docker.sock http://localhost/v1.49/images/alpine/json | jq .Config
{
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
],
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If kernel module br_netfilter isn't loaded on the test host
before rootless test-integration tests that need it, the
rootless daemon hits a permission error when trying to do
the load. So, try to load the module before going rootless.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Networking tests have been creeping towards the limit for a
while, but they're reliably failing in a upcoming PR that adds
a relatively slow test.
Signed-off-by: Rob Murray <rob.murray@docker.com>
pie-mode is supported for windows/arm64 since https://go.dev/cl/452415,
which is part of go1.20. Also update link to Go source for pie-mode support
to match the location for current versions of Go because the package was
moved in https://go.dev/cl/438475.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cross-compiling for arm/v5 was failing;
#56 84.12 /usr/bin/arm-linux-gnueabi-clang -marm -o $WORK/b001/exe/a.out -Wl,--export-dynamic-symbol=_cgo_panic -Wl,--export-dynamic-symbol=_cgo_topofstack -Wl,--export-dynamic-symbol=crosscall2 -Qunused-arguments -Wl,--compress-debug-sections=zlib /tmp/go-link-759578347/go.o /tmp/go-link-759578347/000000.o /tmp/go-link-759578347/000001.o /tmp/go-link-759578347/000002.o /tmp/go-link-759578347/000003.o /tmp/go-link-759578347/000004.o /tmp/go-link-759578347/000005.o /tmp/go-link-759578347/000006.o /tmp/go-link-759578347/000007.o /tmp/go-link-759578347/000008.o /tmp/go-link-759578347/000009.o /tmp/go-link-759578347/000010.o /tmp/go-link-759578347/000011.o /tmp/go-link-759578347/000012.o /tmp/go-link-759578347/000013.o /tmp/go-link-759578347/000014.o /tmp/go-link-759578347/000015.o /tmp/go-link-759578347/000016.o /tmp/go-link-759578347/000017.o /tmp/go-link-759578347/000018.o -O2 -g -O2 -g -O2 -g -lpthread -O2 -g -no-pie -static
#56 84.12 ld.lld: error: undefined symbol: __atomic_load_4
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced 2 more times
#56 84.12
#56 84.12 ld.lld: error: undefined symbol: __atomic_store_4
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(x_cgo_notify_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(x_cgo_set_context_function)
#56 84.12 clang: error: linker command failed with exit code 1 (use -v to see invocation)
From discussion on GitHub;
https://github.com/moby/moby/pull/46982#issuecomment-2206992611
The arm/v5 build failure looks to be due to libatomic not being included
in the link. For reasons probably buried in mailing list archives,
[gcc](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358) and clang don't
bother to implicitly auto-link libatomic. This is not a big deal on many
modern platforms with atomic intrinsics as the compiler generates inline
instruction sequences, avoiding any libcalls into libatomic. ARMv5 is not
one of those platforms: all atomic operations require a libcall.
In theory, adding `CGO_LDFLAGS=-latomic` should fix arm/v5 builds.
While it could be argued that cgo should automatically link against
libatomic in the same way that it automatically links against libpthread,
the Go maintainers would have a valid counter-argument that it should be
the C toolchain's responsibility to link against libatomic automatically,
just like it does with libgcc or compiler-rt.
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
cross-compiling for arm/v5 fails on go1.22; a fix is included for this
in go1.23 (https://github.com/golang/go/issues/65290), but for go1.22
we can set the correct option manually.
1.189 + go build -mod=vendor -modfile=vendor.mod -o /tmp/bundles/binary-daemon/dockerd -tags 'netgo osusergo static_build journald' -ldflags '-w -X "github.com/docker/docker/dockerversion.Version=dev" -X "github.com/docker/docker/dockerversion.GitCommit=HEAD" -X "github.com/docker/docker/dockerversion.BuildTime=2024-08-29T16:59:57.000000000+00:00" -X "github.com/docker/docker/dockerversion.PlatformName=" -X "github.com/docker/docker/dockerversion.ProductName=" -X "github.com/docker/docker/dockerversion.DefaultProductLicense=" -extldflags -static ' -gcflags= github.com/docker/docker/cmd/dockerd
67.78 # runtime/cgo
67.78 gcc_libinit.c:44:8: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:47:6: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:49:10: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:69:9: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:71:3: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
78.20 + rm -f /go/src/github.com/docker/docker/go.mod
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
