Use sub-tests so that the iterations can run in parallel (instead of
sequential), and to make failures show up for the iteration that they're
part of.
Note that changing to subtests means that we'll always run 3 iterations of
the test, and no longer fail early (but the test still fails if any of
those iterations fails.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test has become quite flaky on Windows / Windows with Containerd.
Looking at the test, I noticed that it's running a test three times (according
to the comment "as it failed ~ 50% of the time"). However;
- it uses the `--rm` option to clean up the container after it terminated
- it uses a fixed name for the containers that are started
I had a quick look at the issue that it was created for, and neither of those
options were mentioned in the reported bug (so are just part of the test setup).
I think the test was written when the `--rm` option was still client-side, in which
case the cli would not terminate until it removed the container (making the
remove synchronous). Current versions of docker have moved the `--rm` to the
daemon side, and (if I'm not mistaken) performed asynchronous, and therefore could
potentially cause a conflicting name.
This patch:
- removes the fixed name (the test doesn't require the container to have a
specific name, so we can just use a random name)
- adds logs to capture the stderr and stdout output of the run (so that we're
able to capture failure messages).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I had to check what the actual size was, so added it to the const's documentation.
While at it, also made use of it in a test, so that we're testing against the expected
value, and changed one alias to be consistent with other places where we alias this
import.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Windows Server 2016 (RS1) reached end of support, and Docker Desktop requires
Windows 10 V19H2 (version 1909, build 18363) as a minimum.
This patch makes Windows Server RS5 / ltsc2019 (build 17763) the minimum version
to run the daemon, and removes some hacks for older versions of Windows.
There is one check remaining that checks for Windows RS3 for a workaround
on older versions, but recent changes in Windows seemed to have regressed
on the same issue, so I kept that code for now to check if we may need that
workaround (again);
085c6a98d5/daemon/graphdriver/windows/windows.go (L319-L341)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I think the original intent here was to make passing t optional (62a856e912),
but it looks like that's not done anywhere, so let's remove it.
integration-cli/docker_utils_test.go:81:2: SA5011: possible nil pointer dereference (staticcheck)
c.Helper()
^
integration-cli/docker_utils_test.go:84:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
if c != nil {
^
integration-cli/docker_utils_test.go:106:2: SA5011: possible nil pointer dereference (staticcheck)
c.Helper()
^
integration-cli/docker_utils_test.go:108:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
if c != nil {
^
integration-cli/docker_utils_test.go:116:2: SA5011: possible nil pointer dereference (staticcheck)
c.Helper()
^
integration-cli/docker_utils_test.go:118:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
if c != nil {
^
integration-cli/docker_utils_test.go:126:2: SA5011: possible nil pointer dereference (staticcheck)
c.Helper()
^
integration-cli/docker_utils_test.go:128:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
if c != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use unique names to prevent tests from interfering, using a shorter
name, as there's a maximum length for these.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A number of tests in the TestDockerDaemonSuite create a custom bridge as part
of the test. In some cases, an existing `docker0` bridge could interfere with
those tests. For example, the `TestDaemonICCLinkExpose` and `TestDaemonICCPing`
verify that no "icc" communication is possible, and for this create a new
bridge with a custom IP-range.
However, depending on which tests ran before the test, a default `docker0` bridge
may exist (e.g., if the`TestDefaultGatewayIPv4Implicit`) with the same IP-range,
in which iptables rules may have been set up that allow communication, and thus
make the "icc" tests fail.
This patch removes the `docker0` interface at the start of tests that create
their own bridge to prevent it from interfering.
Note that alternatively, we could update those tests to use an IP-range that's
less likely to overlap, but this may be more brittle (but could still be done
in addition to this change as a follow-up).
To verify these changes;
make DOCKER_GRAPHDRIVER=vfs TEST_SKIP_INTEGRATION=1 TESTFLAGS='-test.run TestDockerDaemonSuite/TestDaemon(DefaultGatewayIPv4|ICC)' test-integration-cli
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The appropriate/nc image was last built over 6 years ago, and uses the
deprecated v2 schema 1 format.
https://github.com/appropriate/docker-nc/tree/master/latest
The image is just a plain "apk install" of netbsd-netcat, but was added
in 1c4286bcff, because at the time the
busybox nc had some bugs.
These appear to be resolved, so we can use the busybox nc, from the
frozen images.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pushing manifest v2, schema 1 images has been deprecated in commit
6302dbbf46 (docker 20.10). It's still used in
some tests to provision a legacy registry to test _pulling_ legacy images
(which is still "supported"), but we should no longer have to validate pushing
for other scenarios.
This patch removes the schema 1 push tests, and inlines the code that was
extracted in non-exported functions (for them to be shared between schema 2 and
schema 1 tests).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Update the frozen images to also be based on Debian bullseye. Using the "slim"
variant (which looks to have all we're currently using), and remove the
buildpack-dep frozen image.
The buildpack-dep image is quite large, and it looks like we only use it to
compile some C binaries, which should work fine on a regular debian image;
docker build -t debian:bullseye-slim-gcc -<<EOF
FROM debian:bullseye-slim
RUN apt-get update && apt-get install -y gcc libc6-dev --no-install-recommends
EOF
docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
debian bullseye-slim-gcc 1851750242af About a minute ago 255MB
buildpack-deps bullseye fe8fece98de2 2 days ago 834MB
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows using the package without having to import the "types" package,
and without having to consume github.com/ishidawataru/sctp.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make the error message slightly more informative, and remove the redundant
`len(config.ArchMap) != 0` check, as iterating over an empty, or 'nil' slice
is a no-op already. This allows to use a slightly more idiomatic "if ok := xx; ok"
condition.
Also move validation to the start of the loop (early return), and explicitly create
a new slice for "names" if the legacy "Name" field is used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "quiet" argument was only used in a single place (at daemon startup), and
every other use had to pass "false" to prevent this function from logging
warnings.
Now that SysInfo contains the warnings that occurred when collecting the
system information, we can make leave it up to the caller to use those
warnings (and log them if wanted).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/devicemapper/devmapper.go:383:28: S1039: unnecessary use of fmt.Sprintf (gosimple)
if err := task.setMessage(fmt.Sprintf("@cancel_deferred_remove")); err != nil {
^
integration/plugin/graphdriver/external_test.go:321:18: S1039: unnecessary use of fmt.Sprintf (gosimple)
http.Error(w, fmt.Sprintf("missing id"), 409)
^
integration-cli/docker_api_stats_test.go:70:31: S1039: unnecessary use of fmt.Sprintf (gosimple)
_, body, err := request.Get(fmt.Sprintf("/info"))
^
integration-cli/docker_cli_build_test.go:4547:19: S1039: unnecessary use of fmt.Sprintf (gosimple)
"--build-arg", fmt.Sprintf("FOO1=fromcmd"),
^
integration-cli/docker_cli_build_test.go:4548:19: S1039: unnecessary use of fmt.Sprintf (gosimple)
"--build-arg", fmt.Sprintf("FOO2="),
^
integration-cli/docker_cli_build_test.go:4549:19: S1039: unnecessary use of fmt.Sprintf (gosimple)
"--build-arg", fmt.Sprintf("FOO3"), // set in env
^
integration-cli/docker_cli_build_test.go:4668:32: S1039: unnecessary use of fmt.Sprintf (gosimple)
cli.WithFlags("--build-arg", fmt.Sprintf("tag=latest")))
^
integration-cli/docker_cli_build_test.go:4690:32: S1039: unnecessary use of fmt.Sprintf (gosimple)
cli.WithFlags("--build-arg", fmt.Sprintf("baz=abc")))
^
pkg/jsonmessage/jsonmessage_test.go:255:4: S1039: unnecessary use of fmt.Sprintf (gosimple)
fmt.Sprintf("ID: status\n"),
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The VolumesService did not have information wether or not a volume
was _created_ or if a volume already existed in the driver, and
the existing volume was used.
As a result, multiple "create" events could be generated for the
same volume. For example:
1. Run `docker events` in a shell to start listening for events
2. Create a volume:
docker volume create myvolume
3. Start a container that uses that volume:
docker run -dit -v myvolume:/foo busybox
4. Check the events that were generated:
2021-02-15T18:49:55.874621004+01:00 volume create myvolume (driver=local)
2021-02-15T18:50:11.442759052+01:00 volume create myvolume (driver=local)
2021-02-15T18:50:11.487104176+01:00 container create 45112157c8b1382626bf5e01ef18445a4c680f3846c5e32d01775dddee8ca6d1 (image=busybox, name=gracious_hypatia)
2021-02-15T18:50:11.519288102+01:00 network connect a19f6bb8d44ff84d478670fa4e34c5bf5305f42786294d3d90e790ac74b6d3e0 (container=45112157c8b1382626bf5e01ef18445a4c680f3846c5e32d01775dddee8ca6d1, name=bridge, type=bridge)
2021-02-15T18:50:11.526407799+01:00 volume mount myvolume (container=45112157c8b1382626bf5e01ef18445a4c680f3846c5e32d01775dddee8ca6d1, destination=/foo, driver=local, propagation=, read/write=true)
2021-02-15T18:50:11.864134043+01:00 container start 45112157c8b1382626bf5e01ef18445a4c680f3846c5e32d01775dddee8ca6d1 (image=busybox, name=gracious_hypatia)
5. Notice that a "volume create" event is created twice;
- once when `docker volume create` was ran
- once when `docker run ...` was ran
This patch moves the generation of (most) events to the volume _store_, and only
generates an event if the volume did not yet exist.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: fa125a3512...b3507428be
- fixed IPv6 iptables rules for enabled firewalld (libnetwork#2609)
- fixes "Docker uses 'iptables' instead of 'ip6tables' for IPv6 NAT rule, crashes"
- Fix regression in docker-proxy
- introduced in "Fix IPv6 Port Forwarding for the Bridge Driver" (libnetwork#2604)
- fixes/addresses: "IPv4 and IPv6 addresses are not bound by default anymore" (libnetwork#2607)
- fixes/addresses "IPv6 is no longer proxied by default anymore" (moby#41858)
- Use hostIP to decide on Portmapper version
- fixes docker-proxy not being stopped correctly
Port mapping of containers now contain separatet mappings for IPv4 and IPv6 addresses, when
listening on "any" IP address. Various tests had to be updated to take multiple mappings into
account.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The normalizing was updated with the output of the "docker port" command
in mind, but we're normalizing the "expected" output, which is passed
without the "->" in front of the mapping, causing some tests to fail;
=== RUN TestDockerSuite/TestPortHostBinding
--- FAIL: TestDockerSuite/TestPortHostBinding (1.21s)
docker_cli_port_test.go:324: assertion failed: error is not nil: |:::9876!=[::]:9876|
=== RUN TestDockerSuite/TestPortList
--- FAIL: TestDockerSuite/TestPortList (0.96s)
docker_cli_port_test.go:25: assertion failed: error is not nil: |:::9876!=[::]:9876|
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In 20.10 we no longer implicitly push all tags and require a
"--all-tags" flag, so add this to the test when the CLI is >= 20.10
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This seems to be testing a strange case, specifically that one can set
the `--net` and `--network` in the same command with the same network.
Indeed this used to work with older CLIs but newer ones error out when
validating the request before sending it to the daemon.
Opening this for discussion because:
1. This doesn't seem to be testing anything at all related to the rest
of the test
2. Not really providing any value here.
3. Is testing that a technically invalid option is successful (whether
the option should be valid as it relates to the CLI accepting it is
debatable).
4. Such a case seems fringe and even a bug in whatever is calling the
CLI with such options.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This old test is failing after an edge-case change in dockerfile
parsing considered a bugfix: https://github.com/moby/buildkit/pull/1559
Instead of fixing the test, I suggest removing it as there are already
tests for it in BuildKit.
Signed-off-by: Tibor Vass <tibor@docker.com>
Rather than bifurcate the test completely, this lets us keep the test
intact with a small function wrapper to allow the compiler to build the
code that'll never be called on Windows, on Windows.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
People keep doing this and getting pwned because they accidentally left
it exposed to the internet.
The warning about doing this has been there forever.
This introduces a sleep after warning.
To disable the extra sleep users must explicitly specify `--tls=false`
or `--tlsverify=false`
Warning also specifies this sleep will be removed in the next release
where the flag will be required if running unauthenticated.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Before this change, the error returned to the user would include the physical
path inside the tmp dir on the daemon host. These paths should be considered
an implementation detail, and provide no value to the user. Printing the tmp
path can confuse users, and will be even more confusing if the daemon is running
remotely (or in a VM, such as on Docker Desktop), in which case the path in the
error message does not exist on the local machine;
echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.57kB
Step 1/2 : FROM busybox
---> 1c35c4412082
Step 2/2 : COPY /some/non-existing/file.txt .
COPY failed: stat /var/lib/docker/tmp/docker-builder405687992/some/non-existing/file.txt: no such file or directory
When copying files from an image or a build stage, using `--from`, the error
is similarly confusing:
echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 4.671kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : COPY --from=busybox /some/non-existing/file.txt .
COPY failed: stat /var/lib/docker/overlay2/ef34239c80526c779b7afaeaedbf11c1b201d7f7681d45613102c4541da0e156/merged/some/non-existing/file.txt: no such file or directory
This patch updates the error messages to be more user-friendly. Changes are slightly
different, depending on if the source was a local path, or an image (or build-stage),
using `--from`.
If `--from` is used, only the path is updated, and we print the relative path
instead of the full path;
echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.583kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : COPY --from=busybox /some/non-existing/file.txt .
COPY failed: stat some/non-existing/file.txt: file does not exist
In other cases, additional information is added to mention "build context" and
".dockerignore", which could provide the user some hints to find the problem:
echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.583kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : COPY /some/non-existing/file.txt .
COPY failed: file not found in build context or excluded by .dockerignore: stat some/non-existing/file.txt: file does not exist
echo -e "FROM busybox\nADD /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.583kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : ADD /some/non-existing/file.txt .
ADD failed: file not found in build context or excluded by .dockerignore: stat some/non-existing/file.txt: file does not exist
This patch only improves the error for the classic builder. Similar changes could
be made for BuildKit, which produces equally, or even more confusing errors;
echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=1 docker build -f- .
[+] Building 1.2s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 85B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 1.2s
=> [internal] load build context 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/2] FROM docker.io/library/busybox@sha256:4f47c01... 0.0s
=> ERROR [2/2] COPY /some/non-existing/file.txt . 0.0s
------
> [2/2] COPY /some/non-existing/file.txt .:
------
failed to compute cache key: failed to walk /var/lib/docker/tmp/buildkit-mount181923793/some/non-existing:
lstat /var/lib/docker/tmp/buildkit-mount181923793/some/non-existing: no such file or directory
echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=1 docker build -f- .
[+] Building 2.5s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 100B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 1.2s
=> FROM docker.io/library/busybox:latest 1.2s
=> => resolve docker.io/library/busybox:latest 1.2s
=> CACHED [stage-0 1/2] FROM docker.io/library/busybox@sha256:4f47c01... 0.0s
=> ERROR [stage-0 2/2] COPY --from=busybox /some/non-existing/file.txt . 0.0s
------
> [stage-0 2/2] COPY --from=busybox /some/non-existing/file.txt .:
------
failed to compute cache key: failed to walk /var/lib/docker/overlay2/2a796d91e46fc038648c6010f062bdfd612ee62b0e8fe77bc632688e3fba32d9/merged/some/non-existing:
lstat /var/lib/docker/overlay2/2a796d91e46fc038648c6010f062bdfd612ee62b0e8fe77bc632688e3fba32d9/merged/some/non-existing: no such file or directory
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
