Use noop tracer provider if the OTEL exporter is not configured.
This makes the OTEL tracing avoid doing unneeded memory allocations for
spans which aren't going to be exported anywhere anyway.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Non-distributable artifacts (foreign layers) were introduced in commit
05bd04350b to accommodate Windows images,
for which the EULA did not allow layers to be distributed through registries
other than those hosted by Microsoft. The concept of foreign / non-distributable
layers was adopted by the OCI distribution spec in [oci#233].
These restrictions were relaxed later to allow distributing these images
through non-public registries, for which a configuration was added in the
daemon in 67fdf574d5. In 2022, Microsoft updated
the EULA and [removed these restrictions altogether][1], and the OCI distribution
spec deprecated the functionality in [oci#965].
In 2023, Microsoft [removed the use of foreign data layers][2] for their images,
making this functionality obsolete.
This patch:
- Deprecates the `--allow-nondistributable-artifacts` daemon flag and corresponding
`allow-nondistributable-artifacts` field in `daemon.json`. Setting either
option will no longer take an effect, but a deprecation warning log is added
to raise awareness about the deprecation. This warning is planned to become
an error in the next release.
- Deprecates the `RegistryConfig.AllowNondistributableArtifactsCIDRs` and
`RegistryConfig.AllowNondistributableArtifactsHostnames` fields in the
`GET /info` API response. For API version v1.48 and lower, the fields are
still included in the response, but always `null`. In API version v1.49 and
higher, the field will be omitted entirely.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsCIDRs`
field.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsHostnames`
field.
- Deprecates the `registry.ServiceOptions.AllowNondistributableArtifacts` field.
[oci#233]: https://github.com/opencontainers/image-spec/pull/233
[oci#965]: https://github.com/opencontainers/image-spec/pull/965
[1]: https://techcommunity.microsoft.com/blog/containers/announcing-windows-container-base-image-redistribution-rights-change/3645201
[2]: https://techcommunity.microsoft.com/blog/containers/announcing-removal-of-foreign-layers-from-windows-container-images/3846833
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pass the context that's used for logging, and add minimal handling of
context-cancellation. Also update logs to use structured fields.
Before this patch:
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
With this patch:
DEBU[2024-12-08T15:33:50.408445543Z] Registering route method=POST path="/networks/{id:.*}/disconnect"
DEBU[2024-12-08T15:33:50.408484335Z] Registering route method=POST path=/networks/prune
DEBU[2024-12-08T15:33:50.408505251Z] Registering route method=DELETE path="/networks/{id:.*}"
Or in JSON format:
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/connect","time":"2024-12-08T15:37:19.235209667Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/disconnect","time":"2024-12-08T15:37:19.235243001Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/prune","time":"2024-12-08T15:37:19.235290876Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that debug-routes are identical to regular routers, we can pass them
the same as those routers. With this, the daemon also logs those routes
as part of its startup (when in debug mode).
Before this patch, only non-debug endpoints would be logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
With this patch, debug endpoints are also logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
DEBU[2024-12-08T15:24:47.324486834Z] Registering GET, /debug/vars
DEBU[2024-12-08T15:24:47.324506751Z] Registering GET, /debug/pprof/
DEBU[2024-12-08T15:24:47.324532126Z] Registering GET, /debug/pprof/cmdline
DEBU[2024-12-08T15:24:47.324549293Z] Registering GET, /debug/pprof/profile
DEBU[2024-12-08T15:24:47.324564501Z] Registering GET, /debug/pprof/symbol
DEBU[2024-12-08T15:24:47.324582043Z] Registering GET, /debug/pprof/trace
DEBU[2024-12-08T15:24:47.324604751Z] Registering GET, /debug/pprof/{name}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was added in eb982e7c00, at
which time networking was not yet implemented for Windows, resulting
in a panic when trying to call network-related endpoints.
That's no longer the case, so we should be able to add network-endpoints
unconditionally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Running a container with "--add-host blah:host-gateway" adds an /etc/hosts
entry for host "blah" and an address on the docker host - to give the
container a convenient way of reaching the host.
If no --host-gateway-ip option is supplied, the IPv4 address of the
default bridge is used - and that's been fine until now, it's a host
address we know will exist. But, in a container that's only connected
to IPv6-only networks, that doesn't work.
So:
- if the default bridge has an IPv6 address, create an additional
/etc/hosts entry with that adddress
- allow two --host-gateway-ip options
- at most one IPv4 and one IPv6 address
- in daemon.json, allow a JSON array value in --host-gateway-ips (plural)
- for a single address, a JSON string is also allowed
For example:
--host-gateway-ip 192.0.2.1 --host-gateway-ip 2001:db8::1111
And the daemon.json version would be:
"host-gateway-ips": ["192.0.2.1", "2001:db8::1111"]
But, this is also still valid:
"host-gateway-ip": "192.0.2.1"
Note that the /etc/hosts entries follow the usual rules. If IPv6 is
disabled in a container (by sysctl, or lack of kernel support), IPv6
addresses are not included in the file. In other cases, IPv4 and IPv6
addresses will both be included, whether or not the container currently
has network endpoints that support IPv4 or IPv6.
buildx has its own code to interpret the host-gateway-ip option. When
it's updated to understand two addresses, moby will need to pass it
both. For now, it passes an IPv4 address if there is one, else IPv6.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Buildkit *can* take a long time to start, we don't want the daemon to fail
to startup because buildkit took too long.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Use the same logic to generate IPAMConf for IPv6 as for IPv4.
- When no fixed-cidr-v6 is specified, rather than error out, use
the default address pools (as for an IPv4 default bridge with no
fixed-cidr, and as for user-defined networks).
- Add daemon option --bip6, similar to --bip.
- Necessary because it's the only way to override an old address
on docker0 (daemon-managed default bridge), as illustrated by
test cases.
- For a user-managed default bridge (--bridge), use IPv6 addresses
on the user's bridge to determine the pool, sub-pool and gateway.
Following the same rules as IPv4.
- Don't set up IPv6 IPAMConf if IPv6 is not enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
If a values is non-nil when we don't expect it, it would be quite
helpful to get an error message explaining what happened.
find . -type f -name "*_test.go" | \
xargs gofmt -w -r "assert.Assert(t, a == nil) -> assert.Assert(t, is.Nil(a))"
find . -type f -name "*_test.go" | \
xargs gofmt -w -r "assert.Check(t, a == nil) -> assert.Check(t, is.Nil(a))"
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The daemon no longer depends on the iptables/ip6tables filter-FORWARD
chain's policy being DROP in order to implement its port filtering
rules.
However, if the daemon enables IP forwarding in the host's system
config, by default it will set the policy to DROP to avoid potential
security issues for other applications/networks.
If docker does need to enable IP forwarding, but other applications
on the host require filter-FORWARD's policies to be ACCEPT, this
option can be used to tell the daemon to leave the policy unchanged.
(Equivalent to enabling IP forwarding before starting the daemon,
but without needing to do that.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
OTEL meter implementation has a memory leak issue which causes each
meter counter invocation to create a new instrument when the meter
provider is not set.
Also add a test, which will fail once a fixed OTEL is vendored.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When a UDP server is running on a multihomed server, as is the case with
pretty much _all_ Docker hosts (eg. eth0 + docker0), the kernel has to
choose which source address is used when replying to a UDP client. But
that process is based on heuristics and is fallible.
If the address picked doesn't match the original destination address
used by the client, it'll drop the datagram and return an ICMP Port
Unreachable.
To prevent that, we need to:
- `setsockopt(IP_PKTINFO)` on proxy's sockets.
- Extract the original destination address from an ancillary message
every time a new 'UDP connection' is 'established' (ie. every time we
insert a new entry into the UDP conntrack table).
- And finally, pass a control message containing the desired source
address to the kernel, every time we send a response back to the
client.
Also, update the inline comment on read errors in `(*UDPProxy).Run()`.
This comment was misleadingly referencing ECONNREFUSED - Linux's UDP
implementation never returns this error (see [1]). Instead, state why
`net.ErrClosed` is perfectly fine and doesn't need to be logged
(although, docker-proxy currently logs to nowhere).
[1]: https://github.com/search?q=repo%3Atorvalds%2Flinux+ECONNREFUSED+path%3A%2F%5Enet%5C%2F%28ipv4%7Cipv6%29%5C%2F%28udp%7Ctcp%29%2F&type=code
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
cmd/dockerd: notifyShutdown: don't use Fatal log to prevent early exit
------------------------------------------------------------------------
Commit 40868e263d added this log for situations
where the daemon failed to start, and where no event-log was created
to discover the failure.
However, it used a `Fatal` log which, when using logrus, not only logs
a message, but also calls the [logger's Exit function][1], which defaults
[to `os.Exit`][2].
The error passed to the `notifyShutdown` function can be the result of
different steps in the service's lifecycle; `cli.start()` starts a long-
lived process, and the error returned can be either because it failed
to start the service, or an error produced while shutting down the service.
We should reduce ambiguity here (which could be for `cli.Start()` to return
whether the service was started in the first place). Regardless, we want
code following the log-entry to be executed, to make sure that codepaths
following the `notifyShutdown` (which could be (defer) statements or cleanup
steps) are executed.
This patch changes the log to an `Error`, which is non-fatal, and should
make sure that at least the `service.stopped` is executed to signal the
service no longer running. The logging is also moved out of the
`notifyShutdown`.
Disclaimer: this is with the assumption that calling `service.stopped` is
always acceptable, even if the service is not running, or in other states.
cmd/dockerd: preNotifyReady: return error instead of logging and exit
------------------------------------------------------------------------
Commit 57aef3b490 added support for running
the daemon as a Windows service. As part of this, it introduced a log
for situations where the service failed to start. However, it used a
`Fatal` log for this purpose which, when using logrus, not only logs
a message, but also calls the [logger's Exit function][1], which defaults
[to `os.Exit`][2].
A result of this is that the daemon does an unclean shutdown, causing
other code (which could be (defer) statements or cleanup steps) not to
be executed.
This patch updates the `preNotifyReady` to return the error instead of
logging and exiting. `preNotifyReady` is executed as part of `cli.start()`,
which has an error-return, and the error that is returned, is already
logged by `runDaemon` (so logging it as part of `preNotifyReady` would
result in the error being logged twice).
[1]: 5098132d84/vendor/github.com/sirupsen/logrus/entry.go (L336-L339)
[2]: 5098132d84/vendor/github.com/sirupsen/logrus/logger.go (L342-L348)
- relates to https://github.com/moby/moby/pull/22340
cmd/dockerd: preNotifyReady: return error instead of logging and exit
Commit 57aef3b490 added support for running
the daemon as a Windows service. As part of this, it introduced a log
for situations where the service failed to start. However, it used a
`Fatal` log for this purpose which, when using logrus, not only logs
a message, but also calls the [logger's Exit function][1], which defaults
[to `os.Exit`][2].
A result of this is that the daemon does an unclean shutdown, causing
other code (which could be (defer) statements or cleanup steps) not to
be executed.
This patch changes the `preNotifyReady` to
[1]: 5098132d84/vendor/github.com/sirupsen/logrus/entry.go (L336-L339)
[2]: 5098132d84/vendor/github.com/sirupsen/logrus/logger.go (L342-L348)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
They're only used within this package, and are not expected to be used
externally. Some exported functions also take non-exported types as
argument, so would not be usable outside of this package either way.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Unlike Linux, which uses fixed locations as default, the Windows daemon uses
paths relative to the data-root as defaults for storing both the PIDFile, and
the daemon configuration file (daemon.json).
The data-root is configurable both through command-line options (`--data-root`),
and through the daemon configuration file (daemon.json). This patch moves Windows-
specific config handling to config-related code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Validate and apply options when creating the CLI, so that starting the
CLI does not have to mutate the config, and to have a clearer separation
between "creating", "validating", and starting the daemon.
This also allows skipping the service-registration code in situations
where we only want to validate the config.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
getDefaultDaemonConfigDir would never return an error and because of that,
neither would getDefaultDaemonConfigFile, so we can remove these error returns.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch moves Windows-specific config for the config-file location to config-
related code to help discoverability.
Unlike Linux, which uses fixed locations as default, the Windows daemon uses
paths relative to the data-root as defaults for storing both the PIDFile, and
the daemon configuration file (daemon.json).
For the PIDfile, additional changes will be needed, as using a PIDfile depends
on whether the daemon is run as a service or not.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon was modified to tell RootlessKit about host port
mappings directly, rather than by running rootlesskit-docker-proxy
to make those updates.
DNAT rules created in rootless mode referred to the host IP address,
rather than the address seen as host address in the rootless network
namespace.
With these changes, port mappings work in rootless mode when
--userland-proxy=false - so, don't gate the RootlessKit API calls
on starting docker-proxy.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In preparation for the daemon passing a listen fd, add command line
option -use-listen-fd to indicate that the fd is present (as fd 4).
If the new flag isn't given, open the listener as normal.
Refactor the TCP and UDP proxies to be constructed with an existing
TCPListener or UDPConn, respectively. Lift the responsibilty of opening
the listener to the entrypoint. Per the Single Responsibility Principle,
this structure affords changing how the listener is created without
having to touch the proxy implementations.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
Configuring CORS headers was deprecated in docker 27.0 through
7ea9acc97f, which disabled them by default
with a temporary `DOCKERD_DEPRECATED_CORS_HEADER` env-var to allow using
the option.
This patch removes the feature altogether; the flag is kept for one more
release to allow printing a more informative error, but can be removed in
the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.
Add a callback that is called by the exporter wrapper.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
