The Namespace keeps some state for each inserted neighbor-table entry
which is used to delete the entry (and any related entries) given only
the IP and MAC address of the entry to delete. This state is not
strictly required as the retained data is a pure function of the
parameters passed to AddNeighbor(), and the kernel can inform us whether
an attempt to add a neighbor entry would conflict with an existing
entry. Get rid of the neighbor state in Namespace. It's just one more
piece of state that can cause lots of grief if it falls out of sync with
ground truth. Require callers to call DeleteNeighbor() with the same
aguments as they had passed to AddNeighbor(). Push the responsibility
for detecting attempts to insert conflicting entries into the neighbor
table onto the kernel by using (*netlink.Handle).NeighAdd() instead of
NeighSet().
Modernize the error messages and logging in DeleteNeighbor() and
AddNeighbor().
Signed-off-by: Cory Snider <csnider@mirantis.com>
func (*Namespace) AddNeighbor is only ever called with the force
parameter set to false. Remove the parameter and eliminate dead code.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Go maintainers started to unconditionally update the minimum go version
for golang.org/x/ dependencies to go1.23, which means that we'll no longer
be able to support any version below that when updating those dependencies;
> all: upgrade go directive to at least 1.23.0 [generated]
>
> By now Go 1.24.0 has been released, and Go 1.22 is no longer supported
> per the Go Release Policy (https://go.dev/doc/devel/release#policy).
>
> For golang/go#69095.
This updates our minimum version to go1.23, as we won't be able to maintain
compatibility with older versions because of the above.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since commit `eaa84bc`, when a container joins a network, we broadcast
unsolicited ARP/NAs to other containers connected to that network. The
initial advertisement is sent synchronously, and then remaining
advertisements (by default, 2) are sent asynchronously at a regular
interval (by default, 1 second).
An OTel span records the time spent sending subsequent advertisements,
but it's attached to the parent span `libnetwork.osl.AddInterface` so it
affects the overall latency reported by the trace.
The real latency (i.e. excluding the latency of that asynchronous
process), is pretty much what CLI users perceive, so it's an important
measure. It's available on the top-most span in the trace, but limiting
the trace to the real time taken by the Engine to process the HTTP
request seems better.
With this change, the initial advertisement is now tracked by a
dedicated span, and a new independent trace (containing a single span)
is created and linked to the 'initial' span / HTTP trace.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Commit 9a6e96f ("Before sending ARPs/NAs, check the bridge is ready")
introduced a check that, if a container's veth is connected to a bridge,
the port is forwarding before sending unsolicited ARP/NA messages to
advertise the container's MAC/IP addresses.
It waited for up-to 200ms, but it seems that's not always enough.
If the port isn't forwarding, the ARP/NA messages will be silently
dropped. But, if it comes up within two seconds, one or both of the
resends will succeed.
So - keep the wait for 200ms but, if anything goes wrong with checking
the bridge port's state, or it's not forwarding after that time, just
log an info/warning level message and continue.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a macvlan's parent interface is down it's not possible
to send NA messages. So, ignore the error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
libnetwork/osl/namespace_linux.go:324:2: naked return in func `DisableARPForVIP` with 28 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
At first glance, it seemed like the Namespace lock was held to make
local copies of Namespace properties, but all those properties all safe
to access concurrently. So, un-alias those props and reduce the scope
of the mutex lock.
This helps understanding where locking is really needed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
To support this, a new netlabel is added: `com.docker.network.endpoint.ifname`.
It gives the ability to specify the interface name to be set by
netdrivers when the interface is added / moved into the container's
network namespace.
All builtin netdrivers support it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Before this commit, `Interface.dstName` was used to store the dest
ifname prefix, and then the final ifname.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Commit aa3a23d (Temporary debug for unsolicited NA) added code to
check for a route from ::1 to ff02::1 if a neighbour advertisement
send failed ... hoping to show that the route appeared fairly quickly
after the first attempt.
The failure didn't happen in CI, perhaps other changes in that PR
changed the timing enough for it not to be an issue. Or, perhaps the
change in commit 9a6e96f (Before sending ARPs/NAs, check the bridge is
ready) was enough to solve the issue.
But, in case it's a timing issue we're just no longer seeing on the CI
hosts - check the route exists before trying to send an NA, and wait
for a short time if not.
Signed-off-by: Rob Murray <rob.murray@docker.com>
We don't enable STP on bridges belonging to bridge networks,
but bridge ports still need to transition from "disabled" to
"forwarding", after the veth device comes "up". Until then,
the bridge will just drop packets.
So, if a container's network is a veth device, and its other
end is slaved to a bridge - wait for the bridge port to
be "forwarding".
Signed-off-by: Rob Murray <rob.murray@docker.com>
Since commit 933fcc9 (Re-remove the SetKey OCI prestart hook),
the network namespace will be set up before endpoints are
created in most cases, apart from build containers.
So, when possible, create the veth with one end in that netns
to save moving it in later. On my host, that saves about 20ms
for each bridge network a container is connected to.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The default is to send 3 messages at 1s intervals.
That can be overridden in "docker network create" using:
-o com.docker.network.advertise_addr_nmsgs=3
-o com.docker.network.advertise_addr_ms=1000
Or, in daemon.json for each driver:
"default-network-opts": {
"bridge": {
"com.docker.network.advertise_addr_nmsgs": "3",
"com.docker.network.advertise_addr_ms": "1000"
}
}
The allowed range is 0-3 for the number of messages, and
100-2000ms for the interval. Setting nmsgs to 0 disables the
gratuitous ARP/NA messages.
The default bridge will always use the built-in defaults,
it cannot be configured.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This was the only location in this file where pkg/errors was used,
and this function already mixed stdlib errors with pkg/errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On live-restore, the Sandbox tries to restore state in the osSbox
by telling it about interface, routes, and gateways that would
have been set up by the previous incarnation of the daemon.
Restoring gateways has been broken since commit 18327745c0 (Allow
separate IPv4/IPv6 gateway endpoints.) ... which didn't properly
deal with searching for the "dstName" of an interface based on its
IPv6 address.
This commit fixes that, and splits the osSbox restore into three
parts:
- Restore the interfaces, including finding the "dstName".
- Restore routes, unchanged, they're just a copy of the sandbox's
StaticRoutes
- Restore gateway info - if the Sandbox's gateway endpoint has an
IP address (v4 or v6, depending on which addr family/families it's
acting as the gateway for), store that. If not, the default route
is bound to the interface, so store that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In L3 modes, the ipvlan driver can't set up a default gateway with
a next hop address, because there's no L2 for it to resolve the
gateway IP into a MAC address.
Instead, it sets up a route to 0.0.0.0 or [::] that's connected to
the network's interface. The end result is the same - the container
has a default route.
So, don't set up routes to 0.0.0.0/:: when applying routes when an
endpoint joins a sandbox, set them up when the endpoint is selected
as the container's gateway. And, drop those routes when another
endpoint becomes the gateway.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Commit 3ec19ff62b introduced a GC goroutine to delete files where netns
were mounted. It was primarly added to work around a race in kernel
3.18-4.0.1. Since no distros we support are using such old kernels,
there's no need to keep this code around.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
A recent change to the vishvananda/netlink package exposes
NLM_F_DUMP_INTR in some netlink responses as an EINTR (with
no data).
Retry the requests when that happens, up to five times, before
returning the error. The limit of five is arbitrary, on most
systems a single retry will be rare but, there's no guarantee
that a retry will succeed. So, on a very busy or misbehaving
system the error may still be returned. In most cases, this
will lead to failure of the operation being attempted (which
may lead to daemon startup failure, network initialisation
failure etc).
Signed-off-by: Rob Murray <rob.murray@docker.com>
26.1.1 added env var DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE to make it
possible to create an IPv4-only network, even with a read-only
"/proc/sys/net" that meant IPv6 could not be disabled on an
interface.
In 27.0 it's easier to enable IPv6, just '--ipv6' when creating the
network - in particular, there's no need to allocate a subnet, because
a unique-local prefix will be assigned by default).
So, this change removes the env-var workaround. Now, the workarounds
are to enable IPv6, mount "/proc/sys/net" read-write, disable IPv6
by default in OS configuration, or remove support for IPv6 from the
kernel.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Until now it's been possible to set per-interface sysctls using, for
example, '--sysctl net.ipv6.conf.eth0.accept_ra=2'. But, the index in
the interface name is allocated serially, and the numbering in a container
with more than one interface may change when a container is restarted.
The change to make it possible to connect a container to more than one
network when it's created increased the ambiguity.
This change adds label "com.docker.network.endpoint.sysctls" to the
DriverOpts in EndpointSettings. This option is explicitly associated
with the interface.
Settings in "--sysctl" for "eth0" are migrated to DriverOpts.
Because using "--sysctl" with any interface apart from "eth0" would have
unpredictable results, it is now an error to use any other interface name
in the top level "--sysctl" option. The error message includes a hint at
how to use the new per-interface setting.
The per-endpoint sysctl name has the interface name replaced by
"IFNAME". For example:
net.ipv6.conf.eth0.accept_ra=2
becomes:
net.ipv6.conf.IFNAME.accept_ra=2
The value of DriverOpts["com.docker.network.endpoint.sysctls"] is a
comma separated list.
Settings from '--sysctl' are applied by the runtime lib during task
creation. So, task creation fails if the endpoint does not exist.
Applying per-endpoint settings during interface configuration means the
endpoint can be created later, which paves the way for removal of the
SetKey OCI prestart hook.
Unlike other DriverOpts, the sysctl label itself is not driver-specific,
but each driver has a chance to check settings/values and raise an error
if a setting would cause it a problem - no such checks have been added
in this initial version. As a future extension, if required, it would be
possible for the driver to echo back valid/extended/modified settings to
libnetwork for it to apply to the interface. (At that point, the syntax
for the options could become driver specific to allow, for example, a
driver to create more than one interface).
Signed-off-by: Rob Murray <rob.murray@docker.com>
If dockerd runs on a host with a read-only /proc/sys/net filesystem,
it isn't able to enable or disable IPv6 on network interfaces when
attaching a container to a network (including initial networks during
container creation).
In release 26.0.2, a read-only /proc/sys/net meant container creation
failed in all cases.
So, don't attempt to enable/disable IPv6 on an interface if it's already
set appropriately.
If it's not possible to enable IPv6 when it's needed, just log (because
that's what libnetwork has always done if IPv6 is disabled in the
kernel).
If it's not possible to disable IPv6 when it needs to be disabled,
refuse to create the container and raise an error that suggests setting
environment variable "DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE=1", to tell
the daemon it's ok to ignore the problem.
Signed-off-by: Rob Murray <rob.murray@docker.com>
No IPAM IPv6 address is given to an interface in a network with
'--ipv6=false', but the kernel would assign a link-local address and,
in a macvlan/ipvlan network, the interface may get a SLAAC-assigned
address.
So, disable IPv6 on the interface to avoid that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"
Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.
But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.
This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Some configuration in a container depends on whether it has support for
IPv6 (including default entries for '::1' etc in '/etc/hosts').
Before this change, the container's support for IPv6 was determined by
whether it was connected to any IPv6-enabled networks. But, that can
change over time, it isn't a property of the container itself.
So, instead, detect IPv6 support by looking for '::1' on the container's
loopback interface. It will not be present if the kernel does not have
IPv6 support, or the user has disabled it in new namespaces by other
means.
Once IPv6 support has been determined for the container, its '/etc/hosts'
is re-generated accordingly.
The daemon no longer disables IPv6 on all interfaces during initialisation.
It now disables IPv6 only for interfaces that have not been assigned an
IPv6 address. (But, even if IPv6 is disabled for the container using the
sysctl 'net.ipv6.conf.all.disable_ipv6=1', interfaces connected to IPv6
networks still get IPv6 addresses that appear in the internal DNS. There's
more to-do!)
Signed-off-by: Rob Murray <rob.murray@docker.com>
For some time, when adding an interface with no IPv6 address (an
interface to a network that does not have IPv6 enabled), we've been
disabling IPv6 on that interface.
As part of a separate change, I'm removing that logic - there's nothing
wrong with having IPv6 enabled on an interface with no routable address.
The difference is that the kernel will assign a link-local address.
TestAddRemoveInterface does this...
- Assign an IPv6 link-local address to one end of a veth interface, and
add it to a namespace.
- Add a bridge with no assigned IPv6 address to the namespace.
- Remove the veth interface from the namespace.
- Put the veth interface back into the namespace, still with an
explicitly assigned IPv6 link local address.
When IPv6 is disabled on the bridge interface, the test passes.
But, when IPv6 is enabled, the bridge gets a kernel assigned link-local
address.
Then, when re-adding the veth interface, the test generates an error in
'osl/interface_linux.go:checkRouteConflict()'. The conflict is between
the explicitly assigned fe80::2 on the veth, and a route for fe80::/64
belonging to the bridge.
So, in preparation for not-disabling IPv6 on these interfaces, use a
unique-local address in the test instead of link-local.
I don't think that changes the intent of the test.
With the change to not-always disable IPv6, it is possible to repro the
problem with a real container, disconnect and re-connect a user-defined
network with '--subnet fe80::/64' while the container's connected to an
IPv4 network. So, strictly speaking, that will be a regression.
But, it's also possible to repro the problem in master, by disconnecting
and re-connecting the fe80::/64 network while another IPv6 network is
connected. So, I don't think it's a problem we need to address, perhaps
other than by prohibiting '--subnet fe80::/64'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
