It's possible to remove a legacy link from running containers.
When that happens, the Sandbox's Endpoints are removed and
re-added.
Since commit 65120d5 ("Create bridge veth in container netns")
the veth device has been created in the container's netns. When
that happens, a flag is set on the Endpoint to note that it
does not need to be moved into the netns.
But, during the Leave/Join (Sandbox.Refresh) the veth is moved
out of the netns. So, clear the flag during the Leave, to note
that it needs to be moved back in during the Join.
Signed-off-by: Rob Murray <rob.murray@docker.com>
There have been numerous reports of the "has active endpoints" error
over the years. Historically, there were some faulty code paths that
could lead to this error, but we believe they all have been fixed by
now.
However, users are still facing this error from time to time. Either
because they forgot that some containers are still running, or because
we still have bugs lying around.
To help users figure whether this error is legitimate, and what triggers
it, add endpoint names (which are just container names) to the error
message.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Add an OTel span processor copying the 'trigger' baggage member
propagated through contexts to all children spans. It's used to identify
what triggered a trace / span (API call, libnet init, etc...)
All code paths that call libnet's `NewNetwork` set this baggage member
with a unique value.
For instance, this can be used to distinguish bridge's `createNetwork`
spans triggered by daemon / libnet initialization from custom network
creation triggerd by an API call.
Two util functions are added to wrap `baggage.New` and
`baggage.NewMemberRaw` to make it easier to deal with baggage and
members by panicking on error. These should not be used with dynamic
values.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Plumb context from the API down to libnet driver method `CreateNetwork`,
and add an OTel span to the bridge driver's `createNetwork` method.
Include a few attributes describing the network configuration (e.g.
IPv4/IPv6, ICC, internal and MTU).
A new util function, `RecordStatus`, is added to the `otelutil` package
to easily record any error, and update the span status accordingly.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Since commit `eaa84bc`, when a container joins a network, we broadcast
unsolicited ARP/NAs to other containers connected to that network. The
initial advertisement is sent synchronously, and then remaining
advertisements (by default, 2) are sent asynchronously at a regular
interval (by default, 1 second).
An OTel span records the time spent sending subsequent advertisements,
but it's attached to the parent span `libnetwork.osl.AddInterface` so it
affects the overall latency reported by the trace.
The real latency (i.e. excluding the latency of that asynchronous
process), is pretty much what CLI users perceive, so it's an important
measure. It's available on the top-most span in the trace, but limiting
the trace to the real time taken by the Engine to process the HTTP
request seems better.
With this change, the initial advertisement is now tracked by a
dedicated span, and a new independent trace (containing a single span)
is created and linked to the 'initial' span / HTTP trace.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Using iptables.OnReloaded to restore individual per-network rules
on firewalld reload means rules for deleted networks pop back in
to existence (because there was no way to delete the callbacks on
network-delete).
So, on firewalld reload, walk over current networks and ask them
to restore their iptables rules.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Report FirewallBackend in "docker info".
It's currently "iptables" or "iptables+firewalld" on Linux, and
omitted on Windows.
Signed-off-by: Rob Murray <rob.murray@docker.com>
endpointCnt is a refcounter used to track how many endpoints use a
network, and how many networks references a config-only network. It's
stored separately from the network.
This is only used to determine if a network can be removed.
This commit removes the `endpointCnt` struct and all its references. The
refcounter is replaced by two lookups in the newly introduced `networks`
and `endpoints` caches added to the `Controller`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The `Controller`'s store is used by:
- `deleteFromStore`
- `getNetworks`
- `getNetworksFromStore`
- `updateToStore`
- … and other methods that can't store / delete / retrieve a Network
Calls to `updateToStore` and `deleteFromStore` have been replaced with
`upsertNetwork` and `deleteNetwork`.
Both `getNetworks` and `getNetworksFromStore` call `cacheNetwork` to
ensure networks loaded from the datastore are kept in-memory.
Finally, `sandboxRestore` was instantiating `Network` itself. These are
cached too.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The `Controller`'s store is used by:
- `deleteFromStore`
- `getEndpointFromStore`
- `getEndpointsFromStore`
- `updateToStore`
- … and other methods that can't store / delete / retrieve an Endpoint
Calls to `updateToStore` and `deleteFromStore` have been replaced with
`upsertEndpoint` and `deleteEndpoint`.
Both `getEndpointFromStore` and `getEndpointsFromStore` call
`cacheEndpoint` to ensure endpoints loaded from the datastore are kept
in-memory.
Finally, `sandboxRestore` was instantiating `Endpoint` itself. These are
cached too.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Since commit f2a183a99, `getEndpointPortMapInfo` is called for all the
endpoints of a container to get its complete list of port mappings. This
is required as multiple endpoints might publish different ports (e.g.
IPv4-only and IPv6-only endpoints mapping an IPv4 and an IPv6 port).
`getEndpointPortMapInfo` calls `(*Endpoint).DriverInfo()` which has a
dodgy behavior: if the endpoint is part of a sandbox that also has an
endpoint for the `docker_gwbridge` network, then `(*Endpoint).DriverInfo()`
returns the DriverInfo of that `docker_gwbridge` endpoint in place of
the current Endpoint's DriverInfo.
On overlay networks, host port-mappings are made through the
`docker_gwbridge` network (which is automatically attached to all Swarm
tasks). This results in duplicated port mappings reported for all Swarm
containers.
Since `getEndpointPortMapInfo` is the only place where
`(*Endpoint).DriverInfo()` is called, just remove that dodgy behavior.
`(*Endpoint).DriverInfo()` has an OS-specific implementation. Unlike the
Linux implementation, on Windows, `DriverInfo()` returns the DriverInfo
of the gateway endpoint _in addition_ to the current Endpoint's
DriverInfo. So it shouldn't be affected by this bug -- don't touch it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Create an iptablesNetwork containing all the info needed to
set up per-network iptables rules, give it methods to do
create the rules, and use it instead of per-rule-type calls
from driver.createNetwork().
Signed-off-by: Rob Murray <rob.murray@docker.com>
Doesn't look like it would ever have worked, but:
- init the dbus connection to avoid a segv
- include the chain name when creating the rule
- remove the test rule if it's created
Signed-off-by: Rob Murray <rob.murray@docker.com>
Firewalld runs in the host netns. So, tests running in
their own netns can't check firewalld's iptables rules.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Commit 9a6e96f ("Before sending ARPs/NAs, check the bridge is ready")
introduced a check that, if a container's veth is connected to a bridge,
the port is forwarding before sending unsolicited ARP/NA messages to
advertise the container's MAC/IP addresses.
It waited for up-to 200ms, but it seems that's not always enough.
If the port isn't forwarding, the ARP/NA messages will be silently
dropped. But, if it comes up within two seconds, one or both of the
resends will succeed.
So - keep the wait for 200ms but, if anything goes wrong with checking
the bridge port's state, or it's not forwarding after that time, just
log an info/warning level message and continue.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a macvlan's parent interface is down it's not possible
to send NA messages. So, ignore the error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
For kernels that don't have CONFIG_IP_NF_RAW, if the env
var DOCKER_INSECURE_NO_IPTABLES_RAW is set to "1", don't
try to create raw rules.
This means direct routing to published ports is possible
from other hosts on the local network, even if the port
is published to a loopback address.
Signed-off-by: Rob Murray <rob.murray@docker.com>
If the userland-proxy is running, packets from one bridge network
addressed to the host port are not DNAT'd - so that docker-proxy
can pick them up, and therefore the packet bypasses the network
isolation rules.
Without the userland-proxy, there's no way for a packet from one
bridge network to bypass the network isolation rules. So, in this
case, DNAT is not skipped - and that at-least allows packets
originating from the network that published the port to access
the host port.
Commit 0546d90 improved support for routed mode networks (allowing
nat-mode networks access to containers in routed-mode networks, as
well as just remote access).
That commit changed the "SKIP DNAT" logic, making sure DNAT was
skipped for a routed-mode network if the userland-proxy was enabled
(so, containers in routed mode networks could access ports published
by other networks).
But, it still skipped DNAT for a routed mode network if the userland
proxy was disabled - packets from the routed mode network aimed at
any other network would be dropped by the network isolation rules
anyway, and containers in a routed mode network don't need access to
ports published from that network (because, by definition, there
can't be any).
However, network isolation rules can be worked-around with a rule
in the DOCKER-USER chain, but the SKIP DNAT rule is harder to deal
with.
So, for routed-mode, only skip DNAT if the userland-proxy is
enabled (just like nat-mode networks).
Signed-off-by: Rob Murray <rob.murray@docker.com>
These chains were added in commit 76417bf ("Don't use ipset"), but
not included in the list of chains that get flushed and deleted
on daemon startup.
Signed-off-by: Rob Murray <rob.murray@docker.com>
