Merge pull request #53847 from corhere/network-test-specs
d/libn/docs: specify the Swarm networking surface
This commit is contained in:
6 files changed
+2205
No files matched your search
@@ -0,0 +1,294 @@
|
||||
Networking test-surface specifications
|
||||
======================================
|
||||
|
||||
Each document here specifies a part of Moby's networking as it is observable from outside the
|
||||
daemon. A document states:
|
||||
|
||||
- what that part is expected to do
|
||||
- what a failing test means
|
||||
- the vantage point from which to observe it
|
||||
- what the verdict of a test depends on
|
||||
|
||||
| Spec | Scope |
|
||||
| --- | --- |
|
||||
| [network-common.md](network-common.md) | What every network driver must do, and the Engine API requests for networks and endpoints that do not depend on the driver. A stub. |
|
||||
| [swarm-networking.md](swarm-networking.md) | The overlay network driver and the Swarm service mesh |
|
||||
| [ipvlan-networking.md](ipvlan-networking.md) | The ipvlan network driver. A stub. |
|
||||
| [bridge-networking.md](bridge-networking.md) | The bridge network driver, and the parts of the daemon's firewall integration that are not specific to Swarm. A stub. |
|
||||
| [container-dns.md](container-dns.md) | Name resolution inside a container on a user-defined network, whatever the network's driver. A stub. |
|
||||
|
||||
A spec describes behavior only. It should change when the behavior of the daemon changes. It should
|
||||
also change when a regression reveals an expectation that, until then, was only an assumption.
|
||||
|
||||
## Who the specs serve
|
||||
|
||||
The specs serve two kinds of test. Functional tests serve the maintainer's story: *"as a Moby
|
||||
maintainer, catch regressions in code changes."* Acceptance tests serve the packager's story: *"as a
|
||||
packager, verify a candidate release works on every platform in my support matrix."*
|
||||
[Depends on](#depends-on) tells which rows have a verdict that can differ between platforms.
|
||||
|
||||
The daemon can have facilities built in for testing, such as a failpoint or a timer override. A
|
||||
packaged build can ship such a facility, enabled only by an environment variable whose name starts
|
||||
with `DOCKER_TEST_`, as the daemon's existing test hooks are.
|
||||
|
||||
### Nodes
|
||||
|
||||
A Linux node can be a system container: a container that boots the platform's own init system and
|
||||
userland, with the daemon's package installed, on the platform's kernel. Examples are a
|
||||
systemd-nspawn machine and a privileged container that runs systemd. The expectations hold on such
|
||||
nodes as on hosts, when the nodes share a Linux bridge as the physical network between them and the
|
||||
host's own firewall does not filter the traffic on that bridge. A Windows node is a Windows host or
|
||||
virtual machine.
|
||||
|
||||
## Expectations
|
||||
|
||||
A spec is a series of numbered sections. Each section has a table of expectations. A row gives the
|
||||
state that the expectation applies to, the expectation, its tier, the vantage point from which to
|
||||
observe it, and what its verdict depends on. After the numbered sections, a spec can list its known
|
||||
divergences & non-goals, and what it deliberately does not assert.
|
||||
|
||||
The "Applies to" cell gives the state before the row's action. Examples are objects that exist,
|
||||
daemon settings, the platform, and what happened to an object before the action. It never names the
|
||||
action itself. A request, a command, an update or a restart that the row is about goes in the
|
||||
"Expectation" cell. When the cell is blank, the row applies whenever its section does.
|
||||
|
||||
An expectation states the outcome of an action at the same level of abstraction as the action. The
|
||||
outcome of an Engine API request is the API's response: its status and body. The outcome of a
|
||||
`docker` command is what the command prints and its exit status. An HTTP status is not visible at
|
||||
the level of a `docker` command. A row about a `docker` command names `cli` among what it depends
|
||||
on. It usually has a partner row about the API request that the command makes.
|
||||
|
||||
An outcome covers only the direct response to the action. A side effect is not part of the outcome.
|
||||
Examples of side effects are a network created or removed, traffic on the wire, and an entry in the
|
||||
daemon's log. A spec states a side effect where it can be observed, whatever the action was. An
|
||||
expectation that an action succeeds also states the side effect that shows the action took effect. A
|
||||
daemon that reports success and does nothing fails such an expectation.
|
||||
|
||||
Unless the row names a range, an expectation about an Engine API request holds at every API version
|
||||
that the daemon accepts by default. These versions go from the daemon's default minimum version to
|
||||
the current version.
|
||||
|
||||
A row names a daemon setting by its `daemon.json` key, such as `firewall-backend` or
|
||||
`features.swarm-nftables`. The matching `dockerd` flag, such as `--firewall-backend`, sets the same
|
||||
setting. A row holds whichever of the two sets it. A row that names a `dockerd` flag, such as
|
||||
`--mtu`, holds only when that flag sets the setting. Any other flag that a row names, such as
|
||||
`--dns`, is a flag of a `docker` command.
|
||||
|
||||
Unless a row, its section or its spec says otherwise, an expectation holds on a node in its baseline
|
||||
state. For a daemon installed from a package or an installer, the baseline is a clean install of a
|
||||
supported OS, a fresh install of the package, and the setup that the product's installation
|
||||
documentation prescribes. The package and that documentation are part of the system under test. For
|
||||
a daemon that is not installed from a package, such as one built in CI, the baseline is a host that
|
||||
meets the host requirements of these specs, with no other configuration. The host requirements are
|
||||
the daemon's own requirements, whatever the distribution ([Host requirements](#host-requirements)).
|
||||
|
||||
The iptables backend means that the daemon's `firewall-backend` is `iptables`, the default. The
|
||||
nftables backend means that it is `nftables`.
|
||||
|
||||
A row's state describes changes from the baseline. A row holds with each daemon setting that it does
|
||||
not name as the baseline leaves it: as the package installs it, or at its default for a daemon that
|
||||
is not installed from a package. It holds at another value only where the row or its section says
|
||||
so. The exception is the firewall backend: a row that does not name a backend holds under both, at
|
||||
the same tier, because the nftables backend behaves like the iptables backend except where a spec
|
||||
says otherwise. Until moby/moby#53125, no row of these specs is asserted on a node in a Swarm under
|
||||
the nftables backend
|
||||
([swarm-networking.md §13](swarm-networking.md#13-known-divergences--non-goals)).
|
||||
|
||||
A driver's rows hold in every scope that the driver supports, local and Swarm, unless a row or its
|
||||
spec says otherwise. A row about a standalone container on a Swarm-scoped network holds where
|
||||
[swarm-networking.md §2.2](swarm-networking.md#22-swarm-scoped-networks) lets the container attach
|
||||
to the network. A row about a network holds whatever options the network has, unless the row or its
|
||||
section names them. The options are those of the network that the row names. The options of a
|
||||
Swarm-scoped network of a node-local driver are those of the config-only network that its
|
||||
`ConfigFrom` names on the node. Without `ConfigFrom`, the network has none of the `Options` given at
|
||||
create. A spec can say that a section's rows hold in several cases, such as the two ends on the same
|
||||
node or on different nodes, or each pairing of the nodes' platforms. A row then holds in every
|
||||
combination of those cases, the network's options and the daemon settings under which the row holds.
|
||||
|
||||
Unless a row or its section says otherwise, the rule has these limits:
|
||||
|
||||
- The daemon does not publish ports on an `internal` network, and does not forward traffic between
|
||||
such a network and addresses outside it. A row about a published port, or about traffic between a
|
||||
container and an address outside the container's networks, holds where the network that carries
|
||||
the traffic is not `internal`. A row about a query that the embedded resolver forwards upstream
|
||||
holds for a querying container that has external access, as [container-dns.md](container-dns.md)
|
||||
defines it.
|
||||
- A row about the addresses that a network allocates, or about its `Status.IPAM`, holds for a
|
||||
network that uses the default IPAM driver.
|
||||
- A row does not cover a network that a known divergence says is not allocated, or that a container
|
||||
cannot attach to.
|
||||
|
||||
The release under test is *N*. The previous release, *N−1*, is each of two releases: the previous
|
||||
patch release of *N*, and the latest patch release of the previous minor release line. A row about
|
||||
*N−1* holds for each of them. A build from a release branch counts as the next patch release of that
|
||||
branch. A build from the main branch, such as a pull request, counts as the next minor release, so
|
||||
its *N−1* is the latest patch release of the latest minor release line. For an x.y.0 release, the
|
||||
two releases are the same. A row about *N−1* holds whether *N−1* runs from its package or from its
|
||||
released binaries. Apart from what the package's upgrade path decides, a row that names `upgrade`
|
||||
also holds when the daemon's binaries are replaced in place of a package upgrade. A row about *N−1*
|
||||
asserts the behavior of *N*. When a known defect in an *N−1* release breaks such a row, the row does
|
||||
not cover the combination that the defect breaks, until *N−1* is a release without the defect. A
|
||||
defect is known when *N* has its fix and that *N−1* release does not.
|
||||
|
||||
A request that a node's daemon carries out itself, such as a request about a container or its
|
||||
endpoints, whatever the scope of the network, or about a network that is not Swarm-scoped, takes
|
||||
effect on that node once the request responds, unless the row names a ceiling. The exception is a
|
||||
node's loss of its local attachment to a Swarm-scoped network after its last container on that
|
||||
network stops or disconnects. That loss is a removal in a Swarm, timed from when the request
|
||||
responds. The effects of `POST /swarm/init`, `POST /swarm/join` and `POST /swarm/leave` on the node
|
||||
that sent the request hold within the leave-and-rejoin ceiling, timed from when the request
|
||||
responds. Any other effect in a Swarm holds within the ceiling of the change's convergence class in
|
||||
[swarm-networking.md §11.2](swarm-networking.md#112-convergence). This includes every effect of a
|
||||
Swarm request, such as a request about a service, a Swarm-scoped network or a node, on every node,
|
||||
the manager that handled the request included. The effect of a removal is in the reassignment class.
|
||||
So is the effect of an addition, such as a new task, service or endpoint, or a task's change to
|
||||
healthy. So is the effect of a change that is neither, such as a change of a service's endpoint
|
||||
mode, networks, published ports or aliases. The exceptions are the addresses, VIPs, published ports
|
||||
and address counts that the API reports, which are in the allocation class. A task's change to
|
||||
unhealthy, the exit of its container, or a change of its desired state away from `running` takes
|
||||
effect on every node, the task's own node included, within the allocation ceiling. The ceiling
|
||||
starts when the task's node emits the container's `health_status` or `die` event, or when `GET
|
||||
/tasks` first reports the new `DesiredState`. What holds once no container of a service is running
|
||||
also holds within the allocation ceiling, from the last of those `die` events. The effect of a
|
||||
change of leader holds within the allocation ceiling too, from the change. The effect of a firewalld
|
||||
reload holds within 30 seconds, the smallest of those ceilings. Some rows send a request that
|
||||
succeeds only once an earlier effect in the row holds. Such a row holds when one attempt at the
|
||||
request, before the ceiling of that effect has passed, gets the expected response. A request that is
|
||||
refused has no effect to wait for. A row about what a refused request leaves unchanged holds once
|
||||
the request responds. Any other row that says that something does not happen holds once its ceiling
|
||||
has passed. A timer override does not shorten a ceiling. A row that says that something does not
|
||||
happen, without an action, holds once a row that says that something does happen in the same setup
|
||||
holds.
|
||||
|
||||
Traffic reaches a destination when the destination receives it. A response from the destination
|
||||
shows that the traffic reached it. Traffic cannot reach a destination when the destination does not
|
||||
receive it within 5 seconds of the send, whatever the sender gets back. This also holds for a row
|
||||
that says that traffic does not reach a destination, or that a destination is not reachable. A
|
||||
destination does not answer when traffic to it gets a refusal or gets no response within 5 seconds.
|
||||
A refusal is a TCP reset or an ICMP unreachable error. A row that names the error holds only for
|
||||
that error.
|
||||
|
||||
### Tier
|
||||
|
||||
Every expectation carries a tier. The tier says what a failing test means and how to respond to it.
|
||||
|
||||
| Tier | A failure means |
|
||||
| --- | --- |
|
||||
| B (behavioral) | *You broke this.* User-visible semantics. A change needs justification. |
|
||||
| C (characterization) | *You changed this. Was that intended?* Internal but load-bearing. A deliberate change is fine. An accidental change is not. |
|
||||
| (omitted) | Incidental. The spec does not assert it at all. Each spec lists such behaviors under *Deliberately not asserted*. |
|
||||
|
||||
A row in a spec's known divergences & non-goals is not an expectation.
|
||||
A numbered C row pins a divergence that is worth a change detector. The "Depends on" cell of a
|
||||
divergence names the facility that decides on which platforms it shows.
|
||||
|
||||
A behavior can carry both tiers, as a B row and a C row. The inner MTU in
|
||||
[swarm-networking.md §3](swarm-networking.md#3-observable-state-inside-a-container) is the model
|
||||
case: it is B as a property of the path and C as a pinned value.
|
||||
|
||||
### Observed from
|
||||
|
||||
A black-box expectation means something only from a stated vantage point. Some rows hold from one
|
||||
vantage point and not from another.
|
||||
|
||||
| Code | Vantage point |
|
||||
| --- | --- |
|
||||
| `task` | Inside a Swarm service task's container |
|
||||
| `ctr` | Inside a standalone container, one that is not a Swarm service task. A spec can narrow which networks the container is attached to. |
|
||||
| `node` | The host network namespace of a node running the daemon |
|
||||
| `underlay` | The physical network between nodes. Its traffic can be observed at any host on it, including at a node's own network interfaces. Traffic on it can come from any host on it other than the node under test, such as another node, or a container outside the node under test that is attached to the physical network. |
|
||||
| `client` | A host other than the node under test that can reach one of the node's addresses. It can be another node of the same Swarm. |
|
||||
| `api:mgr` | Engine API / CLI against a Swarm manager |
|
||||
| `api:wkr` | Engine API / CLI against a Swarm worker |
|
||||
| `api:active` | Engine API / CLI against a Swarm node, manager or worker, whose availability is `active`. Only such a node can run a Swarm service task or attach a standalone container to a Swarm-scoped network. |
|
||||
| `api:solo` | Engine API / CLI against a daemon that is not in a Swarm |
|
||||
| `api` | Engine API / CLI against a daemon, whether or not it is in a Swarm |
|
||||
| `log` | The daemon log on a node. It is the only channel for a whole class of partial failure. See [swarm-networking.md §11.6](swarm-networking.md#116-failure-reporting) |
|
||||
|
||||
In a row, `a` → `b` names two vantage points: traffic sent from `a` is observed at `b`.
|
||||
|
||||
Where a row refers to the node's address, it holds at each address assigned to an interface in the
|
||||
node's host network namespace that the client can reach. A row that says "any address of the node"
|
||||
holds at every one of them. Two limits apply at an IPv6 link-local address of the node. A row about
|
||||
a published port does not hold there when the daemon option `userland-proxy` is false
|
||||
([bridge-networking.md §3.2](bridge-networking.md#32-port-publishing)). A row about the source
|
||||
address that a container observes does not hold there. A row about a published port holds at an
|
||||
address of the node only when the endpoint that the daemon publishes the port on has an address of
|
||||
that family. With `userland-proxy` true, a port published on an endpoint without an IPv6 address is
|
||||
also reachable at the node's IPv6 addresses. A row about the source address that a container or task
|
||||
observes does not hold there.
|
||||
|
||||
A B row about the daemon log refers to the entry with the text that the row's C partner gives. A
|
||||
deliberate change to that text breaks both rows. A row that says that the log does not report
|
||||
something holds when no entry with that text appears.
|
||||
|
||||
### Depends on
|
||||
|
||||
A row names the facilities its verdict depends on, or `none`. A code can have a platform in
|
||||
parentheses after it, such as `os` (Windows). Such a code applies on nodes of that platform only. On
|
||||
the other nodes, the row names `none`.
|
||||
|
||||
| Code | Facility | What varies |
|
||||
| --- | --- | --- |
|
||||
| `os` | What the operating system vendor supplies: the kernel and first-party userland | On Linux: the kernel's netfilter, IPVS, XFRM, VXLAN and SCTP support, nftables and iptables userland, the iptables-nft shim, firewalld, systemd-resolved or NetworkManager, and sysctl defaults. On Windows: the Host Networking Service, the Virtual Filtering Platform and the DNS client. Which features each OS release offers, and how they behave. |
|
||||
| `upgrade` | A package's upgrade path from the previous release | Its service units and maintainer scripts, which stop, start or restart the daemon across the upgrade |
|
||||
| `cli` | The `docker` CLI a packager ships with the daemon | Its version, how it turns commands and flags into API requests, and how it prints the responses |
|
||||
|
||||
What a code means for a row's scope:
|
||||
|
||||
> A row that names a facility holds on every platform in a packager's support matrix, whatever its
|
||||
> tier, with the facility that the platform supplies. Its verdict can differ from one platform to
|
||||
> another. The verdict of a row that names `none` does not differ between platforms that run the same
|
||||
> daemon build.
|
||||
|
||||
A row names a facility if the facility decides the verdict when the real facility is in use. A row
|
||||
whose state fixes how a facility behaves, such as a kernel that does not offer XFRM, names `none`
|
||||
when the daemon alone decides the verdict in that state.
|
||||
|
||||
### Platforms
|
||||
|
||||
Unless a section or a row narrows it, an expectation holds on Linux and Windows nodes alike. Unless
|
||||
a section or a row narrows it, an expectation also holds for traffic between nodes of any mix of the
|
||||
two. A section built on a Linux-only mechanism says so in its introduction. A row that depends on a
|
||||
Linux-only daemon setting, such as the userland proxy or the firewall backend, holds on Linux nodes.
|
||||
The spec lists a platform on which an expectation does not hold yet as a known divergence.
|
||||
|
||||
A row about traffic between nodes holds for each pairing of the two nodes' platforms: Linux to
|
||||
Linux, Linux to Windows, Windows to Linux and Windows to Windows.
|
||||
|
||||
### Host requirements
|
||||
|
||||
A spec can list host requirements in a section of that name. A host requirement is something that a
|
||||
node must provide for the spec's expectations to apply. A row applies only on a host that meets the
|
||||
requirements it relies on, so its "Applies to" cell does not repeat them. A row whose "Applies to"
|
||||
names a host that lacks a requirement states how the daemon behaves on such a host. Such a row also
|
||||
holds where the lack is emulated, for example by a Linux security module policy that denies the
|
||||
facility. The lists give the minimum that a node must provide. Together, the lists of all the specs
|
||||
are the baseline for a daemon that is not installed from a package. A package and its installation
|
||||
documentation meet them on each platform that the package supports.
|
||||
|
||||
Each kind of requirement has a table of its own:
|
||||
|
||||
- Kernel facilities: features of the Linux kernel that the daemon uses, such as VXLAN devices or
|
||||
IPVS. The "Kernel options" column gives the options that provide each facility. They are
|
||||
the options of the mainline Linux kernel, without their `CONFIG_` prefix. A kernel can offer a
|
||||
facility as a module or built in.
|
||||
- Programs: commands that the daemon runs, such as `iptables` or `nft`, and what each must accept.
|
||||
A spec can add the kernel options that a program needs. The daemon does not check them.
|
||||
- Host configuration: state of the host that the daemon relies on and does not set, such as a
|
||||
sysctl that must have a given value, or traffic that the host's firewall must accept.
|
||||
- Windows facilities: services of the Windows operating system that the daemon uses, such as the
|
||||
Host Networking Service.
|
||||
|
||||
The "Needed when" column gives the daemon settings and network options under which the daemon needs
|
||||
the requirement. "Always" means that the daemon needs it whatever its settings. The "Needed for"
|
||||
column names the sections whose rows rely on it.
|
||||
|
||||
Each requirement names the row that pins it, usually a C row. A pinning row states how the daemon
|
||||
behaves on a host that lacks the requirement, such as an error that names the missing facility, or
|
||||
it states host state that an operator can observe, such as a sysctl. When the daemon stops needing
|
||||
the requirement, that row fails, and the change has to update the requirement too. A pinning row
|
||||
does not catch a new requirement. A new requirement shows when other rows fail on a host that lacks
|
||||
it. A requirement without such a row says "Not pinned". A spec does not pin a requirement with a row
|
||||
about the daemon's internal plumbing, such as which kind of interface it creates or the rules it
|
||||
programs in a private network namespace.
|
||||
@@ -0,0 +1,316 @@
|
||||
Bridge networking surface area
|
||||
==============================
|
||||
|
||||
Scope: the bridge network driver, and the parts of the daemon's firewall integration that are not
|
||||
specific to Swarm. This spec describes them as observable from outside the daemon. Every expectation
|
||||
here holds on Linux nodes. A row about `POST /networks/create` or `docker network create` holds for
|
||||
a network that is not Swarm-scoped. For a Swarm-scoped network, the node in a row is the node of the
|
||||
container that the row names. A row about reaching a published port at an address of the node holds
|
||||
on a network whose gateway mode for that address's family is not `routed`, and without
|
||||
`com.docker.network.bridge.host_binding_ipv4`, unless the row names them. A row about IPv4 traffic
|
||||
through the network's gateway, such as egress, a published port or a default route, holds for a
|
||||
network without `com.docker.network.bridge.inhibit_ipv4`, unless the row names it.
|
||||
[swarm-networking.md](swarm-networking.md) covers what is specific to Swarm-scoped bridge networks
|
||||
([§2.2](swarm-networking.md#22-swarm-scoped-networks)) and to `docker_gwbridge`
|
||||
([§8.1](swarm-networking.md#81-gateway-network)).
|
||||
|
||||
This spec is a stub. It holds expectations that belong to the bridge driver. They come from the
|
||||
work on the [Swarm networking surface area](swarm-networking.md) and from reviews of the specs'
|
||||
coverage. This spec is not yet a complete account of the bridge driver. [README.md](README.md)
|
||||
gives its conventions for tiers, vantage points and dependencies.
|
||||
[network-common.md](network-common.md) covers what every network driver must do, such as naming an
|
||||
endpoint's interface.
|
||||
|
||||
## Host requirements
|
||||
|
||||
The conventions for this section are in [README.md](README.md#host-requirements). IPv6 is not a
|
||||
requirement: [§3.6](#36-hosts-without-ipv6) states what holds on a host without it.
|
||||
|
||||
### Kernel facilities
|
||||
|
||||
| Facility | Needed when | Kernel options | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Network namespaces, veth pairs and Linux bridges | Always | `NET_NS`, `VETH`, `BRIDGE` | Every section | The [§3.1](#31-isolation) row about a host without veth pairs |
|
||||
| nftables with the NAT, masquerade, connection-tracking and FIB expressions | The nftables backend | `NF_TABLES`, `NF_TABLES_IPV4`, `NF_TABLES_IPV6`, `NFT_NAT`, `NFT_MASQ`, `NFT_CT`, `NFT_FIB_IPV4`, `NFT_FIB_IPV6`. `NFT_CHAIN_NAT_IPV4`, `NFT_CHAIN_NAT_IPV6`, `NFT_MASQ_IPV4` and `NFT_MASQ_IPV6` on kernels that have them. | [§1.1](#11-base-chain-priorities), [§1.3](#13-cross-generation-cleanup)-[§1.5](#15-rule-content), [§3](#3-data-path) | The [§1.1](#11-base-chain-priorities) row about base-chain hooks and the [§1.5](#15-rule-content) row about the reference nftables rulesets |
|
||||
| Netfilter hooks for bridged traffic (`br_netfilter`) | The daemon option `iptables` true, and either a network that does not allow inter-container communication or the daemon option `userland-proxy` false | `BRIDGE_NETFILTER` | [§3.1](#31-isolation), [§3.2](#32-port-publishing) | The [§3.1](#31-isolation) row about `bridge-nf-call-iptables` |
|
||||
| SCTP | A container that publishes an SCTP port | `IP_SCTP`. `NF_CT_PROTO_SCTP` on kernels that have it. | [§3.2](#32-port-publishing) | The [§3.2](#32-port-publishing) row about a host without SCTP |
|
||||
|
||||
### Programs
|
||||
|
||||
| Program | Needed when | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- |
|
||||
| `iptables`, accepting the filter, nat and raw tables, the `MASQUERADE`, `DNAT` and `SNAT` targets, and the `addrtype`, `conntrack` and `sctp` matches | The iptables backend, with the daemon option `iptables` true | [§1](#1-firewall-anchors--host-integration), [§3](#3-data-path) | The [§1.4](#14-firewall-backend-and-firewalld) row about a host without the `iptables` command, and the [§1.5](#15-rule-content) row about the reference iptables rulesets |
|
||||
| `ip6tables`, accepting the same tables, targets and matches | The iptables backend, with the daemon option `ip6tables` true | [§1](#1-firewall-anchors--host-integration), [§3](#3-data-path) | The [§1.4](#14-firewall-backend-and-firewalld) rows about a host without the `ip6tables` command. No row pins the tables, targets and matches for IPv6: the reference rulesets in [§1.5](#15-rule-content) are for IPv4 only. |
|
||||
| `nft` | The nftables backend | [§1](#1-firewall-anchors--host-integration), [§3](#3-data-path) | The [§1.4](#14-firewall-backend-and-firewalld) row about a host without the `nft` command |
|
||||
|
||||
The daemon does not depend on which implementation of `iptables` and `ip6tables` the host has.
|
||||
The kernel options that the programs need depend on that implementation:
|
||||
|
||||
| Program | Implementation | Kernel options |
|
||||
| --- | --- | --- |
|
||||
| `iptables` | Either | `NETFILTER`, `NETFILTER_ADVANCED`, `NETFILTER_XTABLES`, `NF_CONNTRACK`, `NF_NAT`, `NETFILTER_XT_NAT`, `NETFILTER_XT_TARGET_MASQUERADE`, `NETFILTER_XT_MATCH_ADDRTYPE`, `NETFILTER_XT_MATCH_CONNTRACK`, `NETFILTER_XT_MATCH_SCTP`. Before Linux 5.2, `IP_NF_NAT` and `IP_NF_TARGET_MASQUERADE` in place of `NETFILTER_XT_TARGET_MASQUERADE`. |
|
||||
| `iptables` | iptables-legacy | `IP_NF_IPTABLES`, `IP_NF_FILTER`, `IP_NF_NAT`, `IP_NF_RAW`. `IP_NF_IPTABLES_LEGACY` from Linux 6.9, and `NETFILTER_XTABLES_LEGACY` from Linux 6.17. |
|
||||
| `iptables` | iptables-nft | `NF_TABLES`, `NF_TABLES_IPV4`, `NFT_COMPAT`, `NFT_NAT`. Before Linux 5.1, `NFT_CHAIN_NAT_IPV4` in place of `NFT_NAT`. |
|
||||
| `ip6tables` | Either | `IPV6`, and the options in the first row. Before Linux 5.2, `IP6_NF_NAT` and `IP6_NF_TARGET_MASQUERADE` in place of `NETFILTER_XT_TARGET_MASQUERADE`. |
|
||||
| `ip6tables` | iptables-legacy | `IP6_NF_IPTABLES`, `IP6_NF_FILTER`, `IP6_NF_NAT`, `IP6_NF_RAW`. `IP6_NF_IPTABLES_LEGACY` from Linux 6.9, and `NETFILTER_XTABLES_LEGACY` from Linux 6.17. |
|
||||
| `ip6tables` | iptables-nft | `NF_TABLES`, `NF_TABLES_IPV6`, `NFT_COMPAT`, `NFT_NAT`. Before Linux 5.1, `NFT_CHAIN_NAT_IPV6` in place of `NFT_NAT`. |
|
||||
|
||||
### Host configuration
|
||||
|
||||
| Configuration | Needed when | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- |
|
||||
| `net.ipv4.ip_forward` is 1 | The nftables backend, with the daemon option `ip-forward` true | [§1.4](#14-firewall-backend-and-firewalld), [§3](#3-data-path) | The [§1.4](#14-firewall-backend-and-firewalld) row about the nftables backend on a host where `net.ipv4.ip_forward` is 0 |
|
||||
| `net.ipv6.conf.all.forwarding` and `net.ipv6.conf.default.forwarding` are 1 | The nftables backend, with the daemon option `ip-forward` true, and a bridge network with IPv6 | [§1.4](#14-firewall-backend-and-firewalld), [§3](#3-data-path) | The [§1.4](#14-firewall-backend-and-firewalld) row about the nftables backend on a host where IPv6 forwarding is disabled |
|
||||
| `/proc/sys/net` is writable in the daemon's network namespace | Always | [§3.1](#31-isolation), [§3.2](#32-port-publishing) | The [§3.1](#31-isolation) row about `bridge-nf-call-iptables` and the [§3.2](#32-port-publishing) row about `route_localnet` |
|
||||
| No other table in the host's firewall drops the forwarded traffic of bridge networks. For example, the iptables filter table's `FORWARD` chain has policy ACCEPT. | The nftables backend | [§3](#3-data-path) | The [§1.3](#13-cross-generation-cleanup) row about a `FORWARD` chain with policy DROP |
|
||||
|
||||
## 1. Firewall anchors & host integration
|
||||
|
||||
The content of Moby's firewall rules is not an interface. A backend change replaces exactly this
|
||||
content. [§1.5](#15-rule-content) pins it at tier C only. Two things are interfaces. They are the
|
||||
anchors of this section:
|
||||
|
||||
1. The `DOCKER-USER` iptables chain. Operators attach rules to it. The documentation names it as
|
||||
the place to put policy that survives daemon restarts. The daemon never deletes or modifies its
|
||||
contents.
|
||||
2. The hooks and priorities of the bridge driver's nftables base chains. They decide the evaluation
|
||||
order relative to everything else on the host, such as firewalld, kube-proxy or an operator's
|
||||
own ruleset. A changed priority silently changes who sees a packet first.
|
||||
|
||||
No other name is an anchor. Moby's other iptables chains are not anchors, and no name in nftables
|
||||
is an anchor. An operator's rules can reach Moby's nftables chains only if they are programmed into
|
||||
Moby's own tables. That is not a supported configuration.
|
||||
|
||||
IPv4 is an enabled IP version while the daemon's `iptables` option is true. IPv6 is an enabled IP
|
||||
version while the daemon's `ip6tables` option is true. This applies under either firewall backend,
|
||||
and whether or not `ipv6` is set.
|
||||
|
||||
### 1.1 Base-chain priorities
|
||||
|
||||
These rows cover the host's network namespace only. The specs do not assert the rules that the
|
||||
daemon programs in a container's network namespace
|
||||
([container-dns.md](container-dns.md#deliberately-not-asserted)).
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| The nftables backend | For each enabled IP version, the bridge driver's nftables table `docker-bridges` in that version's family has exactly five base chains. They are on the prerouting hook at raw priority (−300) and at dstnat priority (−100), on the output hook at dstnat priority (−100), on the forward hook at filter priority (0), and on the postrouting hook at srcnat priority (100). | C\* | `node` | `none` |
|
||||
|
||||
\* These expectations are tier C while the nftables backend is experimental. They become tier B
|
||||
when the nftables backend is a supported feature.
|
||||
|
||||
### 1.2 `DOCKER-USER`
|
||||
|
||||
`DOCKER-USER` belongs to the iptables backend. The rows in this section hold under the iptables
|
||||
backend.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | For each enabled IP version, `DOCKER-USER` exists in the filter table and `FORWARD` jumps to it. | B | `node` | `os` |
|
||||
| | A DROP rule that an operator places in `DOCKER-USER` drops matching traffic to a published port. This is also true for a rule that an operator places after a firewalld reload. | B | `client` | `os` |
|
||||
| | `DOCKER-USER` rules that an operator added before a start of the daemon are still there, unchanged, after the start. This includes a restart with `iptables` false. | B | `node`, `client` | `none` |
|
||||
| | After a firewalld reload, for each enabled IP version, `DOCKER-USER` exists and `FORWARD` jumps to it. | B | `node` | `os` |
|
||||
|
||||
### 1.3 Cross-generation cleanup
|
||||
|
||||
This section is about a node that runs release *N−1* and is then upgraded to *N*. After the upgrade,
|
||||
the node can also use a different firewall backend. A generation is the firewall state that one
|
||||
release creates under one firewall backend. A previous generation is such state that a daemon left
|
||||
on the node before the current start of the daemon. After the upgrade, the node must not have two
|
||||
generations of live rules. The rows in this section hold for each enabled IP version. The built-in
|
||||
chains are the chains that iptables itself provides in each table. Examples are the filter table's
|
||||
`FORWARD` chain and the nat table's `PREROUTING` chain.
|
||||
|
||||
Cleanup finds the chains and tables of a previous generation by their names. Thus the daemon itself
|
||||
needs these names, but they are not an interface for operators. A rename thus needs a cleanup path
|
||||
for the old name. Without one, an upgraded node keeps the rules of the previous generation.
|
||||
|
||||
In this section, the daemon restores a network, endpoint or port binding when it sets it up again
|
||||
during its start. It restores each bridge network that exists at the start. With `live-restore`
|
||||
true, it also restores the endpoints and port bindings of the containers that are still running.
|
||||
|
||||
A daemon of one release does not show a rename of a previous release's names. The C row that pins
|
||||
the names that cleanup needs fails on such a rename.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | After the daemon has started, the host netns does not contain any table or chain that a previous generation left there with the other backend. The exceptions are `DOCKER-USER` and the `FORWARD` jump to it, which hold the operator's rules and stay in place. The tables and built-in chains that an `iptables` command creates by itself, such as the filter table and its `FORWARD` chain, are also exceptions. The overlay driver's `docker-overlay` table is also an exception. It is not asserted ([swarm-networking.md §12.1](swarm-networking.md#121-nftables)). This holds for a switch from iptables to nftables and for a switch from nftables to iptables. It holds across a package upgrade from *N−1* to *N* and across a same-binary restart. | B | `node` | `os`, `upgrade` |
|
||||
| The `DOCKER-ISOLATION` chain, which releases older than *N−1* created and *N* does not create, with the `FORWARD` jump to it that those releases installed, beside the chains of *N−1* | After the daemon has started, that chain or table does not remain. `DOCKER-INGRESS` is an exception, and [swarm-networking.md §12.3](swarm-networking.md#123-cross-generation-cleanup) covers it. | B | `node` | `os` |
|
||||
| A rule that the previous generation left in a built-in chain for a network, endpoint or port binding | After a switch from iptables to nftables, the rule does not remain after the daemon has restored that network, endpoint or port binding. This holds across a package upgrade from the previous release and across a same-binary restart. See [§4](#4-known-divergences--non-goals). | B | `node` | `os`, `upgrade` |
|
||||
| | When the daemon starts with the iptables backend, no `docker-bridges` table remains after the start. | C | `node` | `os` |
|
||||
| A jump from the filter table's `FORWARD` chain to `DOCKER-FORWARD` or to `DOCKER-ISOLATION-STAGE-1` | After the daemon starts with the nftables backend, some chains do not remain. These chains are `DOCKER` (in the filter and nat tables), `DOCKER-FORWARD`, `DOCKER-BRIDGE`, `DOCKER-CT`, `DOCKER-INTERNAL`, `DOCKER-ISOLATION-STAGE-1`, `DOCKER-ISOLATION-STAGE-2` and `DOCKER-ISOLATION`. The jumps to these chains from `FORWARD` and from the nat table's `PREROUTING` and `OUTPUT` do not remain either. | C | `node` | `os` |
|
||||
| The rules of the previous generation in built-in chains for a network, endpoint or port binding | After a switch from iptables to nftables, these rules remain until that network, endpoint or port binding is restored. They do not remain after that. | C | `node` | `os` |
|
||||
| | The names that cleanup needs are unchanged. They are the nftables table `docker-bridges`, and the iptables chains `DOCKER` (in the filter and nat tables), `DOCKER-FORWARD`, `DOCKER-BRIDGE`, `DOCKER-CT` and `DOCKER-INTERNAL`. | C | `node` | `none` |
|
||||
| | The policy of the iptables filter table's `FORWARD` chain is not modified when the daemon starts with the nftables backend. This includes a policy of DROP. | C\* | `node` | `none` |
|
||||
| The filter table's `FORWARD` chain with policy DROP | On the first start after a switch from iptables to nftables, the daemon logs a warning that contains `Network traffic for published ports may be dropped, iptables chain FORWARD has policy DROP.` | C\* | `log` | `none` |
|
||||
| | After a switch from iptables to nftables, a DROP rule that an operator placed in `DOCKER-USER` still drops the forwarded traffic that it matches. This includes traffic to published ports. | C\* | `client` | `os` |
|
||||
|
||||
\* These expectations are tier C while the nftables backend is experimental. They become tier B
|
||||
when the nftables backend is a supported feature.
|
||||
|
||||
### 1.4 Firewall backend and firewalld
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A daemon configured with a `firewall-backend` other than `iptables` or `nftables` | The daemon does not start. The daemon log contains an error. | B | `log` | `none` |
|
||||
| A daemon configured with a `firewall-backend` other than `iptables` or `nftables` | The daemon does not start. The daemon log contains an error that contains `invalid firewall-backend`. | C | `log` | `none` |
|
||||
| A node with firewalld running | A published port answers after a firewalld reload. | B | `client` | `os` |
|
||||
| A node with firewalld running | After a firewalld reload, once the daemon has applied its firewall configuration again, `GET /info` at API version 1.49 or later reports a `ReloadedAt` entry in `FirewallBackend.Info`. Its time is no earlier than the second in which the reload started. | C | `api` | `os` |
|
||||
| A node with firewalld running, and a running container on a bridge network that is not `internal`, whose `com.docker.network.bridge.gateway_mode_ipv4` is not `routed`, and whose `com.docker.network.bridge.enable_ip_masquerade` is not `false` | After a firewalld reload, a connection from the container to an IPv4 address of an underlay host succeeds. | B | `ctr` | `os` |
|
||||
| The iptables backend, with `ip-forward` and `iptables` true and `ip-forward-no-drop` false, and a host where `net.ipv4.ip_forward` is 0 | After the daemon has created a bridge network with IPv4, `net.ipv4.ip_forward` is 1. The filter table's `FORWARD` chain also has policy DROP. | B | `node` | `os` |
|
||||
| The iptables backend, with `ip-forward` and `ip6tables` true and `ip-forward-no-drop` false, and a host where `net.ipv6.conf.all.forwarding` or `net.ipv6.conf.default.forwarding` is 0 | After the daemon has created a bridge network with IPv6, both are 1. The ip6tables filter table's `FORWARD` chain also has policy DROP. | B | `node` | `os` |
|
||||
| The iptables backend, with `ip-forward` and `iptables` true and `ip-forward-no-drop` false, firewalld running, and a bridge network with IPv4 that the daemon created on a host where `net.ipv4.ip_forward` was 0 | After a firewalld reload, the filter table's `FORWARD` chain has policy DROP. | B | `node` | `os` |
|
||||
| The iptables backend, with `ip-forward` and `ip6tables` true and `ip-forward-no-drop` false, firewalld running, and a bridge network with IPv6 that the daemon created on a host where `net.ipv6.conf.all.forwarding` or `net.ipv6.conf.default.forwarding` was 0 | After a firewalld reload, the ip6tables filter table's `FORWARD` chain has policy DROP. | B | `node` | `os` |
|
||||
| A daemon with the nftables backend, `ip-forward` true and the default bridge enabled, on a host where `net.ipv4.ip_forward` is 0, and without containers that `live-restore` keeps running | The daemon does not start. It prints an error that contains `IPv4 forwarding is disabled`. | C\* | `log` | `none` |
|
||||
| A daemon with the nftables backend and `ip-forward` true, on a host where `net.ipv6.conf.all.forwarding` or `net.ipv6.conf.default.forwarding` is 0 | `POST /networks/create` for a bridge network with `EnableIPv6` set responds 500, with a message that contains `IPv6 global forwarding is disabled`. See [Host requirements](#host-requirements). | C\* | `api` | `none` |
|
||||
| A daemon with the iptables backend and `iptables` true, on a host without the `iptables` command | The daemon does not start. The daemon log contains `iptables not found`. See [Host requirements](#host-requirements). | C | `log` | `none` |
|
||||
| A daemon with the iptables backend and `ip6tables` true, on a host without the `ip6tables` command | The daemon starts. The daemon log contains `ip6tables is enabled, but cannot set up ip6tables chains`. See [Host requirements](#host-requirements). | C | `log` | `none` |
|
||||
| A daemon with the iptables backend and `ip6tables` true, on a host without the `ip6tables` command | `POST /networks/create` for a bridge network with `EnableIPv6` set responds with a 4xx or 5xx status. The network is not created. | B | `api` | `none` |
|
||||
| A daemon with the nftables backend, on a host without the `nft` command | The daemon does not start. The daemon log contains `failed to find nft tool`. See [Host requirements](#host-requirements). | C\* | `log` | `none` |
|
||||
|
||||
\* This expectation is tier C while the nftables backend is experimental. It becomes tier B when
|
||||
the nftables backend is a supported feature.
|
||||
|
||||
### 1.5 Rule content
|
||||
|
||||
The rows in this subsection pin the content of the bridge driver's rules, so that a change to it is
|
||||
visible in review. A deliberate change to this content is not a breakage.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | For each configuration that the repository keeps a reference iptables ruleset for in `integration/network/bridge/iptablesdoc`, the bridge driver's IPv4 iptables rules match that reference. The reference gives the matches and target of each rule, the order of the rules in each chain, and the names of the chains. The configurations include a Swarm service that publishes a port. | C | `node` | `none` |
|
||||
| | For each configuration that the repository keeps a reference nftables ruleset for in `integration/network/bridge/nftablesdoc`, the bridge driver's IPv4 nftables ruleset matches that reference. The reference gives the matches and verdict of each rule, the order of the rules in each chain, and the table and chain names. | C | `node` | `none` |
|
||||
|
||||
## 2. Network and endpoint options
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | `POST /networks/create` for a bridge network whose `com.docker.network.driver.mtu` is not an integer from 0 to 2⁶³ − 1 responds 400. The message includes the value. | B | `api` | `none` |
|
||||
| | `docker network create --driver bridge` with a `com.docker.network.driver.mtu` option that is not an integer from 0 to 2⁶³ − 1 exits non-zero. It prints an error that includes the value. | B | `api` | `cli` |
|
||||
| | `POST /networks/create` for a bridge network with an MTU that the kernel refuses responds with a 4xx or 5xx status. `GET /networks` then does not list the network. | B | `api` | `none` |
|
||||
| | `POST /networks/create` for a bridge network with an MTU that the kernel refuses responds 500. See [§4](#4-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| | `docker network create --driver bridge` with an MTU that the kernel refuses exits non-zero. `docker network ls` then does not list the network. | B | `api` | `cli` |
|
||||
| A bridge network whose `com.docker.network.driver.mtu` is positive | The interfaces of the network's containers have that MTU. See [§4](#4-known-divergences--non-goals). | B | `ctr` | `os` |
|
||||
| | A bridge network's `com.docker.network.driver.mtu` of 0 leaves the MTU of the bridge and of its containers' interfaces as the kernel set it. | C | `node`, `ctr` | `none` |
|
||||
| A daemon configured with a negative `mtu` | The daemon does not start. The daemon log contains an error. | B | `log` | `none` |
|
||||
| A daemon configured with a negative `mtu` | The daemon does not start. The daemon log contains an error that contains `invalid default MTU`. | C | `log` | `none` |
|
||||
| A daemon configured with an `mtu` that the kernel cannot apply to the default bridge, and without containers that `live-restore` keeps running | The daemon does not start. See [§4](#4-known-divergences--non-goals). | B | `log` | `none` |
|
||||
| A daemon configured with an `mtu` that the kernel cannot apply to the default bridge, with containers that `live-restore` keeps running | The daemon starts. The default bridge keeps its MTU. | B | `log`, `node` | `none` |
|
||||
| A daemon configured with a positive `mtu` | The interfaces of containers on the default bridge have that MTU. See [§4](#4-known-divergences--non-goals). | B | `ctr` | `os` |
|
||||
| A daemon started with `--mtu 0` | The default bridge and its containers' interfaces keep the MTU that the kernel gave them. See [§4](#4-known-divergences--non-goals). | C | `node`, `ctr` | `none` |
|
||||
| A daemon configured with an `mtu` from 2³² to 2⁶³ − 1 whose remainder modulo 2³² the kernel accepts | The daemon starts. The default bridge gets that remainder as its MTU. For example, 4294968796 gives an MTU of 1500. See [§4](#4-known-divergences--non-goals). | C | `log`, `node` | `none` |
|
||||
|
||||
## 3. Data path
|
||||
|
||||
### 3.1 Isolation
|
||||
|
||||
The rows in this subsection hold with `userland-proxy` true and with `userland-proxy` false,
|
||||
unless a row names a value.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A daemon with `allow-direct-routing` false, and a target network whose `com.docker.network.bridge.gateway_mode_ipv4` is `nat` or not set | A container on one bridge network cannot reach a container on another bridge network at that container's IPv4 address, on any port. It reaches the published ports of that container at the node's address. | B | `ctr` → `ctr` | `os` |
|
||||
| A bridge network with `com.docker.network.bridge.enable_icc` set to `false` | A container cannot reach another container on the network at that container's address, on any port. It reaches the published ports of that container at the node's address. | B | `ctr` → `ctr` | `os` |
|
||||
| | The node does not forward traffic between a container on an `internal` bridge network and an address outside the network's subnets, in either direction. This holds even when the container has a default route through the bridge, such as one that a container with `NET_ADMIN` adds itself. Traffic that the node delivers to its own addresses is not forwarded, and this row does not cover it. | B | `ctr`, `underlay` | `os` |
|
||||
| An `internal` bridge network whose `com.docker.network.bridge.gateway_mode_ipv4` is not `isolated`, and without `com.docker.network.bridge.inhibit_ipv4` | A process in the node's host network namespace reaches a container on the network at the container's IPv4 address. | B | `node` | `os` |
|
||||
| An `internal` bridge network whose `com.docker.network.bridge.gateway_mode_ipv4` is not `isolated`, and without `com.docker.network.bridge.inhibit_ipv4` | A container on the network reaches a process in the node's host network namespace that listens on the bridge's IPv4 address. | B | `ctr` | `os` |
|
||||
| An `internal` bridge network whose `com.docker.network.bridge.gateway_mode_ipv4` is not `isolated`, without `com.docker.network.bridge.inhibit_ipv4`, and a container on the network with `NET_ADMIN` that has added a route via the bridge's IPv4 address to an address of the node outside the network's subnets | The container reaches a process in the node's host network namespace that listens on that address. | C | `ctr` | `os` |
|
||||
| An `internal` bridge network with `com.docker.network.bridge.gateway_mode_ipv4` set to `isolated` | The bridge does not have an IPv4 address. A process in the node's host network namespace cannot reach the network's containers at their IPv4 addresses. | B | `node` | `os` |
|
||||
| An `internal` bridge network with `com.docker.network.bridge.gateway_mode_ipv4` set to `isolated` | A container on the network cannot reach a process in the node's host network namespace at any IPv4 address of the node. This holds even when the container has a route to that address that a container with `NET_ADMIN` adds itself. | B | `ctr` | `os` |
|
||||
| A daemon with `allow-direct-routing` false, a container on a bridge network whose `com.docker.network.bridge.gateway_mode_ipv4` is `nat` or not set, and an underlay host with a route to the container's IPv4 address via the node's underlay address | Traffic from that host to the container's IPv4 address arrives on an interface that is not in the network's `com.docker.network.bridge.trusted_host_interfaces`. The traffic does not reach the container on any port. | B | `underlay` → `ctr` | `os` |
|
||||
| A daemon with `allow-direct-routing` true, a container on a bridge network whose `com.docker.network.bridge.gateway_mode_ipv4` is `nat` or not set, and an underlay host with a route to the container's IPv4 address via the node's underlay address | Traffic from that host to the container's IPv4 address reaches the container's published ports only. | B | `underlay` → `ctr` | `os` |
|
||||
| A container on a bridge network whose `com.docker.network.bridge.gateway_mode_ipv4` is `nat` or not set, and an underlay host with a route to the container's IPv4 address via the node's underlay address | When traffic from that host to the container's IPv4 address arrives on an interface in the network's `com.docker.network.bridge.trusted_host_interfaces`, it reaches the container's published ports only. | B | `underlay` → `ctr` | `os` |
|
||||
| A bridge network whose `com.docker.network.bridge.trusted_host_interfaces` names several interfaces separated by colons, such as `eth1:eth2`, and whose `com.docker.network.bridge.gateway_mode_ipv4` is `nat` or not set, and an underlay host with a route to the IPv4 address of a container on the network via the node's underlay address | When traffic from that host to the container's IPv4 address arrives on any of the named interfaces, it reaches the container's published ports. | C | `underlay` → `ctr` | `none` |
|
||||
| A daemon with `iptables` true, on a host where `net.ipv4.ip_forward` is 1, and a bridge network with `com.docker.network.bridge.enable_icc` set to `false`, or any bridge network when `userland-proxy` is false | The sysctl `net.bridge.bridge-nf-call-iptables` is 1. See [Host requirements](#host-requirements). | C | `node` | `none` |
|
||||
| A Linux host whose kernel does not offer veth pairs, and a created or stopped container on a bridge network | `POST /containers/{id}/start` for the container responds 500, with a message that contains `pair interfaces`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
|
||||
### 3.2 Port publishing
|
||||
|
||||
The rows in this subsection hold with `userland-proxy` true and with `userland-proxy` false,
|
||||
unless a row names a value.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | `POST /containers/{id}/start` for a created or stopped container that is on a bridge network and publishes a port without a host port responds 204. `GET /containers/{id}/json` then reports the assigned host port in `NetworkSettings.Ports`. A client reaches the container on that host port at the node's address. | B | `api`, `client` | `os` |
|
||||
| A Linux host whose kernel does not offer SCTP, and a created or stopped container on a bridge network that publishes an SCTP port | `POST /containers/{id}/start` for the container responds with a 4xx or 5xx status, with a message that contains `failed to create socket for` and `/sctp`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
| A container that is on a bridge network and listens on a port that it does not publish | A client that connects to the node's address, on any port, does not reach that listener. | B | `client`, `ctr` | `os` |
|
||||
| | A port published on `127.0.0.1` is reachable from the node at `127.0.0.1`. | B | `node` | `os` |
|
||||
| | A port published on `127.0.0.1` is not reachable from a client at any address of the node. This includes a packet to `127.0.0.1` that a client on the node's link sends in an Ethernet frame addressed to the node's MAC address. | B | `client` | `os` |
|
||||
| | A port published on an address of the node is reachable at that address, from the node and from a client. It is not reachable at the node's other addresses, `127.0.0.1` included. | B | `node`, `client` | `os` |
|
||||
| A bridge network with IPv6 enabled, and a daemon with `ip6tables` true | A port published without a host address is reachable at the node's IPv6 addresses. | B | `client` | `os` |
|
||||
| A bridge network with IPv6 enabled, and a daemon with `ip6tables` true | A client connects to a port published without a host address, at one of the node's IPv6 addresses. The container observes the IPv6 address of the client as the source address. | B | `client` → `ctr` | `os` |
|
||||
| A bridge network with IPv6 enabled, and a daemon with `ip6tables` true and `userland-proxy` false | A port published without a host address is not reachable from a client at an IPv6 link-local address of the node. | C | `client` | `os` |
|
||||
| A daemon with `userland-proxy` false, and a bridge network | The sysctl `net.ipv4.conf.<bridge>.route_localnet` is 1, where `<bridge>` is the network's bridge interface. See [Host requirements](#host-requirements). | C | `node` | `none` |
|
||||
|
||||
### 3.3 Egress
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A bridge network without the options `com.docker.network.bridge.enable_ip_masquerade`, `com.docker.network.bridge.gateway_mode_ipv4` and `com.docker.network.host_ipv4` | An underlay host observes an address of the node as the source address of IPv4 egress from the network's containers. | B | `underlay` | `os` |
|
||||
| A bridge network with `com.docker.network.bridge.enable_ip_masquerade` set to `false`, and without `com.docker.network.bridge.gateway_mode_ipv4` | An underlay host observes the container's address as the source address of IPv4 egress from a container on the network. | B | `underlay` | `os` |
|
||||
| A daemon with `allow-direct-routing` false, and a bridge network with `com.docker.network.bridge.enable_ip_masquerade` set to `false` and without `com.docker.network.bridge.gateway_mode_ipv4` | An underlay host replies to IPv4 egress from a container on the network. The reply arrives on an interface that is not in the network's `com.docker.network.bridge.trusted_host_interfaces`. The reply does not reach the container. | B | `underlay` → `ctr` | `os` |
|
||||
| A daemon with `allow-direct-routing` true, and a bridge network with `com.docker.network.bridge.enable_ip_masquerade` set to `false` and without `com.docker.network.bridge.gateway_mode_ipv4` | An underlay host replies to IPv4 egress from a container on the network. The reply reaches the container. | B | `underlay` → `ctr` | `os` |
|
||||
| A bridge network with `com.docker.network.bridge.enable_ip_masquerade` set to `false` and without `com.docker.network.bridge.gateway_mode_ipv4` | An underlay host replies to IPv4 egress from a container on the network. The reply arrives on an interface in the network's `com.docker.network.bridge.trusted_host_interfaces`. The reply reaches the container. | B | `underlay` → `ctr` | `os` |
|
||||
| A bridge network with `com.docker.network.host_ipv4` set to an address of the node, `com.docker.network.bridge.enable_ip_masquerade` not `false`, and `com.docker.network.bridge.gateway_mode_ipv4` not `routed` | An underlay host observes that address as the source address of IPv4 egress from the network's containers. | B | `underlay` | `os` |
|
||||
|
||||
### 3.4 Gateway modes
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A bridge network with `com.docker.network.bridge.gateway_mode_ipv4` set to `routed` | An underlay host observes the container's address as the source address of IPv4 egress from a container on the network. | B | `underlay` | `os` |
|
||||
| A container on a bridge network with `com.docker.network.bridge.gateway_mode_ipv4` set to `routed`, and a client with a route to the container's IPv4 address via the node's address | Traffic from the client to the container's IPv4 address reaches the container's published ports, and not its other ports. | B | `client` | `os` |
|
||||
| A bridge network with `com.docker.network.bridge.gateway_mode_ipv4` set to `routed` | A client that connects to the node's IPv4 address, on a published port of a container on the network, does not reach the container. | B | `client` | `os` |
|
||||
| A container on a bridge network with `com.docker.network.bridge.gateway_mode_ipv4` set to `nat-unprotected`, and a client with a route to the container's IPv4 address via the node's address | Traffic from the client to the container's IPv4 address reaches every port of the container. | B | `client` | `os` |
|
||||
| | `POST /networks/create` for a bridge network with `Internal` not true and with `com.docker.network.bridge.gateway_mode_ipv4` or `com.docker.network.bridge.gateway_mode_ipv6` set to `isolated` responds with a 4xx or 5xx status. `GET /networks` then does not list the network. | B | `api` | `none` |
|
||||
| | `POST /networks/create` for a bridge network with `Internal` not true and with `com.docker.network.bridge.gateway_mode_ipv4` or `com.docker.network.bridge.gateway_mode_ipv6` set to `isolated` responds 500. The message contains `gateway mode 'isolated' can only be used for an internal network`. See [§4](#4-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| | `docker network create --driver bridge --opt com.docker.network.bridge.gateway_mode_ipv4=isolated` without `--internal` exits non-zero and prints the daemon's error. `docker network ls` then does not list the network. | B | `api` | `cli` |
|
||||
| | `POST /networks/create` for a bridge network with `com.docker.network.bridge.gateway_mode_ipv4` or `com.docker.network.bridge.gateway_mode_ipv6` set to a value other than `nat`, `nat-unprotected`, `routed` and `isolated` responds with a 4xx or 5xx status. `GET /networks` then does not list the network. | B | `api` | `none` |
|
||||
| | `POST /networks/create` for a bridge network with `com.docker.network.bridge.gateway_mode_ipv4` or `com.docker.network.bridge.gateway_mode_ipv6` set to a value other than `nat`, `nat-unprotected`, `routed` and `isolated` responds 400. The message contains `unknown gateway mode`. | C | `api` | `none` |
|
||||
| | `docker network create --driver bridge` with the option `com.docker.network.bridge.gateway_mode_ipv4` set to a value other than `nat`, `nat-unprotected`, `routed` and `isolated` exits non-zero and prints the daemon's error. `docker network ls` then does not list the network. | B | `api` | `cli` |
|
||||
|
||||
### 3.5 Default gateway
|
||||
|
||||
[network-common.md §3](network-common.md#3-default-gateway) covers which endpoint gives a container
|
||||
its default route, for every network driver. The rows in this subsection hold with `userland-proxy`
|
||||
true and with `userland-proxy` false, unless a row names a value.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A container with endpoints on two or more bridge networks that are not `internal` and whose `com.docker.network.bridge.gateway_mode_ipv4` is `nat` or not set | An IPv4 client connects to a published port at the node's address. The connection reaches the container at its address on the network of its IPv4 default route. This holds after a connect or disconnect changes that network. | B | `client`, `ctr` | `os` |
|
||||
|
||||
### 3.6 Hosts without IPv6
|
||||
|
||||
These rows hold on a Linux host booted with `ipv6.disable=1`. Such a host has no IPv6 stack, so it
|
||||
does not have `/proc/sys/net/ipv6`.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A Linux host booted with `ipv6.disable=1`, and a daemon with its default settings | The daemon starts. `GET /info` responds 200. | B | `api` | `none` |
|
||||
| A Linux host booted with `ipv6.disable=1`, and a created container on a bridge network without IPv6 | `POST /containers/{id}/start` for the container responds 204. | B | `api` | `none` |
|
||||
| A Linux host booted with `ipv6.disable=1`, and a running container on a bridge network without IPv6 that publishes a port without a host address | A client reaches the container on that port at the node's IPv4 address. | B | `client` | `os` |
|
||||
| A Linux host booted with `ipv6.disable=1` | `POST /networks/create` for a bridge network with `EnableIPv6` set responds with a 4xx or 5xx status. `GET /networks` then does not list the network. | B | `api` | `none` |
|
||||
|
||||
## 4. Known divergences & non-goals
|
||||
|
||||
| Topic | Current behavior | Depends on |
|
||||
| --- | --- | --- |
|
||||
| Rules left in built-in iptables chains after a switch to nftables | The daemon deletes them only for the networks, endpoints and port bindings that it restores. Rules for anything that it does not restore stay in the built-in chains ([§1.3](#13-cross-generation-cleanup)). | `none` |
|
||||
| Firewalld reload under the nftables backend | Logged as failures. A firewalld reload does not change the daemon's nftables tables. On each reload, the daemon still adds its rules for each bridge network again, and nftables refuses each rule that exists. The daemon log then contains these warning-level entries for each reload: (1) one entry per bridge network that contains `ReapplyNetworkLevelRules is not implemented for nftables`; (2) one entry per endpoint that contains `Failed to reconfigure Endpoint` and `rule exists`, for an endpoint on a network that is not `internal` and whose gateway mode for IPv4 or IPv6 is `nat`, the default, on a daemon with `allow-direct-routing` false; (3) one entry per endpoint that publishes a port that contains `Failed to reconfigure NAT` and `rule exists`. The daemon's tables do not change. | `none` |
|
||||
| Out-of-range bridge MTU error | The error does not name the option. `POST /networks/create` with an MTU that the kernel does not accept responds 500 with `invalid argument`. A request can have IPv6 enabled and an MTU that the kernel accepts for IPv4 but not for IPv6. That request responds 500 with a message that contains `Cannot read IPv6 setup for bridge` and `no such file or directory` ([§2](#2-network-and-endpoint-options)). | `none` |
|
||||
| Bridge MTU from 2³² to 2⁶³ − 1 | The value is applied modulo 2³². This applies to `com.docker.network.driver.mtu` and to the daemon-wide `mtu`. A daemon-wide `mtu` whose remainder the kernel accepts does not stop the daemon. `POST /networks/create` accepts a value whose remainder the kernel accepts. For example, 4294968796 is applied as 1500. `GET /networks/{id}` then reports the requested value in `Options`. `POST /networks/create` refuses a value whose remainder the kernel refuses, as the *Out-of-range bridge MTU error* entry describes ([§2](#2-network-and-endpoint-options)). | `none` |
|
||||
| Daemon-wide `mtu` error | A value that the kernel cannot apply stops the daemon with an error that ends in `error creating default "bridge" network: invalid argument`. This error does not name the setting ([§2](#2-network-and-endpoint-options)). | `none` |
|
||||
| Daemon-wide `mtu` of 0 in daemon.json | Replaced by the default. A `"mtu"` of 0 in daemon.json is replaced by 1500 when the file is merged with the flags. The daemon then sets 1500 on the default bridge and its containers' interfaces. Only `--mtu 0` leaves the MTU that the kernel gave them ([§2](#2-network-and-endpoint-options)). | `none` |
|
||||
| HTTP 500 for refused requests | The daemon refuses each request below on purpose, but the request responds 500. 500 is the status for a fault in the daemon. The requests are: (1) `POST /networks/create` for a bridge network with an MTU that the kernel refuses. (2) `POST /networks/create` for a bridge network with `com.docker.network.bridge.gateway_mode_ipv4` or `com.docker.network.bridge.gateway_mode_ipv6` set to `isolated`, when `Internal` is not true ([§2](#2-network-and-endpoint-options), [§3.4](#34-gateway-modes)). | `none` |
|
||||
|
||||
## Deliberately not asserted
|
||||
|
||||
- The counters of Moby's own rules.
|
||||
- The content of Moby's own chains and tables, and their names, outside the configurations that
|
||||
[§1.5](#15-rule-content) pins. The exceptions are `DOCKER-USER` and the names that
|
||||
[§1.3](#13-cross-generation-cleanup) pins for cleanup. A backend change replaces exactly this
|
||||
content. A B row asserts the packet-level effect of the rules, or an anchor in
|
||||
[§1](#1-firewall-anchors--host-integration), and never the content.
|
||||
- Which packet-filtering technology is in use, except where
|
||||
[§1](#1-firewall-anchors--host-integration) pins a backend-specific anchor.
|
||||
- Whether an operator's `DOCKER-USER` rules survive a firewalld reload. That is firewalld's
|
||||
behavior. After a reload, the daemon recreates the chain and the jump to it. It does not restore
|
||||
the chain's rules.
|
||||
@@ -0,0 +1,199 @@
|
||||
Container DNS surface area
|
||||
==========================
|
||||
|
||||
Scope: name resolution inside a container on a user-defined network, whatever the network's
|
||||
driver. This covers the `/etc/resolv.conf` that the daemon writes for the container and the
|
||||
embedded DNS resolver behind it, as observable from outside the daemon.
|
||||
|
||||
This is a stub. It holds expectations that do not depend on any one driver. They come from the
|
||||
work on the [Swarm networking surface area](swarm-networking.md) and from reviews of the specs'
|
||||
coverage. It is not yet a complete account. Its conventions for tiers, vantage points and
|
||||
dependencies are in [README.md](README.md).
|
||||
|
||||
A container sends its DNS queries to the embedded DNS resolver, which is a DNS server that the
|
||||
daemon runs for the container. The rows state what the embedded resolver answers, and what it
|
||||
forwards to upstream nameservers. A name resolves to an address when the embedded resolver answers a
|
||||
DNS query for the name with that address. A container has external access when its default route is
|
||||
on a network that is not `internal`, and replies to its egress traffic reach it. On a bridge network
|
||||
with `com.docker.network.bridge.enable_ip_masquerade` set to `false`, replies can fail to reach it
|
||||
([bridge-networking.md §3.3](bridge-networking.md#33-egress)). The host's `resolv.conf` is the
|
||||
`/etc/resolv.conf` that the daemon reads, even on a host that runs systemd-resolved.
|
||||
|
||||
On Linux nodes, the containers that follow do not use the embedded resolver, and nothing here
|
||||
applies to them, other than the rows whose state names the default bridge:
|
||||
|
||||
- A container attached only to the default bridge network.
|
||||
- A container that uses the host's network.
|
||||
|
||||
## Host requirements
|
||||
|
||||
These requirements hold on Linux nodes. The conventions for this section are in
|
||||
[README.md](README.md#host-requirements). The embedded resolver needs NAT rules in the container's
|
||||
network namespace under the active firewall backend. It needs them whether or not the daemon's
|
||||
`iptables` setting is true.
|
||||
|
||||
### Kernel facilities
|
||||
|
||||
| Facility | Needed when | Kernel options | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| nftables with the NAT expression | The nftables backend | `NF_TABLES`, `NF_TABLES_IPV4`, `NFT_NAT`. `NFT_CHAIN_NAT_IPV4` on kernels that have it. | Every section | The [bridge-networking.md](bridge-networking.md#host-requirements) nftables facility and its pins |
|
||||
|
||||
### Programs
|
||||
|
||||
| Program | Needed when | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- |
|
||||
| `iptables`, accepting the nat table and the `DNAT` and `SNAT` targets | The iptables backend | Every section | The [§4](#4-port-53-in-the-container) rows about a host without the `iptables` command |
|
||||
| `nft` | The nftables backend | Every section | The [bridge-networking.md](bridge-networking.md#14-firewall-backend-and-firewalld) row about a host without the `nft` command |
|
||||
|
||||
The daemon does not depend on which implementation of `iptables` the host has. The kernel options
|
||||
that the program needs depend on that implementation:
|
||||
|
||||
| Program | Implementation | Kernel options |
|
||||
| --- | --- | --- |
|
||||
| `iptables` | Either | `NETFILTER`, `NETFILTER_XTABLES`, `NF_CONNTRACK`, `NF_NAT`, `NETFILTER_XT_NAT` |
|
||||
| `iptables` | iptables-legacy | `IP_NF_IPTABLES`, `IP_NF_NAT`. `IP_NF_IPTABLES_LEGACY` from Linux 6.9, and `NETFILTER_XTABLES_LEGACY` from Linux 6.17. |
|
||||
| `iptables` | iptables-nft | `NF_TABLES`, `NF_TABLES_IPV4`, `NFT_COMPAT`, `NFT_NAT`. Before Linux 5.1, `NFT_CHAIN_NAT_IPV4` in place of `NFT_NAT`. |
|
||||
|
||||
### Host configuration
|
||||
|
||||
| Configuration | Needed when | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- |
|
||||
| External network access, and nameservers in the host's `resolv.conf` that resolve public names | Always | The [§3](#3-forwarding) row about public DNS names | Not pinned |
|
||||
|
||||
## 1. `resolv.conf`
|
||||
|
||||
This section holds on Linux nodes.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | The `/etc/resolv.conf` nameserver is `127.0.0.11`. | C | `ctr` | `none` |
|
||||
| A container whose DNS options include a valid `ndots` | `/etc/resolv.conf` has the `ndots` value from the container's DNS options. | B | `ctr` | `none` |
|
||||
| A container whose DNS options are empty, and a host `resolv.conf` with a valid `ndots` | `/etc/resolv.conf` has the `ndots` value from the host's `resolv.conf`. | B | `ctr` | `none` |
|
||||
| A container whose DNS options include any option | `/etc/resolv.conf` has none of the options in the host's `resolv.conf`, `ndots` included. | C | `ctr` | `none` |
|
||||
| A container whose options in effect do not include a valid `ndots` | `/etc/resolv.conf` contains `options ndots:0`. This applies when the options do not set `ndots`, and when all of the `ndots` values in them are invalid. | C | `ctr` | `none` |
|
||||
| A container created with search domains | `/etc/resolv.conf` has the search domains that the container was created with. | B | `ctr` | `none` |
|
||||
| A container created without search domains, and a daemon with `dns-search` set | `/etc/resolv.conf` has the daemon's `dns-search`. | B | `ctr` | `none` |
|
||||
| A container created without search domains, and a daemon without `dns-search` | `/etc/resolv.conf` has the search list from the host's `resolv.conf`. | B | `ctr` | `none` |
|
||||
| | `/etc/resolv.conf` does not contain a search domain of `.` given with `--dns-search` or in the daemon's `dns-search`. | B | `ctr` | `none` |
|
||||
| A container whose only search domain, given with `--dns-search` or in the daemon's `dns-search`, is `.` | `/etc/resolv.conf` does not have a `search` line, even if the host's `resolv.conf` has one. | B | `ctr` | `none` |
|
||||
| | Apart from `ndots`, `/etc/resolv.conf` lists the options in effect as they were given. | B | `ctr` | `none` |
|
||||
| A running container | After a process in the container changes the container's `/etc/resolv.conf`, the file keeps the process's content. A connect or disconnect of the container does not change the file. | B | `ctr` | `none` |
|
||||
| A host whose `/etc/resolv.conf` names a nameserver other than 127.0.0.53, and a running container attached only to the default bridge whose `/etc/resolv.conf` has not changed since the daemon wrote it | After `POST /networks/{id}/connect` to a user-defined network responds 200, the container resolves the names of the containers on that network. It resolves other names through the nameservers in the host's `resolv.conf`, loopback nameservers included. See [§5](#5-known-divergences--non-goals). | B | `ctr` | `none` |
|
||||
| A running container attached only to the default bridge, whose `/etc/resolv.conf` has not changed since the daemon wrote it | After `POST /networks/{id}/connect` to a user-defined network responds 200, the `/etc/resolv.conf` nameserver is `127.0.0.11`. | C | `ctr` | `none` |
|
||||
| A running container that started with an endpoint on a user-defined network | The container's `/etc/resolv.conf` keeps the content that it had at start. This holds after later connects and disconnects. | C | `ctr` | `none` |
|
||||
| A running container that started attached only to the default bridge, and that a later connect attached to a user-defined network | The container's `/etc/resolv.conf` keeps the content that it had after that connect. This holds after later connects and disconnects. | C | `ctr` | `none` |
|
||||
| A running container that has had an endpoint on a user-defined network at any time since it started | After the container disconnects from every user-defined network, the embedded resolver continues to answer the container's queries. See [§5](#5-known-divergences--non-goals). | C | `ctr` | `none` |
|
||||
|
||||
A container's DNS options are the options that it was created with. If it was created without DNS
|
||||
options, they are the daemon's `dns-opts`. The options in effect are the container's DNS options
|
||||
when it has any. Otherwise, they are the options in the host's `resolv.conf`. A valid `ndots` is an
|
||||
integer of 0 or more. `/etc/resolv.conf` keeps a valid `ndots` above 15 as given, although glibc
|
||||
treats any `ndots` above 15 as 15.
|
||||
|
||||
## 2. Names and answers
|
||||
|
||||
In this section, the querying container is the container that sends the DNS query. The embedded
|
||||
resolver has records for a name on a network when an endpoint on that network has that name. An
|
||||
endpoint's names are its container's name, short ID and hostname, and its network aliases. In
|
||||
`<name>.<network>`, `<name>` stands for a name and `<network>` for the name of a network. Both are
|
||||
placeholders, not literal text.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | A container's name resolves to its address on a network it shares with the querying container. See [§5](#5-known-divergences--non-goals). | B | `ctr` | `none` |
|
||||
| | A container's name and its network aliases resolve whatever the case of the query: a container named `Web` resolves as `web` and as `WEB`. | B | `ctr` | `none` |
|
||||
| | A network alias given to a container's endpoint resolves to the container's address on that network, for containers on that network only. | B | `ctr` | `none` |
|
||||
| Several containers on a network whose endpoints have the same network alias, where no other network of the querying container has an endpoint with that alias | The alias resolves to the addresses of all of those containers on that network. | B | `ctr` | `none` |
|
||||
| | A container's short ID (the first 12 hex digits of its ID) and its hostname resolve as its name does. | C | `ctr` | `none` |
|
||||
| A container on a user-defined network | `GET /containers/{id}/json` lists in `NetworkSettings.Networks.<network>.DNSNames` the container's name, and then the aliases of its endpoint on that network. The name and each alias resolve on that network. | B | `api`, `ctr` | `none` |
|
||||
| A container on a user-defined network | `NetworkSettings.Networks.<network>.DNSNames` in `GET /containers/{id}/json` lists the container's short ID and then its hostname after the aliases. It does not list a name more than once. | C | `api` | `none` |
|
||||
| A running container whose hostname is not its name | After `POST /containers/{id}/rename` for the container responds 204, `NetworkSettings.Networks.<network>.DNSNames` in `GET /containers/{id}/json` has the new name for each user-defined network. It does not have the old name. Its other entries do not change. | C | `api` | `none` |
|
||||
| Linux nodes with a container attached to the default bridge | `NetworkSettings.Networks.bridge.DNSNames` in `GET /containers/{id}/json` is empty. | C | `api` | `none` |
|
||||
| Linux nodes with a running container attached to the default bridge | After `POST /containers/{id}/rename` for the container responds 204, `NetworkSettings.Networks.bridge.DNSNames` lists the new name, the short ID and, when it differs from the short ID, the hostname. See [§5](#5-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| | A name resolves when an endpoint with that name exists on any network that the querying container is attached to. | B | `ctr` | `os` (Windows) |
|
||||
| | A query for a name does not get the address of an endpoint on a network that the querying container is not attached to. | B | `ctr` | `none` |
|
||||
| A created or stopped standalone container | After `POST /containers/{id}/start` for the container responds 204, its name and its aliases on each of its networks resolve for the other containers on that network. This holds whether or not the container has a healthcheck. | B | `ctr` | `none` |
|
||||
| A running standalone container | After `POST /networks/{id}/connect` for the container responds 200, its name and its aliases on that network resolve for the other containers on the network. This holds whether or not the container has a healthcheck. | B | `ctr` | `none` |
|
||||
| A running container with an endpoint on a user-defined network | After `POST /networks/{id}/disconnect` for the container from that network responds 200, the other containers on that network do not resolve its name or aliases to its address on that network. | B | `ctr` | `none` |
|
||||
| A running container with endpoints on user-defined networks | After `POST /containers/{id}/stop` for the container responds 204, the other containers on each of those networks do not resolve its name or aliases to its address on that network. | B | `ctr` | `none` |
|
||||
| A node that is not in a Swarm, and a running container whose hostname is not its name | After `POST /containers/{id}/rename` for the container responds 204, its new name resolves and its old name does not, for containers on each of its networks. | B | `ctr` | `none` |
|
||||
| A node in a Swarm, and a running container whose hostname is not its name, with an endpoint on a Swarm-scoped network of a multi-host driver, such as `overlay` | After `POST /containers/{id}/rename` for the container responds 204, its new name resolves and its old name does not, for containers on that network on its node. See [§5](#5-known-divergences--non-goals). [swarm-networking.md §5](swarm-networking.md#5-service-discovery) covers containers on other nodes. | B | `ctr` | `none` |
|
||||
| | The embedded resolver answers an A or AAAA query for a name that it has records for from those records. This holds for a single-label name too, whatever `ndots` the options in effect set ([§1](#1-resolvconf)). | B | `ctr` | `none` |
|
||||
| A container on a network with IPv6 that it shares with the querying container, other than an overlay network, where no other network of the querying container has an endpoint with that container's name | An AAAA query for that name returns the container's IPv6 address on the shared network. | B | `ctr` | `none` |
|
||||
| Records in the embedded resolver for a name that are all A records, or all AAAA records | A query of the other type for that name gets an empty `NOERROR` response. The embedded resolver does not forward this query upstream. | B | `ctr` | `none` |
|
||||
| | An MX query for a name that the embedded resolver has A or AAAA records for gets an empty `NOERROR` response. The embedded resolver does not forward this query upstream. | B | `ctr` | `none` |
|
||||
| | The embedded resolver forwards upstream a query of a type other than A, AAAA, MX, PTR and SRV. This holds for a name that it has records for too. | C | `ctr` | `none` |
|
||||
| A Linux node that is not in a Swarm | Several of the querying container's networks can have records for a name. The embedded resolver answers an A or AAAA query for that name from the records of the first network only. The order is: (1) Endpoints with a higher `GwPriority` first. (2) Networks that are not `internal` first. (3) Endpoints with a gateway or default route for more IP versions first. (4) Network names in byte order. When that network does not have an address of the family asked for, the query gets an empty `NOERROR` response. This holds even if another network has one. | C | `ctr` | `none` |
|
||||
| | Over UDP, an answer from the embedded resolver's own records lists all of the name's addresses when it fits in a size limit. The limit is the larger of 512 bytes and the query's EDNS0 buffer size. When the answer does not fit, the embedded resolver truncates it and sets the TC bit. Over TCP, the answer lists all of the name's addresses. | B | `ctr` | `none` |
|
||||
| | In answers from the embedded resolver's own records, the order of a name's addresses varies from one query to the next. | C | `ctr` | `none` |
|
||||
| | The records in an answer from the embedded resolver's own records have a TTL of 600 seconds. This holds for A, AAAA and PTR answers. | C | `ctr` | `none` |
|
||||
| | Reverse (PTR) lookups resolve for the IPv4 and IPv6 addresses of containers on the querying container's networks. | B | `ctr` | `os` (Windows) |
|
||||
| | A forward lookup of the name that a reverse lookup returns for a container's address returns that address. | B | `ctr` | `none` |
|
||||
| | The answer to a reverse lookup of a container's address is `<name>.<network>.`, where `<name>` is the container's name and `<network>` is the network's name. It is never an alias, the short ID or the hostname. | C | `ctr` | `os` (Windows) |
|
||||
| A querying container attached to the network `<network>`, and records for `<name>` on that network | A query for `<name>.<network>` resolves to the addresses of `<name>` on that network only. See [§5](#5-known-divergences--non-goals). | B | `ctr` | `none` |
|
||||
| | In a `<name>.<network>` query, `<network>` matches a network only when its case is the same as the case of the network's name. `<name>` matches whatever its case. | C | `ctr` | `none` |
|
||||
| A network whose name is a valid DNS label and that the querying container is not attached to, where no network of the querying container has an endpoint named `<name>.<network>`, `<network>` being that network's name | The embedded resolver forwards a query for `<name>.<network>` upstream. | C | `ctr` | `none` |
|
||||
| A querying container attached to the network `<network>`, where an endpoint is named `<name>`, and an endpoint named `<name>.<network>` on one of the querying container's networks | `<name>.<network>` resolves to the address of the endpoint with the whole name. It does not resolve to the address of `<name>` on the network `<network>`. | C | `ctr` | `none` |
|
||||
| Linux nodes with a container with `<name>:<alias>` in the `Links` of its endpoint settings for a user-defined network | The alias resolves to the named container's address on that network. It resolves only for queries from the container that has the link. | B | `ctr` | `none` |
|
||||
| Linux nodes | An alias in the `Links` of any of the querying container's endpoints takes precedence over the names and network aliases on the container's networks. The alias resolves to the linked container even when another container on one of those networks has that name. | C | `ctr` | `none` |
|
||||
|
||||
## 3. Forwarding
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A container that has external access | Public DNS names resolve. | B | `ctr` | `os` |
|
||||
| Linux nodes with a container attached only to `internal` networks | The embedded resolver does not forward the container's queries to any nameserver that the container cannot reach from its own network namespace. It also does not forward them to the nameservers in the host's `resolv.conf`. | B | `ctr` | `none` |
|
||||
| Linux nodes with a container attached only to `internal` networks and created without `--dns` nameservers, whose daemon does not have a `dns` setting | A query for a name that the embedded resolver does not have a record for gets SERVFAIL. When the options in effect set `ndots`, a single-label A or AAAA query gets an empty `NOERROR` response instead ([§1](#1-resolvconf), [§5](#5-known-divergences--non-goals)). | C | `ctr` | `none` |
|
||||
| A container that has external access and was created with `--dns` nameservers | The embedded resolver forwards to the nameservers set with `--dns`, in place of the host's nameservers. | B | `ctr` | `os` |
|
||||
| A daemon with a `dns` setting, and a container that has external access and was created without `--dns` nameservers | The embedded resolver forwards to the nameservers in the daemon's `dns` setting, in place of the host's nameservers. | B | `ctr` | `os` |
|
||||
| A container attached only to `internal` networks and created with `--dns` set to the address of a nameserver in another container on one of those networks | The container created with `--dns` resolves names through that nameserver. | B | `ctr` | `os` |
|
||||
| A daemon with a `dns` setting that holds the address of a nameserver in a container on an `internal` network, and another container attached only to `internal` networks, that network among them, and created without `--dns` nameservers | The container created without `--dns` nameservers resolves names through that nameserver. | B | `ctr` | `os` |
|
||||
| Linux nodes with a container that started with an endpoint on a user-defined network, has external access and was created without `--dns` nameservers, whose daemon does not have a `dns` setting | The embedded resolver forwards to the nameservers in the host's `resolv.conf`, loopback nameservers included. It queries them from the host's network namespace. | B | `ctr` | `os` |
|
||||
| Linux nodes | The embedded resolver queries nameservers set with `--dns` or in the daemon's `dns` setting from the container's network namespace. | B | `ctr` | `os` |
|
||||
| | The embedded resolver forwards a query to the next nameserver when the current nameserver does not respond, or responds SERVFAIL or REFUSED. When each nameserver that it forwards to does not respond, or responds SERVFAIL or REFUSED, the client gets SERVFAIL. | B | `ctr` | `none` |
|
||||
| | Over UDP, when no nameserver that the embedded resolver forwards a query to responds, the client gets SERVFAIL within 4 seconds times the number of those nameservers. | C | `ctr` | `none` |
|
||||
| | The embedded resolver returns a nameserver's response other than SERVFAIL or REFUSED, NXDOMAIN included, to the client. It does not ask the next nameserver. | B | `ctr` | `none` |
|
||||
| | The embedded resolver forwards to at most the first three nameservers. | C | `ctr` | `none` |
|
||||
| Linux nodes | The embedded resolver does not forward to a nameserver `127.0.0.11` set with `--dns` or in the daemon's `dns` setting. That nameserver does not count toward the three nameservers. | C | `ctr` | `none` |
|
||||
| Linux nodes with a container whose only nameserver, set with `--dns` or in the daemon's `dns` setting, is `127.0.0.11` | A query for a name that the embedded resolver does not have a record for gets SERVFAIL. When the options in effect set `ndots`, a single-label A or AAAA query gets an empty `NOERROR` response instead ([§1](#1-resolvconf), [§5](#5-known-divergences--non-goals)). | C | `ctr` | `none` |
|
||||
| | The embedded resolver forwards upstream an A or AAAA query for a multi-label name that does not match any container, service or alias on the querying container's networks. This does not hold for `<name>.<network>` when the querying container is attached to a network named `<network>` and the embedded resolver has records for `<name>` on it ([§2](#2-names-and-answers)). | B | `ctr` | `none` |
|
||||
| A container on a Windows node, or a container on a Linux node whose options in effect, per [§1](#1-resolvconf), do not set `ndots` | The embedded resolver forwards a single-label A or AAAA query upstream when it does not have a record for the name. | B | `ctr` | `none` |
|
||||
| A container on a Linux node whose options in effect, per [§1](#1-resolvconf), set `ndots` to 1 or more | A single-label A or AAAA query gets an empty `NOERROR` response when the embedded resolver does not have a record for the name. The embedded resolver does not forward the query upstream. See [§5](#5-known-divergences--non-goals). | C | `ctr` | `none` |
|
||||
|
||||
The row for an `ndots` of 1 or more is tier C because the behavior itself is in question.
|
||||
[moby/moby#53157](https://github.com/moby/moby/issues/53157) proposes to put container names in a
|
||||
domain on the search list. That change would make this behavior unnecessary. An `ndots` of 0, or an
|
||||
invalid `ndots`, has the same effect. That effect is a bug for these values. See
|
||||
[§5](#5-known-divergences--non-goals).
|
||||
|
||||
## 4. Port 53 in the container
|
||||
|
||||
This section holds on Linux nodes.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| | A process in the container can bind UDP and TCP port 53 on `127.0.0.11` without an error. No packet reaches that socket. | C | `ctr` | `os` |
|
||||
| A daemon with the iptables backend and `iptables` false, on a host without the `iptables` command, and a running container with an endpoint on a user-defined network | A DNS query from the container to `127.0.0.11` does not get an answer. See [Host requirements](#host-requirements). | C | `ctr` | `os` |
|
||||
| A daemon with the iptables backend and `iptables` false, on a host without the `iptables` command, and a running container with an endpoint on a user-defined network | The daemon log contains `setting up DNAT/SNAT rules failed`. See [Host requirements](#host-requirements). | C | `log` | `none` |
|
||||
|
||||
## 5. Known divergences & non-goals
|
||||
|
||||
| Topic | Current behavior | Depends on |
|
||||
| --- | --- | --- |
|
||||
| Single-label forwarding with an `ndots` of 0 or an invalid `ndots` | Not forwarded. This applies when the options in effect set `ndots` to 0 or to an invalid value. A single-label A or AAAA query then gets an empty `NOERROR` response when the embedded resolver does not have a record for the name. This is the same as for an `ndots` of 1 or more ([§3](#3-forwarding)). Only an `ndots` of 1 or more is meant to have that effect ([moby/moby#53104](https://github.com/moby/moby/issues/53104)). The effect shows in Docker-in-Docker. The inner daemon reads the outer container's `resolv.conf` as the `resolv.conf` of its host. The inner daemon finds the `ndots:0` that the outer daemon added. The containers of the inner daemon cannot resolve names that only the outer resolver knows. | `none` |
|
||||
| Single-label queries after a live restore | Forwarded. After the daemon restarts with `live-restore` true, the embedded resolver forwards upstream a single-label A or AAAA query that it does not have a record for, from a container that kept running. This occurs whatever `ndots` the options in effect set ([§3](#3-forwarding)). | `none` |
|
||||
| Renaming a container on a Swarm node | Not applied. On a node in a Swarm, `POST /containers/{id}/rename` changes the container's records only on Swarm-scoped networks of multi-host drivers, such as overlay. It does not change them on other networks, such as user-defined bridge networks and Swarm-scoped networks of node-local drivers. The container's new name does not resolve there. Its old name continues to resolve there after the rename. The old name continues to resolve there after the container is removed ([§2](#2-names-and-answers)). | `none` |
|
||||
| Records after a node leaves a Swarm | Lost. Take a node in a Swarm, whether it created the Swarm with `POST /swarm/init` or joined it, that leaves it with `POST /swarm/leave`. After the leave, the names of the containers that were running on the node's user-defined networks before the leave stop resolving there. A container that starts after the leave gets records that resolve ([§2](#2-names-and-answers)). | `none` |
|
||||
| `resolv.conf` after the embedded resolver starts | Not updated. After the embedded resolver starts for a container, `/etc/resolv.conf` does not change again while the container runs. A container that disconnects from every user-defined network keeps `nameserver 127.0.0.11` and the embedded resolver. A container started on the default bridge alone gets the host's nameservers ([§1](#1-resolvconf)). | `none` |
|
||||
| Upstream nameservers after a connect from the default bridge | On a Linux node whose `/etc/resolv.conf` names only 127.0.0.53, take a container that started attached only to the default bridge and later connects to a user-defined network. It forwards to the nameservers in `/run/systemd/resolve/resolv.conf`, not to 127.0.0.53 ([§1](#1-resolvconf), [§3](#3-forwarding)). | `os` |
|
||||
| `DNSNames` on the default bridge after a rename | Filled in. On Linux nodes, `GET /containers/{id}/json` does not report `DNSNames` for the default bridge, which does not have name resolution. After a rename of the running container, it reports the new name, the short ID and the hostname there. These names do not resolve for a container attached to the default bridge alone. On a node that is not in a Swarm, a container attached to the default bridge and a user-defined network resolves the new name and the short ID to the renamed container's address on the default bridge ([§2](#2-names-and-answers)). | `none` |
|
||||
| Reverse-lookup round trip on Windows | Fails. A reverse lookup returns `<name>.<network>`. The embedded resolver on a Windows node does not answer names in that form. It forwards a query for such a name upstream ([§2](#2-names-and-answers)). | `none` |
|
||||
| Nameservers on `internal` networks on Windows | Not applied. A Windows endpoint on an `internal` network does not get the nameservers set with `--dns` or in the daemon's `dns` setting. A Windows container attached only to `internal` networks cannot resolve names through a nameserver on one of them ([§3](#3-forwarding)). | `none` |
|
||||
|
||||
## Deliberately not asserted
|
||||
|
||||
- The comments the daemon writes into `resolv.conf`, such as its header and its record of where
|
||||
`ndots` came from.
|
||||
- The ports that the embedded resolver listens on.
|
||||
- The firewall rules that the daemon programs in a container's network namespace: whether they
|
||||
exist, their content and names, and the hooks and priorities of their base chains.
|
||||
@@ -0,0 +1,48 @@
|
||||
ipvlan networking surface area
|
||||
==============================
|
||||
|
||||
Scope: the ipvlan network driver, as observable from outside the daemon. Every expectation here
|
||||
holds on Linux nodes. A row about `POST /networks/create` or `docker network create` holds for a
|
||||
network that is not Swarm-scoped.
|
||||
|
||||
This spec is a stub. It holds expectations that belong to the ipvlan driver. They come from reviews
|
||||
of the specs' coverage. It is not yet a complete account of the ipvlan driver.
|
||||
[README.md](README.md) gives its conventions for tiers, vantage points and dependencies.
|
||||
[network-common.md](network-common.md) covers what every network driver must do, including the
|
||||
rows for a driver that does not accept an endpoint MAC address.
|
||||
|
||||
## Host requirements
|
||||
|
||||
The conventions for this section are in [README.md](README.md#host-requirements). The ipvlan driver
|
||||
also needs the requirements in [network-common.md](network-common.md#host-requirements). The
|
||||
daemon needs Linux 4.2 or later for an ipvlan network. It checks the kernel version, and does not
|
||||
probe for ipvlan links. The check guards against defects in the L2 mode of ipvlan before Linux 4.2,
|
||||
which a probe cannot detect.
|
||||
|
||||
### Kernel facilities
|
||||
|
||||
| Facility | Needed when | Kernel options | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| ipvlan links, in Linux 4.2 or later | Always | `IPVLAN` | Every section | The [§2](#2-networks-and-interfaces) rows about a kernel older than 4.2 and about a host without ipvlan links |
|
||||
| Dummy links | A network without a `parent` | `DUMMY` | Every section | Not pinned. The error for a missing dummy link does not name the facility. |
|
||||
| VLAN links | A network with a `parent` of the form `<interface>.<VLAN ID>` | `VLAN_8021Q` | Every section | The [§2](#2-networks-and-interfaces) row about a host without VLAN links |
|
||||
|
||||
## 1. Endpoint options
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| An ipvlan network | `POST /containers/{id}/start` for a created or stopped container whose endpoint on that network sets `MacAddress`, created at API version 1.44 or later, responds 500. `POST /networks/{id}/connect` to that network for a running container, with `MacAddress` set, at API version 1.54 or later, also responds 500. Both messages contain `ipvlan interfaces do not support custom mac address assignment`. See [§3](#3-known-divergences--non-goals). | C | `api` | `none` |
|
||||
|
||||
## 2. Networks and interfaces
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A Linux host whose kernel version is older than 4.2 | `POST /networks/create` for an ipvlan network responds 500, with a message that contains `kernel version failed to meet the minimum ipvlan kernel requirement`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
| A Linux host whose kernel does not offer ipvlan links, and a created or stopped container with an endpoint on an ipvlan network | `POST /containers/{id}/start` for the container responds 500, with a message that contains `failed to create the ipvlan port`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
| A Linux host whose kernel does not offer VLAN links | `POST /networks/create` for an ipvlan network whose `parent` is `<interface>.<VLAN ID>`, for an existing interface `<interface>` that does not yet have that VLAN sub-interface, responds 500, with a message that contains `vlan link`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
|
||||
## 3. Known divergences & non-goals
|
||||
|
||||
| Topic | Current behavior | Depends on |
|
||||
| --- | --- | --- |
|
||||
| HTTP 500 for refused requests | The daemon refuses the requests below on purpose, but they respond 500. 500 is the status for a fault in the daemon. The first request is `POST /containers/{id}/start` for a container whose endpoint on an ipvlan network sets `MacAddress`. The second is `POST /networks/{id}/connect` to an ipvlan network for a running container, with `MacAddress` set, at API version 1.54 or later ([§1](#1-endpoint-options)). | `none` |
|
||||
@@ -0,0 +1,211 @@
|
||||
Common networking surface area
|
||||
==============================
|
||||
|
||||
Scope: the behavior that every network driver must have, and the Engine API requests for networks
|
||||
and endpoints that do not depend on the network's driver. This spec describes them as observable
|
||||
from outside the daemon.
|
||||
|
||||
This spec is a stub. It holds expectations that every network driver shares. They come from the work
|
||||
on the other specs and from reviews of the specs' coverage. It is not yet a complete account.
|
||||
[README.md](README.md) gives its conventions for tiers, vantage points and dependencies.
|
||||
[swarm-networking.md](swarm-networking.md) and [bridge-networking.md](bridge-networking.md) cover
|
||||
what is specific to their drivers. [container-dns.md](container-dns.md) covers name resolution.
|
||||
|
||||
In this spec, a predefined network is a network that `GET /networks` with the filter `type=builtin`
|
||||
lists. The rows here also hold for Swarm-scoped networks and on nodes in a Swarm, unless a row
|
||||
narrows them. [swarm-networking.md §2](swarm-networking.md#2-network-lifecycle--api-surface) covers
|
||||
the requests that are specific to Swarm-scoped networks. Where a row of swarm-networking.md says
|
||||
something different about the same request or state, that row holds.
|
||||
|
||||
## Host requirements
|
||||
|
||||
The kernel facilities are requirements of each Linux node. The host configuration is a requirement
|
||||
of each node. The conventions for this section are in [README.md](README.md#host-requirements). The
|
||||
specs of the drivers list what each driver needs:
|
||||
[bridge-networking.md](bridge-networking.md#host-requirements),
|
||||
[swarm-networking.md](swarm-networking.md#host-requirements) and
|
||||
[ipvlan-networking.md](ipvlan-networking.md#host-requirements).
|
||||
|
||||
### Kernel facilities
|
||||
|
||||
| Facility | Needed when | Kernel options | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Network namespaces | Always | `NET_NS` | Every section | The [§6](#6-daemon-restart) rows about network namespaces |
|
||||
| macvlan links | A `macvlan` network | `MACVLAN` | [§2](#2-endpoint-interfaces) | The [§2](#2-endpoint-interfaces) row about a host without macvlan links |
|
||||
| Dummy links | A `macvlan` network without a `parent` | `DUMMY` | [§2](#2-endpoint-interfaces) | Not pinned. The error for a missing dummy link does not name the facility. |
|
||||
| VLAN links | A `macvlan` network with a `parent` of the form `<interface>.<VLAN ID>` | `VLAN_8021Q` | [§2](#2-endpoint-interfaces) | The [§2](#2-endpoint-interfaces) row about a host without VLAN links |
|
||||
|
||||
### Host configuration
|
||||
|
||||
| Configuration | Needed when | Needed for | Pinned by |
|
||||
| --- | --- | --- | --- |
|
||||
| The host's firewall passes the traffic that a row involves, to and from the node, its containers and its published ports. The rules and the firewalld configuration that the daemon programs are not part of the host's firewall. | Always, except in a row whose state names firewalld | Every row about traffic, in every spec | Not pinned |
|
||||
|
||||
## 1. Network API
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| An existing network that is not Swarm-scoped and not predefined | `POST /networks/create` with the network's name responds 409. `GET /networks` then lists only one network with that name. | B | `api` | `none` |
|
||||
| An existing network that is not Swarm-scoped and not predefined | `docker network create` with the network's name exits non-zero and prints the daemon's error. `docker network ls` then lists only one network with that name. | B | `api` | `cli` |
|
||||
| A Linux node that is not in a Swarm | `POST /networks/create` with the name of a predefined network, `default` or `container` responds 403. `POST /networks/create` with a name that starts with `container:` also responds 403. `GET /networks` then does not list a new network with that name. | B | `api:solo` | `none` |
|
||||
| A Linux manager | `POST /networks/create` with the name `bridge` or `host` responds 409. It responds 403 when the name is `none`, `default` or `container`, or starts with `container:`. This holds for the default driver and for `overlay`. | C | `api:mgr` | `none` |
|
||||
| Linux nodes | `docker network create` with the name of a predefined network exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| Linux nodes | `DELETE /networks/{id}` for a predefined network responds 403. `GET /networks` then still lists the network. | B | `api` | `none` |
|
||||
| Linux nodes | `docker network rm` for a predefined network exits non-zero and prints the daemon's error. `docker network ls` then still lists the network. | B | `api` | `cli` |
|
||||
| A network that is not Swarm-scoped, with a running container attached to it | `DELETE /networks/{id}` for the network responds 403. `GET /networks` then still lists the network. | B | `api` | `none` |
|
||||
| A network that is not Swarm-scoped, with a running container attached to it | `docker network rm` for the network exits non-zero and prints the daemon's error. `docker network ls` then still lists the network. | B | `api` | `cli` |
|
||||
| A running container attached to a network | `POST /networks/{id}/connect` to that network, for the container, responds with a 4xx or 5xx status. The container's addresses on that network do not change. | B | `api` | `none` |
|
||||
| A running container attached to a network that is not Swarm-scoped | `POST /networks/{id}/connect` to that network, for the container, responds 403. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A running container attached to a network | `docker network connect` naming that network and the container exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| A created or stopped container whose `NetworkSettings.Networks` lists a network that is not Swarm-scoped | `POST /networks/{id}/connect` to that network, for the container, responds 200. The container's endpoint settings for the network do not change. | C | `api` | `none` |
|
||||
| A node that is not in a Swarm, with a created or stopped container | `POST /networks/{id}/connect` for the container, with an `{id}` that is not the name or an ID prefix of any network, responds 200. `POST /containers/{id}/start` for the container then responds 404 while no network has that name. See [§7](#7-known-divergences--non-goals). | C | `api:solo` | `none` |
|
||||
| A created or stopped container | `POST /networks/{id}/connect` for the container, with an `{id}` that is not the name or an ID prefix of any network, responds 200. A network with that name is then created. `POST /containers/{id}/start` for the container then responds 204, and the container has an address on that network. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A network that is not Swarm-scoped, and a container that is not attached to it | `POST /containers/create` with a static address in `IPAMConfig` that is outside every subnet of that network responds 400, and does not create a container. `POST /networks/{id}/connect` to that network, for the container, with that address, responds 400, and does not attach the container. | B | `api` | `none` |
|
||||
| Linux nodes with a container that is not attached to the network `bridge` | `POST /containers/create` with a static address in `IPAMConfig` for the network `bridge` responds 400, and does not create a container. `POST /networks/{id}/connect` to the network `bridge`, for the container, with a static address in `IPAMConfig`, responds 400, and does not attach the container. | B | `api` | `none` |
|
||||
| | `docker run` and `docker network connect` with `--ip` outside every subnet of a network that is not Swarm-scoped exit non-zero and print the daemon's error. On Linux nodes, they do the same with `--ip` for the network `bridge`. | B | `api` | `cli` |
|
||||
| | At API version 1.43 or earlier, `POST /containers/create` with more than one network in `NetworkingConfig.EndpointsConfig` responds 400, and does not create a container. | B | `api` | `none` |
|
||||
| A container, and a network that it is not attached to | `POST /networks/{id}/disconnect` for the container from that network responds with a 4xx or 5xx status. | B | `api` | `none` |
|
||||
| A container, and a network that it is not attached to | `POST /networks/{id}/disconnect` for the container from that network responds 500, with a message that contains `is not connected to`. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A container, and a network that it is not attached to | `docker network disconnect` naming that network and the container exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| A running container with an endpoint on a network, from its start or from a later connect | `POST /networks/{id}/disconnect` for the container responds 200. `GET /containers/{id}/json` then does not list the network in `NetworkSettings.Networks`, and the container still runs. | B | `api` | `none` |
|
||||
| A running container with an endpoint on a network, from its start or from a later connect | After `POST /networks/{id}/disconnect` for the container responds 200, the container does not have its interface on that network. | B | `ctr` | `os` |
|
||||
| A running container with an endpoint on a network, from its start or from a later connect | `docker network disconnect` for the container exits zero. `docker inspect` for the container then does not list the network in `NetworkSettings.Networks`. | B | `api` | `cli` |
|
||||
| A created or stopped container whose `NetworkSettings.Networks` lists a network | `POST /networks/{id}/disconnect` for the container from that network responds 200. `GET /containers/{id}/json` then does not list the network in `NetworkSettings.Networks`. | B | `api` | `none` |
|
||||
| A created or stopped container whose `NetworkSettings.Networks` lists a network | `docker network disconnect` naming that network and the container exits zero. `docker inspect` for the container then does not list the network in `NetworkSettings.Networks`. | B | `api` | `cli` |
|
||||
| A running container, and a network whose driver does not accept an endpoint MAC address ([§5](#5-driver-properties)) | `POST /networks/{id}/connect` to that network, for the container, with `MacAddress` set, at API version 1.54 or later, fails. `GET /containers/{id}/json` then lists the network in `NetworkSettings.Networks`, and `GET /networks/{id}` does not list the container. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| | `GET /networks/{id}` responds 200 with the network when `{id}` is the network's full ID or its name. It does the same when `{id}` is a prefix of the network's ID that the ID of no other network starts with, and that is not the name of another network. | B | `api` | `none` |
|
||||
| | `docker network inspect` prints the network when it is given the network's full ID or its name. It does the same when it is given a prefix of the network's ID that the ID of no other network starts with, and that is not the name of another network. | B | `api` | `cli` |
|
||||
| | `GET /networks/{id}` responds 404 when `{id}` is not the full ID, the name or a prefix of the ID of any network. | B | `api` | `none` |
|
||||
| Two or more networks whose IDs start with a prefix that is not a network name | `GET /networks/{id}` with that prefix as `{id}` responds with a 4xx or 5xx status. | B | `api` | `none` |
|
||||
| Two or more networks whose IDs start with a prefix that is not a network name | `GET /networks/{id}` with that prefix as `{id}` responds 400, with a message that contains `is ambiguous`. | C | `api` | `none` |
|
||||
| A network whose name is a prefix of the ID of another network that is not Swarm-scoped | `GET /networks/{id}` with that name as `{id}` returns the network with that name. | C | `api` | `none` |
|
||||
| | `GET /networks` with the filter `type=custom` lists the networks that the filter `type=builtin` does not list. | B | `api` | `none` |
|
||||
| Linux nodes | `GET /networks` with the filter `type=builtin` lists `bridge`, `host` and `none`. | C | `api` | `none` |
|
||||
| | `GET /networks` with a filter term that it does not support responds 400. `GET /networks` with a value that a filter term does not accept also responds 400. | B | `api` | `none` |
|
||||
| | `docker network ls --filter` exits non-zero and prints the daemon's error for a filter term that the daemon does not support. It does the same for a value that a filter term does not accept. | B | `api` | `cli` |
|
||||
| | The `type` filter term of `GET /networks` accepts the values `builtin` and `custom`. | C | `api` | `none` |
|
||||
| | `GET /networks` with both `type=builtin` and `type=custom` lists a subset of the networks. The subset can differ from one request to the next. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A network that is not Swarm-scoped and not predefined, with no running container attached to it, other than `docker_gwbridge` on a Linux node in a Swarm that has an ingress network | `GET /networks` with the filter `dangling=true` lists the network. | B | `api` | `none` |
|
||||
| A network that is not Swarm-scoped, and that is predefined or has a running container attached to it | `GET /networks` with the filter `dangling=false` lists the network. | B | `api` | `none` |
|
||||
| A node that is not in a Swarm | `POST /networks/prune` responds 200. It deletes each network that is not Swarm-scoped, not predefined and not config-only, and that no running container is attached to. `NetworksDeleted` in the response lists the names of the networks that it deletes. | B | `api:solo` | `none` |
|
||||
| A node that is not in a Swarm | `docker network prune --force` exits zero, and prints the names of the networks that it deletes. `docker network ls` then does not list those networks. | B | `api:solo` | `cli` |
|
||||
| A node that is not in a Swarm, and a `docker_gwbridge` network that no running container is attached to | `POST /networks/prune` deletes `docker_gwbridge`. See [§7](#7-known-divergences--non-goals). | C | `api:solo` | `none` |
|
||||
| A Linux node in a Swarm that has an ingress network | `POST /networks/prune` does not delete `docker_gwbridge`. `GET /networks/docker_gwbridge` lists `ingress-sbox` in `Containers`. | C | `api:mgr`, `api:wkr` | `none` |
|
||||
| | `POST /networks/create` with `ConfigOnly` true responds 201. `GET /networks/{id}` then reports `ConfigOnly` true. | B | `api` | `none` |
|
||||
| A network that is not Swarm-scoped, created with `ConfigFrom` naming a config-only network | `GET /networks/{id}` for the network reports the config-only network's `IPAM` configuration. [swarm-networking.md §2.2](swarm-networking.md#22-swarm-scoped-networks) covers a Swarm-scoped network. | B | `api` | `none` |
|
||||
| A network created with `ConfigFrom` naming a config-only network | The network's containers get addresses in the subnets of the config-only network's `IPAM` configuration. | B | `ctr` | `none` |
|
||||
| | `POST /networks/create` with `ConfigFrom` naming a network that does not exist responds with a 4xx or 5xx status. `GET /networks` then does not list a network with the request's `Name`. | B | `api` | `none` |
|
||||
| | `POST /networks/create` with `ConfigFrom` naming a network that does not exist responds 404, with a message that contains `configuration network "<name>" does not exist`. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| | `docker network create --config-from` naming a network that does not exist exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| A config-only network, and a network that is not Swarm-scoped whose `ConfigFrom` names it | `DELETE /networks/{id}` for the config-only network responds 403. `GET /networks` then still lists the config-only network. [swarm-networking.md §2.2](swarm-networking.md#22-swarm-scoped-networks) covers a Swarm-scoped network. | B | `api` | `none` |
|
||||
| A config-only network, and a network that is not Swarm-scoped whose `ConfigFrom` names it | `docker network rm` for the config-only network exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| A config-only network | `POST /containers/create` for a container on that network responds 201. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A config-only network, and a created or stopped container that is not attached to it | `POST /networks/{id}/connect` to that network, for the container, responds 200. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A created container on a config-only network | `POST /containers/{id}/start` for the container responds with a 4xx or 5xx status. The container stays in the created state. | B | `api` | `none` |
|
||||
| A created container on a config-only network | `POST /containers/{id}/start` for the container responds 403. The container stays in the created state. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A config-only network | `docker run -d --network` naming that network exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| A running container, and a config-only network | `POST /networks/{id}/connect` to that network, for the container, responds with a 4xx or 5xx status. `GET /networks/{id}` then does not list the container. | B | `api` | `none` |
|
||||
| A running container, and a config-only network | `POST /networks/{id}/connect` to that network, for the container, responds 403, with a message that contains `cannot create endpoint on configuration-only network`. | C | `api` | `none` |
|
||||
| A running container, and a config-only network | After `POST /networks/{id}/connect` to that network, for the container, fails, `GET /containers/{id}/json` lists the network in `NetworkSettings.Networks`. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A running container, and a config-only network | `docker network connect` naming that network and the container exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
|
||||
## 2. Endpoint interfaces
|
||||
|
||||
The rows in this section hold for an endpoint of any network driver that adds an interface to the
|
||||
container ([§5](#5-driver-properties)). An endpoint requests an interface name when its
|
||||
`com.docker.network.endpoint.ifname` is set to that name.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| An endpoint that requests an interface name | The interface has that name. See [§7](#7-known-divergences--non-goals). | B | `task`, `ctr` | `os` |
|
||||
| Linux nodes with an endpoint that does not request an interface name, on a network of a built-in driver without `com.docker.network.container_iface_prefix` | The interface name is `eth<N>`. | C | `task`, `ctr` | `os` |
|
||||
| Linux nodes with a created or stopped container with an endpoint whose `com.docker.network.endpoint.ifname` the OS refuses | `POST /containers/{id}/start` for the container responds with a 4xx or 5xx status. | B | `api` | `none` |
|
||||
| Linux nodes with a running container | `POST /networks/{id}/connect` for the container, with an endpoint whose `com.docker.network.endpoint.ifname` the OS refuses, responds with a 4xx or 5xx status. | B | `api` | `none` |
|
||||
| Linux nodes with a created or stopped container with an endpoint whose `com.docker.network.endpoint.ifname` the OS refuses | `POST /containers/{id}/start` for the container responds 500, with a message that contains `error renaming interface` and the requested name. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| Linux nodes with a running container | `POST /networks/{id}/connect` for the container, with an endpoint whose `com.docker.network.endpoint.ifname` the OS refuses, responds 500. The message contains `error renaming interface` and the requested name. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| Linux nodes | `docker run -d` with a `com.docker.network.endpoint.ifname` that the OS refuses exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| Linux nodes with a running container | `docker network connect` for the container, with a `com.docker.network.endpoint.ifname` that the OS refuses, exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| Linux nodes | At API version 1.44 or later, `POST /containers/create` for a container with two endpoints that request the same interface name responds 201. So does a request for a container with an endpoint that requests `lo`. | C | `api` | `none` |
|
||||
| Linux nodes with a created or stopped container with two endpoints that request the same interface name | `POST /containers/{id}/start` for the container responds with a 4xx or 5xx status. | B | `api` | `none` |
|
||||
| Linux nodes with a created or stopped container with an endpoint that requests `lo` | `POST /containers/{id}/start` for the container responds with a 4xx or 5xx status. | B | `api` | `none` |
|
||||
| Linux nodes with a running container | `POST /networks/{id}/connect` for the container, with an endpoint that requests `lo`, responds with a 4xx or 5xx status. It does the same with an endpoint that requests the name of one of the container's interfaces. | B | `api` | `none` |
|
||||
| Linux nodes with a created or stopped container with two endpoints that request the same interface name, or with one that requests `lo` | `POST /containers/{id}/start` for the container responds 500, with a message that names the interface. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| Linux nodes with a running container | `POST /networks/{id}/connect` for the container, with an endpoint that requests `lo` or the name of one of its interfaces, responds 500. The message names the interface. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| Linux nodes | `docker run -d` exits non-zero and prints the daemon's error when two of the container's endpoints request the same interface name. It does the same when an endpoint requests `lo`. | B | `api` | `cli` |
|
||||
| Linux nodes with a running container | `docker network connect` for the container exits non-zero and prints the daemon's error when the endpoint requests `lo` or the name of one of its interfaces. | B | `api` | `cli` |
|
||||
| A running standalone container whose endpoint on a network has an `IPv4Address` or `IPv6Address` in `IPAMConfig` | Its interface on that network has that address. | B | `ctr` | `os` |
|
||||
| A running standalone container whose endpoint on a network has an `IPv4Address` or `IPv6Address` in `IPAMConfig` | `GET /containers/{id}/json` reports the address as `IPAddress` or `GlobalIPv6Address` in `NetworkSettings.Networks.<network>`. | B | `api` | `none` |
|
||||
| A running container created at API version 1.44 or later with an endpoint `MacAddress` on a network whose driver accepts an endpoint MAC address ([§5](#5-driver-properties)) | Its interface on that network has that MAC address. | B | `ctr` | `os` |
|
||||
| A running container, and a network whose driver accepts an endpoint MAC address ([§5](#5-driver-properties)) | After `POST /networks/{id}/connect` with `MacAddress` set, at API version 1.54 or later, responds 200, the container's interface on the network has that MAC address. | B | `ctr` | `os` |
|
||||
| A running container, and a network whose driver does not accept an endpoint MAC address ([§5](#5-driver-properties)) | `POST /networks/{id}/connect` to that network, for the container, with `MacAddress` set, at API version 1.54 or later, responds with a 4xx or 5xx status. `GET /networks/{id}` then does not list the container. | B | `api` | `none` |
|
||||
| A network whose driver does not accept an endpoint MAC address ([§5](#5-driver-properties)) | At API version 1.44 or later, `POST /containers/create` for a container whose endpoint on that network sets `MacAddress` responds 201. `POST /containers/{id}/start` for the container then responds with a 4xx or 5xx status, and the container stays in the created state. | C | `api` | `none` |
|
||||
| | `docker run -d` with `--mac-address` and with `--network` naming a network whose driver does not accept an endpoint MAC address ([§5](#5-driver-properties)) exits non-zero and prints the daemon's error. | B | `api` | `cli` |
|
||||
| A Linux host whose kernel does not offer macvlan links, and a created or stopped container with an endpoint on a `macvlan` network | `POST /containers/{id}/start` for the container responds 500, with a message that contains `failed to create the macvlan port`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
| A Linux host whose kernel does not offer VLAN links | `POST /networks/create` for a `macvlan` network whose `parent` is `<interface>.<VLAN ID>`, for an existing interface `<interface>` that does not yet have that VLAN sub-interface, responds 500, with a message that contains `vlan link`. See [Host requirements](#host-requirements). | C | `api` | `none` |
|
||||
|
||||
## 3. Default gateway
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A container attached only to `internal` networks | The container does not have a default route. See [§7](#7-known-divergences--non-goals). | B | `task`, `ctr` | `os` |
|
||||
| Linux nodes with a container with endpoints on two or more networks that are not `internal`, one with a higher `GwPriority` than the others | For each IP version that the endpoint gives the container a gateway or default route for, the container's default route goes through that endpoint. | B | `ctr` | `os` |
|
||||
| Linux nodes with a container with endpoints on two or more networks that are not `internal`, all with the same `GwPriority` | For each IP version, the container's default route is through the first endpoint that gives a gateway or default route for that version, in this order: (1) endpoints that give one for both IPv4 and IPv6, then endpoints that give one for a single version; (2) within each group, network names in byte order. | C | `ctr` | `none` |
|
||||
| | At API version 1.47 or earlier, `POST /containers/create` ignores `GwPriority` in `EndpointsConfig`. | B | `api`, `ctr` | `none` |
|
||||
| | `POST /networks/{id}/connect` applies `GwPriority` at every API version. | C | `api`, `ctr` | `none` |
|
||||
|
||||
## 4. `/etc/hosts`
|
||||
|
||||
This section holds on Linux nodes.
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| A container that does not use the host's network | For each of the container's addresses, `/etc/hosts` has a line that maps the address to the container's hostname. The exceptions are addresses on the ingress network and on `docker_gwbridge`. IPv6 addresses are also exceptions while IPv6 is disabled in the container, so that its loopback interface does not have an IPv6 address, for example with `net.ipv6.conf.all.disable_ipv6` set to 1. | B | `task`, `ctr` | `none` |
|
||||
| A container whose `Hostname` contains a dot, or that has a `Domainname` | Each line for the container's addresses gives two names. The first is the full name: the hostname, then a dot and the domain name if the container has one. The second is the part of the hostname before its first dot. | B | `ctr` | `none` |
|
||||
| A running container | After `POST /networks/{id}/connect` for the container responds 200, `/etc/hosts` has the lines for the container's addresses on that network. | B | `ctr` | `none` |
|
||||
| A running container with an endpoint on a network, from its start or from a later connect | After `POST /networks/{id}/disconnect` for the container responds 200, `/etc/hosts` does not have the lines for the container's addresses on that network. | B | `ctr` | `none` |
|
||||
| A daemon with `host-gateway-ips` set | An entry `<name>:host-gateway` in `HostConfig.ExtraHosts` adds a line to `/etc/hosts` for each address in `host-gateway-ips`. Each line maps the address to `<name>`. | B | `ctr` | `none` |
|
||||
| A daemon without `host-gateway-ips` | An entry `<name>:host-gateway` in `HostConfig.ExtraHosts` adds a line to `/etc/hosts` for each gateway address of the default bridge. This holds whatever networks the container is attached to. | C | `ctr` | `none` |
|
||||
| A daemon without `host-gateway-ips` and with `bridge` set to `none`, and a created container with an entry `<name>:host-gateway` in `HostConfig.ExtraHosts` | `POST /containers/{id}/start` for the container responds 500, with a message that contains `unable to derive the IP value for host-gateway`. The container stays in the created state. See [§7](#7-known-divergences--non-goals). | C | `api` | `none` |
|
||||
| A daemon without `host-gateway-ips` and with `bridge` set to `none` | `docker run` with `--add-host <name>:host-gateway` exits non-zero and prints the daemon's error. | C | `api` | `cli` |
|
||||
|
||||
## 5. Driver properties
|
||||
|
||||
Rows in this spec name a property of a network driver, not the driver itself, where the set of
|
||||
drivers with the property can change. This table gives the properties of the built-in drivers. It is
|
||||
tier C: it pins which drivers have each property today. A plugin driver has the properties that it
|
||||
implements. A cell is observed from `ctr`, in a container on a network of that driver, and depends
|
||||
on `os`. The `l2tunnel` and `ics` cells hold on platforms that can create those networks.
|
||||
|
||||
| Driver | Adds an interface to the container | Accepts an endpoint MAC address |
|
||||
| --- | --- | --- |
|
||||
| `bridge` | Yes | Yes |
|
||||
| `overlay` | Yes | Yes |
|
||||
| `macvlan` | Yes | Yes |
|
||||
| `ipvlan` | Yes | No |
|
||||
| `host` | No | Not applicable |
|
||||
| `null` (the `none` network) | No | Not applicable |
|
||||
| Windows: `nat`, `transparent`, `l2bridge`, `l2tunnel`, `ics` | Yes | Yes |
|
||||
|
||||
## 6. Daemon restart
|
||||
|
||||
| Applies to | Expectation | Tier | Observed from | Depends on |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Linux nodes | After `SIGKILL` and restart, every network namespace that the daemon created before the kill is removed, other than an overlay network's namespace and the namespace of a container that live-restore keeps running. This includes the namespaces of load-balancer and ingress sandboxes, and of containers that the daemon does not keep running. An overlay network's namespace is the one that holds the network's VXLAN device. A namespace that the daemon creates after the restart is a different namespace, even at the same path. The row is checked as soon as the daemon's API answers again, without a ceiling, on a node in a Swarm too. See [swarm-networking.md §13](swarm-networking.md#13-known-divergences--non-goals). | B | `node` | `os` |
|
||||
| Linux nodes with a running container, whose daemon has `live-restore` true | After `SIGKILL` and restart, the container keeps its network namespace and its interfaces. | B | `node`, `ctr` | `os` |
|
||||
|
||||
## 7. Known divergences & non-goals
|
||||
|
||||
| Topic | Current behavior | Depends on |
|
||||
| --- | --- | --- |
|
||||
| HTTP 500 for refused requests | Each request below is refused on purpose, but it responds 500. 500 is the status for a fault in the daemon. The requests are: (1) `POST /networks/{id}/disconnect` for a container that is not attached to the network. (2) `POST /containers/{id}/start` and `POST /networks/{id}/connect` for an endpoint with a `com.docker.network.endpoint.ifname` that the OS refuses, or that another endpoint of the container already uses. (3) `POST /containers/{id}/start` for a created or stopped container with a `host-gateway` entry in `HostConfig.ExtraHosts`, when the daemon does not have an address for `host-gateway`. (4) On a node in a Swarm, `POST /containers/{id}/start` for a created or stopped container that `POST /networks/{id}/connect` connected to a network that does not exist ([§1](#1-network-api), [swarm-networking.md §2.2](swarm-networking.md#22-swarm-scoped-networks), [§2](#2-endpoint-interfaces), [§4](#4-etchosts)). | `none` |
|
||||
| Networks that a created or stopped container cannot use | Accepted. For a created or stopped container, `POST /networks/{id}/connect` to a network that does not exist, or to a config-only network, responds 200. `POST /containers/create` for a container on a config-only network also responds 201. Only `POST /containers/{id}/start` refuses the network. For a network that does not exist, it responds 404 with `network <name> not found` on a node that is not in a Swarm. On a node in a Swarm, it responds 500 with a message that contains `network <name> not found`. For a config-only network, it responds 403 with `cannot create endpoint on configuration-only network`. If a network with the missing name is created before the start, the start attaches the container to it ([§1](#1-network-api)). | `none` |
|
||||
| Connecting a running container to a network it is already attached to | Refused with two different statuses. The request responds 403 on a network that is not Swarm-scoped, and 409 on an attachable Swarm-scoped network ([§1](#1-network-api), [swarm-networking.md §2.2](swarm-networking.md#22-swarm-scoped-networks)). | `none` |
|
||||
| HTTP 404 for a missing `ConfigFrom` network | `POST /networks/create` with `ConfigFrom` naming a network that does not exist responds 404. 404 is the status for a request whose own resource does not exist. Here the request is valid, and its body names the missing network ([§1](#1-network-api)). | `none` |
|
||||
| Several values for the `type` filter | Not combined. With both `type=builtin` and `type=custom` given, `GET /networks` tests each network against one of the two values, chosen at random. The response lists a different subset of the networks each time. As with other filter terms, a network is meant to match when it matches either value ([§1](#1-network-api)). | `none` |
|
||||
| Default route on `internal` overlays on Windows | A Windows container attached only to `internal` overlays has a default route ([§3](#3-default-gateway)). | `none` |
|
||||
| `ifname` option on Windows | Ignored. A Windows container's interface name does not follow `com.docker.network.endpoint.ifname` ([§2](#2-endpoint-interfaces)). | `none` |
|
||||
| Interface left behind after a failed connect | If `POST /networks/{id}/connect` for a running container fails at the rename, the interface stays in the container under its generated `veth` name. The interface is down and does not have addresses. Its host-side peer stays attached. The interface and the peer stay until the container's network namespace is destroyed. A failed connect is meant to leave nothing in place ([§2](#2-endpoint-interfaces)). | `none` |
|
||||
| Interface names the kernel rewrites | Accepted without an error. For a name that contains `%d`, the kernel gives the interface the first free name that matches, such as `eth0`. A later disconnect responds 200, but the interface stays in the container. The interface stays up and keeps its address ([§2](#2-endpoint-interfaces)). | `none` |
|
||||
| Interface-name collisions depend on attach order | Take a container with two endpoints. One endpoint asks for `eth0`. The name of the other endpoint is generated. The container starts or fails, depending on the order in which its endpoints are attached. This order is random. The endpoint that asks for the name is meant to get it, whatever the order ([§2](#2-endpoint-interfaces)). | `none` |
|
||||
| Network listed after a failed connect | Left in place. `POST /networks/{id}/connect` for a running container can fail when the daemon creates the endpoint. For example, it fails when the network's driver does not accept the endpoint, or when the network is config-only. After such a failure, `GET /containers/{id}/json` lists the network in `NetworkSettings.Networks`, but the container does not have an endpoint on it. A failed connect is meant to leave nothing in place ([§1](#1-network-api)). | `none` |
|
||||
| `docker_gwbridge` outside a Swarm | Pruned. On a node that is not in a Swarm, `docker_gwbridge` is not a predefined network. `POST /networks/prune` deletes it while no running container is attached to it. This includes a `docker_gwbridge` that an operator creates before the node joins a Swarm, to configure the gateway network ([§1](#1-network-api), [swarm-networking.md §8.1](swarm-networking.md#81-gateway-network)). | `none` |
|
||||
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user