mirror of
https://github.com/moby/moby.git
synced 2026-08-03 14:41:03 +00:00
Generate iptables documentation
In an integration test - run a daemon, capture iptables, and feed them to a markdown text/template describing them. Prep for repeating that, for different network configurations. Fail the test if the generated markdown differs from a "golden" version. (So, at-least the golden markdown will need to be updated if the iptables rules are deliberately changed - hopefully the corresponding description in the template will also be updated.) Signed-off-by: Rob Murray <rob.murray@docker.com>
This commit is contained in:
Rob Murray
159
integration/network/bridge/iptablesdoc/generated/new-daemon.md
Normal file
159
integration/network/bridge/iptablesdoc/generated/new-daemon.md
Normal file
@@ -0,0 +1,159 @@
|
||||
## iptables for a new Daemon
|
||||
|
||||
When the daemon starts, it creates custom chains, and rules for the
|
||||
default bridge network.
|
||||
|
||||
Table `filter`:
|
||||
|
||||
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
|
||||
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0
|
||||
2 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
|
||||
3 0 0 ACCEPT 0 -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
|
||||
4 0 0 DOCKER 0 -- * docker0 0.0.0.0/0 0.0.0.0/0
|
||||
5 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
|
||||
6 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
|
||||
|
||||
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
|
||||
Chain DOCKER (1 references)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
|
||||
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 DOCKER-ISOLATION-STAGE-2 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
|
||||
2 0 0 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0
|
||||
|
||||
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 DROP 0 -- * docker0 0.0.0.0/0 0.0.0.0/0
|
||||
2 0 0 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0
|
||||
|
||||
Chain DOCKER-USER (1 references)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0
|
||||
|
||||
|
||||
<details>
|
||||
<summary>iptables commands</summary>
|
||||
|
||||
-P INPUT ACCEPT
|
||||
-P FORWARD ACCEPT
|
||||
-P OUTPUT ACCEPT
|
||||
-N DOCKER
|
||||
-N DOCKER-ISOLATION-STAGE-1
|
||||
-N DOCKER-ISOLATION-STAGE-2
|
||||
-N DOCKER-USER
|
||||
-A FORWARD -j DOCKER-USER
|
||||
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
|
||||
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
|
||||
-A FORWARD -o docker0 -j DOCKER
|
||||
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
|
||||
-A FORWARD -i docker0 -o docker0 -j ACCEPT
|
||||
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
|
||||
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
|
||||
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
|
||||
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
|
||||
-A DOCKER-USER -j RETURN
|
||||
|
||||
|
||||
</details>
|
||||
|
||||
The FORWARD chain's policy shown above is ACCEPT. However:
|
||||
|
||||
- For IPv4, [setupIPForwarding][1] sets the POLICY to DROP if the sysctl
|
||||
net.ipv4.ip_forward was not set to '1', and the daemon set it itself.
|
||||
- For IPv6, the policy is always DROP.
|
||||
|
||||
[1]: https://github.com/moby/moby/blob/cff4f20c44a3a7c882ed73934dec6a77246c6323/libnetwork/drivers/bridge/setup_ip_forwarding.go#L44
|
||||
|
||||
The FORWARD chain rules are numbered in the output above, they are:
|
||||
|
||||
1. Unconditional jump to DOCKER-USER.
|
||||
This is set up by libnetwork, in [setupUserChain][10].
|
||||
Docker won't add rules to the DOCKER-USER chain, it's only for user-defined rules.
|
||||
It's (mostly) kept at the top of the by deleting it and re-creating after each
|
||||
new network is created, while traffic may be running for other networks.
|
||||
2. Unconditional jump to DOCKER-ISOLATION-STAGE-1.
|
||||
Set up during network creation by [setupIPTables][11], which ensures it appears
|
||||
after the jump to DOCKER-USER (by deleting it and re-creating, while traffic
|
||||
may be running for other networks).
|
||||
3. ACCEPT RELATED,ESTABLISHED packets into a specific bridge network.
|
||||
Allows responses to outgoing requests, and continuation of incoming requests,
|
||||
without needing to process any further rules.
|
||||
This rule is also added during network creation, but the code to do it
|
||||
is in libnetwork, [ProgramChain][12].
|
||||
4. Jump to DOCKER, for any packet destined for a bridge network. Added when
|
||||
the network is created, in [ProgramChain][13] ("filterChain" is the DOCKER chain).
|
||||
The DOCKER chain implements per-port/protocol filtering for each container.
|
||||
5. ACCEPT any packet leaving a network, also set up when the network is created, in
|
||||
[setupIPTablesInternal][14].
|
||||
6. ACCEPT packets flowing between containers within a network, because by default
|
||||
container isolation is disabled. Also set up when the network is created, in
|
||||
[setIcc][15].
|
||||
|
||||
[10]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/firewall_linux.go#L50
|
||||
[11]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L201
|
||||
[12]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/iptables/iptables.go#L270
|
||||
[13]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/iptables/iptables.go#L251-L255
|
||||
[14]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L264
|
||||
[15]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L343
|
||||
|
||||
_With ICC enabled 5 and 6 could be combined, to ACCEPT anything from the bridge.
|
||||
But, when ICC is disabled, rule 6 is DROP, so it would need to be placed before
|
||||
rule 5. Because the rules are generated in different places, that's a slightly
|
||||
bigger change than it should be._
|
||||
|
||||
The DOCKER chain is empty, because there are no containers with port mappings yet.
|
||||
|
||||
The DOCKER-ISOLATION chains implement inter-network isolation, all (unrelated)
|
||||
packets are processed by these chains. The rule are inserted at the head of the
|
||||
chain when a network is created, in [setINC][20].
|
||||
- DOCKER-ISOLATION-STAGE-1 jumps to DOCKER-ISOLATION-STAGE-2 for any packet
|
||||
routed to a docker network that has not come from that docker network.
|
||||
- DOCKER-ISOLATION-STAGE-2 processes all packets leaving a bridge network,
|
||||
packets that are destined for any other network are dropped.
|
||||
|
||||
[20]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L369
|
||||
|
||||
Table nat:
|
||||
|
||||
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
|
||||
|
||||
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
|
||||
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 DOCKER 0 -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
|
||||
|
||||
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 MASQUERADE 0 -- * !docker0 172.17.0.0/16 0.0.0.0/0
|
||||
|
||||
Chain DOCKER (2 references)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 RETURN 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0
|
||||
|
||||
|
||||
<details>
|
||||
<summary>iptables commands</summary>
|
||||
|
||||
-P PREROUTING ACCEPT
|
||||
-P INPUT ACCEPT
|
||||
-P OUTPUT ACCEPT
|
||||
-P POSTROUTING ACCEPT
|
||||
-N DOCKER
|
||||
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
|
||||
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
|
||||
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
|
||||
-A DOCKER -i docker0 -j RETURN
|
||||
|
||||
|
||||
</details>
|
||||
41
integration/network/bridge/iptablesdoc/index.md
Normal file
41
integration/network/bridge/iptablesdoc/index.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Docker Engine's use of iptables
|
||||
|
||||
> [!WARNING]
|
||||
> This is intended for development use - the structure of docker's iptables
|
||||
> (and ip6tables) rules will change between releases, it is not a stable
|
||||
> interface.
|
||||
|
||||
> [!NOTE]
|
||||
> This document is generated by `TestBridgeIptablesDoc` by running a
|
||||
> daemon, creating networks and containers, and capturing iptables.
|
||||
> The iptables are then merged with a text/template for each section.
|
||||
> The resulting document is diffed against one in the repo, so the
|
||||
> test will fail if there are differences in the generated rules (but
|
||||
> changes in the templates may go unnoticed).
|
||||
>
|
||||
> Links to code are permalinks - they will be out of date, and may not
|
||||
> point to the master branch. But, it's difficult to work out where
|
||||
> some of the rules come from, the links are intended as hints.
|
||||
|
||||
ip6tables rules follow the same pattern as iptables rules. So, only the
|
||||
IPv4 rules are shown here.
|
||||
|
||||
The bridge driver deletes its custom chains during its initialisation, in
|
||||
[configure][100]. Rules are then re-created as networks are restored. However,
|
||||
the filter-FORWARD chain is not cleared. The order in which networks are
|
||||
re-created is not the order in which they were originally created. So,
|
||||
rules may be arranged differently following a daemon restart.
|
||||
|
||||
When firewalld is running, if it's reloaded, iptables rules are cleared.
|
||||
The daemon registers handlers for its reload event (received via dbus)
|
||||
to reconstruct the rules.
|
||||
|
||||
The filter-INPUT chain is not used by Docker. Packets arriving from the host's
|
||||
physical network or the host itself hit the filter-FORWARD chain, as they are
|
||||
routed into the bridge network. Similarly, filter-OUTPUT is not used.
|
||||
|
||||
[100]: https://github.com/moby/moby/blob/fe09cab7fe04c3911417061f7c7ef60a8acc6bf3/libnetwork/drivers/bridge/bridge_linux.go#L508
|
||||
|
||||
Scenarios:
|
||||
|
||||
- [New daemon](generated/new-daemon.md)
|
||||
233
integration/network/bridge/iptablesdoc/iptablesdoc_linux_test.go
Normal file
233
integration/network/bridge/iptablesdoc/iptablesdoc_linux_test.go
Normal file
@@ -0,0 +1,233 @@
|
||||
// Package iptablesdoc runs docker, creates networks, runs containers and
|
||||
// captures iptables output for various configurations.
|
||||
//
|
||||
// The iptables output is then used with a markdown text/template from the
|
||||
// "templates" directory for each configuration (for each "section" in "index"),
|
||||
// to generate a markdown document for each section.
|
||||
//
|
||||
// The newly generated documents are placed in:
|
||||
//
|
||||
// bundles/test-integration/TestBridgeIptablesDoc/iptables.md
|
||||
//
|
||||
// If the generated doc differs from the "golden" reference in "generated/",
|
||||
// the test fails. When that happens:
|
||||
//
|
||||
// - check the iptables rules changes in the diff
|
||||
// - update the description in the corresponding "_templ.md" file
|
||||
// - re-run with TESTFLAGS='-update' to update the reference docs
|
||||
package iptablesdoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"text/template"
|
||||
|
||||
containertypes "github.com/docker/docker/api/types/container"
|
||||
networktypes "github.com/docker/docker/api/types/network"
|
||||
"github.com/docker/docker/integration/internal/container"
|
||||
"github.com/docker/docker/integration/internal/network"
|
||||
"github.com/docker/docker/internal/testutils/networking"
|
||||
"github.com/docker/docker/libnetwork/drivers/bridge"
|
||||
"github.com/docker/docker/testutil"
|
||||
"github.com/docker/docker/testutil/daemon"
|
||||
"github.com/docker/go-connections/nat"
|
||||
"gotest.tools/v3/assert"
|
||||
"gotest.tools/v3/golden"
|
||||
"gotest.tools/v3/skip"
|
||||
)
|
||||
|
||||
var (
|
||||
docNetworks = []string{"192.0.2.0/24", "198.51.100.0/24", "203.0.113.0/24"}
|
||||
docGateways = []string{"192.0.2.1", "198.51.100.1", "203.0.113.1"}
|
||||
)
|
||||
|
||||
type ctr struct {
|
||||
name string
|
||||
portMappings nat.PortMap
|
||||
}
|
||||
|
||||
type bridgeNetwork struct {
|
||||
bridge string
|
||||
gwMode string
|
||||
noICC bool
|
||||
internal bool
|
||||
containers []ctr
|
||||
}
|
||||
|
||||
type section struct {
|
||||
name string
|
||||
noUserlandProxy bool
|
||||
networks []bridgeNetwork
|
||||
}
|
||||
|
||||
var index = []section{
|
||||
{
|
||||
name: "new-daemon.md",
|
||||
},
|
||||
}
|
||||
|
||||
// iptCmdType is used to look up iptCmds in the markdown (can't use an int
|
||||
// type, or a new string type, so it's just an alias).
|
||||
type iptCmdType = string
|
||||
|
||||
const (
|
||||
iptCmdLFilter4 iptCmdType = "LFilter4"
|
||||
iptCmdSFilter4 iptCmdType = "SFilter4"
|
||||
iptCmdSFilterForward4 iptCmdType = "SFilterForward4"
|
||||
iptCmdSFilterDocker4 iptCmdType = "SFilterDocker4"
|
||||
iptCmdLNat4 iptCmdType = "LNat4"
|
||||
iptCmdSNat4 iptCmdType = "SNat4"
|
||||
)
|
||||
|
||||
var iptCmds = map[iptCmdType][]string{
|
||||
iptCmdLFilter4: {"iptables", "-nvL", "--line-numbers", "-t", "filter"},
|
||||
iptCmdSFilter4: {"iptables", "-S", "-t", "filter"},
|
||||
iptCmdSFilterForward4: {"iptables", "-S", "FORWARD"},
|
||||
iptCmdSFilterDocker4: {"iptables", "-S", "DOCKER"},
|
||||
iptCmdLNat4: {"iptables", "-nvL", "--line-numbers", "-t", "nat"},
|
||||
iptCmdSNat4: {"iptables", "-S", "-t", "nat"},
|
||||
}
|
||||
|
||||
func TestBridgeIptablesDoc(t *testing.T) {
|
||||
skip.If(t, testEnv.IsRootless)
|
||||
ctx := setupTest(t)
|
||||
|
||||
// Get the full path for "bundles/TestBridgeIptablesDoc".
|
||||
dest := os.Getenv("DOCKER_INTEGRATION_DAEMON_DEST")
|
||||
if dest == "" {
|
||||
dest = os.Getenv("DEST")
|
||||
}
|
||||
dest = filepath.Join(dest, t.Name())
|
||||
|
||||
// Set up an L3Segment, which will have a netns for each "section".
|
||||
addr4 := netip.MustParseAddr("192.168.124.1")
|
||||
addr6 := netip.MustParseAddr("fdc0:36dc:a4dd::1")
|
||||
l3 := networking.NewL3Segment(t, "gen-iptables-doc",
|
||||
netip.PrefixFrom(addr4, 24),
|
||||
netip.PrefixFrom(addr6, 64),
|
||||
)
|
||||
t.Cleanup(func() { l3.Destroy(t) })
|
||||
|
||||
for i, sec := range index {
|
||||
// Create a netns for this section.
|
||||
addr4 = addr4.Next()
|
||||
addr6 = addr6.Next()
|
||||
hostname := fmt.Sprintf("docker%d", i)
|
||||
l3.AddHost(t, hostname, hostname+"-host", "eth0",
|
||||
netip.PrefixFrom(addr4, 24),
|
||||
netip.PrefixFrom(addr6, 64),
|
||||
)
|
||||
host := l3.Hosts[hostname]
|
||||
// Stop the interface, to reduce the chances of stray packets getting counted by iptables.
|
||||
host.Run(t, "ip", "link", "set", "eth0", "down")
|
||||
|
||||
t.Run("gen_"+sec.name, func(t *testing.T) {
|
||||
// t.Parallel() - doesn't speed things up, startup times just extend
|
||||
runTestNet(t, testutil.StartSpan(ctx, t), dest, sec, host)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func runTestNet(t *testing.T, ctx context.Context, bundlesDir string, section section, host networking.Host) {
|
||||
var dArgs []string
|
||||
if section.noUserlandProxy {
|
||||
dArgs = append(dArgs, "--userland-proxy=false")
|
||||
}
|
||||
|
||||
// Start the daemon in its own network namespace.
|
||||
var d *daemon.Daemon
|
||||
host.Do(t, func() {
|
||||
// Run without OTEL because there's no routing from this netns for it - which
|
||||
// means the daemon doesn't shut down cleanly, causing the test to fail.
|
||||
d = daemon.New(t, daemon.WithEnvVars("OTEL_EXPORTER_OTLP_ENDPOINT="))
|
||||
d.StartWithBusybox(ctx, t, dArgs...)
|
||||
t.Cleanup(func() { d.Stop(t) })
|
||||
})
|
||||
|
||||
c := d.NewClientT(t)
|
||||
t.Cleanup(func() { c.Close() })
|
||||
|
||||
assert.Assert(t, len(section.networks) < len(docNetworks), "Don't have enough container network addresses")
|
||||
for i, nw := range section.networks {
|
||||
gwMode := nw.gwMode
|
||||
if gwMode == "" {
|
||||
gwMode = "nat"
|
||||
}
|
||||
netOpts := []func(*networktypes.CreateOptions){
|
||||
network.WithIPAM(docNetworks[i], docGateways[i]),
|
||||
network.WithOption(bridge.BridgeName, nw.bridge),
|
||||
network.WithOption(bridge.IPv4GatewayMode, gwMode),
|
||||
}
|
||||
if nw.noICC {
|
||||
netOpts = append(netOpts, network.WithOption(bridge.EnableICC, "false"))
|
||||
}
|
||||
if nw.internal {
|
||||
netOpts = append(netOpts, network.WithInternal())
|
||||
}
|
||||
network.CreateNoError(ctx, t, c, nw.bridge, netOpts...)
|
||||
t.Cleanup(func() { network.RemoveNoError(ctx, t, c, nw.bridge) })
|
||||
|
||||
for _, ctr := range nw.containers {
|
||||
var exposedPorts []string
|
||||
for ep := range ctr.portMappings {
|
||||
exposedPorts = append(exposedPorts, ep.Port()+"/"+ep.Proto())
|
||||
}
|
||||
id := container.Run(ctx, t, c,
|
||||
container.WithNetworkMode(nw.bridge),
|
||||
container.WithExposedPorts(exposedPorts...),
|
||||
container.WithPortMap(ctr.portMappings),
|
||||
)
|
||||
t.Cleanup(func() {
|
||||
c.ContainerRemove(ctx, id, containertypes.RemoveOptions{Force: true})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
iptablesOutput := runIptables(t, host)
|
||||
generated := generate(t, section.name, iptablesOutput)
|
||||
|
||||
// Write the output to the 'bundles' directory for easy reference.
|
||||
outFile := filepath.Join(bundlesDir, section.name)
|
||||
err := os.WriteFile(outFile, []byte(generated), 0o644)
|
||||
assert.NilError(t, err)
|
||||
t.Log("Wrote ", outFile)
|
||||
|
||||
// Compare against "golden" results.
|
||||
// Use full path so that the directory containing generated docs doesn't
|
||||
// have to be called 'testdata'.
|
||||
wd, err := os.Getwd()
|
||||
assert.NilError(t, err)
|
||||
golden.Assert(t, generated, filepath.Join(wd, "generated", section.name))
|
||||
}
|
||||
|
||||
var rePacketByteCounts = regexp.MustCompile(`\d+ packets, \d+ bytes`)
|
||||
|
||||
func runIptables(t *testing.T, host networking.Host) map[iptCmdType]string {
|
||||
host.Run(t, "iptables", "-Z")
|
||||
host.Run(t, "iptables", "-Z", "-t", "nat")
|
||||
res := map[iptCmdType]string{}
|
||||
for k, cmd := range iptCmds {
|
||||
d := host.Run(t, cmd[0], cmd[1:]...)
|
||||
// In CI, the OUTPUT chain sometimes sees a packet. Remove the counts.
|
||||
d = rePacketByteCounts.ReplaceAllString(d, "0 packets, 0 bytes")
|
||||
// Indent the result, so that it's treated as preformatted markdown.
|
||||
res[k] = strings.ReplaceAll(d, "\n", "\n ")
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
func generate(t *testing.T, name string, data map[iptCmdType]string) string {
|
||||
t.Helper()
|
||||
templ, err := template.New(name).ParseFiles(filepath.Join("templates", name))
|
||||
assert.NilError(t, err)
|
||||
wr := strings.Builder{}
|
||||
err = templ.ExecuteTemplate(&wr, name, data)
|
||||
assert.NilError(t, err)
|
||||
return wr.String()
|
||||
}
|
||||
56
integration/network/bridge/iptablesdoc/main_linux_test.go
Normal file
56
integration/network/bridge/iptablesdoc/main_linux_test.go
Normal file
@@ -0,0 +1,56 @@
|
||||
package iptablesdoc // import "github.com/docker/docker/integration/network/bridge/iptablesdoc"
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/docker/docker/testutil"
|
||||
"github.com/docker/docker/testutil/environment"
|
||||
"go.opentelemetry.io/otel"
|
||||
"go.opentelemetry.io/otel/codes"
|
||||
)
|
||||
|
||||
var (
|
||||
testEnv *environment.Execution
|
||||
baseContext context.Context
|
||||
)
|
||||
|
||||
func TestMain(m *testing.M) {
|
||||
shutdown := testutil.ConfigureTracing()
|
||||
ctx, span := otel.Tracer("").Start(context.Background(), "integration/network/bridge/iptablesdoc.TestMain")
|
||||
baseContext = ctx
|
||||
|
||||
var err error
|
||||
testEnv, err = environment.New(ctx)
|
||||
if err != nil {
|
||||
span.SetStatus(codes.Error, err.Error())
|
||||
span.End()
|
||||
shutdown(ctx)
|
||||
panic(err)
|
||||
}
|
||||
|
||||
err = environment.EnsureFrozenImagesLinux(ctx, testEnv)
|
||||
if err != nil {
|
||||
span.SetStatus(codes.Error, err.Error())
|
||||
span.End()
|
||||
shutdown(ctx)
|
||||
panic(err)
|
||||
}
|
||||
|
||||
testEnv.Print()
|
||||
code := m.Run()
|
||||
if code != 0 {
|
||||
span.SetStatus(codes.Error, "m.Run() returned non-zero exit code")
|
||||
}
|
||||
span.End()
|
||||
shutdown(ctx)
|
||||
os.Exit(code)
|
||||
}
|
||||
|
||||
func setupTest(t *testing.T) context.Context {
|
||||
ctx := testutil.StartSpan(baseContext, t)
|
||||
environment.ProtectAll(ctx, t, testEnv)
|
||||
t.Cleanup(func() { testEnv.Clean(ctx, t) })
|
||||
return ctx
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
## iptables for a new Daemon
|
||||
|
||||
When the daemon starts, it creates custom chains, and rules for the
|
||||
default bridge network.
|
||||
|
||||
Table `filter`:
|
||||
|
||||
{{index . "LFilter4"}}
|
||||
|
||||
<details>
|
||||
<summary>iptables commands</summary>
|
||||
|
||||
{{index . "SFilter4"}}
|
||||
|
||||
</details>
|
||||
|
||||
The FORWARD chain's policy shown above is ACCEPT. However:
|
||||
|
||||
- For IPv4, [setupIPForwarding][1] sets the POLICY to DROP if the sysctl
|
||||
net.ipv4.ip_forward was not set to '1', and the daemon set it itself.
|
||||
- For IPv6, the policy is always DROP.
|
||||
|
||||
[1]: https://github.com/moby/moby/blob/cff4f20c44a3a7c882ed73934dec6a77246c6323/libnetwork/drivers/bridge/setup_ip_forwarding.go#L44
|
||||
|
||||
The FORWARD chain rules are numbered in the output above, they are:
|
||||
|
||||
1. Unconditional jump to DOCKER-USER.
|
||||
This is set up by libnetwork, in [setupUserChain][10].
|
||||
Docker won't add rules to the DOCKER-USER chain, it's only for user-defined rules.
|
||||
It's (mostly) kept at the top of the by deleting it and re-creating after each
|
||||
new network is created, while traffic may be running for other networks.
|
||||
2. Unconditional jump to DOCKER-ISOLATION-STAGE-1.
|
||||
Set up during network creation by [setupIPTables][11], which ensures it appears
|
||||
after the jump to DOCKER-USER (by deleting it and re-creating, while traffic
|
||||
may be running for other networks).
|
||||
3. ACCEPT RELATED,ESTABLISHED packets into a specific bridge network.
|
||||
Allows responses to outgoing requests, and continuation of incoming requests,
|
||||
without needing to process any further rules.
|
||||
This rule is also added during network creation, but the code to do it
|
||||
is in libnetwork, [ProgramChain][12].
|
||||
4. Jump to DOCKER, for any packet destined for a bridge network. Added when
|
||||
the network is created, in [ProgramChain][13] ("filterChain" is the DOCKER chain).
|
||||
The DOCKER chain implements per-port/protocol filtering for each container.
|
||||
5. ACCEPT any packet leaving a network, also set up when the network is created, in
|
||||
[setupIPTablesInternal][14].
|
||||
6. ACCEPT packets flowing between containers within a network, because by default
|
||||
container isolation is disabled. Also set up when the network is created, in
|
||||
[setIcc][15].
|
||||
|
||||
[10]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/firewall_linux.go#L50
|
||||
[11]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L201
|
||||
[12]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/iptables/iptables.go#L270
|
||||
[13]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/iptables/iptables.go#L251-L255
|
||||
[14]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L264
|
||||
[15]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L343
|
||||
|
||||
_With ICC enabled 5 and 6 could be combined, to ACCEPT anything from the bridge.
|
||||
But, when ICC is disabled, rule 6 is DROP, so it would need to be placed before
|
||||
rule 5. Because the rules are generated in different places, that's a slightly
|
||||
bigger change than it should be._
|
||||
|
||||
The DOCKER chain is empty, because there are no containers with port mappings yet.
|
||||
|
||||
The DOCKER-ISOLATION chains implement inter-network isolation, all (unrelated)
|
||||
packets are processed by these chains. The rule are inserted at the head of the
|
||||
chain when a network is created, in [setINC][20].
|
||||
- DOCKER-ISOLATION-STAGE-1 jumps to DOCKER-ISOLATION-STAGE-2 for any packet
|
||||
routed to a docker network that has not come from that docker network.
|
||||
- DOCKER-ISOLATION-STAGE-2 processes all packets leaving a bridge network,
|
||||
packets that are destined for any other network are dropped.
|
||||
|
||||
[20]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L369
|
||||
|
||||
Table nat:
|
||||
|
||||
{{index . "LNat4"}}
|
||||
|
||||
<details>
|
||||
<summary>iptables commands</summary>
|
||||
|
||||
{{index . "SNat4"}}
|
||||
|
||||
</details>
|
||||
@@ -16,7 +16,8 @@ import (
|
||||
// host lives in the current network namespace (eg. where dockerd runs).
|
||||
const CurrentNetns = ""
|
||||
|
||||
func runCommand(t *testing.T, cmd string, args ...string) {
|
||||
func runCommand(t *testing.T, cmd string, args ...string) string {
|
||||
t.Helper()
|
||||
t.Log(strings.Join(append([]string{cmd}, args...), " "))
|
||||
|
||||
var b bytes.Buffer
|
||||
@@ -28,6 +29,7 @@ func runCommand(t *testing.T, cmd string, args ...string) {
|
||||
t.Log(b.String())
|
||||
t.Fatalf("Error: %v", err)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// L3Segment simulates a switched, dual-stack capable network that
|
||||
@@ -113,15 +115,16 @@ func newHost(t *testing.T, nsName, ifname string) Host {
|
||||
}
|
||||
}
|
||||
|
||||
// Run executes the provided command in the host's network namespace.
|
||||
func (h Host) Run(t *testing.T, cmd string, args ...string) {
|
||||
// Run executes the provided command in the host's network namespace
|
||||
// and returns its combined stdout/stderr.
|
||||
func (h Host) Run(t *testing.T, cmd string, args ...string) string {
|
||||
t.Helper()
|
||||
|
||||
if h.ns != CurrentNetns {
|
||||
args = append([]string{"netns", "exec", h.ns, cmd}, args...)
|
||||
cmd = "ip"
|
||||
}
|
||||
runCommand(t, cmd, args...)
|
||||
return runCommand(t, cmd, args...)
|
||||
}
|
||||
|
||||
// Do run the provided function in the host's network namespace.
|
||||
|
||||
Reference in New Issue
Block a user