libnetwork: DNAT gwbridge packets to ingress ports
Swarm ingress ports are published as ordinary port mappings on the ingress load balancer's docker_gwbridge endpoint, so they follow the userland-proxy setting like any other port mapping. With the proxy enabled, the DNAT rule for a port mapping is not applied to packets arriving on the bridge the mapping targets; those packets pass through the INPUT chain to docker-proxy instead. Packets from a task to a published port at its own node's address arrive on docker_gwbridge and take that path, so a host firewall that drops unsolicited input, such as ufw with its default "deny incoming" policy, drops them, even though the same port remains reachable from other nodes and from the host itself. Before ingress ports were published as port mappings, their DNAT rule in the DOCKER-INGRESS chain applied to packets from docker_gwbridge too, and no docker-proxy was started for them. A task could reach a published port on its own node whatever the userland-proxy setting, without ufw's policy getting in the way. A connection from a container to a published port on one of the host's addresses, where the port maps to a container on the same bridge, is a hairpin connection: its packets enter and leave the host through that bridge. docker-proxy makes hairpin connections work by accepting the client's connection in the host and opening a new one to the container. The kernel can handle them without the proxy by DNATing the packets back across the bridge, but that needs br_netfilter: the server's replies are bridged straight back to the client, and conntrack sees them and restores their source address only if br_netfilter passes bridged packets through netfilter. A container that connects to its own published port also needs hairpin mode on its bridge port and a MASQUERADE rule. The bridge driver also uses br_netfilter to block inter-container communication (ICC). Loading br_netfilter has a cost: it passes traffic on every bridge on the host through the host's firewall rules, including bridges that Docker does not manage. And some hosts cannot load br_netfilter at all. The bridge driver therefore loads br_netfilter only when necessary. Add a NoSelfConnect hint to PortBinding and PublishedPort. It tells the bridge driver that nothing behind the endpoint connects to the published port. Packets from the endpoint's own bridge can then be DNATed to the port without hairpin mode or the MASQUERADE rule, needing only br_netfilter. The bridge driver uses the hint to bypass docker-proxy where it can: for a binding with NoSelfConnect set, on a network with ICC disabled, for which the bridge driver has already loaded br_netfilter, the iptables and nftables firewallers DNAT packets from the network's own bridge as well. The existing filter rule that accepts DNATed packets arriving on an ICC-disabled bridge already lets them through. libnetwork sets the hint on ingress ports. They are published on the load balancer's endpoint, whose sandbox libnetwork creates without a container, so nothing behind the endpoint can connect to them. docker_gwbridge also has ICC disabled. Other endpoints on docker_gwbridge can have a container behind them, and only an endpoint's owner knows whether one does, so the hint comes from the owner and is carried on each binding. For ingress ports, the result is a hybrid of the two userland-proxy modes. docker-proxy still listens on the ports, but netfilter now DNATs IPv4 packets arriving on docker_gwbridge before the proxy sees them, as it already did for packets from other nodes. The proxy is left handling IPv6 clients, which it forwards to the load balancer over IPv4, and connections from the host to its loopback addresses. Hairpin connections to ordinary containers' published ports still go through docker-proxy. With the userland proxy disabled, the bridge driver handles them in the kernel: it loads br_netfilter for every network, and since it cannot know whether a container connects to its own published port, it enables hairpin mode on every container's bridge port and adds the MASQUERADE rule. Hairpin mode also sends broadcast and multicast frames back to the container that sent them, and nothing filters them out. Doing the same with the proxy enabled would bring that behaviour, and the cost of br_netfilter, to every container. Closing that gap is part of making operation without the userland proxy match operation with it, apart from servers seeing clients' real addresses, and is a larger change than restoring the ingress ports' earlier behaviour. Signed-off-by: Cory Snider <csnider@mirantis.com> Assisted-By: Claude Opus 5.5
This commit is contained in:
1 parent
ed01f3a432
commit
d95c4fbe4e
13 files changed
+230
-35
No files matched your search
@@ -71,6 +71,22 @@ type NetworkConfigFam struct {
|
||||
Unprotected bool
|
||||
}
|
||||
|
||||
// DNATFromOwnBridge returns true if packets that arrive from the network's own
|
||||
// bridge should be DNATed to pb's container address. If it returns false,
|
||||
// they're left for the userland proxy.
|
||||
//
|
||||
// Without the proxy (config.Hairpin), they're always DNATed. With the proxy,
|
||||
// DNATing them needs two things the proxy avoids. First, br_netfilter: the
|
||||
// container's replies are bridged straight back to the client, so conntrack
|
||||
// only sees them, and restores their source address, if bridged packets go
|
||||
// through netfilter. The network has that when ICC is disabled. Second, for a
|
||||
// container that connects to its own published port, hairpin mode on its
|
||||
// bridge port and a MASQUERADE rule. A binding marked NoSelfConnect doesn't
|
||||
// need those.
|
||||
func (nc NetworkConfig) DNATFromOwnBridge(config Config, pb types.PortBinding) bool {
|
||||
return config.Hairpin || (!nc.ICC && pb.NoSelfConnect)
|
||||
}
|
||||
|
||||
// Firewaller implements firewall rules for bridge networks.
|
||||
type Firewaller interface {
|
||||
// NewNetwork returns an object that can be used to add published ports and legacy
|
||||
|
||||
@@ -15,6 +15,7 @@ import (
|
||||
"github.com/moby/moby/v2/daemon/libnetwork/types"
|
||||
"github.com/moby/moby/v2/internal/testutil/netnsutils"
|
||||
"gotest.tools/v3/assert"
|
||||
is "gotest.tools/v3/assert/cmp"
|
||||
"gotest.tools/v3/golden"
|
||||
"gotest.tools/v3/icmd"
|
||||
"gotest.tools/v3/skip"
|
||||
@@ -265,3 +266,63 @@ func testIptabler(t *testing.T, tn string, config firewaller.Config, netConfig f
|
||||
checkResults("iptables", rnWSL2Mirrored(fmt.Sprintf("%s/cleaned,hairpin=%v", tn, config.Hairpin)), config.IPv4)
|
||||
checkResults("ip6tables", fmt.Sprintf("%s/cleaned,hairpin=%v", tn, config.Hairpin), config.IPv6)
|
||||
}
|
||||
|
||||
// TestDNATFromOwnBridge checks when a port binding's DNAT rule leaves packets
|
||||
// arriving from the network's own bridge to the userland proxy, and that the
|
||||
// rule is deleted with the binding.
|
||||
func TestDNATFromOwnBridge(t *testing.T) {
|
||||
skip.If(t, iptables.UsingFirewalld(), "firewalld is running in the host netns, it can't modify rules in the test's netns")
|
||||
|
||||
testcases := []struct {
|
||||
hairpin, icc, noSelfConnect bool
|
||||
expProxySkip bool
|
||||
}{
|
||||
{hairpin: true, icc: true},
|
||||
{hairpin: true, icc: false, noSelfConnect: true},
|
||||
{icc: true, expProxySkip: true},
|
||||
{icc: true, noSelfConnect: true, expProxySkip: true},
|
||||
{icc: false, expProxySkip: true},
|
||||
{icc: false, noSelfConnect: true},
|
||||
}
|
||||
for _, tc := range testcases {
|
||||
t.Run(fmt.Sprintf("hairpin=%v/icc=%v/noSelfConnect=%v", tc.hairpin, tc.icc, tc.noSelfConnect), func(t *testing.T) {
|
||||
defer netnsutils.SetupTestOSContext(t, netnsutils.WithSetNsHandles(false))()
|
||||
|
||||
fw, err := NewIptabler(t.Context(), firewaller.Config{IPv4: true, Hairpin: tc.hairpin})
|
||||
assert.NilError(t, err)
|
||||
nw, err := fw.NewNetwork(t.Context(), firewaller.NetworkConfig{
|
||||
IfName: "br-dummy",
|
||||
ICC: tc.icc,
|
||||
Masquerade: true,
|
||||
Config4: firewaller.NetworkConfigFam{Prefix: netip.MustParsePrefix("192.168.0.0/24")},
|
||||
})
|
||||
assert.NilError(t, err)
|
||||
pb := types.PortBinding{
|
||||
Proto: types.TCP,
|
||||
IP: net.ParseIP("192.168.0.2"),
|
||||
Port: 80,
|
||||
HostIP: net.IPv4zero,
|
||||
HostPort: 8080,
|
||||
HostPortEnd: 8080,
|
||||
NoSelfConnect: tc.noSelfConnect,
|
||||
}
|
||||
|
||||
dnatRules := func() []string {
|
||||
t.Helper()
|
||||
out, err := iptables.GetIptable(iptables.IPv4).Raw("-t", "nat", "-S", dockerChain)
|
||||
assert.NilError(t, err)
|
||||
return slices.DeleteFunc(strings.Split(string(out), "\n"), func(r string) bool {
|
||||
return !strings.Contains(r, "-j DNAT")
|
||||
})
|
||||
}
|
||||
|
||||
assert.NilError(t, nw.AddPorts(t.Context(), []types.PortBinding{pb}))
|
||||
rules := dnatRules()
|
||||
assert.Assert(t, is.Len(rules, 1))
|
||||
assert.Check(t, strings.Contains(rules[0], "! -i br-dummy") == tc.expProxySkip, "DNAT rule: %s", rules[0])
|
||||
|
||||
assert.NilError(t, nw.DelPorts(t.Context(), []types.PortBinding{pb}))
|
||||
assert.Check(t, is.Len(dnatRules(), 0))
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -102,7 +102,7 @@ func (n *network) setPerPortNAT(ipv iptables.IPVersion, b types.PortBinding, ena
|
||||
"-j", "DNAT",
|
||||
"--to-destination", net.JoinHostPort(b.IP.String(), strconv.Itoa(int(b.Port))),
|
||||
}
|
||||
if !n.ipt.config.Hairpin {
|
||||
if !n.config.DNATFromOwnBridge(n.ipt.config, b) {
|
||||
args = append(args, "!", "-i", n.config.IfName)
|
||||
}
|
||||
if ipv == iptables.IPv6 {
|
||||
|
||||
@@ -242,3 +242,69 @@ func TestHairpinMultipleHostPorts(t *testing.T) {
|
||||
assert.Check(t, is.Equal(countInChain(postRtChain, masqComment), 0))
|
||||
assert.Check(t, is.Equal(countInChain(natChain, dnatComment), 0))
|
||||
}
|
||||
|
||||
// TestDNATFromOwnBridge checks when a port binding's DNAT rule leaves packets
|
||||
// arriving from the network's own bridge to the userland proxy, and that the
|
||||
// rule is deleted with the binding.
|
||||
func TestDNATFromOwnBridge(t *testing.T) {
|
||||
nftables.Enable()
|
||||
defer nftables.Disable()
|
||||
|
||||
testcases := []struct {
|
||||
hairpin, icc, noSelfConnect bool
|
||||
expProxySkip bool
|
||||
}{
|
||||
{hairpin: true, icc: true},
|
||||
{hairpin: true, icc: false, noSelfConnect: true},
|
||||
{icc: true, expProxySkip: true},
|
||||
{icc: true, noSelfConnect: true, expProxySkip: true},
|
||||
{icc: false, expProxySkip: true},
|
||||
{icc: false, noSelfConnect: true},
|
||||
}
|
||||
for _, tc := range testcases {
|
||||
t.Run(fmt.Sprintf("hairpin=%v/icc=%v/noSelfConnect=%v", tc.hairpin, tc.icc, tc.noSelfConnect), func(t *testing.T) {
|
||||
defer netnsutils.SetupTestOSContext(t, netnsutils.WithSetNsHandles(false))()
|
||||
|
||||
fw, err := NewNftabler(t.Context(), firewaller.Config{IPv4: true, Hairpin: tc.hairpin})
|
||||
assert.NilError(t, err)
|
||||
defer fw.Close()
|
||||
nw, err := fw.NewNetwork(t.Context(), firewaller.NetworkConfig{
|
||||
IfName: "br-dummy",
|
||||
ICC: tc.icc,
|
||||
Masquerade: true,
|
||||
Config4: firewaller.NetworkConfigFam{Prefix: netip.MustParsePrefix("192.168.0.0/24")},
|
||||
})
|
||||
assert.NilError(t, err)
|
||||
pb := types.PortBinding{
|
||||
Proto: types.TCP,
|
||||
IP: net.ParseIP("192.168.0.2"),
|
||||
Port: 80,
|
||||
HostIP: net.IPv4zero,
|
||||
HostPort: 8080,
|
||||
HostPortEnd: 8080,
|
||||
NoSelfConnect: tc.noSelfConnect,
|
||||
}
|
||||
|
||||
dnatRules := func() []string {
|
||||
t.Helper()
|
||||
res := icmd.RunCommand("nft", "list", "table", "ip", dockerTable)
|
||||
assert.Assert(t, res.Error, res.Combined())
|
||||
var rules []string
|
||||
for _, r := range strings.Split(res.Stdout(), "\n") {
|
||||
if strings.Contains(r, "dnat to") {
|
||||
rules = append(rules, r)
|
||||
}
|
||||
}
|
||||
return rules
|
||||
}
|
||||
|
||||
assert.NilError(t, nw.AddPorts(t.Context(), []types.PortBinding{pb}))
|
||||
rules := dnatRules()
|
||||
assert.Assert(t, is.Len(rules, 1))
|
||||
assert.Check(t, strings.Contains(rules[0], `iifname != "br-dummy"`) == tc.expProxySkip, "DNAT rule: %s", rules[0])
|
||||
|
||||
assert.NilError(t, nw.DelPorts(t.Context(), []types.PortBinding{pb}))
|
||||
assert.Check(t, is.Len(dnatRules(), 0))
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -144,10 +144,6 @@ func (n *network) setPerPortHairpinForwarding(pbs []types.PortBinding, updater f
|
||||
}
|
||||
|
||||
func (n *network) setPerPortDNAT(pbs []types.PortBinding, updater func(nftables.Obj), ipv nftables.Family) {
|
||||
var proxySkip string
|
||||
if !n.fw.config.Hairpin {
|
||||
proxySkip = fmt.Sprintf("iifname != %s ", n.config.IfName)
|
||||
}
|
||||
var v6LLSkip string
|
||||
if ipv == nftables.IPv6 {
|
||||
v6LLSkip = "ip6 saddr != fe80::/10 "
|
||||
@@ -162,6 +158,10 @@ func (n *network) setPerPortDNAT(pbs []types.PortBinding, updater func(nftables.
|
||||
if (pb.IP.To4() != nil) != (pb.HostIP.To4() != nil) {
|
||||
continue
|
||||
}
|
||||
var proxySkip string
|
||||
if !n.config.DNATFromOwnBridge(n.fw.config, pb) {
|
||||
proxySkip = fmt.Sprintf("iifname != %s ", n.config.IfName)
|
||||
}
|
||||
var daddrMatch string
|
||||
if !pb.HostIP.IsUnspecified() {
|
||||
daddrMatch = fmt.Sprintf("%s daddr %s ", ipv, pb.HostIP)
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/moby/moby/v2/daemon/libnetwork/drivers/bridge/internal/firewaller"
|
||||
"github.com/moby/moby/v2/daemon/libnetwork/drvregistry"
|
||||
"github.com/moby/moby/v2/daemon/libnetwork/portallocator"
|
||||
"github.com/moby/moby/v2/daemon/libnetwork/portmappers/nat"
|
||||
@@ -112,6 +113,23 @@ func TestAddEphemeralPortsTracksSeparately(t *testing.T) {
|
||||
assert.Check(t, is.Equal(env.ep.ephemeralPortMapping[0].Proto, types.TCP))
|
||||
}
|
||||
|
||||
// TestAddEphemeralPortsNoSelfConnect checks that a published port's
|
||||
// NoSelfConnect setting reaches the firewaller, which needs it to decide
|
||||
// whether packets from the bridge are DNATed or left for the userland proxy.
|
||||
func TestAddEphemeralPortsNoSelfConnect(t *testing.T) {
|
||||
defer netnsutils.SetupTestOSContext(t)()
|
||||
env := newEphTestEnv(t, portBindingMode{ipv4: true})
|
||||
|
||||
pp := pubPort(types.TCP, 8080)
|
||||
pp.NoSelfConnect = true
|
||||
err := env.d.AddEphemeralPorts(t.Context(), ephTestNID, ephTestEID, []types.PublishedPort{pp})
|
||||
assert.NilError(t, err)
|
||||
|
||||
fwPorts := env.n.firewallerNetwork.(*firewaller.StubFirewallerNetwork).Ports
|
||||
assert.Assert(t, is.Len(fwPorts, 1))
|
||||
assert.Check(t, fwPorts[0].NoSelfConnect)
|
||||
}
|
||||
|
||||
// TestAddEphemeralPortsNoActiveMode documents a sharp edge rather than asserting
|
||||
// it is desirable: with no NAT family active on the endpoint, addPortMappings
|
||||
// generates no bindings and returns no error, so AddEphemeralPorts reports success
|
||||
|
||||
@@ -471,12 +471,13 @@ func collectFirewallPorts(pbs []portmapperapi.PortBinding) []types.PortBinding {
|
||||
// that can be passed to firewaller.Network for setting up a NAT rule.
|
||||
func toNATBinding(pb portmapperapi.PortBinding) types.PortBinding {
|
||||
return types.PortBinding{
|
||||
IP: pb.IP,
|
||||
Port: pb.Port,
|
||||
Proto: pb.Proto,
|
||||
HostIP: pb.NAT.Addr().AsSlice(),
|
||||
HostPort: pb.NAT.Port(),
|
||||
HostPortEnd: pb.NAT.Port(),
|
||||
IP: pb.IP,
|
||||
Port: pb.Port,
|
||||
Proto: pb.Proto,
|
||||
HostIP: pb.NAT.Addr().AsSlice(),
|
||||
HostPort: pb.NAT.Port(),
|
||||
HostPortEnd: pb.NAT.Port(),
|
||||
NoSelfConnect: pb.NoSelfConnect,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -284,6 +284,10 @@ func publishedPort(pc *PortConfig) (types.PublishedPort, error) {
|
||||
Proto: types.ParseProtocol(strings.ToLower(pc.Protocol.String())),
|
||||
Port: uint16(pc.PublishedPort),
|
||||
HostPort: uint16(pc.PublishedPort),
|
||||
// The ports are published on the load-balancer sandbox's gateway
|
||||
// endpoint. The sandbox has no container, so nothing behind the
|
||||
// endpoint connects to them.
|
||||
NoSelfConnect: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ func TestPublishedPortIgnoresTargetPort(t *testing.T) {
|
||||
assert.NilError(t, err)
|
||||
|
||||
assert.Check(t, is.Equal(a, b))
|
||||
assert.Check(t, is.Equal(a, types.PublishedPort{Proto: types.TCP, Port: 8080, HostPort: 8080}))
|
||||
assert.Check(t, is.Equal(a, types.PublishedPort{Proto: types.TCP, Port: 8080, HostPort: 8080, NoSelfConnect: true}))
|
||||
|
||||
// A differing protocol is a different reservation.
|
||||
udp, err := publishedPort(&PortConfig{Protocol: ProtocolUDP, PublishedPort: 8080, TargetPort: 80})
|
||||
@@ -84,7 +84,7 @@ func TestIngressPortRefsAcrossGenerations(t *testing.T) {
|
||||
ppsB, err := publishedPorts([]*PortConfig{{Protocol: ProtocolTCP, PublishedPort: 8080, TargetPort: 8080}})
|
||||
assert.NilError(t, err)
|
||||
|
||||
want := types.PublishedPort{Proto: types.TCP, Port: 8080, HostPort: 8080}
|
||||
want := types.PublishedPort{Proto: types.TCP, Port: 8080, HostPort: 8080, NoSelfConnect: true}
|
||||
|
||||
// The first generation publishes the reservation.
|
||||
assert.Check(t, is.DeepEqual(refIngressPorts(ppsA), []types.PublishedPort{want}))
|
||||
|
||||
@@ -56,6 +56,11 @@ type PortBinding struct {
|
||||
HostIP net.IP
|
||||
HostPort uint16
|
||||
HostPortEnd uint16
|
||||
// NoSelfConnect is a hint that nothing behind the endpoint connects to
|
||||
// its own published port through the host. Packets from other endpoints
|
||||
// on the same network can then be DNATed to the port without hairpin
|
||||
// mode or masquerading, instead of being left for the userland proxy.
|
||||
NoSelfConnect bool `json:",omitempty"`
|
||||
}
|
||||
|
||||
// PublishedPort is a request to publish a single container port on the host.
|
||||
@@ -67,16 +72,20 @@ type PublishedPort struct {
|
||||
Proto Protocol
|
||||
Port uint16 // container port
|
||||
HostPort uint16 // single host port - there is no range
|
||||
// NoSelfConnect is a hint that nothing behind the endpoint connects to
|
||||
// its own published port through the host. See [PortBinding.NoSelfConnect].
|
||||
NoSelfConnect bool
|
||||
}
|
||||
|
||||
// PortBinding returns the single-port [PortBinding] equivalent of p, with
|
||||
// HostPortEnd pinned to HostPort so it can never denote a range.
|
||||
func (p PublishedPort) PortBinding() PortBinding {
|
||||
return PortBinding{
|
||||
Proto: p.Proto,
|
||||
Port: p.Port,
|
||||
HostPort: p.HostPort,
|
||||
HostPortEnd: p.HostPort,
|
||||
Proto: p.Proto,
|
||||
Port: p.Port,
|
||||
HostPort: p.HostPort,
|
||||
HostPortEnd: p.HostPort,
|
||||
NoSelfConnect: p.NoSelfConnect,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -97,12 +106,13 @@ func (p PortBinding) HostAddr() (net.Addr, error) {
|
||||
// Copy returns a deep copy of the PortBinding.
|
||||
func (p PortBinding) Copy() PortBinding {
|
||||
return PortBinding{
|
||||
Proto: p.Proto,
|
||||
IP: slices.Clone(p.IP),
|
||||
Port: p.Port,
|
||||
HostIP: slices.Clone(p.HostIP),
|
||||
HostPort: p.HostPort,
|
||||
HostPortEnd: p.HostPortEnd,
|
||||
Proto: p.Proto,
|
||||
IP: slices.Clone(p.IP),
|
||||
Port: p.Port,
|
||||
HostIP: slices.Clone(p.HostIP),
|
||||
HostPort: p.HostPort,
|
||||
HostPortEnd: p.HostPortEnd,
|
||||
NoSelfConnect: p.NoSelfConnect,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -113,7 +123,8 @@ func (p PortBinding) Equal(o PortBinding) bool {
|
||||
p.Port == o.Port &&
|
||||
p.HostIP.Equal(o.HostIP) &&
|
||||
p.HostPort == o.HostPort &&
|
||||
p.HostPortEnd == o.HostPortEnd
|
||||
p.HostPortEnd == o.HostPortEnd &&
|
||||
p.NoSelfConnect == o.NoSelfConnect
|
||||
}
|
||||
|
||||
// String returns the PortBinding structure in the form "HostIP:HostPort:IP:Port/Proto",
|
||||
|
||||
@@ -91,7 +91,11 @@ Note that:
|
||||
- The published port is set up as an ordinary port mapping on the ingress
|
||||
load-balancer sandbox's `docker_gwbridge` gateway endpoint (`172.18.0.2`),
|
||||
using the same rules as any other published container port:
|
||||
- a DNAT rule in the nat `DOCKER` chain, and
|
||||
- a DNAT rule in the nat `DOCKER` chain. Unlike a container's port mapping
|
||||
when the userland proxy is running, it doesn't skip packets arriving on
|
||||
`docker_gwbridge`. The load-balancer never connects to its own published
|
||||
port, so a task that reaches the port via its node's address is DNATed
|
||||
like any other client, and doesn't go through docker-proxy, and
|
||||
- ACCEPT rules in the filter `DOCKER` chain. There are two, because
|
||||
`docker_gwbridge` has inter-container communication disabled: one for
|
||||
packets from outside the network, and one for packets from the network
|
||||
@@ -118,7 +122,7 @@ And the corresponding nat table:
|
||||
|
||||
Chain DOCKER (2 references)
|
||||
num pkts bytes target prot opt in out source destination
|
||||
1 0 0 DNAT tcp -- !docker_gwbridge any anywhere anywhere tcp dpt:http-alt to:172.18.0.2:8080
|
||||
1 0 0 DNAT tcp -- any any anywhere anywhere tcp dpt:http-alt to:172.18.0.2:8080
|
||||
|
||||
|
||||
<details>
|
||||
@@ -133,7 +137,7 @@ And the corresponding nat table:
|
||||
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
|
||||
-A POSTROUTING -s 172.18.0.0/16 ! -o docker_gwbridge -j MASQUERADE
|
||||
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
|
||||
-A DOCKER ! -i docker_gwbridge -p tcp -m tcp --dport 8080 -j DNAT --to-destination 172.18.0.2:8080
|
||||
-A DOCKER -p tcp -m tcp --dport 8080 -j DNAT --to-destination 172.18.0.2:8080
|
||||
|
||||
|
||||
</details>
|
||||
@@ -22,7 +22,11 @@ Note that:
|
||||
- The published port is set up as an ordinary port mapping on the ingress
|
||||
load-balancer sandbox's `docker_gwbridge` gateway endpoint (`172.18.0.2`),
|
||||
using the same rules as any other published container port:
|
||||
- a DNAT rule in the nat `DOCKER` chain, and
|
||||
- a DNAT rule in the nat `DOCKER` chain. Unlike a container's port mapping
|
||||
when the userland proxy is running, it doesn't skip packets arriving on
|
||||
`docker_gwbridge`. The load-balancer never connects to its own published
|
||||
port, so a task that reaches the port via its node's address is DNATed
|
||||
like any other client, and doesn't go through docker-proxy, and
|
||||
- ACCEPT rules in the filter `DOCKER` chain. There are two, because
|
||||
`docker_gwbridge` has inter-container communication disabled: one for
|
||||
packets from outside the network, and one for packets from the network
|
||||
|
||||
@@ -995,17 +995,21 @@ func probeUDPPort(t *testing.T, host networking.Host, hostAddr, port string) err
|
||||
|
||||
// TestIngressPortFromTaskOnSameNode checks that a container on the node can
|
||||
// reach a service's published port via one of the node's own addresses, with
|
||||
// and without the userland proxy.
|
||||
// and without the userland proxy, while the node drops unsolicited input.
|
||||
//
|
||||
// The ingress load balancer's published ports are mapped onto its
|
||||
// docker_gwbridge endpoint, and docker_gwbridge is created with inter-container
|
||||
// communication disabled. With docker-proxy, the connection is served from the
|
||||
// host's netns and never crosses that bridge. Without it, the packets are
|
||||
// DNATed across the bridge, so they have to get past the rules that implement
|
||||
// icc=false. Reaching a published port shouldn't depend on the userland proxy,
|
||||
// so both modes are checked and expected to behave the same.
|
||||
// communication disabled. The container's packets are DNATed across that
|
||||
// bridge, so they have to get past the rules that implement icc=false. With
|
||||
// the userland proxy enabled, packets from a container's own bridge are
|
||||
// normally left for docker-proxy instead, which receives them through the
|
||||
// host's INPUT chain, where a host firewall such as ufw drops them. Ingress
|
||||
// ports are DNATed anyway, so that reaching a published port doesn't depend
|
||||
// on the userland proxy or the host's input policy. Both modes are checked
|
||||
// and expected to behave the same.
|
||||
//
|
||||
// Regression test for https://github.com/moby/moby/issues/53713
|
||||
// Regression test for https://github.com/moby/moby/issues/53713 and
|
||||
// https://github.com/moby/moby/issues/53962
|
||||
func TestIngressPortFromTaskOnSameNode(t *testing.T) {
|
||||
skip.If(t, testEnv.IsRemoteDaemon)
|
||||
skip.If(t, testEnv.IsRootless, "rootless mode doesn't support Swarm-mode")
|
||||
@@ -1031,6 +1035,12 @@ func TestIngressPortFromTaskOnSameNode(t *testing.T) {
|
||||
defer l3.Destroy(t)
|
||||
l3.AddHost(t, l3SegHost, "ns-"+l3SegHost, "eth0", netip.MustParsePrefix(hostAddr+"/24"))
|
||||
|
||||
// Drop unsolicited input, like ufw's default "deny incoming" policy.
|
||||
host := l3.Hosts[l3SegHost]
|
||||
host.MustRun(t, "iptables", "-A", "INPUT", "-i", "lo", "-j", "ACCEPT")
|
||||
host.MustRun(t, "iptables", "-A", "INPUT", "-m", "conntrack", "--ctstate", "RELATED,ESTABLISHED", "-j", "ACCEPT")
|
||||
host.MustRun(t, "iptables", "-P", "INPUT", "DROP")
|
||||
|
||||
l3.Hosts[l3SegHost].Do(t, func() {
|
||||
// Not swarm.NewSwarm, because it can't pass extra daemon flags.
|
||||
d := daemon.New(t, daemon.WithSwarmIptables(true))
|
||||
|
||||
Reference in new issue
Block a user