Commit Graph

2570 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
4e39cdd9bb libnetwork: ipvlan: move validation into parseNetworkOptions()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:41 +02:00
Sebastiaan van Stijn
9f0cb20d9f libnetwork: macvlan: move validation into parseNetworkOptions()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:39 +02:00
Sebastiaan van Stijn
b768d69c04 libnetwork: macvlan: processIPAM(): simplify
Remove redundant checks and intermediate variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:37 +02:00
Sebastiaan van Stijn
5d13b38479 libnetwork: macvlan: processIPAM(): remove unused arg and error return
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:36 +02:00
Sebastiaan van Stijn
798021af9f libnetwork: macvlan: set network ID as part of parseNetworkOptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:33 +02:00
Sebastiaan van Stijn
35cba9b1c9 libnetwork: ipvlan: processIPAM(): simplify
Remove redundant checks and intermediate variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:32 +02:00
Sebastiaan van Stijn
8d067bbdb4 libnetwork: ipvlan: processIPAM(): remove unused arg and error return
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:30 +02:00
Sebastiaan van Stijn
a893540b66 libnetwork: ipvlan: set network ID as part of parseNetworkOptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:28 +02:00
Sebastiaan van Stijn
afeb4c7a6e libnetwork: macvlan: use single ipSubnet type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:27 +02:00
Sebastiaan van Stijn
d3e3d43482 libnetwork: ipvlan: use single ipSubnet type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:25 +02:00
Sebastiaan van Stijn
9fba1514d2 libnetwork: macvlan: clean up some consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:23 +02:00
Sebastiaan van Stijn
aec4853498 libnetwork: ipvlan: clean up some consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-01 11:44:17 +02:00
Youfu Zhang
f70a9788c5 drivers/ipvlan: add ipvlan_flag option, support l3s ipvlan_mode
Signed-off-by: Youfu Zhang <zhangyoufu@gmail.com>
2022-06-28 19:28:49 +08:00
Sebastiaan van Stijn
db977355b0 fix typo (cluser -> cluster)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 15:12:14 +02:00
Martin Braun
5edfd6d081 bump netlink library
bump netlink to 1.2.1
change usages of netlink handle .Delete() to Close()
remove superfluous replace in vendor.mod
make requires of github.com/Azure/go-ansiterm direct

Signed-off-by: Martin Braun <braun@neuroforge.de>
2022-06-16 22:25:33 +02:00
Sebastiaan van Stijn
9959eceb9a Merge pull request #42626 from mfeit-internet2/small-ipv4-networks
Support small ipv4 networks
2022-06-07 22:15:19 +02:00
Sebastiaan van Stijn
6e80c027c5 Merge pull request #43409 from vincentbernat/fix/udp-conntrack
bridge: also flush conntrack entries when setting up endpoints
2022-06-03 11:29:25 +02:00
Sebastiaan van Stijn
b4b2a0323b libnetwork/portallocator: un-export consts for defaults
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-02 22:59:32 +02:00
Albin Kerouanton
414dd017b3 Revert "Added API to set ephemeral port allocator range."
Since commit 2c4a868f64, Docker doesn't
use the value of net.ipv4.ip_local_port_range when choosing an ephemeral
port. This change reverts back to the previous behavior.

Fixes #43054.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2022-06-02 22:56:18 +02:00
Sebastiaan van Stijn
5a8304c9c5 Merge pull request #43670 from evol262/ipvs-module-idempotency
Ensure performance tuning is always applied
2022-05-31 22:00:40 +02:00
Ryan Barry
293cfd6c76 Ensure performance tuning is always applied
Previously, with the patch from #43146, it was possible for a
network configured with a single ingress or load balancer on a
distribution which does not have the `ip_vs` kernel module loaded
by default to try to apply sysctls which did not exist yet, and
subsequently dynamically load the module as part of ipvs/netlink.go.

This module is vendored, and not a great place to try to tie back
into core libnetwork functionality, so also ensure that the sysctls
(which are idempotent) are called after ingress/lb creation once
`ipvs` has been initialized.

Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-05-31 11:47:30 -04:00
Ameya Gawde
cba8cf34d2 Set ExternalPortReserved for dummy proxy
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2022-05-25 07:12:43 -07:00
Sebastiaan van Stijn
1aea4c2bbd Merge pull request #43146 from evol262/fix/ingress-namespace-performance
Apply peformance tuning to new sandboxes also
2022-05-18 18:28:30 +02:00
Ryan Barry
0dd3a2eade Apply performance tuning to new sandboxes also
relates to #35082, moby/libnetwork#2491

Previously, values for expire_quiescent_template, conn_reuse_mode,
and expire_nodest_conn were set only system-wide. Also apply them
for new lb_* and ingress_sbox sandboxes, so they are appropriately
propagated

Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-05-17 15:44:49 -04:00
Eng Zer Jun
7873c27cfb all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Samuel Karp
ccb691a427 Merge pull request #43511 from thaJeztah/no_logrus_fatal 2022-04-21 11:33:43 -07:00
Sebastiaan van Stijn
df650a1aeb panic() instead of logrus.Fatal() in init funcs
Some packages were using `logrus.Fatal()` in init functions (which logs the error,
and (by default) calls `os.Exit(1)` after logging).

Given that logrus formatting and outputs have not yet been configured during the
initialization stage, it does not provide much benefits over a plain `panic()`.

This patch replaces some instances of `logrus.Fatal()` with `panic()`, which has
the added benefits of not introducing logrus as a dependency in some of these
packages, and also produces a stacktrace, which could help locating the problem
in the unlikely event an `init()` fails.

Before this change, an error would look like:

    $ dockerd
    FATA[0000] something bad happened

After this change, the same error looks like:

    $ dockerd
    panic: something bad happened

    goroutine 1 [running]:
      github.com/docker/docker/daemon/logger/awslogs.init.0()
        /go/src/github.com/docker/docker/daemon/logger/awslogs/cloudwatchlogs.go:128 +0x89

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 12:15:20 +02:00
Samuel Karp
c5f18aac9f Merge pull request #43505 from thaJeztah/libnetwork_no_string_fields
libnetwork: don't use strings.Fields() to improve performance
2022-04-21 00:57:55 -07:00
Sebastiaan van Stijn
301b252b58 libnetwork: don't use strings.Fields() to improve performance
While looking at this code, I noticed that we were wasting quite some resources
by first constructing a string, only to split it again (with `strings.Fields()`)
into a string slice.

Some conversions were also happening multiple times (int to string, IP-address to
string, etc.)

Setting up networking is known to be costing a considerable amount of time when
starting containers, and while this may only be a small part of that, it doesn't
hurt to save some resources (and readability of the code isn't significantly
impacted).

For example, benchmarking the `redirector()` code before/after:

    BenchmarkParseOld-4   	  137646	      8398 ns/op	    4192 B/op	      75 allocs/op
    BenchmarkParseNew-4   	  629395	      1762 ns/op	    2362 B/op	      24 allocs/op

Average over 10 runs:

    benchstat old.txt new.txt

    name     old time/op    new time/op    delta
    Parse-4    8.43µs ± 2%    1.79µs ± 3%  -78.76%  (p=0.000 n=9+8)

    name     old alloc/op   new alloc/op   delta
    Parse-4    4.19kB ± 0%    2.36kB ± 0%  -43.65%  (p=0.000 n=10+10)

    name     old allocs/op  new allocs/op  delta
    Parse-4      75.0 ± 0%      24.0 ± 0%  -68.00%  (p=0.000 n=10+10)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 14:43:07 +02:00
Trapier Marshall
a46700dbdb Cleanup servicebindings only on Windows
Make the call to cleanupServiceBindings during network deletion
conditional on Windows (where it is required), thereby providing a
performance improvement to network cleanup on Linux.

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:22:16 +00:00
Trapier Marshall
6861aade58 Delay network deletion until after lb cleanup
Removal of PolicyLists from Windows VFP must be performed prior to
removing the HNS network. Otherwise PolicyList removal fails with
HNS error "network not found".

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:22:08 +00:00
Trapier Marshall
556cb3ae81 Log HNS policylist removal failures
Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:21:29 +00:00
Vincent Bernat
e5812117a5 bridge: also flush conntrack entries when setting up endpoints
There is a race condition between the local proxy and iptables rule
setting. When we have a lot of UDP traffic, the kernel will create
conntrack entries to the local proxy and will ignore the iptables
rules set after that.

Related to PR #32505. Fix #8795.

Signed-off-by: Vincent Bernat <vincent@bernat.ch>
2022-03-23 08:34:26 +01:00
Martin Dojcak
feab0cca9f libnetwork/overlay:fix join sandbox deadlock
Operations performed on overlay network sandboxes are handled by
dispatching operations send through a channel. This allows for
asynchronous operations to be performed which, since they are
not called from within another function, are able to operate in
an idempotent manner with a known/measurable starting state from
which an identical series of iterative actions can be performed.

However, it was possible in some cases for an operation dispatched
from this channel to write a message back to the channel in the
case of joining a network when a sufficient volume of sandboxes
were operated on.

A goroutine which is simultaneously reading and writing to an
unbuffered channel can deadlock if it sends a message to a channel
then waits for it to be consumed and completed, since the only
available goroutine is more or less "talking to itself". In order
to break this deadlock, in the observed race, a goroutine is now
created to send the message to the channel.

Signed-off-by: Martin Dojcak <martin.dojcak@lablabs.io>
Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-03-22 11:15:14 -04:00
Sebastiaan van Stijn
1b3fef5333 Windows: require Windows Server RS5 / ltsc2019 (build 17763) as minimum
Windows Server 2016 (RS1) reached end of support, and Docker Desktop requires
Windows 10 V19H2 (version 1909, build 18363) as a minimum.

This patch makes Windows Server RS5 /  ltsc2019 (build 17763) the minimum version
to run the daemon, and removes some hacks for older versions of Windows.

There is one check remaining that checks for Windows RS3 for a workaround
on older versions, but recent changes in Windows seemed to have regressed
on the same issue, so I kept that code for now to check if we may need that
workaround (again);

085c6a98d5/daemon/graphdriver/windows/windows.go (L319-L341)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:28 +01:00
Sebastiaan van Stijn
b9c8eca468 libnetwork/networkdb: remove some redundant fmt.Sprintf()'s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-15 12:56:23 +01:00
Sebastiaan van Stijn
85ed9b8746 libnetwork: fix unhandled errors in tests (ineffassign)
libnetwork/libnetwork_test.go:1014:8: ineffectual assignment to err (ineffassign)
        sbx1, err := controller.NewSandbox(containerID,
              ^
    libnetwork/libnetwork_test.go:1024:8: ineffectual assignment to err (ineffassign)
        sbx2, err := controller.NewSandbox("c2")
              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-08 09:43:27 +01:00
Sebastiaan van Stijn
7b692a421b libnetwork: remove more config bits related to external k/v stores
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:45 +01:00
Sebastiaan van Stijn
745ba3ecbc libnetwork: remove etcd-related code and tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:43 +01:00
Sebastiaan van Stijn
147173b099 libnetwork: remove consul-related code and tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:41 +01:00
Sebastiaan van Stijn
25594c33b9 libnetwork: replace consul with boltdb in test
Based on randomLocalStore() in libnetwork/ipam/allocator_test.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:07 +01:00
Sebastiaan van Stijn
a7d0f3060a libnetwork: remove zookeeper-related code and tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:30 +01:00
Anca Iordache
00f9b23c3a libnetwork: remove external DS-based host discovery
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:26 +01:00
Tianon Gravi
8955d8da89 Merge pull request #42981 from frobnicaty/patch-1
Fix grammar for "does not exist"
2021-12-09 13:36:53 -08:00
frobnicaty
d78b883576 Fix grammar for "does not exist"
as opposed to "does not exists"

Signed-off-by: frobnicaty <92033765+frobnicaty@users.noreply.github.com>
2021-12-03 15:50:13 +00:00
Albin Kerouanton
c721bad8cc Fix flaky TestPortMappingV6Config
Since moby/libnetwork#2635 has been merged, allocatePortsInternal()
checks if IPv6 is enabled by calling IsV6Listenable(). This function
calls `net.Listen("tcp6", "[::1]:0")` and returns false when
net.Listen() fails.

TestPortMappingV6Config() starts by setting up a new net ns to run into
it. The loopback interface is not bring up in this net ns, thus
net.Listen() fails and IsV6Listenable() returns false. This change takes
care of bringing loopback iface up right after moving to the new net ns.

This test has been reported has flaky on s390x in #42468. For some
reason, this test seems to be consistently green on the CI (on amd64
arch) and when running `hack/test/unit` locally. However it consistently
fails when running `TESTFLAGS='-shuffle on' hack/test/unit` locally.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2021-11-16 09:37:34 +01:00
Mark Feit
3a938df4b5 Make the network allocator handle IPv4 blocks too small for network/broadcast addresses.
This was originally in docker/libnetwork#2624, which has been closed since the
code was moved here.

When creating a new network, IPAM's address allocator attempts to reserve the
network and broadcast addresses on IPv4 networks of all sizes. For RFC 3021
point-to-point networks (IPv4 /31s), this consumes both available addresses and
renders any attempt to allocate an address from the block unsuccessful.

This change prevents those reservations from taking place on IPv4 networks having
two or fewer addresses (i.e., /31s and /32s) while retaining the existing behavior
for larger IPv4 blocks and all IPv6 blocks.

In case you're wondering why anyone would allocate /31s:  I work for a network
service provider.  We use a lot of point-to-point networks.  This cuts our
address space utilization for those by 50%, which makes ARIN happy.

This patch modifies the network allocator to recognize when an network is too
small for network and broadcast addresses and skip those reservations.

There are additional unit tests to make sure the functions involved behave as expected.

Try these out:

 * `docker network create --driver bridge --subnet 10.200.1.0/31 --ip-range 10.200.1.0/31 test-31`
 * `docker network create --driver bridge --subnet 10.200.1.0/32 --ip-range 10.200.1.0/32 test-32`

My installation has been running this patch in production with /31s since March.

Signed-off-by: Mark Feit <mfeit@internet2.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-27 13:03:19 +02:00
Sebastiaan van Stijn
eadf839530 fix TestInvalidRemoteDriver() to check underlying error
commit b1a3fe4934 changed how the error was
returned (which is now wrapped), causing the test to fail:

    === RUN   TestInvalidRemoteDriver
        libnetwork_test.go:1289: Did not fail with expected error. Actual error: Plugin does not implement the requested driver: plugin="invalid-network-driver", requested implementation="NetworkDriver"
    --- FAIL: TestInvalidRemoteDriver (0.01s)

Changing the test to use errors.Is()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-25 14:57:18 +02:00
Sebastiaan van Stijn
79d6e935ad libnetwork: some minor refactoring / cleanup
- don't pass the query's quetion.name separately, as we're already
  passing the query itself.
- remove a "fallthrough" in favor of combining the cases in the switch

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-15 15:26:15 +02:00
Sebastiaan van Stijn
d86a331fa4 libnetwork: improve consistency in log messages
- Make sure all log messages have the `[resolver]` prefix
- Use `logrus.WithError()` consistently
- Improve information included in some logs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-15 12:51:02 +02:00