Looks like some packages fail in go module mode, because they require
recent Go versions:
GO111MODULE=on go test -v
# github.com/docker/docker/libnetwork/ipamutils
../../libnetwork/ipamutils/utils.go:46:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipamutils/utils.go:51:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/portallocator
../../libnetwork/portallocator/portallocator.go:179:7: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/netutils
../../libnetwork/netutils/utils_linux.go:66:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/netutils/utils_linux.go:75:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/api/server/router/grpc
../../api/server/router/grpc/grpc.go:56:48: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/container
../../container/view.go:335:47: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/ipams/defaultipam
../../libnetwork/ipams/defaultipam/address_space.go:33:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:53:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:124:10: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:125:21: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:146:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:310:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:311:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/drivers/bridge
../../libnetwork/drivers/bridge/port_mapping_linux.go:76:15: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/drivers/bridge/port_mapping_linux.go:201:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The previous allocator was subnetting address pools eagerly
when the daemon started, and would then just iterate over that
list whenever RequestPool was called. This was leading to high
memory usage whenever IPv6 pools were configured with a target
subnet size too different from the pools prefix size.
For instance: pool = fd00::/8, target size = /64 -- 2 ^ (64-8)
subnets would be generated upfront. This would take approx.
9 * 10^18 bits -- way too much for any human computer in 2024.
Another noteworthy issue, the previous implementation was allocating
a subnet, and then in another layer was checking whether the
allocation was conflicting with some 'reserved networks'. If so,
the allocation would be retried, etc... To make it worse, 'reserved
networks' would be recomputed on every iteration. This is totally
ineffective as there could be 'reserved networks' that fully overlap
a given address pool (or many!).
To fix this issue, a new field `Exclude` is added to `RequestPool`.
It's up to each driver to take it into account. Since we don't know
whether this retry loop is useful for some remote IPAM driver, it's
reimplemented bug-for-bug directly in the remote driver.
The new allocator uses a linear-search algorithm. It takes advantage
of all lists (predefined pools, allocated subnets and reserved
networks) being sorted and logically combines 'allocated' and
'reserved' through a 'double cursor' to iterate on both lists at the
same time while preserving the total order. At the same time, it
iterates over 'predefined' pools and looks for the first empty space
that would be a good fit.
Currently, the size of the allocated subnet is still dictated by
each 'predefined' pools. We should consider hardcoding that size
instead, and let users specify what subnet size they want. This
wasn't possible before as the subnets were generated upfront. This
new allocator should be able to deal with this easily.
The method used for static allocation has been updated to make sure
the ascending order of 'allocated' is preserved. It's bug-for-bug
compatible with the previous implementation.
One consequence of this new algorithm is that we don't keep track
of where the last allocation happened, we just allocate the first
free subnet we find.
Before:
- Allocate: 10.0.1.0/24, 10.0.2.0/24 ; Deallocate: 10.0.1.0/24 ;
Allocate 10.0.3.0/24.
Now, the 3rd allocation would yield 10.0.1.0/24 once again.
As it doesn't change the semantics of the allocator, there's no
reason to worry about that.
Finally, about 'reserved networks'. The heuristics we use are
now properly documented. It was discovered that we don't check
routes for IPv6 allocations -- this can't be changed because
there's no such thing as on-link routes for IPv6.
(Kudos to Rob Murray for coming up with the linear-search idea.)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The function references global shared, mutable state and is no longer
needed. Deleting it brings us one step closer to getting rid of that
pesky shared state.
Signed-off-by: Cory Snider <csnider@mirantis.com>
netlink offers the netlink.LinkNotFoundError type, which we can use with
errors.As() to detect a unused link name.
Additionally, early return if GenerateRandomName fails, as reading
random bytes should be a highly reliable operation, and otherwise the
error would be swallowed by the fall-through return.
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
We only need the content here, not the checksum, so simplifying the code by
just using os.ReadFile().
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The global netlink handle ns.NlHandle() is indirectly cached for the
life of the process by the netutils.CheckRouteOverlaps() function. This
caching behaviour is a problem for the libnetwork unit tests as the
global netlink handle changes every time testutils.SetupTestOSContext()
is called, i.e. at the start of nearly every test case. Route overlaps
can be checked for in the wrong network namespace, causing spurious test
failures e.g. when running the same test twice in a row with -count=2.
Stop the netlink handle from being cached by shadowing the package-scope
variable with a function-scoped one.
Signed-off-by: Cory Snider <csnider@mirantis.com>
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This change brings global default address pool feature into
libnetwork. Idea is to reuse same code flow and functions that were
implemented for local scope default address pool.
Function InitNetworks carries most of the changes. local scope default
address pool init should always happen only once. But Global scope
default address pool can be initialized multiple times.
Signed-off-by: selansen <elango.siva@docker.com>
This fix tries to address the issue raised in:
https://github.com/docker/docker/issues/26341
where multiple addresses in a bridge may cause `--fixed-cidr` to
not have the correct addresses.
The issue is that `netutils.ElectInterfaceAddresses(bridgeName)`
only returns the first IPv4 address.
This fix changes `ElectInterfaceAddresses()` and `addresses()`
so that all IPv4 addresses are returned. This will allow the
possibility of selectively choose the address needed.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Currently ipam/ipamutils has a bunch of dependencies
in osl and netlink which makes the ipam/ipamutils harder
to use independently with other applications. This PR
modularizes ipam/ipamutils into a standalone package
with no OS level dependencies.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Compile the dnet tool for Linux (x86, amd64 and arm)
and Windows (x86 and amd64)
- Moved installation of dependencies into `Dockerfile.build`
- Remove `start-services` from Makefile
- That's the responsibility of Docker or build environment
- Removed utils depending on `netlink` from `netutils/utils.go`
Unable to add `make cross` to CircleCI just yet as there are some
issues to solve that are unrelated to this PR
Also fix `.gitignore` which was not updated after changing the build
image name in #667
Signed-off-by: Dave Tucker <dt@docker.com>
