The default IPAM allocator is unable to represent subnets larger than
64-bits (subnets with a smaller prefix), because it uses a Bitmap
that's limited to 64-bits.
When it's used to represent a 64-bit subnet, the top address can't
be allocated (because bitmap.Bitmap is initialised with the number
of bits it needs to represent in a uint64, so it's one short).
The rest of the daemon doesn't know about these limitations, so
strange things happen when a large IPv6 subnet is used.
No errors are reported, addresses/subnets are just set up incorrectly.
The IPAM code calculates offsets into the bitmap itself, details it
shouldn't need to understand and, because it's working on offsets
into a range it doesn't always notice when it's asked to set a bit
outside the range.
It's unusual to need a big subnet but, for example, it may be useful
for modelling an ISP network, or an ISP's gateway may be in a "/56"
subnet that's outside a 64-bit range used by hosts.
So, use addrset.AddrSet instead of bitmap.Bitmap.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When defaultipam.newPoolData is asked for a pool of 64-bits
or more, it ends up with an overflowed u64 - so, it just
subtracts one to get a nearly-big-enough range (for a 64-bit
subnet).
When defaultipam.getAddress is called with an ipr (sub-pool
range), the range it calls bitmask.SetAnyInRange with is
exclusive of end. So, its end param can't be MaxUint64,
because that's the max value for the top end of the range
and, when checking the range, SetAnyInRange fails.
When fixed-cidr-v6 behaves more like fixed-cidr, it will ask
for a 64-bit range if that's what fixed-cidr-v6 needs. So,
it hits the bug when allocating an address for, for example:
docker network create --ipv6 --subnet fddd::/64 --ip-range fddd::/64 b46
The additional check for "ipr == base" avoids the issue in
this case, by ignoring the ipr/sub-pool range if ipr is the
same as the pool itself (not really a sub-pool).
But, it still fails when ipr!=base. For example:
docker network create --ipv6 --subnet fddd::/56 --ip-range fddd::/64 b46
So, also subtract one from 'end' if it's going to hit the max
value allowed by the Bitmap.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Until this commit, the default local address pool was initialized by the
defaultipam driver if none was provided by libnet / the daemon.
Now, defaultipam errors out if none is passed and instead the daemon is
made responsible for initializing it with the default values if the user
don'te set the related config parameter.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Nothing was validating whether address pools' `base` prefix
were larger than the target subnet `size` they're associated to. As
such invalid address pools would yield no subnet, the error could go
unnoticed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The previous allocator was subnetting address pools eagerly
when the daemon started, and would then just iterate over that
list whenever RequestPool was called. This was leading to high
memory usage whenever IPv6 pools were configured with a target
subnet size too different from the pools prefix size.
For instance: pool = fd00::/8, target size = /64 -- 2 ^ (64-8)
subnets would be generated upfront. This would take approx.
9 * 10^18 bits -- way too much for any human computer in 2024.
Another noteworthy issue, the previous implementation was allocating
a subnet, and then in another layer was checking whether the
allocation was conflicting with some 'reserved networks'. If so,
the allocation would be retried, etc... To make it worse, 'reserved
networks' would be recomputed on every iteration. This is totally
ineffective as there could be 'reserved networks' that fully overlap
a given address pool (or many!).
To fix this issue, a new field `Exclude` is added to `RequestPool`.
It's up to each driver to take it into account. Since we don't know
whether this retry loop is useful for some remote IPAM driver, it's
reimplemented bug-for-bug directly in the remote driver.
The new allocator uses a linear-search algorithm. It takes advantage
of all lists (predefined pools, allocated subnets and reserved
networks) being sorted and logically combines 'allocated' and
'reserved' through a 'double cursor' to iterate on both lists at the
same time while preserving the total order. At the same time, it
iterates over 'predefined' pools and looks for the first empty space
that would be a good fit.
Currently, the size of the allocated subnet is still dictated by
each 'predefined' pools. We should consider hardcoding that size
instead, and let users specify what subnet size they want. This
wasn't possible before as the subnets were generated upfront. This
new allocator should be able to deal with this easily.
The method used for static allocation has been updated to make sure
the ascending order of 'allocated' is preserved. It's bug-for-bug
compatible with the previous implementation.
One consequence of this new algorithm is that we don't keep track
of where the last allocation happened, we just allocate the first
free subnet we find.
Before:
- Allocate: 10.0.1.0/24, 10.0.2.0/24 ; Deallocate: 10.0.1.0/24 ;
Allocate 10.0.3.0/24.
Now, the 3rd allocation would yield 10.0.1.0/24 once again.
As it doesn't change the semantics of the allocator, there's no
reason to worry about that.
Finally, about 'reserved networks'. The heuristics we use are
now properly documented. It was discovered that we don't check
routes for IPv6 allocations -- this can't be changed because
there's no such thing as on-link routes for IPv6.
(Kudos to Rob Murray for coming up with the linear-search idea.)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Prior to this change, cnmallocator would call
`ConfigGlobalScopeDefaultNetworks` right before initializing its
IPAM drivers. This function was mutating some global state used
during drivers init.
This change just remove the global state, and adds an arg to
ipams.Register and defaultipam.Register to pass the global pools
by arguments instead.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
All drivers except the default have a Register function. Before this
change, default's registration was handled by another package. Move
this logic into the driver pkg.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
All drivers except the default ipam driver are stored in ipams/.
Since `default` isn't a valid Go pkg name, this package is
renamed to `defaultipam`, following `windowsipam` example.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
