The previous rule would cause nft to coredump on IPv6:
```console
$ nft add rule ip6 kube-proxy 'svc-ABC123-default/my-svc/tcp/https' meta l4proto tcp dnat ip6 addr . port to fd00:10:244::4 . 6443
Assertion failed: dreg < ctx->reg_low (src/netlink_linearize.c: netlink_gen_expr: 911)
Aborted (core dumped) nft add rule ip6 kube-proxy 'svc-ABC123-default/my-svc/tcp/https' meta l4proto tcp dnat ip6 addr . port to fd00:10:244::4 . 6443
```
The new rule allows IPv6 without coredumping.
We introduced a feature gate "NFTablesNetlink" that is Beta (enabled by default).
When this feature gate is enabled, kube-proxy initializes the knftables client with the `knftables.UseNetlink` option.
This enables using netlink directly for listing rules instead of parsing the output of the `nft` command-line binary, improving performance.
Using a feature gate allows us to safely roll this out while giving users the ability to opt-out in case of problems.
Change-Id: Id68cbd314c40da65664a3d3b30ff78d720043ddc
Add a userspace proxy that accepts TCP connections on loopback NodePort
addresses and forwards them into the nftables data path, gated behind the
new KubeProxyNFTablesLocalhostNodePorts feature gate and enabled when
--nodeport-addresses explicitly includes loopback.
Signed-off-by: Austin Abro <austinabro321@gmail.com>
Extend --nodeport-addresses to accept the "localhost" and "all" keywords
alongside "primary", expanding them to the corresponding loopback and
zero CIDRs.
Signed-off-by: Austin Abro <austinabro321@gmail.com>
Moves endpoint logging from endpointslicecache.go down to endpointschangetracker.go merge and unmerge methods to be consistent with service updates, without losing endpoint tracking information.
Change-Id: I09369a4b32929d8df4b520921d74223090872a60
Logs when a nodePort is configured, cleared, or removed from a service, similar to how endpoints are logged, to improve traceability in kube-proxy.
Change-Id: I75179292cb7ef52649b6fb3081afea1d0c9534af
detectNumCPU sized nf_conntrack_max using github.com/google/cadvisor/lib
machine topology. nf_conntrack_max is host-wide, so it must be sized from the
node's CPU count, not runtime.NumCPU(): the latter honors the process cpuset
and undercounts when kube-proxy runs under a static CPU policy, which is the
behavior cadvisor worked around (kubernetes/kubernetes#99225).
Use cpuset.NumCPU() from k8s.io/utils, which reads the node's online CPU count
from /sys/devices/system/cpu/online, with a runtime.NumCPU() fallback. This
drops the cadvisor dependency from pkg/proxy/conntrack and bumps k8s.io/utils
to pick up cpuset.NumCPU.
Add import-boss .import-restrictions so the lean github.com/google/cadvisor/lib module is allowed only in the directories that consume it -- pkg/kubelet, cmd/kubelet/app, and pkg/proxy/conntrack -- and forbidden everywhere else. The repo-root .import-restrictions default-denies all github.com/google/cadvisor imports; the three consumer subtrees re-allow only .../lib.
Migrate the kubelet (and the kube-proxy conntrack helper) off the full github.com/google/cadvisor module onto the lean github.com/google/cadvisor/lib: repoint info/v1+info/v2 type usage to lib/model and the manager/fs/cache/etc. consumers to lib/*; regenerate the cadvisor.Interface mocks; keep the kubelet-pinned cAdvisor global flags via lib/cadvisorflags.
go.mod: require + replace github.com/google/cadvisor/lib (=> the dims/cadvisor/lib fork for now, until lib is tagged) and drop the full github.com/google/cadvisor module entirely; keep github.com/containerd/containerd/api at v1.11.0 (matching upstream master); add github.com/google/cadvisor to unwanted-dependencies.json unwantedModules so the full module cannot be re-vendored.
test/e2e_node: the one remaining consumer of the full module -- the node-e2e ResourceCollector, which used the v2 HTTP client (client/v2) + v2 API types (info/v2) -- now scrapes the standalone cAdvisor pod's /api/v2.1/stats directly over HTTP+JSON, so test/e2e_node depends on no cAdvisor package. No change to the kubelet's stats surfaces.
The conntrack reconciler skips services without serving endpoints, so
conntrack entries established while endpoints existed are never removed
when a UDP service scales down to zero. The REJECT (iptables) / reject
(nftables) rule installed for such services does not cover those flows:
they are DNATed to the deleted endpoint IP before the rule, which
matches on the service IP, can be evaluated. One-way UDP senders (e.g.
statsd clients) refresh the 30s conntrack timeout with every packet, so
the stale flows blackhole traffic to the deleted pod IP indefinitely;
recovery only happens when the service gets an endpoint again.
This was handled before the reconciler rewrite (kubernetes#127318):
the event-based cleanup cleared entries for every deleted UDP endpoint
regardless of how many endpoints remained.
Process services with an empty endpoints set instead of skipping them,
so every entry directed to their ClusterIP, LoadBalancer IP and
ExternalIP frontends is treated as stale and deleted.
NodePort cleanup is still skipped for services without serving
endpoints: NodePort entries are matched on the destination port only,
and with an empty endpoints set that would also remove UDP flows not
owned by kube-proxy (e.g. traffic to an unrelated host on the same
port).
The classifyLBError function returns lbErrNone when err is nil,
but the test was incorrectly expecting lbErrOther.
Also add lbErrNone to TestLBErrorTypeConstants verification.
GetAllLocalAddressesExcept previously iterated over net.Interfaces() and
called iface.Addrs() for each interface. iface.Addrs() internally performs
a full RTM_GETADDR netlink dump for the entire node and then filters in
user space. With many interfaces and many addresses (for example tens of
thousands of ClusterIPs bound to kube-ipvs0) the cost is
O(N_interfaces * N_addresses) and dominates syncProxyRules latency.
This change replaces the per-interface loop with a single
netlink.AddrList(nil, unix.AF_UNSPEC) call that dumps all addresses on
the node in one RTM_GETADDR request, then filters by LinkIndex in user
space. This makes the call O(N_addresses) and avoids the per-interface
fan-out.
On a production node with 251 interfaces and 19757 addresses, this
reduces GetAllLocalAddressesExcept latency from 34.8s to 60ms (~705x).
ListEntries' result slice is bounded by the number of lines parsed from
the ipset output (`len(strs)`). Pass that as the make() capacity so
the slice doesn't need to be regrown as entries are appended.
No behavior change.
GetVirtualServers and GetRealServers each build a result slice by
appending one entry per source element (`ipvsSvcs` and `dsts`
respectively). Pass the source length as the make() capacity so the
slice does not need to be regrown during the loop.
No behavior change.
Simplify the interface between cmd/kube-proxy and the backends by
passing the complete KubeProxyConfiguration to the backend rather than
having kube-proxy need to know specifically which fields each backend
cares about.
