By restoring the effective capabilities of the thread after setresuid()
we can both:
1. Use the go runtime to setup the uid_map now that we have the
capabilities to do so in the thread again
2. Enable this on distro's which have restrictions around
unprivileged user namespace creation and usage (since the thread
is now privileged)
Let's do it. See [0] for more details on this topic. Unlike
unix::Setresuid()[1], which mimics the glibc implementation and acts on all
threads in the process, unix::Cap{s,g}et() are thread local[2] only as we
want, so we can use that directly.
[0]: https://github.com/containerd/containerd/pull/12317#discussion_r2686960671
[1]: e2fef50def/src/syscall/syscall_linux.go (L1217)
[2]: 6fb913b30f/unix/zsyscall_linux.go (L524)
Signed-off-by: Andrew Halaney <ahalaney@netflix.com>
Right now containerd uses os.StartProcess() to create a dummy ptraced
process via a fork/exec in a new user namespace, with the uid_map/gid_map
setup.
This doesn't work so well with user limits[0] in the kernel, at least if
you expect the container user to have separate isolation from the host's
"normal users". The kernel's ucount mechanism forms a ucount tuple of (ns, kuid_t)
for each user limit. Say containerd runs as uid 0 in the initial user namespace,
and the container runs as uid 100000 from the inital user namespace point of view.
When the container tries to do things like inotify_add_watch(), the
kernel:
1. First verifies that the ucount(container user namespace, 100000)
doesn't exceed its "per user per user namespace limits" and
increments the counter there. This limit is set by
/proc/sys/user/max_inotify_watches when in the namespace
2. Then walks up to the ucount who created this namespace,
ucount(initial user namespace, 0) and increments their counter as
well ensuring they don't exceed their limit. In our example this
is the initial user namespace, this limit is set by
/proc/sys/fs/inotify/max_user_watches as well as
/proc/sys/user/max_inotify_watches in that case.
This is done so a user can't escape per user limits by creating a user
namespace and running as different users in that user namespace. The
accounting always rolls back up to the user who created the user
namespace to ensure this, checking limits at each layer.
This means if you have a rogue container they can consume all of
the containerd user's inotify limits. In practice this means global root
is in danger of being denied usage of resources due to the container,
while other less important users are still well within their budget!
In the inotify case for example systemd will fail to start many new
services due to this.
Let's instead create the user namespace *as the container's initial user
namespace user*. This means that all the attribution for these user
limits rolls up to this unimportant user, preventing one container from
exhausting global root's resources, and further isolating each container
from each other (since each pod in k8s runs as its own disjoint set of
users in the initial user namespace).
This is a bit annoying to do in golang. The best option seems to be what
we have here:
1. Lock OS thread
2. Manually setresuid() (seteuid implementation in golang mimics
glibc's and sets the euid for all of the threads, which causes
issues for other threads running at the same time)
3. Create user namespace
4. Undo the setresuid()
5. Update the uid_map/gid_map as root now (can't do this anymore as
part of (3) since to update the map you must have CAP_SETUID
which the container user will not have)
With this in place you can verify the user namespace is owned by the
proper uid via ioctl(fd, NS_GET_OWNER_UID). Some distros block
unprivleged user namespace creation, for those we'll just continue to
create it as containerd's user.
[0]: https://docs.kernel.org/admin-guide/sysctl/user.html#documentation-for-proc-sys-user
Signed-off-by: Andrew Halaney <ahalaney@netflix.com>
This PR adds opt-in tracing spans/attributes in CRI image pull and selected sandbox-related paths to improve debugging and correlation (e.g., sandbox.id/pod metadata). If maintainers prefer a smaller diff, I’m happy to split this into a pull-only PR plus follow-ups.
• follow-up after pull-only PR
• focuses on task/metadata/sandbox/cni setup spans
Signed-off-by: Cindy Li <cindyli@pinterest.com>
Go 1.24 introduced stricter checks for os.DirFS (via os.Root), which causes failures when /etc/passwd or /etc/group are absolute symlinks pointing outside the mount root (common in NixOS).
This patch introduces a helper that detects absolute symlinks and resolves them relative to the rootfs before opening, preventing the 'path escapes from parent' error.
Fixes#12683
Signed-off-by: Paulo Oliveira <paulo.hco47@gmail.com>
Deprecate the enable_cdi config option for CRI. Change it to a
pointer so we can differentiate between an unset value and one
that is set explicitly to false. Treat an omitted setting (nil
value) as enabled. Mark it deprecated in RELEASES.md. Add a
deprecation warning for it. Log that warning if we start up
with enable_cdi explicitly set to false.
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
Replace manual platform formatting with containerd's platforms.Format()
function to ensure consistent platform string formatting across the
codebase. This removes the TODO comment and handles platform variants
properly.
Fixes the TODO in manifest_printer.go line 124.
Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>
This commit makes all of the recommended changes to use the `testing`
package helper functions instead of doing the equivalent longhand
versions of the same thing.
This change was needed in order to properly detect errors, as the code
would previously skip running `tenv` stating that it had been deprecated
in favor of `usetesting`.
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
Switch to use fs.FS interface over directly requiring path string.
Use os.OpenRoot over continuity RootPath.
Signed-off-by: Derek McGowan <derek@mcg.dev>
UnshareAfterEnterUserns() creates a pidfd via os.StartProcess() with
CLONE_PIDFD but fails to close the file descriptor in any code path,
resulting in a file descriptor leak for every container that uses user
namespace isolation.
The leak occurs because:
- The pidfd is created when PidFD field is set in SysProcAttr
- The original defer block only calls PidfdSendSignal() and
pidfdWaitid()
- No code path calls unix.Close(pidfd) to release the file descriptor
This causes one pidfd leak per container launch when user namespace
isolation is enabled (e.g., Kubernetes pods with hostUsers: false). In
production environments with high container churn, this can exhaust the
system's file descriptor limit.
Fix the leak by adding a defer statement immediately after process
creation that ensures unix.Close(pidfd) is always called, regardless of
which code path is taken. This guarantees cleanup even if the function
returns early due to errors or lack of pidfd support.
This follows the same cleanup pattern already established in
core/mount/mount_idmapped_utils_linux.go:getUsernsFD() which properly
closes its pidfd.
Closes: #12166
Signed-off-by: Jose Fernandez <josef@netflix.com>
Now that we have 1.24.x as go min version, I think
we can remove this go code specific to a lower
version.
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
This variable was introduced in 062c3a00ef,
which didn't describe it as intentional to be able to override the option.
Based on the above, I assume the use of a variable was purely convenience,
the there's no intent for packages to be able to override them, so this
patch changes these to be a regular function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were introduced in c818a6b13d, refactored
in 808b223536 and bdd84abf05,
and moved in a2d1a8a865, but none provided
a motivation for using a variable / alias for these.
Based on the above, I assume the use of a variable was purely convenience,
the there's no intent for packages to be able to override them, so this
patch changes these to be a regular function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use testify for asserting
- Fix various unhandled errors
- Use native t.TempDir() for temporary files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were added as part of b7f673790f,
to provide backward compatibility with go1.16, which we no longer
supports, so we can remove this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove redundant aliases for imports
- rename variables that shadowed imports
- use errors.Is instead of straight error comparing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some of these options are designed to be a no-op when used on a Spec
that doesn't match the platform for the option. However, if the given
plaform was not present, they would panic.
This patch:
- Adds an early-return for options that are only applied on a
specific platform.
- Update the GoDoc for these functions to describe they're a
no-op on other platforms.
- Adds some rudimentary unit-tests to verify their behavior.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rather than exiting early with an error, just output that the content
does not exist locally and continue. This is helpful for displaying
multiplatform images when the content for every platform was not pulled.
If a platform that did not exist locally showed up before one that did,
the inspect would error out before even showing the content.
Signed-off-by: Derek McGowan <derek@mcg.dev>
