When CreateSandbox or StartSandbox fails because the caller's context
was canceled or ran past its deadline, cleanupShim passed that same
context to ShutdownSandbox and derived the shim deletion timeout from
it, so both failed at once and the shim, its sandbox and the mounts
under its bundle were left behind. The cleanup now keeps the values of
the caller's context but not its cancellation, and bounds the shutdown
and the deletion with timeouts of their own; the shutdown one leaves the
shim time to stop what it runs for the sandbox. A closed connection
from a shim that exited before answering ShutdownSandbox is not logged
as a failure, as Shutdown already treats it as success.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
The sandboxed shim shares a pod PID namespace now, so the critest step
for it no longer skips "runtime should support PodPID". The user
namespace specs still skip themselves.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
A pod that shares its PID namespace (shareProcessNamespace) needs a
durable PID 1 for it, the role pause plays. CreateSandbox now runs
containerd-sandbox-init for such a pod through runc: in the sandbox
bundle with a tmpfs rootfs holding only the program, under the pod
cgroup with either cgroup driver, in the pod network, IPC and UTS
namespaces, as an unprivileged user without capabilities, under
containerd's default seccomp profile, with the masked and read-only
/proc paths of a default container spec and the OOM score and CPU
shares of a pause container. Its container id is the sandbox id. The
PID namespace is pinned under the bundle like the IPC and UTS
namespaces and listed in the sandbox spec, so the containers join it
the same way; the sandbox still reports pid 0. StartSandbox execs the
program, StopSandbox kills and deletes it and reports its exit, and an
init that exits on its own stops the sandbox. Every other pod still has
no sandbox process.
The pod SELinux label is not applied to the init yet.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
"shim start" writes the runc options of the runtime handler and the
runtime name into the sandbox bundle, the files runc.NewContainer writes
for a task bundle. The sandbox service reads them to run a process
through the same runc, root and cgroup driver as the containers of the
pod, and "shim delete" reads them to force-delete that runc container
once the shim is gone.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
A minimal PID 1 for the PID namespace of a pod that shares it, run by
containerd-shim-sandboxed-runc-v2 in place of a pause container. It
stays alive so that the namespace does, ignores SIGCHLD so that the
kernel reaps the orphans reparented to it, and removes every signal
handler so that nothing sent from inside the pod can end it or run code
in it: the kernel drops a signal sent to the init of a PID namespace
when init has no handler for it. The Go runtime installs handlers for
almost every signal, including the synchronous faults, SIGPROF and the
signals 32 and 33 that os/signal cannot reset, so those go back to the
default disposition, or are ignored for the others, through
rt_sigaction. Only SIGKILL from the
parent namespace ends it. It makes itself non-dumpable, so that the
other processes of the pod can neither ptrace it nor open its /proc
entries without CAP_SYS_PTRACE. It runs with one P, no garbage
collector and parks in pause(2).
It is built static like the shims and installed with them.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
A sandbox shim may exit as soon as it has shut its sandbox down, before
containerd reads the ShutdownSandbox response, in which case the client
sees ttrpc.ErrClosed. The task Shutdown RPC has the same race and
shimTask.Shutdown treats a closed connection as a successful shutdown;
controllerLocal.Shutdown returned it as an error instead, failing
RemovePodSandbox for a sandbox that was already gone.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
critest.sh takes SANDBOXER, the sandbox controller of the runtime handler
(podsandbox by default; shim adds disable_pause_image_pull, since a
Sandbox API shim needs no pause image). The Linux integration job runs the
critest suite a second time on the runc.v2 legs, with TEST_RUNTIME_PATH
pointing at containerd-shim-sandboxed-runc-v2 and the shim sandboxer. The
one spec that needs a shared pod PID namespace is skipped by name until
that lands; the user namespace specs skip themselves, since the handler
does not advertise user namespaces.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
An experimental runc shim that implements the Sandbox API itself, so
that a pod needs no pause container and no pause image. It serves the
io.containerd.runc.v2 runtime type from a separate binary; a runtime
handler selects it with runtime_path and pairs it with the shim
sandboxer:
[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
runtime_path = "/usr/local/bin/containerd-shim-sandboxed-runc-v2"
sandboxer = "shim"
disable_pause_image_pull = true
The shim is the sandbox. On CreateSandbox a locked thread enters the
network namespace CRI created, unshares the pod IPC and UTS namespaces,
sets the hostname and the pod sysctls from inside them, bind-mounts its
namespace files under the bundle and moves back out; the pins keep the
namespaces alive the way CRI keeps the network namespace alive. The shim
writes the pod shared files and mounts the shm tmpfs under its bundle and
returns them as mounts of the sandbox spec, together with the namespace
paths. StartSandbox reports pid 0. The containers of the pod run through
the task service of containerd-shim-runc-v2, wrapped only so that the
task Shutdown RPC, which containerd sends after every task deletion, does
not end the shim while the pod is alive. StopSandbox and "shim delete"
detach the pins and the shm and remove the files without relying on any
state; ShutdownSandbox ends the shim. The runtime info is the runc shim's
minus the user namespace, which the shim does not support yet.
Not supported yet: a shared pod PID namespace (the containers of such a
pod get their own PID namespaces, with a warning), user namespaces,
user.* sysctls, SELinux labels, cgroup v1 and rootless containerd. The
shim decodes the CRI PodSandboxConfig it receives as options; a
containerd owned configuration message is left as a TODO. The binary is
built and installed with the other Linux commands.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
On a terminal process, a single PTY master file descriptor carries both
stdin and stdout/stderr. The stdin FIFO can reach EOF while the process
is still running, either when a client calls CloseIO to close stdin or
when an empty client stdin stream closes before openStdin's background
writer opens in the kernel. Shutting down and closing the PTY master
when the stdin copy goroutine returns hangs up the terminal, which
sends SIGHUP to the process, causes writes to the slave to fail with
EIO, and causes stdout reads from the master to fail with EBADF.
Close only the stdin FIFO reader when the stdin copy goroutine finishes.
Closing stdin via CloseIO on a terminal process leaves the PTY master
open and the process running without sending SIGHUP.
Assisted-by: Antigravity
Signed-off-by: Samuel Karp <samuelkarp@google.com>
A sandbox controller that reports pid 0 has no process whose
/proc/<pid>/ns entries a container could join and no pause container
whose shared files CRI created. Such a sandbox describes what it holds in
the spec it returns on start.
When the sandbox status reports pid 0, load that spec from the sandbox
store and use it in two places. The pod namespaces come from its
Linux.Namespaces: a namespace listed there is joined by path, one that is
missing is the host namespace when the pod asked for it and an error
otherwise, and a PID namespace entry without a path declares that each
container gets its own. The pod shared files (/etc/hostname, /etc/hosts,
/etc/resolv.conf, /dev/shm) are bind-mounted from the spec mount sources
when nothing exists at the fixed CRI paths, which keep precedence.
Sandboxes with a pid keep the pid derived construction unchanged, and so
do Sandbox API shims that report pid 0 without a Linux section in their
spec, so existing shims see no change.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
WithPodNamespaces derives the namespaces a container joins from the pid
of the sandbox process. A sandbox without a process has to name the
namespaces it holds instead.
WithPodNamespacePaths takes the paths explicitly. The network, IPC and
UTS namespaces of a container are always those of its pod, whatever the
container config says, exactly as they were with pause; the pod's
namespace options decide which of them are host namespaces. Every
namespace the pod shares must have a path: a missing one is an error,
never a fallback to a fresh or host namespace, unless the pod asked for
the host namespace, in which case the entry is removed from the spec. A
sandbox may declare that it holds no PID namespace and that each
container gets its own. WithPodNamespaces delegates to it with the pid
derived paths, so pause based pods produce the same specs as before.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
The files a pod shares with its containers (hostname, hosts, resolv.conf,
the shm tmpfs) are produced by the podsandbox controller from the pod
config. A sandbox implementation outside the CRI server, such as a
Sandbox API shim, has to produce the same files. Put the pure helpers
(file and mount path constants, hostname and resolv.conf content, the shm
mount data) in a leaf package with no CRI server dependencies so that
both can use them. podsandbox is left unchanged for now.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
controllerLocal.Metrics returned the shim's gRPC status error as is,
unlike every other method of the controller. CRI's pod stats listing
tolerates NotFound and Unavailable from a controller but never matched a
status error, so one sandbox whose shim could not report metrics failed
ListPodSandboxStats for every pod on the node.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
A runtime handler can name its shim binary with runtime_path. The task
service replaces the runtime name with that path and the podsandbox
controller passes it to the pause task, but the shim sandboxer never saw
it: CRI stored only the runtime type in the sandbox record and the shim
controller started the binary resolved from that type.
Add a RuntimePath create option to the sandbox controller, have the shim
controller start that binary when it is set (the shim manager already
accepts an absolute path in place of a runtime name), and pass it from
CRI. The gRPC proxy cannot carry it, since ControllerCreateRequest has no
field for it, so a remote sandboxer logs a warning instead.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
On Windows (NTFS/ReFS), extending a file with Truncate does not create
a sparse file by default. When mkfs.ext4 subsequently writes metadata
at scattered offsets across the file, NTFS must zero-fill all
intervening blocks up to the valid data length before each write. For
large filesystem images this is extremely slow.
Fix this by marking the file as sparse via FSCTL_SET_SPARSE before
calling Truncate. This allows NTFS to leave unwritten regions
unallocated, matching the behavior of macOS/Linux where Truncate
already produces sparse files. Failing to set the attribute is not
fatal: it is logged at debug level and mkfs carries on as before. On
other platforms setSparseFile is a no-op.
Benchmark results (8 GB file, Windows 11, NTFS):
Without FSCTL_SET_SPARSE (before):
Create + Truncate: 535µs (on-disk: 8192 MB)
Scattered writes: 4.74s (on-disk: 8192 MB)
Total: 4.74s
With FSCTL_SET_SPARSE (after):
Create + Truncate: 526µs (on-disk: 0 MB)
Scattered writes: 15.6ms (on-disk: 1 MB)
Total: 16.1ms
~300x speedup for the file creation + write pattern, and on-disk usage
drops from 8 GB to only the blocks actually written by mkfs.
Signed-off-by: David Scott <dave@recoil.org>
Assisted-by: Claude Opus 4.6 (1M context)
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Assisted-by: OpenCode (anthropic/claude-sonnet-5-5)
Creating symlinks on Windows requires administrator privileges or
Developer Mode, so NewBundle fails with "A required privilege is not
held by the client" when containerd creates a task without them.
Replace the direct os.Symlink and os.Readlink calls with platform
specific helpers. On non-Windows platforms they keep the existing
symlink behaviour. On Windows, createWorkLink creates a directory
junction using go-winio's reparse point encoding, which does not need
elevated privileges. os.Readlink resolves both symlinks and junctions,
so resolveWorkLink uses it on both platforms.
createWorkLink closes the directory handle before removing the
half-created junction directory on failure. The handle is opened
without sharing, so removing the directory while it is still open would
fail.
Signed-off-by: Craig Gumbley <craiggumbley@gmail.com>
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Assisted-by: OpenCode (anthropic/claude-sonnet-5-5)
The previous wording was confusing, at least to Claude Opus 5.5,
which said:
At least one maintained branch will still support
it [cgroup v1] after that [May 2029]
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Install the pinned release of containerd-runc-shim with cargo install
instead of building the tip of rust-extensions' main, so the Rust legs
run a known shim that does not change underneath the PR. The build no
longer runs inside rust-extensions, whose rust-toolchain.toml used to
pick the toolchain, so pin one here instead.
Building the shim takes a minute or two per Rust leg, and its inputs
are fixed by the pinned crate version, toolchain and runner image, so
keep the installed binary in the actions cache keyed on those and only
run cargo install, and install the toolchain, when it is missing.
CRI only types runtime options for io.containerd.runc.v2, so the Rust
shim got generic options and SystemdCgroup never reached it. Configure
it as the runc runtime's binary through runtime_path instead.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
CRI owns the pod SELinux label, so a label returned by the sandbox
controller is ignored. Log a warning when it differs from the one CRI
allocated.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
The Rust shim now sets up a PTY's stdin without blocking, so
TestContainerPTY runs again. The CRI suite had no way to leave tests
out; it takes a SKIP pattern now, like FOCUS, and the Rust legs use it
for the user namespace tests, which need idmapped mounts the shim does
not do yet. The checkpoint disable script is skipped for the Rust shim
as well: it has no Checkpoint RPC, so the script's second case, a
checkpoint that must succeed with enable_criu unset, cannot pass with
it. Its integration legs already run with -no-criu.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
This reverts commit f078cebbd1.
Propagating State deadline errors causes Get to fail and List to omit
tasks when a shim is temporarily unresponsive, such as during slow
cleanup in Delete. A timeout does not mean the task no longer exists,
and omitting it makes container lifecycle management harder for clients
and CRI.
Restore the previous behavior of returning UNKNOWN on State timeouts,
so tasks remain visible and callers can reconcile their state.
Refs: https://github.com/containerd/containerd/pull/14380#pullrequestreview-5689176711
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Maksym Pavlenko