runc and the OCI hooks it spawns are separate processes, so the trace
context can not reach them over ttrpc. Inject it into their environment
instead, riding on the request context via go-runc's WithExtraEnv, which
does not race with concurrent requests the way setting it on the shim
process would.
The otelttrpc interceptor extracts that context, so chain it first
explicitly instead of leaving it to plugin initialization order. Also
flush the spans the tracing plugin buffered before the shim exits.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Wait for stdout and stderr copying to complete after validating process exit so deferred process deletion does not cancel readers with output still in flight.
Signed-off-by: Chris Ayoub <cayoub@openai.com>
The layer content cache was a single directory, so every source of
pre-converted blobs had to be merged into one tree. A shared read-only fleet
cache, a host-local cache, and a per-image warm cache could not coexist.
Replace layer_content_cache with layer_content_caches, a list. Each directory
is checked in order and the first hit is staged into the snapshot; a layer
found in none of them falls back to the normal download-and-convert path.
Cache directories are no longer required to exist at startup. A missing one is
indistinguishable from an empty one at lookup time (both are simply a miss),
and it may well be mounted after the daemon starts, so the only check left is
that each path is absolute -- a relative one would be symlinked into the
snapshot dir and dangle.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
ctr resolves named users and groups for run and task exec in the
client process. Since EROFS fsview support moved behind registration,
the ctr application does not link the handler and falls back to a
temporary host mount.
Register the handler with the ctr application so EROFS snapshots can
be inspected directly. Non-EROFS mounts continue through the existing
handlers unchanged.
Signed-off-by: Chris Ayoub <cayoub@openai.com>
Adds a new --scrub-logs flag to ctr run on Windows. This flag enables
the ScrubLogs option for the io.containerd.runhcs.v1 shim.
Assisted-by: gemini-cli
Signed-off-by: Samuel Karp <samuelkarp@google.com>
Add a --dmverity flag to `ctr images build-erofs-cache`. When set, each cached
erofs blob is dm-verity formatted (the hash tree is appended in place) and a
.dmverity sidecar is written alongside it. This is required when the erofs
snapshotter runs with dmverity_mode=on, which rejects cache hits that lack a
sidecar; without it such layers would have to be formatted out-of-band.
Extract the differ's dm-verity formatting into a shared dmverity.FormatLayer so
the differ and the cache builder share one implementation; the differ's
formatDmverityLayer now delegates to it.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Add `ctr images build-erofs-cache <image> <cache_dir>`, which reads an
already-pulled image's layers from the content store, converts each into a
directly-mountable erofs blob, and writes them under
<cache_dir>/<algorithm>/<hex>.erofs keyed by the layer's diffID. This
populates the directory the erofs snapshotter's layer_content_cache reads
on pull; because the key is the source diffID, layers shared across images
converge on one blob. No converted image is produced.
Extract the per-layer uncompress + mkfs.erofs step out of LayerConvertFunc
into an exported ConvertLayerToErofs so the image converter and the cache
builder share a single conversion path.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Since Go 1.21, os.MkdirTemp/os.TempDir resolve the temp directory via
Windows' GetTempPath2W. For processes running as SYSTEM (as containerd
does when running under the SCM), that API reads the temp location from
the SystemTemp environment variable rather than TMP/TEMP. As a result,
the existing TMP/TEMP overrides no longer steer the layer-extraction
tempdir for the containerd service, so it falls back to the default
C:\\Windows\\SystemTemp and unpacks on the SystemDrive, reintroducing the
cross-volume copy the 'temp' config option was meant to avoid.
Set SystemTemp to config.TempDir alongside TEMP/TMP so the override keeps
working on Go 1.21+.
Factor the env-var setting out of CreateTopLevelDirectories into a small
setTempDirEnv helper and add a focused unit test (TestSetTempDirEnv) that
verifies the expected variables are set: TEMP/TMP/SystemTemp on Windows,
TMPDIR on other platforms.
Ref: https://cs.opensource.google/go/go/+/refs/tags/go1.21.0:src/os/file_windows.go
Signed-off-by: Maksim An <maksiman@microsoft.com>
The task service guards its containers map with s.mu, and getContainer()
takes it on behalf of effectively every task RPC (State, Connect, Stats,
Wait, Pause, Kill, ...). Create() held s.mu for its whole duration,
including runc.NewContainer(), which runs the actual `runc create`.
`runc create` can be slow on a loaded host. While it runs, any concurrent
task RPC blocks on s.mu. The tasks service applies a 2s timeout to State
(io.containerd.timeout.task.state), so a concurrent State waits on s.mu,
exceeds the deadline, and the ttrpc call is abandoned -- the late shim
reply then shows up as:
ttrpc: received message on inactive stream stream=3
Since deadline errors are now surfaced to clients, this is treated as a
fatal failure and the just-created container is torn down right after
start (observed on Lima/vz: nginx -> Exited (1)).
Move runc.NewContainer() out of the s.mu critical section, mirroring the
runtime v1 shim lock optimization. s.mu is taken only once the container
exists, to guard the map and the remaining (fast) setup, so a slow create
no longer blocks concurrent State and other lookups.
preStart/handleStarted/cleanup only use s.lifecycleMu, so early-exit
handling is unchanged.
See lima-vm/lima#5030.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
When migrating v3 [grpc] config to the v4 server plugin config, only
emit fields that were actually set in the legacy config. This prevents
a sparse imported [grpc] section, such as one only setting
max_send_message_size, from generating address = "" and overwriting the
address migrated from the root config.
Add table-driven coverage for sparse legacy grpc imports, including
address preservation, imported address override, and default address
retention when no imported config specifies address.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Switch the CRI integration layer from containerd's forked Kubernetes helpers
and clients to the upstream Kubernetes modules, and finalize the dependency
update to Kubernetes v0.36.0.
Replace the remaining internal helper copies with upstream packages:
- internal/cri/clock -> k8s.io/utils/clock
- internal/cri/executil -> upstream CRI exec helpers
- internal/cri/resourcequantity -> k8s.io/apimachinery/pkg/api/resource
- internal/cri/setutils -> k8s.io/apimachinery/pkg/util/sets
- internal/cri/types/labels.go -> internal/cri/labels
- integration/cri-api/pkg/apis/services.go -> k8s.io/cri-api/pkg/apis/services.go
Adopt the upstream CRI clients directly:
- add k8s.io/cri-client v0.36.0, k8s.io/cri-streaming v0.36.0, and
k8s.io/streaming v0.36.0 as direct dependencies
- promote k8s.io/utils to a direct dependency and pull in
k8s.io/component-base v0.36.0 indirectly
- keep integration/remote as a thin containerd adapter around cri-client,
because the integration tests still need the stream-shaped
GetContainerEvents RPC
Finalize the Kubernetes dependency update from v0.36.0-rc.0 to v0.36.0,
refresh vendor/, and drop the obsolete internal utility copies.
Also fix the protobuf MessageState mutex-copy vet failures exposed by the new
APIs and close the temporary integration CRI clients explicitly.
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
The new server plugin type allows services which have listeners to be
configured as plugins rather than defined directly in the global
configuration. This provides more configuration consistency and allows
containerd to be extended for new types of api handlers.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The default state directory does not need to be created and only the
configured state directory must be setup.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Send the socket directory from containerd to the shim. The shim still
decides where the socket goes but can use the environment variable
passed from containerd to ensure the socket is placed in the configured
directory with proper permission.
This is needed for some rootless cases which do not have permission to
the default state directory as currently set. The directory being
hardcoded by the shim means it is currently not possible to change the
location the shim will listen at.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Mark converted EROFS manifests with the erofs OS feature and cover
feature-aware manifest selection and unpack routing for erofs images.
Signed-off-by: ChengyuZhu6 <hudson@cyzhu.com>
Avoid using logrus concepts in the API, use slog style log levels with
integer values and 0 meaning the default "info" level.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Revert changes related to reading annotations in runc v2 shim,
because this requires a special path for runc shim. This complicates
migration to the new bootstrap API.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
When cancel() sends SIGTERM to a short-lived logging binary that is
already mid-exit, cmd.Wait() may return 'signal: terminated'. This is
expected behavior since we sent the SIGTERM ourselves, so treat it as
a normal exit.
Fixes flaky TestNewBinaryIOBinaryV2Ready.
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
Introduce functional options to NewLogrusHook to allow optional Trace ID
injection into log fields. This enables log-trace correlation via the
[debug] config without breaking existing external consumers of
pkg/tracing.
Signed-off-by: Hasan Siddiqui <hasiddiqui@google.com>
While incoming gRPC requests are already being traced via the
server-side handler, outgoing RPCs to proxy plugins are missing the
client-side equivalent. Adding otelgrpc.NewClientHandler() ensures trace
context is successfully propagated to the plugins.
Signed-off-by: Hasan Siddiqui <hasiddiqui@google.com>
Correct the typo in the error string (pi -> pid) and use %w for error wrapping instead of the invalid %wd verb when reading the exec PID file.
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Maksym Pavlenko