Adds a new --scrub-logs flag to ctr run on Windows. This flag enables
the ScrubLogs option for the io.containerd.runhcs.v1 shim.
Assisted-by: gemini-cli
Signed-off-by: Samuel Karp <samuelkarp@google.com>
(cherry picked from commit 18a01c0020)
Since Go 1.21, os.MkdirTemp/os.TempDir resolve the temp directory via
Windows' GetTempPath2W. For processes running as SYSTEM (as containerd
does when running under the SCM), that API reads the temp location from
the SystemTemp environment variable rather than TMP/TEMP. As a result,
the existing TMP/TEMP overrides no longer steer the layer-extraction
tempdir for the containerd service, so it falls back to the default
C:\\Windows\\SystemTemp and unpacks on the SystemDrive, reintroducing the
cross-volume copy the 'temp' config option was meant to avoid.
Set SystemTemp to config.TempDir alongside TEMP/TMP so the override keeps
working on Go 1.21+.
Factor the env-var setting out of CreateTopLevelDirectories into a small
setTempDirEnv helper and add a focused unit test (TestSetTempDirEnv) that
verifies the expected variables are set: TEMP/TMP/SystemTemp on Windows,
TMPDIR on other platforms.
Ref: https://cs.opensource.google/go/go/+/refs/tags/go1.21.0:src/os/file_windows.go
Signed-off-by: Maksim An <maksiman@microsoft.com>
The task service guards its containers map with s.mu, and getContainer()
takes it on behalf of effectively every task RPC (State, Connect, Stats,
Wait, Pause, Kill, ...). Create() held s.mu for its whole duration,
including runc.NewContainer(), which runs the actual `runc create`.
`runc create` can be slow on a loaded host. While it runs, any concurrent
task RPC blocks on s.mu. The tasks service applies a 2s timeout to State
(io.containerd.timeout.task.state), so a concurrent State waits on s.mu,
exceeds the deadline, and the ttrpc call is abandoned -- the late shim
reply then shows up as:
ttrpc: received message on inactive stream stream=3
Since deadline errors are now surfaced to clients, this is treated as a
fatal failure and the just-created container is torn down right after
start (observed on Lima/vz: nginx -> Exited (1)).
Move runc.NewContainer() out of the s.mu critical section, mirroring the
runtime v1 shim lock optimization. s.mu is taken only once the container
exists, to guard the map and the remaining (fast) setup, so a slow create
no longer blocks concurrent State and other lookups.
preStart/handleStarted/cleanup only use s.lifecycleMu, so early-exit
handling is unchanged.
See lima-vm/lima#5030.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
When migrating v3 [grpc] config to the v4 server plugin config, only
emit fields that were actually set in the legacy config. This prevents
a sparse imported [grpc] section, such as one only setting
max_send_message_size, from generating address = "" and overwriting the
address migrated from the root config.
Add table-driven coverage for sparse legacy grpc imports, including
address preservation, imported address override, and default address
retention when no imported config specifies address.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Switch the CRI integration layer from containerd's forked Kubernetes helpers
and clients to the upstream Kubernetes modules, and finalize the dependency
update to Kubernetes v0.36.0.
Replace the remaining internal helper copies with upstream packages:
- internal/cri/clock -> k8s.io/utils/clock
- internal/cri/executil -> upstream CRI exec helpers
- internal/cri/resourcequantity -> k8s.io/apimachinery/pkg/api/resource
- internal/cri/setutils -> k8s.io/apimachinery/pkg/util/sets
- internal/cri/types/labels.go -> internal/cri/labels
- integration/cri-api/pkg/apis/services.go -> k8s.io/cri-api/pkg/apis/services.go
Adopt the upstream CRI clients directly:
- add k8s.io/cri-client v0.36.0, k8s.io/cri-streaming v0.36.0, and
k8s.io/streaming v0.36.0 as direct dependencies
- promote k8s.io/utils to a direct dependency and pull in
k8s.io/component-base v0.36.0 indirectly
- keep integration/remote as a thin containerd adapter around cri-client,
because the integration tests still need the stream-shaped
GetContainerEvents RPC
Finalize the Kubernetes dependency update from v0.36.0-rc.0 to v0.36.0,
refresh vendor/, and drop the obsolete internal utility copies.
Also fix the protobuf MessageState mutex-copy vet failures exposed by the new
APIs and close the temporary integration CRI clients explicitly.
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
The new server plugin type allows services which have listeners to be
configured as plugins rather than defined directly in the global
configuration. This provides more configuration consistency and allows
containerd to be extended for new types of api handlers.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The default state directory does not need to be created and only the
configured state directory must be setup.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Send the socket directory from containerd to the shim. The shim still
decides where the socket goes but can use the environment variable
passed from containerd to ensure the socket is placed in the configured
directory with proper permission.
This is needed for some rootless cases which do not have permission to
the default state directory as currently set. The directory being
hardcoded by the shim means it is currently not possible to change the
location the shim will listen at.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Mark converted EROFS manifests with the erofs OS feature and cover
feature-aware manifest selection and unpack routing for erofs images.
Signed-off-by: ChengyuZhu6 <hudson@cyzhu.com>
Avoid using logrus concepts in the API, use slog style log levels with
integer values and 0 meaning the default "info" level.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Revert changes related to reading annotations in runc v2 shim,
because this requires a special path for runc shim. This complicates
migration to the new bootstrap API.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
When cancel() sends SIGTERM to a short-lived logging binary that is
already mid-exit, cmd.Wait() may return 'signal: terminated'. This is
expected behavior since we sent the SIGTERM ourselves, so treat it as
a normal exit.
Fixes flaky TestNewBinaryIOBinaryV2Ready.
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
Introduce functional options to NewLogrusHook to allow optional Trace ID
injection into log fields. This enables log-trace correlation via the
[debug] config without breaking existing external consumers of
pkg/tracing.
Signed-off-by: Hasan Siddiqui <hasiddiqui@google.com>
While incoming gRPC requests are already being traced via the
server-side handler, outgoing RPCs to proxy plugins are missing the
client-side equivalent. Adding otelgrpc.NewClientHandler() ensures trace
context is successfully propagated to the plugins.
Signed-off-by: Hasan Siddiqui <hasiddiqui@google.com>
Correct the typo in the error string (pi -> pid) and use %w for error wrapping instead of the invalid %wd verb when reading the exec PID file.
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
The runc v2 shim's event forwarding has a race condition between
send() and the shutdown callback that closes the events channel.
When a Delete or processExit RPC runs concurrently with shim
shutdown, the following panic occurs:
panic: send on closed channel
goroutine 153 [running]:
github.com/containerd/containerd/v2/cmd/containerd-shim-runc-v2/task.(*service).send(...)
/containerd/cmd/containerd-shim-runc-v2/task/service.go:694
github.com/containerd/containerd/v2/cmd/containerd-shim-runc-v2/task.(*service).Delete(0xc000228120, ...)
/containerd/cmd/containerd-shim-runc-v2/task/service.go:366 +0x229
The root cause is that send() performs a bare channel send on
s.events with no synchronization:
func (s *service) send(evt any) {
s.events <- evt
}
Meanwhile, a shutdown callback registered in NewTaskService closes
the same channel:
sd.RegisterCallback(func(context.Context) error {
close(s.events)
return nil
})
Since shutdownService.Shutdown() runs all callbacks concurrently via
errgroup, close(s.events) can execute while a Delete, Start, Create,
Exec, Pause, Resume, or handleProcessExit goroutine is in send().
This was observed in CI on a Kind cluster (containerd v2.1.1) under
heavy pod churn: the shim panic crashed containerd on the worker
node, breaking the kubectl attach stream to the test pod and causing
the CI job to fail.
Fix this by removing the close(s.events) shutdown callback entirely
and instead using select with s.shutdown.Done() in both send() and
forward(), which is the established pattern for shutdown-safe channel
operations throughout the containerd codebase (see
client/events.go, core/events/exchange/exchange.go). The events
channel is never closed; it is garbage collected when the shim
process exits after shutdown.
In forward(), after s.shutdown.Done() fires, remaining buffered
events are drained with a non-blocking receive loop before
returning, preserving the previous behavior where range over the
closed channel would drain buffered items.
Alternative designs considered:
- recover() in send(): while compact, this pattern is not used
anywhere in the containerd codebase for closed-channel protection,
and silently swallowing panics makes the code harder to reason
about.
- RWMutex guarding the channel send (RLock) and close (WLock): this
would work correctly and allows concurrent sends, but no existing
containerd code uses a mutex to guard channel send+close. It also
adds lock contention on every event send during normal operation.
- select with a dedicated "closing" channel closed before
s.events: this does not work in Go because select panics on a
send to a closed channel regardless of whether other cases are
ready (verified empirically).
Signed-off-by: delthas <delthas@dille.cc>
The 'content fetch' command description has been unchanged since it was
first added over 8 years ago. It described the feature as "experimental"
with "few leaps to make this work", which no longer reflects reality.
Update the help text to accurately describe what the command does: it
pulls image content into the content store without unpacking, and can
be used to pre-fetch images.
Fixes#11754
Signed-off-by: Crawford Ma <crawfordxx@users.noreply.github.com>
Perform the plugin migrations on load to allow stepping through plugin
migration versions to happen alongside migration of the global
configuration object. When the configuration migrations happen
separately, the version in the config can get increasd on load and cause
plugin migration not to occur. This does not cause issues today because
global config migrations only occur for version 0 and 1, which was
before plugin config migration was introduced. Any new version which
does migrations either cannot get called on load or will break plugin
migration later.
This change simplifies configuration load and migration, preventing the
need to migrate the configurations on load and again later when plugins
are loaded. This also allows includes to work at different versions,
which may currently break or cause inconsistent results.
Signed-off-by: Derek McGowan <derek@mcg.dev>
