Pass more complete container status information to NRI, including
exit code, and timestamps for container creation, start, and exit
events.
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
This PR adds opt-in tracing spans/attributes in CRI image pull and selected sandbox-related paths to improve debugging and correlation (e.g., sandbox.id/pod metadata). If maintainers prefer a smaller diff, I’m happy to split this into a pull-only PR plus follow-ups.
• follow-up after pull-only PR
• focuses on task/metadata/sandbox/cni setup spans
Signed-off-by: Cindy Li <cindyli@pinterest.com>
Introduces changes to make pod sandbox updates persistent across
restarts.
This is achieved by:
- Storing the updated Overhead and Resources as an extension on the core
sandbox object and in the in-memory sandbox status store.
- Modifying the sandbox recovery logic to read this extension on startup
(this is not working in recovery unit tests yet and needs fixing).
- Updating the PodSandboxStatus CRI handler to include updated resources
from the sandbox status store.
Signed-off-by: Chris Henzie <chrishenzie@google.com>
The runtimeHandler parameter was added to PullImage() but never used.
Instead, the code relied on an experimental annotation
(io.containerd.cri.runtime-handler) passed in the pod sandbox config.
This annotation was a workaround because CRI's PullImageRequest didn't
include the runtime handler. However, since cri-api v0.29.0, the runtime
handler is available in the API and passed as a parameter to PullImage().
For backward compatibility with CRI clients that don't yet pass the
runtime handler parameter, we fall back to the annotation if the
parameter is empty. The annotation-based fallback is deprecated and
will be removed in containerd 2.5.
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
WithCDI currently emits logs at Info level for every container even when
len(Config.CDIDevices) == 0. Move these to Debug level.
Signed-off-by: Samuel Karp <samuelkarp@google.com>
If a container has CDI devices requested by the dedicated CRI protocol
field, treat this as an error if CDI support is explicitly disabled by
configuration, instead of silently ignoring the requested devices.
Additionally, if CDI support is disabled log a warning about the future
deprecation of the EnableCDI configuration option.
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
Deprecate the enable_cdi config option for CRI. Change it to a
pointer so we can differentiate between an unset value and one
that is set explicitly to false. Treat an omitted setting (nil
value) as enabled. Mark it deprecated in RELEASES.md. Add a
deprecation warning for it. Log that warning if we start up
with enable_cdi explicitly set to false.
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
We have existing detection for concurrent CreateContainer requests, but
the error message is unclear and there is no warning in containerd logs.
This change adds a warning and clarifies the error message.
Signed-off-by: Samuel Karp <samuelkarp@google.com>
Remove unnecessary variable extraction and Interfaces field,
keeping only the Timestamp addition as originally intended.
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
adds a background stats collector that calculates `UsageNanoCores` for containers and pod sandboxes.
- run in the background every second to collect CPU metrics for all containers and sandboxes (similar to what cAdvisor does)
- keep a rolling buffer of CPU samples and calculates the instantaneous CPU usage rate from consecutive samples
- read pod-level CPU stats from the parent cgroup rather than the pause container
- add cgroupv2 Pressure Stall Information for CPU, memory, and IO
- add missing `Timestamp` and `Interfaces` fields
when Kubernetes runs with `PodAndContainerStatsFromCRI=true`, it expects `UsageNanoCores` to be set in stats responses.
This value represents how much CPU is being used right now (as opposed to `UsageCoreNanoSeconds` which is cumulative).
To calculate it, we need to compare CPU samples over time to replicate what is in cadvisor.
we can't yet really test this in CI as some changes in kubernetes has to land for `--feature-gates=PodAndContainerStatsFromCRI=true`
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
The original implementation provided a lot of unfilled or wrong
filled metrics. This tries to do better by only setting things I
am fairly certain are correct.
Signed-off-by: Tim Windelschmidt <tim@monogon.tech>
Co-authored-by: Mike Brown <brownwm@us.ibm.com>
Add nil checks for Memory.Limit, Memory.Reservation, and Memory.Swap
before dereferencing them in extractContainerSpecMetrics. These fields
are optional pointers in OCI runtime spec and can be nil when not set.
- Memory.Limit: Set from Pod resources.limits.memory
- Memory.Reservation: Only set via NRI plugins (rarely used)
- Memory.Swap: Set from memory_swap_limit_in_bytes (optional)
This prevents potential panic when collecting metrics from containers
that don't have all memory limits configured, and makes the code
consistent with CPU resource handling and helpers.go.
Signed-off-by: You Binhao <ymichaelson.0713@gmail.com>
The monitor goroutine (runs (*ContainerIO).Attach.func1) of Attach will
never finish if it attaches to a container without any stdout or stderr
output. Wait for http context cancel and break the pipe actively to
address the issue.
Signed-off-by: wheat2018 <1151937289@qq.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
