mirror of
https://github.com/containerd/containerd.git
synced 2026-08-09 17:39:22 +00:00
842cb99a5e
After runc.Create(init), the container’s cgroup contains a paused init process. Therefore, we should start monitoring OOM events immediately after creation, in case the process goes OOM very quickly. Otherwise, we may encounter flaky cases such as: ```bash Link https://github.com/containerd/containerd/actions/runs/18787603490/job/53609925150?pr=12399 [k8s.io] Container OOM runtime should output OOMKilled reason [It] should terminate with exitCode 137 and reason OOMKilled default: sigs.k8s.io/cri-tools/pkg/validate/container_linux.go:147 default: default: Timeline >> default: STEP: create Privileged podSandbox @ 10/24/25 18:11:50.889 default: STEP: create container @ 10/24/25 18:11:51.691 default: STEP: create a container that will be killed by OOMKiller @ 10/24/25 18:11:51.691 default: STEP: Get image status for image: registry.k8s.io/e2e-test-images/busybox:1.29-2 @ 10/24/25 18:11:51.691 default: STEP: Create container. @ 10/24/25 18:11:51.691 default: Oct 24 18:11:51.718: INFO: Created container "384bb1538123650106d43dd1fb86fa2c13ae32fb1594b0e70d50cc47eddccbe8" default: default: STEP: verifying container status @ 10/24/25 18:11:51.718 default: STEP: start container @ 10/24/25 18:11:51.72 default: STEP: Start container for containerID: 384bb1538123650106d43dd1fb86fa2c13ae32fb1594b0e70d50cc47eddccbe8 @ 10/24/25 18:11:51.72 default: Oct 24 18:11:52.007: INFO: Started container "384bb1538123650106d43dd1fb86fa2c13ae32fb1594b0e70d50cc47eddccbe8" default: default: STEP: container is stopped because of OOM @ 10/24/25 18:11:52.007 default: STEP: Get container status for containerID: 384bb1538123650106d43dd1fb86fa2c13ae32fb1594b0e70d50cc47eddccbe8 @ 10/24/25 18:11:52.007 default: STEP: Get container status for containerID: 384bb1538123650106d43dd1fb86fa2c13ae32fb1594b0e70d50cc47eddccbe8 @ 10/24/25 18:11:56.018 default: STEP: Get container status for containerID: 384bb1538123650106d43dd1fb86fa2c13ae32fb1594b0e70d50cc47eddccbe8 @ 10/24/25 18:11:56.022 default: STEP: exit code is 137 @ 10/24/25 18:11:56.023 default: STEP: reason is OOMKilled @ 10/24/25 18:11:56.023 default: [FAILED] in [It] - sigs.k8s.io/cri-tools/pkg/validate/container_linux.go:165 @ 10/24/25 18:11:56.024 default: STEP: stop PodSandbox @ 10/24/25 18:11:56.024 default: STEP: delete PodSandbox @ 10/24/25 18:11:56.932 default: << Timeline default: default: [FAILED] Expected default: <string>: Error default: to equal default: <string>: OOMKilled default: In [It] at: sigs.k8s.io/cri-tools/pkg/validate/container_linux.go:165 @ 10/24/25 18:11:56.024 ``` NOTE: We have an EventMonitor goroutine that monitors all containers’ OOMKilled events, and a dedicated goroutine that monitors container exit events. There is a race condition where the exit-event goroutine may clean up the task and update its status faster than the OOM event updater. I don’t have a better idea to fully resolve this race condition, but this patch aims to minimize the chance of missing OOM events. Signed-off-by: Wei Fu <fuweid89@gmail.com>