The gRPC sandbox controller service only forwarded the `options` field
when calling the local controller. The `netns_path`, `rootfs`, and
`annotations` fields were silently dropped, causing clients using the
gRPC proxy path to receive incomplete sandbox configurations.
Event topics were missing the leading `/` prefix ("sandboxes/create"
instead of "/sandboxes/create"), causing the event exchange to reject
the publish and return an error to the caller.
Add unit tests for the controller service that exercise all RPC
methods.
Signed-off-by: William Myers <willmyrs@amazon.com>
When a shim becomes unresponsive (e.g., stopped via SIGSTOP), ttrpc
communication times out with `context deadline exceeded`.
Currently, this error is not properly propagated, causing redundant API
calls and slow container listing by client sides.
Specifically, when executing the API to check the task state, it appears
that the `context deadline exceeded` error via ttrpc is not being handled
within `shimTask.State()` and `getProcessState()`.
As a result, when this error occurs, clients such as nerdctl cannot
recognize this error, and it is thought that the issue described below is
occurring:
- https://github.com/containerd/nerdctl/issues/4720
Therefore, this commit adds error handling to ensure timeouts are properly
handled by client sides.
Signed-off-by: Hayato Kiwata <dev@haytok.jp>
Use the erofs differ by default on darwin. This could be default for all
Unix platforms but limit the default changes to fix broken cases for backports.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The sandbox controller should only error out if it cannot find any
sandbox controllers. If it requires the pod sandbox controller to be
initialized, that creates an implicit dependency on all CRI plugins
being initialized. The sandbox controller API can be used without CRI
and therefore should not have this dependency.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Currently the error details are not included in the output error and
there is no log. One of the reasons a the transferer was skipped could
be do to a specific component which is not implemented (such as trying
to use erofs differ) or unsupported image (pulling schema1). This
information is useful to find a bad configuration.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Schema 1 (`application/vnd.docker.distribution.manifest.v1+prettyjws`) has been
officially deprecated since containerd v1.7 (PR 6884), and disabled since v2.0 (PR 9765).
Users who have been seeing warnings like `conversion from schema 1 images is deprecated`
now have to rebuild the image with Schema 2 or OCI.
Schema 2 was introduced in Docker 1.10 (Feb 2016), so most users should have been already
using Schema 2 or OCI.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This implements container restore as described in:
https://kubernetes.io/blog/2022/12/05/forensic-container-checkpointing-alpha/#restore-checkpointed-container-standalone
For detailed step by step instruction also see contrib/checkpoint/checkpoint-restore-cri-test.sh
The code changes are based on changes I have done in Podman around 2018
and CRI-O around 2020.
The history behind restoring container via CRI/Kubernetes probably
requires some explanation. The initial proposal to bring
checkpoint/restore to Kubernetes was looking at pod checkpoint and
restoring and the corresponding CRI changes.
https://github.com/kubernetes-sigs/cri-tools/pull/662https://github.com/kubernetes/kubernetes/pull/97194
After discussing this topic for about two years another approach was
implemented as described in KEP-2008:
https://github.com/kubernetes/enhancements/issues/2008
"Forensic Container Checkpointing" allowed us to separate checkpointing
from restoring. For the "Forensic Container Checkpointing" it is enough
to create a checkpoint of the container. Restoring is not necessary as
the analysis of the checkpoint archive can happen without restoring the
container.
While thinking about a way to restore a container it was by coincidence
that we started to look into restoring containers in Kubernetes via
Create and Start. The way it was done in CRI-O is to figure out during
Create if the container image is a checkpoint image and if that is true
we are using another code path. The same was implemented now with this
change in containerd.
With this change it is possible to restore the container from a
checkpoint tar archive that is created during checkpointing via CRI.
To restore a container via Kubernetes we convert the tar archive to an
OCI image as described in the kubernetes.io blog post from above. Using
this OCI image it is possible to restore a container in Kubernetes.
At this point I think it should be doable to restore containers in
CRI-O and containerd no matter if they have been created by containerd or
CRI-O. The biggest difference is the container metadata and that can
be adapted during restore.
Open items:
* It is not clear to me why restoring a container in containerd goes
through task/Create(). But as the restore code already exists this
change extended the existing code path to restore a container in
task/Create() to also restore a container through the CRI via
Create and Start.
* Automatic image pulling. containerd does not pull images
automatically if created via the CRI. There is an option in
crictl to pull images before starting, but that uses the CRI
image pull interface. It is still a separate pull and create
operation. Restoring containers from an OCI image is a bit
different. The checkpoint OCI image does not include the base
image, but just a reference to the image (NAME@DIGEST).
Using crictl with pulling will enable the pulling of the
checkpoint image, but not of the base image the checkpoint is
based on. So during preparation of the checkpoint containerd
will automatically pull the base image, but I was not able how
to pull an image blockingly in containerd. So there is a for
loop waiting for the container image to appear in the internal
store. I think this probably can be implemented better.
Anyway, this is a first step towards container restored in Kubernetes
when using containerd.
Signed-off-by: Adrian Reber <areber@redhat.com>
* container_update_resources.go: it is Windows and Linux that need special handling
* local*.go: all platforms use the same list of tasks
* temp_unix.go/temp_unsupported.go: Darwin is a Unix
* util_unix.go/util_unsupported.go: use generic unix tag
The only user-visible effect of these changes is that tempMountLocation is now properly handled on Darwin
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
Other similar events were already moved to the metadata store. The
metadata store has more information that can be used for a future
content created event.
Signed-off-by: Derek McGowan <derek@mcg.dev>
- combine consecutive "WithField" calls to "WithFields", as multiple
calls is known to be expensive.
- include a "snapshotter" field in logs to allow correlating actions
with specific snapshotters.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Uses the new github.com/containerd/errdefs/pkg module which is intended
to hold less stable utility functions separately from the stable
github.com/containerd/errdefs error types.
Includes temporary update to hcsshim until a release is cut there
Signed-off-by: Derek McGowan <derek@mcg.dev>
Core should not have a dependency on API types.
This was causing a transative dependency on grpc when importing the core
snapshots package.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The /var/lib/containerd/io.containerd.grpc.v1.introspection/uuid file
stores a UUID to identify the particular containerd daemon responding to
requests. The file should either exist with a UUID, or not exist.
However, it has been observed that the file can be truncated with 0
bytes, which will then fail to be parsed as a valid UUID.
As a defensive practice, detect a 0-length file and overwrite with a new
UUID rather than failing.
Fixes: https://github.com/containerd/containerd/issues/10491
Signed-off-by: Samuel Karp <samuelkarp@google.com>
Allow the api to stay at the same v1 go package name and keep using a
1.x version number. This indicates the API is still at 1.x and allows
sharing proto types with containerd 1.6 and 1.7 releases.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Split service proxy from service plugin.
Make introspection service easier for clients to use.
Update service proxy to support grpc and ttrpc.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Packages related to transfer and unpacking provide core interfaces which
use other core interfaces and part of common functionality.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The metadata store is in the best place to handle events directly after
the database has been updated. This prevents every user of the image
store interface from having to know whether or not they are responsible
for publishing events and avoid double events if the grpc local service
is used.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The new `PlunginInfo()` call can be used for instrospecting the details
of the runtime plugin.
```console
$ ctr plugins inspect-runtime --runtime=io.containerd.runc.v2 --runc-binary=runc
{
"Name": "io.containerd.runc.v2",
"Version": {
"Version": "v2.0.0-beta.0-XX-gXXXXXXXXX.m",
"Revision": "v2.0.0-beta.0-XX-gXXXXXXXXX.m"
},
"Options": {
"binary_name": "runc"
},
"Features": {
"ociVersionMin": "1.0.0",
"ociVersionMax": "1.1.0-rc.2",
...,
},
"Annotations": null
}
```
The shim binary has to support `-info` flag, see `runtime/v2/README.md`
Replaces PR 8509 (`api/services/task: add RuntimeInfo()`)
Co-authored-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
