Commit Graph

193 Commits

Author SHA1 Message Date
Phil Estes
98af40b752 Merge pull request #10722 from henry118/uidmap2
Support multiple uid/gid mappings [2/2]
2025-01-17 18:34:40 +00:00
Fu Wei
c6edc62db4 Merge pull request #10955 from mbaynton/userns-ovl-followup
Make ovl idmap mounts read-only
2025-01-14 21:14:47 +00:00
Jin Dong
fb44e37ff2 Remove confusing warning in cri runtime config migration
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-01-12 02:38:14 +00:00
Maksym Pavlenko
3871c2f265 Merge pull request #11165 from djdongjin/fix-cri-image-snapshotter-loading
Fix runtime platform loading in cri image plugin init
2025-01-10 20:37:22 +00:00
Kazuyoshi Kato
5ad6a150b6 Merge pull request #11189 from djdongjin/move-to-go-native-fuzz
Move fuzz tests to go native fuzz [part1]
2025-01-10 01:27:14 +00:00
Kazuyoshi Kato
f38df49348 Merge pull request #11227 from dmcgowan/unpack-snapshotter-exports
Add snapshotter exports to unpack platform
2025-01-09 00:23:46 +00:00
Derek McGowan
63f6047282 Add snapshotter exports to unpack platform
Allows the unpack operation to be optimized based on the specific
snapshotter configuration.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-07 16:04:44 -08:00
Jin Dong
2de1030299 Move fuzz tests to go native fuzz [part1]
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-12-24 22:48:20 +00:00
Austin Vazquez
bee64b2b93 Remove loop variable copies
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-12-23 23:14:49 -07:00
Mike Baynton
1e3d10dc29 Make ovl idmap mounts read-only
This is a planned follow-on from #10721 primarily at the request of
@fuweid, exchanging MNT_DETACH at unmount time for MOUNT_ATTR_RDONLY at
mount time. The effect is to increase risk of unmount failure due to
EBUSY (as observed in the wild) but add an additional protection that the
then-leaked bind mount does not act as a conduit for inadvertent modification
of the underlying data, including our own efforts to clean up the mountpoint.

Tests covering the lifecycle of the temporary idmap mounts and integrity
of the underlying lower layer data is also included in the normal and
failed-unmount case.

Fixes #10704

Signed-off-by: Mike Baynton <mike@mbaynton.com>
2024-12-21 16:02:33 -06:00
Jin Dong
ef0e709228 Fix runtime platform loading in cri image plugin init
The cri image service init has a bug where, after getting FSPath
for snapshotter_i, it stores it under defaultSnapshotter instead
of snapshotter_i.

Also make a few other refactor:

1. Dedup the snapshotRoot loading for defaultSnapshotter
2. Remove some unnecessary logic in RuntimePlatforms for-loop

Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-12-16 01:27:50 +00:00
Henry Wang
8a030d6537 Update overlay snapshotter to support multiple uid/gid mappings
Signed-off-by: Henry Wang <henwang@amazon.com>
2024-12-11 18:04:11 +00:00
Fu Wei
e21c1a1b2a Merge pull request #11061 from djdongjin/config-migrate-cri-grpc
Complete cri grpc plugin config migration
2024-12-11 15:56:11 +00:00
Jin Dong
8540fed774 complete cri grpc config migration
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-11-27 21:06:00 +00:00
Fu Wei
f6933cbf08 Merge pull request #11062 from dmcgowan/fix-diff-zstd
Update differ to handle zstd media types
2024-11-27 02:01:44 +00:00
Samuel Karp
5b9bd35b18 Merge pull request #11065 from darwin-containers/cleanup-conditional-builds
Unsorted platform conditionals cleanup
2024-11-26 23:35:13 +00:00
Derek McGowan
17f7858b4e Update differ to handle zstd media types
The differ should be able to generate zstd compressed layers when
provided with the zstd media type.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-11-26 14:29:21 -08:00
Marat Radchenko
e9d560f1e8 Unsorted platform conditionals cleanup
* container_update_resources.go: it is Windows and Linux that need special handling
* local*.go: all platforms use the same list of tasks
* temp_unix.go/temp_unsupported.go: Darwin is a Unix
* util_unix.go/util_unsupported.go: use generic unix tag

The only user-visible effect of these changes is that tempMountLocation is now properly handled on Darwin

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2024-11-26 20:05:33 +03:00
Derek McGowan
9e3ab2332b Move content events to metadata
Other similar events were already moved to the metadata store. The
metadata store has more information that can be used for a future
content created event.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-11-15 16:14:05 -08:00
Sebastiaan van Stijn
4594f5cac1 services/snapshots: include name of snapshotter in debug logs
- combine consecutive "WithField" calls to "WithFields", as multiple
  calls is known to be expensive.
- include a "snapshotter" field in logs to allow correlating actions
  with specific snapshotters.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-31 18:00:38 +01:00
Tonis Tiigi
3cc2343de0 local: avoid writing to content root on readonly store
A contentstore can be created on top of readonly path and
should not fail unless there is an attempt to write into it.

Currently this fails because new ingest directory is created
always, meaning for example that you can't create a store to
read blobs from OCI layout without it contaminating the OCI
layout files.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-10-24 18:03:09 -07:00
Derek McGowan
8c98e18a91 Merge pull request #10867 from namespacelabs/main
Lock store writer per digest and writer.
2024-10-24 13:55:07 +00:00
Niklas Gehlen
2535b187a6 Scope writer locks to each writer.
Signed-off-by: Niklas Gehlen <niklas@namespacelabs.com>
2024-10-23 20:40:08 +02:00
Derek McGowan
2f24aa00a5 Update errdefs to 0.3.0
Uses the new github.com/containerd/errdefs/pkg module which is intended
to hold less stable utility functions separately from the stable
github.com/containerd/errdefs error types.

Includes temporary update to hcsshim until a release is cut there

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-10-18 16:04:54 -07:00
Abel Feng
fc5086a74d cri: remove sandbox controller from client
cri will call sandbox controller from the sandboxService, remove the
dependency of client.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-10-16 17:37:07 +08:00
Abel Feng
e4df672ab8 sandbox: add sandbox controller v2
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-10-16 17:37:05 +08:00
Brian Goff
b85909cd4c shim: Move pprof server to plugin
Makes the pprof server a plugin and also gates by the `shim_tracing`
build tag (like otel is).
With this change, `net/http` is no longer a dependency in the shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:50 +00:00
Brian Goff
64d29ebe5b snapshots: core: Remove dependency on api types
Core should not have a dependency on API types.
This was causing a transative dependency on grpc when importing the core
snapshots package.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-02 01:46:19 +00:00
Phil Estes
192679b059 Merge pull request #10659 from djdongjin/remove-duplicated-sandbox-conversion
Remove duplicated sandbox <-> proto conversion functions
2024-09-05 14:57:17 +00:00
Jin Dong
3d7955bf78 remove duplicated descriptor from/to proto
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-31 12:01:33 -07:00
Jin Dong
93d6f0f921 remove duplicated sandbox to proto func
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-31 11:20:32 -07:00
Jin Dong
9c34005724 reduce ptypes.Empty creation by defining it in as a var
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-29 17:27:47 -07:00
Derek McGowan
50b06182f8 Register local content plugin from separate package
Update the local content plugin to register itself in a consistent way
as other plugins. This also allows the separate package to define its
own configuration more cleanly.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-08-22 11:18:30 -07:00
Akhil Mohan
ebc47359ea use format string when using printf like commands
As per https://github.com/golang/go/issues/60529, printf like commands with
non-constant format strings and no args give an error in govet

Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2024-08-14 17:04:53 +05:30
Sebastiaan van Stijn
9776047243 migrate to github.com/moby/sys/userns
Commit 8437c567d8 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 12:48:54 +02:00
ningmingxiao
0e4f2108b5 support to syncfs after pull by using diff plugin
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
2024-08-02 11:47:42 +08:00
Sebastiaan van Stijn
8437c567d8 pkg/userns: deprecate and migrate to github.com/moby/sys/user/userns
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

This patch deprecates the containerd fork of that package, and adds it as
an alias for the moby/sys/user/userns package.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-26 09:47:50 +02:00
Samuel Karp
0dcc51aedf introspection: regenerate UUID if state is empty
The /var/lib/containerd/io.containerd.grpc.v1.introspection/uuid file
stores a UUID to identify the particular containerd daemon responding to
requests.  The file should either exist with a UUID, or not exist.
However, it has been observed that the file can be truncated with 0
bytes, which will then fail to be parsed as a valid UUID.

As a defensive practice, detect a 0-length file and overwrite with a new
UUID rather than failing.

Fixes: https://github.com/containerd/containerd/issues/10491
Signed-off-by: Samuel Karp <samuelkarp@google.com>
2024-07-25 11:02:54 -07:00
Akhil Mohan
300fd770a0 use typeurl funcs for marshalling anypb.Any
Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2024-07-10 22:26:27 +05:30
Fu Wei
3b2a14bb4c Merge pull request #10407 from dmcgowan/fsverity-internal
Move fsverity package to internal
2024-07-01 21:31:55 +00:00
Samuel Karp
ebcbbe5643 Merge pull request #10401 from samuelkarp/nri-panic
cri: ensure NRI API never has nil CRI
2024-07-01 16:37:46 +00:00
Derek McGowan
23170e20df Move fsverity package to internal
This functionality is not directly related to containerd and could move
to external package at some point.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-07-01 09:21:24 -07:00
Fu Wei
f5f11658c5 Merge pull request #9903 from abel-von/add-update-resource-only
sandbox: Add Update API for sandbox controller
2024-07-01 15:16:21 +00:00
Akihiro Suda
7103e381f1 Merge pull request #10007 from Jenkins-J/fsverity-content-verification
Fsverity content verification
2024-06-29 06:32:17 +00:00
Samuel Karp
10aec359a0 cri: ensure NRI API never has nil CRI
A nil CRIImplementation field can cause a nil pointer dereference and
panic during startup recovery.

Prior to this change, the nri.API struct would have a nil cri
(CRIImplementation) field after nri.NewAPI until nri.Register was
called.  Register is called mid-way through initialization of the CRI
plugin, but recovery for containers occurs prior to that.  Container
recovery includes establishing new exit monitors for existing containers
that were discovered.  When a container exits, NRI plugins are given the
opportunity to be notified about the lifecycle event, and this is done
by accessing that CRIImplementation field inside the nri.API.  If a
container exits prior to nri.Register being called, access to the
CRIImplementation field can cause a panic.

Here's the call-path:

* The CRI plugin starts running
  [here](ae71819c4f/pkg/cri/server/service.go (L222))
* It then [calls into](ae71819c4f/pkg/cri/server/service.go (L227))
  `recover()` to recover state from previous runs of containerd
* `recover()` then attempts to recover all containers through
  [`loadContainer()`](ae7d74b9e2/internal/cri/server/restart.go (L175))
* When `loadContainer()` finds a container that is still running, it waits
  for the task (internal containerd object) to exit and sets up
  [exit monitoring](ae7d74b9e2/internal/cri/server/restart.go (L391))
* Any exit that then happens must be
  [handled](ae7d74b9e2/internal/cri/server/events.go (L145))
* Handling an exit includes
  [deleting the Task](ae7d74b9e2/internal/cri/server/events.go (L188))
  and specifying [`nri.WithContainerExit`](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L348))
  to [notify](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L356))
  any subscribed NRI plugins
* NRI plugins need to know information about the pod (not just the sandbox),
  so before a plugin is notified the NRI API package
  [queries the Sandbox Store](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L232))
  through the CRI implementation
* The `cri` implementation member field in the `nri.API` struct is set as part of the
  [`Register()`](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L66)) method
* The `nri.Register()` method is only called
  [much further down in the CRI `Run()` method](ae71819c4f/pkg/cri/server/service.go (L279))

Signed-off-by: Samuel Karp <samuelkarp@google.com>
2024-06-28 15:32:11 -07:00
James Jenkins
ef98c71985 Implement fsverity functionality
Implement calls to the fsverity kernel module, allowing containerd to
enable fsverity on blob data in the content store. This causes fsverity
to veirfy the integrity of blob data when the blob is read.

Signed-off-by: James Jenkins <James.Jenkins@ibm.com>
2024-06-28 09:50:44 -04:00
Akhil Mohan
624aa49d4b increase xfs base image size to 300Mb
from xfsprogs 5.19.0, minimum size of xfs volume has to be 300MB

Ref: https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=6e0ed3d19c54603f0f7d628ea04b550151d8a262
Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2024-06-19 00:07:27 +05:30
Abel Feng
15887d7efc sandbox: add update api for controller
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-06-14 02:31:51 +00:00
Derek McGowan
5d6a94a54c Merge pull request #10243 from dmcgowan/update-shim-manager-config
Cleanup shim manager configuration
2024-06-03 17:22:36 +00:00
Maksym Pavlenko
6383a1ce76 Merge pull request #10258 from bzsuni/bz/update/pause
update pause image to 3.10
2024-05-28 18:01:23 +00:00