Commit Graph

1895 Commits

Author SHA1 Message Date
Akihiro Suda
f209a82ca9 Merge pull request #11860 from thaJeztah/oci_no_panic
pkg/oci: prevent panic for some platform-specific options
2025-05-20 16:53:38 +00:00
Sebastiaan van Stijn
b0052d94a6 pkg/oci: prevent panic for some platform-specific options
Some of these options are designed to be a no-op when used on a Spec
that doesn't match the platform for the option. However, if the given
plaform was not present, they would panic.

This patch:

- Adds an early-return for options that are only applied on a
  specific platform.
- Update the GoDoc for these functions to describe they're a
  no-op on other platforms.
- Adds some rudimentary unit-tests to verify their behavior.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-05-15 00:44:45 +02:00
Samuel Karp
38c9bb93a3 Revert "perf(applyNaive): avoid walking the tree for each file in the same directory"
This reverts commit d8063c30dd.

Signed-off-by: Samuel Karp <samuelkarp@google.com>
2025-05-12 16:36:10 -07:00
Derek McGowan
7360c739fc Fix image inspect skip over missing content
Rather than exiting early with an error, just output that the content
does not exist locally and continue. This is helpful for displaying
multiplatform images when the content for every platform was not pulled.
If a platform that did not exist locally showed up before one that did,
the inspect would error out before even showing the content.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-05-05 16:13:39 -07:00
Derek McGowan
ae9b003d24 Merge pull request #10762 from mxpv/dbg
Enable HTTP debug and trace for transfer based puller
2025-04-24 16:22:02 +00:00
Maksym Pavlenko
17b6e1ef85 Allow streaming to client
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2025-04-23 18:03:29 -07:00
Maksym Pavlenko
bd8e6c727d Enable http debug and tracing for non local puller
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2025-04-23 17:39:57 -07:00
Maksym Pavlenko
27e6c117d9 Move HTTP debug code to pkg
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2025-04-23 17:38:47 -07:00
Sebastiaan van Stijn
1477874494 use lazyregexp to compile regexes on first use
- internal/cri/bandwidth: use lazyregexp to compile regexes on first use
- pkg/identifiers: use lazyregexp to compile regexes on first use
- pkg/progress: use lazyregexp to compile regexes on first use
- pkg/reference: use lazyregexp to compile regexes on first use
- pkg/sys: use lazyregexp to compile regexes on first use

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-22 20:29:14 +02:00
Phil Estes
250593568f Merge pull request #11684 from AkihiroSuda/postpone-deprecation-2.1
config: postpone planned v2.1 deprecations to v2.2
2025-04-17 02:02:48 +00:00
Akihiro Suda
764dcf77a8 config: postpone planned v2.1 deprecations to v2.2
We going to release v2.1 earlier than we have once expected at that time,
so extend the grace period for migration.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2025-04-17 09:14:15 +09:00
Akihiro Suda
499238a52a Remove deprecated dynamic library plugins
Dynamic library plugins have been deprecated since v2.0 (PR 9238)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2025-04-17 09:12:46 +09:00
Akihiro Suda
d9c889568e Remove the support for Schema 1 images
Schema 1 (`application/vnd.docker.distribution.manifest.v1+prettyjws`) has been
officially deprecated since containerd v1.7 (PR 6884), and disabled since v2.0 (PR 9765).

Users who have been seeing warnings like `conversion from schema 1 images is deprecated`
now have to rebuild the image with Schema 2 or OCI.

Schema 2 was introduced in Docker 1.10 (Feb 2016), so most users should have been already
using Schema 2 or OCI.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2025-04-11 09:03:26 +09:00
QiPing Wan
be9ca11a14 fix call fmt.Errorf with wrong error
Signed-off-by: QiPing Wan <alingse@foxmail.com>
2025-04-08 09:09:40 +08:00
ningmingxiao
1dbb7f2ae3 pkg/sys: improve GetLocalListener/CreateUnixSocket error message
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
2025-04-01 12:53:41 +08:00
Jin Dong
42effa3b91 Mark NetworkPluginBinDir as DEPRECATED
To make it as DEPRECATED, this PR does the following:

1. Changes config default to use `NetworkPluginBinDirs`;
2. Mark `NetworkPluginBinDir` as deprecated (in config version 3);
3. Add config migration from 2 to 3, which migrates `bin_dir`
  in version 2 to `bin_dirs` in version 3.

Signed-off-by: Jin Dong <djdongjin95@gmail.com>

[wip] add deprecation warning

Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-03-21 16:59:32 +00:00
Jin Dong
c8effff1a8 Fix CI lint error
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-03-18 01:16:07 +00:00
Derek McGowan
b430e5ac3a Merge commit from fork
validate uid/gid
2025-03-18 01:23:21 +08:00
Craig Ingram
de1341c201 validate uid/gid 2025-03-11 14:44:38 +00:00
Phil Estes
b1924f1fd2 Merge pull request #11337 from azr/tar-rootpath-cache
perf(applyNaive): avoid walking the tree for each file in the same directory
2025-03-07 21:08:05 +00:00
Adrien Delorme
d8063c30dd perf(applyNaive): avoid walking the tree for each file in the same directory
This for example allowed us to gain ~5s while pulling ghcr.io/huggingface/text-generation-inference:3.0.2

Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
Co-Authored-By: Corentin REGAL <143578+co42@users.noreply.github.com>
2025-02-28 18:50:45 +01:00
Akihiro Suda
2841fc8107 Merge pull request #11433 from zhaodiaoer/main
pkg: prevent oom watcher from depending on shim pkg
2025-02-26 01:02:05 +00:00
Samuel Karp
9c2eb67756 Merge pull request #11435 from chrishenzie/ignore-zombies
Ignore defunct verifier procs in test
2025-02-25 22:42:23 +00:00
Chris Henzie
76858ac8e3 Ignore defunct verifier procs in test
This test produces zombies when PID 1 does not properly reap child
processes. Modifies the test to ignore these processes in the meantime.

Signed-off-by: Chris Henzie <chrishenzie@google.com>
2025-02-25 13:29:49 -08:00
Lei Liu
268880bf53 [improve] prevent oom watcher depend on shim pkg.
Signed-off-by: Lei Liu <liulei.pt@bytedance.com>
2025-02-25 22:06:15 +08:00
Kirtana Ashok
6c02321f6e Merge pull request #11179 from ambarve/blocked_cim
Support for importing layers in the block CIM format.
2025-02-24 22:21:10 +00:00
Amit Barve
a1c540085f Support for importing layers in the block CIM format.
Adds a new diff plugin that can import image layers in the block CIM format using the new
block CIM layer writer added in hcsshim repo.

This commit also makes another important change in the way a diff is applied when using
CimFS based layer writers. Currently, the diff plugins call archive.Apply to apply a diff
and pass a function (that can actually apply the diff) as an argument (via
archive.ApplyOptions). This allows the callers to call archive.Apply with either a custom
applier function or if the caller doesn't pass such a function archive.Apply uses the
default naive diff applier.
However, there is drawback to this approach. The applier function passed to the
`archive.Apply` call needs to follow a specific signature. This signature expects it that
all parent layers are represented as an array of strings. In cases like CimFS, we can't
easily represent a set of layers as strings (unless we encode extra data in those strings
in a hacky way). To get around this problem, the diff plugins for CimFS based layers, skip
the archive.Apply call and directly call the layer writer instead.

Signed-off-by: Amit Barve <ambarve@microsoft.com>
2025-02-10 14:10:37 -05:00
Adrien Delorme
c51f5d26f1 perf(zstd): deactivate the low mem decoder
Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
Co-Authored-By: Corentin REGAL <143578+co42@users.noreply.github.com>
2025-02-05 14:39:16 +01:00
Fu Wei
306c47f6e1 Merge pull request #10033 from ambarve/cimfs_layer_refactor
Update cimfs snapshotter & differ for new hcsshim interface
2025-01-22 19:49:36 +00:00
Derek McGowan
bdc847f1eb Remove deprecated WithCDIDevices in oci spec opts
This function has been moved to prevent an unintended dependency on CDI.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-13 22:07:37 -08:00
Derek McGowan
e20f7f4a24 Move CDI device spec out of the OCI package
The CDI device injection spec opt was mistakenly added to the OCI
package which brought in an unintended dependency on CDI and its
transitive dependencies.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-13 21:59:59 -08:00
Amit Barve
b81ace8724 Update cimfs snapshotter & differ for new hcsshim interface
hcsshim recently [updated](microsoft/hcsshim@1d406d0) the interface of APIs that are used
for importing OCI layers. It now expects that the CimFS snapshotter mounts contain the
full cim paths for parent layers. This change updates the cimfs differ & snapshotter to
use that new interface.

Signed-off-by: Amit Barve <ambarve@microsoft.com>
2025-01-10 17:06:57 -05:00
Kazuyoshi Kato
5ad6a150b6 Merge pull request #11189 from djdongjin/move-to-go-native-fuzz
Move fuzz tests to go native fuzz [part1]
2025-01-10 01:27:14 +00:00
Maksym Pavlenko
5e85064da0 Merge pull request #11194 from austinvazquez/remove-loop-variable-copies
Remove loop variable copies
2025-01-02 22:59:58 +00:00
Jin Dong
ee63381887 bump up ttrpc to use its MD.Clone
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-12-30 04:01:17 +00:00
Jin Dong
2de1030299 Move fuzz tests to go native fuzz [part1]
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-12-24 22:48:20 +00:00
Austin Vazquez
bee64b2b93 Remove loop variable copies
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-12-23 23:14:49 -07:00
Maksym Pavlenko
7b5cbd8091 Merge pull request #10851 from athos-ribeiro/test-segfault
test: prevent segfault in imageverifier test
2024-12-13 19:56:46 +00:00
Derek McGowan
17f7858b4e Update differ to handle zstd media types
The differ should be able to generate zstd compressed layers when
provided with the zstd media type.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-11-26 14:29:21 -08:00
Akihiro Suda
588b7a100a testutil: avoid conflict with continuity/testutil
```
=== FAIL: core/metadata  (0.00s)
/tmp/go-build142702247/b1061/metadata.test flag redefined: test.root
panic: /tmp/go-build142702247/b1061/metadata.test flag redefined: test.root

goroutine 1 [running]:
flag.(*FlagSet).Var(0xc0000d6000, {0x132a040, 0x190f3f3}, {0x11b4291, 0x9}, {0x11c7690, 0x1e})
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:1028 +0x54d
flag.BoolVar(...)
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:755
github.com/containerd/containerd/v2/pkg/testutil.init.0()
	/home/runner/work/containerd/containerd/pkg/testutil/helpers.go:36 +0x6b
FAIL	github.com/containerd/containerd/v2/core/metadata	0.028s
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-10-28 11:58:28 +09:00
Sebastiaan van Stijn
fddeb6f3cb pkg/protobuf: fix typo in godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 14:18:53 +02:00
Athos Ribeiro
1617fd72e1 test: prevent segfault in imageverifier test
When VeriftImage returns an error, the first return value may be set to
nil (this is the case for timeouts). When that is the case, it makes no
point trying to access its fields later in the test. Let's use
testify/require instead of testify/assert here to ensure the test halts
upon failure.

Signed-off-by: Athos Ribeiro <athos.ribeiro@canonical.com>
2024-10-19 08:56:50 -03:00
Mike Baynton
347423a114 Request 'allow' setgroups when spawning new userns
Signed-off-by: Mike Baynton <mike@mbaynton.com>
2024-10-17 15:37:36 -05:00
Brian Goff
b85909cd4c shim: Move pprof server to plugin
Makes the pprof server a plugin and also gates by the `shim_tracing`
build tag (like otel is).
With this change, `net/http` is no longer a dependency in the shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:50 +00:00
Brian Goff
b2681dfbdb shim: Move ttrpc interceptors to plugins
This makes it so we don't need to import otelttrpc unless the shim is
compiled with the `shim_tracing` build tag.
This way otel is no longer compiled into the binary at all unless
`shim_tracing` is set.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:47 +00:00
Brian Goff
17d4a1357e Propagate trace contexts to shims
This adds trace context propagation over the grpc/ttrpc calls to a shim.

It also adds the otlp plugin to the runc shim so that it will send
traces to the configured tracer (which is inherited from containerd's
config).
It doesn't look like this is adding any real overhead to the runc shim's
memory usage, however it does add 2MB to the binary size.
As such this is gated by a build tag `shim_tracing`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-09-30 21:44:16 +00:00
Akihiro Suda
8c64a2f6a1 Merge pull request #10607 from fuweid/pin-userns
internal/cri: simplify netns setup with pinned userns
2024-09-19 01:05:41 +00:00
Wei Fu
fd3f3d5a13 pkg/sys: add GetUsernsForNamespace interface
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-09-11 07:21:43 +08:00
Wei Fu
490e45a08a pkg/sys: Add UnshareAfterEnterUserns function
It allows to disassociate parts of its execution context within a user
namespace.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-09-11 07:21:39 +08:00
Samuel Karp
5f37a2c205 fifosync: cross-process synchronization
Signed-off-by: Samuel Karp <samuelkarp@google.com>
2024-09-05 12:48:28 -07:00