Commit Graph
1881 Commits
Author SHA1 Message Date
ningmingxiao 1dbb7f2ae3 pkg/sys: improve GetLocalListener/CreateUnixSocket error message
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
2025-04-01 12:53:41 +08:00
Jin Dong 42effa3b91 Mark NetworkPluginBinDir as DEPRECATED
To make it as DEPRECATED, this PR does the following:

1. Changes config default to use `NetworkPluginBinDirs`;
2. Mark `NetworkPluginBinDir` as deprecated (in config version 3);
3. Add config migration from 2 to 3, which migrates `bin_dir`
  in version 2 to `bin_dirs` in version 3.

Signed-off-by: Jin Dong <djdongjin95@gmail.com>

[wip] add deprecation warning

Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-03-21 16:59:32 +00:00
Jin Dong c8effff1a8 Fix CI lint error
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-03-18 01:16:07 +00:00
Derek McGowan b430e5ac3a Merge commit from fork
validate uid/gid
2025-03-18 01:23:21 +08:00
Craig Ingram de1341c201 validate uid/gid 2025-03-11 14:44:38 +00:00
Phil Estes b1924f1fd2 Merge pull request #11337 from azr/tar-rootpath-cache
perf(applyNaive): avoid walking the tree for each file in the same directory
2025-03-07 21:08:05 +00:00
Adrien DelormeandCorentin REGAL d8063c30dd perf(applyNaive): avoid walking the tree for each file in the same directory
This for example allowed us to gain ~5s while pulling ghcr.io/huggingface/text-generation-inference:3.0.2

Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
Co-Authored-By: Corentin REGAL <143578+co42@users.noreply.github.com>
2025-02-28 18:50:45 +01:00
Akihiro Suda 2841fc8107 Merge pull request #11433 from zhaodiaoer/main
pkg: prevent oom watcher from depending on shim pkg
2025-02-26 01:02:05 +00:00
Samuel Karp 9c2eb67756 Merge pull request #11435 from chrishenzie/ignore-zombies
Ignore defunct verifier procs in test
2025-02-25 22:42:23 +00:00
Chris Henzie 76858ac8e3 Ignore defunct verifier procs in test
This test produces zombies when PID 1 does not properly reap child
processes. Modifies the test to ignore these processes in the meantime.

Signed-off-by: Chris Henzie <chrishenzie@google.com>
2025-02-25 13:29:49 -08:00
Lei Liu 268880bf53 [improve] prevent oom watcher depend on shim pkg.
Signed-off-by: Lei Liu <liulei.pt@bytedance.com>
2025-02-25 22:06:15 +08:00
Kirtana Ashok 6c02321f6e Merge pull request #11179 from ambarve/blocked_cim
Support for importing layers in the block CIM format.
2025-02-24 22:21:10 +00:00
Amit Barve a1c540085f Support for importing layers in the block CIM format.
Adds a new diff plugin that can import image layers in the block CIM format using the new
block CIM layer writer added in hcsshim repo.

This commit also makes another important change in the way a diff is applied when using
CimFS based layer writers. Currently, the diff plugins call archive.Apply to apply a diff
and pass a function (that can actually apply the diff) as an argument (via
archive.ApplyOptions). This allows the callers to call archive.Apply with either a custom
applier function or if the caller doesn't pass such a function archive.Apply uses the
default naive diff applier.
However, there is drawback to this approach. The applier function passed to the
`archive.Apply` call needs to follow a specific signature. This signature expects it that
all parent layers are represented as an array of strings. In cases like CimFS, we can't
easily represent a set of layers as strings (unless we encode extra data in those strings
in a hacky way). To get around this problem, the diff plugins for CimFS based layers, skip
the archive.Apply call and directly call the layer writer instead.

Signed-off-by: Amit Barve <ambarve@microsoft.com>
2025-02-10 14:10:37 -05:00
Adrien DelormeandCorentin REGAL c51f5d26f1 perf(zstd): deactivate the low mem decoder
Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
Co-Authored-By: Corentin REGAL <143578+co42@users.noreply.github.com>
2025-02-05 14:39:16 +01:00
Fu Wei 306c47f6e1 Merge pull request #10033 from ambarve/cimfs_layer_refactor
Update cimfs snapshotter & differ for new hcsshim interface
2025-01-22 19:49:36 +00:00
Derek McGowan bdc847f1eb Remove deprecated WithCDIDevices in oci spec opts
This function has been moved to prevent an unintended dependency on CDI.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-13 22:07:37 -08:00
Derek McGowan e20f7f4a24 Move CDI device spec out of the OCI package
The CDI device injection spec opt was mistakenly added to the OCI
package which brought in an unintended dependency on CDI and its
transitive dependencies.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-13 21:59:59 -08:00
Amit Barve b81ace8724 Update cimfs snapshotter & differ for new hcsshim interface
hcsshim recently [updated](microsoft/hcsshim@1d406d0) the interface of APIs that are used
for importing OCI layers. It now expects that the CimFS snapshotter mounts contain the
full cim paths for parent layers. This change updates the cimfs differ & snapshotter to
use that new interface.

Signed-off-by: Amit Barve <ambarve@microsoft.com>
2025-01-10 17:06:57 -05:00
Kazuyoshi Kato 5ad6a150b6 Merge pull request #11189 from djdongjin/move-to-go-native-fuzz
Move fuzz tests to go native fuzz [part1]
2025-01-10 01:27:14 +00:00
Maksym Pavlenko 5e85064da0 Merge pull request #11194 from austinvazquez/remove-loop-variable-copies
Remove loop variable copies
2025-01-02 22:59:58 +00:00
Jin Dong ee63381887 bump up ttrpc to use its MD.Clone
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-12-30 04:01:17 +00:00
Jin Dong 2de1030299 Move fuzz tests to go native fuzz [part1]
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-12-24 22:48:20 +00:00
Austin Vazquez bee64b2b93 Remove loop variable copies
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-12-23 23:14:49 -07:00
Maksym Pavlenko 7b5cbd8091 Merge pull request #10851 from athos-ribeiro/test-segfault
test: prevent segfault in imageverifier test
2024-12-13 19:56:46 +00:00
Derek McGowan 17f7858b4e Update differ to handle zstd media types
The differ should be able to generate zstd compressed layers when
provided with the zstd media type.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-11-26 14:29:21 -08:00
Akihiro Suda 588b7a100a testutil: avoid conflict with continuity/testutil
```
=== FAIL: core/metadata  (0.00s)
/tmp/go-build142702247/b1061/metadata.test flag redefined: test.root
panic: /tmp/go-build142702247/b1061/metadata.test flag redefined: test.root

goroutine 1 [running]:
flag.(*FlagSet).Var(0xc0000d6000, {0x132a040, 0x190f3f3}, {0x11b4291, 0x9}, {0x11c7690, 0x1e})
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:1028 +0x54d
flag.BoolVar(...)
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:755
github.com/containerd/containerd/v2/pkg/testutil.init.0()
	/home/runner/work/containerd/containerd/pkg/testutil/helpers.go:36 +0x6b
FAIL	github.com/containerd/containerd/v2/core/metadata	0.028s
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-10-28 11:58:28 +09:00
Sebastiaan van Stijn fddeb6f3cb pkg/protobuf: fix typo in godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 14:18:53 +02:00
Athos Ribeiro 1617fd72e1 test: prevent segfault in imageverifier test
When VeriftImage returns an error, the first return value may be set to
nil (this is the case for timeouts). When that is the case, it makes no
point trying to access its fields later in the test. Let's use
testify/require instead of testify/assert here to ensure the test halts
upon failure.

Signed-off-by: Athos Ribeiro <athos.ribeiro@canonical.com>
2024-10-19 08:56:50 -03:00
Mike Baynton 347423a114 Request 'allow' setgroups when spawning new userns
Signed-off-by: Mike Baynton <mike@mbaynton.com>
2024-10-17 15:37:36 -05:00
Brian Goff b85909cd4c shim: Move pprof server to plugin
Makes the pprof server a plugin and also gates by the `shim_tracing`
build tag (like otel is).
With this change, `net/http` is no longer a dependency in the shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:50 +00:00
Brian Goff b2681dfbdb shim: Move ttrpc interceptors to plugins
This makes it so we don't need to import otelttrpc unless the shim is
compiled with the `shim_tracing` build tag.
This way otel is no longer compiled into the binary at all unless
`shim_tracing` is set.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:47 +00:00
Brian Goff 17d4a1357e Propagate trace contexts to shims
This adds trace context propagation over the grpc/ttrpc calls to a shim.

It also adds the otlp plugin to the runc shim so that it will send
traces to the configured tracer (which is inherited from containerd's
config).
It doesn't look like this is adding any real overhead to the runc shim's
memory usage, however it does add 2MB to the binary size.
As such this is gated by a build tag `shim_tracing`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-09-30 21:44:16 +00:00
Akihiro Suda 8c64a2f6a1 Merge pull request #10607 from fuweid/pin-userns
internal/cri: simplify netns setup with pinned userns
2024-09-19 01:05:41 +00:00
Wei Fu fd3f3d5a13 pkg/sys: add GetUsernsForNamespace interface
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-09-11 07:21:43 +08:00
Wei Fu 490e45a08a pkg/sys: Add UnshareAfterEnterUserns function
It allows to disassociate parts of its execution context within a user
namespace.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-09-11 07:21:39 +08:00
Samuel Karp 5f37a2c205 fifosync: cross-process synchronization
Signed-off-by: Samuel Karp <samuelkarp@google.com>
2024-09-05 12:48:28 -07:00
Akihiro Suda cae19b14f3 Merge pull request #10658 from darwin-containers/reorganize-mount-unmount
Reorganize mount/unmount code so it is easier to add Darwin-specific implementation
2024-09-03 01:51:24 +00:00
Akihiro Suda 26d6fd0c3f Merge pull request #10611 from fuweid/getridof-hall-of-shame
core/mount: use ptrace instead of go:linkname
2024-08-30 17:07:33 +00:00
Marat Radchenko bfc1465a2c Reorganize mount/unmount code so it is easier to add Darwin-specific implementation
After these changes, in order to add Darwin bind-mount implementation, one only needs:
* Adjust HasBindMounts definition in mount.go
* Provide implementation in mount_darwin.go

There was no consensus on adding dependency on bindfs, that seems to be the only working solution for bind-mounts on Darwin as of today, in https://github.com/containerd/containerd/pull/8789, that's why the actual implementation is not added in current PR.

As a bonus, Linux FUSE-related code was moved to a separate file and possibly could be reused on FreeBSD, though this needs testing.

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2024-08-30 15:25:06 +03:00
Jin Dong d83184ced1 avoid repeated calls in Tricolor gc
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-26 23:35:24 -07:00
Jin Dong 6b97a08eee add benchmark
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-26 23:35:04 -07:00
Wei Fu 3cd8f9734d core/mount: use ptrace instead of go:linkname
The Go runtime has started to [lock down future uses of linkname][1] since
go1.23. In the go source code, containerd project has been marked in the
comment, [hall of shame][2]. Well, the go:linkname is used to fork no-op
subprocess efficiently. However, since that comment, I would like to use
ptrace and remove go:linkname in the whole repository.

With go1.22 `go:linkname`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 2440            533320 ns/op            1145 B/op         43 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 342           3661616 ns/op           11562 B/op        421 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  2.983s
```

With go1.22 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1785            739557 ns/op            3948 B/op         68 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 328           4024300 ns/op           39601 B/op        671 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.104s
```

With go1.23 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1815            723252 ns/op            4220 B/op         69 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 319           3957157 ns/op           42351 B/op        682 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.051s
```

Diff:

The `ptrace` is slower than `go:linkname` mode. However, it's accepctable.

```
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
                                    │ go122-golinkname │             go122-ptrace              │             go123-ptrace              │
                                    │      sec/op      │    sec/op     vs base                 │    sec/op     vs base                 │
BatchRunGetUsernsFD_Concurrent1-16        533.3µ ± ∞ ¹   739.6µ ± ∞ ¹        ~ (p=1.000 n=1) ²   723.3µ ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16       3.662m ± ∞ ¹   4.024m ± ∞ ¹        ~ (p=1.000 n=1) ²   3.957m ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                   1.397m         1.725m        +23.45%                   1.692m        +21.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │              go122-ptrace               │              go123-ptrace               │
                                    │       B/op       │     B/op       vs base                  │     B/op       vs base                  │
BatchRunGetUsernsFD_Concurrent1-16       1.118Ki ± ∞ ¹   3.855Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   4.121Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16      11.29Ki ± ∞ ¹   38.67Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   41.36Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
geomean                                  3.553Ki         12.21Ki        +243.65%                   13.06Ki        +267.43%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │             go122-ptrace             │             go123-ptrace             │
                                    │    allocs/op     │  allocs/op   vs base                 │  allocs/op   vs base                 │
BatchRunGetUsernsFD_Concurrent1-16         43.00 ± ∞ ¹   68.00 ± ∞ ¹        ~ (p=1.000 n=1) ²   69.00 ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16        421.0 ± ∞ ¹   671.0 ± ∞ ¹        ~ (p=1.000 n=1) ²   682.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                    134.5         213.6        +58.76%                   216.9        +61.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
```

[1]: <https://github.com/golang/go/issues/67401>
[2]: <https://github.com/golang/go/blob/release-branch.go1.23/src/runtime/proc.go#L4820>

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-26 21:19:50 +08:00
Akhil Mohan ebc47359ea use format string when using printf like commands
As per https://github.com/golang/go/issues/60529, printf like commands with
non-constant format strings and no args give an error in govet

Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2024-08-14 17:04:53 +05:30
Maksym Pavlenko 0b02e0c225 Merge pull request #7616 from swagatbora90/trace-cri-runtime
Add tracing spans to CRI runtime service apis
2024-08-09 18:24:47 +00:00
Sebastiaan van Stijn 9776047243 migrate to github.com/moby/sys/userns
Commit 8437c567d8 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 12:48:54 +02:00
Swagat Bora 83c26962c2 Remove extra span created in the instrumented service layer
Signed-off-by: Swagat Bora <sbora@amazon.com>
2024-08-05 02:34:27 +00:00
Swagat Bora c0cdcb34f1 Add spans to CRI runtime service and related client methods
This adds otel spans to CRI service mainly targeting mutating apis which includes:
* Sandbox apis - RunPodSandbox, StopPodSandbox, RemovePodSandbox
* Container apis - CreateContainer, StartContainer, StopContainer, RemoveContainer
* Attach, Exec and Exec Sync
* Containerd client methods: container.go, client.go, process.go and task.go

Signed-off-by: Swagat Bora <sbora@amazon.com>
2024-08-05 02:34:20 +00:00
Kazuyoshi Kato 2ddd3db952 Merge pull request #9858 from w13915984028/fixdebug
Add file name to device type check failure message
2024-07-26 21:20:38 +00:00
Akihiro Suda 7a1f6b2891 Merge pull request #10509 from samuelkarp/pr-9966-follow-up
deprecation: update warnings for CRI config fields
2024-07-26 17:40:59 +00:00
Sebastiaan van Stijn eddd90d7c1 remove deprecated pkg/userns
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-26 09:48:40 +02:00