Commit Graph
15 Commits
Author SHA1 Message Date
Pierluigi Lenoci 2a69c0d2c8 Honor stderrthreshold when logtostderr is enabled
klog has a long-standing bug where setting -logtostderr=true causes the
-stderrthreshold flag to be silently ignored. All log messages are sent
to stderr regardless of their severity.

klog v2.140.0 introduced two new flags that allow callers to opt into the
correct behavior:

  -legacy_stderr_threshold_behavior=false
  -stderrthreshold=INFO

Set them in the CRI plugin's setGLogLevel() function, right after
klog.InitFlags() and before any other flag configuration.

Reference: kubernetes/klog#212
Signed-off-by: Pierluigi Lenoci <pierluigi.lenoci@gmail.com>
2026-04-03 10:53:54 -05:00
Sebastiaan van Stijn 4b6cc97c4d plugins: modernize: any
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -any -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 18:24:15 +01:00
Akihiro Suda 910171e90e Fix directory permissions
- Create /var/lib/containerd with 0o700 (was: 0o711).
- Create config.TempDir with 0o700 (was: 0o711).
- Create /run/containerd/io.containerd.grpc.v1.cri with 0o700 (was: 0o755).
- Create /run/containerd/io.containerd.sandbox.controller.v1.shim with 0o700 (was: 0o711).
- Leave /run/containerd and /run/containerd/io.containerd.runtime.v2.task created with 0o711,
  as required by userns-remapped containers.
  /run/containerd/io.containerd.runtime.v2.task/<NS>/<ID> is created with:
  - 0o700 for non-userns-remapped containers
  - 0o710 for userns-remapped containers with the remapped root group as the owner group.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2025-11-04 16:04:30 +09:00
Enji Cooper f45716efed Clean up issues cited by usetesting package with golangci
This commit makes all of the recommended changes to use the `testing`
package helper functions instead of doing the equivalent longhand
versions of the same thing.

This change was needed in order to properly detect errors, as the code
would previously skip running `tenv` stating that it had been deprecated
in favor of `usetesting`.

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
2025-09-07 14:07:40 -07:00
Mike Brown 5f3f84f567 removes use of klog from containerd repo
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2025-05-02 13:34:46 +00:00
Jin Dong 42effa3b91 Mark NetworkPluginBinDir as DEPRECATED
To make it as DEPRECATED, this PR does the following:

1. Changes config default to use `NetworkPluginBinDirs`;
2. Mark `NetworkPluginBinDir` as deprecated (in config version 3);
3. Add config migration from 2 to 3, which migrates `bin_dir`
  in version 2 to `bin_dirs` in version 3.

Signed-off-by: Jin Dong <djdongjin95@gmail.com>

[wip] add deprecation warning

Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-03-21 16:59:32 +00:00
Jin Dong fb44e37ff2 Remove confusing warning in cri runtime config migration
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2025-01-12 02:38:14 +00:00
Sebastiaan van Stijn 8a8c3e2215 pkg/cri/server/base: log CRI config as embedded JSON
Use the JSON-encoded representation of the config used, which allows
users to reconstruct a (valid) config file from the logs, which may be
more useful for debugging purposes than the internal (Go) representation.

Before this:

    INFO[2023-12-07T15:33:39.914626385Z] starting cri plugin                           config="{PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}"

After this:

    INFO[2023-12-07T15:27:15.862946138Z] starting cri plugin                           config="{\"containerd\":{\"snapshotter\":\"overlayfs\",\"defaultRuntimeName\":\"runc\",\"runtimes\":{\"runc\":{\"runtimeType\":\"io.containerd.runc.v2\",\"runtimePath\":\"\",\"PodAnnotations\":null,\"ContainerAnnotations\":null,\"options\":{\"BinaryName\":\"\",\"CriuImagePath\":\"\",\"CriuWorkPath\":\"\",\"IoGid\":0,\"IoUid\":0,\"NoNewKeyring\":false,\"Root\":\"\",\"ShimCgroup\":\"\"},\"privileged_without_host_devices\":false,\"privileged_without_host_devices_all_devices_allowed\":false,\"baseRuntimeSpec\":\"\",\"cniConfDir\":\"\",\"cniMaxConfNum\":0,\"snapshotter\":\"\",\"sandboxer\":\"podsandbox\"}},\"disableSnapshotAnnotations\":true,\"discardUnpackedLayers\":false,\"ignoreBlockIONotEnabledErrors\":false,\"ignoreRdtNotEnabledErrors\":false},\"cni\":{\"binDir\":\"/opt/cni/bin\",\"confDir\":\"/etc/cni/net.d\",\"maxConfNum\":1,\"setupSerially\":false,\"confTemplate\":\"\",\"ipPref\":\"\"},\"registry\":{\"configPath\":\"\",\"mirrors\":null,\"configs\":null,\"auths\":null,\"headers\":null},\"imageDecryption\":{\"keyModel\":\"node\"},\"disableTCPService\":true,\"streamServerAddress\":\"127.0.0.1\",\"streamServerPort\":\"0\",\"streamIdleTimeout\":\"4h0m0s\",\"enableSelinux\":false,\"selinuxCategoryRange\":1024,\"sandboxImage\":\"registry.k8s.io/pause:3.9\",\"statsCollectPeriod\":10,\"enableTLSStreaming\":false,\"x509KeyPairStreaming\":{\"tlsCertFile\":\"\",\"tlsKeyFile\":\"\"},\"maxContainerLogSize\":16384,\"disableCgroup\":false,\"disableApparmor\":false,\"restrictOOMScoreAdj\":false,\"maxConcurrentDownloads\":3,\"disableProcMount\":false,\"unsetSeccompProfile\":\"\",\"tolerateMissingHugetlbController\":true,\"disableHugetlbController\":true,\"device_ownership_from_security_context\":false,\"ignoreImageDefinedVolumes\":false,\"netnsMountsUnderStateDir\":false,\"enableUnprivilegedPorts\":true,\"enableUnprivilegedICMP\":true,\"enableCDI\":false,\"cdiSpecDirs\":[\"/etc/cdi\",\"/var/run/cdi\"],\"imagePullProgressTimeout\":\"5m0s\",\"drainExecSyncIOTimeout\":\"0s\",\"containerdRootDir\":\"/var/lib/docker/containerd/daemon\",\"containerdEndpoint\":\"/var/run/docker/containerd/containerd.sock\",\"rootDir\":\"/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri\",\"stateDir\":\"/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri\"}"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-29 13:10:54 +02:00
Sebastiaan van Stijn f62edda5a2 pkg/cri/server/base: use structured log for CRI plugin startup
Log the config as a field instead of as part of the log message.

Before this:

    INFO[2023-12-07T14:58:43.515360429Z] loading plugin                                id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1
    INFO[2023-12-07T14:58:43.515787512Z] loading plugin                                id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1
    INFO[2023-12-07T14:58:43.515974429Z] loading plugin                                id=io.containerd.internal.v1.cri type=io.containerd.internal.v1
    INFO[2023-12-07T14:58:43.516037887Z] Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}

After this:

    INFO[2023-12-07T15:33:39.914112719Z] loading plugin                                id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1
    INFO[2023-12-07T15:33:39.914526135Z] loading plugin                                id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1
    INFO[2023-12-07T15:33:39.914580427Z] loading plugin                                id=io.containerd.internal.v1.cri type=io.containerd.internal.v1
    INFO[2023-12-07T15:33:39.914626385Z] starting cri plugin                           config="{PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-29 13:10:51 +02:00
Derek McGowan a086125ae3 Move config version to version package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-14 21:09:44 -08:00
Akihiro Suda 6670695836 Revert "cri: make read-only mounts recursively read-only"
Revert PR 9713, as it appeared to break the compatibility too much
https://github.com/kubernetes/enhancements/pull/3858#issuecomment-1925441072

This reverts commit b2f254fff0.

> Conflicts:
>	internal/cri/opts/spec_linux_opts.go

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-04 01:17:14 +09:00
Maksym Pavlenko bbac058cf3 Move CRI from pkg/ to internal/
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2024-02-02 10:12:08 -08:00
Akihiro Suda b2f254fff0 cri: make read-only mounts recursively read-only
Prior to this commit, `readOnly` volumes were not recursively read-only and
could result in compromise of data;
e.g., even if `/mnt` was mounted as read-only, its submounts such as
`/mnt/usbstorage` were not read-only.

This commit utilizes runc's "rro" bind mount option to make read-only bind
mounts literally read-only. The "rro" bind mount options is implemented by
calling `mount_setattr(2)` with `MOUNT_ATTR_RDONLY` and `AT_RECURSIVE`.

The "rro" bind mount options requires kernel >= 5.12, with runc >= 1.1 or
a compatible runtime such as crun >= 1.4.

When the "rro" bind mount options is not available, containerd falls back
to the legacy non-recursive read-only mounts by default.

The behavior is configurable via `/etc/containerd/config.toml`:
```toml
version = 2
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  # treat_ro_mounts_as_rro ("Enabled"|"IfPossible"|"Disabled")
  # treats read-only mounts as recursive read-only mounts.
  # An empty string means "IfPossible".
  # "Enabled" requires Linux kernel v5.12 or later.
  # This configuration does not apply to non-volume mounts such as "/sys/fs/cgroup".
  treat_ro_mounts_as_rro = ""
```

Replaces:
- kubernetes/enhancements issue 3857
- kubernetes/enhancements PR 3858

Note: this change does not affect non-CRI clients such as ctr, nerdctl, and Docker/Moby.
RRO mounts have been supported since nerdctl v0.14 (containerd/nerdctl PR 511)
and Docker v25 (moby/moby PR 45278).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-01 09:39:36 +09:00
Derek McGowan 65b3922df7 Split streaming config from runtime config
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-28 23:14:59 -08:00
Derek McGowan 9795677fe9 Move cri base plugin to CRI runtime service
Create new plugin type for CRI runtime and image services.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-28 20:57:18 -08:00