The preceding commit advertises the default platform with the erofs OS feature. Move construction of that platform set into snapshotterPlatforms so the client-side matching contract can be tested without changing the advertisement.
EROFS identifies the image layer package format rather than a Linux-only host capability, so the feature-bearing entry remains derived from platforms.DefaultSpec. Test that the advertised set matches both the plain default platform and the same platform carrying the erofs feature.
Assisted-by: Claude Opus 5
Signed-off-by: Martín Fernández <mfernandez@crunchloop.io>
Advertise a second platform - the default platform spec with OSFeatures
["erofs"] - from the EROFS snapshotter plugin, alongside the plain
default platform it already advertised. This mirrors what the EROFS
differ plugin already advertises.
This is required by client.Image.Unpack's checkSnapshotterSupport, which
rejects unpacking an image whose config declares os.features onto a
snapshotter that doesn't advertise a matching platform: without this,
pulling and unpacking an EROFS image layer format specification image
(https://github.com/erofs/erofs-image-spec) - which requires "erofs" in
os.features - with the erofs snapshotter would fail that check.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Martín Fernández <mfernandez@crunchloop.io>
There was no way to tell whether a cache was doing anything: hits and
misses were only visible as trace log lines. Count lookups by result,
hits by the directory that served them so caches with different lifetimes
can be told apart, the blob bytes served, and lookup errors by reason,
which also makes an unreadable cache (a down FUSE mount, say) visible
instead of looking like a cold one.
Also add a span around the differ's Apply, carrying the layer descriptor
and which path handled it. The unpacker traces the fetch and the layer
lifecycle but not the apply, so a slow layer couldn't be pinned on the
download or the conversion.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
A new-* directory can exist before createSnapshot acquires the metadata
writer transaction. A concurrent Remove performs a global orphan scan and
can delete the staging directory because it is not in the ID map yet.
Exclude new-* staging directories from orphan cleanup and use a shared
local constant for directory creation and filtering.
Signed-off-by: Chris Ayoub <cayoub@openai.com>
The layer content cache was a single directory, so every source of
pre-converted blobs had to be merged into one tree. A shared read-only fleet
cache, a host-local cache, and a per-image warm cache could not coexist.
Replace layer_content_cache with layer_content_caches, a list. Each directory
is checked in order and the first hit is staged into the snapshot; a layer
found in none of them falls back to the normal download-and-convert path.
Cache directories are no longer required to exist at startup. A missing one is
indistinguishable from an empty one at lookup time (both are simply a miss),
and it may well be mounted after the daemon starts, so the only check left is
that each path is absolute -- a relative one would be symlinked into the
snapshot dir and dangle.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Replace the snapshots.ErrAlreadyStaged sentinel error with a
read-only-mounts signal: a snapshotter (e.g. erofs serving a layer
content cache hit) now returns Prepare mounts normally, with no
error, when the layer content is already staged into the active
snapshot. The unpacker's isStaged helper checks the last mount's
ReadOnly() to decide whether to skip fetch+apply and just commit.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
The cache used to serve a hit by committing the layer during Prepare and
returning ErrAlreadyExists, which is incompatible with parallel unpacking:
the "rebase" capability defers the parent to Commit time, so a
commit-at-Prepare layer ends up parentless. The erofs snapshotter therefore
had to disable rebase whenever the cache was enabled, making every cache
*miss* fall back to a fully sequential (slower-than-cold) pull.
Introduce snapshots.ErrAlreadyStaged, returned from Prepare to mean "the
active snapshot's content is staged; skip the layer download and apply, but
still Commit it" (where the parent is applied). Unlike ErrAlreadyExists it
does not end the layer's lifecycle. The unpacker handles it by emitting a
status whose bottom half runs the normal serialized commit (rebasing the
parent in), and the metadata snapshotter threads it through Prepare like a
normal active snapshot. The erofs cache hit now stages the blob and returns
ErrAlreadyStaged instead of committing, so it can advertise "rebase"
unconditionally: hits skip download+conversion and misses stay parallel.
Also promote the "rebase" capability string to snapshots.RebaseCap (shared by
the unpacker, erofs, and overlay).
Follow up: fetch is range-based, so only a fully cached contiguous
prefix skips downloads; a miss in a lower layer still pulls everything above
it. Per-layer fetch-skip is left as a follow-up.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
mounts() moved `first` to the merged fsmeta mount's index whenever
mountFsMeta matched, collapsing the overlay's lowerdir range to the
fsmeta alone and dropping any plain lowers already appended above it.
When fsmerge was added, fsmeta was only ever written for the top
parent of the chain being prepared, so the loop matched at i == 0,
where first already equalled the fsmeta's index and the reassignment
was a no-op. It only mattered once the fsmeta could sit below the top
parent, i.e. a chain extending an already-merged sub-chain. The
snapshotter no longer generates fsmeta itself, so today this only
affects externally supplied fsmeta files.
first only marks the start of the lowerdir range, so it must stay at
the first lower's index.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Add a layer_content_cache option to the erofs snapshotter: a directory of
pre-converted, diffID-keyed erofs layer blobs (in production a read-only
mountpoint-s3/FUSE mount) that the snapshotter sources layers from on pull
instead of downloading and converting them per node.
On an image-layer extraction Prepare, if the layer's blob is present in the
cache the snapshotter stages it as a symlink, commits it as the target
chainID in the same transaction, and returns ErrAlreadyExists. This uses the
existing remote-snapshot protocol, so containerd skips both the layer
download and the tar->erofs conversion; no core changes are needed. Any miss
(cache disabled, no snapshot.ref/diff-id labels, blob absent, unreadable
cache) falls through to the normal path, so pulls keep working.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
NewSnapshotter auto-appends "index=off" unless the configured
mount_options already carry an index option — but the check used
hasOption(..., "index", false), which compares against the literal
string "index". No valid mount option ever equals "index" (it is
always valued, "index=on"/"index=off"), so the append happened
unconditionally whenever the kernel exposes the overlay index
parameter.
Mount options are last-wins in the kernel, so a user-configured
"index=on" was silently overridden. Worse, configuring
mount_options = ["index=on", "nfs_export=on"] (the documented way to
make overlay mounts NFS-exportable) produced
"index=on,nfs_export=on,index=off", which the kernel rejects with
EINVAL (nfs_export=on conflicts with an explicit index=off) — breaking
every snapshot mount including image unpack.
Make hasOption match the key of a "key[=value]" option so callers need
not care whether the option takes a value, and add a regression test.
Signed-off-by: AprilNEA <github@sku.moe>
Switch block CIM snapshotter and diff plugin init checks from
IsBlockCimSupported() to IsBlockCimWriteSupported() to allow plugins
to load when the CimWriter.dll is available, even on OS builds
without native block CIM support.
Signed-off-by: Maksim An <maksiman@microsoft.com>
This change updates an incorrect default config value for the block CIM
snapshotter. The `ntfsFormat` should always be false for the block CIM
snapshotter. If the scratch needs to be formatted with NTFS, a format
option named `WithNTFSFormat` can be provided.
Signed-off-by: Amit Barve <ambarve@microsoft.com>
dmsetup.Status() returns a nil DeviceStatus when it fails. The mkfs
failure path records that error, but still unconditionally dereferences
status when logging the pool status.
Avoid the panic by logging a fallback status string when dmsetup.Status()
does not return a status.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Signed-off-by: Mikhail Dmitrichenko <m.dmitrichenko222@gmail.com>
Fix issue 13388
```
[...]
May 12 16:57:23 kind-control-plane kubelet[257]: failed to extract layer (application/vnd.docker.image.rootfs
.diff.tar sha256:6f1cdceb6a3146f0ccb986521156bef8a422cdbb0863396f7f751f575ba308f4) to overlayfs as "extract-920875437
-7QPF sha256:31e64620332e54e3e4fb246d8325ed2c9f1c2cc64a95f0bb23b4b7e82834c95a": failed to mount /var/lib/containerd/t
mpmounts/containerd-mount2180142388: mount source: "overlay", target: "/var/lib/containerd/tmpmounts/containerd-mount
2180142388", fstype: overlay, flags: 0, data: "upperdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/sn
apshots/279/fs", err: invalid argument
[...]
```
This was a regression introduced in PR 13115.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Currently the metadata may be generated after a snapshot is committed,
causing a difference in the overlayfs mount that may cause ESTALE errors.
The committed snapshot must be immutable and should always return the
same set of mounts after commit, even if configuration is changed.
This setting may be added back later to be performed before commit,
either as part of the unpack or as a step before commit. This is getting
pulled out from the upcoming release.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Update the EROFS snapshotter to pass the dm-verity metadata path
through the `X-containerd.dmverity` mount option. Previously, the
system relied on implicit mode detection; providing the explicit
path allows downstream components (such as Kata Containers mount
handlers) to reliably locate and read `.dmverity` files.
Correspondingly, update the EROFS mount handler to parse and utilize
this explicit metadata path for dm-verity device initialization.
This enables the runtime to retrieve root hashes and other necessary
metadata directly, ensuring robust layer integrity verification.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
If fsmerge is enabled and no write is needed, it can return an overlay
mount with a single lowerdir, which is illegal for overlayfs.
For example, it can cause the following Nerdctl error:
: I'm not sure why ctr works, but the issue is real.
```bash
$ nerdctl run --runtime io.containerd.kata.v2 --snapshotter=erofs -it --rm nginx:latest /bin/bash
FATA[0000] failed to mount {Type:overlay Source:overlay Target: Options:[lowerdir=/run/containerd/
io.containerd.mount-manager.v1.bolt/t/7/1]} on "/tmp/initialC2039543827": mount source: "overlay",
target: "/tmp/initialC2039543827", fstype: overlay, flags: 0, data: "lowerdir=/run/containerd/io.
containerd.mount-manager.v1.bolt/t/7/1", err: invalid argument
```
Switch to using a bind mount instead.
Fixes: 9a7500a974 ("Add support for EROFS fsmerge feature")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
The layer blob immutable flag clearing logic was moved before
storage.Remove() call to ensure that immutable files can be properly
removed even if subsequent operations fail after storage.Remove().
The previous order had storage.Remove() called first, which meant if
any subsequent operations failed, there would be no opportunity to
remove the immutable flag on the layer blob files.
Signed-off-by: jinda.ljd <jinda.ljd@alibaba-inc.com>
The traditional mount() syscall has a PAGE_SIZE (typically 4KB) limit
for mount options. Use the new mount API (fsopen/fsconfig/fsmount/
move_mount) introduced in Linux 5.2 to bypass this limitation.
Fixed: #12662
Signed-off-by: ChengyuZhu6 <hudson@cyzhu.com>
This disables the slow_chown feature (nobody in their right mind
is going to be choosing erofs and want to slowly chown each file),
indicates that we support idmaps if the kernel supports it, and makes
sure to chown the upperdir.
This is more or less exactly how the overlay snapshotter does things,
minus the slow_chown part (which has discussions about dropping
altogether at some point anyways).
Signed-off-by: Andrew Halaney <ahalaney@netflix.com>
EROFS has supported a tiny metadata-only image to reference external
blobs since Linux 5.16. This eliminates the need to mount each EROFS
layer one by one and is also useful for VM-based containers (e.g.
nerdbox and Kata containers.)
Similar to LCOW/CimFS, `snapshots.UnpackKeyPrefix` is used to
trigger fsmerge generation (typically < 100 ms) on demand in Prepare().
In the future, we can also generate fsmeta in Commit() of the final
unpacking layer (by introducing an annotation to keep the chainID).
However, in the case of intermediate layer reuse, the Prepare() handling
will still be required.
```toml
[plugins."io.containerd.snapshotter.v1.erofs"]
max_unmerged_layers = 1 # enable fsmerge if image layers >= 2
```
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Non-Linux hosts are not expected to be able to directly mount erofs
snapshotters on the host system. Non-Linux hosts should use block mode
by default with a reasonably set default block size.
Signed-off-by: Derek McGowan <derek@mcg.dev>
We should reject commit request if target parent is not committed.
Just in case we rebase active snapshot on active one.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Rename `default_size_mb` to `default_size` so that it's similar to
Docker's `--storage-opt size=30G` flag. For example,
``` toml
[plugins."io.containerd.snapshotter.v1.erofs"]
default_size = "5GB"
```
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
This commit makes all of the recommended changes to use the `testing`
package helper functions instead of doing the equivalent longhand
versions of the same thing.
This change was needed in order to properly detect errors, as the code
would previously skip running `tenv` stating that it had been deprecated
in favor of `usetesting`.
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
Add new snapshotter & differ plugins that can leverage the block CIM layer writers to
store pulled/imported images in the block CIM format.
Signed-off-by: Amit Barve <ambarve@microsoft.com>
Fu Wei