Add a layer_content_cache option to the erofs snapshotter: a directory of
pre-converted, diffID-keyed erofs layer blobs (in production a read-only
mountpoint-s3/FUSE mount) that the snapshotter sources layers from on pull
instead of downloading and converting them per node.
On an image-layer extraction Prepare, if the layer's blob is present in the
cache the snapshotter stages it as a symlink, commits it as the target
chainID in the same transaction, and returns ErrAlreadyExists. This uses the
existing remote-snapshot protocol, so containerd skips both the layer
download and the tar->erofs conversion; no core changes are needed. Any miss
(cache disabled, no snapshot.ref/diff-id labels, blob absent, unreadable
cache) falls through to the normal path, so pulls keep working.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Force a 4K block size on all platforms rather than only on darwin.
An explicit caller-supplied -b is still respected.
Signed-off-by: Chris Crone <christopher.crone@docker.com>
mkfs.erofs uses TMPDIR for its internal diskbuf temp files. Windows
does not set TMPDIR (only TEMP/TMP), so the MinGW binary falls back
to "/tmp" which resolves to C:\tmp. That directory does not exist on
most Windows machines. mkstemp fails, and erofs_diskbuf_init returns
ENOSPC regardless of actual errno, producing a misleading "No space
left on device" error even on disks with plenty of free space.
Set TMPDIR to the snapshot directory (parent of the output layer file)
for all mkfs.erofs invocations on Windows. This directory is managed
by containerd and guaranteed to exist. On Unix, TMPDIR is left to the
parent process (no change in behavior).
Signed-off-by: Craig Chelnak <craig.chelnak@docker.com>
Change the mkfs.erofs command logging to Debug level in both
ConvertTarErofs and GenerateTarIndexAndAppendTar. This keeps
Info level quiet as intended per project conventions while
still making the commands visible for debugging.
Signed-off-by: Aadhar Agarwal <aadagarwal@microsoft.com>
Use the same approach for appending UUID arguments in GenerateTarIndexAndAppendTar
as done in ConvertTarErofs for consistency between the two modes.
Signed-off-by: Aadhar Agarwal <aadagarwal@microsoft.com>
Derive filesystem UUIDs (`lsblk -o +UUID`) from the OCI layer digests
(although diffIDs are better in principle, but they're unavailable by
differs in advance) rather than generating a random one. This allows
EROFS to uniquely identify each layer using the content-addressable
filesystem UUID.
It can also be used for reproducible builds. To achieve this, configure
`mkfs_options` with `-T0 --mkfs-time` (However, `--mkfs-time` requires
erofs-utils 1.8+; Otherwise, all inode timestamps will be reset w/o it):
``` toml
[plugins."io.containerd.differ.v1.erofs"]
mkfs_options = ["-T0 --mkfs-time"]
```
Fixes: c73c8e5d52 ("Introduce EROFS differ")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Avoid introducing utils package outside of internal. This package
should not be imported by other modules.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Maksym Pavlenko