Commit Graph

6 Commits

Author SHA1 Message Date
Maksym Pavlenko
d983c186f5 Merge pull request #11733 from erofs/erofs-layers
erofs-differ: support EROFS native image layers
2025-04-23 23:33:23 +00:00
Gao Xiang
2f9734fa59 erofs-differ: support EROFS native image layers
If the layer media type is expected as an EROFS native layer (ending
with `.erofs`), copy the content as the layer blob.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2025-04-24 00:26:37 +08:00
Derek McGowan
98eded24b8 Move erofsutils to internal
Avoid introducing utils package outside of internal. This package
should not be imported by other modules.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-04-22 09:03:49 -07:00
Gao Xiang
09f34d18b7 erofs-differ: implement fast differ with DiffDirChanges()
Unlike the walking differ, which implements a generic method to
accommodate all kinds of snapshotters, the EROFS differ is just
implemented for EROFS and EROFS snapshotter so it can utilize the
recent DiffDirChanges() [1] to avoid traversing the entire rootfs
directory in order to improve `nerdctl commit` performance.

Additionally, I think `baseDir` is unnecessary too (in principle,
only `upperdir` is useful for OCI format convention).  However,
addressing this requires more work, so left as is for now.

It's also useful to implement a customized Compare() method for
EROFS differ so that we can dump the native EROFS-formatted blob
to the content store later.

[1] https://github.com/containerd/continuity/pull/145
Signed-off-by: Gao Xiang <xiang@kernel.org>
2025-04-19 11:30:48 +08:00
Gao Xiang
2486d542a5 Introduce EROFS Snapshotter
It allows us to mount each EROFS blob layer (generated by the EROFS
differ) independently, or use the "unpacked" fs/ directories (if
some other differ is used.)

Currently, it's somewhat like the overlay snapshotter, but I tend
to separate the new EROFS logic into a self-contained component,
rather than keeping it tangled in the very beginning.

Existing users who use the overlay snapshotter won't be impacted
at all but they have a chance to use this new snapshotter to
leverage the EROFS filesystem.

Signed-off-by: cardy.tang <zuniorone@gmail.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2025-01-13 16:31:11 +08:00
Gao Xiang
c73c8e5d52 Introduce EROFS differ
The EROFS differ only applies to EROFS layers which are marked by
a special file `.erofslayer` generated by the EROFS snapshotter.

Why it's needed?  Since we'd like to parse []mount.Mount directly
without actual mounting and convert OCI layers into EROFS blobs,
`.erofslayer` gives a hint that the active snapshotter supports
the output blob generated by the EROFS differ.

I'd suggest it could be read together with the next commit.

Signed-off-by: cardy.tang <zuniorone@gmail.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2025-01-13 16:16:54 +08:00