The layer content cache was a single directory, so every source of
pre-converted blobs had to be merged into one tree. A shared read-only fleet
cache, a host-local cache, and a per-image warm cache could not coexist.
Replace layer_content_cache with layer_content_caches, a list. Each directory
is checked in order and the first hit is staged into the snapshot; a layer
found in none of them falls back to the normal download-and-convert path.
Cache directories are no longer required to exist at startup. A missing one is
indistinguishable from an empty one at lookup time (both are simply a miss),
and it may well be mounted after the daemon starts, so the only check left is
that each path is absolute -- a relative one would be symlinked into the
snapshot dir and dangle.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Add a --dmverity flag to `ctr images build-erofs-cache`. When set, each cached
erofs blob is dm-verity formatted (the hash tree is appended in place) and a
.dmverity sidecar is written alongside it. This is required when the erofs
snapshotter runs with dmverity_mode=on, which rejects cache hits that lack a
sidecar; without it such layers would have to be formatted out-of-band.
Extract the differ's dm-verity formatting into a shared dmverity.FormatLayer so
the differ and the cache builder share one implementation; the differ's
formatDmverityLayer now delegates to it.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Add `ctr images build-erofs-cache <image> <cache_dir>`, which reads an
already-pulled image's layers from the content store, converts each into a
directly-mountable erofs blob, and writes them under
<cache_dir>/<algorithm>/<hex>.erofs keyed by the layer's diffID. This
populates the directory the erofs snapshotter's layer_content_cache reads
on pull; because the key is the source diffID, layers shared across images
converge on one blob. No converted image is produced.
Extract the per-layer uncompress + mkfs.erofs step out of LayerConvertFunc
into an exported ConvertLayerToErofs so the image converter and the cache
builder share a single conversion path.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Mark converted EROFS manifests with the erofs OS feature and cover
feature-aware manifest selection and unpack routing for erofs images.
Signed-off-by: ChengyuZhu6 <hudson@cyzhu.com>
Fix issue 11228
`ctr images import --all-platforms` w/o `--local` was failing due to
`unable to initialize unpacker: no unpack platforms defined` error.
W/ `--local`, it unpacks the layers for the strict-default platform.
Now `ctr images import --all-platforms` w/o `--local` unpacks the layers
for the non-strict default platform.
This behavior still differs from `--local`.
i.e., on an arm64 host, arm/v{5,6,7} layers are unpacked too.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
There's a couple spots where we know exactly how large
the destination buffer should be, so pre-size these to
avoid any reallocs to a higher capacity.
Signed-off-by: Danny Canter <danny@dcantah.dev>
Unfortunately, this is a rather large diff, but perhaps worth a one-time
"rip off the bandaid" for v2. This patch removes the use of "gocontext"
as alias for stdLib's "context", and uses "cliContext" for uses of
cli.context.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
ctr currently silently ignores several flags by default (without --local) and
the user can't know which flags are supported until they see the code.
This commit fixes ctr to return an explicit error when it finds an unsupported
flag.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currenlty transfer service doesn't handle plain HTTP connection.
This commit fixes this issue by propagating
`(core/remotes/docker/config).HostOptions.DefaultScheme` from client to the
transfer service.
This commit also fixes ctr to use this feature for "--plain-http" flag.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currently transfer service isn't aware of configurations of hosts directory and
ctr's `--hosts-dir` doesn't work.
This commit fixes this issue by using `config.ConfigureHosts` instead of
`docker.ConfigureDefaultRegistries`.
This commit also fixes ctr to use this feature for "--hosts-dir" flag.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Followed the Migration Guide at https://cli.urfave.org/migrate-v1-to-v2/
The major changes not pointed out in the migration guide are:
- context.Args() no longer produces a []slice, so context.Args().Slice()
in substitued
- All cli.Global***** are deprecated (the migration guide is somewhat
unclear on this)
Signed-off-by: Derek Nola <derek.nola@suse.com>
Vendor in urfave cli/v2
Signed-off-by: Derek Nola <derek.nola@suse.com>
Fix NewStringSlice calls
Signed-off-by: Derek Nola <derek.nola@suse.com>
Packages related to transfer and unpacking provide core interfaces which
use other core interfaces and part of common functionality.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Prior to this commit, `--local=false` had to be explicitly specified to
opt-in to the transfer service
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This adds a new WithLabel function, which allows to set a single label on
a lease, without having to first construct an intermediate map[string]string.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
check if the target tag that is to be created using ctr image tag
is valid and does not contain any forbidden characters.
Signed-off-by: Akhil Mohan <makhil@vmware.com>
The metadata is small and useful for viewing all platforms
for an image and enabling push back to the same registry.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Maksym Pavlenko