Commit Graph

22 Commits

Author SHA1 Message Date
Tonis Tiigi
a7d26b35ec client: add referrers support to client
Allows management of referrer objects when performing
pull, archive export and archive import.

Referrer objects are linked to their subjects via GC
labels. The label is based on sha256 checksum of the
object instead of incremental numbers as referrers are
not immutable and don't have any strict order.

In OCI-layout referrers that are not already in the exported
tree are added to the main index.json with
io.containerd.manifest.subject annotation.

On import such descriptors with that annotation
do not create digest-based images in the image store.

Note that this does not mean all the referrer objects in
the registry are now pulled/exported/imported by default.
The caller of the client pkg functions can choose which
referrer objects should also be handled.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-10-17 13:43:36 -07:00
Kirtana Ashok
12daca5f64 Fix intermittent test failures on Windows CIs
Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
2025-07-14 09:19:59 -07:00
Adrien Delorme
cb53f381e4 fix when multipart fetching and the server does not return content length
Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
2025-06-19 17:45:01 +02:00
Danny Canter
33ee060a35 Use Go 1.19 atomic wrappers everywhere
We've long been able to use these and they have a couple
great benefits:

1. Forces you to always access them atomically. With the pointer
variants it's completely valid to access the regular ol' int64/uint32
etc. without using the atomic.* methods. These wrappers don't provide
access to the underlying value so it forces correct usage always.

2. Conveys intent much better. Seeing the type be atomic.Int32 immediately
lets the reader know that this var will be used in a concurrent context,
and we no longer need comments like "this MUST be accessed atomically"
or similar.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2025-05-03 03:46:20 -07:00
Jonathan A. Sternberg
f25f36c334 proxy: break up writes from the remote writer to avoid grpc limits
The remote content writer proxy already has the capability to break up
large files into multiple writes, but the current API doesn't recognize
when it's about to exceed the limits and attempts to send the data over
grpc in one message instead of breaking it into multiple messages.

This changes the behavior of `Write` to automatically break up the size
of the content based on the max send message size.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-27 10:00:11 -06:00
Austin Vazquez
bee64b2b93 Remove loop variable copies
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-12-23 23:14:49 -07:00
Derek McGowan
2f24aa00a5 Update errdefs to 0.3.0
Uses the new github.com/containerd/errdefs/pkg module which is intended
to hold less stable utility functions separately from the stable
github.com/containerd/errdefs error types.

Includes temporary update to hcsshim until a release is cut there

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-10-18 16:04:54 -07:00
Derek McGowan
2ac2b9c909 Make api a Go sub-module
Allow the api to stay at the same v1 go package name and keep using a
1.x version number. This indicates the API is still at 1.x and allows
sharing proto types with containerd 1.6 and 1.7 releases.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-02 11:03:00 -07:00
Derek McGowan
e1b94c0e7d Move protobuf package under pkg
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-02 10:52:03 -07:00
Derek McGowan
182a5fc653 Merge pull request #9657 from azr/azr/ctt-hlp-read-seeker
content: add a BlobReadSeeker to allow multipart blob uploads
2024-05-02 15:20:17 +00:00
Xinyang Ge
4167416754 Perform file sync outside of lock on Commit
Signed-off-by: Xinyang Ge <xinyang.ge@databricks.com>
2024-04-26 05:42:01 -07:00
Akihiro Suda
5a23e8878c Merge pull request #9917 from AkihiroSuda/mv-testutil
mv internal/testutil pkg/testutil
2024-03-12 12:01:16 +00:00
Akihiro Suda
d9b9160ae1 mv internal/testutil pkg/testutil
The package is consumed by several snapshotter plugins

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-04 17:00:39 +09:00
Derek McGowan
347346e3cf Add ttrpc support to content proxy
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-03-01 14:27:27 -08:00
Adrien Delorme
bb9d923aa6 content: add a BlobReadSeeker func to allow multipart blob streaming
A downstream library (s3) needs a read seeker to be able to do its own multipart upload.

See: https://github.com/moby/buildkit/pull/4551

Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
2024-02-23 10:00:35 +01:00
Justin Chadwell
e4f91c2df0 copy: remove max number of ErrResets
If a writer continually asks to be reset then it should always succeed -
it should be the responsibility of the underlying content.Writer to
stop producing ErrReset after some amount of time and to instead return
the underlying issue - which pushWriter already does today, using the
doWithRetries function.

doWithRetries already has a separate cap for retries of 6 requests (5
retries after the original failure), and it seems like this would be
previously overridden by content.Copy's max number of 5 attempts, hiding
the original error.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:20:10 +01:00
Derek McGowan
fb9b59a843 Switch to new errdefs package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-25 22:18:45 -08:00
Derek McGowan
1c4be2d883 Move pkg/testutil to internal/testutil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:57:28 -08:00
Derek McGowan
4ee6419fad Move pkg/randutil to internal/randutil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:57:10 -08:00
Derek McGowan
c38f2ab724 Move filters to pkg/filters
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:54:54 -08:00
Derek McGowan
44a836c9b5 Move errdefs to pkg/errdefs
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:54:45 -08:00
Derek McGowan
21b4f3f0aa Move content to core/content
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:50:32 -08:00