Most content proxy operations normalize remote RPC errors before
returning them, including stream receive errors from Walk and write
errors from the remote writer. remoteReaderAt.ReadAt was an outlier and
returned raw status errors from Read and Recv.
Callers that use content.ReadBlob through the proxy can then fail
errdefs checks, such as treating concurrent content deletion as
NotFound.
Convert non-EOF read stream errors with errgrpc.ToNative so ReaderAt
matches the rest of the content proxy while preserving io.EOF.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
digest.FromBytes always uses the canonical (sha256) algorithm, so
descriptors using other algorithms (e.g. sha512) would never match
on the embedded data path, falling through to the provider.
Use desc.Digest.Algorithm().FromBytes() to correctly verify data
against whatever algorithm the descriptor specifies.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Allows management of referrer objects when performing
pull, archive export and archive import.
Referrer objects are linked to their subjects via GC
labels. The label is based on sha256 checksum of the
object instead of incremental numbers as referrers are
not immutable and don't have any strict order.
In OCI-layout referrers that are not already in the exported
tree are added to the main index.json with
io.containerd.manifest.subject annotation.
On import such descriptors with that annotation
do not create digest-based images in the image store.
Note that this does not mean all the referrer objects in
the registry are now pulled/exported/imported by default.
The caller of the client pkg functions can choose which
referrer objects should also be handled.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
We've long been able to use these and they have a couple
great benefits:
1. Forces you to always access them atomically. With the pointer
variants it's completely valid to access the regular ol' int64/uint32
etc. without using the atomic.* methods. These wrappers don't provide
access to the underlying value so it forces correct usage always.
2. Conveys intent much better. Seeing the type be atomic.Int32 immediately
lets the reader know that this var will be used in a concurrent context,
and we no longer need comments like "this MUST be accessed atomically"
or similar.
Signed-off-by: Danny Canter <danny@dcantah.dev>
The remote content writer proxy already has the capability to break up
large files into multiple writes, but the current API doesn't recognize
when it's about to exceed the limits and attempts to send the data over
grpc in one message instead of breaking it into multiple messages.
This changes the behavior of `Write` to automatically break up the size
of the content based on the max send message size.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Uses the new github.com/containerd/errdefs/pkg module which is intended
to hold less stable utility functions separately from the stable
github.com/containerd/errdefs error types.
Includes temporary update to hcsshim until a release is cut there
Signed-off-by: Derek McGowan <derek@mcg.dev>
Allow the api to stay at the same v1 go package name and keep using a
1.x version number. This indicates the API is still at 1.x and allows
sharing proto types with containerd 1.6 and 1.7 releases.
Signed-off-by: Derek McGowan <derek@mcg.dev>
If a writer continually asks to be reset then it should always succeed -
it should be the responsibility of the underlying content.Writer to
stop producing ErrReset after some amount of time and to instead return
the underlying issue - which pushWriter already does today, using the
doWithRetries function.
doWithRetries already has a separate cap for retries of 6 requests (5
retries after the original failure), and it seems like this would be
previously overridden by content.Copy's max number of 5 attempts, hiding
the original error.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Paweł Gronowski