Render parsed source identifiers back to their canonical SourceOp form before
source policy evaluation. This lets Git subdir cleanup use the existing source
parser and avoids policy-specific Git parsing.
Add String methods for source identifiers and cover them with unit tests, plus
a client integration regression for canonical Git subdir policy matching.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Without a lease, the manifest may get be deleted by GC
before the SetGCLabels() gets called, causing "not found" error.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Currently attestation chains were always loaded directly from
registry on each pull.
This adds cache capability to resolver so all the pulled manifests
are first pulled to content store and kept there with GC labels
references from the root manifest.
If blob or referrers request already exists in the content store
then local response is used without registry requests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Image manifest content is not needed for signature verification as
the verification is against the top index root. Still report
image manifest digest for more info about the reported attestation
subject but clients need to re-resolve it from the root manifest
for signature verification.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
ResolveImageConfig was changed to ResolveSourceMetadata long time
ago for cross-source implementation but the worker implementation
was still using old method name with conversions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This allows images to be pulled by tag and then
checked against the digest. If digest is added directly
to the image reference, then tag is ignored.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This allows platforms following the new `platforms.FormatAll` function,
which allows for setting the `OSVersion` field of the platform with
`<os>(<ver>)/<arch>`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In this case the current stack trace points to the line
where the context was created. Instead the stack should be
captured when the defer is running so the return path to
the defer call is also part of the stack.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Without this, a single ResolveImageConfig may emit many
"remotes.docker.resolver.HTTPRequest" spans. These can be difficult to
track and organize, especially if a single client performs multiple of
them.
To make traces a bit easier to navigate, these are now grouped under one
top-level span, instead of cluttering up the top-level.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We already have a `reference.Spec` and `reference.Spec.Digest()` does
exactly this. In addition, buildkit was the only external consumer of
the `reference.SplitObject` function, and it may be deprecated and removed
for containerd 2.0.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.
This patch:
- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to enforce using an alias, to prevent accidental use
of the errdefs package in BuildKit or Moby.
- adds a linter check to prevent using the "log" package, which was also
migrated to a separate module.
There are still some uses of the old package in (indirect) dependencies,
which should go away over time.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is more versatile function that works for any source,
not just images.
It can be used together with a policy that switches
between input and output source as well as for adding
additional metadata for other sources in the future.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Trivially created by looking for every reference to .Variant and adding
OSVersion and OSFeatures, except the ones related to the string
representation of a Platform instance.
I then went through and ensured every assignment of OSFeatures that
might leak out, i.e., not local-only or for marhsalling purposes, uses
the append-to-nil idiom to avoid sharing the slice storage and allowing
accidental mutation after-the-fact.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Sources are a pretty neat extension point, except there are a few code
paths that hard-code against each type. This moves code around and
adjusts interfaces so that Source implementations are self-contained and
merely need to be registered with the source.Manager.
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Before this change, ResolveImageConfig was unaware of source policies.
This means that:
1. Images for denied sources may be resolved
2. Image configs may get pulled for sources that are later converted to
a different image
The update makes it so the image resolver first runs a given ref through
the source policy and uses any mutated ref for the actual resolve
(instead of the original ref).
It also returns the mutated ref so it can be used correctly by the
frontend (e.g. don't want to do llb.Image(oldRef@resolvedDigest)).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This breaking api change refactors the LLB API to prevent reference
mangling and demangling throughout OCI access. Once the session and
store IDs have been determined in the dockerfile frontend, we keep them
the same, and attach them as additional properties.
This has the additional effect of making the actual reference used in
the image resolution arbitrary, since we only parse and access the
digest. The rest of the name can be selected to optimize for log
readability.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Instead of using custom parsing mechansisms for references in
oci-layout, we use containerd's reference.Parse or docker distribution's
reference.Parse (depending on where we do the parsing, and what's
consistent with the file where it's already done). These operations are
neater than manually parsing, and have hopefully more consistent error
messages, and better handling of labels (for if/when those are
introduced).
Signed-off-by: Justin Chadwell <me@jedevc.com>
This mirrors the structure of the names for the local cache directory,
as well as the names for the oci exporter (when using a content store).
This ensures that we cannot encounter name collisions (intentionally or
unintentionally).
Signed-off-by: Justin Chadwell <me@jedevc.com>
In the scenario with no session id, then the oci-layout resolver would
still attempt to load a caller with the empty session id. This
inevitably failed, and would fallback to any caller, but this would take
5 seconds to fail with the configured timeout.
In a fresh pull of an OCI image context, this could take up to 15
seconds, as 3 separate calls to the relevant functions would be made.
This patch fixes the issue by correctly identifying this case, and
directly falling through to any caller. Additionally, if a session id is
present, it will always be loaded with no fallback available. To do
this consistently, the helper methods are refactored into a more
consistent withCaller function.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Adds a new `WithLayerLimit` option to `llb.Image`
only pulls specified number of layers instead of
full image.
This can be used in combination with DiffOp/MergeOp
to pull any subset of layers from an image in any order.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.
The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The Parent method will no longer make sense with forthcoming Merge and
Diff support as refs will become capable of having multiple parents. It
was also only ever used externally to get the full chain of refs for
each layer in the ref's chain.
The newly added LayerChain method replaces Parents with a method that
just returns a slice of refs for each layer in the ref's chain. This
will work more seamlessly with Merge and Diff (in which case it returns
the "flattened" ancestors of the ref) in addition to being a bit easier
to use for the exiting cases anyways.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
