Use the resolver-cache ref ID directly when loading a cached HTTP snapshot.
This avoids dereferencing missing metadata for cache entries found during a
concurrent solve.
Reset the unauthorized checksum race request counter per solve iteration so
each retry applies the intended first-request delay.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Move the HTTP snapshot resolver-cache lookup into a helper so the lock
release is deferred after acquisition. This keeps invalid cache entries
and digest mismatch errors from returning while the cache lock is held.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Avoid storing typed nil HTTP metadata in the resolver cache after failed
metadata resolution. Also make resolver cache release functions consistently
callable when no error is returned.
Add a client integration test covering concurrent same-URL HTTP sources with
and without checksum when the server returns 401.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Update non-generated code for the newer lint recommendations by using typed
atomic values, strings.Cut, and slices.Backward where applicable.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Update golangci-lint and adjust code for new gosec diagnostics. Use
root-scoped filesystem operations where appropriate, preserve explicit
user path behavior for SSH keys, and avoid background contexts in
request-scoped cleanup paths.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Support importing git sources from OCI or registry-backed bundle blobs
and exporting resolved checkouts as single-file git bundles.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Allow git sources to fetch a pinned commit without resolving the ref
against the remote tip, while preserving cache keys for canonical
branch refs and covering the behavior with tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add solve-wide compatibility-version support for image and oci
exports, with historical goldens and release compatibility tests.
Backfill version 10 for v0.13-v0.14 git artifact behavior, keep
version 20 as current, and reject unsupported zstd on v10.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Resolve SOURCE_DATE_EPOCH=context in the Dockerfile frontend from the
main build context and pass the resolved numeric epoch through normal
ARG handling and exporter metadata.
Use git commit time for git contexts, HTTP Last-Modified when present,
and newest archive entry mtime for HTTP archives. Leave local contexts
unset.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Use os.OpenRoot for git dir and checkout subdir access, and share root-
relative path normalization between validation and open paths.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Open the snapshot mount as an os.Root and perform file reads
through root-relative APIs in verifySignature and
computeChecksumResponse, consistent with the write path.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Validate user-provided refs once during identifier construction and reject
option-like refs with leading '-'. There is no known attack related to
previous core, patch is to make ref handling more robust and improve
errors.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit e7f8093e1b386ffe711c8468ca8cdde8cfea0c72)
Normalize Git subdir fragments and validate checkout subdir components
so each segment must be a real directory, preventing traversal and symlink escapes.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 8c994eb561a2646b35352e5663afecd225306214)
Move safeFileName from source/http to source/util/pathutil
and apply it to the containerblob source as well. Harden
containerblob/pull.go to use os.OpenRoot for file writes,
preventing path traversal via crafted filenames.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 3d6e587655d72c343f6fdc7268480a900ba45b0c)
Open the snapshot mount as an os.Root and perform file write/chown/chtimes
through root-relative APIs to keep operations constrained to the mount root.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit d568881c97278d87e4f6f01a1f8a67ad807152bb)
Add safeFileName and route all getFileName sources through it.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 9d117af5ab1e1032f75658884384328fea440843)
Add git.mtime=commit option that normalizes all file, symlink,
and directory mtimes in a git snapshot to the resolved commit
timestamp. This enables reproducible builds from git sources.
When SOURCE_DATE_EPOCH is set in the Dockerfile frontend, the
git context automatically uses commit-time mtimes. The URL
query parameter ?mtime=commit|checkout can override this.
New LLB attr (git.mtime) and capability (source.git.mtime) are
registered as experimental. Cache keys include the mtime policy
so that commit-time and checkout-time snapshots are distinct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The `git checkout tree-ish -- pathspec` command run by buildkit to
populate the work tree defaults to overlay mode, so it won't delete objects
which exist in the work tree but don't exist in the tree-ish. It's not a
problem for ordinary file and directory objects because the work tree
starts out empty, but in the case where submodules are deleted or
renamed it will leave stale gitlink references in the index. The
subsequent `git submodule update ...` command will then fail with an
eror like 'No url found for submodule path ... in .gitmodules' as seen
in #4260.
Adding `--no-overlay` ensures that any deleted gitlink references are
removed from the index before the submodule update runs.
Signed-off-by: Brian Ristuccia <brian@ristuccia.com>
Image blob source in LLB allows addressing a single blob
from a container image registry. The difference from the image
source is that image source needs to point to a manifest that
internally points to an array of layer blobs that are all extracted
on top of each other to form a root FS. Contrary, image blob
points to a single blob that is not extracted but downloaded
as a single file into an empty snapshot, similarily how
the HTTP source works.
The main use case for this source is to pin snapshots of
HTTP URLs, upload the downloaded blob into container registry,
and then use a source policy to map a HTTP URL (whose content
might be changed) to the copy of the source as image blob
to ensure immutability.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add detached PGP verification for HTTP sources during metadata resolution
and expose LLB options/caps/attrs for signature validation.
Extract shared OpenPGP verification/parsing logic into util/pgpsign and
reuse it from git signing, plus add integration and source-level tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
These requests allow computing additional checksum algorithms
for the payload of HTTP sources.
Optionally suffix can be passed that is added to payload. This
is needed to make validation of PGP signatures possible remotely.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Without a lease, the manifest may get be deleted by GC
before the SetGCLabels() gets called, causing "not found" error.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Currently attestation chains were always loaded directly from
registry on each pull.
This adds cache capability to resolver so all the pulled manifests
are first pulled to content store and kept there with GC labels
references from the root manifest.
If blob or referrers request already exists in the content store
then local response is used without registry requests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Image manifest content is not needed for signature verification as
the verification is against the top index root. Still report
image manifest digest for more info about the reported attestation
subject but clients need to re-resolve it from the root manifest
for signature verification.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
ResolveImageConfig was changed to ResolveSourceMetadata long time
ago for cross-source implementation but the worker implementation
was still using old method name with conversions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
In (*httpSourceHandler).resolveMetadata method there is
a potential resp.Body leak in case when response status code is not
valid (< 200 or >= 400). Also resp.Body is not closed before exiting
if status code is okay.
This commit adds resp.Body closing before exiting from resolveMetadata
method.
Signed-off-by: Mikhail Dmitrichenko <m.dmitrichenko222@gmail.com>
Make sure remote ref does not change to different commit if
git repo changes in the middle of the build.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
New ResolverCache interface in JobContext allows build jobs to
memorize and synchronize accesses to mutable remote resources.
This is to make sure that when multiple parts of the same build
job, or build job and source metadata resolver access the same
remote resources, it remains the same for the duration of the
single build request, even if data happens to change on the remote side.
Fix such a possible case in the HTTP source. Even if the server
now returns completely different data, if the same URL was accessed
once for the ongoing build, then the initial contents are always
used until the build completes.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
