Validate the default branch returned by ls-remote before using it, so a
malicious remote cannot advertise a HEAD symref such as that would
later be reused as an argument to other git commands.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit a838953fe8776d68379d32a7af0776d538215763)
(cherry picked from commit eb63c321e8)
Update golangci-lint and adjust code for new gosec diagnostics. Use
root-scoped filesystem operations where appropriate, preserve explicit
user path behavior for SSH keys, and avoid background contexts in
request-scoped cleanup paths.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Support importing git sources from OCI or registry-backed bundle blobs
and exporting resolved checkouts as single-file git bundles.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Allow git sources to fetch a pinned commit without resolving the ref
against the remote tip, while preserving cache keys for canonical
branch refs and covering the behavior with tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add solve-wide compatibility-version support for image and oci
exports, with historical goldens and release compatibility tests.
Backfill version 10 for v0.13-v0.14 git artifact behavior, keep
version 20 as current, and reject unsupported zstd on v10.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add git.mtime=commit option that normalizes all file, symlink,
and directory mtimes in a git snapshot to the resolved commit
timestamp. This enables reproducible builds from git sources.
When SOURCE_DATE_EPOCH is set in the Dockerfile frontend, the
git context automatically uses commit-time mtimes. The URL
query parameter ?mtime=commit|checkout can override this.
New LLB attr (git.mtime) and capability (source.git.mtime) are
registered as experimental. Cache keys include the mtime policy
so that commit-time and checkout-time snapshots are distinct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The `git checkout tree-ish -- pathspec` command run by buildkit to
populate the work tree defaults to overlay mode, so it won't delete objects
which exist in the work tree but don't exist in the tree-ish. It's not a
problem for ordinary file and directory objects because the work tree
starts out empty, but in the case where submodules are deleted or
renamed it will leave stale gitlink references in the index. The
subsequent `git submodule update ...` command will then fail with an
eror like 'No url found for submodule path ... in .gitmodules' as seen
in #4260.
Adding `--no-overlay` ensures that any deleted gitlink references are
removed from the index before the submodule update runs.
Signed-off-by: Brian Ristuccia <brian@ristuccia.com>
This is more flexible as from the user standpoint it might not
be easy to know if a version tag is annotated or not.
Downside of this change is that we can't create a cache key
automatically when checksum was set and no keep-git-dir as we
don't know which checksum is being used.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Pin of annotated tag should be tha SHA of tag, not
the commit it is pointing to.
Cache key of annotated tag should be SHA of the tag
if keep-git-dir is enabled and SHA of underlying
commit otherwise.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
If tag was already pulled by --tags or without refs/tags
that creates ambigous reference in the shared repository.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This fails in `gitSourceHandler` because it can't handle *both* SHA-1 and SHA-256 before Git 2.45 (see PR comments), and there's not a simple way to fix that without newer Git or larger refactoring.
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Because subdir filter happened before the submodule update it
resulted in empty directory being filtered and submodule update
being skipped because .gitmodules was already missing.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This fixes current issue when a Git commit is accessed
multiple times through different refs or ref is added
after commit has already been pulled once.
When keep-git-dir option is true, then program can
try to resolve the current reference via .git directory
and because old cache key was only the git commit, previous
.git directory can be reused without any refs inside.
There is no change to the behavior if keep-git-dir is
false as then requests through multiple refs yield to
identical content.
Only the reference in the user provided identifier is added
to the cache key, and that is the only one that can be
expected in .git because of the shallow fetches. We do not
do extra request to find named refs for a commit SHA if that is
provided in the identifier.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
We shouldn't use the cachemount root, we should actually properly
use the worker's specified root which is propagated from the config.
Signed-off-by: Justin Chadwell <me@jedevc.com>
On commit SHA input we currently do a full fetch of
remote so we can pick up the commit by SHA later. This
only pulls in tags that are also part of branches. Extra
flag is needed to also get the tags that are not part of
branches.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Previously, it was very possible for the CacheKey function to return a
sha key that was *not* the checked out commit.
There are two cases that I've encountered where this can happen:
- An annotated tag will have the pin of the tag, and not the underlying
commit, which will be HEAD after the checkout.
- If multiple tags have the same path component (e.g. "mytag" and
"abc/mytag") then the first alphabetical tag will be selected when (in
this case "abc/mytag").
To avoid this kind of case, we can't just search for a single match in
the results for ls-remote. There's no way to filter for just an exact
match, so we need to scan through the output ourselves. Additionally, we
need to dereference the annotated tags by also selecting refs ending in
"^{}" - which have the commit that the tag points at.
Finally, I've improved the test suite around this to check that:
- The cache-key pin is equivalent to the checked out commit
- We can check out non-master branches
- That full ref syntax like "refs/heads/<branch-name>" and
"refs/tags/<tag-name>" (or even "refs/<anything>") can be used.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Move all of the git command line logic into a single object, inspired by
the object already in buildx.
The basic implemenation allows for configuring a git cli for a specific
repository, along with various authorization settings and custom
binaries. Commands can be run for that repository, and a few helpers are
provided for accessing data on it - more to come in the future
hopefully.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Sources are a pretty neat extension point, except there are a few code
paths that hard-code against each type. This moves code around and
adjusts interfaces so that Source implementations are self-contained and
merely need to be registered with the source.Manager.
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
