This fails in `gitSourceHandler` because it can't handle *both* SHA-1 and SHA-256 before Git 2.45 (see PR comments), and there's not a simple way to fix that without newer Git or larger refactoring.
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Because subdir filter happened before the submodule update it
resulted in empty directory being filtered and submodule update
being skipped because .gitmodules was already missing.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Convert usages of `github.com/docker/docker/pkg/idtools` to
`github.com/moby/sys/user` in order to break the dependency between
buildkit and docker.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
The current logic was incorrect in some places so that if first
session randomly chosen by `Any()` returned NotFound then other
sessions were not attempted.
For the main use case of mounting secrets as files the logic
was correct, but it was incorrect for example for the case of
adding secrets as environment variables.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The github.com/docker/docker/pkg/reexec package was moved to
a separate module; migrate to use that module instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fixes current issue when a Git commit is accessed
multiple times through different refs or ref is added
after commit has already been pulled once.
When keep-git-dir option is true, then program can
try to resolve the current reference via .git directory
and because old cache key was only the git commit, previous
.git directory can be reused without any refs inside.
There is no change to the behavior if keep-git-dir is
false as then requests through multiple refs yield to
identical content.
Only the reference in the user provided identifier is added
to the cache key, and that is the only one that can be
expected in .git because of the shallow fetches. We do not
do extra request to find named refs for a commit SHA if that is
provided in the identifier.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
We shouldn't use the cachemount root, we should actually properly
use the worker's specified root which is propagated from the config.
Signed-off-by: Justin Chadwell <me@jedevc.com>
On a build with no-cache, cache mounts were not pruned correctly
if the mount was on top of another ref. This also appeared in
Dockerfile when mode/uid/gid was set because implicit parent
ref is created in these cases in order to change the permissions
of a subdir that is used as a cache mount base.
Because it is not possible to know ahead of time what ref
will become the parent of cache mount during build, all cache
mounts matching the ID that have a parent will be pruned.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
appdefaults_unix.go, constants_unix.go:
/run is a thing only for Linux. Other Unixes use /var/run.
diffapply_linux.go and source_linux.go:
These files use Linux-only API.
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
On commit SHA input we currently do a full fetch of
remote so we can pick up the commit by SHA later. This
only pulls in tags that are also part of branches. Extra
flag is needed to also get the tags that are not part of
branches.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Previously, it was very possible for the CacheKey function to return a
sha key that was *not* the checked out commit.
There are two cases that I've encountered where this can happen:
- An annotated tag will have the pin of the tag, and not the underlying
commit, which will be HEAD after the checkout.
- If multiple tags have the same path component (e.g. "mytag" and
"abc/mytag") then the first alphabetical tag will be selected when (in
this case "abc/mytag").
To avoid this kind of case, we can't just search for a single match in
the results for ls-remote. There's no way to filter for just an exact
match, so we need to scan through the output ourselves. Additionally, we
need to dereference the annotated tags by also selecting refs ending in
"^{}" - which have the commit that the tag points at.
Finally, I've improved the test suite around this to check that:
- The cache-key pin is equivalent to the checked out commit
- We can check out non-master branches
- That full ref syntax like "refs/heads/<branch-name>" and
"refs/tags/<tag-name>" (or even "refs/<anything>") can be used.
Signed-off-by: Justin Chadwell <me@jedevc.com>
It's possible for a git url to contain encoded credentials - even though
this practice is not recommended. For safety, we attempt to censor the
credentials before putting them into cache descriptions and error
messages.
However, we were previously missing one of these, and we would put an
uncensored git url into the git checkout snapshot cache ref description.
Signed-off-by: Justin Chadwell <me@jedevc.com>
This resolves a regression introduced in
50e75e3565. In this previous patch, I'd
incorrectly assumed that scp-like URLs can express a subset of
"standard"-URLs and so we can always safely convert them for
consistency. This isn't true - the URL "git@example.com:foo" should be
resolved to the home directory of the host, however, the converted URL
"ssh://git@example.com/foo" will be resolved to the root of the host.
To resolve this, we need to not perform this conversion. However, we
also need preserve the behaviour of firm distinction between SCP and
normal URL types (so as to keep proper port parsing).
To do this, we add a new GitURL type to the gitutil package. This new
type contains all useful fields shared in common between the standard
libraries url package and our custom scp-style url parsing package. This
keeps the previous property of a single clean interface to all GitURLs,
while also ensuring that we preserve the original URL to pass to the Git
CLI (making sure we strip fragments out, which are used as
buildkit-level metadata).
As a side-effect of this, the client-side calling code for parsing
git urls is simplified (so we don't have to do fragment wrangling at
every call point).
Signed-off-by: Justin Chadwell <me@jedevc.com>
Move all of the git command line logic into a single object, inspired by
the object already in buildx.
The basic implemenation allows for configuring a git cli for a specific
repository, along with various authorization settings and custom
binaries. Commands can be run for that repository, and a few helpers are
provided for accessing data on it - more to come in the future
hopefully.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Sources are a pretty neat extension point, except there are a few code
paths that hard-code against each type. This moves code around and
adjusts interfaces so that Source implementations are self-contained and
merely need to be registered with the source.Manager.
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Buildkit code is mostly generic enough to support FreeBSD, however
there are some quirks / infrastructural pieces that need to be
addressed for full support, to name some
- contenthash.NewFromStat attempts to set Devmajor / Devminor for
regular files, assuming that RDev is zero for regular
files. Unlike on Linux, it's not the case for FreeBSD.
- containerdexecutor.Run uses bind mounts for rootfs. Bind mounts
are not supported in FreeBSD and we should use nullfs instead
- There is no CI job to run tests on FreeBSD
- Some dependencies weren't ported
This change ports buildkit to FreeBSD
Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
