Commit Graph

28 Commits

Author SHA1 Message Date
Jonathan A. Sternberg
1a3fc0aa15 protobuf: remove gogoproto
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-09-26 12:57:45 -05:00
Justin Chadwell
ac7caa8f3b chore: use a better root for computing free disk space
We shouldn't use the cachemount root, we should actually properly
use the worker's specified root which is propagated from the config.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-09-25 10:49:52 +01:00
Tonis Tiigi
610affa5fd exec: fix pruning cache mounts with parent ref on no-cache
On a build with no-cache, cache mounts were not pruned correctly
if the mount was on top of another ref. This also appeared in
Dockerfile when mode/uid/gid was set because implicit parent
ref is created in these cases in order to change the permissions
of a subdir that is used as a cache mount base.

Because it is not possible to know ahead of time what ref
will become the parent of cache mount during build, all cache
mounts matching the ID that have a parent will be pruned.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-09-05 18:09:54 -07:00
Sebastiaan van Stijn
b4a189390e migrate to github.com/moby/sys/userns
Commit 4b85f11164 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 12:36:57 +02:00
Sebastiaan van Stijn
86b550e427 migrate to github.com/moby/sys/user/userns
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 14:11:04 +02:00
Tonis Tiigi
03c7a6d769 lint: fix some testifylint warnings
This does not cover all warning yet but split
into chunks to ease review.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-05-30 11:32:55 -07:00
Akihiro Suda
f668293668 secret mount: avoid setting noexec when an exec bit it set
Fix issue 4668

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-20 06:57:26 +09:00
Tonis Tiigi
8a2a3e83ec replace context.WithCancel with WithCancelCause
Keep stack traces for cancellation errors where possible.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:06 -08:00
Erik Sipsma
45b19d34ef Improve cache related trace logging.
This adds trace logs to the solver's cache manager to assist debugging
cache misses+hits.

It also replace the LazyStackTrace struct with a function that returns a
stack trace string only if trace level is enabled, an empty string
otherwise. This makes it easier to use with logrus hooks that pass the
fields around to different goroutines without needing to try to "unlazy"
the fields first.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-08-07 07:25:40 -07:00
Erik Sipsma
dd53dca59c Add trace logs for cache leaks.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-05-15 16:03:29 -07:00
coryb
b5afdec47c do not mount secrets that are optional and missing from solve opts
Signed-off-by: coryb <cbennett@netflix.com>
2023-01-31 18:27:32 -08:00
coryb
0123d32ba0 mount empty secrets
We should create the mount point for secret files even if the source
secret file is empty.  Otherwise it is hard to debug why secrets
are not showing up when requested in the LLB.  Currently they are just
silently ignored which was pretty confusing.

Signed-off-by: coryb <cbennett@netflix.com>
2022-09-03 11:54:32 -07:00
Justin Chadwell
bad80f4e9c llbsolver: increase test timeout for mount lock tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-02 15:13:30 +01:00
Eng Zer Jun
be6501b654 test: use T.TempDir to create temporary test directory
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-24 02:27:26 +08:00
Sebastiaan van Stijn
21e9e9641e Remove uses of deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-10 00:57:07 +01:00
Kohei Tokunaga
3b7422996a cache: Clean up temporary mount pool on restart
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2022-02-23 16:33:25 +09:00
Erik Sipsma
d73e62f878 Add initial MergeOp implementation.
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-18 11:10:48 -08:00
Erik Sipsma
9321ec2f82 Refactor cache record mount cache.
This is mostly just preparation for merge-op. The existing
Extract method is updated to be usable for unlazying any type of refs
rather than just lazy blobs. The way views are created is simplified and
centralized in one location.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-17 11:02:16 -08:00
CrazyMax
954c4721f7 use bytes as given size for tmpfs mount
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 19:47:17 +02:00
CrazyMax
87e1fa7ecb add size to tmpfs mounts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-18 10:54:52 +02:00
Erik Sipsma
a9f1980ebb Refactor cache metadata interface.
There are a few goals with this refactor:
1. Remove external access to fields that no longer make sense and/or
   won't make sense soon due to other potential changes. For example,
   there can now be multiple blobs associated with a ref (for different
   compression types), so the fact that you could access the "Blob"
   field from the Info method on Ref incorrectly implied there was just
   a single blob for the ref. This is on top of the fact that there is
   no need for external access to blob digests.
2. Centralize use of cache metadata inside the cache package.
   Previously, many parts of the code outside the cache package could
   obtain the bolt storage item for any ref and read/write it directly.
   This made it hard to understand what fields are used and when. Now,
   the Metadata method has been removed from the Ref interface and
   replaced with getters+setters for metadata fields we want to expose
   outside the package, which makes it much easier to track and
   understand. Similar changes have been made to the metadata search
   interface.
3. Use a consistent getter+setter interface for metadata, replacing
   the mix of interfaces like Metadata(), Size(), Info() and other
   inconsistencies.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-08-25 19:15:09 +00:00
Morlay
18b49fd7dc refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
Sebastiaan van Stijn
c325f89fa1 replace uses of deprecated containerd/sys.RunningInUserNS()
This utility was moved to a separate package, which has no dependencies.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 11:14:26 +02:00
Tonis Tiigi
f38972a8ec mounts: handle optional ssh mount
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-02 14:28:51 -08:00
Tonis Tiigi
e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Sebastiaan van Stijn
fbe1346912 replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 09:25:05 +01:00
Cory Bennett
095a919c95 remove unnecessary MountManager interface
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 06:24:34 +00:00
Cory Bennett
3cc8aa0649 move ExecOps mounting logic into new package so logic can be reused from gateway exec
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-15 07:01:42 +00:00