Commit Graph

54 Commits

Author SHA1 Message Date
Tonis Tiigi
d45a506aac cache: rename new prune/gc control fields
Naming that was chosen during review was
reservedSpace, maxUsedSpace and minFreeSpace.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-10-04 17:23:04 -07:00
Tonis Tiigi
a75b5eb15b add capability to detect if new storage filters are supported
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-10-04 17:21:53 -07:00
Tonis Tiigi
25632fb074 client: allow non-octal chmod config for fileop.copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-10-01 12:52:38 -07:00
Justin Chadwell
7e6c20a0db exec: allow specifying non-zero exit codes for execs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-09-17 11:36:07 +01:00
Erik Sipsma
7bc06cb0b7 add support for AlwaysReplaceExistingDestPaths in llb copy
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2024-04-02 12:11:06 -07:00
Justin Chadwell
0eb25a6bf7 exec: allow caller-controlled content-based cache
This allows LLB-directed content-based cache enablement for each mount.

Some mounts may not be explicitly unabled (because it would be unsafe) -
for these cases we explicitly error out.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-06 11:44:25 +00:00
Justin Chadwell
88221133af solver: add cap for multiple exporters
Higher-level clients should be able to check server-side caps to ensure
that when they attempt to use multiple exporters, that the feature is
actually supported.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:04:28 +00:00
Tõnis Tiigi
444cb34af7 Merge pull request #3426 from benlongo/fix-typo-in-exec-mount-bind-readwrite-nooutput
Fix typo in CapExecMountBindReadWriteNoOutput
2022-12-19 22:07:12 -08:00
Ben Longo
5896c5477c Fix typo in CapExecMountBindReadWriteNoOutput.
Signed-off-by: Ben Longo <benlongo9807@gmail.com>
2022-12-16 23:19:02 -05:00
Akihiro Suda
0d3f71ee18 llbsolver: support pinning sources
Alternative to PR 2816 ("dockerfile: support Dockerfile.pin for pinning sources")

This version is implemented on the llbsolver side and agnostic to the LLB frontends.
See `solver/llbsolver/vertex.go:loadLLB()`.

See `docs/build-repro.md` for the usage.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-13 13:01:54 -08:00
Justin Chadwell
d81928a72f solver: delete caps for additional oci properties
We don't need multiple caps for a single feature - the caps that these
were copied by were addded over time, we don't immediately need all of
them to start with, only the main feature one is initially required.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 11:38:50 -08:00
Justin Chadwell
6b265b1bae gateway: add RemoveMountStubsRecursive to exec meta options
This allows a frontend to request a specific for stubs removal.

By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.

This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:06:47 +00:00
Tonis Tiigi
ec0b7b9905 exporter: add SOURCE_DATE_EPOCH support for image exporter
Allows reproducible timestamps for layer and image timestamps.

Implemented as a frontend-opt because in the future same
option could be detected by frontend for custom behavior
and same value should also apply timestamps for FileOps.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-10-12 17:29:01 -07:00
Justin Chadwell
3c64e516ce attestations: enable attestations cap
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-07 17:24:19 +01:00
Justin Chadwell
d392e45ec0 attestations: fixup from review
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-07 17:23:46 +01:00
Pranav Pandit
522573b29c [remotecache] Add Azure Blob Storage support
This adds experimental support for Azure Blob Storage based remote cache to buildkit. For usage instructions please refer to the updated Readme. We have tried to keep it similar to the S3 based implementation while using equivalent Azure Blob Storage specific semantics where appropriate.
This also adds end-to-end tests to exercise the Azure Blob Storage based cache using the [Azurite emulator](https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azurite).

Co-authored-by: Amr Mahdi <amrh@microsoft.com>
Co-authored-by: Pranav Pandit <pranavp@microsoft.com>
Signed-off-by: Pranav Pandit <pranavp@microsoft.com>
2022-08-25 06:20:22 -07:00
Justin Chadwell
d76e401d26 solver: add attestations capability
This will eventually clients to query the server to confirm if
attestations are supported. However, for now, we can keep it disabled
until the gateway api stabilizes.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-11 14:27:04 +01:00
Tõnis Tiigi
a6a114a1a4 Merge pull request #2879 from jedevc/annotations-support
Add annotations support to output
2022-06-16 18:05:17 -07:00
Justin Chadwell
28b9f1e743 solver: add capability for annotations export
Allows clients to query if annotations exporting is supported.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-06-16 13:25:24 +01:00
Avi Deitcher
1fe66075b7 Add OCI source
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2022-06-15 23:04:11 +03:00
Bertrand Paquet
09c5a7c0ee Add s3 remote cache
Signed-off-by: Bertrand Paquet <bertrand.paquet@gmail.com>
2022-05-13 13:13:37 +02:00
Tonis Tiigi
5ac05cb482 Allow pulling partial layer chains from an image
Adds a new `WithLayerLimit` option to `llb.Image`
only pulls specified number of layers instead of
full image.

This can be used in combination with DiffOp/MergeOp
to pull any subset of layers from an image in any order.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-04-21 21:54:13 -07:00
Tonis Tiigi
30a0b02479 exec: allow mounting secret environment variables
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-31 11:22:57 -08:00
Tonis Tiigi
dc21885891 hack: enable more linters
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-19 12:20:30 -08:00
Erik Sipsma
8c1e411d01 Add DiffOp support to solver and client.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-01-06 11:05:51 -08:00
Erik Sipsma
43a74261b3 Remove symlink, not target, in FileOp.Rm.
Before this change, if the path provided to FileOp.Rm was a symlink then
the target of the symlink would be removed instead of the symlink
itself. Now, the symlink will be removed instead. However, any symlinks
present in the parent dirs of the specified path will still be resolved
before calling os.Remove; this change only results in the base of the
specified path not being followed.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-18 12:44:12 -08:00
Erik Sipsma
d73e62f878 Add initial MergeOp implementation.
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-18 11:10:48 -08:00
CrazyMax
c82ef274bf cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
CrazyMax
87e1fa7ecb add size to tmpfs mounts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-18 10:54:52 +02:00
CrazyMax
4e32ae5630 Add support for ulimit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-08 11:50:09 +02:00
Tonis Tiigi
b1c80cf0ab caps: add cap for gha cache backend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-09 20:49:26 -07:00
Tonis Tiigi
baa4fcdb0f add differ support for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:07 -07:00
Akihiro Suda
817955dc8d Merge pull request #2116 from tonistiigi/git-subdir
git: support subdir component
2021-06-08 14:27:50 +09:00
Aaron Lehmann
b27fc6169b Combine include/exclude caps into a single cap
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Aaron Lehmann
b545c281c1 Add capabilities for copy include/exclude patterns
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Tonis Tiigi
3bdb6b5159 git: support subdir component
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-19 22:13:04 -07:00
Alex Couture-Beil
f2c8eb177c Support cloning from non-standard git repos
- fixes assumption that ssh git clones must be via the `git` user.
- allows passing the SSH_AUTH_SOCK from the client to GitSource
- allows passing a known_host entry for ssh

Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-10 15:55:17 -08:00
Tonis Tiigi
c8190b1518 lint: enable golint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
Tonis Tiigi
49de675234 git: support for token authentication
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-16 17:00:06 -07:00
Tonis Tiigi
fc186a8b89 oci: mount whitelist of devices on insecure security mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-01-30 13:07:29 -08:00
Edgar Lee
d81832cf6c Detect support for protobuf array ref on server on frontend return call
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-01-14 17:16:21 -08:00
Edgar Lee
c122adacb5 Implement file action remove for wildcards
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2019-10-30 14:25:00 -07:00
Tonis Tiigi
872bf0b8c8 dockerfile: add cap for detecting correct nooutput support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-08 22:07:34 -07:00
Kunal Kushwaha
a2bbb5ff39 security entitlement support
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2019-03-27 13:57:03 +09:00
Tonis Tiigi
210679d35f client: set cap if fileop used
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 16:35:30 -07:00
Tonis Tiigi
8a4674bab4 fileop: add dockerfile support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:50 -07:00
Tonis Tiigi
ec0e352aae executor: mount cgroups in default spec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-02 23:22:07 -07:00
Akihiro Suda
758a267235 s/CapMount/CapExecMount/g
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:03:36 +09:00
Akihiro Suda
900a1b96c4 define CapMountSSH
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:00:10 +09:00
Ian Campbell
720d8327e2 Push setting of default PATH down into the executor(s)
Setting the default `PATH` in the `llb.State` on the client side means it
depends on the `GOOS` of the buildkit client, rather than of the environment
where it will actually execute.

Instead defer this to execution time and insert the default PATH at that point
if one is not present. Doing this in solver/llbsolver/ops/exec covers all
executors and also avoids breaking the cache.

Client compatibility is handled via a new capability.

Fixes #604

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-28 14:16:47 +01:00