70 Commits

Author SHA1 Message Date
Tonis Tiigi
91cc422d5f network: split exec proxy from net mode
Keep exec network modes limited to sandbox, host, and none, and pass proxy
network configuration separately through solve and executor runtime state.

Proxy execs now use bridge-style egress by default, host egress only for host
network mode with entitlement, and no proxy for none mode. Add integration
coverage for bridge, host, and none proxy behavior across OCI and containerd
workers.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-09 09:11:33 -07:00
Tonis Tiigi
c15eb2679f llb: add passthrough op
Add a passthrough LLB op that builds all inputs and exposes selected
inputs as outputs. Add State.Requires as a client shortcut for requiring
dependencies without returning their snapshots.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-08 11:26:15 -07:00
Tonis Tiigi
3254832159 solver: add proxy network mode
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.

Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.

Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-04 11:23:00 -07:00
Jiří Moravčík
f85c740c2b solver: add per-step CPU and memory resource limits
Add support for setting cgroup resource limits (memory, memory-swap,
cpu-shares, cpu-period, cpu-quota, cpuset-cpus, cpuset-mems) on
individual build steps.

Signed-off-by: Jiří Moravčík <jiri.moravcik@gmail.com>
2026-05-27 11:34:13 +02:00
Jonathan A. Sternberg
ac10b4193f frontend: add required paths to LLB and use it with --parents
This adds an additional `RequiredPaths` that is primarily intended for
use with `COPY --parents`. This parameter specifies expected directories
or files that should exist when performing the checksum. A not found
error will be produced if one of these paths is missing.

This fixes an issue with `COPY --parents` where a non existent directory
that was intended to be copied would be ignored.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-13 14:38:38 -05:00
CrazyMax
6667434ec4 cdi: support optional devices
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2025-02-11 11:21:47 +01:00
CrazyMax
319bf56d8d exec: cdi device support
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2025-02-11 11:21:44 +01:00
Peter Engelbert
20c2d03697 Implement llb.Symlink
* Add file.symlink.create capability and wire it up
* Run codegen for new FileActionSymlink Message
* Add Symlink test
* Add user/group ownership and timestamps to symlink
 ** Symlinks have user/group ownership that are independent of those of the
    target file; in linux, the ownership of the symlink itself is only
    checked when the link resides in a directory with the sticky bit set and
    the link is the subject of removal or renaming. The sticky bit prevents
    files in the directory from being deleted or renamed by non-owners
    (members of the group that owns the file may not delete the file; the
    user must own the file).

    In addition to user/group restrictions, linux symlinks have timestamps
    that are independent of the timestamps on the target file.
* Expose symlink options to `llb` package
* Add symlink integration test
* Use tar exporter for tests
 ** Using the local exporter causes the files to be exported with the
    permissions of the user who does the exporting, instead of retaining
    their file permissions from within the container.

    Using the tar exporter instead preserves the permissions until they can
    be checked.
* Change symlink fields to `oldpath` and `newpath`
 ** Also run `make generated-files`
* Fix typo
* Add doc strings to exported `llb` identifiers
* Remove `requiresLinux` from integration test
* Revert "Remove `requiresLinux` from integration test"
* Add fixes to please the linter

* testFileOpSymlink: check that symlink is created
* Address comments for FileOp llb test
* This commit also fixes a couple of linter complaints.
* Add check for symlink type in tar header
* Address PR review nit

Signed-off-by: Peter Engelbert <pmengelbert@gmail.com>
2025-01-03 17:10:02 -05:00
Tonis Tiigi
25632fb074 client: allow non-octal chmod config for fileop.copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-10-01 12:52:38 -07:00
Jonathan A. Sternberg
1a3fc0aa15 protobuf: remove gogoproto
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-09-26 12:57:45 -05:00
Justin Chadwell
7e6c20a0db exec: allow specifying non-zero exit codes for execs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-09-17 11:36:07 +01:00
Erik Sipsma
7bc06cb0b7 add support for AlwaysReplaceExistingDestPaths in llb copy
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2024-04-02 12:11:06 -07:00
Justin Chadwell
0eb25a6bf7 exec: allow caller-controlled content-based cache
This allows LLB-directed content-based cache enablement for each mount.

Some mounts may not be explicitly unabled (because it would be unsafe) -
for these cases we explicitly error out.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-06 11:44:25 +00:00
Jonathan A. Sternberg
40fb5ce649 llbsolver: unmarshal protobuf objects into the provenance attestation correctly
This modifies how build steps are unmarshaled from JSON into the
provenance attestation. The current method doesn't correctly handle
protobuf attributes that are used with `oneof`.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-11-13 09:57:30 -06:00
Paul "TBBle" Hampson
98e0d8dcff Whenever copying OCI Platform data, include OSVersion and OSFeatures
Trivially created by looking for every reference to .Variant and adding
OSVersion and OSFeatures, except the ones related to the string
representation of a Platform instance.

I then went through and ensured every assignment of OSFeatures that
might leak out, i.e., not local-only or for marhsalling purposes, uses
the append-to-nil idiom to avoid sharing the slice storage and allowing
accidental mutation after-the-fact.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-11-03 12:19:29 +09:00
Justin Chadwell
c5402622bb lint: add protolint config
And tidy up the proto definitions to follow the new linting rules.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-10 13:11:18 +01:00
Tonis Tiigi
1553c2ad95 add language property for sourcemap
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-29 17:17:56 -07:00
Justin Chadwell
6b265b1bae gateway: add RemoveMountStubsRecursive to exec meta options
This allows a frontend to request a specific for stubs removal.

By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.

This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:06:47 +00:00
Tonis Tiigi
d1b0d8a7c6 subrequests: normalize json fields to lowercase
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-05 00:40:41 -07:00
Ce Gao
b150c29ac2 fix: Fix the comment
Signed-off-by: Ce Gao <ce.gao@outlook.com>
2022-03-17 14:46:40 +08:00
Erik Sipsma
cc1499e6d9 Add support for weak ProgressGroup membership.
A ProgressGroup will now only be displayed if a vertex member marked as
non-weak has started.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-02-25 10:24:15 -08:00
Erik Sipsma
0566b9a345 Add support for progress groups.
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.

The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-02-08 11:27:49 -08:00
Tonis Tiigi
30a0b02479 exec: allow mounting secret environment variables
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-31 11:22:57 -08:00
Erik Sipsma
8c1e411d01 Add DiffOp support to solver and client.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-01-06 11:05:51 -08:00
Erik Sipsma
d73e62f878 Add initial MergeOp implementation.
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-18 11:10:48 -08:00
CrazyMax
c82ef274bf cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
CrazyMax
87e1fa7ecb add size to tmpfs mounts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-18 10:54:52 +02:00
CrazyMax
4e32ae5630 Add support for ulimit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-08 11:50:09 +02:00
CrazyMax
6094339341 Add support for shm size
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-01 16:07:29 +02:00
Aaron Lehmann
7660af2f1d Add IncludePatterns and ExcludePatterns options for Copy
Allow include and exclude patterns to be specified for the "copy" op,
similarly to "local".

Depends on https://github.com/tonistiigi/fsutil/pull/101

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Sebastiaan van Stijn
a743d4ba02 Add "ALL_PROXY" to list of default args / proxy env-vars
Relates to a82fff6377/docs/packages.md (proxies)

> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
>     ARG all_proxy

Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-24 18:19:02 +02:00
Lu Jingxiao
5e7ae230b2 Add hostname specifying for building
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
Cory Bennett
355e937e15 update gateway to add ability to run and exec into containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-31 21:13:59 +00:00
Edgar Lee
7a90a36b46 Support multiple source maps and locations for source-level stacktraces
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 16:17:35 -07:00
Tonis Tiigi
d173e3dca8 pb: add more comments
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-19 17:23:03 -07:00
Tonis Tiigi
75d64ffb4a fix proto indentions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:08 -07:00
Tonis Tiigi
6073e6cff3 llb: enable source tracking
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:03 -07:00
Tonis Tiigi
1f9599aba3 llb: move source mapping to llb metadata
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:49:41 -07:00
Kunal Kushwaha
86c7933ac3 proto defination for security entitlements support
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2019-03-27 12:24:31 +09:00
Tonis Tiigi
c6149da2eb fileop: review fixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi
a443cfff05 fileop: resolve review comments
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
Tonis Tiigi
89e6614b38 solver: change uid to uint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
Tonis Tiigi
431d11dda3 llb: add timestamp override to fileop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
Tonis Tiigi
5b4841f308 llb: initial fileop implementation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:01 -07:00
Tonis Tiigi
3938aebdfc llb: make sure to use stable marshaler
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-17 15:19:04 -07:00
Tonis Tiigi
6888956557 sshforward: implement ssh socket forwarding
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 08:57:06 -07:00
Tonis Tiigi
130f5f5ab0 solver: net host with basic entitlements support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:35 -07:00
Tonis Tiigi
4945fe758c llbsolver: add support for extra host records
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
Tonis Tiigi
2da2aff164 llbsolver: add secret mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-19 22:02:05 -07:00
Tonis Tiigi
d4c57aec5d llb: add caps support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-18 17:22:30 -07:00