70 Commits

Author SHA1 Message Date
Tonis Tiigi
64bbec89ca security: validate exec security modes
Reject unknown SecurityMode values before generating executor specs. Ensure
only SecurityMode_INSECURE takes the insecure path, while validated non-insecure
modes keep sandbox security options. Add gateway, raw LLB, and LLB marshal
regression coverage for invalid enum values.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 972895718f963c71388aeebb7cff423ef6963a92)
2026-06-24 10:56:46 -07:00
Jiří Moravčík
f85c740c2b solver: add per-step CPU and memory resource limits
Add support for setting cgroup resource limits (memory, memory-swap,
cpu-shares, cpu-period, cpu-quota, cpuset-cpus, cpuset-mems) on
individual build steps.

Signed-off-by: Jiří Moravčík <jiri.moravcik@gmail.com>
2026-05-27 11:34:13 +02:00
Anthony Nandaa
14acf4c23d wcow: leave default PATH env to be set by the OS
The PATH is very critical during container runs on
Windows. Windows stores the PATH details in its
registry hive, while in unix, this is often
stored in the image's config. See further details
at #5445

Setting a default path like we do on Linux
(which is mostly not used since the PATH is
already set in the configs), works against
the expected build experience, especially when
it comes to installers and commands like `setx`.

Therefore, we skip setting the default PATH on
Windows, and leave it for the OS to load it
from its registry hive.

This also further supports backward compatibilitiy
with the current experience with docker classic
builder.

Users wishing to explicitly store this in the
configs can opt-in by using the ENV PATH= ..
in the Dockerfile, etc.

See also the same practices on Docker Engine
and Containerd:
- da3b31fb2c/oci/defaults.go (L24-L33)
- 041743e8af/pkg/oci/spec_opts_windows.go (L66-L69)

closes #5445

Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
2025-05-10 18:51:38 +03:00
Tonis Tiigi
f6c142ec23 lint: update some sorting code to newer generics based libs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-03-20 10:32:02 -07:00
Tonis Tiigi
b5286f8dcb apply x/tools/modernize fixes
Autogenerated with couple of manual patches.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-03-07 08:18:45 -08:00
CrazyMax
6667434ec4 cdi: support optional devices
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2025-02-11 11:21:47 +01:00
CrazyMax
319bf56d8d exec: cdi device support
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2025-02-11 11:21:44 +01:00
Tonis Tiigi
30413b5de7 llb: avoid concurrent map write on parallel marshal
Calling marshal changes the internal state of the op, for example
addCap() helper adds capability constraints. These can race with
same map being read by another Marshal call. Locking the Marshal
function itself also makes sure that the cache is not recomputed
in this case.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-12-06 17:53:59 -08:00
Jonathan A. Sternberg
41a0a0c37d protobuf: add vtproto as a supplemental marshaler
vtproto is an extra protobuf compiler that generates special methods
suffixed with `VT` that create typed and unrolled marshal and unmarshal
functions similar to gogo that can be used for performance sensitive
code. These extensions are optional for code to use but buildkit uses
them.

A codec is also included to utilize vtproto for grpc code. If the
package `github.com/moby/buildkit/util/grpcutil/encoding/proto` is
imported then vtproto will be used if it exists and otherwise it will
use the standard marshaling and unmarshaling methods.

This codec has an important difference from the default codec. The
default codec will always reset messages before unmarshaling. In most
cases, this is unnecessary and is only relevant for `RecvMsg` on
streams. In most cases, if we are passing in an existing message to this
method, we want to reuse the buffers. This codec will always merge the
message when unmarshaling instead of resetting the input message.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-04 12:52:15 -05:00
Jonathan A. Sternberg
d59218e6e6 llb: deterministic marshaling for protobuf and store results from multiple constraints
This fixes a problem with the new protobuf marshaling with the standard
library. LLB digests are now forced into deterministic marshaling to
ensure they produce the same digest when marshaled multiple times.

In addition, the marshal cache has also been fixed to work in
multi-threaded frontends with multiple different constraints.
Previously, if an LLB vertex was used in multiple goroutines and
marshaled concurrently, the cache would be broken. This could cause
certain problems when a specific node was used multiple times in the
same LLB tree.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-02 11:17:45 -05:00
Jonathan A. Sternberg
1a3fc0aa15 protobuf: remove gogoproto
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-09-26 12:57:45 -05:00
Justin Chadwell
7e6c20a0db exec: allow specifying non-zero exit codes for execs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-09-17 11:36:07 +01:00
a-palchikov
96f746bca3 Implements frontend side of #2122.
This adds the syntax to Dockerfile frontend.
I purposedly chose to use a simple format for this as it's likely going
to be debated. As implemented, the following format is supported:

```
RUN --mount=type=secret,id=MYSECRET,env
```
or, more explicitly:

```
RUN --mount=type=secret,id=MYSECRET,env=true
```

will mount the secret with id MYSECRET as a new environment variable with the same name.

Using 'target', it's possible to create a different environment
variable:
```
RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env
```
will mount 'mysecret' secret as MY_SECRET environment variable.

Any suggestions on making it more ergonomic are welcome.

Signed-off-by: a-palchikov <deemok@gmail.com>
2024-08-19 17:50:48 +02:00
Justin Chadwell
157efce41a chore: set pb.Empty on ssh and secret mounts
These mounts have no vertex inputs, and so should be *explicitly*
marshaled as such.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-08-13 13:17:00 +01:00
adamperlin
e1aa0cecf2 Address review feedback and slightly refactor tests
Signed-off-by: adamperlin <adamp@nanosoft.com>
2024-03-04 22:03:57 +00:00
adamperlin
3d396e4525 Remove input ordering logic from Marshal(), enforce ordering in to (*FileOp).Inputs() and (*ExecOp).Inputs()
Signed-off-by: adamperlin <adamp@nanosoft.com>
2024-02-29 19:33:01 +00:00
Justin Chadwell
0eb25a6bf7 exec: allow caller-controlled content-based cache
This allows LLB-directed content-based cache enablement for each mount.

Some mounts may not be explicitly unabled (because it would be unsafe) -
for these cases we explicitly error out.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-06 11:44:25 +00:00
Brian Goff
206b3a1eeb Add some doc strings for LLB functions
This is just a first round of "would have been useful to me" sort of
documentation and examples for the LLB client API.
This is not in any way exhaustive.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-15 22:22:09 +00:00
Ben Longo
97a45ef6a4 Use SkipOutput instead of -1 for output indexes to clarify semantics.
Signed-off-by: Ben Longo <benlongo9807@gmail.com>
2022-12-16 23:01:13 -05:00
Justin Chadwell
6b265b1bae gateway: add RemoveMountStubsRecursive to exec meta options
This allows a frontend to request a specific for stubs removal.

By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.

This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:06:47 +00:00
Tonis Tiigi
30a0b02479 exec: allow mounting secret environment variables
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-31 11:22:57 -08:00
CrazyMax
c82ef274bf cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
CrazyMax
954c4721f7 use bytes as given size for tmpfs mount
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 19:47:17 +02:00
CrazyMax
87e1fa7ecb add size to tmpfs mounts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-18 10:54:52 +02:00
CrazyMax
4e32ae5630 Add support for ulimit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-08 11:50:09 +02:00
CrazyMax
6094339341 Add support for shm size
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-01 16:07:29 +02:00
Tonis Tiigi
e0124e192c llb: add constraints to async llb
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-22 21:34:43 -07:00
Tonis Tiigi
b4935eb79a llb: add constraints to vertex and validate
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-22 18:47:32 -07:00
Sebastiaan van Stijn
a743d4ba02 Add "ALL_PROXY" to list of default args / proxy env-vars
Relates to a82fff6377/docs/packages.md (proxies)

> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
>     ARG all_proxy

Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-24 18:19:02 +02:00
Tonis Tiigi
ecf070a027 exec: use platform specific default path
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-23 15:30:23 -07:00
Lu Jingxiao
5e7ae230b2 Add hostname specifying for building
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
Tonis Tiigi
c8190b1518 lint: enable golint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
Cory Bennett
2b462fc6ca skip tmpfs mounts when calculating indexes for llb.GetMount
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-06-04 10:54:32 -07:00
Edgar Lee
59fa9fc9a0 Allow multiple source locations to be added as a constraint
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 16:48:44 -07:00
Tonis Tiigi
6073e6cff3 llb: enable source tracking
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:03 -07:00
Tonis Tiigi
0ef961eb87 llb: make generic runoptions stateoptions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-05 22:30:58 -07:00
Tonis Tiigi
1a9d366b49 llb: asyncronous llb graph generation support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-03 14:55:10 -07:00
Tonis Tiigi
1ef979a7bb llb: fix excessive formatting
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-01 16:55:08 -07:00
Tonis Tiigi
d106163985 llb: fix security mode cap check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-27 08:57:36 -07:00
Tonis Tiigi
15ef4e2b8a dockerfile: allow content cache for rw mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 19:40:30 -07:00
Kunal Kushwaha
a2bbb5ff39 security entitlement support
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2019-03-27 13:57:03 +09:00
Tõnis Tiigi
c9dc8352a4 Merge pull request #656 from AkihiroSuda/cap-ssh
define CapMountSSH
2018-10-02 23:21:17 -07:00
Akihiro Suda
758a267235 s/CapMount/CapExecMount/g
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:03:36 +09:00
Akihiro Suda
f542300a10 client/llb: fix cap marshalling
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:02:08 +09:00
Akihiro Suda
900a1b96c4 define CapMountSSH
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:00:10 +09:00
Akihiro Suda
fa92effc4b dockerfile: add support for RUN --mount=type=ssh
* Needs to be compiled with `dfrunmount dfssh`
* Implemented options:
 * `type`(required): needs to be `ssh`
 * `target`(optional): the socket path in the container
 * `id`(optional): id

Test script:

	#!/bin/bash
	set -exu -o pipefail
	REF=localhost:5000/dfssh:latest
	ssh-add -l
	sudo buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=frontend/dockerfile/cmd/dockerfile-frontend \
	  --frontend-opt "build-arg:BUILDTAGS=dfrunmount dfssh" \
	  --exporter=image --exporter-opt name=$REF --exporter-opt push=true
	mkdir -p /tmp/foo
	cd /tmp/foo
	cat << EOF > Dockerfile
	# syntax=$REF
	FROM alpine
	RUN apk add --no-cache openssh-client
	RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
	RUN --mount=type=ssh ssh git@gitlab.com
	# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here
	EOF
	sudo buildctl build --ssh default=$SSH_AUTH_SOCK --progress=plain --frontend=dockerfile.v0 --local context=. --local dockerfile=.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 14:53:43 +09:00
Ian Campbell
720d8327e2 Push setting of default PATH down into the executor(s)
Setting the default `PATH` in the `llb.State` on the client side means it
depends on the `GOOS` of the buildkit client, rather than of the environment
where it will actually execute.

Instead defer this to execution time and insert the default PATH at that point
if one is not present. Doing this in solver/llbsolver/ops/exec covers all
executors and also avoids breaking the cache.

Client compatibility is handled via a new capability.

Fixes #604

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-28 14:16:47 +01:00
Tonis Tiigi
6888956557 sshforward: implement ssh socket forwarding
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 08:57:06 -07:00
Tonis Tiigi
130f5f5ab0 solver: net host with basic entitlements support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:35 -07:00
Tonis Tiigi
f7359f7093 llb: add extrahost support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00