1072 Commits

Author SHA1 Message Date
CrazyMax
409c8da50f exporter: revert attestation manifest push order
This reverts the change from #7012.

Referrer manifests may be pushed before their subject, and the OCI distribution spec requires registries to initially accept a manifest whose subject does not exist yet. Keep BuildKit's default push order aligned with that behavior instead of working around registries that reject this flow.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-08-04 00:03:02 +02:00
CrazyMax
067b18bd03 exporter: fix attestation manifest push order
OCI artifact attestations carry a subject reference to the image manifest.
Push manifests in an order that keeps child manifests before parent indexes
while also pushing any in-stack subject manifest before the attestation
that references it.

Add an integration test variant with a strict registry proxy to cover
registries that reject manifests whose subject doesn't already exist.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-08-03 11:39:07 +02:00
CrazyMax
b1eea810dc test: require image exporter for session finalize
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-07-27 18:41:37 +02:00
CrazyMax
cc977b306b test: avoid local exporter invalid mode integration hang
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-07-27 15:56:11 +02:00
Alberto Garcia Hierro
30a3c27393 exporter: add session export finalization
Add a FinalizeExport RPC to pass the combined exporter response back to
the session provider after all exports complete. This lets clients run
completion work while the build's gateway references remain available.

Keep the callback optional for compatibility with existing session
exporters. Ignore unsupported finalization, propagate callback failures,
and support providers that only register a finalization callback.

Signed-off-by: Alberto Garcia Hierro <alberto.hierro@docker.com>
2026-07-23 16:46:24 +01:00
CrazyMax
e89fccf347 cache/s3: only request required upload checksums
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-07-22 13:23:28 +02:00
Tonis Tiigi
93999f4071 sourcepolicy: normalize parsed source identifiers
Render parsed source identifiers back to their canonical SourceOp form before
source policy evaluation. This lets Git subdir cleanup use the existing source
parser and avoids policy-specific Git parsing.

Add String methods for source identifiers and cover them with unit tests, plus
a client integration regression for canonical Git subdir policy matching.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-07-22 11:36:47 +02:00
Sebastiaan van Stijn
53d4490a3b client: remove use of deprecated proxy.Director (staticcheck)
client/build_test.go:2363:14: SA1019: proxy.Director has been deprecated since Go 1.26 and an alternative has been available since Go 1.20: Use Rewrite instead. (staticcheck)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-07-09 12:45:57 +02:00
Tonis Tiigi
56cf3cc0b5 Revert "exporter: sanitize platform IDs in paths"
This reverts commit 77b65abf49.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-07-07 09:09:47 -07:00
Tonis Tiigi
38f8536ee3 client: split client integration tests by area
Move client integration test declarations out of the large harness files into
focused files for cache, exporters, sources, mounts, gateway behavior, and
related client features.

Keep registration lists grouped by source file, move non-gateway tests out of
the gateway harness, and remove the now-empty build_test.go. This is intended
as a mechanical split with no test logic changes.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-07-06 22:19:20 -07:00
Tõnis Tiigi
c130a532e6 Merge pull request #6910 from tonistiigi/platforms-id-sanitize
exporter: sanitize platform IDs in paths
2026-07-06 15:27:00 -07:00
Tonis Tiigi
77b65abf49 exporter: sanitize platform IDs in paths
Replace Windows path separators and drive separators when platform IDs are
used as local and tar exporter path components. Add a regression test for tar
exporter output generated from frontend-controlled platform metadata.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-07-02 17:24:55 -07:00
CrazyMax
9a16a73a42 exporter: default attestations to OCI artifacts
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-30 18:35:50 +02:00
Tonis Tiigi
64bbec89ca security: validate exec security modes
Reject unknown SecurityMode values before generating executor specs. Ensure
only SecurityMode_INSECURE takes the insecure path, while validated non-insecure
modes keep sandbox security options. Add gateway, raw LLB, and LLB marshal
regression coverage for invalid enum values.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 972895718f963c71388aeebb7cff423ef6963a92)
2026-06-24 10:56:46 -07:00
Tonis Tiigi
51dc1e9093 client: update loopback condition for dockerd worker
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-10 12:39:23 -07:00
Tonis Tiigi
e45a92397c client: update dockerd test conditions after oci-mediatypes default
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-10 10:58:45 -07:00
Tonis Tiigi
fc53e5e9d0 client: fix mergeop requirements in new test for dockerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-10 10:58:45 -07:00
Tõnis Tiigi
0f0aad9e38 Merge pull request #6859 from crazy-max/revert-6810
Revert "executor: fix containerd stdin close race"
2026-06-10 10:04:37 -07:00
CrazyMax
d13b3ac2ce Revert "executor: fix containerd stdin close race"
This reverts commit cb6df1c266.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-10 18:25:51 +02:00
Tonis Tiigi
cd33910d58 network: fix proxy default egress
Use the worker default provider for proxy UNSET egress so host fallback
is preserved when no CNI config is configured.

Extend proxy integration coverage across default, host, bridge, and
default-no-cni worker network modes.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-10 08:33:16 -07:00
CrazyMax
73c67671d3 exporter: use root-backed fsutil receive instead of client-side staging
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-10 12:24:31 +02:00
Tonis Tiigi
57649e9d9a client: add testExportLocalModeMultiPlatformKeepsAllPlatforms test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-10 12:13:15 +02:00
CrazyMax
00764d4f69 exporter: add local exporter mode=delete
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-10 12:13:15 +02:00
CrazyMax
7a2c61eefd Merge pull request #6810 from tonistiigi/containerd-exec-test-fix
test: stabilize worker exec pid1 lifetime
2026-06-10 11:18:19 +02:00
Tonis Tiigi
cb6df1c266 executor: fix containerd stdin close race
Delay containerd stdin CloseIO until BuildKit's stdin reader reaches EOF.
This avoids racing the shim FIFO copy path, where closing stdin before
the copy completes can leave execs stuck or missing stdin data.

Add a client gateway regression that runs concurrent execs writing stdin
into files and verifies every payload is present.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-09 17:28:39 -07:00
Tonis Tiigi
91cc422d5f network: split exec proxy from net mode
Keep exec network modes limited to sandbox, host, and none, and pass proxy
network configuration separately through solve and executor runtime state.

Proxy execs now use bridge-style egress by default, host egress only for host
network mode with entitlement, and no proxy for none mode. Add integration
coverage for bridge, host, and none proxy behavior across OCI and containerd
workers.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-09 09:11:33 -07:00
CrazyMax
63e37e87d1 Merge pull request #6815 from tonistiigi/stdin-fix
executor: avoid runc stdin shutdown hangs
2026-06-09 10:36:53 +02:00
Tonis Tiigi
e9ed800105 exporter: default docker tar to OCI types
Apply the oci-mediatypes default at export time so the Docker tar exporter
does not treat variant=docker as implicit oci-mediatypes=false.

Keep Docker tar manifest.json output for legacy compatibility, and preserve
explicit oci-mediatypes=false behavior with test coverage.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-08 22:20:08 -07:00
Tonis Tiigi
953437b102 executor: avoid runc stdin shutdown hangs
Forward non-tty stdin through an os.Pipe so runc receives an *os.File
instead of the caller's reader. This lets runc exit after the container
process is killed without waiting on Go's internal stdin copy.

Add gateway coverage for graceful pid1 exit, release-based cleanup, and
explicit SIGKILL while pid1 stdin is still open.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-08 18:10:52 -07:00
Tonis Tiigi
c15eb2679f llb: add passthrough op
Add a passthrough LLB op that builds all inputs and exposes selected
inputs as outputs. Add State.Requires as a client shortcut for requiring
dependencies without returning their snapshots.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-08 11:26:15 -07:00
Tonis Tiigi
6f08a4ab4a test: cover proxy network source conversion
Add integration coverage for exec proxy source policy conversion. The test
requests /foo, rewrites it to /bar, and verifies exported content and
provenance materials use the converted source.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-04 11:23:03 -07:00
Tonis Tiigi
d6973c12f6 solver: log proxy network requests
Record each proxied exec request and print a redacted method and URL list in
the exec progress logs after the process completes.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-04 11:23:02 -07:00
Tonis Tiigi
2bdf6abf99 network: capture proxy exec materials
Record successful GET responses through the exec proxy as provenance
materials and report incomplete material coverage as a typed solve error.
Thread proxy policy and capture state through typed executor/network options.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-04 11:23:02 -07:00
Tonis Tiigi
3254832159 solver: add proxy network mode
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.

Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.

Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-04 11:23:00 -07:00
Tõnis Tiigi
2ff20da8b4 Merge pull request #6824 from tonistiigi/oci-mediatypes
exporter/containerimage: default to oci-mediatypes=true
2026-06-04 08:45:50 -07:00
Tonis Tiigi
a3661ca077 test: avoid local source differ release race
Wait before the second solve so previous local source refs can be released
before the reuse assertion runs.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-03 13:45:01 -07:00
Tonis Tiigi
6147354ae5 exporter: add compatibility version 30
Add compatibility-version 30 for the new OCI media type default while keeping
version 20 pinned to the historical image exporter behavior through v0.31.x.

Track omitted oci-mediatypes separately from explicit false, add v30
compatibility goldens, and document the supported compatibility versions.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-03 08:52:37 -07:00
Tonis Tiigi
c4ce866e0f exporter: fix OCI media type test coverage
Allow annotation and attestation exports with OCI media types enabled, fix
the zstd media type map, and update integration tests for the new default.
Keep Docker media types explicit where legacy behavior is under test.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-03 08:52:31 -07:00
CrazyMax
ed3dbc0390 Merge pull request #6823 from tonistiigi/intoto-simplified
attestation: migrate emitted in-toto statements to v1
2026-06-03 17:43:08 +02:00
Tõnis Tiigi
174f4f07dd Merge pull request #6760 from crazy-max/session-empty-fallback
resolver: allow anonymous auth while preserving local image fallback
2026-06-03 08:40:56 -07:00
Bjorn Neergaard
c9f1690554 all: use well-known zstd mediaType constants
When support for zstd was introduced, these mediaTypes were not yet
available in released versions of their respective Go packages. That is
no longer the case.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2026-06-02 23:11:12 -07:00
CrazyMax
bb7ed5c529 attestation: update tests and docs for in-toto v1 statements
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-02 23:04:19 -07:00
Tõnis Tiigi
fbb261b445 Merge pull request #6791 from tonistiigi/fix-http-sigsegv
http: fix resolver cache nil race
2026-06-02 11:22:50 -07:00
Tonis Tiigi
fa1ba86389 source/http: fix resolver cache ref lookup
Use the resolver-cache ref ID directly when loading a cached HTTP snapshot.
This avoids dereferencing missing metadata for cache entries found during a
concurrent solve.

Reset the unauthorized checksum race request counter per solve iteration so
each retry applies the intended first-request delay.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-01 21:12:23 -07:00
Tonis Tiigi
9721df1d32 solver: fix slow cache error result ownership
Clone the existing slow-cache input result when wrapping errors instead of
creating a separate shared owner. This keeps SlowCacheError.Result tied to
the same release chain and avoids releasing the input while the error still
holds it.

Update the containerd cleanup check to ignore empty leases while still
failing on leases that retain resources, and add a deterministic regression.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-06-01 13:50:57 -07:00
CrazyMax
ca9bcf049c containerimage: preserve local fallback for default image resolution
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-05-29 17:17:04 +02:00
Jiří Moravčík
f85c740c2b solver: add per-step CPU and memory resource limits
Add support for setting cgroup resource limits (memory, memory-swap,
cpu-shares, cpu-period, cpu-quota, cpuset-cpus, cpuset-mems) on
individual build steps.

Signed-off-by: Jiří Moravčík <jiri.moravcik@gmail.com>
2026-05-27 11:34:13 +02:00
Tonis Tiigi
10872e1722 http: fix resolver cache nil race
Avoid storing typed nil HTTP metadata in the resolver cache after failed
metadata resolution. Also make resolver cache release functions consistently
callable when no error is returned.

Add a client integration test covering concurrent same-URL HTTP sources with
and without checksum when the server returns 401.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-05-20 12:48:07 -07:00
Tõnis Tiigi
550cf8b8b7 Merge pull request #6774 from tonistiigi/0513-dev-tools-updates
dev tools updates
2026-05-14 17:37:14 -07:00
Tonis Tiigi
98e1113c6d lint: modernize Go lint findings
Update non-generated code for the newer lint recommendations by using typed
atomic values, strings.Cut, and slices.Backward where applicable.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-05-14 10:13:21 -07:00