The first default GC policy is meant to prune the most easily
reproducible cache (local sources, cache mounts, git checkouts) once it
exceeds 512MB and has not been used for 48h. Its filter was written as a
single comma-joined string:
"type==source.local,type==exec.cachemount,type==source.git.checkout"
Each element of the Filters slice is passed to
containerd/filters.ParseAll, where commas within one string are ANDed
together. A record only ever has a single type, so the AND can never be
satisfied: this policy matched nothing and was effectively a no-op.
Split the filter into separate slice elements so they are ORed, matching
the documented intent and the array form already used in user
buildkitd.toml configs.
Signed-off-by: ZRHann <zrhann@foxmail.com>
Traces are now forwarded in a non-blocking goroutine when sent through
the traces exporter. This prevents traces forwarded from the client from
being stalled while waiting for an upstream uploader to appear.
In addition, adds a shutdown context to `appcontext` that will only
cancel when an interrupt has been received twice. One interrupt will
signal the program should clean up and shut down, the second indicates
we should skip shutdown procedures (more forceful), and the third will
indicate that we should immediately terminate the program.
This gives a bit more of a degree of control to shutdown procedures like
the traces and metrics exporter so there's a difference between forcibly
calling exit and just waiting a long time for the shutdown to happen.
Includes a more aggressive shutdown timeout for `buildctl` that is
similar to the export timeout on `docker-buildx` for the tracing
shutdown as another preventative measure to ensure the CLI hangs up at
an appropriate time interval.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Add a proxyNetwork TOML setting and --proxy-network daemon flag to enable
exec proxy enforcement for every build. Wire the default through controller
and solver setup while preserving per-build enablement.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.
Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.
Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
MkdirAll now chmods each created directory to the requested perm so a
non-zero process umask no longer leaves directories with the wrong mode.
buildkitd and buildctl set the umask to 0 at startup, so set the new
copy.UmaskIsZero flag in both to skip the now-redundant per-directory
chmod.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Threads the existing OTEL MeterProvider through llbsolver.Opt and emits
three build-event instruments from the recordBuildHistory finalizer:
- buildkit.builds (counter; labels: status, error_code)
- buildkit.builds.steps (counter; labels: kind)
- buildkit.build.duration (Base2 exponential histogram; labels: status)
The duration histogram uses an exponential aggregation, rendered as a
Prometheus native histogram by the existing exporter, to avoid the
"tens of millions of series" cardinality blow-up reported in #5777.
MeterProvider is passed explicitly through the constructor — buildkit
policy (per the #4957 review) prohibits relying on the OTel global
provider in library packages.
error_code uses gRPC codes.Code.String() for a bounded set;
rec.Error.Message is intentionally never used as a label. The frontend
label is intentionally omitted — client.Build clears req.Frontend on
the wire, so the field is empty for every caller that goes through the
gateway-client API (buildctl, buildx). The metric is forward-compatible
with a future buildkit change that populates rec.Frontend on that path.
A follow-up PR will add observable gauges for worker count and cache
state, plus an operator guide at docs/metrics.md.
Refs #1544; addresses discussion #5777.
Signed-off-by: Ava Barron <abarron@coreweave.com>
Adds a new field to the system configuration that sets the maximum
concurrency for registry connections.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This adds a way to set the log level of buildkitd outside of using
`--debug` or `--trace` which allows increasing the log level rather than
only lowering it.
The `--debug` and `--trace` options are now deprecated along with the
configuration options. A warning will be printed to the log when they
are used.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Expose the builtin Dockerfile frontend version in BuildKit version
APIs and buildctl debug output.
Move Dockerfile version logic into frontend/dockerfile/version and
validate that the builtin version constant matches release tags.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Merge the `tracing/env` and `tracing/exec` package into a single package
with some common types shared between the two.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Wrap finalize calls in inBuilderContext with the same vertex
ID used during export, so push status (layers and manifest)
appears under the "exporting to image" vertex in the progress
stream instead of being reported without a parent context.
This broke when pushing was moved to be parallel step after
main export phase.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Currently it is not possible to update to more
recent version as modernize causes errors in generated code.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This enables client to provide their own functionality for
AuthConfig lookup, rather than just passing in dockercli.ConfigFile.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
history.db currently lacks the automatic corruption recovery that
cache.db has, causing BuildKit to fail on startup if history.db is
corrupted. This is inconsistent since both databases are disposable
(losing history is inconvenient but not fatal).
This commit:
- Extracts the safe database opening logic to util/db/boltutil/SafeOpen
- Updates cache.db to use the shared SafeOpen function
- Applies the same recovery mechanism to history.db
The recovery mechanism backs up corrupted databases and creates fresh
ones, allowing BuildKit to start successfully even after abrupt
shutdowns or snapshot-related corruption (common with NoSync + network
block devices like Ceph RBD).
Fixes startup failures when history.db is corrupted, matching the
resilience already present for cache.db since commit ccc06b7ff.
Signed-off-by: Aditya Maru <aditya.maru@example.com>
Signed-off-by: Claude <adityamaru@gmail.com>
This PR adds comprehensive Windows container support to BuildKit's integration test suite,
enabling cross-platform testing while maintaining full Linux compatibility.
- Updated 35+ integration tests to work on both Windows and Linux
- Uses integration.UnixOrWindows() pattern for platform-specific logic
- Windows: nanoserver images, cmd.exe commands, C:\ paths, CRLF handling
- Linux: preserves existing alpine/busybox images and bash/sh commands
- Added platform-specific adaptations for cache, export, frontend, and CLI tests
- Files: 12 modified (+454/-151 lines)
- Coverage: Major test categories now cross-platform compatible
- Compatibility: 100% backward compatible, no breaking changes
- CI: Enables Windows testing in BuildKit pipeline
- Cache import/export (local, registry, multiple backends)
- Image exporters (containerd, OCI, tar, metadata)
- Dockerfile frontend parsing and builds
- buildctl CLI functionality
- Multi-stage builds and named contexts
Tests requiring POSIX features (uid/gid, tmpfs, file modes) remain Linux-only.
Signed-off-by: Dawei Wei <davwei@microsoft.com>
This allows buildkitd daemon to define additional fields
that are added to all the provenance attestations that
BuildKit creates (by default from /etc/buildkit/provenance.d/).
These custom fields can provide additional context about
the environment BuildKit itself is running (eg. Github workflow)
and are not allowed to collide with the trusted fields created
by BuildKit itself.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Grouping the link digests to parent side (links pointing
to current node) makes more sense than children side (links
pointing to next nodes). This also reduces the duplicates
in the output as previously same digest link could need
plaintext under multiple nodes.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
These endpoints show the contents of current boltdb
cache database together with debug plaintexts if
they have been saved.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This allows opt-in to cache key debug database on
daemon startup.
If enabled, all cache keys generated by builds are
saved into this database together with the plaintexts
of the original data so a reverse lookup can be performed
later to compare two checksums and find out their original
difference. If checksum contains other checksums internally
then these are saved as well. For storage constraints, the
plaintext of file content is not saved but the metadata
portion can be still looked up.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Previously, when no platforms were explicitly defined in the config,
serialization would emit an empty array instead of omitting the field.
When loading this cofig, this prevented the worker from falling back
to default platform detection during initialization. By using `omitempty`,
we ensure the field is excluded when empty, allowing defaults to be
correctly applied.
Fixes#5740.
Signed-off-by: Alberto Garcia Hierro <damaso.hierro@docker.com>
`tldir` flag handling now properly handles the old logic and the new
logic for cert-manager.io without failing.
Improved error message when files are missing.
Co-authored-by: Gleb Nebolyubov <gleb.nebo@gmail.com>
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
