Render parsed source identifiers back to their canonical SourceOp form before
source policy evaluation. This lets Git subdir cleanup use the existing source
parser and avoids policy-specific Git parsing.
Add String methods for source identifiers and cover them with unit tests, plus
a client integration regression for canonical Git subdir policy matching.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
When noCache=true, Platforms() appends to w.WorkerOpt.Platforms.
Concurrent gRPC calls to ListWorkers (each handled in a separate
goroutine) can cause a data race where one goroutine creates matchers
with length N, while another appends new platforms, causing the first
goroutine's range to exceed matchers bounds:
panic: runtime error: index out of range [81] with length 81
Add sync.Mutex to serialize access to Platforms().
Signed-off-by: okhowang(王沛文) <okhowang@tencent.com>
Keep exec network modes limited to sandbox, host, and none, and pass proxy
network configuration separately through solve and executor runtime state.
Proxy execs now use bridge-style egress by default, host egress only for host
network mode with entitlement, and no proxy for none mode. Add integration
coverage for bridge, host, and none proxy behavior across OCI and containerd
workers.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add a passthrough LLB op that builds all inputs and exposes selected
inputs as outputs. Add State.Requires as a client shortcut for requiring
dependencies without returning their snapshots.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Record successful GET responses through the exec proxy as provenance
materials and report incomplete material coverage as a typed solve error.
Thread proxy policy and capture state through typed executor/network options.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.
Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.
Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Update golangci-lint and adjust code for new gosec diagnostics. Use
root-scoped filesystem operations where appropriate, preserve explicit
user path behavior for SSH keys, and avoid background contexts in
request-scoped cleanup paths.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Support importing git sources from OCI or registry-backed bundle blobs
and exporting resolved checkouts as single-file git bundles.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Image blob source in LLB allows addressing a single blob
from a container image registry. The difference from the image
source is that image source needs to point to a manifest that
internally points to an array of layer blobs that are all extracted
on top of each other to form a root FS. Contrary, image blob
points to a single blob that is not extracted but downloaded
as a single file into an empty snapshot, similarily how
the HTTP source works.
The main use case for this source is to pin snapshots of
HTTP URLs, upload the downloaded blob into container registry,
and then use a source policy to map a HTTP URL (whose content
might be changed) to the copy of the source as image blob
to ensure immutability.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
These requests allow computing additional checksum algorithms
for the payload of HTTP sources.
Optionally suffix can be passed that is added to payload. This
is needed to make validation of PGP signatures possible remotely.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
ResolveImageConfig was changed to ResolveSourceMetadata long time
ago for cross-source implementation but the worker implementation
was still using old method name with conversions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Convert usages of `github.com/docker/docker/pkg/idtools` to
`github.com/moby/sys/user` in order to break the dependency between
buildkit and docker.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Current implementation based on leases.SynchronousDelete only works
with the containerd worker and is ignored otherwise. This means that
blobs referenced by history records were left on disc until the
periodic background GC was initialized later.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
We shouldn't use the cachemount root, we should actually properly
use the worker's specified root which is propagated from the config.
Signed-off-by: Justin Chadwell <me@jedevc.com>
On a build with no-cache, cache mounts were not pruned correctly
if the mount was on top of another ref. This also appeared in
Dockerfile when mode/uid/gid was set because implicit parent
ref is created in these cases in order to change the permissions
of a subdir that is used as a cache mount base.
Because it is not possible to know ahead of time what ref
will become the parent of cache mount during build, all cache
mounts matching the ID that have a parent will be pruned.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Because detecting emulator changes can be relatively
expensive, avoid doing it very frequently. A new config
parameter allows controlling if users prefer more or
less frequent updates or want to disable detecting
changes completely and only rely on emultor configuration
at boot time.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This is more versatile function that works for any source,
not just images.
It can be used together with a policy that switches
between input and output source as well as for adding
additional metadata for other sources in the future.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Before this change, ResolveImageConfig was unaware of source policies.
This means that:
1. Images for denied sources may be resolved
2. Image configs may get pulled for sources that are later converted to
a different image
The update makes it so the image resolver first runs a given ref through
the source policy and uses any mutated ref for the actual resolve
(instead of the original ref).
It also returns the mutated ref so it can be used correctly by the
frontend (e.g. don't want to do llb.Image(oldRef@resolvedDigest)).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Migrate history objects to separate namespace to holding
reference to a blob does not interfer with the GC labels
held for same blobs by the containerd image store.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
This centralizes the location where ProgressKey gets set, which works
because it only needs information about the vertex, nothing op-specific.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Before this, the worker's FromRemote method only partially intialized
progress controllers, leaving out the vertex digest. This meant that
only status updates would be sent and vertex start/stops would not be
sent.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Now, when a merge or diff ref is unlazied, the progress will show up
under the vertex for the merge/diff ref. Additionally, any ancestors of
the op that also need to be unlazied as part of unlazying the merge/diff
will show status updates under its vertex in the progress.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Working with strings is error-prone because a platform
can be in multiple string forms and less flexible.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
