Delay containerd stdin CloseIO until BuildKit's stdin reader reaches EOF.
This avoids racing the shim FIFO copy path, where closing stdin before
the copy completes can leave execs stuck or missing stdin data.
Add a client gateway regression that runs concurrent execs writing stdin
into files and verifies every payload is present.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Keep exec network modes limited to sandbox, host, and none, and pass proxy
network configuration separately through solve and executor runtime state.
Proxy execs now use bridge-style egress by default, host egress only for host
network mode with entitlement, and no proxy for none mode. Add integration
coverage for bridge, host, and none proxy behavior across OCI and containerd
workers.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Record successful GET responses through the exec proxy as provenance
materials and report incomplete material coverage as a typed solve error.
Thread proxy policy and capture state through typed executor/network options.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.
Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.
Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Update non-generated code for the newer lint recommendations by using typed
atomic values, strings.Cut, and slices.Backward where applicable.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Use os.OpenRoot for resolv.conf and hosts state file creation, and
adapt executor callers and tests to the root-relative helper API.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add executor.ValidContainerID and enforce it in runc/containerd Run paths.
Only runc executor used the ID in filesystem operations.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 789df2422341960b7549d14ea475add43e73cd74)
Replace duplicate nopCloser, nopWriteCloser, and bufferCloser
type definitions across test and production code with the
shared iohelper.NopWriteCloser type.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Convert usages of `github.com/docker/docker/pkg/idtools` to
`github.com/moby/sys/user` in order to break the dependency between
buildkit and docker.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Like CRI, we should support custom shim paths. We can just add this as a
new config field "path" under the runtime object.
While "name" can be either the name of a custom shim (from which a path
is derived), or a path itself, if configured as a path, no options can
be provided - this is because to derive the type for the options struct,
we need the name to be a member of a well-known set (currently runc and
hcs).
With this patch, it's now possible to configure a runtime with a custom
shim at a non-default path, and include it's options (which was
previously not possible to do in buildkit).
Signed-off-by: Justin Chadwell <me@jedevc.com>
This change splits the containerdexecutor.Run() function into smaller
pieces and enables it to run on Windows.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Buildkit code is mostly generic enough to support FreeBSD, however
there are some quirks / infrastructural pieces that need to be
addressed for full support, to name some
- contenthash.NewFromStat attempts to set Devmajor / Devminor for
regular files, assuming that RDev is zero for regular
files. Unlike on Linux, it's not the case for FreeBSD.
- containerdexecutor.Run uses bind mounts for rootfs. Bind mounts
are not supported in FreeBSD and we should use nullfs instead
- There is no CI job to run tests on FreeBSD
- Some dependencies weren't ported
This change ports buildkit to FreeBSD
Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
In order to support identity mapping and user namespaces, the Moby
project needs to defer the creation of a container's network namespace
to the runtime and hook into the container lifecycle to configure the
network namespace before the user binary is started. The standard way to
do so is by configuring a `createRuntime` OCI lifecycle hook, in which
the OCI runtime executes a specified process in the runtime environment
after the container has been created and before it is started. In the
case of Moby the network namespace needs to be configured from the
daemon process, which necessitates that the hook process communicate
with the daemon process. This is complicated and slow. All the hook
process does is inform the daemon of the container's PID and wait until
the daemon has finished applying the network namespace configuration.
There is an alternative to the `createRuntime` OCI hook which containerd
clients can take advantage of. The `container.NewTask` method is
directly analogous to the OCI create operation, and the `task.Start`
method is directly analogous to the OCI start operation. Any operations
performed between the `NewTask` and `Start` calls are therefore directly
analogous to `createRuntime` OCI hooks, without needing to execute any
external processes! Provide a mechanism for network.Namespace instances
to register a callback function which can be used to configure a
container's network namespace instead of, or in addition to,
`createRuntime` OCI hooks.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This allows a frontend to request a specific for stubs removal.
By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.
This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows) a
process is spawned when the task is created, and deleting the task while
the process is running would leak the process if it was allowed.
Change the deferred `task.Delete` call to pass the `WithProcessKill`
delete option so the cleanup has a chance to succeed in the event that
the `p.Start` call inside `runProcess` returns an error.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This adds netNSPoolSize pool options which allow setting a target
network namespace pool size. buildkitd will create this number of
network namespaces at startup (without blocking). When a container
execution finishes, the network namespace gets returned to the pool. If
the pool goes above the target size, there is a grace period to allow
network namespaces to be reused, and if this passes without reuse, the
extra namespaces will be released.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
This will allow clients to retrieve exit error codes returned during a
solve without parsing the error messages.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
