Commit Graph

81 Commits

Author SHA1 Message Date
Justin Chadwell
7e6c20a0db exec: allow specifying non-zero exit codes for execs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-09-17 11:36:07 +01:00
Alano Terblanche
c3925da302 Refactor containerd NewWorkerOpt & containerdexecutor New parameters
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
2024-06-25 10:24:47 +02:00
Tonis Tiigi
4103099d94 ensure context.WithoutCancel in defer funcs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-06-12 19:18:32 -07:00
CrazyMax
b5506327cf keep local dns in resolv.conf when host network enabled
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-09 09:27:39 +01:00
Tonis Tiigi
09648f4d29 replace WithTimeout with WithTimeoutCause
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:12 -08:00
Tonis Tiigi
8a2a3e83ec replace context.WithCancel with WithCancelCause
Keep stack traces for cancellation errors where possible.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:06 -08:00
Justin Chadwell
148e1ea413 containerd: support custom shim path
Like CRI, we should support custom shim paths. We can just add this as a
new config field "path" under the runtime object.

While "name" can be either the name of a custom shim (from which a path
is derived), or a path itself, if configured as a path, no options can
be provided - this is because to derive the type for the options struct,
we need the name to be a member of a well-known set (currently runc and
hcs).

With this patch, it's now possible to configure a runtime with a custom
shim at a non-default path, and include it's options (which was
previously not possible to do in buildkit).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-07 11:18:51 +00:00
Justin Chadwell
116d0153f5 chore: update getTaskOpts to return multiple opts
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-07 11:16:51 +00:00
Justin Chadwell
8bfd280ab7 Add options to specify containerd runtime
Co-authored-by: Marat Radchenko <marat@slonopotamus.org>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-26 14:20:35 +01:00
Gabriel Adrian Samfira
cc5657c600 Cleanup args, rename jobDetails
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-30 13:40:43 +03:00
Gabriel Adrian Samfira
b6b322b327 Implement executor on Windows
This change splits the containerdexecutor.Run() function into smaller
pieces and enables it to run on Windows.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-15 12:35:45 +03:00
Artem Khramov
c415d85fe0 FreeBSD port
Buildkit code is mostly generic enough to support FreeBSD, however
there are some quirks / infrastructural pieces that need to be
addressed for full support, to name some

-    contenthash.NewFromStat attempts to set Devmajor / Devminor for
    regular files, assuming that RDev is zero for regular
    files. Unlike on Linux, it's not the case for FreeBSD.

-    containerdexecutor.Run uses bind mounts for rootfs. Bind mounts
    are not supported in FreeBSD and we should use nullfs instead

-    There is no CI job to run tests on FreeBSD

-    Some dependencies weren't ported

This change ports buildkit to FreeBSD

Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
2023-07-28 17:02:53 +03:00
Tonis Tiigi
6e87e4b455 resources: add build step resource tracking via cgroups
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-08 15:51:31 -07:00
coryb
a8aa7b200c use bklog.G(ctx) instead of logrus directly
Signed-off-by: coryb <cbennett@netflix.com>
2023-03-15 12:07:44 -07:00
Tõnis Tiigi
e0220af506 Merge pull request #3254 from corhere/c8dexecutor-inprocess-lifecycle-hook
[RFC] containerdexecutor: add network namespace callback
2022-12-22 12:44:31 -08:00
Cory Snider
b5fdf90aee containerdexecutor: add network namespace callback
In order to support identity mapping and user namespaces, the Moby
project needs to defer the creation of a container's network namespace
to the runtime and hook into the container lifecycle to configure the
network namespace before the user binary is started. The standard way to
do so is by configuring a `createRuntime` OCI lifecycle hook, in which
the OCI runtime executes a specified process in the runtime environment
after the container has been created and before it is started. In the
case of Moby the network namespace needs to be configured from the
daemon process, which necessitates that the hook process communicate
with the daemon process. This is complicated and slow. All the hook
process does is inform the daemon of the container's PID and wait until
the daemon has finished applying the network namespace configuration.

There is an alternative to the `createRuntime` OCI hook which containerd
clients can take advantage of. The `container.NewTask` method is
directly analogous to the OCI create operation, and the `task.Start`
method is directly analogous to the OCI start operation. Any operations
performed between the `NewTask` and `Start` calls are therefore directly
analogous to `createRuntime` OCI hooks, without needing to execute any
external processes! Provide a mechanism for network.Namespace instances
to register a callback function which can be used to configure a
container's network namespace instead of, or in addition to,
`createRuntime` OCI hooks.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-08 19:46:23 -05:00
Justin Chadwell
6b265b1bae gateway: add RemoveMountStubsRecursive to exec meta options
This allows a frontend to request a specific for stubs removal.

By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.

This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:06:47 +00:00
Cory Snider
f448313827 containerdexecutor: clean up task if Start() fails
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows) a
process is spawned when the task is created, and deleting the task while
the process is running would leak the process if it was allowed.
Change the deferred `task.Delete` call to pass the `WithProcessKill`
delete option so the cleanup has a chance to succeed in the event that
the `p.Start` call inside `runProcess` returns an error.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-02 18:29:34 -04:00
Akihiro Suda
bd57e5f6b8 Make SELinux labels opt-in (--oci-worker-selinux=<BOOL>)
Fix issue 3202

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-11-02 14:33:01 +09:00
Aaron Lehmann
588a72d04c Add network namespace pool support
This adds netNSPoolSize pool options which allow setting a target
network namespace pool size. buildkitd will create this number of
network namespaces at startup (without blocking). When a container
execution finishes, the network namespace gets returned to the pool. If
the pool goes above the target size, there is a grace period to allow
network namespaces to be reused, and if this passes without reuse, the
extra namespaces will be released.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-14 07:08:31 -07:00
Alex Suraci
e3c9e064a9 cniprovider: pass hostname as K8S_POD_NAME arg
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
2022-08-21 15:43:12 -04:00
Sebastiaan van Stijn
21e9e9641e Remove uses of deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-10 00:57:07 +01:00
Edgar Lee
d21254e7f7 Add events for exec op
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2022-03-02 12:05:35 -08:00
Kohei Tokunaga
21aeba6fd6 Support rootless containerd worker
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2022-02-21 15:39:14 +09:00
Cory Bennett
559d079902 Allow signals to be sent to gateway exec containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2022-02-02 16:57:10 +00:00
Tõnis Tiigi
539be17089 Merge pull request #2428 from tonistiigi/default-user-groups
executor: make sure supplementary groups are set for unset user
2021-12-14 17:02:09 -08:00
CrazyMax
c82ef274bf cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
Tonis Tiigi
e82ccdf958 containerdexecutor: fix setting user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-27 11:26:10 -07:00
Aaron Lehmann
890c3f77b6 Allow ExitError type to be transmitted over GRPC
This will allow clients to retrieve exit error codes returned during a
solve without parsing the error messages.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-28 15:01:05 -07:00
Morlay
18b49fd7dc refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
Akihiro Suda
1a7543a105 Merge pull request #2163 from tonistiigi/detect-otel-env 2021-07-03 01:01:34 +09:00
Cory Bennett
13c842bab7 ensure containerd io is complete and closed before returning
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-06-28 15:39:36 +00:00
Tonis Tiigi
d512920c24 support collecting traces from llb.Exec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
Brian Goff
d9834f872c Add support for apparmor/selinux
Set's an apparmor profile on the OCI spec if one is configred on the
worker.
Adds selinux labels to containers (only added if selinux is enabled on
the system).

This assumes that the specified apparmor profile is already loaded on
the system and does not try to load it or even check if it is loaded.

SELinux support requires the `selinux` build tag to be added.
Likewise, `runc` would require both the `apparmor` and `selinux` build
tags.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Vendored go-selinux to v1.8.0
Fixed tests

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 68bb095353)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-04 22:01:34 +01:00
Cory Bennett
9d3f55c400 move ExitError from solver/errdefs to frontend/gateway/errdefs
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-12-07 01:09:59 +00:00
Sebastiaan van Stijn
67bcfe699a copy containerd.UnknownExitStatus to local const
Copy this const to a local constant to prevent importing the containerd
client in the front-end.

For consistency, I also updated the executor code to use the same const,
although not strictly needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-18 00:11:09 +01:00
Tonis Tiigi
bdcee17437 executor: change mount to struct
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
Tonis Tiigi
e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Tonis Tiigi
17b4ca1edf executor: remove stub files from rootfs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-18 23:39:17 -07:00
Lu Jingxiao
5e7ae230b2 Add hostname specifying for building
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
Cory Bennett
355e937e15 update gateway to add ability to run and exec into containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-31 21:13:59 +00:00
Cory Bennett
19c0077b49 update container resize events in sequence, also move it out of exit/cancel loop to prevent blocking.
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-03 01:37:38 +00:00
Cory Bennett
86e246a874 only warn on resize errors
prevent resize from blocking exit
fix edgecase where kill signal never reaches process

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-01 22:40:57 +00:00
Cory Bennett
f781f83a89 fix containerd executor Run/Exec to close container input on eof from stdin
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-01 17:21:40 +00:00
Cory Bennett
93344a9d24 remove *pixel from winsize struct, tweak ExitError handling for ctx.Err
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 20:22:12 +00:00
Cory Bennett
4b456f17f4 wrap errors from executor Run/Exec to allow access to exit code
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 19:14:30 +00:00
Sebastiaan van Stijn
f26452a847 executor/oci.GetUser(): remove unused context
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-30 11:31:03 +02:00
Cory Bennett
5909d1642e simplify done channel handling, fix other pr comments.
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-11 01:14:37 +00:00
Cory Bennett
5e91dff4ed fix error handling for exec when container fails to start
update run/exec tests for stdin and expected failures
move common tests for runc and container to shared tests package

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-10 22:06:42 +00:00
Cory Bennett
6d58121c11 Update Executor interface for Run and Exec
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-09 23:40:36 +00:00