Commit Graph

67 Commits

Author SHA1 Message Date
Tõnis Tiigi
e0220af506 Merge pull request #3254 from corhere/c8dexecutor-inprocess-lifecycle-hook
[RFC] containerdexecutor: add network namespace callback
2022-12-22 12:44:31 -08:00
Cory Snider
b5fdf90aee containerdexecutor: add network namespace callback
In order to support identity mapping and user namespaces, the Moby
project needs to defer the creation of a container's network namespace
to the runtime and hook into the container lifecycle to configure the
network namespace before the user binary is started. The standard way to
do so is by configuring a `createRuntime` OCI lifecycle hook, in which
the OCI runtime executes a specified process in the runtime environment
after the container has been created and before it is started. In the
case of Moby the network namespace needs to be configured from the
daemon process, which necessitates that the hook process communicate
with the daemon process. This is complicated and slow. All the hook
process does is inform the daemon of the container's PID and wait until
the daemon has finished applying the network namespace configuration.

There is an alternative to the `createRuntime` OCI hook which containerd
clients can take advantage of. The `container.NewTask` method is
directly analogous to the OCI create operation, and the `task.Start`
method is directly analogous to the OCI start operation. Any operations
performed between the `NewTask` and `Start` calls are therefore directly
analogous to `createRuntime` OCI hooks, without needing to execute any
external processes! Provide a mechanism for network.Namespace instances
to register a callback function which can be used to configure a
container's network namespace instead of, or in addition to,
`createRuntime` OCI hooks.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-08 19:46:23 -05:00
Justin Chadwell
6b265b1bae gateway: add RemoveMountStubsRecursive to exec meta options
This allows a frontend to request a specific for stubs removal.

By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.

This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:06:47 +00:00
Cory Snider
f448313827 containerdexecutor: clean up task if Start() fails
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows) a
process is spawned when the task is created, and deleting the task while
the process is running would leak the process if it was allowed.
Change the deferred `task.Delete` call to pass the `WithProcessKill`
delete option so the cleanup has a chance to succeed in the event that
the `p.Start` call inside `runProcess` returns an error.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-02 18:29:34 -04:00
Akihiro Suda
bd57e5f6b8 Make SELinux labels opt-in (--oci-worker-selinux=<BOOL>)
Fix issue 3202

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-11-02 14:33:01 +09:00
Aaron Lehmann
588a72d04c Add network namespace pool support
This adds netNSPoolSize pool options which allow setting a target
network namespace pool size. buildkitd will create this number of
network namespaces at startup (without blocking). When a container
execution finishes, the network namespace gets returned to the pool. If
the pool goes above the target size, there is a grace period to allow
network namespaces to be reused, and if this passes without reuse, the
extra namespaces will be released.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-14 07:08:31 -07:00
Alex Suraci
e3c9e064a9 cniprovider: pass hostname as K8S_POD_NAME arg
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
2022-08-21 15:43:12 -04:00
Sebastiaan van Stijn
21e9e9641e Remove uses of deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-10 00:57:07 +01:00
Edgar Lee
d21254e7f7 Add events for exec op
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2022-03-02 12:05:35 -08:00
Kohei Tokunaga
21aeba6fd6 Support rootless containerd worker
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2022-02-21 15:39:14 +09:00
Cory Bennett
559d079902 Allow signals to be sent to gateway exec containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2022-02-02 16:57:10 +00:00
Tõnis Tiigi
539be17089 Merge pull request #2428 from tonistiigi/default-user-groups
executor: make sure supplementary groups are set for unset user
2021-12-14 17:02:09 -08:00
CrazyMax
c82ef274bf cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
Tonis Tiigi
e82ccdf958 containerdexecutor: fix setting user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-27 11:26:10 -07:00
Aaron Lehmann
890c3f77b6 Allow ExitError type to be transmitted over GRPC
This will allow clients to retrieve exit error codes returned during a
solve without parsing the error messages.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-28 15:01:05 -07:00
Morlay
18b49fd7dc refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
Akihiro Suda
1a7543a105 Merge pull request #2163 from tonistiigi/detect-otel-env 2021-07-03 01:01:34 +09:00
Cory Bennett
13c842bab7 ensure containerd io is complete and closed before returning
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-06-28 15:39:36 +00:00
Tonis Tiigi
d512920c24 support collecting traces from llb.Exec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
Brian Goff
d9834f872c Add support for apparmor/selinux
Set's an apparmor profile on the OCI spec if one is configred on the
worker.
Adds selinux labels to containers (only added if selinux is enabled on
the system).

This assumes that the specified apparmor profile is already loaded on
the system and does not try to load it or even check if it is loaded.

SELinux support requires the `selinux` build tag to be added.
Likewise, `runc` would require both the `apparmor` and `selinux` build
tags.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Vendored go-selinux to v1.8.0
Fixed tests

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 68bb095353)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-04 22:01:34 +01:00
Cory Bennett
9d3f55c400 move ExitError from solver/errdefs to frontend/gateway/errdefs
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-12-07 01:09:59 +00:00
Sebastiaan van Stijn
67bcfe699a copy containerd.UnknownExitStatus to local const
Copy this const to a local constant to prevent importing the containerd
client in the front-end.

For consistency, I also updated the executor code to use the same const,
although not strictly needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-18 00:11:09 +01:00
Tonis Tiigi
bdcee17437 executor: change mount to struct
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
Tonis Tiigi
e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Tonis Tiigi
17b4ca1edf executor: remove stub files from rootfs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-18 23:39:17 -07:00
Lu Jingxiao
5e7ae230b2 Add hostname specifying for building
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
Cory Bennett
355e937e15 update gateway to add ability to run and exec into containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-31 21:13:59 +00:00
Cory Bennett
19c0077b49 update container resize events in sequence, also move it out of exit/cancel loop to prevent blocking.
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-03 01:37:38 +00:00
Cory Bennett
86e246a874 only warn on resize errors
prevent resize from blocking exit
fix edgecase where kill signal never reaches process

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-01 22:40:57 +00:00
Cory Bennett
f781f83a89 fix containerd executor Run/Exec to close container input on eof from stdin
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-01 17:21:40 +00:00
Cory Bennett
93344a9d24 remove *pixel from winsize struct, tweak ExitError handling for ctx.Err
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 20:22:12 +00:00
Cory Bennett
4b456f17f4 wrap errors from executor Run/Exec to allow access to exit code
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 19:14:30 +00:00
Sebastiaan van Stijn
f26452a847 executor/oci.GetUser(): remove unused context
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-30 11:31:03 +02:00
Cory Bennett
5909d1642e simplify done channel handling, fix other pr comments.
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-11 01:14:37 +00:00
Cory Bennett
5e91dff4ed fix error handling for exec when container fails to start
update run/exec tests for stdin and expected failures
move common tests for runc and container to shared tests package

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-10 22:06:42 +00:00
Cory Bennett
6d58121c11 Update Executor interface for Run and Exec
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-09 23:40:36 +00:00
Paul "TBBle" Hampson
b9cf317850 Distinguish containerd failure from process exit code
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-01-30 00:39:11 +11:00
Tonis Tiigi
a0dead0809 fix possible double release on mountable
Refactor the interface to avoid such issues in the future.

BuildKit own mounts are stateless and not affected but
a different mountable implementation could get confused.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-16 08:03:16 -07:00
Tonis Tiigi
2cd19dbc34 executor: ignore workdir if already exists
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-26 16:28:43 -07:00
Tibor Vass
8e692507ee executor: allow setting DNSConfig to be used by resolvconf
This patch allows downstream code to pass a DNSConfig that is
then used by executor/oci.GetResolvConf.

This would allow the BuildKit-based builder in Docker to honor
the docker daemon's DNS configuration, thus fixing a feature gap
with the legacy builder.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-17 20:43:33 +00:00
Tonis Tiigi
7b41906d89 executor: create hosts and resolv.conf with userns root
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-10 16:35:03 -07:00
Tonis Tiigi
858b4c7076 executor: make sure cwd created with correct user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-11 22:11:26 -07:00
Tonis Tiigi
9f53ea3d78 userns support for sources and executor
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:26:04 -07:00
Kunal Kushwaha
a2bbb5ff39 security entitlement support
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2019-03-27 13:57:03 +09:00
Akihiro Suda
c54f4a986d support --oci-worker-no-process-sandbox
Note that this mode allows build executor containers to kill (and potentially ptrace) an arbitrary process in the BuildKit host namespace.
This mode should be enabled only when the BuildKit is running in a container as an unprivileged user.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-08 10:42:52 +09:00
Tonis Tiigi
76692bbe5f executor: clean up static config files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-03 13:59:33 -07:00
Tonis Tiigi
f99352fee1 solver: make sure to return proper canceled errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-26 17:38:16 -07:00
Tonis Tiigi
0940cdc6fe update golint comments
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:06:47 -07:00
Michael Crosby
8eed5bfd15 Provide nil stdin to containerd when not required
This allows builds that inspect stdin to not block and hang forever.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-09-11 14:22:22 -07:00
Anda Xu
b0677e7ef1 allow customized cgroup-parent for runcexecutor
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-05 12:53:57 -07:00