Commit Graph

82 Commits

Author SHA1 Message Date
Tonis Tiigi
09648f4d29 replace WithTimeout with WithTimeoutCause
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:12 -08:00
Tonis Tiigi
8a2a3e83ec replace context.WithCancel with WithCancelCause
Keep stack traces for cancellation errors where possible.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:06 -08:00
Erik Sipsma
fa11bf9e57 executor: fix resource sampler goroutine leak
Before this, the runc executor did not close the cgroupRecord when the
container exited non-zero, which resulted in goroutines leaking.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-07-31 07:52:54 -07:00
Tonis Tiigi
6a2f92d08f resources: add sampler for periodic stat reads
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-08 15:51:35 -07:00
Tonis Tiigi
963f16179f resources: CNI network usage sampling support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-08 15:51:35 -07:00
Tonis Tiigi
6e87e4b455 resources: add build step resource tracking via cgroups
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-08 15:51:31 -07:00
Erik Sipsma
9b0bdb6006 Prevent overflow of runc exit code.
It's possible for the Status field of runc.ExitError to be set to -1, in
which case conversion to uint32 results in the error message to say that
the container exited with code 4294967295 (2^32-1).

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-03-31 19:31:55 -07:00
coryb
6073f5821e runc worker: fix sigkill handling
This fixes the incorrect kill handling introduced in
b76f8c0248.  We need to send the
SIGKILL to the in-container process, not the runc process.  This patch
adds an abstraction over the kill handling:
  * for `runc run` processes use `runc kill`
  * for `runc exec` processes, read pid (in host PID namespace) from
    pidfile created by `runc exec`, then send the signal directly to
    that process.
Also use the kill abstraction when we receive a SIGKILL over the
signal channel for containers created by gateway NewContainer

Signed-off-by: coryb <cbennett@netflix.com>
2023-03-31 12:56:41 -07:00
coryb
b76f8c0248 fix process termination handling for runc exec
This patch makes the process handling consistent between runc.Run and
runc.Exec usage.  Previously runc.Run would use context.Background
for the runc.Run process and would monitor the request context for
shutdown requests, sending a SIGKILL to the container pid1 process. This
allowed runc.Run to gracefully shutdown and reap child processes.  This
logic was not used for runc.Exec where instead we were passing in the
request context to runc.Exec, and if that request context was cancelled
the runc process would immediately terminate preventing runc from reaping
the child process.  In this scenario the extra pid will remain forever
and then when the pid1 process will get wedged in zap_pid_ns_processes
syscall upon shutdown waiting fo the zombie pid to exit.

With this fix both runc.Run and runc.Exec will use context.Background
for runc processes and monitor the request context for shutdown request
triggering a SIGKILL to the pid being monitored by runc.

Signed-off-by: coryb <cbennett@netflix.com>
2023-03-17 12:46:19 -07:00
coryb
a8aa7b200c use bklog.G(ctx) instead of logrus directly
Signed-off-by: coryb <cbennett@netflix.com>
2023-03-15 12:07:44 -07:00
Justin Chadwell
6b265b1bae gateway: add RemoveMountStubsRecursive to exec meta options
This allows a frontend to request a specific for stubs removal.

By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.

This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:06:47 +00:00
Akihiro Suda
bd57e5f6b8 Make SELinux labels opt-in (--oci-worker-selinux=<BOOL>)
Fix issue 3202

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-11-02 14:33:01 +09:00
Aaron Lehmann
588a72d04c Add network namespace pool support
This adds netNSPoolSize pool options which allow setting a target
network namespace pool size. buildkitd will create this number of
network namespaces at startup (without blocking). When a container
execution finishes, the network namespace gets returned to the pool. If
the pool goes above the target size, there is a grace period to allow
network namespaces to be reused, and if this passes without reuse, the
extra namespaces will be released.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-14 07:08:31 -07:00
Alex Suraci
e3c9e064a9 cniprovider: pass hostname as K8S_POD_NAME arg
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
2022-08-21 15:43:12 -04:00
Edgar Lee
d21254e7f7 Add events for exec op
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2022-03-02 12:05:35 -08:00
Cory Bennett
559d079902 Allow signals to be sent to gateway exec containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2022-02-02 16:57:10 +00:00
CrazyMax
c82ef274bf cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
Aaron Lehmann
890c3f77b6 Allow ExitError type to be transmitted over GRPC
This will allow clients to retrieve exit error codes returned during a
solve without parsing the error messages.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-28 15:01:05 -07:00
Morlay
18b49fd7dc refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
Tonis Tiigi
d512920c24 support collecting traces from llb.Exec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
Brian Goff
d9834f872c Add support for apparmor/selinux
Set's an apparmor profile on the OCI spec if one is configred on the
worker.
Adds selinux labels to containers (only added if selinux is enabled on
the system).

This assumes that the specified apparmor profile is already loaded on
the system and does not try to load it or even check if it is loaded.

SELinux support requires the `selinux` build tag to be added.
Likewise, `runc` would require both the `apparmor` and `selinux` build
tags.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Vendored go-selinux to v1.8.0
Fixed tests

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 68bb095353)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-04 22:01:34 +01:00
Cory Bennett
9d3f55c400 move ExitError from solver/errdefs to frontend/gateway/errdefs
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-12-07 01:09:59 +00:00
Sebastiaan van Stijn
67bcfe699a copy containerd.UnknownExitStatus to local const
Copy this const to a local constant to prevent importing the containerd
client in the front-end.

For consistency, I also updated the executor code to use the same const,
although not strictly needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-18 00:11:09 +01:00
Tonis Tiigi
bdcee17437 executor: change mount to struct
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
Tonis Tiigi
e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Akihiro Suda
dda009a58c Merge pull request #1739 from tonistiigi/empty-layer
clear file mount stubs and fix empty layer cases
2020-10-21 00:42:45 +09:00
Cory Bennett
b464f1e315 update go-runc to use runc.ExitError for container exec status
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-20 06:44:29 +00:00
Edgar Lee
5eaecb905c Merge pull request #1731 from coryb/issue-1714
add tty support for runc executor
2020-10-19 14:52:31 -07:00
Tonis Tiigi
17b4ca1edf executor: remove stub files from rootfs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-18 23:39:17 -07:00
Cory Bennett
e308ef7874 add tty support for runc executor
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-16 17:36:56 +00:00
Lu Jingxiao
5e7ae230b2 Add hostname specifying for building
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
Cory Bennett
355e937e15 update gateway to add ability to run and exec into containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-31 21:13:59 +00:00
Cory Bennett
93344a9d24 remove *pixel from winsize struct, tweak ExitError handling for ctx.Err
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 20:22:12 +00:00
Cory Bennett
4b456f17f4 wrap errors from executor Run/Exec to allow access to exit code
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 19:14:30 +00:00
Sebastiaan van Stijn
f26452a847 executor/oci.GetUser(): remove unused context
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-30 11:31:03 +02:00
Tonis Tiigi
57a88b0afb lint: add misspell and unused
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-18 08:13:57 -07:00
Cory Bennett
5909d1642e simplify done channel handling, fix other pr comments.
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-11 01:14:37 +00:00
Cory Bennett
5e91dff4ed fix error handling for exec when container fails to start
update run/exec tests for stdin and expected failures
move common tests for runc and container to shared tests package

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-10 22:06:42 +00:00
Cory Bennett
6d58121c11 Update Executor interface for Run and Exec
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-09 23:40:36 +00:00
Tonis Tiigi
90288ab716 errdefs: update to new packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-24 11:25:44 -07:00
Tonis Tiigi
0ef435687b runcexecutor: update go-runc call after contract change
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-09-11 10:57:51 -07:00
Akihiro Suda
5559a45319 Merge pull request #1139 from tonistiigi/oomscoreadj
runcexecutor: avoid setting oomscoreadj from main process
2019-08-19 15:17:59 +09:00
Tonis Tiigi
a0dead0809 fix possible double release on mountable
Refactor the interface to avoid such issues in the future.

BuildKit own mounts are stateless and not affected but
a different mountable implementation could get confused.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-16 08:03:16 -07:00
Tonis Tiigi
25ebc26d21 runcexecutor: avoid setting oomscoreadj from main process
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-15 23:16:02 -07:00
Tonis Tiigi
2cd19dbc34 executor: ignore workdir if already exists
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-26 16:28:43 -07:00
Tibor Vass
8e692507ee executor: allow setting DNSConfig to be used by resolvconf
This patch allows downstream code to pass a DNSConfig that is
then used by executor/oci.GetResolvConf.

This would allow the BuildKit-based builder in Docker to honor
the docker daemon's DNS configuration, thus fixing a feature gap
with the legacy builder.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-17 20:43:33 +00:00
Tonis Tiigi
7b41906d89 executor: create hosts and resolv.conf with userns root
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-10 16:35:03 -07:00
Tonis Tiigi
e7ad765fcd executor: create directories as 711
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-10 16:34:09 -07:00
Tonis Tiigi
858b4c7076 executor: make sure cwd created with correct user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-11 22:11:26 -07:00
Akihiro Suda
eba3ffc268 runexecutor: support NoPivot
This flag allows DOCKER_BUILDKIT to be running on minikube host: https://github.com/kubernetes/minikube/issues/4143

moby/builder/build-next needs to be updated to set NoPivot when
$DOCKER_RAMDISK is set.

This flag is deprecated from its birth and not exposed to the standalone buildkitd OCI worker.
See https://github.com/kubernetes/minikube/issues/3512

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-05-07 23:22:55 +09:00