This migrates uses of github.com/opencontainers/runc/libcontainer/user
to the new github.com/moby/sys/user module, which was extracted from
runc at commit [opencontainers/runc@a3a0ec4].
This is the initial release of the module, which is a straight copy, but
some changes may be made in the next release (such as fixing camel-casing
in some fields and functions (Uid -> UID).
[opencontainers/runc@a3a0ec4]: a3a0ec48c4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This change splits the containerdexecutor.Run() function into smaller
pieces and enables it to run on Windows.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Before this, the runc executor did not close the cgroupRecord when the
container exited non-zero, which resulted in goroutines leaking.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Buildkit code is mostly generic enough to support FreeBSD, however
there are some quirks / infrastructural pieces that need to be
addressed for full support, to name some
- contenthash.NewFromStat attempts to set Devmajor / Devminor for
regular files, assuming that RDev is zero for regular
files. Unlike on Linux, it's not the case for FreeBSD.
- containerdexecutor.Run uses bind mounts for rootfs. Bind mounts
are not supported in FreeBSD and we should use nullfs instead
- There is no CI job to run tests on FreeBSD
- Some dependencies weren't ported
This change ports buildkit to FreeBSD
Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Protect against data race occuring while both sampler.Run and sub.Close
accesses the same data from multiple threads.
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
It's possible for the Status field of runc.ExitError to be set to -1, in
which case conversion to uint32 results in the error message to say that
the container exited with code 4294967295 (2^32-1).
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This fixes the incorrect kill handling introduced in
b76f8c0248. We need to send the
SIGKILL to the in-container process, not the runc process. This patch
adds an abstraction over the kill handling:
* for `runc run` processes use `runc kill`
* for `runc exec` processes, read pid (in host PID namespace) from
pidfile created by `runc exec`, then send the signal directly to
that process.
Also use the kill abstraction when we receive a SIGKILL over the
signal channel for containers created by gateway NewContainer
Signed-off-by: coryb <cbennett@netflix.com>
This patch makes the process handling consistent between runc.Run and
runc.Exec usage. Previously runc.Run would use context.Background
for the runc.Run process and would monitor the request context for
shutdown requests, sending a SIGKILL to the container pid1 process. This
allowed runc.Run to gracefully shutdown and reap child processes. This
logic was not used for runc.Exec where instead we were passing in the
request context to runc.Exec, and if that request context was cancelled
the runc process would immediately terminate preventing runc from reaping
the child process. In this scenario the extra pid will remain forever
and then when the pid1 process will get wedged in zap_pid_ns_processes
syscall upon shutdown waiting fo the zombie pid to exit.
With this fix both runc.Run and runc.Exec will use context.Background
for runc processes and monitor the request context for shutdown request
triggering a SIGKILL to the pid being monitored by runc.
Signed-off-by: coryb <cbennett@netflix.com>
Port https://github.com/moby/moby/blob/v23.0.1/daemon/oci_linux.go#L430-L460
> // Get the set of mount flags that are set on the mount that contains the given
> // path and are locked by CL_UNPRIVILEGED. This is necessary to ensure that
> // bind-mounting "with options" will not fail with user namespaces, due to
> // kernel restrictions that require user namespace mounts to preserve
> // CL_UNPRIVILEGED locked flags.
Fix issue 3098
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
In order to support identity mapping and user namespaces, the Moby
project needs to defer the creation of a container's network namespace
to the runtime and hook into the container lifecycle to configure the
network namespace before the user binary is started. The standard way to
do so is by configuring a `createRuntime` OCI lifecycle hook, in which
the OCI runtime executes a specified process in the runtime environment
after the container has been created and before it is started. In the
case of Moby the network namespace needs to be configured from the
daemon process, which necessitates that the hook process communicate
with the daemon process. This is complicated and slow. All the hook
process does is inform the daemon of the container's PID and wait until
the daemon has finished applying the network namespace configuration.
There is an alternative to the `createRuntime` OCI hook which containerd
clients can take advantage of. The `container.NewTask` method is
directly analogous to the OCI create operation, and the `task.Start`
method is directly analogous to the OCI start operation. Any operations
performed between the `NewTask` and `Start` calls are therefore directly
analogous to `createRuntime` OCI hooks, without needing to execute any
external processes! Provide a mechanism for network.Namespace instances
to register a callback function which can be used to configure a
container's network namespace instead of, or in addition to,
`createRuntime` OCI hooks.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This allows a frontend to request a specific for stubs removal.
By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.
This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Follows-up removing directory stub mounts by following up the parent
chain, and removing any additionally created directories.
Signed-off-by: Justin Chadwell <me@jedevc.com>
On Linux, an empty directory is usually 4096 bytes, not 0, so we need an
additional explicit check here.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows) a
process is spawned when the task is created, and deleting the task while
the process is running would leak the process if it was allowed.
Change the deferred `task.Delete` call to pass the `WithProcessKill`
delete option so the cleanup has a chance to succeed in the event that
the `p.Start` call inside `runProcess` returns an error.
Signed-off-by: Cory Snider <csnider@mirantis.com>
We only need the content here, not the checksum, so simplifying the code by
just using os.ReadFile(), using resolvconf.Path() for the location.
Also reversing the logic for custom options; The existing code was always
parsing the host's resolv.conf to read the nameservers, searchdomain and
options, but those options were only needed if they were not configured in
DNSConfig. This patch reverses the logic to only parse the resolv.conf if
no options are present in the passed DNSConfig.
There's some more optimisations to make (but changes in libnetwork are needed
for that); resolvconf.FilterResolvDNS() still parse the resolv.conf file, even
if we just generated it (in which case we're generating a resolv.conf, after
which we're parsing the generated file again to update it, which is not ideal).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds netNSPoolSize pool options which allow setting a target
network namespace pool size. buildkitd will create this number of
network namespaces at startup (without blocking). When a container
execution finishes, the network namespace gets returned to the pool. If
the pool goes above the target size, there is a grace period to allow
network namespaces to be reused, and if this passes without reuse, the
extra namespaces will be released.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
