In this case the current stack trace points to the line
where the context was created. Instead the stack should be
captured when the defer is running so the return path to
the defer call is also part of the stack.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
When GracefulStop is called gRPC waits for current requests to finish
before closing. While this is generally the behavior we want, it is
not always same for the History.Listen endpoint. That endpoint is
usually open even if buildkit is not actively processing any builds,
because client may be waiting for new events.
The new logic is that if GracefulStop will happen, history will
close active listeners if there are no active builds. If there are
active builds then active listeners will be closed after all the
active builds have completed their finalizers.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Current implementation based on leases.SynchronousDelete only works
with the containerd worker and is ignored otherwise. This means that
blobs referenced by history records were left on disc until the
periodic background GC was initialized later.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Update default policy to include maximum and free storage controls.
New default policy is combination of all three controls.
Minimum reserved storage: 10GB / 10% (10% was old default)
Maintain free storage: 20%
Maximum allowed storage: 100GB / 80%
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
vtproto is an extra protobuf compiler that generates special methods
suffixed with `VT` that create typed and unrolled marshal and unmarshal
functions similar to gogo that can be used for performance sensitive
code. These extensions are optional for code to use but buildkit uses
them.
A codec is also included to utilize vtproto for grpc code. If the
package `github.com/moby/buildkit/util/grpcutil/encoding/proto` is
imported then vtproto will be used if it exists and otherwise it will
use the standard marshaling and unmarshaling methods.
This codec has an important difference from the default codec. The
default codec will always reset messages before unmarshaling. In most
cases, this is unnecessary and is only relevant for `RecvMsg` on
streams. In most cases, if we are passing in an existing message to this
method, we want to reuse the buffers. This codec will always merge the
message when unmarshaling instead of resetting the input message.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Use golang.org/x/exp/trace to implement an trace recorder that saves the trace
to a circular buffer and can be retrieved at any time.
Debug endpoints have been added under /debug/flight to start and stop the trace
as well as to set its period.
Due to golang.org/x/exp/trace, the minimum go version has been bumped to 1.22
Signed-off-by: Alberto Garcia Hierro <damaso.hierro@docker.com>
appdefaults_unix.go, constants_unix.go:
/run is a thing only for Linux. Other Unixes use /var/run.
diffapply_linux.go and source_linux.go:
These files use Linux-only API.
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
Commit 4b85f11164 migrated the use of the
userns package to the github.com/moby/sys/user module.
After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).
This patch migrates our code to use the new module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].
The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.
[1]: 3778ae603c
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
address #5056
Split out the code for `getRuntimeOptionsType` so as
to reduce the unnecessary imports, hence binary size.
Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
Using the "runc.v2" runtime, it is possible to configure containerd to start
runc with the "systemd_cgroup" flag. This will cause runc to use systemd to
manage the container cgroups. For this configuration to work, runc needs the
cgroup name to be of a special form: "<systemd.slice>:<parent>:<name>".
This is already implemented in the containerd runtime package, provided that
a parent cgroup of the form "<systemd.slice>:<parent>:" is set.
This commit adds the option to configure such a parent cgroup for the
containerd worker. By default, it will still use an empty string as cgroup
parent, keeping existing behaviour.
Using a configuration like:
[worker.containerd]
defaultCgroupParent = "system.slice:buildkit:"
[worker.containerd.runtime]
name = "io.containerd.runc.v2"
[worker.containerd.runtime.options]
SystemdCgroup = true
a user is able to have their container cgroups managed by systemd. This makes
it possible to set global resource constraints on a per-container basis using
systemd drop-in configuration. When using the example above, the following file
restricts every container spawned by buildkit to use only 1 CPU and 1G of RAM:
$ cat /etc/systemd/system/buildkit-.scope.d/limits.conf
[Scope]
CPUQuota=100%
MemoryMax=1G
Signed-off-by: Moritz "WanzenBug" Wanzenböck <moritz@wanzenbug.xyz>
Because detecting emulator changes can be relatively
expensive, avoid doing it very frequently. A new config
parameter allows controlling if users prefer more or
less frequent updates or want to disable detecting
changes completely and only rely on emultor configuration
at boot time.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This commit adds [frontend."dockerfile.v0"] and
[frontend."gateway.v0"] buildkitd.toml configuration sections. Each
frontend can individually be disabled by setting `enabled = false`
(both frontends are enabled by default).
The [frontend."gateway.v0"] section has an `allowedRepositories`
setting. If `allowedRepositories` is empty (the default), all gateway
sources are allowed. Otherwise, only repositories in the list will be
allowed. NOTE: Only the repository name (without tag) is compared.
Change-Id: Ia484401709ef6c13cf3e5a2e4d0e1c6bd0c47d13
Signed-off-by: Ahmon Dancy <adancy@wikimedia.org>
This refactors the detect package with the goal of making it more
similar to otel's `autoexport` package and splitting out the additional
functionality used by buildkit, like the trace recorder and delegated
tracer, to more explicit processors rather than implicit through
`autoexport`.
This removes the global variables for the trace provider and meter
provider along with the global variable for the exporters. This is
replaced with functions that create the exporters. The delegated tracer
has been removed from detect and moved into the normal tracing util
package. This is still used by the command line to send delegated
traces, but it's an explicit exporter that's added rather than implicit.
Some functions have been renamed mostly to force dependent packages to
change their usage rather than have a chance at incorrect usage because
the semantics changed.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This commit is a follow up of #4875, documenting the user of
`--group` flag and `grpc.SecurityDescriptor` toml config.
Also does a minor fix on the `help` for `--group` to specify
Windows named pipe alongside Unix socket.
Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
Initially we had the GC Policy for Windows use only
2 GB (2e9 bytes) of disk space and this was limiting
for some build scenarios that need more than that,
especially ServerCore images.
This commit makes the policy to use percentages
as it is on Linux. Also going for 20%, double that
of Linux, since Windows images tend to be larger.
fixes#4858docker/buildx#2411
Also, refactors the diskSize logic to simplify
it by bringing the `d.AsByte` function back from
platform specific files to `gcpolicy.go`.
Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
The otelgrpc interceptors were deprecated. This updates the areas where
these were used to use the stat handlers instead of the interceptors.
This helps with creating a single method for both unary and stream rpcs
and also ensures we aren't using a deprecated function for the future.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
