Replace the adaptive write watermark and hourly reclaimability checks
with two write-driven triggers. After committed writes, file size is
sampled at most every five minutes and a database at or above the size
watermark is checked for reclaimable space. Every writesPerCheck
writes, reclaimability is checked for any database above sizeWatermark,
so space released by deletions is found even when the file does not
grow. Idle databases are not polled.
After a successful compaction the size watermark is set to the
compacted size plus sizeGrowthPercent, with sizeWatermark as the
minimum, and persisted with the write counter. Compaction is pending
when either reclaim threshold is met. Share the threshold logic
between the scheduler, the compactor and the debug endpoint.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Expose database measurements through GET /debug/compaction and stream a single manual attempt through POST. Resolve requests against registered databases rather than opening user-supplied paths, reject cross-origin browser POSTs, and return busy measurements instead of blocking inspection behind maintenance.
Allow manual compaction whenever the debug listener is enabled, independently of automatic scheduling. Bypass the write watermark while retaining the idle period, reclaimability thresholds, free-space checks, and serialization across databases.
Cancel manual attempts cooperatively when writers arrive, clients disconnect, or the scheduler stops. Reject duplicate requests and deliver progress without blocking database maintenance on a slow HTTP client. Reuse the idle timer and share the scheduler's checkpoint cadence while waiting for a manual attempt.
Document the endpoints in the developer guide and test manual-only and combined modes, inspection during maintenance, origin checks, idle waiting, cancellation, duplicate requests, checkpoint timing, and shutdown.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Configure the otel-collector for `hack/compose` to use it for processing
traces and forwarding to jaeger. This simplifies the configuration and
also opens up the ability to configure additional behavior around traces
and metrics for debugging.
This also removes the automatic override file and the gitignore file. It
wasn't really working well for me and made it difficult to share
extensions to the development environment. I'm going to try something
different for that in a different change.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This introduces a debug variant of the buildkit docker image. This
version builds the binary in a way that disables optimizations so that
it can be run properly with delve. It also builds and installs delve
only in the debug variant of the image.
For the debug variant, a shim `buildkitd` binary is created that execs
with the proper arguments to delve and forwards to the underlying
binary.
In order to use this version, you can set `--build-arg BUILD_VARIANT=debug`
and it will pick up this version of the image. The default is `release`.
The debug variant is only available for linux and the variant is ignored
for any other target os. Most other targets don't support `buildkitd` at
all so the only os this affects is freebsd.
delve is exposed on port 5000 by default and this can be accessed from
the host by using `-p 5000:5000`. If there is a port conflict on the
host, `-p XXXX:5000` can be used instead. It's also highly recommended
that you do `-p 127.0.0.1:5000:5000` instead. This is because docker
defaults to exposing ports to any interface which exposes the debug port
to the open internet. This is generally a good practice for running
buildkit on a tcp address anyway and the same applies to delve.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This gives a new example for the LLB case, explaining the purpose of
what both the selectors and the content-based caching function can be
used for.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Tonis Tiigi