Commit Graph

31 Commits

Author SHA1 Message Date
Tonis Tiigi
7fc428cdfc solver: release unreferenced cache keys after gc
Previously this routine only ran after user ran
prune command or on reboot of the daemon.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-11-20 22:49:54 -08:00
Tonis Tiigi
fbd08c6a42 allow skipping backlinks where walking cache chains for provenance
Backlink walking should not be needed to determine layers for
current build and should be safe to skip. This improves performance
of provenance creation for certain builds.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-05-30 21:26:38 -07:00
Paweł Gronowski
62365aa5e6 util/multiprovider: Implement Info
Require base provides to implement both content.InfoReaderProvider.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 23:16:22 +09:00
Tonis Tiigi
af63defb5f Revert "util/multiprovider: Implement Info"
This reverts commit 55afcdbc0d.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-02-23 14:40:02 -08:00
Paweł Gronowski
55afcdbc0d util/multiprovider: Implement Info
Require base provides to implement both content.InfoReaderProvider.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 18:42:35 +01:00
Justin Chadwell
8d6199a27c chore: update CacheExporterTarget docs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-25 17:33:20 +00:00
Erik Sipsma
45b19d34ef Improve cache related trace logging.
This adds trace logs to the solver's cache manager to assist debugging
cache misses+hits.

It also replace the LazyStackTrace struct with a function that returns a
stack trace string only if trace level is enabled, an empty string
otherwise. This makes it easier to use with logrus hooks that pass the
fields around to different goroutines without needing to try to "unlazy"
the fields first.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-08-07 07:25:40 -07:00
Kohei Tokunaga
085bd8af7f Fix cache cannot reuse lazy layers
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-01-24 16:23:56 +09:00
Tonis Tiigi
9acc6d30eb refactor buildinfo into provenance capture
Change how provenance information is captured from builds.

While previously frontend passed the buildinfo
sources with metadata, now all information is captured
through buildkit. A frontend does not need to implement
buildinfo and can't set incorrect/incomplete buildinfo
for a build result.

All LLB operations can now collect as much provenance
info as they like that will be used when making the
attestation. Previously this was limited to a single Pin
value. For example now we also detect secrets and SSH IDs
that the build uses, or if it accesses network, if local
sources are used etc.. The new design makes sure this
can be easily extended in the future.

Provenance capture can now detect builds that do
multiple separate subsolves in sequence. For example,
first subsolve gathers the sources for the build and
second one builds from immutable sources without a
network connection. If first solve does not participate
in final build result it does not end up in provenance.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-11-15 19:37:03 -08:00
Tonis Tiigi
a20e48f36d provenance: add layers support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-11-15 19:37:03 -08:00
CrazyMax
b4e37a867f buildinfo: refactor
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-14 21:55:18 +01:00
Erik Sipsma
0566b9a345 Add support for progress groups.
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.

The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-02-08 11:27:49 -08:00
Tonis Tiigi
cab33b1e31 exporter: support for compression-level
compression-level option can be set on export to
define the preferred speed vs compression ratio. The
value is a number dependent on the compression algorithm.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-01 15:21:46 -08:00
Kohei Tokunaga
f9e0346b34 Propagate compression options to the inline cache export
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-10-22 15:43:32 +09:00
CrazyMax
5fcc94454f Generate and embed build sources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-18 01:39:35 +02:00
Koichi Shiraishi
671ae38fb7 all: unify the specs-go package import alias to ocispecs
ocispecs means "O"pen "C"ontainer "I"nitiative image-spec/"specs"-go/v1
                      opencontainers          /image-spec/specs-go/v1

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:09 +09:00
Vlad A. Ionescu
b3cf7c43cf Switch to Acquire API.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-12 13:41:33 +03:00
Vlad A. Ionescu
489e17aea9 Add a configuration item to limit parallelism.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-10 15:47:59 +03:00
Tonis Tiigi
b4b0ece384 make sure result refs returned in errors are clone
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 10:49:38 -08:00
Tonis Tiigi
e86356ddf8 solver: add input preprocess capability
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-03 14:49:44 -08:00
Tonis Tiigi
e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Erik Sipsma
55cbd19dec Add support for lazily-pulled blobs in cache manager.
This allows the layers of images to only be pulled if/once they are actually
required.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Tonis Tiigi
2e9987ad16 session: track sessions with a group construct
Avoid hidden session passing and allow one session to drop when
multiple builds share a vertex.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:25 -07:00
Tonis Tiigi
8cfe2de889 solver: evaluate solve results lazily
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-05 15:29:43 -08:00
Edgar Lee
e8326b213b Fixup doc strings for solver types
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2019-11-07 10:00:20 -08:00
Edgar Lee
7846d924ff Improve solver type godocs
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2019-11-06 14:45:26 -08:00
Tonis Tiigi
ac4037a1d9 solver: keep original cache creation date when copying between stores
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-18 11:19:34 -08:00
Tonis Tiigi
97ea41ee6a solver: improve multi-key root caching
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-23 14:27:35 -07:00
Tonis Tiigi
71f5e6dd21 solver: rename solver-next to solver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi
77c2793ebb remove unused packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi
efde4f2340 worker, solver: update interfaces
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-18 16:20:59 +09:00