Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
On a build with no-cache, cache mounts were not pruned correctly
if the mount was on top of another ref. This also appeared in
Dockerfile when mode/uid/gid was set because implicit parent
ref is created in these cases in order to change the permissions
of a subdir that is used as a cache mount base.
Because it is not possible to know ahead of time what ref
will become the parent of cache mount during build, all cache
mounts matching the ID that have a parent will be pruned.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is more versatile function that works for any source,
not just images.
It can be used together with a policy that switches
between input and output source as well as for adding
additional metadata for other sources in the future.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Ensure interactive calls validate same conditions that
the build requests do. Refactor of the build side is to ensure
we use the same validation function for both cases. There
was no validation issue with the LLB validation.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit d1970522d7145be5f4a1f1a028b1910bb527126c)
Running interactive container APIs was done by giving
the gateway implementation access to worker controller
directly, but it should be passed with a build job instead.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 0971dffaab93d91e51af984b44c745b35b3c5b4d)
Before this change, ResolveImageConfig was unaware of source policies.
This means that:
1. Images for denied sources may be resolved
2. Image configs may get pulled for sources that are later converted to
a different image
The update makes it so the image resolver first runs a given ref through
the source policy and uses any mutated ref for the actual resolve
(instead of the original ref).
It also returns the mutated ref so it can be used correctly by the
frontend (e.g. don't want to do llb.Image(oldRef@resolvedDigest)).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Alternative to PR 2816 ("dockerfile: support Dockerfile.pin for pinning sources")
This version is implemented on the llbsolver side and agnostic to the LLB frontends.
See `solver/llbsolver/vertex.go:loadLLB()`.
See `docs/build-repro.md` for the usage.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Change how provenance information is captured from builds.
While previously frontend passed the buildinfo
sources with metadata, now all information is captured
through buildkit. A frontend does not need to implement
buildinfo and can't set incorrect/incomplete buildinfo
for a build result.
All LLB operations can now collect as much provenance
info as they like that will be used when making the
attestation. Previously this was limited to a single Pin
value. For example now we also detect secrets and SSH IDs
that the build uses, or if it accesses network, if local
sources are used etc.. The new design makes sure this
can be easily extended in the future.
Provenance capture can now detect builds that do
multiple separate subsolves in sequence. For example,
first subsolve gathers the sources for the build and
second one builds from immutable sources without a
network connection. If first solve does not participate
in final build result it does not end up in provenance.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Always use the value of the evaluate field to force result generation,
which previously was not performed for frontends. This improves API
consistency, and ensures the value is used regardless of whether
the solver uses a frontend, or a raw definition.
Signed-off-by: Justin Chadwell <me@jedevc.com>
The Result struct in this package is intended to replace the various
Result structs across the codebase, including:
- frontend.Result
- client.Result/gateway.Result
- exporter.Source
These Results are all fundamentally the same, and share the same logic,
with the only difference being the type used in the Ref/Refs properties.
Since these were all separate, we easily encounter the following
problems:
- Repetition of struct and method declarations.
Each separate Result type required it's own properties and methods to
be redeclared. Sometimes these were consistent with each other, and
other times they were not. Unifying all Results together ensures a
lack of duplication, and means that changing the structure of the Result
will not require extensive changes throughout every single declaration.
- Complex conversion between Result types.
At gateway and api boundaries, we are required to convert each Result
into a Result of a different type. Previously, this has been a lengthly
and error prone conversion, requiring knowledge of the exact structure
of the Result, so as to allow traversal. By unifying Results, we can
implement a ConvertResult function to convert from types of Result[A]
to types of Result[B] given a function from A to B, which cleanly
encapsulates the structure of the Result.
- Complex conversion to the protobuf gateway api types.
At the gateway api, we have to convert to pb.Results. Previously, we
had to include logic to convert to and from both frontend.Result and
client.Result. With a generic Result, we can simple convert to a
Result[pb.Ref] and then have one single set of methods to convert
Result[pb.Ref] to a pb.Result.
These problems have been an issue for some time, however, recently with
the addition of attestations, the problem has grown, since the amount of
logic required at conversion boundaries has massively increased.
This patch intends to simplify all of the above, and reduce line-count
and code complexity by using the new-ish go generics features. Care has
been taken to ensure that this patch is as small as possible, and
affects as few Go APIs as possible. This is ensured by using type aliases
for the Result type to re-create the various previous Results, e.g.
frontend.Result/client.Result/exporter.Source, so that little code in
packages that utilize these Results need to be changed. The only major
changes take place at conversion boundaries, where, for the most part,
the code is simplified.
One major caveat is that the result package is made significantly more
complex, due to the generic code, and the need for reflection to perform
comparisons of any types to nil, as interfaces such solver.ResultProxy
(used as a Ref type) do not implement the comparable interface, which is
a limitation of the current go language spec (hopefully to be resolved
in the future).
Signed-off-by: Justin Chadwell <me@jedevc.com>
Previously, a frontend returning two identical references caused a
double-release error, as the gateway expects each ref to be unique.
However, there are scenarios where a frontend might reasonably be
expected to provide identical refs, e.g. for 2 platforms that have
binary compatibility, or for the upcoming attestations use-case.
To solve this issue, we refactor the ResultProxy implementation to
support cloning via splits.
Additionally, we then use the new splitting functionality in the gateway
and forwarder (e.g. dockerfile) frontends, and ensure that the same
ResultProxy is never returned twice in a result. This means that
identical refs over the wire are translated into the correct
shared-memory structures.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Finalize was only used outside the cache package in one place, which
called it with the commit arg set to false. The code path followed
when commit==false turned out to essentially be a no-op because
it set "retain cache" to true if it was already set to true.
It was thus safe to remove the only external call to it and remove it
from the interface. This should be helpful for future efforts to
simplify the equal{Mutable,Immutable} fields in cacheRecord, which exist
due to the "lazy commit" feature that Finalize is tied into.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go
Signed-off-by: Edgar Lee <edgarl@netflix.com>
