Modifies the hack/compose script to use a newer otel collector and try
to suppress miscellaneous logs when it fails to export traces. Disable
jaeger, prometheus, and grafana by default.
It also adds a metrics pipeline that goes to a nop exporter. This is
force the otel collector to start the metrics service and prevent an
error when buildkit exports metrics to the otel collector related to the
metrics service being missing.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Removes the `tools/tools.go` file used as a hack to get `go.mod` to
track tools in favor of the new method introduced in go 1.24 of being
formally supported in the `go.mod` file. This will allow the tools to be
managed with the new `go get tool` and `go install tool` commands.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This moves the aggregation of the various protobuf includes (binaries
and include files) to a separate stage so the files can be retrieved
more easily by external tools.
Since the canonical way of retrieving the protobuf dependencies is to
use this file and the dependencies can be quite complicated to retrieve,
this makes it easier for outside contributors to retrieve the
dependencies and copy them into their own repositories.
This was possible before by using the `tools` target, but that would
also include the entire `golang` image. Now you can retrieve just the
`protobuf` target to get the binaries and the include files.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
One should not use their regular docker credentials here
but make a special config file with only public readonly
token inside.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The prune logic would prune multiple times because one prune could cause
more things to be capable of pruning and change the logic. This was done
through a recursive invocation.
Since go doesn't have support for function tail calls, this would result
in a new stack entry for each loop. This unrolls the logic so the prune
function is invoked iteratively rather than recursively.
`prune` and `pruneOnce` have also had their names swapped. In general,
`pruneOnce` implies that it runs a single prune while `prune` sounds
like the top level function. The current code had this reversed and
`pruneOnce` would call `prune` and `prune` would call itself
recursively.
I've also updated a section in the controller that invoked prune on each
worker. In older versions of Go, the current version was correct because
those versions of Go would reuse the location for each loop which would
cause goroutines to all reference the same worker instead of different
workers.
Recent versions of Go have changed the behavior so this is no longer
needed.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This allows running Dockerfile tests so that the Dockerfile
version and the BuildKit version are from different commits so
that we can test that old Dockerfile releases remain compatible
with the latest BuildKit.
The tests are based on the commit of the Dockerfile frontend as
we can't expect that new test would work on old frontends. In future
we might consider doing it the other way as well but then we need
a way to mark tests that can be ignored if they are not expected to
pass because of a new feature dependency.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
vtproto is an extra protobuf compiler that generates special methods
suffixed with `VT` that create typed and unrolled marshal and unmarshal
functions similar to gogo that can be used for performance sensitive
code. These extensions are optional for code to use but buildkit uses
them.
A codec is also included to utilize vtproto for grpc code. If the
package `github.com/moby/buildkit/util/grpcutil/encoding/proto` is
imported then vtproto will be used if it exists and otherwise it will
use the standard marshaling and unmarshaling methods.
This codec has an important difference from the default codec. The
default codec will always reset messages before unmarshaling. In most
cases, this is unnecessary and is only relevant for `RecvMsg` on
streams. In most cases, if we are passing in an existing message to this
method, we want to reuse the buffers. This codec will always merge the
message when unmarshaling instead of resetting the input message.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
The relative paths option for protoc generators doesn't work well when
it comes to dependencies. This simplifies the code generation to avoid
using `go generate` and to use one global command for protoc generation.
This is similar to https://github.com/docker/buildx/pull/2713 since the
same problems with code generation occur here too.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Use golang.org/x/exp/trace to implement an trace recorder that saves the trace
to a circular buffer and can be retrieved at any time.
Debug endpoints have been added under /debug/flight to start and stop the trace
as well as to set its period.
Due to golang.org/x/exp/trace, the minimum go version has been bumped to 1.22
Signed-off-by: Alberto Garcia Hierro <damaso.hierro@docker.com>
appdefaults_unix.go, constants_unix.go:
/run is a thing only for Linux. Other Unixes use /var/run.
diffapply_linux.go and source_linux.go:
These files use Linux-only API.
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
