101 Commits

Author SHA1 Message Date
Tonis Tiigi
cdb7d70861 Update patch dependencies
Bump patch-level dependencies while leaving
packageurl-go unchanged. This updates AWS config/credentials,
smithy-go, klauspost/compress, and sigstore-go, with required
companion bumps from their module constraints.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-07-21 22:24:51 -07:00
CrazyMax
e4d0dbaf3f chore: update generated files
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-12 09:44:45 +02:00
CrazyMax
86ddfea28e filesync: detect local export multi-platform transfer support
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-11 18:53:07 +02:00
CrazyMax
73c67671d3 exporter: use root-backed fsutil receive instead of client-side staging
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-10 12:24:31 +02:00
CrazyMax
00764d4f69 exporter: add local exporter mode=delete
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2026-06-10 12:13:15 +02:00
Sebastiaan van Stijn
94dcaca630 update protoc to v3.14.0
Updating to the lowest release that includes [protobuf@dfab275], which
removed use of the github.com/golang/protobuf/ptypes/timestamp.Timestamp
type alias (deprecated).

[protobuf@dfab275]: dfab275eca

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-06-05 18:01:59 +02:00
Tonis Tiigi
70b363e54c session: fail stuck session RPCs on health timeout
Bind session RPC contexts to caller lifetime so ongoing RPCs fail when the
session is canceled. Add an integration test that blocks the session tunnel
and verifies the health monitor releases the hung build after timeout.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-04-02 23:29:16 -07:00
Tonis Tiigi
b61489948c tests: use t.Context in unit tests
Replace context.TODO/background calls in targeted unit tests with
t.Context().

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-02-25 16:41:54 -08:00
Tonis Tiigi
ddeb26c5e1 vendor: update outdated patch versions of dependencies
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2026-01-13 22:08:15 -08:00
Tonis Tiigi
09426caaae vendor: update containerd to v2.2.0-rc.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-11-03 11:11:53 -08:00
Jonathan A. Sternberg
f4339fd00b tools: utilize the new tool functionality introduced in go 1.24
Removes the `tools/tools.go` file used as a hack to get `go.mod` to
track tools in favor of the new method introduced in go 1.24 of being
formally supported in the `go.mod` file. This will allow the tools to be
managed with the new `go get tool` and `go install tool` commands.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-10 10:08:03 -05:00
Tonis Tiigi
9a5dff09b5 vendor: update to containerd v2.1.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-05-21 09:09:33 -06:00
Tonis Tiigi
9fcedf9807 update gopls to go1.24 compatible version
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-05-07 18:23:20 -07:00
Tonis Tiigi
c6a1dcd6b5 lint: correcting errorlint and some err113 cases
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-05-02 12:19:22 -07:00
Tonis Tiigi
f1f56c2b77 golangci-lint: v2 upgrade
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-05-02 12:19:17 -07:00
CrazyMax
43fc7e8585 Merge pull request #5897 from tonistiigi/local-metadata-transfer
source: add metadata-only transfer option for local source
2025-04-11 17:50:29 +02:00
Tonis Tiigi
7ae0d6a903 source: add metadata-only transfer option for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-04-11 07:38:29 -07:00
Tonis Tiigi
58faf412f3 exporter: add session exporter capability
This allows client to defer exporter selection to
the the time where build has completed.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-04-10 22:15:19 -07:00
Tonis Tiigi
b5286f8dcb apply x/tools/modernize fixes
Autogenerated with couple of manual patches.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-03-07 08:18:45 -08:00
Sebastiaan van Stijn
d709e4bb1c vendor: google.golang.org/protobuf v1.35.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-14 14:03:31 +01:00
Marat Radchenko
0a5a80cfec Remove pre-Go 1.17 build tags
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2024-11-21 10:58:27 +03:00
Tonis Tiigi
e05a89e0b8 improve stacks of cancels from defers
In this case the current stack trace points to the line
where the context was created. Instead the stack should be
captured when the defer is running so the return path to
the defer call is also part of the stack.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-11-19 18:40:00 -08:00
Tonis Tiigi
c9a25c4d8a pb: regenerate protobuf
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-10-28 12:56:03 -07:00
Jonathan A. Sternberg
c9029975da filesync: reuse data buffer for diffcopy
Reuse the bytes message data buffer for diffcopy to allow memory to be
reused when unmarshaling the bytes.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-21 12:50:46 -05:00
Jonathan A. Sternberg
41a0a0c37d protobuf: add vtproto as a supplemental marshaler
vtproto is an extra protobuf compiler that generates special methods
suffixed with `VT` that create typed and unrolled marshal and unmarshal
functions similar to gogo that can be used for performance sensitive
code. These extensions are optional for code to use but buildkit uses
them.

A codec is also included to utilize vtproto for grpc code. If the
package `github.com/moby/buildkit/util/grpcutil/encoding/proto` is
imported then vtproto will be used if it exists and otherwise it will
use the standard marshaling and unmarshaling methods.

This codec has an important difference from the default codec. The
default codec will always reset messages before unmarshaling. In most
cases, this is unnecessary and is only relevant for `RecvMsg` on
streams. In most cases, if we are passing in an existing message to this
method, we want to reuse the buffers. This codec will always merge the
message when unmarshaling instead of resetting the input message.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-04 12:52:15 -05:00
Jonathan A. Sternberg
ffd8ff490e protobuf: normalize how protobuf files are generated
The relative paths option for protoc generators doesn't work well when
it comes to dependencies. This simplifies the code generation to avoid
using `go generate` and to use one global command for protoc generation.

This is similar to https://github.com/docker/buildx/pull/2713 since the
same problems with code generation occur here too.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-03 17:40:53 -05:00
Jonathan A. Sternberg
1a3fc0aa15 protobuf: remove gogoproto
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-09-26 12:57:45 -05:00
Tonis Tiigi
ab2c224e72 session: remove session name property
This seems to be completely unused.

I believe it is remnant of pre-buildkit session implementation
and was used for either logging of some transfer reuse.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-07-19 17:22:04 -07:00
Anthony Nandaa
e7ceb63b3c wip: tar and local exporter running with privileges
TODO: need to cross-check that there is no way the
SeBackupPrivilege can be abused/exploited.

WIP: how best to handle the files to be exclused
without touching `fsutil`

Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
2024-06-10 17:24:05 +03:00
Tonis Tiigi
03c7a6d769 lint: fix some testifylint warnings
This does not cover all warning yet but split
into chunks to ease review.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-05-30 11:32:55 -07:00
Justin Chadwell
44d62f23aa filesync: append metadata for CopyToCaller
This was a regression introduced in 81b4762291.

Similar in style to 4b56395369, we need to
ensure that CopyToCaller also doesn't overwrite the provided metadata,
but instead appends to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-16 15:55:10 +00:00
Justin Chadwell
1c1777b7c0 exporter: use implicit ids for exporters
We can derive exporter ids from their place in the exporter array in a
SolveRequest - this removes the need to manually generate and handle
multiple sets of IDs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:04:28 +00:00
Justin Chadwell
81b4762291 session: add file send multiplexing
This patch adds multi-plexing to the local file transfer protocol (from
server to client). This is implementation-wise similar to the
multiplexing from the containerd content store transfer protocol, using
a GRPC header to select the appropriate target.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:04:28 +00:00
Justin Chadwell
a80b48544c session: create helper type for exporter file output
Co-authored-by: a-palchikov <deemok@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:03:39 +00:00
Justin Chadwell
578e8105c3 filesync: add doc comments in proto
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:03:39 +00:00
Tonis Tiigi
8a2a3e83ec replace context.WithCancel with WithCancelCause
Keep stack traces for cancellation errors where possible.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:06 -08:00
Tõnis Tiigi
892fbdfd0b Merge pull request #4313 from tonistiigi/session-grpc-message-size-fix
session: fix grpc message size limits for tar streams
2023-11-13 12:48:58 -08:00
Justin Chadwell
11b8c6e0b6 session: modify FSSync provider to take fsutil.FS objects
This patch modifies the function signature of the FSSync provider to
take an fsutil.FS instead of a simple raw path resolved to the client's
root filesystem.

Internally, we were already creating an fsutil.FS to Send to the
buildkit server, however, this abstraction didn't reach the session
attachable parameters, so we couldn't provide our own custom FS
implementation.

The rationale behind this change is to allow providing more abstract
custom filesystem implementations to a BuildKit client. This way, we can
start to build from filesystems that might not be on disk - for example,
we could use our Static filesystem implementation in tests to prevent
creating lots of temporary directories, or we could use our Merge
filesystem implementation to allow easily creating variants of a single
context.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 12:48:34 +01:00
Justin Chadwell
8ba1deb94a vendor: update fsutil to master@f09800878302
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 12:48:33 +01:00
Tonis Tiigi
329e945c25 filesync: split stream data into 3MB chunks to avoid message limits
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-10-09 18:23:48 -07:00
Tõnis Tiigi
eddfcd5265 Merge pull request #4049 from sipsma/append-grpc-headers
filesync: append rather than replace grpc md.
2023-08-29 09:20:02 -07:00
Justin Chadwell
85e2e252ee Merge pull request #4124 from jedevc/chore-refactor-filesync-encode 2023-08-14 15:23:36 +01:00
Justin Chadwell
c5402622bb lint: add protolint config
And tidy up the proto definitions to follow the new linting rules.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-10 13:11:18 +01:00
Justin Chadwell
c02f99dbce chore: tidy up filesync encode headers handling
Co-authored-by: a-palchikov <deemok@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-08 13:34:23 +01:00
CrazyMax
5b8f962c24 filesync: write closer err discarded
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-08 07:53:21 +02:00
Justin Chadwell
1ec36ece35 filesync: remove deprecated override-excludes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-01 17:28:20 +01:00
Erik Sipsma
4b56395369 filesync: append rather than replace grpc md.
Before this, CopyFileWriter just used metadata.NewOutgoingContext to set
metadata, which results in any pre-existing metadata from the provided
context to be removed.

Now, it gets the current metadata and then sets its own on top of that,
so any pre-existing unrelated metadata is retained.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-07-25 13:39:12 -07:00
Tonis Tiigi
48b62404e3 filesync: fix backward compatibility with encoding + and %
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-07-12 09:31:17 -07:00
Justin Chadwell
5230f200ce filesync: escape special query characters
This ensures that files with '%' and '+' can still be properly copied.

To prevent regressions, this also adds in a couple of example test
cases.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-11 15:07:53 +01:00
Tonis Tiigi
a09e2d88dd filesync: mark if options have been encoded to detect old versions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-07-05 23:50:49 -07:00