Commit Graph

49 Commits

Author SHA1 Message Date
Tonis Tiigi
e05a89e0b8 improve stacks of cancels from defers
In this case the current stack trace points to the line
where the context was created. Instead the stack should be
captured when the defer is running so the return path to
the defer call is also part of the stack.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-11-19 18:40:00 -08:00
Jonathan A. Sternberg
1a3fc0aa15 protobuf: remove gogoproto
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-09-26 12:57:45 -05:00
Justin Chadwell
44d62f23aa filesync: append metadata for CopyToCaller
This was a regression introduced in 81b4762291.

Similar in style to 4b56395369, we need to
ensure that CopyToCaller also doesn't overwrite the provided metadata,
but instead appends to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-16 15:55:10 +00:00
Justin Chadwell
1c1777b7c0 exporter: use implicit ids for exporters
We can derive exporter ids from their place in the exporter array in a
SolveRequest - this removes the need to manually generate and handle
multiple sets of IDs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:04:28 +00:00
Justin Chadwell
81b4762291 session: add file send multiplexing
This patch adds multi-plexing to the local file transfer protocol (from
server to client). This is implementation-wise similar to the
multiplexing from the containerd content store transfer protocol, using
a GRPC header to select the appropriate target.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:04:28 +00:00
Justin Chadwell
a80b48544c session: create helper type for exporter file output
Co-authored-by: a-palchikov <deemok@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-05 12:03:39 +00:00
Tonis Tiigi
8a2a3e83ec replace context.WithCancel with WithCancelCause
Keep stack traces for cancellation errors where possible.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-12-06 22:15:06 -08:00
Justin Chadwell
11b8c6e0b6 session: modify FSSync provider to take fsutil.FS objects
This patch modifies the function signature of the FSSync provider to
take an fsutil.FS instead of a simple raw path resolved to the client's
root filesystem.

Internally, we were already creating an fsutil.FS to Send to the
buildkit server, however, this abstraction didn't reach the session
attachable parameters, so we couldn't provide our own custom FS
implementation.

The rationale behind this change is to allow providing more abstract
custom filesystem implementations to a BuildKit client. This way, we can
start to build from filesystems that might not be on disk - for example,
we could use our Static filesystem implementation in tests to prevent
creating lots of temporary directories, or we could use our Merge
filesystem implementation to allow easily creating variants of a single
context.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 12:48:34 +01:00
Justin Chadwell
8ba1deb94a vendor: update fsutil to master@f09800878302
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 12:48:33 +01:00
Tõnis Tiigi
eddfcd5265 Merge pull request #4049 from sipsma/append-grpc-headers
filesync: append rather than replace grpc md.
2023-08-29 09:20:02 -07:00
Justin Chadwell
c02f99dbce chore: tidy up filesync encode headers handling
Co-authored-by: a-palchikov <deemok@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-08 13:34:23 +01:00
CrazyMax
5b8f962c24 filesync: write closer err discarded
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-08 07:53:21 +02:00
Justin Chadwell
1ec36ece35 filesync: remove deprecated override-excludes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-01 17:28:20 +01:00
Erik Sipsma
4b56395369 filesync: append rather than replace grpc md.
Before this, CopyFileWriter just used metadata.NewOutgoingContext to set
metadata, which results in any pre-existing metadata from the provided
context to be removed.

Now, it gets the current metadata and then sets its own on top of that,
so any pre-existing unrelated metadata is retained.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2023-07-25 13:39:12 -07:00
Tonis Tiigi
48b62404e3 filesync: fix backward compatibility with encoding + and %
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-07-12 09:31:17 -07:00
Justin Chadwell
5230f200ce filesync: escape special query characters
This ensures that files with '%' and '+' can still be properly copied.

To prevent regressions, this also adds in a couple of example test
cases.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-11 15:07:53 +01:00
Tonis Tiigi
a09e2d88dd filesync: mark if options have been encoded to detect old versions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-07-05 23:50:49 -07:00
Tonis Tiigi
9d491e7818 filesync: fix handling non-ascii in file paths
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-09 17:53:07 -07:00
Alex Suraci
3548517cf2 allow dynamic LocalDir access
Changes the filesync attachable to accept an interface instead of a
static allowlist of dirs. This way a single session can support syncing
directories not known ahead of time.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
2022-10-26 21:04:57 -04:00
Nick Santos
844618f506 vendor: upgrade github.com/tonistiigi/fsutil
fixes https://github.com/moby/buildkit/issues/2837

Signed-off-by: Nick Santos <nick@tilt.dev>
2022-05-10 12:38:42 -04:00
Edgar Lee
b3e8c63a48 Local should use session ID in op and only fallback to session group if failed before transfer started
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2022-01-28 15:18:43 -08:00
Tonis Tiigi
35fcb28a00 Clean up old TODOs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-14 22:28:08 -07:00
Tonis Tiigi
ee39f5dd68 dockerfile: use none differ for dockerfile/dockerignore
This avoids wrong metadata matches on small files

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-29 16:17:08 -07:00
Tonis Tiigi
baa4fcdb0f add differ support for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:07 -07:00
Tonis Tiigi
bd3354fea3 session: avoid deprecated grpc.Stream type
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
Tonis Tiigi
6bd2d45527 filesync: avoid ignoring close error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-08 17:48:56 -07:00
Tonis Tiigi
96b6a28312 exporter: allow oci exporters visibility to response metadata
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-31 10:21:19 -07:00
Tonis Tiigi
d3597181e0 session: wrap errors with debug info
Make sure to cover the grpc errors origins.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-06 21:15:21 -07:00
Tonis Tiigi
9f53ea3d78 userns support for sources and executor
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:26:04 -07:00
Tonis Tiigi
e8432d677b vendor: update fsutil to 1bdbf123ad
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 19:11:39 -07:00
Ian Campbell
8b69292087 Bump github.com/tonistiigi/fsutil dependency to f567071bed24
7e391b0e78...f567071bed

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-03 11:46:35 +01:00
Tonis Tiigi
c824098183 session: better error on missing local
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-21 11:49:30 -07:00
Tonis Tiigi
5c3cfe5553 vendor: update fsutil to cb2ad947b
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-16 13:47:24 -07:00
Tonis Tiigi
055dcb6c09 local: allow followpaths for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-07 23:23:13 -07:00
Akihiro Suda
9ef8233da1 client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-28 15:48:54 +09:00
Tonis Tiigi
6e40e83d35 Remove net/context dependencies
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-17 09:11:22 -08:00
Tonis Tiigi
08e1c2990c dockerfile: add dockerignore support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:03:35 -08:00
Tonis Tiigi
6edccb7913 exporter: add OCI exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 09:57:28 -08:00
Tonis Tiigi
7699b1a1eb vendor: update containerd to 9649a428e
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-17 19:43:55 -08:00
Tonis Tiigi
59d3b4c15b session: better backwards compatibility with old servers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-01 11:42:42 -07:00
Tonis Tiigi
c2dbdeb457 session: expose map to file send
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-21 22:14:11 -07:00
Tonis Tiigi
9f6d9a9e78 exporter: add local exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-04 17:12:19 -07:00
Tonis Tiigi
8738929b8c solver: implement content based cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-04 10:03:26 -07:00
Derek McGowan
9ce5053a8b Update containerd version
Fix for logrus rename, use fork until fixed in moby.
Removed unused tar stream.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-26 11:07:45 -07:00
Tõnis Tiigi
f0424c367f Merge pull request #80 from tonistiigi/local-source-progress
source: add progress to local sources
2017-07-21 10:28:50 -07:00
Tonis Tiigi
09d7a2b9fa source: add progress to local sources
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-21 10:08:45 -07:00
Tonis Tiigi
cb168bc954 source: add multiple dir support for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-20 21:17:23 -07:00
Tonis Tiigi
5c2e675d18 source: add local file source fundamentals
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-12 23:23:20 -07:00
Tonis Tiigi
fed5c1d9ce session: copy over session package from moby/moby
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-12 22:32:44 -07:00