resolveMetadata stored its result into the resolver cache through
release(v) where v was a *Metadata. On a failed resolve or a cache
hit it set the pointer to nil, but passing a typed nil pointer as an
interface stores a non-nil value, so a nil *Metadata was cached for
that remote#ref. A later resolve of the same key read it back and
dereferenced nil, crashing buildkitd. Because the cache is shared
across the jobs on a solve state, an untrusted Dockerfile or frontend
could crash the daemon, and one build could poison another's.
Store an untyped nil instead, matching the HTTP source, so only a
successfully resolved Metadata is ever cached.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 4dd98355a71d79bdf93bd3456fda3a64847ce1de)
(cherry picked from commit c1399f7266310397f5615f4751147f886afff6d2)
(cherry picked from commit 16c6856432)
Verify that a permissive host Git config cannot enable file-backed
submodules in a remote build context, with and without keeping .git.
Confirm that an unrestricted Git checkout can fetch the same submodule
so the test checks protocol enforcement rather than an invalid fixture.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
When a job merged an edge into another, mergeTo appended the dependency
cache keys to the target edge's secondaryExporters on the scheduler
goroutine. Cache export reads the same slice through the exporter of a
result loaded from that edge, and runs outside the scheduler, e.g. for
build history provenance or cache export while another build is still
running and merging into the shared edge.
Guard secondaryExporters with a mutex and let ExportTo iterate over a
copy, so nested exports run without holding the lock.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Wait for initial frontend data before starting ServeConn so
container startup does not consume the HTTP/2 preface timeout.
Close pipe descriptors on cancellation to unblock the wait.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
- cpu: add amd64 v2/v3/v4 feature bits
- darwin: expose setpriority PRIO_DARWIN_* constants
- unix/darwin: add missing pwritev test case on darwin/arm64
- unix/darwin: add O_EXEC and O_SEARCH on darwin
- unix/darwin: pass fcntl data as unsafe.Pointer in FcntlF{lock,store} on darwin
- unix/linux: don't warn about closed stderr from child process
- unix/linux: unify ppc64x syscall wrappers
- unix: add listns syscall introduced in Linux 6.19
- unix: add Mmsghdr for linux
- unix: add PrctlArg5Ptr
- unix: align the getdents buffer
- unix: extend z/OS support
- unix: keep generated types compilable with current cgo
- unix: pass xattr data as unsafe.Pointer in BSD xattr syscalls
- unix: provide linux/sparc64 assembly for syscalls
- unix: return syscall error in *listxattrNS on BSD
- unix: update glibc to 2.44
- unix: update linux to v7.2
- unix: use the stat64 family and fix Stat_t on linux/sparc64
- windows: drop arm
full diff: https://github.com/golang/sys/compare/v0.48.0...v0.49.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When a build has several exporters, runExporters runs them in parallel
and each one calls the same provenance attestation ContentFunc. Every
call finalized the shared ProvenanceCreator predicate (FinishedOn,
Layers, SysUsage, CompatibilityVersion) while another exporter could be
JSON-encoding it, which the race detector reports as a data race.
Add PredicateJSON, which finalizes and encodes the predicate under a
per-creator mutex and returns the encoded bytes, so no caller keeps a
reference to the shared predicate. Make Predicate unexported so all
callers go through the locked path.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The resize and signal goroutines in runProcess both assigned the err
variable of the enclosing function, so concurrent Resize and Signal
calls on a process raced, and either goroutine could log the other's
error. Use a local error in each goroutine.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
resultProxy.loadResult added the refs of a failed exec to errResults
without holding rp.mu, and borrowed the ExecError so its finalizer would
not release them. If the proxy was released while the load was still
running, for example by the gateway forwarder discarding a lazy ref
whose ReadFile was evaluating it, Release had already run and the refs
were never released. They stayed in use until the daemon restarted, and
prune could not remove them.
Check rp.released under rp.mu after the load and release the refs
right away if the proxy is already gone.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
require.Contains on a map[*Job]struct{} compares the job with every key
using reflect.DeepEqual, which dereferences other jobs and reads their
live progress state while its goroutines run. This raced under -race.
Check membership with a map lookup instead.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
progressWriter.Close set done while other goroutines could still be
writing to the writer, for example a shared vertex writing its snapshot
into the progress of a job that is finishing. Write read the flag
without synchronization. Make done an atomic.Bool.
Add a test that closes a writer while it is being written to; it
reports the race under -race without the fix.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The ExecError of a failed exec is cached by the shared op and returned
to every build that shares the vertex. Each resultProxy then set
OwnerBorrowed on the same error concurrently, which races. Add
ExecError.Borrow that sets the flag under a mutex and use it from
loadResult, Release and the finalizer. Ref ownership is unchanged.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
On the containerd worker the snapshotter is a gRPC client. If a build
is cancelled while finalize is committing a lazy ref, containerd can
apply the commit while the client returns Canceled. finalize then
deleted the lease protecting the committed snapshot and kept
equalMutable, so the record pointed at a mutable snapshot that no
longer existed. Every later build using the record failed with "failed
to stat active key during commit: snapshot ... does not exist" until
the daemon restarted.
Run finalize without cancellation so the commit result is not lost. If
Commit still returns an error, check whether the target snapshot was
committed and, if so, complete the finalize. Keep the lease when that
cannot be determined.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Give frontends a dedicated section with short descriptions to help readers
compare build languages and tools, including Docker Sandbox Kit.
Remove the unavailable JSON Gateway, refresh moved links, and note that
Earthly is no longer actively maintained.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Protect index ID assignment with the cache-key mutex used by trace readers.
Add a concurrent regression test that fails under the race detector.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
testSSHFileDescriptorsClosed read ssh-agent's stderr buffer while the
agent was still running and os/exec's copy goroutine was writing to
it, which races when the test binary is built with -race. Kill the
agent and Wait for it before reading; Wait guarantees the copy has
finished and the full output is available.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
testPrompt polled the container output bytes.Buffer with String() while
the gateway client's output goroutine wrote to the same buffer, which
races when the test binary is built with -race. Wrap the buffer in a
small mutex-protected writer shared by the output streams and the
prompt.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
GetResolvConf read and wrote the package-level notFirstRun and
lastNotEmpty flags inside its flightcontrol callback. flightcontrol
only serializes calls with the same key, so calls for different roots,
or for resolv.conf and resolv-host.conf at once, raced on them. Make
both flags atomic.Bool; the sequential behavior is unchanged.
Add a test that generates resolv.conf for several roots concurrently;
it reports the race under -race without the fix.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
getRecord checked rec.isDead() holding only cm.mu, while finalize
updates equalMutable and the mutable's dead flag under the record
mutex, so a concurrent Get and Finalize of a lazily committed ref
raced. Take the record mutex for the check; lock order stays cm.mu
then record mutex, as in get().
Records restored from metadata also did not follow the rule that
records sharing data share a mutex: the immutable got a new mutex
instead of the equal mutable's, as mutableRef.commit does. Share it,
so finalize and lookups of the mutable synchronize after a restart.
Add tests for both cases; they report the races under -race without
the fix.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Geoff Johnson