Add a passthrough LLB op that builds all inputs and exposes selected
inputs as outputs. Add State.Requires as a client shortcut for requiring
dependencies without returning their snapshots.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Support importing git sources from OCI or registry-backed bundle blobs
and exporting resolved checkouts as single-file git bundles.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Allow git sources to fetch a pinned commit without resolving the ref
against the remote tip, while preserving cache keys for canonical
branch refs and covering the behavior with tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Normalize Git subdir fragments and validate checkout subdir components
so each segment must be a real directory, preventing traversal and symlink escapes.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 8c994eb561a2646b35352e5663afecd225306214)
Add git.mtime=commit option that normalizes all file, symlink,
and directory mtimes in a git snapshot to the resolved commit
timestamp. This enables reproducible builds from git sources.
When SOURCE_DATE_EPOCH is set in the Dockerfile frontend, the
git context automatically uses commit-time mtimes. The URL
query parameter ?mtime=commit|checkout can override this.
New LLB attr (git.mtime) and capability (source.git.mtime) are
registered as experimental. Cache keys include the mtime policy
so that commit-time and checkout-time snapshots are distinct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Introduce util/bkmaps.SyncMap[K, V] as a generic typed
wrapper around sync.Map. Migrate usages in definition,
builder, solver, and contentutil packages, removing
unchecked type assertions at each call site.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Image blob source in LLB allows addressing a single blob
from a container image registry. The difference from the image
source is that image source needs to point to a manifest that
internally points to an array of layer blobs that are all extracted
on top of each other to form a root FS. Contrary, image blob
points to a single blob that is not extracted but downloaded
as a single file into an empty snapshot, similarily how
the HTTP source works.
The main use case for this source is to pin snapshots of
HTTP URLs, upload the downloaded blob into container registry,
and then use a source policy to map a HTTP URL (whose content
might be changed) to the copy of the source as image blob
to ensure immutability.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add detached PGP verification for HTTP sources during metadata resolution
and expose LLB options/caps/attrs for signature validation.
Extract shared OpenPGP verification/parsing logic into util/pgpsign and
reuse it from git signing, plus add integration and source-level tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
These requests allow computing additional checksum algorithms
for the payload of HTTP sources.
Optionally suffix can be passed that is added to payload. This
is needed to make validation of PGP signatures possible remotely.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
ResolveImageConfig was changed to ResolveSourceMetadata long time
ago for cross-source implementation but the worker implementation
was still using old method name with conversions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This adds an additional `RequiredPaths` that is primarily intended for
use with `COPY --parents`. This parameter specifies expected directories
or files that should exist when performing the checksum. A not found
error will be produced if one of these paths is missing.
This fixes an issue with `COPY --parents` where a non existent directory
that was intended to be copied would be ignored.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This allows images to be pulled by tag and then
checked against the digest. If digest is added directly
to the image reference, then tag is ignored.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The Env method returns the current environment variables for the state, not a new State with updated env variables.
Updated doc comment to reflect actual behavior.
Signed-off-by: Michael Dwan <m@dwan.io>
The PATH is very critical during container runs on
Windows. Windows stores the PATH details in its
registry hive, while in unix, this is often
stored in the image's config. See further details
at #5445
Setting a default path like we do on Linux
(which is mostly not used since the PATH is
already set in the configs), works against
the expected build experience, especially when
it comes to installers and commands like `setx`.
Therefore, we skip setting the default PATH on
Windows, and leave it for the OS to load it
from its registry hive.
This also further supports backward compatibilitiy
with the current experience with docker classic
builder.
Users wishing to explicitly store this in the
configs can opt-in by using the ENV PATH= ..
in the Dockerfile, etc.
See also the same practices on Docker Engine
and Containerd:
- da3b31fb2c/oci/defaults.go (L24-L33)
- 041743e8af/pkg/oci/spec_opts_windows.go (L66-L69)closes#5445
Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
Support additional request header fields for HTTP sources, "Accept" and
"User-Agent" as a start.
The "Accept" header is required in some cases and some servers may vary
the response body based on the header value.
The "User-Agent" header may be useful in custom frontends and
potentially other cases.
- llb: Add `llb.Header` and `HTTPInfo.Header` to allow `client/llb`
users to set these header fields on HTTP sources. The argument to
`llb.Header` is a struct to effectively limit header fields to a
subset.
- llb: Define and flag new `source.http.header` capability when
`llb.Header` is used.
- solver: Define new `http.header.` source attribute prefix. Giving
each header field its own attribute (opposed to JSON encoding the
header struct) will allow source policy to make assertions on
individual header fields.
- source/http: Parse `http.header.` attributes into a sorted slice and
include them in cache key digest.
- source/http: Set request headers accordingly.
Signed-off-by: Dan Duvall <dduvall@wikimedia.org>
Support authentication for HTTP sources.
- llb: Define general `llb.AuthOption` interface composed of
`HTTPOption` and `GitOption`.
- llb: Refactor `llb.AuthHeaderSecret` to return an `llb.AuthOption` so
it may be used with both `llb.Git` and `llb.HTTP`.
- llb: Define `HTTPInfo.AuthHeaderSecret`.
- llb: Define and flag new `source.http.auth` capability when
`HTTPInfo.AuthHeaderSecret` is set.
- solver: Define new `http.auth` source attribute.
- source/http: If an `http.auth` attribute is specified, resolve a
secret named by its value and set the "Authorization" request header.
Signed-off-by: Dan Duvall <dduvall@wikimedia.org>
This allows platforms following the new `platforms.FormatAll` function,
which allows for setting the `OSVersion` field of the platform with
`<os>(<ver>)/<arch>`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
* Add file.symlink.create capability and wire it up
* Run codegen for new FileActionSymlink Message
* Add Symlink test
* Add user/group ownership and timestamps to symlink
** Symlinks have user/group ownership that are independent of those of the
target file; in linux, the ownership of the symlink itself is only
checked when the link resides in a directory with the sticky bit set and
the link is the subject of removal or renaming. The sticky bit prevents
files in the directory from being deleted or renamed by non-owners
(members of the group that owns the file may not delete the file; the
user must own the file).
In addition to user/group restrictions, linux symlinks have timestamps
that are independent of the timestamps on the target file.
* Expose symlink options to `llb` package
* Add symlink integration test
* Use tar exporter for tests
** Using the local exporter causes the files to be exported with the
permissions of the user who does the exporting, instead of retaining
their file permissions from within the container.
Using the tar exporter instead preserves the permissions until they can
be checked.
* Change symlink fields to `oldpath` and `newpath`
** Also run `make generated-files`
* Fix typo
* Add doc strings to exported `llb` identifiers
* Remove `requiresLinux` from integration test
* Revert "Remove `requiresLinux` from integration test"
* Add fixes to please the linter
* testFileOpSymlink: check that symlink is created
* Address comments for FileOp llb test
* This commit also fixes a couple of linter complaints.
* Add check for symlink type in tar header
* Address PR review nit
Signed-off-by: Peter Engelbert <pmengelbert@gmail.com>
Calling marshal changes the internal state of the op, for example
addCap() helper adds capability constraints. These can race with
same map being read by another Marshal call. Locking the Marshal
function itself also makes sure that the cache is not recomputed
in this case.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Enables `testDockerfileFromHTTP` integration test
for Windows.
Also fixes a bug where paths were being joined wrongly,
for example:
```go
path.Join("/", "\\context") // would be "/\\context"
```
Addressing tests marked as Revisit for #4485
Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
