diff --git a/README.md b/README.md index 29c15dc3d..11b7c8736 100644 --- a/README.md +++ b/README.md @@ -28,10 +28,16 @@ Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056c Join `#buildkit` channel on [Docker Community Slack](http://dockr.ly/slack) -:information_source: If you are visiting this repo for the usage of BuildKit-only Dockerfile features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)`, please refer to [`frontend/dockerfile/docs/syntax.md`](frontend/dockerfile/docs/syntax.md). +> **Note** +> +> If you are visiting this repo for the usage of BuildKit-only Dockerfile features +> like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)`, please refer to [`frontend/dockerfile/docs/reference.md`](frontend/dockerfile/docs/reference.md) -:information_source: [BuildKit has been integrated to `docker build` since Docker 18.06 .](https://docs.docker.com/develop/develop-images/build_enhancements/) -You don't need to read this document unless you want to use the full-featured standalone version of BuildKit. +> **Note** +> +> [BuildKit has been integrated to `docker build` since Docker 18.09](https://docs.docker.com/develop/develop-images/build_enhancements/). +> You don't need to read this document unless you want to use the full-featured +> standalone version of BuildKit. @@ -44,7 +50,6 @@ You don't need to read this document unless you want to use the full-featured st - [Exploring Dockerfiles](#exploring-dockerfiles) - [Building a Dockerfile with `buildctl`](#building-a-dockerfile-with-buildctl) - [Building a Dockerfile using external frontend:](#building-a-dockerfile-using-external-frontend) - - [Building a Dockerfile with experimental features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)`](#building-a-dockerfile-with-experimental-features-like-run---mounttypebindcachetmpfssecretssh) - [Output](#output) - [Image/Registry](#imageregistry) - [Local directory](#local-directory) @@ -205,10 +210,6 @@ buildctl build \ --opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org ``` -#### Building a Dockerfile with experimental features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)` - -See [`frontend/dockerfile/docs/experimental.md`](frontend/dockerfile/docs/experimental.md). - ### Output By default, the build result and intermediate cache will only remain internally in BuildKit. An output needs to be specified to retrieve the result. diff --git a/docs/experimental-syntaxes.md b/docs/experimental-syntaxes.md index 138050f94..6dbb48754 100644 --- a/docs/experimental-syntaxes.md +++ b/docs/experimental-syntaxes.md @@ -1,4 +1,3 @@ # Dockerfile frontend syntaxes -Documentation for Dockerfile syntaxes can be found in the -[Dockerfile frontend documentation](/frontend/dockerfile/docs/syntax.md) +This page has moved to [Dockerfile reference documentation](/frontend/dockerfile/docs/reference.md) diff --git a/examples/build-using-dockerfile/README.md b/examples/build-using-dockerfile/README.md index b0087ad9c..1ec17bdf1 100644 --- a/examples/build-using-dockerfile/README.md +++ b/examples/build-using-dockerfile/README.md @@ -1,6 +1,5 @@ # `build-using-dockerfile` example -:information_source: [BuildKit has been integrated to `docker build` since Docker 18.06.](https://docs.docker.com/develop/develop-images/build_enhancements/) The `build-using-dockerfile` CLI is just provided as an example for writing a BuildKit client application. For people familiar with `docker build` command, `build-using-dockerfile` is provided as an example for building Dockerfiles with BuildKit using a syntax similar to `docker build`. diff --git a/frontend/dockerfile/builder/build.go b/frontend/dockerfile/builder/build.go index a9e8deeb8..0308c7c0c 100644 --- a/frontend/dockerfile/builder/build.go +++ b/frontend/dockerfile/builder/build.go @@ -61,7 +61,7 @@ const ( keyUlimit = "ulimit" // Don't forget to update frontend documentation if you add - // a new build-arg: frontend/dockerfile/docs/syntax.md + // a new build-arg: frontend/dockerfile/docs/reference.md keyCacheNSArg = "build-arg:BUILDKIT_CACHE_MOUNT_NS" keyContextKeepGitDirArg = "build-arg:BUILDKIT_CONTEXT_KEEP_GIT_DIR" keyHostnameArg = "build-arg:BUILDKIT_SANDBOX_HOSTNAME" diff --git a/frontend/dockerfile/docs/experimental.md b/frontend/dockerfile/docs/experimental.md index 54ab340c6..fc84bdaae 100644 --- a/frontend/dockerfile/docs/experimental.md +++ b/frontend/dockerfile/docs/experimental.md @@ -1,4 +1,3 @@ # Dockerfile frontend syntaxes -This page has moved to -[Dockerfile frontend documentation](/frontend/dockerfile/docs/syntax.md) +This page has moved to [Dockerfile reference documentation](reference.md) diff --git a/frontend/dockerfile/docs/reference.md b/frontend/dockerfile/docs/reference.md index b03a038d9..2af7eb268 100644 --- a/frontend/dockerfile/docs/reference.md +++ b/frontend/dockerfile/docs/reference.md @@ -1,7 +1,7 @@ --- title: Dockerfile reference description: "Dockerfiles use a simple DSL which allows you to automate the steps you would normally manually take to create an image." -keywords: "build, dockerfile, reference" +keywords: build, dockerfile, reference redirect_from: - /reference/builder/ --- @@ -130,7 +130,6 @@ section in the `docker build` command reference. When you're done with your build, you're ready to look into [scanning your image with `docker scan`](https://docs.docker.com/engine/scan/), and [pushing your image to Docker Hub](https://docs.docker.com/docker-hub/repos/). - ## BuildKit Starting with version 18.09, Docker supports a new backend for executing your @@ -147,10 +146,21 @@ implementation. For example, BuildKit can: - Prioritize your build cache for automatic pruning To use the BuildKit backend, you need to set an environment variable -`DOCKER_BUILDKIT=1` on the CLI before invoking `docker build`. +`DOCKER_BUILDKIT=1` on the CLI before invoking `docker build`. [Docker Buildx](https://github.com/docker/buildx) +always enables BuildKit. -To learn about the Dockerfile syntax available to BuildKit-based -builds [refer to the documentation in the BuildKit repository](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md). +### External Dockerfile frontend + +BuildKit supports loading frontends dynamically from container images. To use +an external Dockerfile frontend, the first line of your Dockerfile needs to be +`# syntax=docker/dockerfile:1` pointing to the specific image you want to use. + +BuildKit also ships with Dockerfile frontend builtin but it is recommended to +use an external image to make sure that all users use the same version on the +builder and to pick up bugfixes automatically without waiting for a new version +of BuildKit or Docker engine. + +See the [`syntax` directive section](#syntax) for more information. ## Format @@ -818,6 +828,246 @@ The cache for `RUN` instructions can be invalidated by [`ADD`](#add) and [`COPY` If your system doesn't have support for `dirperm1`, the issue describes a workaround. +## RUN --mount + +> **Note** +> +> Added in [`docker/dockerfile:1.2`](#syntax) + +`RUN --mount` allows you to create mounts that process running as part of the +build can access. This can be used to bind files from other part of the build +without copying, accessing build secrets or ssh-agent sockets, or creating cache +locations to speed up your build. + +Syntax: `--mount=[type=][,option=[,option=]...]` + +### Mount types + +| Type | Description | +|------------------------------------------|-----------------------------------------------------------------------------------------------------------| +| [`bind`](#run---mounttypebind) (default) | Bind-mount context directories (read-only). | +| [`cache`](#run---mounttypecache) | Mount a temporary directory for to cache directories for compilers and package managers. | +| [`secret`](#run---mounttypesecret) | Allow the build container to access SSH keys via SSH agents, with support for passphrases. | +| [`ssh`](#run---mounttypessh) | Allow the build container to access secure files such as private keys without baking them into the image. | + +### RUN --mount=type=bind + +This mount type allows binding directories (read-only) in the context or in an +image to the build container. + +| Option | Description | +|----------------------|--------------------------------------------------------------------------------------| +| `target`[^1] | Mount path. | +| `source` | Source path in the `from`. Defaults to the root of the `from`. | +| `from` | Build stage or image name for the root of the source. Defaults to the build context. | +| `rw`,`readwrite` | Allow writes on the mount. Written data will be discarded. | + +### RUN --mount=type=cache + +This mount type allows the build container to cache directories for compilers +and package managers. + +| Option | Description | +|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | Optional ID to identify separate/different caches. Defaults to value of `target`. | +| `target`[^1] | Mount path. | +| `ro`,`readonly` | Read-only if set. | +| `sharing` | One of `shared`, `private`, or `locked`. Defaults to `shared`. A `shared` cache mount can be used concurrently by multiple writers. `private` creates a new mount if there are multiple writers. `locked` pauses the second writer until the first one releases the mount. | +| `from` | Build stage to use as a base of the cache mount. Defaults to empty directory. | +| `source` | Subpath in the `from` to mount. Defaults to the root of the `from`. | +| `mode` | File mode for new cache directory in octal. Default `0755`. | +| `uid` | User ID for new cache directory. Default `0`. | +| `gid` | Group ID for new cache directory. Default `0`. | + +Contents of the cache directories persists between builder invocations without +invalidating the instruction cache. Cache mounts should only be used for better +performance. Your build should work with any contents of the cache directory as +another build may overwrite the files or GC may clean it if more storage space +is needed. + +#### Example: cache Go packages + +```dockerfile +# syntax=docker/dockerfile:1 +FROM golang +RUN --mount=type=cache,target=/root/.cache/go-build \ + go build ... +``` + +#### Example: cache apt packages + +```dockerfile +# syntax=docker/dockerfile:1 +FROM ubuntu +RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache +RUN --mount=type=cache,target=/var/cache/apt \ + --mount=type=cache,target=/var/lib/apt \ + apt update && apt-get --no-install-recommends install -y gcc +``` + +### RUN --mount=type=tmpfs + +This mount type allows mounting tmpfs in the build container. + +| Option | Description | +|---------------------|-------------------------------------------------------| +| `target`[^1] | Mount path. | +| `size` | Specify an upper limit on the size of the filesystem. | + +### RUN --mount=type=secret + +This mount type allows the build container to access secure files such as +private keys without baking them into the image. + +| Option | Description | +|---------------------|---------------------------------------------------------------------------------------------------| +| `id` | ID of the secret. Defaults to basename of the target path. | +| `target` | Mount path. Defaults to `/run/secrets/` + `id`. | +| `required` | If set to `true`, the instruction errors out when the secret is unavailable. Defaults to `false`. | +| `mode` | File mode for secret file in octal. Default `0400`. | +| `uid` | User ID for secret file. Default `0`. | +| `gid` | Group ID for secret file. Default `0`. | + +#### Example: access to S3 + +```dockerfile +# syntax=docker/dockerfile:1 +FROM python:3 +RUN pip install awscli +RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \ + aws s3 cp s3://... ... +``` + +```console +$ docker buildx build --secret id=aws,src=$HOME/.aws/credentials . +``` + +### RUN --mount=type=ssh + +This mount type allows the build container to access SSH keys via SSH agents, +with support for passphrases. + +| Option | Description | +|---------------------|------------------------------------------------------------------------------------------------| +| `id` | ID of SSH agent socket or key. Defaults to "default". | +| `target` | SSH agent socket path. Defaults to `/run/buildkit/ssh_agent.${N}`. | +| `required` | If set to `true`, the instruction errors out when the key is unavailable. Defaults to `false`. | +| `mode` | File mode for socket in octal. Default `0600`. | +| `uid` | User ID for socket. Default `0`. | +| `gid` | Group ID for socket. Default `0`. | + +#### Example: access to Gitlab + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +RUN apk add --no-cache openssh-client +RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts +RUN --mount=type=ssh \ + ssh -q -T git@gitlab.com 2>&1 | tee /hello +# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here +# with the type of build progress is defined as `plain`. +``` + +```console +$ eval $(ssh-agent) +$ ssh-add ~/.ssh/id_rsa +(Input your passphrase here) +$ docker buildx build --ssh default=$SSH_AUTH_SOCK . +``` + +You can also specify a path to `*.pem` file on the host directly instead of `$SSH_AUTH_SOCK`. +However, pem files with passphrases are not supported. + +## RUN --network + +> **Note** +> +> Added in [`docker/dockerfile:1.1`](#syntax) + +`RUN --network` allows control over which networking environment the command +is run in. + +Syntax: `--network=` + +### Network types + +| Type | Description | +|----------------------------------------------|----------------------------------------| +| [`default`](#run---networkdefault) (default) | Run in the default network. | +| [`none`](#run---networknone) | Run with no network access. | +| [`host`](#run---networkhost) | Run in the host's network environment. | + +### RUN --network=default + +Equivalent to not supplying a flag at all, the command is run in the default +network for the build. + +### RUN --network=none + +The command is run with no network access (`lo` is still available, but is +isolated to this process) + +#### Example: isolating external effects + +```dockerfile +# syntax=docker/dockerfile:1 +FROM python:3.6 +ADD mypackage.tgz wheels/ +RUN --network=none pip install --find-links wheels mypackage +``` + +`pip` will only be able to install the packages provided in the tarfile, which +can be controlled by an earlier build stage. + +### RUN --network=host + +The command is run in the host's network environment (similar to +`docker build --network=host`, but on a per-instruction basis) + +> **Warning** +> +> The use of `--network=host` is protected by the `network.host` entitlement, +> which needs to be enabled when starting the buildkitd daemon with +> `--allow-insecure-entitlement network.host` flag or in [buildkitd config](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md), +> and for a build request with [`--allow network.host` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/#allow). +{:.warning} + +## RUN --security + +> **Note** +> +> Not yet available in stable syntax, use [`docker/dockerfile:1-labs`](#syntax) version. + +### RUN --security=insecure + +With `--security=insecure`, builder runs the command without sandbox in insecure +mode, which allows to run flows requiring elevated privileges (e.g. containerd). +This is equivalent to running `docker run --privileged`. + +> **Warning** +> +> In order to access this feature, entitlement `security.insecure` should be +> enabled when starting the buildkitd daemon with +> `--allow-insecure-entitlement security.insecure` flag or in [buildkitd config](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md), +> and for a build request with [`--allow security.insecure` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/#allow). +{:.warning} + +#### Example: check entitlements + +```dockerfile +# syntax=docker/dockerfile:1-labs +FROM ubuntu +RUN --security=insecure cat /proc/self/status | grep CapEff +``` +```text +#84 0.093 CapEff: 0000003fffffffff +``` + +### RUN --security=sandbox + +Default sandbox mode can be activated via `--security=sandbox`, but that is no-op. + ## CMD The `CMD` instruction has three forms: @@ -1195,7 +1445,7 @@ does not support authentication. > following instructions from the Dockerfile if the contents of `` have > changed. This includes invalidating the cache for `RUN` instructions. > See the [`Dockerfile` Best Practices -guide – Leverage build cache](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) +guide – Leverage build cache](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) > for more information. @@ -1256,6 +1506,10 @@ guide – Leverage build cache](https://docs.docker.com/develop/develop-images/ - If `` doesn't exist, it is created along with all missing directories in its path. +## ADD --link + +See [`COPY --link`](#copy---link). + ## COPY COPY has two forms: @@ -1390,9 +1644,78 @@ attempted to be used instead. > following instructions from the Dockerfile if the contents of `` have > changed. This includes invalidating the cache for `RUN` instructions. > See the [`Dockerfile` Best Practices -guide – Leverage build cache](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) +guide – Leverage build cache](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) > for more information. +## COPY --link + +> **Note** +> +> Added in [`docker/dockerfile:1.4`](#syntax) + +Enabling this flag in `COPY` or `ADD` commands allows you to copy files with +enhanced semantics where your files remain independent on their own layer and +don't get invalidated when commands on previous layers are changed. + +When `--link` is used your source files are copied into an empty destination +directory. That directory is turned into a layer that is linked on top of your +previous state. + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +COPY --link /foo /bar +``` + +Is equivalent of doing two builds: + +```dockerfile +FROM alpine +``` + +and + +```dockerfile +FROM scratch +COPY /foo /bar +``` + +and merging all the layers of both images together. + +### Benefits of using `--link` + +Use `--link` to reuse already built layers in subsequent builds with +`--cache-from` even if the previous layers have changed. This is especially +important for multi-stage builds where a `COPY --from` statement would +previously get invalidated if any previous commands in the same stage changed, +causing the need to rebuild the intermediate stages again. With `--link` the +layer the previous build generated is reused and merged on top of the new +layers. This also means you can easily rebase your images when the base images +receive updates, without having to execute the whole build again. In backends +that support it, BuildKit can do this rebase action without the need to push or +pull any layers between the client and the registry. BuildKit will detect this +case and only create new image manifest that contains the new layers and old +layers in correct order. + +The same behavior where BuildKit can avoid pulling down the base image can also +happen when using `--link` and no other commands that would require access to +the files in the base image. In that case BuildKit will only build the layers +for the `COPY` commands and push them to the registry directly on top of the +layers of the base image. + +### Incompatibilities with `--link=false` + +When using `--link` the `COPY/ADD` commands are not allowed to read any files +from the previous state. This means that if in previous state the destination +directory was a path that contained a symlink, `COPY/ADD` can not follow it. +In the final image the destination path created with `--link` will always be a +path containing only directories. + +If you don't rely on the behavior of following symlinks in the destination +path, using `--link` is always recommended. The performance of `--link` is +equivalent or better than the default behavior and, it creates much better +conditions for cache reuse. + ## ENTRYPOINT ENTRYPOINT has two forms: @@ -2044,6 +2367,36 @@ ARG TARGETPLATFORM RUN echo "I'm building for $TARGETPLATFORM" ``` +### BuildKit built-in build args + +| Arg | Type | Description | +|---------------------------------------|---------|--------------------------------------------------------------------------| +| `BUILDKIT_CACHE_MOUNT_NS` | String | Set optional cache ID namespace. | +| `BUILDKIT_CONTEXT_KEEP_GIT_DIR` | Bool | Trigger git context to keep the `.git` directory. | +| `BUILDKIT_INLINE_BUILDINFO_ATTRS`[^2] | Bool | Inline build info attributes in image config or not. | +| `BUILDKIT_INLINE_CACHE`[^2] | Bool | Inline cache metadata to image config or not. | +| `BUILDKIT_MULTI_PLATFORM` | Bool | Opt into determnistic output regardless of multi-platform output or not. | +| `BUILDKIT_SANDBOX_HOSTNAME` | String | Set the hostname (default `buildkitsandbox`) | +| `BUILDKIT_SYNTAX` | String | Set frontend image | + +#### Example: keep `.git` dir + +When using a Git context, `.git` dir is not kept on git checkouts. It can be +useful to keep it around if you want to retrieve git information during +your build: + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +WORKDIR /src +RUN --mount=target=. \ + make REVISION=$(git rev-parse HEAD) build +``` + +```console +$ docker build --build-arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 https://github.com/user/repo.git#main +``` + ### Impact on build caching `ARG` variables are not persisted into the built image as `ENV` variables are. @@ -2375,6 +2728,87 @@ environment variable expansion semantics could be modified. The `SHELL` instruction can also be used on Linux should an alternate shell be required such as `zsh`, `csh`, `tcsh` and others. +## Here-Documents + +> **Note** +> +> Added in [`docker/dockerfile:1.4`](#syntax) + +Here-documents allow redirection of subsequent Dockerfile lines to the input of +`RUN` or `COPY` commands. If such command contains a [here-document](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04) +the Dockerfile considers the next lines until the line only containing a +here-doc delimiter as part of the same command. + +### Example: Running a multi-line script + +```dockerfile +# syntax=docker/dockerfile:1 +FROM debian +RUN < file1 && < file2 +I am +first +FILE1 +I am +second +FILE2 +``` + +### Example: Creating inline files + +In `COPY` commands source parameters can be replaced with here-doc indicators. +Regular here-doc [variable expansion and tab stripping rules](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04) apply. + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +ARG FOO=bar +COPY <<-EOT /app/foo + hello ${FOO} +EOT +``` + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +COPY <<-"EOT" /app/script.sh + echo hello ${FOO} +EOT +RUN FOO=abc ash /app/script.sh +``` + ## Dockerfile examples For examples of Dockerfiles, refer to: @@ -2382,3 +2816,6 @@ For examples of Dockerfiles, refer to: - The ["build images" section](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) - The ["get started](https://docs.docker.com/get-started/) - The [language-specific getting started guides](https://docs.docker.com/language/) + +[^1]: Value required +[^2]: For Docker-integrated BuildKit (`DOCKER_BUILDKIT=1 docker build`) and `docker buildx build` diff --git a/frontend/dockerfile/docs/syntax.md b/frontend/dockerfile/docs/syntax.md index 33ff1cfbb..fc84bdaae 100644 --- a/frontend/dockerfile/docs/syntax.md +++ b/frontend/dockerfile/docs/syntax.md @@ -1,376 +1,3 @@ # Dockerfile frontend syntaxes -This page documents new BuildKit-only commands added to the Dockerfile frontend. - -## Note for Docker users - -If you are using Docker v18.09 or later, BuildKit mode can be enabled by setting `export DOCKER_BUILDKIT=1` on the client side. - -[Docker Buildx](https://github.com/docker/buildx) always enables BuildKit. - -## Using external Dockerfile frontend - -BuildKit supports loading frontends dynamically from container images. Images for Dockerfile frontends are available at [`docker/dockerfile`](https://hub.docker.com/r/docker/dockerfile/tags/) repository. - -To use the external frontend, the first line of your Dockerfile needs to be `# syntax=docker/dockerfile:1.3` pointing to the -specific image you want to use. - -BuildKit also ships with Dockerfile frontend builtin but it is recommended to use an external image to make sure that all -users use the same version on the builder and to pick up bugfixes automatically without waiting for a new version of BuildKit -or Docker engine. - -The images are published on two channels: *latest* and *labs*. The latest channel uses semver versioning while labs uses an -[incrementing number](https://github.com/moby/buildkit/issues/528). This means the labs channel may remove a feature without -incrementing the major component of a version and you may want to pin the image to a specific revision. Even when syntaxes -change in between releases on labs channel, the old versions are guaranteed to be backward compatible. - - -## Linked copies `COPY --link`, `ADD --link` - -To use this flag set Dockerfile version to at least `1.4`. - -```dockerfile -# syntax=docker/dockerfile:1.4 -``` - -Enabling this flag in `COPY` or `ADD` commands allows you to copy files with enhanced semantics where your files remain independent on their own layer and don't get invalidated when commands on previous layers are changed. - -When `--link` is used your source files are copied into an empty destination directory. That directory is turned into a layer that is linked on top of your previous state. - -```dockerfile -# syntax=docker/dockerfile:1.4 -FROM alpine -COPY --link /foo /bar -``` - -Is equivalent of doing two builds: - -```dockerfile -FROM alpine -``` - -and - -```dockerfile -FROM scratch -COPY /foo /bar -``` - -and merging all the layers of both images together. - -#### Benefits of using `--link` - -Using `--link` allows to reuse already built layers in subsequent builds with `--cache-from` even if the previous layers have changed. This is especially important for multi-stage builds where a `COPY --from` statement would previously get invalidated if any previous commands in the same stage changed, causing the need to rebuild the intermediate stages again. With `--link` the layer the previous build generated is reused and merged on top of the new layers. This also means you can easily rebase your images when the base images receive updates, without having to execute the whole build again. In backends that support it, BuildKit can do this rebase action without the need to push or pull any layers between the client and the registry. BuildKit will detect this case and only create new image manifest that contains the new layers and old layers in correct order. - -The same behavior where BuildKit can avoid pulling down the base image can also happen when using `--link` and no other commands that would require access to the files in the base image. In that case BuildKit will only build the layers for the `COPY` commands and push them to the registry directly on top of the layers of the base image. - -#### Incompatibilities with `--link=false` - -When using `--link` the `COPY/ADD` commands are not allowed to read any files from the previous state. This means that if in previous state the destination directory was a path that contained a symlink, `COPY/ADD` can not follow it. In the final image the destination path created with `--link` will always be a path containing only directories. - -If you don't rely on the behavior of following symlinks in the destination path, using `--link` is always recommended. The performance of `--link` is equivalent or better than the default behavior and it creates much better conditions for cache reuse. - - -## Build Mounts `RUN --mount=...` - -To use this flag set Dockerfile version to at least `1.2` - -``` -# syntax=docker/dockerfile:1.3 -``` - -`RUN --mount` allows you to create mounts that process running as part of the build can access. This can be used to bind -files from other part of the build without copying, accessing build secrets or ssh-agent sockets, or creating cache -locations to speed up your build. - -### `RUN --mount=type=bind` (the default mount type) - -This mount type allows binding directories (read-only) in the context or in an image to the build container. - -|Option |Description| -|---------------------|-----------| -|`target` (required) | Mount path.| -|`source` | Source path in the `from`. Defaults to the root of the `from`.| -|`from` | Build stage or image name for the root of the source. Defaults to the build context.| -|`rw`,`readwrite` | Allow writes on the mount. Written data will be discarded.| - -### `RUN --mount=type=cache` - -This mount type allows the build container to cache directories for compilers and package managers. - -|Option |Description| -|---------------------|-----------| -|`id` | Optional ID to identify separate/different caches. Defaults to value of `target`. | -|`target` (required) | Mount path.| -|`ro`,`readonly` | Read-only if set.| -|`sharing` | One of `shared`, `private`, or `locked`. Defaults to `shared`. A `shared` cache mount can be used concurrently by multiple writers. `private` creates a new mount if there are multiple writers. `locked` pauses the second writer until the first one releases the mount.| -|`from` | Build stage to use as a base of the cache mount. Defaults to empty directory.| -|`source` | Subpath in the `from` to mount. Defaults to the root of the `from`.| -|`mode` | File mode for new cache directory in octal. Default 0755.| -|`uid` | User ID for new cache directory. Default 0.| -|`gid` | Group ID for new cache directory. Default 0.| - -Contents of the cache directories persists between builder invocations without invalidating the -instruction cache. Cache mounts should only be used for better performance. Your build should work -with any contents of the cache directory as another build may overwrite the files or GC may clean -it if more storage space is needed. - - -#### Example: cache Go packages - -```dockerfile -# syntax = docker/dockerfile:1.3 -FROM golang -... -RUN --mount=type=cache,target=/root/.cache/go-build go build ... -``` - -#### Example: cache apt packages - -```dockerfile -# syntax = docker/dockerfile:1.3 -FROM ubuntu -RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache -RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt \ - apt update && apt-get --no-install-recommends install -y gcc -``` - -### `RUN --mount=type=tmpfs` - -This mount type allows mounting tmpfs in the build container. - -|Option |Description| -|---------------------|-----------| -|`target` (required) | Mount path.| -|`size` | Specify an upper limit on the size of the filesystem.| - - -### `RUN --mount=type=secret` - -This mount type allows the build container to access secure files such as private keys without baking them into the image. - -|Option |Description| -|---------------------|-----------| -|`id` | ID of the secret. Defaults to basename of the target path.| -|`target` | Mount path. Defaults to `/run/secrets/` + `id`.| -|`required` | If set to `true`, the instruction errors out when the secret is unavailable. Defaults to `false`.| -|`mode` | File mode for secret file in octal. Default 0400.| -|`uid` | User ID for secret file. Default 0.| -|`gid` | Group ID for secret file. Default 0.| - - -#### Example: access to S3 - -```dockerfile -# syntax = docker/dockerfile:1.3 -FROM python:3 -RUN pip install awscli -RUN --mount=type=secret,id=aws,target=/root/.aws/credentials aws s3 cp s3://... ... -``` - -```console -$ docker build --secret id=aws,src=$HOME/.aws/credentials . -``` - -```console -$ buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. \ - --secret id=aws,src=$HOME/.aws/credentials -``` - -### `RUN --mount=type=ssh` - -This mount type allows the build container to access SSH keys via SSH agents, with support for passphrases. - -|Option |Description| -|---------------------|-----------| -|`id` | ID of SSH agent socket or key. Defaults to "default".| -|`target` | SSH agent socket path. Defaults to `/run/buildkit/ssh_agent.${N}`.| -|`required` | If set to `true`, the instruction errors out when the key is unavailable. Defaults to `false`.| -|`mode` | File mode for socket in octal. Default 0600.| -|`uid` | User ID for socket. Default 0.| -|`gid` | Group ID for socket. Default 0.| - - -#### Example: access to Gitlab - -```dockerfile -# syntax = docker/dockerfile:1.3 -FROM alpine -RUN apk add --no-cache openssh-client -RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts -RUN --mount=type=ssh ssh -q -T git@gitlab.com 2>&1 | tee /hello -# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here -# with the type of build progress is defined as `plain`. -``` - -```console -$ eval $(ssh-agent) -$ ssh-add ~/.ssh/id_rsa -(Input your passphrase here) -$ docker build --ssh default=$SSH_AUTH_SOCK . -``` - -``` -$ buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. \ - --ssh default=$SSH_AUTH_SOCK -``` - -You can also specify a path to `*.pem` file on the host directly instead of `$SSH_AUTH_SOCK`. -However, pem files with passphrases are not supported. - - -## Network modes `RUN --network=none|host|default` - -``` -# syntax=docker/dockerfile:1.3 -``` - -`RUN --network` allows control over which networking environment the command is run in. - -The allowed values are: - -* `none` - The command is run with no network access (`lo` is still available, - but is isolated to this process) -* `host` - The command is run in the host's network environment (similar to - `docker build --network=host`, but on a per-instruction basis) -* `default` - Equivalent to not supplying a flag at all, the command is run in - the default network for the build - -The use of `--network=host` is protected by the `network.host` entitlement, -which needs to be enabled when starting the buildkitd daemon -(`--allow-insecure-entitlement network.host`) and on the build request -(`--allow network.host`). - -#### Example: isolating external effects - -```dockerfile -# syntax = docker/dockerfile:1.3 -FROM python:3.6 -ADD mypackage.tgz wheels/ -RUN --network=none pip install --find-links wheels mypackage -``` - -`pip` will only be able to install the packages provided in the tarfile, which -can be controlled by an earlier build stage. - -## Here-Documents - -This feature is available since `docker/dockerfile:1.4.0` release. - -``` -# syntax=docker/dockerfile:1.4 -``` - -Here-documents allow redirection of subsequent Dockerfile lines to the input of `RUN` or `COPY` commands. -If such command contains a [here-document](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04) -Dockerfile will consider the next lines until the line only containing a here-doc delimiter as part of the same command. - -#### Example: running a multi-line script - -```dockerfile -# syntax = docker/dockerfile:1.4 -FROM debian -RUN < file1 && < file2 -I am -first -FILE1 -I am -second -FILE2 -``` - -#### Example: creating inline files - -In `COPY` commands source parameters can be replaced with here-doc indicators. -Regular here-doc [variable expansion and tab stripping rules](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04) apply. - -```dockerfile -# syntax = docker/dockerfile:1.4 -FROM alpine -ARG FOO=bar -COPY <<-eot /app/foo - hello ${FOO} -eot -``` - -```dockerfile -# syntax = docker/dockerfile:1.4 -FROM alpine -COPY <<-"eot" /app/script.sh - echo hello ${FOO} -eot -RUN FOO=abc ash /app/script.sh -``` - -## Security context `RUN --security=insecure|sandbox` - -To use this flag, set Dockerfile version to `labs` channel. - -``` -# syntax=docker/dockerfile:1.3-labs -``` - -With `--security=insecure`, builder runs the command without sandbox in insecure mode, -which allows to run flows requiring elevated privileges (e.g. containerd). This is equivalent -to running `docker run --privileged`. In order to access this feature, entitlement -`security.insecure` should be enabled when starting the buildkitd daemon -(`--allow-insecure-entitlement security.insecure`) and for a build request -(`--allow security.insecure`). - -Default sandbox mode can be activated via `--security=sandbox`, but that is no-op. - -#### Example: check entitlements - -```dockerfile -# syntax = docker/dockerfile:1.3-labs -FROM ubuntu -RUN --security=insecure cat /proc/self/status | grep CapEff -``` - -``` -#84 0.093 CapEff: 0000003fffffffff -``` - -## Built-in build args - -* `BUILDKIT_CACHE_MOUNT_NS=` set optional cache ID namespace -* `BUILDKIT_CONTEXT_KEEP_GIT_DIR=` trigger git context to keep the `.git` directory -* `BUILDKIT_INLINE_BUILDINFO_ATTRS=`¹ inline build info attributes in image config or not -* `BUILDKIT_INLINE_CACHE=`¹ inline cache metadata to image config or not -* `BUILDKIT_MULTI_PLATFORM=` opt into determnistic output regardless of multi-platform output or not -* `BUILDKIT_SANDBOX_HOSTNAME=` set the hostname (default `buildkitsandbox`) -* `BUILDKIT_SYNTAX=` set frontend image - -> **¹** For Docker-integrated BuildKit (`DOCKER_BUILDKIT=1 docker build`) and `docker buildx` +This page has moved to [Dockerfile reference documentation](reference.md)