diff --git a/docs/deprecated.md b/docs/deprecated.md new file mode 100644 index 0000000000..37ffcbc470 --- /dev/null +++ b/docs/deprecated.md @@ -0,0 +1,1372 @@ +--- +title: Deprecated Docker Engine features +linkTitle: Deprecated features +aliases: ["/engine/misc/deprecated/"] +description: "Deprecated Features." +keywords: "docker, documentation, about, technology, deprecate" +--- + + + +This page provides an overview of features that are deprecated in Engine. Changes +in packaging, and supported (Linux) distributions are not included. To learn +about end of support for Linux distributions, refer to the +[release notes](https://docs.docker.com/engine/release-notes/). + +## Feature deprecation policy + +As changes are made to Docker there may be times when existing features need to +be removed or replaced with newer features. Before an existing feature is removed +it is labeled as "deprecated" within the documentation and remains in Docker for +at least one stable release unless specified explicitly otherwise. After that time +it may be removed. + +Users are expected to take note of the list of deprecated features each release +and plan their migration away from those features, and (if applicable) towards +the replacement features as soon as possible. + +## Deprecated engine features + +The following table provides an overview of the current status of deprecated features: + +- **Deprecated**: the feature is marked "deprecated" and should no longer be used. + + The feature may be removed, disabled, or change behavior in a future release. + The _"Deprecated"_ column contains the release in which the feature was marked + deprecated, whereas the _"Remove"_ column contains a tentative release in which + the feature is to be removed. If no release is included in the _"Remove"_ column, + the release is yet to be decided on. + +- **Removed**: the feature was removed, disabled, or hidden. + + Refer to the linked section for details. Some features are "soft" deprecated, + which means that they remain functional for backward compatibility, and to + allow users to migrate to alternatives. In such cases, a warning may be + printed, and users should not rely on this feature. + +| Status | Feature | Deprecated | Remove | +|------------|------------------------------------------------------------------------------------------------------------------------------------|------------|--------| +| Deprecated | [Support for cgroup v1](#support-for-cgroup-v1) | v29.0 | - | +| Deprecated | [`--pause` option on `docker commit`](#--pause-option-on-docker-commit) | v29.0 | v30.0 | +| Deprecated | [Legacy links environment variables](#legacy-links-environment-variables) | v28.4 | v30.0 | +| Deprecated | [Special handling for quoted values for TLS flags](#special-handling-for-quoted-values-for-tls-flags) | v28.4 | v29.0 | +| Deprecated | [Empty/nil fields in image Config from inspect API](#emptynil-fields-in-image-config-from-inspect-api) | v28.3 | v29.0 | +| Deprecated | [Configuration for pushing non-distributable artifacts](#configuration-for-pushing-non-distributable-artifacts) | v28.0 | v29.0 | +| Deprecated | [`--time` option on `docker stop` and `docker restart`](#--time-option-on-docker-stop-and-docker-restart) | v28.0 | - | +| Removed | [Non-standard fields in image inspect](#non-standard-fields-in-image-inspect) | v27.0 | v28.2 | +| Removed | [API CORS headers](#api-cors-headers) | v27.0 | v28.0 | +| Removed | [Graphdriver plugins (experimental)](#graphdriver-plugins-experimental) | v27.0 | v28.0 | +| Deprecated | [Unauthenticated TCP connections](#unauthenticated-tcp-connections) | v26.0 | v28.0 | +| Removed | [`Container` and `ContainerConfig` fields in Image inspect](#container-and-containerconfig-fields-in-image-inspect) | v25.0 | v26.0 | +| Removed | [Deprecate legacy API versions](#deprecate-legacy-api-versions) | v25.0 | v26.0 | +| Removed | [Container short ID in network Aliases field](#container-short-id-in-network-aliases-field) | v25.0 | v26.0 | +| Removed | [Mount `bind-nonrecursive` option](#mount-bind-nonrecursive-option) | v25.0 | v29.0 | +| Removed | [IsAutomated field, and `is-automated` filter on `docker search`](#isautomated-field-and-is-automated-filter-on-docker-search) | v25.0 | v28.2 | +| Removed | [logentries logging driver](#logentries-logging-driver) | v24.0 | v25.0 | +| Removed | [OOM-score adjust for the daemon](#oom-score-adjust-for-the-daemon) | v24.0 | v25.0 | +| Removed | [BuildKit build information](#buildkit-build-information) | v23.0 | v24.0 | +| Deprecated | [Legacy builder for Linux images](#legacy-builder-for-linux-images) | v23.0 | - | +| Deprecated | [Legacy builder fallback](#legacy-builder-fallback) | v23.0 | - | +| Removed | [Btrfs storage driver on CentOS 7 and RHEL 7](#btrfs-storage-driver-on-centos-7-and-rhel-7) | v20.10 | v23.0 | +| Removed | [Support for encrypted TLS private keys](#support-for-encrypted-tls-private-keys) | v20.10 | v23.0 | +| Removed | [Kubernetes stack and context support](#kubernetes-stack-and-context-support) | v20.10 | v23.0 | +| Removed | [Pulling images from non-compliant image registries](#pulling-images-from-non-compliant-image-registries) | v20.10 | v28.2 | +| Removed | [Linux containers on Windows (LCOW)](#linux-containers-on-windows-lcow-experimental) | v20.10 | v23.0 | +| Deprecated | [BLKIO weight options with cgroups v1](#blkio-weight-options-with-cgroups-v1) | v20.10 | - | +| Removed | [Kernel memory limit](#kernel-memory-limit) | v20.10 | v23.0 | +| Removed | [Classic Swarm and overlay networks using external key/value stores](#classic-swarm-and-overlay-networks-using-cluster-store) | v20.10 | v23.0 | +| Removed | [Support for the legacy `~/.dockercfg` configuration file for authentication](#support-for-legacy-dockercfg-configuration-files) | v20.10 | v23.0 | +| Deprecated | [CLI plugins support](#cli-plugins-support) | v20.10 | - | +| Deprecated | [Dockerfile legacy `ENV name value` syntax](#dockerfile-legacy-env-name-value-syntax) | v20.10 | - | +| Removed | [`docker build --stream` flag (experimental)](#docker-build---stream-flag-experimental) | v20.10 | v20.10 | +| Removed | [`fluentd-async-connect` log opt](#fluentd-async-connect-log-opt) | v20.10 | v28.0 | +| Removed | [Configuration options for experimental CLI features](#configuration-options-for-experimental-cli-features) | v19.03 | v23.0 | +| Removed | [Pushing and pulling with image manifest v2 schema 1](#pushing-and-pulling-with-image-manifest-v2-schema-1) | v19.03 | v28.2 | +| Removed | [`docker engine` subcommands](#docker-engine-subcommands) | v19.03 | v20.10 | +| Removed | [Top-level `docker deploy` subcommand (experimental)](#top-level-docker-deploy-subcommand-experimental) | v19.03 | v20.10 | +| Removed | [`docker stack deploy` using "dab" files (experimental)](#docker-stack-deploy-using-dab-files-experimental) | v19.03 | v20.10 | +| Removed | [Support for the `overlay2.override_kernel_check` storage option](#support-for-the-overlay2override_kernel_check-storage-option) | v19.03 | v24.0 | +| Removed | [AuFS storage driver](#aufs-storage-driver) | v19.03 | v24.0 | +| Removed | [Legacy "overlay" storage driver](#legacy-overlay-storage-driver) | v18.09 | v24.0 | +| Removed | [Device mapper storage driver](#device-mapper-storage-driver) | v18.09 | v25.0 | +| Removed | [Use of reserved namespaces in engine labels](#use-of-reserved-namespaces-in-engine-labels) | v18.06 | v20.10 | +| Removed | [`--disable-legacy-registry` override daemon option](#--disable-legacy-registry-override-daemon-option) | v17.12 | v19.03 | +| Removed | [Interacting with V1 registries](#interacting-with-v1-registries) | v17.06 | v17.12 | +| Removed | [Asynchronous `service create` and `service update` as default](#asynchronous-service-create-and-service-update-as-default) | v17.05 | v17.10 | +| Removed | [`-g` and `--graph` flags on `dockerd`](#-g-and---graph-flags-on-dockerd) | v17.05 | v23.0 | +| Deprecated | [Top-level network properties in NetworkSettings](#top-level-network-properties-in-networksettings) | v1.13 | v17.12 | +| Removed | [`filter` option for `/images/json` endpoint](#filter-option-for-imagesjson-endpoint) | v1.13 | v20.10 | +| Removed | [`repository:shortid` image references](#repositoryshortid-image-references) | v1.13 | v17.12 | +| Removed | [`docker daemon` subcommand](#docker-daemon-subcommand) | v1.13 | v17.12 | +| Removed | [Duplicate keys with conflicting values in engine labels](#duplicate-keys-with-conflicting-values-in-engine-labels) | v1.13 | v17.12 | +| Deprecated | [`MAINTAINER` in Dockerfile](#maintainer-in-dockerfile) | v1.13 | - | +| Deprecated | [API calls without a version](#api-calls-without-a-version) | v1.13 | v17.12 | +| Removed | [Backing filesystem without `d_type` support for overlay/overlay2](#backing-filesystem-without-d_type-support-for-overlayoverlay2) | v1.13 | v17.12 | +| Removed | [`--automated` and `--stars` flags on `docker search`](#--automated-and---stars-flags-on-docker-search) | v1.12 | v20.10 | +| Deprecated | [`-h` shorthand for `--help`](#-h-shorthand-for---help) | v1.12 | v17.09 | +| Removed | [`-e` and `--email` flags on `docker login`](#-e-and---email-flags-on-docker-login) | v1.11 | v17.06 | +| Deprecated | [Separator (`:`) of `--security-opt` flag on `docker run`](#separator--of---security-opt-flag-on-docker-run) | v1.11 | v17.06 | +| Deprecated | [Links on the default bridge network](#links-on-the-default-bridge-network) | v1.10 | - | +| Deprecated | [Ambiguous event fields in API](#ambiguous-event-fields-in-api) | v1.10 | - | +| Removed | [`-f` flag on `docker tag`](#-f-flag-on-docker-tag) | v1.10 | v1.12 | +| Removed | [HostConfig at API container start](#hostconfig-at-api-container-start) | v1.10 | v1.12 | +| Removed | [`--before` and `--since` flags on `docker ps`](#--before-and---since-flags-on-docker-ps) | v1.10 | v1.12 | +| Removed | [Driver-specific log tags](#driver-specific-log-tags) | v1.9 | v1.12 | +| Removed | [Docker Content Trust `ENV` passphrase variables name change](#docker-content-trust-env-passphrase-variables-name-change) | v1.9 | v1.12 | +| Removed | [`/containers/(id or name)/copy` endpoint](#containersid-or-namecopy-endpoint) | v1.8 | v1.12 | +| Removed | [LXC built-in exec driver](#lxc-built-in-exec-driver) | v1.8 | v1.10 | +| Removed | [Old Command Line Options](#old-command-line-options) | v1.8 | v1.10 | +| Removed | [`--api-enable-cors` flag on `dockerd`](#--api-enable-cors-flag-on-dockerd) | v1.6 | v17.09 | +| Removed | [`--run` flag on `docker commit`](#--run-flag-on-docker-commit) | v0.10 | v1.13 | +| Removed | [Three arguments form in `docker import`](#three-arguments-form-in-docker-import) | v0.6.7 | v1.12 | + +### Support for cgroup v1 + +**Deprecated in release: v29.0** + +Support for cgroup v1 is deprecated in the v29.0 release, however, it will continue +to be supported until May 2029. +The latest release in May 2029 may not necessarily support cgroup v1, +but there will be at least one maintained branch with the support for cgroup v1. + +The cgroup version currently in use can be checked by running the `docker info` command: + +```console +$ docker info +<...> +Server: + <...> + Cgroup Version: 2 + <...> +``` + +### `--pause` option on `docker commit` + +**Deprecated in release: v29.0** + +**Target for removal in release: v30.0** + +The `--pause` option is enabled by default since Docker v1.1.0 to prevent +committing containers in an inconsistent state, but can be disabled by +setting the `--pause=false` option. In docker CLI v29.0 this flag is +replaced by a `--no-pause` flag instead. The `--pause` option is still +functional in the v29.0 release, printing a deprecation warning, but +will be removed in docker CLI v30. + +### Legacy links environment variables + +**Deprecated in release: v28.4** + +**Disabled by default in release: v29.0** + +**Target for removal in release: v30.0** + +Containers attached to the default bridge network can specify "legacy links" (e.g. +using `--links` on the CLI) to get access to other containers attached to that +network. The linking container (i.e., the container created with `--links`) automatically +gets environment variables that specify the IP address and port mappings of the linked +container. However, these environment variables are prefixed with the linked +container's names, making them impractical. + +Starting with Docker v29.0, these environment variables are no longer set by +default. Users who still depend on them can start Docker Engine with the +environment variable `DOCKER_KEEP_DEPRECATED_LEGACY_LINKS_ENV_VARS=1` set. + +Support for legacy links environment variables, as well as the `DOCKER_KEEP_DEPRECATED_LEGACY_LINKS_ENV_VARS` +will be removed in Docker Engine v30.0. + +### Special handling for quoted values for TLS flags + +**Deprecated in release: v28.4** + +**Target for removal in release: v29.0** + +The `--tlscacert`, `--tlscert`, and `--tlskey` command-line flags had +non-standard behavior for handling values contained in quotes (`"` or `'`). +Normally, quotes are handled by the shell, for example, in the following +example, the shell takes care of handling quotes before passing the values +to the `docker` CLI: + +```console +docker --some-option "some-value-in-quotes" ... +``` + +However, when passing values using an equal sign (`=`), this may not happen +and values may be handled including quotes; + +```console +docker --some-option="some-value-in-quotes" ... +``` + +This caused issues with "Docker Machine", which used this format as part +of its `docker-machine config` output, and the CLI carried special, non-standard +handling for these flags. + +Docker Machine reached EOL, and this special handling made the processing +of flag values inconsistent with other flags used, so this behavior is +deprecated. Users depending on this behavior are recommended to specify +the quoted values using a space between the flag and its value, as illustrated +above. + +### Empty/nil fields in image Config from inspect API + +**Deprecated in release: v28.3** + +**Target for removal in release: v29.0** + +The `Config` field returned by `docker image inspect` (and the `GET /images/{name}/json` +API endpoint) currently includes certain fields even when they are empty or nil. +Starting in Docker v29.0, the following fields will be omitted from the API response +when they contain empty or default values: + +- `Cmd` +- `Entrypoint` +- `Env` +- `Labels` +- `OnBuild` +- `User` +- `Volumes` +- `WorkingDir` + +Applications consuming the image inspect API should be updated to handle the +absence of these fields gracefully, treating missing fields as having their +default/empty values. + +For API version corresponding to Docker v29.0, these fields will be omitted when +empty. They will continue to be included when using clients that request an older +API version for backward compatibility. + +### Configuration for pushing non-distributable artifacts + +**Deprecated in release: v28.0** + +**Target for removal in release: v29.0** + +Non-distributable artifacts (also called foreign layers) were introduced in +docker v1.12 to accommodate Windows images for which the EULA did not allow +layers to be distributed through registries other than those hosted by Microsoft. +The concept of foreign / non-distributable layers was adopted by the OCI distribution +spec in [oci#233]. These restrictions were relaxed later to allow distributing +these images through non-public registries, for which a configuration was added +in Docker v17.0.6.0. + +In 2022, Microsoft updated the EULA and [removed these restrictions][msft-3645201], +followed by the OCI distribution specification deprecating foreign layers in [oci#965]. +In 2023, Microsoft [removed the use of foreign data layers][msft-3846833] for their images, +making this functionality obsolete. + +Docker v28.0 deprecates the `--allow-nondistributable-artifacts` daemon flag and +corresponding `allow-nondistributable-artifacts` field in `daemon.json`. Setting +either option no longer takes an effect, but a deprecation warning log is added +to raise awareness about the deprecation. This warning is planned to become an +error in the Docker v29.0. + +Users currently using these options are therefore recommended to remove this +option from their configuration to prevent the daemon from starting when +upgrading to Docker v29.0. + +The `AllowNondistributableArtifactsCIDRs` and `AllowNondistributableArtifactsHostnames` +fields in the `RegistryConfig` of the `GET /info` API response are also deprecated. +For API version v1.48 and lower, the fields are still included in the response +but always `null`. In API version v1.49 and higher, the field will be omitted +entirely. + +[oci#233]: https://github.com/opencontainers/image-spec/pull/233 +[oci#965]: https://github.com/opencontainers/image-spec/pull/965 +[msft-3645201]: https://techcommunity.microsoft.com/blog/containers/announcing-windows-container-base-image-redistribution-rights-change/3645201 +[msft-3846833]: https://techcommunity.microsoft.com/blog/containers/announcing-removal-of-foreign-layers-from-windows-container-images/3846833 + +### `--time` option on `docker stop` and `docker restart` + +**Deprecated in release: v28.0** + +The `--time` option for the `docker stop`, `docker container stop`, `docker restart`, +and `docker container restart` commands has been renamed to `--timeout` for +consistency with other uses of timeout options. The `--time` option is now +deprecated and hidden, but remains functional for backward compatibility. +Users are encouraged to migrate to using the `--timeout` option instead. + +### Non-standard fields in image inspect + +**Deprecated in release: v27.0** + +**Removed in release: v28.2** + +The `Config` field returned shown in `docker image inspect` (and as returned by +the `GET /images/{name}/json` API endpoint) returns additional fields that are +not part of the image's configuration and not part of the [Docker image specification] +and [OCI image specification]. + +These fields are never set (and always return the default value for the type), +but are not omitted in the response when left empty. As these fields were not +intended to be part of the image configuration response, they are deprecated, +and will be removed from the API in thee next release. + +The following fields are not part of the underlying image's `Config` field, and +removed in the API response for API v1.50 and newer, corresponding with v28.2. +They continue to be included when using clients that use an older API version: + +- `Hostname` +- `Domainname` +- `AttachStdin` +- `AttachStdout` +- `AttachStderr` +- `Tty` +- `OpenStdin` +- `StdinOnce` +- `Image` +- `NetworkDisabled` (omitted unless set on older API versions) +- `MacAddress` (omitted unless set on older API versions) +- `StopTimeout` (omitted unless set on older API versions) + +[Docker image specification]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32 +[OCI image specification]: https://github.com/opencontainers/image-spec/blob/v1.1.0/specs-go/v1/config.go#L24-L62 + +### Graphdriver plugins (experimental) + +**Deprecated in**: v27.0**. + +**Disabled by default in release: v27.0** + +**Target for removal in release: v28.0** + +[Graphdriver plugins](https://github.com/docker/cli/blob/v26.1.4/docs/extend/plugins_graphdriver.md) +were an experimental feature that allowed extending the Docker Engine with custom +storage drivers for storing images and containers. This feature was not +maintained since its inception. + +Support for graphdriver plugins was disabled by default in v27.0, and removed +in v28.0. Users of this feature are recommended to instead configure the Docker +Engine to use the [containerd image store](https://docs.docker.com/storage/containerd/) +and a custom [snapshotter](https://github.com/containerd/containerd/tree/v1.7.18/docs/snapshotters) + +### API CORS headers + +**Deprecated in release: v27.0** + +**Disabled by default in release: v27.0** + +**Removed in release: v28.0** + +The `api-cors-header` configuration option for the Docker daemon is insecure, +and is therefore deprecated and scheduled for removal. +Incorrectly setting this option could leave a window of opportunity +for unauthenticated cross-origin requests to be accepted by the daemon. + +In Docker Engine v27.0, this flag can still be set, +but it has no effect unless the environment variable +`DOCKERD_DEPRECATED_CORS_HEADER` is also set to a non-empty value. + +This flag has been removed altogether in v28.0. + +This is a breaking change for authorization plugins and other programs +that depend on this option for accessing the Docker API from a browser. +If you need to access the API through a browser, use a reverse proxy. + +### Unauthenticated TCP connections + +**Deprecated in release: v26.0** + +**Target for removal in release: v28.0** + +Configuring the Docker daemon to listen on a TCP address will require mandatory +TLS verification. This change aims to ensure secure communication by preventing +unauthorized access to the Docker daemon over potentially insecure networks. +This mandatory TLS requirement applies to all TCP addresses except `tcp://localhost`. + +In version 27.0 and later, specifying `--tls=false` or `--tlsverify=false` CLI flags +causes the daemon to fail to start if it's also configured to accept remote connections over TCP. +This also applies to the equivalent configuration options in `daemon.json`. + +To facilitate remote access to the Docker daemon over TCP, you'll need to +implement TLS verification. This secures the connection by encrypting data in +transit and providing a mechanism for mutual authentication. + +For environments remote daemon access isn't required, +we recommend binding the Docker daemon to a Unix socket. +For daemons where remote access is required and where TLS encryption is not feasible, +you may want to consider using SSH as an alternative solution. + +For further information, assistance, and step-by-step instructions on +configuring TLS (or SSH) for the Docker daemon, refer to +[Protect the Docker daemon socket](https://docs.docker.com/engine/security/protect-access/). + +### `Container` and `ContainerConfig` fields in Image inspect + +**Deprecated in release: v25.0** + +**Removed in release: v26.0** + +The `Container` and `ContainerConfig` fields returned by `docker inspect` are +mostly an implementation detail of the classic (non-BuildKit) image builder. +These fields are not portable and are empty when using the +BuildKit-based builder (enabled by default since v23.0). +These fields are deprecated in v25.0 and are omitted starting from v26.0 ( +API version v1.45 and up). If image configuration of an image is needed, +you can obtain it from the `Config` field. + +### Deprecate legacy API versions + +**Deprecated in release: v25.0** + +**Target for removal in release: v26.0** + +The Docker daemon provides a versioned API for backward compatibility with old +clients. Docker clients can perform API-version negotiation to select the most +recent API version supported by the daemon (downgrading to and older version of +the API when necessary). API version negotiation was introduced in Docker v1.12.0 +(API 1.24), and clients before that used a fixed API version. + +Docker Engine versions through v25.0 provide support for all [API versions](https://docs.docker.com/engine/api/#api-version-matrix) +included in stable releases for a given platform. For Docker daemons on Linux, +the earliest supported API version is 1.12 (corresponding with Docker Engine +v1.0.0), whereas for Docker daemons on Windows, the earliest supported API +version is 1.24 (corresponding with Docker Engine v1.12.0). + +Support for legacy API versions (providing old API versions on current versions +of the Docker Engine) is primarily intended to provide compatibility with recent, +but still supported versions of the client, which is a common scenario (the Docker +daemon may be updated to the latest release, but not all clients may be up-to-date +or vice versa). Support for API versions before that (API versions provided by +EOL versions of the Docker Daemon) is provided on a "best effort" basis. + +Use of old API versions is rare, and support for legacy API versions +involves significant complexity (Docker 1.0.0 having been released 10 years ago). +Because of this, we'll start deprecating support for legacy API versions. + +Docker Engine v25.0 by default disables API version older than 1.24 (aligning +the minimum supported API version between Linux and Windows daemons). When +connecting with a client that uses an API version older than 1.24, +the daemon returns an error. The following example configures the Docker +CLI to use API version 1.23, which produces an error: + +```console +DOCKER_API_VERSION=1.23 docker version +Error response from daemon: client version 1.23 is too old. Minimum supported API version is 1.24, +upgrade your client to a newer version +``` + +Support for API versions lower than `1.24` has been permanently removed in Docker +Engine v26, and the minimum supported API version will be incrementally raised +in releases following that. + + + + +### Container short ID in network Aliases field + +**Deprecated in release: v25.0** + +**Removed in release: v26.0** + +The `Aliases` field returned by `docker inspect` contains the container short +ID once the container is started. This behavior is deprecated in v25.0 but +kept until the next release, v26.0. Starting with that version, the `Aliases` +field will only contain the aliases set through the `docker container create` +and `docker run` flag `--network-alias`. + +A new field `DNSNames` containing the container name (if one was specified), +the hostname, the network aliases, as well as the container short ID, has been +introduced in v25.0 and should be used instead of the `Aliases` field. + +### Mount `bind-nonrecursive` option + +**Deprecated in release: v25.0** + +**Removed in release: v29.0** + +The `bind-nonrecursive` option was replaced with the [`bind-recursive`] +option (see [cli-4316], [cli-4671]). The option was still accepted, but +printed a deprecation warning: + +```console +bind-nonrecursive is deprecated, use bind-recursive=disabled instead +``` + +In the v29.0 release, this warning is removed, and returned as an error. +Users should use the equivalent `bind-recursive=disabled` option instead. + +[`bind-recursive`]: https://docs.docker.com/engine/storage/bind-mounts/#recursive-mounts +[cli-4316]: https://github.com/docker/cli/pull/4316 +[cli-4671]: https://github.com/docker/cli/pull/4671 + +### IsAutomated field, and `is-automated` filter on `docker search` + +**Deprecated in release: v25.0** + +**Removed in release: v28.2** + +The `is_automated` field has been deprecated by Docker Hub's search API. +Consequently, the `IsAutomated` field in image search will always be set +to `false` in future, and searching for "is-automated=true" will yield no +results. + +The `AUTOMATED` column has been removed from the default `docker search` +and `docker image search` output in v25.0, and the corresponding `IsAutomated` +templating has been removed in v28.2. + +### Logentries logging driver + +**Deprecated in release: v24.0** + +**Removed in release: v25.0** + +The logentries service SaaS was shut down on November 15, 2022, rendering +this logging driver non-functional. Users should no longer use this logging +driver, and the driver has been removed in Docker 25.0. Existing containers +using this logging-driver are migrated to use the "local" logging driver +after upgrading. + +### OOM-score adjust for the daemon + +**Deprecated in release: v24.0** + +**Removed in release: v25.0** + +The `oom-score-adjust` option was added to prevent the daemon from being +OOM-killed before other processes. This option was mostly added as a +convenience, as running the daemon as a systemd unit was not yet common. + +Having the daemon set its own limits is not best-practice, and something +better handled by the process-manager starting the daemon. + +Docker v20.10 and newer no longer adjust the daemon's OOM score by default, +instead setting the OOM-score to the systemd unit (OOMScoreAdjust) that's +shipped with the packages. + +Users currently depending on this feature are recommended to adjust the +daemon's OOM score using systemd or through other means, when starting +the daemon. + +### BuildKit build information + +**Deprecated in release: v23.0** + +**Removed in release: v24.0** + +[Build information](https://github.com/moby/buildkit/blob/v0.11/docs/buildinfo.md) +structures have been introduced in [BuildKit v0.10.0](https://github.com/moby/buildkit/releases/tag/v0.10.0) +and are generated with build metadata that allows you to see all the sources +(images, Git repositories) that were used by the build with their exact +versions and also the configuration that was passed to the build. This +information is also embedded into the image configuration if one is generated. + +### Legacy builder for Linux images + +**Deprecated in release: v23.0** + +Docker v23.0 now uses BuildKit by default to build Linux images, and uses the +[Buildx](https://docs.docker.com/buildx/working-with-buildx/) CLI component for +`docker build`. With this change, `docker build` now exposes all advanced features +that BuildKit provides and which were previously only available through the +`docker buildx` subcommands. + +The Buildx component is installed automatically when installing the `docker` CLI +using our `.deb` or `.rpm` packages, and statically linked binaries are provided +both on `download.docker.com`, and through the [`docker/buildx-bin` image](https://hub.docker.com/r/docker/buildx-bin) +on Docker Hub. Refer the [Buildx section](http://docs.docker.com/go/buildx/) for +detailed instructions on installing the Buildx component. + +This release marks the beginning of the deprecation cycle of the classic ("legacy") +builder for Linux images. No active development will happen on the classic builder +(except for bugfixes). BuildKit development started five Years ago, left the +"experimental" phase since Docker 18.09, and is already the default builder for +[Docker Desktop](https://docs.docker.com/desktop/previous-versions/3.x-mac/#docker-desktop-320). +While we're comfortable that BuildKit is stable for general use, there may be +some changes in behavior. If you encounter issues with BuildKit, we encourage +you to report issues in the [BuildKit issue tracker on GitHub](https://github.com/moby/buildkit/){:target="_blank" rel="noopener" class="_"} + +> Classic builder for building Windows images +> +> BuildKit does not (yet) provide support for building Windows images, and +> `docker build` continues to use the classic builder to build native Windows +> images on Windows daemons. + +### Legacy builder fallback + +**Deprecated in release: v23.0** + +[Docker v23.0 now uses BuildKit by default to build Linux images](#legacy-builder-for-linux-images), +which requires the Buildx component to build images with BuildKit. There may be +situations where the Buildx component is not available, and BuildKit cannot be +used. + +To provide a smooth transition to BuildKit as the default builder, Docker v23.0 +has an automatic fallback for some situations, or produces an error to assist +users to resolve the problem. + +In situations where the user did not explicitly opt-in to use BuildKit (i.e., +`DOCKER_BUILDKIT=1` is not set), the CLI automatically falls back to the classic +builder, but prints a deprecation warning: + +```text +DEPRECATED: The legacy builder is deprecated and will be removed in a future release. + Install the buildx component to build images with BuildKit: + https://docs.docker.com/go/buildx/ +``` + +This situation may occur if the `docker` CLI is installed using the static binaries, +and the Buildx component is not installed or not installed correctly. This fallback +will be removed in a future release, therefore we recommend to [install the Buildx component](https://docs.docker.com/go/buildx/) +and use BuildKit for your builds, or opt-out of using BuildKit with `DOCKER_BUILDKIT=0`. + +If you opted-in to use BuildKit (`DOCKER_BUILDKIT=1`), but the Buildx component +is missing, an error is printed instead, and the `docker build` command fails: + +```text +ERROR: BuildKit is enabled but the buildx component is missing or broken. + Install the buildx component to build images with BuildKit: + https://docs.docker.com/go/buildx/ +``` + +We recommend to [install the Buildx component](https://docs.docker.com/go/buildx/) +to continue using BuildKit for your builds, but alternatively, users can either +unset the `DOCKER_BUILDKIT` environment variable to fall back to the legacy builder, +or opt-out of using BuildKit with `DOCKER_BUILDKIT=0`. + +Be aware that the [classic builder is deprecated](#legacy-builder-for-linux-images) +so both the automatic fallback and opting-out of using BuildKit will no longer +be possible in a future release. + +### Btrfs storage driver on CentOS 7 and RHEL 7 + +**Removed in release: v23.0** + +The `btrfs` storage driver on CentOS and RHEL was provided as a technology preview +by CentOS and RHEL, but has been deprecated since the [Red Hat Enterprise Linux 7.4 release](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/ch-btrfs), +and removed in CentOS 8 and RHEL 8. Users of the `btrfs` storage driver on CentOS +are recommended to migrate to a different storage driver, such as `overlay2`, which +is now the default storage driver. Docker 23.0 continues to provide the `btrfs` +storage driver to allow users to migrate to an alternative driver. The next release +of Docker will no longer provide this driver. + +### Support for encrypted TLS private keys + +**Deprecated in release: v20.10** + +**Removed in release: v23.0** + +Use of encrypted TLS private keys has been deprecated, and has been removed. +Golang has deprecated support for legacy PEM encryption (as specified in +[RFC 1423](https://datatracker.ietf.org/doc/html/rfc1423)), as it is insecure by +design (see [https://go-review.googlesource.com/c/go/+/264159](https://go-review.googlesource.com/c/go/+/264159)). + +This feature allowed using an encrypted private key with a supplied password, +but did not provide additional security as the encryption is known to be broken, +and the key is sitting next to the password in the filesystem. Users are recommended +to decrypt the private key, and store it un-encrypted to continue using it. + +### Kubernetes stack and context support + +**Deprecated in release: v20.10** + +**Removed in release: v23.0** + +Following the deprecation of [Compose on Kubernetes](https://github.com/docker/compose-on-kubernetes), +support for Kubernetes in the `stack` and `context` commands has been removed from +the CLI, and options related to this functionality are now either ignored, or may +produce an error. + +The following command-line flags are removed from the `docker context` subcommands: + +- `--default-stack-orchestrator` - swarm is now the only (and default) orchestrator for stacks. +- `--kubernetes` - the Kubernetes endpoint can no longer be stored in `docker context`. +- `--kubeconfig` - exporting a context as a kubeconfig file is no longer supported. + +The output produced by the `docker context inspect` subcommand no longer contains +information about `StackOrchestrator` and `Kubernetes` endpoints for new contexts. + +The following command-line flags are removed from the `docker stack` subcommands: + +- `--kubeconfig` - using a kubeconfig file as context is no longer supported. +- `--namespace` - configuring the Kubernetes namespace for stacks is no longer supported. +- `--orchestrator` - swarm is now the only (and default) orchestrator for stacks. + +The `DOCKER_STACK_ORCHESTRATOR`, `DOCKER_ORCHESTRATOR`, and `KUBECONFIG` environment +variables, as well as the `stackOrchestrator` option in the `~/.docker/config.json` +CLI configuration file are no longer used, and ignored. + +### Pulling images from non-compliant image registries + +**Deprecated in release: v20.10** + +**Removed in release: v28.2** + +Docker Engine v20.10 and up includes optimizations to verify if images in the +local image cache need updating before pulling, preventing the Docker Engine +from making unnecessary API requests. These optimizations require the container +image registry to conform to the [Open Container Initiative Distribution Specification](https://github.com/opencontainers/distribution-spec). + +While most registries conform to the specification, we encountered some registries +to be non-compliant, resulting in `docker pull` to fail. + +As a temporary solution, Docker Engine v20.10 added a fallback mechanism to +allow `docker pull` to be functional when using a non-compliant registry. A +warning message is printed in this situation: + + WARNING Failed to pull manifest by the resolved digest. This registry does not + appear to conform to the distribution registry specification; falling back to + pull by tag. This fallback is DEPRECATED, and will be removed in a future + release. + +The fallback was added to allow users to either migrate their images to a +compliant registry, or for these registries to become compliant. + +GitHub deprecated the legacy `docker.pkg.github.com` registry, and it was +[sunset on Feb 24th, 2025](https://github.blog/changelog/2025-01-23-legacy-docker-registry-closing-down/) +in favor of GitHub Container Registry (GHCR, ghcr.io), making this fallback +no longer needed. + +### Linux containers on Windows (LCOW) (experimental) + +**Deprecated in release: v20.10** + +**Removed in release: v23.0** + +The experimental feature to run Linux containers on Windows (LCOW) was introduced +as a technical preview in Docker 17.09. While many enhancements were made after +its introduction, the feature never reached completeness, and development has +now stopped in favor of running Docker natively on Linux in WSL2. + +Developers who want to run Linux workloads on a Windows host are encouraged to use +[Docker Desktop with WSL2](https://docs.docker.com/docker-for-windows/wsl/) instead. + +### BLKIO weight options with cgroups v1 + +**Deprecated in release: v20.10** + +Specifying blkio weight (`docker run --blkio-weight` and `docker run --blkio-weight-device`) +is now marked as deprecated when using cgroups v1 because the corresponding features +were [removed in Linux kernel v5.0 and up](https://github.com/torvalds/linux/commit/f382fb0bcef4c37dc049e9f6963e3baf204d815c). +When using cgroups v2, the `--blkio-weight` options are implemented using +[`io.weight](https://github.com/torvalds/linux/blob/v5.0/Documentation/admin-guide/cgroup-v2.rst#io). + +### Kernel memory limit + +**Deprecated in release: v20.10** + +**Removed in release: v23.0** + +Specifying kernel memory limit (`docker run --kernel-memory`) is no longer supported +because the [Linux kernel deprecated `kmem.limit_in_bytes` in v5.4](https://github.com/torvalds/linux/commit/0158115f702b0ba208ab0b5adf44cae99b3ebcc7). +The OCI runtime specification now marks this option as ["NOT RECOMMENDED"](https://github.com/opencontainers/runtime-spec/pull/1093), +and OCI runtimes such as `runc` no longer support this option. + +The Docker API no longer handles the kernel-memory fields, and Docker CLI v29.0 +removes the `--kernel-memory` option. + +### Classic Swarm and overlay networks using cluster store + +**Deprecated in release: v20.10** + +**Removed in release: v23.0** + +Standalone ("classic") Swarm has been deprecated, and with that the use of overlay +networks using an external key/value store. The corresponding`--cluster-advertise`, +`--cluster-store`, and `--cluster-store-opt` daemon options have been removed. + +### Support for legacy `~/.dockercfg` configuration files + +**Deprecated in release: v20.10** + +**Removed in release: v23.0** + +The Docker CLI up until v1.7.0 used the `~/.dockercfg` file to store credentials +after authenticating to a registry (`docker login`). Docker v1.7.0 replaced this +file with a new CLI configuration file, located in `~/.docker/config.json`. When +implementing the new configuration file, the old file (and file-format) was kept +as a fall-back, to assist existing users with migrating to the new file. + +Given that the old file format encourages insecure storage of credentials +(credentials are stored unencrypted), and that no version of the CLI since +Docker v1.7.0 has created this file, support for this file, and its format has +been removed. + +### Configuration options for experimental CLI features + +**Deprecated in release: v19.03** + +**Removed in release: v23.0** + +The `DOCKER_CLI_EXPERIMENTAL` environment variable and the corresponding `experimental` +field in the CLI configuration file are deprecated. Experimental features are +enabled by default, and these configuration options are no longer functional. + +Starting with v23.0, the Docker CLI no longer prints `Experimental` for the client +in the output of `docker version`, and the field has been removed from the JSON +format. + +### CLI plugins support + +**Deprecated in release: v20.10** + +CLI Plugin API is now marked as deprecated. + +### Dockerfile legacy `ENV name value` syntax + +**Deprecated in release: v20.10** + +The Dockerfile `ENV` instruction allows values to be set using either `ENV name=value` +or `ENV name value`. The latter (`ENV name value`) form can be ambiguous, for example, +the following defines a single env-variable (`ONE`) with value `"TWO= THREE=world"`, +but may have intended to be setting three env-vars: + +```dockerfile +ENV ONE TWO= THREE=world +``` + +This format also does not allow setting multiple environment-variables in a single +`ENV` line in the Dockerfile. + +Use of the `ENV name value` syntax is discouraged, and may be removed in a future +release. Users are encouraged to update their Dockerfiles to use the `ENV name=value` +syntax, for example: + +```dockerfile +ENV ONE="" TWO="" THREE="world" +``` + +### `docker build --stream` flag (experimental) + +**Deprecated in release: v20.10** + +**Removed in release: v20.10** + +Docker v17.07 introduced an experimental `--stream` flag on `docker build` which +allowed the build-context to be incrementally sent to the daemon, instead of +unconditionally sending the whole build-context. + +This functionality has been reimplemented as part of BuildKit, which uses streaming +by default and the `--stream` option will be ignored when using the classic builder, +printing a deprecation warning instead. + +Users that want to use this feature are encouraged to enable BuildKit by setting +the `DOCKER_BUILDKIT=1` environment variable or through the daemon or CLI configuration +files. + +### `fluentd-async-connect` log opt + +**Deprecated in release: v20.10** + +**Removed in release: v28.0** + +The `--log-opt fluentd-async-connect` option for the fluentd logging driver is +[deprecated in favor of `--log-opt fluentd-async`](https://github.com/moby/moby/pull/39086). +A deprecation message is logged in the daemon logs if the old option is used: + +```console +fluent#New: AsyncConnect is now deprecated, use Async instead +``` + +Users are encouraged to use the `fluentd-async` option going forward, as support +for the old option has been removed. + +### Pushing and pulling with image manifest v2 schema 1 + +**Deprecated in release: v19.03** + +**Disabled by default in release: v26.0** + +**Removed in release: v28.2** + + +The image manifest [v2 schema 1](https://distribution.github.io/distribution/spec/deprecated-schema-v1/) +and "Docker Image v1" formats were deprecated in favor of the +[v2 schema 2](https://distribution.github.io/distribution/spec/manifest-v2-2/) +and [OCI image spec](https://github.com/opencontainers/image-spec/tree/v1.1.0) +formats. + +These legacy formats should no longer be used, and users are recommended to +update images to use current formats, or to upgrade to more current images. +Starting with Docker v26.0, pulling these images is disabled by default, and +support has been removed in v28.2. Attempting to pull a legacy image now +produces an error: + +```console +$ docker pull ubuntu:10.04 +Error response from daemon: +Docker Image Format v1 and Docker Image manifest version 2, schema 1 support has been removed. +Suggest the author of docker.io/library/ubuntu:10.04 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. +More information at https://docs.docker.com/go/deprecated-image-specs/ +``` + +### `docker engine` subcommands + +**Deprecated in release: v19.03** + +**Removed in release: v20.10** + +The `docker engine activate`, `docker engine check`, and `docker engine update` +provided an alternative installation method to upgrade Docker Community engines +to Docker Enterprise, using an image-based distribution of the Docker Engine. + +This feature was only available on Linux, and only when executed on a local node. +Given the limitations of this feature, and the feature not getting widely adopted, +the `docker engine` subcommands will be removed, in favor of installation through +standard package managers. + +### Top-level `docker deploy` subcommand (experimental) + +**Deprecated in release: v19.03** + +**Removed in release: v20.10** + +The top-level `docker deploy` command (using the "Docker Application Bundle" +(.dab) file format was introduced as an experimental feature in Docker 1.13 / +17.03, but superseded by support for Docker Compose files using the `docker stack deploy` +subcommand. + +### `docker stack deploy` using "dab" files (experimental) + +**Deprecated in release: v19.03** + +**Removed in release: v20.10** + +With no development being done on this feature, and no active use of the file +format, support for the DAB file format and the top-level `docker deploy` command +(hidden by default in 19.03), will be removed, in favour of `docker stack deploy` +using compose files. + +### Support for the `overlay2.override_kernel_check` storage option + +**Deprecated in release: v19.03** + +**Removed in release: v24.0** + +This daemon configuration option disabled the Linux kernel version check used +to detect if the kernel supported OverlayFS with multiple lower dirs, which is +required for the overlay2 storage driver. Starting with Docker v19.03.7, the +detection was improved to no longer depend on the kernel _version_, so this +option was no longer used. + +### AuFS storage driver + +**Deprecated in release: v19.03** + +**Removed in release: v24.0** + +The `aufs` storage driver is deprecated in favor of `overlay2`, and has been +removed in a Docker Engine v24.0. Users of the `aufs` storage driver must +migrate to a different storage driver, such as `overlay2`, before upgrading +to Docker Engine v24.0. + +The `aufs` storage driver facilitated running Docker on distros that have no +support for OverlayFS, such as Ubuntu 14.04 LTS, which originally shipped with +a 3.14 kernel. + +Now that Ubuntu 14.04 is no longer a supported distro for Docker, and `overlay2` +is available to all supported distros (as they are either on kernel 4.x, or have +support for multiple lowerdirs backported), there is no reason to continue +maintenance of the `aufs` storage driver. + +### Legacy overlay storage driver + +**Deprecated in release: v18.09** + +**Removed in release: v24.0** + +The `overlay` storage driver is deprecated in favor of the `overlay2` storage +driver, which has all the benefits of `overlay`, without its limitations (excessive +inode consumption). The legacy `overlay` storage driver has been removed in +Docker Engine v24.0. Users of the `overlay` storage driver should migrate to the +`overlay2` storage driver before upgrading to Docker Engine v24.0. + +The legacy `overlay` storage driver allowed using overlayFS-backed filesystems +on kernels older than v4.x. Now that all supported distributions are able to run `overlay2` +(as they are either on kernel 4.x, or have support for multiple lowerdirs +backported), there is no reason to keep maintaining the `overlay` storage driver. + +### Device mapper storage driver + +**Deprecated in release: v18.09** + +**Disabled by default in release: v23.0** + +**Removed in release: v25.0** + +The `devicemapper` storage driver is deprecated in favor of `overlay2`, and has +been removed in Docker Engine v25.0. Users of the `devicemapper` storage driver +must migrate to a different storage driver, such as `overlay2`, before upgrading +to Docker Engine v25.0. + +The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels +that have no support for other storage drivers (such as overlay2, or btrfs). + +Now that support for `overlay2` is added to all supported distros (as they are +either on kernel 4.x, or have support for multiple lowerdirs backported), there +is no reason to continue maintenance of the `devicemapper` storage driver. + +### Use of reserved namespaces in engine labels + +**Deprecated in release: v18.06** + +**Removed in release: v20.10** + +The namespaces `com.docker.*`, `io.docker.*`, and `org.dockerproject.*` in engine labels +were always documented to be reserved, but there was never any enforcement. + +Usage of these namespaces will now cause a warning in the engine logs to discourage their +use, and will error instead in v20.10 and above. + +### `--disable-legacy-registry` override daemon option + +**Disabled In Release: v17.12** + +**Removed in release: v19.03** + +The `--disable-legacy-registry` flag was disabled in Docker 17.12 and will print +an error when used. For this error to be printed, the flag itself is still present, +but hidden. The flag has been removed in Docker 19.03. + +### Interacting with V1 registries + +**Disabled by default in release: v17.06** + +**Removed in release: v17.12** + +Version 1.8.3 added a flag (`--disable-legacy-registry=false`) which prevents the +Docker daemon from `pull`, `push`, and `login` operations against v1 +registries. Though enabled by default, this signals the intent to deprecate +the v1 protocol. + +Support for the v1 protocol to the public registry was removed in 1.13. Any +mirror configurations using v1 should be updated to use a +[v2 registry mirror](https://docs.docker.com/registry/recipes/mirror/). + +Starting with Docker 17.12, support for V1 registries has been removed, and the +`--disable-legacy-registry` flag can no longer be used, and `dockerd` will fail to +start when set. + +### Asynchronous `service create` and `service update` as default + +**Deprecated in release: v17.05** + +**Disabled by default in release: [v17.10](https://github.com/docker/docker-ce/releases/tag/v17.10.0-ce)** + +Docker 17.05 added an optional `--detach=false` option to make the +`docker service create` and `docker service update` work synchronously. This +option will be enabled by default in Docker 17.10, at which point the `--detach` +flag can be used to use the previous (asynchronous) behavior. + +The default for this option will also be changed accordingly for `docker service rollback` +and `docker service scale` in Docker 17.10. + +### `-g` and `--graph` flags on `dockerd` + +**Deprecated in release: v17.05** + +**Removed in release: v23.0** + +The `-g` or `--graph` flag for the `dockerd` or `docker daemon` command was +used to indicate the directory in which to store persistent data and resource +configuration and has been replaced with the more descriptive `--data-root` +flag. These flags were deprecated and hidden in v17.05, and removed in v23.0. + +### Top-level network properties in NetworkSettings + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Target for removal in release: v17.12** + +When inspecting a container, `NetworkSettings` contains top-level information +about the default ("bridge") network; + +`EndpointID`, `Gateway`, `GlobalIPv6Address`, `GlobalIPv6PrefixLen`, `IPAddress`, +`IPPrefixLen`, `IPv6Gateway`, and `MacAddress`. + +These properties are deprecated in favor of per-network properties in +`NetworkSettings.Networks`. These properties were already "deprecated" in +Docker 1.9, but kept around for backward compatibility. + +Refer to [#17538](https://github.com/docker/docker/pull/17538) for further +information. + +### `filter` option for `/images/json` endpoint + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Removed in release: v20.10** + +The `filter` option to filter the list of image by reference (name or name:tag) +is now implemented as a regular filter, named `reference`. + +### `repository:shortid` image references + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Removed in release: v17.12** + +The `repository:shortid` syntax for referencing images is very little used, +collides with tag references, and can be confused with digest references. + +Support for the `repository:shortid` notation to reference images was removed +in Docker 17.12. + +### `docker daemon` subcommand + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Removed in release: v17.12** + +The daemon is moved to a separate binary (`dockerd`), and should be used instead. + +### Duplicate keys with conflicting values in engine labels + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Removed in release: v17.12** + +When setting duplicate keys with conflicting values, an error will be produced, and the daemon +will fail to start. + +### `MAINTAINER` in Dockerfile + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +`MAINTAINER` was an early very limited form of `LABEL` which should be used instead. + +### API calls without a version + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Target for removal in release: v17.12** + +API versions should be supplied to all API calls to ensure compatibility with +future Engine versions. Instead of just requesting, for example, the URL +`/containers/json`, you must now request `/v1.25/containers/json`. + +### Backing filesystem without `d_type` support for overlay/overlay2 + +**Deprecated in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +**Removed in release: v17.12** + +The overlay and overlay2 storage driver does not work as expected if the backing +filesystem does not support `d_type`. For example, XFS does not support `d_type` +if it is formatted with the `ftype=0` option. + +Support for these setups has been removed, and Docker v23.0 and up now fails to +start when attempting to use the `overlay2` or `overlay` storage driver on a +backing filesystem without `d_type` support. + +Refer to [#27358](https://github.com/docker/docker/issues/27358) for details. + +### `--automated` and `--stars` flags on `docker search` + +**Deprecated in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +**Removed in release: v20.10** + +The `docker search --automated` and `docker search --stars` options are deprecated. +Use `docker search --filter=is-automated=` and `docker search --filter=stars=...` instead. + +### `-h` shorthand for `--help` + +**Deprecated in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +**Target for removal in release: v17.09** + +The shorthand (`-h`) is less common than `--help` on Linux and cannot be used +on all subcommands (due to it conflicting with, e.g. `-h` / `--hostname` on +`docker create`). For this reason, the `-h` shorthand was not printed in the +"usage" output of subcommands, nor documented, and is now marked "deprecated". + +### `-e` and `--email` flags on `docker login` + +**Deprecated in release: [v1.11.0](https://github.com/docker/docker/releases/tag/v1.11.0)** + +**Removed in release: [v17.06](https://github.com/docker/docker-ce/releases/tag/v17.06.0-ce)** + +The `docker login` no longer automatically registers an account with the target registry if the given username doesn't exist. Due to this change, the email flag is no longer required, and will be deprecated. + +### Separator (`:`) of `--security-opt` flag on `docker run` + +**Deprecated in release: [v1.11.0](https://github.com/docker/docker/releases/tag/v1.11.0)** + +**Target for removal in release: v17.06** + +The flag `--security-opt` doesn't use the colon separator (`:`) anymore to divide keys and values, it uses the equal symbol (`=`) for consistency with other similar flags, like `--storage-opt`. + +### Links on the default bridge network + +**Deprecated in release: v1.10** +**Target for removal in release: v30.0** + +The `--link` option on `docker create` and `docker run`, when used with no +`--network` specified, was deprecated in v1.10 and will be removed in a future +release. Custom networks should be used instead. Docker 29.6 added a deprecation +warning when this option is used for the default bridge network. + +Note that the `--link` option is still supported when a non-default network +is used. + +### Ambiguous event fields in API + +**Deprecated in release: [v1.10.0](https://github.com/docker/docker/releases/tag/v1.10.0)** + +The fields `ID`, `Status` and `From` in the events API have been deprecated in favor of a more rich structure. +See the events API documentation for the new format. + +### `-f` flag on `docker tag` + +**Deprecated in release: [v1.10.0](https://github.com/docker/docker/releases/tag/v1.10.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +To make tagging consistent across the various `docker` commands, the `-f` flag on the `docker tag` command is deprecated. It is no longer necessary to specify `-f` to move a tag from one image to another. Nor will `docker` generate an error if the `-f` flag is missing and the specified tag is already in use. + +### HostConfig at API container start + +**Deprecated in release: [v1.10.0](https://github.com/docker/docker/releases/tag/v1.10.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +Passing an `HostConfig` to `POST /containers/{name}/start` is deprecated in favor of +defining it at container creation (`POST /containers/create`). + +### `--before` and `--since` flags on `docker ps` + +**Deprecated in release: [v1.10.0](https://github.com/docker/docker/releases/tag/v1.10.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +The `docker ps --before` and `docker ps --since` options are deprecated. +Use `docker ps --filter=before=...` and `docker ps --filter=since=...` instead. + +### Driver-specific log tags + +**Deprecated in release: [v1.9.0](https://github.com/docker/docker/releases/tag/v1.9.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +Log tags are now generated in a standard way across different logging drivers. +Because of which, the driver specific log tag options `syslog-tag`, `gelf-tag` and +`fluentd-tag` have been deprecated in favor of the generic `tag` option. + +```console +$ docker --log-driver=syslog --log-opt tag="{{.ImageName}}/{{.Name}}/{{.ID}}" +``` + +### Docker Content Trust ENV passphrase variables name change + +**Deprecated in release: [v1.9.0](https://github.com/docker/docker/releases/tag/v1.9.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +Since 1.9, Docker Content Trust Offline key has been renamed to Root key and the Tagging key has been renamed to Repository key. Due to this renaming, we're also changing the corresponding environment variables + +- DOCKER_CONTENT_TRUST_OFFLINE_PASSPHRASE is now named DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE +- DOCKER_CONTENT_TRUST_TAGGING_PASSPHRASE is now named DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE + +### `/containers/(id or name)/copy` endpoint + +**Deprecated in release: [v1.8.0](https://github.com/docker/docker/releases/tag/v1.8.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +The endpoint `/containers/(id or name)/copy` is deprecated in favor of `/containers/(id or name)/archive`. + +### LXC built-in exec driver + +**Deprecated in release: [v1.8.0](https://github.com/docker/docker/releases/tag/v1.8.0)** + +**Removed in release: [v1.10.0](https://github.com/docker/docker/releases/tag/v1.10.0)** + +The built-in LXC execution driver, the lxc-conf flag, and API fields have been removed. + +### Old Command Line Options + +**Deprecated in release: [v1.8.0](https://github.com/docker/docker/releases/tag/v1.8.0)** + +**Removed in release: [v1.10.0](https://github.com/docker/docker/releases/tag/v1.10.0)** + +The flags `-d` and `--daemon` are deprecated. Use the separate `dockerd` binary instead. + +The following single-dash (`-opt`) variant of certain command line options +are deprecated and replaced with double-dash options (`--opt`): + +- `docker attach -nostdin` +- `docker attach -sig-proxy` +- `docker build -no-cache` +- `docker build -rm` +- `docker commit -author` +- `docker commit -run` +- `docker events -since` +- `docker history -notrunc` +- `docker images -notrunc` +- `docker inspect -format` +- `docker ps -beforeId` +- `docker ps -notrunc` +- `docker ps -sinceId` +- `docker rm -link` +- `docker run -cidfile` +- `docker run -dns` +- `docker run -entrypoint` +- `docker run -expose` +- `docker run -link` +- `docker run -lxc-conf` +- `docker run -n` +- `docker run -privileged` +- `docker run -volumes-from` +- `docker search -notrunc` +- `docker search -stars` +- `docker search -t` +- `docker search -trusted` +- `docker tag -force` + +The following double-dash options are deprecated and have no replacement: + +- `docker run --cpuset` +- `docker run --networking` +- `docker ps --since-id` +- `docker ps --before-id` +- `docker search --trusted` + +**Deprecated in release: [v1.5.0](https://github.com/docker/docker/releases/tag/v1.5.0)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +The single-dash (`-help`) was removed, in favor of the double-dash `--help` + +### `--api-enable-cors` flag on `dockerd` + +**Deprecated in release: [v1.6.0](https://github.com/docker/docker/releases/tag/v1.6.0)** + +**Removed in release: [v17.09](https://github.com/docker/docker-ce/releases/tag/v17.09.0-ce)** + +The flag `--api-enable-cors` is deprecated since v1.6.0. Use the flag +`--api-cors-header` instead. + +### `--run` flag on `docker commit` + +**Deprecated in release: [v0.10.0](https://github.com/docker/docker/releases/tag/v0.10.0)** + +**Removed in release: [v1.13.0](https://github.com/docker/docker/releases/tag/v1.13.0)** + +The flag `--run` of the `docker commit` command (and its short version `-run`) were deprecated in favor +of the `--changes` flag that allows to pass `Dockerfile` commands. + +### Three arguments form in `docker import` + +**Deprecated in release: [v0.6.7](https://github.com/docker/docker/releases/tag/v0.6.7)** + +**Removed in release: [v1.12.0](https://github.com/docker/docker/releases/tag/v1.12.0)** + +The `docker import` command format `file|URL|- [REPOSITORY [TAG]]` is deprecated since November 2013. It's no longer supported. diff --git a/docs/extend/EBS_volume.md b/docs/extend/EBS_volume.md new file mode 100644 index 0000000000..88de970ba2 --- /dev/null +++ b/docs/extend/EBS_volume.md @@ -0,0 +1,165 @@ +--- +description: Volume plugin for Amazon EBS +keywords: "API, Usage, plugins, documentation, developer, amazon, ebs, rexray, volume" +--- + + + +# Volume plugin for Amazon EBS + +## A proof-of-concept Rexray plugin + +In this example, a simple Rexray plugin will be created for the purposes of using +it on an Amazon EC2 instance with EBS. It is not meant to be a complete Rexray plugin. + +The example source is available at [https://github.com/tiborvass/rexray-plugin](https://github.com/tiborvass/rexray-plugin). + +To learn more about Rexray: [https://github.com/codedellemc/rexray](https://github.com/codedellemc/rexray) + +## 1. Make a Docker image + +The following is the Dockerfile used to containerize rexray. + +```dockerfile +FROM debian:jessie +RUN apt-get update && apt-get install -y --no-install-recommends wget ca-certificates +RUN wget https://dl.bintray.com/emccode/rexray/stable/0.6.4/rexray-Linux-x86_64-0.6.4.tar.gz -O rexray.tar.gz && tar -xvzf rexray.tar.gz -C /usr/bin && rm rexray.tar.gz +RUN mkdir -p /run/docker/plugins /var/lib/libstorage/volumes +ENTRYPOINT ["rexray"] +CMD ["--help"] +``` + +To build it you can run `image=$(cat Dockerfile | docker build -q -)` and `$image` +will reference the containerized rexray image. + +## 2. Extract rootfs + +```sh +$ TMPDIR=/tmp/rexray # for the purpose of this example +$ # create container without running it, to extract the rootfs from image +$ docker create --name rexray "$image" +$ # save the rootfs to a tar archive +$ docker export -o $TMPDIR/rexray.tar rexray +$ # extract rootfs from tar archive to a rootfs folder +$ ( mkdir -p $TMPDIR/rootfs; cd $TMPDIR/rootfs; tar xf ../rexray.tar ) +``` + +## 3. Add plugin configuration + +We have to put the following JSON to `$TMPDIR/config.json`: + +```json +{ + "Args": { + "Description": "", + "Name": "", + "Settable": null, + "Value": null + }, + "Description": "A proof-of-concept EBS plugin (using rexray) for Docker", + "Documentation": "https://github.com/tiborvass/rexray-plugin", + "Entrypoint": [ + "/usr/bin/rexray", "service", "start", "-f" + ], + "Env": [ + { + "Description": "", + "Name": "REXRAY_SERVICE", + "Settable": [ + "value" + ], + "Value": "ebs" + }, + { + "Description": "", + "Name": "EBS_ACCESSKEY", + "Settable": [ + "value" + ], + "Value": "" + }, + { + "Description": "", + "Name": "EBS_SECRETKEY", + "Settable": [ + "value" + ], + "Value": "" + } + ], + "Interface": { + "Socket": "rexray.sock", + "Types": [ + "docker.volumedriver/1.0" + ] + }, + "Linux": { + "AllowAllDevices": true, + "Capabilities": ["CAP_SYS_ADMIN"], + "Devices": null + }, + "Mounts": [ + { + "Source": "/dev", + "Destination": "/dev", + "Type": "bind", + "Options": ["rbind"] + } + ], + "Network": { + "Type": "host" + }, + "PropagatedMount": "/var/lib/libstorage/volumes", + "User": {}, + "WorkDir": "" +} +``` + +Note a couple of points: +- `PropagatedMount` is needed so that the docker daemon can see mounts done by the +rexray plugin from within the container, otherwise the docker daemon is not able +to mount a docker volume. +- The rexray plugin needs dynamic access to host devices. For that reason, we +have to give it access to all devices under `/dev` and set `AllowAllDevices` to +true for proper access. +- The user of this simple plugin can change only 3 settings: `REXRAY_SERVICE`, +`EBS_ACCESSKEY` and `EBS_SECRETKEY`. This is because of the reduced scope of this +plugin. Ideally other rexray parameters could also be set. + +## 4. Create plugin + +`docker plugin create tiborvass/rexray-plugin "$TMPDIR"` will create the plugin. + +```sh +$ docker plugin ls +ID NAME DESCRIPTION ENABLED +2475a4bd0ca5 tiborvass/rexray-plugin:latest A rexray volume plugin for Docker false +``` + +## 5. Test plugin + +```sh +$ docker plugin set tiborvass/rexray-plugin EBS_ACCESSKEY=$AWS_ACCESSKEY EBS_SECRETKEY=$AWS_SECRETKEY` +$ docker plugin enable tiborvass/rexray-plugin +$ docker volume create -d tiborvass/rexray-plugin my-ebs-volume +$ docker volume ls +DRIVER VOLUME NAME +tiborvass/rexray-plugin:latest my-ebs-volume +$ docker run --rm -v my-ebs-volume:/volume busybox sh -c 'echo bye > /volume/hi' +$ docker run --rm -v my-ebs-volume:/volume busybox cat /volume/hi +bye +``` + +## 6. Push plugin + +First, ensure you are logged in with `docker login`. Then you can run: +`docker plugin push tiborvass/rexray-plugin` to push it like a regular docker +image to a registry, to make it available for others to install via +`docker plugin install tiborvass/rexray-plugin EBS_ACCESSKEY=$AWS_ACCESSKEY EBS_SECRETKEY=$AWS_SECRETKEY`. diff --git a/docs/extend/_index.md b/docs/extend/_index.md new file mode 100644 index 0000000000..8c85b10485 --- /dev/null +++ b/docs/extend/_index.md @@ -0,0 +1,271 @@ +--- +title: Docker Engine managed plugin system +linkTitle: Docker Engine plugins +description: Develop and use a plugin with the managed plugin system +keywords: "API, Usage, plugins, documentation, developer" +aliases: + - "/engine/extend/plugins_graphdriver/" +--- + +- [Installing and using a plugin](index.md#installing-and-using-a-plugin) +- [Developing a plugin](index.md#developing-a-plugin) +- [Debugging plugins](index.md#debugging-plugins) + +Docker Engine's plugin system lets you install, start, stop, and remove +plugins using Docker Engine. + +For information about legacy (non-managed) plugins, refer to +[Understand legacy Docker Engine plugins](legacy_plugins.md). + +> [!NOTE] +> Docker Engine managed plugins are currently not supported on Windows daemons. + +## Installing and using a plugin + +Plugins are distributed as Docker images and can be hosted on Docker Hub or on +a private registry. + +To install a plugin, use the `docker plugin install` command, which pulls the +plugin from Docker Hub or your private registry, prompts you to grant +permissions or capabilities if necessary, and enables the plugin. + +To check the status of installed plugins, use the `docker plugin ls` command. +Plugins that start successfully are listed as enabled in the output. + +After a plugin is installed, you can use it as an option for another Docker +operation, such as creating a volume. + +In the following example, you install the [`rclone` plugin](https://rclone.org/docker/), verify that it is +enabled, and use it to create a volume. + +> [!NOTE] +> This example is intended for instructional purposes only. + +1. Set up the pre-requisite directories. By default they must exist on the host at the following locations: + + - `/var/lib/docker-plugins/rclone/config`. Reserved for the `rclone.conf` config file and must exist even if it's empty and the config file is not present. + - `/var/lib/docker-plugins/rclone/cache`. Holds the plugin state file as well as optional VFS caches. + +2. Install the `rclone` plugin. + + ```console + $ docker plugin install rclone/docker-volume-rclone --alias rclone + + Plugin "rclone/docker-volume-rclone" is requesting the following privileges: + - network: [host] + - mount: [/var/lib/docker-plugins/rclone/config] + - mount: [/var/lib/docker-plugins/rclone/cache] + - device: [/dev/fuse] + - capabilities: [CAP_SYS_ADMIN] + Do you grant the above permissions? [y/N] + ``` + + The plugin requests 5 privileges: + + - It needs access to the `host` network. + - Access to pre-requisite directories to mount to store: + - Your Rclone config files + - Temporary cache data + - Gives access to the FUSE (Filesystem in Userspace) device. This is required because Rclone uses FUSE to mount remote storage as if it were a local filesystem. + - It needs the `CAP_SYS_ADMIN` capability, which allows the plugin to run + the `mount` command. + +2. Check that the plugin is enabled in the output of `docker plugin ls`. + + ```console + $ docker plugin ls + + ID NAME DESCRIPTION ENABLED + aede66158353 rclone:latest Rclone volume plugin for Docker true + ``` + +3. Create a volume using the plugin. + This example mounts the `/remote` directory on host `1.2.3.4` into a + volume named `rclonevolume`. + + This volume can now be mounted into containers. + + ```console + $ docker volume create \ + -d rclone \ + --name rclonevolume \ + -o type=sftp \ + -o path=remote \ + -o sftp-host=1.2.3.4 \ + -o sftp-user=user \ + -o "sftp-password=$(cat file_containing_password_for_remote_host)" + ``` + +4. Verify that the volume was created successfully. + + ```console + $ docker volume ls + + DRIVER NAME + rclone rclonevolume + ``` + +5. Start a container that uses the volume `rclonevolume`. + + ```console + $ docker run --rm -v rclonevolume:/data busybox ls /data + + + ``` + +6. Remove the volume `rclonevolume` + + ```console + $ docker volume rm rclonevolume + + sshvolume + ``` + +To disable a plugin, use the `docker plugin disable` command. To completely +remove it, use the `docker plugin remove` command. For other available +commands and options, see the +[command line reference](https://docs.docker.com/reference/cli/docker/). + +## Developing a plugin + +#### The rootfs directory + +The `rootfs` directory represents the root filesystem of the plugin. In this +example, it was created from a Dockerfile: + +> [!NOTE] +> The `/run/docker/plugins` directory is mandatory inside of the +> plugin's filesystem for Docker to communicate with the plugin. + +```console +$ git clone https://github.com/vieux/docker-volume-sshfs +$ cd docker-volume-sshfs +$ docker build -t rootfsimage . +$ id=$(docker create rootfsimage true) # id was cd851ce43a403 when the image was created +$ sudo mkdir -p myplugin/rootfs +$ sudo docker export "$id" | sudo tar -x -C myplugin/rootfs +$ docker rm -vf "$id" +$ docker rmi rootfsimage +``` + +#### The config.json file + +The `config.json` file describes the plugin. See the [plugins config reference](config.md). + +Consider the following `config.json` file. + +```json +{ + "description": "sshFS plugin for Docker", + "documentation": "https://docs.docker.com/engine/extend/plugins/", + "entrypoint": ["/docker-volume-sshfs"], + "network": { + "type": "host" + }, + "interface": { + "types": ["docker.volumedriver/1.0"], + "socket": "sshfs.sock" + }, + "linux": { + "capabilities": ["CAP_SYS_ADMIN"] + } +} +``` + +This plugin is a volume driver. It requires a `host` network and the +`CAP_SYS_ADMIN` capability. It depends upon the `/docker-volume-sshfs` +entrypoint and uses the `/run/docker/plugins/sshfs.sock` socket to communicate +with Docker Engine. This plugin has no runtime parameters. + +#### Creating the plugin + +A new plugin can be created by running +`docker plugin create ./path/to/plugin/data` where the plugin +data contains a plugin configuration file `config.json` and a root filesystem +in subdirectory `rootfs`. + +After that the plugin `` will show up in `docker plugin ls`. +Plugins can be pushed to remote registries with +`docker plugin push `. + +## Debugging plugins + +Stdout of a plugin is redirected to dockerd logs. Such entries have a +`plugin=` suffix. Here are a few examples of commands for pluginID +`f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62` and their +corresponding log entries in the docker daemon logs. + +```console +$ docker plugin install tiborvass/sample-volume-plugin + +INFO[0036] Starting... Found 0 volumes on startup plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +``` + +```console +$ docker volume create -d tiborvass/sample-volume-plugin samplevol + +INFO[0193] Create Called... Ensuring directory /data/samplevol exists on host... plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +INFO[0193] open /var/lib/docker/plugin-data/local-persist.json: no such file or directory plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +INFO[0193] Created volume samplevol with mountpoint /data/samplevol plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +INFO[0193] Path Called... Returned path /data/samplevol plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +``` + +```console +$ docker run -v samplevol:/tmp busybox sh + +INFO[0421] Get Called... Found samplevol plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +INFO[0421] Mount Called... Mounted samplevol plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +INFO[0421] Path Called... Returned path /data/samplevol plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +INFO[0421] Unmount Called... Unmounted samplevol plugin=f52a3df433b9aceee436eaada0752f5797aab1de47e5485f1690a073b860ff62 +``` + +#### Using runc to obtain logfiles and shell into the plugin. + +Use `runc`, the default docker container runtime, for debugging plugins by +collecting plugin logs redirected to a file. + +```console +$ sudo runc --root /run/docker/runtime-runc/plugins.moby list + +ID PID STATUS BUNDLE CREATED OWNER +93f1e7dbfe11c938782c2993628c895cf28e2274072c4a346a6002446c949b25 15806 running /run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby-plugins/93f1e7dbfe11c938782c2993628c895cf28e2274072c4a346a6002446c949b25 2018-02-08T21:40:08.621358213Z root +9b4606d84e06b56df84fadf054a21374b247941c94ce405b0a261499d689d9c9 14992 running /run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby-plugins/9b4606d84e06b56df84fadf054a21374b247941c94ce405b0a261499d689d9c9 2018-02-08T21:35:12.321325872Z root +c5bb4b90941efcaccca999439ed06d6a6affdde7081bb34dc84126b57b3e793d 14984 running /run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby-plugins/c5bb4b90941efcaccca999439ed06d6a6affdde7081bb34dc84126b57b3e793d 2018-02-08T21:35:12.321288966Z root +``` + +```console +$ sudo runc --root /run/docker/runtime-runc/plugins.moby exec 93f1e7dbfe11c938782c2993628c895cf28e2274072c4a346a6002446c949b25 cat /var/log/plugin.log +``` + +If the plugin has a built-in shell, then exec into the plugin can be done as +follows: + +```console +$ sudo runc --root /run/docker/runtime-runc/plugins.moby exec -t 93f1e7dbfe11c938782c2993628c895cf28e2274072c4a346a6002446c949b25 sh +``` + +#### Using curl to debug plugin socket issues. + +To verify if the plugin API socket that the docker daemon communicates with +is responsive, use curl. In this example, we will make API calls from the +docker host to volume and network plugins using curl 7.47.0 to ensure that +the plugin is listening on the said socket. For a well functioning plugin, +these basic requests should work. Note that plugin sockets are available on the host under `/var/run/docker/plugins/` + +```console +$ curl -H "Content-Type: application/json" -XPOST -d '{}' --unix-socket /var/run/docker/plugins/e8a37ba56fc879c991f7d7921901723c64df6b42b87e6a0b055771ecf8477a6d/plugin.sock http:/VolumeDriver.List + +{"Mountpoint":"","Err":"","Volumes":[{"Name":"myvol1","Mountpoint":"/data/myvol1"},{"Name":"myvol2","Mountpoint":"/data/myvol2"}],"Volume":null} +``` + +```console +$ curl -H "Content-Type: application/json" -XPOST -d '{}' --unix-socket /var/run/docker/plugins/45e00a7ce6185d6e365904c8bcf62eb724b1fe307e0d4e7ecc9f6c1eb7bcdb70/plugin.sock http:/NetworkDriver.GetCapabilities + +{"Scope":"local"} +``` + +When using curl 7.5 and above, the URL should be of the form +`http://hostname/APICall`, where `hostname` is the valid hostname where the +plugin is installed and `APICall` is the call to the plugin API. + +For example, `http://localhost/VolumeDriver.List` diff --git a/docs/extend/config.md b/docs/extend/config.md new file mode 100644 index 0000000000..d2472a39f4 --- /dev/null +++ b/docs/extend/config.md @@ -0,0 +1,227 @@ +--- +description: "How to develop and use a plugin with the managed plugin system" +keywords: "API, Usage, plugins, documentation, developer" +title: Plugin Config Version 1 of Plugin V2 +--- + +This document outlines the format of the V0 plugin configuration. + +Plugin configs describe the various constituents of a Docker engine plugin. +Plugin configs can be serialized to JSON format with the following media types: + +| Config Type | Media Type | +|-------------|-----------------------------------------| +| config | `application/vnd.docker.plugin.v1+json` | + +## Config Field Descriptions + +Config provides the base accessible fields for working with V0 plugin format in +the registry. + +- `description` string + + Description of the plugin + +- `documentation` string + + Link to the documentation about the plugin + +- `interface` PluginInterface + + Interface implemented by the plugins, struct consisting of the following fields: + + - `types` string array + + Types indicate what interface(s) the plugin currently implements. + + Supported types: + + - `docker.volumedriver/1.0` + + - `docker.networkdriver/1.0` + + - `docker.ipamdriver/1.0` + + - `docker.authz/1.0` + + - `docker.logdriver/1.0` + + - `docker.metricscollector/1.0` + + - `socket` string + + Socket is the name of the socket the engine should use to communicate with the plugins. + the socket will be created in `/run/docker/plugins`. + +- `entrypoint` string array + + Entrypoint of the plugin, see [`ENTRYPOINT`](https://docs.docker.com/reference/dockerfile/#entrypoint) + +- `workdir` string + + Working directory of the plugin, see [`WORKDIR`](https://docs.docker.com/reference/dockerfile/#workdir) + +- `network` PluginNetwork + + Network of the plugin, struct consisting of the following fields: + + - `type` string + + Network type. + + Supported types: + + - `bridge` + - `host` + - `none` + +- `mounts` PluginMount array + + Mount of the plugin, struct consisting of the following fields. + See [`MOUNTS`](https://github.com/opencontainers/runtime-spec/blob/master/config.md#mounts). + + - `name` string + + Name of the mount. + + - `description` string + + Description of the mount. + + - `source` string + + Source of the mount. + + - `destination` string + + Destination of the mount. + + - `type` string + + Mount type. + + - `options` string array + + Options of the mount. + +- `ipchost` Boolean + + Access to host ipc namespace. + +- `pidhost` Boolean + + Access to host PID namespace. + +- `propagatedMount` string + + Path to be mounted as rshared, so that mounts under that path are visible to + Docker. This is useful for volume plugins. This path will be bind-mounted + outside of the plugin rootfs so it's contents are preserved on upgrade. + +- `env` PluginEnv array + + Environment variables of the plugin, struct consisting of the following fields: + + - `name` string + + Name of the environment variable. + + - `description` string + + Description of the environment variable. + + - `value` string + + Value of the environment variable. + +- `args` PluginArgs + + Arguments of the plugin, struct consisting of the following fields: + + - `name` string + + Name of the arguments. + + - `description` string + + Description of the arguments. + + - `value` string array + + Values of the arguments. + +- `linux` PluginLinux + + - `capabilities` string array + + Capabilities of the plugin (Linux only), see list [`here`](https://github.com/opencontainers/runc/blob/master/libcontainer/SPEC.md#security) + + - `allowAllDevices` Boolean + + If `/dev` is bind mounted from the host, and allowAllDevices is set to true, the plugin will have `rwm` access to all devices on the host. + + - `devices` PluginDevice array + + Device of the plugin, (Linux only), struct consisting of the following fields. + See [`DEVICES`](https://github.com/opencontainers/runtime-spec/blob/master/config-linux.md#devices). + + - `name` string + + Name of the device. + + - `description` string + + Description of the device. + + - `path` string + + Path of the device. + +## Example Config + +The following example shows the 'tiborvass/sample-volume-plugin' plugin config. + +```json +{ + "Args": { + "Description": "", + "Name": "", + "Settable": null, + "Value": null + }, + "Description": "A sample volume plugin for Docker", + "Documentation": "https://docs.docker.com/engine/extend/plugins/", + "Entrypoint": [ + "/usr/bin/sample-volume-plugin", + "/data" + ], + "Env": [ + { + "Description": "", + "Name": "DEBUG", + "Settable": [ + "value" + ], + "Value": "0" + } + ], + "Interface": { + "Socket": "plugin.sock", + "Types": [ + "docker.volumedriver/1.0" + ] + }, + "Linux": { + "Capabilities": null, + "AllowAllDevices": false, + "Devices": null + }, + "Mounts": null, + "Network": { + "Type": "" + }, + "PropagatedMount": "/data", + "User": {}, + "Workdir": "" +} +``` diff --git a/docs/extend/images/authz_additional_info.png b/docs/extend/images/authz_additional_info.png new file mode 100644 index 0000000000..1a6a6d01d2 Binary files /dev/null and b/docs/extend/images/authz_additional_info.png differ diff --git a/docs/extend/images/authz_allow.png b/docs/extend/images/authz_allow.png new file mode 100644 index 0000000000..f42108040b Binary files /dev/null and b/docs/extend/images/authz_allow.png differ diff --git a/docs/extend/images/authz_chunked.png b/docs/extend/images/authz_chunked.png new file mode 100644 index 0000000000..5bde2c71f9 Binary files /dev/null and b/docs/extend/images/authz_chunked.png differ diff --git a/docs/extend/images/authz_connection_hijack.png b/docs/extend/images/authz_connection_hijack.png new file mode 100644 index 0000000000..f13a2987b2 Binary files /dev/null and b/docs/extend/images/authz_connection_hijack.png differ diff --git a/docs/extend/images/authz_deny.png b/docs/extend/images/authz_deny.png new file mode 100644 index 0000000000..fa4a48584a Binary files /dev/null and b/docs/extend/images/authz_deny.png differ diff --git a/docs/extend/legacy_plugins.md b/docs/extend/legacy_plugins.md new file mode 100644 index 0000000000..11269df404 --- /dev/null +++ b/docs/extend/legacy_plugins.md @@ -0,0 +1,99 @@ +--- +title: Use Docker Engine plugins +aliases: +- "/engine/extend/plugins/" +description: "How to add additional functionality to Docker with plugins extensions" +keywords: "Examples, Usage, plugins, docker, documentation, user guide" +--- + +This document describes the Docker Engine plugins generally available in Docker +Engine. To view information on plugins managed by Docker, +refer to [Docker Engine plugin system](_index.md). + +> [!NOTE] +> Legacy plugins are superseded by Docker Engine's managed plugin system. +> For plugins installed and managed by Docker, use +> [`docker plugin install`](https://docs.docker.com/reference/cli/docker/plugin/install/) +> and the [Docker Engine plugin system](_index.md). The third-party plugins +> listed on this page are provided for historical reference, and archived +> projects are marked accordingly. + +You can extend the capabilities of the Docker Engine by loading third-party +plugins. This page explains the types of plugins and provides links to several +volume and network plugins for Docker. + +## Types of plugins + +Plugins extend Docker's functionality. They come in specific types. For +example, a [volume plugin](plugins_volume.md) might enable Docker +volumes to persist across multiple Docker hosts and a +[network plugin](plugins_network.md) might provide network plumbing. + +Currently Docker supports authorization, volume and network driver plugins. In the future it +will support additional plugin types. + +## Installing a plugin + +Follow the instructions in the plugin's documentation. + +## Finding a plugin + +The sections below provide an overview of third-party plugins that use the +legacy plugin model. + +### Network plugins + +| Plugin | Description | +| :--------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [Contiv Networking](https://github.com/contiv/netplugin) | An open source network plugin to provide infrastructure and security policies for a multi-tenant micro services deployment, while providing an integration to physical network for non-container workload. Contiv Networking implements the remote driver and IPAM APIs available in Docker 1.9 onwards. | +| [Kuryr Network Plugin](https://github.com/openstack/kuryr) | A network plugin is developed as part of the OpenStack Kuryr project and implements the Docker networking (libnetwork) remote driver API by utilizing Neutron, the OpenStack networking service. It includes an IPAM driver as well. | +| [Kathará Network Plugin](https://github.com/KatharaFramework/NetworkPlugin) | Docker Network Plugin used by Kathará, an open source container-based network emulation system for showing interactive demos/lessons, testing production networks in a sandbox environment, or developing new network protocols. | + +### Volume plugins + +| Plugin | Description | +|:---------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Azure File Storage plugin](https://github.com/Azure/azurefile-dockervolumedriver) (archived) | Lets you mount Microsoft [Azure File Storage](https://azure.microsoft.com/blog/azure-file-storage-now-generally-available/) shares to Docker containers as volumes using the SMB 3.0 protocol. [Learn more](https://azure.microsoft.com/blog/persistent-docker-volumes-with-azure-file-storage/). | +| [BeeGFS Volume Plugin](https://github.com/RedCoolBeans/docker-volume-beegfs) | An open source volume plugin to create persistent volumes in a BeeGFS parallel file system. | +| [Blockbridge plugin](https://github.com/blockbridge/blockbridge-docker-volume) | A volume plugin that provides access to an extensible set of container-based persistent storage options. It supports single and multi-host Docker environments with features that include tenant isolation, automated provisioning, encryption, secure deletion, snapshots and QoS. | +| [Contiv Volume Plugin](https://github.com/contiv/volplugin) | An open source volume plugin that provides multi-tenant, persistent, distributed storage with intent based consumption. It has support for Ceph and NFS. | +| [Convoy plugin](https://github.com/rancher/convoy) (archived) | A volume plugin for a variety of storage back-ends including device mapper and NFS. It's a simple standalone executable written in Go and provides the framework to support vendor-specific extensions such as snapshots, backups and restore. | +| [DigitalOcean Block Storage plugin](https://github.com/omallo/docker-volume-plugin-dostorage) | Integrates DigitalOcean's [block storage solution](https://www.digitalocean.com/products/storage/) into the Docker ecosystem by automatically attaching a given block storage volume to a DigitalOcean droplet and making the contents of the volume available to Docker containers running on that droplet. | +| [DRBD plugin](https://www.drbd.org/en/supported-projects/docker) | A volume plugin that provides highly available storage replicated by [DRBD](https://www.drbd.org). Data written to the docker volume is replicated in a cluster of DRBD nodes. | +| [Flocker plugin](https://github.com/ScatterHQ/flocker) (archived) | A volume plugin that provides multi-host portable volumes for Docker, enabling you to run databases and other stateful containers and move them around across a cluster of machines. | +| [Fuxi Volume Plugin](https://github.com/openstack-archive/fuxi) (archived) | A volume plugin that is developed as part of the OpenStack Kuryr project and implements the Docker volume plugin API by utilizing Cinder, the OpenStack block storage service. | +| [gce-docker plugin](https://github.com/mcuadros/gce-docker) (archived) | A volume plugin able to attach, format and mount Google Compute [persistent-disks](https://cloud.google.com/compute/docs/disks/persistent-disks). | +| [GlusterFS plugin](https://github.com/calavera/docker-volume-glusterfs) (archived) | A volume plugin that provides multi-host volumes management for Docker using GlusterFS. | +| [Horcrux Volume Plugin](https://github.com/muthu-r/horcrux) | A volume plugin that allows on-demand, version controlled access to your data. Horcrux is an open-source plugin, written in Go, and supports SCP, [Minio](https://www.minio.io) and Amazon S3. | +| [HPE 3Par Volume Plugin](https://github.com/hpe-storage/python-hpedockerplugin/) | A volume plugin that supports HPE 3Par and StoreVirtual iSCSI storage arrays. | +| [IPFS Volume Plugin](https://github.com/vdemeester/docker-volume-ipfs) (archived) | An open source volume plugin that allows using an [ipfs](https://ipfs.io/) filesystem as a volume. | +| [Keywhiz plugin](https://github.com/calavera/docker-volume-keywhiz) (archived) | A plugin that provides credentials and secret management using Keywhiz as a central repository. | +| [Linode Volume Plugin](https://github.com/linode/docker-volume-linode) | A plugin that adds the ability to manage Linode Block Storage as Docker Volumes from within a Linode. | +| [Local Persist Plugin](https://github.com/CWSpear/local-persist) | A volume plugin that extends the default `local` driver's functionality by allowing you specify a mountpoint anywhere on the host, which enables the files to *always persist*, even if the volume is removed via `docker volume rm`. | +| [NetApp Plugin](https://github.com/NetApp/netappdvp) (nDVP) | A volume plugin that provides direct integration with the Docker ecosystem for the NetApp storage portfolio. The nDVP package supports the provisioning and management of storage resources from the storage platform to Docker hosts, with a robust framework for adding additional platforms in the future. | +| [Netshare plugin](https://github.com/ContainX/docker-volume-netshare) | A volume plugin that provides volume management for NFS 3/4, AWS EFS and CIFS file systems. | +| [Nimble Storage Volume Plugin](https://scod.hpedev.io/docker_volume_plugins/hpe_nimble_storage/index.html) | A volume plug-in that integrates with Nimble Storage Unified Flash Fabric arrays. The plug-in abstracts array volume capabilities to the Docker administrator to allow self-provisioning of secure multi-tenant volumes and clones. | +| [OpenStorage Plugin](https://github.com/libopenstorage/openstorage) | A cluster-aware volume plugin that provides volume management for file and block storage solutions. It implements a vendor neutral specification for implementing extensions such as CoS, encryption, and snapshots. It has example drivers based on FUSE, NFS, NBD and EBS to name a few. | +| [Portworx Volume Plugin](https://github.com/portworx/px-dev) | A volume plugin that turns any server into a scale-out converged compute/storage node, providing container granular storage and highly available volumes across any node, using a shared-nothing storage backend that works with any docker scheduler. | +| [Quobyte Volume Plugin](https://github.com/quobyte/docker-volume) | A volume plugin that connects Docker to [Quobyte](https://www.quobyte.com/containers)'s data center file system, a general-purpose scalable and fault-tolerant storage platform. | +| [REX-Ray plugin](https://github.com/emccode/rexray) | A volume plugin which is written in Go and provides advanced storage functionality for many platforms including VirtualBox, EC2, Google Compute Engine, OpenStack, and EMC. | +| [Virtuozzo Storage and Ploop plugin](https://github.com/virtuozzo/docker-volume-ploop) | A volume plugin with support for Virtuozzo Storage distributed cloud file system as well as ploop devices. | +| [VMware vSphere Storage Plugin](https://github.com/vmware/docker-volume-vsphere) | Docker Volume Driver for vSphere enables customers to address persistent storage requirements for Docker containers in vSphere environments. | + +### Authorization plugins + +| Plugin | Description | +|:---------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Casbin AuthZ Plugin](https://github.com/casbin/casbin-authz-plugin) | An authorization plugin based on [Casbin](https://github.com/casbin/casbin), which supports access control models like ACL, RBAC, ABAC. The access control model can be customized. The policy can be persisted into file or DB. | +| [HBM plugin](https://github.com/kassisol/hbm) | An authorization plugin that prevents from executing commands with certains parameters. | +| [Twistlock AuthZ Broker](https://github.com/twistlock/authz) | A basic extendable authorization plugin that runs directly on the host or inside a container. This plugin allows you to define user policies that it evaluates during authorization. Basic authorization is provided if Docker daemon is started with the --tlsverify flag (username is extracted from the certificate common name). | + +## Troubleshooting a plugin + +If you are having problems with Docker after loading a plugin, ask the authors +of the plugin for help. The Docker team may not be able to assist you. + +## Writing a plugin + +If you are interested in writing a plugin for Docker, or seeing how they work +under the hood, see the [Docker plugins reference](plugin_api.md). diff --git a/docs/extend/plugin_api.md b/docs/extend/plugin_api.md new file mode 100644 index 0000000000..0cf2d9e1d6 --- /dev/null +++ b/docs/extend/plugin_api.md @@ -0,0 +1,186 @@ +--- +title: Docker Plugin API +description: "How to write Docker plugins extensions " +keywords: "API, Usage, plugins, documentation, developer" +--- + +Docker plugins are out-of-process extensions which add capabilities to the +Docker Engine. + +This document describes the Docker Engine plugin API. To view information on +plugins managed by Docker Engine, refer to [Docker Engine plugin system](_index.md). + +This page is intended for people who want to develop their own Docker plugin. +If you just want to learn about or use Docker plugins, look +[here](legacy_plugins.md). + +## What plugins are + +A plugin is a process running on the same or a different host as the Docker daemon, +which registers itself by placing a file on the daemon host in one of the plugin +directories described in [Plugin discovery](#plugin-discovery). + +Plugins have human-readable names, which are short, lowercase strings. For +example, `myplugin`. + +Plugins can run inside or outside containers. Currently running them outside +containers is recommended. + +## Plugin discovery + +Docker discovers plugins by looking for them in the plugin directory whenever a +user or container tries to use one by name. + +There are three types of files which can be put in the plugin directory. + +* `.sock` files are Unix domain sockets. +* `.spec` files are text files containing a URL, such as `unix:///other.sock` or `tcp://localhost:8080`. +* `.json` files are text files containing a full json specification for the plugin. + +Plugins with Unix domain socket files must run on the same host as the Docker daemon. +Plugins with `.spec` or `.json` files can run on a different host if you specify a remote URL. + +Unix domain socket files must be located under `/run/docker/plugins`, whereas +spec files can be located either under `/etc/docker/plugins` or `/usr/lib/docker/plugins`. + +The name of the file (excluding the extension) determines the plugin name. + +For example, a plugin named `myplugin` might create a Unix socket at +`/run/docker/plugins/myplugin.sock`. + +You can define each plugin into a separated subdirectory if you want to isolate definitions from each other. +For example, you can create the `myplugin` socket under `/run/docker/plugins/myplugin/myplugin.sock` and only +mount `/run/docker/plugins/myplugin` inside the `myplugin` container. + +Docker always searches for Unix sockets in `/run/docker/plugins` first. It checks for spec or json files under +`/etc/docker/plugins` and `/usr/lib/docker/plugins` if the socket doesn't exist. The directory scan stops as +soon as it finds the first plugin definition with the given name. + +### JSON specification + +This is the JSON format for a plugin: + +```json +{ + "Name": "plugin-example", + "Addr": "https://example.com/docker/plugin", + "TLSConfig": { + "InsecureSkipVerify": false, + "CAFile": "/usr/shared/docker/certs/example-ca.pem", + "CertFile": "/usr/shared/docker/certs/example-cert.pem", + "KeyFile": "/usr/shared/docker/certs/example-key.pem" + } +} +``` + +The `TLSConfig` field is optional and TLS will only be verified if this configuration is present. + +## Plugin lifecycle + +Plugins should be started before Docker, and stopped after Docker. For +example, when packaging a plugin for a platform which supports `systemd`, you +might use [`systemd` dependencies]( +https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Before=) to +manage startup and shutdown order. + +When upgrading a plugin, you should first stop the Docker daemon, upgrade the +plugin, then start Docker again. + +## Plugin activation + +When a plugin is first referred to -- either by a user referring to it by name +(e.g. `docker run --volume-driver=foo`) or a container already configured to +use a plugin being started -- Docker looks for the named plugin in the plugin +directory and activates it with a handshake. See Handshake API below. + +Plugins are not activated automatically at Docker daemon startup. Rather, +they are activated only lazily, or on-demand, when they are needed. + +## Systemd socket activation + +Plugins may also be socket activated by `systemd`. The official [Plugins helpers](https://github.com/docker/go-plugins-helpers) +natively supports socket activation. In order for a plugin to be socket activated it needs +a `service` file and a `socket` file. + +The `service` file (for example `/lib/systemd/system/your-plugin.service`): + +```systemd +[Unit] +Description=Your plugin +Before=docker.service +After=network.target your-plugin.socket +Requires=your-plugin.socket docker.service + +[Service] +ExecStart=/usr/lib/docker/your-plugin + +[Install] +WantedBy=multi-user.target +``` + +The `socket` file (for example `/lib/systemd/system/your-plugin.socket`): + +```systemd +[Unit] +Description=Your plugin + +[Socket] +ListenStream=/run/docker/plugins/your-plugin.sock + +[Install] +WantedBy=sockets.target +``` + +This will allow plugins to be actually started when the Docker daemon connects to +the sockets they're listening on (for instance the first time the daemon uses them +or if one of the plugin goes down accidentally). + +## API design + +The Plugin API is RPC-style JSON over HTTP, much like webhooks. + +Requests flow from the Docker daemon to the plugin. The plugin needs to +implement an HTTP server and bind this to the Unix socket mentioned in the +"plugin discovery" section. + +All requests are HTTP `POST` requests. + +The API is versioned via an Accept header, which currently is always set to +`application/vnd.docker.plugins.v1+json`. + +## Handshake API + +Plugins are activated via the following "handshake" API call. + +### /Plugin.Activate + +Request: empty body + +Response: + +```json +{ + "Implements": ["VolumeDriver"] +} +``` + +Responds with a list of Docker subsystems which this plugin implements. +After activation, the plugin will then be sent events from this subsystem. + +Possible values are: + +* [`authz`](plugins_authorization.md) +* [`NetworkDriver`](plugins_network.md) +* [`VolumeDriver`](plugins_volume.md) + +## Plugin retries + +Attempts to call a method on a plugin are retried with an exponential backoff +for up to 30 seconds. This may help when packaging plugins as containers, since +it gives plugin containers a chance to start up before failing any user +containers which depend on them. + +## Plugins helpers + +To ease plugins development, we're providing an `sdk` for each kind of plugins +currently supported by Docker at [docker/go-plugins-helpers](https://github.com/docker/go-plugins-helpers). diff --git a/docs/extend/plugins_authorization.md b/docs/extend/plugins_authorization.md new file mode 100644 index 0000000000..605fac16fa --- /dev/null +++ b/docs/extend/plugins_authorization.md @@ -0,0 +1,285 @@ +--- +title: Access authorization plugin +description: "How to create authorization plugins to manage access control to your Docker daemon." +keywords: "security, authorization, authentication, docker, documentation, plugin, extend" +aliases: +- "/engine/extend/authorization/" +--- + +This document describes the Docker Engine plugins available in Docker +Engine. To view information on plugins managed by Docker Engine, +refer to [Docker Engine plugin system](_index.md). + +Docker's out-of-the-box authorization model is all or nothing. Any user with +permission to access the Docker daemon can run any Docker client command. The +same is true for callers using Docker's Engine API to contact the daemon. If you +require greater access control, you can create authorization plugins and add +them to your Docker daemon configuration. Using an authorization plugin, a +Docker administrator can configure granular access policies for managing access +to the Docker daemon. + +Anyone with the appropriate skills can develop an authorization plugin. These +skills, at their most basic, are knowledge of Docker, understanding of REST, and +sound programming knowledge. This document describes the architecture, state, +and methods information available to an authorization plugin developer. + +## Basic principles + +Docker's [plugin infrastructure](plugin_api.md) enables +extending Docker by loading, removing and communicating with +third-party components using a generic API. The access authorization subsystem +was built using this mechanism. + +Using this subsystem, you don't need to rebuild the Docker daemon to add an +authorization plugin. You can add a plugin to an installed Docker daemon. You do +need to restart the Docker daemon to add a new plugin. + +An authorization plugin approves or denies requests to the Docker daemon based +on both the current authentication context and the command context. The +authentication context contains all user details and the authentication method. +The command context contains all the relevant request data. + +Authorization plugins must follow the rules described in [Docker Plugin API](plugin_api.md). +Each plugin must reside within directories described under the +[Plugin discovery](plugin_api.md#plugin-discovery) section. + +> [!NOTE] +> The abbreviations `AuthZ` and `AuthN` mean authorization and authentication +> respectively. + +## Default user authorization mechanism + +If TLS is enabled in the [Docker daemon](https://docs.docker.com/engine/security/https/), the default user authorization flow extracts the user details from the certificate subject name. +That is, the `User` field is set to the client certificate subject common name, and the `AuthenticationMethod` field is set to `TLS`. + +## Basic architecture + +You are responsible for registering your plugin as part of the Docker daemon +startup. You can install multiple plugins and chain them together. This chain +can be ordered. Each request to the daemon passes in order through the chain. +Only when all the plugins grant access to the resource, is the access granted. + +When an HTTP request is made to the Docker daemon through the CLI or via the +Engine API, the authentication subsystem passes the request to the installed +authentication plugin(s). The request contains the user (caller) and command +context. The plugin is responsible for deciding whether to allow or deny the +request. + +The sequence diagrams below depict an allow and deny authorization flow: + +![Authorization Allow flow](images/authz_allow.png) + +![Authorization Deny flow](images/authz_deny.png) + +Each request sent to the plugin includes the authenticated user, the HTTP +headers, and the request/response body. Only the user name and the +authentication method used are passed to the plugin. Most importantly, no user +credentials or tokens are passed. + +> [!NOTE] +> Authorization plugins enforce requests to the Docker daemon's HTTP API only. gRPC method +> calls, whether dispatched natively or upgraded through `POST /grpc`, are not subject to authorization. +> Furthermore, HTTP request/response bodies where the `Content-Type` is `application/json` are forwarded; +> bodies of any other type are not visible to the plugin and cannot be used for enforcement, +> even though the daemon acts on this data. + +For commands that can potentially hijack the HTTP connection (`HTTP +Upgrade`), such as `exec`, the authorization plugin is only called for the +initial HTTP requests. Once the plugin approves the command, authorization is +not applied to the rest of the flow. Specifically, the streaming data is not +passed to the authorization plugins. For commands that return chunked HTTP +response, such as `logs` and `events`, only the HTTP request is sent to the +authorization plugins. + +The Engine's authorization middleware fails closed: when a plugin returns an error or returns `Allow: false`, +the request is denied and the error is surfaced to the client. Plugins should also fail closed: if the plugin +cannot confidently evaluate a request, it should return an error or `Allow: false`. + +> [!WARNING] +> Because the plugin receives the [**raw** request body](#authzpluginauthzreq) from the daemon, it must +> apply the same decoding semantics as the daemon to be sure it evaluates the request the daemon will +> act on. The daemon decodes JSON with Go's [`encoding/json.Unmarshal`](https://pkg.go.dev/encoding/json#Unmarshal). +> +> The same requirement applies to the response body. Plugins that depend on `ResponseBody` +> inspection for redaction or content-filtering should restrict their policies to endpoints +> whose response is produced as a single write (typical of REST-style API responses). For +> commands whose responses are streamed or are likely to exceed the [buffer](#response-body-size-and-partial-buffering) through multiple +> writes, do not rely on `ResponseBody` for security-relevant decisions; perform the filtering +> in a separate layer in front of the daemon. + +### Response body size and partial buffering + +The internal buffer that holds the response body between the daemon's HTTP +handler and the plugin's response authorization callback (`responseModifier`, +defined in [`pkg/authorization/response.go`](https://github.com/moby/moby/blob/master/pkg/authorization/response.go)) +has a fixed capacity of 64 KiB (`maxBufferSize`). + +For most non-streaming endpoints the full response is buffered for plugin +inspection regardless of total size, because Go's `encoding/json` encoder +serializes the complete payload into a single underlying write. The +streaming-response exclusion noted above (for example, `logs` and `events`) +is the practical effect of this 64 KiB threshold combined with the +`io.WriteFlusher` write pattern used by streaming handlers, where each write +is immediately drained to the client and is therefore no longer available +for plugin inspection by the time the handler returns. + +During request/response processing, some authorization flows might +need to do additional queries to the Docker daemon. To complete such flows, +plugins can call the daemon API similar to a regular user. To enable these +additional queries, the plugin must provide the means for an administrator to +configure proper authentication and security policies. + +## Docker client flows + +To enable and configure the authorization plugin, the plugin developer must +support the Docker client interactions detailed in this section. + +### Setting up Docker daemon + +Enable the authorization plugin with a dedicated command line flag in the +`--authorization-plugin=PLUGIN_ID` format. The flag supplies a `PLUGIN_ID` +value. This value can be the plugin’s socket or a path to a specification file. +Authorization plugins can be loaded without restarting the daemon. Refer +to the [`dockerd` documentation](https://docs.docker.com/reference/cli/dockerd/#configuration-reload-behavior) for more information. + +```console +$ dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,... +``` + +Docker's authorization subsystem supports multiple `--authorization-plugin` parameters. + +### Calling authorized command (allow) + +```console +$ docker pull centos +<...> +f1b10cd84249: Pull complete +<...> +``` + +### Calling unauthorized command (deny) + +```console +$ docker pull centos +<...> +docker: Error response from daemon: authorization denied by plugin PLUGIN_NAME: volumes are not allowed. +``` + +### Error from plugins + +```console +$ docker pull centos +<...> +docker: Error response from daemon: plugin PLUGIN_NAME failed with error: AuthZPlugin.AuthZReq: Cannot connect to the Docker daemon. Is the docker daemon running on this host?. +``` + +## API schema and implementation + +In addition to Docker's standard plugin registration method, each plugin +should implement the following two methods: + +* `/AuthZPlugin.AuthZReq` This authorize request method is called before the Docker daemon processes the client request. + +* `/AuthZPlugin.AuthZRes` This authorize response method is called before the response is returned from Docker daemon to the client. + +#### /AuthZPlugin.AuthZReq + +Request + +```json +{ + "User": "The user identification", + "UserAuthNMethod": "The authentication method used", + "RequestMethod": "The HTTP method", + "RequestURI": "The HTTP request URI", + "RequestBody": "Byte array containing the raw HTTP request body", + "RequestHeader": "Byte array containing the raw HTTP request header as a map[string][]string " +} +``` + +Response + +```json +{ + "Allow": "Determined whether the user is allowed or not", + "Msg": "The authorization message", + "Err": "The error message if things go wrong" +} +``` + +#### /AuthZPlugin.AuthZRes + +Request: + +```json +{ + "User": "The user identification", + "UserAuthNMethod": "The authentication method used", + "RequestMethod": "The HTTP method", + "RequestURI": "The HTTP request URI", + "RequestBody": "Byte array containing the raw HTTP request body", + "RequestHeader": "Byte array containing the raw HTTP request header as a map[string][]string", + "ResponseBody": "Byte array containing the raw HTTP response body", + "ResponseHeader": "Byte array containing the raw HTTP response header as a map[string][]string", + "ResponseStatusCode":"Response status code" +} +``` + +Response: + +```json +{ + "Allow": "Determined whether the user is allowed or not", + "Msg": "The authorization message", + "Err": "The error message if things go wrong" +} +``` + +### Request authorization + +Each plugin must support two request authorization messages formats, one from the daemon to the plugin and then from the plugin to the daemon. The tables below detail the content expected in each message. + +#### Daemon -> Plugin + +Name | Type | Description +-----------------------|-------------------|------------------------------------------------------- +User | string | The user identification +Authentication method | string | The authentication method used +Request method | enum | The HTTP method (GET/DELETE/POST) +Request URI | string | The HTTP request URI including API version, as sent by the client (e.g., v.1.17/containers/json) +Request headers | map[string]string | Request headers as key value pairs (without the authorization header) +Request body | []byte | Raw request body + +#### Plugin -> Daemon + +Name | Type | Description +--------|--------|---------------------------------------------------------------------------------- +Allow | bool | Boolean value indicating whether the request is allowed or denied +Msg | string | Authorization message (will be returned to the client in case the access is denied) +Err | string | Error message (will be returned to the client in case the plugin encounter an error. The string value supplied may appear in logs, so should not include confidential information) + +### Response authorization + +The plugin must support two authorization messages formats, one from the daemon to the plugin and then from the plugin to the daemon. The tables below detail the content expected in each message. + +#### Daemon -> Plugin + +Name | Type | Description +----------------------- |------------------ |---------------------------------------------------- +User | string | The user identification +Authentication method | string | The authentication method used +Request method | string | The HTTP method (GET/DELETE/POST) +Request URI | string | The HTTP request URI including API version, as sent by the client (e.g., v.1.17/containers/json) +Request headers | map[string]string | Request headers as key value pairs (without the authorization header) +Request body | []byte | Raw request body +Response status code | int | Status code from the Docker daemon +Response headers | map[string]string | Response headers as key value pairs +Response body | []byte | Raw Docker daemon response body + +#### Plugin -> Daemon + +Name | Type | Description +--------|--------|---------------------------------------------------------------------------------- +Allow | bool | Boolean value indicating whether the response is allowed or denied +Msg | string | Authorization message (will be returned to the client in case the access is denied) +Err | string | Error message (will be returned to the client in case the plugin encounter an error. The string value supplied may appear in logs, so should not include confidential information) diff --git a/docs/extend/plugins_logging.md b/docs/extend/plugins_logging.md new file mode 100644 index 0000000000..2d30571fe3 --- /dev/null +++ b/docs/extend/plugins_logging.md @@ -0,0 +1,215 @@ +--- +title: Docker log driver plugins +description: "Log driver plugins." +keywords: "Examples, Usage, plugins, docker, documentation, user guide, logging" +--- + +This document describes logging driver plugins for Docker. + +Logging drivers enables users to forward container logs to another service for +processing. Docker includes several logging drivers as built-ins, however can +never hope to support all use-cases with built-in drivers. Plugins allow Docker +to support a wide range of logging services without requiring to embed client +libraries for these services in the main Docker codebase. See the +[plugin documentation](legacy_plugins.md) for more information. + +## Create a logging plugin + +The main interface for logging plugins uses the same JSON+HTTP RPC protocol used +by other plugin types. See the +[example](https://github.com/cpuguy83/docker-log-driver-test) plugin for a +reference implementation of a logging plugin. The example wraps the built-in +`jsonfilelog` log driver. + +## LogDriver protocol + +Logging plugins must register as a `LogDriver` during plugin activation. Once +activated users can specify the plugin as a log driver. + +There are two HTTP endpoints that logging plugins must implement: + +### `/LogDriver.StartLogging` + +Signals to the plugin that a container is starting that the plugin should start +receiving logs for. + +Logs will be streamed over the defined file in the request. On Linux this file +is a FIFO. Logging plugins are not currently supported on Windows. + +Request: + +```json +{ + "File": "/path/to/file/stream", + "Info": { + "ContainerID": "123456" + } +} +``` + +`File` is the path to the log stream that needs to be consumed. Each call to +`StartLogging` should provide a different file path, even if it's a container +that the plugin has already received logs for prior. The file is created by +Docker with a randomly generated name. + +`Info` is details about the container that's being logged. This is fairly +free-form, but is defined by the following struct definition: + +```go +type Info struct { + Config map[string]string + ContainerID string + ContainerName string + ContainerEntrypoint string + ContainerArgs []string + ContainerImageID string + ContainerImageName string + ContainerCreated time.Time + ContainerEnv []string + ContainerLabels map[string]string + LogPath string + DaemonName string +} +``` + +`ContainerID` will always be supplied with this struct, but other fields may be +empty or missing. + +Response: + +```json +{ + "Err": "" +} +``` + +If an error occurred during this request, add an error message to the `Err` field +in the response. If no error then you can either send an empty response (`{}`) +or an empty value for the `Err` field. + +The driver should at this point be consuming log messages from the passed in file. +If messages are unconsumed, it may cause the container to block while trying to +write to its stdio streams. + +Log stream messages are encoded as protocol buffers. The protobuf definitions are +in the +[moby repository](https://github.com/moby/moby/blob/master/api/types/plugins/logdriver/entry.proto). + +Since protocol buffers are not self-delimited you must decode them from the stream +using the following stream format: + +```text +[size][message] +``` + +Where `size` is a 4-byte big endian binary encoded uint32. `size` in this case +defines the size of the next message. `message` is the actual log entry. + +A reference golang implementation of a stream encoder/decoder can be found +[here](https://github.com/docker/docker/blob/master/api/types/plugins/logdriver/io.go) + +### `/LogDriver.StopLogging` + +Signals to the plugin to stop collecting logs from the defined file. +Once a response is received, the file will be removed by Docker. You must make +sure to collect all logs on the stream before responding to this request or risk +losing log data. + +Requests on this endpoint does not mean that the container has been removed +only that it has stopped. + +Request: + +```json +{ + "File": "/path/to/file/stream" +} +``` + +Response: + +```json +{ + "Err": "" +} +``` + +If an error occurred during this request, add an error message to the `Err` field +in the response. If no error then you can either send an empty response (`{}`) +or an empty value for the `Err` field. + +## Optional endpoints + +Logging plugins can implement two extra logging endpoints: + +### `/LogDriver.Capabilities` + +Defines the capabilities of the log driver. You must implement this endpoint for +Docker to be able to take advantage of any of the defined capabilities. + +Request: + +```json +{} +``` + +Response: + +```json +{ + "ReadLogs": true +} +``` + +Supported capabilities: + +- `ReadLogs` - this tells Docker that the plugin is capable of reading back logs +to clients. Plugins that report that they support `ReadLogs` must implement the +`/LogDriver.ReadLogs` endpoint + +### `/LogDriver.ReadLogs` + +Reads back logs to the client. This is used when `docker logs ` is +called. + +In order for Docker to use this endpoint, the plugin must specify as much when +`/LogDriver.Capabilities` is called. + +Request: + +```json +{ + "ReadConfig": {}, + "Info": { + "ContainerID": "123456" + } +} +``` + +`ReadConfig` is the list of options for reading, it is defined with the following +golang struct: + +```go +type ReadConfig struct { + Since time.Time + Tail int + Follow bool +} +``` + +- `Since` defines the oldest log that should be sent. +- `Tail` defines the number of lines to read (e.g. like the command `tail -n 10`) +- `Follow` signals that the client wants to stay attached to receive new log messages +as they come in once the existing logs have been read. + +`Info` is the same type defined in `/LogDriver.StartLogging`. It should be used +to determine what set of logs to read. + +Response: + +```text +{{ log stream }} +``` + +The response should be the encoded log message using the same format as the +messages that the plugin consumed from Docker. diff --git a/docs/extend/plugins_metrics.md b/docs/extend/plugins_metrics.md new file mode 100644 index 0000000000..a744406f7d --- /dev/null +++ b/docs/extend/plugins_metrics.md @@ -0,0 +1,79 @@ +--- +title: Docker metrics collector plugins +description: "Metrics plugins." +keywords: "Examples, Usage, plugins, docker, documentation, user guide, metrics" +--- + +Docker exposes internal metrics based on the Prometheus format. Metrics plugins +enable accessing these metrics in a consistent way by providing a Unix +socket at a predefined path where the plugin can scrape the metrics. + +> [!NOTE] +> While the plugin interface for metrics is non-experimental, the naming of the +> metrics and metric labels is still considered experimental and may change in a +> future version. + +## Creating a metrics plugin + +You must currently set `PropagatedMount` in the plugin `config.json` to +`/run/docker`. This allows the plugin to receive updated mounts +(the bind-mounted socket) from Docker after the plugin is already configured. + +## MetricsCollector protocol + +Metrics plugins must register as implementing the`MetricsCollector` interface +in `config.json`. + +On Unix platforms, the socket is located at `/run/docker/metrics.sock` in the +plugin's rootfs. + +`MetricsCollector` must implement two endpoints: + +### `MetricsCollector.StartMetrics` + +Signals to the plugin that the metrics socket is now available for scraping + +Request: + +```json +{} +``` + +The request has no payload. + +Response: + +```json +{ + "Err": "" +} +``` + +If an error occurred during this request, add an error message to the `Err` field +in the response. If no error then you can either send an empty response (`{}`) +or an empty value for the `Err` field. Errors will only be logged. + +### `MetricsCollector.StopMetrics` + +Signals to the plugin that the metrics socket is no longer available. +This may happen when the daemon is shutting down. + +Request: + +```json +{} +``` + +The request has no payload. + +Response: + +```json +{ + "Err": "" +} +``` + +If an error occurred during this request, add an error message to the `Err` field +in the response. If no error then you can either send an empty response (`{}`) +or an empty value for the `Err` field. Errors will only be logged. diff --git a/docs/extend/plugins_network.md b/docs/extend/plugins_network.md new file mode 100644 index 0000000000..0ff0a64b1b --- /dev/null +++ b/docs/extend/plugins_network.md @@ -0,0 +1,64 @@ +--- +title: Docker network driver plugins +description: "Network driver plugins." +keywords: "Examples, Usage, plugins, docker, documentation, user guide" +--- + +This document describes Docker Engine network driver plugins generally +available in Docker Engine. To view information on plugins +managed by Docker Engine, refer to [Docker Engine plugin system](_index.md). + +Docker Engine network plugins enable Engine deployments to be extended to +support a wide range of networking technologies, such as VXLAN, IPVLAN, MACVLAN +or something completely different. Network driver plugins are supported via the +LibNetwork project. Each plugin is implemented as a "remote driver" for +LibNetwork, which shares plugin infrastructure with Engine. Effectively, network +driver plugins are activated in the same way as other plugins, and use the same +kind of protocol. + +## Network plugins and Swarm mode + +[Legacy plugins](legacy_plugins.md) do not work in Swarm mode. However, +plugins written using the [v2 plugin system](_index.md) do work in Swarm mode, as +long as they are installed on each Swarm worker node. + +## Use network driver plugins + +The means of installing and running a network driver plugin depend on the +particular plugin. So, be sure to install your plugin according to the +instructions obtained from the plugin developer. + +Once running however, network driver plugins are used just like the built-in +network drivers: by being mentioned as a driver in network-oriented Docker +commands. For example, + +```console +$ docker network create --driver weave mynet +``` + +Some network driver plugins are listed in [plugins](legacy_plugins.md) + +The `mynet` network is now owned by `weave`, so subsequent commands +referring to that network will be sent to the plugin, + +```console +$ docker run --network=mynet busybox top +``` + +## Find network plugins + +Network plugins are written by third parties, and are published by those +third parties, either on +[Docker Hub](https://hub.docker.com/search?q=&type=plugin) +or on the third party's site. + +## Write a network plugin + +Network plugins implement the [Docker plugin API](plugin_api.md) and the network +plugin protocol + +## Network plugin protocol + +The network driver protocol, in addition to the plugin activation call, is +documented as part of libnetwork: +[https://github.com/moby/moby/blob/master/daemon/libnetwork/docs/remote.md](https://github.com/moby/moby/blob/master/daemon/libnetwork/docs/remote.md). diff --git a/docs/extend/plugins_services.md b/docs/extend/plugins_services.md new file mode 100644 index 0000000000..fb914047b8 --- /dev/null +++ b/docs/extend/plugins_services.md @@ -0,0 +1,184 @@ +--- +keywords: "API, Usage, plugins, documentation, developer" +title: Plugins and Services +--- + + + +# Using Volume and Network plugins in Docker services + +In swarm mode, it is possible to create a service that allows for attaching +to networks or mounting volumes that are backed by plugins. Swarm schedules +services based on plugin availability on a node. + + +### Volume plugins + +In this example, a volume plugin is installed on a swarm worker and a volume +is created using the plugin. In the manager, a service is created with the +relevant mount options. It can be observed that the service is scheduled to +run on the worker node with the said volume plugin and volume. Note that, +node1 is the manager and node2 is the worker. + +1. Prepare manager. In node 1: + + ```console + $ docker swarm init + Swarm initialized: current node (dxn1zf6l61qsb1josjja83ngz) is now a manager. + ``` + +2. Join swarm, install plugin and create volume on worker. In node 2: + + ```console + $ docker swarm join \ + --token SWMTKN-1-aabbccdd00112233aabbccdd00112233aabbccdd00112233aa-aabbccdd00112233... \ + 192.168.99.100:2377 + ``` + + ```console + $ docker plugin install tiborvass/sample-volume-plugin + latest: Pulling from tiborvass/sample-volume-plugin + eb9c16fbdc53: Download complete + Digest: sha256:00b42de88f3a3e0342e7b35fa62394b0a9ceb54d37f4c50be5d3167899994639 + Status: Downloaded newer image for tiborvass/sample-volume-plugin:latest + Installed plugin tiborvass/sample-volume-plugin + ``` + + ```console + $ docker volume create -d tiborvass/sample-volume-plugin --name pluginVol + ``` + +3. Create a service using the plugin and volume. In node1: + + ```console + $ docker service create --name my-service --mount type=volume,volume-driver=tiborvass/sample-volume-plugin,source=pluginVol,destination=/tmp busybox top + + $ docker service ls + z1sj8bb8jnfn my-service replicated 1/1 busybox:latest + ``` + + `docker service ls` shows service 1 instance of service running. + +4. Observe the task getting scheduled in node 2: + + ```console + $ docker ps --format '{{.ID}}\t {{.Status}} {{.Names}} {{.Command}}' + 83fc1e842599 Up 2 days my-service.1.9jn59qzn7nbc3m0zt1hij12xs "top" + ``` + +### Network plugins + +In this example, a global scope network plugin is installed on both the +swarm manager and worker. A service is created with replicated instances +using the installed plugin. We will observe how the availability of the +plugin determines network creation and container scheduling. + +Note that node1 is the manager and node2 is the worker. + + +1. Install a global scoped network plugin on both manager and worker. On node1 + and node2: + + ```console + $ docker plugin install bboreham/weave2 + Plugin "bboreham/weave2" is requesting the following privileges: + - network: [host] + - capabilities: [CAP_SYS_ADMIN CAP_NET_ADMIN] + Do you grant the above permissions? [y/N] y + latest: Pulling from bboreham/weave2 + 7718f575adf7: Download complete + Digest: sha256:2780330cc15644b60809637ee8bd68b4c85c893d973cb17f2981aabfadfb6d72 + Status: Downloaded newer image for bboreham/weave2:latest + Installed plugin bboreham/weave2 + ``` + +2. Create a network using plugin on manager. On node1: + + ```console + $ docker network create --driver=bboreham/weave2:latest globalnet + + $ docker network ls + NETWORK ID NAME DRIVER SCOPE + qlj7ueteg6ly globalnet bboreham/weave2:latest swarm + ``` + +3. Create a service on the manager and have replicas set to 8. Observe that +containers get scheduled on both manager and worker. + + On node 1: + + ```console + $ docker service create --network globalnet --name myservice --replicas=8 mrjana/simpleweb simpleweb +w90drnfzw85nygbie9kb89vpa + ``` + + ```console + $ docker ps + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + 87520965206a mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 5 seconds ago Up 4 seconds myservice.4.ytdzpktmwor82zjxkh118uf1v + 15e24de0f7aa mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 5 seconds ago Up 4 seconds myservice.2.kh7a9n3iauq759q9mtxyfs9hp + c8c8f0144cdc mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 5 seconds ago Up 4 seconds myservice.6.sjhpj5gr3xt33e3u2jycoj195 + 2e8e4b2c5c08 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 5 seconds ago Up 4 seconds myservice.8.2z29zowsghx66u2velublwmrh + ``` + + On node 2: + + ```console + $ docker ps + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + 53c0ae7c1dae mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 2 seconds ago Up Less than a second myservice.7.x44tvvdm3iwkt9kif35f7ykz1 + 9b56c627fee0 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 2 seconds ago Up Less than a second myservice.1.x7n1rm6lltw5gja3ueikze57q + d4f5927ba52c mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 2 seconds ago Up 1 second myservice.5.i97bfo9uc6oe42lymafs9rz6k + 478c0d395bd7 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 2 seconds ago Up Less than a second myservice.3.yr7nkffa48lff1vrl2r1m1ucs + ``` + +4. Scale down the number of instances. On node1: + + ```console + $ docker service scale myservice=0 + myservice scaled to 0 + ``` + +5. Disable and uninstall the plugin on the worker. On node2: + + ```console + $ docker plugin rm -f bboreham/weave2 + bboreham/weave2 + ``` + +6. Scale up the number of instances again. Observe that all containers are +scheduled on the master and not on the worker, because the plugin is not available on the worker anymore. + + On node 1: + + ```console + $ docker service scale myservice=8 + myservice scaled to 8 + ``` + + ```console + $ docker ps + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + cf4b0ec2415e mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 36 seconds myservice.3.r7p5o208jmlzpcbm2ytl3q6n1 + 57c64a6a2b88 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 36 seconds myservice.4.dwoezsbb02ccstkhlqjy2xe7h + 3ac68cc4e7b8 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 35 seconds myservice.5.zx4ezdrm2nwxzkrwnxthv0284 + 006c3cb318fc mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 36 seconds myservice.8.q0e3umt19y3h3gzo1ty336k5r + dd2ffebde435 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 36 seconds myservice.7.a77y3u22prjipnrjg7vzpv3ba + a86c74d8b84b mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 36 seconds myservice.6.z9nbn14bagitwol1biveeygl7 + 2846a7850ba0 mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 37 seconds myservice.2.ypufz2eh9fyhppgb89g8wtj76 + e2ec01efcd8a mrjana/simpleweb@sha256:317d7f221d68c86d503119b0ea12c29de42af0a22ca087d522646ad1069a47a4 "simpleweb" 39 seconds ago Up 38 seconds myservice.1.8w7c4ttzr6zcb9sjsqyhwp3yl + ``` + + On node 2: + + ```console + $ docker ps + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + ``` diff --git a/docs/extend/plugins_volume.md b/docs/extend/plugins_volume.md new file mode 100644 index 0000000000..688350f623 --- /dev/null +++ b/docs/extend/plugins_volume.md @@ -0,0 +1,365 @@ +--- +title: Docker volume plugins +description: "How to manage data with external volume plugins" +keywords: "Examples, Usage, volume, docker, data, volumes, plugin, api" +--- + +Docker Engine volume plugins enable Engine deployments to be integrated with +external storage systems such as Amazon EBS, and enable data volumes to persist +beyond the lifetime of a single Docker host. See the +[plugin documentation](legacy_plugins.md) for more information. + +## Changelog + +### 1.13.0 + +- If used as part of the v2 plugin architecture, mountpoints that are part of + paths returned by the plugin must be mounted under the directory specified by + `PropagatedMount` in the plugin configuration + ([#26398](https://github.com/docker/docker/pull/26398)) + +### 1.12.0 + +- Add `Status` field to `VolumeDriver.Get` response + ([#21006](https://github.com/docker/docker/pull/21006#)) +- Add `VolumeDriver.Capabilities` to get capabilities of the volume driver + ([#22077](https://github.com/docker/docker/pull/22077)) + +### 1.10.0 + +- Add `VolumeDriver.Get` which gets the details about the volume + ([#16534](https://github.com/docker/docker/pull/16534)) +- Add `VolumeDriver.List` which lists all volumes owned by the driver + ([#16534](https://github.com/docker/docker/pull/16534)) + +### 1.8.0 + +- Initial support for volume driver plugins + ([#14659](https://github.com/docker/docker/pull/14659)) + +## Command-line changes + +To give a container access to a volume, use the `--volume` and `--volume-driver` +flags on the `docker container run` command. The `--volume` (or `-v`) flag +accepts a volume name and path on the host, and the `--volume-driver` flag +accepts a driver type. + +```console +$ docker volume create --driver=myplugin volumename + +$ docker container run -it --volume volumename:/data busybox sh +``` + +### `--volume` + +The `--volume` (or `-v`) flag takes a value that is in the format +`:`. The two parts of the value are +separated by a colon (`:`) character. + +- The volume name is a human-readable name for the volume, and cannot begin with + a `/` character. It is referred to as `volume_name` in the rest of this topic. +- The `Mountpoint` is the path on the host (v1) or in the plugin (v2) where the + volume has been made available. + +### `--volume-driver` + +Specifying the `--volume-driver` flag together with a volume name (using +`--volume`) allows you to use plugins to manage volumes for the container. + +The `--volume-driver` flag is used as a default for all volumes created for +the container, including anonymous volumes. Use the [`--mount`] flag with +the [`volume-driver`] option to specify the driver to use for each volume +individually. + +[`--mount`]: https://docs.docker.com/reference/cli/docker/container/run/#mount +[`volume-driver`]: https://docs.docker.com/engine/storage/volumes/#start-a-container-which-creates-a-volume-using-a-volume-driver + +## Create a VolumeDriver + +The container creation endpoint (`/containers/create`) accepts a `VolumeDriver` +field of type `string` allowing to specify the name of the driver. If not +specified, it defaults to `"local"` (the default driver for local volumes). + +## Volume plugin protocol + +If a plugin registers itself as a `VolumeDriver` when activated, it must +provide the Docker Daemon with writeable paths on the host filesystem. The Docker +daemon provides these paths to containers to consume. The Docker daemon makes +the volumes available by bind-mounting the provided paths into the containers. + +> [!NOTE] +> Volume plugins should *not* write data to the `/var/lib/docker/` directory, +> including `/var/lib/docker/volumes`. The `/var/lib/docker/` directory is +> reserved for Docker. + +### `/VolumeDriver.Create` + +Request: + +```json +{ + "Name": "volume_name", + "Opts": {} +} +``` + +Instruct the plugin that the user wants to create a volume, given a user +specified volume name. The plugin does not need to actually manifest the +volume on the filesystem yet (until `Mount` is called). +`Opts` is a map of driver specific options passed through from the user request. + +Response: + +```json +{ + "Err": "" +} +``` + + Respond with a string error if an error occurred. + +### `/VolumeDriver.Remove` + +Request: + +```json +{ + "Name": "volume_name" +} +``` + +Delete the specified volume from disk. This request is issued when a user +invokes `docker rm -v` to remove volumes associated with a container. + +Response: + +```json +{ + "Err": "" +} +``` + +Respond with a string error if an error occurred. + +### `/VolumeDriver.Mount` + +Request: + +```json +{ + "Name": "volume_name", + "ID": "b87d7442095999a92b65b3d9691e697b61713829cc0ffd1bb72e4ccd51aa4d6c" +} +``` + +Docker requires the plugin to provide a volume, given a user specified volume +name. `Mount` is called once per container start. If the same `volume_name` is requested +more than once, the plugin may need to keep track of each new mount request and provision +at the first mount request and deprovision at the last corresponding unmount request. + +`ID` is a unique ID for the caller that is requesting the mount. + +Response: + +- v1 + + ```json + { + "Mountpoint": "/path/to/directory/on/host", + "Err": "" + } + ``` + +- v2 + + ```json + { + "Mountpoint": "/path/under/PropagatedMount", + "Err": "" + } + ``` + +`Mountpoint` is the path on the host (v1) or in the plugin (v2) where the volume +has been made available. + +`Err` is either empty or contains an error string. + +### `/VolumeDriver.Path` + +Request: + +```json +{ + "Name": "volume_name" +} +``` + +Request the path to the volume with the given `volume_name`. + +Response: + +- v1 + + ```json + { + "Mountpoint": "/path/to/directory/on/host", + "Err": "" + } + ``` + +- v2 + + ```json + { + "Mountpoint": "/path/under/PropagatedMount", + "Err": "" + } + ``` + +Respond with the path on the host (v1) or inside the plugin (v2) where the +volume has been made available, and/or a string error if an error occurred. + +`Mountpoint` is optional. However, the plugin may be queried again later if one +is not provided. + +### `/VolumeDriver.Unmount` + +Request: + +```json +{ + "Name": "volume_name", + "ID": "b87d7442095999a92b65b3d9691e697b61713829cc0ffd1bb72e4ccd51aa4d6c" +} +``` + +Docker is no longer using the named volume. `Unmount` is called once per +container stop. Plugin may deduce that it is safe to deprovision the volume at +this point. + +`ID` is a unique ID for the caller that is requesting the mount. + +Response: + +```json +{ + "Err": "" +} +``` + +Respond with a string error if an error occurred. + +### `/VolumeDriver.Get` + +Request: + +```json +{ + "Name": "volume_name" +} +``` + +Get info about `volume_name`. + +Response: + +- v1 + + ```json + { + "Volume": { + "Name": "volume_name", + "Mountpoint": "/path/to/directory/on/host", + "Status": {} + }, + "Err": "" + } + ``` + +- v2 + + ```json + { + "Volume": { + "Name": "volume_name", + "Mountpoint": "/path/under/PropagatedMount", + "Status": {} + }, + "Err": "" + } + ``` + +Respond with a string error if an error occurred. `Mountpoint` and `Status` are +optional. + + +### /VolumeDriver.List + +Request: + +```json +{} +``` + +Get the list of volumes registered with the plugin. + +Response: + +- v1 + + ```json + { + "Volumes": [ + { + "Name": "volume_name", + "Mountpoint": "/path/to/directory/on/host" + } + ], + "Err": "" + } + ``` + +- v2 + + ```json + { + "Volumes": [ + { + "Name": "volume_name", + "Mountpoint": "/path/under/PropagatedMount" + } + ], + "Err": "" + } + ``` + +Respond with a string error if an error occurred. `Mountpoint` is optional. + +### /VolumeDriver.Capabilities + +Request: + +```json +{} +``` + +Get the list of capabilities the driver supports. + +The driver is not required to implement `Capabilities`. If it is not +implemented, the default values are used. + +Response: + +```json +{ + "Capabilities": { + "Scope": "global" + } +} +``` + +Supported scopes are `global` and `local`. Any other value in `Scope` will be +ignored, and `local` is used. `Scope` allows cluster managers to handle the +volume in different ways. For instance, a scope of `global`, signals to the +cluster manager that it only needs to create the volume once instead of on each +Docker host. More capabilities may be added in the future. diff --git a/docs/reference/dockerd.md b/docs/reference/dockerd.md new file mode 100644 index 0000000000..61a342c655 --- /dev/null +++ b/docs/reference/dockerd.md @@ -0,0 +1,1475 @@ +--- +title: "dockerd" +description: "The daemon command description and usage" +keywords: "container, daemon, runtime" +aliases: +- /engine/reference/commandline/daemon/ +--- + + + +# daemon + +```markdown +Usage: dockerd [OPTIONS] + +A self-sufficient runtime for containers. + +Options: + --add-runtime runtime Register an additional OCI compatible runtime (default []) + --allow-direct-routing Allow remote access to published ports on container IP addresses + --apparmor-profile string Path to AppArmor profile template + --authorization-plugin list Authorization plugins to load + --bip string IPv4 address for the default bridge + --bip6 string IPv6 address for the default bridge + -b, --bridge string Attach containers to a network bridge + --bridge-accept-fwmark string In bridge networks, accept packets with this firewall mark/mask + --cdi-spec-dir list CDI specification directories to use + --cgroup-parent string Set parent cgroup for all containers + --config-file string Daemon configuration file (default "/etc/docker/daemon.json") + --containerd string containerd grpc address + --containerd-namespace string Containerd namespace to use (default "moby") + --containerd-plugins-namespace string Containerd namespace to use for plugins (default "plugins.moby") + --cpu-rt-period int Limit the CPU real-time period in microseconds for the + parent cgroup for all containers (not supported with cgroups v2) + --cpu-rt-runtime int Limit the CPU real-time runtime in microseconds for the + parent cgroup for all containers (not supported with cgroups v2) + --cri-containerd start containerd with cri + --data-root string Root directory of persistent Docker state (default "/var/lib/docker") + -D, --debug Enable debug mode + --default-address-pool pool-options Default address pools for node specific local networks + --default-cgroupns-mode string Default mode for containers cgroup namespace ("host" | "private") (default "private") + --default-gateway ip Default gateway IPv4 address for the default bridge network + --default-gateway-v6 ip Default gateway IPv6 address for the default bridge network + --default-ipc-mode string Default mode for containers ipc ("shareable" | "private") (default "private") + --default-network-opt mapmap Default network options (default map[]) + --default-runtime string Default OCI runtime for containers (default "runc") + --default-shm-size bytes Default shm size for containers (default 64MiB) + --default-stop-timeout int Set the default container stop timeout in seconds (0 to terminate immediately) (default 10) + --default-ulimit ulimit Default ulimits for containers (default []) + --dns list DNS server to use + --dns-opt list DNS options to use + --dns-search list DNS search domains to use + --exec-opt list Runtime execution options + --exec-root string Root directory for execution state files (default "/var/run/docker") + --experimental Enable experimental features + --feature map Enable feature in the daemon + --firewall-backend string Firewall backend to use, iptables or nftables + --fixed-cidr string IPv4 subnet for the default bridge network + --fixed-cidr-v6 string IPv6 subnet for the default bridge network + -G, --group string Group for the unix socket (default "docker") + --help Print usage + -H, --host list Daemon socket(s) to connect to + --host-gateway-ip list IP addresses that the special 'host-gateway' string in --add-host resolves to. + Defaults to the IP addresses of the default bridge + --http-proxy string HTTP proxy URL to use for outgoing traffic + --https-proxy string HTTPS proxy URL to use for outgoing traffic + --icc Enable inter-container communication for the default bridge network (default true) + --init Run an init in the container to forward signals and reap processes + --init-path string Path to the docker-init binary + --insecure-registry list Enable insecure registry communication + --ip ip Host IP for port publishing from the default bridge network (default 0.0.0.0) + --ip-forward Enable IP forwarding in system configuration (default true) + --ip-forward-no-drop Do not set the filter-FORWARD policy to DROP when enabling IP forwarding + --ip-masq Enable IP masquerading for the default bridge network (default true) + --ip6tables Enable addition of ip6tables rules (default true) + --iptables Enable addition of iptables rules (default true) + --ipv6 Enable IPv6 networking for the default bridge network + --label list Set key=value labels to the daemon + --live-restore Enable live restore of docker when containers are still running + --log-driver string Default driver for container logs (default "json-file") + --log-format string Set the logging format ("text"|"json") (default "text") + -l, --log-level string Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info") + --log-opt map Default log driver options for containers (default map[]) + --max-concurrent-downloads int Set the max concurrent downloads (default 3) + --max-concurrent-uploads int Set the max concurrent uploads (default 5) + --max-download-attempts int Set the max download attempts for each pull (default 5) + --metrics-addr string Set default address and port to serve the metrics api on + --mtu int Set the MTU for the default "bridge" network (default 1500) + --network-control-plane-mtu int Network Control plane MTU (default 1500) + --no-new-privileges Set no-new-privileges by default for new containers + --no-proxy string Comma-separated list of hosts or IP addresses for which the proxy is skipped + --node-generic-resource list Advertise user-defined resource + -p, --pidfile string Path to use for daemon PID file (default "/var/run/docker.pid") + --raw-logs Full timestamps without ANSI coloring + --registry-mirror list Preferred registry mirror + --rootless Enable rootless mode; typically used with RootlessKit + --seccomp-profile string Path to seccomp profile. Set to "unconfined" to disable the default seccomp profile (default "builtin") + --selinux-enabled Enable selinux support + --shutdown-timeout int Set the default shutdown timeout (default 15) + -s, --storage-driver string Storage driver to use + --storage-opt list Storage driver options + --swarm-default-advertise-addr string Set default address or interface for swarm advertised address + --tls Use TLS; implied by --tlsverify + --tlscacert string Trust certs signed only by this CA (default "~/.docker/ca.pem") + --tlscert string Path to TLS certificate file (default "~/.docker/cert.pem") + --tlskey string Path to TLS key file (default "~/.docker/key.pem") + --tlsverify Use TLS and verify the remote + --userland-proxy Use userland proxy for loopback traffic (default true) + --userland-proxy-path string Path to the userland proxy binary + --userns-remap string User/Group setting for user namespaces + --validate Validate daemon configuration and exit + -v, --version Print version information and quit +``` + +Options with [] may be specified multiple times. + +## Description + +`dockerd` is the persistent process that manages containers. Docker +uses different binaries for the daemon and client. To run the daemon you +type `dockerd`. + +To run the daemon with debug output, use `dockerd --debug` or add `"debug": true` +to [the `daemon.json` file](#daemon-configuration-file). + +> [!NOTE] +> **Enabling experimental features** +> +> Enable experimental features by starting `dockerd` with the `--experimental` +> flag or adding `"experimental": true` to the `daemon.json` file. + +### Environment variables + +The following list of environment variables are supported by the `dockerd` daemon. +Some of these environment variables are supported both by the Docker Daemon and +the `docker` CLI. Refer to [Environment variables](https://docs.docker.com/reference/cli/docker/#environment-variables) +to learn about environment variables supported by the `docker` CLI. + +| Variable | Description | +| :------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `DOCKER_CERT_PATH` | Location of your authentication keys. This variable is used both by the [`docker` CLI](https://docs.docker.com/reference/cli/docker/) and the `dockerd` daemon. | +| `DOCKER_DRIVER` | The storage driver to use. | +| `DOCKER_RAMDISK` | If set this disables `pivot_root`. | +| `DOCKER_TLS_VERIFY` | When set Docker uses TLS and verifies the remote. This variable is used both by the [`docker` CLI](https://docs.docker.com/reference/cli/docker/) and the `dockerd` daemon. | +| `DOCKER_TMPDIR` | Location for temporary files created by the daemon. | +| `HTTP_PROXY` | Proxy URL for HTTP requests unless overridden by NoProxy. See the [Go specification](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config) for details. | +| `HTTPS_PROXY` | Proxy URL for HTTPS requests unless overridden by NoProxy. See the [Go specification](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config) for details. | +| `MOBY_DISABLE_PIGZ` | Disables the use of [`unpigz`](https://linux.die.net/man/1/pigz) to decompress layers in parallel when pulling images, even if it is installed. | +| `NO_PROXY` | Comma-separated values specifying hosts that should be excluded from proxying. See the [Go specification](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config) for details. | + +## Examples + +### Proxy configuration + +> [!NOTE] +> Refer to the [Docker Desktop manual](https://docs.docker.com/desktop/networking/#httphttps-proxy-support) +> if you are running [Docker Desktop](https://docs.docker.com/desktop/). + +If you are behind an HTTP proxy server, for example in corporate settings, +you may have to configure the Docker daemon to use the proxy server for +operations such as pulling and pushing images. The daemon can be configured +in three ways: + +1. Using environment variables (`HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`). +2. Using the `http-proxy`, `https-proxy`, and `no-proxy` fields in the + [daemon configuration file](#daemon-configuration-file) (Docker Engine version 23.0 or later). +3. Using the `--http-proxy`, `--https-proxy`, and `--no-proxy` command-line + options. (Docker Engine version 23.0 or later). + +The command-line and configuration file options take precedence over environment +variables. Refer to [control and configure Docker with systemd](https://docs.docker.com/engine/daemon/proxy/) +to set these environment variables on a host using `systemd`. + +### Daemon socket option + +The Docker daemon can listen for [Docker Engine API](https://docs.docker.com/engine/api/) +requests via three different types of Socket: `unix`, `tcp`, and `fd`. + +By default, a `unix` domain socket (or IPC socket) is created at +`/var/run/docker.sock`, requiring either `root` permission, or `docker` group +membership. + +If you need to access the Docker daemon remotely, you need to enable the tcp +Socket. When using a TCP socket, the Docker daemon provides un-encrypted and +un-authenticated direct access to the Docker daemon by default. You should secure +the daemon either using the [built in HTTPS encrypted socket](https://docs.docker.com/engine/security/protect-access/), +or by putting a secure web proxy in front of it. You can listen on port `2375` on all +network interfaces with `-H tcp://0.0.0.0:2375`, or on a particular network +interface using its IP address: `-H tcp://192.168.59.103:2375`. It is +conventional to use port `2375` for un-encrypted, and port `2376` for encrypted +communication with the daemon. + +> [!NOTE] +> If you're using an HTTPS encrypted socket, keep in mind that only +> TLS version 1.0 and higher is supported. Protocols SSLv3 and below are not +> supported for security reasons. + +On systemd based systems, you can communicate with the daemon via +[systemd socket activation](https://0pointer.de/blog/projects/socket-activation.html), +with `dockerd -H fd://`. Using `fd://` works for most setups, but +you can also specify individual sockets: `dockerd -H fd://3`. If the +specified socket activated files aren't found, the daemon exits. You can +find examples of using systemd socket activation with Docker and systemd in the +[Docker source tree](https://github.com/docker/docker/tree/master/contrib/init/systemd/). + +You can configure the Docker daemon to listen to multiple sockets at the same +time using multiple `-H` options: + +The example below runs the daemon listening on the default Unix socket, and +on 2 specific IP addresses on this host: + +```console +$ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2 +``` + +The Docker client honors the `DOCKER_HOST` environment variable to set the +`-H` flag for the client. Use **one** of the following commands: + +```console +$ docker -H tcp://0.0.0.0:2375 ps +``` + +```console +$ export DOCKER_HOST="tcp://0.0.0.0:2375" + +$ docker ps +``` + +Setting the `DOCKER_TLS_VERIFY` environment variable to any value other than +the empty string is equivalent to setting the `--tlsverify` flag. The following +are equivalent: + +```console +$ docker --tlsverify ps +# or +$ export DOCKER_TLS_VERIFY=1 +$ docker ps +``` + +The Docker client honors the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` +environment variables (or the lowercase versions thereof). `HTTPS_PROXY` takes +precedence over `HTTP_PROXY`. + +The Docker client supports connecting to a remote daemon via SSH: + +```console +$ docker -H ssh://me@example.com:22/var/run/docker.sock ps +$ docker -H ssh://me@example.com:22 ps +$ docker -H ssh://me@example.com ps +$ docker -H ssh://example.com ps +``` + +To use SSH connection, you need to set up `ssh` so that it can reach the +remote host with public key authentication. Password authentication is not +supported. If your key is protected with passphrase, you need to set up +`ssh-agent`. + +#### Bind Docker to another host/port or a Unix socket + +> [!WARNING] +> Changing the default `docker` daemon binding to a TCP port or Unix `docker` +> user group introduces security risks, as it may allow non-root users to gain +> root access on the host. Make sure you control access to `docker`. If you are +> binding to a TCP port, anyone with access to that port has full Docker +> access; so it's not advisable on an open network. +{ .warning } + +With `-H` it's possible to make the Docker daemon to listen on a specific IP +and port. By default, it listens on `unix:///var/run/docker.sock` to allow +only local connections by the root user. You could set it to `0.0.0.0:2375` or +a specific host IP to give access to everybody, but that isn't recommended +because someone could gain root access to the host where the daemon is running. + +Similarly, the Docker client can use `-H` to connect to a custom port. +The Docker client defaults to connecting to `unix:///var/run/docker.sock` +on Linux, and `tcp://127.0.0.1:2376` on Windows. + +`-H` accepts host and port assignment in the following format: + +```text +tcp://[host]:[port][path] or unix://path +``` + +For example: + +- `tcp://` -> TCP connection to `127.0.0.1` on either port `2376` when TLS encryption + is on, or port `2375` when communication is in plain text. +- `tcp://host:2375` -> TCP connection on + host:2375 +- `tcp://host:2375/path` -> TCP connection on + host:2375 and prepend path to all requests +- `unix://path/to/socket` -> Unix socket located + at `path/to/socket` + +`-H`, when empty, defaults to the same value as +when no `-H` was passed in. + +`-H` also accepts short form for TCP bindings: `host:` or `host:port` or `:port` + +Run Docker in daemon mode: + +```console +$ sudo /dockerd -H 0.0.0.0:5555 & +``` + +Download an `ubuntu` image: + +```console +$ docker -H :5555 pull ubuntu +``` + +You can use multiple `-H`, for example, if you want to listen on both +TCP and a Unix socket + +```console +$ sudo dockerd -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock & +# Download an ubuntu image, use default Unix socket +$ docker pull ubuntu +# OR use the TCP port +$ docker -H tcp://127.0.0.1:2375 pull ubuntu +``` + +### Daemon storage-driver + +On Linux, the Docker daemon has support for several different image layer storage +drivers: `overlay2`, `fuse-overlayfs`, `btrfs`, and `zfs`. + +`overlay2` is the preferred storage driver for all currently supported Linux distributions, +and is selected by default. Unless users have a strong reason to prefer another storage driver, +`overlay2` should be used. + +You can find out more about storage drivers and how to select one in [Select a storage driver](https://docs.docker.com/engine/storage/drivers/select-storage-driver/). + +On Windows, the Docker daemon only supports the `windowsfilter` storage driver. + +### Options per storage driver + +Particular storage-driver can be configured with options specified with +`--storage-opt` flags. Options for `zfs` start with `zfs`, and options for +`btrfs` start with `btrfs`. + +#### ZFS options + +##### `zfs.fsname` + +Specifies the ZFS filesystem that the daemon should use to create its datasets. +By default, the ZFS filesystem in `/var/lib/docker` is used. + +###### Example + +```console +$ sudo dockerd -s zfs --storage-opt zfs.fsname=zroot/docker +``` + +#### Btrfs options + +##### `btrfs.min_space` + +Specifies the minimum size to use when creating the subvolume which is used +for containers. If user uses disk quota for btrfs when creating or running +a container with **--storage-opt size** option, Docker should ensure the +**size** can't be smaller than **btrfs.min_space**. + +###### Example + +```console +$ sudo dockerd -s btrfs --storage-opt btrfs.min_space=10G +``` + +#### Overlay2 options + +##### `overlay2.size` + +Sets the default max size of the container. It is supported only when the +backing filesystem is `xfs` and mounted with `pquota` mount option. Under these +conditions the user can pass any size less than the backing filesystem size. + +###### Example + +```console +$ sudo dockerd -s overlay2 --storage-opt overlay2.size=1G +``` + + +#### Windowsfilter options + +##### `size` + +Specifies the size to use when creating the sandbox which is used for containers. +Defaults to 20G. + +###### Example + +```powershell +C:\> dockerd --storage-opt size=40G +``` + +### Runtime options + +The Docker daemon relies on a +[OCI](https://github.com/opencontainers/runtime-spec) compliant runtime +(invoked via the `containerd` daemon) as its interface to the Linux +kernel `namespaces`, `cgroups`, and `SELinux`. + +#### Configure container runtimes + +By default, the Docker daemon uses runc as a container runtime. +You can configure the daemon to add additional runtimes. + +containerd shims installed on `PATH` can be used directly, without the need +to edit the daemon's configuration. For example, if you install the Kata +Containers shim (`containerd-shim-kata-v2`) on `PATH`, then you can select that +runtime with `docker run` without having to edit the daemon's configuration: + +```console +$ docker run --runtime io.containerd.kata.v2 +``` + +Container runtimes that don't implement containerd shims, or containerd shims +installed outside of `PATH`, must be registered with the daemon, either via the +configuration file or using the `--add-runtime` command line flag. + +For examples on how to use other container runtimes, see +[Alternative container runtimes](https://docs.docker.com/engine/daemon/alternative-runtimes/) + +##### Configure runtimes using `daemon.json` + +To register and configure container runtimes using the daemon's configuration +file, add the runtimes as entries under `runtimes`: + +```json +{ + "runtimes": { + "": {} + } +} +``` + +The key of the entry (`` in the previous example) represents the name +of the runtime. This is the name that you reference when you run a container, +using `docker run --runtime `. + +The runtime entry contains an object specifying the configuration for your +runtime. The properties of the object depends on what kind of runtime you're +looking to register: + +- If the runtime implements its own containerd shim, the object shall contain + a `runtimeType` field and an optional `options` field. + + ```json + { + "runtimes": { + "": { + "runtimeType": "", + "options": {} + } + } + } + ``` + + See [Configure shims](#configure-containerd-shims). + +- If the runtime is designed to be a drop-in replacement for runc, + the object contains a `path` field, and an optional `runtimeArgs` field. + + ```json + { + "runtimes": { + "": { + "path": "/path/to/bin", + "runtimeArgs": ["...args"] + } + } + } + ``` + + See [Configure runc drop-in replacements](#configure-runc-drop-in-replacements). + +After changing the runtimes configuration in the configuration file, +you must reload or restart the daemon for changes to take effect: + +```console +$ sudo systemctl reload dockerd +``` + +##### Configure containerd shims + +If the runtime that you want to register implements a containerd shim, +or if you want to register a runtime which uses the runc shim, +use the following format for the runtime entry: + +```json +{ + "runtimes": { + "": { + "runtimeType": "", + "options": {} + } + } +} +``` + +`runtimeType` refers to either: + +- A fully qualified name of a containerd shim. + + The fully qualified name of a shim is the same as the `runtime_type` used to + register the runtime in containerd's CRI configuration. + For example, `io.containerd.runsc.v1`. + +- The path of a containerd shim binary. + + This option is useful if you installed the containerd shim binary outside of + `PATH`. + +`options` is optional. It lets you specify the runtime configuration that you +want to use for the shim. The configuration parameters that you can specify in +`options` depends on the runtime you're registering. For most shims, +the supported configuration options are `TypeUrl` and `ConfigPath`. +For example: + +```json +{ + "runtimes": { + "gvisor": { + "runtimeType": "io.containerd.runsc.v1", + "options": { + "TypeUrl": "io.containerd.runsc.v1.options", + "ConfigPath": "/etc/containerd/runsc.toml" + } + } + } +} +``` + +You can configure multiple runtimes using the same runtimeType. For example: + +```json +{ + "runtimes": { + "gvisor-foo": { + "runtimeType": "io.containerd.runsc.v1", + "options": { + "TypeUrl": "io.containerd.runsc.v1.options", + "ConfigPath": "/etc/containerd/runsc-foo.toml" + } + }, + "gvisor-bar": { + "runtimeType": "io.containerd.runsc.v1", + "options": { + "TypeUrl": "io.containerd.runsc.v1.options", + "ConfigPath": "/etc/containerd/runsc-bar.toml" + } + } + } +} +``` + +The `options` field takes a special set of configuration parameters when used +with `"runtimeType": "io.containerd.runc.v2"`. For more information about runc +parameters, refer to the runc configuration section in +[CRI Plugin Config Guide](https://github.com/containerd/containerd/blob/v1.7.2/docs/cri/config.md#full-configuration). + +##### Configure runc drop-in replacements + +If the runtime that you want to register can act as a drop-in replacement for +runc, you can register the runtime either using the daemon configuration file, +or using the `--add-runtime` flag for the `dockerd` cli. + +When you use the configuration file, the entry uses the following format: + +```json +{ + "runtimes": { + "": { + "path": "/path/to/binary", + "runtimeArgs": ["...args"] + } + } +} +``` + +Where `path` is either the absolute path to the runtime executable, or the name +of an executable installed on `PATH`: + +```json +{ + "runtimes": { + "runc": { + "path": "runc" + } + } +} +``` + +And `runtimeArgs` lets you optionally pass additional arguments to the runtime. +Entries with this format use the containerd runc shim to invoke a custom +runtime binary. + +When you use the `--add-runtime` CLI flag, use the following format: + +```console +$ sudo dockerd --add-runtime = +``` + +Defining runtime arguments via the command line is not supported. + +For an example configuration for a runc drop-in replacement, see +[Alternative container runtimes > youki](https://docs.docker.com/engine/daemon/alternative-runtimes/#youki) + +##### Configure the default container runtime + +You can specify either the name of a fully qualified containerd runtime shim, +or the name of a registered runtime. You can specify the default runtime either +using the daemon configuration file, or using the `--default-runtime` flag for +the `dockerd` cli. + +When you use the configuration file, the entry uses the following format: + +```json +{ + "default-runtime": "io.containerd.runsc.v1" +} +``` + +When you use the `--default-runtime` CLI flag, use the following format: + +```console +$ dockerd --default-runtime io.containerd.runsc.v1 +``` + +#### Run containerd standalone + +By default, the Docker daemon automatically starts `containerd`. If you want to +control `containerd` startup, manually start `containerd` and pass the path to +the `containerd` socket using the `--containerd` flag. For example: + +```console +$ sudo dockerd --containerd /run/containerd/containerd.sock +``` + +#### Configure cgroup driver + +You can configure how the runtime should manage container cgroups, using the +`--exec-opt native.cgroupdriver` CLI flag. + +You can only specify `cgroupfs` or `systemd`. If you specify +`systemd` and it is not available, the system errors out. If you omit the +`native.cgroupdriver` option,` cgroupfs` is used on cgroup v1 hosts, `systemd` +is used on cgroup v2 hosts with systemd available. + +This example sets the `cgroupdriver` to `systemd`: + +```console +$ sudo dockerd --exec-opt native.cgroupdriver=systemd +``` + +Setting this option applies to all containers the daemon launches. + +#### Configure container isolation technology (Windows) + +For Windows containers, you can specify the default container isolation +technology to use, using the `--exec-opt isolation` flag. + +The following example makes `hyperv` the default isolation technology: + +```console +> dockerd --exec-opt isolation=hyperv +``` + +If no isolation value is specified on daemon start, on Windows client, +the default is `hyperv`, and on Windows server, the default is `process`. + +### Daemon DNS options + +To set the DNS server for all Docker containers, use: + +```console +$ sudo dockerd --dns 8.8.8.8 +``` + +To set the DNS search domain for all Docker containers, use: + +```console +$ sudo dockerd --dns-search example.com +``` + +### Insecure registries + +In this section, "registry" refers to a private registry, and `myregistry:5000` +is a placeholder example of a private registry. + +Docker considers a private registry either secure or insecure. +A secure registry uses TLS and a copy of its CA certificate is placed on the +Docker host at `/etc/docker/certs.d/myregistry:5000/ca.crt`. An insecure +registry is either not using TLS (i.e., listening on plain text HTTP), or is +using TLS with a CA certificate not known by the Docker daemon. The latter can +happen when the certificate wasn't found under +`/etc/docker/certs.d/myregistry:5000/`, or if the certificate verification +failed (i.e., wrong CA). + +By default, Docker assumes all registries to be secure, except for local registries. +Communicating with an insecure registry isn't possible +if Docker assumes that registry is secure. In order to communicate with an +insecure registry, the Docker daemon requires `--insecure-registry` in one of +the following two forms: + +* `--insecure-registry myregistry:5000` tells the Docker daemon that + myregistry:5000 should be considered insecure. +* `--insecure-registry 10.1.0.0/16` tells the Docker daemon that all registries + whose domain resolve to an IP address is part of the subnet described by the + CIDR syntax, should be considered insecure. + +The flag can be used multiple times to allow multiple registries to be marked +as insecure. + +If an insecure registry isn't marked as insecure, `docker pull`, +`docker push`, and `docker search` result in error messages, prompting +the user to either secure or pass the `--insecure-registry` flag to the Docker +daemon as described above. + +Local registries, whose IP address falls in the 127.0.0.0/8 range, are +automatically marked as insecure as of Docker 1.3.2. It isn't recommended to +rely on this, as it may change in the future. + +Enabling `--insecure-registry`, i.e., allowing un-encrypted and/or untrusted +communication, can be useful when running a local registry. However, +because its use creates security vulnerabilities it should only be enabled for +testing purposes. For increased security, users should add their CA to their +system's list of trusted CAs instead of enabling `--insecure-registry`. + +#### Legacy Registries + +Operations against registries supporting only the legacy v1 protocol are no longer +supported. Specifically, the daemon doesn't attempt to push, pull or sign in +to v1 registries. The exception to this is `search` which can still be performed +on v1 registries. + +### Running a Docker daemon behind an HTTPS_PROXY + +When running inside a LAN that uses an `HTTPS` proxy, the proxy's certificates +replace Docker Hub's certificates. These certificates must be added to your +Docker host's configuration: + +1. Install the `ca-certificates` package for your distribution +2. Ask your network admin for the proxy's CA certificate and append them to + `/etc/pki/tls/certs/ca-bundle.crt` +3. Then start your Docker daemon with `HTTPS_PROXY=http://username:password@proxy:port/ dockerd`. + The `username:` and `password@` are optional - and are only needed if your + proxy is set up to require authentication. + +This only adds the proxy and authentication to the Docker daemon's requests. +To use the proxy when building images and running containers, see +[Configure Docker to use a proxy server](https://docs.docker.com/engine/cli/proxy/) + +### Default container stop timeout + +The `--default-stop-timeout` flag sets the number of seconds to wait for a +container to stop when the container doesn't have a `--stop-timeout` configured. +The default is `10` seconds on Linux and `30` seconds on Windows. A value of `0` +doesn't wait before forcefully terminating the container. Negative values are +invalid. + +You can also configure the timeout with the `default-stop-timeout` option in the +[daemon configuration file](#daemon-configuration-file). You can reload this +option without restarting the daemon. + +### Default `ulimit` settings + +The `--default-ulimit` flag lets you set the default `ulimit` options to use for +all containers. It takes the same options as `--ulimit` for `docker run`. If +these defaults aren't set, `ulimit` settings are inherited from the Docker daemon. +Any `--ulimit` options passed to `docker run` override the daemon defaults. + +Be careful setting `nproc` with the `ulimit` flag, as `nproc` is designed by Linux to +set the maximum number of processes available to a user, not to a container. +For details, see [`docker run` reference](https://docs.docker.com/reference/cli/docker/container/run/#ulimit). + +### Access authorization + +Docker's access authorization can be extended by authorization plugins that your +organization can purchase or build themselves. You can install one or more +authorization plugins when you start the Docker `daemon` using the +`--authorization-plugin=PLUGIN_ID` option. + +```console +$ sudo dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,... +``` + +The `PLUGIN_ID` value is either the plugin's name or a path to its specification +file. The plugin's implementation determines whether you can specify a name or +path. Consult with your Docker administrator to get information about the +plugins available to you. + +Once a plugin is installed, requests made to the `daemon` through the +command line or Docker's Engine API are allowed or denied by the plugin. +If you have multiple plugins installed, each plugin, in order, must +allow the request for it to complete. + +For information about how to create an authorization plugin, refer to the +[authorization plugin](https://docs.docker.com/engine/extend/plugins_authorization/) section. + +### Configure the default AppArmor profile + +The `--apparmor-profile` option sets the path to an AppArmor profile definition +for Docker's default `docker-default` container profile. This option is only +supported on Linux hosts. The file can be a static profile or a Go template. + +The equivalent `daemon.json` configuration is: + +```json +{ + "apparmor-profile": "/etc/docker/apparmor/docker-default" +} +``` + +Restart the daemon to apply changes. For template values, safe file placement, +security considerations, and verification, see +[Customize the default AppArmor profile](https://docs.docker.com/engine/security/apparmor/#customize-the-default-profile). + +### Daemon user namespace options + +The Linux kernel +[user namespace support](https://man7.org/linux/man-pages/man7/user_namespaces.7.html) +provides additional security by enabling a process, and therefore a container, +to have a unique range of user and group IDs which are outside the traditional +user and group range utilized by the host system. One of the most important +security improvements is that, by default, container processes running as the +`root` user have expected administrative privileges it expects (with some restrictions) +inside the container, but are effectively mapped to an unprivileged `uid` on +the host. + +For details about how to use this feature, as well as limitations, see +[Isolate containers with a user namespace](https://docs.docker.com/engine/security/userns-remap/). + +### Configure host gateway IP + +The Docker daemon supports a special `host-gateway` value for the `--add-host` +flag for the `docker run` and `docker build` commands. This value resolves to +addresses on the host, so that containers can connect to services running on the +host. + +By default, `host-gateway` resolves to the IPv4 address of the default bridge, +and its IPv6 address if it has one. + +You can configure this to resolve to a different IP using the `--host-gateway-ip` +flag for the dockerd command line interface, or the `host-gateway-ip` key in +the daemon configuration file. + +To supply both IPv4 and IPv6 addresses on the command line, use two +`--host-gateway-ip` options. + +To supply addresses in the daemon configuration file, use `"host-gateway-ips"` +with a JSON array, as shown below. For compatibility with older versions of the +daemon, a single IP address can also be specified as a JSON string in option +`"host-gateway-ip"`. + +```console +$ cat > /etc/docker/daemon.json +{ "host-gateway-ips": ["192.0.2.1", "2001:db8::1111"]} +$ sudo systemctl restart docker +$ docker run -it --add-host host.docker.internal:host-gateway \ + busybox ping host.docker.internal +PING host.docker.internal (192.0.2.1): 56 data bytes +$ docker run -it --add-host host.docker.internal:host-gateway \ + busybox ping -6 host.docker.internal +PING host.docker.internal (2001:db8::1111): 56 data bytes +``` + +### Configure CDI devices + +Container Device Interface (CDI) is a +[standardized](https://github.com/cncf-tags/container-device-interface/blob/main/SPEC.md) +mechanism for container runtimes to create containers which are able to +interact with third party devices. + +CDI is currently only supported for Linux containers and is enabled by default +since Docker Engine 28.3.0. + +The Docker daemon supports running containers with CDI devices if the requested +device specifications are available on the filesystem of the daemon. + +The default specification directories are: + +- `/etc/cdi/` for static CDI Specs +- `/var/run/cdi` for generated CDI Specs + +#### Set custom locations + +To set custom locations for CDI specifications, use the +`cdi-spec-dirs` option in the `daemon.json` configuration file, or the +`--cdi-spec-dir` flag for the `dockerd` CLI: + +```json +{ + "cdi-spec-dirs": ["/etc/cdi/", "/var/run/cdi"] +} +``` + +You can view the configured CDI specification directories using the `docker info` command. + +#### Disable CDI devices + +The feature in enabled by default. To disable it, use the `cdi` options in the `daemon.json` file: + +```json +"features": { + "cdi": false +}, +``` + +To check the status of the CDI devices, run `docker info`. + +#### Daemon logging format {#log-format} + +The `--log-format` option or "log-format" option in the [daemon configuration file](#daemon-configuration-file) +lets you set the format for logs produced by the daemon. The logging format should +only be configured either through the `--log-format` command line option or +through the "log-format" field in the configuration file; using both +the command-line option and the "log-format" field in the configuration +file produces an error. If this option is not set, the default is "text". + +The following example configures the daemon through the `--log-format` command +line option to use `json` formatted logs; + +```console +$ dockerd --log-format=json +# ... +{"level":"info","msg":"API listen on /var/run/docker.sock","time":"2024-09-16T11:06:08.558145428Z"} +``` + +The following example shows a `daemon.json` configuration file with the +"log-format" set; + +```json +{ + "log-format": "json" +} +``` + +### Miscellaneous options + +IP masquerading uses address translation to allow containers without a public +IP to talk to other machines on the internet. This may interfere with some +network topologies, and can be disabled with `--ip-masq=false`. + +Docker supports soft links for the Docker data directory (`/var/lib/docker`) and +for `/var/lib/docker/tmp`. The `DOCKER_TMPDIR` and the data directory can be +set like this: + +```console +$ export DOCKER_TMPDIR=/mnt/disk2/tmp +$ sudo -E dockerd --data-root /var/lib/docker -H unix:// +```` + +#### Default cgroup parent + +The `--cgroup-parent` option lets you set the default cgroup parent +for containers. If this option isn't set, it defaults to `/docker` for +the cgroupfs driver, and `system.slice` for the systemd cgroup driver. + +If the cgroup has a leading forward slash (`/`), the cgroup is created +under the root cgroup, otherwise the cgroup is created under the daemon +cgroup. + +Assuming the daemon is running in cgroup `daemoncgroup`, +`--cgroup-parent=/foobar` creates a cgroup in +`/sys/fs/cgroup/memory/foobar`, whereas using `--cgroup-parent=foobar` +creates the cgroup in `/sys/fs/cgroup/memory/daemoncgroup/foobar` + +The systemd cgroup driver has different rules for `--cgroup-parent`. systemd +represents hierarchy by slice and the name of the slice encodes the location in +the tree. So `--cgroup-parent` for systemd cgroups should be a slice name. A +name can consist of a dash-separated series of names, which describes the path +to the slice from the root slice. For example, `--cgroup-parent=user-a-b.slice` +means the memory cgroup for the container is created in +`/sys/fs/cgroup/memory/user.slice/user-a.slice/user-a-b.slice/docker-.scope`. + +This setting can also be set per container, using the `--cgroup-parent` +option on `docker create` and `docker run`, and takes precedence over +the `--cgroup-parent` option on the daemon. + +#### Daemon metrics + +The `--metrics-addr` option takes a TCP address to serve the metrics API. + +To serve the metrics API on `localhost:9323` you would specify `--metrics-addr 127.0.0.1:9323`, +allowing you to make requests on the API at `127.0.0.1:9323/metrics` to receive metrics in the +[prometheus](https://prometheus.io/docs/instrumenting/exposition_formats/) format. + +Port `9323` is the [default port associated with Docker +metrics](https://github.com/prometheus/prometheus/wiki/Default-port-allocations) +to avoid collisions with other Prometheus exporters and services. + +If you are running a Prometheus server you can add this address to your scrape configs +to have Prometheus collect metrics on Docker. For more information, see +[Collect Docker metrics with Prometheus](https://docs.docker.com/engine/daemon/prometheus/). + +#### Node generic resources + +The `--node-generic-resources` option takes a list of key-value +pair (`key=value`) that allows you to advertise user defined resources +in a Swarm cluster. + +The current expected use case is to advertise NVIDIA GPUs so that services +requesting `NVIDIA-GPU=[0-16]` can land on a node that has enough GPUs for +the task to run. + +Example of usage: + +```json +{ + "node-generic-resources": [ + "NVIDIA-GPU=UUID1", + "NVIDIA-GPU=UUID2" + ] +} +``` + +### Enable feature in the daemon (--feature) {#feature} + +The `--feature` option lets you enable or disable a feature in the daemon. +This option corresponds with the "features" field in the [daemon.json configuration file](#daemon-configuration-file). +Features should only be configured either through the `--feature` command line +option or through the "features" field in the configuration file; using both +the command-line option and the "features" field in the configuration +file produces an error. The feature option can be specified multiple times +to configure multiple features. The `--feature` option accepts a name and +optional boolean value. When omitting the value, the default is `true`. + +The following example runs the daemon with the `cdi` and `containerd-snapshotter` +features enabled. The `cdi` option is provided with a value; + +```console +$ dockerd --feature cdi=true --feature containerd-snapshotter +``` + +The following example is the equivalent using the `daemon.json` configuration +file; + +```json +{ + "features": { + "cdi": true, + "containerd-snapshotter": true + } +} +``` + +### Daemon configuration file + +The `--config-file` option allows you to set any configuration option +for the daemon in a JSON format. This file uses the same flag names as keys, +except for flags that allow several entries, where it uses the plural +of the flag name, e.g., `labels` for the `label` flag. + +The options set in the configuration file must not conflict with options set +using flags. The Docker daemon fails to start if an option is duplicated between +the file and the flags, regardless of their value. This is intentional, and avoids +silently ignore changes introduced in configuration reloads. +For example, the daemon fails to start if you set daemon labels +in the configuration file and also set daemon labels via the `--label` flag. +Options that are not present in the file are ignored when the daemon starts. + +The `--validate` option allows to validate a configuration file without +starting the Docker daemon. A non-zero exit code is returned for invalid +configuration files. + +```console +$ dockerd --validate --config-file=/tmp/valid-config.json +configuration OK + +$ echo $? +0 + +$ dockerd --validate --config-file /tmp/invalid-config.json +unable to configure the Docker daemon with file /tmp/invalid-config.json: the following directives don't match any configuration option: unknown-option + +$ echo $? +1 +``` + +##### On Linux + +The default location of the configuration file on Linux is `/etc/docker/daemon.json`. +If the file does not exist, you need to create it first. Use the `--config-file` +flag to specify a non-default location. + +The following is a full example of the allowed configuration options on Linux: + +```json +{ + "allow-direct-routing": false, + "apparmor-profile": "", + "authorization-plugins": [], + "bip": "", + "bip6": "", + "bridge": "", + "bridge-accept-fwmark": "", + "builder": { + "gc": { + "enabled": true, + "defaultReservedSpace": "10GB", + "policy": [ + { "maxUsedSpace": "512MB", "keepDuration": "48h", "filter": [ "type=source.local" ] }, + { "reservedSpace": "10GB", "maxUsedSpace": "100GB", "keepDuration": "1440h" }, + { "reservedSpace": "50GB", "minFreeSpace": "20GB", "maxUsedSpace": "200GB", "all": true } + ] + } + }, + "cgroup-parent": "", + "containerd": "/run/containerd/containerd.sock", + "containerd-namespace": "docker", + "containerd-plugins-namespace": "docker-plugins", + "data-root": "", + "debug": true, + "default-address-pools": [ + { + "base": "172.30.0.0/16", + "size": 24 + }, + { + "base": "172.31.0.0/16", + "size": 24 + } + ], + "default-cgroupns-mode": "private", + "default-gateway": "", + "default-gateway-v6": "", + "default-network-opts": {}, + "default-runtime": "runc", + "default-shm-size": "64M", + "default-stop-timeout": 10, + "default-ulimits": { + "nofile": { + "Hard": 64000, + "Name": "nofile", + "Soft": 64000 + } + }, + "dns": [], + "dns-opts": [], + "dns-search": [], + "exec-opts": [], + "exec-root": "", + "experimental": false, + "features": { + "cdi": true, + "containerd-snapshotter": true + }, + "firewall-backend": "", + "fixed-cidr": "", + "fixed-cidr-v6": "", + "group": "", + "host-gateway-ip": "", + "hosts": [], + "proxies": { + "http-proxy": "http://proxy.example.com:80", + "https-proxy": "https://proxy.example.com:443", + "no-proxy": "*.test.example.com,.example.org" + }, + "icc": false, + "init": false, + "init-path": "/usr/libexec/docker-init", + "insecure-registries": [], + "ip": "0.0.0.0", + "ip-forward": false, + "ip-masq": false, + "iptables": false, + "ip6tables": false, + "ipv6": false, + "labels": [], + "live-restore": true, + "log-driver": "json-file", + "log-format": "text", + "log-level": "", + "log-opts": { + "cache-disabled": "false", + "cache-max-file": "5", + "cache-max-size": "20m", + "cache-compress": "true", + "env": "os,customer", + "labels": "somelabel", + "max-file": "5", + "max-size": "10m" + }, + "max-concurrent-downloads": 3, + "max-concurrent-uploads": 5, + "max-download-attempts": 5, + "mtu": 0, + "no-new-privileges": false, + "node-generic-resources": [ + "NVIDIA-GPU=UUID1", + "NVIDIA-GPU=UUID2" + ], + "pidfile": "", + "raw-logs": false, + "registry-mirrors": [], + "runtimes": { + "cc-runtime": { + "path": "/usr/bin/cc-runtime" + }, + "custom": { + "path": "/usr/local/bin/my-runc-replacement", + "runtimeArgs": [ + "--debug" + ] + } + }, + "seccomp-profile": "", + "selinux-enabled": false, + "shutdown-timeout": 15, + "storage-driver": "", + "storage-opts": [], + "swarm-default-advertise-addr": "", + "tls": true, + "tlscacert": "", + "tlscert": "", + "tlskey": "", + "tlsverify": true, + "userland-proxy": false, + "userland-proxy-path": "/usr/libexec/docker-proxy", + "userns-remap": "" +} +``` + +> [!NOTE] +> You can't set options in `daemon.json` that have already been set on +> daemon startup as a flag. +> On systems that use systemd to start the Docker daemon, `-H` is already set, so +> you can't use the `hosts` key in `daemon.json` to add listening addresses. +> See [custom Docker daemon options](https://docs.docker.com/engine/daemon/proxy/#systemd-unit-file) +> for an example on how to configure the daemon using systemd drop-in files. + +##### On Windows + +The default location of the configuration file on Windows is +`%programdata%\docker\config\daemon.json`. If the file does not exist, you need +to create it first. Use the `--config-file` flag to specify a non-default +location. + +The following is a full example of the allowed configuration options on Windows: + +```json +{ + "authorization-plugins": [], + "bridge": "", + "containerd": "\\\\.\\pipe\\containerd-containerd", + "containerd-namespace": "docker", + "containerd-plugins-namespace": "docker-plugins", + "data-root": "", + "debug": true, + "default-network-opts": {}, + "default-runtime": "", + "default-stop-timeout": 30, + "default-ulimits": {}, + "dns": [], + "dns-opts": [], + "dns-search": [], + "exec-opts": [], + "experimental": false, + "features": {}, + "fixed-cidr": "", + "group": "", + "host-gateway-ip": "", + "hosts": [], + "insecure-registries": [], + "labels": [], + "log-driver": "", + "log-format": "text", + "log-level": "", + "max-concurrent-downloads": 3, + "max-concurrent-uploads": 5, + "max-download-attempts": 5, + "mtu": 0, + "pidfile": "", + "raw-logs": false, + "registry-mirrors": [], + "shutdown-timeout": 15, + "storage-driver": "", + "storage-opts": [], + "swarm-default-advertise-addr": "", + "tlscacert": "", + "tlscert": "", + "tlskey": "", + "tlsverify": true +} +``` + +The `default-runtime` option is by default unset, in which case dockerd automatically detects the runtime. +This detection is based on if the `containerd` flag is set. + +Accepted values: + +- `com.docker.hcsshim.v1` - This is the built-in runtime that Docker has used since Windows supported was first added and uses the v1 HCS API's in Windows. +- `io.containerd.runhcs.v1` - This is uses the containerd `runhcs` shim to run the container and uses the v2 HCS API's in Windows. + +#### Feature options + +The optional field `features` in `daemon.json` lets you enable or disable specific +daemon features. + +```json +{ + "features": { + "some-feature": true, + "some-disabled-feature-enabled-by-default": false + } +} +``` + +The list of feature options include: + +- `containerd-snapshotter`: when set to `true`, the daemon uses containerd + snapshotters instead of the classic storage drivers for storing image and + container data. For more information, see + [containerd storage](https://docs.docker.com/engine/storage/containerd/). +- `windows-dns-proxy`: when set to `true`, the daemon's internal DNS resolver + will forward requests to external servers. Without this, most applications + running in the container will still be able to use secondary DNS servers + configured in the container itself, but `nslookup` won't be able to resolve + external names. The current default is `false`, it will change to `true` in + a future release. This option is only allowed on Windows. + + > [!WARNING] + > The `windows-dns-proxy` feature flag will be removed in a future release. + +#### Configuration reload behavior + +Some options can be reconfigured when the daemon is running without requiring +to restart the process. The daemon uses the `SIGHUP` signal in Linux to reload, +and a global event in Windows with the key `Global\docker-daemon-config-$PID`. +You can modify the options in the configuration file, but the daemon still +checks for conflicting settings with the specified CLI flags. The daemon fails +to reconfigure itself if there are conflicts, but it won't stop execution. + +The list of currently supported options that can be reconfigured is this: + +| Option | Description | +| ---------------------------------- | ----------------------------------------------------------------------------------------------------------- | +| `debug` | Toggles debug mode of the daemon. | +| `labels` | Replaces the daemon labels with a new set of labels. | +| `live-restore` | Toggles [live restore](https://docs.docker.com/engine/daemon/live-restore/). | +| `max-concurrent-downloads` | Configures the max concurrent downloads for each pull. | +| `max-concurrent-uploads` | Configures the max concurrent uploads for each push. | +| `max-download-attempts` | Configures the max download attempts for each pull. | +| `default-runtime` | Configures the runtime to be used if not is specified at container creation. | +| `default-stop-timeout` | Configures the timeout for stopping containers that have no container-specific timeout. | +| `runtimes` | Configures the list of available OCI runtimes that can be used to run containers. | +| `authorization-plugin` | Specifies the authorization plugins to use. | +| `insecure-registries` | Specifies a list of registries that the daemon should consider insecure. | +| `registry-mirrors` | Specifies a list of registry mirrors. | +| `shutdown-timeout` | Configures the daemon's existing configuration timeout with a new timeout for shutting down all containers. | +| `features` | Enables or disables specific features. | + +### Run multiple daemons + +> [!NOTE] +> Running multiple daemons on a single host is considered experimental. +> You may encounter unsolved problems, and things may not work as expected in some cases. + +This section describes how to run multiple Docker daemons on a single host. To +run multiple daemons, you must configure each daemon so that it doesn't +conflict with other daemons on the same host. You can set these options either +by providing them as flags, or by using a [daemon configuration file](#daemon-configuration-file). + +The following daemon options must be configured for each daemon: + +```text +-b, --bridge= Attach containers to a network bridge +--exec-root=/var/run/docker Root of the Docker execdriver +--data-root=/var/lib/docker Root of persisted Docker data +-p, --pidfile=/var/run/docker.pid Path to use for daemon PID file +-H, --host=[] Daemon socket(s) to connect to +--iptables=true Enable addition of iptables rules +--config-file=/etc/docker/daemon.json Daemon configuration file +--tlscacert="~/.docker/ca.pem" Trust certs signed only by this CA +--tlscert="~/.docker/cert.pem" Path to TLS certificate file +--tlskey="~/.docker/key.pem" Path to TLS key file +``` + +When your daemons use different values for these flags, you can run them on the same host without any problems. +It is important that you understand the meaning of these options and to use them correctly. + +- The `-b, --bridge=` flag is set to `docker0` as default bridge network. + It is created automatically when you install Docker. + If you aren't using the default, you must create and configure the bridge manually, or set it to 'none': `--bridge=none` +- `--exec-root` is the path where the container state is stored. + The default value is `/var/run/docker`. + Specify the path for your running daemon here. +- `--data-root` is the path where persisted data such as images, volumes, and + cluster state are stored. The default value is `/var/lib/docker`. To avoid any + conflict with other daemons, set this parameter separately for each daemon. +- `-p, --pidfile=/var/run/docker.pid` is the path where the process ID of the daemon is stored. + Specify the path for your PID file here. +- `--host=[]` specifies where the Docker daemon listens for client connections. + If unspecified, it defaults to `/var/run/docker.sock`. +- `--iptables=false` prevents the Docker daemon from adding iptables rules. If + multiple daemons manage iptables rules, they may overwrite rules set by another + daemon. Be aware that disabling this option requires you to manually add + iptables rules to expose container ports. If you prevent Docker from adding + iptables rules, Docker also doesn't add IP masquerading rules, even if you set + `--ip-masq` to `true`. Without IP masquerading rules, Docker containers can't + connect to external hosts or the internet when using network other than default bridge. +- `--config-file=/etc/docker/daemon.json` is the path where configuration file is stored. + You can use it instead of daemon flags. Specify the path for each daemon. +- `--tls*` Docker daemon supports `--tlsverify` mode that enforces encrypted and authenticated remote connections. + The `--tls*` options enable use of specific certificates for individual daemons. + +Example script for a separate “bootstrap” instance of the Docker daemon without network: + +```console +$ sudo dockerd \ + -H unix:///var/run/docker-bootstrap.sock \ + -p /var/run/docker-bootstrap.pid \ + --iptables=false \ + --ip-masq=false \ + --bridge=none \ + --data-root=/var/lib/docker-bootstrap \ + --exec-root=/var/run/docker-bootstrap +``` + +### Default network options + +The `default-network-opts` key in the `daemon.json` configuration file, and the +equivalent `--default-network-opt` CLI flag, let you specify default values for +driver network driver options for new networks. + +The following example shows how to configure options for the `bridge` driver +using the `daemon.json` file. + +```json +{ + "default-network-opts": { + "bridge": { + "com.docker.network.bridge.host_binding_ipv4": "127.0.0.1", + "com.docker.network.driver.mtu": "1234" + } + } +} +``` + +This example uses the `bridge` network driver. Refer to the +[bridge network driver page](https://docs.docker.com/engine/network/drivers/bridge/#options) +for an overview of available driver options. + +After changing the configuration and restarting the daemon, new networks that +you create use these option configurations as defaults. + +```console +$ docker network create mynet +$ docker network inspect mynet --format "{{json .Options}}" +{"com.docker.network.bridge.host_binding_ipv4":"127.0.0.1","com.docker.network.driver.mtu":"1234"} +``` + +Note that changing this daemon configuration doesn't affect pre-existing +networks. + +Using the `--default-network-opt` CLI flag is useful for testing and debugging +purposes, but you should prefer using the `daemon.json` file for persistent +daemon configuration. The CLI flag expects a value with the following format: +`driver=opt=value`, for example: + +```console +$ sudo dockerd \ + --default-network-opt bridge=com.docker.network.bridge.host_binding_ipv4=127.0.0.1 \ + --default-network-opt bridge=com.docker.network.driver.mtu=1234 +```