Commit Graph

310 Commits

Author SHA1 Message Date
Kir Kolyshkin
b66d4b567a image/v1: rm regexp use
Replace the regexp checking ID validity with a for loop.

The benefits are:
 - faster (up to 10x faster with less allocations);
 - no init overhead to compile the regexp.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2024-07-16 12:46:57 -07:00
Sebastiaan van Stijn
d0aa3eaccf Migrate to github.com/containerd/platforms module
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.

The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-02 21:20:29 +02:00
Sebastiaan van Stijn
e19e6cf7f4 pkg/stringid: deprecate ValidateID
This function is only used for the legacy v1 image format.

Deprecate the function, and make image/v1 self-contained.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 15:34:38 +02:00
Paweł Gronowski
ad0f263eb5 tarexport: Plumb ctx, add OTEL spans, handle cancellation
Pass `context.Context` through `tarexport.Load` and `tarexport.Save`.
Create OTEL spans for the most time consuming operations.

Also, handle context cancellations to actually end saving/loading when
the operation is cancelled - before this PR the daemon would still be
performing the operation even though the user already cancelled it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-05-14 13:08:33 +02:00
Brian Goff
9160b9fda6 save: Remove platform from config descriptor
This was brought up by bmitch that its not expected to have a platform
object in the config descriptor.
Also checked with tianon who agreed, its not _wrong_ but is unexpected
and doesn't neccessarily make sense to have it there.

Also, while technically incorrect, ECR is throwing an error when it sees
this.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-04-02 17:15:52 +00:00
Paweł Gronowski
e8496b1ee4 imageService: Extract common code from MakeImageCache
Both containerd and graphdriver image service use the same code to
create the cache - they only supply their own `cacheAdaptor` struct.

Extract the shared code to `cache.New`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 15:39:42 +01:00
Paweł Gronowski
d66177591e c8d/cache: Use the same cache logic as graphdrivers
Implement the cache adaptor for containerd image store and use the same
cache logic.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:06 +01:00
Paweł Gronowski
bf30fee58a image/cache: Refactor backend specific code
Move image store backend specific code out of the cache code and move it
to a separate interface to allow using the same cache code with
containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:04 +01:00
Sebastiaan van Stijn
03a17a2887 migrate image spec to github.com/moby/docker-image-spec
The specification was migrated to a separate module:
https://github.com/moby/docker-image-spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-09 19:12:18 +01:00
Sebastiaan van Stijn
b71c2792d2 image/spec: remove link to docs.docker.com "registry" specification
This spec is not directly relevant for the image spec, and the Docker
documentation no longer includes the actual specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-08 17:39:18 +01:00
Paweł Gronowski
2c01d53d96 image/cache: Use Platform from ocispec
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 14:26:51 +01:00
Sebastiaan van Stijn
203ffb1c09 Merge pull request #47330 from vvoland/cache-fix-older-windows
image/cache: Ignore Build and Revision on Windows
2024-02-06 13:02:00 +01:00
Sebastiaan van Stijn
2156635843 Merge pull request #47232 from vvoland/fix-save-manifests
image/save: Fix untagged images not present in index.json
2024-02-05 19:06:54 +01:00
Paweł Gronowski
91ea04089b image/cache: Ignore Build and Revision on Windows
The compatibility depends on whether `hyperv` or `process` container
isolation is used.
This fixes cache not being used when building images based on older
Windows versions on a newer Windows host.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 17:41:21 +01:00
Paweł Gronowski
d131f00fff image/save: Fix untagged images not present in index.json
Saving an image via digested reference, ID or truncated ID doesn't store
the image reference in the archive. This also causes the save code to
not add the image's manifest to the index.json.
This commit explicitly adds the untagged manifests to the index.json if
no tagged manifests were added.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-29 16:26:37 +01:00
Paweł Gronowski
877ebbe038 image/cache: Check image platform
Make sure the cache candidate platform matches the requested.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:05 +01:00
Paweł Gronowski
96ac22768a image/cache: Restrict cache candidates to locally built images
Restrict cache candidates only to images that were built locally.
This doesn't affect builds using `--cache-from`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:03 +01:00
Paweł Gronowski
c6156dc51b daemon/imageStore: Mark images built locally
Store additional image property which makes it possible to distinguish
if image was built locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:00 +01:00
Paweł Gronowski
537348763f image/cache: Compare all config fields
Add checks for some image config fields that were missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:03:58 +01:00
Sebastiaan van Stijn
f7e2357745 image: remove deprecated IDFromDigest
This function was deprecated in 456ea1bb1d
(Docker v24.0), and is no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:54:59 +01:00
Paweł Gronowski
17fd6562bf image/save: Fix layers order in OCI manifest
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:48:12 +01:00
Paweł Gronowski
4979605212 image/save: Change layers type to DiffID
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:47:57 +01:00
Paweł Gronowski
9f50e06993 image/save: Refactor saveLayer
`diffIDPaths` is not used and can be removed.

`savedConfig` stores if the config was already saved (ID of the image is
the ID of the config).
`savedLayers` stores if the layer (diff ID) was already saved.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 11:14:15 +01:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Ben Lovy
341e7b35a6 Updated manifest v2 URLs in image spec README
Signed-off-by: Ben Lovy <ben@deciduously.com>
2023-12-28 16:13:26 -05:00
Sebastiaan van Stijn
f05dce8d3a replace some strings with OCI consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-17 16:34:52 +02:00
Sebastiaan van Stijn
cff4f20c44 migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Sebastiaan van Stijn
a3c97beee0 image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
1148a24e64 migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
9c4e82435e Merge pull request #46351 from thaJeztah/api_events_actions_enum
api/types/events: define "Action" type and consts
2023-09-05 11:11:42 +02:00
Paweł Gronowski
366a5f1d74 image/spec: Add Go structs
Add Go structs describing the image spec which extend the OCI types.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 12:29:45 +02:00
Sebastiaan van Stijn
e9b3a16cdc Merge pull request #46381 from vvoland/image-spec-healthcheck-startinterval
image/spec: Add Healthcheck.StartInterval (v1.3)
2023-08-31 12:23:32 +02:00
Paweł Gronowski
536826d66f image/spec: Add Healthcheck.StartInterval (v1.3)
This field was added in 2216d3ca8d.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 11:22:54 +02:00
Paweł Gronowski
6dd78a9ac2 image/spec: Fix typo in "superseded"
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 10:35:14 +02:00
Sebastiaan van Stijn
d864d1eb97 image/spec: add back versioned files
Add back files at the old locations, as there may be external links
referencing the specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:17:17 +02:00
Sebastiaan van Stijn
7aa0bc5790 merge image-spec v1.2 into specs.md
This merges the v1.2 specs to provide a single history of the
specification.

To view the combined history:

    git log --follow image/spec/spec.md

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:11:14 +02:00
Sebastiaan van Stijn
823d0b3765 merge image-spec v1.1 into specs.md
This merges the v1.1 specs to provide a single history of the
specification.

To view the combined history:

    git log --follow image/spec/spec.md
2023-08-30 16:09:30 +02:00
Sebastiaan van Stijn
b8de6c01b5 image/spec: rename v1.2.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:06:36 +02:00
Sebastiaan van Stijn
c3f2bd1009 image/spec: rename v1.1.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:06:07 +02:00
Sebastiaan van Stijn
7ff38506ab image/spec: rename v1.0.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:05:34 +02:00
Sebastiaan van Stijn
32d5e6312d image/spec: use permalink to versioned documents
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:04:08 +02:00
Sebastiaan van Stijn
7b7019a4da image/spec: add missing "Shell" field (v1.2)
This field was added in b18ae8c9cc, which
was part of v1.12.0-rc1 and later, which used image spec v1.2.0.

This patch amends the v1.2 spec to include the missing field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 14:33:11 +02:00
Sebastiaan van Stijn
5380f3f0c4 image/spec: add missing "ArgsEscaped" field (v1.1, v1.2)
This field was added in 9db5db1b94, which
was part of v1.10.0-rc1 and later, which used image spec v1.1.0.

It's worth noting that documentation for the v1.1.0 image spec was not
yet available until commit 4fa0eccd10,
which was included in v1.12.0-rc1 and up. The `ArgsEscaped` field was
also adopted by the OCI image spec since [v1.1.0-rc3][1], but considered
deprecated, and not recommended to be used.

This patch amends the v1.1 and v1.2 specifications to describe the field.

[1]: 59780aa569

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 13:25:00 +02:00
Sebastiaan van Stijn
f33370219b image/spec: add missing "OnBuild" field
This field was added in commit 9f994c9646,
which was merged before the image-spec v1.0.0 was released (which happened
in commit 79910625f0).

This patch backfills the specifications to describe the property.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 13:04:26 +02:00
Sebastiaan van Stijn
e19edd3f25 image/spec: minor fixes and touch-ups in markdown files
- remove some trailing commas, which made the JSON invalid (some of these
  were fixed in the 1.2 spec, but not in older versions).
- synchronise some formatting / phrasing between versions, to make them
  easier to compare.
- remove non-breaking spaces (`NBSP`) in example outputs, and replace
  them with regular spaces.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 11:18:20 +02:00
Sebastiaan van Stijn
0f871f8cb7 api/types/events: define "Action" type and consts
Define consts for the Actions we use for events, instead of "ad-hoc" strings.
Having these consts makes it easier to find where specific events are triggered,
makes the events less error-prone, and allows documenting each Action (if needed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 00:38:08 +02:00
Sebastiaan van Stijn
87cedbe443 image: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:57:22 +02:00
Paweł Gronowski
01214bafd2 c8d/builder: Don't drop fields from created image
Previous image created a new partially filled image.
This caused child images to lose their parent's layers.

Instead of creating a new object and trying to replace its fields, just
clone the original passed image and change its ID to the manifest
digest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-24 09:44:12 +02:00
Sebastiaan van Stijn
5e2a1195d7 swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00