Commit Graph

44 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
fdd61821da use consistent alias for docker image spec
This package was aliased as "imagespec" in some places, and "dockerspec"
in other places, which made it easy to confuse.

Change all uses of this package to be aliased as "dockerspec" and configure
an "importas" linting check to enforce it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-10-06 14:24:41 +02:00
Derek McGowan
f74e5d48b3 Create github.com/moby/moby/v2 module
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-31 10:13:29 -07:00
Sebastiaan van Stijn
83510a26b3 api/types: move backend types to daemon/server
The "backend" types in API were designed to decouple the API server
implementation from the daemon, or other parts of the code that
back the API server. This would allow the daemon to evolve (e.g.
functionality moved to different subsystems) without that impacting
the API server's implementation.

Now that the API server is no longer part of the API package (module),
there is no benefit to having it in the API module. The API server
may evolve (and require changes in the backend), which has no direct
relation with the API module (types, responses); the backend definition
is, however, coupled to the API server implementation.

It's worth noting that, while "technically" possible to use the API
server package, and implement an alternative backend implementation,
this has never been a prime objective. The backend definition was
never considered "stable", and we don't expect external users to
(attempt) to use it as such.

This patch moves the backend types to the daemon/server package,
so that they can evolve with the daemon and API server implementation
without that impacting the API module (which we intend to be stable,
following SemVer).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-07-28 00:03:04 +02:00
Derek McGowan
f24455c90b Move image to daemon/internal/image
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-24 12:11:02 -07:00
Derek McGowan
afd6487b2e Create github.com/moby/moby/api module
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-21 09:30:05 -07:00
Matthieu MOREL
381d9d0723 fix use-errors-new from revive
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-26 12:07:38 +00:00
Derek McGowan
d0154d3e59 Update to use github.com/moby/go-archive
Update use of idtools to moby/user for archive and other deprecated uses

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-04-08 17:35:05 -07:00
Derek McGowan
0aa8fe0bf9 Update to containerd v2.0.2, buildkit v0.19.0-rc2
Update buildkit version to commit which uses 2.0

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-15 14:09:30 +01:00
Derek McGowan
a63602472d Use local cleanup method rather than dependency
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-14 21:36:58 -08:00
Derek McGowan
e6170a5c4f Fix lease management during image operations
Ensure that leases have a reasonable expiration and are
cleaned up during prune

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-11-25 06:41:29 -08:00
Paweł Gronowski
638172417c container: Add ImagePlatform field and deprecate OS
Change the persistent container metadata to store the whole platform
(as defined by OCI) instead of only the operating system.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-19 13:55:54 +01:00
Sebastiaan van Stijn
5343c7b451 remove internal/compatcontext and use context instead
This internal package was added in f6e44bc0e8
to preserve compatibility with go1.20 and older. At the time, our vendor.mod
still had go1.18 as minimum version requirement (see [1]), which got updated to go1.20
in 16063c7456, and go1.21 in f90b03ee5d

The version of BuildKit we use already started using context.WithoutCancel,
without a fallback, so we no longer can provide compatibility with older
versions of Go, which makes our compatiblity package redundant.

This patch removes the package, and updates our code to use stdlib's context
instead.

[1]: f6e44bc0e8/vendor.mod (L7)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 13:29:39 +02:00
Sebastiaan van Stijn
86f7762d48 vendor: github.com/containerd/containerd v1.7.18
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.

This patch:

- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to prevent using the "log" package, which was also
  migrated to a separate module.

There are still some uses of the old package in (indirect) dependencies,
which should go away over time.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-05 09:21:00 +02:00
Paweł Gronowski
74e2f23e1a daemon/c8d: Use i.images and i.content
Use `image.Store` and `content.Store` stored in the ImageService struct
instead of fetching it every time from containerd client.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:27 +01:00
Sebastiaan van Stijn
03a17a2887 migrate image spec to github.com/moby/docker-image-spec
The specification was migrated to a separate module:
https://github.com/moby/docker-image-spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-09 19:12:18 +01:00
Paul "TBBle" Hampson
0dc07ccc3a Unmount RWLayer during Commit
Needed for Diff on Windows. Don't remount it afterwards as the layer is
going to be released anyway.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:27 +01:00
Paweł Gronowski
8390bc5683 c8d/cache: Use ContainerConfig from content store
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:55 +01:00
Paweł Gronowski
f760cb4f97 c8d/builder: Store ContainerConfig
Serialize ContainerConfig to content store and store its digest in
label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:54 +01:00
Djordje Lukic
71ebfc7c63 c8d: make the cache in classic builder work
In order for the cache in the classic builder to work we need to:
- use the came comparison function as the graph drivers implementation
- save the container config when commiting the image
- use all images to search a 'FROM "scratch"' image
- load all images if `cacheFrom` is empty

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:50 +01:00
Paweł Gronowski
28f48ce1ac c8d/commit-builder: Extract common code
Extract duplicated image creation code to a function.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-11 13:30:40 +01:00
Derek McGowan
f74ca4ed36 Update commit id unmap to directly use active mounts
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-22 10:14:44 -08:00
Djordje Lukic
cf5a3bc531 c8d: Fix image commit with userns mapping
The remapping in the commit code was in the wrong place, we would create
a diff and then remap the snapshot, but the descriptor created in
"CreateDiff" was still pointing to the old snapshot, we now remap the
snapshot before creating a diff. Also make sure we don't lose any
capabilities, they used to be lost after the chown.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-12-20 17:30:57 +01:00
Djordje Lukic
d06af368ad c8d: "unremap" the user namespace on commit
We remap the snapshot when we create a container, we have to to the
inverse when we commit the container into an image

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 13:28:45 +01:00
Sebastiaan van Stijn
cff4f20c44 migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Paweł Gronowski
6f27bef9fc daemon/c8d: Use non cancellable context in defers
Fixes leases not being released when operation was cancelled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-03 15:27:40 +02:00
Paweł Gronowski
f22b112005 c8d/commit: Unpack committed image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:14 +02:00
Paweł Gronowski
081b74b97d c8d/builder: Store parent in c8d image label
Images built by classic builder will have an additional label (in the
containerd image object, not image config) pointing to a parent of that
image.

This allows to differentiate intermediate images (dangling
images created as a result of a each Dockerfile instruction) from the
final images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 09:47:53 +02:00
Paweł Gronowski
0ffa3dd870 daemon/c8d: Use Docker imagespec
This makes the c8d code which creates/reads OCI types not lose
Docker-specific features like ONBUILD or Healthcheck.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 17:15:05 +02:00
Sebastiaan van Stijn
28d201feb7 daemon/containerd: log errors when releasing leases
Log a warning if we encounter an error when releasing leases. While it
may not have direct consequences, failing to release the lease should be
unexpected, so let's make them visible.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 12:54:14 +02:00
Paweł Gronowski
eb56493f4e c8d/commit: Don't produce an empty layer
If the diff is empty and don't produce an empty layer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:18 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
b42e367045 vendor: github.com/opencontainers/image-spec v1.1.0-rc3
full diff: https://github.com/opencontainers/image-spec/compare/3a7f492d3f1b...v1.1.0-rc3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 02:34:50 +02:00
Laura Brehm
bd6868557d c8d/builder: implement cache
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 12:02:09 +01:00
Laura Brehm
e46674b6a7 c8d: implement classic builder
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 11:57:01 +01:00
Laura Brehm
e8be792130 c8d: fix missing image history
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-09 16:19:05 +01:00
Sebastiaan van Stijn
9613da63dc c8d: commit: generateCommitImageConfig: don't merge image config
daemon.CreateImageFromContainer() already constructs a new config by taking
the image config, applying custom options (`docker commit --change ..`) (if
any), and merging those with the containers' configuration, so there is
no need to merge options again.

e22758bfb2/daemon/commit.go (L152-L158)

This patch removes the merge logic from generateCommitImageConfig, and
removes the unused arguments and error-return.

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Co-authored-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 02:53:54 +02:00
Paweł Gronowski
9fe5ac6614 c8d: Handle nil container.ImageManifest
Don't panic when processing containers created under fork containerd
integration (this field was added in the upstream and didn't exist in
fork).

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-26 14:54:01 +02:00
Sebastiaan van Stijn
465cbccdaf c8d: use the container's configured snapshotter where possible
While we currently do not provide an option to specify the snapshotter to use
for individual containers (we may want to add this option in future), currently
it already is possible to configure the snapshotter in the daemon configuration,
which could (likely) cause issues when changing and restarting the daemon.

This patch updates some code-paths that have the container available to use
the snapshotter that's configured for the container (instead of the default
snapshotter configured).

There are still code-paths to be looked into, and a tracking ticket as well as
some TODO's were added for those.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 14:09:21 +02:00
Paweł Gronowski
24804e67f5 c8d/commit: Use manifest digest for dangling image
Previously commit incorrectly used image config digest as an image id
for the new image which isn't consistent with the image target.
This changes it to use manifest digest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-30 14:13:13 +02:00
Laura Brehm
a34060cdb4 Resolve and store manifest when creating container
This addresses the previous issue with the containerd store where, after a container is created, we can't deterministically resolve which image variant was used to run it (since we also don't store what platform the image was fetched for).

This is required for things like `docker commit`, and computing the containers layer size later, since we need to resolve the specific image variant.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-03-06 15:13:36 +01:00
Nicolas De Loof
168ca2dcc8 Introduce support for docker commit
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Co-authored-by: Laura Brehm <laurabrehm@hey.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-06 15:11:36 +01:00
Nicolas De Loof
9b5c21309b images: Pass context to commit related operations
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-18 12:23:32 +01:00
Nicolas De Loof
4dd86a0b33 containerd-integration: prefer error over panic where possible
- prefer error over panic where possible
- ContainerChanges is not implemented by snapshotter-based ImageService

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-07 22:01:49 +02:00
Sebastiaan van Stijn
9c8b0b6050 daemon/containerd: split methods to separate files
This splits the ImageService methods to separate files, to closer
match the existing implementation, and to reduce the amount of code
per file, making it easier to read, and to reduce merge conflicts if
new functionality is added.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:46:31 +02:00