Commit Graph
114 Commits
Author SHA1 Message Date
Maksym Pavlenko d7bebd827a Merge pull request #12889 from 1seal/codex/descurls-strip-sensitive-headers
docker fetcher: strip sensitive headers on descriptor URLs
2026-08-24 22:08:21 +00:00
Oleh Konko 51cf999e92 core/remotes/docker: normalize descriptor URL origins
Signed-off-by: Oleh Konko <security@1seal.org>
2026-08-07 19:00:34 +04:00
Oleh Konko 5b3ce72589 core/remotes/docker: strip sensitive headers on desc.urls fetch
Signed-off-by: Oleh Konko <security@1seal.org>
2026-07-15 22:53:53 +04:00
Paweł Gronowski 80975e2c75 remotes/docker: Propagate registry warnings to resolver
According to the OCI distribution spec registries may include
informational warnings in HTTP Warning headers:

- https://github.com/opencontainers/distribution-spec/blob/e612a6e1e1bc717f9fa7e1feb4f05c8b6568754a/spec.md#warnings
- https://www.rfc-editor.org/rfc/rfc7234#section-5.5

This change implements support for handling these warnings and
propagating them to the resolver.

This patch adds a new, optional WarningHandler interface field to
ResolverOptions that allows callers to receive and process warnings sent
by registries via HTTP Warning headers with warn-code 299.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2026-07-13 12:15:30 +02:00
cshung 5c66703ee3 remotes: surface OCI error body on HEAD 403 via GET fallback
When a registry returns 403 Forbidden on a HEAD request (e.g., manifest
resolve or push existence check), the diagnostic error body is lost
because HEAD responses carry no body per HTTP spec. This leaves users
with an opaque "403 Forbidden" message and no actionable guidance.

Add a follow-up GET on HEAD 403 to retrieve the registry's OCI error
body. The existing unexpectedResponseErr machinery already parses the
body into structured errors — it just needs the body to be present. The
fallback lives in a shared withGETErrorBody helper used by both the
pusher and resolver: it only enriches when the GET also returns 403, and
preserves the original HEAD request's method and status while borrowing
just the body, so the resulting error's status and body stay consistent.

Scoped to 403 only because it is rare (CMK key disabled, IP firewall,
RBAC misconfiguration) and its body is highly diagnostic, while other
status codes either already use GET or have bodies that add no value.

Fixes #8969

Signed-off-by: Andrew Au <cshung@gmail.com>
2026-07-07 21:19:39 +00:00
Derek McGowan 20af2e324a resolver: retry on transient network errors
Allow the last host to retry on transient network errors to incrase the
likelihood of the operation succeeding and help reduce flaky tests.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2026-06-12 16:36:08 -07:00
Mike Brown 5fa03e6bbb Merge pull request #13164 from Mujib-Ahasan/add-ResponseHeaderTimeout
Add: ResponseHeaderTimeout to image pull HTTP transport
2026-04-20 13:20:04 +00:00
Samuel Karp c6cf634443 Merge pull request #12262 from doddi/fix-check-status-code-on-fetch
Add check for status code for GET requests
2026-04-15 23:36:52 +00:00
Derek McGowan 83044a43a1 Merge pull request #13128 from thaJeztah/windows_system_pool
core/remotes/docker: use SystemCertPool on Windows
2026-04-12 16:58:22 +00:00
Derek McGowan 17cdec2981 Merge pull request #12206 from wjordan/push-namespace
Add registry host namespace query parameter to mirror push requests
2026-04-07 22:21:22 +00:00
Mujib Ahasan 574a60700e Add: ResponseHeaderTimeout to image pull HTTP transport
Signed-off-by: Mujib Ahasan <ahasanmujib8@gmail.com>
2026-04-05 02:58:02 +05:30
Sebastiaan van Stijn dc609cf4b4 core/remotes/docker: use SystemCertPool on Windows
This code was added in [containerd@17b6050] (containerd v1.4.0-beta.0), at
which time containerd was using go1.13.8, which didn't support system-pools
([x509.SystemCertPool]) on Windows, so an empty pool was constructed, similar
to [docker/go-connections@55aadc3].

Support for system pools on Windows originally added in Go 1.8 (through
[golang/go@05471e9]), but reverted, and re-implemented in Go 1.18 (through
[golang/go@3544082]).

Go 1.18 and up now implement this, but, unlike Linux, which uses a pure-Go
implementation, certificate validation is handled by the system:

> On macOS and Windows, certificate verification is handled by system APIs,
> but the package aims to apply consistent validation rules across operating
> systems.

On macOS and Windows, x509.SystemCertPool returns an empty Pool, with the
`systemPool` set to `true` (see [loadSystemRoots]). This must be considered
an implementation detail; custom CAs can be appended to this pool, and handled
as usual.

This patch removes the special handling on Windows, allowing it to use the
system-pool, aligned with Linux.

[containerd@17b6050]: https://github.com/containerd/containerd/commit/17b6050d20bf7d25d15e26ed867f774e8c7c72fe
[docker/go-connections@55aadc3]: https://github.com/docker/go-connections/commit/55aadc3cc561684699edcdd0921b9293c3ee6b49
[golang/go@05471e9]: https://github.com/golang/go/commit/05471e9ee64a300bd2dcc4582ee1043c055893bb
[golang/go@3544082]: https://github.com/golang/go/commit/3544082f75fd3d2df7af237ed9aef3ddd499ab9c
[x509.SystemCertPool]: https://pkg.go.dev/crypto/x509#SystemCertPool
[loadSystemRoots]: https://cs.opensource.google/go/go/+/refs/tags/go1.26.1:src/crypto/x509/root_windows.go;l=15-17

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-26 13:00:13 +01:00
Maksym Pavlenko 1abba0c961 Merge pull request #13017 from jzhn/main
fix: avoid content storage pollution by limiting the fallback on ref resolution
2026-03-25 19:08:53 +00:00
Justin Chadwell 4f35b756e2 chore: Add explicit digest requirement to docker pusher
The `push` function below assumes that digest and mediatypes are
populated and set. If they aren't, then the requests made are malformed,
attempting to invoke `HEAD /blobs/` (instead of `HEAD /blobs/<digest>`).
Additionally, if we *were* to move past this point, we'd then populate
an empty digest in the query parameter, and even provide invalid HTTP
mediatypes.

However, the `WithDescriptor` `WriterOpt` specifically notes that "Write
does not require any field of desc to be set". It's very easy for the
caller to read this as an optional field, to skip it, and then get
confusing HTTP errors from inside the `push` function.

We can avoid this by explicitly validating that the descriptor is valid
and provide early feedback.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2026-03-25 10:56:00 -07:00
Sebastiaan van Stijn 6b58f13443 replace some uses of interface{} in (go)docs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-17 13:38:25 +01:00
Sebastiaan van Stijn fc3165188e core: modernize: stringsbuilder
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -stringsbuilder -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 18:24:16 +01:00
Sebastiaan van Stijn 92b0b289e3 core: modernize: any
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -any -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 18:24:14 +01:00
Sebastiaan van Stijn 18c74abd56 *: modernize: slicescontains
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -slicescontains -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 18:24:14 +01:00
Sebastiaan van Stijn 1754af311e *: modernize: stringsseq
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -stringsseq -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 18:24:13 +01:00
Sebastiaan van Stijn 0ecd8f43ed core: modernize: stringscut
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -stringscut -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 15:02:33 +01:00
Sebastiaan van Stijn 3723a67092 core: modernize: rangeint
go install golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest
    modernize -rangeint -fix ./...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-03-15 14:56:35 +01:00
Joseph Zhang 9b7fa61316 fix:avoid content storage pollution by limiting the fallback on ref resolution
Signed-off-by: Joseph Zhang <jzhang@absolute.com>
2026-03-13 10:33:16 -07:00
Sebastiaan van Stijn 642be181dc core/remotes/docker: include "method" and "url" in logs
This code would produce "info" logs on failures, but the logs did only
include the host and HTTP-status, but not which URL was tried, which
limited their functionality when debugging issues;

    INFO[0000] fetch failed after status: 404 Not Found      host=registry-1.docker.io spanID=30cdc3d09e370346 traceID=851a780f994f420f657e18fa6b8d0a77
    INFO[0001] fetch failed after status: 404 Not Found      host=registry-1.docker.io spanID=30cdc3d09e370346 traceID=851a780f994f420f657e18fa6b8d0a77
    INFO[0002] fetch failed after status: 404 Not Found      host=registry-1.docker.io spanID=30cdc3d09e370346 traceID=851a780f994f420f657e18fa6b8d0a77

With this patch, both the "url" and "method" are included. The existing
"host" field is kept for now, but duplicated in the URL (note that the
existing "host" path only includes the hostname, but not custom paths
used);

    INFO[0001] fetch failed after status: 404 Not Found      host=registry-1.docker.io method=HEAD url="https://registry-1.docker.io/v2/dockersamples/labspace-content-dev/manifests/sha256:512d98f1e4e60663a36123f91254db63d0acadf323f81588379fb52ee1da2fa0" spanID=5d7d1f4c83ef8ee4 traceID=584a3a9b5bee402f90b76281a3fc2176
    INFO[0003] fetch failed after status: 404 Not Found      host=registry-1.docker.io method=HEAD url="https://registry-1.docker.io/v2/dockersamples/labspace-content-dev/manifests/sha256:708aa78a398943ba8c5917de90bc8d07afd7de9de174ff77f55d20d15e8987bb" spanID=5d7d1f4c83ef8ee4 traceID=584a3a9b5bee402f90b76281a3fc2176
    INFO[0004] fetch failed after status: 404 Not Found      host=registry-1.docker.io method=HEAD url="https://registry-1.docker.io/v2/dockersamples/labspace-content-dev/manifests/sha256:593c44044ef21eda81ff7150a0466539875c0315a1606538a15e30c593f025ea" spanID=5d7d1f4c83ef8ee4 traceID=584a3a9b5bee402f90b76281a3fc2176

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-02-05 01:10:00 +01:00
Sebastiaan van Stijn 64cc8cdec1 core/remotes/docker: add request.sanitizedURL for logging and errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-02-05 01:09:54 +01:00
Adrien Delorme e86523ecdb multipart fetch stability fixes
Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
2026-01-23 09:45:04 +01:00
Aadhar Agarwal d6ee6f69b2 Fix ST1005 lint violations: lowercase error strings
Error strings should not be capitalized (unless beginning with proper
nouns or acronyms) or end with punctuation, per Go style guide and
staticcheck ST1005.

This change lowercases the first letter of error messages in fmt.Errorf
calls throughout the codebase, while preserving uppercase for acronyms
at the start of error strings (e.g., NRI, RDT, CDI, UID, GID, VHD, FUSE).

Also fixes typo 'faild' -> 'failed' in resolver_test.go.

Changes:
- core/metadata: target.Digest, target.Size, target.MediaType
- core/remotes/docker: fetcher.Seek, fetcherByDigest
- integration: various test error messages
- internal/cri/server: containerConfig
- plugins/snapshots: erofs

Fixes #12011

Signed-off-by: Aadhar Agarwal <aadagarwal@microsoft.com>
2026-01-07 00:26:28 +00:00
markdodgson bf5fe06f8d Use len for stripping http://
Signed-off-by: markdodgson <mdodgson@sonatype.com>
2025-12-22 09:32:04 +00:00
Brian Goff 3c9a0bd316 Revert "Implement io.ReaderAt on docker fetch reader"
This reverts commit 4bf1705a88.

This caused issues for downstream clients who were wrapping the type
contingent on it implementing io.ReaderAt.
Consequently this is causing headaches due to increased round trips with
the remote.

Meanwhile I only added this as a convenience for implementing
content.Provider in the remote which can done even without the original
change, just... less conveniently.

In hindsight, this was just a bad change as it has a rather wide impact
and the actual implementation isn't an optimized ReaderAt.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2025-11-17 15:55:42 -08:00
Derek McGowan 565151652e Add size check to referrers response
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-11-03 12:10:54 -08:00
Will Jordan e95b753058 Add namespace to push requests
If a host is configured that's different to the registry host namespace (e.g. a mirror), then
containerd will append the registry host namespace to requests as a query parameter called `ns`.
Set this query parameter on push requests for proxy mirror support.

Signed-off-by: Will Jordan <will.jordan@gmail.com>
2025-10-30 09:43:24 -07:00
Derek McGowan ec37347d85 Merge pull request #12435 from dmcgowan/referrers-interface
Update referrers interface to support more options
2025-10-29 12:56:29 -07:00
Derek McGowan 42336c7de6 Update referrers interface to support more options
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-10-28 22:37:05 -07:00
Tonis Tiigi 9840ad93e1 docker: fix addQuery with multiple calls
In previous implementation every call would
replace the whole query instead of adding to it.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-10-28 18:25:00 -07:00
Derek McGowan 1684c4f46d Merge pull request #9371 from makkes/lenient-digest-hdr
Make dockerPusher more compliant with distribution spec
2025-10-16 16:43:30 +00:00
Derek McGowan 36c22f45c8 Update referrers fallback logic to always have a fallback
Always attempt the fallback on hosts configured with the referrers
capability. Fallback for any non-404 error.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-10-07 15:38:26 -07:00
Derek McGowan a7537cb8a5 Add referrers to default registries
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-09-24 17:59:19 -07:00
Derek McGowan b668614b55 Add referrers to host config
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-09-24 17:59:19 -07:00
Tonis Tiigi c1b1297c6c Update capabilities for referrers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2025-09-24 17:59:18 -07:00
Derek McGowan 09b4ac136d Add referrers fetcher to remotes
Adds an implementation of the referrers API from
OCI distribution-spec 1.1.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-09-24 17:59:18 -07:00
markdodgson 2e856be039 Check for error status code on response to a get request
Signed-off-by: markdodgson <mdodgson@sonatype.com>
2025-09-23 11:24:51 +01:00
Henry Wang 635907e63a Ensure errContentRangeIgnored error when range-get request is ignored by registry
Signed-off-by: Henry Wang <henwang@amazon.com>
2025-09-16 19:47:28 -07:00
Max Jonas Werner 5243cdd2dc Check expected digest when committing as a sanity check
This commit also adds an info-level log message for the case where the
registry doesn't send a digest header since the computed digest is a
valuable part of our content Commit interface.

Signed-off-by: Max Jonas Werner <max@coppersoft.com>
2025-07-21 09:56:42 +02:00
Max Jonas Werner f2b9ff67f3 Make dockerPusher more compliant with distribution spec
The distribution spec says this about the Docker-Content-Digest
header:

> These headers are OPTIONAL and clients SHOULD NOT depend on them

This commit changes the behaviour of the `dockerPusher` in a way that
makes it more compliant with that requirement in that it allows for it
to be missing from HTTP responses.

Signed-off-by: Max Jonas Werner <max@coppersoft.com>
2025-07-21 09:56:38 +02:00
Derek McGowan 4420b5a49a Ensure fetcher always closes body and properly calls release
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-18 15:30:51 -07:00
jinda.ljd 1b821ca040 fix(dockerFetcher): resolve deadlock issue in dockerFetcher open
Signed-off-by: jinda.ljd <jinda.ljd@alibaba-inc.com>
2025-07-18 10:42:44 +08:00
Derek McGowan 4d89721f23 Merge pull request #11623 from yashsingh74/update-golangci-v2
ci: bump golangci from 6.5.2 to 7.0.0
2025-07-01 18:06:14 +00:00
Derek McGowan 99df502341 Merge pull request #11981 from cpuguy83/remote_readerat
Implement io.ReaderAt on docker fetch reader
2025-06-23 22:40:24 +00:00
Adrien Delorme cb53f381e4 fix when multipart fetching and the server does not return content length
Signed-off-by: Adrien Delorme <azr@users.noreply.github.com>
2025-06-19 17:45:01 +02:00
Derek McGowan 7d394c9e2c Merge pull request #11998 from dmcgowan/avoid-unconditional-range-fetch
Fix fetch always adding range to requests
2025-06-19 22:03:46 +08:00
Derek McGowan d3516916aa Fix fetch always adding range to requests
Add condition for adding range header to avoid unnecessary header when
parallel pull is not being used. Some registries do not properly handle
the range header.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-06-18 22:22:54 -07:00