Commit Graph

542 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
4f5ec9994c pkg/reexec: deprecate and migrate to github.com/moby/sys/reexec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 10:12:58 +01:00
Sebastiaan van Stijn
544b1574e2 builder/dockerfile: fix shadowed variables (govet)
builder/dockerfile/builder.go:361:2: shadow: declaration of "dispatchRequest" shadows declaration at line 189 (govet)
        dispatchRequest := newDispatchRequest(b, dockerfile.EscapeToken, nil, NewBuildArgs(b.options.BuildArgs), newStagesBuildResults())
        ^
    builder/dockerfile/copy.go:217:2: shadow: declaration of "copyInfo" shadows declaration at line 39 (govet)
        copyInfo, err := copyInfoForFile(o.source, origPath)
        ^
    builder/dockerfile/dispatchers.go:97:2: shadow: declaration of "copyInstruction" shadows declaration at line 60 (govet)
        copyInstruction, err := copier.createCopyInstruction(c.SourcesAndDest, "ADD")
        ^
    builder/dockerfile/dispatchers.go:124:2: shadow: declaration of "copyInstruction" shadows declaration at line 60 (govet)
        copyInstruction, err := copier.createCopyInstruction(c.SourcesAndDest, "COPY")
        ^
    builder/dockerfile/dispatchers.go:162:3: shadow: declaration of "v" shadows declaration at line 161 (govet)
            v, err := d.getExpandedString(d.shlex, v)
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-06 12:54:52 +01:00
Sebastiaan van Stijn
81a1ca0217 builder/dockerfile: parseChownFlag: fix non-constant format string in call (govet)
builder/dockerfile/internals_linux.go:38:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
            return idtools.Identity{}, errors.Wrapf(err, "can't find uid for user "+userStr)
                                                         ^
    builder/dockerfile/internals_linux.go:42:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
            return idtools.Identity{}, errors.Wrapf(err, "can't find gid for group "+grpStr)
                                                         ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-27 10:23:40 +02:00
Tonis Tiigi
1787c364e0 vendor: update buildkit to v0.15.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-07-08 20:35:36 -07:00
Sebastiaan van Stijn
d0aa3eaccf Migrate to github.com/containerd/platforms module
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.

The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-02 21:20:29 +02:00
Sebastiaan van Stijn
4a89963f1e builder: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 23:18:46 +02:00
Sebastiaan van Stijn
5a4595466b Merge pull request #48008 from thaJeztah/deprecate_runconfig_DefaultDaemonNetworkMode
runconfig: deprecate DefaultDaemonNetworkMode, move to daemon/network
2024-06-18 14:13:07 +02:00
Sebastiaan van Stijn
8e91b64e07 runconfig: deprecate DefaultDaemonNetworkMode, move to daemon/network
This function returns the default network to use for the daemon platform;
moving this to a location separate from runconfig, which is planned to
be dismantled and moved to the API.

While it might be convenient to move this utility inside api/types/container,
we don't want to advertise this function too widely, as the default returned
can ONLY be considered correct when ran on the daemon-side. An alternative
would be to introduce an argument (daemonPlatform), which isn't very convenient
to use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 17:32:56 +02:00
Sebastiaan van Stijn
437e1ae15e runconfig/opts: deprecate ConvertKVStringsToMap and move internal
This utility is only used in two places, and simple enough to duplicate.
There's no external consumers, and a copy of this utility exists in docker/cli
for use on the client side, so we could consider skipping deprecation,
but just to be on the safe side ':)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 16:23:39 +02:00
Sebastiaan van Stijn
0fb6360fa7 builder/dockerfile: createDestInfo: remove platform arg (LCOW left-over)
This was added in 7a7357dae1 as part of the
LCOW implementation. LCOW has been removed, and this option was no longer
in use because of that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 14:03:01 +02:00
Sebastiaan van Stijn
c5d95fdb04 builder/dockerfile: fix some minor linting issues
- explicitly suppress some errors
- use fmt.Fprintln instead of manually appending a newline
- remove an outdated TODO; looking at the suggestion, it's not a
  realistic option

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 14:02:55 +02:00
Sebastiaan van Stijn
6fa6812c51 builder/dockerfile: rename vars that shadowed types and builtins
- imageMount was shadowing the imageMount type
- copy was shadowing the copy builtin
- container was shadowing the container import

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 14:01:46 +02:00
Sebastiaan van Stijn
074932966d builder/dockerfile: remove endsInSlash utility
It was only used in a single location, and other locations were shadowing
the function through local variables. As it's a one-liner, inlining the
code may be just as transparent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 13:51:44 +02:00
Paweł Gronowski
995604236e builder: Adjust usage of shlex.ProcessWord
1b1c5bc08ad81add007eb647e66ed0929693f3a0 extended the function signature
with one additional return value.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:16 +02:00
Paweł Gronowski
438371e1fe builder: Pass nil linter to instructions.Parse
eea0b41bf4fb1d69e109ff5ff8045c63f0c0d510 added a new argument to
`instructions.Parse` to support issuing linter warnings.

Classic builder uses it to parse the Dockerfile instructions and its
usage needs adjustment.

The classic builder is deprecated and we won't be adding any new
features to it, so we just pass a nil linter callback.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:15 +02:00
Sebastiaan van Stijn
cd1ed46d73 don't depend on containerd platform.Parse to return a typed error
We currently depend on the containerd platform-parsing to return typed
errdefs errors; the new containerd platforms module does not return such
errors, and documents that errors returned should not be used as sentinel
errors; c1438e911a/errors.go (L21-L30)

Let's type these errors ourselves, so that we don't depend on the error-types
returned by containerd, and consider that eny platform string that results in
an error is an invalid parameter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 10:18:29 +02:00
Paweł Gronowski
a4d5b6b4d0 builder/normalizeWorkdir: Always return cleaned path
The `normalizeWorkdir` function has two branches, one that returns a
result of `filepath.Join` which always returns a cleaned path, and
another one where the input string is returned unmodified.

To make these two outputs consistent, also clean the path in the second
branch.

This also makes the cleaning of the container workdir explicit in the
`normalizeWorkdir` function instead of relying on the
`SetupWorkingDirectory` to mutate it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-19 12:42:19 +02:00
Albin Kerouanton
4eed3dcdfe api: normalize the default NetworkMode
The NetworkMode "default" is now normalized into the value it
aliases ("bridge" on Linux and "nat" on Windows) by the
ContainerCreate endpoint, the legacy image builder, Swarm's
cluster executor and by the container restore codepath.

builder-next is left untouched as it already uses the normalized
value (ie. bridge).

Going forward, this will make maintenance easier as there's one
less NetworkMode to care about.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-28 12:34:23 +01:00
Cory Snider
5bcd2f6860 builder/dockerfile: ADD with best-effort xattrs
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.

The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-12 13:31:44 -05:00
Sebastiaan van Stijn
8758d08bb4 api: remove handling of HostConfig on POST /containers/{id}/start (api < v1.24)
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Paweł Gronowski
96d461d27e builder/windows: Don't set ArgsEscaped for RUN cache probe
Previously this was done indirectly - the `compare` function didn't
check the `ArgsEscaped`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:07 +01:00
Paweł Gronowski
877ebbe038 image/cache: Check image platform
Make sure the cache candidate platform matches the requested.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:05 +01:00
Sebastiaan van Stijn
1f9098e7d0 builder: remove redundant ExecBackend.ContainerKill()
The `ExecBackend.ContainerKill()` function was called before removing a build-
container.

This function is backed by `daemon.ContainerKill()` which, if no signal is passed,
performed a `daemon.Kill()`, using `SIGKILL` as signal. However, the
`ExecBackend.ContainerRm()` (backed by `daemonContainerRm()`), which is called
after this, is executed with the `ForceRemove` option set, which calls
`daemon.cleanupContainer()` with `ForceRemove` set, which also results in
`daemon.Kill()` being called:
1a0c15abbb/daemon/delete.go (L84-L95)

This makes the `ExecBackend.ContainerKill()` redundant, so removing this from
the interface.

While looking at this code, one (possible) race-condition was found in
`daemon.cleanupContainer()`, where `daemon.Kill()` could return a `errdefs.Conflict`
if the container was already stopped. An extra check was added for this case to
prevent `daemon.cleanupContainer()` from terminating early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:36:41 +01:00
Sebastiaan van Stijn
0aa96c5512 builder/dockerfile: containerManager.RemoveAll() prevent partial cleanup
Prevent cleanup from terminating early when failing to remove a container;

- continue trying to remove remaining containers
- ignore errors due to containers that were not found

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:34:34 +01:00
Sebastiaan van Stijn
03a8188a9a builder/dockerfile: remove containerManager.removeContainer()
This was just a very thin wrapper for backend.ContainerRm(), and the
error it returned was not handled, so moving this code inline.

Moving it inline also allows differentiating the error message to
distinguish the "removing all intermediate containers" from "removing container"
(when cancelling a build).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:32:54 +01:00
Sebastiaan van Stijn
b8f2caa80a pkg/containerfs: deprecate ResolveScopedPath
If was a very shallow wrapper around symlink.FollowSymlinkInScope, so inline
that code instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:32:31 +01:00
Sebastiaan van Stijn
484e6b784c api/types: move ContainerCreateConfig, ContainerRmConfig to api/types/backend
The `ContainerCreateConfig` and `ContainerRmConfig` structs are used for
options to be passed to the backend, and are not used in client code.

Thess struct currently is intended for internal use only (for example, the
`AdjustCPUShares` is an internal implementation details to adjust the container's
config when older API versions are used).

Somewhat ironically, the signature of the Backend has a nicer UX than that
of the client's `ContainerCreate` signature (which expects all options to
be passed as separate arguments), so we may want to update that signature
to be closer to what the backend is using, but that can be left as a future
exercise.

This patch moves the `ContainerCreateConfig` and `ContainerRmConfig` structs
to the backend package to prevent it being imported in the client, and to make
it more clear that this is part of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 16:41:36 +01:00
Frank Villaro-Dixon
123ade3763 builder/dockerfile: errmsg: quote build target
The build target is not quoted and it makes it difficult for some
persons to see what the problem is.

By quoting it we emphasize that the target name is variable.

Signed-off-by: Frank Villaro-Dixon <frank.villarodixon@merkle.com>
2023-11-02 09:38:14 +01:00
Sebastiaan van Stijn
df3a321164 migrate to github.com/moby/sys/user
The github.com/opencontainers/runc/libcontainer/user package was moved
to a separate module. While there's still uses of the old module in
our code-base, runc itself is migrating to the new module, and deprecated
the old package (for runc 1.2).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 15:45:02 +02:00
Sebastiaan van Stijn
cff4f20c44 migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Sebastiaan van Stijn
a3c97beee0 image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
5dc44f03f0 classic builder: align "removing intermediate container" output
This is something that stood out to me: removing the intermediate
container is part of a build step, but unlike the other output from
the build, wasn't indented (and prefixed with `--->`) to be shown
as part of the build.

This patch adds the `--->` prefix, to make it clearer what step the
removal was part of.

While at it, I also updated the message itself: this output is printed
_after_ the intermediate container has been removed, so we may as well
make it match reality, so I changed "removing" to "removed".

Before:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> a416a98b71e2
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in a1a65b9365ac
    Removing intermediate container a1a65b9365ac
     ---> 8c6b57ebebdd
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 9fa977b763a5
    Removing intermediate container 9fa977b763a5
     ---> 795c1f2fc7b9
    Successfully built 795c1f2fc7b9

After:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> fc9db2894f4e
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in 38d7c34c2178
     ---> Removed intermediate container 38d7c34c2178
     ---> 7c0dbc45111d
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 629620285d4c
     ---> Removed intermediate container 629620285d4c
     ---> b92f70f2e57d
    Successfully built b92f70f2e57d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 21:27:56 +02:00
Jose Diaz-Gonzalez
079a9d4562 Sort unconsumed build arguments before usage
Golang map iteration order is not guaranteed, so in some cases the built slice has it's output of order as well. This means that testing for exact warning messages in docker build output would result in random test failures, making it more annoying for end-users to test against this functionality.

Signed-off-by: Jose Diaz-Gonzalez <email@josediazgonzalez.com>
2023-07-09 03:23:06 -04:00
Sebastiaan van Stijn
202907b14c builder/dockerfile: use string-literals for easier grep'ing
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:25:59 +02:00
Sebastiaan van Stijn
3e0565e17c builder/dockerfile: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
ab35df454d remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Akihiro Suda
9e6370819b Merge pull request #45473 from zhangguanzhang/master-commentStr
[builder] make chownComment easy to read
2023-05-12 09:35:10 +09:00
Laura Brehm
e46674b6a7 c8d: implement classic builder
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 11:57:01 +01:00
Sebastiaan van Stijn
3f7fde76c2 Merge pull request #45507 from corhere/cleanup-reexec-inits
[chore] clean up reexec.Init() calls
2023-05-10 11:49:11 +02:00
Cory Snider
4e0319c878 [chore] clean up reexec.Init() calls
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)

Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-09 19:13:17 -04:00
Jeyanthinath Muthuram
307b09e7eb fixing consistent aliases for OCI spec imports
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 15:27:52 +05:30
zhangguanzhang
43b1afb2cf make chownComment easy to read
Signed-off-by: zhangguanzhang <zhangguanzhang@qq.com>
2023-05-05 19:22:49 +08:00
Nicolas De Loof
9b5c21309b images: Pass context to commit related operations
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-18 12:23:32 +01:00
Sebastiaan van Stijn
4f81f9d5ea builder/dockerfile: use example.com for tests
Trying to remove the "docker.io" domain from locations where it's not relevant.
In these cases, this domain was used as a "random" domain for testing or example
purposes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 15:28:09 +01:00
Sebastiaan van Stijn
59880cb855 builder/dockerfile: use strings.Cut()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 11:09:00 +01:00
Sebastiaan van Stijn
c63ea32a17 pkg/ioutils: TempDir: move to pkg/longpath
This utility wasn't very related to all other utilities in pkg/ioutils.
Moving it to longpath to also make it more clear what it does.

It looks like there's only a single (public) external consumer of this
utility, and only used in a test, and it's not 100% clear if it was
intentional to use our package, of if it was a case of "I actually meant
`io/ioutil.MkdirTemp`" so we could consider skipping the alias.

While moving the package, I also renamed `TempDir` to `MkdirTemp`, which
is the signature it matches in "os" from stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-20 23:24:12 +01:00
Sebastiaan van Stijn
9f3e5eead5 pkg/system: deprecate DefaultPathEnv, move to oci
This patch:

- Deprecates pkg/system.DefaultPathEnv
- Moves the implementation inside oci
- Adds TODOs to align the default in the Builder with the one used elsewhere

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 17:02:50 +01:00
Paweł Gronowski
66a0289081 builder: Don't store context in struct
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:46 +01:00
Nicolas De Loof
def549c8f6 imageservice: Add context to various methods
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:40 +01:00