1124 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
7239c72eca remove uses of deprecated go-archive consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-12-19 17:51:03 +01:00
Paweł Gronowski
3df05205f4 modernize: Use range int
Added in Go 1.22

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-12-15 18:56:34 +01:00
Paweł Gronowski
a25907b485 modernize: Prefer strings.SplitSeq instead of Split
Avoids extra allocations. Added in Go 1.24.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-12-15 18:56:33 +01:00
Paweł Gronowski
f66f555ad4 graphdriver/windows: Potential fix for access denied
Use `winio.RunWithPrivileges` to get the privileges.
It's better because it also locks the Go runtime thread so if the Go
scheduler decides to run this code on a different thread, it will still
have the expected privileges.

A naive attempt at fixing an error experienced by Docker Desktop user
when using Windows containers:

```
failed to register layer: re-exec error: exit status 1: output: hcsshim::ProcessUtilityVMImage \\?\C:\ProgramData\Docker\windowsfilter\<hash1>\UtilityVM: Access is denied.
failed to register layer: re-exec error: exit status 1: output: hcsshim::ProcessBaseLayer \\?\C:\ProgramData\Docker\windowsfilter\<hash2>: Access is denied.
failed to register layer: re-exec error: exit status 1: output: hcsshim::ProcessBaseLayer \\?\C:\ProgramData\Docker\windowsfilter\<hash3>: Access is denied.
```

Unfortunately I can't reproduce the issue on a Windows VM, but this
definitely won't hurt.

At least, I verified that the daemon still works on Windows.

Inspired by: https://github.com/containerd/containerd/issues/8206

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-09-01 16:16:28 +02:00
Paweł Gronowski
555e3939c9 daemon: Fix forceful switch to containerd image store
When no explicit driver was specified, the containerd store by default
was also applied to existing graphdriver setups.

Fix this and add a test.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-08-20 11:20:50 +02:00
Derek McGowan
632de98f75 Enable containerd snapshotters by default
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-08-08 12:07:41 -07:00
Derek McGowan
9f5f4f5a42 Add containerd migration to daemon startup
Add layer migration on startup
Use image size threshold rather than image count
Add daemon integration test
Add test for migrating to containerd snapshotters
Add vfs migration
Add tar export for containerd migration
Add containerd migration test with save and load

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-08-08 12:07:23 -07:00
Sebastiaan van Stijn
c98e5cb60b update github links to moby/moby
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-08-01 01:48:55 +02:00
Derek McGowan
f74e5d48b3 Create github.com/moby/moby/v2 module
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-31 10:13:29 -07:00
Sebastiaan van Stijn
5535e81a79 pkg/system: move to daemon/internal
It has no external users, and this package still has too many different
responsibilities, some of which may be available elsewhere, so moving it
internal so that we can decide to dismantle it further.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-07-29 11:48:57 +02:00
Sebastiaan van Stijn
ca1c5ee08f pkg/stringid: move to daemon, and provide copy in client
The stringid package is used in many places; while it's trivial
to implement a similar utility, let's just provide it as a utility
package in the client, removing the daemon-specific logic.

For integration tests, I opted to use the implementation in the
client, as those should not ideally not make assumptions about
the daemon implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-07-25 13:39:32 +02:00
Derek McGowan
50743e3f38 Move internal/containerfs to daemon/internal/containerfs
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-24 12:13:10 -07:00
Derek McGowan
daeaac0d3c Move internal/directory to daemon/internal/directory
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-15 10:45:15 -07:00
Derek McGowan
33d824b838 Move quota to daemon/internal/quota
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-15 10:45:00 -07:00
Sebastiaan van Stijn
0d24798529 daemon/graphdriver: remove error or deprecated graphdriver-plugins
This error was added in 555dac5e14 to produce
an error for the deprecated graphdriver-plugins.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-07-11 01:24:39 +02:00
Sebastiaan van Stijn
7767525ccd daemon/graphdriver: remove deprecated GetDriver()
This was deprecated in 84cabde357, which
was part of v28.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-07-11 01:16:46 +02:00
Sebastiaan van Stijn
1fe4a0a7ad daemon/graphdriver: remove redundant init()
This was added in a63ff8da46, but looks
like the only reason was to just have the var initialized, so let's
do so when we create the var.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-07-11 01:10:47 +02:00
Matthieu MOREL
381d9d0723 fix use-errors-new from revive
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-26 12:07:38 +00:00
Matthieu MOREL
6d737371b8 fix comparison rule from errorlint
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-13 08:26:56 +00:00
Sebastiaan van Stijn
bd20bfdc41 all: remove redundant import-aliases for "go-winio"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-06-05 12:11:30 +02:00
Sebastiaan van Stijn
9876c9fbcf daemon/graphdriver: remove // import comments
These comments were added to enforce using the correct import path for
our packages ("github.com/docker/docker", not "github.com/moby/moby").
However, when working in go module mode (not GOPATH / vendor), they have
no effect, so their impact is limited.

Remove these imports in preparation of migrating our code to become an
actual go module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-05-30 15:59:11 +02:00
Sebastiaan van Stijn
48220008d8 daemon/graphdriver: remove/rename err-returns and remove naked returns
Prevent accidentally shadowing these errors, which are used in defers, and
while at it, also fixed some linting warnings about unhandled errors.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-28 15:06:34 +02:00
Derek McGowan
d0154d3e59 Update to use github.com/moby/go-archive
Update use of idtools to moby/user for archive and other deprecated uses

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-04-08 17:35:05 -07:00
Sebastiaan van Stijn
6422ff2804 deprecate pkg/atomicwriter, migrate to github.com/moby/sys/atomicwriter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-04 23:07:00 +02:00
Sebastiaan van Stijn
42ca9154e9 layer: remove StoreOptions.ExperimentalEnabled
I noticed that the only reason we kept this was so that we could produce
a more targeted error for the deprecated storage-driver plugins, but it's
very unlikely someone used those, and if they did, we already had the
"DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS" added as requirement. Let's
just produce an error if that option is set (and remove that altogether in
a later release, but just that check doesn't add significant complexity).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-03-07 22:39:55 +01:00
Sebastiaan van Stijn
eeb5651de5 daemon/graohdriver/fuse-overlayfs/: fix naked returns and slight refactor
daemon/graphdriver/fuse-overlayfs/fuseoverlayfs.go:527:2: naked return in func `fusermountU` with 25 lines of code (nakedret)
        return
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-03-04 13:56:15 +01:00
Sebastiaan van Stijn
02b4610246 daemon/graohdriver: fix naked returns
daemon/graphdriver/fsdiff.go:140:3: naked return in func `ApplyDiff` with 20 lines of code (nakedret)
            return
            ^
    daemon/graphdriver/fsdiff.go:149:3: naked return in func `ApplyDiff` with 20 lines of code (nakedret)
            return
            ^
    daemon/graphdriver/fsdiff.go:153:2: naked return in func `ApplyDiff` with 20 lines of code (nakedret)
        return
        ^
    daemon/graphdriver/fsdiff.go:164:3: naked return in func `DiffSize` with 15 lines of code (nakedret)
            return
            ^
    daemon/graphdriver/fsdiff.go:169:3: naked return in func `DiffSize` with 15 lines of code (nakedret)
            return
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-03-04 13:56:15 +01:00
Sebastiaan van Stijn
15895d8ead daemon/graphdriver: rename vars that shadowed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-02-28 21:36:09 +01:00
Derek McGowan
0aa8fe0bf9 Update to containerd v2.0.2, buildkit v0.19.0-rc2
Update buildkit version to commit which uses 2.0

Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-15 14:09:30 +01:00
Sebastiaan van Stijn
5b18a7914c deprecate pkg/parsers.ParseKeyValueOpt and move internal
Move the utility to where it's used, and deprecate the implementation
in pkg/parsers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 13:14:09 +01:00
Sebastiaan van Stijn
7864454792 pkg/ioutils: move atomic file-writers to a separate (pkg/atomicwriter) package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-08 10:36:32 +01:00
Sebastiaan van Stijn
1e060d3315 daemon/graphdriver/windows: remove uses of deprecated system.MkdirAll
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-21 14:38:23 +01:00
Sebastiaan van Stijn
4f5ec9994c pkg/reexec: deprecate and migrate to github.com/moby/sys/reexec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 10:12:58 +01:00
François Scala
e7d15d4d58 daemon/graphdriver/zfs: ignore non-existent dataset on removal
Ignore "dataset does not exist" error in Remove function

Signed-off-by: François Scala <github@arcenik.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 14:44:21 +01:00
Sebastiaan van Stijn
222b3a9e5c Merge pull request #48834 from vvoland/gounits-alias
all: Remove redundant `units` alias for `go-units`
2024-11-07 19:37:19 +01:00
Paweł Gronowski
c0045476b8 all: Remove redundant units alias for go-units
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-07 15:55:16 +01:00
Sebastiaan van Stijn
a28e6e2d21 daemon/graphdriver/btrfs: Driver.parseStorageOpt: fix shadowed variables (govet)
daemon/graphdriver/btrfs/btrfs.go:562:3: shadow: declaration of "key" shadows declaration at line 561 (govet)
            key := strings.ToLower(key)
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-06 12:54:52 +01:00
Sebastiaan van Stijn
d058e09200 daemon/graphdriver/overlay2: fix shadowed variable (govet)
daemon/graphdriver/overlay2/overlay.go:430:3: shadow: declaration of "key" shadows declaration at line 429 (govet)
            key := strings.ToLower(key)
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-06 12:54:50 +01:00
Sebastiaan van Stijn
7b0ef10a9a migrate to github.com/moby/sys/userns
Commit 2ce811e632 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 10:52:33 +02:00
Sebastiaan van Stijn
2ce811e632 migrate to github.com/moby/sys/user/userns
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 12:47:34 +02:00
Sebastiaan van Stijn
688301caf4 daemon/graphdriver: remove Capabilities, CapabilityDriver
Capabilities were implemented in aa96c3176b,
as part of work on an external graphdriver-plugin. Given that none of
the builtin graphdrivers use this option, and support for graphdriver-
plugins has been removed in 555dac5e14,
we can remove this functionality.

This patch:

- removes the CapabilityDriver interface, which has no implementations
- removes the Capabilities type
- layer: remove layerStore.useTarSplit. This field was previously set
  through the driver's Capabilities, but always enabled for the builtin
  graphdrivers,

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-08 18:23:21 -05:00
Sebastiaan van Stijn
c4dcaa0d9a Merge pull request #48092 from thaJeztah/fsmagic_internal
daemon/graphdriver: split, internalize packages to separate snapshotters and graphdrivers
2024-07-08 16:09:59 -05:00
Sebastiaan van Stijn
0f3273e71a daemon/graphdriver: move RefCounter to an internal package
The RefCounter is used in both graphdrivers and snapshotters. Move it
to a separate package to help decoupling snapshotters and graphdrivers,
and make it internal, as it's not intended to be used as a generic utility
package (we can still make it public if there would be a need).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-01 23:53:28 +02:00
Sebastiaan van Stijn
3a3bb1cb50 pkg/directory: deprecate, and move to internal
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-01 20:44:31 +02:00
Sebastiaan van Stijn
e55897977c daemon/graphdriver: simplify Checker, remove NewFsChecker, NewDefaultChecker
The Checker interface was introduced in 1ba05cdb6a
as an optimization to allow passing a simplified check for situations that
don't require mountinfo.Mounted to be executed (as that may result in parsing
 `/proc/self/mountinfo`).

The Checker was defined as an interface with a single `IsMounted` method,
possibly with the intent to allow for additional kind of checks to be added.
No new additions were made since its inception 9 Years ago, and if a need would
arrive, could probably be implemented as part of the check.

This patch simplifies the definition to a function, removing the need to
implement a wrapper struct just to satisfy the interface. The `Checker`
type definition is somewhat redundant, but is kept to have a place to
provide GoDoc.

The `NewFsChecker` and `NewDefaultChecker` utilities are removed as part
of this change, favoring a local definition for storage-drivers that
used them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-01 19:15:26 +02:00
Sebastiaan van Stijn
49f6e004f1 daemon/graphdriver: move FsMagic utilities to an internal package
These utilities were used in both graphdrivers and snapshotters. Move them
to a separate package, to help decoupling snapshotters and graphdrivers,
and make it internal, as it's not intended to be used as a generic utility
package (we can still make it public if there would be a need).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-01 19:15:20 +02:00
Sebastiaan van Stijn
f2970e5358 pkg/containerfs: move to internal
The only external consumer are the `graphdriver` and `graphdriver/shim`
packages in github.com/docker/go-plugins-helpers, which depended on
[ContainerFS][1], which was removed in 9ce2b30b81.

graphdriver-plugins were deprecated in 6da604aa6a,
and support for them removed in 555dac5e14,
so removing this should not be an issue.

Ideally this package would've been moved inside `daemon/internal`, but it's used
by the `daemon` (cleanupContainer), `plugin` package, and by `graphdrivers`,
so needs to be in the top-level `internal/` package.

[1]: 6eecb7beb6/graphdriver/api.go (L218)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-30 19:13:23 +02:00
Sebastiaan van Stijn
0d16821922 Merge pull request #48083 from thaJeztah/fix_rootless_pull
daemon/graphdriver/overlay2: set TarOptions.InUserNS for native differ (fix "failed to Lchown "/dev/console")
2024-06-28 19:33:15 +02:00
Sebastiaan van Stijn
6521057bb2 daemon/graphdriver/overlay2: set TarOptions.InUserNS for native differ
Commits b2fd67de77 (and the follow-up commit
f6b80253b8) updated doesSupportNativeDiff to
detect whether the host can support native overlay diffing with userns
enabled.

As a result, [useNaiveDiff] would now return "false" in cases where it
previously would return "true" (and thus skip). However, [overlay2],
unlike [fuse-overlay] did not take user namespaces into account, when
using the native differ, and it therefore did not set the InUserNS option
in TarOptions.

As a result pkg/archive.createTarFile would attempt tocreate [device-nodes]
through [handleTarTypeBlockCharFifo] which would fail, but the resulting
error `EPERM` would be discarded, and `createTarFile` would not return
early, therefor attempting to [os.LChown] the missing file, ultimately
resulting in an error:

    failed to Lchown "/dev/console" for UID 0, GID 0: lchown /dev/console: no such file or directory

This patch fixes the missing option in overlay.

[useNaiveDiff]: 47eebd718f/daemon/graphdriver/overlay2/overlay.go (L248-L256)
[overlay2]: 47eebd718f/daemon/graphdriver/overlay2/overlay.go (L684-L689)
[fuse-overlay]: 47eebd718f/daemon/graphdriver/fuse-overlayfs/fuseoverlayfs.go (L456-L462)
[device-nodes]: ff1e2c0de7/pkg/archive/archive.go (L713-L720)
[handleTarTypeBlockCharFifo]: 47eebd718f/pkg/archive/archive_unix.go (L110-L114)
[os.LChown]: ff1e2c0de7/pkg/archive/archive.go (L762-L773)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 12:38:40 +02:00
Sebastiaan van Stijn
84cabde357 daemon/graphdriver: deprecate GetDriver(), and remove its use
The exported function was only used in tests, and identical in use when
using New with a name provided. Deprecate it, and remove the uses of it
in our (integration-)tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 20:05:44 +02:00