Compare commits

...

512 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
77446557b0 Merge pull request #49487 from austinvazquez/cherry-pick-838ae09a2337e6561b40d13be6ddf43005a92a9e-to-27.x
[27.x backport] Dockerfile: update runc binary to v1.2.5
2025-02-18 18:08:52 +01:00
Sebastiaan van Stijn
20755a757c Dockerfile: update runc binary to v1.2.5
This is the fifth patch release in the 1.2.z series of runc. It primarily fixes
an issue caused by an upstream systemd bug.

* There was a regression in systemd v230 which made the way we define
  device rule restrictions require a systemctl daemon-reload for our
  transient units. This caused issues for workloads using NVIDIA GPUs.
  Workaround the upstream regression by re-arranging how the unit properties
  are defined.
* Dependency github.com/cyphar/filepath-securejoin is updated to v0.4.1,
  to allow projects that vendor runc to bump it as well.
* CI: fixed criu-dev compilation.
* Dependency golang.org/x/net is updated to 0.33.0.

full diff: https://github.com/opencontainers/runc/compare/v1.2.4...v1.2.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 838ae09a23)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2025-02-18 15:19:10 +00:00
Sebastiaan van Stijn
15c6aaf0e9 Merge pull request #49394 from vvoland/update-go-27.x
[27.x] update to go1.22.12
2025-02-06 09:38:21 +01:00
Paweł Gronowski
a584f0b227 update to go1.22.12
This minor release include 1 security fix following the security policy:

- crypto/elliptic: timing sidechannel for P-256 on ppc64le

  Due to the usage of a variable time instruction in the assembly implementation
  of an internal function, a small number of bits of secret scalars are leaked on
  the ppc64le architecture. Due to the way this function is used, we do not
  believe this leakage is enough to allow recovery of the private key when P-256
  is used in any well known protocols.

This is CVE-2025-22866 and Go issue https://go.dev/issue/71383.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.12

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-04 20:31:16 +01:00
Sebastiaan van Stijn
5df78b0645 Merge pull request #49384 from vvoland/49362-27.x
[27.x backport] github: Clarify release notes description
2025-02-04 17:24:07 +01:00
Paweł Gronowski
b4d7cb0141 Merge pull request #49383 from vvoland/49361-27.x
[27.x backport] gha/validate-pr: Also run when PR has new commits
2025-02-03 16:38:09 +00:00
Paweł Gronowski
ced95906a4 github: Clarify release notes description
Error out if the release notes section is filled for PRs without the
`impact/` label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 79b0e89628)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-03 17:27:13 +01:00
Paweł Gronowski
48d2940834 gha/validate-pr: Also run when PR has new commits
Otherwise, the workflow will won't be rerun even if it was failing
before.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 7ac0e34dba)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-03 16:28:55 +01:00
Sebastiaan van Stijn
4c9b3b011a Merge pull request #49317 from thaJeztah/27.x_backport_bump_dev_tools
[27.x backport] Dockerfile: dev-container: update CLI v27.5.0, buildx v0.20.0, compose v2.32.4
2025-01-22 00:46:50 +01:00
Sebastiaan van Stijn
0da7a2656e Dockerfile: update compose to v2.32.4
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.30.3...v2.32.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 836a5fac10)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 17:42:37 +01:00
Sebastiaan van Stijn
4c8ec29141 Dockerfile: update buildx to v0.20.0
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/v0.18.0..v0.20.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b75a75017d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 17:42:36 +01:00
Sebastiaan van Stijn
fbc854d9d3 Dockerfile: update docker CLI to v27.5.0
Update the Docker CLI used in the dev-container
full diff: https://github.com/docker/cli/compare/v27.3.1...v27.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e2a828ed5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 17:42:34 +01:00
Sebastiaan van Stijn
36c72d4c3d Merge pull request #49322 from thaJeztah/27.x_backport_bake-v6
[27.x backport] ci: update bake-action to v6, ci(bin-image): fix bake build
2025-01-21 17:42:00 +01:00
CrazyMax
e85906c6a8 ci(bin-image): fix bake build
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit d86920b9b3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 16:43:21 +01:00
CrazyMax
542e33c421 ci: update bake-action to v6
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit f8a973ba4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 16:42:59 +01:00
Paweł Gronowski
c0df67813a Merge pull request #49314 from vvoland/49313-27.x
[27.x backport] gha: Adjust release branches
2025-01-21 10:54:03 +01:00
Paweł Gronowski
95d18194a4 gha: Adjust release branches
Adjust all workflows to also run on branches like `27.x`

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5e9ee1a849)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-01-20 18:22:23 +01:00
Sebastiaan van Stijn
13eca88597 Merge pull request #49312 from thaJeztah/27.x_bump_golang_1.22.11
[27.x] update to go1.22.11 (fix CVE-2024-45341, CVE-2024-45336)
2025-01-20 17:34:41 +01:00
Sebastiaan van Stijn
c83862c541 update to go1.22.11 (fix CVE-2024-45341, CVE-2024-45336)
go1.22.11 (released 2025-01-16) includes security fixes to the crypto/x509 and
net/http packages, as well as bug fixes to the runtime. See the Go 1.22.11
milestone on our issue tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.22.11+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.10...go1.22.11

Hello gophers,

We have just released Go versions 1.23.5 and 1.22.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- crypto/x509: usage of IPv6 zone IDs can bypass URI name constraints

  A certificate with a URI which has a IPv6 address with a zone ID may
  incorrectly satisfy a URI name constraint that applies to the certificate
  chain.

  Certificates containing URIs are not permitted in the web PKI, so this
  only affects users of private PKIs which make use of URIs.

  Thanks to Juho Forsén of Mattermost for reporting this issue.

  This is CVE-2024-45341 and Go issue https://go.dev/issue/71156.

- net/http: sensitive headers incorrectly sent after cross-domain redirect

  The HTTP client drops sensitive headers after following a cross-domain redirect.
  For example, a request to a.com/ containing an Authorization header which is
  redirected to b.com/ will not send that header to b.com.

  In the event that the client received a subsequent same-domain redirect, however,
  the sensitive headers would be restored. For example, a chain of redirects from
  a.com/, to b.com/1, and finally to b.com/2 would incorrectly send the Authorization
  header to b.com/2.

  Thanks to Kyle Seely for reporting this issue.

  This is CVE-2024-45336 and Go issue https://go.dev/issue/70530.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 15:59:42 +01:00
Sebastiaan van Stijn
78bccb9709 Merge pull request #49308 from thaJeztah/27.x_backport_gha-arm64
[27.x backport] ci: switch from jenkins to gha for arm64 build and tests (and set correct go version for branch)
2025-01-20 15:59:00 +01:00
Sebastiaan van Stijn
0d4f27db11 Merge pull request #49306 from thaJeztah/27.x_backport_49240_ignore_br_netfilter_error
[27.x backport] Allow users to ignore missing br_netfilter
2025-01-20 12:37:13 +01:00
Sebastiaan van Stijn
e0b88bc9a8 Merge pull request #49307 from thaJeztah/27.x_backport_bridge_startup_error
[27.x backport] Recover from default bridge init failure
2025-01-20 11:53:13 +01:00
Sebastiaan van Stijn
174ce6b376 Merge pull request #49304 from AkihiroSuda/cherrypick-49303-27
[27.x] Dockerfile: update RootlessKit to v2.3.2
2025-01-20 11:46:11 +01:00
Sebastiaan van Stijn
6c832d05c4 gha: set arm64 GO_VERSION to 1.22.10
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:48:57 +01:00
CrazyMax
36706f0e5f ci: switch from jenkins to gha for arm64 build and tests
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 8c236de735)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:45:24 +01:00
Rob Murray
b33299da85 Bridge: on network delete, make sure it's deleted from store
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 22c02219de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:35:36 +01:00
Rob Murray
d57309014d Move integration/network/bridge_test.go to bridge subdir
Also make it Linux-only, as the bridge driver is Linux only and
all of the tests had skips for Windows.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 24f53eba7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:35:26 +01:00
Rob Murray
ad04c7b55a Allow users to ignore missing br_netfilter
Since commit 0f8fc31, the bridge driver will try to load kernel
module br_netfilter if the userland proxy is disabled. If it fails,
we're in unknown territory, so it's treated as an error. At the
very least, containers will not be able to access host ports
mapped to other containers in the same network.

Before that, and before commit 5c499fc delayed the module load
until it was needed - it was loaded unconditionally, but errors
were only logged.

So, on systems where the module is not available, or could not be
loaded/configured, no error was reported and things "worked" (as
long as you didn't try to use something that didn't work).

That behaviour has been useful to some. So, make it possible to
ignore the problem by setting env var:
  DOCKER_IGNORE_BR_NETFILTER_ERROR=1

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit e7bd60ee2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:21:34 +01:00
Akihiro Suda
70a866e37e Dockerfile: update RootlessKit to v2.3.2
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.3.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit aa565ec1ef)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2025-01-20 17:41:26 +09:00
Sebastiaan van Stijn
38b84dce32 Merge pull request #49255 from thaJeztah/27.x_backport_ioutils_more_deprecations
[27.x backport] pkg/ioutils: deprecate NopWriter, NopWriteClose
2025-01-10 15:39:09 +01:00
Sebastiaan van Stijn
ae821132d3 Merge pull request #49253 from thaJeztah/27.x_backport_bump_containerd_binary_1.7.25
[27.x backport] Dockerfile: update containerd to v1.7.25
2025-01-10 14:56:06 +01:00
Sebastiaan van Stijn
8a2fd51c84 pkg/ioutils: deprecate NopWriteCloser
It was only used internally, and has no external consumers; deprecate
it to be removed in the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3faa170371)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-10 14:37:48 +01:00
Sebastiaan van Stijn
cf37b661dd pkg/ioutils: deprecate NopWriter
It's not used, and users can use io.Discard instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fa3c553e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-10 14:36:25 +01:00
Sebastiaan van Stijn
c83f658829 Dockerfile: update containerd to v1.7.25
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.25
full diff: https://github.com/containerd/containerd/compare/v1.7.24...v1.7.25

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c12bfda3cd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-10 13:26:40 +01:00
Paweł Gronowski
931be8ef77 Merge pull request #49246 from thaJeztah/27.x_backport_ioutils_deprecations
[27.x backport] pkg/ioutils: deprecate unused types and functions
2025-01-10 10:13:07 +01:00
Sebastiaan van Stijn
2e1aee3326 Merge pull request #49249 from robmry/backport-27.x/fix_unit_tests_for_nftables_host
[27.x backport] Fix unit tests for an nftables host
2025-01-10 00:13:22 +01:00
Rob Murray
f53d72e0be Fix unit tests for an nftables host
On a freshly rebooted Linux host that's configured to use nftables
with the iptables front-end, "make test-unit" fails "TestUserChains"
on the first run - it's ok in subsequent runs.

The unit tests run in moby's dev container.

The first test in TestUserChain runs with ip6tables disabled, so the
bridge driver doesn't try to load the ip6_tables module. Then, because
the module isn't loaded (it wasn't needed by the daemon running on the
nftables host when it started), the test fails because it can't check
what's in the ip6tables filter chain.

The next test in TestUserChain does the same thing, but with ip6tables
enabled. So the module gets loaded by the bridge driver, and everything
works normally after that.

The dev container used to try to load the module on startup, but that
was removed in commit 2af19b6b ("Don't try to modprobe ip6_tables in
the moby dev container"), as part of a change to give the daemon a way
to load modules itself.

Rather that put back the dev container's code to load ip6_tables on
startup (which would mean the daemon's module-loading code not getting
to run on nftables/firewalld hosts) ...

Run the tests in TestUserChains in a different order, with iptables
enabled in the first test will make it happen to work. At least for
now.

It's not ideal, but we'll be switching to nftables soon, so the issue
will go away.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 8f9c09edd4)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-01-09 20:42:22 +00:00
Akihiro Suda
55f705597c Merge pull request #49243 from thaJeztah/27.x_backport_bump_runc_binary_1.2.4
[27.x backport] Dockerfile: update runc binary to v1.2.4
2025-01-10 05:27:04 +09:00
Akihiro Suda
36ad318a2d Merge pull request #49247 from thaJeztah/27.x_backport_deprecate_runtime_numcpu
[27.x backport] pkg/sysinfo: Deprecate NumCPU
2025-01-10 05:25:51 +09:00
Paweł Gronowski
5b7034f4c6 pkg/sysinfo: Deprecate NumCPU
Deprecate in favor of `runtime.NumCPU` as the behavior is the same now.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3db72b255d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:20:47 +01:00
Sebastiaan van Stijn
9b974d599a daemon: don't repeatedly call NumCPU if not needed
sysinfo.NumCPU returns the number of CPUs which are currently online,
which involves some syscalls and parsing on Windows.

Change the code to only retrieve this information when needed, and
memoize the result to prevent calling this function multiple times.

Ideally, we'd obtain this information from daemon.RawSysInfo(), but
that uses a sync.Once, which could return outdated information.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ddd885a961)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:19:21 +01:00
Sebastiaan van Stijn
41697f61ec pkg/ioutils: deprecate NopFlusher
Apart from being used internally for NewWriteFlusher, it's only used
in a single location outside of this package. Copy the implementation
where it's used, and mark it deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 818a180fce)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:10:51 +01:00
Sebastiaan van Stijn
d9c2cde3be pkg/ioutils: deprecate WriteCounter, NewWriteCounter
it was moved to pkg/ioutils in c30a55f14d,
and only had a single use at the time in [engine/Env.WriteTo]. That use
was removed in 531f4122bd, which removed
the engine package.

[engine/Env.WriteTo]: c30a55f14d/engine/env.go (L260-L264)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 383503d382)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:10:51 +01:00
Sebastiaan van Stijn
a78ae66d0e pkg/ioutils: deprecate BytesPipe, NewBytesPipe, ErrClosed
These types are only used internally in container/streams and have no
external consumers. Deprecate them in preparation of moving them to
a subpackage of container/streams.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 60971a66b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:10:45 +01:00
Sebastiaan van Stijn
95122b4bde Dockerfile: update runc binary to v1.2.4
This is the fourth patch release of the 1.2.z release branch of runc. It
includes a fix for a regression introduced in 1.2.0 related to the
default device list.

- Re-add tun/tap devices to built-in allowed devices lists.

 In runc 1.2.0 we removed these devices from the default allow-list
 (which were added seemingly by accident early in Docker's history) as
 a precaution in order to try to reduce the attack surface of device
 inodes available to most containers. At the time we thought
 that the vast majority of users using tun/tap would already be
 specifying what devices they need (such as by using --device with
 Docker/Podman) as opposed to doing the mknod manually, and thus
 there would've been no user-visible change.

 Unfortunately, it seems that this regressed a noticeable number of
 users (and not all higher-level tools provide easy ways to specify
 devices to allow) and so this change needed to be reverted. Users
 that do not need these devices are recommended to explicitly disable
 them by adding deny rules in their container configuration.

full diff: https://github.com/opencontainers/runc/compare/v1.2.3...v1.2.4
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aad7bcedd2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 13:56:33 +01:00
Sebastiaan van Stijn
d9de1e24ad Merge pull request #49239 from vvoland/49228-27.x
[27.x backport] Clear RWLayer reference under container lock
2025-01-08 21:25:37 +01:00
Tadeusz Dudkiewicz
0ec92ba6a4 Clear RWLayer reference under container lock
Previously the RWLayer reference was cleared without holding the
container lock. This could lead to goroutine panics in various places
that use the container.RWLayer because nil checks introduced in #36242
where not sufficient as the reference could change right before the use.

Fixes #49227

Signed-off-by: Tadeusz Dudkiewicz <tadeusz.dudkiewicz@rtbhouse.com>
(cherry picked from commit 97dc3056c6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-01-08 19:00:12 +01:00
Sebastiaan van Stijn
43fc912ef5 Merge pull request #49194 from rumpl/27.x_backport_c8d-fix-tar-build
[27.x backport]  Use the roundtripper during build
2025-01-03 19:17:33 +01:00
Sebastiaan van Stijn
19d1277ff7 Merge pull request #49209 from thaJeztah/27.x_backport_deprecate_GetTotalUsedFds
[27.x backport] pkg/fileutils: deprecate GetTotalUsedFds
2025-01-03 18:40:43 +01:00
Sebastiaan van Stijn
914696818b Merge pull request #49212 from thaJeztah/27.x_backport_hcsshim_stringmatch
[27.x backport] libnetwork/drivers/windows: fix error-matching for hcsshim "not found"
2025-01-03 18:38:36 +01:00
Sebastiaan van Stijn
a167bc5c9a libnetwork/drivers/windows: fix error-matching for hcsshim "not found"
This code has some gnarly string-matching to detect "not found" errors
returned by hcsshim.

Hcsshim at some point changed this error to lowercase;
6d67a30859

It looks like we ran into that problem in integration tests, which was
fixed in c530c9cbb0 when updating hcsshim,
however, it was only fixed in tests, and hiding the actual issue in our
code.

It looks like hcsshim has some utilities to detect error-types, such as the
IsElementNotFoundError function in hcn, which is the newer API that also wraps
the "HNS" service;
d9a4231b9d/hcn/hcnerrors.go (L75-L77)

But unfortunately, the hns API used by us, does not return typed errors, and
returns HNS errors as a untyped formatted string.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5b31a5b370)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-03 17:38:32 +01:00
Sebastiaan van Stijn
dcedefde22 Merge pull request #49204 from thaJeztah/27.x_backport_non_const_formatting
[27.x backport] fix non-constant format string (caught by go1.24)
2025-01-03 17:36:51 +01:00
Sebastiaan van Stijn
4c7dc9347d Merge pull request #49203 from thaJeztah/27.x_backport_go1.24_prep
[27.x backport] daemon: adjust tests for changes in go1.24 JSON errors
2025-01-03 17:36:21 +01:00
Sebastiaan van Stijn
bdd3a95a13 pkg/fileutils: deprecate GetTotalUsedFds
This function is only used internally and has no external consumers.
Mark it deprecated to be removed in the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e45f20352d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-03 17:23:36 +01:00
Sebastiaan van Stijn
5f81af6eaa libnetwork/drivers/bridge: format errors with '%v' for consistency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1f4c9f23c3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:23:43 +01:00
Sebastiaan van Stijn
0f45f60e9e libnetwork/drivers/windows: fix non-constant format string
Also updated some existing ones to use `%v` instead of `%s` for consistency.

caught by go1.24

    # github.com/docker/docker/libnetwork/drivers/windows/overlay
    # github.com/docker/docker/libnetwork/drivers/windows/overlay/ov_network_windows.go:206:32: non-constant format string in call to github.com/docker/docker/libnetwork/types.ForbiddenErrorf
    FAIL    github.com/docker/docker/libnetwork/drivers/windows/overlay [build failed]

    # github.com/docker/docker/libnetwork/drivers/windows
    # github.com/docker/docker/libnetwork/drivers/windows/windows.go:449:33: non-constant format string in call to github.com/docker/docker/libnetwork/types.ForbiddenErrorf
    FAIL    github.com/docker/docker/libnetwork/drivers/windows [build failed]

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3b8ba71fe3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:23:42 +01:00
Sebastiaan van Stijn
17e8cb2b06 distribution: fix non-constant format string
caught by go1.24

    # github.com/docker/docker/distribution
    # github.com/docker/docker/distribution/pull_v2_windows.go:145:35: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Debugf
    FAIL    github.com/docker/docker/distribution [build failed]

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c96e45375)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:23:42 +01:00
Sebastiaan van Stijn
60f9e333b6 daemon: adjust tests for changes in go1.24 JSON errors
These tests failed because the error message changed in go1.24 through
https://go.dev/cl/606956.

    === Failed
    === FAIL: daemon/config TestDaemonConfigurationFeatures/invalid_config_value (0.00s)
        config_linux_test.go:182: assertion failed: expected error "json: cannot unmarshal string into Go struct field Config.features of type bool", got "json: cannot unmarshal string into Go struct field Config.CommonConfig.features of type bool"

    === FAIL: daemon/config TestDaemonConfigurationFeatures (0.00s)

    === FAIL: daemon/config TestDaemonConfigurationHostGatewayIP/config_not_array (0.00s)
        config_linux_test.go:354: assertion failed: expected error "json: cannot unmarshal string into Go struct field Config.host-gateway-ips of type []netip.Addr", got "json: cannot unmarshal string into Go struct field Config.CommonConfig.DNSConfig.host-gateway-ips of type []netip.Addr"

Relax the tests a bit to accept errors produced by either go1.24 or older.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1c37a4454b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:21:23 +01:00
Sebastiaan van Stijn
9cf331393c Merge pull request #49200 from robmry/backport-27.x/49179_restore_windows_network_labels
[27.x backport] Restore labels when re-creating Windows networks
2025-01-02 23:14:50 +01:00
Rob Murray
95b2f2ed49 Restore labels when re-creating Windows networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 25009ed5c0)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-01-02 19:14:24 +00:00
Djordje Lukic
623af65208 Use the roundtripper during build
The roundtripper is responsible for giving back the build context when
it comes from a tar directly. So we add it to the source manager of the
containerd worker.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2025-01-02 10:10:55 +01:00
Sebastiaan van Stijn
c23af29109 Merge pull request #48949 from thaJeztah/27.x_backport_buildkit-v0.18.0-rc1
[27.x backport] vendor: update buildkit to v0.18.2
2024-12-20 17:48:41 +01:00
Sebastiaan van Stijn
b979cf91aa Merge pull request #49145 from thaJeztah/27.x_backport_swagger_updates
[27.x backport] swagger / API-docs updates
2024-12-19 20:45:31 +01:00
Sebastiaan van Stijn
771d1dc44e docs/api: allow for an empty string for Isolation (api v1.25-v1.47)
Backport the changes from 26049febb2
to all versions used in the documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 364e4790e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 17:30:52 +01:00
Adam Lamers
68b39f825d docs: Update example section for SwarmJoinRequest
- Updated the example section in `swagger.yaml` for the `DataPathAddr` parameter in `SwarmJoinRequest`.
- Updated corresponding example sections in `docs/api/v1.*`

Signed-off-by: Adam Lamers <adam.lamers@wmsdev.pl>
(cherry picked from commit 1aecca8bbd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 17:30:41 +01:00
Daniel Guns
4767dbd713 api: Allow for an empty string for Isolation in Swagger specs This adds an empty string as a valid option for the Isolation field when inspecting a container. On non windows systems, this is always empty, so no error should be returned. Fixes #47452
Signed-off-by: Daniel Guns <danbguns@gmail.com>
(cherry picked from commit 26049febb2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 17:30:28 +01:00
Tonis Tiigi
4359c8b08e vendor: update buildkit to v0.18.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit cd81985bfa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:12 +01:00
Tonis Tiigi
bbd14fc508 vendor: update buildkit to v0.18.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 504ee465fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Tonis Tiigi
fd86c39cc3 vendor: update buildkit to v0.18.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit b4769e86a4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Tonis Tiigi
1fd14d845f vendor: update buildkit to v0.18.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 90250ce93a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Sebastiaan van Stijn
68b0c87e05 vendor: github.com/moby/buildkit 94d0f6ed85e5 (master / v0.18.0-rc.2)
full diff: https://github.com/moby/buildkit/compare/v0.18.0-rc1...94d0f6ed85e504e1851495fa98485ffbd9cfddb0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0666d4a585)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Tonis Tiigi
74600d8425 vendor: update buildkit to v0.18.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 48b237f7af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:08 +01:00
Sebastiaan van Stijn
5b334e0e93 Merge pull request #49136 from thaJeztah/27.x_backport_bump_containerd_hcsshim
[27.x backport] vendor: github.com/containerd/containerd v1.7.23, hcsshim v0.12.8
2024-12-19 16:25:42 +01:00
Sebastiaan van Stijn
09b6837cb8 Merge pull request #49137 from thaJeztah/27.x_backport_builder-gcpolicy-empty-keepbytes
[27.x backport] builder: fall back to defaultKeepStorage if keepStorage is unset for GC policy
2024-12-19 16:25:26 +01:00
David Karlsson
4830f6db66 builder: fall back to defaultKeepStorage if keepStorage is unset for GC policy
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
(cherry picked from commit b08ff81204)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 14:38:04 +01:00
Sebastiaan van Stijn
0e43d1f642 vendor: github.com/containerd/containerd v1.7.23, hcsshim v0.12.8
full diff:

- https://github.com/containerd/containerd/compare/v1.7.22...v1.7.23
- https://github.com/microsoft/hcsshim/compare/v0.12.7...v0.12.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5f39567e56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:22 +01:00
Sebastiaan van Stijn
800d735171 vendor: github.com/Microsoft/hcsshim v0.12.7
- full diff: https://github.com/Microsoft/hcsshim/compare/v0.12.5...v0.12.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 93255bd748)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:22 +01:00
Sebastiaan van Stijn
a57c1848b6 vendor: sigs.k8s.io/yaml v1.4.0
full diff: https://github.com/kubernetes-sigs/yaml/compare/v1.3.0...v1.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3dc042b1df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:21 +01:00
Sebastiaan van Stijn
26eeb6e414 vendor: github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161
full diff: d185dfc1b5...306776ec81

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c98c6d4f08)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:21 +01:00
Akihiro Suda
f6f7c9f231 vendor.mod: github.com/microsoft/hcsshim v0.12.5
https://github.com/microsoft/hcsshim/compare/v0.11.7...v0.12.5

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit f49fad75eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:18 +01:00
Sebastiaan van Stijn
29f12c5f77 Merge pull request #49135 from thaJeztah/27.x_backport_migrate_reexec
[27.x backport] pkg/reexec: deprecate and migrate to github.com/moby/sys/reexec
2024-12-19 13:54:08 +01:00
Sebastiaan van Stijn
053aadc451 Merge pull request #49134 from thaJeztah/27.x_backport_bump_otel
[27.x backport] vendor: go.opentelemetry.io/otel v1.28.0, go.opentelemetry.io/contrib v0.53.0
2024-12-19 13:51:12 +01:00
Sebastiaan van Stijn
bdecc9d58e pkg/reexec: deprecate and migrate to github.com/moby/sys/reexec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f5ec9994c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:45 +01:00
Sebastiaan van Stijn
46a8e5f8c5 pkg/reexec: Command: separate public API from implementation
Move the exported `Command` to a platform-agnostic file, and un-export
the platform-specific implementations. This allows us to maintain the
GoDoc in a single place, describing platform-specific differences where
needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8fd177d79b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:39 +01:00
Sebastiaan van Stijn
d175e4f1f1 pkg/reexec: make platform-agnostic (again)
The reexec package originally was platform-agnostic, but gained some
Linux-specific handling in 1cb17f03d0.

When Windows support was implemented in Docker, the pkg/reexec package
was adjusted accordingly in 64715c4f33,
which now made the package with with either Linux or Windows, with various
other platforms (freebsd, solaris, darwin) being added back in separate
changes.

Based on the history above, this package should be platform-agnostic, except
for Linux-specific changes introduced in 1cb17f03d0
and 5aee8807a6.

This patch:

- removes the stub-implementation to make it functional on other platforms.
- renames the files for consistency

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6568c06d12)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:39 +01:00
Sebastiaan van Stijn
ed5c0a79e7 pkg/reexec: use const for name of test binary
Also use a slightly different name, because "reexec" is used so
widely as term in this package, making it somewhat confusing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7672d60033)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:39 +01:00
Paweł Gronowski
4bf113f9bd all: Remove redundant units alias for go-units
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit c0045476b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:34 +01:00
Albin Kerouanton
dde37eb363 Merge pull request #48938 from thaJeztah/27.x_backport_update_netlink
[27.x backport] vendor: vendor: github.com/vishvananda/netlink v1.3.x (084abd93d) and fix compatibility
2024-12-19 12:36:47 +01:00
Sebastiaan van Stijn
7a512a9531 vendor: go.opentelemetry.io/contrib/instrumentation/xxx v0.53.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2a3acc104)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:02:52 +01:00
Sebastiaan van Stijn
cf4e196e77 vendor: go.opentelemetry.io/otel v1.28.0
aligning all related packages to v1.28.0 as well

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d9a2ca7b49)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:02:51 +01:00
Sebastiaan van Stijn
2a3cc83f36 vendor: google.golang.org/genproto/googleapis/api f6361c86f094
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 070d6c75a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:02:51 +01:00
Rob Murray
4141379d07 Discard ErrDumpInterrupted and return data after maxRetries
Returning possibly inconsistent data avoids retrying indefinitely, and
matches the behaviour of vishvananda/netlink prior to version 1.2.1, in
which the NLM_F_DUMP_INTR flag was ignored.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a0a0bbae6c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
4c50153e2e Fix comments
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 2f02f456f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
c5e72db8e8 vendor: github.com/vishvananda/netlink v1.3.1-0.20240922070040-084abd93d350
Particularly for:
- Preserve results when NLM_F_DUMP_INTR is set (1018).
- Fix SetSendTimeout/SetReceiveTimeout (1012).

full diff: https://github.com/vishvananda/netlink/compare/v1.3.0...084abd93d350

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit fd0d008504)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
14b30aeb61 Add linting rules to spot use of un-wrapped netlink functions.
Spot netlink functions that may return EINTR because
network configuration changed during a state dump, and
point at the nlutil wrappers.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit edaa0eb56d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
87fc62a648 Fix merge problem
Commit f9c0103 (WSL2 mirrored-mode loopback) uses netlink funcs
that were removed/wrapped by commit 00bf437.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 333cfa6402)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:36 +01:00
Rob Murray
3ca9a3be70 Add nlutil functions to retry on netlink EINTR
A recent change to the vishvananda/netlink package exposes
NLM_F_DUMP_INTR in some netlink responses as an EINTR (with
no data).

Retry the requests when that happens, up to five times, before
returning the error. The limit of five is arbitrary, on most
systems a single retry will be rare but, there's no guarantee
that a retry will succeed. So, on a very busy or misbehaving
system the error may still be returned. In most cases, this
will lead to failure of the operation being attempted (which
may lead to daemon startup failure, network initialisation
failure etc).

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 00bf437d84)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:33 +01:00
Sebastiaan van Stijn
4729e446ce vendor: github.com/vishvananda/netlink v1.3.0
full diff: https://github.com/vishvananda/netlink/compare/v1.2.1...v1.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2c498c6a74)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:26:35 +01:00
Sebastiaan van Stijn
cbe4488a6b vendor: github.com/vishvananda/netlink v1.2.1
Local changes related to:

- c96b03b4be
- dedb7ec04b

full diff: https://github.com/vishvananda/netlink/compare/v1.2.1-beta.2...v1.2.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6d0a5e31d7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:26:32 +01:00
Sebastiaan van Stijn
eaec884bcb Merge pull request #48939 from thaJeztah/27.x_backport_bump_prometheus
[27.x backport] vendor: github.com/prometheus/client_golang v1.20.5
2024-12-19 11:25:50 +01:00
Sebastiaan van Stijn
caa122e0fc Merge pull request #48943 from thaJeztah/27.x_backport_bump_buildkit_deps
[27.x backport] vendor: github.com/go-logr/logr v1.4.2, github.com/cenkalti/backoff/v4 v4.3.0
2024-12-19 11:25:03 +01:00
Sebastiaan van Stijn
c710b88579 Merge pull request #49119 from thaJeztah/27.x_backport_libnetwork_deprecate_Passthrough
[27.x backport] libnetwork/iptables: deprecate type IPV, Passthrough
2024-12-17 13:50:31 +01:00
Sebastiaan van Stijn
eda0a20deb libnetwork/iptables: deprecate Passthrough
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d688389f4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-17 12:44:40 +01:00
Albin Kerouanton
b51622dd0d libnet/iptables: deprecate type IPV
The iptables package has two different types to specify the IP version:
IPVersion, used by iptables code, and IPV, used by firewalld code.
Both are representing the ip version as a string.

For iptables, the case doesn't matter because the string is never used
as-is. However, for firewalld the case matters.

Make the IPV type an alias of IPVersion, and deprecate it. Also change
the case used in IPVersion strings to make IPV consts aliases of
IPVersion consts.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 27deff4da1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-17 12:44:34 +01:00
Sebastiaan van Stijn
829ac832da Merge pull request #49104 from thaJeztah/27.x_backport_update_swagger_headers
[27.x backport] docs/api: document correct case for Api-Version header
2024-12-16 18:57:46 +01:00
Albin Kerouanton
bd7da11e95 Merge pull request #49110 from thaJeztah/27.x_backport_fix_setupIPChains_defer
[27.x backport] libnetwork/drivers/bridge: setupIPChains: fix defer checking wrong err
2024-12-16 18:49:52 +01:00
Sebastiaan van Stijn
135b1449cd Merge pull request #49105 from thaJeztah/27.x_backport_testing-suse-apparmor
[27.x backport] integration-cli: don't skip AppArmor tests on SLES
2024-12-16 18:17:10 +01:00
Sebastiaan van Stijn
08de719cf2 libnetwork/drivers/bridge: setupIPChains: fix defer checking wrong err
The output variable was renamed in 0503cf2510,
but that commit failed to change this defer, which was now checking the
wrong error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 01a55860c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 16:49:35 +01:00
Sebastiaan van Stijn
2a6231993b Merge pull request #49100 from thaJeztah/27.x_backport_deprecate_pkg_system
[27.x backport] pkg/system: deprecate types and functions that are only used internally
2024-12-16 16:05:28 +01:00
Aleksa Sarai
6855ca1e62 integration-cli: don't skip AppArmor tests on SLES
This partially reverts e440831802 ("fix and skip some tests based on
API version"), which caused the integration-cli tests to skip all
AppArmor-related tests on SUSE.

It's not really clear why this was done originally, but I have verified
that on modern SLE 12 and SLE 15 systems the AppArmor tests pass without
any adjustments needed.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
(cherry picked from commit 1a453abfb1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 15:14:57 +01:00
Sebastiaan van Stijn
224b30556d docs/api: document correct case for Api-Version header
This header is sent in its canonical format; update the docs to
reflect this.

Follow-up to 76a5ca1d4d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4e9df46a6e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 14:14:11 +01:00
maggie44
5b92b3e7e0 Accurately reflect the canonical casing of API-Version and OS-Type headers
Go automatically canonicalises HTTP headers, meaning the string `API-Version` passed as a header has always been returned as `Api-Version`. Similarly, `OSType` is returned as `Ostype`.

This commit updates the documentation to reflect this behaviour and modifies the codebase to ensure that input strings are aligned with their canonical output values.

Signed-off-by: maggie44 <64841595+maggie44@users.noreply.github.com>
(cherry picked from commit 76a5ca1d4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 14:14:11 +01:00
Sebastiaan van Stijn
1df9e896f2 pkg/system: deprecate types and functions that are only used internally
These types and functions are only used internally (through pkg/archive).
Deprecate them, and mark them for removal.

This deprecates the `Lstat()`, `Mkdev()`, `Mknod()`, `FromStatT()`
and `Stat()` functions, and related `StatT` type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f0ce367e1e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 11:41:55 +01:00
Sebastiaan van Stijn
24f2f4fe06 Merge pull request #49090 from thaJeztah/27.x_backport_no_netfilter_warnings
[27.x backport] daemon: info: remove bridge-nf-call-iptables / ip6tables warnings
2024-12-13 12:09:32 +01:00
Sebastiaan van Stijn
8a8ab0d567 Merge pull request #49043 from robmry/backport-27.x/modprobeless
[27.x backport] Try to load kernel modules, without modprobe
2024-12-13 12:03:26 +01:00
Sebastiaan van Stijn
d1201f6dd8 Merge pull request #49085 from thaJeztah/27.x_backport_bump_runc_binary_1.2.3
[27.x backport] Dockerfile: update runc binary to v1.2.3
2024-12-13 11:50:42 +01:00
Sebastiaan van Stijn
9e272da0f2 Merge pull request #49080 from thaJeztah/27.x_backport_bump_xx
[27.x backport] update xx to v1.6.1 for compatibility with alpine 3.21
2024-12-13 11:50:07 +01:00
Sebastiaan van Stijn
f2209d1aae Merge pull request #49084 from thaJeztah/27.x_backport_bump_md2man
[27.x backport] man: vendor: github.com/cpuguy83/go-md2man v2.0.5
2024-12-13 11:49:41 +01:00
Sebastiaan van Stijn
3a1e495a0f daemon: info: remove bridge-nf-call-iptables / ip6tables warnings
Historically, the `bridge` network-driver would detect whether netfiltering
was enabled in the kernel or, if disabled, try to do a `modprobe` when
initializing the driver. This approach became problematic, as loading the
module was not always performed  at startup depending on daemon configuration,
or the daemon may have failed to load the module. The `/info` response
would include a warning  to inform the user that some functionality may not
be available;

    WARNING: bridge-nf-call-iptables is disabled
    WARNING: bridge-nf-call-ip6tables is disabled

Starting with db25b0dcd0, detecting whether
netfiltering  is enabled now [happens when needed][1], which was further improved
on to not depend  on `modprobe` in 264c15bfc4 and
4740820716.

Because of the above, the `/info` output would now return warnings in any
situation where netfiltering was not enabled on the host before the daemon
started, which may be either _incorrect_ (i.e., the module may have been
loaded afterwards), or irrelevant, because netfiltering is not needed in
all situations.

This patch removes the warnings from the `/info` response,

[1]: 944e403502/libnetwork/drivers/bridge/setup_bridgenetfiltering.go (L16-L77)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5c358743af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 10:16:02 +01:00
Sebastiaan van Stijn
4082abad8b Dockerfile: update runc binary to v1.2.3
This is the third patch release of the 1.2.z release branch of runc. It
primarily fixes some minor regressions introduced in 1.2.0.

- Fixed a regression in use of securejoin.MkdirAll, where multiple
  runc processes racing to create the same mountpoint in a shared rootfs
  would result in spurious EEXIST errors. In particular, this regression
  caused issues with BuildKit.
- Fixed a regression in eBPF support for pre-5.6 kernels after upgrading
  Cilium's eBPF library version to 0.16 in runc.

full diff: https://github.com/opencontainers/runc/compare/v1.2.2...v1.2.3
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec5c9e06e3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 01:33:45 +01:00
Sebastiaan van Stijn
a379aae924 man: vendor: github.com/cpuguy83/go-md2man v2.0.5
full diff: https://github.com/cpuguy83/go-md2man/compare/v2.0.4...v2.0.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a7da6fb2a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 01:30:06 +01:00
Sebastiaan van Stijn
c3968c6ab7 update xx to v1.6.1 for compatibility with alpine 3.21
This fixes compatibility with alpine 3.21

- Fix additional possible `xx-cc`/`xx-cargo` compatibility issue with Alpine 3.21
- Support for Alpine 3.21
- Fix `xx-verify` with `file` 5.46+
- Fix possible error taking lock in `xx-apk` in latest Alpine without `coreutils`

full diff: https://github.com/tonistiigi/xx/compare/v1.5.0...v1.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 89899b71a0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 00:40:41 +01:00
Sebastiaan van Stijn
dfce8756b7 Merge pull request #49079 from vvoland/49078-27.x
[27.x backport] otel: Avoid excessive memory allocations if not configured
2024-12-12 21:57:07 +01:00
Paweł Gronowski
2e6ca227cf otel: Use non-noop tracer provider for grpc
Needed for Buildkit history

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d8358ebc87)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 20:11:32 +01:00
Paweł Gronowski
a5f86f9279 integration/build_traces: Create own tracer provider
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 899360b649)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 20:11:31 +01:00
Paweł Gronowski
3a05c5dcad otel: Avoid excessive memory allocations if not configured
Use noop tracer provider if the OTEL exporter is not configured.
This makes the OTEL tracing avoid doing unneeded memory allocations for
spans which aren't going to be exported anywhere anyway.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4847557d1b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 20:11:28 +01:00
Rob Murray
adb8773adb Jenkins: don't modprobe kernel module
These modprobes were added as a workaround in commit cce5dfe1, but
dockerd should now be able to load the modules it needs.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 15ba03c8fc)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:42 +00:00
Rob Murray
ff176d42d5 Don't try to modprobe ip6_tables in the moby dev container
dockerd will now do this itself, if ip6tables is enabled.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 2af19b6b7c)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:42 +00:00
Rob Murray
86126b9af1 Try to load kernel module ip6_tables, when necessary.
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit f2e1f52a04)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:40 +00:00
Rob Murray
58318d26fc Use ioctl to try to trigger kernel module loads
An ioctl() call to get the "interface index" for a kernel module triggers
the kernel to try to load the module, if the process is running with
CAP_SYS_MODULE. This tends to be more reliable than "modprobe" for
docker-in-docker.

If the ioctl() method fails, fall back to trying "modprobe".

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 4740820716)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:08 +00:00
Sebastiaan van Stijn
92a83937d0 Merge pull request #49027 from thaJeztah/27.x_backport_cdi-rootless
[27.x backport] Dockerd rootless: make {/etc,/var/run}/cdi available
2024-12-04 15:17:19 +01:00
Sebastiaan van Stijn
9163aa379a Merge pull request #49026 from thaJeztah/27.x_update_go_1.22.10
[27.x] update to go1.22.10
2024-12-04 13:15:15 +01:00
Rafael Fernández López
4775621ab6 Dockerd rootless: make {/etc,/var/run}/cdi available
When dockerd is executed with the `dockerd-rootless.sh` script, make
/etc/cdi and /var/run/cdi available to the daemon if they exist.

This makes it possible to enable the CDI integration in rootless mode.

Fixes: #47676

Signed-off-by: Rafael Fernández López <ereslibre@ereslibre.es>
(cherry picked from commit 4e30acb63f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-04 12:04:13 +01:00
Sebastiaan van Stijn
0176f4a5c3 Merge pull request #49024 from thaJeztah/27.x_vendor_buildkit_0.17.3
[27.x] vendor: github.com/moby/buildkit v0.17.3
2024-12-04 12:00:06 +01:00
Sebastiaan van Stijn
0e34b3956b update to go1.22.10
go1.22.10 (released 2024-12-03) includes fixes to the runtime and the syscall
package. See the Go 1.22.10 milestone on our issue tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.22.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.9...go1.22.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-04 10:16:39 +01:00
Sebastiaan van Stijn
7919b806e7 [27.x] vendor: github.com/moby/buildkit v0.17.3
Fix issue where builds from older versions of clients/frontends could result
in missing "no-cache" behavior or original Dockerfile commands could be missing
in progress output

full diff: 80e01a9dc7...v0.17.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-04 09:42:20 +01:00
Sebastiaan van Stijn
a92d4c5a57 Merge pull request #49013 from vvoland/49006-27.x
[27.x backport] registry: deprecate RepositoryInfo.Class
2024-12-02 13:32:41 +01:00
Sebastiaan van Stijn
1cc127466d Merge pull request #49010 from vvoland/49009-27.x
[27.x backport] c8d/tag: Don't log a warning if the source image is not dangling
2024-12-02 13:31:19 +01:00
Sebastiaan van Stijn
525b929947 registry: deprecate RepositoryInfo.Class
The Class field was added in a12b466183 because
Docker Hub registry required a special scope to be set for pulling plugins;

    HTTP/1.1 401 Unauthorized
    ...
    Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository(plugin):vieux/sshfs:pull",error="insufficient_scope"

This is no longer a requirement, and the field is no longer set.

This patch deprecates the field and removes its use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5f91c769f5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-02 11:53:28 +01:00
Paweł Gronowski
d6d43b2912 c8d/tag: Don't log a warning if the source image is not dangling
After the image is tagged, the engine attempts to delete a dangling
image of the source image, so the image is no longer dangling.

When the source image is not dangling, the removal errors out (as
expected), but a warning is logged to the daemon log:

```
time="2024-12-02T10:44:25.386957553Z" level=warning msg="unexpected error when deleting dangling image" error="NotFound: image \"moby-dangling@sha256:54d8c2251c811295690b53af7767ecaf246f1186c36e4f2b2a63e0bfa42df045\": not found" imageID="sha256:54d8c2251c811295690b53af7767ecaf246f1186c36e4f2b2a63e0bfa42df045" spanID=bd10a21a07830d72 tag="docker.io/library/test:latest" traceID=4cf61671c2dc6da3dc7a09c0c6ac4e16
```

Remove that log as it causes unnecessary confusion, as the failure is
expected.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a93f6c61db)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-02 11:49:18 +01:00
Austin Vazquez
0f86fa4502 Merge pull request #48995 from thaJeztah/27.x_backport_modprobe_br_netfilter
[27.x backport] Jenkinsfile: modprobe br_netfilter
2024-11-29 19:44:04 -08:00
Sebastiaan van Stijn
e3564b52af Jenkinsfile: modprobe br_netfilter
Make sure the module is loaded, as we're not able to load it from within
the dev-container;

    time="2024-11-29T20:40:42Z" level=error msg="Running modprobe br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/5.15.0-1072-aws\n" error="exit status 1"

Also moving these steps _before_ the "print info" step, so that docker info
doesn't show warnings that bridge-nf-call-iptables and bridge-nf-call-ip6tables
are not loaded.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cce5dfe1e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-29 23:27:00 +01:00
Sebastiaan van Stijn
7f11005a3d Merge pull request #48968 from thaJeztah/27.x_backport_43080-zfs-destroy-missing-volume-fails
[27.x backport] daemon/graphdriver/zfs: ignore non-existent dataset on removal
2024-11-27 22:52:51 +01:00
Sebastiaan van Stijn
7f7efed8c9 Merge pull request #48966 from thaJeztah/27.x_backport_br_net-fix
[27.x backport] Fix br_netfilter module loading logic
2024-11-27 22:52:26 +01:00
Sebastiaan van Stijn
b9fd5ee243 Merge pull request #48969 from thaJeztah/27.x_backport_integration_add_wait
[27.x backport] integration: add wait
2024-11-27 22:24:42 +01:00
Akihiro Suda
46f6b9a259 integration: add wait
Cherry-picked several WIP commits from
b0a592798f/

Originally-authored-by: Rodrigo Campos <rodrigoca@microsoft.com>
Co-Authored-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fb6e650ab9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 16:28:51 +01:00
François Scala
18aa104041 daemon/graphdriver/zfs: ignore non-existent dataset on removal
Ignore "dataset does not exist" error in Remove function

Signed-off-by: François Scala <github@arcenik.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e7d15d4d58)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 16:22:13 +01:00
Shreenidhi Shedi
052f7d6461 Fix br_netfilter module loading logic
Checking for `/proc/sys/net/bridge` directory alone is not enough to
decide if bridge, br_netfilter module to be loaded.
Check for specific file for br_netfilter module and then
do modprobe if the file is not found under `/proc/sys/net/bridge`

Loading br_netfilter implicitly loads bridge module.

fixes: #48948

Signed-off-by: Shreenidhi Shedi <shreenidhi.shedi@broadcom.com>
(cherry picked from commit 264c15bfc4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 15:45:23 +01:00
Sebastiaan van Stijn
33b6b826c2 Merge pull request #48963 from thaJeztah/27.x_backport_carry_buildkit_init_timeout
[27.x backport] Remove buildkit init timeout
2024-11-27 11:42:55 +01:00
Brian Goff
151c7565a9 Remove buildkit init timeout
Buildkit *can* take a long time to start, we don't want the daemon to fail
to startup because buildkit took too long.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit b7f43c3729)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 09:32:54 +01:00
Sebastiaan van Stijn
eb19aa01ec Merge pull request #48957 from thaJeztah/27.x_backport_containerd-lease-cleanup
[27.x backport] Fix lease management during pull and export
2024-11-26 21:29:31 +01:00
Derek McGowan
1abc35f098 Add label on PrepareSnapshot to warn about non-expiring leases
Currently when preparing a snapshot for a container, a lease is used to
hold that snapshot for the lifespan of a container. That is workaround
to preserve the snapshot when a container is recreated, however, the
containerd object should be able to hold this reference itself.

Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit 4becdaca72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 14:20:15 +01:00
Derek McGowan
1d1da55382 Update containerd image builder to use lease prune label
Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit eef6b83e53)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 14:20:15 +01:00
Derek McGowan
a3afccd409 Fix lease management during image operations
Ensure that leases have a reasonable expiration and are
cleaned up during prune

Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit e6170a5c4f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 14:20:12 +01:00
Sebastiaan van Stijn
8a30a5e39c Merge pull request #48954 from thaJeztah/27.x_update_buildkit_0.17.3
[27.x] vendor: github.com/moby/buildkit 80e01a9dc7c1 (v0.17.3-dev)
2024-11-26 14:11:01 +01:00
Sebastiaan van Stijn
33e5f87d0f [27.x] vendor: github.com/moby/buildkit 80e01a9dc7c1 (v0.17.3-dev)
Use view transaction for metadata read to prevent dockerd failing to fully
start with a "context deadline exceeded error" with containerd snapshotter
and many builds/images.

full diff: https://github.com/moby/buildkit/compare/v0.17.2...80e01a9dc7c1f5bab680bab7b43059ad7a413301

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 11:04:34 +01:00
Sebastiaan van Stijn
55b81a15bc Merge pull request #48942 from thaJeztah/27.x_buildkit_0.17.2
[27.x] vendor: github.com/moby/buildkit v0.17.2
2024-11-25 17:54:05 +01:00
Sebastiaan van Stijn
465eb6aee2 vendor: github.com/cenkalti/backoff/v4 v4.3.0
full diff: https://github.com/cenkalti/backoff/compare/v4.2.1...v4.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f31188bc4c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 12:35:09 +01:00
Sebastiaan van Stijn
e4d5bae9c5 vendor: github.com/go-logr/logr v1.4.2
full diff: https://github.com/go-logr/logr/compare/v1.4.1...v1.4.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c3cab4170b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 12:34:00 +01:00
Sebastiaan van Stijn
752c0453d6 vendor: github.com/moby/buildkit v0.17.2
full diff: https://github.com/moby/buildkit/compare/v0.17.1...v0.17.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 12:29:05 +01:00
Sebastiaan van Stijn
339572db5d vendor: github.com/prometheus/client_golang v1.20.5
full diff: https://github.com/prometheus/client_golang/compare/v1.17.0...v1.20.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 95959f7000)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 11:10:24 +01:00
Sebastiaan van Stijn
4be68aaf6d Merge pull request #48934 from thaJeztah/27.x_backport_update-containerd-1.7.24
[27.x backport] Update containerd to v1.7.24
2024-11-22 18:20:26 +01:00
Sebastiaan van Stijn
af93467718 Merge pull request #48935 from thaJeztah/27.x_backport_bump_typeurl
[27.x backport] vendor: github.com/containerd/typeurl v2.2.3
2024-11-22 17:48:29 +01:00
Sebastiaan van Stijn
01f0737540 Merge pull request #48933 from thaJeztah/27.x_backport_update_go-actions-cache
[27.x backport] vendor: github.com/tonistiigi/go-actions-cache 394979b8119e
2024-11-22 17:12:25 +01:00
Sebastiaan van Stijn
ef390e92f9 Merge pull request #48931 from vvoland/48930-27.x
[27.x backport] vendor: resenje.org/singleflight v0.4.3
2024-11-22 17:11:42 +01:00
Sebastiaan van Stijn
6cab48bd9b vendor: github.com/containerd/typeurl v2.2.3
- add a `no_gogo` build-tag to allow compiling without support for gogoproto (github.com/gogo/protobuf)
- fix panic when type is not found and no handler is hit.
- fix unmarshalling of registered types

full diff: https://github.com/containerd/typeurl/compare/v2.2.0...v2.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0af6203b46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 14:48:10 +01:00
Derek McGowan
a64812939f Update containerd to v1.7.24
Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit a650dbd951)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 14:44:09 +01:00
Sebastiaan van Stijn
b6e1eb0dde Merge pull request #48926 from thaJeztah/27.x_bump_continuity
[27.x backport] vendor: github.com/containerd/continuity v0.4.5
2024-11-22 14:29:30 +01:00
Sebastiaan van Stijn
179c7c7596 vendor: github.com/tonistiigi/go-actions-cache 394979b8119e
updates to github.com/golang-jwt/jwt/v5

full diff: 58651d5e11...394979b811

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44f9eec1ae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 14:13:15 +01:00
Paweł Gronowski
7f9c5b278a vendor: resenje.org/singleflight v0.4.3
full diff: https://resenje.org/singleflight/compare/v0.4.1...v0.4.3

Changes:
- Fix incorrect `Forget` behavior
- Make panic behavior consistent with x/sync package

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 1551d95727)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-22 12:20:19 +01:00
Sebastiaan van Stijn
2684eacb65 Merge pull request #48919 from austinvazquez/cherry-pick-runtime-updates-to-27.x
[27.x backport] Dockerfile: update containerd v1.7.24, runc v1.2.2
2024-11-22 10:17:46 +01:00
Sebastiaan van Stijn
b90c6e922c vendor: github.com/containerd/continuity v0.4.5
full diff: https://github.com/containerd/continuity/compare/v0.4.4...v0.4.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d23bc11b97)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 10:12:04 +01:00
Sebastiaan van Stijn
e956d4a370 Dockerfile: update to runc v1.2.2
- 1.2.2 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.2
- 1.2.1 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.1
- 1.2.0 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.0

Breaking changes and deprecations are included below;

Breaking changes:

Several aspects of how mount options work has been adjusted in a way that
could theoretically break users that have very strange mount option strings.
This was necessary to fix glaring issues in how mount options were being
treated. The key changes are:

- Mount options on bind-mounts that clear a mount flag are now always
  applied. Previously, if a user requested a bind-mount with only clearing
  options (such as rw,exec,dev) the options would be ignored and the
  original bind-mount options would be set. Unfortunately this also means
  that container configurations which specified only clearing mount options
  will now actually get what they asked for, which could break existing
  containers (though it seems unlikely that a user who requested a specific
  mount option would consider it "broken" to get the mount options they
  asked foruser who requested a specific mount option would consider it
  "broken" to get the mount options they asked for). This also allows us to
  silently add locked mount flags the user did not explicitly request to be
  cleared in rootless mode, allowing for easier use of bind-mounts for
  rootless containers.
- Container configurations using bind-mounts with superblock mount flags
  (i.e. filesystem-specific mount flags, referred to as "data" in
  mount(2), as opposed to VFS generic mount flags like MS_NODEV) will
  now return an error. This is because superblock mount flags will also
  affect the host mount (as the superblock is shared when bind-mounting),
  which is obviously not acceptable. Previously, these flags were silently
  ignored so this change simply tells users that runc cannot fulfil their
  request rather than just ignoring it.

Deprecated

- runc option --criu is now ignored (with a warning), and the option will
  be removed entirely in a future release. Users who need a non-standard
  criu binary should rely on the standard way of looking up binaries in
  $PATH.
- runc kill option -a is now deprecated. Previously, it had to be specified
  to kill a container (with SIGKILL) which does not have its own private PID
  namespace (so that runc would send SIGKILL to all processes). Now, this is
  done automatically.
- github.com/opencontainers/runc/libcontainer/user is now deprecated, please
  use github.com/moby/sys/user instead. It will be removed in a future
  release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e257856116)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-11-21 18:09:01 +00:00
Austin Vazquez
b888333fe1 Dockerfile: update containerd to v1.7.24
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
(cherry picked from commit 8cecf3a71c)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-11-21 18:08:32 +00:00
Austin Vazquez
7fca82b32c Merge pull request #48913 from thaJeztah/27.x_backport_vendor-golang-jwt-v4.5.1
[27.x backport] vendor: github.com/golang-jwt/jwt/v4@v4.5.1
2024-11-21 04:53:26 -08:00
Austin Vazquez
6b04fcf560 vendor: github.com/golang-jwt/jwt/v4@v4.5.1
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
(cherry picked from commit 1eccc326de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-21 10:00:25 +01:00
Sebastiaan van Stijn
a080c3f32f Merge pull request #48899 from thaJeztah/27.x_backport_bump_grpc
[27.x backport] vendor: google.golang.org/grpc v1.66.3
2024-11-19 09:55:16 +01:00
Sebastiaan van Stijn
0f0b83a00b vendor: google.golang.org/grpc v1.66.3
- transport: Fix reporting of bytes read while reading headers
- xds/server: Fix xDS Server leak

full diff: https://github.com/grpc/grpc-go/compare/v1.66.2...v1.66.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4b26582bc6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-19 00:33:07 +01:00
Sebastiaan van Stijn
7bb4004fc8 Merge pull request #48880 from thaJeztah/27.x_containerd_1.7.23
[27.x] vendor: github.com/containerd/containerd v1.7.23
2024-11-15 15:43:10 +01:00
Sebastiaan van Stijn
2406bbff50 Merge pull request #48886 from thaJeztah/27.x_backport_cdi_0.8.0
[27.x] vendor: tags.cncf.io/container-device-interface v0.8.0
2024-11-15 15:01:07 +01:00
Sebastiaan van Stijn
1ef4219557 Merge pull request #48885 from thaJeztah/27.x_backport_more_build_tags
[27.x] update go:build tags to use go1.22, fix missing go:build tags
2024-11-15 15:00:39 +01:00
Sebastiaan van Stijn
b859373da2 vendor: tags.cncf.io/container-device-interface v0.8.0
Breaking change: The .ToOCI() functions in the specs-go package have been
removed. This removes the dependency on the OCI runtime specification from
the CDI specification definition itself.

What's Changed

- Add workflow to mark prs and issues as stale
- Remove the ToOCI functions from the specs-go package
- docs: add a pointer to community meetings in our docs.
- Bump spec version to v0.8.0
- Update spec version in README

Full diff:  https://github.com/cncf-tags/container-device-interface/compare/v0.7.2...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2f0180934d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:25:01 +01:00
Sebastiaan van Stijn
2f816b2596 [27.x] daemon/containerd: add missing go:build tag
This was introduced in c902e1a18701ac8e903314c688564e59cd1be997;

    make BIND_DIR=. shell
    make -C ./internal/gocompat/
    GO111MODULE=on go test -v
    # github.com/docker/docker/daemon/containerd
    ../../daemon/containerd/image.go:113:17: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../daemon/containerd/image.go:113:32: "github.com/distribution/reference".Named to satisfy comparable requires go1.20 or later (-lang was set to go1.16; check go.mod)
    ../../daemon/containerd/image_list.go:505:16: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:18:58 +01:00
Sebastiaan van Stijn
3e03c979da update go:build tags to use go1.22
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.

go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b453aa65fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:01:33 +01:00
Sebastiaan van Stijn
346ac21032 daemon/logger/loggerutils: add //go:build directives to prevent downgrading to go1.16
commit 77f2d90e27 introduced the slices import,
which uses generics, but failed to add the go:build directives.

    # github.com/docker/docker/daemon/logger/loggerutils
    ../../daemon/logger/loggerutils/logfile.go:770:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fb6da4f4b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:00:52 +01:00
Sebastiaan van Stijn
f54e5b0b19 libnetwork/networkdb: add //go:build directives to prevent downgrading to go1.16
commit 2847c4b7fe switched networkdb to use
go-immutable-radix v2, which uses generics, but failed to add the go:build
directives.

    # github.com/docker/docker/libnetwork/networkdb
    ../../libnetwork/networkdb/networkdb.go:47:19: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/networkdb/networkdb.go:259:33: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/networkdb/networkdb.go:269:25: function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/networkdb/networkdb.go:270:27: function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5403e3f4de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:00:52 +01:00
Sebastiaan van Stijn
6ec5197103 [27.x] vendor: github.com/containerd/containerd v1.7.23
similar to 5f39567e56 on master, but without
the hcsshim vendor updates, as it's pinned to v0.11.x in this branch.

full diff: https://github.com/containerd/containerd/compare/v1.7.22...v1.7.23

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 10:23:30 +01:00
Sebastiaan van Stijn
5765e9f35b Merge pull request #48842 from cpuguy83/27_backport_47983_log_reading
[27.x]: Make log reading more robust to errors
2024-11-14 13:53:22 +01:00
Sebastiaan van Stijn
158680383c Merge pull request #48863 from thaJeztah/27.x_backport_deprecate_pkg_platform
[27.x backport] deprecate pkg/platform and move internal
2024-11-14 13:40:57 +01:00
Sebastiaan van Stijn
1b40ddb635 Merge pull request #48864 from thaJeztah/27.x_backport_try-fix-flaky-plugin-client-test
[27.x backport] Increase test handler sleep, replace deprecated assert
2024-11-14 13:40:37 +01:00
Sebastiaan van Stijn
02e2e0426b Merge pull request #48867 from thaJeztah/27.x_backport_bump_compose_buildx
[27.x backport] Dockerfile: update buildx to v0.18.0, compose to v2.30.3
2024-11-14 13:40:15 +01:00
Sebastiaan van Stijn
31c8d23e7e Dockerfile: update compose to v2.30.3
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.29.7...v2.30.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a2c48e231)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-14 10:53:20 +01:00
Sebastiaan van Stijn
2ee3796e3f Dockerfile: update buildx to v0.18.0
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/0.17.1..0.18.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ee95c7bd89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-14 10:53:20 +01:00
Adam Simon
2acf9893c6 Increase test handler sleep, replace deprecated assert
- increase mock handler processing time to 50ms to try to prevent it from finishing before the 10ms client timeout occurs
- replace deprecated error type assertion

Signed-off-by: Adam Simon <adamsimon85100@gmail.com>
(cherry picked from commit 28dc2f6fac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-13 21:22:23 +01:00
Sebastiaan van Stijn
edfb2048ac deprecate pkg/platform and move internal
Functions in this package are only used internally in the daemon for
the `/info` endpoint (Architecture), and as part of `stats` (NumProcs).

I was not able to find external consumers, but deprecating the package
first, so that we can remove / dismantle the package in a follow-up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b034dc41a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-13 18:45:59 +01:00
Cory Snider
5ab7960b46 logger/journald: plumb contexts into reader
Co-authored-by:: Cory Snider <csnider@mirantis.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 6d94122217)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:10:48 +00:00
Brian Goff
857fc4730c logfile: Close reader when caller cancels
This allows for an individual decode operation to be cancelled while the
log reader is reading data from a log file by closing the underlying file.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 9b6ba18fc9)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:02:25 +00:00
Brian Goff
a7badb2bb5 Logfile: Add tracing spans
This plumbs a context down the stack and handles cancellation as needed
so that we can have correlated traces from the API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit dbf6873f45)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:02:16 +00:00
Brian Goff
d48c403efa Logfile: skip files that are corrupted
When there is an error in parsing an individual log file just close the
log and move on to the next one instead of erroring our the entire
request.

I investigated trying to error correct and scan ahead for corrupted log
files but found this is too much of a risk of parsing things we
shouldn't be and hence why this is just dropping the rest of the file.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1b46faf233)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:01:43 +00:00
Brian Goff
c9a7391911 Refactor logfile reads
This simplifies how we manage log files, especially rotated ones.
It also fixes a long-standing issue to lazily open rotated files so we
don't needlessly start decompressing files that we don't need.

Much of this is just setting things up for commits following this one.

It uses ReaderAtSize for managing all files to be tailed and manages
cleanups by passing closures.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 77f2d90e27)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:01:35 +00:00
Brian Goff
49f5431807 Fix time comparison in sharedtemp implementation
Times cannot be compared with `==` and instead should use the `t.Equal`
function.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit b37c8a03c0)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:01:27 +00:00
Sebastiaan van Stijn
b2c682be5f Merge pull request #48840 from akerouanton/remove-dns-answers-from-otel-spans
[27.4] libnet: don't put external DNS answers in OTel spans
2024-11-08 19:25:42 +01:00
Albin Kerouanton
173a9f2091 libnet: don't put external DNS answers in OTel spans
When containers make DNS resolution, and the domain name they're trying
to resolve doesn't match any running container, the DNS query is
forwarded to upstream servers. In that case, when we receive a response,
we put it in an OTel spans.

This was useful to debug DNS resolution on GHA, but it leads to
excessive memory usage when DNS resolution happen in a tight loop. So,
keep the OTel event signaling that a response was received, but drop the
answer from the OTel span.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-11-08 17:32:39 +01:00
Brian Goff
c14efb8dee Merge pull request #48820 from thaJeztah/27.x_backport_vendor-buildkit-v0.17.0
[27.x] vendor: update buildkit to v0.17.1, pin github.com/microsoft/hcsshim to v0.11.x
2024-11-08 07:58:05 -08:00
Tonis Tiigi
15b69c4200 vendor: update buildkit to v0.17.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 756586e437)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:32 +01:00
Tonis Tiigi
3280e50f65 vendor: update buildkit to v0.17.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 0e72863b9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:14 +01:00
Tonis Tiigi
00774deb96 vendor: update buildkit to v0.17.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 2cc21208e6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:14 +01:00
Sebastiaan van Stijn
0a67347003 vendor: google.golang.org/protobuf v1.34.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d2557466c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:13 +01:00
Sebastiaan van Stijn
033bd73bba [27.x] pin github.com/Microsoft/hcsshim to v0.11.x
We pin hcsshim to the v0.11.x version matching the version of containerd 1.7.
hcsshim v0.12.x comes with many (indirect) dependency updates that are not
desirable for minor releases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:11 +01:00
Sebastiaan van Stijn
e6b23cf105 Merge pull request #48832 from vvoland/update-go-27.x
[27.x] update to go1.22.9
2024-11-07 19:57:49 +01:00
Paweł Gronowski
0e4ab47f23 update to go1.22.9
- https://github.com/golang/go/issues?q=milestone%3AGo1.22.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.8...go1.22.9

go1.22.9 (released 2024-11-06) includes fixes to the linker. See the
[Go 1.22.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.22.9+label%3ACherryPickApproved)
milestone for details.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-07 13:20:09 +01:00
Paweł Gronowski
896f76e2e1 Merge pull request #48829 from vvoland/48357-27.x
[27.x backport] api/swagger: Improve description for platform in images/push
2024-11-06 15:25:54 +00:00
Paweł Gronowski
d3b9ec90ba api/swagger: Improve description for platform in images/push
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5014c90516)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-06 13:17:05 +01:00
Sebastiaan van Stijn
15ad1e4917 Merge pull request #48785 from vvoland/48777-27.x
[27.x] c8d/inspect: Fix duplicate RepoDigests
2024-10-30 11:01:22 +01:00
Paweł Gronowski
c902e1a187 c8d/inspect: Fix duplicate RepoDigests
Multiple images with the same repository name but different tag caused
the `RepoDigests` to contain duplicated entries for each of the image.

Deduplicate the slice before setting the `RepoDigests` field.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ba454f573b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-29 10:52:37 +01:00
Sebastiaan van Stijn
bad984fa46 Merge pull request #48768 from thaJeztah/27.x_backport_fix_debug_level
[27.x backport] volume: VolumesService.Create: fix log-level for debug logs
2024-10-26 20:18:24 +02:00
Sebastiaan van Stijn
b089f36796 volume: VolumesService.Create: fix log-level for debug logs
These log-entries were added in 10d57fde44,
but it looks like I accidentally left them as Error-logs following some
debugging (whoops!).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 352b4ff2f1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 17:43:27 +02:00
Sebastiaan van Stijn
5c383ad4dd Merge pull request #48764 from laurazard/27.x-backport-idle-conns
[27.x backport] client: prevent idle connections leaking FDs
2024-10-25 20:54:11 +02:00
Laura Brehm
afd7c9913f client: prevent idle connections leaking FDs
Patch from af6ada910f

Without this change, if a long-lived process uses the client and creates
connections, these connections are not released and grow over time.

We can also look into addressing this issue from the server side, but it
doesn't hurt for the `client` package to have good defaults and not
cause this.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit 5c72a95a30)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-10-25 18:13:24 +01:00
Sebastiaan van Stijn
b37ecae729 Merge pull request #48758 from vvoland/c8d-httpfallback-adjust
[27.x] c8d/httpfallback: Handle connection errors
2024-10-25 16:28:58 +02:00
Sebastiaan van Stijn
be5296767a Merge pull request #48755 from thaJeztah/27.x_backport_fix_anonymous_volumes_labels
[27.x backport] volume/mounts: fix anonymous volume not being labeled
2024-10-25 15:17:17 +02:00
Paweł Gronowski
de0135965e c8d/httpfallback: Handle connection errors
Adjust the httpFallback implementation to also handle non-TLS related
errors which can also happen when issuing a HTTPS requested to HTTP-only
registries.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-25 13:44:15 +02:00
Sebastiaan van Stijn
ddb56edd00 volume/mounts: fix anonymous volume not being labeled
`Parser.ParseMountRaw()` labels anonymous volumes with a `AnonymousLabel` label
(`com.docker.volume.anonymous`) label based on whether a volume has a name
(named volume) or no name (anonymous) (see [1]).

However both `VolumesService.Create()` (see [1]) and `Parser.ParseMountRaw()`
(see [2], [3]) were generating a random name for anonymous volumes. The latter
is called before `VolumesService.Create()` is called, resulting in such volumes
not being labeled as anonymous.

Generating the name was originally done in Create (fc7b904dce),
but duplicated in b3b7eb2723 with the introduction
of the new Mounts field in HostConfig. Duplicating this effort didn't have a
real effect until (`Create` would just skip generating the name), until
618f26ccbc introduced the `AnonymousLabel` in
(v24.0.0, backported to v23.0.0).

Parsing generally should not fill in defaults / generate names, so this patch;

- Removes generating volume names from  `Parser.ParseMountRaw()`
- Adds a debug-log entry to `VolumesService.Create()`
- Touches up some logs to use structured logs for easier correlating logs

With this patch applied:

    docker run --rm --mount=type=volume,target=/toto hello-world

    DEBU[2024-10-24T22:50:36.359990376Z] creating anonymous volume                     volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
    DEBU[2024-10-24T22:50:36.360069209Z] probing all drivers for volume                volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
    DEBU[2024-10-24T22:50:36.360341209Z] Registering new volume reference              driver=local volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02

[1]: 032721ff75/volume/service/service.go (L72-L83)
[2]: 032721ff75/volume/mounts/linux_parser.go (L330-L336)
[3]: 032721ff75/volume/mounts/windows_parser.go (L394-L400)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 10d57fde44)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-25 02:07:03 +02:00
Sebastiaan van Stijn
885b6e6103 Merge pull request #48742 from austinvazquez/cherry-pick-6be2074aefa60c3301dd728e7f9f6335a372c55a-to-27.x
[27.x backport] daemon: use OwnCgroupPath in withCgroups
2024-10-24 21:56:48 +02:00
Kir Kolyshkin
b7bc29af50 daemon: use OwnCgroupPath in withCgroups
Note: this usage comes from commit 56f77d5ade (part of PR 23430).

cgroups.InitCgroupPath is removed from runc (see [1]), and it is
suggested that users use OwnCgroupPath instead, because using init's is
problematic when in host PID namespace (see [2]) and is generally not
the right thing to do (see [3]).

[1]: https://github.com/opencontainers/runc/commit/fd5debf3
[2]: https://github.com/opencontainers/runc/commit/2b28b3c2
[3]: https://github.com/opencontainers/runc/commit/54e20217

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 6be2074aef)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-24 02:58:44 +00:00
Sebastiaan van Stijn
08674fb469 Merge pull request #48719 from thaJeztah/27.x_backport_deprecate_daemon_exists
[27.x backport] daemon: deprecate Daemon.Exists and Daemon.IsPaused
2024-10-22 15:12:19 +02:00
Sebastiaan van Stijn
1e11f2a1c0 daemon: deprecate Daemon.IsPaused
This function was poorly documented as it uses fuzzy matching under the hood,
and it's no longer used. Mark it as deprecated, and to be removed in the
next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac6e32cb5c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-21 17:22:31 +02:00
Sebastiaan van Stijn
0183f5dd13 daemon: deprecate Daemon.Exists
This function was poorly documented as it uses fuzzy matching under the hood,
and it's no longer used. Mark it as deprecated, and to be removed in the
next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d47c31ffdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-21 17:22:27 +02:00
Rob Murray
b6b3898090 Merge pull request #48714 from robmry/27.x-backport/48560-setup-user-chains
[27.x backport] Fix: setup user chains even if there are running containers
2024-10-21 14:25:31 +01:00
Andrés Maldonado
749f9ccee4 Fix: setup user chains even if there are running containers
Currently, the DOCKER-USER chains are set up on firewall reload or network
creation. If there are running containers at startup, configureNetworking won't
be called (daemon/daemon_unix.go), so the user chains won't be setup.

This commit puts the setup logic on a separate function, and calls it on the
original place and on initNetworkController.

Signed-off-by: Andrés Maldonado <maldonado@codelutin.com>
(cherry picked from commit a8bfa83667)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-21 12:40:26 +01:00
Sebastiaan van Stijn
e041d76796 Merge pull request #48712 from thaJeztah/27.x_backport_keep_manifest_order
[27.x backport] api: GET /images/json: preserve original manifest order
2024-10-21 13:17:54 +02:00
Sebastiaan van Stijn
8deb557451 api: GET /images/json: preserve original manifest order
The `manifests` option, as used for the `--tree` option on `docker image ls`
currently sorts manifests to put those that are present first. The intent was
to present "available" images at the top of each tree, followed by images that
were not pulled.

However, there's some limitations to this. First of all, the current approach
makes the output non-deterministic as the order in which variants are pulled
determines the order in which they're presented, i.e., the last pulled variant
is returned first (I omitted some variants in the example for brevity);

Here's the result of pulling `linux/riscv64`, then pulling `linux/arm64`;

    docker pull --platform=linux/riscv64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       10.6MB         3.37MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    └─ linux/arm64/v8       9cee2b382fe2           0B             0B

    docker pull --platform=linux/arm64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    └─ linux/amd64          33735bd63cf8           0B             0B

Repeating the steps but in reverse order results in the output to be reversed;

    docker image rm alpine:latest
    docker pull --platform=linux/arm64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       13.6MB         4.09MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    └─ linux/riscv64        80cde017a105           0B             0B

    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    └─ linux/amd64          33735bd63cf8           0B             0B

The second limitation is that order sometimes matters; when matching a
platform from a manifest-index, implementations may find multiple suitable
candidates. In most cases the _most_ suitable candidate can be selected
(e.g., prefer `linux/arm/v7` over `linux/arm/v6`), but manifest-indices do
allow multiple entries for the same platform, in which case implementations
match the first entry found.

While these situations will be less common (and usually due to incorect use
of tooling such as `docker manifest`), being able to observe the order in
which manifests appeared in the index can help debugging or help the user
understand why a specific variant was selected.

We should therefore not re-order these manifests, and return them in the
order in which they appeared. If we decide to present "present" variants
before "non-present" variants, we can do this ordering on the client side.

With this patch applied;

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker pull --quiet --platform=linux/arm64 alpine:latest
    docker image ls --tree alpine

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    ├─ linux/arm/v6         50f635c8b04d           0B             0B
    ├─ linux/arm/v7         f2f82d424957           0B             0B
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    ├─ linux/386            b3e87f642f5c           0B             0B
    ├─ linux/ppc64le        c7a6800e3dc5           0B             0B
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    └─ linux/s390x          2b5b26e09ca2           0B             0B

Which matches the order of the manifests in the index:

    docker buildx imagetools inspect --raw alpine:latest | jq -c .manifests[].platform
    {"architecture":"amd64","os":"linux"}
    {"architecture":"arm","os":"linux","variant":"v6"}
    {"architecture":"arm","os":"linux","variant":"v7"}
    {"architecture":"arm64","os":"linux","variant":"v8"}
    {"architecture":"386","os":"linux"}
    {"architecture":"ppc64le","os":"linux"}
    {"architecture":"riscv64","os":"linux"}
    {"architecture":"s390x","os":"linux"}

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d122ea0aea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-21 10:15:34 +02:00
Sebastiaan van Stijn
3bbb3c7f7f Merge pull request #48697 from austinvazquez/cherry-pick-pr-48668-to-27.x
[27.x backport] container: deprecate ErrNameReserved, ErrNameNotReserved, use errdefs instead
2024-10-19 17:46:15 +02:00
Sebastiaan van Stijn
c9cbe0e142 Merge pull request #48698 from austinvazquez/cherry-pick-pr-48654-to-27.x
[27.x backport] gha: more limits, update alpine version, and some minor improvements
2024-10-19 17:45:54 +02:00
Sebastiaan van Stijn
e4c067af91 gha: shorter time limits for smoke, validate
- validate-prepare and smoke-prepare took 10 seconds; limiting to 10 minutes
- smoke tests took less than 3 minutes; limiting to 10 minutes
- validate: most took under a minute, but "deprecate-integration-cli" took
  14 minutes; limiting to 30 minutes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a051aba82e)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:33 -07:00
Sebastiaan van Stijn
525b19ff04 gha: use "ubuntu-24.04" instead of "ubuntu-latest"
To be more explicit on what we're using.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 91c448bfb5)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:24 -07:00
Sebastiaan van Stijn
ceb41b83f3 gha: dco: small tweaks to running the container
- add `--quiet` to suppress pull progress output
- use `./` instead of `$(pwd)` now that relative paths are supported
- set the working directory on the container, so that we don't have to `cd`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9a14299540)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:13 -07:00
Sebastiaan van Stijn
2690b70a09 gha: dco: update ALPINE_VERSION to 3.20
Alpine 3.16 has been EOL for some time. Update to the latest version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cb98d759d)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:00 -07:00
Sebastiaan van Stijn
f53efc688f gha: build (binary), build (dynbinary): limit to 20 minutes
Regular runs are under 5 minutes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cfe0d2a131)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:04:34 -07:00
Sebastiaan van Stijn
166565e3ac gha: dco: limit to 10 minutes
Regular runs are under a minute.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e75f7aca2f)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:04:21 -07:00
Sebastiaan van Stijn
57a72c3c9d container: deprecate ErrNameReserved, ErrNameNotReserved
Use errdefs definitions instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cf90ca73f)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:59:20 -07:00
Sebastiaan van Stijn
8804551902 container: viewDB.ReserveName, view.GetID: return errdefs errors
Follow-up to 94dea2018e. Change these to return
errdefs types, which could allow us to move away from the sentinel errors
defined in the package, and instead use errdefs definitions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0200c58c4a)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:59:12 -07:00
Sebastiaan van Stijn
202aac0eb4 container: viewDB.withTxn: don't wrap errors
The withTxn function takes a custom function to execute; we should not
wrap those errors as the only responsibility of this function is to
execute the given function in a transaction.

This was introduced in 6549a270e9, and
an oversight of me.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0603bd9577)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:56:27 -07:00
Sebastiaan van Stijn
be8be11708 container: TestNames: don't string-match error assertions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0d36ab1875)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:56:11 -07:00
Sebastiaan van Stijn
b81bbc3aab Merge pull request #48695 from thaJeztah/27.x_backport_fix_48678
[27.x backport] dockerd-rootless-setuptool.sh: let --force ignore smoke test errors
2024-10-18 23:55:35 +02:00
Akihiro Suda
bedbe333bf dockerd-rootless-setuptool.sh: let --force ignore smoke test errors
Now `dockerd-rootless-setuptool.sh install --force` ignores errors
from `rootlesskit`.

This might be useful when installing Rootless Docker into a container
image with `RUN` instructions.

Fix issue 48678

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit fafdcd1194)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-18 22:22:54 +02:00
Sebastiaan van Stijn
cc70286271 Merge pull request #48693 from vvoland/48690-27.x
[27.x backport] cmd/dockerd: Add workaround for OTEL meter leak
2024-10-18 22:21:08 +02:00
Paweł Gronowski
ffb5bd905a cmd/dockerd: Add workaround for OTEL meter leak
OTEL meter implementation has a memory leak issue which causes each
meter counter invocation to create a new instrument when the meter
provider is not set.

Also add a test, which will fail once a fixed OTEL is vendored.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit cca7085464)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 17:06:12 +02:00
Sebastiaan van Stijn
b3b66cbbd7 Merge pull request #48691 from vvoland/48631-27.x
[27.x backport] Touch-up some errors for missing platforms
2024-10-18 17:05:24 +02:00
Paweł Gronowski
12cc7f7696 daemon/containerd: getPushDescriptor: fix formatting of platform in errors
The platform was printed in its raw format, which didn't produce a very
readable output;

Before this change:

    $ docker image push --platform=linux/amd64 alpine:arm64
    Error response from daemon: no suitable image manifest found for platform {amd64 linux [] }

After this change:

    $ docker image push --platform=linux/amd64 alpine:arm64
    Error response from daemon: no suitable image manifest found for platform linux/amd64

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:49:38 +02:00
Paweł Gronowski
01412dacf7 daemon/containerd: touch-up platform not found error
- Changed "match" to "provide", in an attempt to indicate that the image is
  a multi-platform image that doesn't contain the given platform.
- Remove the "wanted" and instead make the requested platforms to be a part
  of the sentence, but within braces.

Before this patch:

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker image history --platform=linux/amd64 alpine
    Error response from daemon: image with reference alpine:latest was found but does not match the specified platform: wanted linux/amd64

With this patch:

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker image history --platform=linux/amd64 alpine
    Error response from daemon: image with reference alpine:latest was found but does not provide the specified platform (linux/amd64)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:48:54 +02:00
Sebastiaan van Stijn
3c93981344 images: GetImage: touch-up error message for missing platform
Slightly touching up the error message, because the "wanted/actual" output
felt too much like a assertion failure in a test-case.

- Format the image-reference using "familiar" format, which shows the
  image's short name for official images.
- Move the actual and requested platforms to be a part of the sentence,
  but within braces.

Before this patch:

    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: image with reference docker.io/library/alpine:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux/riscv64

With this patch:

    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: image with reference alpine:latest was found but its platform (linux/riscv64) does not match the specified platform (linux/amd64)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8681b3c2ac)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:48:32 +02:00
Sebastiaan van Stijn
03225b1a41 PushImage: remove misleading error about --platform without containerd
Without containerd store enabled, we were discarding underlying errors,
and instead informing the user that `--platform` is not suported;

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: graphdriver backed image store doesn't support multiplatform images

However, that's not the case; platform filtering works, but if the image
is not the right platform, the push fails (which is the same as would
happen with the containerd image store enabled).

    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: image with reference docker.io/library/alpine:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux/riscv64

When specifying the platform and that platform matches, it finds the image,
and the push continue;

    docker image push --platform=linux/riscv64 alpine:latest
    The push refers to repository [docker.io/library/alpine]
    3fd4750fd687: Layer already exists
    ...

(The above example obviously fails because I don't have permissions to
push official images).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d31c241ea5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:48:30 +02:00
Paweł Gronowski
2ecd6971be Merge pull request #48687 from thaJeztah/27.x_backport_bump_swarmkit
[27.x backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20241017191044-e8ecf83ee08e
2024-10-18 12:19:16 +02:00
Sebastiaan van Stijn
a5555a93dd Merge pull request #48688 from thaJeztah/27.x_backport_build_split_history_db
[27.x backport] build: create distinct history db for each store
2024-10-18 11:27:15 +02:00
CrazyMax
5376990d4d build: create distinct history db for each store
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 58f4e916a3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 22:50:16 +02:00
Sebastiaan van Stijn
34d2e5e7e6 vendor: github.com/moby/swarmkit/v2 v2.0.0-20241017191044-e8ecf83ee08e
- add Unwrap error to custom error types
- removes dependency on github.com/rexray/gocsi
- fix CSI plugin load issue

full diff: ea1a7cec35...e8ecf83ee0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21b0d5f0c1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 22:31:26 +02:00
Sebastiaan van Stijn
bd021c3310 Merge pull request #48685 from robmry/backport-27.x/48664_br_netfilter_noproxy
[27.x backport] Enable bridge netfiltering if userland-proxy=false
2024-10-17 21:14:36 +02:00
Sebastiaan van Stijn
2f5d5faedd Merge pull request #48684 from robmry/backport-27.x/nodad_bridge_address
[27.x backport] Set NODAD on bridge IPv6 addresses
2024-10-17 21:13:55 +02:00
Rob Murray
0f8fc3105e Enable bridge netfiltering if userland-proxy=false
In release 27.0, ip6tables was enabled by default. That caused a
problem on some hosts where iptables was explicitly disabled and
loading the br_netfilter module (which loads with its nf-call-iptables
settings enabled) caused user-defined iptables rules to block traffic
on bridges, breaking inter-container communication.

In 27.3.0, commit 5c499fc4b2 delayed
loading of the br_netfilter module until it was needed. The load
now happens in the function that sets bridge-nf-call-ip[6]tables when
needed. It was only called for icc=false networks.

However, br_netfilter is also needed when userland-proxy=false.
Without it, packets addressed to a host-mapped port for a container
on the same network are not DNAT'd properly (responses have the server
container's address instead of the host's).

That means, in all releases including 26.x, if br_netfilter was loaded
before the daemon started - and the OS/user/other-application had
disabled bridge-nf-call-ip[6]tables, it would not be enabled by the
daemon. So, ICC would fail for host-mapped ports with the userland-proxy
disabled.

The change in 27.3.0 made this worse - previously, loading br_netfilter
whenever iptables/ip6tables was enabled meant that bridge-netfiltering
got enabled, even though the daemon didn't check it was enabled.

So... check that br_netfilter is loaded, with bridge-nf-call-ip[6]tables
enabled, if userland-proxy=false.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 0548fe251c)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-17 16:39:01 +01:00
Rob Murray
505c4c32be Set NODAD on bridge IPv6 addresses
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 38f0483ea7)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-17 16:13:32 +01:00
Sebastiaan van Stijn
6060b750e1 Merge pull request #48656 from thaJeztah/27.x_backport_docs_update_api_links
[27.x backport] update links to API documentation
2024-10-15 14:51:42 +02:00
Paweł Gronowski
b42c0bea9b Merge pull request #48666 from thaJeztah/27.x_backport_api_resize_required
[27.x backport] docs: api: document w (width) and h (height) query params as required
2024-10-15 11:01:49 +02:00
Sebastiaan van Stijn
c00b3108e5 docs: api: document w (width) and h (height) query params as required
Query parameters default to being _optional_ in swagger, but for these
endpoints, they were required. Omitting these parameters (or passing an
empty value) produces a 400 (bad request);

    docker run -d --name foo nginx:alpine

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize'
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h=100'

This patch updates the swagger to reflect that these parameters are required.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d9627b6159)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-14 23:25:25 +02:00
Sebastiaan van Stijn
e39ced5c00 api: document w (width) and h (height) query params as required
Query parameters default to being _optional_ in swagger, but for these
endpoints, they were required. Omitting these parameters (or passing an
empty value) produces a 400 (bad request);

    docker run -d --name foo nginx:alpine

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize'
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h=100'

This patch amends the swagger to reflect that these parameters are required.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a3a7e8ee8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-14 23:25:22 +02:00
Sebastiaan van Stijn
3c258e6011 pkg/authorization: update link to API documentation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a2361ea09)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 18:56:00 +02:00
Sebastiaan van Stijn
29f0dcd902 client: update link to API documentation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5b149d3faa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 18:56:00 +02:00
Sebastiaan van Stijn
21c974b281 docs/api: version-history: update documentation links
URLs were updated on docs.docker.com

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f8f143a27)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 18:55:29 +02:00
Sebastiaan van Stijn
fb64e14bbf Merge pull request #48646 from austinvazquez/cherry-pick-c68c9aed8cb3916669de6d7f2c564279ec83663f-to-27.x
[27.x backport] gha: restrict cross and bin-image to 20 minutes
2024-10-12 16:07:34 +02:00
Sebastiaan van Stijn
0bc91d5dc8 gha: restrict cross and bin-image to 20 minutes
We had a couple of runs where these jobs got stuck and github
actions didn't allow terminating them, so that they were only
terminated after 120 minutes.

These jobs usually complete in 5 minutes, so let's give them
a shorter timeout. 20 minutes should be enough (don't @ me).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c68c9aed8c)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-12 01:16:03 +00:00
Sebastiaan van Stijn
b434599c2d Merge pull request #48635 from thaJeztah/27.x_backport_bump_buildkit
[27.x backport] vendor: github.com/moby/buildkit v0.17.0-rc1
2024-10-11 23:00:13 +02:00
Sebastiaan van Stijn
d755573aba vendor: github.com/moby/buildkit v0.17.0-rc1
full diff:

- https://github.com/moby/buildkit/compare/v0.16.0...v0.17.0-rc1
- 91a3fc4684...3f140a1299
- 9b4947da39...701f63a606

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fe04c142d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-11 18:39:10 +02:00
Sebastiaan van Stijn
2388a908d5 Merge pull request #48633 from thaJeztah/27.x_backport_gha_timeout
[27.x backport] gha: add guardrails timeouts on all jobs
2024-10-10 20:32:43 +02:00
Sebastiaan van Stijn
32afa26719 gha: add guardrails timeouts on all jobs
We had a few "runaway jobs" recently, where the job got stuck, and kept
running for 6 hours (in one case even 24 hours, probably due some github
outage). Some of those jobs could not be terminated.

While running these actions on public repositories doesn't cost us, it's
still not desirable to have jobs running for that long (as they can still
hold up the queue).

This patch adds a blanket "2 hours" time-limit to all jobs that didn't
have a limit set. We should look at tweaking those limits to actually
expected duration, but having a default at least is a start.

Also changed the position of some existing timeouts so that we have a
consistent order in which it's set; making it easier to spot locations
where no limit is defined.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7e2783d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 18:42:01 +02:00
Sebastiaan van Stijn
875144c174 Merge pull request #48630 from thaJeztah/27.x_backport_integration_remove_TestBuildWithSession
[27.x backport] integration/build: remove TestBuildWithSession, and fsutil direct dependency
2024-10-10 17:33:38 +02:00
Sebastiaan van Stijn
8dc3ce5606 integration/build: remove TestBuildWithSession, and fsutil direct dependency
This test was originally used to test the experimental `--stream` option with
the classic builder. This feature was removed in Docker 20.10 through
6ca3ec88ae, which also disabled the test
(adding a note to rewrite the test for BuildKit).

The above is 5 years ago, and we now run the BuildKit integration tests
as part of our CI, which (I assume) should cover this functionality.

Removing this test, as it depends on github.com/tonistiigi/fsutil, which
does not have a stable API, and frequently cannot be updated independently
from BuildKit.

Also adding a linter-rule to prevent it from accidentally being imported;

    errdefs/defs.go:3:8: import 'github.com/tonistiigi/fsutil' is not allowed from list 'main': The fsutil module does not have a stable API, so we should not have a direct dependency unless necessary. (depguard)
    import _ "github.com/tonistiigi/fsutil"
           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fa6d3c230)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 14:38:19 +02:00
Sebastiaan van Stijn
e9ea285eec Merge pull request #48621 from thaJeztah/27.x_backport_bump_buildkit_deps
[27.x backport] vendor assorted dependencies in preparation of BuildKit v0.17
2024-10-10 14:36:34 +02:00
Sebastiaan van Stijn
136fdfcdfc vendor: google.golang.org/grpc v1.66.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2e699872bc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:14 +02:00
Sebastiaan van Stijn
6eed19d656 vendor: google.golang.org/protobuf v1.34.1
full diff: https://github.com/protocolbuffers/protobuf-go/compare/v1.33.0...v1.34.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e007ef71c1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:14 +02:00
Sebastiaan van Stijn
f29cc7c2cf vendor: github.com/cespare/xxhash/v2 v2.3.0
full diff: https://github.com/cespare/xxhash/compare/v2.2.0...v2.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30b6e58242)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
5f92ab397b vendor: golang.org/x/tools v0.25.0
full diff: https://github.com/golang/tools/compare/v0.24.0...v0.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4592ce42b6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
9af7d06698 vendor: golang.org/x/mod v0.21.0
full diff: https://github.com/golang/mod/compare/v0.20.0...v0.21.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 36549fb56e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
8fef24d9d4 vendor: golang.org/x/net v0.29.0
no changes in vendored code

full diff: https://github.com/golang/net/compare/v0.28.0...v0.29.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cc80652bcd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
efc5252041 vendor: golang.org/x/crypto v0.27.0
full diff: https://github.com/golang/crypto/compare/v0.26.0...v0.27.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d6db83acc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
a8e92fa662 vendor: golang.org/x/text v0.18.0
no changes in vendored code

full diff: https://github.com/golang/text/compare/v0.17.0...v0.18.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6e32888caa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:12 +02:00
Sebastiaan van Stijn
80cd75d7d5 vendor: golang.org/x/sys v0.25.0
full diff: https://github.com/golang/sys/compare/v0.24.0...v0.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 51cf0ed303)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:12 +02:00
Sebastiaan van Stijn
a2a544202b vendor: go.etcd.io/bbolt v1.3.11
BoltDB

- Fix the freelist.allocs isn't rollbacked when a tx is rollbacked.

CMD

- Add -gobench-output option for bench command to adapt to benchstat.

Other

- Bump go version to 1.22.x.
- This patch also added dmflakey package, which can be reused by other projects. See #812.

full diff: https://github.com/etcd-io/bbolt/compare/v1.3.10...v1.3.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7565ba40)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:10 +02:00
Sebastiaan van Stijn
ef4072a113 Merge pull request #48624 from thaJeztah/27.x_backport_fix_buildkit_go_version
[27.x backport] gha: buildkit: make sure expected Go version is installed
2024-10-10 13:16:41 +02:00
Sebastiaan van Stijn
418a79d51b gha: buildkit: make sure expected Go version is installed
The buildkit workflow uses Go to determine the version of Buildkit to run
integration-tests for. It currently uses on the default version that's
installed on the GitHub actions runners (1.21.13 currently), but this fails
if the go.mod/vendor.mod specify a higher version of Go as required version.

If this fails, the BUILDKIT_REF and REPO env-vars are not set / empty,
resulting in the workflow checking out the current (moby) repository instead
of buildkit, which fails.

This patch adds a step to explicitly install the expected version of Go.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 02d4fc3234)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 11:43:13 +02:00
Akihiro Suda
ee1210fa86 Merge pull request #48580 from austinvazquez/cherry-pick-ca4c68ab956993b47fd0046b4d96eceab8b9a261-to-27.x
[27.x backport] update to go1.22.8
2024-10-07 15:39:17 +09:00
Sebastiaan van Stijn
43580ccbc1 update to go1.22.8
go1.22.8 (released 2024-10-01) includes fixes to cgo, and the maps and syscall
packages. See the Go 1.22.8 milestone on our issue tracker for details;

- https://github.com/golang/go/issues?q=milestone%3AGo1.22.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.7...go1.22.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ca4c68ab95)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-04 19:44:30 +00:00
Sebastiaan van Stijn
78ab948df8 Merge pull request #48578 from austinvazquez/cherry-pick-docker-cli-27.3.1-compose-2.29.7-to-27.x
[27.x backport] Dockerfile: update docker CLI to v27.3.1, compose to v2.29.7
2024-10-04 14:00:01 +02:00
Sebastiaan van Stijn
af8b897560 Dockerfile: update compose to v2.29.7
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.29.4...v2.29.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7624a6bfa5)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-04 05:47:49 +00:00
Sebastiaan van Stijn
d91a922ef3 Dockerfile: update docker CLI to v27.3.1
Update the Docker CLI used in the dev-container

full diff: https://github.com/docker/cli/compare/v27.2.1...v27.3.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 233dce3dba)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-04 05:47:35 +00:00
Sebastiaan van Stijn
aec336e45d Merge pull request #48574 from thaJeztah/27.x_backport_add_codeql
[27.x backport] gha: add CodeQL Analysis workflow
2024-10-03 16:23:19 +02:00
Sebastiaan van Stijn
0138291345 gha: add CodeQL Analysis workflow
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 450200b4a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-03 13:20:02 +02:00
Sebastiaan van Stijn
41ca978a0a Merge pull request #48525 from thaJeztah/27.x_backport_govulncheck_permissions
[27.x backport] gha: govulncheck: make sure read permissions are set
2024-09-18 12:25:38 +02:00
Sebastiaan van Stijn
a6b772b24c gha: govulncheck: make sure read permissions are set
If any permission is set, any permission not included in the list is
implicitly set to "none".

see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

The govulncheck check need read permissions, which is not problematic
for public repositories, but may be needed when running in a private
fork (such as those used for security releases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f1d739de5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-18 10:36:14 +02:00
Sebastiaan van Stijn
856359cb77 Merge pull request #48514 from robmry/backport-27.x/wsl2_mirrored_loopback0_workaround
[27.x backport] Do not DNAT packets from WSL2's loopback0
2024-09-17 15:25:41 +02:00
Rob Murray
cd21af7e41 Do not DNAT packets from WSL2's loopback0
When running WSL2 with mirrored mode networking, add an iptables
rule to skip DNAT for packets arriving on interface loopback0 that
are addressed to a localhost address - they're from the Windows
host.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit f9c0103413)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-09-17 10:03:08 +01:00
Sebastiaan van Stijn
8516f3b0c1 Merge pull request #48510 from thaJeztah/27.x_backport_bump_buildx_compose
[27.x backport] Dockerfile: update buildx to v0.17.1, compose to v2.29.4
2024-09-16 17:08:16 +02:00
Sebastiaan van Stijn
3a7779ad61 Merge pull request #48511 from robmry/backport-27.x/48375_bridge_netfiltering
[27.x backport] Only enable bridge netfiltering when needed
2024-09-16 16:55:10 +02:00
Rob Murray
5c499fc4b2 Only enable bridge netfiltering when needed
Kernel module br_netfilter is loaded when the daemon starts with
either iptables or ip6tables enabled. That automatically sets:
  net.bridge.bridge-nf-call-arptables = 1
  net.bridge.bridge-nf-call-ip6tables = 1
  net.bridge.bridge-nf-call-iptables = 1

So, when:
- docker was running happily with iptables=false, and
- no explicit ip6tables=false, and
- br_netfilter was not loaded
... the change in moby 27.0 to enable ip6tables by default, resulted
in net.bridge.bridge-nf-call-iptables being enabled.

If the host also had a firewall with default-drop on its forward
chain - that resulted in packets getting dropped between containers
on a bridge network.

So, only try to load br_netfilter when it's needed - it's only needed
to implement "--icc=false", which can only be used when iptables or
ip6tables is enabled.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit db25b0dcd0)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-09-16 14:59:40 +01:00
Sebastiaan van Stijn
98f24aaf8a Merge pull request #48506 from thaJeztah/27.x_backport_man_dockerd_logformat
[27.x backport] man: dockerd: add description for --log-format option
2024-09-16 15:50:11 +02:00
Sebastiaan van Stijn
8adc8e405d Dockerfile: update compose to v2.29.4
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.29.2...v2.29.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 605f51172a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 14:56:28 +02:00
Sebastiaan van Stijn
576fc88b1a Dockerfile: update buildx to v0.17.1
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/0.17.0...0.17.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3d4831a23f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 14:56:27 +02:00
Sebastiaan van Stijn
e928a0d22d man: dockerd: add description for --log-format option
This option was added in a08abec9f8,
as part of Docker v25.0, but did not update the docs and manpage.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 45a9dde660)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 14:13:27 +02:00
Sebastiaan van Stijn
aed88ada36 Merge pull request #48503 from thaJeztah/27.x_backport_dockerd_typo
[27.x backport] dockerd: fix docs, improve validation and improve coverage of "--feature" flag
2024-09-16 14:12:17 +02:00
Sebastiaan van Stijn
82cf138590 internal/opts: SetOpts: invalidate empty option-names
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c2fc1f4a40)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
487a2f5314 internal/opts: SetOpts.Set: remove redundant var assignment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 908bb959e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
6d4a507acc internal/opts: SetOpts,NamedSetOpts: test for optional value
The value is optional for SetOpts (and NamedSetOpts), and implied
"true" when omitted.

This patch adds a test-case for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 758cca6036)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
bae46f6463 man: dockerd: value is optional for --feature flag
The --feature flag allows the boolean value to be omitted.
If only a name is provided, the default is "true".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 50e83a0713)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
176e04d73b man: fix duplicate word in --feature flag description
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2b6550bb2e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
7c0c43a239 Merge pull request #48499 from austinvazquez/cherry-pick-8b0e94ffaf7ea7d42391a3961e795b33976256c9-27.x
[27.x backport] Update dlv in the dev-env
2024-09-16 10:51:15 +02:00
Rob Murray
06c9c72269 Update dlv in the dev-env
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 8b0e94ffaf)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-09-14 07:25:40 -07:00
Sebastiaan van Stijn
bf60e5cced Merge pull request #48488 from thaJeztah/27.x_backport_c8d-prune-keeplastref
[27.x backport] c8d/prune: Keep the last tagged image instead of creating dangling image
2024-09-12 23:34:15 +02:00
Sebastiaan van Stijn
16294f36a2 Merge pull request #48490 from thaJeztah/27.x_backport_dockerd_man_features
[27.x backport] man: update dockerd man-page to include --feature flag
2024-09-12 23:33:52 +02:00
Sebastiaan van Stijn
15fbba2169 man: update dockerd man-page to include --feature flag
commit f13c08246d introduced
this flag, but did not yet update the manpage.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fa06acc851)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:51:02 +02:00
Paweł Gronowski
b262a95d67 c8d/prune: Keep deletion order stable
When untagging multiple images targetting the same digest, delete the
images in lexographic order to be consistent with graphdrivers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fefa98ae90)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:46:40 +02:00
Paweł Gronowski
39c17094b0 c8d/prune: Add OTEL spans
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 23e79a261e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:46:19 +02:00
Sebastiaan van Stijn
343800d7be adjust TestPruneDontDeleteUsedImage for 27.x branch
The 27.x branch does not have 7053007f71
adjusting the test to use the old location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:45:10 +02:00
Sebastiaan van Stijn
6753acddfe Merge pull request #48487 from thaJeztah/27.x_backport_add-feature-flags-daemon
[27.x backport] Add feature to daemon flags
2024-09-12 21:32:43 +02:00
Sebastiaan van Stijn
9c94a4d3cd Merge pull request #48485 from austinvazquez/cherry-pick-adb00d3d555f9dae37305c872dc32f6b654088d9-to-27.x
[27.x backport] TestIPRangeAt64BitLimit: remove colon after XFAIL to help grepping
2024-09-12 20:03:59 +02:00
Paweł Gronowski
95ffe8493e c8d/prune: Keep the last tagged image instead of creating dangling image
Don't turn images into dangling when they are used by containers created
with an image specified by an ID only (e.g. `docker run 82d1e9d`).

Keep the last image reference with the same target when all other
references would be pruned.

If the container was created with a digested and tagged reference (e.g.
`docker run alpine:latest@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1`),
the `alpine:latest` image won't get untagged.

This change makes the behavior consistent with the graphdriver
implementation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e4c2eb9d8a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 19:34:24 +02:00
Derek McGowan
c40cd54593 Add feature to daemon flags
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f13c08246d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 19:30:17 +02:00
Sebastiaan van Stijn
dbee7b16df TestIPRangeAt64BitLimit: remove colon after XFAIL to help grepping
When looking for failures in CI, I always search for `FAIL:` (with a
trailing colon) to find tests that fail. This test has some test-cases
that are currently expected to fail, but due to the colon would also
be included when searching;

    === RUN   TestIPRangeAt64BitLimit/ipRange_at_end_of_64-bit_subnet
        bridge_test.go:196: XFAIL: Container startup failed with error: Error response from daemon: no available IPv6 addresses on this network's address pools: test64bl (b014e28c35c14cc34514430a8cfe1c97632c7988c56d89cea46abb10fa32229d)
    === RUN   TestIPRangeAt64BitLimit/ipRange_at_64-bit_boundary_inside_56-bit_subnet
        bridge_test.go:196: XFAIL: Container startup failed with error: Error response from daemon: no available IPv6 addresses on this network's address pools: test64bl (fb70301550d7a2d1d3425f5c1010a9ef487a9a251221a2d68ac49d257b249013)

Remove the trailing `:` so that searching for unexpected failures does not
include these tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit adb00d3d55)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-09-12 16:40:57 +00:00
Sebastiaan van Stijn
6c084b6646 Merge pull request #48479 from thaJeztah/27.x_backport_rootlesskit
[27.x backport] Dockerfile: update RootlessKit to v2.3.1
2024-09-12 17:34:19 +02:00
Sebastiaan van Stijn
81e74f3e09 Merge pull request #48483 from thaJeztah/27.x_backport_fix-gpus-0
[27.x backport] Explicitly disable nvidia device injection for --gpus=0
2024-09-12 16:42:28 +02:00
Evan Lezar
bd71b5d319 Explicitly disable nvidia device injection for --gpus=0
This change ensures that when --gpus=0 is selected, the injection of
NVIDIA device nodes and libraries is disabled by setting the
NVIDIA_VISIBLE_DEVICES environment variable to void instead of
leaving this unspecfied.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
(cherry picked from commit 5128007116)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 13:21:53 +02:00
Akihiro Suda
a5478dde0e Dockerfile: update RootlessKit to v2.3.1
https://github.com/rootless-containers/rootlesskit/compare/v2.0.2...v2.3.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 43036f350a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-11 18:27:47 +02:00
Sebastiaan van Stijn
c717e4cd2a Merge pull request #48481 from laurazard/backport-typos
[27.x backport] Fix typos
2024-09-11 17:51:34 +02:00
Sebastiaan van Stijn
8956bdd7b3 Merge pull request #48477 from vvoland/48475-27.x
[27.x backport] Dockerfile: Update CLI, buildx and compose
2024-09-11 17:20:21 +02:00
Nathan Baulch
80d0013217 Fix typos
Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
(cherry picked from commit 59eba0ae13)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-09-11 15:36:28 +01:00
Paweł Gronowski
ce86fade0d Dockerfile: update compose to v2.29.2
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/cli/compare/v2.29.0...v2.29.2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 962da27cdf)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 12:29:02 +02:00
Paweł Gronowski
682b96c479 Dockerfile: update buildx to 0.17.0
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/0.16.1...0.17.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e5e8addeee)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 12:29:01 +02:00
Paweł Gronowski
c3aaaf2be0 Dockerfile: update docker CLI to v27.2.1
Update the Docker CLI used in the dev-container

full diff: https://github.com/docker/cli/compare/v27.0.2...v27.2.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d291722110)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 12:28:57 +02:00
Sebastiaan van Stijn
20bff974a0 Merge pull request #48469 from thaJeztah/27.x_backport_vendor_buildkit_v0.16
[27.x backport] vendor: update buildkit to v0.16.0
2024-09-11 10:14:27 +02:00
Sebastiaan van Stijn
ecf2f8420e Merge pull request #48459 from thaJeztah/27.x_backport_fix_push_docs
[27.x backport] api: swagger: fix documentation for image push endpoint
2024-09-11 10:13:58 +02:00
Sebastiaan van Stijn
b8e36aafe7 Merge pull request #48470 from thaJeztah/27.x_backport_api_fix_deprecation_version
[27.x backport] api/swagger: update deprecation version for erroneous fields
2024-09-11 10:13:15 +02:00
CrazyMax
ce07360a1c vendor: update buildkit to v0.16.0
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 71e449de0a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 20:21:53 +02:00
Sebastiaan van Stijn
cdbfb9c71a Merge pull request #48468 from thaJeztah/27.x_backport_bump_containerd_1.7.22
[27.x backport] Dockerfile/vendor: update containerd to v1.7.22
2024-09-10 19:07:38 +02:00
Sebastiaan van Stijn
ab99508c07 docs/api: update deprecation version for erroneous fields (v1.46, v1.47)
commit af0cdc36c7 officially marked these
fields as deprecated and to be removed in API v1.47 (which was targeted
for v28.0). We shipped v1.47 with the v27.2 release, but did not yet
remove the erroneous fields.

This patch updates the version to v1.48.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 09fc6ab2d9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 16:32:10 +02:00
Sebastiaan van Stijn
c7a8942d6d api/swagger: update deprecation version for erroneous fields
commit af0cdc36c7 officially marked these
fields as deprecated and to be removed in API v1.47 (which was targeted
for v28.0). We shipped v1.47 with the v27.2 release, but did not yet
remove the erroneous fields.

This patch updates the version to v1.48.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3df03d8e66)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 16:32:07 +02:00
Tonis Tiigi
57ba0b05e6 vendor: update buildkit to v0.16.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 6f37e5a168)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:38 +02:00
CrazyMax
16bf28ec2d test: update since session signature has changed
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 5ebc179cb1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
CrazyMax
a0650bf67c vendor: update buildkit to v0.16.0-rc1
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit f4261dbb75)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
Akihiro Suda
f7ea6bb4d9 vendor.mod: golang.org/x/net v0.28.0, etc.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d6446d81d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
Akihiro Suda
da2b59d3a2 vendor.mod: golang.org/x/time v0.6.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 65fdd363a0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
Akihiro Suda
3e60519c9c vendor.mod: golang.org/x/sys v0.24.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 4bfdfd6f72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Akihiro Suda
e6a4c5f840 vendor.mod: golang.org/x/sync v0.8.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 78d0ef5e87)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Akihiro Suda
2514b02677 vendor.mod: golang.org/x/mod v0.20.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 492fe40052)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Akihiro Suda
69f9ab2958 vendor.mod: drop direct dependency on golang.org/x/exp
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d7ff6b5085)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Sebastiaan van Stijn
70c8d41445 vendor: github.com/containerd/nydus-snapshotter v0.14.0
- removes use of deprecated github.com/containerd/containerd/errdefs package
- removes use of deprecated github.com/containerd/containerd/platforms package
- removes use of deprecated github.com/containerd/containerd/reference/docker package
- switch to dario.cat/mergo v1.0.0 dependency
- remove use of deprecated CRI Alpha API

full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.13.14...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af84ddec13)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:25 +02:00
Sebastiaan van Stijn
54e4cdc044 vendor: github.com/containerd/nydus-snapshotter v0.13.14
- removes use of deprecated github.com/containerd/containerd/log package

full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.13.7...v0.13.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e5c0c4d77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:31:44 +02:00
Sebastiaan van Stijn
dca7a36092 vendor: google.golang.org/grpc v1.62.0
diffs:

- https://github.com/grpc/grpc-go/compare/v1.59.0...v1.62.0
- 83a465c022...ef4313101c
- https://github.com/googleapis/google-api-go-client/compare/v0.149.0...v0.155.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d44f869a2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:27:53 +02:00
Sebastiaan van Stijn
458110ccd0 vendor: golang.org/x/time v0.5.0
full diff: https://github.com/golang/time/compare/v0.3.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 716ec490db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:27:52 +02:00
Sebastiaan van Stijn
27b5c33847 vendor: github.com/moby/sys/sequential v0.6.0
full diff: https://github.com/moby/sys/compare/sequential/v0.5.0...sequential/v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21fd5a3f30)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:22:34 +02:00
Sebastiaan van Stijn
6e77ead91e vendor: github.com/moby/sys/symlink v0.3.0
full diff: https://github.com/moby/sys/compare/symlink/v0.2.0...symlink/v0.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6a6b49f2de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:20:47 +02:00
Sebastiaan van Stijn
0890631ac7 vendor: github.com/moby/sys/signal v0.7.1
full diff: https://github.com/moby/sys/compare/signal/v0.7.0...signal/v0.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68636d56fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:17:40 +02:00
Sebastiaan van Stijn
508173c39d vendor: github.com/moby/sys/mount v0.3.4
full diff: https://github.com/moby/sys/compare/mount/v0.3.3...mount/v0.3.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44b266ab59)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:17:40 +02:00
Sebastiaan van Stijn
9b26c10e8b vendor: github.com/moby/sys/mountinfo v0.7.2
full diff: https://github.com/moby/sys/compare/mountinfo/v0.7.1...mountinfo/v0.7.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc4a12874e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:17:40 +02:00
Sebastiaan van Stijn
2f39dca20a Merge pull request #48463 from gdams/27
[27.x backport] seccomp: add riscv64 mapping to seccomp_linux.go
2024-09-10 14:14:46 +02:00
Sebastiaan van Stijn
d3884731cd Dockerfile: update containerd binary to v1.7.22
Update the containerd binary that's used in CI and static binaries

- Update to go1.22.7, go1.23.1
- CRI: Cumulative stats can't decrease
- Fix bug where init exits were being dropped
- Update runc binary to 1.1.14

- diff: https://github.com/containerd/containerd/compare/v1.7.21...v1.7.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92195c1333)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:04:19 +02:00
Sebastiaan van Stijn
cdbc2b6869 vendor: github.com/containerd/containerd v1.7.22
no significant changes in vendored code

- diff: https://github.com/containerd/containerd/compare/v1.7.21...v1.7.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b4a3e8a307)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:04:16 +02:00
George Adams
d0462c0e80 seccomp: add riscv64 mapping to seccomp_linux.go
Signed-off-by: George Adams <georgeadams1995@gmail.com>
(cherry picked from commit 1161b790cf)
Signed-off-by: George Adams <georgeadams1995@gmail.com>
2024-09-10 11:35:03 +01:00
Sebastiaan van Stijn
fc774f6dc0 docs: api: fix documentation for image push endpoint (API v1.25-v1.47)
The documentation was incorrect and didn't properly document the use of
tags;

- Image push currently only accepts an image-name, not an ID / digest.
- When giving a name, it's expected to be without tag included; when
  including a tag, it is ignored.
- The tag parameter is required when pushing a single image (i.e., it
  does not default to "latest"); omitting the tag parameter will push
  all tags of the given image.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aca38a4218)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 10:49:25 +02:00
Sebastiaan van Stijn
84d78a8ad5 api: swagger: fix documentation for image push endpoint
The documentation was incorrect and didn't properly document the use of
tags;

- Image push currently only accepts an image-name, not an ID / digest.
- When giving a name, it's expected to be without tag included; when
  including a tag, it is ignored.
- The tag parameter is required when pushing a single image (i.e., it
  does not default to "latest"); omitting the tag parameter will push
  all tags of the given image.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07e31e393a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 10:49:25 +02:00
Sebastiaan van Stijn
efe9fbd800 Merge pull request #48451 from vvoland/48450-27.x
[27.x backport] gha/bin-image: Also run on branches like `27.x`
2024-09-09 21:28:51 +02:00
Paweł Gronowski
e16197e51d gha/bin-image: Also run on branches like 27.x
We moved to the major release branches with a `.x` suffix and forgot to
adjust this workflow.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 47d5ce0ef8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-09 18:58:17 +02:00
Sebastiaan van Stijn
8b539b8df2 Merge pull request #48433 from vvoland/48432-27.x
[27.x backport] update to go1.22.7
2024-09-06 11:57:40 +02:00
Paweł Gronowski
37bf6e81f9 update to go1.22.7
- https://github.com/golang/go/issues?q=milestone%3AGo1.22.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.6...go1.22.7

These minor releases include 3 security fixes following the security policy:

- go/parser: stack exhaustion in all Parse* functions

    Calling any of the Parse functions on Go source code which contains deeply nested literals can cause a panic due to stack exhaustion.

    This is CVE-2024-34155 and Go issue https://go.dev/issue/69138.

- encoding/gob: stack exhaustion in Decoder.Decode

    Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion.

    This is a follow-up to CVE-2022-30635.

    Thanks to Md Sakib Anwar of The Ohio State University (anwar.40@osu.edu) for reporting this issue.

    This is CVE-2024-34156 and Go issue https://go.dev/issue/69139.

- go/build/constraint: stack exhaustion in Parse

    Calling Parse on a "// +build" build tag line with deeply nested expressions can cause a panic due to stack exhaustion.

    This is CVE-2024-34158 and Go issue https://go.dev/issue/69141.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.23.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a2e14dd8bd)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-05 17:49:47 +02:00
Sebastiaan van Stijn
fefdb1bf24 Merge pull request #48430 from vvoland/48425-27.x
[27.x backport] vendor: github.com/opencontainers/runc v1.1.14
2024-09-04 11:57:15 +02:00
Sebastiaan van Stijn
3a4c589290 vendor: github.com/opencontainers/runc v1.1.14
diff: https://github.com/opencontainers/runc/compare/v1.1.13...v1.1.14

Release Notes:

- Fix CVE-2024-45310 / GHSA-jfvp-7x6p-h2pv, a low-severity attack that allowed maliciously configured containers to create empty files and directories on the host.
- Add support for Go 1.23.
- Revert "allow overriding VERSION value in Makefile" and add EXTRA_VERSION.
- rootfs: consolidate mountpoint creation logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b5ec31f090)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-03 17:52:57 +02:00
Paweł Gronowski
6eeb414adf Merge pull request #48426 from thaJeztah/27.x_backport_bump_runc_1.1.14
[27.x backport] update runc binary to 1.1.14
2024-09-03 14:13:08 +00:00
Sebastiaan van Stijn
54af846a0c Merge pull request #48416 from thaJeztah/27.x_backport_update_go_1.22
[27.x backport] update to go1.22.6
2024-09-03 16:12:38 +02:00
Sebastiaan van Stijn
10cf2d64b6 update runc binary to 1.1.14
Update the runc binary that's used in CI and for the static packages.

diff: https://github.com/opencontainers/runc/compare/v1.1.13...v1.1.14

Release Notes:

- Fix CVE-2024-45310, a low-severity attack that allowed maliciously configured containers to create empty files and directories on the host.
- Add support for Go 1.23.
- Revert "allow overriding VERSION value in Makefile" and add EXTRA_VERSION.
- rootfs: consolidate mountpoint creation logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2189aa2426)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-03 15:13:42 +02:00
Paweł Gronowski
7b86f40aa2 Merge pull request #48423 from vvoland/48422-27.x
[27.x backport] docs/api: add documentation for API v1.47
2024-09-03 10:58:45 +00:00
Paweł Gronowski
84cfcb0893 docs/api: add documentation for API v1.47
API version used by the v27.2.0 release.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4e0d1dfc17)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-03 11:58:32 +02:00
Sebastiaan van Stijn
edeb6ba9fb golangci-lint: temporarily disable G115: integer overflow conversion
it produces many hits, some of which may be false positives, but we need to
look into these, e.g.;

    container/container.go:517:72: G115: integer overflow conversion int -> uint32 (gosec)
        shouldRestart, _, _ := container.RestartManager().ShouldRestart(uint32(container.ExitCode()), container.HasBeenManuallyStopped, container.FinishedAt.Sub(container.StartedAt))
                                                                              ^
    container/view.go:401:25: G115: integer overflow conversion int -> uint16 (gosec)
                        PrivatePort: uint16(p),
                                           ^
    container/view.go:413:25: G115: integer overflow conversion int -> uint16 (gosec)
                        PrivatePort: uint16(p),
                                           ^
    container/view.go:414:25: G115: integer overflow conversion int -> uint16 (gosec)
                        PublicPort:  uint16(h),
                                           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f5108e9c6b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:32 +02:00
Sebastiaan van Stijn
adb0b8536d update golangci-lint to v1.60.2
Update to add go1.23 support

full diff: https://github.com/golangci/golangci-lint/compare/v1.59.1...v1.60.2
Changelog: https://golangci-lint.run/product/changelog/#1602

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b11bb507b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:31 +02:00
Sebastiaan van Stijn
01f7ec0f21 update to go1.22.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3df59c9dcf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:31 +02:00
Cory Snider
442462ad4e hack/make/.binary: set CGO_LDFLAGS=-latomic for arm/v5
cross-compiling for arm/v5 was failing;

    #56 84.12 /usr/bin/arm-linux-gnueabi-clang -marm -o $WORK/b001/exe/a.out -Wl,--export-dynamic-symbol=_cgo_panic -Wl,--export-dynamic-symbol=_cgo_topofstack -Wl,--export-dynamic-symbol=crosscall2 -Qunused-arguments -Wl,--compress-debug-sections=zlib /tmp/go-link-759578347/go.o /tmp/go-link-759578347/000000.o /tmp/go-link-759578347/000001.o /tmp/go-link-759578347/000002.o /tmp/go-link-759578347/000003.o /tmp/go-link-759578347/000004.o /tmp/go-link-759578347/000005.o /tmp/go-link-759578347/000006.o /tmp/go-link-759578347/000007.o /tmp/go-link-759578347/000008.o /tmp/go-link-759578347/000009.o /tmp/go-link-759578347/000010.o /tmp/go-link-759578347/000011.o /tmp/go-link-759578347/000012.o /tmp/go-link-759578347/000013.o /tmp/go-link-759578347/000014.o /tmp/go-link-759578347/000015.o /tmp/go-link-759578347/000016.o /tmp/go-link-759578347/000017.o /tmp/go-link-759578347/000018.o -O2 -g -O2 -g -O2 -g -lpthread -O2 -g -no-pie -static
    #56 84.12 ld.lld: error: undefined symbol: __atomic_load_4
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced 2 more times
    #56 84.12
    #56 84.12 ld.lld: error: undefined symbol: __atomic_store_4
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(x_cgo_notify_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(x_cgo_set_context_function)
    #56 84.12 clang: error: linker command failed with exit code 1 (use -v to see invocation)

From discussion on GitHub;
https://github.com/moby/moby/pull/46982#issuecomment-2206992611

The arm/v5 build failure looks to be due to libatomic not being included
in the link. For reasons probably buried in mailing list archives,
[gcc](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358) and clang don't
bother to implicitly auto-link libatomic. This is not a big deal on many
modern platforms with atomic intrinsics as the compiler generates inline
instruction sequences, avoiding any libcalls into libatomic. ARMv5 is not
one of those platforms: all atomic operations require a libcall.

In theory, adding `CGO_LDFLAGS=-latomic` should fix arm/v5 builds.

While it could be argued that cgo should automatically link against
libatomic in the same way that it automatically links against libpthread,
the Go maintainers would have a valid counter-argument that it should be
the C toolchain's responsibility to link against libatomic automatically,
just like it does with libgcc or compiler-rt.

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 4cd5c2b643)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:31 +02:00
Cory Snider
a3eca63b5b hack/make/.binary: set CCGO_CFLAGS=-Wno-atomic-alignment for arm/v5
cross-compiling for arm/v5 fails on go1.22; a fix is included for this
in go1.23 (https://github.com/golang/go/issues/65290), but for go1.22
we can set the correct option manually.

    1.189 + go build -mod=vendor -modfile=vendor.mod -o /tmp/bundles/binary-daemon/dockerd -tags 'netgo osusergo static_build journald' -ldflags '-w -X "github.com/docker/docker/dockerversion.Version=dev" -X "github.com/docker/docker/dockerversion.GitCommit=HEAD" -X "github.com/docker/docker/dockerversion.BuildTime=2024-08-29T16:59:57.000000000+00:00" -X "github.com/docker/docker/dockerversion.PlatformName=" -X "github.com/docker/docker/dockerversion.ProductName=" -X "github.com/docker/docker/dockerversion.DefaultProductLicense=" -extldflags -static ' -gcflags= github.com/docker/docker/cmd/dockerd
    67.78 # runtime/cgo
    67.78 gcc_libinit.c:44:8: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:47:6: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:49:10: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:69:9: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:71:3: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    78.20 + rm -f /go/src/github.com/docker/docker/go.mod

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit e853c093bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:28 +02:00
Sebastiaan van Stijn
7177a05666 Merge pull request #48415 from vvoland/48414-27.x
[27.x backport] c8d/pull: Same error message for non-matching platform
2024-09-02 17:28:47 +02:00
Paweł Gronowski
d036255b69 c8d/pull: Same error message for non-matching platform
Use the same error message as the graphdrivers image store backend.
It's more informative as it also includes the requested platform and
won't break clients checking doing error check with string-matching.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit c7f8557310)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-02 13:33:37 +02:00
Tianon Gravi
88288fedff Merge pull request #48389 from corhere/backport-27.x/dockerd-manpage-install
[27.x backport] man: create parent directories in install recipe
2024-08-29 14:35:07 -07:00
Cory Snider
b9f56706c5 man: create parent directories in install recipe
Support the use of `make install` in packaging scripts, where the
$mandir tree might not exist under $DESTDIR.

For portability, create the parent directories using a separate install
command instead of relying on the non-portable `-D` flag.

Set errexit so the install target fails if any install step fails.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 88b118688e)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-29 16:20:24 -04:00
Sebastiaan van Stijn
7c1f8db55e Merge pull request #48402 from vvoland/48399-27.x
[27.x backport] c8d/list: Don't exclude non-container images
2024-08-29 20:53:17 +02:00
Paweł Gronowski
12d4379ad1 c8d/list: Don't exclude non-container images
Before this, the image list would not show images that are not a valid
container image, but could be a valid artifact.

While they're not directly usable by docker, we should still show them
so the user can still discover them and at least be able to delete them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit f4e82e2fb0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-29 14:27:18 +02:00
Sebastiaan van Stijn
801f77bbef Merge pull request #48395 from thaJeztah/27.x_backport_fix_linting_issues
[27.x backport] Fix linting issues in preparation of Go and GolangCI-lint update
2024-08-28 15:48:50 +02:00
Sebastiaan van Stijn
0fbb78c187 daemon: fix non-constant format string in call (govet)
daemon/daemon.go:942:21: printf: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Errorf (govet)
            log.G(ctx).Errorf(err.Error())
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1ad5b5abb2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:11 +02:00
Sebastiaan van Stijn
2bc17a1e5c api/types: fix non-constant format string in call (govet)
api/types/container/hostconfig.go:328:43: printf: non-constant format string in call to fmt.Errorf (govet)
                return &errInvalidParameter{fmt.Errorf(msg)}
                                                       ^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 005b488506)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:11 +02:00
Sebastiaan van Stijn
fe01a5fc30 api/server/router: fix non-constant format string in call (govet)
api/server/router/container/container_routes.go:943:22: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
                                  ^
    api/server/router/image/image_routes.go:144:50: printf: non-constant format string in call to github.com/docker/docker/pkg/streamformatter.FormatStatus (govet)
                output.Write(streamformatter.FormatStatus("", id.String()))
                                                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fd3a53c12)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
bf7ca4dc4b container/stream: fix non-constant format string in call (govet)
container/stream/streams.go:111:21: printf: non-constant format string in call to fmt.Errorf (govet)
            return fmt.Errorf(strings.Join(errors, "\n"))
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a93233b88)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
405a163620 libnetwork/drivers/bridge: fix non-constant format string in call (govet)
libnetwork/drivers/bridge/setup_ip_tables_linux.go:385:23: printf: non-constant format string in call to fmt.Errorf (govet)
                    return fmt.Errorf(msg)
                                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 068c1bf3be)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
812b97d559 volume/testutils: fix non-constant format string in call (govet)
volume/testutils/testutils.go:98:26: printf: non-constant format string in call to fmt.Errorf (govet)
            return nil, fmt.Errorf(opts["error"])
                                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f434cdd14a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
c4eef2a58c builder/dockerfile: parseChownFlag: fix non-constant format string in call (govet)
builder/dockerfile/internals_linux.go:38:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
            return idtools.Identity{}, errors.Wrapf(err, "can't find uid for user "+userStr)
                                                         ^
    builder/dockerfile/internals_linux.go:42:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
            return idtools.Identity{}, errors.Wrapf(err, "can't find gid for group "+grpStr)
                                                         ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 81a1ca0217)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
8dec1f216f layer: ignore G602: slice index out of range (gosec)
This looks to be a false positive;

    layer/layer.go:202:47: G602: slice index out of range (gosec)
            return createChainIDFromParent(ChainID(dgsts[0]), dgsts[1:]...)
                                                        ^
    layer/layer.go:205:69: G602: slice index out of range (gosec)
        dgst := digest.FromBytes([]byte(string(parent) + " " + string(dgsts[0])))
                                                                           ^
    layer/layer.go:206:53: G602: slice bounds out of range (gosec)
        return createChainIDFromParent(ChainID(dgst), dgsts[1:]...)
                                                           ^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b56c58a860)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:09 +02:00
Sebastiaan van Stijn
068e82bdd2 internal/cleanups: fix non-constant format string in call (govet)
internal/cleanups/composite_test.go:46:9: printf: non-constant format string in call to (*testing.common).Logf (govet)
        t.Logf(err.Error())
               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c308bd3e0e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
8ae439ddaa libnetwork/cnmallocator: fix non-constant format string in call (govet)
libnetwork/cnmallocator/drivers_ipam.go:43:31: printf: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Infof (govet)
            log.G(context.TODO()).Infof("Swarm initialized global default address pool to: " + str.String())
                                        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b60a7047d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
c27f7de662 cmd/dockerd: fix non-constant format string in call (govet)
cmd/dockerd/required.go:17:24: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Errorf (govet)
            return errors.Errorf("\n" + strings.TrimRight(cmd.UsageString(), "\n"))
                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06bfe8bab3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
ec8ff9cbbd libnetwork: fix non-constant format string in call (govet)
libnetwork/controller.go:1054:32: printf: non-constant format string in call to github.com/docker/docker/libnetwork/types.NotFoundErrorf (govet)
                return types.NotFoundErrorf(err.Error())
                                            ^
    libnetwork/controller.go:1073:32: printf: non-constant format string in call to github.com/docker/docker/libnetwork/types.NotFoundErrorf (govet)
                return types.NotFoundErrorf(err.Error())
                                            ^
    libnetwork/sandbox_externalkey_unix.go:113:21: printf: non-constant format string in call to fmt.Errorf (govet)
            return fmt.Errorf(string(buf[0:n]))
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6008c42ca2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
e5bc931f17 integration-cli: fix non-constant format string in call (govet)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b79a4696ee)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
6c3b1992d4 integration-cli: DockerSwarmSuite: rm redundant Fprintf, handle errors
Also fix some unhandled errors.

    integration-cli/docker_cli_swarm_test.go:697:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"failed to add veth pair: `+err.Error()+`"}`)
                               ^
    integration-cli/docker_cli_swarm_test.go:731:18: printf: non-constant format string in call to fmt.Fprintf (govet)
            fmt.Fprintf(w, `{"LocalDefaultAddressSpace":"`+lAS+`", "GlobalDefaultAddressSpace": "`+gAS+`"}`)
                           ^
    integration-cli/docker_cli_swarm_test.go:742:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"Unknown address space in pool request: `+poolRequest.AddressSpace+`"}`)
                               ^
    integration-cli/docker_cli_swarm_test.go:746:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"PoolID":"`+poolID+`", "Pool":"`+pool+`"}`)
                               ^
    integration-cli/docker_cli_swarm_test.go:763:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Address":"`+gw+`"}`)
                               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bbacbec26)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:56 +02:00
Sebastiaan van Stijn
83171f6b5a integration-cli: DockerNetworkSuite: rm redundant Fprintf, handle errors
Also rename some variables that shadowed imports, and fix some
unhandled errors.

    integration-cli/docker_cli_network_unix_test.go:102:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"failed to add veth pair: `+err.Error()+`"}`)
                               ^
    integration-cli/docker_cli_network_unix_test.go:136:18: printf: non-constant format string in call to fmt.Fprintf (govet)
            fmt.Fprintf(w, `{"LocalDefaultAddressSpace":"`+lAS+`", "GlobalDefaultAddressSpace": "`+gAS+`"}`)
                           ^
    integration-cli/docker_cli_network_unix_test.go:147:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"Unknown address space in pool request: `+poolRequest.AddressSpace+`"}`)
                               ^
    integration-cli/docker_cli_network_unix_test.go:151:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"PoolID":"`+poolID+`", "Pool":"`+pool+`"}`)
                               ^
    integration-cli/docker_cli_network_unix_test.go:168:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Address":"`+gw+`"}`)
                               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3ca38f0b5e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:55 +02:00
Sebastiaan van Stijn
4cfa21bb92 integration-cli: use erors.New() instead of fmt.Errorf
integration-cli/benchmark_test.go:49:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:62:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:68:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:73:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:78:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:84:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:94:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2b7a687554)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:55 +02:00
Sebastiaan van Stijn
28bdd819d0 libnetwork: TestDNSOptions: remove redundant skip check
libnetwork/sandbox_dns_unix_test.go:17:13: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
        skip.If(t, runtime.GOOS == "windows", "test only works on linux")
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7b36f8953)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:55 +02:00
Sebastiaan van Stijn
cb2e98cfbf integration-cli: remove redundant platform checks
This condition was added in 0215a62d5b, which
removed pkg/homedir as abstraction, but didn't consider that this test
is currently only ran on Unix.

    integration-cli/docker_cli_run_unix_test.go:254:5: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
        if runtime.GOOS == "windows" {
           ^
    integration-cli/docker_cli_run_unix_test.go:338:5: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
        if runtime.GOOS == "windows" {
           ^

Added a TODO, because this functionality should also be tested on Windows,
probably as part of tests in docker/cli instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bd7835cb6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:53 +02:00
Paweł Gronowski
3ab5c7d003 Merge pull request #48383 from vvoland/48382-27.x
[27.x backport] Dockerfile/vendor: update containerd to v1.7.21
2024-08-27 16:00:14 +02:00
Paweł Gronowski
875e8aeef2 vendor: github.com/containerd/containerd v1.7.21
full diff: https://github.com/containerd/containerd/compare/v1.7.20...v1.7.21

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a88efd7359)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-27 14:07:56 +02:00
Paweł Gronowski
1900e4d8eb Dockerfile: update containerd binary to v1.7.21 (static binaries and CI only)
Update the containerd binary that's used in CI and static binaries

- full diff: https://github.com/containerd/containerd/compare/v1.7.20...v1.7.21
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.21

```markdown changelog
Update containerd (static binaries only) to [v1.7.21](https://github.com/containerd/containerd/releases/tag/v1.7.21)
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit de4fc1c927)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-27 13:43:36 +02:00
Sebastiaan van Stijn
cd7746d30b Merge pull request #48380 from vvoland/48374-27.x
[27.x backport] c8d/pull: Keep the replaced image as dangling
2024-08-27 13:08:51 +02:00
Paweł Gronowski
2a13a384b8 Merge pull request #48376 from vvoland/48293-27.x
[27.x backport] c8d/load: Multi-platform fixes
2024-08-27 11:39:06 +02:00
Sebastiaan van Stijn
9fd71f5d0e Merge pull request #48378 from corhere/backport-27.x/dockerd-manpage
[27.x backport] Move dockerd man page back from docker/cli
2024-08-27 10:43:54 +02:00
Paweł Gronowski
ecd2b6ff09 c8d/image: Add hostPlatformMatcher
Subset of 842c5c584e which only adds the
`hostPlatformMatcher` method.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-27 10:17:50 +02:00
Cory Snider
d5b03423d1 man: support bringing your own go-md2man
Set the GO_MD2MAN make variable to elide building go-md2man from
vendored sources and use the specified command instead.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit edfde78355)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Cory Snider
56c5c23114 man: build dockerd man pages using make
Vendor the go-md2man tool used to generate the man pages so that the
only dependency is a Go toolchain.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 05d7008419)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Grace Choi
77b2eb5734 Removed all mentions of "please" from docs and messages
Signed-off-by: Grace Choi <gracechoi@utexas.edu>
Signed-off-by: Pranjal Rai <pranjalrai@utexas.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b4cee5c3ee)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
David Karlsson
805becdc7e docs: add default-network-opt daemon option
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
(cherry picked from commit f1ec84314d)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
a5828ac742 docs: remove devicemapper
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 23812190c3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Kir Kolyshkin
f7246a0e2c man/dockerd.8: assorted formatting fixes
Mostly, this makes sure that literals (such as true, false, host,
private, examples of options usage etc.) are typeset in bold, except for
filenames, which are typeset in italic.

While at it,
 - remove some default values from synopsis as it should not
   be there;
 - fix man pages references (page name in bold, volume number in
   regular).

This is not a complete fix, but a step in the right direction.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 690d166632)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Kir Kolyshkin
f110e779f6 man/dockerd.8: escape asterisks and underscores
1. Escape asterisks and underscores, that have special meaning in
   Markdown. While most markdown processors are smart enough to
   distinguish whether it's a literal * or _ or a formatting directive,
   escaping makes things more explicit.

2. Fix using wrong level of headings in some dm options (most are ####,
   but some were #####).

3. Do not use sub-heading for examples in some dm options (this is how
   it's done in the rest of the man page).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 374b779dd1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
edbcbf8da7 docs: update dockerd usage output for new proxy-options
Adds documentation for the options that were added in
427c7cc5f8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 386d0c0fbc)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Ashly Mathew
b7cc16b0b0 Fix styling of arguments
Signed-off-by: Ashly Mathew <ashlymathew93@gmail.com>
(cherry picked from commit 54971ac807)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Luis Henrique Mulinari
ecbc4f05bb Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in moby/moby#44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
(cherry picked from commit a8b8f9b288)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
43298ad298 docs: remove documentation about deprecated cluster-store
This removes documentation related to legacy overlay networks using
an external k/v store.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68e9223289)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Anca Iordache
8cc7f26f56 Document --validate daemon option
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
(cherry picked from commit 6c702167bf)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
255eaa6647 Update man-page source MarkDown to work with go-md2man v2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af45195a21)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Akihiro Suda
ee27f4cd7f docs: update for cgroup v2 and rootless
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 562a6d2b13)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Rob Gulewich
c1d3c952e7 docker run: specify cgroup namespace mode with --cgroupns
Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
(cherry picked from commit 7cf2132655)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Lukas Heeren
fc9029a2e2 daemon: document --max-download-attempts option
update docs based on PR 39949

Signed-off-by: Lukas Heeren <lukas-heeren@hotmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1cbcd5d47a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
taiji-tech
115b10a467 Update document links and title.
Signed-off-by: taiji-tech <csuhqg@foxmail.com>
(cherry picked from commit 3cfa74724c)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
selansen
488872bcb4 Allow user to specify default address pools for docker networks This is separate commit for CLI files to address PR 36054
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit 462f38bd8b)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
f623030fac Update docs and completion-scripts for deprecated features
- the `--disable-legacy-registry` daemon flag was removed
- duplicate keys with conflicting values for engine labels
  now produce an error instead of a warning.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 13ff896b38)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Renaud Gaubert
ac7032bff9 Added docs for dockerd
Signed-off-by: Renaud Gaubert <renaud.gaubert@gmail.com>
(cherry picked from commit f3c3b05b50)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Abdur Rehman
161006302f fix a number of minor typos
Fix 19 typos, grammatical errors and duplicated words.

These fixes have minimal impact on the code as these are either in the
doc files or in comments inside the code files.

Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
(cherry picked from commit 20f8455562)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Kir Kolyshkin
a6de17d230 Introduce/document new IPC modes
This builds (and depends) on https://github.com/moby/moby/pull/34087

Version 2:
 - remove --ipc argument validation (it is now done by daemon)
 - add/document 'none' value
 - docs/reference/run.md: add a table with better modes description
 - dockerd(8) typesetting fixes

Version 3:
 - remove ipc mode tests from cli/command/container/opts_test.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c23d4b017a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Aleksa Sarai
6513e80c84 docs: add documentation for dm.libdm_log_level
This is a new option added specifically to allow for debugging of bugs
in Docker's storage drivers or libdm itself.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit 25baee8ab9)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Cory Snider
3d917f3fd6 Restore dockerd man page
Prepare to move the dockerd man page back to this repository from
docker/cli, retaining history.

This partially reverts commit b5579a4ce3.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 7d3f09a9c3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Paweł Gronowski
e854a5c201 c8d/pull: Replace pointer to interface with interface
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 0afe684685)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:47 +02:00
Paweł Gronowski
ea58dab95e c8d/pull: Keep the replaced image as dangling
With graphdrivers, the old image was still kept as a dangling image.
Keep the same behavior with containerd.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit db40a6132b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:43 +02:00
Paweł Gronowski
0a38589add c8d/load: Only unpack host platform images
When loading a multi-platform image, it's not necessary to unpack all
platforms, especially those which have a completely different OS.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6ebe6a7353)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:10 +02:00
Paweł Gronowski
7c069d3021 c8d/load: Don't fail whole operation if unpack failed
Log the error to the progress output instead.
The image is still loaded into the content store and image service even
if the unpacking failed, so don't error out the whole operation to avoid
missing the load events for other image names loaded from the same
archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 728894b7d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:08 +02:00
Sebastiaan van Stijn
b27de4ef16 Merge pull request #48369 from vvoland/48367-27.x
[27.x backport] c8d/list: Fix race condition when traversing containers
2024-08-26 13:04:26 +02:00
Paweł Gronowski
5002faebe8 integration/TestAPIImagesListManifests: Check Containers
Verify that the ImageData.Containers contains the ID of the container
using that image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 55f693e7b7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 10:41:22 +02:00
Paweł Gronowski
a15a309832 c8d/list: Update benchmark to also have containers
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 29a2f6d339)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 10:41:21 +02:00
Paweł Gronowski
fd5cede287 c8d/list: Fix race condition when traversing containers
Use a regular for loop instead of ApplyAll which spawns a separate
goroutine for each separate container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a5d75f6d27)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 10:41:19 +02:00
Sebastiaan van Stijn
c050bc3739 Merge pull request #48364 from austinvazquez/cherry-pick-3cd28504dec017ef38f1a7abc141a493b9319757-to-27.x
[27.x backport] govulncheck to report known vulnerabilities
2024-08-23 22:18:00 +02:00
Sebastiaan van Stijn
de22458d0f Merge pull request #48363 from austinvazquez/cherry-pick-c4ba1f47187fb77646d906c512084a185036fd51-to-27.x
[27.x backport] Dockerfile: update xx to v1.5.0
2024-08-23 22:17:23 +02:00
CrazyMax
65c4e49aff govulncheck to report known vulnerabilities
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 3cd28504de)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-08-22 13:57:15 +00:00
Sebastiaan van Stijn
7ebb277873 Dockerfile: update xx to v1.5.0
full diff: https://github.com/tonistiigi/xx/compare/v1.4.0...v1.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c4ba1f4718)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-08-22 13:46:45 +00:00
Paweł Gronowski
9942d656ba Merge pull request #48346 from vvoland/47526-27.x
[27.x backport] c8d: Multi-platform image list
2024-08-16 18:47:36 +02:00
Paweł Gronowski
ad5eb875d4 c8d/list: Don't require opts.ContainerCount for manifest containers
The `GET /images/json` requires an optional `container-count` parameter
which set the `Containers` property of in the ImageSummary to a number
of containers using that image.

This was also propagated to the new manifest list property which
includes a list of all the container IDs that are using this specific
image manifest.

Disconnect the `ImageData.Containers` property from this option and
always include it by default without an explicit opt-in.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b93cf37dcd)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:44 +02:00
Paweł Gronowski
3d845e0e8c c8d/list: Add test for total and content size
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 495fab8e66)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:42 +02:00
Paweł Gronowski
3563a707d0 c8d/list: Fix Total size calculation
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 469c2ef3ec)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:41 +02:00
Paweł Gronowski
89757f83ff api/list: Expose manifests
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).

There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 050afe1e1a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:40 +02:00
Paweł Gronowski
bb2fec6425 api: Bump default version to 1.47
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 85e9102dc9)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:39 +02:00
Paweł Gronowski
0f8fcec1d9 swagger: Disable ImageSummary model generation
Our version of go-swagger doesn't handle the `omitempty` correctly for
the new field.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit efb3c50799)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:37 +02:00
Paweł Gronowski
1a342adda7 Merge pull request #48344 from vvoland/48324-27.x
[27.x backport] fix deprecation comments, and update some godoc
2024-08-16 16:59:01 +02:00
Sebastiaan van Stijn
1ec5e86154 api/types/registry: fix godoc, and add some doc-links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e77e543b58)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 12:56:26 +02:00
Sebastiaan van Stijn
62f32e9a97 plugin: fix deprecation comments
These must have a whitespace before them, otherwise they are ignored.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 218c08b283)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 12:56:24 +02:00
Sebastiaan van Stijn
68484b732a Merge pull request #48341 from tonistiigi/v27-update-buildkit-v0.15.2
[27.x] vendor: update buildkit to v0.15.2
2024-08-15 21:50:09 +02:00
Tonis Tiigi
830c76c6f2 vendor: update buildkit to v0.15.2
Also brings in fix for moby/buildkit#5242

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit c459986399)
2024-08-15 18:42:29 +03:00
Sebastiaan van Stijn
8f969bf61c Merge pull request #48332 from vvoland/48281-27.x
[27.x backport] Migrate per-endpoint sysctls until 28.0.0
2024-08-15 10:39:38 +02:00
Sebastiaan van Stijn
290663ede5 Merge pull request #48333 from vvoland/48081-27.x
[27.x backport] do another run of gofumpt
2024-08-15 10:16:52 +02:00
Sebastiaan van Stijn
354bf75675 libcontainerd: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 080a8e1b6b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:42 +02:00
Sebastiaan van Stijn
4ab7d90669 pkg/plugins: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 56fa45773f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:41 +02:00
Sebastiaan van Stijn
c11b2d9c7d pkg/archive: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0e2d40c24a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:40 +02:00
Sebastiaan van Stijn
ccdc79d55a libnetwork: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 84e43da752)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:39 +02:00
Sebastiaan van Stijn
35b1a30028 layer: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bb1b766ddb)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:37 +02:00
Sebastiaan van Stijn
9f63aa7435 internal: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07469b4509)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:36 +02:00
Sebastiaan van Stijn
4d16ac993e integration: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e50a96a78)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:34 +02:00
Sebastiaan van Stijn
6d5266a650 integration-cli: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c3ac7fee26)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:32 +02:00
Sebastiaan van Stijn
4084dac566 daemon: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e765dd90ee)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:30 +02:00
Sebastiaan van Stijn
c36ab4c2ca daemon/containerd: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 181101c4a8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:29 +02:00
Sebastiaan van Stijn
904867593b daemon/config: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 46b0102da4)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:11 +02:00
Sebastiaan van Stijn
72876770d0 builder: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a89963f1e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:10 +02:00
Sebastiaan van Stijn
e8109ee4da api/types: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8768145519)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:08 +02:00
Sebastiaan van Stijn
ed65e1224e api/server: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92346bcec6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:06 +02:00
Rob Murray
d54aff9312 API 1.46: end per-interface sysctl migration in major release
Rather than in API 1.47.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a86a9e3aa4)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:29:23 +02:00
Rob Murray
8f12906274 Migrate per-endpoint sysctls until 28.0.0
Commit 0071832226 introduced
per-endpoint sysctls, and migration to them from the top-level
'--sysctl' option.

The migration was intended to be short-term, disabled in the
next major release and code was added to check for the next
API version. But now, the API version will be bumped in a
minor release - this breaking change needs to wait until the
next major release, and we don't yet know the API version
number for that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 17adc1478b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:29:19 +02:00
Albin Kerouanton
5955778fe3 Merge pull request #48326 from robmry/backport-27.2/64bit_iprange_fix
[27.2 backport] Allow 64-bit --ip-range
2024-08-14 09:24:04 +02:00
Rob Murray
c53be2f3d5 Allow --ip-range ending on a 64-bit boundary
When defaultipam.newPoolData is asked for a pool of 64-bits
or more, it ends up with an overflowed u64 - so, it just
subtracts one to get a nearly-big-enough range (for a 64-bit
subnet).

When defaultipam.getAddress is called with an ipr (sub-pool
range), the range it calls bitmask.SetAnyInRange with is
exclusive of end. So, its end param can't be MaxUint64,
because that's the max value for the top end of the range
and, when checking the range, SetAnyInRange fails.

When fixed-cidr-v6 behaves more like fixed-cidr, it will ask
for a 64-bit range if that's what fixed-cidr-v6 needs. So,
it hits the bug when allocating an address for, for example:

  docker network create --ipv6 --subnet fddd::/64 --ip-range fddd::/64 b46

The additional check for "ipr == base" avoids the issue in
this case, by ignoring the ipr/sub-pool range if ipr is the
same as the pool itself (not really a sub-pool).

But, it still fails when ipr!=base. For example:

  docker network create --ipv6 --subnet fddd::/56 --ip-range fddd::/64 b46

So, also subtract one from 'end' if it's going to hit the max
value allowed by the Bitmap.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 496b457ad8)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-08-13 15:40:48 +01:00
Sebastiaan van Stijn
f9522e5e96 Merge pull request #48315 from vvoland/48169-27.x
[27.x backport] rm regexp use
2024-08-10 15:59:46 +02:00
Sebastiaan van Stijn
a037b7250c Merge pull request #48314 from vvoland/48275-27.x
[27.x backport] api/swagger: fix x-nullable for SystemInfo.Containerd (api v1.46)
2024-08-09 17:18:14 +02:00
Kir Kolyshkin
fc0150b962 daemon/containerd: rm use of regexp
Replace the regexp check with a function.

Keep the use of regexp.QuoteMeta.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 508939821b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:08:05 +02:00
Kir Kolyshkin
73c01d0b6a image/v1: rm regexp use
Replace the regexp checking ID validity with a for loop.

The benefits are:
 - faster (up to 10x faster with less allocations);
 - no init overhead to compile the regexp.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b66d4b567a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:08:04 +02:00
Kir Kolyshkin
c93fe4a27d layer: rm regexp use
Replace the regexp checking ID validity with a function. The benefits
are:

 - function is faster (up to 10x faster with less allocations);
 - no init overhead to compile the regexp;

Add a test case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1c0dc8a94f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:08:02 +02:00
Sebastiaan van Stijn
31459c8268 docs/api: swagger: fix x-nullable for SystemInfo.Containerd (api v1.46)
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.

Rendering the docs (`make swagger-docs`) showed a warning:

> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7dec1c67a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:00:24 +02:00
Sebastiaan van Stijn
35d430c62e api/swagger: fix x-nullable for SystemInfo.Containerd
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.

Rendering the docs (`make swagger-docs`) showed a warning:

> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 66b5b8bfa8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:00:21 +02:00
Sebastiaan van Stijn
f5fa0908ef Merge pull request #48308 from thaJeztah/27.x_backport_migrate_userns
[27.x backport] migrate to github.com/moby/sys/userns
2024-08-08 12:34:48 +02:00
Sebastiaan van Stijn
a17f5d4f10 Merge pull request #48294 from austinvazquez/cherry-pick-2b5ffa0b63c76e8bb4ebb253d7e4db5c7af918c0-to-27.x
[27.x backport] gha: set permissions to read-only by default
2024-08-08 11:59:30 +02:00
Sebastiaan van Stijn
80a59c2f1a migrate to github.com/moby/sys/userns
Commit 2ce811e632 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b0ef10a9a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 11:04:52 +02:00
Tianon Gravi
95db7055cc Merge pull request #48301 from vvoland/48300-27.x
[27.x backport] update to go1.21.13
2024-08-07 16:16:41 -07:00
Paweł Gronowski
e7fe276c00 update to go1.21.13
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.13+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.12...go1.21.13

go1.21.13 (released 2024-08-06) includes fixes to the go command, the
covdata command, and the bytes package. See the Go 1.21.13 milestone on
our issue tracker for details.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b24c2e95e5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-07 11:45:54 +02:00
Sebastiaan van Stijn
e8cd19e810 gha: set permissions to read-only by default
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2b5ffa0b63)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-08-06 15:55:59 +00:00
Paweł Gronowski
45d37a0ca9 Merge pull request #48246 from vvoland/48239-27.x
[27.1 backport] vendor: update buildkit to v0.15.1
2024-07-26 18:11:02 +02:00
CrazyMax
e0c52e0ba6 vendor: update buildkit to v0.15.1
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 1baf8f9e60)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-26 16:29:21 +02:00
Paweł Gronowski
b9be986e35 Merge pull request #48245 from thaJeztah/27.1_backport_buildkit_fix_grpc_control_api_sizes
[27.1 backport] api/server/router/grpc: NewRouter: set correct MaxRecvMsgSize, MaxSendMsgSize
2024-07-26 16:10:33 +02:00
Sebastiaan van Stijn
efb67b16b0 api/server/router/grpc: NewRouter: set correct MaxRecvMsgSize, MaxSendMsgSize
[buildkit@29b4b1a537][1] applied changes to `buildkitd` to set the correct
defaults, which should be 16MB, but used the library defaults. Without that
change, builds using large Dockerfiles would fail with a `ResourceExhausted`
error;

    => [internal] load build definition from Dockerfile
     => => transferring dockerfile: 896.44kB
    ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (44865299 vs. 16777216)

However those changes were applied to the `buildkitd` code, which is the
daemon when running BuildKit standalone (or in a container through the
`container` driver). When running a build with the BuildKit builder compiled
into the Docker Engine, that code is not used, so the BuildKit changes did
not fix the issue.

This patch applies the same changes as were made in [buildkit@29b4b1a537][1]
to the gRPC endpoint provided by the dockerd daemon.

[1]: 29b4b1a537

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cdbfae1d3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-26 14:28:27 +02:00
Paweł Gronowski
741e23b913 Merge pull request #48238 from thaJeztah/27.1_backport_migrate_userns
[27.1 backport] migrate to github.com/moby/sys/user/userns
2024-07-26 10:05:53 +02:00
Sebastiaan van Stijn
f96e26f68d migrate to github.com/moby/sys/user/userns
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2ce811e632)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 14:15:19 +02:00
Sebastiaan van Stijn
78b59867f2 vendor: github.com/moby/sys/user v0.2.0
full diff: https://github.com/moby/sys/compare/user/v0.1.0...user/v0.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 91dfc326cf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 14:15:14 +02:00
Akihiro Suda
7d861e889c Merge pull request #48235 from thaJeztah/27.1_backport_vendor_flock
[27.1 backport] vendor: github.com/gofrs/flock v0.12.1
2024-07-25 17:49:46 +09:00
Sebastiaan van Stijn
17e1108324 Merge pull request #48228 from vvoland/47789-27.0
[27.0 backport] hack: explicitly control enabling the journald logging driver
2024-07-25 01:39:49 +02:00
Sebastiaan van Stijn
018137b01a vendor: github.com/gofrs/flock v0.12.1
- fix: missing read-write flag in reopenFDOnError
  fixes a regression that could result in a `ERROR: bad file descriptor`.

b659e1e00a
introduced a regression where `f.flag` would not be in read-write mode
[1]  but read-only [2] which breaks people using NFS protocol.

[1]: b659e1e00a (diff-87c2c4fe0fb43f4b38b4bee45c1b54cfb694c61e311f93b369caa44f6c1323ffR192)
[2]: b659e1e00a (diff-22145325dded38eb5288ed3321a113d8260ccc70747ee04d4551bfd2fba975fdR69)

full diff: https://github.com/gofrs/flock/compare/v0.12.0...v0.12.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e2ccf8046)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 00:45:40 +02:00
Sebastiaan van Stijn
650e06ac75 vendor: golang.org/x/sys v0.22.0
full diff: https://github.com/golang/sys/compare/v0.21.0...v0.22.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 077b32ac4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 00:45:40 +02:00
Akihiro Suda
7f5494dc97 Merge pull request #48233 from AkihiroSuda/cherrypick-48216-27
[27.x backport] dockerd-rootless-setuptool.sh: move RootlessKit smoke test
2024-07-25 02:53:28 +09:00
Akihiro Suda
bfe5339c7e dockerd-rootless-setuptool.sh: move RootlessKit smoke test
`dockerd-rootless-setuptool.sh check` now skips the smoke test for
running RootlessKit.

Fix docker/docker-install issue 417

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e2237240f5)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-07-25 00:37:02 +09:00
Paweł Gronowski
a4046c4ca0 Merge pull request #48221 from thaJeztah/27.1_backport_readme_commercial_support
[27.1 backport] README: replace obsolete Docker EE mention
2024-07-24 11:48:07 +02:00
William Hubbs
99471ac2fe hack: explicitly control enabling the journald logging driver
Without this, the dependency on systemd is said to be "automagic", which
can lead to breakage, for example, if a binary package of docker is
built on a system that has systemd installed then installed on a system
that does not have systemd installed.

for example: https://bugs.gentoo.org/914076

Signed-off-by: William Hubbs <w.d.hubbs@gmail.com>
(cherry picked from commit 499c842c52)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-24 10:56:45 +02:00
Cory Snider
b9b43b3bdf README: replace obsolete Docker EE mention
Docker EE is no more. Point users looking for commercial support at the
currently-maintained commercial products based on the Moby project:
Docker Desktop and Mirantis Container Runtime.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit b37c983d31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-23 22:05:49 +02:00
2038 changed files with 166771 additions and 91986 deletions

View File

@@ -19,11 +19,14 @@ Please provide the following information:
**- How to verify it**
**- Description for the changelog**
**- Human readable description for the release notes**
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog.
It must be placed inside the below triple backticks section:
It must be placed inside the below triple backticks section.
NOTE: Only fill this section if changes introduced in this PR are user-facing.
The PR must have a relevant impact/ label.
-->
```markdown changelog

View File

@@ -3,15 +3,25 @@ name: .dco
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
env:
ALPINE_VERSION: 3.16
ALPINE_VERSION: "3.20"
jobs:
run:
runs-on: ubuntu-20.04
timeout-minutes: 10 # guardrails timeout for the whole job
steps:
-
name: Checkout
@@ -39,10 +49,12 @@ jobs:
name: Validate
run: |
docker run --rm \
-v "$(pwd):/workspace" \
--quiet \
-v ./:/workspace \
-w /workspace \
-e VALIDATE_REPO \
-e VALIDATE_BRANCH \
alpine:${{ env.ALPINE_VERSION }} sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && cd /workspace && hack/validate/dco'
alpine:${{ env.ALPINE_VERSION }} sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && hack/validate/dco'
env:
VALIDATE_REPO: ${{ github.server_url }}/${{ github.repository }}.git
VALIDATE_BRANCH: ${{ steps.base-ref.outputs.result }}

View File

@@ -3,6 +3,15 @@ name: .test-prepare
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
outputs:
@@ -13,6 +22,7 @@ on:
jobs:
run:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
outputs:
matrix: ${{ steps.set.outputs.matrix }}
steps:

View File

@@ -3,6 +3,15 @@ name: .test
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
inputs:
@@ -12,7 +21,7 @@ on:
default: "graphdriver"
env:
GO_VERSION: "1.21.12"
GO_VERSION: "1.22.12"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
ITG_CLI_MATRIX_SIZE: 6
@@ -25,8 +34,8 @@ env:
jobs:
unit:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
steps:
-
name: Checkout
@@ -43,7 +52,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -80,8 +89,8 @@ jobs:
unit-report:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- unit
@@ -108,8 +117,8 @@ jobs:
docker-py:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
steps:
-
name: Checkout
@@ -129,7 +138,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -165,8 +174,8 @@ jobs:
integration-flaky:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
steps:
-
name: Checkout
@@ -183,7 +192,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -197,8 +206,8 @@ jobs:
integration:
runs-on: ${{ matrix.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
@@ -241,7 +250,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -295,8 +304,8 @@ jobs:
integration-report:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- integration
@@ -324,6 +333,7 @@ jobs:
integration-cli-prepare:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
@@ -359,8 +369,8 @@ jobs:
integration-cli:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
needs:
- integration-cli-prepare
strategy:
@@ -386,7 +396,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -439,8 +449,8 @@ jobs:
integration-cli-report:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- integration-cli

View File

@@ -3,6 +3,16 @@ name: .windows
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
inputs:
@@ -19,7 +29,7 @@ on:
default: false
env:
GO_VERSION: "1.21.12"
GO_VERSION: "1.22.12"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
WINDOWS_BASE_IMAGE: mcr.microsoft.com/windows/servercore
@@ -33,6 +43,7 @@ env:
jobs:
build:
runs-on: ${{ inputs.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -112,7 +123,7 @@ jobs:
unit-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120
timeout-minutes: 120 # guardrails timeout for the whole job
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -193,7 +204,8 @@ jobs:
retention-days: 1
unit-test-report:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
if: always()
needs:
- unit-test
@@ -219,7 +231,8 @@ jobs:
find /tmp/artifacts -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
integration-test-prepare:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
@@ -253,8 +266,8 @@ jobs:
integration-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
timeout-minutes: 120
needs:
- build
- integration-test-prepare
@@ -512,7 +525,8 @@ jobs:
retention-days: 1
integration-test-report:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
if: always()
needs:

276
.github/workflows/arm64.yml vendored Normal file
View File

@@ -0,0 +1,276 @@
name: arm64
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
env:
GO_VERSION: "1.22.12"
TESTSTAT_VERSION: v0.1.25
DESTDIR: ./build
SETUP_BUILDX_VERSION: edge
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
DOCKER_EXPERIMENTAL: 1
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build:
runs-on: ubuntu-22.04-arm
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
strategy:
fail-fast: false
matrix:
target:
- binary
- dynbinary
steps:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v6
with:
targets: ${{ matrix.target }}
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
build-dev:
runs-on: ubuntu-22.04-arm
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
steps:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
*.cache-from=type=gha,scope=dev-arm64
*.cache-to=type=gha,scope=dev-arm64,mode=max
*.output=type=cacheonly
test-unit:
runs-on: ubuntu-22.04-arm
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev-arm64
-
name: Test
run: |
make -o build test-unit
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-unit-arm64-graphdriver
path: /tmp/reports/*
retention-days: 1
test-unit-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- test-unit
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: vendor.sum
-
name: Download reports
uses: actions/download-artifact@v4
with:
pattern: test-reports-unit-arm64-*
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
test-integration:
runs-on: ubuntu-22.04-arm
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev-arm64
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION_CLI: 1
TESTCOVERAGE: 1
-
name: Prepare reports
if: always()
run: |
reportsPath="/tmp/reports/arm64-graphdriver"
mkdir -p bundles $reportsPath
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
curl -sSLf localhost:16686/api/traces?service=integration-test-client > $reportsPath/jaeger-trace.json
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-integration-arm64-graphdriver
path: /tmp/reports/*
retention-days: 1
test-integration-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- test-integration
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: vendor.sum
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: test-reports-integration-arm64-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY

View File

@@ -1,5 +1,14 @@
name: bin-image
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,6 +19,7 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
tags:
- 'v*'
pull_request:
@@ -31,6 +41,7 @@ jobs:
prepare:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
outputs:
platforms: ${{ steps.platforms.outputs.matrix }}
steps:
@@ -48,7 +59,7 @@ jobs:
## push semver tag v23.0.0
# moby/moby-bin:23.0.0
# moby/moby-bin:latest
## push semver prelease tag v23.0.0-beta.1
## push semver prerelease tag v23.0.0-beta.1
# moby/moby-bin:23.0.0-beta.1
## push on master
# moby/moby-bin:master
@@ -83,6 +94,7 @@ jobs:
build:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
- prepare
@@ -92,16 +104,16 @@ jobs:
matrix:
platform: ${{ fromJson(needs.prepare.outputs.platforms) }}
steps:
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Download meta bake definition
uses: actions/download-artifact@v4
@@ -128,8 +140,9 @@ jobs:
-
name: Build
id: bake
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
source: .
files: |
./docker-bake.hcl
/tmp/bake-meta.json
@@ -157,6 +170,7 @@ jobs:
merge:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- build
if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') && github.event_name != 'pull_request' && github.repository == 'moby/moby'

View File

@@ -1,5 +1,14 @@
name: buildkit
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,10 +19,11 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
env:
GO_VERSION: "1.21.12"
GO_VERSION: "1.22.12"
DESTDIR: ./build
SETUP_BUILDX_VERSION: latest
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
@@ -24,12 +34,10 @@ jobs:
build:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
@@ -39,7 +47,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: binary
-
@@ -53,7 +61,7 @@ jobs:
test:
runs-on: ubuntu-20.04
timeout-minutes: 120
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- build
env:
@@ -93,6 +101,11 @@ jobs:
uses: actions/checkout@v4
with:
path: moby
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: BuildKit ref
run: |
@@ -130,8 +143,9 @@ jobs:
docker info
-
name: Build test image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
source: .
workdir: ./buildkit
targets: integration-tests
set: |

View File

@@ -1,5 +1,14 @@
name: ci
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,6 +19,7 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
env:
@@ -23,6 +33,7 @@ jobs:
build:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
strategy:
@@ -32,11 +43,6 @@ jobs:
- binary
- dynbinary
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
@@ -46,7 +52,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: ${{ matrix.target }}
-
@@ -59,7 +65,8 @@ jobs:
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
prepare-cross:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
outputs:
@@ -81,6 +88,7 @@ jobs:
cross:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
- prepare-cross
@@ -89,11 +97,6 @@ jobs:
matrix:
platform: ${{ fromJson(needs.prepare-cross.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Prepare
run: |
@@ -108,7 +111,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: all
set: |
@@ -121,3 +124,33 @@ jobs:
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
govulncheck:
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
permissions:
# required to write sarif report
security-events: write
# required to check out the repository
contents: read
steps:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Run
uses: docker/bake-action@v6
with:
targets: govulncheck
env:
GOVULNCHECK_FORMAT: sarif
-
name: Upload SARIF report
if: ${{ github.event_name != 'pull_request' && github.repository == 'moby/moby' }}
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ${{ env.DESTDIR }}/govulncheck.out

71
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,71 @@
name: codeql
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
tags:
- 'v*'
pull_request:
# The branches below must be a subset of the branches above
branches: ["master"]
schedule:
# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday)
# │ │ │ │ │
# │ │ │ │ │
# │ │ │ │ │
# * * * * *
- cron: '0 9 * * 4'
jobs:
codeql:
runs-on: ubuntu-24.04
timeout-minutes: 10
permissions:
actions: read
contents: read
security-events: write
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 2
# CodeQL 2.16.4's auto-build added support for multi-module repositories,
# and is trying to be smart by searching for modules in every directory,
# including vendor directories. If no module is found, it's creating one
# which is ... not what we want, so let's give it a "go.mod".
# see: https://github.com/docker/cli/pull/4944#issuecomment-2002034698
- name: Create go.mod
run: |
ln -s vendor.mod go.mod
ln -s vendor.sum go.sum
- name: Update Go
uses: actions/setup-go@v5
with:
go-version: "1.22.12"
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: go
- name: Autobuild
uses: github/codeql-action/autobuild@v3
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:go"

View File

@@ -1,5 +1,14 @@
name: test
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,10 +19,11 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
env:
GO_VERSION: "1.21.12"
GO_VERSION: "1.22.12"
GIT_PAGER: "cat"
PAGER: "cat"
SETUP_BUILDX_VERSION: latest
@@ -25,6 +35,7 @@ jobs:
build-dev:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
strategy:
@@ -40,9 +51,6 @@ jobs:
if [ "${{ matrix.mode }}" = "systemd" ]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
fi
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
@@ -52,7 +60,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -77,6 +85,7 @@ jobs:
validate-prepare:
runs-on: ubuntu-20.04
timeout-minutes: 10 # guardrails timeout for the whole job
needs:
- validate-dco
outputs:
@@ -98,7 +107,7 @@ jobs:
validate:
runs-on: ubuntu-20.04
timeout-minutes: 120
timeout-minutes: 30 # guardrails timeout for the whole job
needs:
- validate-prepare
- build-dev
@@ -124,7 +133,7 @@ jobs:
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -136,6 +145,7 @@ jobs:
smoke-prepare:
runs-on: ubuntu-20.04
timeout-minutes: 10 # guardrails timeout for the whole job
needs:
- validate-dco
outputs:
@@ -157,6 +167,7 @@ jobs:
smoke:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- smoke-prepare
strategy:
@@ -164,9 +175,6 @@ jobs:
matrix:
platform: ${{ fromJson(needs.smoke-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Prepare
run: |
@@ -184,7 +192,7 @@ jobs:
buildkitd-flags: --debug
-
name: Test
uses: docker/bake-action@v4
uses: docker/bake-action@v6
with:
targets: binary-smoketest
set: |

View File

@@ -1,12 +1,22 @@
name: validate-pr
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
pull_request:
types: [opened, edited, labeled, unlabeled]
types: [opened, edited, labeled, unlabeled, synchronize]
jobs:
check-area-label:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
steps:
- name: Missing `area/` label
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/') && !contains(join(github.event.pull_request.labels.*.name, ','), 'area/')
@@ -17,9 +27,10 @@ jobs:
run: exit 0
check-changelog:
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/')
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
env:
HAS_IMPACT_LABEL: ${{ contains(join(github.event.pull_request.labels.*.name, ','), 'impact/') }}
PR_BODY: |
${{ github.event.pull_request.body }}
steps:
@@ -31,15 +42,23 @@ jobs:
# Strip empty lines
desc=$(echo "$block" | awk NF)
if [ -z "$desc" ]; then
echo "::error::Changelog section is empty. Please provide a description for the changelog."
exit 1
fi
if [ "$HAS_IMPACT_LABEL" = "true" ]; then
if [ -z "$desc" ]; then
echo "::error::Changelog section is empty. Please provide a description for the changelog."
exit 1
fi
len=$(echo -n "$desc" | wc -c)
if [[ $len -le 6 ]]; then
echo "::error::Description looks too short: $desc"
exit 1
len=$(echo -n "$desc" | wc -c)
if [[ $len -le 6 ]]; then
echo "::error::Description looks too short: $desc"
exit 1
fi
else
if [ -n "$desc" ]; then
echo "::error::PR has a changelog description, but no changelog label"
echo "::error::Please add the relevant 'impact/' label to the PR or remove the changelog description"
exit 1
fi
fi
echo "This PR will be included in the release notes with the following note:"
@@ -47,6 +66,7 @@ jobs:
check-pr-branch:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
env:
PR_TITLE: ${{ github.event.pull_request.title }}
steps:

View File

@@ -1,5 +1,14 @@
name: windows-2019
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

View File

@@ -1,5 +1,14 @@
name: windows-2022
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,6 +19,7 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
jobs:

View File

@@ -6,6 +6,7 @@ linters:
- gosec
- gosimple
- govet
- forbidigo
- importas
- ineffassign
- misspell
@@ -31,6 +32,15 @@ linters-settings:
- "true" # some tests use this as expected output
- "false" # some tests use this as expected output
- "root" # for tests using "ls" output with files owned by "root:root"
forbidigo:
forbid:
- pkg: github.com/vishvananda/netlink$
p: ^netlink\.(Handle\.)?(AddrList|BridgeVlanList|ChainList|ClassList|ConntrackTableList|ConntrackDeleteFilter$|ConntrackDeleteFilters|DevLinkGetDeviceList|DevLinkGetAllPortList|DevlinkGetDeviceParams|FilterList|FouList|GenlFamilyList|GTPPDPList|LinkByName|LinkByAlias|LinkList|LinkSubscribeWithOptions|NeighList$|NeighProxyList|NeighListExecute|NeighSubscribeWithOptions|LinkGetProtinfo|QdiscList|RdmaLinkList|RdmaLinkByName|RdmaLinkDel|RouteList|RouteListFilteredIter|RuleListFiltered$|RouteSubscribeWithOptions|RuleList$|RuleListFiltered|SocketGet|SocketDiagTCPInfo|SocketDiagTCP|SocketDiagUDPInfo|SocketDiagUDP|UnixSocketDiagInfo|UnixSocketDiag|VDPAGetDevConfigList|VDPAGetDevList|VDPAGetMGMTDevList|XfrmPolicyList|XfrmStateList)
msg: Use internal nlwrap package for EINTR handling.
- pkg: github.com/docker/docker/internal/nlwrap$
p: ^nlwrap.Handle.(BridgeVlanList|ChainList|ClassList|ConntrackDeleteFilter$|DevLinkGetDeviceList|DevLinkGetAllPortList|DevlinkGetDeviceParams|FilterList|FouList|GenlFamilyList|GTPPDPList|LinkByAlias|LinkSubscribeWithOptions|NeighList$|NeighProxyList|NeighListExecute|NeighSubscribeWithOptions|LinkGetProtinfo|QdiscList|RdmaLinkList|RdmaLinkByName|RdmaLinkDel|RouteListFilteredIter|RuleListFiltered$|RouteSubscribeWithOptions|RuleList$|RuleListFiltered|SocketGet|SocketDiagTCPInfo|SocketDiagTCP|SocketDiagUDPInfo|SocketDiagUDP|UnixSocketDiagInfo|UnixSocketDiag|VDPAGetDevConfigList|VDPAGetDevList|VDPAGetMGMTDevList)
msg: Add a wrapper to nlwrap.Handle for EINTR handling and update the list in .golangci.yml.
analyze-types: true
importas:
# Do not allow unaliased imports of aliased packages.
no-unaliased: true
@@ -45,6 +55,11 @@ linters-settings:
govet:
check-shadowing: false
gosec:
excludes:
- G115 # FIXME temporarily suppress 'G115: integer overflow conversion': it produces many hits, some of which may be false positives, and need to be looked at; see https://github.com/moby/moby/issues/48358
depguard:
rules:
main:
@@ -57,10 +72,16 @@ linters-settings:
desc: Use "gotest.tools/v3/assert" instead
- pkg: "github.com/stretchr/testify/suite"
desc: Do not use
- pkg: github.com/containerd/containerd/errdefs
- pkg: "github.com/containerd/containerd/errdefs"
desc: The errdefs package has moved to a separate module, https://github.com/containerd/errdefs
- pkg: github.com/containerd/containerd/log
- pkg: "github.com/containerd/containerd/log"
desc: The logs package has moved to a separate module, https://github.com/containerd/log
- pkg: "github.com/containerd/containerd/pkg/userns"
desc: Use github.com/moby/sys/userns instead.
- pkg: "github.com/opencontainers/runc/libcontainer/userns"
desc: Use github.com/moby/sys/userns instead.
- pkg: "github.com/tonistiigi/fsutil"
desc: The fsutil module does not have a stable API, so we should not have a direct dependency unless necessary.
revive:
rules:
# FIXME make sure all packages have a description. Currently, there's many packages without.
@@ -78,7 +99,7 @@ issues:
# (unlike the "include" option), the "exclude" option does not take exclusion
# ID's.
#
# These exclusion patterns are copied from the default excluses at:
# These exclusion patterns are copied from the default excludes at:
# https://github.com/golangci/golangci-lint/blob/v1.46.2/pkg/config/issues.go#L10-L104
# EXC0001
@@ -129,11 +150,27 @@ issues:
path: "api/types/(volume|container)/"
linters:
- revive
# FIXME temporarily suppress these until we migrated these to internal.
- text: "SA1019: fileutils\\.GetTotalUsedFds"
linters:
- staticcheck
# FIXME temporarily suppress these (see https://github.com/gotestyourself/gotest.tools/issues/272)
- text: "SA1019: (assert|cmp|is)\\.ErrorType is deprecated"
linters:
- staticcheck
# FIXME temporarily suppress these until https://github.com/moby/moby/pull/49072 is merged, which removes their use.
- text: "SA1019: system\\.(FromStatT|Mkdev|Mknod|StatT)"
path: "pkg/archive/"
linters:
- staticcheck
# FIXME temporarily suppress these until they are moved internal to container/streams.
- text: "SA1019: ioutils\\.(ErrClosed|BytesPipe|NewBytesPipe)"
path: "container/stream/"
linters:
- staticcheck
- text: "ineffectual assignment to ctx"
source: "ctx[, ].*=.*\\(ctx[,)]"
linters:

View File

@@ -1,19 +1,19 @@
# syntax=docker/dockerfile:1.7
ARG GO_VERSION=1.21.12
ARG GO_VERSION=1.22.12
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
ARG XX_VERSION=1.4.0
ARG XX_VERSION=1.6.1
ARG VPNKIT_VERSION=0.5.0
ARG DOCKERCLI_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_VERSION=v27.0.2
ARG DOCKERCLI_VERSION=v27.5.0
# cli version used for integration-cli tests
ARG DOCKERCLI_INTEGRATION_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_INTEGRATION_VERSION=v17.06.2-ce
ARG BUILDX_VERSION=0.16.1
ARG COMPOSE_VERSION=v2.29.0
ARG BUILDX_VERSION=0.20.0
ARG COMPOSE_VERSION=v2.32.4
ARG SYSTEMD="false"
ARG DOCKER_STATIC=1
@@ -147,7 +147,7 @@ RUN git init . && git remote add origin "https://github.com/go-delve/delve.git"
# from the https://github.com/go-delve/delve repository.
# It can be used to run Docker with a possibility of
# attaching debugger to it.
ARG DELVE_VERSION=v1.21.1
ARG DELVE_VERSION=v1.23.0
RUN git fetch -q --depth 1 origin "${DELVE_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS delve-supported
@@ -196,7 +196,7 @@ RUN git init . && git remote add origin "https://github.com/containerd/container
# When updating the binary version you may also need to update the vendor
# version to pick up bug fixes or new APIs, however, usually the Go packages
# are built from a commit from the master branch.
ARG CONTAINERD_VERSION=v1.7.20
ARG CONTAINERD_VERSION=v1.7.25
RUN git fetch -q --depth 1 origin "${CONTAINERD_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerd-build
@@ -229,7 +229,7 @@ FROM binary-dummy AS containerd-windows
FROM containerd-${TARGETOS} AS containerd
FROM base AS golangci_lint
ARG GOLANGCI_LINT_VERSION=v1.59.1
ARG GOLANGCI_LINT_VERSION=v1.60.2
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/golangci/golangci-lint/cmd/golangci-lint@${GOLANGCI_LINT_VERSION}" \
@@ -287,7 +287,7 @@ RUN git init . && git remote add origin "https://github.com/opencontainers/runc.
# that is used. If you need to update runc, open a pull request in the containerd
# project first, and update both after that is merged. When updating RUNC_VERSION,
# consider updating runc in vendor.mod accordingly.
ARG RUNC_VERSION=v1.1.13
ARG RUNC_VERSION=v1.2.5
RUN git fetch -q --depth 1 origin "${RUNC_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS runc-build
@@ -356,7 +356,7 @@ FROM base AS rootlesskit-src
WORKDIR /usr/src/rootlesskit
RUN git init . && git remote add origin "https://github.com/rootless-containers/rootlesskit.git"
# When updating, also update vendor.mod and hack/dockerfile/install/rootlesskit.installer accordingly.
ARG ROOTLESSKIT_VERSION=v2.0.2
ARG ROOTLESSKIT_VERSION=v2.3.2
RUN git fetch -q --depth 1 origin "${ROOTLESSKIT_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS rootlesskit-build

View File

@@ -5,7 +5,7 @@
# This represents the bare minimum required to build and test Docker.
ARG GO_VERSION=1.21.12
ARG GO_VERSION=1.22.12
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"

View File

@@ -161,10 +161,10 @@ FROM ${WINDOWS_BASE_IMAGE}:${WINDOWS_BASE_IMAGE_TAG}
# Use PowerShell as the default shell
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG GO_VERSION=1.21.12
ARG GO_VERSION=1.22.12
ARG GOTESTSUM_VERSION=v1.8.2
ARG GOWINRES_VERSION=v0.3.1
ARG CONTAINERD_VERSION=v1.7.20
ARG CONTAINERD_VERSION=v1.7.25
# Environment variable notes:
# - GO_VERSION must be consistent with 'Dockerfile' used by Linux.

165
Jenkinsfile vendored
View File

@@ -1,165 +0,0 @@
#!groovy
pipeline {
agent none
options {
buildDiscarder(logRotator(daysToKeepStr: '30'))
timeout(time: 2, unit: 'HOURS')
timestamps()
}
parameters {
booleanParam(name: 'arm64', defaultValue: true, description: 'ARM (arm64) Build/Test')
booleanParam(name: 'dco', defaultValue: true, description: 'Run the DCO check')
}
environment {
DOCKER_BUILDKIT = '1'
DOCKER_EXPERIMENTAL = '1'
DOCKER_GRAPHDRIVER = 'overlay2'
CHECK_CONFIG_COMMIT = '33a3680e08d1007e72c3b3f1454f823d8e9948ee'
TESTDEBUG = '0'
TIMEOUT = '120m'
}
stages {
stage('pr-hack') {
when { changeRequest() }
steps {
script {
echo "Workaround for PR auto-cancel feature. Borrowed from https://issues.jenkins-ci.org/browse/JENKINS-43353"
def buildNumber = env.BUILD_NUMBER as int
if (buildNumber > 1) milestone(buildNumber - 1)
milestone(buildNumber)
}
}
}
stage('DCO-check') {
when {
beforeAgent true
expression { params.dco }
}
agent { label 'arm64 && ubuntu-2004' }
steps {
sh '''
docker run --rm \
-v "$WORKSPACE:/workspace" \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
alpine sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && cd /workspace && hack/validate/dco'
'''
}
}
stage('Build') {
parallel {
stage('arm64') {
when {
beforeAgent true
expression { params.arm64 }
}
agent { label 'arm64 && ubuntu-2004' }
environment {
TEST_SKIP_INTEGRATION_CLI = '1'
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh 'docker build --force-rm -t docker:${GIT_COMMIT} .'
}
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Integration tests") {
environment { TEST_SKIP_INTEGRATION_CLI = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=arm64-integration
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
}
}
}
}

View File

@@ -90,7 +90,7 @@ DOCKER_ENVS := \
# note: BINDDIR is supported for backwards-compatibility here
BIND_DIR := $(if $(BINDDIR),$(BINDDIR),$(if $(DOCKER_HOST),,bundles))
# DOCKER_MOUNT can be overriden, but use at your own risk!
# DOCKER_MOUNT can be overridden, but use at your own risk!
ifndef DOCKER_MOUNT
DOCKER_MOUNT := $(if $(BIND_DIR),-v "$(CURDIR)/$(BIND_DIR):/go/src/github.com/docker/docker/$(BIND_DIR)")
DOCKER_MOUNT := $(if $(DOCKER_BINDDIR_MOUNT_OPTS),$(DOCKER_MOUNT):$(DOCKER_BINDDIR_MOUNT_OPTS),$(DOCKER_MOUNT))

View File

@@ -32,7 +32,7 @@ New projects can be added if they fit with the community goals. Docker is commit
However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.
The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful.
The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.
The releases are supported by the maintainers, community and users, on a best efforts basis only. For customers who want enterprise or commercial support, [Docker Desktop](https://www.docker.com/products/docker-desktop/) and [Mirantis Container Runtime](https://www.mirantis.com/software/mirantis-container-runtime/) are the appropriate products for these use cases.
-----

View File

@@ -3,7 +3,7 @@ package api // import "github.com/docker/docker/api"
// Common constants for daemon and client.
const (
// DefaultVersion of the current REST API.
DefaultVersion = "1.46"
DefaultVersion = "1.47"
// MinSupportedAPIVersion is the minimum API version that can be supported
// by the API server, specified as "major.minor". Note that the daemon

View File

@@ -67,8 +67,8 @@ func (e versionUnsupportedError) InvalidParameter() {}
func (v VersionMiddleware) WrapHandler(handler func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error) func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
w.Header().Set("Server", fmt.Sprintf("Docker/%s (%s)", v.serverVersion, runtime.GOOS))
w.Header().Set("API-Version", v.defaultAPIVersion)
w.Header().Set("OSType", runtime.GOOS)
w.Header().Set("Api-Version", v.defaultAPIVersion)
w.Header().Set("Ostype", runtime.GOOS)
apiVersion := vars["version"]
if apiVersion == "" {

View File

@@ -141,6 +141,6 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
hdr := resp.Result().Header
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/1.2.3"))
assert.Check(t, is.Contains(hdr.Get("Server"), runtime.GOOS))
assert.Check(t, is.Equal(hdr.Get("API-Version"), api.DefaultVersion))
assert.Check(t, is.Equal(hdr.Get("OSType"), runtime.GOOS))
assert.Check(t, is.Equal(hdr.Get("Api-Version"), api.DefaultVersion))
assert.Check(t, is.Equal(hdr.Get("Ostype"), runtime.GOOS))
}

View File

@@ -339,8 +339,12 @@ type flusher interface {
Flush()
}
type nopFlusher struct{}
func (f *nopFlusher) Flush() {}
func wrapOutputBufferedUntilRequestRead(rc io.ReadCloser, out io.Writer) (io.ReadCloser, io.Writer) {
var fl flusher = &ioutils.NopFlusher{}
var fl flusher = &nopFlusher{}
if f, ok := out.(flusher); ok {
fl = f
}

View File

@@ -769,12 +769,14 @@ func handleSysctlBC(
netIfSysctl := fmt.Sprintf("net.%s.%s.IFNAME.%s=%s", spl[1], spl[2], spl[4], v)
// Find the EndpointConfig to migrate settings to, if not already found.
if ep == nil {
/* TODO(robmry) - apply this to the API version used in 28.0.0
// Per-endpoint sysctls were introduced in API version 1.46. Migration is
// needed, but refuse to do it automatically for newer versions of the API.
if versions.GreaterThan(version, "1.46") {
if versions.GreaterThan(version, "1.??") {
return "", fmt.Errorf("interface specific sysctl setting %q must be supplied using driver option '%s'",
k, netlabel.EndpointSysctls)
}
*/
var err error
ep, err = epConfigForNetMode(version, hostConfig.NetworkMode, netConfig)
if err != nil {
@@ -938,9 +940,11 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
if multiplexed && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = types.MediaTypeMultiplexedStream
}
fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
// FIXME(thaJeztah): we should not ignore errors here; see https://github.com/moby/moby/pull/48359#discussion_r1725562802
fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: %v\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n", contentType)
} else {
fmt.Fprintf(conn, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n")
// FIXME(thaJeztah): we should not ignore errors here; see https://github.com/moby/moby/pull/48359#discussion_r1725562802
fmt.Fprint(conn, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n")
}
go notifyClosed(ctx, conn, cancel)

View File

@@ -273,15 +273,17 @@ func TestHandleSysctlBC(t *testing.T) {
"net.ipv6.conf.all.disable_ipv6": "0",
},
},
/* TODO(robmry) - enable this test for the API version used in 28.0.0
{
name: "migration disabled for newer api",
apiVersion: "1.47",
apiVersion: "1.??",
networkMode: "mynet",
sysctls: map[string]string{
"net.ipv6.conf.eth0.accept_ra": "2",
},
expError: "must be supplied using driver option 'com.docker.network.endpoint.sysctls'",
},
*/
{
name: "only migrate eth0",
apiVersion: "1.46",

View File

@@ -1,5 +1,5 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.21
//go:build go1.22
package grpc // import "github.com/docker/docker/api/server/router/grpc"
@@ -9,13 +9,14 @@ import (
"os"
"strings"
"github.com/containerd/containerd/defaults"
"github.com/containerd/log"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/internal/otelutil"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/moby/buildkit/util/stack"
"github.com/moby/buildkit/util/tracing"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"go.opentelemetry.io/otel"
"golang.org/x/net/http2"
"google.golang.org/grpc"
)
@@ -28,10 +29,13 @@ type grpcRouter struct {
// NewRouter initializes a new grpc http router
func NewRouter(backends ...Backend) router.Router {
tp, _ := otelutil.NewTracerProvider(context.Background(), false)
opts := []grpc.ServerOption{
grpc.StatsHandler(tracing.ServerStatsHandler(otelgrpc.WithTracerProvider(otel.GetTracerProvider()))),
grpc.StatsHandler(tracing.ServerStatsHandler(otelgrpc.WithTracerProvider(tp))),
grpc.ChainUnaryInterceptor(unaryInterceptor, grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
grpc.MaxRecvMsgSize(defaults.DefaultMaxRecvMsgSize),
grpc.MaxSendMsgSize(defaults.DefaultMaxSendMsgSize),
}
r := &grpcRouter{

View File

@@ -142,7 +142,7 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
id, progressErr = ir.backend.ImportImage(ctx, tagRef, platform, comment, layerReader, r.Form["changes"])
if progressErr == nil {
output.Write(streamformatter.FormatStatus("", id.String()))
_, _ = output.Write(streamformatter.FormatStatus("", "%v", id.String()))
}
}
if progressErr != nil {
@@ -220,7 +220,6 @@ func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter
}
platform = p
}
}
if err := ir.backend.PushImage(ctx, ref, platform, metaHeaders, authConfig, output); err != nil {
@@ -424,10 +423,16 @@ func (ir *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
sharedSize = httputils.BoolValue(r, "shared-size")
}
var manifests bool
if versions.GreaterThanOrEqualTo(version, "1.47") {
manifests = httputils.BoolValue(r, "manifests")
}
images, err := ir.backend.Images(ctx, imagetypes.ListOptions{
All: httputils.BoolValue(r, "all"),
Filters: imageFilters,
SharedSize: sharedSize,
Manifests: manifests,
})
if err != nil {
return err

View File

@@ -75,13 +75,13 @@ func (e invalidRequestError) Error() string {
func (e invalidRequestError) InvalidParameter() {}
type ambigousResultsError string
type ambiguousResultsError string
func (e ambigousResultsError) Error() string {
func (e ambiguousResultsError) Error() string {
return "network " + string(e) + " is ambiguous"
}
func (ambigousResultsError) InvalidParameter() {}
func (ambiguousResultsError) InvalidParameter() {}
func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
@@ -182,7 +182,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
}
if len(listByFullName) > 1 {
return errors.Wrapf(ambigousResultsError(term), "%d matches found based on name", len(listByFullName))
return errors.Wrapf(ambiguousResultsError(term), "%d matches found based on name", len(listByFullName))
}
// Find based on partial ID, returns true only if no duplicates
@@ -192,7 +192,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
}
if len(listByPartialID) > 1 {
return errors.Wrapf(ambigousResultsError(term), "%d matches found based on ID prefix", len(listByPartialID))
return errors.Wrapf(ambiguousResultsError(term), "%d matches found based on ID prefix", len(listByPartialID))
}
return libnetwork.ErrNoSuchNetwork(term)

View File

@@ -151,5 +151,4 @@ func adjustForAPIVersion(cliVersion string, service *swarm.ServiceSpec) {
service.TaskTemplate.ContainerSpec.OomScoreAdj = 0
}
}
}

View File

@@ -1,5 +1,5 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.21
//go:build go1.22
package system // import "github.com/docker/docker/api/server/router/system"

View File

@@ -19,10 +19,10 @@ produces:
consumes:
- "application/json"
- "text/plain"
basePath: "/v1.46"
basePath: "/v1.47"
info:
title: "Docker Engine API"
version: "1.46"
version: "1.47"
x-logo:
url: "https://docs.docker.com/assets/images/logo-docker-main.png"
description: |
@@ -55,8 +55,8 @@ info:
the URL is not supported by the daemon, a HTTP `400 Bad Request` error message
is returned.
If you omit the version-prefix, the current version of the API (v1.46) is used.
For example, calling `/info` is the same as calling `/v1.46/info`. Using the
If you omit the version-prefix, the current version of the API (v1.47) is used.
For example, calling `/info` is the same as calling `/v1.47/info`. Using the
API without a version-prefix is deprecated and will be removed in a future release.
Engine releases in the near future should support this version of the API,
@@ -393,7 +393,7 @@ definitions:
Make the mount non-recursively read-only, but still leave the mount recursive
(unless NonRecursive is set to `true` in conjunction).
Addded in v1.44, before that version all read-only mounts were
Added in v1.44, before that version all read-only mounts were
non-recursive by default. To match the previous behaviour this
will default to `true` for clients on versions prior to v1.44.
type: "boolean"
@@ -1195,6 +1195,7 @@ definitions:
- "default"
- "process"
- "hyperv"
- ""
MaskedPaths:
type: "array"
description: |
@@ -1384,7 +1385,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always empty. It must not be used, and will be removed in API v1.47.
> always empty. It must not be used, and will be removed in API v1.48.
type: "string"
example: ""
Domainname:
@@ -1394,7 +1395,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always empty. It must not be used, and will be removed in API v1.47.
> always empty. It must not be used, and will be removed in API v1.48.
type: "string"
example: ""
User:
@@ -1408,7 +1409,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always false. It must not be used, and will be removed in API v1.47.
> always false. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1419,7 +1420,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always false. It must not be used, and will be removed in API v1.47.
> always false. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1430,7 +1431,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always false. It must not be used, and will be removed in API v1.47.
> always false. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1457,7 +1458,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always false. It must not be used, and will be removed in API v1.47.
> always false. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1468,7 +1469,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always false. It must not be used, and will be removed in API v1.47.
> always false. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1479,7 +1480,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always false. It must not be used, and will be removed in API v1.47.
> always false. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1516,7 +1517,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always empty. It must not be used, and will be removed in API v1.47.
> always empty. It must not be used, and will be removed in API v1.48.
type: "string"
default: ""
example: ""
@@ -1555,7 +1556,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always omitted. It must not be used, and will be removed in API v1.47.
> always omitted. It must not be used, and will be removed in API v1.48.
type: "boolean"
default: false
example: false
@@ -1567,7 +1568,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always omitted. It must not be used, and will be removed in API v1.47.
> always omitted. It must not be used, and will be removed in API v1.48.
type: "string"
default: ""
example: ""
@@ -1601,7 +1602,7 @@ definitions:
<p><br /></p>
> **Deprecated**: this field is not part of the image specification and is
> always omitted. It must not be used, and will be removed in API v1.47.
> always omitted. It must not be used, and will be removed in API v1.48.
type: "integer"
default: 10
x-nullable: true
@@ -2216,7 +2217,7 @@ definitions:
Created:
description: |
Date and time at which the image was created as a Unix timestamp
(number of seconds sinds EPOCH).
(number of seconds since EPOCH).
type: "integer"
x-nullable: false
example: "1644009612"
@@ -2265,6 +2266,19 @@ definitions:
x-nullable: false
type: "integer"
example: 2
Manifests:
description: |
Manifests is a list of manifests available in this image.
It provides a more detailed view of the platform-specific image manifests
or other image-attached data like build attestations.
WARNING: This is experimental and may change at any time without any backward
compatibility.
type: "array"
x-nullable: false
x-omitempty: true
items:
$ref: "#/definitions/ImageManifestSummary"
AuthConfig:
type: "object"
@@ -2500,7 +2514,7 @@ definitions:
example: false
Attachable:
description: |
Wheter a global / swarm scope network is manually attachable by regular
Whether a global / swarm scope network is manually attachable by regular
containers from workers in swarm mode.
type: "boolean"
default: false
@@ -3723,7 +3737,7 @@ definitions:
example: "json-file"
Options:
description: |
Driver-specific options for the selectd log driver, specified
Driver-specific options for the selected log driver, specified
as key/value pairs.
type: "object"
additionalProperties:
@@ -4167,6 +4181,7 @@ definitions:
- "default"
- "process"
- "hyperv"
- ""
Init:
description: |
Run an init inside the container that forwards signals and reaps
@@ -5318,7 +5333,7 @@ definitions:
description: |
The default (and highest) API version that is supported by the daemon
type: "string"
example: "1.46"
example: "1.47"
MinAPIVersion:
description: |
The minimum API version that is supported by the daemon
@@ -5334,7 +5349,7 @@ definitions:
The version Go used to compile the daemon, and the version of the Go
runtime in use.
type: "string"
example: "go1.21.12"
example: "go1.22.7"
Os:
description: |
The operating system that the daemon is running on ("linux" or "windows")
@@ -5737,6 +5752,7 @@ definitions:
- "default"
- "hyperv"
- "process"
- ""
InitBinary:
description: |
Name and, optional, path of the `docker-init` binary.
@@ -5807,8 +5823,6 @@ definitions:
type: "string"
example:
- "WARNING: No memory limit support"
- "WARNING: bridge-nf-call-iptables is disabled"
- "WARNING: bridge-nf-call-ip6tables is disabled"
CDISpecDirs:
description: |
List of directories where (Container Device Interface) CDI
@@ -5830,13 +5844,13 @@ definitions:
- "/var/run/cdi"
Containerd:
$ref: "#/definitions/ContainerdInfo"
x-nullable: true
ContainerdInfo:
description: |
Information for connecting to the containerd instance that is used by the daemon.
This is included for debugging purposes only.
type: "object"
x-nullable: true
properties:
Address:
description: "The address of the containerd socket."
@@ -6644,6 +6658,120 @@ definitions:
additionalProperties:
type: "string"
ImageManifestSummary:
x-go-name: "ManifestSummary"
description: |
ImageManifestSummary represents a summary of an image manifest.
type: "object"
required: ["ID", "Descriptor", "Available", "Size", "Kind"]
properties:
ID:
description: |
ID is the content-addressable ID of an image and is the same as the
digest of the image manifest.
type: "string"
example: "sha256:95869fbcf224d947ace8d61d0e931d49e31bb7fc67fffbbe9c3198c33aa8e93f"
Descriptor:
$ref: "#/definitions/OCIDescriptor"
Available:
description: Indicates whether all the child content (image config, layers) is fully available locally.
type: "boolean"
example: true
Size:
type: "object"
x-nullable: false
required: ["Content", "Total"]
properties:
Total:
type: "integer"
format: "int64"
example: 8213251
description: |
Total is the total size (in bytes) of all the locally present
data (both distributable and non-distributable) that's related to
this manifest and its children.
This equal to the sum of [Content] size AND all the sizes in the
[Size] struct present in the Kind-specific data struct.
For example, for an image kind (Kind == "image")
this would include the size of the image content and unpacked
image snapshots ([Size.Content] + [ImageData.Size.Unpacked]).
Content:
description: |
Content is the size (in bytes) of all the locally present
content in the content store (e.g. image config, layers)
referenced by this manifest and its children.
This only includes blobs in the content store.
type: "integer"
format: "int64"
example: 3987495
Kind:
type: "string"
example: "image"
enum:
- "image"
- "attestation"
- "unknown"
description: |
The kind of the manifest.
kind | description
-------------|-----------------------------------------------------------
image | Image manifest that can be used to start a container.
attestation | Attestation manifest produced by the Buildkit builder for a specific image manifest.
ImageData:
description: |
The image data for the image manifest.
This field is only populated when Kind is "image".
type: "object"
x-nullable: true
x-omitempty: true
required: ["Platform", "Containers", "Size", "UnpackedSize"]
properties:
Platform:
$ref: "#/definitions/OCIPlatform"
description: |
OCI platform of the image. This will be the platform specified in the
manifest descriptor from the index/manifest list.
If it's not available, it will be obtained from the image config.
Containers:
description: |
The IDs of the containers that are using this image.
type: "array"
items:
type: "string"
example: ["ede54ee1fda366ab42f824e8a5ffd195155d853ceaec74a927f249ea270c7430", "abadbce344c096744d8d6071a90d474d28af8f1034b5ea9fb03c3f4bfc6d005e"]
Size:
type: "object"
x-nullable: false
required: ["Unpacked"]
properties:
Unpacked:
type: "integer"
format: "int64"
example: 3987495
description: |
Unpacked is the size (in bytes) of the locally unpacked
(uncompressed) image content that's directly usable by the containers
running this image.
It's independent of the distributable content - e.g.
the image might still have an unpacked data that's still used by
some container even when the distributable/compressed content is
already gone.
AttestationData:
description: |
The image data for the attestation manifest.
This field is only populated when Kind is "attestation".
type: "object"
x-nullable: true
x-omitempty: true
required: ["For"]
properties:
For:
description: |
The digest of the image manifest that this attestation is for.
type: "string"
example: "sha256:95869fbcf224d947ace8d61d0e931d49e31bb7fc67fffbbe9c3198c33aa8e93f"
paths:
/containers/json:
get:
@@ -7585,7 +7713,7 @@ paths:
* Memory usage % = `(used_memory / available_memory) * 100.0`
* cpu_delta = `cpu_stats.cpu_usage.total_usage - precpu_stats.cpu_usage.total_usage`
* system_cpu_delta = `cpu_stats.system_cpu_usage - precpu_stats.system_cpu_usage`
* number_cpus = `lenght(cpu_stats.cpu_usage.percpu_usage)` or `cpu_stats.online_cpus`
* number_cpus = `length(cpu_stats.cpu_usage.percpu_usage)` or `cpu_stats.online_cpus`
* CPU usage % = `(cpu_delta / system_cpu_delta) * number_cpus * 100.0`
operationId: "ContainerStats"
produces: ["application/json"]
@@ -7749,10 +7877,12 @@ paths:
type: "string"
- name: "h"
in: "query"
required: true
description: "Height of the TTY session in characters"
type: "integer"
- name: "w"
in: "query"
required: true
description: "Width of the TTY session in characters"
type: "integer"
tags: ["Container"]
@@ -8622,6 +8752,11 @@ paths:
description: "Show digest information as a `RepoDigests` field on each image."
type: "boolean"
default: false
- name: "manifests"
in: "query"
description: "Include `Manifests` in the image summary."
type: "boolean"
default: false
tags: ["Image"]
/build:
post:
@@ -9094,13 +9229,37 @@ paths:
parameters:
- name: "name"
in: "path"
description: "Image name or ID."
description: |
Name of the image to push. For example, `registry.example.com/myimage`.
The image must be present in the local image store with the same name.
The name should be provided without tag; if a tag is provided, it
is ignored. For example, `registry.example.com/myimage:latest` is
considered equivalent to `registry.example.com/myimage`.
Use the `tag` parameter to specify the tag to push.
type: "string"
required: true
- name: "tag"
in: "query"
description: "The tag to associate with the image on the registry."
description: |
Tag of the image to push. For example, `latest`. If no tag is provided,
all tags of the given image that are present in the local image store
are pushed.
type: "string"
- name: "platform"
type: "string"
in: "query"
description: |
JSON-encoded OCI platform to select the platform-variant to push.
If not provided, all available variants will attempt to be pushed.
If the daemon provides a multi-platform image store, this selects
the platform-variant to push to the registry. If the image is
a single-platform image, or if the multi-platform image does not
provide a variant matching the given platform, an error is returned.
Example: `{"os": "linux", "architecture": "arm", "variant": "v5"}`
- name: "X-Registry-Auth"
in: "header"
description: |
@@ -9110,11 +9269,6 @@ paths:
details.
type: "string"
required: true
- name: "platform"
in: "query"
description: "Select a platform-specific manifest to be pushed. OCI platform (JSON encoded)"
type: "string"
x-nullable: true
tags: ["Image"]
/images/{name}/tag:
post:
@@ -9410,7 +9564,7 @@ paths:
type: "string"
example: "OK"
headers:
API-Version:
Api-Version:
type: "string"
description: "Max API Version the server supports"
Builder-Version:
@@ -9466,7 +9620,7 @@ paths:
type: "string"
example: "(empty)"
headers:
API-Version:
Api-Version:
type: "string"
description: "Max API Version the server supports"
Builder-Version:
@@ -10060,10 +10214,12 @@ paths:
type: "string"
- name: "h"
in: "query"
required: true
description: "Height of the TTY session in characters"
type: "integer"
- name: "w"
in: "query"
required: true
description: "Width of the TTY session in characters"
type: "integer"
tags: ["Exec"]
@@ -11479,6 +11635,7 @@ paths:
example:
ListenAddr: "0.0.0.0:2377"
AdvertiseAddr: "192.168.1.1:2377"
DataPathAddr: "192.168.1.1"
RemoteAddrs:
- "node1:2377"
JoinToken: "SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-7p73s1dx5in4tatdymyhg9hu2"

View File

@@ -1,6 +1,7 @@
package container // import "github.com/docker/docker/api/types/container"
import (
"errors"
"fmt"
"strings"
@@ -9,7 +10,7 @@ import (
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/strslice"
"github.com/docker/go-connections/nat"
units "github.com/docker/go-units"
"github.com/docker/go-units"
)
// CgroupnsMode represents the cgroup namespace mode of the container
@@ -325,12 +326,12 @@ func ValidateRestartPolicy(policy RestartPolicy) error {
if policy.MaximumRetryCount < 0 {
msg += " and cannot be negative"
}
return &errInvalidParameter{fmt.Errorf(msg)}
return &errInvalidParameter{errors.New(msg)}
}
return nil
case RestartPolicyOnFailure:
if policy.MaximumRetryCount < 0 {
return &errInvalidParameter{fmt.Errorf("invalid restart policy: maximum retry count cannot be negative")}
return &errInvalidParameter{errors.New("invalid restart policy: maximum retry count cannot be negative")}
}
return nil
case "":

View File

@@ -196,7 +196,7 @@ func (args Args) Match(field, source string) bool {
}
// GetBoolOrDefault returns a boolean value of the key if the key is present
// and is intepretable as a boolean value. Otherwise the default value is returned.
// and is interpretable as a boolean value. Otherwise the default value is returned.
// Error is not nil only if the filter values are not valid boolean or are conflicting.
func (args Args) GetBoolOrDefault(key string, defaultValue bool) (bool, error) {
fieldValues, ok := args.fields[key]

View File

@@ -0,0 +1,99 @@
package image
import (
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
type ManifestKind string
const (
ManifestKindImage ManifestKind = "image"
ManifestKindAttestation ManifestKind = "attestation"
ManifestKindUnknown ManifestKind = "unknown"
)
type ManifestSummary struct {
// ID is the content-addressable ID of an image and is the same as the
// digest of the image manifest.
//
// Required: true
ID string `json:"ID"`
// Descriptor is the OCI descriptor of the image.
//
// Required: true
Descriptor ocispec.Descriptor `json:"Descriptor"`
// Indicates whether all the child content (image config, layers) is
// fully available locally
//
// Required: true
Available bool `json:"Available"`
// Size is the size information of the content related to this manifest.
// Note: These sizes only take the locally available content into account.
//
// Required: true
Size struct {
// Content is the size (in bytes) of all the locally present
// content in the content store (e.g. image config, layers)
// referenced by this manifest and its children.
// This only includes blobs in the content store.
Content int64 `json:"Content"`
// Total is the total size (in bytes) of all the locally present
// data (both distributable and non-distributable) that's related to
// this manifest and its children.
// This equal to the sum of [Content] size AND all the sizes in the
// [Size] struct present in the Kind-specific data struct.
// For example, for an image kind (Kind == ManifestKindImage),
// this would include the size of the image content and unpacked
// image snapshots ([Size.Content] + [ImageData.Size.Unpacked]).
Total int64 `json:"Total"`
} `json:"Size"`
// Kind is the kind of the image manifest.
//
// Required: true
Kind ManifestKind `json:"Kind"`
// Fields below are specific to the kind of the image manifest.
// Present only if Kind == ManifestKindImage.
ImageData *ImageProperties `json:"ImageData,omitempty"`
// Present only if Kind == ManifestKindAttestation.
AttestationData *AttestationProperties `json:"AttestationData,omitempty"`
}
type ImageProperties struct {
// Platform is the OCI platform object describing the platform of the image.
//
// Required: true
Platform ocispec.Platform `json:"Platform"`
Size struct {
// Unpacked is the size (in bytes) of the locally unpacked
// (uncompressed) image content that's directly usable by the containers
// running this image.
// It's independent of the distributable content - e.g.
// the image might still have an unpacked data that's still used by
// some container even when the distributable/compressed content is
// already gone.
//
// Required: true
Unpacked int64 `json:"Unpacked"`
}
// Containers is an array containing the IDs of the containers that are
// using this image.
//
// Required: true
Containers []string `json:"Containers"`
}
type AttestationProperties struct {
// For is the digest of the image manifest that this attestation is for.
For digest.Digest `json:"For"`
}

View File

@@ -76,6 +76,9 @@ type ListOptions struct {
// ContainerCount indicates whether container count should be computed.
ContainerCount bool
// Manifests indicates whether the image manifests should be returned.
Manifests bool
}
// RemoveOptions holds parameters to remove images.

View File

@@ -1,10 +1,5 @@
package image
// This file was generated by the swagger tool.
// Editing this file might prove futile when you re-run the swagger generate command
// Summary summary
// swagger:model Summary
type Summary struct {
// Number of containers using this image. Includes both stopped and running
@@ -17,7 +12,7 @@ type Summary struct {
Containers int64 `json:"Containers"`
// Date and time at which the image was created as a Unix timestamp
// (number of seconds sinds EPOCH).
// (number of seconds since EPOCH).
//
// Required: true
Created int64 `json:"Created"`
@@ -47,6 +42,14 @@ type Summary struct {
// Required: true
ParentID string `json:"ParentId"`
// Manifests is a list of image manifests available in this image. It
// provides a more detailed view of the platform-specific image manifests or
// other image-attached data like build attestations.
//
// WARNING: This is experimental and may change at any time without any backward
// compatibility.
Manifests []ManifestSummary `json:"Manifests,omitempty"`
// List of content-addressable digests of locally available image manifests
// that the image is referenced from. Multiple manifests can refer to the
// same image.

View File

@@ -102,7 +102,6 @@ func TestEndpointIPAMConfigWithOutOfRangeAddrs(t *testing.T) {
}
})
}
}
func TestEndpointIPAMConfigWithInvalidConfig(t *testing.T) {

View File

@@ -34,10 +34,9 @@ type AuthConfig struct {
}
// EncodeAuthConfig serializes the auth configuration as a base64url encoded
// RFC4648, section 5) JSON string for sending through the X-Registry-Auth header.
// ([RFC4648, section 5]) JSON string for sending through the X-Registry-Auth header.
//
// For details on base64url encoding, see:
// - RFC4648, section 5: https://tools.ietf.org/html/rfc4648#section-5
// [RFC4648, section 5]: https://tools.ietf.org/html/rfc4648#section-5
func EncodeAuthConfig(authConfig AuthConfig) (string, error) {
buf, err := json.Marshal(authConfig)
if err != nil {
@@ -46,15 +45,14 @@ func EncodeAuthConfig(authConfig AuthConfig) (string, error) {
return base64.URLEncoding.EncodeToString(buf), nil
}
// DecodeAuthConfig decodes base64url encoded (RFC4648, section 5) JSON
// DecodeAuthConfig decodes base64url encoded ([RFC4648, section 5]) JSON
// authentication information as sent through the X-Registry-Auth header.
//
// This function always returns an AuthConfig, even if an error occurs. It is up
// This function always returns an [AuthConfig], even if an error occurs. It is up
// to the caller to decide if authentication is required, and if the error can
// be ignored.
//
// For details on base64url encoding, see:
// - RFC4648, section 5: https://tools.ietf.org/html/rfc4648#section-5
// [RFC4648, section 5]: https://tools.ietf.org/html/rfc4648#section-5
func DecodeAuthConfig(authEncoded string) (*AuthConfig, error) {
if authEncoded == "" {
return &AuthConfig{}, nil
@@ -69,7 +67,7 @@ func DecodeAuthConfig(authEncoded string) (*AuthConfig, error) {
// clients and API versions. Current clients and API versions expect authentication
// to be provided through the X-Registry-Auth header.
//
// Like DecodeAuthConfig, this function always returns an AuthConfig, even if an
// Like [DecodeAuthConfig], this function always returns an [AuthConfig], even if an
// error occurs. It is up to the caller to decide if authentication is required,
// and if the error can be ignored.
func DecodeAuthConfigBody(rdr io.ReadCloser) (*AuthConfig, error) {

View File

@@ -122,7 +122,7 @@ type CAConfig struct {
SigningCAKey string `json:",omitempty"`
// If this value changes, and there is no specified signing cert and key,
// then the swarm is forced to generate a new root certificate ane key.
// then the swarm is forced to generate a new root certificate and key.
ForceRotate uint64 `json:",omitempty"`
}

View File

@@ -484,4 +484,6 @@ type BuildCachePruneOptions struct {
All bool
KeepStorage int64
Filters filters.Args
// FIXME(thaJeztah): add new options; see https://github.com/moby/moby/issues/48639
}

View File

@@ -414,7 +414,7 @@ type Info struct {
// the Volume has not been successfully created yet.
VolumeID string `json:",omitempty"`
// AccessibleTopolgoy is the topology this volume is actually accessible
// AccessibleTopology is the topology this volume is actually accessible
// from.
AccessibleTopology []Topology `json:",omitempty"`
}

View File

@@ -1,5 +1,5 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.21
//go:build go1.22
package containerimage

View File

@@ -38,6 +38,7 @@ import (
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
grpcmetadata "google.golang.org/grpc/metadata"
"google.golang.org/protobuf/proto"
)
type errMultipleFilterValues struct{}
@@ -162,16 +163,29 @@ func (b *Builder) DiskUsage(ctx context.Context) ([]*types.BuildCache, error) {
Description: r.Description,
InUse: r.InUse,
Shared: r.Shared,
Size: r.Size_,
CreatedAt: r.CreatedAt,
LastUsedAt: r.LastUsedAt,
UsageCount: int(r.UsageCount),
Size: r.Size,
CreatedAt: func() time.Time {
if r.CreatedAt != nil {
return r.CreatedAt.AsTime()
}
return time.Time{}
}(),
LastUsedAt: func() *time.Time {
if r.LastUsedAt == nil {
return nil
}
t := r.LastUsedAt.AsTime()
return &t
}(),
UsageCount: int(r.UsageCount),
})
}
return items, nil
}
// Prune clears all reclaimable build cache
// Prune clears all reclaimable build cache.
//
// FIXME(thaJeztah): wire up new options https://github.com/moby/moby/issues/48639
func (b *Builder) Prune(ctx context.Context, opts types.BuildCachePruneOptions) (int64, []string, error) {
ch := make(chan *controlapi.UsageRecord)
@@ -197,10 +211,10 @@ func (b *Builder) Prune(ctx context.Context, opts types.BuildCachePruneOptions)
eg.Go(func() error {
defer close(ch)
return b.controller.Prune(&controlapi.PruneRequest{
All: pi.All,
KeepDuration: int64(pi.KeepDuration),
KeepBytes: pi.KeepBytes,
Filter: pi.Filter,
All: pi.All,
KeepDuration: int64(pi.KeepDuration),
ReservedSpace: pi.ReservedSpace,
Filter: pi.Filter,
}, &pruneProxy{
streamProxy: streamProxy{ctx: ctx},
ch: ch,
@@ -211,7 +225,7 @@ func (b *Builder) Prune(ctx context.Context, opts types.BuildCachePruneOptions)
var cacheIDs []string
eg.Go(func() error {
for r := range ch {
size += r.Size_
size += r.Size
cacheIDs = append(cacheIDs, r.ID)
}
return nil
@@ -381,7 +395,7 @@ func (b *Builder) Build(ctx context.Context, opt backend.BuildConfig) (*builder.
exporterAttrs["name"] = strings.Join(nameAttr, ",")
}
cache := controlapi.CacheOptions{}
cache := &controlapi.CacheOptions{}
if inlineCache := opt.Options.BuildArgs["BUILDKIT_INLINE_CACHE"]; inlineCache != nil {
if b, err := strconv.ParseBool(*inlineCache); err == nil && b {
cache.Exports = append(cache.Exports, &controlapi.CacheOptionsEntry{
@@ -402,7 +416,7 @@ func (b *Builder) Build(ctx context.Context, opt backend.BuildConfig) (*builder.
}
if opt.Options.NetworkMode == "host" {
req.Entitlements = append(req.Entitlements, entitlements.EntitlementNetworkHost)
req.Entitlements = append(req.Entitlements, string(entitlements.EntitlementNetworkHost))
}
aux := streamformatter.AuxFormatter{Writer: opt.ProgressWriter.Output}
@@ -437,7 +451,7 @@ func (b *Builder) Build(ctx context.Context, opt backend.BuildConfig) (*builder.
eg.Go(func() error {
for sr := range ch {
dt, err := sr.Marshal()
dt, err := proto.Marshal(sr)
if err != nil {
return err
}
@@ -624,6 +638,7 @@ func toBuildkitUlimits(inp []*container.Ulimit) (string, error) {
return strings.Join(ulimits, ","), nil
}
// FIXME(thaJeztah): wire-up new fields; see https://github.com/moby/moby/issues/48639
func toBuildkitPruneInfo(opts types.BuildCachePruneOptions) (client.PruneInfo, error) {
var until time.Duration
untilValues := opts.Filters.Get("until") // canonical
@@ -679,9 +694,9 @@ func toBuildkitPruneInfo(opts types.BuildCachePruneOptions) (client.PruneInfo, e
}
}
return client.PruneInfo{
All: opts.All,
KeepDuration: until,
KeepBytes: opts.KeepStorage,
Filter: []string{strings.Join(bkFilter, ",")},
All: opts.All,
KeepDuration: until,
ReservedSpace: opts.KeepStorage,
Filter: []string{strings.Join(bkFilter, ",")},
}, nil
}

View File

@@ -25,7 +25,7 @@ import (
wlabel "github.com/docker/docker/builder/builder-next/worker/label"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/daemon/graphdriver"
units "github.com/docker/go-units"
"github.com/docker/go-units"
"github.com/moby/buildkit/cache"
"github.com/moby/buildkit/cache/metadata"
"github.com/moby/buildkit/cache/remotecache"
@@ -43,6 +43,8 @@ import (
containerdsnapshot "github.com/moby/buildkit/snapshot/containerd"
"github.com/moby/buildkit/solver"
"github.com/moby/buildkit/solver/bboltcachestorage"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/apicaps"
"github.com/moby/buildkit/util/archutil"
"github.com/moby/buildkit/util/entitlements"
"github.com/moby/buildkit/util/network/netproviders"
@@ -55,9 +57,6 @@ import (
"go.etcd.io/bbolt"
bolt "go.etcd.io/bbolt"
"go.opentelemetry.io/otel/sdk/trace"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/apicaps"
)
func newController(ctx context.Context, rt http.RoundTripper, opt Opt) (*control.Controller, error) {
@@ -86,7 +85,7 @@ func newSnapshotterController(ctx context.Context, rt http.RoundTripper, opt Opt
return nil, err
}
historyDB, historyConf, err := openHistoryDB(opt.Root, opt.BuilderConfig.History)
historyDB, historyConf, err := openHistoryDB(opt.Root, "history_c8d.db", opt.BuilderConfig.History)
if err != nil {
return nil, err
}
@@ -149,7 +148,7 @@ func newSnapshotterController(ctx context.Context, rt http.RoundTripper, opt Opt
}
wo.Executor = exec
w, err := mobyworker.NewContainerdWorker(ctx, wo, opt.Callbacks)
w, err := mobyworker.NewContainerdWorker(ctx, wo, opt.Callbacks, rt)
if err != nil {
return nil, err
}
@@ -194,11 +193,12 @@ func newSnapshotterController(ctx context.Context, rt http.RoundTripper, opt Opt
LeaseManager: wo.LeaseManager,
ContentStore: wo.ContentStore,
TraceCollector: getTraceExporter(ctx),
GarbageCollect: w.GarbageCollect,
})
}
func openHistoryDB(root string, cfg *config.BuilderHistoryConfig) (*bolt.DB, *bkconfig.HistoryConfig, error) {
db, err := bbolt.Open(filepath.Join(root, "history.db"), 0o600, nil)
func openHistoryDB(root string, fn string, cfg *config.BuilderHistoryConfig) (*bolt.DB, *bkconfig.HistoryConfig, error) {
db, err := bbolt.Open(filepath.Join(root, fn), 0o600, nil)
if err != nil {
return nil, nil, err
}
@@ -344,7 +344,7 @@ func newGraphDriverController(ctx context.Context, rt http.RoundTripper, opt Opt
return nil, err
}
historyDB, historyConf, err := openHistoryDB(opt.Root, opt.BuilderConfig.History)
historyDB, historyConf, err := openHistoryDB(opt.Root, "history.db", opt.BuilderConfig.History)
if err != nil {
return nil, err
}
@@ -382,6 +382,7 @@ func newGraphDriverController(ctx context.Context, rt http.RoundTripper, opt Opt
Layers: layers,
Platforms: archutil.SupportedPlatforms(true),
LeaseManager: lm,
GarbageCollect: mdb.GarbageCollect,
Labels: getLabels(opt, nil),
}
@@ -421,6 +422,7 @@ func newGraphDriverController(ctx context.Context, rt http.RoundTripper, opt Opt
HistoryDB: historyDB,
HistoryConfig: historyConf,
TraceCollector: getTraceExporter(ctx),
GarbageCollect: w.GarbageCollect,
})
}
@@ -435,7 +437,7 @@ func getGCPolicy(conf config.BuilderConfig, root string) ([]client.PruneInfo, er
if conf.GC.DefaultKeepStorage != "" {
defaultKeepStorage, err = units.RAMInBytes(conf.GC.DefaultKeepStorage)
if err != nil {
return nil, errors.Wrapf(err, "could not parse '%s' as Builder.GC.DefaultKeepStorage config", conf.GC.DefaultKeepStorage)
return nil, errors.Wrapf(err, "failed to parse defaultKeepStorage")
}
}
@@ -444,13 +446,18 @@ func getGCPolicy(conf config.BuilderConfig, root string) ([]client.PruneInfo, er
} else {
gcPolicy = make([]client.PruneInfo, len(conf.GC.Policy))
for i, p := range conf.GC.Policy {
b, err := units.RAMInBytes(p.KeepStorage)
if err != nil {
return nil, err
var b int64
if p.KeepStorage != "" {
b, err = units.RAMInBytes(p.KeepStorage)
if err != nil {
return nil, errors.Wrapf(err, "failed to parse keepStorage")
}
}
if b == 0 {
b = defaultKeepStorage
}
// FIXME(thaJeztah): wire up new options https://github.com/moby/moby/issues/48639
gcPolicy[i], err = toBuildkitPruneInfo(types.BuildCachePruneOptions{
All: p.All,
KeepStorage: b,

View File

@@ -2,11 +2,14 @@ package worker
import (
"context"
nethttp "net/http"
"github.com/containerd/log"
"github.com/docker/docker/builder/builder-next/exporter"
"github.com/moby/buildkit/client"
bkexporter "github.com/moby/buildkit/exporter"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/source/http"
"github.com/moby/buildkit/worker/base"
)
@@ -17,11 +20,21 @@ type ContainerdWorker struct {
}
// NewContainerdWorker instantiates a local worker.
func NewContainerdWorker(ctx context.Context, wo base.WorkerOpt, callbacks exporter.BuildkitCallbacks) (*ContainerdWorker, error) {
func NewContainerdWorker(ctx context.Context, wo base.WorkerOpt, callbacks exporter.BuildkitCallbacks, rt nethttp.RoundTripper) (*ContainerdWorker, error) {
bw, err := base.NewWorker(ctx, wo)
if err != nil {
return nil, err
}
hs, err := http.NewSource(http.Opt{
CacheAccessor: bw.CacheManager(),
Transport: rt,
})
if err == nil {
bw.SourceManager.Register(hs)
} else {
log.G(ctx).Warnf("Could not register builder http source: %s", err)
}
return &ContainerdWorker{Worker: bw, callbacks: callbacks}, nil
}

View File

@@ -27,26 +27,27 @@ func DefaultGCPolicy(p string, defaultKeepBytes int64) []client.PruneInfo {
tempCacheKeepBytes = minTempCacheKeepBytes
}
// FIXME(thaJeztah): wire up new options https://github.com/moby/moby/issues/48639
return []client.PruneInfo{
// if build cache uses more than 512MB delete the most easily reproducible data after it has not been used for 2 days
{
Filter: []string{"type==source.local,type==exec.cachemount,type==source.git.checkout"},
KeepDuration: 48 * time.Hour,
KeepBytes: tempCacheKeepBytes,
Filter: []string{"type==source.local,type==exec.cachemount,type==source.git.checkout"},
KeepDuration: 48 * time.Hour,
ReservedSpace: tempCacheKeepBytes,
},
// remove any data not used for 60 days
{
KeepDuration: 60 * 24 * time.Hour,
KeepBytes: keep,
KeepDuration: 60 * 24 * time.Hour,
ReservedSpace: keep,
},
// keep the unshared build cache under cap
{
KeepBytes: keep,
ReservedSpace: keep,
},
// if previous policies were insufficient start deleting internal data to keep build cache under cap
{
All: true,
KeepBytes: keep,
All: true,
ReservedSpace: keep,
},
}
}

View File

@@ -8,8 +8,10 @@ import (
"time"
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/gc"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/rootfs"
cerrdefs "github.com/containerd/errdefs"
"github.com/containerd/log"
"github.com/containerd/platforms"
imageadapter "github.com/docker/docker/builder/builder-next/adapters/containerimage"
@@ -76,6 +78,7 @@ type Opt struct {
ContentStore *containerdsnapshot.Store
CacheManager cache.Manager
LeaseManager *leaseutil.Manager
GarbageCollect func(context.Context) (gc.Stats, error)
ImageSource *imageadapter.Source
DownloadManager *xfer.LayerDownloadManager
V2MetadataService distmetadata.V2MetadataService
@@ -183,6 +186,14 @@ func (w *Worker) BuildkitVersion() client.BuildkitVersion {
}
}
func (w *Worker) GarbageCollect(ctx context.Context) error {
if w.Opt.GarbageCollect == nil {
return nil
}
_, err := w.Opt.GarbageCollect(ctx)
return err
}
// Close closes the worker and releases all resources
func (w *Worker) Close() error {
return nil
@@ -353,13 +364,13 @@ func (w *Worker) GetRemotes(ctx context.Context, ref cache.ImmutableRef, createI
}
// PruneCacheMounts removes the current cache snapshots for specified IDs
func (w *Worker) PruneCacheMounts(ctx context.Context, ids []string) error {
func (w *Worker) PruneCacheMounts(ctx context.Context, ids map[string]bool) error {
mu := mounts.CacheMountsLocker()
mu.Lock()
defer mu.Unlock()
for _, id := range ids {
mds, err := mounts.SearchCacheDir(ctx, w.CacheManager(), id)
for id, nested := range ids {
mds, err := mounts.SearchCacheDir(ctx, w.CacheManager(), id, nested)
if err != nil {
return err
}
@@ -572,5 +583,5 @@ func (p *emptyProvider) ReaderAt(ctx context.Context, dec ocispec.Descriptor) (c
}
func (p *emptyProvider) Info(ctx context.Context, d digest.Digest) (content.Info, error) {
return content.Info{}, errors.Errorf("Info not implemented for empty provider")
return content.Info{}, errors.Wrapf(cerrdefs.ErrNotImplemented, "Info not implemented for empty provider")
}

View File

@@ -8,8 +8,8 @@ import (
winio "github.com/Microsoft/go-winio"
"github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/reexec"
"github.com/docker/docker/pkg/system"
"github.com/moby/sys/reexec"
"github.com/pkg/errors"
"golang.org/x/sys/windows"
)

View File

@@ -8,8 +8,8 @@ import (
"github.com/docker/docker/builder/remotecontext"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/reexec"
"github.com/moby/buildkit/frontend/dockerfile/instructions"
"github.com/moby/sys/reexec"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
"gotest.tools/v3/skip"

View File

@@ -27,25 +27,25 @@ func parseChownFlag(ctx context.Context, builder *Builder, state *dispatchState,
passwdPath, err := symlink.FollowSymlinkInScope(filepath.Join(ctrRootPath, "etc", "passwd"), ctrRootPath)
if err != nil {
return idtools.Identity{}, errors.Wrapf(err, "can't resolve /etc/passwd path in container rootfs")
return idtools.Identity{}, errors.Wrap(err, "can't resolve /etc/passwd path in container rootfs")
}
groupPath, err := symlink.FollowSymlinkInScope(filepath.Join(ctrRootPath, "etc", "group"), ctrRootPath)
if err != nil {
return idtools.Identity{}, errors.Wrapf(err, "can't resolve /etc/group path in container rootfs")
return idtools.Identity{}, errors.Wrap(err, "can't resolve /etc/group path in container rootfs")
}
uid, err := lookupUser(userStr, passwdPath)
if err != nil {
return idtools.Identity{}, errors.Wrapf(err, "can't find uid for user "+userStr)
return idtools.Identity{}, errors.Wrap(err, "can't find uid for user "+userStr)
}
gid, err := lookupGroup(grpStr, groupPath)
if err != nil {
return idtools.Identity{}, errors.Wrapf(err, "can't find gid for group "+grpStr)
return idtools.Identity{}, errors.Wrap(err, "can't find gid for group "+grpStr)
}
// convert as necessary because of user namespaces
chownPair, err := identityMapping.ToHost(idtools.Identity{UID: uid, GID: gid})
if err != nil {
return idtools.Identity{}, errors.Wrapf(err, "unable to convert uid/gid to host mapping")
return idtools.Identity{}, errors.Wrap(err, "unable to convert uid/gid to host mapping")
}
return chownPair, nil
}

View File

@@ -17,7 +17,6 @@ func createTestTempDir(t *testing.T, dir, prefix string) (string, func()) {
return path, func() {
err = os.RemoveAll(path)
if err != nil {
t.Fatalf("Error when removing directory %s: %s", path, err)
}

View File

@@ -7,7 +7,7 @@ import (
"github.com/docker/docker/builder"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/reexec"
"github.com/moby/sys/reexec"
"github.com/pkg/errors"
"gotest.tools/v3/skip"
)
@@ -33,7 +33,6 @@ func TestCloseRootDirectory(t *testing.T) {
src := makeTestArchiveContext(t, contextDir)
err = src.Close()
if err != nil {
t.Fatalf("Error while executing Close: %s", err)
}

View File

@@ -17,7 +17,6 @@ func createTestTempDir(t *testing.T, dir, prefix string) (string, func()) {
return path, func() {
err = os.RemoveAll(path)
if err != nil {
t.Fatalf("Error when removing directory %s: %s", path, err)
}

View File

@@ -2,7 +2,7 @@
Package client is a Go client for the Docker Engine API.
For more information about the Engine API, see the documentation:
https://docs.docker.com/engine/api/
https://docs.docker.com/reference/api/engine/
# Usage
@@ -247,6 +247,14 @@ func (cli *Client) tlsConfig() *tls.Config {
func defaultHTTPClient(hostURL *url.URL) (*http.Client, error) {
transport := &http.Transport{}
// Necessary to prevent long-lived processes using the
// client from leaking connections due to idle connections
// not being released.
// TODO: see if we can also address this from the server side,
// or in go-connections.
// see: https://github.com/moby/moby/issues/45539
transport.MaxIdleConns = 6
transport.IdleConnTimeout = 30 * time.Second
err := sockets.ConfigureTransport(transport, hostURL.Scheme, hostURL.Host)
if err != nil {
return nil, err

View File

@@ -371,7 +371,7 @@ func TestNegotiateAPIVersionAutomatic(t *testing.T) {
var pingVersion string
httpClient := newMockClient(func(req *http.Request) (*http.Response, error) {
resp := &http.Response{StatusCode: http.StatusOK, Header: http.Header{}}
resp.Header.Set("API-Version", pingVersion)
resp.Header.Set("Api-Version", pingVersion)
resp.Body = io.NopCloser(strings.NewReader("OK"))
return resp, nil
})

View File

@@ -11,6 +11,11 @@ import (
)
// ImageList returns a list of images in the docker host.
//
// Experimental: Setting the [options.Manifest] will populate
// [image.Summary.Manifests] with information about image manifests.
// This is experimental and might change in the future without any backward
// compatibility.
func (cli *Client) ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error) {
var images []image.Summary
@@ -47,6 +52,9 @@ func (cli *Client) ImageList(ctx context.Context, options image.ListOptions) ([]
if options.SharedSize && versions.GreaterThanOrEqualTo(cli.version, "1.42") {
query.Set("shared-size", "1")
}
if options.Manifests && versions.GreaterThanOrEqualTo(cli.version, "1.47") {
query.Set("manifests", "1")
}
serverResp, err := cli.get(ctx, "/images/json", query, nil)
defer ensureReaderClosed(serverResp)

View File

@@ -56,8 +56,8 @@ func parsePingResponse(cli *Client, resp serverResponse) (types.Ping, error) {
err := cli.checkResponseErr(resp)
return ping, errdefs.FromStatusCode(err, resp.statusCode)
}
ping.APIVersion = resp.header.Get("API-Version")
ping.OSType = resp.header.Get("OSType")
ping.APIVersion = resp.header.Get("Api-Version")
ping.OSType = resp.header.Get("Ostype")
if resp.header.Get("Docker-Experimental") == "true" {
ping.Experimental = true
}

View File

@@ -24,7 +24,7 @@ func TestPingFail(t *testing.T) {
resp := &http.Response{StatusCode: http.StatusInternalServerError}
if withHeader {
resp.Header = http.Header{}
resp.Header.Set("API-Version", "awesome")
resp.Header.Set("Api-Version", "awesome")
resp.Header.Set("Docker-Experimental", "true")
resp.Header.Set("Swarm", "inactive")
}
@@ -72,7 +72,7 @@ func TestPingSuccess(t *testing.T) {
client: newMockClient(func(req *http.Request) (*http.Response, error) {
resp := &http.Response{StatusCode: http.StatusOK}
resp.Header = http.Header{}
resp.Header.Set("API-Version", "awesome")
resp.Header.Set("Api-Version", "awesome")
resp.Header.Set("Docker-Experimental", "true")
resp.Header.Set("Swarm", "active/manager")
resp.Body = io.NopCloser(strings.NewReader("OK"))
@@ -122,7 +122,7 @@ func TestPingHeadFallback(t *testing.T) {
resp.StatusCode = tc.status
}
resp.Header = http.Header{}
resp.Header.Add("API-Version", strings.Join(reqs, ", "))
resp.Header.Add("Api-Version", strings.Join(reqs, ", "))
return resp, nil
}),
}

View File

@@ -4,6 +4,7 @@ import (
"runtime"
"github.com/docker/docker/daemon/config"
dopts "github.com/docker/docker/internal/opts"
"github.com/docker/docker/opts"
"github.com/docker/docker/registry"
"github.com/spf13/pflag"
@@ -28,6 +29,7 @@ func installCommonConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
flags.StringVar(&conf.ExecRoot, "exec-root", conf.ExecRoot, "Root directory for execution state files")
flags.StringVar(&conf.ContainerdAddr, "containerd", "", "containerd grpc address")
flags.BoolVar(&conf.CriContainerd, "cri-containerd", false, "start containerd with cri")
flags.Var(dopts.NewNamedSetOpts("features", conf.Features), "feature", "Enable feature in the daemon")
flags.Var(opts.NewNamedMapMapOpts("default-network-opts", conf.DefaultNetworkOpts, nil), "default-network-opt", "Default network options")
flags.IntVar(&conf.MTU, "mtu", conf.MTU, `Set the MTU for the default "bridge" network`)

View File

@@ -44,6 +44,7 @@ import (
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/daemon/listeners"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/internal/otelutil"
"github.com/docker/docker/libcontainerd/supervisor"
dopts "github.com/docker/docker/opts"
"github.com/docker/docker/pkg/authorization"
@@ -64,8 +65,6 @@ import (
"github.com/spf13/pflag"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/sdk/resource"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
"tags.cncf.io/container-device-interface/pkg/cdi"
)
@@ -245,7 +244,7 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
// Initialize the trace recorder for buildkit.
detect.Recorder = detect.NewTraceRecorder()
tp := newTracerProvider(ctx)
tp, otelShutdown := otelutil.NewTracerProvider(ctx, true)
otel.SetTracerProvider(tp)
log.G(ctx).Logger.AddHook(tracing.NewLogrusHook())
@@ -270,7 +269,7 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
}
// Note that CDI is not inherently linux-specific, there are some linux-specific assumptions / implementations in the code that
// queries the properties of device on the host as wel as performs the injection of device nodes and their access permissions into the OCI spec.
// queries the properties of device on the host as well as performs the injection of device nodes and their access permissions into the OCI spec.
//
// In order to lift this restriction the following would have to be addressed:
// - Support needs to be added to the cdi package for injecting Windows devices: https://tags.cncf.io/container-device-interface/issues/28
@@ -297,16 +296,13 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
log.G(ctx).Info("Daemon has completed initialization")
routerCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
// Get a the current daemon config, because the daemon sets up config
// during initialization. We cannot user the cli.Config for that reason,
// as that only holds the config that was set by the user.
//
// FIXME(thaJeztah): better separate runtime and config data?
daemonCfg := d.Config()
routerOpts, err := newRouterOptions(routerCtx, &daemonCfg, d, c)
routerOpts, err := newRouterOptions(ctx, &daemonCfg, d, c)
if err != nil {
return err
}
@@ -364,7 +360,7 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
return errors.Wrap(err, "shutting down due to ServeAPI error")
}
if err := tp.Shutdown(context.Background()); err != nil {
if err := otelShutdown(context.WithoutCancel(ctx)); err != nil {
log.G(ctx).WithError(err).Error("Failed to shutdown OTEL tracing")
}
@@ -395,20 +391,6 @@ func setOTLPProtoDefault() {
}
}
func newTracerProvider(ctx context.Context) *sdktrace.TracerProvider {
opts := []sdktrace.TracerProviderOption{
sdktrace.WithResource(resource.Default()),
sdktrace.WithSyncer(detect.Recorder),
}
if exp, err := detect.NewSpanExporter(ctx); err != nil {
log.G(ctx).WithError(err).Warn("Failed to initialize tracing, skipping")
} else if !detect.IsNoneSpanExporter(exp) {
opts = append(opts, sdktrace.WithBatcher(exp))
}
return sdktrace.NewTracerProvider(opts...)
}
type routerOptions struct {
sessionManager *session.Manager
buildBackend *buildbackend.Backend

View File

@@ -9,7 +9,7 @@ import (
"testing"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/pkg/reexec"
"github.com/moby/sys/reexec"
"golang.org/x/sys/unix"
"gotest.tools/v3/assert"
)

View File

@@ -1,12 +1,14 @@
package main
import (
"runtime"
"testing"
"github.com/containerd/log"
"github.com/docker/docker/daemon/config"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/spf13/pflag"
"go.opentelemetry.io/otel"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
"gotest.tools/v3/fs"
@@ -284,3 +286,29 @@ func TestCDISpecDirs(t *testing.T) {
})
}
}
// TestOtelMeterLeak tests for a memory leak in the OTEL meter implementation.
// Once the fixed OTEL is vendored, this test will fail - the workaround
// and this test should be removed then.
func TestOtelMeterLeak(t *testing.T) {
meter := otel.Meter("foo")
var before runtime.MemStats
runtime.ReadMemStats(&before)
const counters = 10 * 1000 * 1000
for i := 0; i < counters; i++ {
_, _ = meter.Int64Counter("bar")
}
var after runtime.MemStats
runtime.ReadMemStats(&after)
allocs := after.Mallocs - before.Mallocs
t.Log("Allocations:", allocs)
if allocs < 10 {
// TODO: Remove Workaround OTEL memory leak in cmd/dockerd/daemon.go
t.Fatal("Allocations count decreased. OTEL leak workaround is no longer needed!")
}
}

View File

@@ -9,11 +9,14 @@ import (
"github.com/containerd/log"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/pkg/reexec"
"github.com/docker/docker/pkg/rootless"
"github.com/moby/buildkit/util/apicaps"
"github.com/moby/sys/reexec"
"github.com/moby/term"
"github.com/spf13/cobra"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/metric/noop"
)
var honorXDG bool
@@ -82,6 +85,12 @@ func main() {
// Fixes https://github.com/docker/docker/issues/19728
signal.Ignore(syscall.SIGPIPE)
// Workaround OTEL memory leak
// See: https://github.com/open-telemetry/opentelemetry-go-contrib/issues/5190
// The need for this workaround is checked by the TestOtelMeterLeak test
// TODO: Remove this workaround after upgrading to v1.30.0
otel.SetMeterProvider(noop.MeterProvider{})
// Set terminal emulation based on platform as required.
_, stdout, stderr := term.StdStreams()
onError := func(err error) {

View File

@@ -3,7 +3,7 @@ package main
import (
"testing"
"github.com/docker/docker/pkg/reexec"
"github.com/moby/sys/reexec"
)
func TestMain(m *testing.M) {

View File

@@ -14,7 +14,7 @@ func NoArgs(cmd *cobra.Command, args []string) error {
}
if cmd.HasSubCommands() {
return errors.Errorf("\n" + strings.TrimRight(cmd.UsageString(), "\n"))
return errors.New("\n" + strings.TrimRight(cmd.UsageString(), "\n"))
}
return errors.Errorf(

View File

@@ -35,7 +35,7 @@ import (
"github.com/docker/docker/restartmanager"
"github.com/docker/docker/volume"
volumemounts "github.com/docker/docker/volume/mounts"
units "github.com/docker/go-units"
"github.com/docker/go-units"
agentexec "github.com/moby/swarmkit/v2/agent/exec"
"github.com/moby/sys/signal"
"github.com/moby/sys/symlink"
@@ -325,7 +325,7 @@ func (container *Container) SetupWorkingDirectory(rootIdentity idtools.Identity)
}
// GetResourcePath evaluates `path` in the scope of the container's BaseFS, with proper path
// sanitisation. Symlinks are all scoped to the BaseFS of the container, as
// sanitization. Symlinks are all scoped to the BaseFS of the container, as
// though the container's BaseFS was `/`.
//
// The BaseFS of a container is the host-facing path which is bind-mounted as
@@ -368,7 +368,7 @@ func cleanScopedPath(path string) string {
}
// GetRootResourcePath evaluates `path` in the scope of the container's root, with proper path
// sanitisation. Symlinks are all scoped to the root of the container, as
// sanitization. Symlinks are all scoped to the root of the container, as
// though the container's root was `/`.
//
// The root of a container is the host-facing configuration metadata directory.

View File

@@ -9,7 +9,7 @@ import (
"github.com/docker/docker/api/types"
libcontainerdtypes "github.com/docker/docker/libcontainerd/types"
units "github.com/docker/go-units"
"github.com/docker/go-units"
)
// State holds the current container state, and has methods to get and

View File

@@ -2,6 +2,7 @@ package stream // import "github.com/docker/docker/container/stream"
import (
"context"
"errors"
"fmt"
"io"
"strings"
@@ -86,29 +87,35 @@ func (c *Config) NewInputPipes() {
// NewNopInputPipe creates a new input pipe that will silently drop all messages in the input.
func (c *Config) NewNopInputPipe() {
c.stdinPipe = ioutils.NopWriteCloser(io.Discard)
c.stdinPipe = &nopWriteCloser{io.Discard}
}
type nopWriteCloser struct {
io.Writer
}
func (w *nopWriteCloser) Close() error { return nil }
// CloseStreams ensures that the configured streams are properly closed.
func (c *Config) CloseStreams() error {
var errors []string
var errs []string
if c.stdin != nil {
if err := c.stdin.Close(); err != nil {
errors = append(errors, fmt.Sprintf("error close stdin: %s", err))
errs = append(errs, fmt.Sprintf("error close stdin: %s", err))
}
}
if err := c.stdout.Clean(); err != nil {
errors = append(errors, fmt.Sprintf("error close stdout: %s", err))
errs = append(errs, fmt.Sprintf("error close stdout: %s", err))
}
if err := c.stderr.Clean(); err != nil {
errors = append(errors, fmt.Sprintf("error close stderr: %s", err))
errs = append(errs, fmt.Sprintf("error close stderr: %s", err))
}
if len(errors) > 0 {
return fmt.Errorf(strings.Join(errors, "\n"))
if len(errs) > 0 {
return errors.New(strings.Join(errs, "\n"))
}
return nil

View File

@@ -1,5 +1,5 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.21
//go:build go1.22
package container // import "github.com/docker/docker/container"
@@ -30,8 +30,12 @@ const (
var (
// ErrNameReserved is an error which is returned when a name is requested to be reserved that already is reserved
//
// Deprecated: check for [errdefs.Conflict] errors instead (using [errdefs.IsConflict].
ErrNameReserved = errors.New("name is reserved")
// ErrNameNotReserved is an error which is returned when trying to find a name that is not reserved
//
// Deprecated: check for [errdefs.NotFound] errors instead (using [errdefs.IsNotFound].
ErrNameNotReserved = errors.New("name is not reserved")
)
@@ -112,6 +116,7 @@ func NewViewDB() (*ViewDB, error) {
// GetByPrefix returns a container with the given ID prefix. It returns an
// error if an empty prefix was given or if multiple containers match the prefix.
// It returns an [errdefs.NotFound] if the given s yielded no results.
func (db *ViewDB) GetByPrefix(s string) (string, error) {
if s == "" {
return "", errdefs.InvalidParameter(errors.New("prefix can't be empty"))
@@ -152,7 +157,7 @@ func (db *ViewDB) withTxn(cb func(*memdb.Txn) error) error {
err := cb(txn)
if err != nil {
txn.Abort()
return errdefs.System(err)
return err
}
txn.Commit()
return nil
@@ -183,10 +188,9 @@ func (db *ViewDB) Delete(c *Container) error {
})
}
// ReserveName registers a container ID to a name
// ReserveName is idempotent
// Attempting to reserve a container ID to a name that already exists results in an `ErrNameReserved`
// A name reservation is globally unique
// ReserveName registers a container ID to a name. ReserveName is idempotent,
// but returns an [errdefs.Conflict] when attempting to reserve a container ID
// to a name that already is reserved.
func (db *ViewDB) ReserveName(name, containerID string) error {
return db.withTxn(func(txn *memdb.Txn) error {
s, err := txn.First(memdbNamesTable, memdbIDIndex, name)
@@ -195,7 +199,7 @@ func (db *ViewDB) ReserveName(name, containerID string) error {
}
if s != nil {
if s.(nameAssociation).containerID != containerID {
return ErrNameReserved
return errdefs.Conflict(ErrNameReserved) //nolint:staticcheck // ignore SA1019: ErrNameReserved is deprecated.
}
return nil
}
@@ -235,6 +239,7 @@ func (v *View) All() ([]Snapshot, error) {
}
// Get returns an item by id. Returned objects must never be modified.
// It returns an [errdefs.NotFound] if the given id was not found.
func (v *View) Get(id string) (*Snapshot, error) {
s, err := v.txn.First(memdbContainersTable, memdbIDIndex, id)
if err != nil {
@@ -266,13 +271,14 @@ func (v *View) getNames(containerID string) []string {
}
// GetID returns the container ID that the passed in name is reserved to.
// It returns an [errdefs.NotFound] if the given id was not found.
func (v *View) GetID(name string) (string, error) {
s, err := v.txn.First(memdbNamesTable, memdbIDIndex, name)
if err != nil {
return "", errdefs.System(err)
}
if s == nil {
return "", ErrNameNotReserved
return "", errdefs.NotFound(ErrNameNotReserved) //nolint:staticcheck // ignore SA1019: ErrNameNotReserved is deprecated.
}
return s.(nameAssociation).containerID, nil
}

View File

@@ -9,6 +9,7 @@ import (
"github.com/docker/docker/api/types"
containertypes "github.com/docker/docker/api/types/container"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/stringid"
"github.com/google/uuid"
"gotest.tools/v3/assert"
@@ -115,7 +116,10 @@ func TestNames(t *testing.T) {
assert.Check(t, db.ReserveName("name1", "containerid1"))
assert.Check(t, db.ReserveName("name1", "containerid1")) // idempotent
assert.Check(t, db.ReserveName("name2", "containerid2"))
assert.Check(t, is.Error(db.ReserveName("name2", "containerid3"), ErrNameReserved.Error()))
err = db.ReserveName("name2", "containerid3")
assert.Check(t, is.ErrorType(err, errdefs.IsConflict))
assert.Check(t, is.ErrorIs(err, ErrNameReserved)) //nolint:staticcheck // ignore SA1019: ErrNameReserved is deprecated.
// Releasing a name allows the name to point to something else later.
assert.Check(t, db.ReleaseName("name2"))
@@ -132,7 +136,8 @@ func TestNames(t *testing.T) {
assert.Check(t, is.Equal("containerid3", id))
_, err = view.GetID("notreserved")
assert.Check(t, is.Error(err, ErrNameNotReserved.Error()))
assert.Check(t, is.ErrorType(err, errdefs.IsNotFound))
assert.Check(t, is.ErrorIs(err, ErrNameNotReserved)) //nolint:staticcheck // ignore SA1019: ErrNameNotReserved is deprecated.
// Releasing and re-reserving a name doesn't affect the snapshot.
assert.Check(t, db.ReleaseName("name2"))

View File

@@ -269,13 +269,6 @@ init() {
# - sysctl: "net.ipv4.ip_unprivileged_port_start"
# - external binary: slirp4netns
# - external binary: fuse-overlayfs
# check RootlessKit functionality. RootlessKit will print hints if something is still unsatisfied.
# (e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint)
if ! rootlesskit true; then
ERROR "RootlessKit failed, see the error messages and https://rootlesscontaine.rs/getting-started/common/ ."
exit 1
fi
}
# CLI subcommand: "check"
@@ -400,7 +393,20 @@ cli_ctx_rm() {
# CLI subcommand: "install"
cmd_entrypoint_install() {
init
# requirements are already checked in init()
# Most requirements are already checked in init(), except the smoke test below for RootlessKit.
# https://github.com/docker/docker-install/issues/417
# check RootlessKit functionality. RootlessKit will print hints if something is still unsatisfied.
# (e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint)
if ! rootlesskit true; then
if [ -z "$OPT_FORCE" ]; then
ERROR "RootlessKit failed, see the error messages and https://rootlesscontaine.rs/getting-started/common/ . Set --force to ignore."
exit 1
else
WARNING "RootlessKit failed, see the error messages and https://rootlesscontaine.rs/getting-started/common/ ."
fi
fi
if [ -z "$SYSTEMD" ]; then
install_nonsystemd
else

View File

@@ -54,6 +54,30 @@ if ! [ -d "$HOME" ]; then
exit 1
fi
mount_directory() {
if [ -z "$_DOCKERD_ROOTLESS_CHILD" ]; then
echo "mount_directory should be called from the child context. Otherwise data loss is at risk" >&2
exit 1
fi
DIRECTORY="$1"
if [ ! -d "$DIRECTORY" ]; then
return
fi
# Bind mount directory: this makes this directory visible to
# Dockerd, even if it is originally a symlink, given Dockerd does
# not always follow symlinks. Some directories might also be
# "copied-up", meaning that they will also be writable on the child
# namespace; this will be the case only if they are provided as
# --copy-up to the rootlesskit.
DIRECTORY_REALPATH=$(realpath "$DIRECTORY")
MOUNT_OPTIONS="${2:---bind}"
rm -rf "$DIRECTORY"
mkdir -p "$DIRECTORY"
mount $MOUNT_OPTIONS "$DIRECTORY_REALPATH" "$DIRECTORY"
}
rootlesskit=""
for f in docker-rootlesskit rootlesskit; do
if command -v $f > /dev/null 2>&1; then
@@ -139,6 +163,25 @@ if [ -z "$_DOCKERD_ROOTLESS_CHILD" ]; then
"$0" "$@"
else
[ "$_DOCKERD_ROOTLESS_CHILD" = 1 ]
# The Container Device Interface (CDI) specs can be found by default
# under {/etc,/var/run}/cdi. More information at:
# https://github.com/cncf-tags/container-device-interface
#
# In order to use the Container Device Interface (CDI) integration,
# the CDI paths need to exist before the Docker daemon is started in
# order for it to read the CDI specification files. Otherwise, a
# Docker daemon restart will be required for the daemon to discover
# them.
#
# If another set of CDI paths (other than the default /etc/cdi and
# /var/run/cdi) are configured through the Docker configuration file
# (using "cdi-spec-dirs"), they need to be bind mounted in rootless
# mode; otherwise the Docker daemon won't have access to the CDI
# specification files.
mount_directory /etc/cdi
mount_directory /var/run/cdi
# remove the symlinks for the existing files in the parent namespace if any,
# so that we can create our own files in our mount namespace.
rm -f /run/docker /run/containerd /run/xtables.lock
@@ -153,10 +196,7 @@ else
if [ "$(stat -c %T -f /etc)" = "tmpfs" ] && [ -L "/etc/ssl" ]; then
# Workaround for "x509: certificate signed by unknown authority" on openSUSE Tumbleweed.
# https://github.com/rootless-containers/rootlesskit/issues/225
realpath_etc_ssl=$(realpath /etc/ssl)
rm -f /etc/ssl
mkdir /etc/ssl
mount --rbind ${realpath_etc_ssl} /etc/ssl
mount_directory /etc/ssl "--rbind"
fi
exec "$dockerd" "$@"

View File

@@ -141,7 +141,7 @@ func (daemon *Daemon) containerAttach(c *container.Container, cfg *stream.Attach
if !ok {
return logger.ErrReadLogsNotSupported{}
}
logs := cLog.ReadLogs(logger.ReadConfig{Tail: -1})
logs := cLog.ReadLogs(context.TODO(), logger.ReadConfig{Tail: -1})
defer logs.ConsumerGone()
LogLoop:

View File

@@ -34,7 +34,7 @@ func newCDIDeviceDriver(cdiSpecDirs ...string) *deviceDriver {
// We create a spec updater that always returns an error.
// This error will be returned only when a CDI device is requested.
// This ensures that daemon startup is not blocked by a CDI registry initialization failure or being disabled
// by configuratrion.
// by configuration.
errorOnUpdateSpec := func(s *specs.Spec, dev *deviceInstance) error {
return fmt.Errorf("CDI device injection failed: %w", err)
}

View File

@@ -396,7 +396,7 @@ func TestServiceConvertFromGRPCCredentialSpec(t *testing.T) {
}
}
func TestServiceConvertToGRPCNetworkAtachmentRuntime(t *testing.T) {
func TestServiceConvertToGRPCNetworkAttachmentRuntime(t *testing.T) {
someid := "asfjkl"
s := swarmtypes.ServiceSpec{
TaskTemplate: swarmtypes.TaskSpec{

View File

@@ -19,7 +19,7 @@ func TestWaitNodeAttachment(t *testing.T) {
// and add some attachments to it
attachmentStore := emptyDaemon.GetAttachmentStore()
// create a set of attachments to put into the attahcment store
// create a set of attachments to put into the attachment store
attachments := map[string]string{
"network1": "10.1.2.3/24",
}

View File

@@ -3,13 +3,14 @@ package cluster // import "github.com/docker/docker/daemon/cluster"
import (
"net"
"github.com/docker/docker/internal/nlwrap"
"github.com/vishvananda/netlink"
)
func (c *Cluster) resolveSystemAddr() (net.IP, error) {
// Use the system's only device IP address, or fail if there are
// multiple addresses to choose from.
interfaces, err := netlink.LinkList()
interfaces, err := nlwrap.LinkList()
if err != nil {
return nil, err
}
@@ -26,7 +27,7 @@ func (c *Cluster) resolveSystemAddr() (net.IP, error) {
continue
}
addrs, err := netlink.AddrList(intf, netlink.FAMILY_ALL)
addrs, err := nlwrap.AddrList(intf, netlink.FAMILY_ALL)
if err != nil {
continue
}

View File

@@ -305,6 +305,7 @@ func New() (*Config, error) {
},
ContainerdNamespace: DefaultContainersNamespace,
ContainerdPluginNamespace: DefaultPluginNamespace,
Features: make(map[string]bool),
DefaultRuntime: StockRuntimeName,
MinAPIVersion: defaultMinAPIVersion,
},

View File

@@ -4,6 +4,7 @@ import (
"testing"
"github.com/docker/docker/api/types/container"
dopts "github.com/docker/docker/internal/opts"
"github.com/docker/docker/opts"
"github.com/spf13/pflag"
"gotest.tools/v3/assert"
@@ -121,6 +122,72 @@ func TestDaemonConfigurationMergeShmSize(t *testing.T) {
assert.Check(t, is.Equal(int64(expectedValue), cc.ShmSize.Value()))
}
func TestDaemonConfigurationFeatures(t *testing.T) {
tests := []struct {
name, config, flags string
expectedValue map[string]bool
expectedErr string
}{
{
name: "enable from file",
config: `{"features": {"containerd-snapshotter": true}}`,
expectedValue: map[string]bool{"containerd-snapshotter": true},
},
{
name: "enable from flags",
config: `{}`,
flags: "containerd-snapshotter=true",
expectedValue: map[string]bool{"containerd-snapshotter": true},
},
{
name: "disable from file",
config: `{"features": {"containerd-snapshotter": false}}`,
expectedValue: map[string]bool{"containerd-snapshotter": false},
},
{
name: "disable from flags",
config: `{}`,
flags: "containerd-snapshotter=false",
expectedValue: map[string]bool{"containerd-snapshotter": false},
},
{
name: "conflict",
config: `{"features": {"containerd-snapshotter": true}}`,
flags: "containerd-snapshotter=true",
expectedErr: `the following directives are specified both as a flag and in the configuration file: features: (from flag: map[containerd-snapshotter:true], from file: map[containerd-snapshotter:true])`,
},
{
name: "invalid config value",
config: `{"features": {"containerd-snapshotter": "not-a-boolean"}}`,
expectedErr: `json: cannot unmarshal string into Go struct field`,
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
c, err := New()
assert.NilError(t, err)
configFile := makeConfigFile(t, tc.config)
flags := pflag.NewFlagSet("test", pflag.ContinueOnError)
flags.Var(dopts.NewNamedSetOpts("features", c.Features), "feature", "Enable feature in the daemon")
if tc.flags != "" {
err = flags.Set("feature", tc.flags)
assert.NilError(t, err)
}
cc, err := MergeDaemonConfigurations(c, flags, configFile)
if tc.expectedErr != "" {
assert.ErrorContains(t, err, tc.expectedErr)
} else {
assert.NilError(t, err)
assert.Check(t, is.DeepEqual(tc.expectedValue, cc.Features))
}
})
}
}
func TestUnixGetInitPath(t *testing.T) {
testCases := []struct {
config *Config

View File

@@ -557,7 +557,6 @@ func TestValidateMinAPIVersion(t *testing.T) {
}
})
}
}
func TestConfigInvalidDNS(t *testing.T) {

View File

@@ -70,12 +70,16 @@ func (daemon *Daemon) GetContainer(prefixOrName string) (*container.Container, e
// Exists returns a true if a container of the specified ID or name exists,
// false otherwise.
//
// Deprecated: use [Daemon.GetContainer] to look up a container by ID, Name, or ID-prefix. This function will be removed in the next release.
func (daemon *Daemon) Exists(id string) bool {
c, _ := daemon.GetContainer(id)
return c != nil
}
// IsPaused returns a bool indicating if the specified container is paused.
//
// Deprecated: use [Daemon.GetContainer] to look up a container by ID, Name, or ID-prefix, and use [container.State.IsPaused]. This function will be removed in the next release.
func (daemon *Daemon) IsPaused(id string) bool {
c, _ := daemon.GetContainer(id)
return c.State.IsPaused()

View File

@@ -1,5 +1,5 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.21
//go:build go1.22
package daemon // import "github.com/docker/docker/daemon"

View File

@@ -44,7 +44,7 @@ func (c cacheAdaptor) Get(id image.ID) (*image.Image, error) {
return nil, fmt.Errorf("resolveImage: %w", err)
}
var errFound = errors.New("success")
errFound := errors.New("success")
err = c.is.walkImageManifests(ctx, c8dImg, func(img *ImageManifest) error {
desc, err := img.Config(ctx)
if err != nil {

View File

@@ -1,3 +1,6 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.22
package containerd
import (
@@ -19,6 +22,7 @@ import (
"github.com/docker/docker/daemon/images"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/image"
"github.com/docker/docker/internal/sliceutil"
imagespec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
@@ -26,8 +30,6 @@ import (
"golang.org/x/sync/semaphore"
)
var truncatedID = regexp.MustCompile(`^(sha256:)?([a-f0-9]{4,64})$`)
var errInconsistentData error = errors.New("consistency error: data changed during operation, retry")
// GetImage returns an image corresponding to the image referred to by refOrID.
@@ -111,7 +113,7 @@ func (i *ImageService) GetImage(ctx context.Context, refOrID string, options bac
}
img.Details = &image.Details{
References: refs,
References: sliceutil.Dedup(refs),
Size: size,
Metadata: nil,
Driver: i.snapshotter,
@@ -202,10 +204,8 @@ func (i *ImageService) GetImageManifest(ctx context.Context, refOrID string, opt
}
if options.Platform != nil {
if plat == nil {
return nil, errdefs.NotFound(errors.Errorf("image with reference %s was found but does not match the specified platform: wanted %s, actual: nil", refOrID, platforms.Format(*options.Platform)))
} else if !platform.Match(*plat) {
return nil, errdefs.NotFound(errors.Errorf("image with reference %s was found but does not match the specified platform: wanted %s, actual: %s", refOrID, platforms.Format(*options.Platform), platforms.Format(*plat)))
if plat == nil || !platform.Match(*plat) {
return nil, errdefs.NotFound(errors.Errorf("image with reference %s was found but does not provide the specified platform (%s)", refOrID, platforms.FormatAll(*options.Platform)))
}
}
@@ -326,9 +326,8 @@ func (i *ImageService) resolveImage(ctx context.Context, refOrID string) (contai
}
}
// If the identifier could be a short ID, attempt to match
if truncatedID.MatchString(refOrID) {
idWithoutAlgo := strings.TrimPrefix(refOrID, "sha256:")
// If the identifier could be a short ID, attempt to match.
if idWithoutAlgo := checkTruncatedID(refOrID); idWithoutAlgo != "" { // Valid ID.
filters := []string{
fmt.Sprintf("name==%q", ref), // Or it could just look like one.
"target.digest~=" + strconv.Quote(fmt.Sprintf(`^sha256:%s[0-9a-fA-F]{%d}$`, regexp.QuoteMeta(idWithoutAlgo), 64-len(idWithoutAlgo))),
@@ -435,7 +434,7 @@ func (i *ImageService) resolveAllReferences(ctx context.Context, refOrID string)
var dgst digest.Digest
var img *containerdimages.Image
if truncatedID.MatchString(refOrID) {
if idWithoutAlgo := checkTruncatedID(refOrID); idWithoutAlgo != "" { // Valid ID.
if d, ok := parsed.(reference.Digested); ok {
if cimg, err := i.images.Get(ctx, d.String()); err == nil {
img = &cimg
@@ -451,7 +450,6 @@ func (i *ImageService) resolveAllReferences(ctx context.Context, refOrID string)
dgst = d.Digest()
}
} else {
idWithoutAlgo := strings.TrimPrefix(refOrID, "sha256:")
name := reference.TagNameOnly(parsed.(reference.Named)).String()
filters := []string{
fmt.Sprintf("name==%q", name), // Or it could just look like one.
@@ -551,3 +549,20 @@ func (i *ImageService) resolveAllReferences(ctx context.Context, refOrID string)
return img, imgs, nil
}
// checkTruncatedID checks id for validity. If id is invalid, an empty string
// is returned; otherwise, the ID without the optional "sha256:" prefix is
// returned. The validity check is equivalent to
// regexp.MustCompile(`^(sha256:)?([a-f0-9]{4,64})$`).MatchString(id).
func checkTruncatedID(id string) string {
id = strings.TrimPrefix(id, "sha256:")
if l := len(id); l < 4 || l > 64 {
return ""
}
for _, c := range id {
if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
return ""
}
}
return id
}

View File

@@ -106,10 +106,11 @@ func (i *ImageService) GetImageAndReleasableLayer(ctx context.Context, refOrID s
}
}
ctx, _, err := i.client.WithLease(ctx, leases.WithRandomID(), leases.WithExpiration(1*time.Hour))
ctx, release, err := i.withLease(ctx, true)
if err != nil {
return nil, nil, fmt.Errorf("failed to create lease for commit: %w", err)
}
defer release()
// TODO(laurazard): do we really need a new method here to pull the image?
imgDesc, err := i.pullForBuilder(ctx, refOrID, opts.AuthConfig, opts.Output, opts.Platform)
@@ -234,9 +235,9 @@ func newROLayerForImage(ctx context.Context, imgDesc *ocispec.Descriptor, i *Ima
func createLease(ctx context.Context, lm leases.Manager) (context.Context, leases.Lease, error) {
lease, err := lm.Create(ctx,
leases.WithExpiration(time.Hour*24),
leases.WithExpiration(leaseExpireDuration),
leases.WithLabels(map[string]string{
"org.mobyproject.lease.classicbuilder": "true",
pruneLeaseLabel: "true",
}),
)
if err != nil {
@@ -493,17 +494,11 @@ func (i *ImageService) createImageOCI(ctx context.Context, imgToCreate imagespec
parentDigest digest.Digest, layers []ocispec.Descriptor,
containerConfig container.Config,
) (dimage.ID, error) {
// Necessary to prevent the contents from being GC'd
// between writing them here and creating an image
ctx, release, err := i.client.WithLease(ctx, leases.WithRandomID(), leases.WithExpiration(1*time.Hour))
ctx, release, err := i.withLease(ctx, false)
if err != nil {
return "", err
}
defer func() {
if err := release(context.WithoutCancel(ctx)); err != nil {
log.G(ctx).WithError(err).Warn("failed to release lease created for create")
}
}()
defer release()
manifestDesc, ccDesc, err := writeContentsForImage(ctx, i.snapshotter, i.content, imgToCreate, layers, containerConfig)
if err != nil {

View File

@@ -13,7 +13,6 @@ import (
// getImagesWithLabel returns all images that have the matching label key and value.
func (i *ImageService) getImagesWithLabel(ctx context.Context, labelKey string, labelValue string) ([]image.ID, error) {
imgs, err := i.images.List(ctx, "labels."+labelKey+"=="+labelValue)
if err != nil {
return []image.ID{}, errdefs.System(errors.Wrap(err, "failed to list all images"))
}

View File

@@ -12,7 +12,6 @@ import (
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/diff"
"github.com/containerd/containerd/leases"
"github.com/containerd/containerd/mount"
"github.com/containerd/containerd/pkg/cleanup"
"github.com/containerd/containerd/snapshots"
@@ -67,16 +66,11 @@ func (i *ImageService) CommitImage(ctx context.Context, cc backend.CommitConfig)
sn = i.client.SnapshotService(container.Driver)
)
// Don't gc me and clean the dirty data after 1 hour!
ctx, release, err := i.client.WithLease(ctx, leases.WithRandomID(), leases.WithExpiration(1*time.Hour))
ctx, release, err := i.withLease(ctx, false)
if err != nil {
return "", fmt.Errorf("failed to create lease for commit: %w", err)
}
defer func() {
if err := release(context.WithoutCancel(ctx)); err != nil {
log.G(ctx).WithError(err).Warn("failed to release lease created for commit")
}
}()
defer release()
diffLayerDesc, diffID, err := i.createDiff(ctx, cc.ContainerID, sn, cs, differ)
if err != nil {

View File

@@ -254,7 +254,6 @@ func TestImageDelete(t *testing.T) {
}
})
}
}
type testContainerStore struct{}

View File

@@ -67,19 +67,14 @@ func (i *ImageService) ExportImage(ctx context.Context, names []string, outStrea
archive.WithSkipMissing(i.content),
}
leasesManager := i.client.LeasesService()
lease, err := leasesManager.Create(ctx, leases.WithRandomID())
ctx, done, err := i.withLease(ctx, false)
if err != nil {
return errdefs.System(err)
}
defer func() {
if err := leasesManager.Delete(ctx, lease); err != nil {
log.G(ctx).WithError(err).Warn("cleaning up lease")
}
}()
defer done()
addLease := func(ctx context.Context, target ocispec.Descriptor) error {
return leaseContent(ctx, i.content, leasesManager, lease, target)
return i.leaseContent(ctx, i.content, target)
}
exportImage := func(ctx context.Context, target ocispec.Descriptor, ref reference.Named) error {
@@ -131,7 +126,6 @@ func (i *ImageService) ExportImage(ctx context.Context, names []string, outStrea
for _, img := range imgs {
ref, err := reference.ParseNamed(img.Name)
if err != nil {
log.G(ctx).WithFields(log.Fields{
"image": img.Name,
@@ -207,7 +201,13 @@ func (i *ImageService) ExportImage(ctx context.Context, names []string, outStrea
// leaseContent will add a resource to the lease for each child of the descriptor making sure that it and
// its children won't be deleted while the lease exists
func leaseContent(ctx context.Context, store content.Store, leasesManager leases.Manager, lease leases.Lease, desc ocispec.Descriptor) error {
func (i *ImageService) leaseContent(ctx context.Context, store content.Store, desc ocispec.Descriptor) error {
lid, ok := leases.FromContext(ctx)
if !ok {
return nil
}
lease := leases.Lease{ID: lid}
leasesManager := i.client.LeasesService()
return containerdimages.Walk(ctx, containerdimages.HandlerFunc(func(ctx context.Context, desc ocispec.Descriptor) ([]ocispec.Descriptor, error) {
_, err := store.Info(ctx, desc.Digest)
if err != nil {
@@ -291,6 +291,17 @@ func (i *ImageService) LoadImage(ctx context.Context, inTar io.ReadCloser, outSt
return nil
}
imgPlat, err := platformImg.ImagePlatform(ctx)
if err != nil {
logger.WithError(err).Warn("failed to read image platform, skipping unpack")
return nil
}
// Only unpack the image if it matches the host platform
if !i.hostPlatformMatcher().Match(imgPlat) {
return nil
}
unpacked, err := platformImg.IsUnpacked(ctx, i.snapshotter)
if err != nil {
logger.WithError(err).Warn("failed to check if image is unpacked")
@@ -299,7 +310,6 @@ func (i *ImageService) LoadImage(ctx context.Context, inTar io.ReadCloser, outSt
if !unpacked {
err = platformImg.Unpack(ctx, i.snapshotter)
if err != nil {
return errdefs.System(err)
}
@@ -307,12 +317,14 @@ func (i *ImageService) LoadImage(ctx context.Context, inTar io.ReadCloser, outSt
logger.WithField("alreadyUnpacked", unpacked).WithError(err).Debug("unpack")
return nil
})
if err != nil {
return errors.Wrap(err, "failed to unpack loaded image")
}
fmt.Fprintf(progress, "%s: %s\n", loadedMsg, name)
i.LogImageEvent(img.Target.Digest.String(), img.Target.Digest.String(), events.ActionLoad)
if err != nil {
// The image failed to unpack, but is already imported, log the error but don't fail the whole load.
fmt.Fprintf(progress, "Error unpacking image %s: %v\n", name, err)
}
}
return nil

View File

@@ -327,7 +327,7 @@ func (i *ImageService) unpackImage(ctx context.Context, snapshotter string, img
return nil
}
// detectCompression dectects the reader compression type.
// detectCompression detects the reader compression type.
func detectCompression(bufRd *bufio.Reader) (archive.Compression, error) {
bs, err := bufRd.Peek(10)
if err != nil && err != io.EOF {

View File

@@ -1,3 +1,6 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.22
package containerd
import (
@@ -21,8 +24,9 @@ import (
"github.com/docker/docker/api/types/filters"
imagetypes "github.com/docker/docker/api/types/image"
timetypes "github.com/docker/docker/api/types/time"
"github.com/docker/docker/container"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/internal/sliceutil"
"github.com/moby/buildkit/util/attestation"
dockerspec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/identity"
@@ -209,6 +213,7 @@ func (i *ImageService) Images(ctx context.Context, opts imagetypes.ListOptions)
func (i *ImageService) imageSummary(ctx context.Context, img images.Image, platformMatcher platforms.MatchComparer,
opts imagetypes.ListOptions, tagsByDigest map[digest.Digest][]string,
) (_ *imagetypes.Summary, allChainIDs []digest.Digest, _ error) {
var manifestSummaries []imagetypes.ManifestSummary
// Total size of the image including all its platform
var totalSize int64
@@ -223,67 +228,148 @@ func (i *ImageService) imageSummary(ctx context.Context, img images.Image, platf
var best *ImageManifest
var bestPlatform ocispec.Platform
err := i.walkImageManifests(ctx, img, func(img *ImageManifest) error {
if isPseudo, err := img.IsPseudoImage(ctx); isPseudo || err != nil {
err := i.walkReachableImageManifests(ctx, img, func(img *ImageManifest) error {
target := img.Target()
logger := log.G(ctx).WithFields(log.Fields{
"image": img.Name(),
"digest": target.Digest,
"manifest": target,
})
available, err := img.CheckContentAvailable(ctx)
if err != nil && !errdefs.IsNotFound(err) {
logger.WithError(err).Warn("checking availability of platform specific manifest failed")
return nil
}
available, err := img.CheckContentAvailable(ctx)
mfstSummary := imagetypes.ManifestSummary{
ID: target.Digest.String(),
Available: available,
Descriptor: target,
Kind: imagetypes.ManifestKindUnknown,
}
if opts.Manifests {
defer func() {
manifestSummaries = append(manifestSummaries, mfstSummary)
}()
}
contentSize, err := img.Size(ctx)
if err != nil {
log.G(ctx).WithFields(log.Fields{
"error": err,
"manifest": img.Target(),
"image": img.Name(),
}).Warn("checking availability of platform specific manifest failed")
if !cerrdefs.IsNotFound(err) {
logger.WithError(err).Warn("failed to determine size")
}
} else {
mfstSummary.Size.Content = contentSize
totalSize += contentSize
mfstSummary.Size.Total += contentSize
}
isPseudo, err := img.IsPseudoImage(ctx)
// Ignore not found error as it's expected in case where the image is
// not fully available. Otherwise, just continue to the next manifest,
// so we don't error out the whole list in case the error is related to
// the content itself (e.g. corrupted data) or just manifest kind that
// we don't know about (yet).
if err != nil && !errdefs.IsNotFound(err) {
logger.WithError(err).Debug("pseudo image check failed")
return nil
}
logger = logger.WithField("isPseudo", isPseudo)
if isPseudo {
if img.IsAttestation() {
if s := target.Annotations[attestation.DockerAnnotationReferenceDigest]; s != "" {
dgst, err := digest.Parse(s)
if err != nil {
logger.WithError(err).Warn("failed to parse attestation digest")
return nil
}
mfstSummary.Kind = imagetypes.ManifestKindAttestation
mfstSummary.AttestationData = &imagetypes.AttestationProperties{For: dgst}
}
}
return nil
}
mfstSummary.Kind = imagetypes.ManifestKindImage
mfstSummary.ImageData = &imagetypes.ImageProperties{}
if target.Platform != nil {
mfstSummary.ImageData.Platform = *target.Platform
}
if !available {
return nil
}
conf, err := img.Config(ctx)
if err != nil {
return err
logger.WithError(err).Warn("failed to read image config")
return nil
}
var dockerImage dockerspec.DockerOCIImage
if err := readConfig(ctx, i.content, conf, &dockerImage); err != nil {
return err
logger.WithError(err).Warn("failed to read image config")
return nil
}
target := img.Target()
if target.Platform == nil {
mfstSummary.ImageData.Platform = dockerImage.Platform
}
diffIDs, err := img.RootFS(ctx)
if err != nil {
return err
logger.WithError(err).Warn("failed to read image config")
return nil
}
chainIDs := identity.ChainIDs(diffIDs)
ts, _, err := i.singlePlatformSize(ctx, img)
unpackedSize, imgContentSize, err := i.singlePlatformSize(ctx, img)
if err != nil {
return err
logger.WithError(err).Warn("failed to determine platform specific size")
return nil
}
totalSize += ts
// If the image-specific content size calculation produces different result
// than the "generic" one, adjust the total size with the difference.
// Note: This shouldn't happen unless the implementation changes or the
// content is added/removed during the list operation.
if contentSize != imgContentSize {
logger.WithFields(log.Fields{
"contentSize": contentSize,
"imgContentSize": imgContentSize,
}).Warn("content size calculation mismatch")
mfstSummary.Size.Content = contentSize
// contentSize was already added to total, adjust it by the difference
// between the newly calculated size and the old size.
d := imgContentSize - contentSize
totalSize += d
mfstSummary.Size.Total += d
}
mfstSummary.ImageData.Size.Unpacked = unpackedSize
mfstSummary.Size.Total += unpackedSize
totalSize += unpackedSize
allChainsIDs = append(allChainsIDs, chainIDs...)
if opts.ContainerCount {
i.containers.ApplyAll(func(c *container.Container) {
if c.ImageManifest != nil && c.ImageManifest.Digest == target.Digest {
containersCount++
}
})
}
var platform ocispec.Platform
if target.Platform != nil {
platform = *target.Platform
} else {
platform = dockerImage.Platform
for _, c := range i.containers.List() {
if c.ImageManifest != nil && c.ImageManifest.Digest == target.Digest {
mfstSummary.ImageData.Containers = append(mfstSummary.ImageData.Containers, c.ID)
containersCount++
}
}
platform := mfstSummary.ImageData.Platform
// Filter out platforms that don't match the requested platform. Do it
// after the size, container count and chainIDs are summed up to have
// the single combined entry still represent the whole multi-platform
@@ -305,17 +391,25 @@ func (i *ImageService) imageSummary(ctx context.Context, img images.Image, platf
"error": err,
"image": img.Name,
}).Warn("unexpected image target (neither a manifest nor index)")
return nil, nil, nil
} else {
return nil, nil, err
}
return nil, nil, err
}
if best == nil {
// TODO we should probably show *something* for images we've pulled
// but are 100% shallow or an empty manifest list/index
// ("tianon/scratch:index" is an empty example image index and
// "tianon/scratch:list" is an empty example manifest list)
return nil, nil, nil
target := img.Target
return &imagetypes.Summary{
ID: target.Digest.String(),
RepoDigests: []string{target.Digest.String()},
RepoTags: tagsByDigest[target.Digest],
Size: totalSize,
// -1 indicates that the value has not been set (avoids ambiguity
// between 0 (default) and "not set". We cannot use a pointer (nil)
// for this, as the JSON representation uses "omitempty", which would
// consider both "0" and "nil" to be "empty".
SharedSize: -1,
Containers: -1,
}, nil, nil
}
image, err := i.singlePlatformImage(ctx, i.content, tagsByDigest[best.RealTarget.Digest], best)
@@ -323,6 +417,7 @@ func (i *ImageService) imageSummary(ctx context.Context, img images.Image, platf
return nil, nil, err
}
image.Size = totalSize
image.Manifests = manifestSummaries
if opts.ContainerCount {
image.Containers = containersCount
@@ -330,7 +425,7 @@ func (i *ImageService) imageSummary(ctx context.Context, img images.Image, platf
return image, allChainsIDs, nil
}
func (i *ImageService) singlePlatformSize(ctx context.Context, imgMfst *ImageManifest) (totalSize int64, contentSize int64, _ error) {
func (i *ImageService) singlePlatformSize(ctx context.Context, imgMfst *ImageManifest) (unpackedSize int64, contentSize int64, _ error) {
// TODO(thaJeztah): do we need to take multiple snapshotters into account? See https://github.com/moby/moby/issues/45273
snapshotter := i.snapshotterService(i.snapshotter)
@@ -356,10 +451,7 @@ func (i *ImageService) singlePlatformSize(ctx context.Context, imgMfst *ImageMan
return -1, -1, err
}
// totalSize is the size of the image's packed layers and snapshots
// (unpacked layers) combined.
totalSize = contentSize + unpackedUsage.Size
return totalSize, contentSize, nil
return unpackedUsage.Size, contentSize, nil
}
func (i *ImageService) singlePlatformImage(ctx context.Context, contentStore content.Store, repoTags []string, imageManifest *ImageManifest) (*imagetypes.Summary, error) {
@@ -401,15 +493,19 @@ func (i *ImageService) singlePlatformImage(ctx context.Context, contentStore con
return nil, err
}
totalSize, _, err := i.singlePlatformSize(ctx, imageManifest)
unpackedSize, contentSize, err := i.singlePlatformSize(ctx, imageManifest)
if err != nil {
return nil, errors.Wrapf(err, "failed to calculate size of image %s", imageManifest.Name())
}
// totalSize is the size of the image's packed layers and snapshots
// (unpacked layers) combined.
totalSize := contentSize + unpackedSize
summary := &imagetypes.Summary{
ParentID: rawImg.Labels[imageLabelClassicBuilderParent],
ID: target.String(),
RepoDigests: repoDigests,
RepoDigests: sliceutil.Dedup(repoDigests),
RepoTags: repoTags,
Size: totalSize,
Labels: cfg.Config.Labels,

View File

@@ -7,6 +7,7 @@ import (
"math/rand"
"os"
"path/filepath"
"slices"
"sort"
"strconv"
"testing"
@@ -22,6 +23,7 @@ import (
"github.com/containerd/log/logtest"
"github.com/containerd/platforms"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/container"
daemonevents "github.com/docker/docker/daemon/events"
"github.com/docker/docker/internal/testutils/specialimage"
"github.com/opencontainers/go-digest"
@@ -44,7 +46,13 @@ func imagesFromIndex(index ...*ocispec.Index) []images.Image {
}
func BenchmarkImageList(b *testing.B) {
populateStore := func(ctx context.Context, is *ImageService, dir string, count int) {
populateStore := func(ctx context.Context, is *ImageService, dir string,
count int,
// % chance for each image to spawn containers
containerChance int,
// Maximum container count if the image is decided to spawn containers (chance above)
maxContainerCount int,
) {
// Use constant seed for reproducibility
src := rand.NewSource(1982731263716)
@@ -59,15 +67,34 @@ func BenchmarkImageList(b *testing.B) {
idx, err := specialimage.RandomSinglePlatform(dir, platform, src)
assert.NilError(b, err)
r1 := int(src.Int63())
r2 := int(src.Int63())
imgs := imagesFromIndex(idx)
for _, desc := range imgs {
_, err := is.images.Create(ctx, desc)
assert.NilError(b, err)
if r1%100 >= containerChance {
continue
}
containersCount := r2 % maxContainerCount
for j := 0; j < containersCount; j++ {
id := digest.FromString(desc.Name + strconv.Itoa(i)).String()
target := desc.Target
is.containers.Add(id, &container.Container{
ID: id,
ImageManifest: &target,
})
}
}
}
}
for _, count := range []int{10, 100, 1000} {
count := count
csDir := b.TempDir()
ctx := namespaces.WithNamespace(context.TODO(), "testing-"+strconv.Itoa(count))
@@ -78,7 +105,11 @@ func BenchmarkImageList(b *testing.B) {
}
is := fakeImageService(b, ctx, cs)
populateStore(ctx, is, csDir, count)
// Every generated image has a 10% chance to spawn up to 5 containers
const containerChance = 10
const maxContainerCount = 5
populateStore(ctx, is, csDir, count, containerChance, maxContainerCount)
b.Run(strconv.Itoa(count)+"-images", func(b *testing.B) {
for i := 0; i < b.N; i++ {
@@ -89,6 +120,76 @@ func BenchmarkImageList(b *testing.B) {
}
}
func TestImageListCheckTotalSize(t *testing.T) {
ctx := namespaces.WithNamespace(context.TODO(), "testing")
blobsDir := t.TempDir()
cs := &blobsDirContentStore{blobs: filepath.Join(blobsDir, "blobs/sha256")}
twoplatform, mfstsDescs, err := specialimage.MultiPlatform(blobsDir, "test:latest", []ocispec.Platform{
{OS: "linux", Architecture: "arm64"},
{OS: "linux", Architecture: "amd64"},
})
assert.NilError(t, err)
ctx = logtest.WithT(ctx, t)
service := fakeImageService(t, ctx, cs)
_, err = service.images.Create(ctx, imagesFromIndex(twoplatform)[0])
assert.NilError(t, err)
all, err := service.Images(ctx, imagetypes.ListOptions{Manifests: true})
assert.NilError(t, err)
assert.Check(t, is.Len(all, 1))
assert.Check(t, is.Len(all[0].Manifests, 2))
// TODO: The test snapshotter doesn't do anything, so the size is always 0.
assert.Check(t, is.Equal(all[0].Manifests[0].ImageData.Size.Unpacked, int64(0)))
assert.Check(t, is.Equal(all[0].Manifests[1].ImageData.Size.Unpacked, int64(0)))
mfstArm64 := mfstsDescs[0]
mfstAmd64 := mfstsDescs[1]
indexSize := blobSize(t, ctx, cs, twoplatform.Manifests[0].Digest)
arm64ManifestSize := blobSize(t, ctx, cs, mfstArm64.Digest)
amd64ManifestSize := blobSize(t, ctx, cs, mfstAmd64.Digest)
var arm64Mfst, amd64Mfst ocispec.Manifest
assert.NilError(t, readConfig(ctx, cs, mfstArm64, &arm64Mfst))
assert.NilError(t, readConfig(ctx, cs, mfstAmd64, &amd64Mfst))
// MultiPlatform should produce a single layer. If these fail, the test needs to be adjusted.
assert.Assert(t, is.Len(arm64Mfst.Layers, 1))
assert.Assert(t, is.Len(amd64Mfst.Layers, 1))
arm64ConfigSize := blobSize(t, ctx, cs, arm64Mfst.Config.Digest)
amd64ConfigSize := blobSize(t, ctx, cs, amd64Mfst.Config.Digest)
arm64LayerSize := blobSize(t, ctx, cs, arm64Mfst.Layers[0].Digest)
amd64LayerSize := blobSize(t, ctx, cs, amd64Mfst.Layers[0].Digest)
allTotalSize := indexSize +
arm64ManifestSize + amd64ManifestSize +
arm64ConfigSize + amd64ConfigSize +
arm64LayerSize + amd64LayerSize
assert.Check(t, is.Equal(all[0].Size, allTotalSize-indexSize))
assert.Check(t, is.Equal(all[0].Manifests[0].Size.Content, arm64ManifestSize+arm64ConfigSize+arm64LayerSize))
assert.Check(t, is.Equal(all[0].Manifests[1].Size.Content, amd64ManifestSize+amd64ConfigSize+amd64LayerSize))
// TODO: This should also include the Size.Unpacked, but the test snapshotter doesn't do anything yet
assert.Check(t, is.Equal(all[0].Manifests[0].Size.Total, amd64ManifestSize+amd64ConfigSize+amd64LayerSize))
assert.Check(t, is.Equal(all[0].Manifests[1].Size.Total, amd64ManifestSize+amd64ConfigSize+amd64LayerSize))
}
func blobSize(t *testing.T, ctx context.Context, cs content.Store, dgst digest.Digest) int64 {
info, err := cs.Info(ctx, dgst)
assert.NilError(t, err)
return info.Size
}
func TestImageList(t *testing.T) {
ctx := namespaces.WithNamespace(context.TODO(), "testing")
@@ -106,6 +207,9 @@ func TestImageList(t *testing.T) {
configTarget, err := specialimage.ConfigTarget(blobsDir)
assert.NilError(t, err)
textplain, err := specialimage.TextPlain(blobsDir)
assert.NilError(t, err)
cs := &blobsDirContentStore{blobs: filepath.Join(blobsDir, "blobs/sha256")}
for _, tc := range []struct {
@@ -123,6 +227,10 @@ func TestImageList(t *testing.T) {
assert.Check(t, is.Equal(all[0].ID, multilayer.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[0].RepoTags, []string{"multilayer:latest"}))
assert.Check(t, is.Len(all[0].Manifests, 1))
assert.Check(t, all[0].Manifests[0].Available)
assert.Check(t, is.Equal(all[0].Manifests[0].Kind, imagetypes.ManifestKindImage))
},
},
{
@@ -133,6 +241,18 @@ func TestImageList(t *testing.T) {
assert.Check(t, is.Equal(all[0].ID, twoplatform.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[0].RepoTags, []string{"twoplatform:latest"}))
i := all[0]
assert.Check(t, is.Len(i.Manifests, 2))
assert.Check(t, is.Equal(i.Manifests[0].Kind, imagetypes.ManifestKindImage))
if assert.Check(t, i.Manifests[0].ImageData != nil) {
assert.Check(t, is.Equal(i.Manifests[0].ImageData.Platform.Architecture, "amd64"))
}
assert.Check(t, is.Equal(i.Manifests[1].Kind, imagetypes.ManifestKindImage))
if assert.Check(t, i.Manifests[1].ImageData != nil) {
assert.Check(t, is.Equal(i.Manifests[1].ImageData.Platform.Architecture, "arm64"))
}
},
},
{
@@ -146,23 +266,48 @@ func TestImageList(t *testing.T) {
assert.Check(t, is.Equal(all[1].ID, twoplatform.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[1].RepoTags, []string{"twoplatform:latest"}))
assert.Check(t, is.Len(all[0].Manifests, 1))
assert.Check(t, is.Len(all[1].Manifests, 2))
assert.Check(t, is.Equal(all[0].Manifests[0].Kind, imagetypes.ManifestKindImage))
assert.Check(t, is.Equal(all[1].Manifests[0].Kind, imagetypes.ManifestKindImage))
assert.Check(t, is.Equal(all[1].Manifests[1].Kind, imagetypes.ManifestKindImage))
},
},
{
name: "three images, one is an empty index",
images: imagesFromIndex(multilayer, emptyIndex, twoplatform),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 2))
assert.Check(t, is.Len(all, 3))
},
},
{
// Make sure an invalid image target doesn't break the whole operation
name: "one good image, second has config as a target",
images: imagesFromIndex(multilayer, configTarget),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 1))
assert.Check(t, is.Len(all, 2))
sort.Slice(all, func(i, j int) bool {
return slices.Contains(all[i].RepoTags, "multilayer:latest")
})
assert.Check(t, is.Equal(all[0].ID, multilayer.Manifests[0].Digest.String()))
assert.Check(t, is.Len(all[0].Manifests, 1))
assert.Check(t, is.Equal(all[1].ID, configTarget.Manifests[0].Digest.String()))
assert.Check(t, is.Len(all[1].Manifests, 0))
},
},
{
name: "a non-container image manifest",
images: imagesFromIndex(textplain),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 1))
assert.Check(t, is.Equal(all[0].ID, textplain.Manifests[0].Digest.String()))
assert.Assert(t, is.Len(all[0].Manifests, 0))
},
},
} {
@@ -176,7 +321,9 @@ func TestImageList(t *testing.T) {
assert.NilError(t, err)
}
all, err := service.Images(ctx, tc.opts)
opts := tc.opts
opts.Manifests = true
all, err := service.Images(ctx, opts)
assert.NilError(t, err)
sort.Slice(all, func(i, j int) bool {
@@ -192,7 +339,6 @@ func TestImageList(t *testing.T) {
tc.check(t, all)
})
}
}
func fakeImageService(t testing.TB, ctx context.Context, cs content.Store) *ImageService {
@@ -206,7 +352,7 @@ func fakeImageService(t testing.TB, ctx context.Context, cs content.Store) *Imag
service := &ImageService{
images: metadata.NewImageStore(mdb),
containers: emptyTestContainerStore(),
containers: container.NewMemoryStore(),
content: cs,
eventsService: daemonevents.New(),
snapshotterServices: snapshotters,

View File

@@ -2,14 +2,18 @@ package containerd
import (
"context"
"sort"
"strings"
containerdimages "github.com/containerd/containerd/images"
"github.com/containerd/containerd/leases"
"github.com/containerd/containerd/tracing"
cerrdefs "github.com/containerd/errdefs"
"github.com/containerd/log"
"github.com/distribution/reference"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/container"
"github.com/docker/docker/errdefs"
"github.com/hashicorp/go-multierror"
"github.com/opencontainers/go-digest"
@@ -57,11 +61,42 @@ func (i *ImageService) ImagesPrune(ctx context.Context, fltrs filters.Args) (*im
return nil, err
}
// Prune leases
leaseManager := i.client.LeasesService()
pullLeases, err := leaseManager.List(ctx, pruneLeaseFilter)
if err != nil {
return nil, err
}
for i, lease := range pullLeases {
var opts []leases.DeleteOpt
if i == len(pullLeases)-1 {
opts = append(opts, leases.SynchronousDelete)
}
if err := leaseManager.Delete(ctx, lease, opts...); err != nil {
return nil, err
}
}
return i.pruneUnused(ctx, filterFunc, danglingOnly)
}
// pruneUnused deletes images that are dangling or unused by any container.
// The behavior is controlled by the danglingOnly parameter.
// If danglingOnly is true, only dangling images are deleted.
// Otherwise, all images unused by any container are deleted.
//
// Additionally, the filterFunc parameter is used to filter images that should
// be considered for deletion.
//
// Container created with images specified by an ID only (e.g. `docker run 82d1e9d`)
// will keep at least one image tag with that ID.
//
// In case a digested and tagged reference was used (e.g. `docker run alpine:latest@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1`),
// the alpine:latest image will be kept.
func (i *ImageService) pruneUnused(ctx context.Context, filterFunc imageFilterFunc, danglingOnly bool) (*image.PruneReport, error) {
report := image.PruneReport{}
ctx, span := tracing.StartSpan(ctx, "ImageService.pruneUnused")
span.SetAttributes(tracing.Attribute("danglingOnly", danglingOnly))
defer span.End()
allImages, err := i.images.List(ctx)
if err != nil {
@@ -85,16 +120,52 @@ func (i *ImageService) pruneUnused(ctx context.Context, filterFunc imageFilterFu
if canBePruned {
imagesToPrune[img.Name] = img
}
}
}
usedDigests := filterImagesUsedByContainers(ctx, i.containers.List(), imagesToPrune)
// Sort images by name to make the behavior deterministic and consistent with graphdrivers.
sorted := make([]string, 0, len(imagesToPrune))
for name := range imagesToPrune {
sorted = append(sorted, name)
}
sort.Strings(sorted)
// Make sure we don't delete the last image of a particular digest used by any container.
for _, name := range sorted {
img := imagesToPrune[name]
dgst := img.Target.Digest
if digestRefCount[dgst] > 1 {
digestRefCount[dgst] -= 1
continue
}
if _, isUsed := usedDigests[dgst]; isUsed {
delete(imagesToPrune, name)
}
}
return i.pruneAll(ctx, imagesToPrune)
}
// filterImagesUsedByContainers removes image names that are used by containers
// and returns a map of used image digests.
func filterImagesUsedByContainers(ctx context.Context,
allContainers []*container.Container,
imagesToPrune map[string]containerdimages.Image,
) (usedDigests map[digest.Digest]struct{}) {
ctx, span := tracing.StartSpan(ctx, "filterImagesUsedByContainers")
span.SetAttributes(tracing.Attribute("count", len(allContainers)))
defer span.End()
// Image specified by digests that are used by containers.
usedDigests := map[digest.Digest]struct{}{}
usedDigests = map[digest.Digest]struct{}{}
// Exclude images used by existing containers
for _, ctr := range i.containers.List() {
// If the original image was deleted, make sure we don't delete the dangling image
for _, ctr := range allContainers {
// If the original image was force deleted, make sure we don't delete the dangling image
delete(imagesToPrune, danglingImageName(ctr.ImageID.Digest()))
// Config.Image is the image reference passed by user.
@@ -105,41 +176,48 @@ func (i *ImageService) pruneUnused(ctx context.Context, filterFunc imageFilterFu
// but both will have ImageID="sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1"
imageDgst := ctr.ImageID.Digest()
// If user didn't specify an explicit image, mark the digest as used.
// If user used an full or truncated ID instead of an explicit image name, mark the digest as used.
normalizedImageID := "sha256:" + strings.TrimPrefix(ctr.Config.Image, "sha256:")
if strings.HasPrefix(imageDgst.String(), normalizedImageID) {
fullOrTruncatedID := strings.HasPrefix(imageDgst.String(), normalizedImageID)
digestedRef := strings.HasSuffix(ctr.Config.Image, "@"+imageDgst.String())
if fullOrTruncatedID || digestedRef {
usedDigests[imageDgst] = struct{}{}
continue
}
ref, err := reference.ParseNormalizedNamed(ctr.Config.Image)
log.G(ctx).WithFields(log.Fields{
"ctr": ctr.ID,
"image": ref,
"imageRef": ref,
"imageID": imageDgst,
"nameParseErr": err,
}).Debug("filtering container's image")
if err == nil {
// If user provided a specific image name, exclude that image.
name := reference.TagNameOnly(ref)
delete(imagesToPrune, name.String())
}
}
// Create dangling images for images that will be deleted but are still in use.
for _, img := range imagesToPrune {
dgst := img.Target.Digest
digestRefCount[dgst] -= 1
if digestRefCount[dgst] == 0 {
if _, isUsed := usedDigests[dgst]; isUsed {
if err := i.ensureDanglingImage(ctx, img); err != nil {
return &report, errors.Wrapf(err, "failed to create ensure dangling image for %s", img.Name)
}
// Also exclude repo:tag image if repo:tag@sha256:digest reference was used.
_, isDigested := name.(reference.Digested)
tagged, isTagged := name.(reference.NamedTagged)
if isDigested && isTagged {
named, _ := reference.ParseNormalizedNamed(tagged.Name())
namedTagged, _ := reference.WithTag(named, tagged.Tag())
delete(imagesToPrune, namedTagged.String())
}
}
}
return usedDigests
}
// pruneAll deletes all images in the imagesToPrune map.
func (i *ImageService) pruneAll(ctx context.Context, imagesToPrune map[string]containerdimages.Image) (*image.PruneReport, error) {
report := image.PruneReport{}
ctx, span := tracing.StartSpan(ctx, "ImageService.pruneAll")
span.SetAttributes(tracing.Attribute("count", len(imagesToPrune)))
defer span.End()
possiblyDeletedConfigs := map[digest.Digest]struct{}{}
var errs error

View File

@@ -39,6 +39,12 @@ func (i *ImageService) PullImage(ctx context.Context, baseRef reference.Named, p
}()
out := streamformatter.NewJSONProgressOutput(outStream, false)
ctx, done, err := i.withLease(ctx, true)
if err != nil {
return err
}
defer done()
if !reference.IsNameOnly(baseRef) {
return i.pullTag(ctx, baseRef, platform, metaHeaders, authConfig, out)
}
@@ -79,10 +85,32 @@ func (i *ImageService) pullTag(ctx context.Context, ref reference.Named, platfor
resolver, _ := i.newResolverFromAuthConfig(ctx, authConfig, ref)
opts = append(opts, containerd.WithResolver(resolver))
old, err := i.resolveDescriptor(ctx, ref.String())
oldImage, err := i.resolveImage(ctx, ref.String())
if err != nil && !errdefs.IsNotFound(err) {
return err
}
// Will be set to the new image after pull succeeds.
var outNewImg containerd.Image
if oldImage.Target.Digest != "" {
err = i.leaseContent(ctx, i.content, oldImage.Target)
if err != nil {
return errdefs.System(fmt.Errorf("failed to lease content: %w", err))
}
// If the pulled image is different than the old image, we will keep the old image as a dangling image.
defer func() {
if outNewImg != nil {
if outNewImg.Target().Digest != oldImage.Target.Digest {
if err := i.ensureDanglingImage(ctx, oldImage); err != nil {
log.G(ctx).WithError(err).Warn("failed to keep the previous image as dangling")
}
}
}
}()
}
p := platforms.Default()
if platform != nil {
p = platforms.Only(*platform)
@@ -100,7 +128,6 @@ func (i *ImageService) pullTag(ctx context.Context, ref reference.Named, platfor
pp := pullProgress{store: i.content, showExists: true}
finishProgress := jobs.showProgress(ctx, out, pp)
var outNewImg *containerd.Image
defer func() {
finishProgress()
@@ -114,9 +141,10 @@ func (i *ImageService) pullTag(ctx context.Context, ref reference.Named, platfor
// Status: Downloaded newer image for hello-world:latest
// docker.io/library/hello-world:latest
if outNewImg != nil {
img := *outNewImg
img := outNewImg
progress.Message(out, "", "Digest: "+img.Target().Digest.String())
writeStatus(out, reference.FamiliarString(ref), old.Digest != img.Target().Digest)
newer := oldImage.Target.Digest != img.Target().Digest
writeStatus(out, reference.FamiliarString(ref), newer)
}
}()
@@ -184,6 +212,18 @@ func (i *ImageService) pullTag(ctx context.Context, ref reference.Named, platfor
}
return errdefs.NotFound(fmt.Errorf("pull access denied for %s, repository does not exist or may require 'docker login'", reference.FamiliarName(ref)))
}
if cerrdefs.IsNotFound(err) {
// Transform "no match for platform in manifest" error returned by containerd into
// the same message as the graphdrivers backend.
// The one returned by containerd doesn't contain the platform and is much less informative.
if strings.Contains(err.Error(), "platform") {
platformStr := platforms.DefaultString()
if platform != nil {
platformStr = platforms.Format(*platform)
}
return errdefs.NotFound(fmt.Errorf("no matching manifest for %s in the manifest list entries: %w", platformStr, err))
}
}
return err
}
@@ -202,7 +242,7 @@ func (i *ImageService) pullTag(ctx context.Context, ref reference.Named, platfor
}
i.LogImageEvent(reference.FamiliarString(ref), reference.FamiliarName(ref), events.ActionPull)
outNewImg = &img
outNewImg = img
return nil
}

View File

@@ -270,7 +270,7 @@ func (i *ImageService) getPushDescriptor(ctx context.Context, img containerdimag
switch len(presentMatchingManifests) {
case 0:
return ocispec.Descriptor{}, errdefs.NotFound(fmt.Errorf("no suitable image manifest found for platform %s", *platform))
return ocispec.Descriptor{}, errdefs.NotFound(fmt.Errorf("no suitable image manifest found for platform %s", platforms.FormatAll(*platform)))
case 1:
// Only one manifest is available AND matching the requested platform.
@@ -302,7 +302,7 @@ func (i *ImageService) getPushDescriptor(ctx context.Context, img containerdimag
return ocispec.Descriptor{}, errdefs.Conflict(errors.Errorf("multiple matching manifests found but no specific platform requested"))
}
return ocispec.Descriptor{}, errdefs.Conflict(errors.Errorf("multiple manifests found for platform %s", *platform))
return ocispec.Descriptor{}, errdefs.Conflict(errors.Errorf("multiple manifests found for platform %s", platforms.FormatAll(*platform)))
}
}

View File

@@ -1,5 +1,5 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.21
//go:build go1.22
package containerd
@@ -7,6 +7,7 @@ import (
"context"
"fmt"
"path/filepath"
"slices"
"testing"
containerdimages "github.com/containerd/containerd/images"
@@ -15,7 +16,6 @@ import (
"github.com/docker/docker/errdefs"
"github.com/docker/docker/internal/testutils/specialimage"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"golang.org/x/exp/slices"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
@@ -204,7 +204,7 @@ func TestImagePushIndex(t *testing.T) {
imgSvc.defaultPlatformOverride = platforms.Only(defaultDaemonPlatform)
}
idx, err := specialimage.MultiPlatform(csDir, "multiplatform:latest", tc.indexPlatforms)
idx, _, err := specialimage.MultiPlatform(csDir, "multiplatform:latest", tc.indexPlatforms)
assert.NilError(t, err)
imgs := imagesFromIndex(idx)

View File

@@ -58,6 +58,11 @@ func (i *ImageService) PrepareSnapshot(ctx context.Context, id string, parentIma
parentSnapshot = identity.ChainID(diffIDs).String()
}
// TODO: Consider a better way to do this. It is better to have a container directly
// reference a snapshot, however, that is not done today because a container may
// removed and recreated with nothing holding the snapshot in between. Consider
// removing this lease and only temporarily holding a lease on re-create, using
// non-expiring leases introduces the possibility of leaking resources.
ls := i.client.LeasesService()
lease, err := ls.Create(ctx, leases.WithID(id))
if err != nil {

View File

@@ -68,7 +68,9 @@ func (i *ImageService) TagImage(ctx context.Context, imageID image.ID, newTag re
// Delete the source dangling image, as it's no longer dangling.
if err := i.images.Delete(context.WithoutCancel(ctx), danglingImageName(targetImage.Target.Digest)); err != nil {
logger.WithError(err).Warn("unexpected error when deleting dangling image")
if !cerrdefs.IsNotFound(err) {
logger.WithError(err).Warn("unexpected error when deleting dangling image")
}
}
return nil

Some files were not shown because too many files have changed in this diff Show More