Compare commits

...

3987 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
77446557b0 Merge pull request #49487 from austinvazquez/cherry-pick-838ae09a2337e6561b40d13be6ddf43005a92a9e-to-27.x
[27.x backport] Dockerfile: update runc binary to v1.2.5
2025-02-18 18:08:52 +01:00
Sebastiaan van Stijn
20755a757c Dockerfile: update runc binary to v1.2.5
This is the fifth patch release in the 1.2.z series of runc. It primarily fixes
an issue caused by an upstream systemd bug.

* There was a regression in systemd v230 which made the way we define
  device rule restrictions require a systemctl daemon-reload for our
  transient units. This caused issues for workloads using NVIDIA GPUs.
  Workaround the upstream regression by re-arranging how the unit properties
  are defined.
* Dependency github.com/cyphar/filepath-securejoin is updated to v0.4.1,
  to allow projects that vendor runc to bump it as well.
* CI: fixed criu-dev compilation.
* Dependency golang.org/x/net is updated to 0.33.0.

full diff: https://github.com/opencontainers/runc/compare/v1.2.4...v1.2.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 838ae09a23)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2025-02-18 15:19:10 +00:00
Sebastiaan van Stijn
15c6aaf0e9 Merge pull request #49394 from vvoland/update-go-27.x
[27.x] update to go1.22.12
2025-02-06 09:38:21 +01:00
Paweł Gronowski
a584f0b227 update to go1.22.12
This minor release include 1 security fix following the security policy:

- crypto/elliptic: timing sidechannel for P-256 on ppc64le

  Due to the usage of a variable time instruction in the assembly implementation
  of an internal function, a small number of bits of secret scalars are leaked on
  the ppc64le architecture. Due to the way this function is used, we do not
  believe this leakage is enough to allow recovery of the private key when P-256
  is used in any well known protocols.

This is CVE-2025-22866 and Go issue https://go.dev/issue/71383.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.12

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-04 20:31:16 +01:00
Sebastiaan van Stijn
5df78b0645 Merge pull request #49384 from vvoland/49362-27.x
[27.x backport] github: Clarify release notes description
2025-02-04 17:24:07 +01:00
Paweł Gronowski
b4d7cb0141 Merge pull request #49383 from vvoland/49361-27.x
[27.x backport] gha/validate-pr: Also run when PR has new commits
2025-02-03 16:38:09 +00:00
Paweł Gronowski
ced95906a4 github: Clarify release notes description
Error out if the release notes section is filled for PRs without the
`impact/` label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 79b0e89628)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-03 17:27:13 +01:00
Paweł Gronowski
48d2940834 gha/validate-pr: Also run when PR has new commits
Otherwise, the workflow will won't be rerun even if it was failing
before.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 7ac0e34dba)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-02-03 16:28:55 +01:00
Sebastiaan van Stijn
4c9b3b011a Merge pull request #49317 from thaJeztah/27.x_backport_bump_dev_tools
[27.x backport] Dockerfile: dev-container: update CLI v27.5.0, buildx v0.20.0, compose v2.32.4
2025-01-22 00:46:50 +01:00
Sebastiaan van Stijn
0da7a2656e Dockerfile: update compose to v2.32.4
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.30.3...v2.32.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 836a5fac10)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 17:42:37 +01:00
Sebastiaan van Stijn
4c8ec29141 Dockerfile: update buildx to v0.20.0
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/v0.18.0..v0.20.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b75a75017d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 17:42:36 +01:00
Sebastiaan van Stijn
fbc854d9d3 Dockerfile: update docker CLI to v27.5.0
Update the Docker CLI used in the dev-container
full diff: https://github.com/docker/cli/compare/v27.3.1...v27.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e2a828ed5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 17:42:34 +01:00
Sebastiaan van Stijn
36c72d4c3d Merge pull request #49322 from thaJeztah/27.x_backport_bake-v6
[27.x backport] ci: update bake-action to v6, ci(bin-image): fix bake build
2025-01-21 17:42:00 +01:00
CrazyMax
e85906c6a8 ci(bin-image): fix bake build
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit d86920b9b3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 16:43:21 +01:00
CrazyMax
542e33c421 ci: update bake-action to v6
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit f8a973ba4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-21 16:42:59 +01:00
Paweł Gronowski
c0df67813a Merge pull request #49314 from vvoland/49313-27.x
[27.x backport] gha: Adjust release branches
2025-01-21 10:54:03 +01:00
Paweł Gronowski
95d18194a4 gha: Adjust release branches
Adjust all workflows to also run on branches like `27.x`

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5e9ee1a849)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-01-20 18:22:23 +01:00
Sebastiaan van Stijn
13eca88597 Merge pull request #49312 from thaJeztah/27.x_bump_golang_1.22.11
[27.x] update to go1.22.11 (fix CVE-2024-45341, CVE-2024-45336)
2025-01-20 17:34:41 +01:00
Sebastiaan van Stijn
c83862c541 update to go1.22.11 (fix CVE-2024-45341, CVE-2024-45336)
go1.22.11 (released 2025-01-16) includes security fixes to the crypto/x509 and
net/http packages, as well as bug fixes to the runtime. See the Go 1.22.11
milestone on our issue tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.22.11+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.10...go1.22.11

Hello gophers,

We have just released Go versions 1.23.5 and 1.22.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- crypto/x509: usage of IPv6 zone IDs can bypass URI name constraints

  A certificate with a URI which has a IPv6 address with a zone ID may
  incorrectly satisfy a URI name constraint that applies to the certificate
  chain.

  Certificates containing URIs are not permitted in the web PKI, so this
  only affects users of private PKIs which make use of URIs.

  Thanks to Juho Forsén of Mattermost for reporting this issue.

  This is CVE-2024-45341 and Go issue https://go.dev/issue/71156.

- net/http: sensitive headers incorrectly sent after cross-domain redirect

  The HTTP client drops sensitive headers after following a cross-domain redirect.
  For example, a request to a.com/ containing an Authorization header which is
  redirected to b.com/ will not send that header to b.com.

  In the event that the client received a subsequent same-domain redirect, however,
  the sensitive headers would be restored. For example, a chain of redirects from
  a.com/, to b.com/1, and finally to b.com/2 would incorrectly send the Authorization
  header to b.com/2.

  Thanks to Kyle Seely for reporting this issue.

  This is CVE-2024-45336 and Go issue https://go.dev/issue/70530.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 15:59:42 +01:00
Sebastiaan van Stijn
78bccb9709 Merge pull request #49308 from thaJeztah/27.x_backport_gha-arm64
[27.x backport] ci: switch from jenkins to gha for arm64 build and tests (and set correct go version for branch)
2025-01-20 15:59:00 +01:00
Sebastiaan van Stijn
0d4f27db11 Merge pull request #49306 from thaJeztah/27.x_backport_49240_ignore_br_netfilter_error
[27.x backport] Allow users to ignore missing br_netfilter
2025-01-20 12:37:13 +01:00
Sebastiaan van Stijn
e0b88bc9a8 Merge pull request #49307 from thaJeztah/27.x_backport_bridge_startup_error
[27.x backport] Recover from default bridge init failure
2025-01-20 11:53:13 +01:00
Sebastiaan van Stijn
174ce6b376 Merge pull request #49304 from AkihiroSuda/cherrypick-49303-27
[27.x] Dockerfile: update RootlessKit to v2.3.2
2025-01-20 11:46:11 +01:00
Sebastiaan van Stijn
6c832d05c4 gha: set arm64 GO_VERSION to 1.22.10
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:48:57 +01:00
CrazyMax
36706f0e5f ci: switch from jenkins to gha for arm64 build and tests
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 8c236de735)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:45:24 +01:00
Rob Murray
b33299da85 Bridge: on network delete, make sure it's deleted from store
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 22c02219de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:35:36 +01:00
Rob Murray
d57309014d Move integration/network/bridge_test.go to bridge subdir
Also make it Linux-only, as the bridge driver is Linux only and
all of the tests had skips for Windows.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 24f53eba7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:35:26 +01:00
Rob Murray
ad04c7b55a Allow users to ignore missing br_netfilter
Since commit 0f8fc31, the bridge driver will try to load kernel
module br_netfilter if the userland proxy is disabled. If it fails,
we're in unknown territory, so it's treated as an error. At the
very least, containers will not be able to access host ports
mapped to other containers in the same network.

Before that, and before commit 5c499fc delayed the module load
until it was needed - it was loaded unconditionally, but errors
were only logged.

So, on systems where the module is not available, or could not be
loaded/configured, no error was reported and things "worked" (as
long as you didn't try to use something that didn't work).

That behaviour has been useful to some. So, make it possible to
ignore the problem by setting env var:
  DOCKER_IGNORE_BR_NETFILTER_ERROR=1

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit e7bd60ee2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-20 10:21:34 +01:00
Akihiro Suda
70a866e37e Dockerfile: update RootlessKit to v2.3.2
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.3.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit aa565ec1ef)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2025-01-20 17:41:26 +09:00
Sebastiaan van Stijn
38b84dce32 Merge pull request #49255 from thaJeztah/27.x_backport_ioutils_more_deprecations
[27.x backport] pkg/ioutils: deprecate NopWriter, NopWriteClose
2025-01-10 15:39:09 +01:00
Sebastiaan van Stijn
ae821132d3 Merge pull request #49253 from thaJeztah/27.x_backport_bump_containerd_binary_1.7.25
[27.x backport] Dockerfile: update containerd to v1.7.25
2025-01-10 14:56:06 +01:00
Sebastiaan van Stijn
8a2fd51c84 pkg/ioutils: deprecate NopWriteCloser
It was only used internally, and has no external consumers; deprecate
it to be removed in the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3faa170371)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-10 14:37:48 +01:00
Sebastiaan van Stijn
cf37b661dd pkg/ioutils: deprecate NopWriter
It's not used, and users can use io.Discard instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fa3c553e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-10 14:36:25 +01:00
Sebastiaan van Stijn
c83f658829 Dockerfile: update containerd to v1.7.25
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.25
full diff: https://github.com/containerd/containerd/compare/v1.7.24...v1.7.25

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c12bfda3cd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-10 13:26:40 +01:00
Paweł Gronowski
931be8ef77 Merge pull request #49246 from thaJeztah/27.x_backport_ioutils_deprecations
[27.x backport] pkg/ioutils: deprecate unused types and functions
2025-01-10 10:13:07 +01:00
Sebastiaan van Stijn
2e1aee3326 Merge pull request #49249 from robmry/backport-27.x/fix_unit_tests_for_nftables_host
[27.x backport] Fix unit tests for an nftables host
2025-01-10 00:13:22 +01:00
Rob Murray
f53d72e0be Fix unit tests for an nftables host
On a freshly rebooted Linux host that's configured to use nftables
with the iptables front-end, "make test-unit" fails "TestUserChains"
on the first run - it's ok in subsequent runs.

The unit tests run in moby's dev container.

The first test in TestUserChain runs with ip6tables disabled, so the
bridge driver doesn't try to load the ip6_tables module. Then, because
the module isn't loaded (it wasn't needed by the daemon running on the
nftables host when it started), the test fails because it can't check
what's in the ip6tables filter chain.

The next test in TestUserChain does the same thing, but with ip6tables
enabled. So the module gets loaded by the bridge driver, and everything
works normally after that.

The dev container used to try to load the module on startup, but that
was removed in commit 2af19b6b ("Don't try to modprobe ip6_tables in
the moby dev container"), as part of a change to give the daemon a way
to load modules itself.

Rather that put back the dev container's code to load ip6_tables on
startup (which would mean the daemon's module-loading code not getting
to run on nftables/firewalld hosts) ...

Run the tests in TestUserChains in a different order, with iptables
enabled in the first test will make it happen to work. At least for
now.

It's not ideal, but we'll be switching to nftables soon, so the issue
will go away.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 8f9c09edd4)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-01-09 20:42:22 +00:00
Akihiro Suda
55f705597c Merge pull request #49243 from thaJeztah/27.x_backport_bump_runc_binary_1.2.4
[27.x backport] Dockerfile: update runc binary to v1.2.4
2025-01-10 05:27:04 +09:00
Akihiro Suda
36ad318a2d Merge pull request #49247 from thaJeztah/27.x_backport_deprecate_runtime_numcpu
[27.x backport] pkg/sysinfo: Deprecate NumCPU
2025-01-10 05:25:51 +09:00
Paweł Gronowski
5b7034f4c6 pkg/sysinfo: Deprecate NumCPU
Deprecate in favor of `runtime.NumCPU` as the behavior is the same now.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3db72b255d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:20:47 +01:00
Sebastiaan van Stijn
9b974d599a daemon: don't repeatedly call NumCPU if not needed
sysinfo.NumCPU returns the number of CPUs which are currently online,
which involves some syscalls and parsing on Windows.

Change the code to only retrieve this information when needed, and
memoize the result to prevent calling this function multiple times.

Ideally, we'd obtain this information from daemon.RawSysInfo(), but
that uses a sync.Once, which could return outdated information.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ddd885a961)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:19:21 +01:00
Sebastiaan van Stijn
41697f61ec pkg/ioutils: deprecate NopFlusher
Apart from being used internally for NewWriteFlusher, it's only used
in a single location outside of this package. Copy the implementation
where it's used, and mark it deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 818a180fce)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:10:51 +01:00
Sebastiaan van Stijn
d9c2cde3be pkg/ioutils: deprecate WriteCounter, NewWriteCounter
it was moved to pkg/ioutils in c30a55f14d,
and only had a single use at the time in [engine/Env.WriteTo]. That use
was removed in 531f4122bd, which removed
the engine package.

[engine/Env.WriteTo]: c30a55f14d/engine/env.go (L260-L264)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 383503d382)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:10:51 +01:00
Sebastiaan van Stijn
a78ae66d0e pkg/ioutils: deprecate BytesPipe, NewBytesPipe, ErrClosed
These types are only used internally in container/streams and have no
external consumers. Deprecate them in preparation of moving them to
a subpackage of container/streams.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 60971a66b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 18:10:45 +01:00
Sebastiaan van Stijn
95122b4bde Dockerfile: update runc binary to v1.2.4
This is the fourth patch release of the 1.2.z release branch of runc. It
includes a fix for a regression introduced in 1.2.0 related to the
default device list.

- Re-add tun/tap devices to built-in allowed devices lists.

 In runc 1.2.0 we removed these devices from the default allow-list
 (which were added seemingly by accident early in Docker's history) as
 a precaution in order to try to reduce the attack surface of device
 inodes available to most containers. At the time we thought
 that the vast majority of users using tun/tap would already be
 specifying what devices they need (such as by using --device with
 Docker/Podman) as opposed to doing the mknod manually, and thus
 there would've been no user-visible change.

 Unfortunately, it seems that this regressed a noticeable number of
 users (and not all higher-level tools provide easy ways to specify
 devices to allow) and so this change needed to be reverted. Users
 that do not need these devices are recommended to explicitly disable
 them by adding deny rules in their container configuration.

full diff: https://github.com/opencontainers/runc/compare/v1.2.3...v1.2.4
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aad7bcedd2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-09 13:56:33 +01:00
Sebastiaan van Stijn
d9de1e24ad Merge pull request #49239 from vvoland/49228-27.x
[27.x backport] Clear RWLayer reference under container lock
2025-01-08 21:25:37 +01:00
Tadeusz Dudkiewicz
0ec92ba6a4 Clear RWLayer reference under container lock
Previously the RWLayer reference was cleared without holding the
container lock. This could lead to goroutine panics in various places
that use the container.RWLayer because nil checks introduced in #36242
where not sufficient as the reference could change right before the use.

Fixes #49227

Signed-off-by: Tadeusz Dudkiewicz <tadeusz.dudkiewicz@rtbhouse.com>
(cherry picked from commit 97dc3056c6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2025-01-08 19:00:12 +01:00
Sebastiaan van Stijn
43fc912ef5 Merge pull request #49194 from rumpl/27.x_backport_c8d-fix-tar-build
[27.x backport]  Use the roundtripper during build
2025-01-03 19:17:33 +01:00
Sebastiaan van Stijn
19d1277ff7 Merge pull request #49209 from thaJeztah/27.x_backport_deprecate_GetTotalUsedFds
[27.x backport] pkg/fileutils: deprecate GetTotalUsedFds
2025-01-03 18:40:43 +01:00
Sebastiaan van Stijn
914696818b Merge pull request #49212 from thaJeztah/27.x_backport_hcsshim_stringmatch
[27.x backport] libnetwork/drivers/windows: fix error-matching for hcsshim "not found"
2025-01-03 18:38:36 +01:00
Sebastiaan van Stijn
a167bc5c9a libnetwork/drivers/windows: fix error-matching for hcsshim "not found"
This code has some gnarly string-matching to detect "not found" errors
returned by hcsshim.

Hcsshim at some point changed this error to lowercase;
6d67a30859

It looks like we ran into that problem in integration tests, which was
fixed in c530c9cbb0 when updating hcsshim,
however, it was only fixed in tests, and hiding the actual issue in our
code.

It looks like hcsshim has some utilities to detect error-types, such as the
IsElementNotFoundError function in hcn, which is the newer API that also wraps
the "HNS" service;
d9a4231b9d/hcn/hcnerrors.go (L75-L77)

But unfortunately, the hns API used by us, does not return typed errors, and
returns HNS errors as a untyped formatted string.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5b31a5b370)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-03 17:38:32 +01:00
Sebastiaan van Stijn
dcedefde22 Merge pull request #49204 from thaJeztah/27.x_backport_non_const_formatting
[27.x backport] fix non-constant format string (caught by go1.24)
2025-01-03 17:36:51 +01:00
Sebastiaan van Stijn
4c7dc9347d Merge pull request #49203 from thaJeztah/27.x_backport_go1.24_prep
[27.x backport] daemon: adjust tests for changes in go1.24 JSON errors
2025-01-03 17:36:21 +01:00
Sebastiaan van Stijn
bdd3a95a13 pkg/fileutils: deprecate GetTotalUsedFds
This function is only used internally and has no external consumers.
Mark it deprecated to be removed in the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e45f20352d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-03 17:23:36 +01:00
Sebastiaan van Stijn
5f81af6eaa libnetwork/drivers/bridge: format errors with '%v' for consistency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1f4c9f23c3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:23:43 +01:00
Sebastiaan van Stijn
0f45f60e9e libnetwork/drivers/windows: fix non-constant format string
Also updated some existing ones to use `%v` instead of `%s` for consistency.

caught by go1.24

    # github.com/docker/docker/libnetwork/drivers/windows/overlay
    # github.com/docker/docker/libnetwork/drivers/windows/overlay/ov_network_windows.go:206:32: non-constant format string in call to github.com/docker/docker/libnetwork/types.ForbiddenErrorf
    FAIL    github.com/docker/docker/libnetwork/drivers/windows/overlay [build failed]

    # github.com/docker/docker/libnetwork/drivers/windows
    # github.com/docker/docker/libnetwork/drivers/windows/windows.go:449:33: non-constant format string in call to github.com/docker/docker/libnetwork/types.ForbiddenErrorf
    FAIL    github.com/docker/docker/libnetwork/drivers/windows [build failed]

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3b8ba71fe3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:23:42 +01:00
Sebastiaan van Stijn
17e8cb2b06 distribution: fix non-constant format string
caught by go1.24

    # github.com/docker/docker/distribution
    # github.com/docker/docker/distribution/pull_v2_windows.go:145:35: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Debugf
    FAIL    github.com/docker/docker/distribution [build failed]

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c96e45375)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:23:42 +01:00
Sebastiaan van Stijn
60f9e333b6 daemon: adjust tests for changes in go1.24 JSON errors
These tests failed because the error message changed in go1.24 through
https://go.dev/cl/606956.

    === Failed
    === FAIL: daemon/config TestDaemonConfigurationFeatures/invalid_config_value (0.00s)
        config_linux_test.go:182: assertion failed: expected error "json: cannot unmarshal string into Go struct field Config.features of type bool", got "json: cannot unmarshal string into Go struct field Config.CommonConfig.features of type bool"

    === FAIL: daemon/config TestDaemonConfigurationFeatures (0.00s)

    === FAIL: daemon/config TestDaemonConfigurationHostGatewayIP/config_not_array (0.00s)
        config_linux_test.go:354: assertion failed: expected error "json: cannot unmarshal string into Go struct field Config.host-gateway-ips of type []netip.Addr", got "json: cannot unmarshal string into Go struct field Config.CommonConfig.DNSConfig.host-gateway-ips of type []netip.Addr"

Relax the tests a bit to accept errors produced by either go1.24 or older.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1c37a4454b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-01-02 23:21:23 +01:00
Sebastiaan van Stijn
9cf331393c Merge pull request #49200 from robmry/backport-27.x/49179_restore_windows_network_labels
[27.x backport] Restore labels when re-creating Windows networks
2025-01-02 23:14:50 +01:00
Rob Murray
95b2f2ed49 Restore labels when re-creating Windows networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 25009ed5c0)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-01-02 19:14:24 +00:00
Djordje Lukic
623af65208 Use the roundtripper during build
The roundtripper is responsible for giving back the build context when
it comes from a tar directly. So we add it to the source manager of the
containerd worker.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2025-01-02 10:10:55 +01:00
Sebastiaan van Stijn
c23af29109 Merge pull request #48949 from thaJeztah/27.x_backport_buildkit-v0.18.0-rc1
[27.x backport] vendor: update buildkit to v0.18.2
2024-12-20 17:48:41 +01:00
Sebastiaan van Stijn
b979cf91aa Merge pull request #49145 from thaJeztah/27.x_backport_swagger_updates
[27.x backport] swagger / API-docs updates
2024-12-19 20:45:31 +01:00
Sebastiaan van Stijn
771d1dc44e docs/api: allow for an empty string for Isolation (api v1.25-v1.47)
Backport the changes from 26049febb2
to all versions used in the documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 364e4790e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 17:30:52 +01:00
Adam Lamers
68b39f825d docs: Update example section for SwarmJoinRequest
- Updated the example section in `swagger.yaml` for the `DataPathAddr` parameter in `SwarmJoinRequest`.
- Updated corresponding example sections in `docs/api/v1.*`

Signed-off-by: Adam Lamers <adam.lamers@wmsdev.pl>
(cherry picked from commit 1aecca8bbd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 17:30:41 +01:00
Daniel Guns
4767dbd713 api: Allow for an empty string for Isolation in Swagger specs This adds an empty string as a valid option for the Isolation field when inspecting a container. On non windows systems, this is always empty, so no error should be returned. Fixes #47452
Signed-off-by: Daniel Guns <danbguns@gmail.com>
(cherry picked from commit 26049febb2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 17:30:28 +01:00
Tonis Tiigi
4359c8b08e vendor: update buildkit to v0.18.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit cd81985bfa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:12 +01:00
Tonis Tiigi
bbd14fc508 vendor: update buildkit to v0.18.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 504ee465fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Tonis Tiigi
fd86c39cc3 vendor: update buildkit to v0.18.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit b4769e86a4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Tonis Tiigi
1fd14d845f vendor: update buildkit to v0.18.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 90250ce93a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Sebastiaan van Stijn
68b0c87e05 vendor: github.com/moby/buildkit 94d0f6ed85e5 (master / v0.18.0-rc.2)
full diff: https://github.com/moby/buildkit/compare/v0.18.0-rc1...94d0f6ed85e504e1851495fa98485ffbd9cfddb0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0666d4a585)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:11 +01:00
Tonis Tiigi
74600d8425 vendor: update buildkit to v0.18.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 48b237f7af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 16:26:08 +01:00
Sebastiaan van Stijn
5b334e0e93 Merge pull request #49136 from thaJeztah/27.x_backport_bump_containerd_hcsshim
[27.x backport] vendor: github.com/containerd/containerd v1.7.23, hcsshim v0.12.8
2024-12-19 16:25:42 +01:00
Sebastiaan van Stijn
09b6837cb8 Merge pull request #49137 from thaJeztah/27.x_backport_builder-gcpolicy-empty-keepbytes
[27.x backport] builder: fall back to defaultKeepStorage if keepStorage is unset for GC policy
2024-12-19 16:25:26 +01:00
David Karlsson
4830f6db66 builder: fall back to defaultKeepStorage if keepStorage is unset for GC policy
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
(cherry picked from commit b08ff81204)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 14:38:04 +01:00
Sebastiaan van Stijn
0e43d1f642 vendor: github.com/containerd/containerd v1.7.23, hcsshim v0.12.8
full diff:

- https://github.com/containerd/containerd/compare/v1.7.22...v1.7.23
- https://github.com/microsoft/hcsshim/compare/v0.12.7...v0.12.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5f39567e56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:22 +01:00
Sebastiaan van Stijn
800d735171 vendor: github.com/Microsoft/hcsshim v0.12.7
- full diff: https://github.com/Microsoft/hcsshim/compare/v0.12.5...v0.12.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 93255bd748)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:22 +01:00
Sebastiaan van Stijn
a57c1848b6 vendor: sigs.k8s.io/yaml v1.4.0
full diff: https://github.com/kubernetes-sigs/yaml/compare/v1.3.0...v1.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3dc042b1df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:21 +01:00
Sebastiaan van Stijn
26eeb6e414 vendor: github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161
full diff: d185dfc1b5...306776ec81

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c98c6d4f08)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:21 +01:00
Akihiro Suda
f6f7c9f231 vendor.mod: github.com/microsoft/hcsshim v0.12.5
https://github.com/microsoft/hcsshim/compare/v0.11.7...v0.12.5

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit f49fad75eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 13:57:18 +01:00
Sebastiaan van Stijn
29f12c5f77 Merge pull request #49135 from thaJeztah/27.x_backport_migrate_reexec
[27.x backport] pkg/reexec: deprecate and migrate to github.com/moby/sys/reexec
2024-12-19 13:54:08 +01:00
Sebastiaan van Stijn
053aadc451 Merge pull request #49134 from thaJeztah/27.x_backport_bump_otel
[27.x backport] vendor: go.opentelemetry.io/otel v1.28.0, go.opentelemetry.io/contrib v0.53.0
2024-12-19 13:51:12 +01:00
Sebastiaan van Stijn
bdecc9d58e pkg/reexec: deprecate and migrate to github.com/moby/sys/reexec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f5ec9994c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:45 +01:00
Sebastiaan van Stijn
46a8e5f8c5 pkg/reexec: Command: separate public API from implementation
Move the exported `Command` to a platform-agnostic file, and un-export
the platform-specific implementations. This allows us to maintain the
GoDoc in a single place, describing platform-specific differences where
needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8fd177d79b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:39 +01:00
Sebastiaan van Stijn
d175e4f1f1 pkg/reexec: make platform-agnostic (again)
The reexec package originally was platform-agnostic, but gained some
Linux-specific handling in 1cb17f03d0.

When Windows support was implemented in Docker, the pkg/reexec package
was adjusted accordingly in 64715c4f33,
which now made the package with with either Linux or Windows, with various
other platforms (freebsd, solaris, darwin) being added back in separate
changes.

Based on the history above, this package should be platform-agnostic, except
for Linux-specific changes introduced in 1cb17f03d0
and 5aee8807a6.

This patch:

- removes the stub-implementation to make it functional on other platforms.
- renames the files for consistency

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6568c06d12)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:39 +01:00
Sebastiaan van Stijn
ed5c0a79e7 pkg/reexec: use const for name of test binary
Also use a slightly different name, because "reexec" is used so
widely as term in this package, making it somewhat confusing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7672d60033)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:39 +01:00
Paweł Gronowski
4bf113f9bd all: Remove redundant units alias for go-units
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit c0045476b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:52:34 +01:00
Albin Kerouanton
dde37eb363 Merge pull request #48938 from thaJeztah/27.x_backport_update_netlink
[27.x backport] vendor: vendor: github.com/vishvananda/netlink v1.3.x (084abd93d) and fix compatibility
2024-12-19 12:36:47 +01:00
Sebastiaan van Stijn
7a512a9531 vendor: go.opentelemetry.io/contrib/instrumentation/xxx v0.53.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2a3acc104)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:02:52 +01:00
Sebastiaan van Stijn
cf4e196e77 vendor: go.opentelemetry.io/otel v1.28.0
aligning all related packages to v1.28.0 as well

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d9a2ca7b49)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:02:51 +01:00
Sebastiaan van Stijn
2a3cc83f36 vendor: google.golang.org/genproto/googleapis/api f6361c86f094
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 070d6c75a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 12:02:51 +01:00
Rob Murray
4141379d07 Discard ErrDumpInterrupted and return data after maxRetries
Returning possibly inconsistent data avoids retrying indefinitely, and
matches the behaviour of vishvananda/netlink prior to version 1.2.1, in
which the NLM_F_DUMP_INTR flag was ignored.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a0a0bbae6c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
4c50153e2e Fix comments
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 2f02f456f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
c5e72db8e8 vendor: github.com/vishvananda/netlink v1.3.1-0.20240922070040-084abd93d350
Particularly for:
- Preserve results when NLM_F_DUMP_INTR is set (1018).
- Fix SetSendTimeout/SetReceiveTimeout (1012).

full diff: https://github.com/vishvananda/netlink/compare/v1.3.0...084abd93d350

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit fd0d008504)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
14b30aeb61 Add linting rules to spot use of un-wrapped netlink functions.
Spot netlink functions that may return EINTR because
network configuration changed during a state dump, and
point at the nlutil wrappers.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit edaa0eb56d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:37 +01:00
Rob Murray
87fc62a648 Fix merge problem
Commit f9c0103 (WSL2 mirrored-mode loopback) uses netlink funcs
that were removed/wrapped by commit 00bf437.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 333cfa6402)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:36 +01:00
Rob Murray
3ca9a3be70 Add nlutil functions to retry on netlink EINTR
A recent change to the vishvananda/netlink package exposes
NLM_F_DUMP_INTR in some netlink responses as an EINTR (with
no data).

Retry the requests when that happens, up to five times, before
returning the error. The limit of five is arbitrary, on most
systems a single retry will be rare but, there's no guarantee
that a retry will succeed. So, on a very busy or misbehaving
system the error may still be returned. In most cases, this
will lead to failure of the operation being attempted (which
may lead to daemon startup failure, network initialisation
failure etc).

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 00bf437d84)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:28:33 +01:00
Sebastiaan van Stijn
4729e446ce vendor: github.com/vishvananda/netlink v1.3.0
full diff: https://github.com/vishvananda/netlink/compare/v1.2.1...v1.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2c498c6a74)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:26:35 +01:00
Sebastiaan van Stijn
cbe4488a6b vendor: github.com/vishvananda/netlink v1.2.1
Local changes related to:

- c96b03b4be
- dedb7ec04b

full diff: https://github.com/vishvananda/netlink/compare/v1.2.1-beta.2...v1.2.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6d0a5e31d7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-19 11:26:32 +01:00
Sebastiaan van Stijn
eaec884bcb Merge pull request #48939 from thaJeztah/27.x_backport_bump_prometheus
[27.x backport] vendor: github.com/prometheus/client_golang v1.20.5
2024-12-19 11:25:50 +01:00
Sebastiaan van Stijn
caa122e0fc Merge pull request #48943 from thaJeztah/27.x_backport_bump_buildkit_deps
[27.x backport] vendor: github.com/go-logr/logr v1.4.2, github.com/cenkalti/backoff/v4 v4.3.0
2024-12-19 11:25:03 +01:00
Sebastiaan van Stijn
c710b88579 Merge pull request #49119 from thaJeztah/27.x_backport_libnetwork_deprecate_Passthrough
[27.x backport] libnetwork/iptables: deprecate type IPV, Passthrough
2024-12-17 13:50:31 +01:00
Sebastiaan van Stijn
eda0a20deb libnetwork/iptables: deprecate Passthrough
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d688389f4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-17 12:44:40 +01:00
Albin Kerouanton
b51622dd0d libnet/iptables: deprecate type IPV
The iptables package has two different types to specify the IP version:
IPVersion, used by iptables code, and IPV, used by firewalld code.
Both are representing the ip version as a string.

For iptables, the case doesn't matter because the string is never used
as-is. However, for firewalld the case matters.

Make the IPV type an alias of IPVersion, and deprecate it. Also change
the case used in IPVersion strings to make IPV consts aliases of
IPVersion consts.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 27deff4da1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-17 12:44:34 +01:00
Sebastiaan van Stijn
829ac832da Merge pull request #49104 from thaJeztah/27.x_backport_update_swagger_headers
[27.x backport] docs/api: document correct case for Api-Version header
2024-12-16 18:57:46 +01:00
Albin Kerouanton
bd7da11e95 Merge pull request #49110 from thaJeztah/27.x_backport_fix_setupIPChains_defer
[27.x backport] libnetwork/drivers/bridge: setupIPChains: fix defer checking wrong err
2024-12-16 18:49:52 +01:00
Sebastiaan van Stijn
135b1449cd Merge pull request #49105 from thaJeztah/27.x_backport_testing-suse-apparmor
[27.x backport] integration-cli: don't skip AppArmor tests on SLES
2024-12-16 18:17:10 +01:00
Sebastiaan van Stijn
08de719cf2 libnetwork/drivers/bridge: setupIPChains: fix defer checking wrong err
The output variable was renamed in 0503cf2510,
but that commit failed to change this defer, which was now checking the
wrong error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 01a55860c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 16:49:35 +01:00
Sebastiaan van Stijn
2a6231993b Merge pull request #49100 from thaJeztah/27.x_backport_deprecate_pkg_system
[27.x backport] pkg/system: deprecate types and functions that are only used internally
2024-12-16 16:05:28 +01:00
Aleksa Sarai
6855ca1e62 integration-cli: don't skip AppArmor tests on SLES
This partially reverts e440831802 ("fix and skip some tests based on
API version"), which caused the integration-cli tests to skip all
AppArmor-related tests on SUSE.

It's not really clear why this was done originally, but I have verified
that on modern SLE 12 and SLE 15 systems the AppArmor tests pass without
any adjustments needed.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
(cherry picked from commit 1a453abfb1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 15:14:57 +01:00
Sebastiaan van Stijn
224b30556d docs/api: document correct case for Api-Version header
This header is sent in its canonical format; update the docs to
reflect this.

Follow-up to 76a5ca1d4d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4e9df46a6e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 14:14:11 +01:00
maggie44
5b92b3e7e0 Accurately reflect the canonical casing of API-Version and OS-Type headers
Go automatically canonicalises HTTP headers, meaning the string `API-Version` passed as a header has always been returned as `Api-Version`. Similarly, `OSType` is returned as `Ostype`.

This commit updates the documentation to reflect this behaviour and modifies the codebase to ensure that input strings are aligned with their canonical output values.

Signed-off-by: maggie44 <64841595+maggie44@users.noreply.github.com>
(cherry picked from commit 76a5ca1d4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 14:14:11 +01:00
Sebastiaan van Stijn
1df9e896f2 pkg/system: deprecate types and functions that are only used internally
These types and functions are only used internally (through pkg/archive).
Deprecate them, and mark them for removal.

This deprecates the `Lstat()`, `Mkdev()`, `Mknod()`, `FromStatT()`
and `Stat()` functions, and related `StatT` type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f0ce367e1e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-16 11:41:55 +01:00
Sebastiaan van Stijn
24f2f4fe06 Merge pull request #49090 from thaJeztah/27.x_backport_no_netfilter_warnings
[27.x backport] daemon: info: remove bridge-nf-call-iptables / ip6tables warnings
2024-12-13 12:09:32 +01:00
Sebastiaan van Stijn
8a8ab0d567 Merge pull request #49043 from robmry/backport-27.x/modprobeless
[27.x backport] Try to load kernel modules, without modprobe
2024-12-13 12:03:26 +01:00
Sebastiaan van Stijn
d1201f6dd8 Merge pull request #49085 from thaJeztah/27.x_backport_bump_runc_binary_1.2.3
[27.x backport] Dockerfile: update runc binary to v1.2.3
2024-12-13 11:50:42 +01:00
Sebastiaan van Stijn
9e272da0f2 Merge pull request #49080 from thaJeztah/27.x_backport_bump_xx
[27.x backport] update xx to v1.6.1 for compatibility with alpine 3.21
2024-12-13 11:50:07 +01:00
Sebastiaan van Stijn
f2209d1aae Merge pull request #49084 from thaJeztah/27.x_backport_bump_md2man
[27.x backport] man: vendor: github.com/cpuguy83/go-md2man v2.0.5
2024-12-13 11:49:41 +01:00
Sebastiaan van Stijn
3a1e495a0f daemon: info: remove bridge-nf-call-iptables / ip6tables warnings
Historically, the `bridge` network-driver would detect whether netfiltering
was enabled in the kernel or, if disabled, try to do a `modprobe` when
initializing the driver. This approach became problematic, as loading the
module was not always performed  at startup depending on daemon configuration,
or the daemon may have failed to load the module. The `/info` response
would include a warning  to inform the user that some functionality may not
be available;

    WARNING: bridge-nf-call-iptables is disabled
    WARNING: bridge-nf-call-ip6tables is disabled

Starting with db25b0dcd0, detecting whether
netfiltering  is enabled now [happens when needed][1], which was further improved
on to not depend  on `modprobe` in 264c15bfc4 and
4740820716.

Because of the above, the `/info` output would now return warnings in any
situation where netfiltering was not enabled on the host before the daemon
started, which may be either _incorrect_ (i.e., the module may have been
loaded afterwards), or irrelevant, because netfiltering is not needed in
all situations.

This patch removes the warnings from the `/info` response,

[1]: 944e403502/libnetwork/drivers/bridge/setup_bridgenetfiltering.go (L16-L77)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5c358743af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 10:16:02 +01:00
Sebastiaan van Stijn
4082abad8b Dockerfile: update runc binary to v1.2.3
This is the third patch release of the 1.2.z release branch of runc. It
primarily fixes some minor regressions introduced in 1.2.0.

- Fixed a regression in use of securejoin.MkdirAll, where multiple
  runc processes racing to create the same mountpoint in a shared rootfs
  would result in spurious EEXIST errors. In particular, this regression
  caused issues with BuildKit.
- Fixed a regression in eBPF support for pre-5.6 kernels after upgrading
  Cilium's eBPF library version to 0.16 in runc.

full diff: https://github.com/opencontainers/runc/compare/v1.2.2...v1.2.3
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec5c9e06e3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 01:33:45 +01:00
Sebastiaan van Stijn
a379aae924 man: vendor: github.com/cpuguy83/go-md2man v2.0.5
full diff: https://github.com/cpuguy83/go-md2man/compare/v2.0.4...v2.0.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a7da6fb2a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 01:30:06 +01:00
Sebastiaan van Stijn
c3968c6ab7 update xx to v1.6.1 for compatibility with alpine 3.21
This fixes compatibility with alpine 3.21

- Fix additional possible `xx-cc`/`xx-cargo` compatibility issue with Alpine 3.21
- Support for Alpine 3.21
- Fix `xx-verify` with `file` 5.46+
- Fix possible error taking lock in `xx-apk` in latest Alpine without `coreutils`

full diff: https://github.com/tonistiigi/xx/compare/v1.5.0...v1.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 89899b71a0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-13 00:40:41 +01:00
Sebastiaan van Stijn
dfce8756b7 Merge pull request #49079 from vvoland/49078-27.x
[27.x backport] otel: Avoid excessive memory allocations if not configured
2024-12-12 21:57:07 +01:00
Paweł Gronowski
2e6ca227cf otel: Use non-noop tracer provider for grpc
Needed for Buildkit history

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d8358ebc87)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 20:11:32 +01:00
Paweł Gronowski
a5f86f9279 integration/build_traces: Create own tracer provider
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 899360b649)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 20:11:31 +01:00
Paweł Gronowski
3a05c5dcad otel: Avoid excessive memory allocations if not configured
Use noop tracer provider if the OTEL exporter is not configured.
This makes the OTEL tracing avoid doing unneeded memory allocations for
spans which aren't going to be exported anywhere anyway.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4847557d1b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-12 20:11:28 +01:00
Rob Murray
adb8773adb Jenkins: don't modprobe kernel module
These modprobes were added as a workaround in commit cce5dfe1, but
dockerd should now be able to load the modules it needs.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 15ba03c8fc)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:42 +00:00
Rob Murray
ff176d42d5 Don't try to modprobe ip6_tables in the moby dev container
dockerd will now do this itself, if ip6tables is enabled.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 2af19b6b7c)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:42 +00:00
Rob Murray
86126b9af1 Try to load kernel module ip6_tables, when necessary.
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit f2e1f52a04)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:40 +00:00
Rob Murray
58318d26fc Use ioctl to try to trigger kernel module loads
An ioctl() call to get the "interface index" for a kernel module triggers
the kernel to try to load the module, if the process is running with
CAP_SYS_MODULE. This tends to be more reliable than "modprobe" for
docker-in-docker.

If the ioctl() method fails, fall back to trying "modprobe".

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 4740820716)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-12-06 14:26:08 +00:00
Sebastiaan van Stijn
92a83937d0 Merge pull request #49027 from thaJeztah/27.x_backport_cdi-rootless
[27.x backport] Dockerd rootless: make {/etc,/var/run}/cdi available
2024-12-04 15:17:19 +01:00
Sebastiaan van Stijn
9163aa379a Merge pull request #49026 from thaJeztah/27.x_update_go_1.22.10
[27.x] update to go1.22.10
2024-12-04 13:15:15 +01:00
Rafael Fernández López
4775621ab6 Dockerd rootless: make {/etc,/var/run}/cdi available
When dockerd is executed with the `dockerd-rootless.sh` script, make
/etc/cdi and /var/run/cdi available to the daemon if they exist.

This makes it possible to enable the CDI integration in rootless mode.

Fixes: #47676

Signed-off-by: Rafael Fernández López <ereslibre@ereslibre.es>
(cherry picked from commit 4e30acb63f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-04 12:04:13 +01:00
Sebastiaan van Stijn
0176f4a5c3 Merge pull request #49024 from thaJeztah/27.x_vendor_buildkit_0.17.3
[27.x] vendor: github.com/moby/buildkit v0.17.3
2024-12-04 12:00:06 +01:00
Sebastiaan van Stijn
0e34b3956b update to go1.22.10
go1.22.10 (released 2024-12-03) includes fixes to the runtime and the syscall
package. See the Go 1.22.10 milestone on our issue tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.22.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.9...go1.22.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-04 10:16:39 +01:00
Sebastiaan van Stijn
7919b806e7 [27.x] vendor: github.com/moby/buildkit v0.17.3
Fix issue where builds from older versions of clients/frontends could result
in missing "no-cache" behavior or original Dockerfile commands could be missing
in progress output

full diff: 80e01a9dc7...v0.17.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-12-04 09:42:20 +01:00
Sebastiaan van Stijn
a92d4c5a57 Merge pull request #49013 from vvoland/49006-27.x
[27.x backport] registry: deprecate RepositoryInfo.Class
2024-12-02 13:32:41 +01:00
Sebastiaan van Stijn
1cc127466d Merge pull request #49010 from vvoland/49009-27.x
[27.x backport] c8d/tag: Don't log a warning if the source image is not dangling
2024-12-02 13:31:19 +01:00
Sebastiaan van Stijn
525b929947 registry: deprecate RepositoryInfo.Class
The Class field was added in a12b466183 because
Docker Hub registry required a special scope to be set for pulling plugins;

    HTTP/1.1 401 Unauthorized
    ...
    Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository(plugin):vieux/sshfs:pull",error="insufficient_scope"

This is no longer a requirement, and the field is no longer set.

This patch deprecates the field and removes its use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5f91c769f5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-02 11:53:28 +01:00
Paweł Gronowski
d6d43b2912 c8d/tag: Don't log a warning if the source image is not dangling
After the image is tagged, the engine attempts to delete a dangling
image of the source image, so the image is no longer dangling.

When the source image is not dangling, the removal errors out (as
expected), but a warning is logged to the daemon log:

```
time="2024-12-02T10:44:25.386957553Z" level=warning msg="unexpected error when deleting dangling image" error="NotFound: image \"moby-dangling@sha256:54d8c2251c811295690b53af7767ecaf246f1186c36e4f2b2a63e0bfa42df045\": not found" imageID="sha256:54d8c2251c811295690b53af7767ecaf246f1186c36e4f2b2a63e0bfa42df045" spanID=bd10a21a07830d72 tag="docker.io/library/test:latest" traceID=4cf61671c2dc6da3dc7a09c0c6ac4e16
```

Remove that log as it causes unnecessary confusion, as the failure is
expected.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a93f6c61db)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-12-02 11:49:18 +01:00
Austin Vazquez
0f86fa4502 Merge pull request #48995 from thaJeztah/27.x_backport_modprobe_br_netfilter
[27.x backport] Jenkinsfile: modprobe br_netfilter
2024-11-29 19:44:04 -08:00
Sebastiaan van Stijn
e3564b52af Jenkinsfile: modprobe br_netfilter
Make sure the module is loaded, as we're not able to load it from within
the dev-container;

    time="2024-11-29T20:40:42Z" level=error msg="Running modprobe br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/5.15.0-1072-aws\n" error="exit status 1"

Also moving these steps _before_ the "print info" step, so that docker info
doesn't show warnings that bridge-nf-call-iptables and bridge-nf-call-ip6tables
are not loaded.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cce5dfe1e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-29 23:27:00 +01:00
Sebastiaan van Stijn
7f11005a3d Merge pull request #48968 from thaJeztah/27.x_backport_43080-zfs-destroy-missing-volume-fails
[27.x backport] daemon/graphdriver/zfs: ignore non-existent dataset on removal
2024-11-27 22:52:51 +01:00
Sebastiaan van Stijn
7f7efed8c9 Merge pull request #48966 from thaJeztah/27.x_backport_br_net-fix
[27.x backport] Fix br_netfilter module loading logic
2024-11-27 22:52:26 +01:00
Sebastiaan van Stijn
b9fd5ee243 Merge pull request #48969 from thaJeztah/27.x_backport_integration_add_wait
[27.x backport] integration: add wait
2024-11-27 22:24:42 +01:00
Akihiro Suda
46f6b9a259 integration: add wait
Cherry-picked several WIP commits from
b0a592798f/

Originally-authored-by: Rodrigo Campos <rodrigoca@microsoft.com>
Co-Authored-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fb6e650ab9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 16:28:51 +01:00
François Scala
18aa104041 daemon/graphdriver/zfs: ignore non-existent dataset on removal
Ignore "dataset does not exist" error in Remove function

Signed-off-by: François Scala <github@arcenik.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e7d15d4d58)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 16:22:13 +01:00
Shreenidhi Shedi
052f7d6461 Fix br_netfilter module loading logic
Checking for `/proc/sys/net/bridge` directory alone is not enough to
decide if bridge, br_netfilter module to be loaded.
Check for specific file for br_netfilter module and then
do modprobe if the file is not found under `/proc/sys/net/bridge`

Loading br_netfilter implicitly loads bridge module.

fixes: #48948

Signed-off-by: Shreenidhi Shedi <shreenidhi.shedi@broadcom.com>
(cherry picked from commit 264c15bfc4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 15:45:23 +01:00
Sebastiaan van Stijn
33b6b826c2 Merge pull request #48963 from thaJeztah/27.x_backport_carry_buildkit_init_timeout
[27.x backport] Remove buildkit init timeout
2024-11-27 11:42:55 +01:00
Brian Goff
151c7565a9 Remove buildkit init timeout
Buildkit *can* take a long time to start, we don't want the daemon to fail
to startup because buildkit took too long.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit b7f43c3729)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-27 09:32:54 +01:00
Sebastiaan van Stijn
eb19aa01ec Merge pull request #48957 from thaJeztah/27.x_backport_containerd-lease-cleanup
[27.x backport] Fix lease management during pull and export
2024-11-26 21:29:31 +01:00
Derek McGowan
1abc35f098 Add label on PrepareSnapshot to warn about non-expiring leases
Currently when preparing a snapshot for a container, a lease is used to
hold that snapshot for the lifespan of a container. That is workaround
to preserve the snapshot when a container is recreated, however, the
containerd object should be able to hold this reference itself.

Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit 4becdaca72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 14:20:15 +01:00
Derek McGowan
1d1da55382 Update containerd image builder to use lease prune label
Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit eef6b83e53)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 14:20:15 +01:00
Derek McGowan
a3afccd409 Fix lease management during image operations
Ensure that leases have a reasonable expiration and are
cleaned up during prune

Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit e6170a5c4f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 14:20:12 +01:00
Sebastiaan van Stijn
8a30a5e39c Merge pull request #48954 from thaJeztah/27.x_update_buildkit_0.17.3
[27.x] vendor: github.com/moby/buildkit 80e01a9dc7c1 (v0.17.3-dev)
2024-11-26 14:11:01 +01:00
Sebastiaan van Stijn
33e5f87d0f [27.x] vendor: github.com/moby/buildkit 80e01a9dc7c1 (v0.17.3-dev)
Use view transaction for metadata read to prevent dockerd failing to fully
start with a "context deadline exceeded error" with containerd snapshotter
and many builds/images.

full diff: https://github.com/moby/buildkit/compare/v0.17.2...80e01a9dc7c1f5bab680bab7b43059ad7a413301

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-26 11:04:34 +01:00
Sebastiaan van Stijn
55b81a15bc Merge pull request #48942 from thaJeztah/27.x_buildkit_0.17.2
[27.x] vendor: github.com/moby/buildkit v0.17.2
2024-11-25 17:54:05 +01:00
Sebastiaan van Stijn
465eb6aee2 vendor: github.com/cenkalti/backoff/v4 v4.3.0
full diff: https://github.com/cenkalti/backoff/compare/v4.2.1...v4.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f31188bc4c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 12:35:09 +01:00
Sebastiaan van Stijn
e4d5bae9c5 vendor: github.com/go-logr/logr v1.4.2
full diff: https://github.com/go-logr/logr/compare/v1.4.1...v1.4.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c3cab4170b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 12:34:00 +01:00
Sebastiaan van Stijn
752c0453d6 vendor: github.com/moby/buildkit v0.17.2
full diff: https://github.com/moby/buildkit/compare/v0.17.1...v0.17.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 12:29:05 +01:00
Sebastiaan van Stijn
339572db5d vendor: github.com/prometheus/client_golang v1.20.5
full diff: https://github.com/prometheus/client_golang/compare/v1.17.0...v1.20.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 95959f7000)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-25 11:10:24 +01:00
Sebastiaan van Stijn
4be68aaf6d Merge pull request #48934 from thaJeztah/27.x_backport_update-containerd-1.7.24
[27.x backport] Update containerd to v1.7.24
2024-11-22 18:20:26 +01:00
Sebastiaan van Stijn
af93467718 Merge pull request #48935 from thaJeztah/27.x_backport_bump_typeurl
[27.x backport] vendor: github.com/containerd/typeurl v2.2.3
2024-11-22 17:48:29 +01:00
Sebastiaan van Stijn
01f0737540 Merge pull request #48933 from thaJeztah/27.x_backport_update_go-actions-cache
[27.x backport] vendor: github.com/tonistiigi/go-actions-cache 394979b8119e
2024-11-22 17:12:25 +01:00
Sebastiaan van Stijn
ef390e92f9 Merge pull request #48931 from vvoland/48930-27.x
[27.x backport] vendor: resenje.org/singleflight v0.4.3
2024-11-22 17:11:42 +01:00
Sebastiaan van Stijn
6cab48bd9b vendor: github.com/containerd/typeurl v2.2.3
- add a `no_gogo` build-tag to allow compiling without support for gogoproto (github.com/gogo/protobuf)
- fix panic when type is not found and no handler is hit.
- fix unmarshalling of registered types

full diff: https://github.com/containerd/typeurl/compare/v2.2.0...v2.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0af6203b46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 14:48:10 +01:00
Derek McGowan
a64812939f Update containerd to v1.7.24
Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit a650dbd951)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 14:44:09 +01:00
Sebastiaan van Stijn
b6e1eb0dde Merge pull request #48926 from thaJeztah/27.x_bump_continuity
[27.x backport] vendor: github.com/containerd/continuity v0.4.5
2024-11-22 14:29:30 +01:00
Sebastiaan van Stijn
179c7c7596 vendor: github.com/tonistiigi/go-actions-cache 394979b8119e
updates to github.com/golang-jwt/jwt/v5

full diff: 58651d5e11...394979b811

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44f9eec1ae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 14:13:15 +01:00
Paweł Gronowski
7f9c5b278a vendor: resenje.org/singleflight v0.4.3
full diff: https://resenje.org/singleflight/compare/v0.4.1...v0.4.3

Changes:
- Fix incorrect `Forget` behavior
- Make panic behavior consistent with x/sync package

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 1551d95727)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-22 12:20:19 +01:00
Sebastiaan van Stijn
2684eacb65 Merge pull request #48919 from austinvazquez/cherry-pick-runtime-updates-to-27.x
[27.x backport] Dockerfile: update containerd v1.7.24, runc v1.2.2
2024-11-22 10:17:46 +01:00
Sebastiaan van Stijn
b90c6e922c vendor: github.com/containerd/continuity v0.4.5
full diff: https://github.com/containerd/continuity/compare/v0.4.4...v0.4.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d23bc11b97)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-22 10:12:04 +01:00
Sebastiaan van Stijn
e956d4a370 Dockerfile: update to runc v1.2.2
- 1.2.2 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.2
- 1.2.1 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.1
- 1.2.0 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.0

Breaking changes and deprecations are included below;

Breaking changes:

Several aspects of how mount options work has been adjusted in a way that
could theoretically break users that have very strange mount option strings.
This was necessary to fix glaring issues in how mount options were being
treated. The key changes are:

- Mount options on bind-mounts that clear a mount flag are now always
  applied. Previously, if a user requested a bind-mount with only clearing
  options (such as rw,exec,dev) the options would be ignored and the
  original bind-mount options would be set. Unfortunately this also means
  that container configurations which specified only clearing mount options
  will now actually get what they asked for, which could break existing
  containers (though it seems unlikely that a user who requested a specific
  mount option would consider it "broken" to get the mount options they
  asked foruser who requested a specific mount option would consider it
  "broken" to get the mount options they asked for). This also allows us to
  silently add locked mount flags the user did not explicitly request to be
  cleared in rootless mode, allowing for easier use of bind-mounts for
  rootless containers.
- Container configurations using bind-mounts with superblock mount flags
  (i.e. filesystem-specific mount flags, referred to as "data" in
  mount(2), as opposed to VFS generic mount flags like MS_NODEV) will
  now return an error. This is because superblock mount flags will also
  affect the host mount (as the superblock is shared when bind-mounting),
  which is obviously not acceptable. Previously, these flags were silently
  ignored so this change simply tells users that runc cannot fulfil their
  request rather than just ignoring it.

Deprecated

- runc option --criu is now ignored (with a warning), and the option will
  be removed entirely in a future release. Users who need a non-standard
  criu binary should rely on the standard way of looking up binaries in
  $PATH.
- runc kill option -a is now deprecated. Previously, it had to be specified
  to kill a container (with SIGKILL) which does not have its own private PID
  namespace (so that runc would send SIGKILL to all processes). Now, this is
  done automatically.
- github.com/opencontainers/runc/libcontainer/user is now deprecated, please
  use github.com/moby/sys/user instead. It will be removed in a future
  release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e257856116)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-11-21 18:09:01 +00:00
Austin Vazquez
b888333fe1 Dockerfile: update containerd to v1.7.24
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
(cherry picked from commit 8cecf3a71c)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-11-21 18:08:32 +00:00
Austin Vazquez
7fca82b32c Merge pull request #48913 from thaJeztah/27.x_backport_vendor-golang-jwt-v4.5.1
[27.x backport] vendor: github.com/golang-jwt/jwt/v4@v4.5.1
2024-11-21 04:53:26 -08:00
Austin Vazquez
6b04fcf560 vendor: github.com/golang-jwt/jwt/v4@v4.5.1
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
(cherry picked from commit 1eccc326de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-21 10:00:25 +01:00
Sebastiaan van Stijn
a080c3f32f Merge pull request #48899 from thaJeztah/27.x_backport_bump_grpc
[27.x backport] vendor: google.golang.org/grpc v1.66.3
2024-11-19 09:55:16 +01:00
Sebastiaan van Stijn
0f0b83a00b vendor: google.golang.org/grpc v1.66.3
- transport: Fix reporting of bytes read while reading headers
- xds/server: Fix xDS Server leak

full diff: https://github.com/grpc/grpc-go/compare/v1.66.2...v1.66.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4b26582bc6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-19 00:33:07 +01:00
Sebastiaan van Stijn
7bb4004fc8 Merge pull request #48880 from thaJeztah/27.x_containerd_1.7.23
[27.x] vendor: github.com/containerd/containerd v1.7.23
2024-11-15 15:43:10 +01:00
Sebastiaan van Stijn
2406bbff50 Merge pull request #48886 from thaJeztah/27.x_backport_cdi_0.8.0
[27.x] vendor: tags.cncf.io/container-device-interface v0.8.0
2024-11-15 15:01:07 +01:00
Sebastiaan van Stijn
1ef4219557 Merge pull request #48885 from thaJeztah/27.x_backport_more_build_tags
[27.x] update go:build tags to use go1.22, fix missing go:build tags
2024-11-15 15:00:39 +01:00
Sebastiaan van Stijn
b859373da2 vendor: tags.cncf.io/container-device-interface v0.8.0
Breaking change: The .ToOCI() functions in the specs-go package have been
removed. This removes the dependency on the OCI runtime specification from
the CDI specification definition itself.

What's Changed

- Add workflow to mark prs and issues as stale
- Remove the ToOCI functions from the specs-go package
- docs: add a pointer to community meetings in our docs.
- Bump spec version to v0.8.0
- Update spec version in README

Full diff:  https://github.com/cncf-tags/container-device-interface/compare/v0.7.2...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2f0180934d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:25:01 +01:00
Sebastiaan van Stijn
2f816b2596 [27.x] daemon/containerd: add missing go:build tag
This was introduced in c902e1a18701ac8e903314c688564e59cd1be997;

    make BIND_DIR=. shell
    make -C ./internal/gocompat/
    GO111MODULE=on go test -v
    # github.com/docker/docker/daemon/containerd
    ../../daemon/containerd/image.go:113:17: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../daemon/containerd/image.go:113:32: "github.com/distribution/reference".Named to satisfy comparable requires go1.20 or later (-lang was set to go1.16; check go.mod)
    ../../daemon/containerd/image_list.go:505:16: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:18:58 +01:00
Sebastiaan van Stijn
3e03c979da update go:build tags to use go1.22
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.

go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b453aa65fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:01:33 +01:00
Sebastiaan van Stijn
346ac21032 daemon/logger/loggerutils: add //go:build directives to prevent downgrading to go1.16
commit 77f2d90e27 introduced the slices import,
which uses generics, but failed to add the go:build directives.

    # github.com/docker/docker/daemon/logger/loggerutils
    ../../daemon/logger/loggerutils/logfile.go:770:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fb6da4f4b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:00:52 +01:00
Sebastiaan van Stijn
f54e5b0b19 libnetwork/networkdb: add //go:build directives to prevent downgrading to go1.16
commit 2847c4b7fe switched networkdb to use
go-immutable-radix v2, which uses generics, but failed to add the go:build
directives.

    # github.com/docker/docker/libnetwork/networkdb
    ../../libnetwork/networkdb/networkdb.go:47:19: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/networkdb/networkdb.go:259:33: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/networkdb/networkdb.go:269:25: function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/networkdb/networkdb.go:270:27: function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5403e3f4de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 13:00:52 +01:00
Sebastiaan van Stijn
6ec5197103 [27.x] vendor: github.com/containerd/containerd v1.7.23
similar to 5f39567e56 on master, but without
the hcsshim vendor updates, as it's pinned to v0.11.x in this branch.

full diff: https://github.com/containerd/containerd/compare/v1.7.22...v1.7.23

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-15 10:23:30 +01:00
Sebastiaan van Stijn
5765e9f35b Merge pull request #48842 from cpuguy83/27_backport_47983_log_reading
[27.x]: Make log reading more robust to errors
2024-11-14 13:53:22 +01:00
Sebastiaan van Stijn
158680383c Merge pull request #48863 from thaJeztah/27.x_backport_deprecate_pkg_platform
[27.x backport] deprecate pkg/platform and move internal
2024-11-14 13:40:57 +01:00
Sebastiaan van Stijn
1b40ddb635 Merge pull request #48864 from thaJeztah/27.x_backport_try-fix-flaky-plugin-client-test
[27.x backport] Increase test handler sleep, replace deprecated assert
2024-11-14 13:40:37 +01:00
Sebastiaan van Stijn
02e2e0426b Merge pull request #48867 from thaJeztah/27.x_backport_bump_compose_buildx
[27.x backport] Dockerfile: update buildx to v0.18.0, compose to v2.30.3
2024-11-14 13:40:15 +01:00
Sebastiaan van Stijn
31c8d23e7e Dockerfile: update compose to v2.30.3
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.29.7...v2.30.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a2c48e231)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-14 10:53:20 +01:00
Sebastiaan van Stijn
2ee3796e3f Dockerfile: update buildx to v0.18.0
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/0.17.1..0.18.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ee95c7bd89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-14 10:53:20 +01:00
Adam Simon
2acf9893c6 Increase test handler sleep, replace deprecated assert
- increase mock handler processing time to 50ms to try to prevent it from finishing before the 10ms client timeout occurs
- replace deprecated error type assertion

Signed-off-by: Adam Simon <adamsimon85100@gmail.com>
(cherry picked from commit 28dc2f6fac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-13 21:22:23 +01:00
Sebastiaan van Stijn
edfb2048ac deprecate pkg/platform and move internal
Functions in this package are only used internally in the daemon for
the `/info` endpoint (Architecture), and as part of `stats` (NumProcs).

I was not able to find external consumers, but deprecating the package
first, so that we can remove / dismantle the package in a follow-up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b034dc41a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-13 18:45:59 +01:00
Cory Snider
5ab7960b46 logger/journald: plumb contexts into reader
Co-authored-by:: Cory Snider <csnider@mirantis.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 6d94122217)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:10:48 +00:00
Brian Goff
857fc4730c logfile: Close reader when caller cancels
This allows for an individual decode operation to be cancelled while the
log reader is reading data from a log file by closing the underlying file.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 9b6ba18fc9)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:02:25 +00:00
Brian Goff
a7badb2bb5 Logfile: Add tracing spans
This plumbs a context down the stack and handles cancellation as needed
so that we can have correlated traces from the API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit dbf6873f45)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:02:16 +00:00
Brian Goff
d48c403efa Logfile: skip files that are corrupted
When there is an error in parsing an individual log file just close the
log and move on to the next one instead of erroring our the entire
request.

I investigated trying to error correct and scan ahead for corrupted log
files but found this is too much of a risk of parsing things we
shouldn't be and hence why this is just dropping the rest of the file.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1b46faf233)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:01:43 +00:00
Brian Goff
c9a7391911 Refactor logfile reads
This simplifies how we manage log files, especially rotated ones.
It also fixes a long-standing issue to lazily open rotated files so we
don't needlessly start decompressing files that we don't need.

Much of this is just setting things up for commits following this one.

It uses ReaderAtSize for managing all files to be tailed and manages
cleanups by passing closures.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 77f2d90e27)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:01:35 +00:00
Brian Goff
49f5431807 Fix time comparison in sharedtemp implementation
Times cannot be compared with `==` and instead should use the `t.Equal`
function.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit b37c8a03c0)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-11-09 17:01:27 +00:00
Sebastiaan van Stijn
b2c682be5f Merge pull request #48840 from akerouanton/remove-dns-answers-from-otel-spans
[27.4] libnet: don't put external DNS answers in OTel spans
2024-11-08 19:25:42 +01:00
Albin Kerouanton
173a9f2091 libnet: don't put external DNS answers in OTel spans
When containers make DNS resolution, and the domain name they're trying
to resolve doesn't match any running container, the DNS query is
forwarded to upstream servers. In that case, when we receive a response,
we put it in an OTel spans.

This was useful to debug DNS resolution on GHA, but it leads to
excessive memory usage when DNS resolution happen in a tight loop. So,
keep the OTel event signaling that a response was received, but drop the
answer from the OTel span.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-11-08 17:32:39 +01:00
Brian Goff
c14efb8dee Merge pull request #48820 from thaJeztah/27.x_backport_vendor-buildkit-v0.17.0
[27.x] vendor: update buildkit to v0.17.1, pin github.com/microsoft/hcsshim to v0.11.x
2024-11-08 07:58:05 -08:00
Tonis Tiigi
15b69c4200 vendor: update buildkit to v0.17.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 756586e437)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:32 +01:00
Tonis Tiigi
3280e50f65 vendor: update buildkit to v0.17.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 0e72863b9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:14 +01:00
Tonis Tiigi
00774deb96 vendor: update buildkit to v0.17.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 2cc21208e6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:14 +01:00
Sebastiaan van Stijn
0a67347003 vendor: google.golang.org/protobuf v1.34.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d2557466c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:13 +01:00
Sebastiaan van Stijn
033bd73bba [27.x] pin github.com/Microsoft/hcsshim to v0.11.x
We pin hcsshim to the v0.11.x version matching the version of containerd 1.7.
hcsshim v0.12.x comes with many (indirect) dependency updates that are not
desirable for minor releases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-08 15:03:11 +01:00
Sebastiaan van Stijn
e6b23cf105 Merge pull request #48832 from vvoland/update-go-27.x
[27.x] update to go1.22.9
2024-11-07 19:57:49 +01:00
Paweł Gronowski
0e4ab47f23 update to go1.22.9
- https://github.com/golang/go/issues?q=milestone%3AGo1.22.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.8...go1.22.9

go1.22.9 (released 2024-11-06) includes fixes to the linker. See the
[Go 1.22.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.22.9+label%3ACherryPickApproved)
milestone for details.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-07 13:20:09 +01:00
Paweł Gronowski
896f76e2e1 Merge pull request #48829 from vvoland/48357-27.x
[27.x backport] api/swagger: Improve description for platform in images/push
2024-11-06 15:25:54 +00:00
Paweł Gronowski
d3b9ec90ba api/swagger: Improve description for platform in images/push
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5014c90516)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-11-06 13:17:05 +01:00
Sebastiaan van Stijn
15ad1e4917 Merge pull request #48785 from vvoland/48777-27.x
[27.x] c8d/inspect: Fix duplicate RepoDigests
2024-10-30 11:01:22 +01:00
Paweł Gronowski
c902e1a187 c8d/inspect: Fix duplicate RepoDigests
Multiple images with the same repository name but different tag caused
the `RepoDigests` to contain duplicated entries for each of the image.

Deduplicate the slice before setting the `RepoDigests` field.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ba454f573b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-29 10:52:37 +01:00
Sebastiaan van Stijn
bad984fa46 Merge pull request #48768 from thaJeztah/27.x_backport_fix_debug_level
[27.x backport] volume: VolumesService.Create: fix log-level for debug logs
2024-10-26 20:18:24 +02:00
Sebastiaan van Stijn
b089f36796 volume: VolumesService.Create: fix log-level for debug logs
These log-entries were added in 10d57fde44,
but it looks like I accidentally left them as Error-logs following some
debugging (whoops!).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 352b4ff2f1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 17:43:27 +02:00
Sebastiaan van Stijn
5c383ad4dd Merge pull request #48764 from laurazard/27.x-backport-idle-conns
[27.x backport] client: prevent idle connections leaking FDs
2024-10-25 20:54:11 +02:00
Laura Brehm
afd7c9913f client: prevent idle connections leaking FDs
Patch from af6ada910f

Without this change, if a long-lived process uses the client and creates
connections, these connections are not released and grow over time.

We can also look into addressing this issue from the server side, but it
doesn't hurt for the `client` package to have good defaults and not
cause this.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit 5c72a95a30)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-10-25 18:13:24 +01:00
Sebastiaan van Stijn
b37ecae729 Merge pull request #48758 from vvoland/c8d-httpfallback-adjust
[27.x] c8d/httpfallback: Handle connection errors
2024-10-25 16:28:58 +02:00
Sebastiaan van Stijn
be5296767a Merge pull request #48755 from thaJeztah/27.x_backport_fix_anonymous_volumes_labels
[27.x backport] volume/mounts: fix anonymous volume not being labeled
2024-10-25 15:17:17 +02:00
Paweł Gronowski
de0135965e c8d/httpfallback: Handle connection errors
Adjust the httpFallback implementation to also handle non-TLS related
errors which can also happen when issuing a HTTPS requested to HTTP-only
registries.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-25 13:44:15 +02:00
Sebastiaan van Stijn
ddb56edd00 volume/mounts: fix anonymous volume not being labeled
`Parser.ParseMountRaw()` labels anonymous volumes with a `AnonymousLabel` label
(`com.docker.volume.anonymous`) label based on whether a volume has a name
(named volume) or no name (anonymous) (see [1]).

However both `VolumesService.Create()` (see [1]) and `Parser.ParseMountRaw()`
(see [2], [3]) were generating a random name for anonymous volumes. The latter
is called before `VolumesService.Create()` is called, resulting in such volumes
not being labeled as anonymous.

Generating the name was originally done in Create (fc7b904dce),
but duplicated in b3b7eb2723 with the introduction
of the new Mounts field in HostConfig. Duplicating this effort didn't have a
real effect until (`Create` would just skip generating the name), until
618f26ccbc introduced the `AnonymousLabel` in
(v24.0.0, backported to v23.0.0).

Parsing generally should not fill in defaults / generate names, so this patch;

- Removes generating volume names from  `Parser.ParseMountRaw()`
- Adds a debug-log entry to `VolumesService.Create()`
- Touches up some logs to use structured logs for easier correlating logs

With this patch applied:

    docker run --rm --mount=type=volume,target=/toto hello-world

    DEBU[2024-10-24T22:50:36.359990376Z] creating anonymous volume                     volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
    DEBU[2024-10-24T22:50:36.360069209Z] probing all drivers for volume                volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
    DEBU[2024-10-24T22:50:36.360341209Z] Registering new volume reference              driver=local volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02

[1]: 032721ff75/volume/service/service.go (L72-L83)
[2]: 032721ff75/volume/mounts/linux_parser.go (L330-L336)
[3]: 032721ff75/volume/mounts/windows_parser.go (L394-L400)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 10d57fde44)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-25 02:07:03 +02:00
Sebastiaan van Stijn
885b6e6103 Merge pull request #48742 from austinvazquez/cherry-pick-6be2074aefa60c3301dd728e7f9f6335a372c55a-to-27.x
[27.x backport] daemon: use OwnCgroupPath in withCgroups
2024-10-24 21:56:48 +02:00
Kir Kolyshkin
b7bc29af50 daemon: use OwnCgroupPath in withCgroups
Note: this usage comes from commit 56f77d5ade (part of PR 23430).

cgroups.InitCgroupPath is removed from runc (see [1]), and it is
suggested that users use OwnCgroupPath instead, because using init's is
problematic when in host PID namespace (see [2]) and is generally not
the right thing to do (see [3]).

[1]: https://github.com/opencontainers/runc/commit/fd5debf3
[2]: https://github.com/opencontainers/runc/commit/2b28b3c2
[3]: https://github.com/opencontainers/runc/commit/54e20217

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 6be2074aef)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-24 02:58:44 +00:00
Sebastiaan van Stijn
08674fb469 Merge pull request #48719 from thaJeztah/27.x_backport_deprecate_daemon_exists
[27.x backport] daemon: deprecate Daemon.Exists and Daemon.IsPaused
2024-10-22 15:12:19 +02:00
Sebastiaan van Stijn
1e11f2a1c0 daemon: deprecate Daemon.IsPaused
This function was poorly documented as it uses fuzzy matching under the hood,
and it's no longer used. Mark it as deprecated, and to be removed in the
next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac6e32cb5c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-21 17:22:31 +02:00
Sebastiaan van Stijn
0183f5dd13 daemon: deprecate Daemon.Exists
This function was poorly documented as it uses fuzzy matching under the hood,
and it's no longer used. Mark it as deprecated, and to be removed in the
next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d47c31ffdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-21 17:22:27 +02:00
Rob Murray
b6b3898090 Merge pull request #48714 from robmry/27.x-backport/48560-setup-user-chains
[27.x backport] Fix: setup user chains even if there are running containers
2024-10-21 14:25:31 +01:00
Andrés Maldonado
749f9ccee4 Fix: setup user chains even if there are running containers
Currently, the DOCKER-USER chains are set up on firewall reload or network
creation. If there are running containers at startup, configureNetworking won't
be called (daemon/daemon_unix.go), so the user chains won't be setup.

This commit puts the setup logic on a separate function, and calls it on the
original place and on initNetworkController.

Signed-off-by: Andrés Maldonado <maldonado@codelutin.com>
(cherry picked from commit a8bfa83667)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-21 12:40:26 +01:00
Sebastiaan van Stijn
e041d76796 Merge pull request #48712 from thaJeztah/27.x_backport_keep_manifest_order
[27.x backport] api: GET /images/json: preserve original manifest order
2024-10-21 13:17:54 +02:00
Sebastiaan van Stijn
8deb557451 api: GET /images/json: preserve original manifest order
The `manifests` option, as used for the `--tree` option on `docker image ls`
currently sorts manifests to put those that are present first. The intent was
to present "available" images at the top of each tree, followed by images that
were not pulled.

However, there's some limitations to this. First of all, the current approach
makes the output non-deterministic as the order in which variants are pulled
determines the order in which they're presented, i.e., the last pulled variant
is returned first (I omitted some variants in the example for brevity);

Here's the result of pulling `linux/riscv64`, then pulling `linux/arm64`;

    docker pull --platform=linux/riscv64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       10.6MB         3.37MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    └─ linux/arm64/v8       9cee2b382fe2           0B             0B

    docker pull --platform=linux/arm64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    └─ linux/amd64          33735bd63cf8           0B             0B

Repeating the steps but in reverse order results in the output to be reversed;

    docker image rm alpine:latest
    docker pull --platform=linux/arm64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       13.6MB         4.09MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    └─ linux/riscv64        80cde017a105           0B             0B

    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    └─ linux/amd64          33735bd63cf8           0B             0B

The second limitation is that order sometimes matters; when matching a
platform from a manifest-index, implementations may find multiple suitable
candidates. In most cases the _most_ suitable candidate can be selected
(e.g., prefer `linux/arm/v7` over `linux/arm/v6`), but manifest-indices do
allow multiple entries for the same platform, in which case implementations
match the first entry found.

While these situations will be less common (and usually due to incorect use
of tooling such as `docker manifest`), being able to observe the order in
which manifests appeared in the index can help debugging or help the user
understand why a specific variant was selected.

We should therefore not re-order these manifests, and return them in the
order in which they appeared. If we decide to present "present" variants
before "non-present" variants, we can do this ordering on the client side.

With this patch applied;

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker pull --quiet --platform=linux/arm64 alpine:latest
    docker image ls --tree alpine

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    ├─ linux/arm/v6         50f635c8b04d           0B             0B
    ├─ linux/arm/v7         f2f82d424957           0B             0B
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    ├─ linux/386            b3e87f642f5c           0B             0B
    ├─ linux/ppc64le        c7a6800e3dc5           0B             0B
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    └─ linux/s390x          2b5b26e09ca2           0B             0B

Which matches the order of the manifests in the index:

    docker buildx imagetools inspect --raw alpine:latest | jq -c .manifests[].platform
    {"architecture":"amd64","os":"linux"}
    {"architecture":"arm","os":"linux","variant":"v6"}
    {"architecture":"arm","os":"linux","variant":"v7"}
    {"architecture":"arm64","os":"linux","variant":"v8"}
    {"architecture":"386","os":"linux"}
    {"architecture":"ppc64le","os":"linux"}
    {"architecture":"riscv64","os":"linux"}
    {"architecture":"s390x","os":"linux"}

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d122ea0aea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-21 10:15:34 +02:00
Sebastiaan van Stijn
3bbb3c7f7f Merge pull request #48697 from austinvazquez/cherry-pick-pr-48668-to-27.x
[27.x backport] container: deprecate ErrNameReserved, ErrNameNotReserved, use errdefs instead
2024-10-19 17:46:15 +02:00
Sebastiaan van Stijn
c9cbe0e142 Merge pull request #48698 from austinvazquez/cherry-pick-pr-48654-to-27.x
[27.x backport] gha: more limits, update alpine version, and some minor improvements
2024-10-19 17:45:54 +02:00
Sebastiaan van Stijn
e4c067af91 gha: shorter time limits for smoke, validate
- validate-prepare and smoke-prepare took 10 seconds; limiting to 10 minutes
- smoke tests took less than 3 minutes; limiting to 10 minutes
- validate: most took under a minute, but "deprecate-integration-cli" took
  14 minutes; limiting to 30 minutes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a051aba82e)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:33 -07:00
Sebastiaan van Stijn
525b19ff04 gha: use "ubuntu-24.04" instead of "ubuntu-latest"
To be more explicit on what we're using.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 91c448bfb5)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:24 -07:00
Sebastiaan van Stijn
ceb41b83f3 gha: dco: small tweaks to running the container
- add `--quiet` to suppress pull progress output
- use `./` instead of `$(pwd)` now that relative paths are supported
- set the working directory on the container, so that we don't have to `cd`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9a14299540)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:13 -07:00
Sebastiaan van Stijn
2690b70a09 gha: dco: update ALPINE_VERSION to 3.20
Alpine 3.16 has been EOL for some time. Update to the latest version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cb98d759d)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:05:00 -07:00
Sebastiaan van Stijn
f53efc688f gha: build (binary), build (dynbinary): limit to 20 minutes
Regular runs are under 5 minutes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cfe0d2a131)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:04:34 -07:00
Sebastiaan van Stijn
166565e3ac gha: dco: limit to 10 minutes
Regular runs are under a minute.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e75f7aca2f)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 22:04:21 -07:00
Sebastiaan van Stijn
57a72c3c9d container: deprecate ErrNameReserved, ErrNameNotReserved
Use errdefs definitions instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cf90ca73f)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:59:20 -07:00
Sebastiaan van Stijn
8804551902 container: viewDB.ReserveName, view.GetID: return errdefs errors
Follow-up to 94dea2018e. Change these to return
errdefs types, which could allow us to move away from the sentinel errors
defined in the package, and instead use errdefs definitions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0200c58c4a)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:59:12 -07:00
Sebastiaan van Stijn
202aac0eb4 container: viewDB.withTxn: don't wrap errors
The withTxn function takes a custom function to execute; we should not
wrap those errors as the only responsibility of this function is to
execute the given function in a transaction.

This was introduced in 6549a270e9, and
an oversight of me.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0603bd9577)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:56:27 -07:00
Sebastiaan van Stijn
be8be11708 container: TestNames: don't string-match error assertions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0d36ab1875)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-18 21:56:11 -07:00
Sebastiaan van Stijn
b81bbc3aab Merge pull request #48695 from thaJeztah/27.x_backport_fix_48678
[27.x backport] dockerd-rootless-setuptool.sh: let --force ignore smoke test errors
2024-10-18 23:55:35 +02:00
Akihiro Suda
bedbe333bf dockerd-rootless-setuptool.sh: let --force ignore smoke test errors
Now `dockerd-rootless-setuptool.sh install --force` ignores errors
from `rootlesskit`.

This might be useful when installing Rootless Docker into a container
image with `RUN` instructions.

Fix issue 48678

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit fafdcd1194)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-18 22:22:54 +02:00
Sebastiaan van Stijn
cc70286271 Merge pull request #48693 from vvoland/48690-27.x
[27.x backport] cmd/dockerd: Add workaround for OTEL meter leak
2024-10-18 22:21:08 +02:00
Paweł Gronowski
ffb5bd905a cmd/dockerd: Add workaround for OTEL meter leak
OTEL meter implementation has a memory leak issue which causes each
meter counter invocation to create a new instrument when the meter
provider is not set.

Also add a test, which will fail once a fixed OTEL is vendored.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit cca7085464)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 17:06:12 +02:00
Sebastiaan van Stijn
b3b66cbbd7 Merge pull request #48691 from vvoland/48631-27.x
[27.x backport] Touch-up some errors for missing platforms
2024-10-18 17:05:24 +02:00
Paweł Gronowski
12cc7f7696 daemon/containerd: getPushDescriptor: fix formatting of platform in errors
The platform was printed in its raw format, which didn't produce a very
readable output;

Before this change:

    $ docker image push --platform=linux/amd64 alpine:arm64
    Error response from daemon: no suitable image manifest found for platform {amd64 linux [] }

After this change:

    $ docker image push --platform=linux/amd64 alpine:arm64
    Error response from daemon: no suitable image manifest found for platform linux/amd64

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:49:38 +02:00
Paweł Gronowski
01412dacf7 daemon/containerd: touch-up platform not found error
- Changed "match" to "provide", in an attempt to indicate that the image is
  a multi-platform image that doesn't contain the given platform.
- Remove the "wanted" and instead make the requested platforms to be a part
  of the sentence, but within braces.

Before this patch:

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker image history --platform=linux/amd64 alpine
    Error response from daemon: image with reference alpine:latest was found but does not match the specified platform: wanted linux/amd64

With this patch:

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker image history --platform=linux/amd64 alpine
    Error response from daemon: image with reference alpine:latest was found but does not provide the specified platform (linux/amd64)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:48:54 +02:00
Sebastiaan van Stijn
3c93981344 images: GetImage: touch-up error message for missing platform
Slightly touching up the error message, because the "wanted/actual" output
felt too much like a assertion failure in a test-case.

- Format the image-reference using "familiar" format, which shows the
  image's short name for official images.
- Move the actual and requested platforms to be a part of the sentence,
  but within braces.

Before this patch:

    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: image with reference docker.io/library/alpine:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux/riscv64

With this patch:

    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: image with reference alpine:latest was found but its platform (linux/riscv64) does not match the specified platform (linux/amd64)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8681b3c2ac)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:48:32 +02:00
Sebastiaan van Stijn
03225b1a41 PushImage: remove misleading error about --platform without containerd
Without containerd store enabled, we were discarding underlying errors,
and instead informing the user that `--platform` is not suported;

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: graphdriver backed image store doesn't support multiplatform images

However, that's not the case; platform filtering works, but if the image
is not the right platform, the push fails (which is the same as would
happen with the containerd image store enabled).

    docker image push --platform=linux/amd64 alpine:latest
    Error response from daemon: image with reference docker.io/library/alpine:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux/riscv64

When specifying the platform and that platform matches, it finds the image,
and the push continue;

    docker image push --platform=linux/riscv64 alpine:latest
    The push refers to repository [docker.io/library/alpine]
    3fd4750fd687: Layer already exists
    ...

(The above example obviously fails because I don't have permissions to
push official images).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d31c241ea5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-10-18 13:48:30 +02:00
Paweł Gronowski
2ecd6971be Merge pull request #48687 from thaJeztah/27.x_backport_bump_swarmkit
[27.x backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20241017191044-e8ecf83ee08e
2024-10-18 12:19:16 +02:00
Sebastiaan van Stijn
a5555a93dd Merge pull request #48688 from thaJeztah/27.x_backport_build_split_history_db
[27.x backport] build: create distinct history db for each store
2024-10-18 11:27:15 +02:00
CrazyMax
5376990d4d build: create distinct history db for each store
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 58f4e916a3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 22:50:16 +02:00
Sebastiaan van Stijn
34d2e5e7e6 vendor: github.com/moby/swarmkit/v2 v2.0.0-20241017191044-e8ecf83ee08e
- add Unwrap error to custom error types
- removes dependency on github.com/rexray/gocsi
- fix CSI plugin load issue

full diff: ea1a7cec35...e8ecf83ee0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21b0d5f0c1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-17 22:31:26 +02:00
Sebastiaan van Stijn
bd021c3310 Merge pull request #48685 from robmry/backport-27.x/48664_br_netfilter_noproxy
[27.x backport] Enable bridge netfiltering if userland-proxy=false
2024-10-17 21:14:36 +02:00
Sebastiaan van Stijn
2f5d5faedd Merge pull request #48684 from robmry/backport-27.x/nodad_bridge_address
[27.x backport] Set NODAD on bridge IPv6 addresses
2024-10-17 21:13:55 +02:00
Rob Murray
0f8fc3105e Enable bridge netfiltering if userland-proxy=false
In release 27.0, ip6tables was enabled by default. That caused a
problem on some hosts where iptables was explicitly disabled and
loading the br_netfilter module (which loads with its nf-call-iptables
settings enabled) caused user-defined iptables rules to block traffic
on bridges, breaking inter-container communication.

In 27.3.0, commit 5c499fc4b2 delayed
loading of the br_netfilter module until it was needed. The load
now happens in the function that sets bridge-nf-call-ip[6]tables when
needed. It was only called for icc=false networks.

However, br_netfilter is also needed when userland-proxy=false.
Without it, packets addressed to a host-mapped port for a container
on the same network are not DNAT'd properly (responses have the server
container's address instead of the host's).

That means, in all releases including 26.x, if br_netfilter was loaded
before the daemon started - and the OS/user/other-application had
disabled bridge-nf-call-ip[6]tables, it would not be enabled by the
daemon. So, ICC would fail for host-mapped ports with the userland-proxy
disabled.

The change in 27.3.0 made this worse - previously, loading br_netfilter
whenever iptables/ip6tables was enabled meant that bridge-netfiltering
got enabled, even though the daemon didn't check it was enabled.

So... check that br_netfilter is loaded, with bridge-nf-call-ip[6]tables
enabled, if userland-proxy=false.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 0548fe251c)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-17 16:39:01 +01:00
Rob Murray
505c4c32be Set NODAD on bridge IPv6 addresses
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 38f0483ea7)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-10-17 16:13:32 +01:00
Sebastiaan van Stijn
6060b750e1 Merge pull request #48656 from thaJeztah/27.x_backport_docs_update_api_links
[27.x backport] update links to API documentation
2024-10-15 14:51:42 +02:00
Paweł Gronowski
b42c0bea9b Merge pull request #48666 from thaJeztah/27.x_backport_api_resize_required
[27.x backport] docs: api: document w (width) and h (height) query params as required
2024-10-15 11:01:49 +02:00
Sebastiaan van Stijn
c00b3108e5 docs: api: document w (width) and h (height) query params as required
Query parameters default to being _optional_ in swagger, but for these
endpoints, they were required. Omitting these parameters (or passing an
empty value) produces a 400 (bad request);

    docker run -d --name foo nginx:alpine

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize'
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h=100'

This patch updates the swagger to reflect that these parameters are required.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d9627b6159)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-14 23:25:25 +02:00
Sebastiaan van Stijn
e39ced5c00 api: document w (width) and h (height) query params as required
Query parameters default to being _optional_ in swagger, but for these
endpoints, they were required. Omitting these parameters (or passing an
empty value) produces a 400 (bad request);

    docker run -d --name foo nginx:alpine

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize'
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h='
    {"message":"strconv.Atoi: parsing \"\": invalid syntax"}

    curl -XPOST --unix-socket /var/run/docker.sock 'http://localhost/v1.47/containers/foo/resize?w=100&h=100'

This patch amends the swagger to reflect that these parameters are required.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a3a7e8ee8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-14 23:25:22 +02:00
Sebastiaan van Stijn
3c258e6011 pkg/authorization: update link to API documentation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a2361ea09)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 18:56:00 +02:00
Sebastiaan van Stijn
29f0dcd902 client: update link to API documentation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5b149d3faa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 18:56:00 +02:00
Sebastiaan van Stijn
21c974b281 docs/api: version-history: update documentation links
URLs were updated on docs.docker.com

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f8f143a27)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-12 18:55:29 +02:00
Sebastiaan van Stijn
fb64e14bbf Merge pull request #48646 from austinvazquez/cherry-pick-c68c9aed8cb3916669de6d7f2c564279ec83663f-to-27.x
[27.x backport] gha: restrict cross and bin-image to 20 minutes
2024-10-12 16:07:34 +02:00
Sebastiaan van Stijn
0bc91d5dc8 gha: restrict cross and bin-image to 20 minutes
We had a couple of runs where these jobs got stuck and github
actions didn't allow terminating them, so that they were only
terminated after 120 minutes.

These jobs usually complete in 5 minutes, so let's give them
a shorter timeout. 20 minutes should be enough (don't @ me).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c68c9aed8c)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-12 01:16:03 +00:00
Sebastiaan van Stijn
b434599c2d Merge pull request #48635 from thaJeztah/27.x_backport_bump_buildkit
[27.x backport] vendor: github.com/moby/buildkit v0.17.0-rc1
2024-10-11 23:00:13 +02:00
Sebastiaan van Stijn
d755573aba vendor: github.com/moby/buildkit v0.17.0-rc1
full diff:

- https://github.com/moby/buildkit/compare/v0.16.0...v0.17.0-rc1
- 91a3fc4684...3f140a1299
- 9b4947da39...701f63a606

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fe04c142d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-11 18:39:10 +02:00
Sebastiaan van Stijn
2388a908d5 Merge pull request #48633 from thaJeztah/27.x_backport_gha_timeout
[27.x backport] gha: add guardrails timeouts on all jobs
2024-10-10 20:32:43 +02:00
Sebastiaan van Stijn
32afa26719 gha: add guardrails timeouts on all jobs
We had a few "runaway jobs" recently, where the job got stuck, and kept
running for 6 hours (in one case even 24 hours, probably due some github
outage). Some of those jobs could not be terminated.

While running these actions on public repositories doesn't cost us, it's
still not desirable to have jobs running for that long (as they can still
hold up the queue).

This patch adds a blanket "2 hours" time-limit to all jobs that didn't
have a limit set. We should look at tweaking those limits to actually
expected duration, but having a default at least is a start.

Also changed the position of some existing timeouts so that we have a
consistent order in which it's set; making it easier to spot locations
where no limit is defined.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7e2783d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 18:42:01 +02:00
Sebastiaan van Stijn
875144c174 Merge pull request #48630 from thaJeztah/27.x_backport_integration_remove_TestBuildWithSession
[27.x backport] integration/build: remove TestBuildWithSession, and fsutil direct dependency
2024-10-10 17:33:38 +02:00
Sebastiaan van Stijn
8dc3ce5606 integration/build: remove TestBuildWithSession, and fsutil direct dependency
This test was originally used to test the experimental `--stream` option with
the classic builder. This feature was removed in Docker 20.10 through
6ca3ec88ae, which also disabled the test
(adding a note to rewrite the test for BuildKit).

The above is 5 years ago, and we now run the BuildKit integration tests
as part of our CI, which (I assume) should cover this functionality.

Removing this test, as it depends on github.com/tonistiigi/fsutil, which
does not have a stable API, and frequently cannot be updated independently
from BuildKit.

Also adding a linter-rule to prevent it from accidentally being imported;

    errdefs/defs.go:3:8: import 'github.com/tonistiigi/fsutil' is not allowed from list 'main': The fsutil module does not have a stable API, so we should not have a direct dependency unless necessary. (depguard)
    import _ "github.com/tonistiigi/fsutil"
           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fa6d3c230)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 14:38:19 +02:00
Sebastiaan van Stijn
e9ea285eec Merge pull request #48621 from thaJeztah/27.x_backport_bump_buildkit_deps
[27.x backport] vendor assorted dependencies in preparation of BuildKit v0.17
2024-10-10 14:36:34 +02:00
Sebastiaan van Stijn
136fdfcdfc vendor: google.golang.org/grpc v1.66.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2e699872bc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:14 +02:00
Sebastiaan van Stijn
6eed19d656 vendor: google.golang.org/protobuf v1.34.1
full diff: https://github.com/protocolbuffers/protobuf-go/compare/v1.33.0...v1.34.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e007ef71c1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:14 +02:00
Sebastiaan van Stijn
f29cc7c2cf vendor: github.com/cespare/xxhash/v2 v2.3.0
full diff: https://github.com/cespare/xxhash/compare/v2.2.0...v2.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30b6e58242)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
5f92ab397b vendor: golang.org/x/tools v0.25.0
full diff: https://github.com/golang/tools/compare/v0.24.0...v0.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4592ce42b6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
9af7d06698 vendor: golang.org/x/mod v0.21.0
full diff: https://github.com/golang/mod/compare/v0.20.0...v0.21.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 36549fb56e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
8fef24d9d4 vendor: golang.org/x/net v0.29.0
no changes in vendored code

full diff: https://github.com/golang/net/compare/v0.28.0...v0.29.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cc80652bcd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
efc5252041 vendor: golang.org/x/crypto v0.27.0
full diff: https://github.com/golang/crypto/compare/v0.26.0...v0.27.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d6db83acc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:13 +02:00
Sebastiaan van Stijn
a8e92fa662 vendor: golang.org/x/text v0.18.0
no changes in vendored code

full diff: https://github.com/golang/text/compare/v0.17.0...v0.18.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6e32888caa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:12 +02:00
Sebastiaan van Stijn
80cd75d7d5 vendor: golang.org/x/sys v0.25.0
full diff: https://github.com/golang/sys/compare/v0.24.0...v0.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 51cf0ed303)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:12 +02:00
Sebastiaan van Stijn
a2a544202b vendor: go.etcd.io/bbolt v1.3.11
BoltDB

- Fix the freelist.allocs isn't rollbacked when a tx is rollbacked.

CMD

- Add -gobench-output option for bench command to adapt to benchstat.

Other

- Bump go version to 1.22.x.
- This patch also added dmflakey package, which can be reused by other projects. See #812.

full diff: https://github.com/etcd-io/bbolt/compare/v1.3.10...v1.3.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7565ba40)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 13:17:10 +02:00
Sebastiaan van Stijn
ef4072a113 Merge pull request #48624 from thaJeztah/27.x_backport_fix_buildkit_go_version
[27.x backport] gha: buildkit: make sure expected Go version is installed
2024-10-10 13:16:41 +02:00
Sebastiaan van Stijn
418a79d51b gha: buildkit: make sure expected Go version is installed
The buildkit workflow uses Go to determine the version of Buildkit to run
integration-tests for. It currently uses on the default version that's
installed on the GitHub actions runners (1.21.13 currently), but this fails
if the go.mod/vendor.mod specify a higher version of Go as required version.

If this fails, the BUILDKIT_REF and REPO env-vars are not set / empty,
resulting in the workflow checking out the current (moby) repository instead
of buildkit, which fails.

This patch adds a step to explicitly install the expected version of Go.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 02d4fc3234)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-10 11:43:13 +02:00
Akihiro Suda
ee1210fa86 Merge pull request #48580 from austinvazquez/cherry-pick-ca4c68ab956993b47fd0046b4d96eceab8b9a261-to-27.x
[27.x backport] update to go1.22.8
2024-10-07 15:39:17 +09:00
Sebastiaan van Stijn
43580ccbc1 update to go1.22.8
go1.22.8 (released 2024-10-01) includes fixes to cgo, and the maps and syscall
packages. See the Go 1.22.8 milestone on our issue tracker for details;

- https://github.com/golang/go/issues?q=milestone%3AGo1.22.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.7...go1.22.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ca4c68ab95)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-04 19:44:30 +00:00
Sebastiaan van Stijn
78ab948df8 Merge pull request #48578 from austinvazquez/cherry-pick-docker-cli-27.3.1-compose-2.29.7-to-27.x
[27.x backport] Dockerfile: update docker CLI to v27.3.1, compose to v2.29.7
2024-10-04 14:00:01 +02:00
Sebastiaan van Stijn
af8b897560 Dockerfile: update compose to v2.29.7
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.29.4...v2.29.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7624a6bfa5)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-04 05:47:49 +00:00
Sebastiaan van Stijn
d91a922ef3 Dockerfile: update docker CLI to v27.3.1
Update the Docker CLI used in the dev-container

full diff: https://github.com/docker/cli/compare/v27.2.1...v27.3.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 233dce3dba)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-10-04 05:47:35 +00:00
Sebastiaan van Stijn
aec336e45d Merge pull request #48574 from thaJeztah/27.x_backport_add_codeql
[27.x backport] gha: add CodeQL Analysis workflow
2024-10-03 16:23:19 +02:00
Sebastiaan van Stijn
0138291345 gha: add CodeQL Analysis workflow
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 450200b4a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-03 13:20:02 +02:00
Sebastiaan van Stijn
41ca978a0a Merge pull request #48525 from thaJeztah/27.x_backport_govulncheck_permissions
[27.x backport] gha: govulncheck: make sure read permissions are set
2024-09-18 12:25:38 +02:00
Sebastiaan van Stijn
a6b772b24c gha: govulncheck: make sure read permissions are set
If any permission is set, any permission not included in the list is
implicitly set to "none".

see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

The govulncheck check need read permissions, which is not problematic
for public repositories, but may be needed when running in a private
fork (such as those used for security releases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f1d739de5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-18 10:36:14 +02:00
Sebastiaan van Stijn
856359cb77 Merge pull request #48514 from robmry/backport-27.x/wsl2_mirrored_loopback0_workaround
[27.x backport] Do not DNAT packets from WSL2's loopback0
2024-09-17 15:25:41 +02:00
Rob Murray
cd21af7e41 Do not DNAT packets from WSL2's loopback0
When running WSL2 with mirrored mode networking, add an iptables
rule to skip DNAT for packets arriving on interface loopback0 that
are addressed to a localhost address - they're from the Windows
host.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit f9c0103413)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-09-17 10:03:08 +01:00
Sebastiaan van Stijn
8516f3b0c1 Merge pull request #48510 from thaJeztah/27.x_backport_bump_buildx_compose
[27.x backport] Dockerfile: update buildx to v0.17.1, compose to v2.29.4
2024-09-16 17:08:16 +02:00
Sebastiaan van Stijn
3a7779ad61 Merge pull request #48511 from robmry/backport-27.x/48375_bridge_netfiltering
[27.x backport] Only enable bridge netfiltering when needed
2024-09-16 16:55:10 +02:00
Rob Murray
5c499fc4b2 Only enable bridge netfiltering when needed
Kernel module br_netfilter is loaded when the daemon starts with
either iptables or ip6tables enabled. That automatically sets:
  net.bridge.bridge-nf-call-arptables = 1
  net.bridge.bridge-nf-call-ip6tables = 1
  net.bridge.bridge-nf-call-iptables = 1

So, when:
- docker was running happily with iptables=false, and
- no explicit ip6tables=false, and
- br_netfilter was not loaded
... the change in moby 27.0 to enable ip6tables by default, resulted
in net.bridge.bridge-nf-call-iptables being enabled.

If the host also had a firewall with default-drop on its forward
chain - that resulted in packets getting dropped between containers
on a bridge network.

So, only try to load br_netfilter when it's needed - it's only needed
to implement "--icc=false", which can only be used when iptables or
ip6tables is enabled.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit db25b0dcd0)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-09-16 14:59:40 +01:00
Sebastiaan van Stijn
98f24aaf8a Merge pull request #48506 from thaJeztah/27.x_backport_man_dockerd_logformat
[27.x backport] man: dockerd: add description for --log-format option
2024-09-16 15:50:11 +02:00
Sebastiaan van Stijn
8adc8e405d Dockerfile: update compose to v2.29.4
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/compose/compare/v2.29.2...v2.29.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 605f51172a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 14:56:28 +02:00
Sebastiaan van Stijn
576fc88b1a Dockerfile: update buildx to v0.17.1
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/0.17.0...0.17.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3d4831a23f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 14:56:27 +02:00
Sebastiaan van Stijn
e928a0d22d man: dockerd: add description for --log-format option
This option was added in a08abec9f8,
as part of Docker v25.0, but did not update the docs and manpage.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 45a9dde660)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 14:13:27 +02:00
Sebastiaan van Stijn
aed88ada36 Merge pull request #48503 from thaJeztah/27.x_backport_dockerd_typo
[27.x backport] dockerd: fix docs, improve validation and improve coverage of "--feature" flag
2024-09-16 14:12:17 +02:00
Sebastiaan van Stijn
82cf138590 internal/opts: SetOpts: invalidate empty option-names
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c2fc1f4a40)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
487a2f5314 internal/opts: SetOpts.Set: remove redundant var assignment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 908bb959e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
6d4a507acc internal/opts: SetOpts,NamedSetOpts: test for optional value
The value is optional for SetOpts (and NamedSetOpts), and implied
"true" when omitted.

This patch adds a test-case for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 758cca6036)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
bae46f6463 man: dockerd: value is optional for --feature flag
The --feature flag allows the boolean value to be omitted.
If only a name is provided, the default is "true".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 50e83a0713)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
176e04d73b man: fix duplicate word in --feature flag description
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2b6550bb2e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-16 11:04:04 +02:00
Sebastiaan van Stijn
7c0c43a239 Merge pull request #48499 from austinvazquez/cherry-pick-8b0e94ffaf7ea7d42391a3961e795b33976256c9-27.x
[27.x backport] Update dlv in the dev-env
2024-09-16 10:51:15 +02:00
Rob Murray
06c9c72269 Update dlv in the dev-env
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 8b0e94ffaf)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-09-14 07:25:40 -07:00
Sebastiaan van Stijn
bf60e5cced Merge pull request #48488 from thaJeztah/27.x_backport_c8d-prune-keeplastref
[27.x backport] c8d/prune: Keep the last tagged image instead of creating dangling image
2024-09-12 23:34:15 +02:00
Sebastiaan van Stijn
16294f36a2 Merge pull request #48490 from thaJeztah/27.x_backport_dockerd_man_features
[27.x backport] man: update dockerd man-page to include --feature flag
2024-09-12 23:33:52 +02:00
Sebastiaan van Stijn
15fbba2169 man: update dockerd man-page to include --feature flag
commit f13c08246d introduced
this flag, but did not yet update the manpage.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fa06acc851)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:51:02 +02:00
Paweł Gronowski
b262a95d67 c8d/prune: Keep deletion order stable
When untagging multiple images targetting the same digest, delete the
images in lexographic order to be consistent with graphdrivers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fefa98ae90)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:46:40 +02:00
Paweł Gronowski
39c17094b0 c8d/prune: Add OTEL spans
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 23e79a261e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:46:19 +02:00
Sebastiaan van Stijn
343800d7be adjust TestPruneDontDeleteUsedImage for 27.x branch
The 27.x branch does not have 7053007f71
adjusting the test to use the old location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 21:45:10 +02:00
Sebastiaan van Stijn
6753acddfe Merge pull request #48487 from thaJeztah/27.x_backport_add-feature-flags-daemon
[27.x backport] Add feature to daemon flags
2024-09-12 21:32:43 +02:00
Sebastiaan van Stijn
9c94a4d3cd Merge pull request #48485 from austinvazquez/cherry-pick-adb00d3d555f9dae37305c872dc32f6b654088d9-to-27.x
[27.x backport] TestIPRangeAt64BitLimit: remove colon after XFAIL to help grepping
2024-09-12 20:03:59 +02:00
Paweł Gronowski
95ffe8493e c8d/prune: Keep the last tagged image instead of creating dangling image
Don't turn images into dangling when they are used by containers created
with an image specified by an ID only (e.g. `docker run 82d1e9d`).

Keep the last image reference with the same target when all other
references would be pruned.

If the container was created with a digested and tagged reference (e.g.
`docker run alpine:latest@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1`),
the `alpine:latest` image won't get untagged.

This change makes the behavior consistent with the graphdriver
implementation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e4c2eb9d8a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 19:34:24 +02:00
Derek McGowan
c40cd54593 Add feature to daemon flags
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f13c08246d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 19:30:17 +02:00
Sebastiaan van Stijn
dbee7b16df TestIPRangeAt64BitLimit: remove colon after XFAIL to help grepping
When looking for failures in CI, I always search for `FAIL:` (with a
trailing colon) to find tests that fail. This test has some test-cases
that are currently expected to fail, but due to the colon would also
be included when searching;

    === RUN   TestIPRangeAt64BitLimit/ipRange_at_end_of_64-bit_subnet
        bridge_test.go:196: XFAIL: Container startup failed with error: Error response from daemon: no available IPv6 addresses on this network's address pools: test64bl (b014e28c35c14cc34514430a8cfe1c97632c7988c56d89cea46abb10fa32229d)
    === RUN   TestIPRangeAt64BitLimit/ipRange_at_64-bit_boundary_inside_56-bit_subnet
        bridge_test.go:196: XFAIL: Container startup failed with error: Error response from daemon: no available IPv6 addresses on this network's address pools: test64bl (fb70301550d7a2d1d3425f5c1010a9ef487a9a251221a2d68ac49d257b249013)

Remove the trailing `:` so that searching for unexpected failures does not
include these tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit adb00d3d55)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-09-12 16:40:57 +00:00
Sebastiaan van Stijn
6c084b6646 Merge pull request #48479 from thaJeztah/27.x_backport_rootlesskit
[27.x backport] Dockerfile: update RootlessKit to v2.3.1
2024-09-12 17:34:19 +02:00
Sebastiaan van Stijn
81e74f3e09 Merge pull request #48483 from thaJeztah/27.x_backport_fix-gpus-0
[27.x backport] Explicitly disable nvidia device injection for --gpus=0
2024-09-12 16:42:28 +02:00
Evan Lezar
bd71b5d319 Explicitly disable nvidia device injection for --gpus=0
This change ensures that when --gpus=0 is selected, the injection of
NVIDIA device nodes and libraries is disabled by setting the
NVIDIA_VISIBLE_DEVICES environment variable to void instead of
leaving this unspecfied.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
(cherry picked from commit 5128007116)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-12 13:21:53 +02:00
Akihiro Suda
a5478dde0e Dockerfile: update RootlessKit to v2.3.1
https://github.com/rootless-containers/rootlesskit/compare/v2.0.2...v2.3.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 43036f350a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-11 18:27:47 +02:00
Sebastiaan van Stijn
c717e4cd2a Merge pull request #48481 from laurazard/backport-typos
[27.x backport] Fix typos
2024-09-11 17:51:34 +02:00
Sebastiaan van Stijn
8956bdd7b3 Merge pull request #48477 from vvoland/48475-27.x
[27.x backport] Dockerfile: Update CLI, buildx and compose
2024-09-11 17:20:21 +02:00
Nathan Baulch
80d0013217 Fix typos
Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
(cherry picked from commit 59eba0ae13)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-09-11 15:36:28 +01:00
Paweł Gronowski
ce86fade0d Dockerfile: update compose to v2.29.2
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/cli/compare/v2.29.0...v2.29.2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 962da27cdf)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 12:29:02 +02:00
Paweł Gronowski
682b96c479 Dockerfile: update buildx to 0.17.0
Update the buildx cli plugin used in the dev-container

full diff: https://github.com/docker/buildx/compare/0.16.1...0.17.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e5e8addeee)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 12:29:01 +02:00
Paweł Gronowski
c3aaaf2be0 Dockerfile: update docker CLI to v27.2.1
Update the Docker CLI used in the dev-container

full diff: https://github.com/docker/cli/compare/v27.0.2...v27.2.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d291722110)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-11 12:28:57 +02:00
Sebastiaan van Stijn
20bff974a0 Merge pull request #48469 from thaJeztah/27.x_backport_vendor_buildkit_v0.16
[27.x backport] vendor: update buildkit to v0.16.0
2024-09-11 10:14:27 +02:00
Sebastiaan van Stijn
ecf2f8420e Merge pull request #48459 from thaJeztah/27.x_backport_fix_push_docs
[27.x backport] api: swagger: fix documentation for image push endpoint
2024-09-11 10:13:58 +02:00
Sebastiaan van Stijn
b8e36aafe7 Merge pull request #48470 from thaJeztah/27.x_backport_api_fix_deprecation_version
[27.x backport] api/swagger: update deprecation version for erroneous fields
2024-09-11 10:13:15 +02:00
CrazyMax
ce07360a1c vendor: update buildkit to v0.16.0
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 71e449de0a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 20:21:53 +02:00
Sebastiaan van Stijn
cdbfb9c71a Merge pull request #48468 from thaJeztah/27.x_backport_bump_containerd_1.7.22
[27.x backport] Dockerfile/vendor: update containerd to v1.7.22
2024-09-10 19:07:38 +02:00
Sebastiaan van Stijn
ab99508c07 docs/api: update deprecation version for erroneous fields (v1.46, v1.47)
commit af0cdc36c7 officially marked these
fields as deprecated and to be removed in API v1.47 (which was targeted
for v28.0). We shipped v1.47 with the v27.2 release, but did not yet
remove the erroneous fields.

This patch updates the version to v1.48.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 09fc6ab2d9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 16:32:10 +02:00
Sebastiaan van Stijn
c7a8942d6d api/swagger: update deprecation version for erroneous fields
commit af0cdc36c7 officially marked these
fields as deprecated and to be removed in API v1.47 (which was targeted
for v28.0). We shipped v1.47 with the v27.2 release, but did not yet
remove the erroneous fields.

This patch updates the version to v1.48.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3df03d8e66)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 16:32:07 +02:00
Tonis Tiigi
57ba0b05e6 vendor: update buildkit to v0.16.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 6f37e5a168)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:38 +02:00
CrazyMax
16bf28ec2d test: update since session signature has changed
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 5ebc179cb1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
CrazyMax
a0650bf67c vendor: update buildkit to v0.16.0-rc1
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit f4261dbb75)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
Akihiro Suda
f7ea6bb4d9 vendor.mod: golang.org/x/net v0.28.0, etc.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d6446d81d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
Akihiro Suda
da2b59d3a2 vendor.mod: golang.org/x/time v0.6.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 65fdd363a0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:37 +02:00
Akihiro Suda
3e60519c9c vendor.mod: golang.org/x/sys v0.24.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 4bfdfd6f72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Akihiro Suda
e6a4c5f840 vendor.mod: golang.org/x/sync v0.8.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 78d0ef5e87)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Akihiro Suda
2514b02677 vendor.mod: golang.org/x/mod v0.20.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 492fe40052)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Akihiro Suda
69f9ab2958 vendor.mod: drop direct dependency on golang.org/x/exp
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d7ff6b5085)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:36 +02:00
Sebastiaan van Stijn
70c8d41445 vendor: github.com/containerd/nydus-snapshotter v0.14.0
- removes use of deprecated github.com/containerd/containerd/errdefs package
- removes use of deprecated github.com/containerd/containerd/platforms package
- removes use of deprecated github.com/containerd/containerd/reference/docker package
- switch to dario.cat/mergo v1.0.0 dependency
- remove use of deprecated CRI Alpha API

full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.13.14...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af84ddec13)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:32:25 +02:00
Sebastiaan van Stijn
54e4cdc044 vendor: github.com/containerd/nydus-snapshotter v0.13.14
- removes use of deprecated github.com/containerd/containerd/log package

full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.13.7...v0.13.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e5c0c4d77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:31:44 +02:00
Sebastiaan van Stijn
dca7a36092 vendor: google.golang.org/grpc v1.62.0
diffs:

- https://github.com/grpc/grpc-go/compare/v1.59.0...v1.62.0
- 83a465c022...ef4313101c
- https://github.com/googleapis/google-api-go-client/compare/v0.149.0...v0.155.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d44f869a2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:27:53 +02:00
Sebastiaan van Stijn
458110ccd0 vendor: golang.org/x/time v0.5.0
full diff: https://github.com/golang/time/compare/v0.3.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 716ec490db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:27:52 +02:00
Sebastiaan van Stijn
27b5c33847 vendor: github.com/moby/sys/sequential v0.6.0
full diff: https://github.com/moby/sys/compare/sequential/v0.5.0...sequential/v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21fd5a3f30)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:22:34 +02:00
Sebastiaan van Stijn
6e77ead91e vendor: github.com/moby/sys/symlink v0.3.0
full diff: https://github.com/moby/sys/compare/symlink/v0.2.0...symlink/v0.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6a6b49f2de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:20:47 +02:00
Sebastiaan van Stijn
0890631ac7 vendor: github.com/moby/sys/signal v0.7.1
full diff: https://github.com/moby/sys/compare/signal/v0.7.0...signal/v0.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68636d56fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:17:40 +02:00
Sebastiaan van Stijn
508173c39d vendor: github.com/moby/sys/mount v0.3.4
full diff: https://github.com/moby/sys/compare/mount/v0.3.3...mount/v0.3.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44b266ab59)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:17:40 +02:00
Sebastiaan van Stijn
9b26c10e8b vendor: github.com/moby/sys/mountinfo v0.7.2
full diff: https://github.com/moby/sys/compare/mountinfo/v0.7.1...mountinfo/v0.7.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc4a12874e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:17:40 +02:00
Sebastiaan van Stijn
2f39dca20a Merge pull request #48463 from gdams/27
[27.x backport] seccomp: add riscv64 mapping to seccomp_linux.go
2024-09-10 14:14:46 +02:00
Sebastiaan van Stijn
d3884731cd Dockerfile: update containerd binary to v1.7.22
Update the containerd binary that's used in CI and static binaries

- Update to go1.22.7, go1.23.1
- CRI: Cumulative stats can't decrease
- Fix bug where init exits were being dropped
- Update runc binary to 1.1.14

- diff: https://github.com/containerd/containerd/compare/v1.7.21...v1.7.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92195c1333)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:04:19 +02:00
Sebastiaan van Stijn
cdbc2b6869 vendor: github.com/containerd/containerd v1.7.22
no significant changes in vendored code

- diff: https://github.com/containerd/containerd/compare/v1.7.21...v1.7.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.22

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b4a3e8a307)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 14:04:16 +02:00
George Adams
d0462c0e80 seccomp: add riscv64 mapping to seccomp_linux.go
Signed-off-by: George Adams <georgeadams1995@gmail.com>
(cherry picked from commit 1161b790cf)
Signed-off-by: George Adams <georgeadams1995@gmail.com>
2024-09-10 11:35:03 +01:00
Sebastiaan van Stijn
fc774f6dc0 docs: api: fix documentation for image push endpoint (API v1.25-v1.47)
The documentation was incorrect and didn't properly document the use of
tags;

- Image push currently only accepts an image-name, not an ID / digest.
- When giving a name, it's expected to be without tag included; when
  including a tag, it is ignored.
- The tag parameter is required when pushing a single image (i.e., it
  does not default to "latest"); omitting the tag parameter will push
  all tags of the given image.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aca38a4218)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 10:49:25 +02:00
Sebastiaan van Stijn
84d78a8ad5 api: swagger: fix documentation for image push endpoint
The documentation was incorrect and didn't properly document the use of
tags;

- Image push currently only accepts an image-name, not an ID / digest.
- When giving a name, it's expected to be without tag included; when
  including a tag, it is ignored.
- The tag parameter is required when pushing a single image (i.e., it
  does not default to "latest"); omitting the tag parameter will push
  all tags of the given image.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07e31e393a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-10 10:49:25 +02:00
Sebastiaan van Stijn
efe9fbd800 Merge pull request #48451 from vvoland/48450-27.x
[27.x backport] gha/bin-image: Also run on branches like `27.x`
2024-09-09 21:28:51 +02:00
Paweł Gronowski
e16197e51d gha/bin-image: Also run on branches like 27.x
We moved to the major release branches with a `.x` suffix and forgot to
adjust this workflow.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 47d5ce0ef8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-09 18:58:17 +02:00
Sebastiaan van Stijn
8b539b8df2 Merge pull request #48433 from vvoland/48432-27.x
[27.x backport] update to go1.22.7
2024-09-06 11:57:40 +02:00
Paweł Gronowski
37bf6e81f9 update to go1.22.7
- https://github.com/golang/go/issues?q=milestone%3AGo1.22.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.6...go1.22.7

These minor releases include 3 security fixes following the security policy:

- go/parser: stack exhaustion in all Parse* functions

    Calling any of the Parse functions on Go source code which contains deeply nested literals can cause a panic due to stack exhaustion.

    This is CVE-2024-34155 and Go issue https://go.dev/issue/69138.

- encoding/gob: stack exhaustion in Decoder.Decode

    Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion.

    This is a follow-up to CVE-2022-30635.

    Thanks to Md Sakib Anwar of The Ohio State University (anwar.40@osu.edu) for reporting this issue.

    This is CVE-2024-34156 and Go issue https://go.dev/issue/69139.

- go/build/constraint: stack exhaustion in Parse

    Calling Parse on a "// +build" build tag line with deeply nested expressions can cause a panic due to stack exhaustion.

    This is CVE-2024-34158 and Go issue https://go.dev/issue/69141.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.23.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a2e14dd8bd)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-05 17:49:47 +02:00
Sebastiaan van Stijn
fefdb1bf24 Merge pull request #48430 from vvoland/48425-27.x
[27.x backport] vendor: github.com/opencontainers/runc v1.1.14
2024-09-04 11:57:15 +02:00
Sebastiaan van Stijn
3a4c589290 vendor: github.com/opencontainers/runc v1.1.14
diff: https://github.com/opencontainers/runc/compare/v1.1.13...v1.1.14

Release Notes:

- Fix CVE-2024-45310 / GHSA-jfvp-7x6p-h2pv, a low-severity attack that allowed maliciously configured containers to create empty files and directories on the host.
- Add support for Go 1.23.
- Revert "allow overriding VERSION value in Makefile" and add EXTRA_VERSION.
- rootfs: consolidate mountpoint creation logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b5ec31f090)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-03 17:52:57 +02:00
Paweł Gronowski
6eeb414adf Merge pull request #48426 from thaJeztah/27.x_backport_bump_runc_1.1.14
[27.x backport] update runc binary to 1.1.14
2024-09-03 14:13:08 +00:00
Sebastiaan van Stijn
54af846a0c Merge pull request #48416 from thaJeztah/27.x_backport_update_go_1.22
[27.x backport] update to go1.22.6
2024-09-03 16:12:38 +02:00
Sebastiaan van Stijn
10cf2d64b6 update runc binary to 1.1.14
Update the runc binary that's used in CI and for the static packages.

diff: https://github.com/opencontainers/runc/compare/v1.1.13...v1.1.14

Release Notes:

- Fix CVE-2024-45310, a low-severity attack that allowed maliciously configured containers to create empty files and directories on the host.
- Add support for Go 1.23.
- Revert "allow overriding VERSION value in Makefile" and add EXTRA_VERSION.
- rootfs: consolidate mountpoint creation logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2189aa2426)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-03 15:13:42 +02:00
Paweł Gronowski
7b86f40aa2 Merge pull request #48423 from vvoland/48422-27.x
[27.x backport] docs/api: add documentation for API v1.47
2024-09-03 10:58:45 +00:00
Paweł Gronowski
84cfcb0893 docs/api: add documentation for API v1.47
API version used by the v27.2.0 release.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4e0d1dfc17)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-03 11:58:32 +02:00
Sebastiaan van Stijn
edeb6ba9fb golangci-lint: temporarily disable G115: integer overflow conversion
it produces many hits, some of which may be false positives, but we need to
look into these, e.g.;

    container/container.go:517:72: G115: integer overflow conversion int -> uint32 (gosec)
        shouldRestart, _, _ := container.RestartManager().ShouldRestart(uint32(container.ExitCode()), container.HasBeenManuallyStopped, container.FinishedAt.Sub(container.StartedAt))
                                                                              ^
    container/view.go:401:25: G115: integer overflow conversion int -> uint16 (gosec)
                        PrivatePort: uint16(p),
                                           ^
    container/view.go:413:25: G115: integer overflow conversion int -> uint16 (gosec)
                        PrivatePort: uint16(p),
                                           ^
    container/view.go:414:25: G115: integer overflow conversion int -> uint16 (gosec)
                        PublicPort:  uint16(h),
                                           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f5108e9c6b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:32 +02:00
Sebastiaan van Stijn
adb0b8536d update golangci-lint to v1.60.2
Update to add go1.23 support

full diff: https://github.com/golangci/golangci-lint/compare/v1.59.1...v1.60.2
Changelog: https://golangci-lint.run/product/changelog/#1602

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b11bb507b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:31 +02:00
Sebastiaan van Stijn
01f7ec0f21 update to go1.22.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3df59c9dcf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:31 +02:00
Cory Snider
442462ad4e hack/make/.binary: set CGO_LDFLAGS=-latomic for arm/v5
cross-compiling for arm/v5 was failing;

    #56 84.12 /usr/bin/arm-linux-gnueabi-clang -marm -o $WORK/b001/exe/a.out -Wl,--export-dynamic-symbol=_cgo_panic -Wl,--export-dynamic-symbol=_cgo_topofstack -Wl,--export-dynamic-symbol=crosscall2 -Qunused-arguments -Wl,--compress-debug-sections=zlib /tmp/go-link-759578347/go.o /tmp/go-link-759578347/000000.o /tmp/go-link-759578347/000001.o /tmp/go-link-759578347/000002.o /tmp/go-link-759578347/000003.o /tmp/go-link-759578347/000004.o /tmp/go-link-759578347/000005.o /tmp/go-link-759578347/000006.o /tmp/go-link-759578347/000007.o /tmp/go-link-759578347/000008.o /tmp/go-link-759578347/000009.o /tmp/go-link-759578347/000010.o /tmp/go-link-759578347/000011.o /tmp/go-link-759578347/000012.o /tmp/go-link-759578347/000013.o /tmp/go-link-759578347/000014.o /tmp/go-link-759578347/000015.o /tmp/go-link-759578347/000016.o /tmp/go-link-759578347/000017.o /tmp/go-link-759578347/000018.o -O2 -g -O2 -g -O2 -g -lpthread -O2 -g -no-pie -static
    #56 84.12 ld.lld: error: undefined symbol: __atomic_load_4
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced 2 more times
    #56 84.12
    #56 84.12 ld.lld: error: undefined symbol: __atomic_store_4
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(x_cgo_notify_runtime_init_done)
    #56 84.12 >>> referenced by gcc_libinit.c
    #56 84.12 >>>               /tmp/go-link-759578347/000009.o:(x_cgo_set_context_function)
    #56 84.12 clang: error: linker command failed with exit code 1 (use -v to see invocation)

From discussion on GitHub;
https://github.com/moby/moby/pull/46982#issuecomment-2206992611

The arm/v5 build failure looks to be due to libatomic not being included
in the link. For reasons probably buried in mailing list archives,
[gcc](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358) and clang don't
bother to implicitly auto-link libatomic. This is not a big deal on many
modern platforms with atomic intrinsics as the compiler generates inline
instruction sequences, avoiding any libcalls into libatomic. ARMv5 is not
one of those platforms: all atomic operations require a libcall.

In theory, adding `CGO_LDFLAGS=-latomic` should fix arm/v5 builds.

While it could be argued that cgo should automatically link against
libatomic in the same way that it automatically links against libpthread,
the Go maintainers would have a valid counter-argument that it should be
the C toolchain's responsibility to link against libatomic automatically,
just like it does with libgcc or compiler-rt.

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 4cd5c2b643)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:31 +02:00
Cory Snider
a3eca63b5b hack/make/.binary: set CCGO_CFLAGS=-Wno-atomic-alignment for arm/v5
cross-compiling for arm/v5 fails on go1.22; a fix is included for this
in go1.23 (https://github.com/golang/go/issues/65290), but for go1.22
we can set the correct option manually.

    1.189 + go build -mod=vendor -modfile=vendor.mod -o /tmp/bundles/binary-daemon/dockerd -tags 'netgo osusergo static_build journald' -ldflags '-w -X "github.com/docker/docker/dockerversion.Version=dev" -X "github.com/docker/docker/dockerversion.GitCommit=HEAD" -X "github.com/docker/docker/dockerversion.BuildTime=2024-08-29T16:59:57.000000000+00:00" -X "github.com/docker/docker/dockerversion.PlatformName=" -X "github.com/docker/docker/dockerversion.ProductName=" -X "github.com/docker/docker/dockerversion.DefaultProductLicense=" -extldflags -static ' -gcflags= github.com/docker/docker/cmd/dockerd
    67.78 # runtime/cgo
    67.78 gcc_libinit.c:44:8: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:47:6: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:49:10: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:69:9: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    67.78 gcc_libinit.c:71:3: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
    78.20 + rm -f /go/src/github.com/docker/docker/go.mod

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit e853c093bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-09-02 18:37:28 +02:00
Sebastiaan van Stijn
7177a05666 Merge pull request #48415 from vvoland/48414-27.x
[27.x backport] c8d/pull: Same error message for non-matching platform
2024-09-02 17:28:47 +02:00
Paweł Gronowski
d036255b69 c8d/pull: Same error message for non-matching platform
Use the same error message as the graphdrivers image store backend.
It's more informative as it also includes the requested platform and
won't break clients checking doing error check with string-matching.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit c7f8557310)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-09-02 13:33:37 +02:00
Tianon Gravi
88288fedff Merge pull request #48389 from corhere/backport-27.x/dockerd-manpage-install
[27.x backport] man: create parent directories in install recipe
2024-08-29 14:35:07 -07:00
Cory Snider
b9f56706c5 man: create parent directories in install recipe
Support the use of `make install` in packaging scripts, where the
$mandir tree might not exist under $DESTDIR.

For portability, create the parent directories using a separate install
command instead of relying on the non-portable `-D` flag.

Set errexit so the install target fails if any install step fails.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 88b118688e)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-29 16:20:24 -04:00
Sebastiaan van Stijn
7c1f8db55e Merge pull request #48402 from vvoland/48399-27.x
[27.x backport] c8d/list: Don't exclude non-container images
2024-08-29 20:53:17 +02:00
Paweł Gronowski
12d4379ad1 c8d/list: Don't exclude non-container images
Before this, the image list would not show images that are not a valid
container image, but could be a valid artifact.

While they're not directly usable by docker, we should still show them
so the user can still discover them and at least be able to delete them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit f4e82e2fb0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-29 14:27:18 +02:00
Sebastiaan van Stijn
801f77bbef Merge pull request #48395 from thaJeztah/27.x_backport_fix_linting_issues
[27.x backport] Fix linting issues in preparation of Go and GolangCI-lint update
2024-08-28 15:48:50 +02:00
Sebastiaan van Stijn
0fbb78c187 daemon: fix non-constant format string in call (govet)
daemon/daemon.go:942:21: printf: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Errorf (govet)
            log.G(ctx).Errorf(err.Error())
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1ad5b5abb2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:11 +02:00
Sebastiaan van Stijn
2bc17a1e5c api/types: fix non-constant format string in call (govet)
api/types/container/hostconfig.go:328:43: printf: non-constant format string in call to fmt.Errorf (govet)
                return &errInvalidParameter{fmt.Errorf(msg)}
                                                       ^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 005b488506)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:11 +02:00
Sebastiaan van Stijn
fe01a5fc30 api/server/router: fix non-constant format string in call (govet)
api/server/router/container/container_routes.go:943:22: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
                                  ^
    api/server/router/image/image_routes.go:144:50: printf: non-constant format string in call to github.com/docker/docker/pkg/streamformatter.FormatStatus (govet)
                output.Write(streamformatter.FormatStatus("", id.String()))
                                                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fd3a53c12)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
bf7ca4dc4b container/stream: fix non-constant format string in call (govet)
container/stream/streams.go:111:21: printf: non-constant format string in call to fmt.Errorf (govet)
            return fmt.Errorf(strings.Join(errors, "\n"))
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a93233b88)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
405a163620 libnetwork/drivers/bridge: fix non-constant format string in call (govet)
libnetwork/drivers/bridge/setup_ip_tables_linux.go:385:23: printf: non-constant format string in call to fmt.Errorf (govet)
                    return fmt.Errorf(msg)
                                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 068c1bf3be)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
812b97d559 volume/testutils: fix non-constant format string in call (govet)
volume/testutils/testutils.go:98:26: printf: non-constant format string in call to fmt.Errorf (govet)
            return nil, fmt.Errorf(opts["error"])
                                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f434cdd14a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
c4eef2a58c builder/dockerfile: parseChownFlag: fix non-constant format string in call (govet)
builder/dockerfile/internals_linux.go:38:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
            return idtools.Identity{}, errors.Wrapf(err, "can't find uid for user "+userStr)
                                                         ^
    builder/dockerfile/internals_linux.go:42:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
            return idtools.Identity{}, errors.Wrapf(err, "can't find gid for group "+grpStr)
                                                         ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 81a1ca0217)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:10 +02:00
Sebastiaan van Stijn
8dec1f216f layer: ignore G602: slice index out of range (gosec)
This looks to be a false positive;

    layer/layer.go:202:47: G602: slice index out of range (gosec)
            return createChainIDFromParent(ChainID(dgsts[0]), dgsts[1:]...)
                                                        ^
    layer/layer.go:205:69: G602: slice index out of range (gosec)
        dgst := digest.FromBytes([]byte(string(parent) + " " + string(dgsts[0])))
                                                                           ^
    layer/layer.go:206:53: G602: slice bounds out of range (gosec)
        return createChainIDFromParent(ChainID(dgst), dgsts[1:]...)
                                                           ^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b56c58a860)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:17:09 +02:00
Sebastiaan van Stijn
068e82bdd2 internal/cleanups: fix non-constant format string in call (govet)
internal/cleanups/composite_test.go:46:9: printf: non-constant format string in call to (*testing.common).Logf (govet)
        t.Logf(err.Error())
               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c308bd3e0e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
8ae439ddaa libnetwork/cnmallocator: fix non-constant format string in call (govet)
libnetwork/cnmallocator/drivers_ipam.go:43:31: printf: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Infof (govet)
            log.G(context.TODO()).Infof("Swarm initialized global default address pool to: " + str.String())
                                        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b60a7047d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
c27f7de662 cmd/dockerd: fix non-constant format string in call (govet)
cmd/dockerd/required.go:17:24: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Errorf (govet)
            return errors.Errorf("\n" + strings.TrimRight(cmd.UsageString(), "\n"))
                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06bfe8bab3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
ec8ff9cbbd libnetwork: fix non-constant format string in call (govet)
libnetwork/controller.go:1054:32: printf: non-constant format string in call to github.com/docker/docker/libnetwork/types.NotFoundErrorf (govet)
                return types.NotFoundErrorf(err.Error())
                                            ^
    libnetwork/controller.go:1073:32: printf: non-constant format string in call to github.com/docker/docker/libnetwork/types.NotFoundErrorf (govet)
                return types.NotFoundErrorf(err.Error())
                                            ^
    libnetwork/sandbox_externalkey_unix.go:113:21: printf: non-constant format string in call to fmt.Errorf (govet)
            return fmt.Errorf(string(buf[0:n]))
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6008c42ca2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
e5bc931f17 integration-cli: fix non-constant format string in call (govet)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b79a4696ee)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:16:28 +02:00
Sebastiaan van Stijn
6c3b1992d4 integration-cli: DockerSwarmSuite: rm redundant Fprintf, handle errors
Also fix some unhandled errors.

    integration-cli/docker_cli_swarm_test.go:697:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"failed to add veth pair: `+err.Error()+`"}`)
                               ^
    integration-cli/docker_cli_swarm_test.go:731:18: printf: non-constant format string in call to fmt.Fprintf (govet)
            fmt.Fprintf(w, `{"LocalDefaultAddressSpace":"`+lAS+`", "GlobalDefaultAddressSpace": "`+gAS+`"}`)
                           ^
    integration-cli/docker_cli_swarm_test.go:742:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"Unknown address space in pool request: `+poolRequest.AddressSpace+`"}`)
                               ^
    integration-cli/docker_cli_swarm_test.go:746:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"PoolID":"`+poolID+`", "Pool":"`+pool+`"}`)
                               ^
    integration-cli/docker_cli_swarm_test.go:763:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Address":"`+gw+`"}`)
                               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bbacbec26)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:56 +02:00
Sebastiaan van Stijn
83171f6b5a integration-cli: DockerNetworkSuite: rm redundant Fprintf, handle errors
Also rename some variables that shadowed imports, and fix some
unhandled errors.

    integration-cli/docker_cli_network_unix_test.go:102:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"failed to add veth pair: `+err.Error()+`"}`)
                               ^
    integration-cli/docker_cli_network_unix_test.go:136:18: printf: non-constant format string in call to fmt.Fprintf (govet)
            fmt.Fprintf(w, `{"LocalDefaultAddressSpace":"`+lAS+`", "GlobalDefaultAddressSpace": "`+gAS+`"}`)
                           ^
    integration-cli/docker_cli_network_unix_test.go:147:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Error":"Unknown address space in pool request: `+poolRequest.AddressSpace+`"}`)
                               ^
    integration-cli/docker_cli_network_unix_test.go:151:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"PoolID":"`+poolID+`", "Pool":"`+pool+`"}`)
                               ^
    integration-cli/docker_cli_network_unix_test.go:168:19: printf: non-constant format string in call to fmt.Fprintf (govet)
                fmt.Fprintf(w, `{"Address":"`+gw+`"}`)
                               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3ca38f0b5e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:55 +02:00
Sebastiaan van Stijn
4cfa21bb92 integration-cli: use erors.New() instead of fmt.Errorf
integration-cli/benchmark_test.go:49:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:62:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:68:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:73:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:78:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:84:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^
    integration-cli/benchmark_test.go:94:27: printf: non-constant format string in call to fmt.Errorf (govet)
                            chErr <- fmt.Errorf(out)
                                                ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2b7a687554)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:55 +02:00
Sebastiaan van Stijn
28bdd819d0 libnetwork: TestDNSOptions: remove redundant skip check
libnetwork/sandbox_dns_unix_test.go:17:13: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
        skip.If(t, runtime.GOOS == "windows", "test only works on linux")
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7b36f8953)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:55 +02:00
Sebastiaan van Stijn
cb2e98cfbf integration-cli: remove redundant platform checks
This condition was added in 0215a62d5b, which
removed pkg/homedir as abstraction, but didn't consider that this test
is currently only ran on Unix.

    integration-cli/docker_cli_run_unix_test.go:254:5: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
        if runtime.GOOS == "windows" {
           ^
    integration-cli/docker_cli_run_unix_test.go:338:5: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
        if runtime.GOOS == "windows" {
           ^

Added a TODO, because this functionality should also be tested on Windows,
probably as part of tests in docker/cli instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bd7835cb6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-28 10:15:53 +02:00
Paweł Gronowski
3ab5c7d003 Merge pull request #48383 from vvoland/48382-27.x
[27.x backport] Dockerfile/vendor: update containerd to v1.7.21
2024-08-27 16:00:14 +02:00
Paweł Gronowski
875e8aeef2 vendor: github.com/containerd/containerd v1.7.21
full diff: https://github.com/containerd/containerd/compare/v1.7.20...v1.7.21

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a88efd7359)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-27 14:07:56 +02:00
Paweł Gronowski
1900e4d8eb Dockerfile: update containerd binary to v1.7.21 (static binaries and CI only)
Update the containerd binary that's used in CI and static binaries

- full diff: https://github.com/containerd/containerd/compare/v1.7.20...v1.7.21
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.21

```markdown changelog
Update containerd (static binaries only) to [v1.7.21](https://github.com/containerd/containerd/releases/tag/v1.7.21)
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit de4fc1c927)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-27 13:43:36 +02:00
Sebastiaan van Stijn
cd7746d30b Merge pull request #48380 from vvoland/48374-27.x
[27.x backport] c8d/pull: Keep the replaced image as dangling
2024-08-27 13:08:51 +02:00
Paweł Gronowski
2a13a384b8 Merge pull request #48376 from vvoland/48293-27.x
[27.x backport] c8d/load: Multi-platform fixes
2024-08-27 11:39:06 +02:00
Sebastiaan van Stijn
9fd71f5d0e Merge pull request #48378 from corhere/backport-27.x/dockerd-manpage
[27.x backport] Move dockerd man page back from docker/cli
2024-08-27 10:43:54 +02:00
Paweł Gronowski
ecd2b6ff09 c8d/image: Add hostPlatformMatcher
Subset of 842c5c584e which only adds the
`hostPlatformMatcher` method.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-27 10:17:50 +02:00
Cory Snider
d5b03423d1 man: support bringing your own go-md2man
Set the GO_MD2MAN make variable to elide building go-md2man from
vendored sources and use the specified command instead.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit edfde78355)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Cory Snider
56c5c23114 man: build dockerd man pages using make
Vendor the go-md2man tool used to generate the man pages so that the
only dependency is a Go toolchain.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 05d7008419)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Grace Choi
77b2eb5734 Removed all mentions of "please" from docs and messages
Signed-off-by: Grace Choi <gracechoi@utexas.edu>
Signed-off-by: Pranjal Rai <pranjalrai@utexas.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b4cee5c3ee)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
David Karlsson
805becdc7e docs: add default-network-opt daemon option
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
(cherry picked from commit f1ec84314d)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
a5828ac742 docs: remove devicemapper
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 23812190c3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Kir Kolyshkin
f7246a0e2c man/dockerd.8: assorted formatting fixes
Mostly, this makes sure that literals (such as true, false, host,
private, examples of options usage etc.) are typeset in bold, except for
filenames, which are typeset in italic.

While at it,
 - remove some default values from synopsis as it should not
   be there;
 - fix man pages references (page name in bold, volume number in
   regular).

This is not a complete fix, but a step in the right direction.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 690d166632)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Kir Kolyshkin
f110e779f6 man/dockerd.8: escape asterisks and underscores
1. Escape asterisks and underscores, that have special meaning in
   Markdown. While most markdown processors are smart enough to
   distinguish whether it's a literal * or _ or a formatting directive,
   escaping makes things more explicit.

2. Fix using wrong level of headings in some dm options (most are ####,
   but some were #####).

3. Do not use sub-heading for examples in some dm options (this is how
   it's done in the rest of the man page).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 374b779dd1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
edbcbf8da7 docs: update dockerd usage output for new proxy-options
Adds documentation for the options that were added in
427c7cc5f8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 386d0c0fbc)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Ashly Mathew
b7cc16b0b0 Fix styling of arguments
Signed-off-by: Ashly Mathew <ashlymathew93@gmail.com>
(cherry picked from commit 54971ac807)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Luis Henrique Mulinari
ecbc4f05bb Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in moby/moby#44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
(cherry picked from commit a8b8f9b288)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
43298ad298 docs: remove documentation about deprecated cluster-store
This removes documentation related to legacy overlay networks using
an external k/v store.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68e9223289)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Anca Iordache
8cc7f26f56 Document --validate daemon option
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
(cherry picked from commit 6c702167bf)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
255eaa6647 Update man-page source MarkDown to work with go-md2man v2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af45195a21)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Akihiro Suda
ee27f4cd7f docs: update for cgroup v2 and rootless
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 562a6d2b13)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Rob Gulewich
c1d3c952e7 docker run: specify cgroup namespace mode with --cgroupns
Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
(cherry picked from commit 7cf2132655)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Lukas Heeren
fc9029a2e2 daemon: document --max-download-attempts option
update docs based on PR 39949

Signed-off-by: Lukas Heeren <lukas-heeren@hotmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1cbcd5d47a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
taiji-tech
115b10a467 Update document links and title.
Signed-off-by: taiji-tech <csuhqg@foxmail.com>
(cherry picked from commit 3cfa74724c)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
selansen
488872bcb4 Allow user to specify default address pools for docker networks This is separate commit for CLI files to address PR 36054
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit 462f38bd8b)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Sebastiaan van Stijn
f623030fac Update docs and completion-scripts for deprecated features
- the `--disable-legacy-registry` daemon flag was removed
- duplicate keys with conflicting values for engine labels
  now produce an error instead of a warning.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 13ff896b38)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Renaud Gaubert
ac7032bff9 Added docs for dockerd
Signed-off-by: Renaud Gaubert <renaud.gaubert@gmail.com>
(cherry picked from commit f3c3b05b50)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Abdur Rehman
161006302f fix a number of minor typos
Fix 19 typos, grammatical errors and duplicated words.

These fixes have minimal impact on the code as these are either in the
doc files or in comments inside the code files.

Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
(cherry picked from commit 20f8455562)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Kir Kolyshkin
a6de17d230 Introduce/document new IPC modes
This builds (and depends) on https://github.com/moby/moby/pull/34087

Version 2:
 - remove --ipc argument validation (it is now done by daemon)
 - add/document 'none' value
 - docs/reference/run.md: add a table with better modes description
 - dockerd(8) typesetting fixes

Version 3:
 - remove ipc mode tests from cli/command/container/opts_test.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c23d4b017a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Aleksa Sarai
6513e80c84 docs: add documentation for dm.libdm_log_level
This is a new option added specifically to allow for debugging of bugs
in Docker's storage drivers or libdm itself.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit 25baee8ab9)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Cory Snider
3d917f3fd6 Restore dockerd man page
Prepare to move the dockerd man page back to this repository from
docker/cli, retaining history.

This partially reverts commit b5579a4ce3.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 7d3f09a9c3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-08-26 16:22:55 -04:00
Paweł Gronowski
e854a5c201 c8d/pull: Replace pointer to interface with interface
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 0afe684685)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:47 +02:00
Paweł Gronowski
ea58dab95e c8d/pull: Keep the replaced image as dangling
With graphdrivers, the old image was still kept as a dangling image.
Keep the same behavior with containerd.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit db40a6132b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:43 +02:00
Paweł Gronowski
0a38589add c8d/load: Only unpack host platform images
When loading a multi-platform image, it's not necessary to unpack all
platforms, especially those which have a completely different OS.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6ebe6a7353)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:10 +02:00
Paweł Gronowski
7c069d3021 c8d/load: Don't fail whole operation if unpack failed
Log the error to the progress output instead.
The image is still loaded into the content store and image service even
if the unpacking failed, so don't error out the whole operation to avoid
missing the load events for other image names loaded from the same
archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 728894b7d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 18:39:08 +02:00
Sebastiaan van Stijn
b27de4ef16 Merge pull request #48369 from vvoland/48367-27.x
[27.x backport] c8d/list: Fix race condition when traversing containers
2024-08-26 13:04:26 +02:00
Paweł Gronowski
5002faebe8 integration/TestAPIImagesListManifests: Check Containers
Verify that the ImageData.Containers contains the ID of the container
using that image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 55f693e7b7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 10:41:22 +02:00
Paweł Gronowski
a15a309832 c8d/list: Update benchmark to also have containers
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 29a2f6d339)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 10:41:21 +02:00
Paweł Gronowski
fd5cede287 c8d/list: Fix race condition when traversing containers
Use a regular for loop instead of ApplyAll which spawns a separate
goroutine for each separate container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a5d75f6d27)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-26 10:41:19 +02:00
Sebastiaan van Stijn
c050bc3739 Merge pull request #48364 from austinvazquez/cherry-pick-3cd28504dec017ef38f1a7abc141a493b9319757-to-27.x
[27.x backport] govulncheck to report known vulnerabilities
2024-08-23 22:18:00 +02:00
Sebastiaan van Stijn
de22458d0f Merge pull request #48363 from austinvazquez/cherry-pick-c4ba1f47187fb77646d906c512084a185036fd51-to-27.x
[27.x backport] Dockerfile: update xx to v1.5.0
2024-08-23 22:17:23 +02:00
CrazyMax
65c4e49aff govulncheck to report known vulnerabilities
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 3cd28504de)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-08-22 13:57:15 +00:00
Sebastiaan van Stijn
7ebb277873 Dockerfile: update xx to v1.5.0
full diff: https://github.com/tonistiigi/xx/compare/v1.4.0...v1.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c4ba1f4718)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-08-22 13:46:45 +00:00
Paweł Gronowski
9942d656ba Merge pull request #48346 from vvoland/47526-27.x
[27.x backport] c8d: Multi-platform image list
2024-08-16 18:47:36 +02:00
Paweł Gronowski
ad5eb875d4 c8d/list: Don't require opts.ContainerCount for manifest containers
The `GET /images/json` requires an optional `container-count` parameter
which set the `Containers` property of in the ImageSummary to a number
of containers using that image.

This was also propagated to the new manifest list property which
includes a list of all the container IDs that are using this specific
image manifest.

Disconnect the `ImageData.Containers` property from this option and
always include it by default without an explicit opt-in.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b93cf37dcd)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:44 +02:00
Paweł Gronowski
3d845e0e8c c8d/list: Add test for total and content size
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 495fab8e66)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:42 +02:00
Paweł Gronowski
3563a707d0 c8d/list: Fix Total size calculation
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 469c2ef3ec)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:41 +02:00
Paweł Gronowski
89757f83ff api/list: Expose manifests
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).

There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 050afe1e1a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:40 +02:00
Paweł Gronowski
bb2fec6425 api: Bump default version to 1.47
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 85e9102dc9)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:39 +02:00
Paweł Gronowski
0f8fcec1d9 swagger: Disable ImageSummary model generation
Our version of go-swagger doesn't handle the `omitempty` correctly for
the new field.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit efb3c50799)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 17:28:37 +02:00
Paweł Gronowski
1a342adda7 Merge pull request #48344 from vvoland/48324-27.x
[27.x backport] fix deprecation comments, and update some godoc
2024-08-16 16:59:01 +02:00
Sebastiaan van Stijn
1ec5e86154 api/types/registry: fix godoc, and add some doc-links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e77e543b58)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 12:56:26 +02:00
Sebastiaan van Stijn
62f32e9a97 plugin: fix deprecation comments
These must have a whitespace before them, otherwise they are ignored.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 218c08b283)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-16 12:56:24 +02:00
Sebastiaan van Stijn
68484b732a Merge pull request #48341 from tonistiigi/v27-update-buildkit-v0.15.2
[27.x] vendor: update buildkit to v0.15.2
2024-08-15 21:50:09 +02:00
Tonis Tiigi
830c76c6f2 vendor: update buildkit to v0.15.2
Also brings in fix for moby/buildkit#5242

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit c459986399)
2024-08-15 18:42:29 +03:00
Sebastiaan van Stijn
8f969bf61c Merge pull request #48332 from vvoland/48281-27.x
[27.x backport] Migrate per-endpoint sysctls until 28.0.0
2024-08-15 10:39:38 +02:00
Sebastiaan van Stijn
290663ede5 Merge pull request #48333 from vvoland/48081-27.x
[27.x backport] do another run of gofumpt
2024-08-15 10:16:52 +02:00
Sebastiaan van Stijn
354bf75675 libcontainerd: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 080a8e1b6b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:42 +02:00
Sebastiaan van Stijn
4ab7d90669 pkg/plugins: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 56fa45773f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:41 +02:00
Sebastiaan van Stijn
c11b2d9c7d pkg/archive: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0e2d40c24a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:40 +02:00
Sebastiaan van Stijn
ccdc79d55a libnetwork: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 84e43da752)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:39 +02:00
Sebastiaan van Stijn
35b1a30028 layer: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bb1b766ddb)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:37 +02:00
Sebastiaan van Stijn
9f63aa7435 internal: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07469b4509)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:36 +02:00
Sebastiaan van Stijn
4d16ac993e integration: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e50a96a78)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:34 +02:00
Sebastiaan van Stijn
6d5266a650 integration-cli: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c3ac7fee26)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:32 +02:00
Sebastiaan van Stijn
4084dac566 daemon: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e765dd90ee)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:30 +02:00
Sebastiaan van Stijn
c36ab4c2ca daemon/containerd: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 181101c4a8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:29 +02:00
Sebastiaan van Stijn
904867593b daemon/config: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 46b0102da4)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:11 +02:00
Sebastiaan van Stijn
72876770d0 builder: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a89963f1e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:10 +02:00
Sebastiaan van Stijn
e8109ee4da api/types: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8768145519)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:08 +02:00
Sebastiaan van Stijn
ed65e1224e api/server: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92346bcec6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:33:06 +02:00
Rob Murray
d54aff9312 API 1.46: end per-interface sysctl migration in major release
Rather than in API 1.47.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a86a9e3aa4)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:29:23 +02:00
Rob Murray
8f12906274 Migrate per-endpoint sysctls until 28.0.0
Commit 0071832226 introduced
per-endpoint sysctls, and migration to them from the top-level
'--sysctl' option.

The migration was intended to be short-term, disabled in the
next major release and code was added to check for the next
API version. But now, the API version will be bumped in a
minor release - this breaking change needs to wait until the
next major release, and we don't yet know the API version
number for that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 17adc1478b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-14 19:29:19 +02:00
Albin Kerouanton
5955778fe3 Merge pull request #48326 from robmry/backport-27.2/64bit_iprange_fix
[27.2 backport] Allow 64-bit --ip-range
2024-08-14 09:24:04 +02:00
Rob Murray
c53be2f3d5 Allow --ip-range ending on a 64-bit boundary
When defaultipam.newPoolData is asked for a pool of 64-bits
or more, it ends up with an overflowed u64 - so, it just
subtracts one to get a nearly-big-enough range (for a 64-bit
subnet).

When defaultipam.getAddress is called with an ipr (sub-pool
range), the range it calls bitmask.SetAnyInRange with is
exclusive of end. So, its end param can't be MaxUint64,
because that's the max value for the top end of the range
and, when checking the range, SetAnyInRange fails.

When fixed-cidr-v6 behaves more like fixed-cidr, it will ask
for a 64-bit range if that's what fixed-cidr-v6 needs. So,
it hits the bug when allocating an address for, for example:

  docker network create --ipv6 --subnet fddd::/64 --ip-range fddd::/64 b46

The additional check for "ipr == base" avoids the issue in
this case, by ignoring the ipr/sub-pool range if ipr is the
same as the pool itself (not really a sub-pool).

But, it still fails when ipr!=base. For example:

  docker network create --ipv6 --subnet fddd::/56 --ip-range fddd::/64 b46

So, also subtract one from 'end' if it's going to hit the max
value allowed by the Bitmap.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 496b457ad8)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-08-13 15:40:48 +01:00
Sebastiaan van Stijn
f9522e5e96 Merge pull request #48315 from vvoland/48169-27.x
[27.x backport] rm regexp use
2024-08-10 15:59:46 +02:00
Sebastiaan van Stijn
a037b7250c Merge pull request #48314 from vvoland/48275-27.x
[27.x backport] api/swagger: fix x-nullable for SystemInfo.Containerd (api v1.46)
2024-08-09 17:18:14 +02:00
Kir Kolyshkin
fc0150b962 daemon/containerd: rm use of regexp
Replace the regexp check with a function.

Keep the use of regexp.QuoteMeta.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 508939821b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:08:05 +02:00
Kir Kolyshkin
73c01d0b6a image/v1: rm regexp use
Replace the regexp checking ID validity with a for loop.

The benefits are:
 - faster (up to 10x faster with less allocations);
 - no init overhead to compile the regexp.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b66d4b567a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:08:04 +02:00
Kir Kolyshkin
c93fe4a27d layer: rm regexp use
Replace the regexp checking ID validity with a function. The benefits
are:

 - function is faster (up to 10x faster with less allocations);
 - no init overhead to compile the regexp;

Add a test case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1c0dc8a94f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:08:02 +02:00
Sebastiaan van Stijn
31459c8268 docs/api: swagger: fix x-nullable for SystemInfo.Containerd (api v1.46)
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.

Rendering the docs (`make swagger-docs`) showed a warning:

> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7dec1c67a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:00:24 +02:00
Sebastiaan van Stijn
35d430c62e api/swagger: fix x-nullable for SystemInfo.Containerd
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.

Rendering the docs (`make swagger-docs`) showed a warning:

> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 66b5b8bfa8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-09 10:00:21 +02:00
Sebastiaan van Stijn
f5fa0908ef Merge pull request #48308 from thaJeztah/27.x_backport_migrate_userns
[27.x backport] migrate to github.com/moby/sys/userns
2024-08-08 12:34:48 +02:00
Sebastiaan van Stijn
a17f5d4f10 Merge pull request #48294 from austinvazquez/cherry-pick-2b5ffa0b63c76e8bb4ebb253d7e4db5c7af918c0-to-27.x
[27.x backport] gha: set permissions to read-only by default
2024-08-08 11:59:30 +02:00
Sebastiaan van Stijn
80a59c2f1a migrate to github.com/moby/sys/userns
Commit 2ce811e632 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b0ef10a9a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 11:04:52 +02:00
Tianon Gravi
95db7055cc Merge pull request #48301 from vvoland/48300-27.x
[27.x backport] update to go1.21.13
2024-08-07 16:16:41 -07:00
Paweł Gronowski
e7fe276c00 update to go1.21.13
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.13+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.12...go1.21.13

go1.21.13 (released 2024-08-06) includes fixes to the go command, the
covdata command, and the bytes package. See the Go 1.21.13 milestone on
our issue tracker for details.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b24c2e95e5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-08-07 11:45:54 +02:00
Sebastiaan van Stijn
e8cd19e810 gha: set permissions to read-only by default
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2b5ffa0b63)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-08-06 15:55:59 +00:00
Paweł Gronowski
45d37a0ca9 Merge pull request #48246 from vvoland/48239-27.x
[27.1 backport] vendor: update buildkit to v0.15.1
2024-07-26 18:11:02 +02:00
CrazyMax
e0c52e0ba6 vendor: update buildkit to v0.15.1
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 1baf8f9e60)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-26 16:29:21 +02:00
Paweł Gronowski
b9be986e35 Merge pull request #48245 from thaJeztah/27.1_backport_buildkit_fix_grpc_control_api_sizes
[27.1 backport] api/server/router/grpc: NewRouter: set correct MaxRecvMsgSize, MaxSendMsgSize
2024-07-26 16:10:33 +02:00
Sebastiaan van Stijn
efb67b16b0 api/server/router/grpc: NewRouter: set correct MaxRecvMsgSize, MaxSendMsgSize
[buildkit@29b4b1a537][1] applied changes to `buildkitd` to set the correct
defaults, which should be 16MB, but used the library defaults. Without that
change, builds using large Dockerfiles would fail with a `ResourceExhausted`
error;

    => [internal] load build definition from Dockerfile
     => => transferring dockerfile: 896.44kB
    ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (44865299 vs. 16777216)

However those changes were applied to the `buildkitd` code, which is the
daemon when running BuildKit standalone (or in a container through the
`container` driver). When running a build with the BuildKit builder compiled
into the Docker Engine, that code is not used, so the BuildKit changes did
not fix the issue.

This patch applies the same changes as were made in [buildkit@29b4b1a537][1]
to the gRPC endpoint provided by the dockerd daemon.

[1]: 29b4b1a537

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cdbfae1d3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-26 14:28:27 +02:00
Paweł Gronowski
741e23b913 Merge pull request #48238 from thaJeztah/27.1_backport_migrate_userns
[27.1 backport] migrate to github.com/moby/sys/user/userns
2024-07-26 10:05:53 +02:00
Sebastiaan van Stijn
f96e26f68d migrate to github.com/moby/sys/user/userns
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2ce811e632)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 14:15:19 +02:00
Sebastiaan van Stijn
78b59867f2 vendor: github.com/moby/sys/user v0.2.0
full diff: https://github.com/moby/sys/compare/user/v0.1.0...user/v0.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 91dfc326cf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 14:15:14 +02:00
Akihiro Suda
7d861e889c Merge pull request #48235 from thaJeztah/27.1_backport_vendor_flock
[27.1 backport] vendor: github.com/gofrs/flock v0.12.1
2024-07-25 17:49:46 +09:00
Sebastiaan van Stijn
17e1108324 Merge pull request #48228 from vvoland/47789-27.0
[27.0 backport] hack: explicitly control enabling the journald logging driver
2024-07-25 01:39:49 +02:00
Sebastiaan van Stijn
018137b01a vendor: github.com/gofrs/flock v0.12.1
- fix: missing read-write flag in reopenFDOnError
  fixes a regression that could result in a `ERROR: bad file descriptor`.

b659e1e00a
introduced a regression where `f.flag` would not be in read-write mode
[1]  but read-only [2] which breaks people using NFS protocol.

[1]: b659e1e00a (diff-87c2c4fe0fb43f4b38b4bee45c1b54cfb694c61e311f93b369caa44f6c1323ffR192)
[2]: b659e1e00a (diff-22145325dded38eb5288ed3321a113d8260ccc70747ee04d4551bfd2fba975fdR69)

full diff: https://github.com/gofrs/flock/compare/v0.12.0...v0.12.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e2ccf8046)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 00:45:40 +02:00
Sebastiaan van Stijn
650e06ac75 vendor: golang.org/x/sys v0.22.0
full diff: https://github.com/golang/sys/compare/v0.21.0...v0.22.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 077b32ac4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-25 00:45:40 +02:00
Akihiro Suda
7f5494dc97 Merge pull request #48233 from AkihiroSuda/cherrypick-48216-27
[27.x backport] dockerd-rootless-setuptool.sh: move RootlessKit smoke test
2024-07-25 02:53:28 +09:00
Akihiro Suda
bfe5339c7e dockerd-rootless-setuptool.sh: move RootlessKit smoke test
`dockerd-rootless-setuptool.sh check` now skips the smoke test for
running RootlessKit.

Fix docker/docker-install issue 417

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e2237240f5)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-07-25 00:37:02 +09:00
Paweł Gronowski
a4046c4ca0 Merge pull request #48221 from thaJeztah/27.1_backport_readme_commercial_support
[27.1 backport] README: replace obsolete Docker EE mention
2024-07-24 11:48:07 +02:00
William Hubbs
99471ac2fe hack: explicitly control enabling the journald logging driver
Without this, the dependency on systemd is said to be "automagic", which
can lead to breakage, for example, if a binary package of docker is
built on a system that has systemd installed then installed on a system
that does not have systemd installed.

for example: https://bugs.gentoo.org/914076

Signed-off-by: William Hubbs <w.d.hubbs@gmail.com>
(cherry picked from commit 499c842c52)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-24 10:56:45 +02:00
Cory Snider
b9b43b3bdf README: replace obsolete Docker EE mention
Docker EE is no more. Point users looking for commercial support at the
currently-maintained commercial products based on the Moby project:
Docker Desktop and Mirantis Container Runtime.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit b37c983d31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-23 22:05:49 +02:00
Sebastiaan van Stijn
cc13f95251 Merge commit from fork
[27.0] AuthZ plugin security fixes
2024-07-23 21:36:28 +02:00
Sebastiaan van Stijn
a21b1a2d12 Merge pull request #48196 from thaJeztah/27.1_backport_vendor_containerd_1.7.20
[27.1 backport] vendor: github.com/containerd/containerd v1.7.20
2024-07-19 16:42:08 +02:00
Sebastiaan van Stijn
1bc907c97c vendor: github.com/containerd/containerd v1.7.20
full diff: https://github.com/containerd/containerd/compare/v1.7.19...v1.7.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 55a5f3fcaa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 15:40:23 +02:00
Sebastiaan van Stijn
4bb4575ffb Merge pull request #48191 from thaJeztah/27.1_backport_update_containerd_binary_1.7.20
[27.1 backport] update containerd binary to v1.7.20
2024-07-19 13:44:08 +02:00
Sebastiaan van Stijn
df7f275db6 Merge pull request #48195 from thaJeztah/27.1_backport_fix_pr_title_check
[27.1 backport] gha: check-pr-branch: fix branch check regression
2024-07-19 12:34:25 +02:00
Sebastiaan van Stijn
1c0885d60d gha: check-pr-branch: fix branch check regression
This check was updated in f460110ef5, but
introduced some bugs;

- the regular expressions were meant to define a capturing group, but
  the braces (`(`, `)`) were escaped (they previously were used by
  `sed`, which requires different escaping), so no value was captured.
- the check itself was not updated to use the resulting `$target_branch`
  env-var, so was comparing against the `$GITHUB_BASE_REF` (which is
  the branch name before stripping minor versions).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e0b98a3222)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 12:29:22 +02:00
Paweł Gronowski
fb3ec9fc96 Merge pull request #48187 from thaJeztah/27.1_backport_bump_buildx_compose
[27.0 backport] Dockerfile: update buildx to v0.16.1, compose to v2.29.0
2024-07-19 10:05:14 +02:00
Sebastiaan van Stijn
ed83a9e3a1 update containerd binary to v1.7.20
Update the containerd binary that's used in CI and for the static packages.

release notes: https://github.com/containerd/containerd/releases/tag/v1.7.20
full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fbbda057ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 02:35:39 +02:00
Sebastiaan van Stijn
71b59bf442 Merge pull request #48178 from thaJeztah/27.1_backport_relax_pr_check
[27.1 backport] gha: check-pr-branch: verify major version only
2024-07-19 02:26:06 +02:00
Sebastiaan van Stijn
f8f926b719 Merge pull request #48185 from thaJeztah/27.1_backport_internalize_pkg_directory
[27.0 backport] deprecate packages that are to be removed in the next release
2024-07-19 02:06:21 +02:00
Sebastiaan van Stijn
422ef48c2f gha: check-pr-branch: verify major version only
We'll be using release branches for minor version updates, so instead
of (e.g.) a 27.0 branch, we'll be using 27.x and continue using the
branch for minor version updates.

This patch changes the validation step to only compare against the
major version.

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f460110ef5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 01:21:30 +02:00
Sebastiaan van Stijn
c9d37a9198 [27.1] pkg/rootless/specconv: deprecate, and add temporary aliases
There's no (known) external consumers of this, but let's add a
deprecation for the 27.1 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:23 +02:00
Sebastiaan van Stijn
1f16a44b3d pkg/rootless/specconv: move to internal
This package is only used by the daemon, so move it to the internal
rootless package instead.

Note that technically this could be in daemon/internal, but as there's
already an existing internal/rootless package (which needs to be in the
top-level internal package because it's also used by /plugin), I'm moving
it there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit efdaca2792)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:23 +02:00
Sebastiaan van Stijn
c8f1317585 pkg/directory: deprecate, and move to internal
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3a3bb1cb50)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:23 +02:00
Sebastiaan van Stijn
68587c38fe pkg/directory: fix comment, and remove import comments
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 80900bdbcd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:23 +02:00
Sebastiaan van Stijn
d1ea2b1fec [27.1] pkg/containerfs: deprecate, and add temporary aliases
There's no (known) external consumers of this, but let's add a
deprecation for the 27.1 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:23 +02:00
Sebastiaan van Stijn
31c1b7dc17 pkg/containerfs: move to internal
The only external consumer are the `graphdriver` and `graphdriver/shim`
packages in github.com/docker/go-plugins-helpers, which depended on
[ContainerFS][1], which was removed in 9ce2b30b81.

graphdriver-plugins were deprecated in 6da604aa6a,
and support for them removed in 555dac5e14,
so removing this should not be an issue.

Ideally this package would've been moved inside `daemon/internal`, but it's used
by the `daemon` (cleanupContainer), `plugin` package, and by `graphdrivers`,
so needs to be in the top-level `internal/` package.

[1]: 6eecb7beb6/graphdriver/api.go (L218)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2970e5358)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:22 +02:00
Sebastiaan van Stijn
6231ea4a34 pkg/containerfs: cleanup GoDoc, and make Windows a proper wrapper
- Improve some GoDoc to use docs links
- Change the Windows stub to an actual wrapper function, as we don't
  want it to be updateable, and it currently shows as "variable" on
  pkg.go.dev, which is confusing.
- Remove "import" comments in preparation of moving this package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a3e6ce95c4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:22 +02:00
Sebastiaan van Stijn
dc33eb81d8 pkg/containerfs: remove CleanScopedPath and make it internal
The container package is the only consumer of this function in our code
and there's no known external users;
https://grep.app/search?q=.CleanScopedPath%28&filter[lang][0]=Go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e2ae6907c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:23:22 +02:00
Sebastiaan van Stijn
51433d65c0 Merge pull request #48184 from thaJeztah/27.1_backport_rm_deprecated_executiondriver
[27.0 backport] api/types/system: remove Info.ExecutionDriver
2024-07-19 00:11:22 +02:00
Sebastiaan van Stijn
f3bd9da62a Merge pull request #48183 from thaJeztah/27.1_backport_bump_google_deps
[27.0 backport] vendor: cloud.google.com/go/logging v1.9.0
2024-07-19 00:10:49 +02:00
Sebastiaan van Stijn
bc6ae42031 Dockerfile: update compose to v2.29.0
This is the version used in the dev-container, and for testing.

release notes: https://github.com/docker/compose/releases/tag/v2.29.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a42f7fd717)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:10:16 +02:00
Sebastiaan van Stijn
af8866f324 Dockerfile: update buildx to v0.16.1
This is the version used in the dev-container, and for testing.

release notes:
https://github.com/docker/buildx/releases/tag/v0.16.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 97b51c6b72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-19 00:10:15 +02:00
Sebastiaan van Stijn
5e4ddd81a2 api/types/system: remove Info.ExecutionDriver
The execution-driver was replaced with containerd since docker 1.11 (API
v1.23) in 9c4570a958, after which the value
was no longer set. The field was left in the type definition.
Commit 1fb1136fec removed its use from the
CLI and [docker/engine-api@39c7d7e] removed it from the API type, followed
by an update to the API docs in 3c6ef4c29d.

Changes to the API types were not pulled into the engine until v1.13, and
probably because of that gated it on API version < 1.25 instead of < 1.24
(see 6d98e344c7); setting a "not supported"
value for older versions.

Based on the above; this field was deprecated in API v1.23, and empty
since then. Given that the minimum API version supported by the engine
is not v1.24, we can safely remove it.

[docker/engine-api@39c7d7e]: 39c7d7ec19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e4d792a06d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-18 23:08:18 +02:00
Sebastiaan van Stijn
147eaae6b7 Merge pull request #48181 from vvoland/48156-27.0
[27.0 backport] Fix API version in TestSetInterfaceSysctl
2024-07-18 22:58:35 +02:00
Sebastiaan van Stijn
c7e4d181a1 vendor: cloud.google.com/go/logging v1.9.0
removes dependency on appengine, among others

full diff: https://github.com/googleapis/google-cloud-go/compare/logging/v1.0.1...logging/v1.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fa71a4cfc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-18 22:32:11 +02:00
Sebastiaan van Stijn
3d7e824bc2 vendor: golang.org/x/oauth2 v0.21.0
removes dependency on appengine, among others

full diff: https://github.com/golang/oauth2/compare/v0.11.0...v0.21.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit eafad2cb86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-18 22:32:11 +02:00
Sebastiaan van Stijn
d66b76d2e6 vendor: cloud.google.com/go/compute/metadata v0.3.0
full diff: https://github.com/googleapis/google-cloud-go/compare/compute/metadata/v0.2.3...compute/metadata/v0.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b782b8ff7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-18 22:32:11 +02:00
Sebastiaan van Stijn
0e678a85d7 Merge pull request #48182 from vvoland/48078-27.0
[27.0 backport] c8d/build: Log `image tag` event when image was built with Buildkit
2024-07-18 21:47:07 +02:00
Brian Goff
3db1544179 Merge pull request #48175 from thaJeztah/27.1_backports
[27.0 backport] vendor: update buildkit to v0.15.0
2024-07-18 19:46:04 +00:00
Paweł Gronowski
03dc388f63 c8d/build: Log image tag event when image was built with Buildkit
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 53bc396ef4)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-18 17:46:34 +02:00
Paweł Gronowski
5ee23b6050 builder-next: Add ImageNamedByBuildkit callback
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.

Add a callback that is called by the exporter wrapper.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 1506bbcfe8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-18 17:46:32 +02:00
Paweł Gronowski
53c521a6b2 builder-next: Don't return error from exported callback
This is only a callback that notifies about event so there is no way to
react to the error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit bce76d486e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-18 17:46:29 +02:00
Rob Murray
eccccd7577 Fix API version in TestSetInterfaceSysctl
The test checks that it's possible to set a per-interface sysctl
using '--sysctl' - but, after API v1.46, it's not (and driver option
'com.docker.network.endpoint.sysctls' must be used instead).

Test added in commit fde80fe2
Per-interface sysctls added, with API changes, in commit 00718322

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit f649fd0c97)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-18 17:34:58 +02:00
Sebastiaan van Stijn
d9e3d1b815 update containerd binary to v1.7.19
Update the containerd binary that's used in CI and for the static packages.

- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.19
- full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19

Welcome to the v1.7.19 release of containerd!

The nineteenth patch release for containerd 1.7 contains various updates and
splits the main module from the api module in preparation for the same change
in containerd 2.0. Splitting the modules will allow 1.7 and 2.x to both exist
as transitive dependencies without running into API registration errors.
Projects should use this version as the minimum 1.7 version in preparing to
use containerd 2.0 or to be imported alongside it.

Highlights

- Fix support for OTLP config
- Add API go module
- Remove overlayfs volatile option on temp mounts
- Update runc binary to v1.1.13
- Migrate platforms package to github.com/containerd/platforms
- Migrate reference/docker package to github.com/distribution/reference

Container Runtime Interface (CRI)

- Fix panic in NRI from nil CRI reference
- Fix Windows HPC working directory

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 398e15b7de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 23:12:42 +02:00
Tonis Tiigi
b91e20cc2e vendor: update buildkit to v0.15.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 68bd630830)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 23:00:07 +02:00
Tonis Tiigi
505be6557b vendor: update buildkit to v0.15.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 89781912c1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:59:33 +02:00
Tonis Tiigi
b1613dc2a1 vendor: update buildkit to v0.15.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 1787c364e0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:58:28 +02:00
Sebastiaan van Stijn
52f6163746 vendor: golang.org/x/net v0.25.0
full diff: https://github.com/golang/net/compare/v0.24.0...v0.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 066b7fa83c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:57:41 +02:00
Sebastiaan van Stijn
c70e404e9e vendor: golang.org/x/crypto v0.23.0
full diff: https://github.com/golang/crypto/compare/v0.22.0...v0.23.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7721408db7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:57:34 +02:00
Sebastiaan van Stijn
d7a3f01421 vendor: golang.org/x/text v0.15.0
no changes in vendored files

full diff: https://github.com/golang/text/compare/v0.14.0...v0.15.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f43436e6b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:57:27 +02:00
Sebastiaan van Stijn
0f2f9e0049 vendor: golang.org/x/sys v0.21.0
full diff: https://github.com/golang/sys/compare/v0.19.0...v0.21.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 342ce515ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:57:17 +02:00
Sebastiaan van Stijn
45a1c34202 vendor: github.com/klauspost/compress v1.17.9
full diff: https://github.com/klauspost/compress/compare/v1.17.4...v1.17.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2e58a29023)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:57:10 +02:00
Sebastiaan van Stijn
7b31435cf8 Migrate to github.com/containerd/platforms module
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.

The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d0aa3eaccf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:52:58 +02:00
Sebastiaan van Stijn
99df4fee0b vendor: github.com/containerd/containerd v1.7.19
Highlights

- Fix support for OTLP config
- Add API go module
- Remove overlayfs volatile option on temp mounts
- Update runc binary to v1.1.13
- Migrate platforms package to github.com/containerd/platforms
- Migrate reference/docker package to github.com/distribution/reference

Container Runtime Interface (CRI)

- Fix panic in NRI from nil CRI reference
- Fix Windows HPC working directory

full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8983957ac5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:52:50 +02:00
Sebastiaan van Stijn
9f08d1e357 vendor: github.com/microsoft/hcsshim v0.11.7
- Fix process handle leak when launching a job container
- Add EndpointState attribute to the HNSEndpoint struct to support
  hyperv containers for k8s
- Add support for loadbalancer policy update in hns
- Changes for checking the global version for modify policy version support
- OutBoundNATPolicy Schema changes (add MaxPortPoolUsage to OutboundNatPolicySetting)

full diff: https://github.com/microsoft/hcsshim/compare/v0.11.5...v0.11.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a2fe103f0d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-17 22:52:28 +02:00
Jameson Hyde
d1bbb61844 If url includes scheme, urlPath will drop hostname, which would not match the auth check
Signed-off-by: Jameson Hyde <jameson.hyde@docker.com>
(cherry picked from commit 754fb8d9d03895ae3ab60d2ad778152b0d835206)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5282cb25d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-15 18:46:29 +02:00
Jameson Hyde
0835eaa5a1 Authz plugin security fixes for 0-length content and path validation
Signed-off-by: Jameson Hyde <jameson.hyde@docker.com>

fix comments

(cherry picked from commit 9659c3a52bac57e615b5fb49b0652baca448643e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2ac8a479c5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-15 18:46:27 +02:00
Akihiro Suda
73ce798d3b Merge pull request #48155 from vvoland/v27.0-48154
[27.0 backport] docs/api: Add missing `
2024-07-11 23:22:30 +09:00
Paweł Gronowski
b039de78d7 docs/api: Add missing `
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 7f04a603f6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-11 14:28:18 +02:00
Sebastiaan van Stijn
7fa33a539a Merge pull request #48141 from AkihiroSuda/cherrypick-48134-27
[27.0 backport] rootless: add `Requires=dbus.socket`
2024-07-08 15:05:47 -05:00
Akihiro Suda
7d99ebe418 rootless: add Requires=dbus.socket
On a cgroup v2 host with systemd, dbus is needed to avoid the following error:
```
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed
: unable to start container process: unable to apply cgroup configuration: unable to start unit "docker-170a4183e351e69835b82cc3134b97c8cbb0e6d3a6
16d5a0fb0ea473075062ad.scope" (properties [{Name:Description Value:"libcontainer container 170a4183e351e69835b82cc3134b97c8cbb0e6d3a616d5a0fb0ea47
3075062ad"} {Name:Slice Value:"user.slice"} {Name:Delegate Value:true} {Name:PIDs Value:@au [2872]} {Name:MemoryAccounting Value:true} {Name:CPUAc
counting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Value:false}]): Interactive authen
tication required.: unknown.
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 206445fa4f)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-07-08 17:41:41 +09:00
Akihiro Suda
e7e0428218 Merge pull request #48122 from vvoland/v27.0-48120
[27.0 backport] update to go1.21.12
2024-07-04 10:57:45 +09:00
Paweł Gronowski
540b29c0c6 update to go1.21.12
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.12+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.11...go1.21.12

These minor releases include 1 security fixes following the security policy:

net/http: denial of service due to improper 100-continue handling

The net/http HTTP/1.1 client mishandled the case where a server responds to a request with an "Expect: 100-continue" header with a non-informational (200 or higher) status. This mishandling could leave a client connection in an invalid state, where the next request sent on the connection will fail.

An attacker sending a request to a net/http/httputil.ReverseProxy proxy can exploit this mishandling to cause a denial of service by sending "Expect: 100-continue" requests which elicit a non-informational response from the backend. Each such request leaves the proxy with an invalid connection, and causes one subsequent request using that connection to fail.

Thanks to Geoff Franks for reporting this issue.

This is CVE-2024-24791 and Go issue https://go.dev/issue/67555.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.21.12

**- Description for the changelog**

```markdown changelog
Update Go runtime to 1.21.12
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4d1d7c3ebe)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-03 14:20:05 +02:00
Sebastiaan van Stijn
662f78c0b1 Merge pull request #48090 from thaJeztah/27.0_backport_48067_fix_specific_ipv6_portmap_proxy_to_ipv4
[27.0 backport] Fix incorrect validation of port mapping
2024-06-28 23:16:49 +02:00
Sebastiaan van Stijn
b86d9bdaf3 Merge pull request #48086 from thaJeztah/27.0_backport_fix_rootless_pull
[27.0 backport] daemon/graphdriver/overlay2: set TarOptions.InUserNS for native differ (fix "failed to Lchown "/dev/console")
2024-06-28 22:40:07 +02:00
Sebastiaan van Stijn
0dbc3ac59e Merge pull request #48087 from thaJeztah/27.0_backport_gofmt
[27.0 backport] fix some gofmt issues reported by goreportcard
2024-06-28 21:11:01 +02:00
Rob Murray
276a648ec3 Fix incorrect validation of port mapping
Regression introduced in 01eecb6.

A port mapping from a specific IPv6 host address can be used
by a container on an IPv4-only network, docker-proxy makes the
connection.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit dfbcddb9f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 21:07:29 +02:00
Sebastiaan van Stijn
22aa07b28f Merge pull request #48089 from robmry/backport-27.0/48069_fix_overlapping_subnets
[27.0 backport] Fix duplicate subnet allocations
2024-06-28 18:26:59 +02:00
Rob Murray
23b8b023dd Fix duplicate subnet allocations
Keep allocated subnets in-order, so that they're not mistakenly
reallocated due to a gap in the list where misplaced subnets should
have been.

Introduced in 9d288b5.

The iterator over allocated subnets was incremented too early, this
change moves it past three clauses in addrSpace.allocatePredefinedPool().
The three new unit tests correspond to a separate failure caused by
incrementing before each of them.

(cherry picked from commit 4de54ee14c)
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-28 16:24:47 +01:00
Sebastiaan van Stijn
bf222d635b fix some gofmt issues reported by goreportcard
https://goreportcard.com/report/github.com/docker/docker

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6ada1cff02)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 16:48:01 +02:00
Sebastiaan van Stijn
f8231b52d3 daemon/graphdriver/overlay2: set TarOptions.InUserNS for native differ
Commits b2fd67de77 (and the follow-up commit
f6b80253b8) updated doesSupportNativeDiff to
detect whether the host can support native overlay diffing with userns
enabled.

As a result, [useNaiveDiff] would now return "false" in cases where it
previously would return "true" (and thus skip). However, [overlay2],
unlike [fuse-overlay] did not take user namespaces into account, when
using the native differ, and it therefore did not set the InUserNS option
in TarOptions.

As a result pkg/archive.createTarFile would attempt tocreate [device-nodes]
through [handleTarTypeBlockCharFifo] which would fail, but the resulting
error `EPERM` would be discarded, and `createTarFile` would not return
early, therefor attempting to [os.LChown] the missing file, ultimately
resulting in an error:

    failed to Lchown "/dev/console" for UID 0, GID 0: lchown /dev/console: no such file or directory

This patch fixes the missing option in overlay.

[useNaiveDiff]: 47eebd718f/daemon/graphdriver/overlay2/overlay.go (L248-L256)
[overlay2]: 47eebd718f/daemon/graphdriver/overlay2/overlay.go (L684-L689)
[fuse-overlay]: 47eebd718f/daemon/graphdriver/fuse-overlayfs/fuseoverlayfs.go (L456-L462)
[device-nodes]: ff1e2c0de7/pkg/archive/archive.go (L713-L720)
[handleTarTypeBlockCharFifo]: 47eebd718f/pkg/archive/archive_unix.go (L110-L114)
[os.LChown]: ff1e2c0de7/pkg/archive/archive.go (L762-L773)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6521057bb2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 16:31:54 +02:00
Sebastiaan van Stijn
b951474404 pkg/archive: createTarFile: consistently use the same value for userns
createTarFile accepts a opts (TarOptions) argument to specify whether
userns is enabled; whe should consider always detecting locally, but
at least make sure we're consistently working with the same value within
this function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 969993a729)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 16:31:53 +02:00
Sebastiaan van Stijn
c5794e23ec pkg/archive: handleTarTypeBlockCharFifo: don't discard EPERM errors
This function was discarding EPERM errors if it detected that userns was
enabled; move such checks to the caller-site, so that they can decide
how to handle the error (which, in case of userns may be to log and ignore).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 379ce56cd8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 16:31:53 +02:00
Sebastiaan van Stijn
02e24483be pkg/archive: getWhiteoutConverter: don't error with userns enabled
Since 838047a1f5, the overlayWhiteoutConverter
is supported with userns enabled, so we no longer need this check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af85e47343)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 16:31:53 +02:00
Sebastiaan van Stijn
b70040a8fc Merge pull request #48074 from vvoland/v27.0-48073
[27.0 backport] Dockerfile: update compose to v2.28.1, update cli to v27.0.2
2024-06-27 18:00:44 +02:00
Paweł Gronowski
838330bac3 Dockerfile: update docker CLI to v27.0.2
Update the Docker CLI used in the dev-container

full diff: https://github.com/docker/cli/compare/v26.1.0...v27.0.2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3928165cf7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-27 14:55:28 +02:00
Paweł Gronowski
e419e22f29 Dockerfile: update compose to v2.28.1
Update the compose cli plugin used in the dev-container

full diff: https://github.com/docker/cli/compare/v2.27.1...v2.28.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 790035f754)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-27 14:55:26 +02:00
Paweł Gronowski
e953d76450 Merge pull request #48060 from thaJeztah/27.0_backport_api_deprecate_ContainerJSONBase_Node
[27.0 backport] api/types: deprecate ContainerJSONBase.Node, ContainerNode
2024-06-26 20:30:43 +02:00
Paweł Gronowski
861fde8cc9 Merge pull request #48061 from thaJeztah/27_backport_bump_golangci_lint
[27.0 backport] update golangci-lint to v1.59.1
2024-06-26 19:14:38 +02:00
Sebastiaan van Stijn
3557077867 update golangci-lint to v1.59.1
full diff: https://github.com/golangci/golangci-lint/compare/v1.55.2...v1.59.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 95fae036ae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-26 14:09:41 +02:00
Sebastiaan van Stijn
c95b917167 pkg/archive: reformat code to make #nosec comment work again
Looks like the way it picks up #nosec comments changed, causing the
linter error to re-appear;

    pkg/archive/archive_linux.go:57:17: G305: File traversal when extracting zip/tar archive (gosec)
                    Name:       filepath.Join(hdr.Name, WhiteoutOpaqueDir),
                                ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d4160d5aa7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-26 14:09:41 +02:00
Sebastiaan van Stijn
c0ff08acbd builder/remotecontext: reformat code to make #nosec comment work again
Looks like the way it picks up #nosec comments changed, causing the
linter error to re-appear;

    builder/remotecontext/remote.go:48:17: G107: Potential HTTP request made with variable url (gosec)
        if resp, err = http.Get(address); err != nil {
                       ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 04bf0e3d69)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-26 14:09:41 +02:00
Sebastiaan van Stijn
4587688258 api/types: deprecate ContainerJSONBase.Node, ContainerNode
The `Node` field and related `ContainerNode` type were used by the classic
(standalone) Swarm API. API documentation for this field was already removed
in 234d5a78fe (API 1.41 / docker 20.10), and
as the Docker Engine didn't implement these fields for the Swarm API, it
would always have been unset / nil.

Let's do a quick deprecation, and remove it on the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1fc9236119)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-26 14:05:47 +02:00
Paweł Gronowski
ff1e2c0de7 Merge pull request #48050 from thaJeztah/deprecate_graphdriver_plugins
deprecate experimental Graphdriver plugins, and disable by default
2024-06-21 15:12:12 +02:00
Sebastiaan van Stijn
6da604aa6a deprecate experimental Graphdriver plugins, and disable by default
Graphdriver plugins] are an experimental feature that allow extending the
Docker Engine with custom storage drivers for storing images and containers.
This feature was not maintained since its inception, and will no longer be
supported in upcoming releases.

Users of this feature are recommended to instead configure the Docker Engine
to use the [containerd image store], and a custom [snapshotter].

This patch:

- Disables loading graphdriver plugins by default, producing an error instead.
- Introduces a temporary `DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS` environment
  variable to re-enable the deprecated features; this allows users to still
  use the feature on a v27.0 daemon, but disabling it by default will give a
  strong message that it will no longer be supported.

[Graphdriver plugins]: https://github.com/docker/cli/blob/v26.1.4/docs/extend/plugins_graphdriver.md
[containerd image store]: https://docs.docker.com/storage/containerd/
[snapshotter]: https://github.com/containerd/containerd/tree/v1.7.18/docs/snapshotters

Before this patch (ignore the "Unable to load plugin" errors, as there's no plugin);

    dockerd --experimental -s my-driver
    ...
    INFO[2024-06-21T10:42:49.574901255Z] containerd successfully booted in 0.011384s
    INFO[2024-06-21T10:42:50.575891922Z] [graphdriver] trying configured driver: my-driver
    WARN[2024-06-21T10:42:50.576121547Z] Unable to locate plugin: my-driver, retrying in 1s
    WARN[2024-06-21T10:42:51.577131506Z] Unable to locate plugin: my-driver, retrying in 2s
    WARN[2024-06-21T10:42:53.582637715Z] Unable to locate plugin: my-driver, retrying in 4s

With this patch:

    dockerd --experimental -s my-driver
    ...
    INFO[2024-06-21T10:32:35.123078845Z] [graphdriver] trying configured driver: my-driver
    ERRO[2024-06-21T10:32:35.123127012Z] Failed to GetDriver graph                     driver=my-driver error="DEPRECATED: Experimental graphdriver plugins are deprecated, and disabled by default. This feature will be removed in the next release. See https://docs.docker.com/go/deprecated/" home-dir=/var/lib/docker
    INFO[2024-06-21T10:32:35.124735595Z] stopping healthcheck following graceful shutdown  module=libcontainerd
    INFO[2024-06-21T10:32:35.124743137Z] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
    failed to start daemon: error initializing graphdriver: driver not supported: my-driver

With the `DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS` env-var set:

    DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS=1 dockerd --experimental -s my-driver
    ...
    INFO[2024-06-21T10:35:04.149901970Z] containerd successfully booted in 0.013614s
    INFO[2024-06-21T10:35:05.148195845Z] [graphdriver] trying configured driver: my-driver
    WARN[2024-06-21T10:35:05.150647679Z] Unable to locate plugin: my-driver, retrying in 1s
    WARN[2024-06-21T10:35:06.152531221Z] Unable to locate plugin: my-driver, retrying in 2s
    WARN[2024-06-21T10:35:08.158452389Z] Unable to locate plugin: my-driver, retrying in 4s
    WARN[2024-06-21T10:35:12.163699293Z] Unable to locate plugin: my-driver, retrying in 8s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-21 13:08:31 +02:00
Paweł Gronowski
81b2027979 Merge pull request #48049 from thaJeztah/fix_swagger_tmpfsopts
api: swagger: fix definition of TmpFsOptions (API v1.46)
2024-06-21 12:22:26 +02:00
Paweł Gronowski
97f6a9d005 Merge pull request #48045 from thaJeztah/bump_ttrpc_1.2.5
vendor: github.com/containerd/ttrpc v1.2.5
2024-06-21 12:19:51 +02:00
Paweł Gronowski
3aace758b9 Merge pull request #48046 from thaJeztah/daemon_no_logrus
cmd/dockerd: initMiddlewares: use containerd/logs
2024-06-21 12:19:31 +02:00
Sebastiaan van Stijn
ce5571f343 api: swagger: fix definition of TmpFsOptions (API v1.46)
Since it's a [][]string, there should only be two levels of array
in the OpenAPI spec. Also, the outermost level array shouldn't have
properties: (it should have items: instead).

Co-authored-by: Mark Yen <mark.yen@suse.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-21 09:17:57 +02:00
Sebastiaan van Stijn
a9ab04603e cmd/dockerd: initMiddlewares: use containerd/logs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-20 23:06:40 +02:00
Tianon Gravi
418eed6e4e Merge pull request #47804 from cpuguy83/more_paths_docker_proxy
Lookup docker-proxy in libexec paths
2024-06-20 13:52:21 -07:00
Sebastiaan van Stijn
e355e10011 vendor: github.com/containerd/ttrpc v1.2.5
full diff: https://github.com/containerd/ttrpc/compare/v1.2.4...v1.2.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-20 22:52:19 +02:00
Brian Goff
f8c088be05 Lookup docker-proxy in libexec paths
This allows distros to put docker-proxy under libexec paths as is done
for docker-init.

Also expands the lookup to to not require a `docker/` subdir in libexec
subdir.
Since it is a generic helper that may be used for something else in the
future, this is only done for binaries with a `docker-`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-06-20 19:26:54 +00:00
Paweł Gronowski
018d93decf Merge pull request #47961 from gtomitsuka/gabriel/data-races
client: Make version negotiation thread-safe
2024-06-20 13:55:10 +02:00
Albin Kerouanton
1a1f3cff45 Merge pull request #48011 from thaJeztah/deprecate_runconfig_IsPreDefinedNetwork
runconfig: deprecate IsPreDefinedNetwork
2024-06-20 12:59:08 +02:00
Sebastiaan van Stijn
202de333a4 Merge pull request #48040 from thaJeztah/move_stats
api/types: move stats-types to api/types/container
2024-06-20 11:30:31 +02:00
Sebastiaan van Stijn
d22d8a78f1 runconfig: deprecate IsPreDefinedNetwork
Move the function internal to the daemon, where it's used. Deliberately
not mentioning the new location, as this function should not be used
externally.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-20 11:10:54 +02:00
Sebastiaan van Stijn
a24af26aba Merge pull request #48037 from thaJeztah/registry_cleanups
registry: minor cleanups
2024-06-20 10:27:16 +02:00
Sebastiaan van Stijn
b5d3c47a37 Merge pull request #48033 from thaJeztah/api_update_examples
docs/api: update some example values to be more accurate (API v1.46)
2024-06-20 10:26:52 +02:00
Sebastiaan van Stijn
0a4277abf4 api/types: move stats-types to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-20 10:21:42 +02:00
Paweł Gronowski
fe60fa241b Merge pull request #48039 from thaJeztah/fixo_typo
api/types: fix typo in GoDoc
2024-06-20 10:15:35 +02:00
Paweł Gronowski
5ace798cab Merge pull request #48041 from thaJeztah/rename_statsresponse
api/types: rename container.StatsResponse to StatsResponseReader
2024-06-20 10:12:17 +02:00
Sebastiaan van Stijn
91a2a574d7 api/types: rename container.StatsResponse to StatsResponseReader
commit 17c3269a37 moved the ContainerStats
type to the container package, and renamed it to StatsResponse. However,
this name is chosen poorly, as it documents it to be the response of
the API endpoint, but is more accurately a wrapper around a reader,
used to read a (stream of) StatsJSON. We want to change StatsJSON
to StatsResponse, as it's more consistent with other response types.

As 17c3269a37 did not make it into a
non-pre-release, we can still change this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-20 08:54:52 +02:00
Akihiro Suda
485e1c7be7 Merge pull request #48035 from thaJeztah/more_mailmap
update one more entry in mailmap and AUTHORS
2024-06-20 09:16:16 +09:00
Sebastiaan van Stijn
6fafc8762f api/types: fix typo in GoDoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-20 01:39:32 +02:00
Brian Goff
3a36cad0bd Merge pull request #48034 from cyphar/volume-atomic-write
volume: use AtomicWriteFile to save volume options
2024-06-19 18:39:49 +00:00
Sebastiaan van Stijn
42cb29f6ea registry: Search.searchUnfiltered: inline variable
The scopes variable was used in one location; inline it where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-19 17:05:48 +02:00
Sebastiaan van Stijn
43d4a4c63e registry: v2AuthHTTPClient: inline some vars and slight refactor
- inline the auth.TokenHandlerOptions in the auth.NewTokenHandlerWithOptions call
- construct a authHandlers slice to make it more clear that this is a variadic
  list of authentication-handlers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-19 17:04:59 +02:00
Sebastiaan van Stijn
bf35f3d8c8 update one more entry in mailmap and AUTHORS
Found back the PR related to this contributors, and they addressed
their name in an intermediate rebase, but it got lost in a later one.

While at it, also fixed an entry next to it :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-19 12:48:41 +02:00
Aleksa Sarai
b4c20da143 volume: use AtomicWriteFile to save volume options
If the system (or Docker) crashes while saivng the volume options, on
restart the daemon will error out when trying to read the options file
because it doesn't contain valid JSON.

In such a crash scenario, the new volume will be treated as though it
has the default options configuration. This is not ideal, but volumes
created on very old Docker versions (pre-1.11[1], circa 2016) do not
have opts.json and so doing some kind of cleanup when loading the volume
store (even if we take care to only delete empty volumes) could delete
existing volumes carried over from very old Docker versions that users
would not expect to disappear.

Ultimately, if a user creates a volume and the system crashes, a volume
that has the wrong config is better than Docker not being able to start.

[1]: commit b05b237075 ("Support mount opts for `local` volume driver")

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2024-06-19 18:57:51 +10:00
Akihiro Suda
11179de64c Merge pull request #48031 from thaJeztah/bump_cobra
vendor: github.com/spf13/cobra v1.8.1
2024-06-19 14:19:41 +09:00
Akihiro Suda
9e85d1cb41 Merge pull request #48032 from thaJeztah/update_mailmap
update .mailmap and AUTHORS
2024-06-19 14:19:09 +09:00
Sebastiaan van Stijn
39b4448e12 docs/api: update some example values to be more accurate (API v1.46)
Update daemon versions, and minimum supported API version to be more
representative to what the API would return.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-19 00:51:14 +02:00
Sebastiaan van Stijn
37b57c2ae0 api: swagger: update some example values to be more accurate
Update daemon versions, and minimum supported API version to be more
representative to what the API would return.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-19 00:50:23 +02:00
Sebastiaan van Stijn
2a7bb2a7bd update .mailmap and AUTHORS
I noticed some duplicates made their way in, in
084219a5f9 and some authors
didn't have git configured properly to include the name
they used for the sign-off

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 23:31:38 +02:00
Sebastiaan van Stijn
e7347f8a8c Merge pull request #48030 from thaJeztah/bump_buildx
Dockerfile: update buildx to v0.15.1
2024-06-18 22:59:07 +02:00
Sebastiaan van Stijn
68b8e97849 vendor: github.com/spf13/cobra v1.8.1
- release notes: https://github.com/spf13/cobra/releases/tag/v1.8.1
- full diff: https://github.com/spf13/cobra/compare/v1.8.0...v1.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 22:17:41 +02:00
Sebastiaan van Stijn
b5cc61a243 Dockerfile: update buildx to v0.15.1
This is the version used in the dev-container, and for testing.

release notes:
https://github.com/docker/buildx/releases/tag/v0.15.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 21:36:25 +02:00
Sebastiaan van Stijn
59b119f94e Merge pull request #47994 from thaJeztah/simplify_getDummyName
libnetwork: ipvlan, macvlan: cleanup getDummyName utility
2024-06-18 21:25:55 +02:00
Albin Kerouanton
3e85c9d517 Merge pull request #48025 from robmry/fix_port_mapped_hairpin
Fix hairpin between networks with mapped port
2024-06-18 19:46:16 +02:00
Sebastiaan van Stijn
f741ca857c libnetwork/drivers/macvlan: getDummyName don't use stringid.TruncateID
The stringid.TruncateID utility is used to provide a consistent length
for "short IDs" (containers, networks). While the dummy interfaces need
a short identifier, they use their own format and don't have to follow
the same length as is used for "short IDs" elsewhere.

In addition, stringid.TruncateID has an additional check for the given
ID to contain colons (":"), which won't be the case for network-IDs that
are passed to it, so this check is redundant.

This patch moves the truncating local to the getDummyName function, so
that it can define its own semantics, independent of changes elsewhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 19:40:15 +02:00
Sebastiaan van Stijn
d241862f36 libnetwork/drivers/macvlan: move truncating ID to getDummyName
The function description mentions that the returned value will contain
a truncated ID, but the function was only prepending the prefix, which
meant that callers had to be aware that truncating is necessary.

This patch moves truncating the ID into the utility to make its use
less error-prone, and to make the code a bite more DRY.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 19:40:12 +02:00
Sebastiaan van Stijn
dab918b2b0 libnetwork/drivers/ipvlan: getDummyName don't use stringid.TruncateID
The stringid.TruncateID utility is used to provide a consistent length
for "short IDs" (containers, networks). While the dummy interfaces need
a short identifier, they use their own format and don't have to follow
the same length as is used for "short IDs" elsewhere.

In addition, stringid.TruncateID has an additional check for the given
ID to contain colons (":"), which won't be the case for network-IDs that
are passed to it, so this check is redundant.

This patch moves the truncating local to the getDummyName function, so
that it can define its own semantics, independent of changes elsewhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 19:34:25 +02:00
Sebastiaan van Stijn
b8c80b19de libnetwork/drivers/ipvlan: move truncating ID to getDummyName
The function description mentions that the returned value will contain
a truncated ID, but the function was only prepending the prefix, which
meant that callers had to be aware that truncating is necessary.

This patch moves truncating the ID into the utility to make its use
less error-prone, and to make the code a bite more DRY.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 19:34:23 +02:00
Albin Kerouanton
fb8d8a9fe5 Merge pull request #47318 from andrewbaxter/47317-allow-macvlan-dup-parent
Allow multiple macvlan networks to share a parent
2024-06-18 19:32:24 +02:00
Sebastiaan van Stijn
b5b7ddfdd5 Merge pull request #48028 from tonistiigi/update-buildkit-v0.14.1
vendor: update buildkit to v0.14.1
2024-06-18 19:12:51 +02:00
Tonis Tiigi
8599213b52 vendor: update buildkit to v0.14.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-06-18 09:12:55 -07:00
Rob Murray
2df4391473 Fix hairpin between networks with mapped port
Following changes to the port mapping code, the DNAT iptables rule was
inserted into the nat table rather than appended.

This meant DNAT was applied before the rule that should have skipped
it when a packet was from a bridge network.

So, packets sent from a container on one network to a mapped port on
the host's address were DNAT'd before docker-proxy could pick them up,
then they were dropped by a rule intended to isolate the networks.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-18 16:14:16 +01:00
Sebastiaan van Stijn
fd71cbfab5 Merge pull request #48026 from thaJeztah/api_v1.46_docs
docs: add API documentation for API v1.46
2024-06-18 15:28:03 +02:00
Sebastiaan van Stijn
9402ea1c8e Merge pull request #48023 from thaJeztah/local_ulimits_alias
api/types/container: provide alias for github.com/docker/go-units.Ulimit
2024-06-18 14:19:43 +02:00
Sebastiaan van Stijn
b06b6b3648 docs: add API documentation for API v1.46
This is the API version to be released with v27.0, and the API
is now frozen for this release, so we can create the documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 14:15:13 +02:00
Sebastiaan van Stijn
f174059a25 Merge pull request #48019 from thaJeztah/executor_err_handling
daemon/cluster/executor/container: fix error-handling
2024-06-18 14:13:46 +02:00
Paweł Gronowski
ea48d90399 Merge pull request #48024 from vvoland/update-authors
update AUTHORS
2024-06-18 14:13:14 +02:00
Sebastiaan van Stijn
5a4595466b Merge pull request #48008 from thaJeztah/deprecate_runconfig_DefaultDaemonNetworkMode
runconfig: deprecate DefaultDaemonNetworkMode, move to daemon/network
2024-06-18 14:13:07 +02:00
Sebastiaan van Stijn
f160cd0087 Merge pull request #48016 from thaJeztah/deprecate_runconfig_opts
runconfig/opts: deprecate ConvertKVStringsToMap and move internal
2024-06-18 14:10:58 +02:00
Sebastiaan van Stijn
517fb0991e api/types/container: provide alias for github.com/docker/go-units.Ulimit
This type is included in various types used in the API, but comes from
a separate module. The go-units module may be moving to the moby org,
and it is yet to be decided if the Ulimit type is a good fit for that
module (which deals with more generic units, such as "size" and "duration"
otherwise).

This patch introduces an alias to help during the transition of this type
to it's new location. The alias makes sure that existing code continues
to work (at least for now), but we need to start updating such code after
this PR is merged.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 13:18:20 +02:00
Sebastiaan van Stijn
c3217300e2 Merge pull request #48022 from thaJeztah/leftover_nits
daemon, daemon/cluster, integration/container: minor linting issues and cleanups
2024-06-18 13:17:41 +02:00
Sebastiaan van Stijn
ad716b223b integration/container: use consistent alias for import
The canonical alias is "containertypes" for this import.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 13:11:37 +02:00
Sebastiaan van Stijn
25f44885ed daemon/cluster/executor/container: use consistent alias for import
The canonical alias is "containertypes" for this import.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 13:11:34 +02:00
Paweł Gronowski
084219a5f9 update AUTHORS
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-18 13:01:11 +02:00
Sebastiaan van Stijn
f09f756851 daemon/cluster: minor linting issues and cleanup
- rename variables that shadowed imports
- remove some intermediate vars
- slight reformating for readability

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-18 10:42:56 +02:00
Rob Murray
9e389b1eac Merge pull request #48020 from robmry/revert_internal_resolver_for_default_bridge
Revert internal resolver for default bridge
2024-06-18 09:04:57 +01:00
Sebastiaan van Stijn
4ea464d1a7 Merge pull request #47950 from psaintlaurent/ENGINE-903
Add OOMScoreAdj to the moby API
2024-06-17 22:58:24 +02:00
Rob Murray
74d77d8811 Revert "Internal resolver for default bridge network"
This reverts commit 18f4f775ed.

Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).

Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.

By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-17 20:19:20 +01:00
Rob Murray
6d35673504 Revert "No default nameservers for internal resolver"
This reverts commit d365702dbd.

Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).

Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.

By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-17 20:19:10 +01:00
Sebastiaan van Stijn
01efb9a5ab daemon/cluster/executor/container: fix error-handling
While working on this file, I noticed the `isContainerCreateNameConflict`,
`isUnknownContainer`, and `isStoppedContainer` utilities, which are used
to perform error-type detection through string-matching.

These utilities were added in 534a90a993,
as part of the initial implementation of the Swarm executor in Docker.
At that time, the Docker API client did not return typed errors, and
various part of the code depended on string matching, which is brittle,
and it looks like `isContainerCreateNameConflict` at least is already
broken since c9d0a77657, which changed
the error-message.

Starting with ebcb7d6b40, we use typed
errors through the errdefs package, so we can replace these utilities:

The `isUnknownContainer` utility is replace by `errdefs.IsNotFound`,
which is returned if the object is not found. Interestingly, this utility
was checking for containers only (`No such container`), but was also
used for an `removeNetworks` call. Tracking back history of that use to
verify if it was _intentionally_ checking for a "container not found"
error;

- This check added in the initial implementation 534a90a993
- Moved from `controller.Remove` to `container.Shutdown` to make sure the
  sandbox was removed in 680d0ba4ab
- And finally touched again in 70fa7b6a3f,
  which was a follow-up to the previous one, and fixed the conditions
  to prevent returning early before the network was removed.

None of those patches mention that these errors are related to containers,
and checking the codepath that's executed, we can only expect a
`libmetwork.ErrNoSuchNetwork` to be returned, so this looks to have been
a bug.

The `isStoppedContainer` utility is replaced by `errdefs.IsNotModified`,
which is the error (status) returned in situations where the container
is already stopped; caf502a0bc/daemon/stop.go (L30-L35)
This is the only

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 21:14:47 +02:00
plaurent
b640582436 Add OomScoreAdj options
Signed-off-by: plaurent <patrick@saint-laurent.us>
2024-06-17 12:01:06 -04:00
Sebastiaan van Stijn
8e91b64e07 runconfig: deprecate DefaultDaemonNetworkMode, move to daemon/network
This function returns the default network to use for the daemon platform;
moving this to a location separate from runconfig, which is planned to
be dismantled and moved to the API.

While it might be convenient to move this utility inside api/types/container,
we don't want to advertise this function too widely, as the default returned
can ONLY be considered correct when ran on the daemon-side. An alternative
would be to introduce an argument (daemonPlatform), which isn't very convenient
to use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 17:32:56 +02:00
Paweł Gronowski
caf502a0bc Merge pull request #47239 from cpuguy83/containerd_image_info
Set containerd container image ref
2024-06-17 17:02:24 +02:00
Sebastiaan van Stijn
d88ae86a16 Merge pull request #48014 from thaJeztah/daemon_rename_imports
daemon: rename some inconsistent import aliases
2024-06-17 16:40:53 +02:00
Paweł Gronowski
9f4cd92e07 Merge pull request #47929 from vvoland/image-create
daemon: Emit Image Create event when image is built
2024-06-17 16:30:35 +02:00
Sebastiaan van Stijn
437e1ae15e runconfig/opts: deprecate ConvertKVStringsToMap and move internal
This utility is only used in two places, and simple enough to duplicate.
There's no external consumers, and a copy of this utility exists in docker/cli
for use on the client side, so we could consider skipping deprecation,
but just to be on the safe side ':)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 16:23:39 +02:00
Sebastiaan van Stijn
b3e236d3b5 daemon: rename some inconsistent import aliases
These used aliases that weren't used elsewhere, so renaming / removing
to keep some consistency. Some local variables were renamed to prevent
shadowing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 14:50:09 +02:00
Brian Goff
2851ddc44c Add containerd image ref to created containers
This populates the "Image" field on containerd containers, but only when
using the containerd image store.
This allows containerd clients to look up the image information.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 14:45:17 +02:00
Brian Goff
812f319a57 Add containerd connection info to info endpoint (API v1.46)
This will be used in the next commit to test that changes are propagated
to the containerd store.
It is also just generally useful for debugging purposes.

- docs/api: update version history
- daemon: add fillContainerdInfo utility
- api: update swagger file with new types

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 14:43:39 +02:00
Paweł Gronowski
1327342b14 hack: Ignore deprecate-integration-cli validation
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:15 +02:00
Paweł Gronowski
7b8f4922a5 daemon: Emit Image Create event when image is built
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:12 +02:00
Paweł Gronowski
09eb6ec4f1 builder/exporter: Wrap exporter to hook the image export
Buildkit doesn't call the engine API when it builds an image without
tag. Wrap the exporter returned by the worker that calls a callback when
a new image is exported from buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:11 +02:00
Paweł Gronowski
0e84482ef5 builder-next: Move exporter wrapper to exporter package
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-17 14:05:09 +02:00
Sebastiaan van Stijn
eb360efeb5 Merge pull request #48013 from thaJeztah/reformat_networkmodes
api/types/container: reformat to align windows and unix implementations
2024-06-17 13:20:56 +02:00
Sebastiaan van Stijn
7012c4a129 Merge pull request #47996 from thaJeztah/pkg_longpath_deprecate_Prefix
pkg/longpath: deprecate Prefix const, and use early returns in AddPrefix
2024-06-17 13:18:33 +02:00
Sebastiaan van Stijn
b2441c7419 Merge pull request #45052 from cpuguy83/attach_fd_leak
Fix attach goroutine/fd leak when no I/O is ready
2024-06-17 13:16:08 +02:00
Sebastiaan van Stijn
08aebce331 Merge pull request #48012 from thaJeztah/daemon_less_shadow
daemon: rename variables that shadowed imports
2024-06-17 12:59:09 +02:00
Sebastiaan van Stijn
53c521bdc9 Merge pull request #47993 from thaJeztah/builder_dockerfile_cleanups
builder/dockerfile: assorted linting fixes, and remove LCOW leftover
2024-06-17 12:06:25 +02:00
Sebastiaan van Stijn
48ff86ec64 pkg/longpath: AddPrefix: use early returns
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 12:04:26 +02:00
Sebastiaan van Stijn
424c22390e pkg/longpath: deprecate Prefix const
This const was exported because it was in use by pkg/symlink. This
dependency was removed in a48c6e3005,
after which this const was only used internally.

This patch deprecates the const and introduces a non-exported const
to use.

There are no known external consumers of this const, so we may skip
deprecating it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 12:03:27 +02:00
Sebastiaan van Stijn
2f45cbf69f api/types/container: NetworkMode align code between Windows and Linux
Change the order of declarations betwen both implementations for easier
comparing of differences.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 11:53:19 +02:00
Sebastiaan van Stijn
7b56fa8dc0 api/types/container: NetworkMode.NetworkName: use switch
- Use a switch instead of if/else for readability and to reduce
  the risk of duplicates in the checks.
- Align order between Windows and Linux implementation for easier
  comparing of differences in the implementation.
- Add a check for `IsHost()` in the Windows implementation which
  would never occur currently, but is implemented.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 11:49:37 +02:00
Sebastiaan van Stijn
680e6d6e1c Merge pull request #48000 from thaJeztah/nosec_comments
pkg/archive. pkg/tarsum: format #nosec comments to standard format
2024-06-17 11:46:25 +02:00
Sebastiaan van Stijn
c114b5e6f0 Merge pull request #47997 from thaJeztah/pkg_archive_no_longpath
pkg/archive, pkg/chrootarchive: remove dependency on pkg/longpath
2024-06-17 11:43:59 +02:00
Sebastiaan van Stijn
4014b893e4 Merge pull request #48007 from thaJeztah/runconfig_drop_old_api_versions
runconfig: remove code for API < v1.18, deprecate SetDefaultNetModeIfBlank, ContainerConfigWrapper
2024-06-17 11:43:15 +02:00
Sebastiaan van Stijn
9c7f20e255 Merge pull request #48003 from thaJeztah/pkg_archive_cleanup
pkg/archive: assorted minor refactors and cleanups
2024-06-17 11:11:42 +02:00
Sebastiaan van Stijn
7b438c5c31 daemon: rename variables that shadowed imports
Not a full list yet, but renaming to prevent shadowing, and to use a more
consistent short form (ctr for container).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 11:06:06 +02:00
Sebastiaan van Stijn
21a7686862 Merge pull request #47995 from thaJeztah/stringid_cleanups
pkg/stringid: deprecate ValidateID, IsShortID
2024-06-17 10:55:06 +02:00
Sebastiaan van Stijn
e788628e2e Merge pull request #48005 from thaJeztah/pkg_archive_rm_kernelversion_check
pkg/archive: TestChangesDirsEmpty, TestChangesDirsMutated: no kernel-version check
2024-06-17 10:34:36 +02:00
Albin Kerouanton
a5fede8a51 Merge pull request #48006 from thaJeztah/libnetwork_nosversion
libnetwork: remove special handling for Windows 14393 (RS1, V1607, LTSC2016)
2024-06-17 10:25:28 +02:00
Sebastiaan van Stijn
4c7228663c api/server/router/container.postCommit: add TODO about use of CreateRequest
The commit endpoint accepts a container.Config, but uses the decoder to
unmarshal the request. The decoder uses a CreateRequest, which is a superset,
and also contains HostConfig and network.NetworkConfig. Those structs are
discarded in the router, but decoder.DecodeConfig also performs validation,
so a request containing those additional fields would result in a validation
error.

We should rewrite this code to only unmarshal what's expected.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:46 +02:00
Sebastiaan van Stijn
c692af36c3 runconfig: deprecate ContainerConfigWrapper, move to api/types/container
Move the type to api/types/container.CreateRequest, together with other
types used by the container API endpoints.

The Decoder, and related validation code is kept in the runconfig package
for now, but should likely be moved elsewhere (inside the API).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:46 +02:00
Sebastiaan van Stijn
afdfe4ff86 runconfig: deprecate SetDefaultNetModeIfBlank
Remove uses of this function and mark it deprecated. There's no known
consumers of this function, but let's stay on the safe side, and mark
it deprected for 1 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:46 +02:00
Sebastiaan van Stijn
4af9f418a3 runconfig: remove ContainerConfigWrapper.getHostConfig() utility
This utility used to be responsible for backward compatibility with old
API versions, but was reduced to a single line. Inline the code, and
inline the SetDefaultNetModeIfBlank code, which in itself also was
just 3 lines of code.

A platform check was added to only set the default network conditionally,
but other paths in the codebase don't perform this conditionally, so a
TODO was added, to verify if this behavior is needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:46 +02:00
Sebastiaan van Stijn
98bd08c534 runconfig: remove redundant import-alias
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:46 +02:00
Sebastiaan van Stijn
e42503213d runconfig: ContainerConfigWrapper: unify Linux and Windows implementations
Now that the backward-compatibility code has been removed, the Linux and
Windows implementations of this struct are identical, so the platform-
specific code can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:45 +02:00
Sebastiaan van Stijn
2954b05b03 runconfig: remove conversion code for API < v1.18
The runconfig package contained compatibility code to unmarshal API
requests on API < v1.18, and to convert them to current API versions.
These fields were marked as deprecated, but didn't mention relevant API
versions, so some digging was needed to track back history;

API versions before 1.18 accepted top-level `Memory`, `MemorySwap`,
`CpuShares`, and `Cpuset` fields as part of the container create requests.
These fields were not considered "portable", and therefore moved to the
`HostConfig` struct in 837eec064d. The
API version at that time was [v1.18]. For backward-compatibility, the
existing top-level fields were kept, and conversion code was added in
[ContainerHostConfigFromJob] to copy their values to `HostConfig` if
present.

A refactor in 767df67e31 introduced a new
`ContainerConfigWrapper` struct, which embedded the container-config and
a (non-exported) `hostConfigWrapper`. This resulted in an incompatibility
when compiling with gccgo, sn eb97de7dee
removed the non-exported `hostConfigWrapper`, instead embedding the
`HostConfig` and adding a `CpuSet` field. The API version at that time
was [v1.19].

With the introduction of Windows containers, which did not need conversion
code as it never supported previous API versions, the `ContainerConfigWrapper`
was split to Linux and Windows implementation in f6ed590596.
This change introduced a `SetDefaultNetModeIfBlank` function to set the
default network-mode on Linux. Windows did not have a default network,
but did require a separate `ValidateNetMode` implemenation.

The `ContainerConfigWrapper` was expanded to include `NetworkingConfig`
in 2bb3fc1bc5 for API [v1.22], but did
not involve backward-compatiblity / conversion code.

Based on the above, all conversion code present in runconfig is related
to API versions [v1.18] or before. 19a04efa2f,
and other commits in [moby PR 47155] removed support for API < v1.24, so
this conversion code is no longer needed.

This patch removes the legacy fields from the `ContainerConfigWrapper`,
and removes the corresponding conversion code. The `InnerHostConfig` field
is also renamed, as it is no longer shadowed by the `container.HostConfig`
that was embedded for backward-compatibility.

[v1.18]: 837eec064d/api/common.go (L18)
[v1.19]: 767df67e31/api/common.go (L20)
[v1.22]: 2bb3fc1bc5/api/common.go (L21)
[moby PR 47155]: https://github.com/moby/moby/pull/47155
[ContainerHostConfigFromJob]: 837eec064d/runconfig/hostconfig.go (L149-L162)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:45 +02:00
Sebastiaan van Stijn
37f4616751 integration-cli: fix TestCreateWithTooLowMemoryLimit: using deprecated API fields
This test was depending on top-level fields that were deprecated since
API v1.18. These fields are no longer sent by current clients.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 00:26:31 +02:00
Sebastiaan van Stijn
964aba696c libnetwork: windows/overlay: remove endpointRequest wrapper
This wrapper is now a plain alias for hcsshim.HNSEndpointRequest, so let's
remove the extra abstraction.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 19:02:18 +02:00
Sebastiaan van Stijn
c316ed0c22 libnetwork: remove special handling for Windows 14393 (RS1, V1607, LTSC2016)
This synchronisation was added in [libnetwork@0a61693]:

> Adding synchronization around peerAdd and peerDelete to prevent network
> connectivity issue
>
> When multiple networks are present in a Swarm Cluster, multiple peerAdd
> or peerDelete calls are an issue for different remote endpoints. These
> threads are updating the remote endpoint to HNS parallelly. In 2016 HNS
> code base, we don't have synchronization around remoteEndpoint addition
> and deletion. So serializing the peerAdd and peerDelete calls from docker
> network driver.

We no longer support and test Windows 2016, as it reached EOL / end of
[standard support][1], so we can remove this special condition.

[libnetwork@0a61693]: c90114ce7c
[1]: https://en.wikipedia.org/wiki/Windows_10,_version_1607

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 18:59:21 +02:00
Akihiro Suda
ec4bac431c Merge pull request #47999 from thaJeztah/deprecate_pkg_dmesg
pkg/dmesg: deprecate, and use internal utility instead
2024-06-17 01:50:53 +09:00
Akihiro Suda
ff652c82e9 Merge pull request #48001 from thaJeztah/pkg_archive_deprecate_CanonicalTarNameForPath
pkg/archive: deprecate CanonicalTarNameForPath
2024-06-17 01:50:21 +09:00
Akihiro Suda
19257effaa Merge pull request #48002 from thaJeztah/pkg_archive_deprecate_TempArchive
pkg/archive: deprecate NewTempArchive, TempArchive
2024-06-17 01:49:58 +09:00
Sebastiaan van Stijn
3108165c94 pkg/archive: TestChangesDirsEmpty, TestChangesDirsMutated: no kernel-version check
TestChangesDirsEmpty and TestChangesDirsMutated fail on Windows V19H1 (1903)
and up, possibly due to changes in the kernel:

    === FAIL: github.com/docker/docker/pkg/archive TestChangesDirsEmpty (0.21s)
    changes_test.go:261: Reported changes for identical dirs: [{\dirSymlink C}]

    === FAIL: github.com/docker/docker/pkg/archive TestChangesDirsMutated (0.14s)
    changes_test.go:391: unexpected change "C \\dirSymlink" "\\dirnew"

commit 8f4b3b0ad4 added a version-dependent
skip for those tests, but as we no longer run CI on versions before V19H1,
we can remove the kernel-version check, and skip it on Windows unconditionally.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 18:40:50 +02:00
Sebastiaan van Stijn
17ca8b62bd pkg/archive: remove uses of iota
While using iota can be convenient, it can also make it harder to grasp
what value is assigned. Use of iota also makes changing values implicit;
changing the order of these consts implicitly means their value changes.

This can be problematic, as some of these consts are a plain `int` and
while golang is strong-typed, it does allow plain `int` values to be
used for such values.

For example, `archive.Tar` accepts a `Compression` as second argument,
but allows a plain int to be passed, so both of these are equivalent;

    archive.Tar(contextDir, archive.Uncompressed)
    archive.Tar(contextDir, 0)

This patch removes the use of `iota`, and instead explicitly setting a
value for each to prevent accidental changes in their value, which can
be hard to discover.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 17:14:49 +02:00
Sebastiaan van Stijn
94caeeb401 pkg/archive: don't call system.Lgetxattr on unsupported platforms
[pkg/system.Lgetxattr] is only implemented on Linux, and always produces
an ErrNotSupportedPlatform on other platforms.

This patch removes the call to this function, but intentionally leaves
it commented-out as a reminder to include this code if this would ever
be refactored and implemented on other platforms.

[pkg/system.Lgetxattr]: d1273b2b4a/pkg/system/xattrs_unsupported.go (L1-L8)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 17:14:49 +02:00
Sebastiaan van Stijn
c565a3a1fe pkg/archive: collectFileInfo: don't create FileInfo if unused
The system.Lstat may fail, in which case it would be discarded,
so let's move it later.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 17:14:49 +02:00
Sebastiaan van Stijn
7ce1edd7c6 pkg/archive: deprecate NewTempArchive, TempArchive
These were added in baacae8345, but are
currently only used in tests inside pkg/archive. There are no external
users of this function, so we should deprecated them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 17:07:47 +02:00
Sebastiaan van Stijn
92b8d93f47 pkg/archive: deprecate CanonicalTarNameForPath
Commit d59758450b changed this function to
be a wrapper for `filepath.ToSlash`. It was used in the CLI for the classic
builder, but is no longer used in our codebase.

However, there may still be some consumers that copied the CLI code for the
classic builder that didn't synchronise their implementation yet, so let's
deprecate this function to give them a warning that they should no longer
use this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 16:56:25 +02:00
Sebastiaan van Stijn
0ffc0c95e6 pkg/tarsum: format #nosec comments to standard format
gosec uses a non-standard format for "automated" comments to suppress
false positives (such comments should not have a leading space, but
are not allowed to start with a non-alphabetical character). However,
current versions of gosec do allow a leading space.

This patch reformats the comments to prevent them from being changed
by IDEs when reformating code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 14:31:27 +02:00
Sebastiaan van Stijn
cb19b59b65 pkg/archive: format #nosec comments to standard format
gosec uses a non-standard format for "automated" comments to suppress
false positives (such comments should not have a leading space, but
are not allowed to start with a non-alphabetical character). However,
current versions of gosec do allow a leading space.

This patch reformats the comments to prevent them from being changed
by IDEs when reformating code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 14:31:18 +02:00
Sebastiaan van Stijn
805ccd2365 pkg/dmesg: deprecate, and use internal utility instead
This package was originally added in 46833ee1c3
for use in the devicemapper graphdriver. The devicemapper graphdriver was
deprecated and has been removed. The only remaining consumer is an integration
test.

Deprecate the package and mark it for removal in the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 12:41:39 +02:00
Sebastiaan van Stijn
c7f4abc14a pkg/dmesg: use unix.SYSLOG_ACTION_READ_ALL instead of local variable
This value was originally added in 46833ee1c3,
at which time golang.org/x/sys/unix didn't have utilities for this syscall.
A later patch switched the implementation to use the golang/x/sys/unix
implementation in 2841b05b71, but kept the
local variable.

golang.org/x/sys now has a const for this, so let's use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-16 12:41:39 +02:00
Sebastiaan van Stijn
d1273b2b4a Merge pull request #46809 from dperny/add-exec-option-to-tmpfs
Rebase #36720 "Add exec option to tmpfs"
2024-06-15 22:32:59 +02:00
andrew
528ffa9cae Allow multiple macvlan networks to share a parent
The only case where macvlan interfaces are unable to share a parent is
when the macvlan mode is passthru. This change tightens the check to
that situation.

It also makes the error message more specific to avoid suggesting that
sharing parents is never correct.

Signed-off-by: Andrew Baxter <423qpsxzhh8k3h@s.rendaw.me>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 19:42:11 +02:00
Sebastiaan van Stijn
9389f76f6d pkg/chrootarchive: remove dependency on pkg/longpath
Copy the function to the package, so that we don't have a dependency
on pkg/longpath.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 17:32:08 +02:00
Sebastiaan van Stijn
f657a75bf6 pkg/archive: remove dependency on pkg/longpath
Copy the function to the package, so that we don't have a dependency
on pkg/longpath.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 17:30:13 +02:00
Sebastiaan van Stijn
2100a70741 pkg/stringid: deprecate IsShortID
This function is no longer used, and has no external users. Deprecated
the function and mark if for removal for the next release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 15:35:07 +02:00
Sebastiaan van Stijn
e19e6cf7f4 pkg/stringid: deprecate ValidateID
This function is only used for the legacy v1 image format.

Deprecate the function, and make image/v1 self-contained.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 15:34:38 +02:00
Sebastiaan van Stijn
0fb6360fa7 builder/dockerfile: createDestInfo: remove platform arg (LCOW left-over)
This was added in 7a7357dae1 as part of the
LCOW implementation. LCOW has been removed, and this option was no longer
in use because of that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 14:03:01 +02:00
Sebastiaan van Stijn
c5d95fdb04 builder/dockerfile: fix some minor linting issues
- explicitly suppress some errors
- use fmt.Fprintln instead of manually appending a newline
- remove an outdated TODO; looking at the suggestion, it's not a
  realistic option

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 14:02:55 +02:00
Sebastiaan van Stijn
6fa6812c51 builder/dockerfile: rename vars that shadowed types and builtins
- imageMount was shadowing the imageMount type
- copy was shadowing the copy builtin
- container was shadowing the container import

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 14:01:46 +02:00
Sebastiaan van Stijn
074932966d builder/dockerfile: remove endsInSlash utility
It was only used in a single location, and other locations were shadowing
the function through local variables. As it's a one-liner, inlining the
code may be just as transparent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-15 13:51:44 +02:00
Sebastiaan van Stijn
98fecb0d54 Merge pull request #47991 from corhere/healthcheck-startinterval-swarm
api: adjust health start interval on swarm update
2024-06-15 10:30:57 +02:00
Cory Snider
c8e7fcf91a api: adjust health start interval on swarm update
The health-check start interval added in API v1.44, and the start
interval option is ignored when creating a Swarm service using an older
API version. However, due to an oversight, the option is not ignored
when older API clients _update_ a Swarm service. Fix this oversight by
moving the adjustment code into the adjustForAPIVersion function used by
both the createService and updateService handler functions.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-06-14 18:05:55 -04:00
Paweł Gronowski
ba69bd9c1e Merge pull request #47984 from akerouanton/daemon-restore-clear-net-state
daemon: restore: clear net state for stopped containers
2024-06-14 21:36:52 +02:00
Arash Deshmeh
dd1ca95ef9 Add exec option to API TmpfsOptions
Includes two commits from Arash Deshmeh:

add exec option to API TmpfsOptions and the related volume functions

Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>

feature: daemon handles tmpfs mounts exec option

Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>

Updated by Drew Erny

Signed-off-by: Drew Erny <derny@mirantis.com>
2024-06-14 12:11:20 -05:00
Albin Kerouanton
c467e4f08d Merge pull request #47989 from robmry/ipv6_bridge_route_noerrlog
Don't log an error about route-add for IPv6 bridge
2024-06-14 18:50:20 +02:00
Albin Kerouanton
07053a0991 testutil/daemon: Wait() until the daemon is Kill()'ed
`Daemon.Kill()` was sending a SIGKILL to the daemon process but wasn't
waiting until the process was really killed. While the race window is
really small, better safe than sorry.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 18:35:31 +02:00
Albin Kerouanton
955b923352 daemon: releaseNetwork: clear SandboxID, SandboxKey
When the container stops or during `restore`, `daemon.releaseNetwork` is
used to clear all net-related state carried by a container. However, the
fields `SandboxID` and `SandboxKey` are never cleared. On the next start,
these fields will be replaced with new values. There's no point in
preserving these data since they became invalid as soon as the container
stopped.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 18:35:31 +02:00
Albin Kerouanton
e3c5665d21 daemon: restore: clear net state for stopped containers
When the daemon crashes, the host unexpectedly reboot, or the daemon
restarts with live-restore enabled, running containers might stop and the
on-disk state for containers might diverge from reality. All these
situations are currently handled by the daemon's `restore` method.

That method calls `daemon.Cleanup()` for all the dead containers. In
turn, `Cleanup` calls `daemon.releaseNetwork()`. However, this last
method won't do anything because it expects the `netController` to be
initialized when it's called. That's not the case in the `restore` code
path -- the `netController` is initialized _after_ cleaning up dead
containers.

There's a chicken-egg problem here, and fixing that would require some
important architectural changes (eg. change the way libnet's controller
is initialized).

Since `releaseNetwork()` early exits, dead containers won't ever have
their networking state cleaned. This led to bugs in Docker Desktop,
among other things.

Fix that by calling `releaseNetwork` after initializing the
`netController`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 18:35:31 +02:00
Rob Murray
17a59a7506 Don't log an error about route-add for IPv6 bridge
setupBridgeIPv6 attempts to add a route to a new network while
the bridge device is 'down', so it always fails (and the route
is added anyway when the bridge is set 'up').

I'm almost sure the RouteAdd can be removed but, this close to
the moby 27.0 release, only sure-enough to demote the log message
from error to debug.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-14 16:34:23 +01:00
Paweł Gronowski
34c3569768 Merge pull request #47985 from vvoland/bk-patchimageconfig-panic
builder/mobyexporter: Add missing nil check
2024-06-14 17:05:21 +02:00
Albin Kerouanton
1882da852e Merge pull request #47906 from akerouanton/libnet-add-otel-spans-v3
api, daemon, libnet: Create OTel spans at various places
2024-06-14 17:03:56 +02:00
Albin Kerouanton
57c6a5e691 libnet: SetKey: propagate traces from API to SetKey reexec
The `Sandbox.SetKey()` method is called through an OCI prestart hook
which then calls back the daemon through a UNIX socket. This method is
responsible for provisioning interfaces, etc... into the sandbox.

A new EnvironCarrier is used to propagate the trace context to the
prestart hook, which then marhsals an OTel MapCarrier into the JSON
payload sent back to the daemon. That way, every spans created from
`SetKey()` are correctly parented to the original `ContainerStart` API
call.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 15:25:08 +02:00
Albin Kerouanton
b7186bdfc8 libnet: Sandbox: add ctx to SetKey
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 15:25:08 +02:00
Albin Kerouanton
6c71ebd82c libcontainerd: Start: add ctx
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 15:25:07 +02:00
Albin Kerouanton
2d8c4265c7 libcontainerd: NewTask: add ctx
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 15:25:07 +02:00
Albin Kerouanton
19f72d6fc4 libnet: add more OTel spans
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 15:25:07 +02:00
Albin Kerouanton
224d7291df container: add a span to CheckpointTo
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-14 15:25:07 +02:00
Paweł Gronowski
642242a26b builder/mobyexporter: Add missing nil check
Add a nil check to handle a case where the image config JSON would
deserialize into a nil map.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-14 15:25:04 +02:00
Sebastiaan van Stijn
076c976e28 Merge pull request #47981 from gtomitsuka/move-debug-to-dockerd
cli/debug: move to "cmd/dockerd/debug"
2024-06-14 09:53:34 +02:00
Gabriel Tomitsuka
44f19518f9 move the cli/debug package to cmd/dockerd/debug
Signed-off-by: Gabriel Tomitsuka <gabriel@tomitsuka.com>
2024-06-13 21:53:44 +00:00
Sebastiaan van Stijn
078c3a237c Merge pull request #47979 from thaJeztah/gofmt_builder
builder/builder-next: gofmt
2024-06-13 23:35:04 +02:00
Sebastiaan van Stijn
fef34669f6 Merge pull request #47976 from thaJeztah/bump_runc_1.1.13
update runc binary and vendor to v1.1.13
2024-06-13 23:06:31 +02:00
Sebastiaan van Stijn
e4e40558ba builder/builder-next: gofmt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 21:50:12 +02:00
Gabriel Tomitsuka
88e5e4cfb4 Prevent data race during version negotiation
Signed-off-by: Gabriel Tomitsuka <gabriel@tomitsuka.com>
2024-06-13 19:37:11 +00:00
Sebastiaan van Stijn
9101392309 update runc binary to v1.1.13
Update the runc binary that's used in CI and for the static packages.

full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13

Release notes:

* If building with Go 1.22.x, make sure to use 1.22.4 or a later version.

* Support go 1.22.4+.
* runc list: fix race with runc delete.
* Fix set nofile rlimit error.
* libct/cg/fs: fix setting rt_period vs rt_runtime.
* Fix a debug msg for user ns in nsexec.
* script/*: fix gpg usage wrt keyboxd.
* CI fixes and misc backports.
* Fix codespell warnings.

* Silence security false positives from golang/net.
* libcontainer: allow containers to make apps think fips is enabled/disabled for testing.
* allow overriding VERSION value in Makefile.
* Vagrantfile.fedora: bump Fedora to 39.
* ci/cirrus: rm centos stream 8.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 21:25:10 +02:00
Sebastiaan van Stijn
ec24e19d69 vendor: github.com/opencontainers/runc v1.1.13
full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13

Release notes:

* If building with Go 1.22.x, make sure to use 1.22.4 or a later version.

* Support go 1.22.4+.
* runc list: fix race with runc delete.
* Fix set nofile rlimit error.
* libct/cg/fs: fix setting rt_period vs rt_runtime.
* Fix a debug msg for user ns in nsexec.
* script/*: fix gpg usage wrt keyboxd.
* CI fixes and misc backports.
* Fix codespell warnings.

* Silence security false positives from golang/net.
* libcontainer: allow containers to make apps think fips is enabled/disabled for testing.
* allow overriding VERSION value in Makefile.
* Vagrantfile.fedora: bump Fedora to 39.
* ci/cirrus: rm centos stream 8.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 21:25:06 +02:00
Sebastiaan van Stijn
ff5cc18482 Merge pull request #47977 from thaJeztah/bump_runc_deps
vendor: golang.org/x/sys v0.19.0, golang.org/x/crypto v0.22.0, golang.org/x/net v0.24.0
2024-06-13 21:11:12 +02:00
Sebastiaan van Stijn
7106a96be2 Merge pull request #47973 from thaJeztah/fix_gocompat
Add more go:build statements to prevent downgrading Go language version, and update to go1.21
2024-06-13 20:54:45 +02:00
Tianon Gravi
b5bc84119e Merge pull request #47960 from robmry/dev_container_ip6_tables
Dev container: try to load kernel module ip6_tables
2024-06-13 11:22:45 -07:00
Sebastiaan van Stijn
d20739b6fe vendor: golang.org/x/net v0.24.0
no changes in vendored files

full diff: https://github.com/golang/net/compare/v0.23.0...v0.24.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 17:48:51 +02:00
Sebastiaan van Stijn
535898dd9a vendor: golang.org/x/crypto v0.22.0
full diff: https://github.com/golang/crypto/compare/v0.21.0...v0.22.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 17:48:03 +02:00
Sebastiaan van Stijn
4b2aa9f875 vendor: golang.org/x/sys v0.19.0
full diff: https://github.com/golang/sys/compare/v0.18.0...v0.19.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 17:47:00 +02:00
Albin Kerouanton
cec0d50361 libnet: add ctx to Sandbox.Destroy()
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 17:13:43 +02:00
Albin Kerouanton
af23a024a1 libnet: Endpoint: add ctx to Join and Leave
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 17:00:05 +02:00
Albin Kerouanton
566026af8f libnet: Controller: add ctx to store methods
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:59:05 +02:00
Albin Kerouanton
9391052700 libnet: Add ctx to NewSandbox
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:52:48 +02:00
Albin Kerouanton
4924f56e7b libnet/driverapi: Add ctx to ProgramExternalConnectivity
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:51:55 +02:00
Albin Kerouanton
c5c1d133ef libnet/driverapi: Add ctx to Join
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:45:54 +02:00
Albin Kerouanton
8dcded102e libnet: add OTel spans to CreateEndpoint
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 16:45:31 +02:00
Sebastiaan van Stijn
450f18d3ca Merge pull request #47971 from thaJeztah/vendor_no_gocompat
hack/vendor.sh: remove redundant  -compat 1.18
2024-06-13 15:13:18 +02:00
Sebastiaan van Stijn
cf376170ed Add more go:build statements to prevent downgrading Go language version
Looks like some packages fail in go module mode, because they require
recent Go versions:

    GO111MODULE=on go test -v
    # github.com/docker/docker/libnetwork/ipamutils
    ../../libnetwork/ipamutils/utils.go:46:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipamutils/utils.go:51:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/libnetwork/portallocator
    ../../libnetwork/portallocator/portallocator.go:179:7: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/libnetwork/netutils
    ../../libnetwork/netutils/utils_linux.go:66:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/netutils/utils_linux.go:75:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/api/server/router/grpc
    ../../api/server/router/grpc/grpc.go:56:48: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/container
    ../../container/view.go:335:47: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

    # github.com/docker/docker/libnetwork/ipams/defaultipam
    ../../libnetwork/ipams/defaultipam/address_space.go:33:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipams/defaultipam/address_space.go:53:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipams/defaultipam/address_space.go:124:10: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipams/defaultipam/address_space.go:125:21: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipams/defaultipam/address_space.go:146:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipams/defaultipam/address_space.go:310:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/ipams/defaultipam/address_space.go:311:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/libnetwork/drivers/bridge
    ../../libnetwork/drivers/bridge/port_mapping_linux.go:76:15: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/drivers/bridge/port_mapping_linux.go:201:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 14:59:54 +02:00
Sebastiaan van Stijn
b7d5a42168 Update go:build comments to go1.21
Match the minimum version that's specified on our vendor.mod.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 14:59:54 +02:00
Sebastiaan van Stijn
c0c0bed9ab Merge pull request #47970 from thaJeztah/replace_compatcontext
remove internal/compatcontext and use context instead
2024-06-13 14:43:29 +02:00
Sebastiaan van Stijn
5343c7b451 remove internal/compatcontext and use context instead
This internal package was added in f6e44bc0e8
to preserve compatibility with go1.20 and older. At the time, our vendor.mod
still had go1.18 as minimum version requirement (see [1]), which got updated to go1.20
in 16063c7456, and go1.21 in f90b03ee5d

The version of BuildKit we use already started using context.WithoutCancel,
without a fallback, so we no longer can provide compatibility with older
versions of Go, which makes our compatiblity package redundant.

This patch removes the package, and updates our code to use stdlib's context
instead.

[1]: f6e44bc0e8/vendor.mod (L7)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 13:29:39 +02:00
Sebastiaan van Stijn
13c3384303 hack/vendor.sh: remove redundant -compat 1.18
This was added to use a specific format for the vendor.mod/go.mod
file, but we should no longer need this, as go1.21 is now the
minimum.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-13 13:24:07 +02:00
Albin Kerouanton
f20fd3c8a0 golangci-lint: ignore ineffassign & staticcheck on ctx shadowing
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-13 11:29:35 +02:00
Rob Murray
52333f3a34 Merge pull request #47871 from robmry/portmapper_fixes_and_nonat
Portmapper improvements, and options to disable NAT
2024-06-13 09:12:53 +01:00
Rob Murray
09777ade5a Merge pull request #47963 from robmry/47773_remove_ipv6_disable_escape_hatch
Remove ipv6 disable escape hatch
2024-06-12 19:13:31 +01:00
Rob Murray
d0790fd03e Trivial tidying in osl.setIPv6()
- Removed unnecessary variable 'enable'.
- Replaced a couple of fmt's with string concatenation.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-12 16:53:15 +01:00
Rob Murray
b7f1036cd9 Remove the option to ignore failure to disable ipv6
26.1.1 added env var DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE to make it
possible to create an IPv4-only network, even with a read-only
"/proc/sys/net" that meant IPv6 could not be disabled on an
interface.

In 27.0 it's easier to enable IPv6, just '--ipv6' when creating the
network - in particular, there's no need to allocate a subnet, because
a unique-local prefix will be assigned by default).

So, this change removes the env-var workaround. Now, the workarounds
are to enable IPv6, mount "/proc/sys/net" read-write, disable IPv6
by default in OS configuration, or remove support for IPv6 from the
kernel.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-12 16:28:29 +01:00
Rob Murray
44d00e3b9b Dev container: try to load kernel module ip6_tables
On an nftables host, the ip6_tables kernel module may not be loaded,
but it needs to be for dockerd to run (with ip6tables now enabled by
default).

If ip6tables doesn't work, try the dind official image's trick for
loading the module using "ip link show".

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-12 14:34:20 +01:00
Sebastiaan van Stijn
4fec999c11 Merge pull request #47956 from thaJeztah/cleanup_getDefaultNetworkSettings
daemon: cleanup getDefaultNetworkSettings
2024-06-12 13:47:55 +02:00
Paweł Gronowski
bcd280a3ed Merge pull request #47959 from vvoland/buildkit-update
Dockerfile: update buildx to v0.15.0
2024-06-12 10:35:59 +02:00
Paweł Gronowski
d0a135772e Dockerfile: update buildx to v0.15.0
- 0.15.0 release notes: https://github.com/docker/buildx/releases/tag/v0.15.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-12 09:42:18 +02:00
Sebastiaan van Stijn
c6b12f72a0 Merge pull request #47954 from tonistiigi/update-buildkit-v0.14.0
vendor: update buildkit to v0.14.0
2024-06-11 23:46:56 +02:00
Sebastiaan van Stijn
215410316f daemon: cleanup getDefaultNetworkSettings
Small cleanup of this function;

- change to a regular function, as it does not depend on the daemon
- use an early return
- explicitly refer to EndpointSettings.EndpointSettings, not the top-
  level EndpointSettings.
- use a struct-literal.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-11 23:40:35 +02:00
Rob Murray
01eecb6cdf Validate port bindings for gateway_mode=routed
When bridge driver opt com.docker.network.bridge.gatway_mode_ipv[46]
is set to "routed", there is no NAT.

When there's no NAT, there's no meaning to the HostPort field in a
port mapping (all the port mapping does is open the container's port),
and the HostIP field is only used to determine the address family.

So, check port bindings, and raise errors if fields are unexpectedly
set when the mapping only applies to a gateway_mode=routed network.
Zero-addresses are allowed, to say the mapping/open-port should be
IPv4-only or IPv6-only, and host ports are not allowed.

A mapping with no host address, so it applies to IPv4 and IPv6 when
the default binding is 0.0.0.0, may include a host port if either
uses NAT. The port number is ignored for the directly-routed family.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 22:33:58 +01:00
Rob Murray
2a291c1855 Improve PortBinding.String()
Display a PortBinding in a format that's more like the one
used in the CLI, but includes the container IP if known.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 22:33:58 +01:00
Rob Murray
a1f8bbeeef Remove unused bridge.driver.portAllocator
It was added so that tests could replace it before it was picked
up and used by a new network's PortMapper, so that tests were isolated
from each other. Now the PortMapper is not used by the bridge driver,
neither is driver's portAllocator.

Instead of replacing the driver.portAllocator in tests, reset the
singleton instance using its ReleaseAll().

Un-export portallocator.NewInstance, now the tests aren't using it.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 22:33:58 +01:00
Rob Murray
043db3be68 Bind the same port for multiple addresses
Without this change, if a port mapping did not specify a host address
and the network was IPv6-enabled, the same port would be allocated for
mappings from '0.0.0.0' and '::'. But, if the port mapping was specified
with explicit addresses even, for example:
  -p 0.0.0.0:8080-8083:80 -p '[::]:8083-8080:80'

This change looks for port mappings that only differ in the host IP
address, and makes sure it allocates the same port for all of them. If
it can't, it fails with an error.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 22:33:58 +01:00
Rob Murray
20c99e4156 Option to disable NAT for IPv4/IPv6 for a bridge network.
Add bridge driver options...
  com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>
  com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>

If set to "routed", no NAT or masquerade rules are set up for port
mappings.

When NAT is disabled, the mapping is shown in 'inspect' output with
no host port number. For example, for "-p 80" with NAT disabled for
IPv6 but not IPv4:

    "80/tcp": [
        {
            "HostIp": "0.0.0.0",
            "HostPort": "32768"
        },
        {
            "HostIp": "::",
            "HostPort": ""
        }

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 22:33:58 +01:00
Tonis Tiigi
18ff5ef537 vendor: update buildkit to v0.14.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-06-11 12:08:34 -07:00
Rob Murray
e05848c002 Set up bridge-specific iptables rules in the bridge driver
Use the bridge driver's iptables types to set up portmapping related
iptables rules - instead of using iptables.Forward, which is bridge
specific code in the iptables package.

Remove iptables.Forward() and its unit test, the bridge driver's
version is covered by TestAddPortMappings.

Remove hairpinMode from iptables.ChainInfo hairpinMode relates to bridge
driver specific behaviour, that is now implemented in the bridge driver.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 16:50:16 +00:00
Rob Murray
4f09af6267 Allocate same port for IPv4/IPv6 for 'any interface' mappings.
The bridge driver now does its own port-mapping, rather than using the
portmapper module (which ran as two completely separate instances, for
IPv4 and IPv6).

When asked for a mapping from any host address (0.0.0.0/0) with a range
of host ports, the same port will be allocated for IPv4 and IPv6, or the
mapping will fail with an error if that's not possible.

The bridge driver now manages its own port mappings. So, remove
linux-specific PortMapper code and make what's left Windows-only.

Also, replace the portmapper.userlandProxy interface with StartProxy().

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 16:50:15 +00:00
Rob Murray
03577884d9 Retry port mapping for a range if ports are in-use
For a port mapping like '-p 8080-8083:80', when some non-docker process
is using a port in the range, try other ports in the range. And, don't
do that on live-restore.

Because the port mapping may fail on live-restore, leaving no ports
mapped for the endpoint - update the view of mapped ports shown in
'inspect' output. (The wrong mappings will still be shown in 'docker ps',
the container will be left running and connected to the network, it just
won't work. There's plenty of scope for better error handling here.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 16:50:15 +00:00
Rob Murray
931eea20ff Add portallocator.RequestPortsInRange()
Similar to portallocator.RequestPortInRange(), but it attempts to
allocate the same port for multiple IP addresses.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-11 16:50:15 +00:00
Sebastiaan van Stijn
87794b3030 Merge pull request #47946 from thaJeztah/remove_platforms_platform_alias
remove uses of platforms.Platform alias
2024-06-11 13:38:13 +02:00
Sebastiaan van Stijn
e0b762ed1b daemon/containerd: fix duplicate import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-11 11:49:57 +02:00
Sebastiaan van Stijn
7f2ed139fe remove uses of platforms.Platform alias
It's an alias for the OCI-spec type, which was only there for
convenience, but will be deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-11 11:48:01 +02:00
Sebastiaan van Stijn
9d9488468f Merge pull request #47943 from vvoland/c8d-multiplatform-push-2
c8d/push: Fix small whoopsies
2024-06-10 22:14:18 +02:00
Akihiro Suda
89431adcd1 Merge pull request #47500 from AkihiroSuda/fix-47499
seccomp: allow specifying a custom profile with `--privileged`
2024-06-11 05:07:53 +09:00
Akihiro Suda
896de6d426 seccomp: allow specifying a custom profile with --privileged
`--privileged --security-opt seccomp=<CUSTOM.json>` was ignoring
`<CUSTOM.json>`.

Fix issue 47499

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-06-11 03:37:54 +09:00
Sebastiaan van Stijn
22c212d208 Merge pull request #47941 from thaJeztah/api_image_inspect_deprecate_fields
api: deprecate erroneous Config fields in `GET /images/{name}/json` response
2024-06-10 19:19:04 +02:00
Paweł Gronowski
2ccce36d10 c8d/progress: Allow updating "Unavailable" ids
They might still change to "Mounted from" or "Already exists" when
containerd updates the status in tracker.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 19:14:23 +02:00
Paweł Gronowski
e2326c27b5 c8d/push: Fix wrong Originalindex descriptor in aux error
The target variable was already overwritten with the new value. Use the
original value instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 19:14:21 +02:00
Sebastiaan van Stijn
0566e38cbf Merge pull request #47605 from jonasgeiler/43626-rootless-native-overlay-diff
rootless: overlay2: support native overlay diff when using rootless-mode in kernel 5.11 and above
2024-06-10 18:57:19 +02:00
Sebastiaan van Stijn
1513068d8c Merge pull request #47679 from vvoland/c8d-multiplatform-push
c8d/push: Support `--platform` switch
2024-06-10 18:38:47 +02:00
Sebastiaan van Stijn
af0cdc36c7 api: deprecate erroneous Config fields in GET /images/{name}/json response
The `Config` field returned by this endpoint (used for "image inspect") returns
additional fields that are not part of the image's configuration and not part of
the [Docker Image Spec] and the [OCI Image Spec].

These additional fields are included in the response, due to an
implementation detail, where the [api/types.ImageInspec] type used
for the response is using the [container.Config] type.

The [container.Config] type is a superset of the image config, and while the
image's Config is used as a _template_ for containers created from the image,
the additional fields are set at runtime (from options passed when creating
the container) and not taken from the image Config.

These fields are never set (and always return the default value for the type),
but are not omitted in the response when left empty. As these fields were not
intended to be part of the image configuration response, they are deprecated,
and will be removed from the API.

The following fields are currently included in the API response, but
are not part of the underlying image's Config, and deprecated:

- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)

[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
[OCI Image Spec]: https://github.com/opencontainers/image-spec/blob/v1.1.0/specs-go/v1/config.go#L24-L62
[api/types.ImageInspec]: https://github.com/moby/moby/blob/v26.1.4/api/types/types.go#L87-L104
[container.Config]: https://github.com/moby/moby/blob/v26.1.4/api/types/container/config.go#L47-L82

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 16:56:17 +02:00
Sebastiaan van Stijn
a736d0701c Merge pull request #47936 from thaJeztah/api_types_container_types
api/types: move more types to sub-packages
2024-06-10 16:51:49 +02:00
Sebastiaan van Stijn
4d40d770cd Merge pull request #47942 from thaJeztah/api_swagger_update_image_config
docs: api: use separate definition for Image.Config (api v1.39 - v1.45)
2024-06-10 16:45:52 +02:00
Sebastiaan van Stijn
58641c7b5c docs: api: use separate definition for Image.Config (api v1.39 - v1.45)
The Image.Config field currently reuses the ContainerConfig definition,
matching the Go implementation, which also uses that type.

However, the ContainerConfig type contains various fields that are not
part of the image config, and would never be set. The Image.Config is
used as template / default values for containers started from the image,
but will only use the fields that are part of the [Docker image spec].

This patch updates the swagger files used in the documentation to use a
separate `ImageConfig` definition for the Image.Config field. The new
definition is a copy of the existing `ContainerConfig` type, but with
updated descriptions for fields, and with an example response that omits
the fields that should not be used.

The following fields are currently included in the `Config` field of the API
response, but are not part of the underlying image's config:

- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)

[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 15:38:18 +02:00
Sebastiaan van Stijn
5e0e34fafd api: swagger: use separate definition for Image.Config
The Image.Config field currently reuses the ContainerConfig definition,
matching the Go implementation, which also uses that type.

However, the ContainerConfig type contains various fields that are not
part of the image config, and would never be set. The Image.Config is
used as template / default values for containers started from the image,
but will only use the fields that are part of the [Docker image spec].

This patch updates the swagger files used in the documentation to use a
separate `ImageConfig` definition for the Image.Config field. The new
definition is a copy of the existing `ContainerConfig` type, but with
updated descriptions for fields, and with an example response that omits
the fields that should not be used.

The following fields are currently included in the `Config` field of the API
response, but are not part of the underlying image's config:

- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)

[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 15:37:27 +02:00
Sebastiaan van Stijn
a24e3f2ac6 Merge pull request #47799 from j2walker/47648-dameon-health-start-interval-default-value-fix
Changed default value of the startInterval to 5s
2024-06-10 14:36:02 +02:00
Paweł Gronowski
8d96d759bb c8d/image_manifest: IsPseudoImage return true for unknown/unknown platform
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:17 +02:00
Paweł Gronowski
b4d2283c89 api/push: Ignore Platform on older APIs
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:15 +02:00
Paweł Gronowski
68a63d0611 c8d/push: Extract missing content note to an Aux progress
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:14 +02:00
Paweł Gronowski
0a31437208 c8d/push: Support platform selection
Add a OCI platform fields as parameters to the `POST /images/{id}/push`
that allow to specify a specific-platform manifest to be pushed instead
of the whole image index.

When no platform was requested and pushing whole index failed, fallback
to pushing a platform-specific manifest with a best candidate (if it's
possible to choose one).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:13 +02:00
Paweł Gronowski
999f1c63db testutils/specialimage: Add MultiPlatform
Add utility that allows to construct an image with the specified
platforms.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:12 +02:00
Paweł Gronowski
c16d676266 c8d/blobsDirContentStore: Return ErrNotExists
Translate os.ErrNotExist into cerrdefs.ErrNotExists

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:11 +02:00
Paweł Gronowski
85249a8401 c8d/image_manifest: Add helper functions
This adds the common helper functions used by the recent
multiplatform-related PRs.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-10 13:35:10 +02:00
Jack Walker
c514952774 Changed default value of the startInterval to 5s
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Jack Walker <90711509+j2walker@users.noreply.github.com>
2024-06-10 13:23:26 +02:00
Paweł Gronowski
ac064904b8 Merge pull request #47927 from crazy-max/ci-buildkit-debug
ci: enable debug for buildkit container builder
2024-06-10 11:57:41 +02:00
Sebastiaan van Stijn
04110fa774 Merge pull request #47919 from laurazard/fix-deprecated-otel
otel: remove deprecated usages of `otelgrpc`
2024-06-10 11:53:26 +02:00
Sebastiaan van Stijn
6e514e8993 Merge pull request #47932 from thaJeztah/reexec_clean
pkg/reexec: cleanup and remove some dependencies
2024-06-10 11:31:01 +02:00
Sebastiaan van Stijn
aa22d137e9 Merge pull request #47937 from thaJeztah/client_fix_test_typos
client: fix typos in test-names and godoc
2024-06-10 10:37:28 +02:00
Sebastiaan van Stijn
b6ee4b66ad Merge pull request #47931 from thaJeztah/graphdriver_remove_Mounted
daemon/graphdriver: remove redundant Mounted function
2024-06-10 10:22:11 +02:00
Sebastiaan van Stijn
6c2934f373 api/types: move ImageLoadResponse to api/types/image
This moves the type, but we should consider removing this type, and just
returning an io.ReadCloser

This type was added in 9fd2c0feb0c131d01d727d50baa7183b976c7bdc;

> Make docker load to output json when the response content type is json
> Swarm hijacks the response from docker load and returns JSON rather
> than plain text like the Engine does. This makes the API library to return
> information to figure that out.

However the "load" endpoint unconditionally returns JSON;
7b9d2ef6e5/api/server/router/image/image_routes.go (L248-L255)

Commit 96d7db665b made the response-type depend
on whether "quiet" was set, but this logic got changed in a follow-up
2f27632cde, which made the JSON response-type
unconditionally, but the output produced depend on whether"quiet" was set.

We should deprecated the "quiet" option, as it's really a client
responsibility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:25 +02:00
Sebastiaan van Stijn
eb675cce71 api/types: move ImageImportSource to api/types/image
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:25 +02:00
Sebastiaan van Stijn
f6cc76ceb9 api/types: move ImageSearchOptions to api/types/registry
Note that RequestPrivilegeFunc could not be referenced, as it would
introduce a circular import, so copying the definition instead.

Also combining the other search-related types in the package to be in
the same file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:24 +02:00
Sebastiaan van Stijn
b5f15bc0aa api/types: move EventsOptions to api/types/events
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:24 +02:00
Sebastiaan van Stijn
ecb24afaaf api/types: move ImagesPruneReport to api/types/image
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:24 +02:00
Sebastiaan van Stijn
162ef4f8d1 api/types: move VolumesPruneReport to api/types/volume
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:24 +02:00
Sebastiaan van Stijn
17c3269a37 api/types: move ContainerStats to api/types/container
This is the response type; other types related to stats are left
for now, but should be moved (as well as utilities ported from
the CLI repository).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:21:22 +02:00
Sebastiaan van Stijn
fd1d8f323b api/types: move CopyToContainerOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:20:47 +02:00
Sebastiaan van Stijn
47d7c9e31d api/types: move ContainerPathStat to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:20:45 +02:00
Sebastiaan van Stijn
db2f1acd5d api/types: move ContainersPruneReport to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:47 +02:00
Sebastiaan van Stijn
5b27e71521 api/types: move ContainerExecInspect to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Sebastiaan van Stijn
d91638e295 api/types: move ExecStartCheck to api/types/container
This moves the type to api/types/container and creates an alias for
exec attach; ContainerExecAttach currently uses the same type as
ContainerExecStart, but does not all the same options (and some
options cannot be used).

We need to split the actual types, but lets start with aliasing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Sebastiaan van Stijn
452e134001 api/types: move ExecStartOptions to api/types/backend
It's a type used by the backend, so moving it there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Sebastiaan van Stijn
cd76e3e7f8 api/types: move ExecConfig to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-10 10:19:46 +02:00
Paweł Gronowski
3d2ee590a0 Merge pull request #47935 from thaJeztah/remove_TestContainerAPICopyNotExistsAnyMore
integration-cli: remove DockerAPISuite.TestContainerAPICopyNotExistsAnyMore
2024-06-10 10:18:00 +02:00
Sebastiaan van Stijn
8c34c63d81 Merge pull request #47939 from thaJeztah/api_remove_container_containerconfig
docs: api: image inspect: remove Container and ContainerConfig
2024-06-10 10:06:08 +02:00
Sebastiaan van Stijn
3434a8ef6e Merge pull request #47938 from thaJeztah/vendor_pty
vendor: github.com/creack/pty v1.1.21
2024-06-10 10:05:43 +02:00
Sebastiaan van Stijn
e314cbdab8 Merge pull request #47862 from thaJeztah/exec_router_nits
api/server/router/container: minor nits in exec router
2024-06-10 10:05:18 +02:00
Sebastiaan van Stijn
ac27a5379b docs: api: image inspect: remove Container and ContainerConfig
The Container and ContainerConfig fields have been deprecated, and removed
since API v1.45 in commit 03cddc62f4.

This patch fixes the swagger and documentation to no longer mention them
as they are no longer returned by API v1.45 and higher.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-09 23:17:37 +02:00
Sebastiaan van Stijn
52580b2673 vendor: github.com/creack/pty v1.1.21
full diff: https://github.com/creack/pty/compare/v1.1.18...v1.1.21

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-09 22:39:33 +02:00
Sebastiaan van Stijn
fa95f8a070 client: fix typos in test-names and godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-09 22:35:04 +02:00
Sebastiaan van Stijn
311c949871 Merge pull request #47731 from racequite/master
chore: fix function names in comment
2024-06-09 22:33:09 +02:00
Akihiro Suda
7b9d2ef6e5 Merge pull request #47934 from thaJeztah/vendor_reference_0.6
vendor: github.com/distribution/reference v0.6.0
2024-06-09 05:15:13 +09:00
Akihiro Suda
59875a9218 Merge pull request #47933 from thaJeztah/bump_bbolt_1.3.10
vendor: go.etcd.io/bbolt v1.3.10
2024-06-09 05:14:57 +09:00
Sebastiaan van Stijn
08939f21ad integration-cli: remove DockerAPISuite.TestContainerAPICopyNotExistsAnyMore
This test was added in 428328908dc529b1678fb3d8b033fb0591a294e3;

> Deprecate /containers/(id or name)/copy endpoint
> This endpoint has been deprecated since 1.8. Return an error starting
> from this API version (1.24) in order to make sure it's not used for the
> next API version and so that we can remove it sometimes later.

We deprecated and removed those older API versions, and the test was
effectively only verifying that a non-existing endpoint returns a 404,
so let's remove it.

This also removes api/types.CopyConfig, which was only used in this
test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 21:00:25 +02:00
Sebastiaan van Stijn
75843620a6 api/server/router/container: minor nits in exec router
- remove intermediate variable
- format a "todo" comment as an actual todo ':)
- explicitly suppress some unhandled errors to keep linters happy

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 20:54:38 +02:00
Sebastiaan van Stijn
18e1afd1a1 vendor: github.com/distribution/reference v0.6.0
full diff: https://github.com/distribution/reference/compare/v0.5.0...v0.6.0

- remove deprecated SplitHostname
- refactor splitDockerDomain to include more documentation
- fix typo in readme
- Exclude domain from name length check

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 17:51:50 +02:00
Sebastiaan van Stijn
7529e95f6f Merge pull request #47918 from robmry/allow_startup_with_no_ip6tables
Allow startup with no kernel support for ip6_tables
2024-06-08 16:10:54 +02:00
Sebastiaan van Stijn
7501b90a22 vendor: go.etcd.io/bbolt v1.3.10
- Remove deprecated UnsafeSlice and use unsafe.Slice
- Stabilize the behaviour of Prev when the cursor already points to
  the first element
    - Fix Cursor.Prev() out of range issues in v1.3.9
    - Relates to boltdb/bolt/issues/357 (Cursor inconsistent when mixing
      cursor.Delete() with Put() in same transaction)
- Bump go version to 1.21.9

full diff: https://github.com/etcd-io/bbolt/compare/v1.3.9...v1.3.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 15:16:45 +02:00
Sebastiaan van Stijn
cf796aa56a pkg/reexec: touch-up GoDoc, and remove "import" comments
Touch-up some GoDoc in the package, and remove "import" comments.

This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. The "import" comments are ignored when
used in go module mode so have little benefit. Let's remove them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 14:18:13 +02:00
Sebastiaan van Stijn
d20a074f33 pkg/reexec: remove gotest.tools from tests
This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. While it's not too problematic to have
this dependency, the tests only used basic assertions from gotest.tools,
which could be easily re-implemented without the dependency.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 14:18:12 +02:00
Sebastiaan van Stijn
defd5a08f4 pkg/reexec: unify non-Linux implementation of Command
The Windows, Darwin, and FreeBSD implementations were identical, other
than their GoDoc to be different. Unify them so that we don't have to
maintain separate GoDoc for each.

It's worth noting that FreeBSD also supports Pdeathsig, so could be
using the same implementation as Linux. However, we don't test/maintain
the FreeBSD implementation, and it would require updating to GoDoc to
be more specific about the use of `/proc/self/exe`, so keeping the
status quo for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 14:18:12 +02:00
Sebastiaan van Stijn
004451c812 pkg/reexec: unify implementation of Self() and remove stub
This combines the implementations of the Self function, to allow having
a single GoDoc to document the behavior. The naiveSelf function is kept,
because it's used in unit-tests.

There is a minor change in behavior, as this patch removes the stub for
unsupported platforms (non-linux, windows, freebsd or darwin), which will
now use `os.Args[0]`. The stub was added in 21537b818d
to fix compilation of https://github.com/ethereum/go-ethereum on OpenBSD,
which had docker/docker as dependency. It looks like that repository no
longer has this dependency, and as this was only to make the code
compilable, is unlikely to be a problem.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 14:18:11 +02:00
Sebastiaan van Stijn
79bff9461c pkg/reexec: don't mix syscall and golang.org/x/sys package
commit 069fdc8a08 changed most uses of
the syscall package to switch utsname from unsigned to signed (see
069fdc8a08). Those don't seem to be
impacting the code used here, so either stdlib or golang.org/x/sys/unix
should work for this case.

I chose stdlib's syscall package for this case, in case we'd decide to
move this package to a separate module (and want to limit its dependencies).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 14:18:11 +02:00
Sebastiaan van Stijn
a445f7fa8a daemon/graphdriver: fix GoDoc for ProtoDriver.GetMetadata
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 12:20:55 +02:00
Sebastiaan van Stijn
a76b768fea daemon/graphdriver: remove redundant Mounted function
This function largely identical to GetFSMagic, except for suppressing
ENOENT errors. The only consumer of this function was fsChecker.IsMounted,
which would ignore errors either way, and only use the "success" case to
check if the detected filesystem-type was the expected one.

This patch;

- rewrites fsChecker.IsMounted to use GetFSMagic instead
- removes the now unused Mounted function

As we consider daemon/graphdriver to be "internal", and as there are no
public consumers of this, we can remove this function without deprecating
first.

The freebsd implementation also seemed to be broken, as it mixed syscall
with golang.org/x/sys/unix, which used incompatible types. I left the file
in place for now, but we can consider removing it altogether as there's no
active development on making freebsd functional.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-08 12:13:56 +02:00
Rob Murray
837b3f9576 Allow startup with no kernel support for ip6_tables
Before "ip6tables" was enabled by default, dockerd would start normally
when:
- the kernel had no IPv6 support, or
- docker is running as docker-in-docker, and the host doesn't have kernel
  module 'ip6_tables' loaded.

Now, the bridge driver will try to set up its ip6tables chains and it'll
fail. By not treating that as an error, the daemon will start and IPv4
will work normally.

A subsequent attempt to create an IPv6 network will fail with an error
about ip6tables. At that point, the user's options are:
- set "ip6tables":false in daemon config
- in the DinD case, "modprobe ip6_tables" on the host, or start dockerd
  on the host with ip6tables enabled (causing the kernel module load).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-07 17:40:38 +01:00
Sebastiaan van Stijn
4fb17cb3af Merge pull request #47928 from akerouanton/rob-maintainer
Add Rob Murray (@robmry) as maintainer
2024-06-07 17:54:39 +02:00
Albin Kerouanton
4df4f83f23 Merge pull request #47926 from akerouanton/revert-47837
Revert "libnet/i/defaultipam: Disambiguate PoolID string format"
2024-06-07 16:32:39 +02:00
Albin Kerouanton
fd3fa4b28a Add Rob Murray (@robmry) as maintainer
Rob is currently a curator, and has been actively contributing to this
repo for 7 months now.

Beside day-to-day triaging and bug fixing, Rob is an instrumental
contributor to libnetwork, and amongst other things, to the ongoing work
on IPv6 improvements.

I nominated Rob as maintainer, and votes passed, so opening a PR to make
it official.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-07 16:18:46 +02:00
Laura Brehm
854130eee0 deps: remove grpc-ecosystem/go-grpc-middleware
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-06-07 14:35:52 +01:00
Laura Brehm
49ca0d0d03 otel: remove deprecated usages of otelgrpc
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-06-07 14:35:51 +01:00
Albin Kerouanton
1243f9da6d Revert "libnet/i/defaultipam: Disambiguate PoolID string format"
This reverts commit 9369132879.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-07 15:32:13 +02:00
Albin Kerouanton
6af0646236 Revert "libnet/i/defaultipam: Use InternalErrorf instead of InvalidParameterErrof"
This reverts commit 5a2fa59688.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-07 15:31:54 +02:00
CrazyMax
4aa85cd159 ci: enable debug for buildkit container builder
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-06-07 15:30:22 +02:00
Sebastiaan van Stijn
59996a493c Merge pull request #47866 from cncal/return_container_annotations
api/server: ContainerList returns container annotations
2024-06-07 14:14:12 +02:00
Sebastiaan van Stijn
aa2c7de9b9 Merge pull request #47924 from thaJeztah/api_checkduplicates_optional
api/types/network: make CheckDuplicate optional
2024-06-07 13:35:07 +02:00
Sebastiaan van Stijn
fc9dd6acb4 api/types/network: make CheckDuplicate optional
The CheckDuplicate option is no longer part of the current API; it's
only used by the client when connecting to old API versions, which need
to have this field set.

This patch:

- Removes the CheckDuplicate from the API documentation, as the API
  describes the current version of the API (which does not have this
  field).
- Moves the CheckDuplicate field to the CreateRequest type; this is
  the type used for the network create request. The CheckDuplicate
  is not an option that's set by the user, and set internally by
  the client, so removing it from the CreateOptions struct moves
  it entirely internal.
- Change the CheckDuplicate field to be a pointer; this makes the
  "omitempty" become active, and the client will no longer include
  the field in the request JSON unless it's set (API < 1.44).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-07 12:06:39 +02:00
Sebastiaan van Stijn
f6d8ac04ba Merge pull request #47921 from thaJeztah/api_move_network_create
api/types: move NetworkCreate, NetworkCreateRequest, NetworksPruneReport to api/types/network
2024-06-07 12:06:28 +02:00
Sebastiaan van Stijn
e5f9484ab6 api/types: move NetworksPruneReport to api/types/network
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-07 11:14:52 +02:00
Sebastiaan van Stijn
ad6edc139f api/types: move NetworkCreate, NetworkCreateRequest to api/types/network
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-07 11:11:15 +02:00
Sebastiaan van Stijn
b75bca3868 Merge pull request #47922 from tonistiigi/20240606-update-buildkit
vendor: update buildkit to v0.14.0-rc2
2024-06-07 08:47:43 +02:00
Tonis Tiigi
4f61fa21cb vendor: update buildkit to v0.14.0-rc2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-06-06 18:59:02 -07:00
cncal
ca0529f984 api/server: ContainerList returns container annotations
Allow clients (e.g. cri-dockerd) to fetch container annotations in
ContainerList api.

Signed-off-by: cncal <flycalvin@qq.com>
2024-06-07 09:50:11 +08:00
Sebastiaan van Stijn
181e70cc07 Merge pull request #47920 from thaJeztah/bump_appengine
vendor: google.golang.org/appengine v1.6.8
2024-06-06 20:47:26 +02:00
Sebastiaan van Stijn
00f18ef7a4 Merge pull request #47867 from akerouanton/api-EnableIPv6-override
api: Make EnableIPv6 optional (impl #1 - pointer-based)
2024-06-06 20:20:29 +02:00
Sebastiaan van Stijn
6f32dc19f1 Merge pull request #47683 from vvoland/buildkit-update
vendor: github.com/moby/buildkit v0.14.0-rc2-dev
2024-06-06 16:59:14 +02:00
Sebastiaan van Stijn
b3c8216873 vendor: google.golang.org/appengine v1.6.8
removes use of the deprecated "golang.org/x/net/context" package

full diff: https://github.com/golang/appengine/compare/v1.6.7...v1.6.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-06 15:28:58 +02:00
Paweł Gronowski
0b5f7b9ff8 builder-next: Adjust NewGatewayFrontend invocation
b5c50afa882e2b34aba880fd5028615e2ef94e07 changed the signature of
NewGatewayFrontend to include a slice of allowed repositories.

Docker does not allow to specify this option, so don't place any
restrictions by passing an empty slice.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:18 +02:00
Jonathan A. Sternberg
fa03db1b82 builder: Update detect usage for new detect API from buildkit
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:17 +02:00
Paweł Gronowski
995604236e builder: Adjust usage of shlex.ProcessWord
1b1c5bc08ad81add007eb647e66ed0929693f3a0 extended the function signature
with one additional return value.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:16 +02:00
Paweł Gronowski
438371e1fe builder: Pass nil linter to instructions.Parse
eea0b41bf4fb1d69e109ff5ff8045c63f0c0d510 added a new argument to
`instructions.Parse` to support issuing linter warnings.

Classic builder uses it to parse the Dockerfile instructions and its
usage needs adjustment.

The classic builder is deprecated and we won't be adding any new
features to it, so we just pass a nil linter callback.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:15 +02:00
Paweł Gronowski
3bcbb561ad vendor: update github.com/moby/buildkit to v0.14.0-rc2-dev
- full diff: https://github.com/moby/buildkit/compare/v0.13.1...v0.14.0-rc2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-06-06 11:20:13 +02:00
Albin Kerouanton
f3f20c3a86 Merge pull request #47602 from robmry/internal_resolver_for_default_bridge
Add resolver for default bridge, remove default nameservers
2024-06-06 10:39:24 +02:00
Rob Murray
d365702dbd No default nameservers for internal resolver
Don't fall-back to Google's DNS servers in a network that has an
internal resolver.

Now the default bridge uses the internal resolver, the only reason a
network started by the daemon should end up without any upstream
servers is if the host's resolv.conf doesn't list any.  In this case,
the '--dns' option can be used to explicitly configure nameservers
for a container if necessary.

(Note that buildkit's containers do not have an internal resolver, so
they will still set up Google's nameservers if the host has no
resolvers that can be used in the container's namespace.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-05 20:27:24 +01:00
Rob Murray
18f4f775ed Internal resolver for default bridge network
Until now, containers on the default bridge network have been configured
to talk directly to external DNS servers - their resolv.conf files have
either been populated with nameservers from the host's resolv.conf, or
with servers from '--dns' (or with Google's nameservers as a fallback).

This change makes the internal bridge more like other networks by using
the internal resolver.  But, the internal resolver is not populated with
container names or aliases - it's only for external DNS lookups.

Containers on the default network, on a host that has a loopback
resolver (like systemd's on 127.0.0.53) will now use that resolver
via the internal resolver. So, the logic used to find systemd's current
set of resolvers is no longer needed by the daemon.

Legacy links work just as they did before, using '/etc/hosts' and magic.

(Buildkit does not use libnetwork, so it can't use the internal resolver.
But it does use libnetwork/resolvconf's logic to configure resolv.conf.
So, code to set up resolv.conf for a legacy networking without an internal
resolver can't be removed yet.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-06-05 20:27:24 +01:00
Albin Kerouanton
c6aaabc9fc Merge pull request #47887 from thaJeztah/move_more_network_api_types_2
api/types: migrate NetworkResource to api/types/network
2024-06-05 15:48:24 +02:00
Sebastiaan van Stijn
c5c4abbf59 Merge pull request #47909 from thaJeztah/vendor_containerd_1.7.18
vendor: github.com/containerd/containerd v1.7.18
2024-06-05 12:33:04 +02:00
Paweł Gronowski
1a4efd2c74 Merge pull request #47910 from thaJeztah/bump_containerd_binary_1.7.18
update containerd binary to v1.7.18
2024-06-05 11:56:40 +02:00
Sebastiaan van Stijn
5318c38eae update containerd binary to v1.7.18
Update the containerd binary that's used in CI and for the static packages.

- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.18
- full diff: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18

Welcome to the v1.7.18 release of containerd!

The eighteenth patch release for containerd 1.7 contains various updates along
with an updated version of Go. Go 1.22.4 and 1.21.11 include a fix for a symlink
time of check to time of use race condition during directory removal.

Highlights

- Update Go version to 1.21.11
- Remove uses of platforms.Platform alias
- Migrate log imports to github.com/containerd/log
- Migrate errdefs package to github.com/containerd/errdefs
- Fix usage of "unknown" platform

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-05 10:18:42 +02:00
Paweł Gronowski
189dc1b47e Merge pull request #47902 from thaJeztah/bump_go1.21.11
update to go1.21.11
2024-06-05 10:14:19 +02:00
Sebastiaan van Stijn
86f7762d48 vendor: github.com/containerd/containerd v1.7.18
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.

This patch:

- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to prevent using the "log" package, which was also
  migrated to a separate module.

There are still some uses of the old package in (indirect) dependencies,
which should go away over time.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-05 09:21:00 +02:00
Sebastiaan van Stijn
91e2c29865 update to go1.21.11
go1.21.11 (released 2024-06-04) includes security fixes to the archive/zip
and net/netip packages, as well as bug fixes to the compiler, the go command,
the runtime, and the os package. See the Go 1.21.11 milestone on our issue
tracker for details;

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.11+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.10...go1.21.11

From the security announcement;

We have just released Go versions 1.22.4 and 1.21.11, minor point releases.
These minor releases include 2 security fixes following the security policy:

- archive/zip: mishandling of corrupt central directory record

  The archive/zip package's handling of certain types of invalid zip files
  differed from the behavior of most zip implementations. This misalignment
  could be exploited to create an zip file with contents that vary depending
  on the implementation reading the file. The archive/zip package now rejects
  files containing these errors.

  Thanks to Yufan You for reporting this issue.

  This is CVE-2024-24789 and Go issue https://go.dev/issue/66869.

- net/netip: unexpected behavior from Is methods for IPv4-mapped IPv6 addresses

  The various Is methods (IsPrivate, IsLoopback, etc) did not work as expected
  for IPv4-mapped IPv6 addresses, returning false for addresses which would
  return true in their traditional IPv4 forms.

  Thanks to Enze Wang of Alioth and Jianjun Chen of Zhongguancun Lab
  for reporting this issue.

  This is CVE-2024-24790 and Go issue https://go.dev/issue/67680.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-04 23:23:27 +02:00
Albin Kerouanton
216e426ec2 Merge pull request #47901 from thaJeztah/cluster_remove_getRequestContext
daemon/cluster: remove Cluster.getRequestContext()
2024-06-04 22:18:50 +02:00
Albin Kerouanton
163c6ca9ad api: Make EnableIPv6 optional
Currently, starting dockerd with
`--default-network-opt=bridge=com.docker.network.enable_ipv6=true` has
no effect as `NetworkCreateRequest.EnableIPv6` is a basic bool.

This change makes it a `*bool` to make it optional. If clients don't
specify it, the default-network-opt will be applied.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-06-04 21:36:18 +02:00
Sebastiaan van Stijn
83c532c9b7 daemon/cluster: remove Cluster.getRequestContext()
This method was added in 534a90a993 as
part of adding the Swarm cluster backend, and later updated in commit
85b1fdf15c to use a swarmRequestTimeout
const for the timeout.

Nothing in this utility depends on the Cluster struct, and the abstraction
makes it appear as more than it is, which is just a wrapper for
context.WithTimeout().

Let's remove the abstraction to make it less magical.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-04 18:09:58 +02:00
Sebastiaan van Stijn
69b2a05d27 api/types: migrate NetworkResource to api/types/network
This moves the type to the api/types/network package, but also introduces
a "Summary" alias; the intent here is to allow diverging the types used
for "list" and "inspect" operations, as list operations may only be
producing a subset of the fields available.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-04 17:44:46 +02:00
Sebastiaan van Stijn
a865453b1a Merge pull request #47900 from thaJeztah/api_types_rm_deprecated_aliases
api/types: remove aliases for deprecated Image types
2024-06-04 17:17:32 +02:00
Sebastiaan van Stijn
3306034c64 api/types: remove aliases for deprecated Image types
These aliases were added in ac2a028dcc,
which was part of the v26.0 and v26.1 releases. We can remove the
aliases, assuming users that depended on this have migrated to the
new location of these types.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-04 16:01:22 +02:00
Sebastiaan van Stijn
989426d303 Merge pull request #47897 from thaJeztah/rm_deprecated_inspectoptions
integration/network: remove used of deprecated NetworkInspectOptions
2024-06-04 14:13:51 +02:00
Sebastiaan van Stijn
70bac42113 integration/network: remove used of deprecated NetworkInspectOptions
The types.NetworkInspectOptions type was moved to the networks package
in 5bea0c38bc and deprecated, but use of it
was re-introduced in cd3804655a, which was
merged out-of-order.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-04 11:53:00 +02:00
Albin Kerouanton
cd3804655a Merge pull request #47853 from akerouanton/libnet-ipam-default-ula
libnet/i/defaultipam: use ULA prefix by default
2024-06-04 00:05:41 +02:00
Lei Jitang
58aac7773d Merge pull request #47888 from thaJeztah/opts_remove_alias
opts: remove alias for ipamutils
2024-06-03 19:17:07 +08:00
Sebastiaan van Stijn
8c400f4f37 Merge pull request #47886 from thaJeztah/cleanup_httpstatus_fromerror
api/server/httpstatus: FromError: remove redundant checks and cleanup
2024-06-03 12:35:55 +02:00
Sebastiaan van Stijn
1ec92ea60b opts: remove alias for ipamutils
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-03 12:27:40 +02:00
Sebastiaan van Stijn
1f359403fe Merge pull request #47881 from thaJeztah/client_cleanups
client: minor cleanup, linting- and bug-fixes
2024-06-03 11:43:27 +02:00
Sebastiaan van Stijn
e6f41e22a7 client: Client.doRequest: fix closing filehandle and reversed errors
commit 1a5dafb31e improved the error messages
produced by adding a check if the client is using as an elevated user. For
this, it attempts to open `\\.\PHYSICALDRIVE0`.

However, it looks like closing the file landed in the wrong branch of the
condition, so the file-handle would not be closed when the os.Open succeeded.

Looking further into this check, it appears the conditions were reversed;
if the check _fails_, it means the user is not running with elevated
permissions, but the check would use elevatedErr == nil.

Fix both by changing the condition to `elevatedErr != nil`.

While at it, also changing the string to use a string-literal, to reduce
the amount of escaping needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-03 10:52:15 +02:00
Sebastiaan van Stijn
9110ef1eec client: ensureReaderClosed: make linters happier
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-03 10:52:15 +02:00
Sebastiaan van Stijn
57f597b866 client: Client.NetworkInspectWithRaw: minor cleanup
Make this code slightly more idiomatic, and make it clear in what cases
we don't return an actual response, but an empty / default struct.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-03 10:52:15 +02:00
Sebastiaan van Stijn
42f4db26c7 api/server/httpstatus: FromError: remove redundant checks and cleanup
- remove redundant `if statusCode == 0 {` check, which would always be true
- use early returns in the switch
- move all conditions into the switch, and scope the `statusCode` variable
  to conditions where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-03 09:47:50 +02:00
Sebastiaan van Stijn
4318ab0b33 Merge pull request #47882 from thaJeztah/move_more_network_api_types
api/types: migrate NetworkListOptions to api/types/network
2024-06-01 15:50:40 +02:00
Sebastiaan van Stijn
f78dac35e5 api/types: migrate NetworkListOptions to api/types/network
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-31 17:41:31 +02:00
Sebastiaan van Stijn
e622cea556 Merge pull request #47873 from thaJeztah/move_more_network_api_types
api/types: move more networking-types to api/types/network
2024-05-30 21:56:42 +02:00
Albin Kerouanton
d18b88fd32 daemon: add ULA prefix by default
So far, Moby only had IPv4 prefixes in its 'default-address-pools'. To
get dynamic IPv6 subnet allocations, users had to redefine this
parameter to include IPv6 base network(s). This is needlessly complex
and against Moby's 'batteries-included' principle.

This change generates a ULA base network by deriving a ULA Global ID
from the Engine's Host ID and put that base network into
'default-address-pools'. This Host ID is stable over time (except if
users remove their '/var/lib/docker/engine-id') and thus the GID is
stable too.

This ULA base network won't be put into 'default-address-pools' if users
have manually configured it.

This is loosely based on https://datatracker.ietf.org/doc/html/rfc4193#section-3.2.2.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-29 22:38:27 +02:00
Sebastiaan van Stijn
68bf0e7625 api/types: migrate EndpointResource to api/types/network
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-29 19:22:37 +02:00
Sebastiaan van Stijn
5bea0c38bc api/types: migrate NetworkInspectOptions to api/types/network
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-29 19:22:37 +02:00
Sebastiaan van Stijn
245d12175f api/types: migrate NetworkConnect, NetworkDisconnect to api/types/network
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-29 19:22:37 +02:00
Sebastiaan van Stijn
89624e09e6 api/types: migrate NetworkCreateResponse to network.CreateResponse
Migrate the type to the network package, and generate it from swagger.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-29 19:22:34 +02:00
Albin Kerouanton
2ebf19129f Merge pull request #47686 from robmry/47639_per-interface-sysctls
Per-interface sysctls
2024-05-29 10:54:05 +02:00
Albin Kerouanton
32418e9753 daemon: set the default local addr pool if none configured
Until this commit, the default local address pool was initialized by the
defaultipam driver if none was provided by libnet / the daemon.

Now, defaultipam errors out if none is passed and instead the daemon is
made responsible for initializing it with the default values if the user
don'te set the related config parameter.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-29 10:47:59 +02:00
Rob Murray
0071832226 Add per-endpoint sysctls to DriverOpts
Until now it's been possible to set per-interface sysctls using, for
example, '--sysctl net.ipv6.conf.eth0.accept_ra=2'. But, the index in
the interface name is allocated serially, and the numbering in a container
with more than one interface may change when a container is restarted.
The change to make it possible to connect a container to more than one
network when it's created increased the ambiguity.

This change adds label "com.docker.network.endpoint.sysctls" to the
DriverOpts in EndpointSettings. This option is explicitly associated
with the interface.

Settings in "--sysctl" for "eth0" are migrated to DriverOpts.

Because using "--sysctl" with any interface apart from "eth0" would have
unpredictable results, it is now an error to use any other interface name
in the top level "--sysctl" option. The error message includes a hint at
how to use the new per-interface setting.

The per-endpoint sysctl name has the interface name replaced by
"IFNAME". For example:
    net.ipv6.conf.eth0.accept_ra=2
becomes:
    net.ipv6.conf.IFNAME.accept_ra=2

The value of DriverOpts["com.docker.network.endpoint.sysctls"] is a
comma separated list.

Settings from '--sysctl' are applied by the runtime lib during task
creation. So, task creation fails if the endpoint does not exist.
Applying per-endpoint settings during interface configuration means the
endpoint can be created later, which paves the way for removal of the
SetKey OCI prestart hook.

Unlike other DriverOpts, the sysctl label itself is not driver-specific,
but each driver has a chance to check settings/values and raise an error
if a setting would cause it a problem - no such checks have been added
in this initial version. As a future extension, if required, it would be
possible for the driver to echo back valid/extended/modified settings to
libnetwork for it to apply to the interface. (At that point, the syntax
for the options could become driver specific to allow, for example, a
driver to create more than one interface).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-29 08:59:48 +01:00
Sebastiaan van Stijn
56a43a7618 Merge pull request #47865 from thaJeztah/api_docs_network_config_only
api: networking: document Scope, ConfigOnly, ConfigFrom, Peers
2024-05-28 23:52:05 +02:00
Sebastiaan van Stijn
ce0ccc09ff Merge pull request #45313 from akerouanton/deprecate-cors-headers
Deprecate dockerd api-cors-header parameter
2024-05-28 23:42:53 +02:00
Sebastiaan van Stijn
4e07c49336 Merge pull request #47092 from thaJeztah/bump_docker_py
update docker-py to 7.1.0
2024-05-28 20:42:00 +02:00
Sebastiaan van Stijn
e5532c52aa Merge pull request #47861 from thaJeztah/integration_nits
integration/system, integration/container: minor cleanups
2024-05-28 19:56:41 +02:00
Sebastiaan van Stijn
c96a2dbb54 Merge pull request #47860 from thaJeztah/test_nits
integration/internal/swarm, testutil/fakestorage: fix minor (linting) issues
2024-05-28 19:55:06 +02:00
Bjorn Neergaard
9a7d8c8660 Merge pull request #47863 from thaJeztah/platforms_err_handling
don't depend on containerd platform.Parse to return a typed error
2024-05-28 08:01:52 -06:00
Sebastiaan van Stijn
347bb4122a update docker-py to 7.1.0
full diff: https://github.com/docker/docker-py/compare/7.0.0...7.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-28 11:38:54 +02:00
Sebastiaan van Stijn
69f1e2a923 docs/api: add Scope, ConfigOnly, ConfigFrom, Peers (v1.41 - v1.45)
- api: swagger: Network: inline examples, and add ConfigOnly, ConfigFrom

  These fields were added in 9ee7b4dda9, but
  not documented in the API docs / swagger.

  Also move the example values per-field to reduce the risk of the example
  given from diverging with the actual struct that's used for the request.

- api: swagger: POST /networks/create: document Scope, ConfigOnly, ConfigFrom

  Adds missing documentation for Scope, ConfigOnly, and ConfigFrom. The ConfigOnly
  and ConfigFrom fields were added in 9ee7b4dda9,
  but not documented in the API docs / swagger.

- api: swagger: Network: add Peers

  Add documentation for the Peers field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 21:30:28 +02:00
Sebastiaan van Stijn
53542fefd5 api: swagger: Network: add Peers
Add documentation for the Peers field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 21:28:48 +02:00
Sebastiaan van Stijn
44125530bf api: swagger: POST /networks/create: document Scope, ConfigOnly, ConfigFrom
Adds missing documentation for Scope, ConfigOnly, and ConfigFrom. The ConfigOnly
and ConfigFrom fields were added in 9ee7b4dda9,
but not documented in the API docs / swagger.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 21:28:48 +02:00
Sebastiaan van Stijn
8b7a54f622 api: swagger: Network: inline examples, and add ConfigOnly, ConfigFrom
These fields were added in 9ee7b4dda9, but
not documented in the API docs / swagger.

Also move the example values per-field to reduce the risk of the example
given from diverging with the actual struct that's used for the request.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 21:28:41 +02:00
Sebastiaan van Stijn
6cab6d0302 docs/api: POST /networks/create: inline examples per-field (v1.41 - v1.45)
Move the example values per-field to reduce the risk of the example given
from diverging with the actual struct that's used for the request.

This patch updates older API versions (went back to v1.41).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 17:49:06 +02:00
Sebastiaan van Stijn
51885166b9 api: swagger: POST /networks/create: inline examples per-field
Move the example values per-field to reduce the risk of the example given
from diverging with the actual struct that's used for the request.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 17:36:05 +02:00
Sebastiaan van Stijn
d64e220afb api/types: NetworkCreate: add GoDoc
GoDoc is mostly copied from NetworkResource, which is the equivalent for
retrieving the information.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 16:25:18 +02:00
Sebastiaan van Stijn
cd1ed46d73 don't depend on containerd platform.Parse to return a typed error
We currently depend on the containerd platform-parsing to return typed
errdefs errors; the new containerd platforms module does not return such
errors, and documents that errors returned should not be used as sentinel
errors; c1438e911a/errors.go (L21-L30)

Let's type these errors ourselves, so that we don't depend on the error-types
returned by containerd, and consider that eny platform string that results in
an error is an invalid parameter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-27 10:18:29 +02:00
Sebastiaan van Stijn
56086c9952 integration/container: remove redundant type-conversion, and minor cleanup
- Remove redundant conversion to strslice.StrSlice
- Use assert.Check where possible to not fail early
- Remove instances of types.ExecStartCheck that used default values
- Minor code-formatting cleanup

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-25 14:58:52 +02:00
Sebastiaan van Stijn
070e90a94c integration/system: remove redundant type-conversion, and minor cleanup
- Remove redundant conversion to strslice.StrSlice
- Use assert.Assert instead of assert.Check to fail early if value is nil
- Minor code-formatting cleanup

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-25 14:56:44 +02:00
Sebastiaan van Stijn
667094924d testutil/fakestorage: fix minor (linting) issues
- fix typo in comment
- rename variable that collided with an import
- add log for an unhandled error
- slightly improve error-logs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-25 14:00:09 +02:00
Sebastiaan van Stijn
4a074c809e integration/internal/swarm: remove unused ContainerPoll
This was added in ee6959addc to account
for arm (32) requiring a longer timeout at the time, but it was never
used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-25 13:57:39 +02:00
Sebastiaan van Stijn
fe78d6d9da integration/internal/swarm: rename vars that collided with imports
- rename the client var to not collide with the imported client package
- remove an intermediate startCheck variable

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-25 13:55:59 +02:00
Brian Goff
4619b14403 refactor: rename attach config var
This var for the incoming attach request.
Just within this one function we also have `cfg`, and `ctr` already, so
`c` just makes things more confusing.
Not to mention `c` is usually referencing a container object in other
parts of the code.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-24 21:32:18 +00:00
Brian Goff
2d134c5abd Fix goroutine/fd leak when client disconnects
In cases where the client disconnects and there is nothing to read from
a stdio stream after that disconnect, the copy goroutines and file
descriptors are leaked because `io.Copy` is just blocked waiting for
data from the container's I/O stream.

This fix only applies to Linux.
Windows will need a separate fix.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-24 21:32:15 +00:00
Sebastiaan van Stijn
ceefb7d0b9 Merge pull request #47855 from thaJeztah/bump_buildx
Dockerfile: update buildx to v0.14.1, compose v2.27.1
2024-05-24 19:00:37 +02:00
Albin Kerouanton
62ddd3dea8 Merge pull request #47747 from robmry/non-experimental-ip6tables
Enable 'ip6tables' by default, don't require 'experimental'.
2024-05-24 18:38:16 +02:00
Sebastiaan van Stijn
8361baf8d9 Dockerfile: update compose to v2.27.1
release notes: https://github.com/docker/compose/releases/tag/v2.27.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-24 14:53:58 +02:00
Sebastiaan van Stijn
387be6ec91 Dockerfile: update buildx to v0.14.1
- 0.14.1 release notes: https://github.com/docker/buildx/releases/tag/v0.14.1
- 0.14.0 release notes: https://github.com/docker/buildx/releases/tag/v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-24 14:44:01 +02:00
Sebastiaan van Stijn
5cd2e6a1b7 Merge pull request #47854 from dperny/fix-manager-promote-race
Fix issue where node promotion could fail
2024-05-23 21:14:58 +02:00
Brian Goff
b0f7117b31 Merge pull request #47850 from weebney/patch-1
Replace dead RFC8878 hyperlink in documentation
2024-05-23 15:57:40 +00:00
Albin Kerouanton
d16a425f0f Merge pull request #47768 from akerouanton/libnet-ipam-linear-allocator
libnet/ipams/default: introduce a linear allocator
2024-05-23 11:07:12 +02:00
Albin Kerouanton
500eff0ae9 libnet/i/defaultipam: improve address pools validation
Nothing was validating whether address pools' `base` prefix
were larger than the target subnet `size` they're associated to. As
such invalid address pools would yield no subnet, the error could go
unnoticed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-23 08:26:07 +02:00
Albin Kerouanton
0c022307e9 libnet/i/defaultipam: Unmap IPv4-mapped IPv6 addrs
This ensures such address pools are part of the IPv4 address space.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-23 08:26:00 +02:00
Albin Kerouanton
9d288b5b43 libnet/i/defaultipam: introduce a linear allocator
The previous allocator was subnetting address pools eagerly
when the daemon started, and would then just iterate over that
list whenever RequestPool was called. This was leading to high
memory usage whenever IPv6 pools were configured with a target
subnet size too different from the pools prefix size.

For instance: pool = fd00::/8, target size = /64 -- 2 ^ (64-8)
subnets would be generated upfront. This would take approx.
9 * 10^18 bits -- way too much for any human computer in 2024.

Another noteworthy issue, the previous implementation was allocating
a subnet, and then in another layer was checking whether the
allocation was conflicting with some 'reserved networks'. If so,
the allocation would be retried, etc... To make it worse, 'reserved
networks' would be recomputed on every iteration. This is totally
ineffective as there could be 'reserved networks' that fully overlap
a given address pool (or many!).

To fix this issue, a new field `Exclude` is added to `RequestPool`.
It's up to each driver to take it into account. Since we don't know
whether this retry loop is useful for some remote IPAM driver, it's
reimplemented bug-for-bug directly in the remote driver.

The new allocator uses a linear-search algorithm. It takes advantage
of all lists (predefined pools, allocated subnets and reserved
networks) being sorted and logically combines 'allocated' and
'reserved' through a 'double cursor' to iterate on both lists at the
same time while preserving the total order. At the same time, it
iterates over 'predefined' pools and looks for the first empty space
that would be a good fit.

Currently, the size of the allocated subnet is still dictated by
each 'predefined' pools. We should consider hardcoding that size
instead, and let users specify what subnet size they want. This
wasn't possible before as the subnets were generated upfront. This
new allocator should be able to deal with this easily.

The method used for static allocation has been updated to make sure
the ascending order of 'allocated' is preserved. It's bug-for-bug
compatible with the previous implementation.

One consequence of this new algorithm is that we don't keep track
of where the last allocation happened, we just allocate the first
free subnet we find.

Before:

- Allocate: 10.0.1.0/24, 10.0.2.0/24 ; Deallocate: 10.0.1.0/24 ;
Allocate 10.0.3.0/24.

Now, the 3rd allocation would yield 10.0.1.0/24 once again.

As it doesn't change the semantics of the allocator, there's no
reason to worry about that.

Finally, about 'reserved networks'. The heuristics we use are
now properly documented. It was discovered that we don't check
routes for IPv6 allocations -- this can't be changed because
there's no such thing as on-link routes for IPv6.

(Kudos to Rob Murray for coming up with the linear-search idea.)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-23 08:24:51 +02:00
Albin Kerouanton
5f183b9b3a Merge pull request #47837 from akerouanton/libnet-ipam-disambiguate-PoolID
libnet/i/defaultipam: Disambiguate PoolID string format
2024-05-22 22:52:15 +02:00
weebney
a9ebb0c267 Replace dead RFC8878 go doc hyperlink
Fixes #45952

Signed-off-by: weebney <weebney@gmail.com>
2024-05-22 14:27:15 -04:00
Sebastiaan van Stijn
274b2932a1 Merge pull request #47838 from dmcgowan/update-containerd-v1.7.17
Update containerd to v1.7.17
2024-05-22 17:36:21 +02:00
Drew Erny
16e5c41591 Fix issue where node promotion could fail
If a node is promoted right after another node is demoted, there exists
the possibility of a race, by which the newly promoted manager attempts
to connect to the newly demoted manager for its initial Raft membership.
This connection fails, and the whole swarm Node object exits.

At this point, the daemon nodeRunner sees the exit and restarts the
Node.

However, if the address of the no-longer-manager is recorded in the
nodeRunner's config.joinAddr, the Node again attempts to connect to the
no-longer-manager, and crashes again. This repeats. The solution is to
remove the node entirely and rejoin the Swarm as a new node.

This change erases config.joinAddr from the restart of the nodeRunner,
if the node has previously become Ready. The node becoming Ready
indicates that at some point, it did successfully join the cluster, in
some fashion. If it has successfully joined the cluster, then Swarm has
its own persistent record of known manager addresses. If no joinAddr is
provided, then Swarm will choose from its persisted list of managers to
join, and will join a functioning manager.

Signed-off-by: Drew Erny <derny@mirantis.com>
2024-05-22 08:48:03 -05:00
Albin Kerouanton
5a2fa59688 libnet/i/defaultipam: Use InternalErrorf instead of InvalidParameterErrof
InvalidParameterErrorf was used whenever an invalid value was found
during PoolID unmarshaling. This error is converted to a 400 HTTP code
by the HTTP server.

However, users never provide PoolIDs directly -- these are constructed
from user-supplied values which are already validated when the PoolID is
marshaled. Hence, if such erroneous value is found, it's an internal
error and should be converted to a 500.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-22 10:02:20 +02:00
Albin Kerouanton
9369132879 libnet/i/defaultipam: Disambiguate PoolID string format
Prior to this change PoolID microformat was using slashes to separate
fields. Those fields include subnet prefixes in CIDR notation, which
also include a slash. This makes future evolution harder than it should
be.

This change introduces a 'v2' microformat based on JSON. This has two
advantages:

1. Fields are clearly named to ensure each value is associated to the
right field.
2. Field values and separators are clearly distinguished to remove any
ambiguity.

The 'v1' encoding will be kept until the next major MCR LTS is released.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-22 10:02:19 +02:00
Sebastiaan van Stijn
8470add2fe Merge pull request #47840 from vvoland/update-containerd
Dockerfile: update containerd binary to v1.7.17 (static binaries and CI only)
2024-05-21 23:16:00 +02:00
Albin Kerouanton
081f6ba39a Merge pull request #47826 from robmry/windns_proxy_default
Default to "windows-dns-proxy":true
2024-05-21 17:19:09 +02:00
Sebastiaan van Stijn
c3a40873f9 Merge pull request #47820 from akerouanton/libnet-store-is-never-nil-followup
libnet: Controller: more c.store clean-ups
2024-05-21 15:27:40 +02:00
Albin Kerouanton
6d21574535 libnet: Controller: drop getStore()
This method does nothing more than `return c.store`. It has no value and
adds an unecessary level of indirection. Let's ditch it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-21 12:54:57 +02:00
Albin Kerouanton
49888559cc libnet: Controller: drop closeStores
Previous commit made it clear that c.store can't be nil. Hence,
`c.store.Close()` can be called without checking if c.store is nil.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-21 12:43:48 +02:00
Albin Kerouanton
2336363e28 libnet: init datastore in ctrler constructor
This was done in a separate method, called by the ctrler constructor.
This method was returning a nil datastore when c.cfg was nil -- but that
can't happen in practice!

This was giving the impression that the controller could be run without
a datastore properly configured. It's not the case, so make it explicit
by instantiating the datastore before `Controller`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-21 12:43:48 +02:00
Paweł Gronowski
3847da374b integration/TestDiskUsage: Make 4096 also a 'empty' value
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-05-21 12:35:01 +02:00
Albin Kerouanton
145a73a36c Merge pull request #47818 from akerouanton/libnet-d-bridge-dont-parse-MacAddress-netlabel
libnet/d/bridge: don't parse the MacAddress netlabel
2024-05-21 10:12:16 +02:00
Rob Murray
1e29f9b12f Move EndpointSettings.DriverOpts from op-state to config
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-17 11:30:14 +01:00
Rob Murray
a35716f5b9 Factor out selection of endpoint for config migration
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-17 11:30:14 +01:00
Paweł Gronowski
4f0cb7d964 Dockerfile: update containerd binary to v1.7.17 (static binaries and CI only)
Update the containerd binary that's used in CI and static binaries

- full diff: https://github.com/containerd/containerd/compare/v1.7.15...v1.7.17
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.17

```markdown changelog
Update containerd (static binaries only) to [v1.7.17](https://github.com/containerd/containerd/releases/tag/v1.7.17)
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-05-17 11:07:11 +02:00
Derek McGowan
e459487feb Update containerd to v1.7.17
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-16 15:24:38 -07:00
Sebastiaan van Stijn
06e3a49d66 Merge pull request #47796 from cpuguy83/fix_superfluous_write_header
Explicity write http headers on streaming endpoints
2024-05-16 23:12:57 +02:00
Brian Goff
707ab48cbb Explicity write http headers on streaming endpoints
This works around issues with the otel http handler wrapper causing
multiple calls to `WriteHeader` when a `Flush` is called before `Write`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-16 18:00:02 +00:00
Brian Goff
50d3028464 Fix fd leak/goroutine when attaching stdin only
When only stdin is attached the goroutine can only ever exit if:

1. The container pipe is closed while trying to write to it
2. The client closes the stdin read pipe

This is because `io.Copy` does a read on the read side then a write to
the write side.
If reading from the client's stdin pipe blocks, the goroutine will never
get notified that the container pipe is closed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-16 16:49:07 +00:00
Sebastiaan van Stijn
ae976b998b Merge pull request #47629 from vvoland/tarexport-tracing-ctx-cancel
tarexport: Plumb ctx, add OTEL spans, handle cancellation
2024-05-14 14:14:51 +02:00
Paweł Gronowski
ad0f263eb5 tarexport: Plumb ctx, add OTEL spans, handle cancellation
Pass `context.Context` through `tarexport.Load` and `tarexport.Save`.
Create OTEL spans for the most time consuming operations.

Also, handle context cancellations to actually end saving/loading when
the operation is cancelled - before this PR the daemon would still be
performing the operation even though the user already cancelled it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-05-14 13:08:33 +02:00
Albin Kerouanton
5505c851f1 Merge pull request #47821 from robmry/internal_network_with_dns
Forward DNS requests into --internal networks
2024-05-14 12:22:49 +02:00
Rob Murray
33f9a5329a Default to "windows-dns-proxy":true
In 26.1, we added daemon feature flag "windows-dns-proxy" which could
be set to "true" to make "nslookup" work in Windows containers, by
forwarding requests from the internal resolver to the container's
external DNS servers.

This changes the default to forwarding-enabled - it can be disabled by
via daemon.json using ...
  "features": { "windows-dns-proxy": false }

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-13 11:11:20 +01:00
Rob Murray
77a47dba3b Forward DNS requests into --internal networks
A recent change to prevent containers only connected to --internal
networks from communicating with external DNS servers inadvertently
prevented the daemon's internal DNS server from forwarding requests
within an internal network to a containerised DNS server.

Relax the check, so that only requests that need to be forwarded
from the host's network namespace are dropped.

External DNS servers remain unreachable from the internal network.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 19:36:55 +01:00
Albin Kerouanton
cd08d377c5 Merge pull request #47819 from akerouanton/libnet-d-remote-replace-errorWithRollback
libnet/d/remote: replace errorWithRollback
2024-05-10 13:26:41 +02:00
Albin Kerouanton
6fbae5ff26 libnet/d/bridge: don't parse the MacAddress netlabel
Libnet's method `(*Network).createEndpoint()` is already parsing this
netlabel to set the field `ep.iface.mac`. Later on, this same method
invoke the driver's method `CreateEndpoint` with an `InterfaceInfo` arg
and an `options` arg (an opaque map of driver otps).

The `InterfaceInfo` interface contains a `MacAddress()` method that
returns `ep.iface.mac`. And the opaque map may contain the key
`netlabel.MacAddress`.

Prior to this change, the bridge driver was calling `MacAddress()`. If
no value was returned, it'd fall back to the option set in the `options`
map, or generate a MAC address based on the IP address.

However, the expected type of the `options` value is a `net.HardwareAddr`.
This is what's set by the daemon when handing over the endpoint config
to libnet controller. If the value is a string, as is the case if the
MAC address is provided through `EndpointsSettings.DriverOpts`, it
produces an error.

As such, the opaque option and the `MacAddress()` are necessarily the
same -- either nothing or a `net.HardwareAddr`. No need to keep both.

Moreover, the struct `endpointConfiguration` was only used to store that
netlabel value. Drop it too.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-10 12:45:12 +02:00
Rob Murray
41ddc47bbf Don't explicitly enable ip6tables in tests
Tests no longer need to use "--experimental --ip6tables", now ip6tables
is the default behaviour.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 10:17:30 +01:00
Rob Murray
07ccaf028d Enable 'ip6tables' by default, don't require 'experimental'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Rob Murray
5705cbf6e3 Make it an error to set up filtering on an unnamed bridge
In setupIPv6BridgeNetFiltering(), the bridge should always be named.
Don't fall back to checking the "default" setting for a new bridge.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Rob Murray
d6b6a5122f Enable filtering on IPv6 bridges with no IPv6 address
Check forwarding, then set bridge-nf-call-ip6tables, on a bridge
if IPv6 is enabled - even if no IPv6 address has been assigned.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Rob Murray
4df97f2e1e Gate setting of bridge-nf-call-ip6tables on "--ip6tables=true".
The code to enable "bridge-nf-call-iptables" or "bridge-nf-call-ip6tables"
was gated on "--iptables=true", it didn't check "--ip6tables=true".

So, split the top level call into IPv4/IPv6 so that the iptables-enable
settings can be checked independently, and simplfied the implementation.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Rob Murray
8751562d3f Set up IPv6 n/w isolation rules when --ip6tables=true
bridgeNetwork.isolateNetwork() checks "--iptables=true" and
"--ip6tables=true" before doing anything with IPv4 and IPv6
respectively.  But, it was only called if "--iptables=true".

Now, it's called if "--ip6tables=true", even if "--iptables=false".

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Rob Murray
23fd15985b Allow "--ip6tables=true" when "--iptables=false"
The bridge driver's setupIPChains() had an initial sanity check that
"--iptables=true".

But, it's called with "version=IPv6" when "--iptables=false" and
"--ip6tables=true" - the sanity test needed to allow for that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Rob Murray
9a8ffe38fc Disable ip6tables in tests that disable iptables
Tests that start a daemon disable iptables, to avoid conflicts with
other tests running in parallel and also creating iptables chains.

Do the same for ip6tables, in prep for them being enabled by-default.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-10 09:21:21 +01:00
Albin Kerouanton
75821a7d9a Merge pull request #47787 from robmry/47778_preserve_kernel_ll_addrs
Preserve kernel-assigned IPv6 link-local addresses on a bridge network's bridge
2024-05-10 10:18:11 +02:00
Albin Kerouanton
a9ded90030 Merge pull request #47788 from robmry/bad_integration-cli_ipv6_tests
Fix/remove broken integration-cli IPv6 tests
2024-05-10 10:17:48 +02:00
Albin Kerouanton
5952920380 libnet/d/remote: replace errorWithRollback
Use defer funcs instead.

For no apparant reasons, a few error cases in the Join method were not
triggering a rollback. This is now fixed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-10 10:15:11 +02:00
Sebastiaan van Stijn
64da0e0b13 Merge pull request #47810 from akerouanton/libnet-store-is-never-nil
libnet: don't check if ctrler store is nil
2024-05-09 13:10:12 +02:00
Albin Kerouanton
7216541b17 libnet: don't check if ctrler store is nil
Since commit befff0e1, `(*Controller).getStore()` never returns nil
except if `c.store` isn't initialized yet. This can't happen unless
`New()` returned an error and it wasn't proper caught.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-05-08 12:37:30 +02:00
Albin Kerouanton
7ea9acc97f cmd/dockerd: deprecate api-cors-header
CORS headers were originally added by 6d5bdff.

These headers could be set without any Authz plugin enabled
beforehand, making this feature quite dangerous.

This commit marks the daemon flag `api-cors-header` as deprecated
and requires the env var `DOCKERD_DEPRECATED_CORS_HEADER` to be
set. When enabled, the daemon will write a deprecation warning to
the logs and the endpoint `GET /info` will return the same
deprecation warning.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-08 12:12:09 +02:00
Paweł Gronowski
4554d871d7 Merge pull request #47805 from vvoland/update-go
update to go1.21.10
2024-05-08 10:58:54 +02:00
Paweł Gronowski
6c97e0e0b5 update to go1.21.10
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.9...go1.21.10

These minor releases include 2 security fixes following the security policy:

- cmd/go: arbitrary code execution during build on darwin
On Darwin, building a Go module which contains CGO can trigger arbitrary code execution when using the Apple version of ld, due to usage of the -lto_library flag in a "#cgo LDFLAGS" directive.
Thanks to Juho Forsén of Mattermost for reporting this issue.
This is CVE-2024-24787 and Go issue https://go.dev/issue/67119.

- net: malformed DNS message can cause infinite loop
A malformed DNS message in response to a query can cause the Lookup functions to get stuck in an infinite loop.
Thanks to long-name-let-people-remember-you on GitHub for reporting this issue, and to Mateusz Poliwczak for bringing the issue to our attention.
This is CVE-2024-24788 and Go issue https://go.dev/issue/66754.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.3

**- Description for the changelog**

```markdown changelog
Update Go runtime to 1.21.10
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-05-08 09:31:46 +02:00
Albin Kerouanton
4d525c9009 Merge pull request #47744 from robmry/47716_no_dns_req_to_self
Do not forward DNS requests to self.
2024-05-07 18:24:41 +02:00
Albin Kerouanton
da3f60bfe4 Merge pull request #47745 from robmry/firewalld_forwarding_policy
Add firewalld policy "docker-forwarding".
2024-05-07 15:52:26 +02:00
Paweł Gronowski
440836a8cf Merge pull request #47003 from LarsSven/fix-container-start-time
Move StartedAt time to before starting the container
2024-05-07 14:58:27 +02:00
Paweł Gronowski
8e14f278c4 Merge pull request #47651 from vvoland/api-bump
API: bump version to 1.46
2024-05-07 10:40:07 +02:00
Sebastiaan van Stijn
9314eaff2f Merge pull request #47797 from cpuguy83/bundles_aint_phony
Makefile: bundles is not PHONY
2024-05-06 08:02:24 +02:00
Brian Goff
ac71ac1c92 Merge pull request #47664 from crazybolillo/47516-crazybolillo 2024-05-03 14:23:15 -07:00
Brian Goff
72eb615490 Makefile: bundles is not PHONY
This was changed recently so that the bundles target is always run, but
`mkdir bundles` fails when bundles exists...

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-05-03 21:01:12 +00:00
Sebastiaan van Stijn
a73e63cfa6 Merge pull request #47656 from imalasong/pr/1
Makefile: refactoring .PHONY
2024-05-03 00:19:31 +02:00
Sebastiaan van Stijn
5d03db29d8 Merge pull request #47749 from woky/apparmor-runc
apparmor: Allow confined runc to kill containers
2024-05-02 20:50:06 +02:00
Rob Murray
fda708f55d Delete broken/unused test requirement helper "IPv6"
It'd only return true on a host with no IPv6 in its kernel.

So, removed, having fixed the two tests that used it.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 19:28:16 +01:00
Rob Murray
4aff2fbc98 Remove integration-cli TestDaemonSuite/TestDaemonIPv6Enabled
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.

TestDaemonIPv6Enabled tried to run with IPv6 on the default bridge,
but didn't set up a "fixed-cidr-v6" - so the daemon wouldn't start.

It then tried to check the bridge had address "fe80::1", which it
expected to work because it had just used setupV6() to add that
address.

Then it  checked that "LinkLocalIPv6Address" was set in container
inspect output, but it wouldn't be (the field is deprecated).

There are working IPv6 tests in the suite (TestDaemonIPv6FixedCIDR,
TestDaemonIPv6FixedCIDRAndMac, TestDaemonIPv6HostMode) - and there's
more coverage in the network integration tests.

So, deleted the test as it didn't seem worth salvaging.

Also deleted now-unused helper functions setupV6(), teardownV6().

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 19:26:06 +01:00
Rob Murray
346a7c07a0 Fix TestDockerSwarmSuite/TestSwarmInitIPv6
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.

If the test had run, it'd have failed because:
- it used "--listen-add", but the option is "--listen-addr"
  - so, the daemon wouldn't have started
- it tried to use "--join ::1"
  - address "::1" was interpreted as host:port so the Dial() failed,
    it needed to be "[::1]".
  - it didn't supply a  join token

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 19:09:54 +01:00
Rob Murray
b11e95f5bc Don't delete IPv6 multicast addresses from a bridge
Multicast addresses aren't added by the daemon so, if they're present,
it's because they were explicitly added - possibly to a user-managed
bridge. So, don't remove.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 17:46:08 +01:00
Rob Murray
a5f82ba4bf Disallow IPv6 multicast as bridge n/w subnet
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 17:20:56 +01:00
Rob Murray
aa3a86c038 Refactor IPv6 subnet validation
- Remove package variable bridge.bridgeIPv6
- Use netip in more places
- Improve error messages from fixed-cidr-v6 checks

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 17:20:56 +01:00
Rob Murray
f46473b29c Do not remove kernel-ll addresses from bridges
Make the behaviour enabled by env var DOCKER_BRIDGE_PRESERVE_KERNEL_LL
the default...
- don't remove kernel assigned link-local addresses
  - or any address in fe80::/64
- don't assign fe80::1 to a bridge

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 17:20:56 +01:00
Paweł Gronowski
9d07820b22 Merge pull request #47771 from robmry/dont_delete_kernel_ll_addrs
Option to avoid deleting the kernel_ll address from bridges.
2024-04-30 11:27:30 +02:00
Sebastiaan van Stijn
22892d2210 Merge pull request #47572 from avoidaway/master
chore: remove repetitive words
2024-04-30 08:53:53 +02:00
Sebastiaan van Stijn
8cbd20246c Merge pull request #47769 from robmry/47751_readonly_procsysnet
Allow for a read-only "/proc/sys/net".
2024-04-29 21:43:45 +02:00
Rob Murray
01ea18f1e3 Allow for a read-only "/proc/sys/net".
If dockerd runs on a host with a read-only /proc/sys/net filesystem,
it isn't able to enable or disable IPv6 on network interfaces when
attaching a container to a network (including initial networks during
container creation).

In release 26.0.2, a read-only /proc/sys/net meant container creation
failed in all cases.

So, don't attempt to enable/disable IPv6 on an interface if it's already
set appropriately.

If it's not possible to enable IPv6 when it's needed, just log (because
that's what libnetwork has always done if IPv6 is disabled in the
kernel).

If it's not possible to disable IPv6 when it needs to be disabled,
refuse to create the container and raise an error that suggests setting
environment variable "DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE=1", to tell
the daemon it's ok to ignore the problem.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-29 16:17:06 +01:00
Rob Murray
57ada4b848 Option to avoid deleting the kernel_ll address from bridges.
If env var DOCKER_BRIDGE_PRESERVE_KERNEL_LL=1, don't assign fe80::1/64
to a bridge, and don't delete any link local address with prefix fe80::/64.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-28 17:01:59 +01:00
Albin Kerouanton
48d769bf2f Merge pull request #47727 from akerouanton/libnet-ipam-cleanup
libnet/ipam: Various clean-ups
2024-04-26 22:42:53 +02:00
Albin Kerouanton
c5376e534c libnet/ipams/null: move driver name to its pkg
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
f2387f3632 libnet/ipams/defaultipam: move driver name to its pkg
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
0db56de78e libnet/ipamutils: no more global state
Prior to this change, cnmallocator would call
`ConfigGlobalScopeDefaultNetworks` right before initializing its
IPAM drivers. This function was mutating some global state used
during drivers init.

This change just remove the global state, and adds an arg to
ipams.Register and defaultipam.Register to pass the global pools
by arguments instead.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
3c9718144f libnet/ipams: register all drivers
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
eda47500fc libnet/ipams: Unconditionally call windowsipam.Register
This function is made a no-op on non-windows platform.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
ae9e4319b0 libnet/ipams/windowsipam: that driver knows its name
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
8cec9f0dca libnet/ipams/defaultipam: add a Register fn
All drivers except the default have a Register function. Before this
change, default's registration was handled by another package. Move
this logic into the driver pkg.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
218394cada libnet/ipams/builtin: move to libnet/ipams
Packages in libnet/ipams are drivers, except builtin -- it's used
to register drivers. Move files one level up and delete this pkg.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
29f2ca04e0 libnet: move ipam pkg to ipam/defaultipam
All drivers except the default ipam driver are stored in ipams/.
Since `default` isn't a valid Go pkg name, this package is
renamed to `defaultipam`, following `windowsipam` example.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
e8644c3e0e libnet/ipam: default-address-pools as Register arg
Prior to this change, daemon's `default-address-pools` param would
be passed to `SetDefaultIPAddressPool()` to set a global var named
`defaultAddressPool`. This var would then be retrieved during the
`default` IPAM driver registration. Both steps were executed in
close succession during libnet's controller initialization.

This change removes the global var and just pass the user-defined
`default-address-pools` to the `default` driver's `Register` fn.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
1d5a12dfb1 integration-cli: createNetwork: add t.Helper()
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:29 +02:00
Albin Kerouanton
115de5ff3d libnet/ipamapi: add in/out structs for RequestPool
The `RequestPool` method has many args and named returns. This
makes the code hard to follow at times. This commit adds one struct,
`PoolRequest`, to replace these args, and one struct, `AllocatedPool`,
to replace these named returns.

Both structs' fields are properly documented to better define their
semantics, and their relationship with address allocation.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 17:28:26 +02:00
Albin Kerouanton
82aae0fe50 libnet/netutils: remove dead util NetworkRange
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 16:10:39 +02:00
Albin Kerouanton
37a81cd04d libnet/ipam: split v4/v6 address spaces
Address spaces are a continuum of addresses that can be used for a
specific purpose (ie. 'local' for unmanaged containers, 'global for
Swarm). v4 and v6 addresses aren't of the same size -- hence
combining them into a single address space doesn't form a continuum.
Better set them apart into two different address spaces.

Also, the upcoming rewrite of `addrSpace` will benefit from that
split.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 16:07:57 +02:00
Albin Kerouanton
199c72cb5d libnet/ipam: remove dead DumpDatabase()
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 16:06:40 +02:00
Albin Kerouanton
df88857e6c libnet/ipam: put addrSpace into a separate file
`addrSpace` methods are currently scattered in two different files.
As upcoming work will rewrite some of these methods, better put them
into a separate file.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 16:06:40 +02:00
Albin Kerouanton
a047d4b1df libnet/ipam: un-embed mutex from addrSpace
Embedding `sync.Mutex` into a struct is considered a bad practice
as it makes the mutex methods part of the embedding struct's API.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-04-26 16:06:40 +02:00
Sebastiaan van Stijn
16b2c223ce Merge pull request #47536 from Benehiko/docker-client-ctx-reduced
feat: ctx to client API
2024-04-26 15:21:00 +02:00
Paweł Gronowski
dda4fec99a Merge pull request #47763 from dmcgowan/update-containerd-1.7.16
vendor: update containerd to v1.7.16
2024-04-26 11:00:28 +02:00
Derek McGowan
eeec716e33 Update containerd to v1.7.16
Includes fix for HTTP fallback

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-04-25 15:35:15 -07:00
Jonas Geiler
efca9303a4 refactor: updated native diff error message
Signed-off-by: Jonas Geiler <git@jonasgeiler.com>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
2024-04-25 21:30:10 +02:00
Rob Murray
ff8de5e156 Add firewalld policy "docker-forwarding".
Allow forwarding from any firewalld zone to the 'docker' zone.

This makes it possible to use routable IPv6 addresses on a bridge
network, with masquerading disabled, and have the host forward packets
to it.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-24 16:44:43 +01:00
Tomáš Virtus
5ebe2c0d6b apparmor: Allow confined runc to kill containers
/usr/sbin/runc is confined with "runc" profile[1] introduced in AppArmor
v4.0.0. This change breaks stopping of containers, because the profile
assigned to containers doesn't accept signals from the "runc" peer.
AppArmor >= v4.0.0 is currently part of Ubuntu Mantic (23.10) and later.

In the case of Docker, this regression is hidden by the fact that
dockerd itself sends SIGKILL to the running container after runc fails
to stop it. It is still a regression, because graceful shutdowns of
containers via "docker stop" are no longer possible, as SIGTERM from
runc is not delivered to them. This can be seen in logs from dockerd
when run with debug logging enabled and also from tracing signals with
killsnoop utility from bcc[2] (in bpfcc-tools package in Debian/Ubuntu):

  Test commands:

    root@cloudimg:~# docker run -d --name test redis
    ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
    root@cloudimg:~# docker stop test

  Relevant syslog messages (with wrapped long lines):

    Apr 23 20:45:26 cloudimg kernel: audit:
      type=1400 audit(1713905126.444:253): apparmor="DENIED"
      operation="signal" class="signal" profile="docker-default" pid=9289
      comm="runc" requested_mask="receive" denied_mask="receive"
      signal=kill peer="runc"
    Apr 23 20:45:36 cloudimg dockerd[9030]:
      time="2024-04-23T20:45:36.447016467Z"
      level=warning msg="Container failed to exit within 10s of kill - trying direct SIGKILL"
      container=ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
      error="context deadline exceeded"

  Killsnoop output after "docker stop ...":

    root@cloudimg:~# killsnoop-bpfcc
    TIME      PID      COMM             SIG  TPID     RESULT
    20:51:00  9631     runc             3    9581     -13
    20:51:02  9637     runc             9    9581     -13
    20:51:12  9030     dockerd          9    9581     0

This change extends the docker-default profile with rules that allow
receiving signals from processes that run confined with either runc or
crun profile (crun[4] is an alternative OCI runtime that's also confined
in AppArmor >= v4.0.0, see [1]). It is backward compatible because the
peer value is a regular expression (AARE) so the referenced profile
doesn't have to exist for this profile to successfully compile and load.

Note that the runc profile has an attachment to /usr/sbin/runc. This is
the path where the runc package in Debian/Ubuntu puts the binary. When
the docker-ce package is installed from the upstream repository[3], runc
is installed as part of the containerd.io package at /usr/bin/runc.
Therefore it's still running unconfined and has no issues sending
signals to containers.

[1] https://gitlab.com/apparmor/apparmor/-/commit/2594d936
[2] https://github.com/iovisor/bcc/blob/master/tools/killsnoop.py
[3] https://download.docker.com/linux/ubuntu
[4] https://github.com/containers/crun

Signed-off-by: Tomáš Virtus <nechtom@gmail.com>
2024-04-24 13:07:48 +02:00
Rob Murray
87506142d8 Do not forward DNS requests to self.
If a container is configured with the internal DNS resolver's own
address as an external server, try the next ext server rather than
recursing (return SERVFAIL if there are no other servers).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-24 09:15:35 +01:00
Sebastiaan van Stijn
faf84d7f0a Merge pull request #47743 from thaJeztah/bump_go_winio
vendor: github.com/Microsoft/go-winio v0.6.2 (for go1.22 compatibility)
2024-04-23 14:47:40 +02:00
Paweł Gronowski
70475b371b Merge pull request #47739 from thaJeztah/vendor_ebpf
vendor: github.com/cilium/ebpf v0.12.3
2024-04-23 12:58:02 +02:00
Paweł Gronowski
c95b0a97c5 Merge pull request #47742 from vvoland/update-cli
Dockerfile: update docker CLI to v26.1.0
2024-04-23 12:52:47 +02:00
racequite
147f701bd1 chore: fix function names in comment
Signed-off-by: Rui JingAn <quiterace@gmail.com>
2024-04-23 17:49:41 +08:00
Sebastiaan van Stijn
e3c59640d5 vendor: github.com/Microsoft/go-winio v0.6.2
- fileinfo: internally fix FileBasicInfo memory alignment (fixes compatibility
  with go1.22)

full diff: https://github.com/Microsoft/go-winio/compare/v0.6.1...v0.6.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-23 11:00:52 +02:00
Sebastiaan van Stijn
2140e7e0f5 vendor: golang.org/x/tools v0.16.0
It's not used in our code, but some dependencies have a "tools.go" to
force it; updating to a version that doesn't depend on golang.org/x/sys/execabs

full diff: https://github.com/golang/tools/compare/v0.14.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-23 11:00:03 +02:00
Sebastiaan van Stijn
85c9900377 vendor: golang.org/x/mod v0.17.0
no changes in vendored codee

full diff: https://github.com/golang/mod/compare/v0.13.0...v0.17.0

- modfile: do not collapse if there are unattached comments within blocks
- modfile: fix crash on AddGoStmt in empty File
- modfile: improve directory path detection and error text consistency
- modfile: use new go version string format in WorkFile.add error
- sumdb: replace globsMatchPath with module.MatchPrefixPatterns
- sumdb/tlog: make NewTiles only generate strictly necessary tiles

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-23 10:53:17 +02:00
Paweł Gronowski
e314113ad7 Dockerfile: update docker CLI to v26.1.0
Update the CLI that's used in the dev-container

- full diff: https://github.com/docker/cli/compare/v26.0.0...v26.1.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-23 10:47:49 +02:00
Brian Goff
c8af8ebe4a Merge pull request #47738 from vvoland/c8d-walk-image-badimagetarget
c8d/list: Ignore unexpected image target
2024-04-22 07:45:14 -07:00
Sebastiaan van Stijn
df831c943f vendor: github.com/cilium/ebpf v0.12.3
full diff: https://github.com/cilium/ebpf/compare/v0.11.0...v0.12.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-22 13:52:44 +02:00
Paweł Gronowski
7d95fe8db5 c8d/list: Ignore unexpected image target
Don't fail-fast when encountering an image that targets an unexpected
descriptor (neither a manifest nor index). Log a warning instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-22 12:58:46 +02:00
Bjorn Neergaard
801fd16e3e Merge pull request #47735 from cpuguy83/better_walk_error
Include more details in errNotManifestOrIndex
2024-04-19 18:16:12 -07:00
Brian Goff
6667e96dad Include more details in errnotManifestOrIndex
This error is returned when attempting to walk a descriptor that
*should* be an index or a manifest.
Without this the error is not very helpful sicne there's no way to tell
what triggered it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-04-19 21:13:05 +00:00
Paweł Gronowski
ee8b788538 Merge pull request #47734 from krissetto/image-history-timestamp-dereference
fix: avoid nil dereference on image history `Created` value
2024-04-19 14:23:02 +02:00
Paweł Gronowski
96c9353e9b Merge pull request #47723 from vvoland/builder-fix-workdir-slash
builder: Fix `WORKDIR` with a trailing slash causing a cache miss
2024-04-19 13:56:40 +02:00
Christopher Petito
ab570ab3d6 nil dereference fix on image history Created value
Issue was caused by the changes here https://github.com/moby/moby/pull/45504
First released in v25.0.0-beta.1

Signed-off-by: Christopher Petito <47751006+krissetto@users.noreply.github.com>
2024-04-19 10:44:30 +00:00
Paweł Gronowski
7532420f3b container/SetupWorkingDirectory: Don't mutate config
Don't mutate the container's `Config.WorkingDir` permanently with a
cleaned path when creating a working directory.

Move the `filepath.Clean` to the `translateWorkingDir` instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-19 12:42:20 +02:00
Paweł Gronowski
a4d5b6b4d0 builder/normalizeWorkdir: Always return cleaned path
The `normalizeWorkdir` function has two branches, one that returns a
result of `filepath.Join` which always returns a cleaned path, and
another one where the input string is returned unmodified.

To make these two outputs consistent, also clean the path in the second
branch.

This also makes the cleaning of the container workdir explicit in the
`normalizeWorkdir` function instead of relying on the
`SetupWorkingDirectory` to mutate it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-19 12:42:19 +02:00
Paweł Gronowski
e829cca0ee Merge pull request #47584 from robmry/upstream_dns_windows
Windows DNS resolver forwarding
2024-04-19 11:34:50 +02:00
Sebastiaan van Stijn
82d8f8d6e6 Merge pull request from GHSA-x84c-p2g9-rqv9
Disable IPv6 for endpoints in '--ipv6=false' networks.
2024-04-18 17:50:35 +02:00
Rob Murray
6c68be24a2 Windows DNS resolver forwarding
Make the internal DNS resolver for Windows containers forward requests
to upsteam DNS servers when it cannot respond itself, rather than
returning SERVFAIL.

Windows containers are normally configured with the internal resolver
first for service discovery (container name lookup), then external
resolvers from '--dns' or the host's networking configuration.

When a tool like ping gets a SERVFAIL from the internal resolver, it
tries the other nameservers. But, nslookup does not, and with this
change it does not need to.

The internal resolver learns external server addresses from the
container's HNSEndpoint configuration, so it will use the same DNS
servers as processes in the container.

The internal resolver for Windows containers listens on the network's
gateway address, and each container may have a different set of external
DNS servers. So, the resolver uses the source address of the DNS request
to select external resolvers.

On Windows, daemon.json feature option 'windows-no-dns-proxy' can be used
to prevent the internal resolver from forwarding requests (restoring the
old behaviour).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-16 18:57:28 +01:00
Sebastiaan van Stijn
b7c059886c Merge pull request #47706 from elezar/bump-container-device-interface
Update tags.cncf.io/container-device-interface to v0.7.1
2024-04-16 15:46:49 +02:00
Evan Lezar
745e2356ab Update tags.cncf.io/container-device-interface to v0.7.1
This also bumps the maximum supported CDI specification to v0.7.0.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2024-04-16 12:06:23 +02:00
Sebastiaan van Stijn
29f24a828b Merge pull request #47719 from thaJeztah/vendor_runtime_spec
vendor: github.com/opencontainers/runtime-spec v1.2.0
2024-04-16 11:50:50 +02:00
Sebastiaan van Stijn
0d6a1a212b vendor: github.com/opencontainers/runtime-spec v1.2.0
- deprecate Prestart hook
- deprecate kernel memory limits

Additions

- config: add idmap and ridmap mount options
- config.md: allow empty mappings for [r]idmap
- features-linux: Expose idmap information
- mount: Allow relative mount destinations on Linux
- features: add potentiallyUnsafeConfigAnnotations
- config: add support for org.opencontainers.image annotations

Minor fixes:

- config: improve bind mount and propagation doc

full diff: https://github.com/opencontainers/runtime-spec/compare/v1.1.0...v1.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-15 17:56:26 +02:00
Sebastiaan van Stijn
f5209d23a8 daemon: add nolint-comments for deprecated kernel-memory options, hooks
This adds some nolint-comments for the deprecated kernel-memory options; we
deprecated these, but they could technically still be accepted by alternative
runtimes.

    daemon/daemon_unix.go:108:3: SA1019: memory.Kernel is deprecated: kernel-memory limits are not supported in cgroups v2, and were obsoleted in [kernel v5.4]. This field should no longer be used, as it may be ignored by runtimes. (staticcheck)
            memory.Kernel = &config.KernelMemory
            ^
    daemon/update_linux.go:63:3: SA1019: memory.Kernel is deprecated: kernel-memory limits are not supported in cgroups v2, and were obsoleted in [kernel v5.4]. This field should no longer be used, as it may be ignored by runtimes. (staticcheck)
            memory.Kernel = &resources.KernelMemory
            ^

Prestart hooks are deprecated, and more granular hooks should be used instead.
CreateRuntime are the closest equivalent, and executed in the same locations
as Prestart-hooks, but depending on what these hooks do, possibly one of the
other hooks could be used instead (such as CreateContainer or StartContainer).
As these hooks are still supported, this patch adds nolint comments, but adds
some TODOs to consider migrating to something else;

    daemon/nvidia_linux.go:86:2: SA1019: s.Hooks.Prestart is deprecated: use [Hooks.CreateRuntime], [Hooks.CreateContainer], and [Hooks.StartContainer] instead, which allow more granular hook control during the create and start phase. (staticcheck)
        s.Hooks.Prestart = append(s.Hooks.Prestart, specs.Hook{
        ^

    daemon/oci_linux.go:76:5: SA1019: s.Hooks.Prestart is deprecated: use [Hooks.CreateRuntime], [Hooks.CreateContainer], and [Hooks.StartContainer] instead, which allow more granular hook control during the create and start phase. (staticcheck)
                    s.Hooks.Prestart = append(s.Hooks.Prestart, specs.Hook{
                    ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-15 17:55:47 +02:00
Paweł Gronowski
442f29a699 Merge pull request #47711 from vvoland/swarm-subpath
daemon/cluster/executor: Add volume `Subpath`
2024-04-15 17:45:20 +02:00
Rob Murray
f07644e17e Add netiputil.AddrPortFromNet()
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-15 14:51:20 +01:00
Paweł Gronowski
d3c051318f daemon/cluster/executor: Add volume Subpath
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-15 14:14:32 +02:00
Paweł Gronowski
5368c3a04f vendor: github.com/moby/swarmkit/v2 master (f3ffc0881d0e)
full diff: 911c97650f...f3ffc0881d

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-15 14:14:30 +02:00
Lei Jitang
8d5d655db0 Merge pull request #47708 from ViToni/fix_typos
Fix typo
2024-04-13 10:33:58 +08:00
Victor Toni
f51e18f58e Fix typo
Signed-off-by: Victor Toni <victor.toni@gmail.com>
2024-04-11 00:19:05 +02:00
Rob Murray
57dd56726a Disable IPv6 for endpoints in '--ipv6=false' networks.
No IPAM IPv6 address is given to an interface in a network with
'--ipv6=false', but the kernel would assign a link-local address and,
in a macvlan/ipvlan network, the interface may get a SLAAC-assigned
address.

So, disable IPv6 on the interface to avoid that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 17:11:20 +01:00
Paweł Gronowski
f9dfd139ec Merge pull request #47657 from siepkes/illumos_fix
Minor fix for illumos support
2024-04-10 12:35:14 +02:00
Paweł Gronowski
051d587447 Merge pull request #47677 from robmry/47662_ipvlan_l3_dns
Enable external DNS for ipvlan-l3, and disable it for macvlan/ipvlan with no parent interface
2024-04-10 12:23:40 +02:00
Rob Murray
9954d7c6bd Run ipvlan tests even if 'modprobe ipvlan' fails
This reverts commit a77e147d32.

The ipvlan integration tests have been skipped in CI because of a check
intended to ensure the kernel has ipvlan support - which failed, but
seems to be unnecessary (probably because kernels have moved on).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 08:52:08 +01:00
Rob Murray
cd7240f6d9 Stop macvlan with no parent from using ext-dns
We document that an macvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an macvlan
network was still configured with a gateway. So, DNS proxying would
be enabled in the internal resolver (and, if the host's resolver
was on a localhost address, requests to external resolvers from the
host's network namespace would succeed).

This change disables configuration of a gateway for a macvlan Endpoint
if no parent interface is specified.

(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway will still be set up. Documentation
will need to be updated to note that '--internal' should be used to
prevent DNS request forwarding in this case.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 08:51:00 +01:00
Rob Murray
17b8631545 Enable DNS proxying for ipvlan-l3
The internal DNS resolver should only forward requests to external
resolvers if the libnetwork.Sandbox served by the resolver has external
network access (so, no forwarding for '--internal' networks).

The test for external network access was whether the Sandbox had an
Endpoint with a gateway configured.

However, an ipvlan-l3 networks with external network access does not
have a gateway, it has a default route bound to an interface.

Also, we document that an ipvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an ipvlan-l2
network was configured with a gateway. So, DNS proxying would be enabled
in the internal resolver (and, if the host's resolver was on a localhost
address, requests to external resolvers from the host's network
namespace would succeed).

So, this change adjusts the test for enabling DNS proxying to include
a check for '--internal' (as a shortcut) and, for non-internal networks,
checks for a default route as well as a gateway. It also disables
configuration of a gateway or a default route for an ipvlan Endpoint if
no parent interface is specified.

(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway/default route will still be set up
and external DNS proxying will be enabled. The network must be
configured as '--internal' to prevent that from happening.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 08:50:57 +01:00
Sebastiaan van Stijn
8383c487c6 Merge pull request #47691 from vvoland/vendor-master-containerd-v1.7.15
vendor: github.com/containerd/containerd v1.7.15
2024-04-09 12:08:31 +02:00
imalasong
194cbd6e7d Makefile: refactoring .PHONY
Signed-off-by: xiaochangbai <704566072@qq.com>
2024-04-09 09:26:31 +08:00
Sebastiaan van Stijn
7a54a16740 Merge pull request #47647 from vvoland/ci-backport-title
github/ci: Check if backport is opened against the expected branch
2024-04-08 19:15:37 +02:00
Sebastiaan van Stijn
2fabb28813 Merge pull request #47689 from vvoland/update-containerd
update containerd binary to v1.7.15
2024-04-08 19:07:18 +02:00
Paweł Gronowski
5ae5969739 vendor: github.com/containerd/containerd v1.7.15
full diff: https://github.com/containerd/containerd/compare/v1.7.14...v1.7.15

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 14:33:32 +02:00
Paweł Gronowski
3485cfbb1e update containerd binary to v1.7.15
Update the containerd binary that's used in CI

- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.15
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.15

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 13:01:28 +02:00
Tianon Gravi
3b81ca4969 Merge pull request #47661 from cpuguy83/oci_tar_no_platform
save: Remove platform from config descriptor
2024-04-05 15:38:30 -07:00
Sebastiaan van Stijn
80572929e1 Merge pull request #47682 from vvoland/ci-check-changelog-error
ci/validate-pr: Use `::error::` command to print errors
2024-04-05 23:05:00 +02:00
Paweł Gronowski
fb92caf2aa ci/validate-pr: Use ::error:: command to print errors
This will make Github render the log line as an error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-05 14:56:50 +02:00
Paweł Gronowski
61269e718f github/ci: Check if backport is opened against the expected branch
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-05 11:55:14 +02:00
Sebastiaan van Stijn
d25b0bd7ea Merge pull request #47673 from thaJeztah/vendor_x_net
vendor: golang.org/x/net v0.23.0
2024-04-04 14:31:05 +02:00
Rob Murray
d8b768149b Move dummy DNS server to integration/internal/network
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-04 12:02:22 +01:00
Sebastiaan van Stijn
6d30487d2e Merge pull request #47670 from vvoland/update-go
update to go1.21.9
2024-04-04 12:11:14 +02:00
Paweł Gronowski
329d403e20 update to go1.21.9
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.

These minor releases include 1 security fixes following the security policy:

- http2: close connections when receiving too many headers

Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.

Set a limit on the amount of excess header frames we will process before
closing a connection.

Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.

This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-04 10:13:50 +02:00
Sebastiaan van Stijn
d66589496e vendor: golang.org/x/net v0.23.0
full diff: https://github.com/golang/net/compare/v0.22.0...v0.23.0

Includes a fix for CVE-2023-45288, which is also addressed in go1.22.2
and go1.21.9;

> http2: close connections when receiving too many headers
>
> Maintaining HPACK state requires that we parse and process
> all HEADERS and CONTINUATION frames on a connection.
> When a request's headers exceed MaxHeaderBytes, we don't
> allocate memory to store the excess headers but we do
> parse them. This permits an attacker to cause an HTTP/2
> endpoint to read arbitrary amounts of data, all associated
> with a request which is going to be rejected.
>
> Set a limit on the amount of excess header frames we
> will process before closing a connection.
>
> Thanks to Bartek Nowotarski for reporting this issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-03 20:42:29 +02:00
Sebastiaan van Stijn
e1ca74361b vendor: golang.org/x/net v0.22.0, golang.org/x/crypto v0.21.0
full diffs changes relevant to vendored code:

- https://github.com/golang/net/compare/v0.18.0...v0.22.0
    - websocket: add support for dialing with context
    - http2: remove suspicious uint32->v conversion in frame code
    - http2: send an error of FLOW_CONTROL_ERROR when exceed the maximum octets
- https://github.com/golang/crypto/compare/v0.17.0...v0.21.0
    - internal/poly1305: drop Go 1.12 compatibility
    - internal/poly1305: improve sum_ppc64le.s
    - ocsp: don't use iota for externally defined constants

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-03 20:38:05 +02:00
Jasper Siepkes
cf933115b6 Minor fix for illumos support
illumos is the opensource continuation of OpenSolaris after Oracle
closed to source it (again).

For example use see: https://github.com/openbao/openbao/pull/205.

Signed-off-by: Jasper Siepkes <siepkes@serviceplanet.nl>
2024-04-03 15:58:27 +02:00
Albin Kerouanton
9fa76786ab Merge pull request #47431 from akerouanton/api-normalize-default-NetworkMode
api: normalize the default NetworkMode
2024-04-03 15:44:24 +02:00
Antonio Aguilar
57a12a372f Update GoDoc for ioutils on atomic writers
Unlike its stdlib counterparts, AtomicFileWriter does not take into
consideration umask due to its use of chmod. Failure to recognize this
may cause subtle problems like the one described in #47498.

Therefore the documentation has been updated to let users know that
umask is not taken into consideration when using AtomicFileWriter.

Closes #47516.

Signed-off-by: Antonio Aguilar <antonio@zoftko.com>
2024-04-02 23:27:04 -06:00
Brian Goff
9160b9fda6 save: Remove platform from config descriptor
This was brought up by bmitch that its not expected to have a platform
object in the config descriptor.
Also checked with tianon who agreed, its not _wrong_ but is unexpected
and doesn't neccessarily make sense to have it there.

Also, while technically incorrect, ECR is throwing an error when it sees
this.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-04-02 17:15:52 +00:00
Paweł Gronowski
8599f2a3fb Merge pull request #47658 from cpuguy83/fix_error_wrap_local_logs
Fix cases where we are wrapping a nil error
2024-04-02 10:28:09 +02:00
Brian Goff
0a48d26fbc Fix cases where we are wrapping a nil error
This was using `errors.Wrap` when there was no error to wrap, meanwhile
we are supposed to be creating a new error.

Found this while investigating some log corruption issues and
unexpectedly getting a nil reader and a nil error from `getTailReader`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-04-01 21:30:43 +00:00
avoidaway
98d51b510d chore: remove repetitive words
chore: remove repetitive words

Signed-off-by: avoidaway <cmoman@126.com>
2024-03-30 22:17:47 +08:00
Paweł Gronowski
8bbba6315f API: bump version to 1.46
Docker 26.0 was released with API v1.45, so any change in the API should
now target v1.46.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-29 11:07:35 +01:00
Bjorn Neergaard
330a6f959f Merge pull request #47645 from vvoland/community-slack
CONTRIBUTING.md: update Slack link
2024-03-28 14:25:58 -06:00
Albin Kerouanton
c4689034fd daemon: don't call NetworkMode.IsDefault()
Previous commit made this unnecessary.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-28 12:35:47 +01:00
Albin Kerouanton
4eed3dcdfe api: normalize the default NetworkMode
The NetworkMode "default" is now normalized into the value it
aliases ("bridge" on Linux and "nat" on Windows) by the
ContainerCreate endpoint, the legacy image builder, Swarm's
cluster executor and by the container restore codepath.

builder-next is left untouched as it already uses the normalized
value (ie. bridge).

Going forward, this will make maintenance easier as there's one
less NetworkMode to care about.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-28 12:34:23 +01:00
Paweł Gronowski
c187f95fe1 CONTRIBUTING.md: update Slack link
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-28 09:37:18 +01:00
Albin Kerouanton
a33b302d54 Merge pull request #47635 from robmry/backport-26.0/47619_restore_prestart_hook
[26.0 backport] Restore the SetKey prestart hook.
2024-03-28 08:24:48 +00:00
Paweł Gronowski
484480f56a Merge pull request #47636 from crazy-max/rm-artifacts-upload
ci: update workflow artifacts retention
2024-03-27 12:30:30 +01:00
CrazyMax
aff003139c ci: update workflow artifacts retention
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-03-27 10:57:01 +01:00
Rob Murray
1014f481de Restore the SetKey prestart hook.
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"

Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.

But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.

This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-27 08:50:25 +00:00
Albin Kerouanton
d57b899904 Merge pull request #47621 from robmry/47619_restore_prestart_hook
Restore the SetKey prestart hook.
2024-03-27 08:12:18 +00:00
Rob Murray
fde80fe2e7 Restore the SetKey prestart hook.
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"

Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.

But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.

This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-25 19:35:55 +00:00
Bjorn Neergaard
bfdb8918f9 Merge pull request #47613 from availhang/master
chore: fix mismatched function names in godoc
2024-03-22 15:48:57 -06:00
Alano Terblanche
80d92fd450 feat: ctx to client API
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
2024-03-22 13:06:43 +01:00
Sebastiaan van Stijn
83ae9927fb Merge pull request #47603 from neersighted/authors_mailmap
AUTHORS,.mailmap: update with recent contributors
2024-03-22 10:13:28 +01:00
George Ma
14a8fac092 chore: fix mismatched function names in godoc
Signed-off-by: George Ma <mayangang@outlook.com>
2024-03-22 16:24:31 +08:00
Jonas Geiler
838047a1f5 archive: fix ConvertRead/ConvertWrite functions for rootless docker & native diff overlay
Signed-off-by: Jonas Geiler <git@jonasgeiler.com>
2024-03-22 01:25:21 +01:00
Jonas Geiler
aef6905e01 overlay2: better documentation of user namespace behavior when checking native diff support
Signed-off-by: Jonas Geiler <git@jonasgeiler.com>
2024-03-21 22:17:54 +01:00
Jonas Geiler
f6b80253b8 overlay2: get rid of unnecessary kernel version check
Signed-off-by: Jonas Geiler <git@jonasgeiler.com>
2024-03-21 20:44:44 +01:00
Brian Goff
59c5059081 Merge pull request #47443 from corhere/cnmallocator/lift-n-shift
Vendor dependency cycle-free swarmkit
2024-03-21 12:29:46 -07:00
Sebastiaan van Stijn
1552e30a05 Merge pull request #47595 from tonistiigi/dockerfile-dlv-update
Dockerfile: avoid hardcoding arch combinations for delve
2024-03-21 15:46:47 +01:00
Paweł Gronowski
c64314fd55 Merge pull request #47610 from vvoland/dockerfile-cli-v26
Dockerfile: update docker CLI to v26.0.0
2024-03-21 15:35:41 +01:00
Bjorn Neergaard
61e2199b78 AUTHORS,.mailmap: update with recent contributors
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-03-21 07:34:37 -06:00
Paweł Gronowski
ea72f9f72c Dockerfile: update docker CLI to v26.0.0
Update the CLI that's used in the dev-container

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc2...v26.0.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-21 11:39:15 +01:00
Jonas Geiler
b2fd67de77 overlay2: support rootless native overlay diff in kernel 5.11 and above
Signed-off-by: Jonas Geiler <git@jonasgeiler.com>
2024-03-21 01:40:38 +01:00
Bjorn Neergaard
8b79278316 Merge pull request #47599 from neersighted/short_id_aliases_removal
api: document changed behavior of the `Aliases` field in v1.45
2024-03-20 08:33:39 -06:00
Bjorn Neergaard
22726fb63b api: document changed behavior of the Aliases field in v1.45
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-03-20 08:23:48 -06:00
Bjorn Neergaard
963e1f3eed Merge pull request #47597 from vvoland/c8d-list-fix-shared-size
c8d/list: Fix shared size calculation
2024-03-20 07:26:09 -06:00
Paweł Gronowski
3312b82515 c8d/list: Add a test case for images sharing a top layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 13:17:56 +01:00
Paweł Gronowski
ad8a5a5732 c8d/list: Fix diffIDs being outputted instead of chainIDs
The `identity.ChainIDs` call was accidentally removed in
b37ced2551.

This broke the shared size calculation for images with more than one
layer that were sharing the same compressed layer.

This was could be reproduced with:
```
$ docker pull docker.io/docker/desktop-kubernetes-coredns:v1.11.1
$ docker pull docker.io/docker/desktop-kubernetes-etcd:3.5.10-0
$ docker system df
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:50 +01:00
Paweł Gronowski
0c2d83b5fb c8d/list: Handle unpacked layers when calculating shared size
After a535a65c4b the size reported by the
image list was changed to include all platforms of that image.

This made the "shared size" calculation consider all diff ids of all the
platforms available in the image which caused "snapshot not found"
errors when multiple images were sharing the same layer which wasn't
unpacked.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:28 +01:00
Tonis Tiigi
f696e0d2a7 Dockerfile: avoid hardcoding arch combinations for delve
This is better because every possible platform combination
does not need to be defined in the Dockerfile. If built
for platform where Delve is not supported then it is just
skipped.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-03-19 10:22:35 -07:00
Paweł Gronowski
330d777c53 Merge pull request #47591 from vvoland/api-1.45
docs/api: add documentation for API v1.45
2024-03-19 14:27:45 +01:00
Paweł Gronowski
3d2a56e7cf docs/api: add documentation for API v1.45
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v26.0.0 release.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 13:37:05 +01:00
Paweł Gronowski
4531a371f2 Merge pull request #47580 from vvoland/c8d-list-slow
c8d/list: Generate image summary concurrently
2024-03-19 13:32:52 +01:00
Paweł Gronowski
731a64069f c8d/list: Generate image summary concurrently
Run `imageSummary` concurrently to avoid being IO blocked on the
containerd gRPC.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:58 +01:00
Paweł Gronowski
dade279565 c8d/list: Add Images benchmark
Benchmark the `Images` implementation (image list) against an image
store with 10, 100 and 1000 random images. Currently the images are
single-platform only.

The images are generated randomly, but a fixed seed is used so the
actual testing data will be the same across different executions.

Because the content store is not a real containerd image store but a
local implementation, a small delay (500us) is added to each content
store method call. This is to simulate a real-world usage where each
containerd client call requires a gRPC call.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:56 +01:00
Sebastiaan van Stijn
23e1af45c6 Merge pull request #47582 from vvoland/vendor-buildkit-0.13.1
vendor: github.com/moby/buildkit v0.13.1
2024-03-18 21:53:15 +01:00
Paweł Gronowski
e7c60a30e6 vendor: github.com/moby/buildkit v0.13.1
full diff: https://github.com/moby/buildkit/compare/v0.13.0...v0.13.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 20:16:09 +01:00
Bjorn Neergaard
641e341eed Merge pull request #47538 from robmry/libnet-resolver-nxdomain
libnet: Don't forward to upstream resolvers on internal nw
2024-03-18 11:22:59 -06:00
Sebastiaan van Stijn
dd146571ea Merge pull request #47568 from vvoland/c8d-list-fix
c8d/list: Fix premature `Images` return
2024-03-18 15:28:09 +01:00
Paweł Gronowski
fe70ee9477 Merge pull request #47577 from vvoland/c8d-list-labels-filter
c8d/list: Don't setup label filter if it's not specified
2024-03-18 15:13:40 +01:00
Sebastiaan van Stijn
307962dbd5 Merge pull request #47578 from thaJeztah/fix_resolvconf_go_version
resolvconf: add //go:build directives to prevent downgrading to go1.16 language
2024-03-18 14:00:03 +01:00
Sebastiaan van Stijn
7e56442cee Merge pull request #47574 from thaJeztah/bump_tools
Dockerfile: update docker CLI to v26.0.0-rc2, docker compose v2.25.0
2024-03-18 13:59:42 +01:00
Sebastiaan van Stijn
ebf300c165 Merge pull request #47579 from vvoland/flaky-testdiskusage
integration: Remove Parallel from TestDiskUsage
2024-03-18 13:59:28 +01:00
Paweł Gronowski
2e4ebf032a c8d/list: Pass ctx to setupLabelFilter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 13:50:45 +01:00
Paweł Gronowski
153de36b3f c8d/list: Add empty index test case
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:50:10 +01:00
Sebastiaan van Stijn
4ff655f4b8 resolvconf: add //go:build directives to prevent downgrading to go1.16 language
Commit 8921897e3b introduced the uses of `clear()`,
which requires go1.21, but Go is downgrading this file to go1.16 when used in
other projects (due to us not yet being a go module);

    0.175 + xx-go build '-gcflags=' -ldflags '-X github.com/moby/buildkit/version.Version=b53a13e -X github.com/moby/buildkit/version.Revision=b53a13e4f5c8d7e82716615e0f23656893df89af -X github.com/moby/buildkit/version.Package=github.com/moby/buildkit -extldflags '"'"'-static'"'" -tags 'osusergo netgo static_build seccomp ' -o /usr/bin/buildkitd ./cmd/buildkitd
    181.8 # github.com/docker/docker/libnetwork/internal/resolvconf
    181.8 vendor/github.com/docker/docker/libnetwork/internal/resolvconf/resolvconf.go:509:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-18 12:28:21 +01:00
Paweł Gronowski
1c03312378 integration: Remove Parallel from TestDiskUsage
Check if removing the Parallel execution from that test fixes its
flakiness.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:11:34 +01:00
Paweł Gronowski
f512dba037 c8d/list: Fix premature Images return
52a80b40e2 extracted the `imageSummary`
function but introduced a bug causing the whole caller function to
return if the image should be skipped.

`imageSummary` returns a nil error and nil image when the image doesn't
have any platform or all its platforms are not available locally.
In this case that particular image should be skipped, instead of failing
the whole image list operation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:43:12 +01:00
Paweł Gronowski
89dc2860ba c8d/list: Handle missing configs in label filter
Don't error out the filter if an image config is missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:29:16 +01:00
Paweł Gronowski
6f3892dc99 c8d/list: Don't setup label filter if it's not specified
Don't run filter function which would only run through the images
reading theirs config without checking any label anyway.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:23:31 +01:00
Paweł Gronowski
a9bca45e92 Merge pull request #47575 from thaJeztah/bump_shfmt
Dockerfile: update mvdan/shfmt to v3.8.0
2024-03-18 09:26:35 +01:00
Sebastiaan van Stijn
fe8fb9b9a1 Dockerfile: update mvdan/shfmt to v3.8.0
- full diff: https://github.com/mvdan/sh/compare/v3.7.0...v3.8.0
- 3.7.0 release notes: https://github.com/mvdan/sh/releases/tag/v3.7.0
- 3.8.0 release notes: https://github.com/mvdan/sh/releases/tag/v3.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-17 13:36:43 +01:00
Sebastiaan van Stijn
70e46f2c7c Merge pull request #47559 from AkihiroSuda/fix-47436
rootless: fix `open /etc/docker/plugins: permission denied`
2024-03-16 15:54:09 +01:00
Sebastiaan van Stijn
23339a6147 Merge pull request #47570 from thaJeztah/bump_xx_1.4
Dockerfile: update xx to v1.4.0
2024-03-16 15:53:49 +01:00
Sebastiaan van Stijn
4bd30829d1 Dockerfile: update docker compose to v2.25.0
Update the version of compose that's used in the dev-container.

- full diff: https://github.com/docker/compose/compare/v2.24.7...v2.25.0
- release notes: https://github.com/docker/compose/releases/tag/v2.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-16 14:22:42 +01:00
Sebastiaan van Stijn
971562b005 Dockerfile: update docker CLI to v26.0.0-rc2
Update the CLI that's used in the dev-container to the latest rc

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc1...v26.0.0-rc2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-16 14:22:30 +01:00
Akihiro Suda
d742659877 rootless: fix open /etc/docker/plugins: permission denied
Fix issue 47436

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-16 22:03:34 +09:00
Sebastiaan van Stijn
4f46c44725 Dockerfile: update xx to v1.4.0
full diff: https://github.com/tonistiigi/xx/compare/v1.2.1...v1.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 19:59:48 +01:00
Bjorn Neergaard
1f539a6e85 Merge pull request #47569 from thaJeztah/make_fix_empty_check
Makefile: generate-files: fix check for empty TMP_OUT
2024-03-15 12:07:00 -06:00
Bjorn Neergaard
959c2ee6cf Merge pull request #47558 from AkihiroSuda/fix-47248
plugin: fix mounting /etc/hosts when running in UserNS
2024-03-15 12:06:48 -06:00
Sebastiaan van Stijn
25c9e6e8df Makefile: generate-files: fix check for empty TMP_OUT
commit c655b7dc78 added a check to make sure
the TMP_OUT variable was not set to an empty value, as such a situation would
perform an `rm -rf /**` during cleanup.

However, it was a bit too eager, because Makefile conditionals (`ifeq`) are
evaluated when parsing the Makefile, which happens _before_ the make target
is executed.

As a result `$@_TMP_OUT` was always empty when the `ifeq` was evaluated,
making it not possible to execute the `generate-files` target.

This patch changes the check to use a shell command to evaluate if the var
is set to an empty value.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 17:54:54 +01:00
Akihiro Suda
762ec4b60c plugin: fix mounting /etc/hosts when running in UserNS
Fix `error mounting "/etc/hosts" to rootfs at "/etc/hosts": mount
/etc/hosts:/etc/hosts (via /proc/self/fd/6), flags: 0x5021: operation
not permitted`.

This error was introduced in 7d08d84b03
(`dockerd-rootless.sh: set rootlesskit --state-dir=DIR`) that changed
the filesystem of the state dir from /tmp to /run (in a typical setup).

Fix issue 47248

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-15 22:16:34 +09:00
Sebastiaan van Stijn
979f03f9f6 Merge pull request #47567 from thaJeztah/move_rootless_mountopts
daemon: move getUnprivilegedMountFlags to internal package
2024-03-15 14:13:23 +01:00
Sebastiaan van Stijn
7b414f5703 daemon: move getUnprivilegedMountFlags to internal package
This code is currently only used in the daemon, but is also needed in other
places. We should consider moving this code to github.com/moby/sys, so that
BuildKit can also use the same implementation instead of maintaining a fork;
moving it to internal allows us to reuse this code inside the repository, but
does not allow external consumers to depend on it (which we don't want as
it's not a permanent location).

As our code only uses this in linux files, I did not add a stub for other
platforms (but we may decide to do that in the moby/sys repository).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 12:55:09 +01:00
Sebastiaan van Stijn
ff05850e7e Merge pull request #47563 from vvoland/buildkit-runc-override
builder-next: Add env-var to override runc used by buildkit
2024-03-14 20:17:01 +01:00
Sebastiaan van Stijn
cdf70c0a51 Merge pull request #47430 from vvoland/inspect-remove-container
api/image-inspect: Remove Container and ContainerConfig
2024-03-14 19:27:43 +01:00
Sebastiaan van Stijn
40c681355e Merge pull request #47562 from thaJeztah/update_protobuf
vendor: google.golang.org/protobuf v1.33.0, github.com/golang/protobuf v1.5.4
2024-03-14 19:14:00 +01:00
Albin Kerouanton
790c3039d0 libnet: Don't forward to upstream resolvers on internal nw
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.

This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-14 17:46:48 +00:00
Paweł Gronowski
10bdc7136c builder-next: Add env-var to override runc used by buildkit
Adds an experimental `DOCKER_BUILDKIT_RUNC_COMMAND` variable that allows
to specify different runc-compatible binary to be used by the buildkit's
runc executor.

This allows runtimes like sysbox be used for the containers spawned by
buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-14 17:35:26 +01:00
Paweł Gronowski
a8abb67c5e Merge pull request #47561 from thaJeztah/bump_tools
Dockerfile: update buildx to v0.13.1,  compose v2.24.7
2024-03-14 13:46:24 +01:00
Sebastiaan van Stijn
1ca89d7eae vendor: google.golang.org/protobuf v1.33.0, github.com/golang/protobuf v1.5.4
full diffs:

- https://github.com/protocolbuffers/protobuf-go/compare/v1.31.0...v1.33.0
- https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4

From the Go security announcement list;

> Version v1.33.0 of the google.golang.org/protobuf module fixes a bug in
> the google.golang.org/protobuf/encoding/protojson package which could cause
> the Unmarshal function to enter an infinite loop when handling some invalid
> inputs.
>
> This condition could only occur when unmarshaling into a message which contains
> a google.protobuf.Any value, or when the UnmarshalOptions.UnmarshalUnknown
> option is set. Unmarshal now correctly returns an error when handling these
> inputs.
>
> This is CVE-2024-24786.

In a follow-up post;

> A small correction: This vulnerability applies when the UnmarshalOptions.DiscardUnknown
> option is set (as well as when unmarshaling into any message which contains a
> google.protobuf.Any). There is no UnmarshalUnknown option.
>
> In addition, version 1.33.0 of google.golang.org/protobuf inadvertently
> introduced an incompatibility with the older github.com/golang/protobuf
> module. (https://github.com/golang/protobuf/issues/1596) Users of the older
> module should update to github.com/golang/protobuf@v1.5.4.

govulncheck results in our code:

    govulncheck ./...
    Scanning your code and 1221 packages across 204 dependent modules for known vulnerabilities...

    === Symbol Results ===

    Vulnerability #1: GO-2024-2611
        Infinite loop in JSON unmarshaling in google.golang.org/protobuf
      More info: https://pkg.go.dev/vuln/GO-2024-2611
      Module: google.golang.org/protobuf
        Found in: google.golang.org/protobuf@v1.31.0
        Fixed in: google.golang.org/protobuf@v1.33.0
        Example traces found:
          #1: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Peek
          #2: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Read
          #3: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls protojson.Unmarshal

    Your code is affected by 1 vulnerability from 1 module.
    This scan found no other vulnerabilities in packages you import or modules you
    require.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 13:12:54 +01:00
Sebastiaan van Stijn
f40bdf5f63 Dockerfile: update compose to v2.24.7
full diff: https://github.com/docker/compose/compare/v2.24.5...v2.24.7

release notes:

- https://github.com/docker/compose/releases/tag/v2.24.6
- https://github.com/docker/compose/releases/tag/v2.24.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 12:40:26 +01:00
Sebastiaan van Stijn
3f73d23ea0 Dockerfile: update buildx to v0.13.1
release notes:

- https://github.com/docker/buildx/releases/tag/v0.13.1
- https://github.com/docker/buildx/releases/tag/v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 12:38:06 +01:00
Paweł Gronowski
77b05b97f4 Merge pull request #47556 from vvoland/deprecate-notls
Be more explicit about non-TLS TCP access deprecation
2024-03-14 12:07:42 +01:00
Lei Jitang
e3bc82f7d4 Merge pull request #47542 from eriksjolund/47407-clarify-git-clone
set-up-git.md: clarify URL in git clone command
2024-03-14 17:24:02 +08:00
Sebastiaan van Stijn
342923b01c Merge pull request #47555 from rumpl/feat-c8d-prom
c8d: Prometheus metrics
2024-03-13 17:35:14 +01:00
Sebastiaan van Stijn
15122b3b1c Merge pull request #47350 from vvoland/cache-refactor
c8d/cache: Use the same cache logic as graphdrivers
2024-03-13 17:19:36 +01:00
Djordje Lukic
388ecf65bc c8d: Send push metrics to prom
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
bb3ab1edb7 c8d: Send pull metrics to prom
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
da245cab15 c8d: Send history metrics to prometheus
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
1cfd763214 c8d: Send image delete metrics to prometheus
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
0ce714a085 images: Export the image actions prometheus counter
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:36 +01:00
Paweł Gronowski
bcb4794eea Be more explicit about non-TLS TCP access deprecation
Turn warnings into a deprecation notice and highlight that it will
prevent daemon startup in future releases.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 14:22:10 +01:00
Paweł Gronowski
0d5ef431a1 docker-py: Temporarily skip test_commit and test_commit_with_changes
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 13:03:48 +01:00
Paweł Gronowski
03cddc62f4 api/image-inspect: Remove Container and ContainerConfig
Don't include these fields starting from API v1.45.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 13:02:29 +01:00
Akihiro Suda
825635a5bf Merge pull request #47552 from thaJeztah/vendor_containerd_1.7.14
vendor: github.com/containerd/containerd v1.7.14
2024-03-13 11:57:52 +09:00
Lars Andringa
d4f61f92fd Move StartedAt time to before starting the container
Signed-off-by: Lars Andringa <l.s.andringa@rug.nl>
Signed-off-by: LarsSven <l.s.andringa@rug.nl>

Replaced boolean parameter by IsZero check

Signed-off-by: LarsSven <l.s.andringa@rug.nl>

Separated SetRunning into two functions

Signed-off-by: LarsSven <l.s.andringa@rug.nl>

Apply suggestions from code review

Documentation fixes

Co-authored-by: Paweł Gronowski <me@woland.xyz>
Signed-off-by: LarsSven <l.s.andringa@rug.nl>
2024-03-12 16:20:21 +01:00
Sebastiaan van Stijn
ec19fd6fed vendor: github.com/containerd/containerd v1.7.14
- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.14
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.14

Welcome to the v1.7.14 release of containerd!

The fourteenth patch release for containerd 1.7 contains various fixes and updates.

Highlights

- Update builds to use go 1.21.8
- Fix various timing issues with docker pusher
- Register imagePullThroughput and count with MiB
- Move high volume event logs to Trace level

Container Runtime Interface (CRI)

- Handle pod transition states gracefully while listing pod stats

Runtime

- Update runc-shim to process exec exits before init

Dependency Changes

- github.com/containerd/nri v0.4.0 -> v0.6.0
- github.com/containerd/ttrpc v1.2.2 -> v1.2.3
- google.golang.org/genproto/googleapis/rpc 782d3b101e98 -> cbb8c96f2d6d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-12 12:46:19 +01:00
Sebastiaan van Stijn
d19f6d4b6d vendor: github.com/containerd/ttrpc v1.2.3
full diff: https://github.com/containerd/ttrpc/compare/v1.2.2..v1.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-12 12:37:44 +01:00
Sebastiaan van Stijn
b8165a9cd1 Merge pull request #47494 from vvoland/devcontainer-golang
devcontainer: Add Golang extension
2024-03-11 17:50:13 +01:00
Erik Sjölund
a6a445d86b set-up-git.md: clarify URL in git clone command
Fixes https://github.com/moby/moby/issues/47407

Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
2024-03-09 16:42:44 +01:00
Sebastiaan van Stijn
0fb845858d Merge pull request #47505 from akerouanton/fix-TestBridgeICC-ipv6
inte/networking:  ping with -6 specified when needed
2024-03-08 18:33:46 +01:00
Paweł Gronowski
db2263749b Merge pull request #47530 from vvoland/flaky-liverestore
volume: Don't decrement refcount below 0
2024-03-08 12:28:10 +01:00
Sebastiaan van Stijn
1abf17c779 Merge pull request #47512 from robmry/46329_internal_resolver_ipv6_upstream
Add IPv6 nameserver to the internal DNS's upstreams.
2024-03-07 21:21:12 +01:00
Paweł Gronowski
294fc9762e volume: Don't decrement refcount below 0
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.

This makes sure we don't underflow the refcount (uint64) when
decrementing it.

The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 20:42:20 +01:00
Paweł Gronowski
eef352b565 devcontainer: Use a separate devcontainer target
Use a separate `devcontainer` Dockerfile target, this allows to include
the `gopls` in the devcontainer so it doesn't have to be installed by
the Go vscode extension.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 20:39:56 +01:00
Paweł Gronowski
f4c696eef1 Merge pull request #47449 from vvoland/c8d-list-single
c8d/list: Add test and combine size
2024-03-07 18:49:19 +01:00
Albin Kerouanton
5a009cdd5b inte/networking: add isIPv6 flag
Make sure the `ping` command used by `TestBridgeICC` actually has
the `-6` flag when it runs IPv6 test cases. Without this flag,
IPv6 connectivity isn't tested properly.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-07 17:55:53 +01:00
Paweł Gronowski
2f1a32e3e5 c8d/list: Skip images with non matching platform
Currently this won't have any real effect because the platform matcher
matches all platform and is only used for sorting.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:12 +01:00
Paweł Gronowski
72f1f82f28 c8d/list: Remove outdated TODO
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:10 +01:00
Paweł Gronowski
52a80b40e2 c8d/list: Extract imageSummary function
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:09 +01:00
Paweł Gronowski
288a14e264 c8d/list: Simplify "best" image selection
Don't save all present images,  inline the sorting into the loop
instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:07 +01:00
Paweł Gronowski
b37ced2551 c8d/list: Count containers by their manifest
Move containers counting out of `singlePlatformImage` and count them
based on the `ImageManifest` property.

(also remove ChainIDs calculation as they're no longer used)

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:26:53 +01:00
Paweł Gronowski
a535a65c4b c8d/list: Combine size
Multi-platform images are coalesced into one entry now.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:32 +01:00
Paweł Gronowski
582de4bc3c c8d/list: Add TestImageList
Add unit test for `Images` implementation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:31 +01:00
Paweł Gronowski
a6e7e67d3a specialimage: Return optional ocispec.Index
To ease accessing image descriptors in tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:30 +01:00
Paweł Gronowski
1b108bdfeb daemon/c8d: Cache SnapshotService
Avoid fetching `SnapshotService` from client every time. Fetch it once
and then store when creating the image service.

This also allows to pass custom snapshotter implementation for unit
testing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:29 +01:00
Paweł Gronowski
74e2f23e1a daemon/c8d: Use i.images and i.content
Use `image.Store` and `content.Store` stored in the ImageService struct
instead of fetching it every time from containerd client.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:27 +01:00
Paweł Gronowski
e8496b1ee4 imageService: Extract common code from MakeImageCache
Both containerd and graphdriver image service use the same code to
create the cache - they only supply their own `cacheAdaptor` struct.

Extract the shared code to `cache.New`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 15:39:42 +01:00
Paweł Gronowski
d66177591e c8d/cache: Use the same cache logic as graphdrivers
Implement the cache adaptor for containerd image store and use the same
cache logic.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:06 +01:00
Paweł Gronowski
bf30fee58a image/cache: Refactor backend specific code
Move image store backend specific code out of the cache code and move it
to a separate interface to allow using the same cache code with
containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:04 +01:00
Paweł Gronowski
608d77d740 Merge pull request #47497 from robmry/resolvconf_fixes
Fix 'resolv.conf' parsing issues
2024-03-07 13:05:10 +01:00
Paweł Gronowski
66adfc729a Merge pull request #47521 from robmry/no_ipv6_addr_when_ipv6_disabled
Don't configure IPv6 addr/gw when IPv6 disabled.
2024-03-07 12:57:27 +01:00
Sebastiaan van Stijn
ab4b5a4890 Merge pull request #47519 from thaJeztah/dupword
golangci-lint: enable dupword linter
2024-03-07 12:41:54 +01:00
Sebastiaan van Stijn
f5a5e3f203 golangci-lint: enable dupword linter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 11:44:27 +01:00
Sebastiaan van Stijn
773f792b88 Merge pull request #47523 from tonistiigi/snapshot-lock-fix
builder-next: fix missing lock in ensurelayer
2024-03-07 11:17:25 +01:00
Sebastiaan van Stijn
4adc40ac40 fix duplicate words (dupwords)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 10:57:03 +01:00
Rob Murray
8921897e3b Ignore bad ndots in host resolv.conf
Rather than error out if the host's resolv.conf has a bad ndots option,
just ignore it. Still validate ndots supplied via '--dns-option' and
treat failure as an error.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-07 09:27:34 +00:00
Tonis Tiigi
37545cc644 builder-next: fix missing lock in ensurelayer
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.

In that case the reference count got mixed up and on
release only one of these layers was actually released.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-03-06 23:11:32 -08:00
Bjorn Neergaard
6c10086976 Merge pull request #47520 from vvoland/buildkit-v13-leaseutil
builder-next/export: Use leaseutil for descref lease
2024-03-06 11:55:02 -07:00
Rob Murray
ef5295cda4 Don't configure IPv6 addr/gw when IPv6 disabled.
When IPv6 is disabled in a container by, for example, using the --sysctl
option - an IPv6 address/gateway is still allocated. Don't attempt to
apply that config because doing so enables IPv6 on the interface.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-06 18:32:31 +00:00
Paweł Gronowski
49b77753cb builder-next/export: Use leaseutil for descref lease
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 17:47:37 +01:00
Paweł Gronowski
d91665a2ac Merge pull request #47511 from vvoland/buildkit-v13.0
vendor: github.com/moby/buildkit v0.13.0
2024-03-06 17:41:55 +01:00
Sebastiaan van Stijn
4e53936f0a Merge pull request #47509 from thirdkeyword/master
remove repetitive words
2024-03-06 13:52:16 +01:00
Sebastiaan van Stijn
cb8c8e9631 Merge pull request #47498 from Dzejrou/lower-perm-fix
daemon: overlay2: remove world writable permission from the lower file
2024-03-06 13:09:30 +01:00
Paweł Gronowski
c4fc6c3371 builder-next/executor: Replace removed network.Sample
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 12:02:12 +01:00
Paweł Gronowski
0f30791a0d vendor: github.com/moby/buildkit v0.13.0
full diff: https://github.com/moby/buildkit/compare/v0.13.0-rc3...v0.13.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 11:54:43 +01:00
Rob Murray
4e8d9a4522 Add IPv6 nameserver to the internal DNS's upstreams.
When configuring the internal DNS resolver - rather than keep IPv6
nameservers read from the host's resolv.conf in the container's
resolv.conf, treat them like IPv4 addresses and use them as upstream
resolvers.

For IPv6 nameservers, if there's a zone identifier in the address or
the container itself doesn't have IPv6 support, mark the upstream
addresses for use in the host's network namespace.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-06 10:47:18 +00:00
Albin Kerouanton
7c7e453255 Merge pull request #47474 from robmry/47441_mac_addr_config_migration
Don't create endpoint config for MAC addr config migration
2024-03-06 11:04:17 +01:00
thirdkeyword
06628e383a remove repetitive words
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-06 18:03:51 +08:00
Sebastiaan van Stijn
4046928978 Merge pull request #47504 from AkihiroSuda/rootlesskit-2.0.2
update RootlessKit to 2.0.2
2024-03-06 10:12:32 +01:00
Albin Kerouanton
21835a5696 inte/networking: rename linkLocal flag into isLinkLocal
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-06 00:16:08 +01:00
Akihiro Suda
b32cfc3b3a dockerd-rootless-setuptool.sh: check RootlessKit functionality
RootlessKit will print hints if something is still unsatisfied.

e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint
rootless-containers/rootlesskit@33c3e7ca6c

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:43:00 +09:00
Akihiro Suda
49fd8df9b9 Dockerfile: update RootlessKit to v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:38:55 +09:00
Akihiro Suda
72ec187dfe go.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:38:00 +09:00
Akihiro Suda
83cda67f73 go.mod: golang.org/x/sys v0.18.0
https://github.com/golang/sys/compare/v0.16.0...v0.18.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:37:37 +09:00
Paweł Gronowski
460b4aebdf Merge pull request #47502 from vvoland/go-1.21.8
update to go1.21.8
2024-03-05 21:58:11 +01:00
Paweł Gronowski
57b7ffa7f6 update to go1.21.8
go1.21.8 (released 2024-03-05) includes 5 security fixes

- crypto/x509: Verify panics on certificates with an unknown public key algorithm (CVE-2024-24783, https://go.dev/issue/65390)
- net/http: memory exhaustion in Request.ParseMultipartForm (CVE-2023-45290, https://go.dev/issue/65383)
- net/http, net/http/cookiejar: incorrect forwarding of sensitive headers and cookies on HTTP redirect (CVE-2023-45289, https://go.dev/issue/65065)
- html/template: errors returned from MarshalJSON methods may break template escaping (CVE-2024-24785, https://go.dev/issue/65697)
- net/mail: comments in display names are incorrectly handled (CVE-2024-24784, https://go.dev/issue/65083)

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.1

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.7...go1.21.8

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-05 19:17:18 +01:00
Jaroslav Jindrak
cadb124ab6 daemon: overlay2: remove world writable permission from the lower file
In de2447c, the creation of the 'lower' file was changed from using
os.Create to using ioutils.AtomicWriteFile, which ignores the system's
umask. This means that even though the requested permission in the
source code was always 0666, it was 0644 on systems with default
umask of 0022 prior to de2447c, so the move to AtomicFile potentially
increased the file's permissions.

This is not a security issue because the parent directory does not
allow writes into the file, but it can confuse security scanners on
Linux-based systems into giving false positives.

Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
2024-03-05 14:25:50 +01:00
Sebastiaan van Stijn
046827c657 Merge pull request #47485 from vvoland/vendor-dns
vendor: github.com/miekg/dns v1.1.57
2024-03-04 11:55:01 +01:00
Sebastiaan van Stijn
04c9d7f6a3 Merge pull request #47465 from vvoland/v26-remove-deprecated
api/search: Reset `is_automated` to false
2024-03-04 11:27:24 +01:00
Paweł Gronowski
b2921509e5 api/search: Reset is_automated field to false
The field will still be present in the response, but will always be
`false`.
Searching for `is-automated=true` will yield no results, while
`is-automated=false` will effectively be a no-op.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-04 10:15:59 +01:00
Rob Murray
f04f69e366 Accumulate resolv.conf options
If there are multiple "options" lines, keep the options from all of
them.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-01 16:59:28 +00:00
Rob Murray
7f69142aa0 resolv.conf comments have '#' or ';' in the first column
When a '#' or ';' appears anywhere else, it's not a comment marker.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-01 16:58:04 +00:00
Sebastiaan van Stijn
97a5435d33 Merge pull request #47477 from robmry/resolvconf_gocompat
Remove slices.Clone() calls to avoid Go bug
2024-03-01 17:28:01 +01:00
Rob Murray
91d9307738 Replace uses of slices.Clone()
Avoid https://github.com/golang/go/issues/64759

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-01 15:27:29 +00:00
Paweł Gronowski
12dea3fa9e devcontainer: Add Golang extension automatically
When using devcontainers in VSCode, install the Go extension
automatically in the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-01 14:54:35 +01:00
Sebastiaan van Stijn
137a9d6a4c Merge pull request #47395 from robmry/47370_windows_natnw_dns_test
Test DNS on Windows 'nat' networks
2024-03-01 13:02:52 +01:00
Paweł Gronowski
9f4e824a6e vendor: github.com/miekg/dns v1.1.57
full diff: https://github.com/github.com/miekg/dns/compare/v1.1.43...v1.1.57

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-01 10:14:22 +01:00
Albin Kerouanton
f8e6801533 Merge pull request #47478 from fopina/patch-1
fix typo in error message
2024-03-01 08:53:43 +01:00
Filipe Pina
ef681124ca fix typo in error message
Signed-off-by: Filipe Pina <hzlu1ot0@duck.com>
2024-02-29 23:27:00 +00:00
Cory Snider
7ebd88d2d9 hack: block imports of vendored testify packages
While github.com/stretchr/testify is not used directly by any of the
repository code, it is a transitive dependency via Swarmkit and
therefore still easy to use without having to revendor. Add lint rules
to ban importing testify packages to make sure nobody does.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-29 16:20:15 -05:00
Cory Snider
4f30a930ad libn/cnmallocator: migrate tests to gotest.tools/v3
Apply command gotest.tools/v3/assert/cmd/gty-migrate-from-testify to the
cnmallocator package to be consistent with the assertion library used
elsewhere in moby.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-29 16:14:02 -05:00
Rob Murray
a580544d82 Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-29 17:02:19 +00:00
Sebastiaan van Stijn
b8aa8579ca Merge pull request #47352 from serhii-nakon/allow_host_loopback
Allow to enable host loopback and use 10.0.2.2 to connect to the host (OPTIONALLY)
2024-02-29 17:58:28 +01:00
Paweł Gronowski
225ccc0cfd Merge pull request #47473 from vvoland/cli-v26
Dockerfile: Update dev cli to v26.0.0-rc1
2024-02-29 16:02:16 +01:00
Paweł Gronowski
d19d98b136 Merge pull request #47475 from thaJeztah/nothing_to_see_here_move_along_move_along
distribution/xfer: fix pull progress message
2024-02-29 14:46:41 +01:00
Sebastiaan van Stijn
ebf3f8c7fe distribution/xfer: fix pull progress message
This message accidentally changed in ac2a028dcc
because my IDE's "refactor tool" was a bit over-enthusiastic. It also went and
updated the tests accordingly, so CI didn't catch this :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-29 14:02:55 +01:00
Sebastiaan van Stijn
a242208be3 Merge pull request #47457 from vvoland/ci-report-timeout
ci: Update `teststat` to v0.1.25
2024-02-29 13:39:09 +01:00
Paweł Gronowski
2af2496c8c Dockerfile: Update dev cli to v26.0.0-rc1
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 12:45:17 +01:00
Paweł Gronowski
fc0e5401f2 ci: Update teststat to v0.1.25
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 10:06:07 +01:00
Bjorn Neergaard
8517c3386c Merge pull request #47458 from vvoland/ci-reports-better-find
ci: Make `find` for test reports more specific
2024-02-29 01:47:07 -07:00
Sebastiaan van Stijn
6c3b3523c9 Merge pull request #47041 from robmry/46968_refactor_resolvconf
Refactor 'resolv.conf' generation.
2024-02-29 09:33:55 +01:00
Bjorn Neergaard
d40b140c08 Merge pull request #47440 from thaJeztah/fix_ping_connection_errs
client: fix connection-errors being shadowed by API version errors
2024-02-28 13:33:49 -07:00
Sebastiaan van Stijn
81428bf11b Merge pull request #47459 from thaJeztah/disable_schema1
disable pulling legacy image formats by default
2024-02-28 17:12:31 +01:00
Sebastiaan van Stijn
230cb53d3b Merge pull request #47462 from vvoland/integration-testdaemonproxy-reset-otel
integration: Reset `OTEL_EXPORTER_OTLP_ENDPOINT` for sub-daemons
2024-02-28 17:11:54 +01:00
Cory Snider
7b0ab1011c Vendor dependency cycle-free swarmkit
Moby imports Swarmkit; Swarmkit no longer imports Moby. In order to
accomplish this feat, Swarmkit has introduced a new plugin.Getter
interface so it could stop importing our pkg/plugingetter package. This
new interface is not entirely compatible with our
plugingetter.PluginGetter interface, necessitating a thin adapter.

Swarmkit had to jettison the CNM network allocator to stop having to
import libnetwork as the cnmallocator package is deeply tied to
libnetwork. Move the CNM network allocator into libnetwork, where it
belongs. The package had a short an uninteresting Git history in the
Swarmkit repository so no effort was made to retain history.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-28 09:46:45 -05:00
Sebastiaan van Stijn
3ca1d751e5 Merge pull request #47461 from vvoland/vendor-buildkit-0.13.0-rc3
vendor: github.com/moby/buildkit v0.13.0-rc3
2024-02-28 14:12:43 +01:00
Sebastiaan van Stijn
589dc5e647 Merge pull request #47456 from huang-jl/fix_restore_digest
libcontainerd: change the digest used when restoring
2024-02-28 14:05:40 +01:00
Sebastiaan van Stijn
62b33a2604 disable pulling legacy image formats by default
This patch disables pulling legacy (schema1 and schema 2, version 1) images by
default.

A `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE` environment-variable is
introduced to allow re-enabling this feature, aligning with the environment
variable used in containerd 2.0 (`CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`).

With this patch, attempts to pull a legacy image produces an error:

With graphdrivers:

    docker pull docker:1.0
    1.0: Pulling from library/docker
    [DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

With the containerd image store enabled, output is slightly different
as it returns the error before printing the `1.0: pulling ...`:

    docker pull docker:1.0
    Error response from daemon: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

Using the "distribution" endpoint to resolve the digest for an image also
produces an error:

    curl -v --unix-socket /var/run/docker.sock http://foo/distribution/docker.io/library/docker:1.0/json
    *   Trying /var/run/docker.sock:0...
    * Connected to foo (/var/run/docker.sock) port 80 (#0)
    > GET /distribution/docker.io/library/docker:1.0/json HTTP/1.1
    > Host: foo
    > User-Agent: curl/7.88.1
    > Accept: */*
    >
    < HTTP/1.1 400 Bad Request
    < Api-Version: 1.45
    < Content-Type: application/json
    < Docker-Experimental: false
    < Ostype: linux
    < Server: Docker/dev (linux)
    < Date: Tue, 27 Feb 2024 16:09:42 GMT
    < Content-Length: 354
    <
    {"message":"[DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"}
    * Connection #0 to host foo left intact

Starting the daemon with the `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`
env-var set to a non-empty value allows pulling the image;

    docker pull docker:1.0
    [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
    b0a0e6710d13: Already exists
    d193ad713811: Already exists
    ba7268c3149b: Already exists
    c862d82a67a2: Already exists
    Digest: sha256:5e7081837926c7a40e58881bbebc52044a95a62a2ea52fb240db3fc539212fe5
    Status: Image is up to date for docker:1.0
    docker.io/library/docker:1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-28 13:57:43 +01:00
Paweł Gronowski
5fe96e234d integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:07 +01:00
Paweł Gronowski
84eecc4a30 Revert "integration/TestDaemonProxy: Remove OTEL span"
This reverts commit 56aeb548b2.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:03 +01:00
Paweł Gronowski
261dccc98a builder-next: Add Info to emptyProvider
To satisfy the `content.InfoReaderProvider` interface.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:20:55 +01:00
Paweł Gronowski
2c9c5e1c03 vendor: github.com/moby/buildkit v0.13.0-rc3
full diff: https://github.com/moby/buildkit/compare/v0.13.0-rc2...v0.13.0-rc3

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:05:02 +01:00
serhii.n
b649e272bb Allow to enable host loopback and use 10.0.2.2 to connect to the host (OPTIONALLY)
This should allow to enable host loopback by setting
DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK to false,
defaults true.

Signed-off-by: serhii.n <serhii.n@thescimus.com>
2024-02-28 00:52:35 +02:00
Paweł Gronowski
e4de4dea5c ci: Make find for test reports more specific
Don't use all `*.json` files blindly, take only these that are likely to
be reports from go test.
Also, use `find ... -exec` instead of piping results to `xargs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 23:38:03 +01:00
Sebastiaan van Stijn
b37f8c8070 Merge pull request #47460 from thaJeztah/bump_bolt
vendor: go.etcd.io/bbolt v1.3.9
2024-02-27 20:01:52 +01:00
Sebastiaan van Stijn
9be820d8ca vendor: go.etcd.io/bbolt v1.3.9
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.7...v1.3.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-27 18:24:01 +01:00
Sebastiaan van Stijn
f6fa6ff9ed Merge pull request #47391 from vvoland/rro-backwards-compatible
api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-02-27 18:04:46 +01:00
Sebastiaan van Stijn
220835106b Merge pull request #47364 from vvoland/buildkit-v13
vendor: github.com/moby/buildkit v0.13.0-rc2
2024-02-27 16:38:04 +01:00
Paweł Gronowski
2c25ca9dba Merge pull request #47455 from vvoland/c8d-skip-last-windows-tests
c8d/windows: Temporarily skip two failing tests
2024-02-27 14:01:31 +01:00
Paweł Gronowski
94f9f39b24 Merge pull request #47454 from vvoland/c8d-pull-pullingfslayer-truncated
c8d/pull: Output truncated id for `Pulling fs layer`
2024-02-27 13:28:38 +01:00
huang-jl
da643c0b8a libcontainerd: change the digest used when restoring
For current implementation of Checkpoint Restore (C/R) in docker, it
will write the checkpoint to content store. However, when restoring
libcontainerd uses .Digest().Encoded(), which will remove the info
of alg, leading to error.

Signed-off-by: huang-jl <1046678590@qq.com>
2024-02-27 20:17:31 +08:00
Rob Murray
9083c2f10d Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-27 11:40:11 +00:00
Paweł Gronowski
44167988c3 c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 12:04:21 +01:00
Paweł Gronowski
2d31532a00 otel: Default metrics protocol to http/protobuf
Buildkit added support for exporting metrics in:
7de2e4fb32

Explicitly set the protocol for exporting metrics like we do for the
traces. We need that because Buildkit defaults to grpc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:12 +01:00
CrazyMax
60358bfcab ci(buildkit): dedicated step to build test image
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:11 +01:00
Paweł Gronowski
f5722da5e0 mobyexporter: Store temporary config descriptor
Temporarily store the produced config descriptor for the buildkit
history to work.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:09 +01:00
Paweł Gronowski
951e42cd60 builder-next: Replace ResolveImageConfig with ResolveSourceMetadata
30c069cb03
removed the `ResolveImageConfig` method in favor of more generic
`ResolveSourceMetadata` that can also support other things than images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:37 +01:00
Paweł Gronowski
e01a1c5d09 builder/mobyexporter: Set image.name response key
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:35 +01:00
Paweł Gronowski
fa467caf4d builder-next/mobyexporter: Use OptKeyName const
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:33 +01:00
Paweł Gronowski
59ad1690f7 builder-next: Adjust to source changes
Adjust to cache sources changes from:
6b27487fec

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:17 +01:00
Paweł Gronowski
b04a2dad6b builder/controller: Adjust NewWorkerOpt call
8bfd280ab7
added a new argument that allows to specify different runtime.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:15 +01:00
Paweł Gronowski
bc6d88c09a cmd/dockerd: Fix overriding OTEL resource
e358792815
changed that field to a function and added an `OverrideResource`
function that allows to override it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:14 +01:00
Paweł Gronowski
a79bb1e832 builder-next/exporter: Sync with new signature
1c1777b7c0
added an explicit id argument to the Resolve method.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:12 +01:00
Paweł Gronowski
e68f71259a integration/build: Use fsutil.NewFS
StaticDirSource definition changed and can no longer be initialized from
the composite literal.

a80b48544c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:10 +01:00
Paweł Gronowski
dd6992617e integration/build: Use new buildkit progressui
Introduced in: 37131781d7

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:09 +01:00
Paweł Gronowski
31545c3b67 vendor: github.com/moby/buildkit v0.13.0-rc2
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:07 +01:00
CrazyMax
f90b03ee5d go.mod: bump to go 1.21 and use local toolchain when vendoring
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:25:20 +01:00
Paweł Gronowski
16aa7dd67f c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:09:16 +01:00
Sebastiaan van Stijn
8cdb5a9070 Merge pull request #47450 from neersighted/image_created_omitempty
api: omit missing Created field from ImageInspect response
2024-02-26 20:06:52 +01:00
Sebastiaan van Stijn
ffd294ebcc Merge pull request #45967 from tianon/c8d-image-list
c8d: Adjust "image list" to return only a single item for each image store entry
2024-02-26 20:05:29 +01:00
Bjorn Neergaard
881260148f api: omit missing Created field from ImageInspect response
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-26 10:26:15 -07:00
Paweł Gronowski
432390320e api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 11:37:30 +01:00
Sebastiaan van Stijn
c70d7905fb Merge pull request #47432 from vvoland/c8d-pull-fslayer
c8d/pull: Progress fixes
2024-02-26 10:38:00 +01:00
Sebastiaan van Stijn
0eecd59153 Merge pull request #47245 from thaJeztah/bump_otel
vendor: OTEL v0.46.1 / v1.21.0
2024-02-23 17:47:27 +01:00
Sebastiaan van Stijn
6aea26b431 client: fix connection-errors being shadowed by API version mismatch errors
Commit e6907243af applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

    docker -H unix:///no/such/socket.sock secret ls
    "secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

    docker -H unix:///no/such/socket.sock secret ls
    Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 15:17:10 +01:00
Sebastiaan van Stijn
913478b428 client: doRequest: make sure we return a connection-error
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 15:13:22 +01:00
Sebastiaan van Stijn
901b90593d client: NegotiateAPIVersion: do not ignore (connection) errors from Ping
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 14:31:55 +01:00
Sebastiaan van Stijn
349abc64ed client: fix TestPingWithError
This test was added in 27ef09a46f, which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.

The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.

Running the test also shows that returning a response is unexpected, and
ignored:

    === RUN   TestPingWithError
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    --- PASS: TestPingWithError (0.00s)
    PASS

This patch updates the test to remove the response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 14:25:52 +01:00
Sebastiaan van Stijn
24fe934a7b Merge pull request #47423 from vvoland/ci-check-changelog
ci: Require changelog description
2024-02-23 12:24:13 +01:00
Paweł Gronowski
05b883bdc8 mounts/validate: Don't check source exists with CreateMountpoint
Don't error out when mount source doesn't exist and mounts has
`CreateMountpoint` option enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-23 11:20:55 +01:00
Sebastiaan van Stijn
c516804d6f vendor: OTEL v0.46.1 / v1.21.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 10:11:07 +01:00
Sebastiaan van Stijn
445d500aeb Merge pull request #47410 from vvoland/test-daemonproxy-no-otel
integration/TestDaemonProxy: Remove OTEL span
2024-02-22 22:19:53 +01:00
Sebastiaan van Stijn
1ffaf469ba Merge pull request #47175 from corhere/best-effort-xattrs-classic-builder
builder/dockerfile: ADD with best-effort xattrs
2024-02-22 20:14:22 +01:00
Albin Kerouanton
842d1b3c12 Merge pull request #47433 from akerouanton/libnet-ds-extra-space-in-err
libnet/ds: remove extra space in error msg
2024-02-22 19:38:26 +01:00
Albin Kerouanton
83c02f7a11 libnet/ds: remove extra space in error msg
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-22 18:49:28 +01:00
Paweł Gronowski
14df52b709 c8d/pull: Don't emit Downloading with 0 progress
To align with the graphdrivers behavior and don't send unnecessary
progress messages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:16 +01:00
Paweł Gronowski
ff5f780f2b c8d/pull: Emit Pulling fs layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:15 +01:00
Paweł Gronowski
5689dabfb3 pkg/streamformatter: Make progressOutput concurrency safe
Sync access to the underlying `io.Writer` with a mutex.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:13 +01:00
Sebastiaan van Stijn
7d081179e9 Merge pull request #47422 from akerouanton/libnet-ds-DeleteIdempotent
libnet: Replace DeleteAtomic in retry loops with Delete
2024-02-22 17:24:05 +01:00
Paweł Gronowski
3865c63d45 Merge pull request #47426 from vvoland/vendor-continuity
vendor: github.com/containerd/continuity v0.4.3
2024-02-22 14:28:41 +01:00
Paweł Gronowski
1d473549e8 ci: Require changelog description
Any PR that is labeled with any `impact/*` label should have a
description for the changelog and an `area/*` label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 13:40:23 +01:00
Paweł Gronowski
b2aaf5c2b0 vendor: github.com/containerd/continuity v0.4.3
full diff: https://github.com/containerd/continuity/compare/v0.4.3...v0.4.2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 10:24:52 +01:00
Albin Kerouanton
cbd45e83cf libnet: Replace DeleteAtomic in retry loops with DeleteIdempotent
A common pattern in libnetwork is to delete an object using
`DeleteAtomic`, ie. to check the optimistic lock, but put in a retry
loop to refresh the data and the version index used by the optimistic
lock.

This commit introduces a new `Delete` method to delete without
checking the optimistic lock. It focuses only on the few places where
it's obvious the calling code doesn't rely on the side-effects of the
retry loop (ie. refreshing the object to be deleted).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-22 08:22:09 +01:00
Sebastiaan van Stijn
cba87125b2 Merge pull request #47405 from vvoland/validate-vendor-nopager
validate/vendor: Disable pager for git diff
2024-02-21 17:16:11 +01:00
CrazyMax
2a41ce93fe Merge pull request #47409 from crazy-max/ci-codecov-token
ci: set codecov token
2024-02-21 16:32:15 +01:00
Sebastiaan van Stijn
c42ae61e62 Merge pull request #47417 from thaJeztah/resolver_improve_logs
libnetwork: resolve: use structured logs for DNS error
2024-02-21 10:41:06 +01:00
Sebastiaan van Stijn
d9e082ff54 libnetwork: resolve: use structured logs for DNS error
I noticed that this log didn't use structured logs;

    [resolver] failed to query DNS server: 10.115.11.146:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:46361->10.115.11.146:53: i/o timeout
    [resolver] failed to query DNS server: 10.44.139.225:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:53991->10.44.139.225:53: i/o timeout

But other logs did;

    DEBU[2024-02-20T15:48:51.026704088Z] [resolver] forwarding query                   client-addr="udp:172.19.0.2:39661" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t A"
    DEBU[2024-02-20T15:48:51.028331088Z] [resolver] forwarding query                   client-addr="udp:172.19.0.2:35163" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t AAAA"
    DEBU[2024-02-20T15:48:51.057329755Z] [resolver] received AAAA record "2a00:1450:400e:801::200e" for "google.com." from udp:192.168.65.7
    DEBU[2024-02-20T15:48:51.057666880Z] [resolver] received A record "142.251.36.14" for "google.com." from udp:192.168.65.7

As we're already constructing a logger with these fields, we may as well use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-20 17:01:06 +01:00
Paweł Gronowski
8761bffcaf Makefile: Pass PAGER/GIT_PAGER variable
Allow to override the PAGER/GIT_PAGER variables inside the container.
Use `cat` as pager when running in Github Actions (to avoid things like
`git diff` stalling the CI).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 11:52:58 +01:00
Paweł Gronowski
56aeb548b2 integration/TestDaemonProxy: Remove OTEL span
Don't use OTEL tracing in this test because we're testing the HTTP proxy
behavior here and we don't want OTEL to interfere.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 10:21:53 +01:00
CrazyMax
38827ba290 ci: set codecov token
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-20 08:58:27 +01:00
Sebastiaan van Stijn
9d1541526c Merge pull request #47361 from robmry/47331_swarm_ipam_validation
Don't enforce new validation rules for existing networks
2024-02-16 16:58:33 +01:00
Sebastiaan van Stijn
7bf8d2606e Merge pull request #47356 from robmry/47329_restore_internal_bridge_addr
Make 'internal' bridge networks accessible from host
2024-02-16 13:23:38 +01:00
Sebastiaan van Stijn
bf053be997 Merge pull request #47373 from vvoland/aws-v1.24.1
vendor: bump github.com/aws/aws-sdk-go-v2 to v1.24.1
2024-02-15 12:00:47 +01:00
Sebastiaan van Stijn
101241c804 Merge pull request #47382 from robmry/run_macvlan_ipvlan_tests
Run the macvlan/ipvlan integration tests
2024-02-14 23:29:38 +01:00
Paweł Gronowski
bddd892e91 c8d: Adjust "image list" to return only a single item for each image store entry
This will return a single entry for each name/value pair, and for now
all the "image specific" metadata (labels, config, size) should be
either "default platform" or "first platform we have locally" (which
then matches the logic for commands like `docker image inspect`, etc)
with everything else (just ID, maybe?) coming from the manifest
list/index.

That leaves room for the longer-term implementation to add new fields to
describe the _other_ images that are part of the manifest list/index.

Co-authored-by: Tianon Gravi <admwiggin@gmail.com>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 18:44:37 +01:00
Paweł Gronowski
2aa13e950d awslogs: Replace depreacted WithEndpointResolver usage
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 18:38:05 +01:00
Paweł Gronowski
70a4a9c969 vendor: bump github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to v1.32.0
v1.33.0 is also available, but it would also cause
`github.com/aws/aws-sdk-go-v2` change from v1.24.1 to v1.25.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 17:23:55 +01:00
Sebastiaan van Stijn
a0f12f96eb Merge pull request #47374 from tianon/api-inspect-created
Set `Created` to `0001-01-01T00:00:00Z` on older API versions
2024-02-14 17:09:07 +01:00
Akihiro Suda
3d354593c9 Merge pull request #47385 from thaJeztah/update_go_1.21.7
update to go1.21.7
2024-02-15 00:26:10 +09:00
Rob Murray
9faf4855d5 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:09:45 +00:00
Rob Murray
4eb95d01bc Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:08:10 +00:00
Sebastiaan van Stijn
7c2975d2df update to go1.21.7
go1.21.7 (released 2024-02-06) includes fixes to the compiler, the go command,
the runtime, and the crypto/x509 package. See the Go 1.21.7 milestone on our
issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.6...go1.21.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-14 12:56:06 +01:00
Paweł Gronowski
903412d0fc api/history: Mention empty Created
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 11:00:17 +01:00
Sebastiaan van Stijn
b8a71a1c44 Merge pull request #47375 from robmry/47370_windows_nat_network_dns
Set up DNS names for Windows default network
2024-02-13 13:47:28 +01:00
Rob Murray
443f56efb0 Set up DNS names for Windows default network
DNS names were only set up for user-defined networks. On Linux, none
of the built-in networks (bridge/host/none) have built-in DNS, so they
don't need DNS names.

But, on Windows, the default network is "nat" and it does need the DNS
names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-12 21:11:44 +00:00
Tianon Gravi
b4fbe226e8 Set Created to 0001-01-01T00:00:00Z on older API versions
This matches the prior behavior before 2a6ff3c24f.

This also updates the Swagger documentation for the current version to note that the field might be the empty string and what that means.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2024-02-12 12:39:16 -08:00
Cory Snider
5bcd2f6860 builder/dockerfile: ADD with best-effort xattrs
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.

The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-12 13:31:44 -05:00
Paweł Gronowski
999f90ac1c vendor: bump github.com/aws/aws-sdk-go-v2 to v1.24.1
In preparation for buildkit v0.13

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-12 18:22:32 +01:00
Akihiro Suda
a60546b084 Merge pull request #47371 from thaJeztah/bump_nydus
vendor: github.com/containerd/nydus-snapshotter v0.13.7
2024-02-12 18:53:05 +09:00
Sebastiaan van Stijn
ef7766304c vendor: github.com/containerd/nydus-snapshotter v0.13.7
Update to the latest patch release, which contains changes from v0.13.5 to
remove the reference package from "github.com/docker/distribution", which
is now a separate module.

full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.8.2...v0.13.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:25:28 +01:00
Sebastiaan van Stijn
6932939326 vendor: google.golang.org/genproto/googleapis/rpc 49dd2c1f3d0b
manually aligned the indirect dependencies to be on the same commit

diff: b8732ec382...49dd2c1f3d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:25:26 +01:00
Sebastiaan van Stijn
10a72f2504 vendor: cloud.google.com/go/logging v1.8.1
full diff: https://github.com/googleapis/google-cloud-go/compare/logging/v1.7.0...logging/v1.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:24:51 +01:00
Sebastiaan van Stijn
a60fef0c41 vendor: golang.org/x/exp v0.0.0-20231006140011-7918f672742d
full diff: c95f2b4c22...7918f67274

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:13:32 +01:00
Bjorn Neergaard
86b86412a1 Merge pull request #47362 from thaJeztah/migrate_image_spec
migrate image spec to github.com/moby/docker-image-spec
2024-02-09 14:22:42 -07:00
Sebastiaan van Stijn
03a17a2887 migrate image spec to github.com/moby/docker-image-spec
The specification was migrated to a separate module:
https://github.com/moby/docker-image-spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-09 19:12:18 +01:00
Rob Murray
a26c953b94 Add comment explaining network-create flow for Swarm
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-09 11:56:46 +00:00
Rob Murray
571af915d5 Don't enforce new validation rules for existing networks
Non-swarm networks created before network-creation-time validation
was added in 25.0.0 continued working, because the checks are not
re-run.

But, swarm creates networks when needed (with 'agent=true'), to
ensure they exist on each agent - ignoring the NetworkNameError
that says the network already existed.

By ignoring validation errors on creation of a network with
agent=true, pre-existing swarm networks with IPAM config that would
fail the new checks will continue to work too.

New swarm (overlay) networks are still validated, because they are
initially created with 'agent=false'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-09 11:56:46 +00:00
Sebastiaan van Stijn
97478c99f8 Merge pull request #47360 from thaJeztah/image_spec_clean
image/spec: remove link to docs.docker.com "registry" specification
2024-02-08 18:50:18 +01:00
Sebastiaan van Stijn
b71c2792d2 image/spec: remove link to docs.docker.com "registry" specification
This spec is not directly relevant for the image spec, and the Docker
documentation no longer includes the actual specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-08 17:39:18 +01:00
Sebastiaan van Stijn
57e8352c9e Merge pull request #47359 from vvoland/c8d-1.7.13
vendor: github.com/containerd/containerd v1.7.13
2024-02-08 16:25:16 +01:00
Paweł Gronowski
4ab11a1148 vendor: github.com/containerd/containerd v1.7.13
No major changes, it just adds `content.InfoReaderProvider` interface.

full diff: https://github.com/containerd/containerd/compare/v1.7.12...v1.7.13

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-08 15:04:04 +01:00
Sebastiaan van Stijn
23d80f729e Merge pull request #46981 from thaJeztah/bump_prometheus
vendor: github.com/prometheus/client_golang v1.17.0
2024-02-07 23:06:06 +01:00
Rob Murray
419f5a6372 Make 'internal' bridge networks accessible from host
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).

This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-07 19:12:10 +00:00
Sebastiaan van Stijn
475019d70a vendor: github.com/prometheus/procfs v0.12.0
- https://github.com/prometheus/procfs/compare/v0.11.1...v0.12.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-07 02:40:09 +01:00
Sebastiaan van Stijn
63c354aae2 vendor: github.com/prometheus/client_golang v1.17.0
full diffs:

- https://github.com/prometheus/client_golang/compare/v1.14.0...v1.17.0
- https://github.com/prometheus/client_model/compare/v0.3.0...v0.5.0
- https://github.com/prometheus/common/compare/v0.42.0...v0.44.0
- https://github.com/prometheus/procfs/compare/v0.9.0...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-07 02:40:07 +01:00
Sebastiaan van Stijn
9e075f3808 Merge pull request #47155 from thaJeztah/remove_deprecated_api_versions
api: remove deprecated API versions (API < v1.24)
2024-02-07 01:43:04 +01:00
Rob Murray
beb97f7fdf Refactor 'resolv.conf' generation.
Replace regex matching/replacement and re-reading of generated files
with a simple parser, and struct to remember and manipulate the file
content.

Annotate the generated file with a header comment saying the file is
generated, but can be modified, and a trailing comment describing how
the file was generated and listing external nameservers.

Always start with the host's resolv.conf file, whether generating config
for host networking, or with/without an internal resolver - rather than
editing a file previously generated for a different use-case.

Resolves an issue where rewrites of the generated file resulted in
default IPv6 nameservers being unnecessarily added to the config.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-06 22:26:12 +00:00
Sebastiaan van Stijn
d2f12e6d51 Merge pull request #47336 from rumpl/history-config
c8d: Use the same logic to get the present images
2024-02-06 19:42:51 +01:00
Sebastiaan van Stijn
14503ccebd api/server/middleware: NewVersionMiddleware: add validation
Make sure the middleware cannot be initialized with out of range versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:45 +01:00
Sebastiaan van Stijn
e1897cbde4 api/server/middleware:use API-consts in tests
Use the API consts to have more realistic values in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:45 +01:00
Sebastiaan van Stijn
0fef6e1c99 api/server/middleware: VersionMiddleware: improve docs
Improve documentation and rename fields and variables to be more descriptive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:45 +01:00
Sebastiaan van Stijn
6b01719ffb api: add MinSupportedAPIVersion const
This const contains the minimum API version that can be supported by the
API server. The daemon is currently configured to use the same version,
but we may increment the _configured_ minimum version when deprecating
old API versions in future.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
19a04efa2f api: remove API < v1.24
Commit 08e4e88482 (Docker Engine v25.0.0)
deprecated API version v1.23 and lower, but older API versions could be
enabled through the DOCKER_MIN_API_VERSION environment variable.

This patch removes all support for API versions < v1.24.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
8758d08bb4 api: remove handling of HostConfig on POST /containers/{id}/start (api < v1.24)
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
ffd877f948 api: remove plain-text error-responses (api < v1.24)
Commit 322e2a7d05 changed the format of errors
returned by the API to be in JSON format for API v1.24. Older versions of
the API returned errors in plain-text format.

API v1.23 and older are deprecated, so we can remove support for plain-text
error responses.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
b3a0ff9944 api: remove POST /containers/{id}/copy endpoint (api < v1.23)
This endpoint was deprecated in API v1.20 (Docker Engine v1.8.0) in
commit db9cc91a9e, in favor of the
`PUT /containers/{id}/archive` and `HEAD /containers/{id}/archive`
endpoints, and disabled in API v1.24 (Docker Engine v1.12.0) through
commit 428328908d.

This patch removes the endpoint, and the associated `daemon.ContainerCopy`
method in the backend.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
83f790cccc api: POST /exec/{id}/start: remove support for API < v1.21
API v1.21 (Docker Engine v1.9.0) enforces the request to have a JSON
content-type on exec start (see 45dc57f229).
An exception was added in 0b5e628e14 to
make this check conditional (supporting API < 1.21).

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
d1974aa492 api: remove code for container stats on api < v1.21
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.20 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
ed93110e11 api: update test to reflect reality on Windows
The TestInspectAPIContainerResponse mentioned that Windows does not
support API versions before v1.25.

While technically, no stable release existed for Windows with API versions
before that (see f811d5b128), API version
v1.24 was enabled in e4af39aeb3, to have
a consistend fallback version for API version negotiation.

This patch updates the test to reflect that change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
570d5a9645 api: remove code for ContainerInspect on api v1.20
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.20.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
f0dd554e3c api: remove code for ContainerInspect on api < v1.20
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.19 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
dfdf2adf0c api: POST /containers/{id}/kill: remove handling for api < 1.20
API v1.20 and up produces an error when signalling / killing a non-running
container (see c92377e300). Older API versions
allowed this, and an exception was added in 621e3d8587.

API v1.23 and older are deprecated, so we can remove this handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:42 +01:00
Sebastiaan van Stijn
2970b320aa api: remove code for adjusting CPU shares (api < v1.19)
API versions before 1.19 allowed CpuShares that were greater than the maximum
or less than the minimum supported by the kernel, and relied on the kernel to
do the right thing.

Commit ed39fbeb2a introduced code to adjust the
CPU shares to be within the accepted range when using API version 1.18 or
lower.

API v1.23 and older are deprecated, so we can remove support for this
functionality.

Currently, there's no validation for CPU shares to be within an acceptable
range; a TODO was added to add validation for this option, and to use the
`linuxMinCPUShares` and `linuxMaxCPUShares` consts for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:33 +01:00
Sebastiaan van Stijn
ef25f0aa52 api: POST /build: remove version-gate for "pull" (api < v1.16)
The "pull" option was added in API v1.16 (Docker Engine v1.4.0) in commit
054e57a622, which gated the option by API
version.

API v1.23 and older are deprecated, so we can remove the gate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Sebastiaan van Stijn
7fa116830b api: POST /build: remove version-gate for "rm", "force-rm" (api < v1.16)
The "rm" option was made the default in API v1.12 (Docker Engine v1.0.0)
in commit b60d647172, and "force-rm" was
added in 667e2bd4ea.

API v1.23 and older are deprecated, so we can remove these gates.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Sebastiaan van Stijn
1b1147e46b api: POST /commit: remove version-gate for "pause" (api < v1.16)
The "pause" flag was added in API v1.13 (Docker Engine v1.1.0), and is
enabled by default (see 17d870bed5).

API v1.23 and older are deprecated, so we can remove the version-gate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Sebastiaan van Stijn
d26bdfe226 runconfig: remove fixtures for api < v1.19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:24 +01:00
Djordje Lukic
f1e6958295 c8d: Use the same logic to get the present images
Inspect and history used two different ways to find the present images.
This made history fail in some cases where image inspect would work (if
a configuration of a manifest wasn't found in the content store).

With this change we now use the same logic for both inspect and history.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-02-06 16:35:53 +01:00
Sebastiaan van Stijn
27ac2beca0 Merge pull request #47342 from vvoland/cache-ocispec-platforms
image/cache: Use Platform from ocispec
2024-02-06 15:45:49 +01:00
Sebastiaan van Stijn
9e10605e77 Merge pull request #47341 from thaJeztah/seccomp_updates
profiles/seccomp: add syscalls for kernel v5.17 - v6.6, match containerd's profile
2024-02-06 15:22:16 +01:00
Paweł Gronowski
2c01d53d96 image/cache: Use Platform from ocispec
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 14:26:51 +01:00
Sebastiaan van Stijn
d69729e053 seccomp: add futex_wake syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 9f6c532f59

    futex: Add sys_futex_wake()

    To complement sys_futex_waitv() add sys_futex_wake(). This syscall
    implements what was previously known as FUTEX_WAKE_BITSET except it
    uses 'unsigned long' for the bitmask and takes FUTEX2 flags.

    The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:12:40 +01:00
Sebastiaan van Stijn
10d344d176 seccomp: add futex_wait syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cb8c4312af

    futex: Add sys_futex_wait()

    To complement sys_futex_waitv()/wake(), add sys_futex_wait(). This
    syscall implements what was previously known as FUTEX_WAIT_BITSET
    except it uses 'unsigned long' for the value and bitmask arguments,
    takes timespec and clockid_t arguments for the absolute timeout and
    uses FUTEX2 flags.

    The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:12:40 +01:00
Sebastiaan van Stijn
df57a080b6 seccomp: add futex_requeue syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 0f4b5f9722

    futex: Add sys_futex_requeue()

    Finish off the 'simple' futex2 syscall group by adding
    sys_futex_requeue(). Unlike sys_futex_{wait,wake}() its arguments are
    too numerous to fit into a regular syscall. As such, use struct
    futex_waitv to pass the 'source' and 'destination' futexes to the
    syscall.

    This syscall implements what was previously known as FUTEX_CMP_REQUEUE
    and uses {val, uaddr, flags} for source and {uaddr, flags} for
    destination.

    This design explicitly allows requeueing between different types of
    futex by having a different flags word per uaddr.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:12:31 +01:00
Sebastiaan van Stijn
8826f402f9 seccomp: add map_shadow_stack syscall (kernel v6.6, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: c35559f94e

    x86/shstk: Introduce map_shadow_stack syscall

    When operating with shadow stacks enabled, the kernel will automatically
    allocate shadow stacks for new threads, however in some cases userspace
    will need additional shadow stacks. The main example of this is the
    ucontext family of functions, which require userspace allocating and
    pivoting to userspace managed stacks.

    Unlike most other user memory permissions, shadow stacks need to be
    provisioned with special data in order to be useful. They need to be setup
    with a restore token so that userspace can pivot to them via the RSTORSSP
    instruction. But, the security design of shadow stacks is that they
    should not be written to except in limited circumstances. This presents a
    problem for userspace, as to how userspace can provision this special
    data, without allowing for the shadow stack to be generally writable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:02:33 +01:00
Sebastiaan van Stijn
6f242f1a28 seccomp: add fchmodat2 syscall (kernel v6.6, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 09da082b07

    fs: Add fchmodat2()

    On the userspace side fchmodat(3) is implemented as a wrapper
    function which implements the POSIX-specified interface. This
    interface differs from the underlying kernel system call, which does not
    have a flags argument. Most implementations require procfs [1][2].

    There doesn't appear to be a good userspace workaround for this issue
    but the implementation in the kernel is pretty straight-forward.

    The new fchmodat2() syscall allows to pass the AT_SYMLINK_NOFOLLOW flag,
    unlike existing fchmodat.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 13:59:04 +01:00
Sebastiaan van Stijn
4d0d5ee10d seccomp: add cachestat syscall (kernel v6.5, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cf264e1329

    NAME
        cachestat - query the page cache statistics of a file.

    SYNOPSIS
        #include <sys/mman.h>

        struct cachestat_range {
            __u64 off;
            __u64 len;
        };

        struct cachestat {
            __u64 nr_cache;
            __u64 nr_dirty;
            __u64 nr_writeback;
            __u64 nr_evicted;
            __u64 nr_recently_evicted;
        };

        int cachestat(unsigned int fd, struct cachestat_range *cstat_range,
            struct cachestat *cstat, unsigned int flags);

    DESCRIPTION
        cachestat() queries the number of cached pages, number of dirty
        pages, number of pages marked for writeback, number of evicted
        pages, number of recently evicted pages, in the bytes range given by
        `off` and `len`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 13:57:00 +01:00
Sebastiaan van Stijn
1251982cf7 seccomp: add set_mempolicy_home_node syscall (kernel v5.17, libseccomp v2.5.4)
This syscall is gated by CAP_SYS_NICE, matching the profile in containerd.

containerd: a6e52c74fa
libseccomp: d83cb7ac25
kernel: c6018b4b25

    mm/mempolicy: add set_mempolicy_home_node syscall
    This syscall can be used to set a home node for the MPOL_BIND and
    MPOL_PREFERRED_MANY memory policy.  Users should use this syscall after
    setting up a memory policy for the specified range as shown below.

      mbind(p, nr_pages * page_size, MPOL_BIND, new_nodes->maskp,
            new_nodes->size + 1, 0);
      sys_set_mempolicy_home_node((unsigned long)p, nr_pages * page_size,
                    home_node, 0);

    The syscall allows specifying a home node/preferred node from which
    kernel will fulfill memory allocation requests first.
    ...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 13:53:15 +01:00
Sebastiaan van Stijn
203ffb1c09 Merge pull request #47330 from vvoland/cache-fix-older-windows
image/cache: Ignore Build and Revision on Windows
2024-02-06 13:02:00 +01:00
Sebastiaan van Stijn
cae5d323e1 Merge pull request #47332 from AkihiroSuda/rootlesskit-2.0.1
Update Rootlesskit to v2.0.1
2024-02-06 10:38:19 +01:00
Akihiro Suda
7f1b700227 Dockerfile: update RootlessKit to v2.0.1
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.1

Fix issue 47327 (`rootless lxc-user-nic: /etc/resolv.conf missing ip`)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-06 11:51:47 +09:00
Akihiro Suda
f1730a6512 go.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.1
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-06 11:51:00 +09:00
Akihiro Suda
f7192bb0b4 vendor.mod: github.com/google/uuid v1.6.0
https://github.com/google/uuid/releases/tag/v1.6.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-06 11:50:00 +09:00
Sebastiaan van Stijn
2156635843 Merge pull request #47232 from vvoland/fix-save-manifests
image/save: Fix untagged images not present in index.json
2024-02-05 19:06:54 +01:00
Paweł Gronowski
91ea04089b image/cache: Ignore Build and Revision on Windows
The compatibility depends on whether `hyperv` or `process` container
isolation is used.
This fixes cache not being used when building images based on older
Windows versions on a newer Windows host.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 17:41:21 +01:00
Paweł Gronowski
2ef0b53e51 integration/save: Add tests checking OCI archive output
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 11:17:58 +01:00
Sebastiaan van Stijn
6b83319773 Merge pull request #47299 from laurazard/plugin-install-digest
plugins: Fix panic when fetching by digest
2024-02-05 09:39:05 +01:00
Sebastiaan van Stijn
ee3c710f22 Merge pull request #47319 from coolljt0725/fix_broken_links
Fix broken links in project
2024-02-05 09:31:17 +01:00
Laura Brehm
74d51e8553 plugins: fix panic installing from repo w/ digest
Only print the tag when the received reference has a tag, if
we can't cast the received tag to a `reference.Tagged` then
skip printing the tag as it's likely a digest.

Fixes panic when trying to install a plugin from a reference
with a digest such as
`vieux/sshfs@sha256:1d3c3e42c12138da5ef7873b97f7f32cf99fb6edde75fa4f0bcf9ed277855811`

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-02-04 20:23:06 +00:00
Lei Jitang
e910a79e2b Remove 'VERSION' from the text
We removed `VERSION` file in this PR:
https://github.com/moby/moby/pull/35368

Signed-off-by: Lei Jitang <leijitang@outlook.com>
2024-02-04 11:59:03 +08:00
Lei Jitang
9fcea5b933 Fix broken links in project/README.md
Related to https://github.com/moby/moby/pull/37104,
`RELEASE-CHECKLIST.md` has been removed by this PR

Signed-off-by: Lei Jitang <leijitang@outlook.com>
2024-02-04 11:57:53 +08:00
Sebastiaan van Stijn
e61c425cc2 Merge pull request #47315 from thaJeztah/update_dev_cli_compose
Dockerfile: update docker-cli to v25.0.2, docker compose v2.24.5
2024-02-03 15:23:01 +01:00
Sebastiaan van Stijn
10d6f5213a Dockerfile: update docker compose to v2.24.5
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.24.3...v2.24.5
- release notes: https://github.com/docker/compose/releases/tag/v2.24.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-03 13:51:49 +01:00
Sebastiaan van Stijn
9c92c07acf Dockerfile: update dev-shell version of the cli to v25.0.2
Update the docker CLI that's available for debugging in the dev-shell
to the v25 release.

full diff: https://github.com/docker/cli/compare/v25.0.1...v25.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-03 13:50:18 +01:00
Sebastiaan van Stijn
0616b4190e Merge pull request #47309 from akerouanton/libnet-bridge-mtu-ignore-einval
libnet: bridge: ignore EINVAL when configuring bridge MTU
2024-02-03 11:36:19 +01:00
Albin Kerouanton
89470a7114 libnet: bridge: ignore EINVAL when configuring bridge MTU
Since 964ab7158c, we explicitly set the bridge MTU if it was specified.
Unfortunately, kernel <v4.17 have a check preventing us to manually set
the MTU to anything greater than 1500 if no links is attached to the
bridge, which is how we do things -- create the bridge, set its MTU and
later on, attach veths to it.

Relevant kernel commit: 804b854d37

As we still have to support CentOS/RHEL 7 (and their old v3.10 kernels)
for a few more months, we need to ignore EINVAL if the MTU is > 1500
(but <= 65535).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 19:32:45 +01:00
Sebastiaan van Stijn
701dd989f1 Merge pull request #47302 from akerouanton/libnet-ds-PersistConnection
libnet: boltdb: remove PersistConnection
2024-02-02 19:05:09 +01:00
Brian Goff
7c4828f8fb Merge pull request #47256 from corhere/journald/quit-when-youre-ahead
daemon/logger/journald: quit waiting when the logger closes
2024-02-02 09:10:42 -08:00
Sebastiaan van Stijn
7964cae9e8 Merge pull request #47306 from akerouanton/revert-automatically-enable-ipv6
Revert "daemon: automatically set network EnableIPv6 if needed"
2024-02-02 16:29:49 +01:00
Albin Kerouanton
e37172c613 api/t/network: ValidateIPAM: ignore v6 subnet when IPv6 is disabled
Commit 4f47013feb introduced a new validation step to make sure no
IPv6 subnet is configured on a network which has EnableIPv6=false.

Commit 5d5eeac310 then removed that validation step and automatically
enabled IPv6 for networks with a v6 subnet. But this specific commit
was reverted in c59e93a67b and now the error introduced by 4f47013feb
is re-introduced.

But it turns out some users expect a network created with an IPv6
subnet and EnableIPv6=false to actually have no IPv6 connectivity.
This restores that behavior.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 10:34:39 +01:00
Albin Kerouanton
c59e93a67b Revert "daemon: automatically set network EnableIPv6 if needed"
This reverts commit 5d5eeac310.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 10:34:26 +01:00
Albin Kerouanton
83af50aee3 libnet: boltdb: inline getDBhandle()
Previous commit made getDBhandle a one-liner returning a struct
member -- making it useless. Inline it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:19:07 +01:00
Albin Kerouanton
4d7c11c208 libnet: boltdb: remove PersistConnection
This parameter was used to tell the boltdb kvstore not to open/close
the underlying boltdb db file before/after each get/put operation.

Since d21d0884ae, we've a single datastore instance shared by all
components that need it. That commit set `PersistConnection=true`.
We can now safely remove this param altogether, and remove all the
code that was opening and closing the db file before and after each
operation -- it's dead code!

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:19:07 +01:00
Albin Kerouanton
8070a9aa66 libnet: drop TestMultipleControllersWithSameStore
This test is non-representative of what we now do in libnetwork.
Since the ability of opening the same boltdb database multiple
times in parallel will be dropped in the next commit, just remove
this test.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:19:07 +01:00
Albin Kerouanton
ca683c1c77 Merge pull request #47233 from robmry/47146-duplicate_mac_addrs2
Only restore a configured MAC addr on restart.
2024-02-02 09:08:17 +01:00
Albin Kerouanton
025967efd0 Merge pull request #47293 from robmry/47229-internal-bridge-firewalld
Add internal n/w bridge to firewalld docker zone
2024-02-02 08:36:27 +01:00
Albin Kerouanton
add2c4c79b Merge pull request #47285 from corhere/libn/one-datastore-to-rule-them-all
libnetwork: share a single datastore with drivers
2024-02-02 08:03:01 +01:00
Sebastiaan van Stijn
8604cc400d Merge pull request #47242 from robmry/remove_etchosts_build_unused_params
Remove unused params from etchosts.Build()
2024-02-02 01:09:10 +01:00
Brian Goff
e240ba44b7 Merge pull request #47300 from corhere/libc8d/fix-startup-data-race
libcontainerd/supervisor: fix data race
2024-02-01 15:02:00 -08:00
Laura Brehm
82dda18898 tests: add plugin install test w/ digest
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:

```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-02-01 23:00:38 +00:00
Cory Snider
dd20bf4862 libcontainerd/supervisor: fix data race
The monitorDaemon() goroutine calls startContainerd() then blocks on
<-daemonWaitCh to wait for it to exit. The startContainerd() function
would (re)initialize the daemonWaitCh so a restarted containerd could be
waited on. This implementation was race-free because startContainerd()
would synchronously initialize the daemonWaitCh before returning. When
the call to start the managed containerd process was moved into the
waiter goroutine, the code to initialize the daemonWaitCh struct field
was also moved into the goroutine. This introduced a race condition.

Move the daemonWaitCh initialization to guarantee that it happens before
the startContainerd() call returns.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-01 15:53:18 -05:00
Sebastiaan van Stijn
f5cf22ca99 Merge pull request #47259 from vvoland/api-build-version
api: Document `version` in `/build`
2024-02-01 19:16:25 +01:00
Paweł Gronowski
0c3b8ccda7 api: Document version in /build
It was introduced in API v1.38 but wasn't documented.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-01 17:00:07 +01:00
Sebastiaan van Stijn
810ef4d0b6 Merge pull request #47244 from thaJeztah/bump_grpc
vendor: google.golang.org/grpc v1.59.0
2024-02-01 15:44:15 +01:00
Rob Murray
2cc627932a Add internal n/w bridge to firewalld docker zone
Containers attached to an 'internal' bridge network are unable to
communicate when the host is running firewalld.

Non-internal bridges are added to a trusted 'docker' firewalld zone, but
internal bridges were not.

DOCKER-ISOLATION iptables rules are still configured for an internal
network, they block traffic to/from addresses outside the network's subnet.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 11:49:53 +00:00
Sebastiaan van Stijn
968f05910a Merge pull request #47263 from crazy-max/bump-actions
ci: bump remaining gha to latest stable
2024-02-01 11:39:29 +01:00
Rob Murray
8c64b85fb9 No inspect 'Config.MacAddress' unless configured.
Do not set 'Config.MacAddress' in inspect output unless the MAC address
is configured.

Also, make sure it is filled in for a configured address on the default
network before the container is started (by translating the network name
from 'default' to 'config' so that the address lookup works).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:57:35 +00:00
Rob Murray
dae33031e0 Only restore a configured MAC addr on restart.
The API's EndpointConfig struct has a MacAddress field that's used for
both the configured address, and the current address (which may be generated).

A configured address must be restored when a container is restarted, but a
generated address must not.

The previous attempt to differentiate between the two, without adding a field
to the API's EndpointConfig that would show up in 'inspect' output, was a
field in the daemon's version of EndpointSettings, MACOperational. It did
not work, MACOperational was set to true when a configured address was
used. So, while it ensured addresses were regenerated, it failed to preserve
a configured address.

So, this change removes that code, and adds DesiredMacAddress to the wrapped
version of EndpointSettings, where it is persisted but does not appear in
'inspect' results. Its value is copied from MacAddress (the API field) when
a container is created.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:55:54 +00:00
CrazyMax
a2026ee442 ci: update to docker/bake-action@v4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 09:33:02 +01:00
CrazyMax
5a3c463a37 ci: update to codecov/codecov-action@v4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 09:33:02 +01:00
CrazyMax
9babc02283 ci: update to actions/download-artifact@v4 and actions/upload-artifact@v4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 09:33:02 +01:00
CrazyMax
a83557d747 ci: update to actions/cache@v3
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 08:29:54 +01:00
Cory Snider
2200c0137f libnetwork/datastore: don't parse file path
File paths can contain commas, particularly paths returned from
t.TempDir() in subtests which include commas in their names. There is
only one datastore provider and it only supports a single address, so
the only use of parsing the address is to break tests in mysterious
ways.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-31 21:26:28 -05:00
Cory Snider
d21d0884ae libnetwork: share a single datastore with drivers
The bbolt library wants exclusive access to the boltdb file and uses
file locking to assure that is the case. The controller and each network
driver that needs persistent storage instantiates its own unique
datastore instance, backed by the same boltdb file. The boltdb kvstore
implementation works around multiple access to the same boltdb file by
aggressively closing the boltdb file between each transaction. This is
very inefficient. Have the controller pass its datastore instance into
the drivers and enable the PersistConnection option to disable closing
the boltdb between transactions.

Set data-dir in unit tests which instantiate libnetwork controllers so
they don't hang trying to lock the default boltdb database file.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-31 21:08:34 -05:00
Sebastiaan van Stijn
3e230cfdcc Merge pull request from GHSA-xw73-rw38-6vjc
image/cache: Restrict cache candidates to locally built images
2024-02-01 01:12:23 +01:00
Sebastiaan van Stijn
f42b8ae8db Merge pull request #47278 from thaJeztah/bump_containerd_binary_1.7.13
update containerd binary to v1.7.13
2024-02-01 00:03:58 +01:00
Sebastiaan van Stijn
7a920fd275 Merge pull request #47268 from thaJeztah/bump_runc_binary_1.1.12
update runc binary to v1.1.12
2024-01-31 22:50:35 +01:00
Sebastiaan van Stijn
584e60e820 Merge pull request #47273 from vvoland/vendor-bk-0.12.5
vendor: github.com/moby/buildkit v0.12.5
2024-01-31 22:24:49 +01:00
Sebastiaan van Stijn
8ee908e47c Merge pull request #47272 from thaJeztah/bump_runc_1.1.12
vendor: github.com/opencontainers/runc v1.1.12
2024-01-31 22:17:42 +01:00
Sebastiaan van Stijn
835cdcac95 update containerd binary to v1.7.13
Update the containerd binary that's used in CI

- full diff: https://github.com/containerd/containerd/compare/v1.7.12...v1.7.13
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 22:01:50 +01:00
Paweł Gronowski
f4a93b6993 vendor: github.com/moby/buildkit v0.12.5
full diff: https://github.com/moby/buildkit/compare/v0.12.4...v0.12.5

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-31 21:24:50 +01:00
Sebastiaan van Stijn
b20dccba5e vendor: github.com/opencontainers/runc v1.1.12
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.12
- full diff: https://github.com/opencontainers/runc/compare/v1.1.11...v1.1.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 21:17:56 +01:00
Sebastiaan van Stijn
44bf407d4d update runc binary to v1.1.12
Update the runc binary that's used in CI and for the static packages, which
includes a fix for [CVE-2024-21626].

- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.12
- full diff: https://github.com/opencontainers/runc/compare/v1.1.11...v1.1.12

[CVE-2024-21626]: https://github.com/opencontainers/runc/security/advisories/GHSA-xr7r-f8xq-vfvv

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 21:05:31 +01:00
Sebastiaan van Stijn
8a81b9d35f Merge pull request #47264 from vvoland/ci-fix-makeps1-templatefail
hack/make.ps1: Fix go list pattern
2024-01-31 21:01:08 +01:00
Paweł Gronowski
ecb217cf69 hack/make.ps1: Fix go list pattern
The double quotes inside a single quoted string don't need to be
escaped.
Looks like different Powershell versions are treating this differently
and it started failing unexpectedly without any changes on our side.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-31 19:54:27 +01:00
Sebastiaan van Stijn
2df4755725 Merge pull request #47250 from thaJeztah/update_actions
gha: update actions to account for node 16 deprecation
2024-01-31 12:45:29 +01:00
Sebastiaan van Stijn
3a8191225a gha: update to crazy-max/ghaction-github-runtime@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff: https://github.com/crazy-max/ghaction-github-runtime/compare/v2.2.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
08251978a8 gha: update to docker/login-action@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff https://github.com/docker/login-action/compare/v2.2.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
5d396e0533 gha: update to docker/setup-qemu-action@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff https://github.com/docker/setup-qemu-action/compare/v2.2.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
4a1839ef1d gha: update to docker/bake-action@v4
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff https://github.com/docker/bake-action/compare/v2.3.0...v4.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
b7fd571b0a gha: update to docker/setup-buildx-action@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff: https://github.com/docker/setup-buildx-action/compare/v2.10.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:16 +01:00
Sebastiaan van Stijn
00a2626b56 gha: update to docker/metadata-action@v5
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff: https://github.com/docker/metadata-action/compare/v4.6.0...v5.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:11 +01:00
Sebastiaan van Stijn
e27a785f43 gha: update to actions/setup-go@v5
- full diff: https://github.com/actions/setup-go/compare/v3.5.0...v5.0.0

v5

In scope of this release, we change Nodejs runtime from node16 to node20.
Moreover, we update some dependencies to the latest versions.

Besides, this release contains such changes as:

- Fix hosted tool cache usage on windows
- Improve documentation regarding dependencies caching

V4

The V4 edition of the action offers:

- Enabled caching by default
- The action will try to enable caching unless the cache input is explicitly
  set to false.

Please see "Caching dependency files and build outputs" for more information:
https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:46:17 +01:00
Sebastiaan van Stijn
fb53ee6ba3 gha: update to actions/github-script@v7
- full diff: https://github.com/actions/github-script/compare/v6.4.1...v7.0.1

breaking changes: https://github.com/actions/github-script?tab=readme-ov-file#v7

> Version 7 of this action updated the runtime to Node 20
> https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions
>
> All scripts are now run with Node 20 instead of Node 16 and are affected
> by any breaking changes between Node 16 and 20
>
> The previews input now only applies to GraphQL API calls as REST API previews
> are no longer necessary
> https://github.blog/changelog/2021-10-14-rest-api-preview-promotions/.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:46:17 +01:00
Sebastiaan van Stijn
0ffddc6bb8 gha: update to actions/checkout@v4
Release notes:

- https://github.com/actions/checkout/compare/v3.6.0...v4.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:46:14 +01:00
Cory Snider
987fe37ed1 d/logger/journald: quit waiting when logger closes
If a reader has caught up to the logger and is waiting for the next
message, it should stop waiting when the logger is closed. Otherwise
the reader will unnecessarily wait the full closedDrainTimeout for no
log messages to arrive.

This case was overlooked when the journald reader was recently
overhauled to be compatible with systemd 255, and the reader tests only
failed when a logical race happened to settle in such a way to exercise
the bugged code path. It was only after implicit flushing on close was
added to the journald test harness that the Follow tests would
repeatably fail due to this bug. (No new regression tests are needed.)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 17:57:12 -05:00
Cory Snider
d53b7d7e46 d/logger/journald: sync logger on close in tests
The journald reader test harness injects an artificial asynchronous
delay into the logging pipeline: a logged message won't be written to
the journal until at least 150ms after the Log() call returns. If a test
returns while log messages are still in flight to be written, the logs
may attempt to be written after the TempDir has been cleaned up, leading
to spurious errors.

The logger read tests which interleave writing and reading have to
include explicit synchronization points to work reliably with this delay
in place. On the other hand, tests should not be required to sync the
logger explicitly before returning. Override the Close() method in the
test harness wrapper to wait for in-flight logs to be flushed to disk.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 17:18:46 -05:00
Cory Snider
39c5c16521 d/logger/loggertest: improve TestConcurrent
- Check the return value when logging messages
- Log the stream (stdout/stderr) and list of messages that were not read
- Wait until the logger is closed before returning early (panic/fatal)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 17:18:12 -05:00
Cory Snider
5792bf7ab3 d/logger/journald: log journal-remote cmd output
Writing the systemd-journal-remote command output directly to os.Stdout
and os.Stderr makes it nearly impossible to tell which test case the
output is related to when the tests are not run in verbose mode. Extend
the journald sender fake to redirect output to the test log so they
interleave with the rest of the test output.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 14:50:57 -05:00
Cory Snider
982e777d49 d/logger/journald: fix data race in test harness
The Go race detector was detecting a data race when running the
TestLogRead/Follow/Concurrent test against the journald logging driver.
The race was in the test harness, specifically syncLogger. The waitOn
field would be reassigned each time a log entry is sent to the journal,
which is not concurrency-safe. Make it concurrency-safe using the same
patterns that are used in the log follower implementation to synchronize
with the logger.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 14:34:15 -05:00
Sebastiaan van Stijn
f472dda2e9 Merge pull request #47236 from akerouanton/remove-sb-leave-options-param
libnet: remove arg `options` from (*Endpoint).Leave()
2024-01-30 16:57:36 +01:00
Sebastiaan van Stijn
ca40ac030c vendor: google.golang.org/grpc v1.59.0
full diff:

- https://github.com/grpc/grpc-go/compare/v1.58.3...v1.59.0
- 782d3b101e...b8732ec382
- https://github.com/googleapis/google-cloud-go/compare/v0.110.4...v0.110.7
- https://github.com/googleapis/google-cloud-go/compare/compute/v1.21.0...compute/v1.23.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-29 18:59:21 +01:00
Sebastiaan van Stijn
0818a476e5 vendor: github.com/go-logr/logr v1.3.0
full diff: https:// github.com/go-logr/logr/compare/v1.2.4...v1.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-29 18:51:54 +01:00
Sebastiaan van Stijn
a0b53f6fd2 vendor: golang.org/x/net v0.18.0
full diff: https://github.com/golang/net/compare/v0.17.0...v0.18.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-29 18:49:40 +01:00
Rob Murray
2ddec74d59 Remove unused params from etchosts.Build()
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-29 15:37:08 +00:00
Albin Kerouanton
794f7127ef Merge pull request #47062 from robmry/35954-default_ipv6_enabled
Detect IPv6 support in containers, generate '/etc/hosts' accordingly.
2024-01-29 16:31:35 +01:00
Paweł Gronowski
5e13f54f57 c8d/save: Handle digested reference same as ID
When saving an image treat `image@sha256:abcdef...` the same as
`abcdef...`, this makes it:

- Not export the digested tag as the image name
- Not try to export all tags from the image repository

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-29 16:29:05 +01:00
Paweł Gronowski
d131f00fff image/save: Fix untagged images not present in index.json
Saving an image via digested reference, ID or truncated ID doesn't store
the image reference in the archive. This also causes the save code to
not add the image's manifest to the index.json.
This commit explicitly adds the untagged manifests to the index.json if
no tagged manifests were added.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-29 16:26:37 +01:00
Albin Kerouanton
7d4ee14147 Merge pull request #47234 from corhere/libn/overlay-peerdb-unused-flags
libnetwork/d/overlay: drop unused `miss` flags from peerAdd
2024-01-29 13:02:08 +01:00
Brian Goff
0f507ef624 Merge pull request #47019 from corhere/fix-journald-logs-systemd-255
logger/journald: fix tailing logs with systemd 255
2024-01-28 12:22:16 -08:00
Albin Kerouanton
21136865ac libnet: remove arg options from (*Endpoint).Leave()
This arg is never set by any caller. Better remove it

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-27 09:26:36 +01:00
Cory Snider
a8e8a4cdad libn/d/overlay: drop miss flags from peerAddOp
as all callers unconditionally set them to false.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 15:43:57 -05:00
Cory Snider
6ee58c2d29 libnetwork/d/overlay: drop miss flags from peerAdd
as all callers unconditionally set them to false.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 15:38:13 -05:00
Cory Snider
905477c8ae logger/journald: drop errDrainDone sentinel
errDrainDone is a sentinel error which is never supposed to escape the
package. Consequently, it needs to be filtered out of returns all over
the place, adding boilerplate. Forgetting to filter out these errors
would be a logic bug which the compiler would not help us catch. Replace
it with boolean multi-valued returns as they can't be accidentally
ignored or propagated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 12:42:09 -05:00
Cory Snider
d70fe8803c logger/journald: wait no longer than the deadline
While it doesn't really matter if the reader waits for an extra
arbitrary period beyond an arbitrary hardcoded timeout, it's also
trivial and cheap to implement, and nice to have.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 12:42:04 -05:00
Cory Snider
e94ec8068d logger/journald: use deadline for drain timeout
The journald reader uses a timer to set an upper bound on how long to
wait for the final log message of a stopped container. However, the
timer channel is only received from in non-blocking select statements!
There isn't enough benefit of using a timer to offset the cost of having
to manage the timer resource. Setting a deadline and comparing the
current time is just as effective, without having to manage the
lifecycle of any runtime resources.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 11:47:02 -05:00
Cory Snider
71bfffdad1 l/journald: make tests compatible with systemd 255
Synthesize a boot ID for journal entries fed into
systemd-journal-remote, as required by systemd 255.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 11:47:02 -05:00
Cory Snider
931568032a daemon/logger/loggertest: expand log-follow tests
Following logs with a non-negative tail when the container log is empty
is broken on the journald driver when used with systemd 255. Add tests
which cover this edge case to our loggertest suite.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 11:47:02 -05:00
Sebastiaan van Stijn
ee6cbc540e Merge pull request #47188 from thaJeztah/cleanup_newRouterOptions
cmd/dockerd: newRouterOptions: pass cluster as argument, and slight cleanup
2024-01-26 16:49:19 +01:00
Sebastiaan van Stijn
6d34bb71a0 Merge pull request #47230 from thaJeztah/update_dev_cli_compose
Dockerfile: update docker-cli to v25.0.1, docker compose v2.24.3
2024-01-26 10:18:58 +01:00
Sebastiaan van Stijn
388ba9a69c Dockerfile: update docker compose to v2.24.3
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.24.2...v2.24.3
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 09:30:17 +01:00
Sebastiaan van Stijn
3eb1527fdb Dockerfile: update dev-shell version of the cli to v25.0.1
Update the docker CLI that's available for debugging in the dev-shell
to the v25 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 09:23:14 +01:00
Sebastiaan van Stijn
93861bb5e6 Merge pull request #47227 from dvdksn/docs-api-remove-dead-links
docs: remove dead links from api version history
2024-01-25 22:59:25 +01:00
David Karlsson
7f94acb6ab docs: remove dead links from api verison history
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-01-25 20:15:24 +01:00
Sebastiaan van Stijn
97a5614d23 Merge pull request #47224 from s4ke/bump-swarmkit-generic-resources#main
Fix HasResource inverted boolean error - vendor swarmkit v2.0.0-20240125134710-dcda100a8261
2024-01-25 18:49:52 +01:00
Martin Braun
5c2eda6f71 vendor swarmkit v2.0.0-20240125134710-dcda100a8261
Signed-off-by: Martin Braun <braun@neuroforge.de>
2024-01-25 16:26:04 +01:00
Paweł Gronowski
96d461d27e builder/windows: Don't set ArgsEscaped for RUN cache probe
Previously this was done indirectly - the `compare` function didn't
check the `ArgsEscaped`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:07 +01:00
Paweł Gronowski
877ebbe038 image/cache: Check image platform
Make sure the cache candidate platform matches the requested.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:05 +01:00
Paweł Gronowski
96ac22768a image/cache: Restrict cache candidates to locally built images
Restrict cache candidates only to images that were built locally.
This doesn't affect builds using `--cache-from`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:03 +01:00
Paweł Gronowski
c6156dc51b daemon/imageStore: Mark images built locally
Store additional image property which makes it possible to distinguish
if image was built locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:00 +01:00
Paweł Gronowski
537348763f image/cache: Compare all config fields
Add checks for some image config fields that were missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:03:58 +01:00
Sebastiaan van Stijn
c7b3cb101b Merge pull request #47213 from thaJeztah/more_gocompat
add more //go:build directives to prevent downgrading to go1.16 language
2024-01-25 14:37:29 +01:00
Sebastiaan van Stijn
69d2923e4e Merge pull request #46419 from vvoland/pkg-pools-close-noop
pkg/ioutils: Make subsequent Close calls a no-op
2024-01-25 14:24:43 +01:00
Sebastiaan van Stijn
86198815a2 Merge pull request #47209 from corhere/sliceutil-map
internal/sliceutil: add utilities to map values
2024-01-25 11:51:50 +01:00
Sebastiaan van Stijn
d864df5a1d Merge pull request #47208 from akerouanton/libnet-ds-remove-unused-key-params
libnet/ds: remove unused param `key` from `GetObject` and `List`
2024-01-25 11:46:20 +01:00
Sebastiaan van Stijn
bd4ff31775 add more //go:build directives to prevent downgrading to go1.16 language
This is a follow-up to 2cf230951f, adding
more directives to adjust for some new code added since:

Before this patch:

    make -C ./internal/gocompat/
    GO111MODULE=off go generate .
    GO111MODULE=on go mod tidy
    GO111MODULE=on go test -v

    # github.com/docker/docker/internal/sliceutil
    internal/sliceutil/sliceutil.go:3:12: type parameter requires go1.18 or later (-lang was set to go1.16; check go.mod)
    internal/sliceutil/sliceutil.go:3:14: predeclared comparable requires go1.18 or later (-lang was set to go1.16; check go.mod)
    internal/sliceutil/sliceutil.go:4:19: invalid map key type T (missing comparable constraint)

    # github.com/docker/docker/libnetwork
    libnetwork/endpoint.go:252:17: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

    # github.com/docker/docker/daemon
    daemon/container_operations.go:682:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    daemon/inspect.go:42:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

With this patch:

    make -C ./internal/gocompat/
    GO111MODULE=off go generate .
    GO111MODULE=on go mod tidy
    GO111MODULE=on go test -v
    === RUN   TestModuleCompatibllity
        main_test.go:321: all packages have the correct go version specified through //go:build
    --- PASS: TestModuleCompatibllity (0.00s)
    PASS
    ok  	gocompat	0.031s
    make: Leaving directory '/go/src/github.com/docker/docker/internal/gocompat'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-25 11:18:44 +01:00
Cory Snider
e245fb76de internal/sliceutil: add utilities to map values
Functional programming for the win! Add a utility function to map the
values of a slice, along with a curried variant, to tide us over until
equivalent functionality gets added to the standard library
(https://go.dev/issue/61898)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-24 17:56:29 -05:00
Albin Kerouanton
3147a013fb libnet/ds: remove unused param key from List
Since 43dccc6 the `key` param is never used and can be safely
removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-24 22:42:18 +01:00
Albin Kerouanton
f7ef0e9fc7 libnet/ds: remove unused param key from GetObject
Since 43dccc6 the `key` param is never used and can be safely
removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-24 22:42:18 +01:00
Sebastiaan van Stijn
e8346c53d9 Merge pull request #46786 from rumpl/c8d-userns-namespace
c8d: Use a specific containerd namespace when userns are remapped
2024-01-24 20:36:40 +01:00
Djordje Lukic
3a617e5463 c8d: Use a specific containerd namespace when userns are remapped
We need to isolate the images that we are remapping to a userns, we
can't mix them with "normal" images. In the graph driver case this means
we create a new root directory where we store the images and everything
else, in the containerd case we can use a new namespace.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-01-24 15:46:16 +01:00
Sebastiaan van Stijn
43ffb1ee9d Merge pull request #47148 from thaJeztah/api_remove_deprecated_types
api/types: remove deprecated type-aliases
2024-01-24 12:40:27 +01:00
Sebastiaan van Stijn
115c7673dc Merge pull request #47198 from thaJeztah/image_remove_IDFromDigest
image: remove deprecated IDFromDigest
2024-01-24 12:36:42 +01:00
Sebastiaan van Stijn
f7e2357745 image: remove deprecated IDFromDigest
This function was deprecated in 456ea1bb1d
(Docker v24.0), and is no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:54:59 +01:00
Sebastiaan van Stijn
314ea05f8d Merge pull request #47197 from corhere/libn/carry-typo-fixes
libnetwork: carry typo fixes from moby/libnetwork repo
2024-01-24 01:31:08 +01:00
Sebastiaan van Stijn
13f46948dd api/types: remove deprecated container-types
These types were deprecated in v25.0, and moved to api/types/container;

This patch removes the aliases for;

- api/types.ResizeOptions (deprecated in 95b92b1f97)
- api/types.ContainerAttachOptions (deprecated in 30f09b4a1a)
- api/types.ContainerCommitOptions (deprecated in 9498d897ab)
- api/types.ContainerRemoveOptions (deprecated in 0f77875220)
- api/types.ContainerStartOptions (deprecated in 7bce33eb0f)
- api/types.ContainerListOptions (deprecated in 9670d9364d)
- api/types.ContainerLogsOptions (deprecated in ebef4efb88)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
4b09bc2145 api/types: remove deprecated service-types
These types were deprecated in v25.0, and moved to api/types/swarm;

This patch removes the aliases for;

- api/types.ServiceUpdateResponse (deprecated in 5b3e6555a3)
- api/types.ServiceCreateResponse (deprecated in ec69501e94)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
49637d0206 api/types: remove deprecated image-types
These types were deprecated in 48cacbca24
(v25.0), and moved to api/types/image.

This patch removes the aliases for;

- api/types.ImageDeleteResponseItem
- api/types.ImageSummary
- api/types.ImageMetadata

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
eccb1a3eb8 api/types: remove deprecated checkpoint-types
These types were deprecated in b688af2226
(v25.0), and moved to api/types/checkpoint.

This patch removes the aliases for;

- api/types.CheckpointCreateOptions
- api/types.CheckpointListOptions
- api/types.CheckpointDeleteOptions
- api/types.Checkpoint

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
0b1921649f api/types: remove deprecated system info types and functions
These types were deprecated in c90229ed9a
(v25.0), and moved to api/types/system.

This patch removes the aliases for;

- api/types.Info
- api/types.Commit
- api/types.PluginsInfo
- api/types.NetworkAddressPool
- api/types.Runtime
- api/types.SecurityOpt
- api/types.KeyValue
- api/types.DecodeSecurityOptions

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:24 +01:00
Sebastiaan van Stijn
4544bedea3 Merge pull request #47139 from thaJeztah/api_move_image_options
api/types: move image options to api/types/image
2024-01-24 01:26:55 +01:00
Cory Snider
6f44138269 libnetwork: fix tiny grammar mistake on design.md
Co-authored-by: Farhim Ferdous <37705070+AluBhorta@users.noreply.github.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-23 18:26:45 -05:00
Cory Snider
9a41cc58d9 libnetwork: fix typo in iptables.go
Co-authored-by: Ikko Ashimine <eltociear@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-23 18:25:08 -05:00
Sebastiaan van Stijn
ac2a028dcc api/types: move image options to api/types/image
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 00:10:33 +01:00
Sebastiaan van Stijn
8906adc8d4 Merge pull request #47138 from thaJeztah/move_image_backend_opt
api/types/image: move GetImageOpts to api/types/backend
2024-01-23 23:41:38 +01:00
Sebastiaan van Stijn
0bb84f5cef Merge pull request #47195 from akerouanton/fix-multiple-rename-error
daemon: rename: don't reload endpoint from datastore
2024-01-23 23:41:07 +01:00
Albin Kerouanton
80c44b4b2e daemon: rename: don't reload endpoint from datastore
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.

This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.

As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).

In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 22:53:21 +01:00
Sebastiaan van Stijn
2da96d6c12 Merge pull request #47121 from voloder/master
Make sure that make doesn't rm -rf the system out of existence
2024-01-23 19:03:06 +01:00
Paweł Gronowski
0b64499a24 Merge pull request #47194 from vvoland/volume-cifs-resolve-optout-2
volume/local: Fix CIFS urls with spaces, add tests
2024-01-23 18:58:52 +01:00
Sebastiaan van Stijn
9763709c05 Merge pull request #47181 from akerouanton/fix-aliases-on-default-bridge
daemon: only add short cid to aliases for custom networks
2024-01-23 18:28:33 +01:00
Paweł Gronowski
250886741b volume/local: Fix cifs url containing spaces
Unescapes the URL to avoid passing an URL encoded address to the kernel.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:11 +01:00
Paweł Gronowski
f4beb130b0 volume/local: Add tests for parsing nfs/cifs mounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:09 +01:00
Paweł Gronowski
df43311f3d volume/local: Break early if addr was specified
I made a mistake in the last commit - after resolving the IP from the
passed `addr` for CIFS it would still resolve the `device` part.

Apply only one name resolution

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:17:53 +01:00
Albin Kerouanton
9f37672ca8 daemon: only add short cid to aliases for custom networks
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.

This commit reintroduces the old logic.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 17:07:40 +01:00
Sebastiaan van Stijn
c25773ecbf cmd/dockerd: newRouterOptions: pass cluster as argument, and slight cleanup
- pass the cluster as an argument instead of manually setting it after
  creating the router-options
- remove the "opts" variable, to prevent it accidentally being used (with
  the assumption that's the value returned)
- use a struct-literal for the returned options.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 16:54:54 +01:00
Sebastiaan van Stijn
f19f233ca5 Merge pull request #47187 from thaJeztah/fix_gateway_ip
fix "host-gateway-ip" label not set for builder workers
2024-01-23 16:52:35 +01:00
Sebastiaan van Stijn
ca67dbd12c Merge pull request #47185 from vvoland/volume-cifs-resolve-optout
volume/local: Make host resolution backwards compatible
2024-01-23 16:23:40 +01:00
Paweł Gronowski
cac52f7173 Merge pull request #47167 from vvoland/c8d-prefer-default-platform-snapshot
c8d/snapshot: Create any platform if not specified
2024-01-23 15:25:17 +01:00
Sebastiaan van Stijn
00c9785e2e fix "host-gateway-ip" label not set for builder workers
Commit 21e50b89c9 added a label on the buildkit
worker to advertise the host-gateway-ip. This option can be either set by the
user in the daemon config, or otherwise defaults to the gateway-ip.

If no value is set by the user, discovery of the gateway-ip happens when
initializing the network-controller (`NewDaemon`, `daemon.restore()`).

However d222bf097c changed how we handle the
daemon config. As a result, the `cli.Config` used when initializing the
builder only holds configuration information form the daemon config
(user-specified or defaults), but is not updated with information set
by `NewDaemon`.

This patch adds an accessor on the daemon to get the current daemon config.
An alternative could be to return the config by `NewDaemon` (which should
likely be a _copy_ of the config).

Before this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: <nil>

After this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.18.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 14:58:01 +01:00
Paweł Gronowski
0d51cf9db8 volume/local: Make host resolution backwards compatible
Commit 8ae94cafa5 added a DNS resolution
of the `device` part of the volume option.

The previous way to resolve the passed hostname was to use `addr`
option, which was handled by the same code path as the `nfs` mount type.

The issue is that `addr` is also an SMB module option handled by kernel
and passing a hostname as `addr` produces an invalid argument error.

To fix that, restore the old behavior to handle `addr` the same way as
before, and only perform the new DNS resolution of `device` if there is
no `addr` passed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 14:49:05 +01:00
Sebastiaan van Stijn
1786517338 Merge pull request #47179 from thaJeztah/update_compose
Dockerfile: update docker compose to v2.24.2
2024-01-23 11:25:58 +01:00
Sebastiaan van Stijn
22a504935f Merge pull request #45474 from thaJeztah/testing_cleanups
assorted test fixes and cleanups
2024-01-23 10:01:27 +01:00
Sebastiaan van Stijn
05d952b246 Dockerfile: update docker compose to v2.24.2
Update the version of compose used in CI to the latest version.

- full diff: docker/compose@v2.24.1...v2.24.2
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 09:51:41 +01:00
Sebastiaan van Stijn
d86d24de35 Merge pull request #47174 from corhere/richer-xattr-errors
pkg/system: return even richer xattr errors
2024-01-23 09:46:12 +01:00
Sebastiaan van Stijn
20bd690844 integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
34668a5945 pkg/archive: fixe some unclosed file-handles in tests
Also fixing a "defer in loop" warning, instead changing to use
sub-tests, and simplifying some code, using os.WriteFile() instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
1090aaaedd libnetwork: fix some unclosed file-handles in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
c482383458 fix some leaking mounts in tests
This should help with errors such as:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride3702360633/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

    === RUN   TestSysctlOverrideHost
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverrideHost226485533/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverrideHost (0.00s)

    === RUN   TestDockerSuite/TestRunWithVolumesIsRecursive
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestDockerSuiteTestRunWithVolumesIsRecursive1156692230/001/tmpfs: device or resource busy
        --- FAIL: TestDockerSuite/TestRunWithVolumesIsRecursive (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:23 +01:00
Cory Snider
43bf65c174 pkg/system: return even richer xattr errors
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-22 15:25:10 -05:00
Sebastiaan van Stijn
a3a42c459e api/types/image: move GetImageOpts to api/types/backend
The `GetImageOpts` struct is used for options to be passed to the backend,
and are not used in client code. This struct currently is intended for internal
use only.

This patch moves the `GetImageOpts` struct to the backend package to prevent
it being imported in the client, and to make it more clear that this is part
of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 20:45:21 +01:00
Sebastiaan van Stijn
c87e0ad209 Merge pull request #47168 from robmry/47146-duplicate_mac_addrs
Remove generated MAC addresses on restart.
2024-01-22 19:48:24 +01:00
Rob Murray
cd53b7380c Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-22 17:52:20 +00:00
Paweł Gronowski
fb19f1fc20 c8d/snapshot: Create any platform if not specified
With containerd snapshotters enabled `docker run` currently fails when
creating a container from an image that doesn't have the default host
platform without an explicit `--platform` selection:

```
$ docker run image:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

This is confusing and the graphdriver behavior is much better here,
because it runs whatever platform the image has, but prints a warning:

```
$ docker run image:amd64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
```

This commits changes the containerd snapshotter behavior to be the same
as the graphdriver. This doesn't affect container creation when platform
is specified explicitly.

```
$ docker run --rm --platform linux/arm64 asdf:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 16:15:07 +01:00
Sebastiaan van Stijn
3602ba0afd Merge pull request #47162 from vvoland/25-fix-swarm-startinterval
daemon/cluster/executer: Add missing `StartInterval`
2024-01-22 15:51:37 +01:00
Sebastiaan van Stijn
f0eef50273 Merge pull request #47159 from akerouanton/fix-bad-http-code
daemon: return an InvalidParameter error when ep settings are wrong
2024-01-22 15:06:06 +01:00
Sebastiaan van Stijn
b6a5c2968b Merge pull request #47160 from vvoland/save-fix-oci-diffids
image/save: Fix layers order in OCI manifest
2024-01-22 15:05:06 +01:00
Paweł Gronowski
6100190e5c daemon/cluster/executer: Add missing StartInterval
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 14:42:17 +01:00
Paweł Gronowski
17fd6562bf image/save: Fix layers order in OCI manifest
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:48:12 +01:00
Paweł Gronowski
4979605212 image/save: Change layers type to DiffID
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:47:57 +01:00
Albin Kerouanton
fcc651972e daemon: return an InvalidParameter error when ep settings are wrong
Since v25.0 (commit ff50388), we validate endpoint settings when
containers are created, instead of doing so when containers are started.
However, a container created prior to that release would still trigger
validation error at start-time. In such case, the API returns a 500
status code because the Go error isn't wrapped into an InvalidParameter
error. This is now fixed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-22 12:48:23 +01:00
Sebastiaan van Stijn
42a07883f7 Merge pull request #47154 from thaJeztah/integration_cli_api_versions
integration-cli: adjust inspect tests to use current API version
2024-01-22 11:07:45 +01:00
Sebastiaan van Stijn
5a3a101af2 Merge pull request #47151 from thaJeztah/fix_TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs
integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
2024-01-22 10:23:50 +01:00
Sebastiaan van Stijn
6803dd8c90 Merge pull request #47124 from thaJeztah/remove_deprecated_api_docs
docs: remove documentation for deprecated API versions (v1.23 and before)
2024-01-22 10:23:31 +01:00
Sebastiaan van Stijn
82acb922cb Merge pull request #47147 from thaJeztah/integration_minor_refactor
integration: improve some asserts, and add asserts for unhandled errs
2024-01-22 10:23:03 +01:00
Akihiro Suda
e5dbe10e65 Merge pull request #47123 from thaJeztah/cli_25
Dockerfile: update docker-cli to v25.0.0, docker compose v2.24.1
2024-01-22 11:05:43 +09:00
Akihiro Suda
8528ed3409 Merge pull request #47128 from thaJeztah/remove_pkg_loopback
remove deprecated pkg/loopback (utility package for devicemapper)
2024-01-22 11:05:15 +09:00
Akihiro Suda
570b8a794c Merge pull request #47129 from thaJeztah/pkg_system_deprecated
pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
2024-01-22 11:04:25 +09:00
Akihiro Suda
5ad5334c2e Merge pull request #47130 from thaJeztah/pkg_homedir_deprecated
pkg/homedir: remove deprecated Key() and GetShortcutString()
2024-01-22 11:03:54 +09:00
Akihiro Suda
417826376f Merge pull request #47131 from thaJeztah/pkg_containerfs_deprecated
pkg/containerfs: remove deprecated ResolveScopedPath
2024-01-22 11:03:23 +09:00
Akihiro Suda
e30610aaa3 Merge pull request #47143 from thaJeztah/golang_x_updates
vendor: assorted golang.org/x/... updates
2024-01-22 11:02:48 +09:00
Sebastiaan van Stijn
a0466ca8e1 integration-cli: TestInspectAPIMultipleNetworks: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
13a384a6fa integration-cli: TestInspectAPIBridgeNetworkSettings121: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
52e3fff828 integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
This test was added in 75f6929b44, but pinned
to the API version that was current at the time (v1.20), which is now
deprecated.

Update the test to use the current API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:37:31 +01:00
Sebastiaan van Stijn
521123944a docs/api: remove version matrices from swagger files
These tables linked to deprecated API versions, and an up-to-date version of
the matrix is already included at https://docs.docker.com/engine/api/#api-version-matrix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:17:53 +01:00
Sebastiaan van Stijn
d54be2ee6d docs: remove documentation for deprecated API versions < v1.23
These versions are deprecated in v25.0.0, and disabled by default,
see 08e4e88482.

Users that need to refer to documentation for older API versions,
can use archived versions of the documentation on GitHub:

- API v1.23 and before: https://github.com/moby/moby/tree/v25.0.0/docs/api
- API v1.17 and before: https://github.com/moby/moby/tree/v1.9.1/docs/reference/api

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:12:42 +01:00
Sebastiaan van Stijn
64a6cc3afd integration/build: improve some asserts, and add asserts for unhandled errs
- add some asserts for unhandled errors
- use consts for fixed values, and slightly re-format Dockerfile contentt
- inline one-line Dockerfiles
- fix some vars to be properly camel-cased
- improve assert for error-types;

Before:

    === RUN   TestBuildPlatformInvalid
        build_test.go:685: assertion failed: expression is false: errdefs.IsInvalidParameter(err)
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

After:

    === RUN   TestBuildPlatformInvalid
        build_test.go:689: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
a88cd68d3e integration/images: improve some asserts, and add asserts for unhandled errs
Before:

    === FAIL: amd64.integration.image TestImagePullPlatformInvalid (0.01s)
        pull_test.go:37: assertion failed: expression is false: errdefs.IsInvalidParameter(err)

After:

    === RUN   TestImagePullPlatformInvalid
        pull_test.go:37: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
27e85c7b68 Merge pull request #47141 from thaJeztah/internalize_pkg_platforms
pkg/platforms: internalize in daemon/containerd
2024-01-20 23:47:48 +01:00
Sebastiaan van Stijn
a404017a86 vendor: golang.org/x/tools v0.14.0
full diff: https://github.com/golang/tools/comopare/v0.13.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:31 +01:00
Sebastiaan van Stijn
41a2aa2ee2 vendor: golang.org/x/oauth2 v0.11.0
full diff: https://github.com/golang/oauth2/comopare/v0.10.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:31 +01:00
Sebastiaan van Stijn
2799417da1 vendor: golang.org/x/mod v0.13.0, golang.org/x/tools v0.13.0
full diff:

- https://github.com/golang/mod/comopare/v0.11.0...v0.13.0
- https://github.com/golang/tools/comopare/v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:30 +01:00
Sebastiaan van Stijn
407ad89ff0 vendor: golang.org/x/sync v0.5.0
full diff: https://github.com/golang/sync/comopare/v0.3.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:30 +01:00
Sebastiaan van Stijn
94b4765363 pkg/platforms: internalize in daemon/containerd
This matcher was only used internally in the containerd implementation of
the image store. Un-export it, and make it a local utility in that package
to prevent external use.

This package was introduced in 1616a09b61
(v24.0), and there are no known external consumers of this package, so there
should be no need to deprecate / alias the old location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:28:56 +01:00
Sebastiaan van Stijn
c5187380d8 Merge pull request #46252 from thaJeztah/libnetwork_sandbox_sorting
libnetwork: Sandbox.ResolveName: refactor ordering of endpoints
2024-01-20 17:41:35 +01:00
Sebastiaan van Stijn
0a9bc3b507 libnetwork: Sandbox.ResolveName: refactor ordering of endpoints
When resolving names in swarm mode, services with exposed ports are
connected to user overlay network, ingress network, and local (docker_gwbridge)
networks. Name resolution should prioritize returning the VIP/IPs on user
overlay network over ingress and local networks.

Sandbox.ResolveName implemented this by taking the list of endpoints,
splitting the list into 3 separate lists based on the type of network
that the endpoint was attached to (dynamic, ingress, local), and then
creating a new list, applying the networks in that order.

This patch refactors that logic to use a custom sorter (sort.Interface),
which makes the code more transparent, and prevents iterating over the
list of endpoints multiple times.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 12:41:33 +01:00
Sebastiaan van Stijn
17c3829528 Merge pull request #47132 from corhere/allow-container-ip-outside-subpool
libnetwork: loosen container IPAM validation
2024-01-20 11:29:51 +01:00
Cory Snider
058b30023f libnetwork: loosen container IPAM validation
Permit container network attachments to set any static IP address within
the network's IPAM master pool, including when a subpool is configured.
Users have come to depend on being able to statically assign container
IP addresses which are guaranteed not to collide with automatically-
assigned container addresses.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:18:15 -05:00
Sebastiaan van Stijn
dfdd5169a2 Merge pull request #46113 from akerouanton/remove-deprecated-oom-score-adjust
daemon: remove --oom-score-adjust flag
2024-01-20 01:35:37 +01:00
Sebastiaan van Stijn
844ca49743 pkg/containerfs: remove deprecated ResolveScopedPath
This function was deprecated in b8f2caa80a
(v25.0), and is no longer in use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:27:07 +01:00
Sebastiaan van Stijn
2767d9ba05 pkg/homedir: remove deprecated Key() and GetShortcutString()
These were deprecated in ddd9665289 (v25.0),
and 3c1de2e667, and are no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:11:44 +01:00
Sebastiaan van Stijn
f16a2179a6 pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
These were deprecated in a3c97beee0 (v25.0.0),
and are no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:01:11 +01:00
Sebastiaan van Stijn
e2086b941f remove deprecated pkg/loopback (utility package for devicemapper)
This package was introduced in af59752712
as a utility package for devicemapper, which was removed in commit
dc11d2a2d8 (v25.0.0), and the package
was deprecated in bf692d47fb.

This patch removes the package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 00:55:09 +01:00
Albin Kerouanton
f07c45e4f2 daemon: remove --oom-score-adjust flag
This flag was marked deprecated in commit 5a922dc16 (released in v24.0)
and to be removed in the next release.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-20 00:40:28 +01:00
Sebastiaan van Stijn
307fe9c716 Dockerfile: update docker compose to v2.24.1
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.24.0...v2.24.1
- release notes: https://github.com/docker/compose/releases/tag/v2.24.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 23:27:59 +01:00
Sebastiaan van Stijn
dfced4b557 Dockerfile: update dev-shell version of the cli to v25.0.0
Update the docker CLI that's available for debugging in the dev-shell
to the v25 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 23:25:12 +01:00
Sebastiaan van Stijn
75fd873fe6 Merge pull request #47009 from cpuguy83/swarm_rotate_key_flake
De-flake TestSwarmClusterRotateUnlockKey... again... maybe?
2024-01-19 23:13:34 +01:00
voloder
c655b7dc78 Assert temp output directory is not an empty string
Signed-off-by: voloder <110066198+voloder@users.noreply.github.com>
2024-01-19 22:45:54 +01:00
Rob Murray
a8f7c5ee48 Detect IPv6 support in containers.
Some configuration in a container depends on whether it has support for
IPv6 (including default entries for '::1' etc in '/etc/hosts').

Before this change, the container's support for IPv6 was determined by
whether it was connected to any IPv6-enabled networks. But, that can
change over time, it isn't a property of the container itself.

So, instead, detect IPv6 support by looking for '::1' on the container's
loopback interface. It will not be present if the kernel does not have
IPv6 support, or the user has disabled it in new namespaces by other
means.

Once IPv6 support has been determined for the container, its '/etc/hosts'
is re-generated accordingly.

The daemon no longer disables IPv6 on all interfaces during initialisation.
It now disables IPv6 only for interfaces that have not been assigned an
IPv6 address. (But, even if IPv6 is disabled for the container using the
sysctl 'net.ipv6.conf.all.disable_ipv6=1', interfaces connected to IPv6
networks still get IPv6 addresses that appear in the internal DNS. There's
more to-do!)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-19 20:24:07 +00:00
Cory Snider
0046b16d87 daemon: set libnetwork sandbox key w/o OCI hook
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:23:12 +00:00
Sebastiaan van Stijn
31ccdbb7a8 Merge pull request #45687 from vvoland/volume-mount-subpath
volumes: Implement subpath mount
2024-01-19 18:41:12 +01:00
Paweł Gronowski
5bbcc41c20 volumes/subpath: Plumb context
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:21 +01:00
Paweł Gronowski
cb1af229f2 daemon/populateVolumes: Support volume subpath
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:20 +01:00
Paweł Gronowski
349a52b279 container: Change comment into debug log
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:18 +01:00
Paweł Gronowski
42afac91d7 internal/safepath: Add windows implementation
All components of the path are locked before the check, and
released once the path is already mounted.
This makes it impossible to replace the mounted directory until it's
actually mounted in the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:17 +01:00
Paweł Gronowski
5841ed4e5e internal/safepath: Adapt k8s openat2 fallback
Adapts the function source code to the Moby codebase.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:16 +01:00
Paweł Gronowski
56bb143a4d internal/safepath: Import k8s safeopen function
For use as a soft fallback if Openat2 is not available.
Source: 55fb1805a1/pkg/volume/util/subpath/subpath_linux.go

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:14 +01:00
Paweł Gronowski
3784316d46 internal/safepath: Handle EINTR in unix syscalls
Handle EINTR by retrying the syscall.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:13 +01:00
Paweł Gronowski
9a0cde66ba internal/safepath: Add linux implementation
All subpath components are opened with openat, relative to the base
volume directory and checked against the volume escape.
The final file descriptor is mounted from the /proc/self/fd/<fd> to a
temporary mount point owned by the daemon and then passed to the
underlying container runtime.
Temporary mountpoint is removed after the container is started.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:12 +01:00
Paweł Gronowski
bfb810445c volumes: Implement subpath mount
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume
directory.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:10 +01:00
Paweł Gronowski
f2e1105056 Introduce a helper that collects cleanup functions
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:17 +01:00
Paweł Gronowski
f07387466a daemon/oci: Extract side effects from withMounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:16 +01:00
Paweł Gronowski
9c8752505f volume/mounts: Rename errors in defer block
To make it easier to distinguish if an output variable is modified.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:15 +01:00
Sebastiaan van Stijn
17d80442cc Merge pull request #47111 from robmry/fix_TestAddRemoveInterface
Fix libnetwork/osl test TestAddRemoveInterface
2024-01-19 16:15:49 +01:00
Sebastiaan van Stijn
d35e923a4c Merge pull request #47116 from thaJeztah/minor_nits
assorted minor linting issues
2024-01-19 16:09:12 +01:00
Sebastiaan van Stijn
67de5c84d3 Merge pull request #47118 from vvoland/api-1.45
API: bump version to 1.45
2024-01-19 15:35:47 +01:00
Paweł Gronowski
5bcbedb7ee API: bump version to 1.45
Docker 25.0 was released with API v1.44, so any change in the API should
now target v1.45.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 14:42:19 +01:00
Sebastiaan van Stijn
35789fce99 daemon.images: ImageService.getImage: use named fields in struct literals
Prevent things from breaking if additional fields are added to this struct.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 13:11:40 +01:00
Sebastiaan van Stijn
7c1914411f daemon/images: ImageService.manifestMatchesPlatform: optimize logger
We constructed a "function level" logger, which was used once "as-is", but
also added additional Fields in a loop (for each resource), effectively
overwriting the previous one for each iteration. Adding additional
fields can result in some overhead, so let's construct a "logger" only for
inside the loop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 13:08:30 +01:00
Sebastiaan van Stijn
5581efe7cd rename "ociimage" var to be proper camelCase
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:50:13 +01:00
Sebastiaan van Stijn
66cf6e3a7a rename "image" vars to prevent conflicts with imports
We have many "image" packages, so these vars easily conflict/shadow
imports. Let's rename them (and in some cases use a const) to
prevent that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:49:53 +01:00
Tianon Gravi
4a40d10b60 Merge pull request #47109 from whalelines/git-url-regex
Fix isGitURL regular expression
2024-01-18 14:02:57 -08:00
Rob Murray
c72e458a72 Fix libnetwork/osl test TestAddRemoveInterface
For some time, when adding an interface with no IPv6 address (an
interface to a network that does not have IPv6 enabled), we've been
disabling IPv6 on that interface.

As part of a separate change, I'm removing that logic - there's nothing
wrong with having IPv6 enabled on an interface with no routable address.
The difference is that the kernel will assign a link-local address.

TestAddRemoveInterface does this...
- Assign an IPv6 link-local address to one end of a veth interface, and
  add it to a namespace.
- Add a bridge with no assigned IPv6 address to the namespace.
- Remove the veth interface from the namespace.
- Put the veth interface back into the namespace, still with an
  explicitly assigned IPv6 link local address.

When IPv6 is disabled on the bridge interface, the test passes.

But, when IPv6 is enabled, the bridge gets a kernel assigned link-local
address.

Then, when re-adding the veth interface, the test generates an error in
'osl/interface_linux.go:checkRouteConflict()'. The conflict is between
the explicitly assigned fe80::2 on the veth, and a route for fe80::/64
belonging to the bridge.

So, in preparation for not-disabling IPv6 on these interfaces, use a
unique-local address in the test instead of link-local.

I don't think that changes the intent of the test.

With the change to not-always disable IPv6, it is possible to repro the
problem with a real container, disconnect and re-connect a user-defined
network with '--subnet fe80::/64' while the container's connected to an
IPv4 network. So, strictly speaking, that will be a regression.

But, it's also possible to repro the problem in master, by disconnecting
and re-connecting the fe80::/64 network while another IPv6 network is
connected. So, I don't think it's a problem we need to address, perhaps
other than by prohibiting '--subnet fe80::/64'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-18 21:01:41 +00:00
David Dooling
768146b1b0 Fix isGitURL regular expression
Escape period (.) so regular expression does not match any character before "git".

Signed-off-by: David Dooling <david.dooling@docker.com>
2024-01-18 14:14:08 -06:00
Paweł Gronowski
615dfdf672 Merge pull request #47106 from vvoland/c8d-windows-pull-error-msg
c8d/integration: Adjust error in TestPullLinuxImageFailsOnWindows
2024-01-18 17:28:52 +01:00
Paweł Gronowski
7cbf01b994 Merge pull request #47105 from vvoland/c8d-mount-refcount-default
c8d/mount: Use ref-counted mounter by default
2024-01-18 17:26:22 +01:00
Paweł Gronowski
05695c2458 c8d/integation: Adjust error in TestPullLinuxImageFailsOnWindows
Message is different with containerd backend. The Linux test
`TestPullLinuxImageFailsOnLinux` was adjusted before, but we missed this
one.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-18 16:13:23 +01:00
Paweł Gronowski
ae6468b4b9 c8d/mount: Use ref-counted mounter by default
All commonly used filesystems should use ref-counted mounter, so make it
the default instead of having to whitelist them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-18 15:38:51 +01:00
Sebastiaan van Stijn
c183fcf116 Merge pull request #47104 from rumpl/run-snapshotter-linux-tests
ci: Always run snapshotter tests on Linux
2024-01-18 15:24:53 +01:00
Sebastiaan van Stijn
982f85b281 Merge pull request #47101 from AkihiroSuda/refactor-47100
net=host: remove /var/run/docker/netns/default from OCI config
2024-01-18 14:55:53 +01:00
Djordje Lukic
bd481592ff ci: Always run snapshotter tests on Linux
Now that we have a green CI on linux we can enable this for all PRs.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-01-18 11:16:57 +01:00
Akihiro Suda
ed15f1d717 net=host: remove /var/run/docker/netns/default from OCI config
Prior to this commit, a container running with `--net=host` had
`{"type":"network","path":"/var/run/docker/netns/default"}` in
the ``.linux.namespaces` field of the OCI Runtime Config,
but this wasn't needed.

Close issue 47100

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-18 18:09:52 +09:00
Sebastiaan van Stijn
4f9c865edd Merge pull request #46158 from elezar/refactor-rootless-tempdir
Add testutil.TempDir function
2024-01-17 17:38:21 +01:00
Sebastiaan van Stijn
436bf27e6c Merge pull request #46539 from TBBle/containerd_image_store_pr46402
c8d: Just enough Windows support to run the test suite
2024-01-17 17:31:51 +01:00
Sebastiaan van Stijn
038729a7ab Merge pull request #47091 from thaJeztah/unskip_some_tests
docker-py: unskip some tests
2024-01-17 17:08:05 +01:00
Sebastiaan van Stijn
8b0468e9ee Merge pull request #47090 from thaJeztah/re_enable_check
Revert "temporary: Disable deprecate-integration-cli validation"
2024-01-17 17:05:39 +01:00
Sebastiaan van Stijn
568c064795 docker-py: unskip some tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 16:44:06 +01:00
Paul "TBBle" Hampson
e8f4bfb374 Root.Path for a process-isolated WCOW container must be the Volume GUID
The actual divergence is due to differences in the snapshotter and
graphfilter mount behaviour on Windows, but the snapshotter behaviour is
better, so we deal with it here rather than changing the snapshotter
behaviour.

We're relying on the internals of containerd's Windows mount
implementation here. Unless this code flow is replaced, future work is
to move getBackingDeviceForContainerdMount into containerd's mount
implementation.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:32 +01:00
Paul "TBBle" Hampson
ec041193f9 Windows doesn't have 'host' mode networking, so use 'auto'.
That means 'null', not that we can call builder-next on Windows. If and
when we do get builder-next going, this will need to be solved properly
in some way.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:30 +01:00
Paul "TBBle" Hampson
66325f7271 Implement GetLayerFolders for the containerd image store
The existing API ImageService.GetLayerFolders didn't have access to the
ID of the container, and once we have that, the snapshotter Mounts API
provides all the information we need here.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:28 +01:00
Paul "TBBle" Hampson
0dc07ccc3a Unmount RWLayer during Commit
Needed for Diff on Windows. Don't remount it afterwards as the layer is
going to be released anyway.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:27 +01:00
Paul "TBBle" Hampson
a1f6b64e24 Don't call setupInit in a temp mount if setupInit is nil
This is consistent with layerStore's CreateRWLayer behaviour.

Potentially this can be refactored to avoid creating the -init layer,
but as noted in layerStore's initMount, this name may be special, and
should be cleared-out all-at-once.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:25 +01:00
Paul "TBBle" Hampson
efadb70ef8 The Windows snapshotter and graphdriver have different names
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:24 +01:00
Paul "TBBle" Hampson
641050c93f Typo fixes
* conatinerd => containerd
* ROLayer => RWLayer

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:22 +01:00
Sebastiaan van Stijn
83de55b370 Merge pull request #47084 from AkihiroSuda/dockerd-rootless-setuptool-nsenter
dockerd-rootless-setuptool.sh: add `nsenter` subcommand (for debugging)
2024-01-17 16:16:23 +01:00
Sebastiaan van Stijn
508a2d979a Revert "temporary: Disable deprecate-integration-cli validation"
This reverts commit bdc7d0c2db.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 15:59:52 +01:00
Sebastiaan van Stijn
1fd682930a Merge pull request #46634 from rumpl/c8d-classic-builder-cache
c8d: make the cache in classic builder work
2024-01-17 15:51:16 +01:00
Evan Lezar
f7065ab207 Add testutil.TempDir function
This change adds a TempDir function that ensures the correct permissions for
the fake-root user in rootless mode.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2024-01-17 15:44:47 +01:00
Akihiro Suda
84d2229ffe dockerd-rootless-setuptool.sh: add nsenter subcommand
Usage: `dockerd-rootless-setuptool.sh nsenter -- ip a`

Expected to be used for debugging.

Ported from nerdctl's `containerd-rootless-setuptool.sh`
https://github.com/containerd/nerdctl/blob/v1.7.2/extras/rootless/containerd-rootless-setuptool.sh#L142-L147

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Akihiro Suda
7d08d84b03 dockerd-rootless.sh: set rootlesskit --state-dir=DIR
Now the state dir is set to `${XDG_RUNTIME_DIR}/dockerd-rootless`.

This is similar to `${XDG_RUNTIME_DIR}/containerd-rootless` used in nerdctl:
https://github.com/containerd/nerdctl/blob/v1.7.2/extras/rootless/containerd-rootless.sh#L35

Prior to this commit, the state dir was unset and a random dir under `/tmp` was used.
(e.g., `/tmp/rootlesskit1869901982`)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Akihiro Suda
ae9fbcb82c testutil/daemon: shorten XDG_RUNTIME_DIR
XDG_RUNTIME_DIR will contain sockets so its path mustn't be too long.
Prior to this commit, it was set to very long path like
`/go/src/github.com/docker/docker/bundles/test-integration/TestDiskUsage/de4fb36576d7d/xdgrun`

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Paweł Gronowski
bdc7d0c2db temporary: Disable deprecate-integration-cli validation
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:58 +01:00
Paweł Gronowski
8390bc5683 c8d/cache: Use ContainerConfig from content store
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:55 +01:00
Paweł Gronowski
f760cb4f97 c8d/builder: Store ContainerConfig
Serialize ContainerConfig to content store and store its digest in
label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:54 +01:00
Paweł Gronowski
a5a15c7782 c8d/cache: Optimize FROM scratch case
Consider only images that were built `FROM scratch` as valid candidates
for the `FROM scratch` + INSTRUCTION build step.

The images are marked as `FROM scratch` based by the classic builder
with a special label. It must be a new label instead of empty parent
label, because empty label values are not persisted.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:53 +01:00
Djordje Lukic
71ebfc7c63 c8d: make the cache in classic builder work
In order for the cache in the classic builder to work we need to:
- use the came comparison function as the graph drivers implementation
- save the container config when commiting the image
- use all images to search a 'FROM "scratch"' image
- load all images if `cacheFrom` is empty

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:50 +01:00
Sebastiaan van Stijn
af4f6c124d Merge pull request #47088 from vvoland/testing-setupTest-parallel
environment: Error when t.Parallel was called before Protect
2024-01-17 14:57:19 +01:00
Sebastiaan van Stijn
60ed73bb91 Merge pull request #47044 from corhere/lazy-health-status
Lazily checkpoint container health status to disk
2024-01-17 14:12:38 +01:00
Sebastiaan van Stijn
d3e08fe3cf Merge pull request #47087 from neersighted/cdi_feature
cdi: use separate feature-flag
2024-01-17 14:11:41 +01:00
Paweł Gronowski
24da5233dd integration: Fix Parallel before setupTest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 13:06:05 +01:00
Bjorn Neergaard
d22c775e04 cdi: use separate feature-flag
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 11:31:37 +01:00
Paweł Gronowski
fa4deb02af environment: Error when t.Parallel was called before Protect
Protecting the environment relies on the shared state (containers,
images, etc) which might already be mutated by other tests if the test
opted in into the Parallel execution before Protect was called.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 10:54:11 +01:00
Cory Snider
97d32bb7d7 daemon: stop checkpointing health probes to disk
The health status and probe log of containers are not mission-criticial
data which must survive a crash. It is not worth prematrely wearing out
consumer-grade flash storage by overwriting and fsync()ing the container
config on after every probe. Update only the live Container object and
the ViewDB replica on every container health probe instead. It will
eventually get checkpointed along with some other state (or config)
change. Running containers will not be checkpointed on daemon shutdown
when live-restore is enabled, but it does not matter: the health status
and probe log will be zeroed out when the daemon starts back up.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-16 14:09:40 -05:00
Sebastiaan van Stijn
2c47a6df0d Merge pull request #46978 from vvoland/c8d-fix-export
c8d/exporter: Use WithSkipMissing
2024-01-16 20:04:17 +01:00
Brian Goff
353bccdf22 Merge pull request #47051 from corhere/reinit-health-on-live-restore 2024-01-16 10:08:20 -08:00
Sebastiaan van Stijn
9045bea756 Merge pull request #47082 from vvoland/c8d-integration-parallel-fix
integration/image: Move Parallel after setupTest
2024-01-16 18:41:30 +01:00
Sebastiaan van Stijn
8afd6daa3a Merge pull request #47086 from AkihiroSuda/fix-47085
(*Daemon).fillRootlessVersion: fix nil panic
2024-01-16 18:34:30 +01:00
Akihiro Suda
a6d9462cb1 (*Daemon).fillRootlessVersion: fix nil panic
Fix issue 47085

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 00:50:53 +09:00
Akihiro Suda
5c4189e6b2 Merge pull request #47083 from AkihiroSuda/follow-up-47076
dockerd-rootless.sh: fix inconsistent description about "builtin" driver
2024-01-17 00:19:38 +09:00
Akihiro Suda
e954cdc188 dockerd-rootless.sh: fix inconsistent description about "builtin" driver
The "builtin" port driver was marked as "Slow" in the row for the lxc-user-nic
network driver, while it was marked as "Fast" in other rows.

It had to be consistently marked as "Fast" regardless to the network driver.
It is still not as fast as rootful.

Follow-up to PR 47076
Fixes: b5a5ecf4a3

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 23:11:40 +09:00
Paweł Gronowski
4ff8436ebd Merge pull request #47081 from vvoland/c8d-integration-cli-skipcontainerconfig
c8d/integration-cli: Skip tests checking `ContainerConfig`
2024-01-16 14:50:02 +01:00
Paweł Gronowski
9de132dbf6 integration/image: Move Parallel after setupTest
setupTest should be called before Parallel as it modifies the test
environment which might produce:

```
fatal error: concurrent map writes

goroutine 143 [running]:
github.com/docker/docker/testutil/environment.(*Execution).ProtectContainer(...)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:59
github.com/docker/docker/testutil/environment.ProtectContainers({0x12e8d98, 0xc00040e420}, {0x12f2878?, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:68 +0xb1
github.com/docker/docker/testutil/environment.ProtectAll({0x12e8d98, 0xc00040e210}, {0x12f2878, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:45 +0xf3
github.com/docker/docker/integration/image.setupTest(0xc0004fc340)
	/go/src/github.com/docker/docker/integration/image/main_test.go:46 +0x59
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:45:26 +01:00
Paweł Gronowski
f0efb90f21 c8d/integration-cli: Skip tests checking ContainerConfig
Field is not supported with containerd integration and deprecated in
API 1.44

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:03:58 +01:00
Paweł Gronowski
2a02403a94 c8d/exporter: Use WithSkipMissing
Save the unmodified manifest list to keep the image ID of the
multi-platform images when not all platforms are present.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 13:06:43 +01:00
Sebastiaan van Stijn
a42726f866 Merge pull request #47078 from vvoland/docker-save-fix-layers-2
image/save: Derive Descriptor from actual tar archive
2024-01-16 12:55:55 +01:00
Sebastiaan van Stijn
b12cf05c82 Merge pull request #47076 from AkihiroSuda/rootlesskit-v2
Update RootlessKit to v2.0.0 (support Pasta)
2024-01-16 12:12:25 +01:00
Paweł Gronowski
9f50e06993 image/save: Refactor saveLayer
`diffIDPaths` is not used and can be removed.

`savedConfig` stores if the config was already saved (ID of the image is
the ID of the config).
`savedLayers` stores if the layer (diff ID) was already saved.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 11:14:15 +01:00
Akihiro Suda
b5a5ecf4a3 dockerd-rootless.sh: add comments about pasta
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:30 +09:00
Akihiro Suda
f5d05b43d7 Dockerfile: update RootlessKit to v2.0.0
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0

=== Pasta ===
RootlessKit v2 adds the support for pasta (https://passt.top/passt/).
Pasta is similar to slirp4netns but its port forwarder achieves better
throughput than slirp4netns port driver.

It is still not faster than RootlessKit's `builtin` port driver, but unlike the
`builtin` port driver, pasta can retain source IP address information.

Network driver | Port driver    | Net throughput | Port throughput | Src IP | No SUID | Note
---------------|----------------|----------------|-----------------|--------|---------|--------------------------------------------
slirp4netns    | builtin        | Slow           | Fast          |      |       | Default in typical setup
vpnkit         | builtin        | Slow           | Fast          |      |       | Default when slirp4netns is not installed
slirp4netns    | slirp4netns    | Slow           | Slow            |      |       |
**pasta**      | **implicit**   | Slow           | Fast          |      |       | Experimental
lxc-user-nic   | builtin        | Fast         | Slow            |      |       | Experimental
(bypass4netns) | (bypass4netns) | Fast         | Fast          |      |       | (Not integrated to RootlessKit)

=== Detach-netns ===
Aside from pasta, RootlessKit v2 also brings the support for
"detach-netns" mode, which leaves the runtime in the host network namespace to
eliminate the slirp overhead for pull/push and to allow accessing the "real"
127.0.0.1.

See containerd/nerdctl PR 2723 for how detach-netns is being adopted in
nerdctl v2.

Integrating detach-netns into Docker/Moby will need an extra work and will be
deferred to Docker v26 (or later).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:29 +09:00
Akihiro Suda
ba86626642 vendor.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.0
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:27 +09:00
Cory Snider
312450d079 integration: test container healthcheck is reset
Update the TestDaemonRestartKilContainers integration test to assert
that a container's healthcheck status is always reset to the Starting
state after a daemon restart, even when the container is live-restored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-15 15:30:22 -05:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Sebastiaan van Stijn
3eba4216e0 Merge pull request #47080 from thaJeztah/bump_mountinfo
vendor: github.com/moby/sys/mountinfo v0.7.1
2024-01-15 13:11:50 +01:00
Sebastiaan van Stijn
6d90f1168b Merge pull request #47079 from thaJeztah/bump_x_sys
vendor: golang.org/x/sys v0.16.0
2024-01-15 13:11:35 +01:00
Sebastiaan van Stijn
c4d2968183 vendor: github.com/moby/sys/mountinfo v0.7.1
full diff: https://github.com/moby/sys/compare/mountinfo/v0.6.2...mountinfo/v0.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 12:09:02 +01:00
Sebastiaan van Stijn
075e1141b8 vendor: golang.org/x/sys v0.16.0
full diff: https://github.com/golang/sys/compare/v0.15.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 12:05:29 +01:00
Sebastiaan van Stijn
87041b207f Merge pull request #46980 from thaJeztah/bump_compare
vendor: github.com/google/go-cmp v0.6.0, github.com/google/uuid v1.5.0
2024-01-15 11:55:41 +01:00
Sebastiaan van Stijn
770271d98f Merge pull request #47075 from rapsealk/docs/doc-comments-on-volume-mounts-mountpoint-spec
doc: Correct typo in doc comments for volume/mounts/mounts/MountPoint
2024-01-15 10:44:47 +01:00
Sebastiaan van Stijn
a759acd958 vendor: github.com/google/uuid v1.5.0
full diff: https://github.com/google/uuid/compare/v1.3.1...v1.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 10:40:15 +01:00
Sebastiaan van Stijn
3bcb01a84f vendor: github.com/google/go-cmp v0.6.0
full diff: https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 10:40:13 +01:00
Jeongseok Kang
36dd2f029f doc: Correct typo in doc comments for volume/mounts/mounts/MountPoint
Signed-off-by: Jeongseok Kang <piono623@naver.com>
2024-01-15 09:25:09 +09:00
Sebastiaan van Stijn
608238ead6 Merge pull request #46979 from thaJeztah/move_mergo
vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
2024-01-12 21:14:59 +01:00
Sebastiaan van Stijn
3a15c6c9d7 Merge pull request #47070 from thaJeztah/bump_containerd_binary
update containerd binary to v1.7.12
2024-01-12 21:14:07 +01:00
Sebastiaan van Stijn
8f89f10b40 Merge pull request #47068 from vvoland/docker-save-fix-layers
image/save: Fix missing layers in manifest
2024-01-12 19:39:56 +01:00
Sebastiaan van Stijn
6e2e7641f9 Merge pull request #47067 from thaJeztah/bump_buildx_compose
Dockerfile: update buildx to v0.12.1
2024-01-12 18:37:10 +01:00
Sebastiaan van Stijn
92175dd78c vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
Switch github.com/imdario/mergo to dario.cat/mergo v1.0.0, because
the module was renamed, and reached v1.0.0

full diff: https://github.com/imdario/mergo/compare/v0.3.13...v1.0.0

vendor: github.com/containerd/containerd v1.7.12

- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 18:09:24 +01:00
Sebastiaan van Stijn
cd1709b0d4 update containerd binary to v1.7.12
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 17:56:37 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Sebastiaan van Stijn
0790864064 Dockerfile: update buildx to v0.12.1
Update the version of buildx used in CI to the latest version.

- full diff: https://github.com/docker/buildx/compare/v0.12.0...v0.12.1
- release notes: https://github.com/docker/buildx/releases/tag/v0.12.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 16:13:34 +01:00
Sebastiaan van Stijn
ac6c791880 Merge pull request #47066 from thaJeztah/bump_compose
Dockerfile: update docker compose to v2.24.0
2024-01-12 16:12:58 +01:00
Sebastiaan van Stijn
a9d0128bd7 Dockerfile: update docker compose to v2.24.0
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.20.1...v2.24.0
- release notes: https://github.com/docker/compose/releases/tag/v2.24.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 14:43:40 +01:00
Sebastiaan van Stijn
92cc0d6257 Merge pull request #47060 from vvoland/c8d-builder-commit-shareimagecreate
c8d/commit-builder: Extract common code
2024-01-11 14:37:20 +01:00
Paweł Gronowski
28f48ce1ac c8d/commit-builder: Extract common code
Extract duplicated image creation code to a function.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-11 13:30:40 +01:00
Sebastiaan van Stijn
ef1a85ca5e Merge pull request #47052 from corhere/libc8d/create-unstarted-tasks
libcontainerd: create unstarted tasks
2024-01-10 20:41:53 +01:00
Cory Snider
659d7b190f libcontainerd: create unstarted tasks
Split task creation and start into two separate method calls in the
libcontainerd API. Clients now have the opportunity to inspect the
freshly-created task and customize its runtime environment before
starting execution of the user-specified binary.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-10 13:50:26 -05:00
Sebastiaan van Stijn
7af5682b3c Merge pull request #47053 from thaJeztah/update_golang_1.21.6
update to go1.21.6
2024-01-10 17:51:03 +01:00
Sebastiaan van Stijn
0fb6ba4ccc Merge pull request #47048 from thaJeztah/fix_version_output
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
2024-01-10 17:15:08 +01:00
Sebastiaan van Stijn
d7141cfd06 update to go1.21.6
go1.21.6 (released 2024-01-09) includes fixes to the compiler, the runtime, and
the crypto/tls, maps, and runtime/pprof packages. See the Go 1.21.6 milestone on
our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.6+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.5...go1.21.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-10 09:46:37 +01:00
Cory Snider
0e62dbadcd daemon: reinit health monitor on live-restore
The container may have been running without health probes for an
indeterminate amount of time. The container may have become unhealthy in
the interim. We should probe it sooner than in steady-state, while also
giving it some leeway to recover from e.g. timed-out connections. This
is easy to achieve by probing the container like a freshly-started one.
The original author of health-checks came to the same conclusion; the
health monitor was reinitialized on live-restored containers before
v17.11.0, when health monitoring of live-restored containers was
accidentally broken. Revert to the original behavior.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-09 11:03:06 -05:00
Sebastiaan van Stijn
afc7e581e6 Merge pull request #47049 from vvoland/c8d-fix-labels-check-filter
c8d/getImageLabelByDigest: Fix misspelled `labels` check
2024-01-09 15:55:57 +01:00
Sebastiaan van Stijn
72303ef11a Merge pull request #47050 from thaJeztah/rob_curator
Add Rob Murray (robmry) to curators
2024-01-09 15:40:54 +01:00
Sebastiaan van Stijn
4aa1bea04a Merge pull request #47046 from thaJeztah/bump_rootlesskit
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
2024-01-09 13:28:37 +01:00
Sebastiaan van Stijn
71c9aec834 Add Rob Murray (robmry) to curators
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:20:44 +01:00
Sebastiaan van Stijn
e8e20c0897 daemon/config: setPlatformDefaults: use debug for missing userland-proxy
commit 4f9db655ed moved looking up the
userland-proxy binary to early in the startup process, and introduced
a log-message if the binary was missing.

However, a side-effect of this was this message would also be printed
when running "--version";

    dockerd --version
    time="2024-01-09T09:18:53.705271292Z" level=warning msg="failed to lookup default userland-proxy binary" error="exec: \"docker-proxy\": executable file not found in $PATH"
    Docker version v25.0.0-rc.1, build 9cebefa717

We should look if we can avoid this, but let's change the message to be
a debug message as a short-term workaround.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:18:04 +01:00
Paweł Gronowski
18a0e144c2 c8d/getImageLabelByDigest: Fix misspelled labels check
It should be `labels.*` not `label.*`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-09 13:16:56 +01:00
Sebastiaan van Stijn
e27ffdab0f Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
Commit 0b1c1877c5 updated the version in
hack/dockerfile/install/rootlesskit.installer, but forgot to update the
version in Dockerfile.

Also updating both to use a tag, instead of commit. While it's good to pin by
an immutable reference, I think it's reasonably safe to use the tag, which is
easier to use, and what we do for other binaries, such as runc as well.

Full diff: https://github.com/rootless-containers/rootlesskit/compare/v1.1.0...v1.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 09:54:47 +01:00
Sebastiaan van Stijn
c41f94eda3 Merge pull request #47042 from thaJeztah/remove_tarsplit_deadcode
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
2024-01-08 22:40:14 +01:00
Sebastiaan van Stijn
22b08a10ff Merge pull request #47032 from thaJeztah/otel_align_versions
vendor: otelhttptrace v0.45.0 to align with other OTEL packages
2024-01-08 17:07:16 +01:00
Sebastiaan van Stijn
97d91aff7b Merge pull request #47027 from thaJeztah/bump_cgroups
vendor: github.com/containerd/cgroups/v3 v3.0.3
2024-01-08 13:09:33 +01:00
Sebastiaan van Stijn
13225400c8 Merge pull request #47039 from thaJeztah/bump_tarsplit
vendor: github.com/vbatts/tar-split v0.11.5
2024-01-08 13:09:07 +01:00
Sebastiaan van Stijn
adba0c2b63 Merge pull request #46202 from thaJeztah/daemon_deferrrrr
daemon: Daemon.connectToNetwork, Daemon.ContainerRename: improve err-handling in defers
2024-01-08 12:40:45 +01:00
Sebastiaan van Stijn
782fe1fe82 layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
The only use is in `builder/builder-next/adapters/snapshot.EnsureLayer()`,
which always calls the function with an _empty_ `oldTarDataPath`;
7082aecd54/builder/builder-next/adapters/snapshot/layer.go (L81)

When called with an empty `oldTarDataPath`, this function was an alias for
`checksumForGraphIDNoTarsplit`, so let's make it that.

Note that this code was added in 500e77bad0, as
part of the migration from "v1" images to "v2" (content-addressable) images.

Given that the remaining code lives in a "migration" file, possibly more code
can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 12:28:41 +01:00
Sebastiaan van Stijn
1dd415df1a vendor: github.com/vbatts/tar-split v0.11.5
full diff: https://github.com/vbatts/tar-split/compare/v0.11.3...v0.11.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 11:40:15 +01:00
Sebastiaan van Stijn
7082aecd54 Merge pull request #47030 from knight42/feat/check-wrapped-err
Make errdefs.Is<SomeError> helper functions work with wrapped errors
2024-01-08 09:40:32 +01:00
Sebastiaan van Stijn
e6f46e637c Merge pull request #43697 from thaJeztah/builder_remove_removecontainer
Builder: some improvements on cleaning up containers
2024-01-08 09:37:58 +01:00
Sebastiaan van Stijn
b7042cb767 Merge pull request #47031 from thaJeztah/go_connection_0.5.0
vendor: github.com/docker/go-connections v0.5.0
2024-01-06 11:12:34 +01:00
Sebastiaan van Stijn
372cd16b88 vendor: otelhttptrace v0.45.0 to align with other OTEL packages
This package / module is used by BuildKit's detect package; the buildkit
version we use (v0.12.x) [is using OTEL v0.40.0][1], whereas Moby and Containerd
moved to v0.45.0.

Go module [minimum-version-selection (MVS)][1] will pick the minimum required
version, and because there's no "reverse" dependency from the "main" OTEL modules
to this module, it selects the lowest version (specified in the BuildKit module).

Usually this would be a _good_ thing, but we know that OTEL likes all versions
to be aligned, so let's make it so.

full diff: https://github.com/open-telemetry/opentelemetry-go-contrib/compare/instrumentation/net/http/httptrace/otelhttptrace/v0.40.0...instrumentation/net/http/httptrace/otelhttptrace/v0.45.0

[1]: https://github.com/moby/buildkit/blob/v0.12.4-2-g3b6880d2a/go.mod#L73-L83
[2]: https://go.dev/ref/mod#glos-minimal-version-selection

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-06 11:04:13 +01:00
Sebastiaan van Stijn
22597efa06 vendor: github.com/docker/go-connections v0.5.0
no diff, as the tag is the same commit as we used already;
https://github.com/docker/go-connections/compare/fa09c952e3ea...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 18:20:12 +01:00
Sebastiaan van Stijn
1f9098e7d0 builder: remove redundant ExecBackend.ContainerKill()
The `ExecBackend.ContainerKill()` function was called before removing a build-
container.

This function is backed by `daemon.ContainerKill()` which, if no signal is passed,
performed a `daemon.Kill()`, using `SIGKILL` as signal. However, the
`ExecBackend.ContainerRm()` (backed by `daemonContainerRm()`), which is called
after this, is executed with the `ForceRemove` option set, which calls
`daemon.cleanupContainer()` with `ForceRemove` set, which also results in
`daemon.Kill()` being called:
1a0c15abbb/daemon/delete.go (L84-L95)

This makes the `ExecBackend.ContainerKill()` redundant, so removing this from
the interface.

While looking at this code, one (possible) race-condition was found in
`daemon.cleanupContainer()`, where `daemon.Kill()` could return a `errdefs.Conflict`
if the container was already stopped. An extra check was added for this case to
prevent `daemon.cleanupContainer()` from terminating early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:36:41 +01:00
Sebastiaan van Stijn
0aa96c5512 builder/dockerfile: containerManager.RemoveAll() prevent partial cleanup
Prevent cleanup from terminating early when failing to remove a container;

- continue trying to remove remaining containers
- ignore errors due to containers that were not found

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:34:34 +01:00
Sebastiaan van Stijn
03a8188a9a builder/dockerfile: remove containerManager.removeContainer()
This was just a very thin wrapper for backend.ContainerRm(), and the
error it returned was not handled, so moving this code inline.

Moving it inline also allows differentiating the error message to
distinguish the "removing all intermediate containers" from "removing container"
(when cancelling a build).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:32:54 +01:00
Sebastiaan van Stijn
6b1baf8dd2 Merge pull request #47028 from vvoland/ci-mobybin-strip-tag-v
hack/make: Strip "v" tag prefix name from VERSION
2024-01-05 16:18:08 +01:00
Sebastiaan van Stijn
a38b5d7430 daemon: Daemon.ContainerRename: move vars closer to where they're used
Also break-up some "if" statements that were hiding that they were updating
existing variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 16:00:47 +01:00
Sebastiaan van Stijn
00a863b25e daemon: Daemon.connectToNetwork: rename named return to prevent shadowing
The output var was used in a `defer`, but named `err` and shadowed in various
places. Rename the var to a more explicit name to make clear where it's used
and to prevent it being accidentally shadowed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 15:56:17 +01:00
Jian Zeng
0f337e9c14 feat: make errdefs.IsXXX helper functions work with wrapped errors
Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>
2024-01-05 22:54:13 +08:00
Paweł Gronowski
dd41f86339 hack/make: Refactor VERSION normalization
Turn the if-else tree into a `case`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-05 13:21:13 +01:00
Sebastiaan van Stijn
2af17cd3f5 vendor: github.com/containerd/cgroups/v3 v3.0.3
release notes:

- NewSystemd handles UnitExists when starting units
- makefile fixes
- cgroups2: export memory max usage and swap max usage
- build(deps): bump github.com/cilium/ebpf from v0.9.1 to v0.11.0
- support psi
- feat: add Threads for cgroupv2
- Linux.Swap is defined as memory+swap combined, while in cgroup v2 swap is a separate value
- fix(): support re-enabling oom killer refs #307 by @kestrelcjx

full diff: https://github.com/containerd/cgroups/compare/v3.0.2...v3.0.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 13:18:14 +01:00
Paweł Gronowski
9a5393d500 hack/make: Strip "v" tag prefix name from VERSION
To make the version format in the `moby-bin` consistent with the
version we use in the release pipeline.

```diff
Server: Docker Engine - Community
 Engine:
-  Version:          v25.0.0
+  Version:          25.0.0
...
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-05 13:17:47 +01:00
Sebastiaan van Stijn
12128189eb vendor: github.com/cilium/ebpf v0.11.0
full diff: https://github.com/cilium/ebpf/compare/v0.9.1...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 13:14:49 +01:00
Sebastiaan van Stijn
c3c1ee0150 Merge pull request #47024 from thaJeztah/fix_daemon_side_digest_resolve
daemon/cluster: Cluster.imageWithDigestString: include mirrors to resolve digest
2024-01-05 12:13:35 +01:00
Sebastiaan van Stijn
07d2ad30e5 daemon/cluster: Cluster.imageWithDigestString: include mirrors to resolve digest
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.

For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.

This patch brings the daemon-side lookup of image-manifests on-par with the
client-side lookup (the GET /distribution endpoint) as used in API 1.30 and
higher.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 16:53:45 +01:00
Sebastiaan van Stijn
9cebefa717 Merge pull request #47023 from thaJeztah/fix_distribution_mirrors
api: fix "GET /distribution" endpoint ignoring mirrors
2024-01-04 16:38:22 +01:00
Sebastiaan van Stijn
8aacbb3ba9 api: fix "GET /distribution" endpoint ignoring mirrors
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.

If the daemon is configured to use a mirror for the default (Docker Hub)
registry, did not fall back to querying the upstream if the mirror did not
contain the given reference.

For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.

This problem was caused by the logic used in GetRepository, which had an
optimization to only return the first registry it was successfully able to
configure (and connect to), with the assumption that the mirror either contained
all images used, or to be configured as a pull-through mirror.

This patch:

- Introduces a GetRepositories method, which returns all candidates (both
  mirror(s) and upstream).
- Updates the endpoint to try all

Before this patch:

    # the daemon is configured to use a mirror for Docker Hub
    cat /etc/docker/daemon.json
    { "registry-mirrors": ["http://localhost:5000"]}

    # start the mirror (empty registry, not configured as pull-through mirror)
    docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2

    # querying the endpoint fails, because the image-manifest is not found in the mirror:
    curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json
    {
      "message": "manifest unknown: manifest unknown"
    }

With this patch applied:

    # the daemon is configured to use a mirror for Docker Hub
    cat /etc/docker/daemon.json
    { "registry-mirrors": ["http://localhost:5000"]}

    # start the mirror (empty registry, not configured as pull-through mirror)
    docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2

    # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry):
    curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq .
    {
      "Descriptor": {
        "mediaType": "application/vnd.oci.image.index.v1+json",
        "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e",
        "size": 3849
      },
      "Platforms": [
        {
          "architecture": "amd64",
          "os": "linux"
        }
      ]
    }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 15:46:32 +01:00
Sebastiaan van Stijn
e9042385ac Merge pull request #47022 from akerouanton/libnet-bridge-dead-check-on-stale-default-nw
libnet/d/bridge: dead code: no conflict on stale default nw
2024-01-04 14:39:14 +01:00
Albin Kerouanton
b9e27acabc libnet/d/bridge: dead code: no conflict on stale default nw
A check was added to the bridge driver to detect when it was called to
create the default bridge nw whereas a stale default bridge already
existed. In such case, the bridge driver was deleting the stale network
before re-creating it. This check was introduced in docker/libnetwork@6b158eac6a
to fix an issue related to newly introduced live-restore.

However, since commit docker/docker@ecffb6d58c,
the daemon doesn't even try to create default networks if there're
active sandboxes (ie. due to live-restore).

Thus, now it's impossible for the default bridge network to be stale and
to exists when the driver's CreateNetwork() method is called. As such,
the check introduced in the first commit mentioned above is dead code
and can be safely removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-04 11:50:04 +01:00
Paweł Gronowski
585d74bad1 pkg/ioutils: Make subsequent Close attempts noop
Turn subsequent `Close` calls into a no-op and produce a warning with an
optional stack trace (if debug mode is enabled).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-04 11:21:04 +01:00
Albin Kerouanton
0a26cdf344 libnet/d/bridge: remove dead ActiveEndpointsError
This error is unused since docker/libnetwork@6b158eac6.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-04 11:12:53 +01:00
Brian Goff
12c7411b6b Merge pull request #46750 from thaJeztah/daemon_start_log 2024-01-03 17:19:59 -08:00
Brian Goff
fbdc02534a De-flake TestSwarmClusterRotateUnlockKey... again... maybe?
This hopefully makes the test less flakey (or removes any flake that
would be caused by the test itself).

1. Adds tail of cluster daemon logs when there is a test failure so we
   can more easily see what may be happening
2. Scans the daemon logs to check if the key is rotated before
   restarting the daemon. This is a little hacky but a little better
   than assuming it is done after a hard-coded 3 seconds.
3. Cleans up the `node ls` check such that it uses a poll function

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-01-04 00:18:58 +00:00
Brian Goff
e2b78f074b Merge pull request #47020 from thaJeztah/fix_go_compilerbug
daemon: work around go1.21 compiler bug
2024-01-03 16:10:00 -08:00
Sebastiaan van Stijn
cc6d826fdd Merge pull request #46971 from thaJeztah/api_docs
docs/api: add documentation for API v1.44
2024-01-04 00:39:05 +01:00
Cory Snider
6d2c4f87af daemon: work around go1.21 compiler bug
The Go 1.21.5 compiler has a bug: per-file language version override
directives do not take effect when instantiating generic functions which
have certain nontrivial type constraints. Consequently, a module-mode
project with Moby as a dependency may fail to compile when the compiler
incorrectly applies go1.16 semantics to the generic function call.

As the offending function is trivial and is only used in one place, work
around the issue by converting it to a concretely-typed function.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-04 00:02:39 +01:00
Sebastiaan van Stijn
69c23cc0c9 docs/api: add documentation for API v1.44
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v25.0.0 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-03 23:45:54 +01:00
Sebastiaan van Stijn
e7001c1455 Merge pull request #47015 from thaJeztah/api_fields_deprecation
api: Deprecate outdated fields in NetworkSettingsBase
2024-01-03 19:41:59 +01:00
Sebastiaan van Stijn
7df554acb0 Merge pull request #46974 from akerouanton/fud-debug-log
libnet: Improve the debug log written when the extKeyListener is stopped
2024-01-03 19:28:58 +01:00
Sebastiaan van Stijn
08632253d9 Merge pull request #46972 from dmcgowan/fix-userns-capabilities
c8d: Fix image commit with userns mapping (carry)
2024-01-03 19:18:12 +01:00
Albin Kerouanton
ce574949f7 api: Deprecate outdated fields in NetworkSettingsBase
The following fields are never written and are now marked as deprecated:

- `HairpinMode`
- `LinkLocalIPv6Address`
- `LinkLocalIPv6PrefixLen`
- `SecondaryIPAddress`
- `SecondaryIPv6Addresses`

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-03 18:51:17 +01:00
Sebastiaan van Stijn
57a37ed583 Merge pull request #47016 from thaJeztah/bump_golangci_lint
update golangci-lint to v1.55.2
2024-01-03 18:45:37 +01:00
Sebastiaan van Stijn
1f6c42c678 Merge pull request #47006 from thaJeztah/deprecate_ResolveScopedPath
pkg/containerfs: unify CleanScopedPath implementation, and deprecate ResolveScopedPath
2024-01-03 15:38:52 +01:00
Sebastiaan van Stijn
f8159163c1 Merge pull request #47014 from thaJeztah/daemon_deadcode
daemon: remove dead code relying on SecondaryIPAddress
2024-01-03 14:20:46 +01:00
Sebastiaan van Stijn
d5a3fccb06 update golangci-lint to v1.55.2
- full diff: https://github.com/golangci/golangci-lint/compare/v1.54.2...v1.55.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-03 13:41:37 +01:00
Sebastiaan van Stijn
7ad5396f79 Merge pull request #47013 from thaJeztah/update_authors
update AUTHORS
2024-01-03 13:16:40 +01:00
Albin Kerouanton
9b184e1268 daemon: remove dead code relying on SecondaryIPAddress
Since commit d0e0c13b60, `NetworkSettings.SecondaryAddress` is never
written.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-03 10:19:21 +01:00
Sebastiaan van Stijn
7b0f8122b2 update AUTHORS
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-03 10:12:37 +01:00
Akihiro Suda
b7ba0615d0 Merge pull request #47005 from thaJeztah/remove_pkg_aaparser
remove pkg/aaparser, and inline remaining functionality in profiles/apparmor
2024-01-03 03:43:25 +09:00
Akihiro Suda
dbe86d9d37 Merge pull request #47007 from thaJeztah/bump_runc_binary
update runc binary to v1.1.11
2024-01-03 03:42:02 +09:00
Sebastiaan van Stijn
0be7a1e33b Merge pull request #47004 from thaJeztah/portmapper_rm_err_return
libnetwork/portallocator: PortAllocator.ReleasePort: remove unused err-return
2024-01-02 18:20:35 +01:00
Sebastiaan van Stijn
8fa736945a Merge pull request #47008 from thaJeztah/vendor_runc_1.1.11
vendor: github.com/opencontainers/runc v1.1.11
2024-01-02 17:59:31 +01:00
Sebastiaan van Stijn
fc8fcf85a2 vendor: github.com/opencontainers/runc v1.1.11
This is the eleventh patch release in the 1.1.z release branch of runc.
It primarily fixes a few issues with runc's handling of containers that
are configured to join existing user namespaces, as well as improvements
to cgroupv2 support.

- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2.
  Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
  For cgroupv1, Usage and Failcnt are set by subtracting memory usage
  from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
  are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.

- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.11
- full diff: https://github.com/opencontainers/runc/compare/v1.1.10...v1.1.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 16:29:43 +01:00
Sebastiaan van Stijn
5fa4cfcabf update runc binary to v1.1.11
This is the eleventh patch release in the 1.1.z release branch of runc.
It primarily fixes a few issues with runc's handling of containers that
are configured to join existing user namespaces, as well as improvements
to cgroupv2 support.

- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2.
  Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
  For cgroupv1, Usage and Failcnt are set by subtracting memory usage
  from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
  are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.

- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.11
- full diff: https://github.com/opencontainers/runc/compare/v1.1.10...v1.1.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 16:09:16 +01:00
Sebastiaan van Stijn
84ba2558e2 Merge pull request #46976 from robmry/bridge_todos
Validate IPv6 address in libnetwork's bridge driver, remove unused error types.
2024-01-02 16:03:16 +01:00
Sebastiaan van Stijn
b8f2caa80a pkg/containerfs: deprecate ResolveScopedPath
If was a very shallow wrapper around symlink.FollowSymlinkInScope, so inline
that code instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:32:31 +01:00
Sebastiaan van Stijn
90f37f48e2 pkg/containerfs: unify CleanScopedPath implementation
Use stdlib's filepath.VolumeName to get the volume-name (if present) instead
of a self-crafted implementation, and unify the implementations for Windows
and Unix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:26:29 +01:00
Sebastiaan van Stijn
6fae583dba pkg/aaparser: remove, and integrate into profiles/apparmor
This package provided utilities to obtain the apparmor_parser version, as well
as loading a profile.

Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used, which made LoadProfile the only utility remaining in this package.

LoadProfile appears to have no external consumers, and the only use in our code
is "profiles/apparmor".

This patch moves the remaining code (LoadProfile) to profiles/apparmor as a
non-exported function, and deletes the package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:15:49 +01:00
Sebastiaan van Stijn
53d405c37f pkg/aaparser: remove deprecated GetVersion utility
Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used.

This patch removes the deprecated utility, and inlines the remaining code into
the LoadProfile function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 14:10:37 +01:00
Sebastiaan van Stijn
f714730c40 libnetwork/portallocator: PortAllocator.ReleasePort: remove unused err-return
This function never returned an error, and was not matching an interface, so
remove the error-return.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 11:00:22 +01:00
Sebastiaan van Stijn
a5b4670c79 Merge pull request #47001 from thaJeztah/portmapper_fix_release
portmapper: fix defers to prevent potentially unreleased ports
2024-01-02 10:58:24 +01:00
Sebastiaan van Stijn
abdaf7c2b5 Merge pull request #47000 from thaJeztah/userland_proxy_validation
portmapper: move userland-proxy lookup to daemon config
2024-01-02 10:55:13 +01:00
Sebastiaan van Stijn
4f9db655ed portmapper: move userland-proxy lookup to daemon config
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).

This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

However, the container would still be created (but invalid);

    docker ps -a
    CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS    PORTS     NAMES
    869f41d7e94f   nginx:alpine   "/docker-entrypoint.…"   10 seconds ago   Created             romantic_wiles

This patch changes how the userland-proxy is configured;

- The path of the userland-proxy is now looked up / configured at daemon
  startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
  produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
  is now required to be set. It no longer looks up the executable, and
  produces an error if no path was provided. While this change was not
  required, it makes the daemon config the canonical source of truth, instead
  of logic spread accross multiplee locations.

Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;

- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
  preference in PATH)

With this patch:

Validating the config produces an error if the binary is not found:

    dockerd --validate
    WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    userland-proxy is enabled, but userland-proxy-path is not set

Disabling userland-proxy prints a warning, but validates as "OK":

    dockerd --userland-proxy=false --validate
    WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    configuration OK

Speficying a non-absolute path produces an error:

    dockerd --userland-proxy-path=docker-proxy --validate
    invalid userland-proxy-path: must be an absolute path: docker-proxy

Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

Specifying an invalid userland-proxy-path produces an error as well:

    dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
    userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory

    mkdir -p /usr/local/bin/not-a-file
    dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory

    touch /usr/local/bin/not-an-executable
    dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied

Same when using the daemon.json config-file;

    echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary

    dockerd --userland-proxy-path=hello --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 16:23:18 +01:00
Sebastiaan van Stijn
488f2bd141 Merge pull request #46998 from deciduously/update-manifest-v2-urls
Updated manifest v2 URLs in image spec README
2023-12-29 15:23:11 +01:00
Sebastiaan van Stijn
214ab2caef libnetwork/portmapper: PortMapper.MapRange: inline "cleanup" closure
The cleanup function never returns an error, so didn't add much value. This
patch removes the closure, and calls it inline to remove the extra
indirection, and removes the error which would never be returned.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 14:30:42 +01:00
Sebastiaan van Stijn
6ae6dcfc53 libnetwork/portmapper: PortMapper.MapRange: fix defer
The defer was set after the switch, but various code-paths inside the switch
could return with an error after the port was allocated / reserved, which
could result in those ports not being released.

This patch moves the defer into each individual branch of the switch to set
it immediately after succesfully reserving the port.

We can also remove a redundant ReleasePort from the cleanup function, as
it's only called if an error occurs, and the defers already take care of
that.

Note that the cleanup function was handling errors returned by ReleasePort,
but this function never returns an error, so it was fully redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 14:26:56 +01:00
Sebastiaan van Stijn
8712c6df22 libnetwork/portmapper: PortMapper.MapRange: rename err-return
Prevent accidentally shadowing the error, which is used in a defer.
Also re-format the code to make it more clear we're not acting on
a locally-scoped "allocatedHostPort" variable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 14:26:06 +01:00
Ben Lovy
341e7b35a6 Updated manifest v2 URLs in image spec README
Signed-off-by: Ben Lovy <ben@deciduously.com>
2023-12-28 16:13:26 -05:00
Sebastiaan van Stijn
60b7e2bbbe Merge pull request #46997 from thaJeztah/bump_crun
Dockerfile: update crun to 1.12
2023-12-28 21:06:51 +01:00
Sebastiaan van Stijn
882b79b254 Dockerfile: update crun to 1.12
full diff: https://github.com/containers/crun/compare/1.8.7...1.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-28 10:50:28 +01:00
Sebastiaan van Stijn
1ffc120558 Merge pull request #46991 from thaJeztah/bump_compress
vendor: github.com/klauspost/compress v1.17.4
2023-12-28 10:14:35 +01:00
Sebastiaan van Stijn
dc23efd3a2 vendor: github.com/klauspost/compress v1.17.4
full diff: https://github.com/klauspost/compress/compare/v1.17.2...v1.17.4

v1.17.4:

- huff0: Speed up symbol counting
- huff0: Remove byteReader
- gzhttp: Allow overriding decompression on transport
- gzhttp: Clamp compression level
- gzip: Error out if reserved bits are set

v1.17.3:

- fse: Fix max header size
- zstd: Improve better/best compression
- gzhttp: Fix missing content type on Close

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-27 10:23:44 +01:00
Derek McGowan
f74ca4ed36 Update commit id unmap to directly use active mounts
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-22 10:14:44 -08:00
Sebastiaan van Stijn
46f7ab808b Merge pull request #46977 from crazy-max/fix-push-tag
ci(bin-image): fix merge job run condition
2023-12-21 22:35:40 +01:00
CrazyMax
0252a6f475 ci(bin-image): fix merge job run condition
All underlying jobs inherit from the status of all parent jobs
in the tree, not just the very parent. We need to apply the same
kind of special condition.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-21 16:58:36 +01:00
CrazyMax
1ea1d561c7 ci: do not run ci workflow on push tag events
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-21 16:53:46 +01:00
Rob Murray
141cb65e51 Check, then assume an IPv6 bridge has a subnet.
If IPv6 is enabled for a bridge network, by the time configuration
is applied, the bridge will always have an address. Assert that, by
raising an error when the configuration is validated.

Use that to simplify the logic used to calculate which addresses
should be assigned to a bridge. Also remove a redundant check in
setupGatewayIPv6() and the error associated with it.

Fix unit tests that enabled IPv6, but didn't supply an IPv6 IPAM
address/pool. Before this change, these tests passed but silently
left the bridge without an IPv6 address.

(The daemon already ensured there was an IPv6 address, this change
does not add a new restriction on config at that level.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-21 15:26:34 +00:00
Rob Murray
437bc829bf Don't try to validate incomplete network config.
Some checks in 'networkConfiguration.Validate()' were not running as
expected, they'd always pass - because 'parseNetworkOptions()' called
it before 'config.processIPAM()' had added IP addresses and gateways.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-21 15:16:26 +00:00
Rob Murray
52d9b0cb56 Remove unused error types.
Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-21 12:47:59 +00:00
Albin Kerouanton
f9135cdeb5 libnet: Improve the debug log written when the extKeyListener is stopped
This log message was quite spreading FUD whereas it's absolutely benign.
Reword it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-21 12:38:08 +01:00
Sebastiaan van Stijn
7bc56c5365 Merge pull request #46853 from akerouanton/libnet-ep-dns-names
libnet: Endpoint: remove isAnonymous & myAliases
2023-12-20 19:53:16 +01:00
Albin Kerouanton
13915f6521 libnet: document what Network.networkType represents
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-20 19:04:37 +01:00
Albin Kerouanton
6a2542dacf libnet: remove Endpoint.anonymous
No more concept of "anonymous endpoints". The equivalent is now an
endpoint with no DNSNames set.

Some of the code removed by this commit was mutating user-supplied
endpoint's Aliases to add container's short ID to that list. In order to
preserve backward compatibility for the ContainerInspect endpoint, this
commit also takes care of adding that short ID (and the container
hostname) to `EndpointSettings.Aliases` before returning the response.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-20 19:04:37 +01:00
Djordje Lukic
cf5a3bc531 c8d: Fix image commit with userns mapping
The remapping in the commit code was in the wrong place, we would create
a diff and then remap the snapshot, but the descriptor created in
"CreateDiff" was still pointing to the old snapshot, we now remap the
snapshot before creating a diff. Also make sure we don't lose any
capabilities, they used to be lost after the chown.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-12-20 17:30:57 +01:00
Sebastiaan van Stijn
388216fc45 Merge pull request #46850 from robmry/46829-allow_ipv6_subnet_change
Allow overlapping change in bridge's IPv6 network.
2023-12-19 18:35:13 +01:00
Sebastiaan van Stijn
69b7952b8b Merge pull request #46963 from corhere/libn/resolver-pack-servfail
libnetwork: write ServFail if DNS reply msg is bad
2023-12-19 18:33:27 +01:00
Sebastiaan van Stijn
43a82dab90 Merge pull request #46840 from dmcgowan/c8d-rmi-cleanup
containerd: Image delete fixes and cleanup
2023-12-19 18:07:16 +01:00
Sebastiaan van Stijn
c378b1bf4c Merge pull request #46966 from vvoland/c8d-dockerpy-skip-commit
c8d/docker-py: Temporarily skip test_commit_*
2023-12-19 17:52:05 +01:00
Cory Snider
5eaf898fcb libnetwork: write ServFail if DNS reply msg is bad
If the resolver's DNSBackend returns a name that cannot be marshaled
into a well-formed DNS message, the resolver will only discover this
when it attempts to write the reply message and it fails with an error.
No reply message is sent, leaving the client to wait out its timeout and
the user in the dark about what went wrong.

When writing the intended reply message fails, retry once with a
ServFail response to inform the client and user that the DNS query was
not resolved due to a problem with to the resolver, not the network.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-19 11:24:33 -05:00
Cory Snider
1da85f7bdc libnetwork: assert DNS replies are well-formed
The well-formedness of a DNS message is only checked when it is
serialized, through the (*dns.Msg).Pack() method. Add a call to Pack()
to our tstwriter mock to mirror the behaviour of the real
dns.ResponseWriter implementation. And fix tests which generated
ill-formed DNS query messages.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-19 11:13:35 -05:00
Paweł Gronowski
f6a160d34e c8d/docker-py: Temporarily skip test_commit_*
These tests check deprecated fields. Reenable them once a new docker-py
is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-19 16:28:20 +01:00
Sebastiaan van Stijn
ae726dcd2b Merge pull request #46965 from vvoland/c8d-dockerpy-skip-imagesaveload
c8d/docker-py: Temporarily skip test_save_and_load*
2023-12-19 14:58:48 +01:00
Derek McGowan
cf1ea9237c Add dangling image reference on delete when last image has children
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-19 05:56:43 -08:00
Paweł Gronowski
9603adb5b4 c8d/docker-py: Temporarily skip test_save_and_load*
They fail because exporting an image which targets a manifest list when
only one platform is available exports only the platform-specific
manifest so the ID of the loaded image is different (ID of the platform
manifest, not manifest list).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-19 11:07:24 +01:00
Albin Kerouanton
7a9b680a9c libnet: remove Endpoint.myAliases
This property is now unused, let's get rid of it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
523b907359 daemon: no more IsAnonymousEndpoint
The semantics of an "anonymous" endpoint has always been weird: it was
set on endpoints which name shouldn't be taken into account when
inserting DNS records into libnetwork's `Controller.svcRecords` (and
into the NetworkDB). However, in that case the endpoint's aliases would
still be used to create DNS records; thus, making those "anonymous
endpoints" not so anonymous.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
8b7af1d0fc libnet: update dnsNames on ContainerRename
The `(*Endpoint).rename()` method is changed to only mutate `ep.name`
and let a new method `(*Endpoint).UpdateDNSNames()` handle DNS updates.

As a consequence, the rollback code that was part of
`(*Endpoint).rename()` is now removed, and DNS updates are now
rolled back by `ContainerRename`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
3bb13c7eb4 libnet: Use Endpoint.dnsNames to create DNS records
Instead of special-casing anonymous endpoints, use the list of DNS names
associated to the endpoint.

`(*Endpoint).isAnonymous()` has no more uses, so let's delete it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:37 +01:00
Sebastiaan van Stijn
581cba2a56 Merge pull request #46964 from thaJeztah/bump_crypto_0.17
vendor: golang.org/x/crypto v0.17.0
2023-12-19 10:17:11 +01:00
Albin Kerouanton
f5cc497eac libnet: populate Endpoint.dnsNames on UnmarshalJSON
This new property will be empty if the daemon was upgraded with
live-restore enabled. To not break DNS resolutions for restored
containers, we need to populate dnsNames based on endpoint's myAliases &
anonymous properties.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:16:05 +01:00
Albin Kerouanton
ab8968437b daemon: build the list of endpoint's DNS names
Instead of special-casing anonymous endpoints in libnetwork, let the
daemon specify what (non fully qualified) DNS names should be associated
to container's endpoints.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:16:04 +01:00
Derek McGowan
0c6e9121b2 Add support for removing repo and digest
When repo and digest is provided, remove all references within a
repository for the given digest.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 16:33:53 -08:00
Sebastiaan van Stijn
c22fc41ddf vendor: golang.org/x/crypto v0.17.0
update the package, which contains a fix in the ssh package.

full diff: https://github.com/golang/crypto/compare/v0.16.0...v0.17.0

from the security mailing:

> Hello gophers,
>
> Version v0.17.0 of golang.org/x/crypto fixes a protocol weakness in the
> golang.org/x/crypto/ssh package that allowed a MITM attacker to compromise
> the integrity of the secure channel before it was established, allowing
> them to prevent transmission of a number of messages immediately after
> the secure channel was established without either side being aware.
>
> The impact of this attack is relatively limited, as it does not compromise
> confidentiality of the channel. Notably this attack would allow an attacker
> to prevent the transmission of the SSH2_MSG_EXT_INFO message, disabling a
> handful of newer security features.
>
> This protocol weakness was also fixed in OpenSSH 9.6.
>
> Thanks to Fabian Bäumer, Marcus Brinkmann, and Jörg Schwenk from Ruhr
> University Bochum for reporting this issue.
>
> This is CVE-2023-48795 and Go issue https://go.dev/issue/64784.
>
> Cheers,
> Roland on behalf of the Go team

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-19 00:30:17 +01:00
Sebastiaan van Stijn
0751141003 Merge pull request #46920 from dmcgowan/client-hijack-cleanup
Replace use of httputil in client hijack
2023-12-18 23:21:26 +01:00
Derek McGowan
87c87bccb5 Update cleanup logic to use resolve all images
Ensure that when removing an image, an image is checked consistently
against the images with the same target digest. Add unit testing around
delete.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 14:04:48 -08:00
Derek McGowan
529d19bad8 Add resolve all images and unit test
Add single resolve function to get a consistent list of images matching
the same digest.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 14:04:46 -08:00
Derek McGowan
eb9ce779f6 Update OTEL in client hijack to use transport
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:37 -08:00
Derek McGowan
c84e889a69 Remove httputil lint exceptions
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:37 -08:00
Derek McGowan
35a51fd697 Update authz plugin test to not use httputil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Derek McGowan
bb4e9c72d7 Replace use of httputil in client hijack
Simplify the hijack process by just performing the http request/response
on the connection and returning the raw conn after success. The client
conn from httputil is deprecated and easily replaced.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Albin Kerouanton
dc1e73cbbf libnet: add a new dnsNames property to Endpoint
This new property is meant to replace myAliases and anonymous
properties.

The end goal is to get rid of both properties by letting the daemon
determine what (non fully qualified) DNS names should be associated to
them.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-18 18:38:25 +01:00
Albin Kerouanton
1eb0751803 daemon: endpoints on default nw aren't anonymous
They just happen to exist on a network that doesn't support DNS-based
service discovery (ie. no embedded DNS servers are started for them).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-18 18:38:25 +01:00
Paweł Gronowski
74cf9a0f60 Merge pull request #46912 from vvoland/c8d-inspect-parent
c8d/inspect: Fill `Parent` property
2023-12-18 17:47:01 +01:00
Rob Murray
27f3abd893 Allow overlapping change in bridge's IPv6 network.
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.

(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)

IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.

Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.

Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-18 16:10:41 +00:00
Sebastiaan van Stijn
ba321d318b Merge pull request #46879 from martinjirku/add_taskstatus_to_api_yml
add TaskStatus to swagger docs
2023-12-18 16:33:07 +01:00
Paweł Gronowski
18b1aec0ee c8d/inspect: Fill Parent property
Before this change `ParentId` was filled for images when calling the
`/images/json` (image list) endpoint but was not for the
`/images/<image>/json` (image inspect).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-18 16:23:19 +01:00
Sebastiaan van Stijn
ec259ddfee Merge pull request #46959 from thaJeztah/update_crypto
vendor: golang.org/x/crypto v0.16.0, golang.org/x/sys v0.15.0, golang.org/x/text v0.14.0
2023-12-18 14:36:14 +01:00
Sebastiaan van Stijn
2b0d667b14 Merge pull request #46955 from akerouanton/libnet-remove-unused-cmd
libnet: Remove unused cmd/readme_test
2023-12-18 14:17:07 +01:00
Sebastiaan van Stijn
e48b58e1be vendor: golang.org/x/crypto v0.16.0
full diff: https://github.com/golang/crypto/compare/v0.14.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-18 13:39:50 +01:00
Sebastiaan van Stijn
44511f9e43 vendor: golang.org/x/text v0.14.0
full diff: https://github.com/golang/text/compare/v0.13.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-18 13:38:46 +01:00
Sebastiaan van Stijn
2cdd4906ac vendor: golang.org/x/sys v0.15.0
full diff: https://github.com/golang/sys/compare/v0.13.0...v0.15.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-18 13:37:57 +01:00
Albin Kerouanton
d6a656cf7f libnet: Remove unused cmd/readme_test
This command was originally added by ea7f555446
to test the code snippet put into libnet's README.md. Nothing compiles
this file and it doesn't add any value to the project. So better remove
it than maintaining it.

This commit also removes the code snippet from libnet's README.md for
the same reasons.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-16 13:06:15 +01:00
Sebastiaan van Stijn
1997933508 Merge pull request #46933 from vvoland/c8d-tag-label-duplicate
c8d/tag: Remove duplicate label copy
2023-12-15 19:10:25 +01:00
Sebastiaan van Stijn
7e1c8a5be3 Merge pull request #46943 from thaJeztah/typo_fixes
typo fixes
2023-12-15 18:39:40 +01:00
Sebastiaan van Stijn
2f80a91b3e Merge pull request #46936 from crazy-max/fix-dco-on-tag
ci(bin-image): fix conditional run for skipped job
2023-12-15 18:38:53 +01:00
Sebastiaan van Stijn
21334593d1 Merge pull request #46942 from thaJeztah/go_compat
add //go:build directives to prevent downgrading to go1.16 language
2023-12-15 18:37:15 +01:00
Sebastiaan van Stijn
d90e5f283a Merge pull request #46935 from vvoland/c8d-dockerpy-skip-swuash
c8d/docker-py: Skip test_build_squash
2023-12-15 17:24:32 +01:00
Akihiro Suda
aa5d1a8897 typo fixes
follow-up to 9e6d012e9c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-15 17:21:34 +01:00
Sebastiaan van Stijn
79c759393f Merge pull request #46647 from AkihiroSuda/runtime-status
docker info: expose runtime features ("rro" mount mode, etc.), in machine-parsable format
2023-12-15 17:19:37 +01:00
Sebastiaan van Stijn
f58f02f3e6 Merge pull request #46930 from rumpl/devcontainer
Add devcontainer configuration
2023-12-15 17:09:11 +01:00
Sebastiaan van Stijn
2cf230951f add //go:build directives to prevent downgrading to go1.16 language
This repository is not yet a module (i.e., does not have a `go.mod`). This
is not problematic when building the code in GOPATH or "vendor" mode, but
when using the code as a module-dependency (in module-mode), different semantics
are applied since Go1.21, which switches Go _language versions_ on a per-module,
per-package, or even per-file base.

A condensed summary of that logic [is as follows][1]:

- For modules that have a go.mod containing a go version directive; that
  version is considered a minimum _required_ version (starting with the
  go1.19.13 and go1.20.8 patch releases: before those, it was only a
  recommendation).
- For dependencies that don't have a go.mod (not a module), go language
  version go1.16 is assumed.
- Likewise, for modules that have a go.mod, but the file does not have a
  go version directive, go language version go1.16 is assumed.
- If a go.work file is present, but does not have a go version directive,
  language version go1.17 is assumed.

When switching language versions, Go _downgrades_ the language version,
which means that language features (such as generics, and `any`) are not
available, and compilation fails. For example:

    # github.com/docker/cli/cli/context/store
    /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)

Note that these fallbacks are per-module, per-package, and can even be
per-file, so _(indirect) dependencies_ can still use modern language
features, as long as their respective go.mod has a version specified.

Unfortunately, these failures do not occur when building locally (using
vendor / GOPATH mode), but will affect consumers of the module.

Obviously, this situation is not ideal, and the ultimate solution is to
move to go modules (add a go.mod), but this comes with a non-insignificant
risk in other areas (due to our complex dependency tree).

We can revert to using go1.16 language features only, but this may be
limiting, and may still be problematic when (e.g.) matching signatures
of dependencies.

There is an escape hatch: adding a `//go:build` directive to files that
make use of go language features. From the [go toolchain docs][2]:

> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a `go.mod` file with a go line such as `go 1.21` or `go 1.21.3`.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding `//go:build go1.22` to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.

This patch adds `//go:build` directives to those files using recent additions
to the language. It's currently using go1.19 as version to match the version
in our "vendor.mod", but we can consider being more permissive ("any" requires
go1.18 or up), or more "optimistic" (force go1.21, which is the version we
currently use to build).

For completeness sake, note that any file _without_ a `//go:build` directive
will continue to use go1.16 language version when used as a module.

[1]: 58c28ba286/src/cmd/go/internal/gover/version.go (L9-L56)
[2]: https://go.dev/doc/toolchain

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-15 15:24:15 +01:00
Akihiro Suda
9e6d012e9c docker info: expose runtime features ("rro" mount mode, etc.)
Fix issue 46580

```console
$ curl -s --unix-socket /var/run/docker.sock http://docker/v1.44/info | jq .Runtimes
{
  "crun": {
    "path": "/usr/local/bin/crun",
    "status": {
      "org.opencontainers.runtime-spec.features": "{\"ociVersionMin\":\"1.0.0\",...}"
    }
  },
  "io.containerd.runc.v2": {
    "path": "runc",
    "status": {
      "org.opencontainers.runtime-spec.features": "{\"ociVersionMin\":\"1.0.0\",...}"
    }
  },
  "runc": {
    "path": "runc",
    "status": {
      "org.opencontainers.runtime-spec.features": "{\"ociVersionMin\":\"1.0.0\",...}"
    }
  },
  "runsc": {
    "path": "/usr/local/bin/runsc"
  }
}
```

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-12-15 11:48:46 +09:00
Sebastiaan van Stijn
bd70d66a62 Merge pull request #46939 from vvoland/api-deprecate-image-container
api: Deprecate `Container` and `ContainerConfig` for `/images/{id}/json`
2023-12-14 20:54:59 +01:00
Paweł Gronowski
1602e2f4f1 api: Deprecate Container and ContainerConfig for /images/{id}/json
These fields were an implementation detail of the classic image builder
and are empty when using buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-14 17:27:41 +01:00
CrazyMax
d91bf690ef ci(bin-image): fix conditional run for skipped job
When the doc job is skipped, the dependent ones will be skipped
as well. To fix this issue we need to apply special conditions
to always run dependent jobs but not if canceled or failed.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-14 15:46:52 +01:00
CrazyMax
61d5e5ca9a ci(test): do not run on push tag events
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-14 15:43:38 +01:00
Paweł Gronowski
f8275cb303 c8d/docker-py: Skip test_build_squash
build --squash is an experimental feature that is not implemented in the
containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-14 09:27:31 +01:00
Djordje Lukic
279e6658ac Add devcontainer configuration
We already have everything needed to work inside a container, with this
configuration file developing in moby is even easier: the IDE will ask
you if you want to run everything inside a container and set it up for
you. No need to know that you have to run "BIN_DIR=. make shell" any
more.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-12-13 13:27:52 +01:00
Paweł Gronowski
d11a772080 c8d/tag: Remove duplicate label copy
No need to copy the parent label from the source dangling image, because
it will already be copied from the source image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-13 12:41:28 +01:00
Sebastiaan van Stijn
f3cc93630e Merge pull request #46926 from thaJeztah/carry_logentries_removal
logger: remove logentries driver (carry 44442)
2023-12-13 10:14:42 +01:00
Sebastiaan van Stijn
77ea269b86 Merge pull request #46925 from thaJeztah/logentries_docs_changes
api / docs: remove mentions of logentries logging driver
2023-12-13 10:11:41 +01:00
Sebastiaan van Stijn
3b1d9f1a26 add validation and migration for deprecated logentries driver
A validation step was added to prevent the daemon from considering "logentries"
as a dynamically loaded plugin, causing it to continue trying to load the plugin;

    WARN[2023-12-12T21:53:16.866857127Z] Unable to locate plugin: logentries, retrying in 1s
    WARN[2023-12-12T21:53:17.868296836Z] Unable to locate plugin: logentries, retrying in 2s
    WARN[2023-12-12T21:53:19.874259254Z] Unable to locate plugin: logentries, retrying in 4s
    WARN[2023-12-12T21:53:23.879869881Z] Unable to locate plugin: logentries, retrying in 8s

But would ultimately be returned as an error to the user:

    docker container create --name foo --log-driver=logentries nginx:alpine
    Error response from daemon: error looking up logging plugin logentries: plugin "logentries" not found

With the additional validation step, an error is returned immediately:

    docker container create --log-driver=logentries busybox
    Error response from daemon: the logentries logging driver has been deprecated and removed

A migration step was added on container restore. Containers using the
"logentries" logging driver are migrated to use the "local" logging driver:

    WARN[2023-12-12T22:38:53.108349297Z] migrated deprecated logentries logging driver  container=4c9309fedce75d807340ea1820cc78dc5c774d7bfcae09f3744a91b84ce6e4f7 error="<nil>"

As an alternative to the validation step, I also considered using a "stub"
deprecation driver, however this would not result in an error when creating
the container, and only produce an error when starting:

    docker container create --name foo --log-driver=logentries nginx:alpine
    4c9309fedce75d807340ea1820cc78dc5c774d7bfcae09f3744a91b84ce6e4f7

    docker start foo
    Error response from daemon: failed to create task for container: failed to initialize logging driver: the logentries logging driver has been deprecated and removed
    Error: failed to start containers: foo

For containers, this validation is added in the backend (daemon). For services,
this was not sufficient, as SwarmKit would try to schedule the task, which
caused a close loop;

    docker service create --log-driver=logentries --name foo nginx:alpine
    zo0lputagpzaua7cwga4lfmhp
    overall progress: 0 out of 1 tasks
    1/1: no suitable node (missing plugin on 1 node)
    Operation continuing in background.

    DEBU[2023-12-12T22:50:28.132732757Z] Calling GET /v1.43/tasks?filters=%7B%22_up-to-date%22%3A%7B%22true%22%3Atrue%7D%2C%22service%22%3A%7B%22zo0lputagpzaua7cwga4lfmhp%22%3Atrue%7D%7D
    DEBU[2023-12-12T22:50:28.137961549Z] Calling GET /v1.43/nodes
    DEBU[2023-12-12T22:50:28.340665007Z] Calling GET /v1.43/services/zo0lputagpzaua7cwga4lfmhp?insertDefaults=false
    DEBU[2023-12-12T22:50:28.343437632Z] Calling GET /v1.43/tasks?filters=%7B%22_up-to-date%22%3A%7B%22true%22%3Atrue%7D%2C%22service%22%3A%7B%22zo0lputagpzaua7cwga4lfmhp%22%3Atrue%7D%7D
    DEBU[2023-12-12T22:50:28.345201257Z] Calling GET /v1.43/nodes

So a validation was added in the service create and update endpoints;

    docker service create --log-driver=logentries --name foo nginx:alpine
    Error response from daemon: the logentries logging driver has been deprecated and removed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-13 01:10:05 +01:00
Sebastiaan van Stijn
3a14b1235c docs/api: remove logentries
The service was discontinued on November 15, 2022, so
remove mentions of this driver in the API docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 22:59:32 +01:00
Sebastiaan van Stijn
30f739db44 api/swagger: remove logentries
The service was discontinued on November 15, 2022, so
remove mentions of this driver in the API docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 22:59:31 +01:00
Bjorn Neergaard
cd41f655f9 logger: remove logentries driver
The Logentries service will be discontinued next week:

> Dear Logentries user,
>
> We have identified you as the owner of, or collaborator of, a Logentries account.
>
> The Logentries service will be discontinued on November 15th, 2022. This means that your Logentries account access will be removed and all your log data will be permanently deleted on this date.
>
> Next Steps
> If you are interested in an alternative Rapid7 log management solution, InsightOps will be available for purchase through December 16th, 2022. Please note, there is no support to migrate your existing Logentries account to InsightOps.
>
> Thank you for being a valued user of Logentries.
>
> Thank you,
> Rapid7 Customer Success

There is no reason to preserve this code in Moby as a result.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 22:51:23 +01:00
Sebastiaan van Stijn
92884c25b3 Merge pull request #46924 from thaJeztah/vendor_singleflight
vendor: resenje.org/singleflight v0.4.1
2023-12-12 16:54:01 +01:00
Sebastiaan van Stijn
dbdfc71026 vendor: resenje.org/singleflight v0.4.1
- fix race when returning call.shared

full diff: https://github.com/janos/singleflight/compare/v0.4.0...v0.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 16:07:13 +01:00
Sebastiaan van Stijn
e8c72adf54 Merge pull request #46830 from thaJeztah/vendor_containerd_1.7.9
vendor: OTEL v1.19.0 / v0.45.0, containerd v1.7.11
2023-12-12 14:56:12 +01:00
Sebastiaan van Stijn
4d2a324fce update to go.opentelemetry.io/otel/semconv/v1.21.0, remove "httpconv" uses
This commit switches our code to use semconv 1.21, which is the version matching
the OTEL modules, as well as the containerd code.

The BuildKit 0.12.x module currently uses an older version of the OTEL modules,
and uses the semconv 0.17 schema. Mixing schema-versions is problematic, but
we still want to consume BuildKit's "detect" package to wire-up other parts
of OTEL.

To align the versions in our code, this patch sets the BuildKit detect.Resource
with the correct semconv version.

It's worth noting that the BuildKit package has a custom "serviceNameDetector";
https://github.com/moby/buildkit/blob/v0.12.4/util/tracing/detect/detect.go#L153-L169

Whith is merged with OTEL's default resource:
https://github.com/moby/buildkit/blob/v0.12.4/util/tracing/detect/detect.go#L100-L107

There's no need to duplicate that code, as OTEL's `resource.Default()` already
provides this functionality:

- It uses fromEnv{} detector internally: https://github.com/open-telemetry/opentelemetry-go/blob/v1.19.0/sdk/resource/resource.go#L208
- fromEnv{} detector reads OTEL_SERVICE_NAME: https://github.com/open-telemetry/opentelemetry-go/blob/v1.19.0/sdk/resource/env.go#L53

This patch also removes uses of the httpconv package, which is no longer included
in semconv 1.21 and now an internal package. Removing the use of this package
means that hijacked connections will not have the HTTP attributes on the Moby
client span, which isn't ideal, but a limited loss that'd impact exec/attach.
The span itself will still exist, it just won't the additional attributes that
are added by that package.

Alternatively, the httpconv call COULD remain - it will not error and will send
syntactically valid spans but we would be mixing & matching semconv versions,
so won't be compliant.

Some parts of the httpconv package were preserved through a very minimal local
implementation; a variant of `httpconv.ClientStatus(resp.StatusCode))` is added
to set the span status (`span.SetStatus()`). The `httpconv` package has complex
logic for this, but mostly drills down to HTTP status range (1xx/2xx/3xx/4xx/5xx)
to determine if the status was successfull or non-successful (4xx/5xx).

The additional logic it provided was to validate actual status-codes, and to
convert "bogus" status codes in "success" ranges (1xx, 2xx) into an error. That
code seemed over-reaching (and not accounting for potential future _valid_
status codes). Let's assume we only get valid status codes.

- https://github.com/open-telemetry/opentelemetry-go/blob/v1.21.0/semconv/v1.17.0/httpconv/http.go#L85-L89
- https://github.com/open-telemetry/opentelemetry-go/blob/v1.21.0/semconv/internal/v2/http.go#L322-L330
- https://github.com/open-telemetry/opentelemetry-go/blob/v1.21.0/semconv/internal/v2/http.go#L356-L404

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 12:27:37 +01:00
Sebastiaan van Stijn
7d991b6e42 vendor: github.com/moby/buildkit v0.12.5-0.20231208203051-3b6880d2a00f
full diff: https://github.com/moby/buildkit/compare/v0.12.4...3b6880d2a00fd6080019118ef40515be905e58db

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 11:58:48 +01:00
Sebastiaan van Stijn
fcf03cdfa9 vendor: github.com/containerd/containerd v1.7.11
full diff: https://github.com/containerd/containerd/compare/v1.7.10...v1.7.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 11:58:47 +01:00
Sebastiaan van Stijn
7028a03ffa vendor: github.com/containerd/containerd v1.7.10
full diff: https://github.com/containerd/containerd/compare/v1.7.9...v1.7.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 11:58:47 +01:00
Sebastiaan van Stijn
49ad10261e vendor: github.com/containerd/containerd v1.7.9
full diff: https://github.com/containerd/containerd/compare/v1.7.8...v1.7.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 11:58:47 +01:00
Sebastiaan van Stijn
c14bd4f1ad vendor: vendor: upgrade OpenTelemetry to v1.19.0 / v0.45.0
Upgrade to the latest OpenTelemetry libraries; this will unblock a lot of
downstream projects in the ecosystem to upgrade, as some of the parts here
were pre-1.0/unstable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-12 11:58:44 +01:00
Sebastiaan van Stijn
65973c6c40 Merge pull request #46923 from thaJeztah/update_securejoin
vendor: github.com/cyphar/filepath-securejoin v0.2.4
2023-12-11 21:33:08 +01:00
Sebastiaan van Stijn
291eaf0a77 vendor: github.com/cyphar/filepath-securejoin v0.2.4
update the dependency to v0.2.4 to prevent scanners from flagging the
vulnerability (GHSA-6xv5-86q9-7xr8 / GO-2023-2048). Note that that vulnerability
only affects Windows, and is currently only used in runc/libcontainer, so should
not impact our use (as that code is Linux-only).

full diff: https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-11 13:59:37 +01:00
Sebastiaan van Stijn
86cd6da76e Merge pull request #46916 from thaJeztah/bump_docker_py
testing: update docker-py to 7.0.0
2023-12-11 12:02:23 +01:00
Sebastiaan van Stijn
a69ffc3e20 Merge pull request #46915 from thaJeztah/containerd_binary_1.7.11
update containerd binary to 1.7.11
2023-12-09 02:22:16 +01:00
Sebastiaan van Stijn
b524ed2dda testing: update docker-py to 7.0.0
full diff: https://github.com/docker/docker-py/compare/7.0.0b1...7.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-09 01:36:38 +01:00
Sebastiaan van Stijn
33d2ec08ba update containerd binary to 1.7.11
- full diff: https://github.com/containerd/containerd/compare/v1.7.10...v1.7.11
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.11

Welcome to the v1.7.11 release of containerd!

The eleventh patch release for containerd 1.7 contains various fixes and
updates including one security issue.

Notable Updates

- Fix Windows default path overwrite issue
- Update push to always inherit distribution sources from parent
- Update shim to use net dial for gRPC shim sockets
- Fix otel version incompatibility
- Fix Windows snapshotter blocking snapshot GC on remove failure
- Mask /sys/devices/virtual/powercap path in runtime spec and deny in
  default apparmor profile [GHSA-7ww5-4wqc-m92c]

Deprecation Warnings

- Emit deprecation warning for AUFS snapshotter
- Emit deprecation warning for v1 runtime
- Emit deprecation warning for deprecated CRI configs
- Emit deprecation warning for CRI v1alpha1 usage
- Emit deprecation warning for CRIU config in CRI

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-09 01:26:44 +01:00
Sebastiaan van Stijn
c51a262e34 update containerd binary to 1.7.10
- full diff: https://github.com/containerd/containerd/compare/v1.7.9...v1.7.10
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.10

Welcome to the v1.7.10 release of containerd!

The tenth patch release for containerd 1.7 contains various fixes and
updates.

Notable Updates

- Enhance container image unpack client logs
- cri: fix using the pinned label to pin image
- fix: ImagePull should close http connection if there is no available data to read.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-09 01:25:57 +01:00
Sebastiaan van Stijn
fc8d9cb8f7 Merge pull request #46914 from thaJeztah/vendor_containerd_deps
vendor: felixge/httpsnoop v1.0.4, grpc-ecosystem/grpc-gateway/v2 v2.16.0
2023-12-08 20:09:56 +01:00
Sebastiaan van Stijn
f66f464802 Merge pull request #46913 from thaJeztah/local_logs_timezone
daemon/logger/local: always use UTC for timestamps
2023-12-08 19:56:09 +01:00
Sebastiaan van Stijn
12c70b4386 vendor: github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0
full diff: https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.11.3...v2.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-08 18:59:40 +01:00
Sebastiaan van Stijn
a0615f6839 vendor: github.com/felixge/httpsnoop v1.0.4
full diff: https://github.com/felixge/httpsnoop/compare/v1.0.3...v1.0.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-08 18:59:36 +01:00
Sebastiaan van Stijn
312fbb6d41 Merge pull request #46911 from vvoland/c8d-prune-dangling-default
c8d/prune: Default `dangling` to true, familiarize untagged images
2023-12-08 18:54:29 +01:00
Sebastiaan van Stijn
afe281964d daemon/logger/local: always use UTC for timestamps
When reading logs, timestamps should always be presented in UTC. Unlike
the "json-file" and other logging drivers, the "local" logging driver
was using local time.

Thanks to Roman Valov for reporting this issue, and locating the bug.

Before this change:

    echo $TZ
    Europe/Amsterdam

    docker run -d --log-driver=local nginx:alpine
    fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3

    docker logs --timestamps fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3
    2023-12-08T18:16:56.291023422+01:00 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
    2023-12-08T18:16:56.291056463+01:00 /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
    2023-12-08T18:16:56.291890130+01:00 /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
    ...

With this patch:

    echo $TZ
    Europe/Amsterdam

    docker run -d --log-driver=local nginx:alpine
    14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5

    docker logs --timestamps 14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5
    2023-12-08T17:18:46.635967625Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
    2023-12-08T17:18:46.635989792Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
    2023-12-08T17:18:46.636897417Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
    ...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-08 18:31:30 +01:00
Paweł Gronowski
bea729c030 c8d/prune: Familiarize image names that were untagged
To align with the graphdriver implementation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-08 16:39:33 +01:00
Paweł Gronowski
8ba8a59697 c8d/prune: Default dangling filter to true
If no `dangling` filter is specified, prune should only delete dangling
images.

This wasn't visible by doing `docker image prune` because the CLI
explicitly sets this filter to true.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-08 13:58:09 +01:00
Sebastiaan van Stijn
659e7b52df Merge pull request #46896 from vvoland/TestPruneDontDeleteUsedDangling-separate-daemon
integration/prune: Run in a separate daemon
2023-12-07 20:39:27 +01:00
Sebastiaan van Stijn
294274e10e Merge pull request #46847 from thaJeztah/uncontainerise_TestDaemonNoSpaceLeftOnDeviceError
un-containerize TestDaemonNoSpaceLeftOnDeviceError
2023-12-07 20:02:29 +01:00
Paweł Gronowski
eaaf1ea96d integration/prune: Run in a separate daemon
Isolate the prune effects by running the test in a separate daemon.
This minimizes the impact of/on other integration tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-07 17:33:26 +01:00
Sebastiaan van Stijn
abf9d0b07f Merge pull request #46907 from vvoland/c8d-fix-TestBuildClearCmd
c8d/integration-cli: Adjust TestBuildClearCmd
2023-12-07 17:29:37 +01:00
Sebastiaan van Stijn
3b4207896d Merge pull request #46752 from corhere/libn/diagnostics-handler-ctx
libnetwork/diagnostic: clean up Server type
2023-12-07 15:55:41 +01:00
Paweł Gronowski
90dfb1c8ad c8d/integration-cli: Adjust TestBuildClearCmd
Config serialization performed by the graphdriver implementation
maintained the distinction between an empty array and having no Cmd set.

With containerd integration we serialize the OCI types directly that use
the `omitempty` option which doesn't persist that distinction.

Considering that both values should have exactly the same semantics (no
cmd being passed) it should be fine if in this case the Cmd would be
null instead of an empty array.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-07 14:17:35 +01:00
Sebastiaan van Stijn
82a318db5f Merge pull request #46894 from vvoland/hack-load-specialimages
hack: Load special images on demand
2023-12-07 11:28:38 +01:00
Rob Murray
0f9f9a132e Move 'netip' utils from 'ipam' to 'internal'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-06 17:13:40 +00:00
Cory Snider
1931a1bdc7 libnetwork/diagnostic: lock mutex in help handler
Acquire the mutex in the help handler to synchronize access to the
handlers map. While a trivial issue---a panic in the request handler if
the node joins a swarm at just the right time, which would only result
in an HTTP 500 response---it is also a trivial race condition to fix.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-06 11:20:47 -05:00
Cory Snider
424ae36046 libnetwork/diagnostic: use standard http.Handler
We don't need C-style callback functions which accept a void* context
parameter: Go has closures. Drop the unnecessary httpHandlerCustom type
and refactor the diagnostic server handler functions into closures which
capture whatever context they need implicitly.

If the node leaves and rejoins a swarm, the cluster agent and its
associated NetworkDB are discarded and replaced with new instances. Upon
rejoin, the agent registers its NetworkDB instance with the diagnostic
server. These handlers would all conflict with the handlers registered
by the previous NetworkDB instance. Attempting to register a second
handler on a http.ServeMux with the same pattern will panic, which the
diagnostic server would historically deal with by ignoring the duplicate
handler registration. Consequently, the first NetworkDB instance to be
registered would "stick" to the diagnostic server for the lifetime of
the process, even after it is replaced with another instance. Improve
duplicate-handler registration such that the most recently-registered
handler for a pattern is used for all subsequent requests.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-06 11:19:59 -05:00
Paweł Gronowski
bc94dfc7d2 hack: Load special images on demand
Rewrite `.build-empty-images` shell script that produced special images
(emptyfs with no layers, and empty danglign image) to a Go functions
that construct the same archives in a temporary directory.

Use them to load these images on demand only in the tests that need
them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-06 17:16:37 +01:00
Paweł Gronowski
68eead6628 Merge pull request #46901 from thaJeztah/move_more_backend_types
api/types: move Plugin-types and NetworkListConfig to api/types/backend
2023-12-06 14:51:49 +01:00
Sebastiaan van Stijn
debcb76939 Merge pull request #46895 from vvoland/integration-build-cleanenv
integration/build: Use setupTest
2023-12-06 02:26:43 +01:00
Sebastiaan van Stijn
7cb1efebec api/types: move NetworkListConfig to api/types/backend
This struct is intended for internal use only for the backend, and is
not intended to be used externally.

This moves the plugin-related `NetworkListConfig` types to the backend
package to prevent it being imported in the client, and to make it more
clear that this is part of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-06 02:21:21 +01:00
Sebastiaan van Stijn
a58b0a3d9c api/types: move Plugin-types to api/types/backend
These structs are intended for internal use only for the backend, and are
not intended to be used externally.

This moves the plugin-related `PluginRmConfig`, `PluginEnableConfig`, and
`PluginDisableConfig` types to the backend package to prevent them being
imported in the client, and to make it more clear that this is part of
internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-06 02:16:02 +01:00
Sebastiaan van Stijn
4046ae5e2f Merge pull request #46787 from thaJeztah/update_golang_1.21.4
update to go1.21.5
2023-12-06 01:03:19 +01:00
Sebastiaan van Stijn
6af9204938 Merge pull request #46887 from thaJeztah/minimum_api_version
daemon: raise default minimum API version to v1.24
2023-12-06 00:15:03 +01:00
Sebastiaan van Stijn
862caf826c update to go1.21.5
go1.21.5 (released 2023-12-05) includes security fixes to the go command,
and the net/http and path/filepath packages, as well as bug fixes to the
compiler, the go command, the runtime, and the crypto/rand, net, os, and
syscall packages. See the Go 1.21.5 milestone on our issue tracker for
details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.4...go1.21.5

from the security mailing:

[security] Go 1.21.5 and Go 1.20.12 are released

Hello gophers,

We have just released Go versions 1.21.5 and 1.20.12, minor point releases.

These minor releases include 3 security fixes following the security policy:

- net/http: limit chunked data overhead

  A malicious HTTP sender can use chunk extensions to cause a receiver
  reading from a request or response body to read many more bytes from
  the network than are in the body.

  A malicious HTTP client can further exploit this to cause a server to
  automatically read a large amount of data (up to about 1GiB) when a
  handler fails to read the entire body of a request.

  Chunk extensions are a little-used HTTP feature which permit including
  additional metadata in a request or response body sent using the chunked
  encoding. The net/http chunked encoding reader discards this metadata.
  A sender can exploit this by inserting a large metadata segment with
  each byte transferred. The chunk reader now produces an error if the
  ratio of real body to encoded bytes grows too small.

  Thanks to Bartek Nowotarski for reporting this issue.

  This is CVE-2023-39326 and Go issue https://go.dev/issue/64433.

- cmd/go: go get may unexpectedly fallback to insecure git

  Using go get to fetch a module with the ".git" suffix may unexpectedly
  fallback to the insecure "git://" protocol if the module is unavailable
  via the secure "https://" and "git+ssh://" protocols, even if GOINSECURE
  is not set for said module. This only affects users who are not using
  the module proxy and are fetching modules directly (i.e. GOPROXY=off).

  Thanks to David Leadbeater for reporting this issue.

  This is CVE-2023-45285 and Go issue https://go.dev/issue/63845.

- path/filepath: retain trailing \ when cleaning paths like \\?\c:\

  Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the
  volume name in Windows paths starting with \\?\, resulting in
  filepath.Clean(\\?\c:\) returning \\?\c: rather than \\?\c:\ (among
  other effects). The previous behavior has been restored.

  This is an update to CVE-2023-45283 and Go issue https://go.dev/issue/64028.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:27:58 +01:00
Sebastiaan van Stijn
0bf6ffba43 update to go1.21.4
go1.21.4 (released 2023-11-07) includes security fixes to the path/filepath
package, as well as bug fixes to the linker, the runtime, the compiler, and
the go/types, net/http, and runtime/cgo packages. See the Go 1.21.4 milestone
on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.4+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.3...go1.21.4

from the security mailing:

[security] Go 1.21.4 and Go 1.20.11 are released

Hello gophers,

We have just released Go versions 1.21.4 and 1.20.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- path/filepath: recognize `\??\` as a Root Local Device path prefix.

  On Windows, a path beginning with `\??\` is a Root Local Device path equivalent
  to a path beginning with `\\?\`. Paths with a `\??\` prefix may be used to
  access arbitrary locations on the system. For example, the path `\??\c:\x`
  is equivalent to the more common path c:\x.

  The filepath package did not recognize paths with a `\??\` prefix as special.

  Clean could convert a rooted path such as `\a\..\??\b` into
  the root local device path `\??\b`. It will now convert this
  path into `.\??\b`.

  `IsAbs` did not report paths beginning with `\??\` as absolute.
  It now does so.

  VolumeName now reports the `\??\` prefix as a volume name.

  `Join(`\`, `??`, `b`)` could convert a seemingly innocent
  sequence of path elements into the root local device path
  `\??\b`. It will now convert this to `\.\??\b`.

  This is CVE-2023-45283 and https://go.dev/issue/63713.

- path/filepath: recognize device names with trailing spaces and superscripts

  The `IsLocal` function did not correctly detect reserved names in some cases:

  - reserved names followed by spaces, such as "COM1 ".
  - "COM" or "LPT" followed by a superscript 1, 2, or 3.

  `IsLocal` now correctly reports these names as non-local.

  This is CVE-2023-45284 and https://go.dev/issue/63713.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:27:08 +01:00
Sebastiaan van Stijn
08e4e88482 daemon: raise default minimum API version to v1.24
The daemon currently provides support for API versions all the way back
to v1.12, which is the version of the API that shipped with docker 1.0. On
Windows, the minimum supported version is v1.24.

Such old versions of the client are rare, and supporting older API versions
has accumulated significant amounts of code to remain backward-compatible
(which is largely untested, and a "best-effort" at most).

This patch updates the minimum API version to v1.24, which is the fallback
API version used when API-version negotiation fails. The intent is to start
deprecating older API versions, but no code is removed yet as part of this
patch, and a DOCKER_MIN_API_VERSION environment variable is added, which
allows overriding the minimum version (to allow restoring the behavior from
before this patch).

With this patch the daemon defaults to API v1.24 as minimum:

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.24)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

Trying to use an older version of the API produces an error:

    DOCKER_API_VERSION=1.23 docker version
    Client:
     Version:           24.0.2
     API version:       1.23 (downgraded from 1.43)
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default
    Error response from daemon: client version 1.23 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

To restore the previous minimum, users can start the daemon with the
DOCKER_MIN_API_VERSION environment variable set:

    DOCKER_MIN_API_VERSION=1.12 dockerd

API 1.12 is the oldest supported API version on Linux;

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.12)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

When using the `DOCKER_MIN_API_VERSION` with a version of the API that
is not supported, an error is produced when starting the daemon;

    DOCKER_MIN_API_VERSION=1.11 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: 1.11

    DOCKER_MIN_API_VERSION=1.45 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: maximum supported API version is 1.44: 1.45

Specifying a malformed API version also produces the same error;

    DOCKER_MIN_API_VERSION=hello dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: hello

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:11:02 +01:00
Sebastiaan van Stijn
4fa5a79833 Merge pull request #46898 from thaJeztah/backend_types
api/types: move ContainerCreateConfig, ContainerRmConfig to api/types/backend
2023-12-05 19:44:22 +01:00
Sebastiaan van Stijn
484e6b784c api/types: move ContainerCreateConfig, ContainerRmConfig to api/types/backend
The `ContainerCreateConfig` and `ContainerRmConfig` structs are used for
options to be passed to the backend, and are not used in client code.

Thess struct currently is intended for internal use only (for example, the
`AdjustCPUShares` is an internal implementation details to adjust the container's
config when older API versions are used).

Somewhat ironically, the signature of the Backend has a nicer UX than that
of the client's `ContainerCreate` signature (which expects all options to
be passed as separate arguments), so we may want to update that signature
to be closer to what the backend is using, but that can be left as a future
exercise.

This patch moves the `ContainerCreateConfig` and `ContainerRmConfig` structs
to the backend package to prevent it being imported in the client, and to make
it more clear that this is part of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 16:41:36 +01:00
Sebastiaan van Stijn
6a810b4259 Merge pull request #46897 from thaJeztah/bump_criu_debian
Dockerfile: update criu stage to Debian 12 (Bookworm)
2023-12-05 15:40:30 +01:00
Sebastiaan van Stijn
c786c3a974 Dockerfile: update criu stage to Debian 12 (Bookworm)
Follow-up to e72c4818c4, which updated the
Dockerfile to use Debian 12 "bookworm", but forgot to update the package
repository to use for the CRIU packages. Note that the criu stage is currently
not built by default (see d3d2823edf), so to
verify the stage, it needs to be built manually;

    docker build --target=criu .

This patch adds an extra `criu --version` to the build, so that it's verified
to be "functional".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 14:09:57 +01:00
Paweł Gronowski
d5eba1bfe5 integration/build: Use setupTest
These tests build new images, setupTest sets up the test cleanup
function that clears the test environment from created images,
containers, etc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-05 12:49:30 +01:00
Sebastiaan van Stijn
2a38569337 Merge pull request #46891 from thaJeztah/cleanup_api_versions
integration / integration-cil: cleanup and fix API versions used in tests
2023-12-05 10:41:56 +01:00
Sebastiaan van Stijn
713c7d49a1 integration(-cli): remove skips for old daemon versions (<20.10)
This removes various skips that accounted for running the integration tests
against older versions of the daemon before 20.10 (API version v1.41). Those
versions are EOL, and we don't run tests against them.

This reverts most of e440831802, and similar
PRs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 01:03:50 +01:00
Sebastiaan van Stijn
7f68e3107e integration-cli: TestAPIClientVersionOldNotSupported: use daemon API version
Use the minimum API version as advertised by the test-daemon, instead of the
hard-coded API version from code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:58:47 +01:00
Sebastiaan van Stijn
97549d923c integration/container: TestInfoAPIVersioned: adjust API version
This test was using API version 1.20 to test old behavior, but the actual change
in behavior was API v1.25; see commit 6d98e344c7
and 63b5a37203.
This updates the test to use API v1.24 to test the old behavior.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:55 +01:00
Sebastiaan van Stijn
410feab962 integration/container: TestKillDifferentUserContainer: adjust API version
TestKillDifferentUserContainer was migrated from integration-cli in
commit 0855922cd3. Before migration, it
was not using a specific API version, so we can assume "current"
API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:52 +01:00
Cory Snider
757a004a90 libnetwork/diagnostic: drop Init method
Fold it into the constructor, because that's what the constructor is
supposed to do.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-04 15:13:17 -05:00
Cory Snider
f270057e0c libnetwork/diagnostic: un-embed sync.Mutex field
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-04 15:13:17 -05:00
Sebastiaan van Stijn
7dc5f9d0b1 Merge pull request #46884 from thaJeztah/remove_apt_mirror
Dockerfile: remove APT_MIRROR build-arg
2023-12-04 21:02:04 +01:00
Sebastiaan van Stijn
aa2fa2caa3 un-containerize TestDaemonNoSpaceLeftOnDeviceError
Commit 59b83d8aae containerized these steps,
as they didn't work well on Debian Jessie:

> Because the `mount` here will sometimes fail when run in `debian:jessie`,
> which is what the environrment hosting the test suite is running if run
> from the `Makefile`.
> Also, why the heck not containerize it, all the things.

Follow-up commits, such as 228d74842f, and
1c5806cf57 updated the Debian distro, but
also updated this comment, losing the original context (the issue was
(originally) related to Debian Jessie).

This patch changes the test back to not use containers, which seems to
work fine (at least "it worked on my machine").

    make TEST_IGNORE_CGROUP_CHECK=1 TEST_FILTER=TestDaemonNoSpaceLeftOnDeviceError DOCKER_GRAPHDRIVER=overlay2 test-integration

    === RUN   TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError
        check_test.go:589: [df36ad96a412b] daemon is not started
    --- PASS: TestDockerDaemonSuite (5.12s)
        --- PASS: TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError (5.12s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 18:51:15 +01:00
Sebastiaan van Stijn
49f3e7d233 Dockerfile: remove APT_MIRROR build-arg
This option was originally added in 8ec8564691,
at which time the upstream debian package repositories were not always
reliable, so using a mirror helped with CI stability and performance.

Debian's package repositories are a lot more reliable now, so there's no
longer a need to use a mirror.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 13:52:58 +01:00
Sebastiaan van Stijn
df59a357ec Merge pull request #46877 from robmry/early_error_for_cifs_url_with_port
Don't allow port in CIFS URL
2023-12-04 12:44:02 +01:00
Sebastiaan van Stijn
bcc0315003 Merge pull request #46882 from vvoland/vendor-buildkit-0.12.4
vendor: github.com/moby/buildkit v0.12.4
2023-12-04 12:42:05 +01:00
Rob Murray
baca6cb417 Don't allow port in CIFS URL
When creating a CIFS volume, generate an error if the device URL
includes a port number, for example:
   --opt device="//some.server.com:2345/thepath"

The port must be specified in the port option instead, for example:
    --opt o=username=USERNAME,password=PASSWORD,vers=3,sec=ntlmsspi,port=1234

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-04 10:08:09 +00:00
Paweł Gronowski
c8134fa046 vendor: github.com/moby/buildkit v0.12.4
full diff: https://github.com/moby/buildkit/compare/v0.12.3...v0.12.4

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-04 10:56:20 +01:00
Martin Jirku
a189832454 add TaskStatus to swagger docs
Add the TaskStatus, PortStatus and ContainerStatus to api docs. TaskStatus was moved to the swagger definitions root from anonymous type definition, and PortStatus and Container Status are its dependencies.

Signed-off-by: Martin Jirku <martin@jirku.sk>
2023-12-03 16:12:32 +01:00
Sebastiaan van Stijn
029519a149 Merge pull request #46876 from thaJeztah/bump_mux
vendor: github.com/gorilla/mux v1.8.1
2023-12-01 16:26:10 +01:00
Sebastiaan van Stijn
faa843fcef vendor: github.com/gorilla/mux v1.8.1
full diff: https://github.com/gorilla/mux/compare/v1.8.0...v1.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-01 12:48:09 +01:00
Sebastiaan van Stijn
f179243876 Merge pull request #46875 from thaJeztah/bump_cobra
vendor: github.com/spf13/cobra v1.8.0
2023-12-01 12:09:44 +01:00
Sebastiaan van Stijn
028ed1bb22 vendor: github.com/spf13/cobra v1.8.0
full diff: https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-01 11:01:59 +01:00
Sebastiaan van Stijn
bb99b3f02e Merge pull request #46871 from thaJeztah/less_cobra_customisation
cmd/dockerd: remove various cobra templating
2023-12-01 10:58:07 +01:00
Sebastiaan van Stijn
d3533eee68 Merge pull request #46849 from robmry/37937_explicit_bridge_mtu
Explicitly set MTU on bridge devices.
2023-11-30 22:30:11 +01:00
Tianon Gravi
488fe76eb8 Merge pull request #45789 from thaJeztah/update_debian
Dockerfile: update to Debian "bookworm", and update frozen images to debian:bookworm-slim
2023-11-30 12:33:48 -08:00
Sebastiaan van Stijn
75546e1e46 Merge pull request #46865 from vvoland/c8d-fix-DockerRegistryAuthTokenSuite
c8d/integration-cli: Adjust DockerRegistryAuthTokenSuite
2023-11-30 17:36:45 +01:00
Sebastiaan van Stijn
ec7c2b784a Merge pull request #46268 from thaJeztah/detachcontext
container: internalize InitAttachContext
2023-11-30 17:03:34 +01:00
Sebastiaan van Stijn
08035dc805 Merge pull request #46868 from vvoland/c8d-skip-TestListDanglingImagesWithDigests
c8d/integration-cli: Skip part of TestListDanglingImagesWithDigests
2023-11-30 16:16:02 +01:00
Sebastiaan van Stijn
e18f5a5304 container: internalize InitAttachContext
Move the initialization logic to the attachContext itself, so that
the container doesn't have to be aware about mutexes and other logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 15:26:53 +01:00
Sebastiaan van Stijn
c8b9dfb25e Merge pull request #46867 from thaJeztah/log_info
daemon: kill: use log level to "warn" if container doesn't exit in time
2023-11-30 14:41:32 +01:00
Paweł Gronowski
fcb89da9c2 c8d/integration-cli: Skip TestListDanglingImagesWithDigests
Skip TestListDanglingImagesWithDigests which tests graphdriver
implementation specific behavior of `docker images --filter
dangling=true`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-30 14:01:22 +01:00
Sebastiaan van Stijn
61d94850a7 cmd/dockerd: make cobra templates a const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 13:40:15 +01:00
Sebastiaan van Stijn
0f163170a7 cmd/dockerd: remove various cobra templating
This removes various templating functions that were added for the
docker CLI. These are not needed for the dockerd binary, which does
not have subcommands or management commands.

Revert "Only hide commands if the env variable is set."

This reverts commit a7c8bcac2b.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 13:40:10 +01:00
Sebastiaan van Stijn
faecc65cd3 Merge pull request #46863 from michaelkebe/fix/cifs-volume-resolve-hostname
cifs volume resolves hostname correctly
2023-11-30 12:27:46 +01:00
Paweł Gronowski
ae6960cf91 Merge pull request #46857 from vvoland/liverestore-fix-46308
liverestore: Don't remove `--rm` containers on restart
2023-11-30 12:08:13 +01:00
Sebastiaan van Stijn
3bfb6a9420 frozen images: update to debian:bookworm-slim
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 12:01:06 +01:00
Sebastiaan van Stijn
fd40dfaf58 Dockerfile: remove uses of DEBIAN_FRONTEND
We used DEBIAN_FRONTEND in some places to prevent installation of packages
from being blocked. However, debian bookworm now [includes a fix][1] for
situations like this (it was specifically reported for Docker situations <3),
so we can get rid of these.

Thanks to Tianon for noticing this, and for linking to the Debian ticket!

[1]: https://bugs.debian.org/929417

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:51:18 +01:00
Sebastiaan van Stijn
e72c4818c4 Dockerfile: update to Debian "bookworm" (current stable)
Also switch yamllint to be installed from debian's packages, which are
currently at v1.29.0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:50:34 +01:00
Michael Kebe
8ae94cafa5 cifs volume resolves hostname correctly.
Co-authored-by: Paweł Gronowski <me@woland.xyz>
Signed-off-by: Michael Kebe <michael.kebe@gmail.com>
2023-11-30 11:32:04 +01:00
Paweł Gronowski
3a0af5ad30 integration/TestLiveRestore: Wait for process to exit
Replace `time.Sleep` with a poll that checks if process no longer exists
to avoid possible race condition.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-30 10:15:58 +01:00
Brian Goff
718fafed26 Merge pull request #46859 from thaJeztah/fix_TestDaemonICC_tests
integration-cli: fix TestDaemonICC tests for newer iptables versions
2023-11-29 07:22:15 -08:00
Sebastiaan van Stijn
61d8f57f2a daemon: kill: use log level to "warn" if container doesn't exit in time
I noticed this log being logged as an error, but the kill logic actually
proceeds after this (doing a "direct" kill instead). While usually containers
are expected to be exiting within the given timeout, I don't think this
needs to be logged as an error (an error is returned after we fail to
kill the container).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-29 16:18:34 +01:00
Paweł Gronowski
e262cd38ad c8d/integration-cli: Adjust DockerRegistryAuthTokenSuite
The auth service error response is not a part of the spec and containerd
doesn't parse it like the Docker's distribution does.

Check for containerd specific errors instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-29 14:28:18 +01:00
Sebastiaan van Stijn
5dde37c846 Merge pull request #46861 from tonistiigi/gc-time-filter
builder-next: fix timing filter for default policy
2023-11-29 12:20:46 +01:00
Tonis Tiigi
49d088d9ce builder-next: fix timing filter for default policy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-11-28 22:37:30 -08:00
Sebastiaan van Stijn
c3eed9fa3e integration-cli: fix TestDaemonICC tests for newer iptables versions
Debian Woodworm ships with a newer version of iptables, which caused two
tests to fail:

    === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)
    docker_cli_daemon_test.go:841: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge6.*ext-bridge6", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge6 !ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge6 ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n"
    --- FAIL: TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)

    === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCPing (1.19s)
    docker_cli_daemon_test.go:803: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge5.*ext-bridge5", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge5 !ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge5 ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n"
    --- FAIL: TestDockerDaemonSuite/TestDaemonICCPing (1.19s)

Both the `TestDaemonICCPing`, and `TestDaemonICCLinkExpose` test were introduced
in dd0666e64f. These tests called `iptables` with
the `-n` (`--numeric`) option, which prevents it from doing a reverse-DNS lookup
as an optimization.

However, the `-n` option did not have an effect to the `prot` column before
commit [da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa] (iptables < v1.8.9 or v1.8.8).
Newer versions, such as the iptables version shipping with Debian Woodworm do,
so we need to update the expected output for this version.

This patch removes the `-n` option, to keep the test more portable, also when
run non-containerized, and removes the use of regular expressions to check the
result, as these regular expressions were quite permissive (using `.*` wild-
card matching). Instead, we're getting the

With this change;

make DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestDaemonICC TEST_IGNORE_CGROUP_CHECK=1 test-integration
...
--- PASS: TestDockerDaemonSuite (139.11s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCLinkExpose (54.62s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCPing (84.48s)

[da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa]: https://git.netfilter.org/iptables/commit/?id=da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-28 18:58:03 +01:00
Paweł Gronowski
c5ea3d595c liverestore: Don't remove --rm containers on restart
When live-restore is enabled, containers with autoremove enabled
shouldn't be forcibly killed when engine restarts.
They still should be removed if they exited while the engine was down
though.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-28 12:59:38 +01:00
Sebastiaan van Stijn
f6533a1df1 Merge pull request #46852 from thaJeztah/fix_systemdind_apparmor
hack/dind-systemd: make AppArmor work with systemd enabled
2023-11-27 22:14:13 +01:00
Sebastiaan van Stijn
2249db0c73 Merge pull request #46848 from vvoland/c8d-skip-TestSaveCheckTimes
c8d/integration: Adjust TestSaveCheckTimes
2023-11-27 19:54:45 +01:00
Sebastiaan van Stijn
bcf52efbb6 Merge pull request #46855 from vvoland/c8d-fix-TestCrossRepositoryLayerPush
integration-cli/TestCrossRepositoryLayerPush: Change repo name
2023-11-27 19:46:27 +01:00
Paweł Gronowski
9b399814e7 integration-cli/TestCrossRepositoryLayerPush: Change repo name
Change the repo name used as for an intermediate image so it doesn't
try to mount from the image pushed by `TestBuildMultiStageImplicitPull`.

Before this patch, this test failed because the distribution.source
labels are not cleared between tests and the busybox content still has
the distribution.source label pointing to the `dockercli/testf`
repository which is no longer present in the test registry.
So both `dockercli/busybox` and `dockercli/testf` are equally valid
mount candidates for `dockercli/crossrepopush` and containerd algorithm
just happens to select the last one.

This changes the repo name to not have the common repository component
(`dockercli`) with the `dockercli/testf` repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-27 18:12:50 +01:00
Sebastiaan van Stijn
2ef69899de Merge pull request #46854 from thaJeztah/quota_update_size
quota: increase sparse test-image to 300MB
2023-11-27 17:12:31 +01:00
Sebastiaan van Stijn
9709b7e458 quota: increase sparse test-image to 300MB
Starting with [6e0ed3d19c54603f0f7d628ea04b550151d8a262], the minimum
allowed size is now 300MB. Given that this is a sparse image, and
the size of the image is irrelevant to the test (we check for
limits defined through project-quotas, not the size of the
device itself), we can raise the size of this image.

[6e0ed3d19c54603f0f7d628ea04b550151d8a262]: https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=6e0ed3d19c54603f0f7d628ea04b550151d8a262

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-27 16:08:08 +01:00
Sebastiaan van Stijn
65cfcc28ab hack/dind: update comments around AppArmor
Provide more context to the steps we're doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-27 14:48:51 +01:00
Sebastiaan van Stijn
cfb8ca520a hack/dind-systemd: make AppArmor work with systemd enabled
On bookworm, AppArmor failed to start inside the container, which can be
seen at startup of the dev-container:

    Created symlink /etc/systemd/system/systemd-firstboot.service → /dev/null.
    Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
    Created symlink /etc/systemd/system/multi-user.target.wants/docker-entrypoint.service → /etc/systemd/system/docker-entrypoint.service.
    hack/dind-systemd: starting /lib/systemd/systemd --show-status=false --unit=docker-entrypoint.target
    systemd 252.17-1~deb12u1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
    Detected virtualization docker.
    Detected architecture x86-64.
    modprobe@configfs.service: Deactivated successfully.
    modprobe@dm_mod.service: Deactivated successfully.
    modprobe@drm.service: Deactivated successfully.
    modprobe@efi_pstore.service: Deactivated successfully.
    modprobe@fuse.service: Deactivated successfully.
    modprobe@loop.service: Deactivated successfully.
    apparmor.service: Starting requested but asserts failed.
    proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 49 (systemd-binfmt)
    + source /etc/docker-entrypoint-cmd
    ++ hack/make.sh dynbinary test-integration

When checking "aa-status", an error was printed that the filesystem was
not mounted:

    aa-status
    apparmor filesystem is not mounted.
    apparmor module is loaded.

Checking if "local-fs.target" was loaded, that seemed to be the case;

    systemctl status local-fs.target
    ● local-fs.target - Local File Systems
         Loaded: loaded (/lib/systemd/system/local-fs.target; static)
         Active: active since Mon 2023-11-27 10:48:38 UTC; 18s ago
           Docs: man:systemd.special(7)

However, **on the host**, "/sys/kernel/security" has a mount, which was not
present inside the container:

    mount | grep securityfs
    securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)

Interestingly, on `debian:bullseye`, this was not the case either; no
`securityfs` mount was present inside the container, and apparmor actually
failed to start, but succeeded silently:

    mount | grep securityfs
    systemctl start apparmor
    systemctl status apparmor
    ● apparmor.service - Load AppArmor profiles
         Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
         Active: active (exited) since Mon 2023-11-27 11:59:09 UTC; 44s ago
           Docs: man:apparmor(7)
                 https://gitlab.com/apparmor/apparmor/wikis/home/
        Process: 43 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=0/SUCCESS)
       Main PID: 43 (code=exited, status=0/SUCCESS)
            CPU: 10ms

    Nov 27 11:59:09 9519f89cade1 apparmor.systemd[43]: Not starting AppArmor in container

Same, using the `/etc/init.d/apparmor` script:

    /etc/init.d/apparmor start
    Starting apparmor (via systemctl): apparmor.service.
    echo $?
    0

And apparmor was not actually active:

    aa-status
    apparmor module is loaded.
    apparmor filesystem is not mounted.

    aa-enabled
    Maybe - policy interface not available.

After further investigating, I found that the non-systemd dind script
had a mount for AppArmor, which was added in 31638ab2ad

The systemd variant was missing this mount, which may have gone unnoticed
because `debian:bullseye` was silently ignoring this when starting the
apparmor service.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-27 14:47:59 +01:00
Paweł Gronowski
4cd2654a9d Merge pull request #46770 from vvoland/c8d-unmount-empty-basefs
daemon/c8d: Unmount container fs after unclean shutdown
2023-11-27 13:52:00 +01:00
Paweł Gronowski
203bac0ec4 daemon/c8d: Unmount container fs after unclean shutdown
BaseFS is not serialized and is lost after an unclean shutdown. Unmount
method in the containerd image service implementation will not work
correctly in that case.
This patch will allow Unmount to restore the BaseFS if the target is
still mounted.

The reason it works with graphdrivers is that it doesn't directly
operate on BaseFS. It uses RWLayer, which is explicitly restored
immediately as soon as container is loaded.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-27 12:33:33 +01:00
Rob Murray
964ab7158c Explicitly set MTU on bridge devices.
This is purely cosmetic - if a non-default MTU is configured, the bridge
will have the default MTU=1500 until a container's 'veth' is connected
and an MTU is set on the veth. That's a disconcerting, it looks like the
config has been ignored - so, set the bridge's MTU explicitly.

Fixes #37937

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-11-27 11:18:54 +00:00
Paweł Gronowski
05523e289b c8d/integration: Adjust TestSaveCheckTimes
The graphdriver implementation sets the ModTime of all image content to
match the `Created` time from the image config, whereas the containerd's
archive export code just leaves it empty (zero).

Adjust the test in the case where containerd integration is enabled to
check if config file ModTime is equal to zero (UNIX epoch) instead.

This behaviour is not a part of the Docker Image Specification and the
intention behind introducing it was to make the `docker save` produce
the same archive regardless of the time it was performed.

It would also be a bit problematic with the OCI archive layout which can
contain multiple images referencing the same content.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-24 14:49:02 +01:00
Sebastiaan van Stijn
ce1ee98aba Merge pull request #46447 from akerouanton/api-predefined-networks
api: Add consts for predefined networks
2023-11-24 12:26:48 +01:00
Albin Kerouanton
5ce8eee0a4 Merge pull request #46846 from akerouanton/refactor/container-rename-move-log-args
daemon: ContainerRename: move log args to log fields
2023-11-24 11:56:38 +01:00
Albin Kerouanton
b1676a289c daemon: ContainerRename: move log args to log fields
Also, err `e` is renamed into the more standard `err` as the defer
already uses `retErr` to avoid clashes (changed in f5a611a74).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-24 11:05:02 +01:00
Sebastiaan van Stijn
cfdca8dc1d Merge pull request #46844 from akerouanton/fix/windows-adapter-dns-param
daemon: windows: set DNS config on all adapters
2023-11-23 22:41:19 +01:00
Albin Kerouanton
d2865f1e8a daemon: win: set DNS config on all adapters
DNS config is a property of each adapter on Windows, thus we've a
dedicated `EndpointOption` for that.

The list of `EndpointOption` that should be applied to a given endpoint
is built by `buildCreateEndpointOptions`. This function contains a
seemingly flawed condition that adds the DNS config _iff_:

1. the network isn't internal ;
2. no ports are published / exposed through another sandbox endpoint ;

While 1. does make sense, there's actually no justification for 2.,
hence this commit remove this part of the condition.

This logic flaw has been made obvious by 0fd0e82, but it was originally
introduced by d1e0a78. Commit and PR comments don't mention why this is
done like so. Most probably, this was overlooked both by the original
author and the PR reviewers.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-23 18:40:58 +01:00
Albin Kerouanton
35eba19a65 Merge pull request #46843 from akerouanton/fix/bad-mac-address
daemon: build ports-related ep options in a dedicated func
2023-11-23 18:39:17 +01:00
Sebastiaan van Stijn
2f65748927 Merge pull request #46790 from corhere/libn/overlay-ipv6-vtep
libnetwork/drivers/overlay: support IPv6 transport
2023-11-23 18:23:27 +01:00
Albin Kerouanton
0fd0e8255f daemon: build ports-related ep options in a dedicated func
The `buildCreateEndpointOptions` does a lot of things to build the list
of `libnetwork.EndpointOption` from the `EndpointSettings` spec. To skip
ports-related options, an early return was put in the middle of that
function body.

Early returns are generally great, but put in the middle of a 150-loc
long function that does a lot, they're just a potential footgun. And I'm
the one who pulled the trigger in 052562f. Since this commit, generic
options won't be applied to endpoints if there's already one with
exposed/published ports. As a consequence, only the first endpoint can
have a user-defined MAC address right now.

Instead of moving up the code line that adds generic options, a better
change IMO is to move ports-related options, and the early-return gating
those options, to a dedicated func to make `buildCreateEndpointOptions`
slightly easier to read and reason about.

There was actually one oddity in the original
`buildCreateEndpointOptions`: the early-return also gates the addition
of `CreateOptionDNS`. These options are Windows-specific; a comment is
added to explain that. But the oddity is really: why are we checking if
an endpoint with exposed / published ports joined this sandbox to decide
whether we want to configure DNS server on the endpoint's adapter? Well,
this early-return was most probably overlooked by the original author
and by reviewers at the time these options were added (in commit d1e0a78)

Let's fix that in a follow-up commit.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-23 16:26:01 +01:00
Sebastiaan van Stijn
54fcd40aa4 Merge pull request #46227 from thaJeztah/supervisor_ignore_errs
libcontainerd/supervisor: explicitly ignore process kill errors
2023-11-22 08:40:45 +01:00
Akihiro Suda
b93a532bcd Merge pull request #46368 from thaJeztah/docker_py_latest
testing: update docker-py to 7.0.0b1
2023-11-22 10:40:35 +09:00
Sebastiaan van Stijn
7786f8512b Revert "testing: temporarily pin docker-py tests to use "bullseye""
This reverts commit 19d860fa9d.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-21 23:53:45 +01:00
Sebastiaan van Stijn
4394c61e6c testing: update docker-py to 7.0.0b1
https://github.com/docker/docker-py/compare/6.1.3...7.0.0b1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-21 23:53:13 +01:00
Albin Kerouanton
8e84bc3931 Merge pull request #46481 from akerouanton/fix-deprecation-message-servicespec-networks
api/t/swarm: Fix deprecation for ServiceSpec.Networks
2023-11-21 14:24:15 +01:00
Albin Kerouanton
f877360dc1 api/t/swarm: Fix deprecation for ServiceSpec.Networks
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-21 10:54:53 +01:00
Sebastiaan van Stijn
f678459593 Merge pull request #46835 from vvoland/dockerfile-buildx-update-0.12
Dockerfile: update buildx to v0.12.0
2023-11-20 21:14:46 +01:00
Paweł Gronowski
681c94ca17 Dockerfile: update buildx to v0.12.0
Update the version of buildx we use in the dev-container to v0.12.0

Release notes: https://github.com/docker/buildx/releases/tag/v0.12.0

Full diff: https://github.com/docker/buildx/compare/v0.11.1...v0.12.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-20 16:03:41 +01:00
Sebastiaan van Stijn
ab099ab076 Merge pull request #46833 from thaJeztah/bump_hcsshim
vendor: github.com/Microsoft/hcsshim v0.11.4
2023-11-20 14:06:02 +01:00
Paweł Gronowski
d154421092 Merge pull request #46444 from cpuguy83/docker_info_slow
Plumb context through info endpoint
2023-11-20 12:10:30 +01:00
Sebastiaan van Stijn
2d79f7653f Merge pull request #46831 from thaJeztah/update_containerd_binary_1.7.9
update containerd binary to v1.7.9
2023-11-20 11:24:02 +01:00
Sebastiaan van Stijn
0af1650158 Merge pull request #46154 from crazy-max/fix-dco-tag-events
ci(bin-image): skip dco on push tag events
2023-11-19 13:43:22 +01:00
Sebastiaan van Stijn
68e73ceb67 update containerd binary to v1.7.9
full diff: https://github.com/containerd/containerd/compare/v1.7.8...v1.7.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.9

Notable Updates

- update runc binary to v1.1.10
- vendor: upgrade OpenTelemetry to v1.19.0 / v0.45.0
- Expose usage of cri-api v1alpha2
- integration: deflake TestIssue9103
- fix: shimv1 leak issue
- cri: add deprecation warnings for mirrors, auths, and configs
- Update hcsshim tag to v0.11.4
- Expose usage of deprecated features

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-19 13:34:04 +01:00
Sebastiaan van Stijn
2f28582b55 vendor: github.com/Microsoft/hcsshim v0.11.4
full diff: https://github.com/Microsoft/hcsshim/compare/v0.11.1...v0.11.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-19 13:12:22 +01:00
CrazyMax
f4776ef9df ci(bin-image): skip dco on push tag events
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-11-19 12:53:16 +01:00
Sebastiaan van Stijn
75324c4cc8 Merge pull request #46814 from thaJeztah/update_authors
update authors and mailmap
2023-11-16 22:24:19 +01:00
Sebastiaan van Stijn
604f4eed65 Merge pull request #46819 from dmcgowan/c8d-fix-pull-by-digest
c8d: fix support for pull by digest
2023-11-16 20:00:07 +01:00
Sebastiaan van Stijn
847f3060d4 Merge pull request #46803 from thaJeztah/daemon_no_custom_opts
daemon/config: change DNSConfig.DNS to a []net.IP
2023-11-15 14:23:49 +01:00
Sebastiaan van Stijn
93fffa299c Merge pull request #46818 from thaJeztah/fix_readme_example
client: update example in readme
2023-11-14 15:22:19 +01:00
Sebastiaan van Stijn
e6ae462268 Merge pull request #46812 from robmry/46810-vfs_faster_copy_of_hard_links
graphdriver/copy: faster copy of hard links
2023-11-14 13:18:30 +01:00
Sebastiaan van Stijn
8641d2da3b client: update example in readme
The example still used the deprecated types.ContainerListOptions;
also slightly updated the example to show both stopped and running
containers, so that the example works even if no container is running.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-14 12:27:35 +01:00
Derek McGowan
0ab7267ae3 Fix support for pull by digest
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-11-13 21:38:51 -08:00
Sebastiaan van Stijn
ba46bda149 update authors and mailmap
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 22:12:58 +01:00
Sebastiaan van Stijn
f13d8c2026 Merge pull request #46724 from rhansen/host_ipv6
New `host_ipv6` bridge option to SNAT IPv6 connections
2023-11-13 21:50:17 +01:00
Rob Murray
a3fdad84ad graphdriver/copy: faster copy of hard links
The DirCopy() function in "graphdriver/copy/copy.go" has a special case for
skip file-attribute copying when making a hard link to an already-copied
file, if "copyMode == Hardlink". Do the same for copies of hard-links in
the source filesystem.

Significantly speeds up vfs's copy of a BusyBox filesystem (which
consists mainly of hard links to a single binary), making moby's
integration tests run more quickly and more reliably in a dev container.

Fixes #46810

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-11-13 18:22:03 +00:00
Sebastiaan van Stijn
6af7d6ea2d Merge pull request #46808 from thaJeztah/vendor_runc_1.1.10
vendor: github.com/opencontainers/runc v1.1.10
2023-11-13 17:24:35 +01:00
Sebastiaan van Stijn
c89668f8e5 Merge pull request #46805 from thaJeztah/bump_runc_binary_1.1.10
update runc binary to v1.1.10
2023-11-13 16:58:28 +01:00
Sebastiaan van Stijn
b275f95f37 Merge pull request #46684 from smoser/fix/no-colors-in-check-config-to-file
contrib/check-config: Only add color if output is a terminal.
2023-11-13 16:33:43 +01:00
Sebastiaan van Stijn
9b02a84a5a Merge pull request #46785 from rumpl/c8d-userns-commit
c8d: handle user namespace remapping on commit
2023-11-13 16:31:32 +01:00
Sebastiaan van Stijn
fb53da508f vendor: github.com/opencontainers/runc v1.1.10
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10

This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.

- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
  Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
  of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
  (fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
  configuration. This issue is not a security issue because it requires a
  malicious config.json, which is outside of our threat model.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 16:26:29 +01:00
Sebastiaan van Stijn
15bcc707e6 update runc binary to v1.1.10
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10

This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.

- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
  Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
  of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
  (fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
  configuration. This issue is not a security issue because it requires a
  malicious config.json, which is outside of our threat model.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 16:01:02 +01:00
Sebastiaan van Stijn
84036d3e18 daemon/config: change DNSConfig.DNS to a []net.IP
Use a strong type for the DNS IP-addresses so that we can use flags.IPSliceVar,
instead of implementing our own option-type and validation.

Behavior should be the same, although error-messages have slightly changed:

Before this patch:

    dockerd --dns 1.1.1.1oooo --validate
    Status: invalid argument "1.1.1.1oooo" for "--dns" flag: 1.1.1.1oooo is not an ip address
    See 'dockerd --help'., Code: 125

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1"]}

    dockerd --dns 2.2.2.2 --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: dns: (from flag: [2.2.2.2], from file: [1.1.1.1])

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1oooo"]}

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: 1.1.1.1ooooo is not an ip address

With this patch:

    dockerd --dns 1.1.1.1oooo --validate
    Status: invalid argument "1.1.1.1oooo" for "--dns" flag: invalid string being converted to IP address: 1.1.1.1oooo
    See 'dockerd --help'., Code: 125

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1"]}

    dockerd --dns 2.2.2.2 --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: dns: (from flag: [2.2.2.2], from file: [1.1.1.1])

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1oooo"]}

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid IP address: 1.1.1.1oooo

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 12:22:51 +01:00
Sebastiaan van Stijn
34e923e3e3 Merge pull request #46801 from thaJeztah/improve_ValidateIPAddress
opts: ValidateIPAddress: improve error, godoc, and tests
2023-11-13 12:22:38 +01:00
Sebastiaan van Stijn
1eadfb0e28 opts: ValidateIPAddress: improve error, godoc, and tests
- document accepted values
- add test-coverage for the function's behavior (including whitespace handling),
  and use sub-tests.
- improve error-message to use uppercase for "IP", and to use a common prefix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-11 15:31:19 +01:00
Sebastiaan van Stijn
3b423ea778 Merge pull request #46799 from thaJeztah/bump_go_connections
vendor: github.com/docker/go-connections fa09c952e3ea (v0.5.0-dev)
2023-11-11 00:49:56 +01:00
Sebastiaan van Stijn
981a878cf5 vendor: github.com/docker/go-connections fa09c952e3ea (v0.5.0-dev)
0b8c1f4e07...fa09c952e3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-10 23:23:37 +01:00
Brian Goff
677d41aa3b Plumb context through info endpoint
I was trying to find out why `docker info` was sometimes slow so
plumbing a context through to propagate trace data through.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-11-10 20:09:25 +00:00
Tianon Gravi
7c8d02d1bc Merge pull request #46797 from cpuguy83/fix_missing_scope
Fix missing import for "scope" package
2023-11-09 16:15:16 -08:00
Brian Goff
f0b89e63b9 Fix missing import for "scope" package
I believe this happened due to conflicting PR's that got merged without
CI re-running between them.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-11-09 22:48:01 +00:00
Brian Goff
63b2a2138d Merge pull request #41084 from thaJeztah/40495_update_goconnections_negotiate
Bump go-connections for TLS 1.3 support
2023-11-09 14:33:18 -08:00
Brian Goff
99638f5f76 Merge pull request #46713 from p-fruck/docs/api-network-connect-403 2023-11-09 12:48:59 -08:00
Sebastiaan van Stijn
774e677cc8 Merge pull request #46781 from kb2ma/doc-swagger-port
Update API README to include port in URL for locally built Swagger docs
2023-11-09 20:34:22 +01:00
Brian Goff
524eef5d75 Merge pull request #46681 from corhere/libn/datastore-misc-cleanups 2023-11-09 11:31:30 -08:00
Cory Snider
33564a0c03 libnetwork/d/overlay: support IPv6 transport
The forwarding database (fdb) of Linux VXLAN links are restricted to
entries with destination VXLAN tunnel endpoint (VTEP) address of a
single address family. Which address family is permitted is set when the
link is created and cannot be modified. The overlay network driver
creates VXLAN links such that the kernel only allows fdb entries to be
created with IPv4 destination VTEP addresses. If the Swarm is configured
with IPv6 advertise addresses, creating fdb entries for remote peers
fails with EAFNOSUPPORT (address family not supported by protocol).

Make overlay networks functional over IPv6 transport by configuring the
VXLAN links for IPv6 VTEPs if the local node's advertise address is an
IPv6 address. Make encrypted overlay networks secure over IPv6 transport
by applying the iptables rules to the ip6tables when appropriate.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-09 12:04:47 -05:00
Cory Snider
e1d85da306 libnetwork/d/overlay: parse discovery data eagerly
Parse the address strings once and use the binary representation
internally.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-09 12:04:47 -05:00
Sebastiaan van Stijn
5bff90cea2 Merge pull request #46789 from akerouanton/updateSvcRecord-early-return
libnet: early-return from updateSvcRecord if no addr available
2023-11-09 14:14:55 +01:00
Albin Kerouanton
d47b3ef4c9 libnet: early return from updateSvcRecord if no addr available
Early return if the iface or its address is nil to make the whole
function slightly easier to read.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-08 20:45:15 +01:00
Albin Kerouanton
7e66d9900c Merge pull request #46788 from akerouanton/container-rename-refactor
daemon: ContainerRename: use named error-return
2023-11-08 20:37:22 +01:00
Albin Kerouanton
f5a611a74c daemon: ContainerRename: use named error-return
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-08 18:53:13 +01:00
Sam Whited
8074e7a437 Bump go-connections for TLS 1.3 support
Fixes 40495

Signed-off-by: Sam Whited <sam@samwhited.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-08 17:58:50 +01:00
Sebastiaan van Stijn
5504ef7a84 Merge pull request #46784 from rumpl/image-builder-cleanup
c8d: cleanup imports in the image builder file
2023-11-08 17:27:52 +01:00
Sebastiaan van Stijn
ea729aea4a Merge pull request #46577 from RachitSharma2001/46576-dockerimagelsuntil
Add until filter to docker image ls
2023-11-08 16:58:08 +01:00
Sebastiaan van Stijn
587d509ddd Merge pull request #46777 from thaJeztah/fix_redirects
docs/api: update redirect metadata for hugo
2023-11-08 14:19:59 +01:00
Djordje Lukic
d06af368ad c8d: "unremap" the user namespace on commit
We remap the snapshot when we create a container, we have to to the
inverse when we commit the container into an image

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 13:28:45 +01:00
Djordje Lukic
a7acfffea6 test: Add a test for userns remapped daemon commit
The files in an image created by a commit should have the right uid and gid set

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 12:33:03 +01:00
Djordje Lukic
876d4e5484 c8d: cleanup imports in the image builder file
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 12:31:03 +01:00
Rachit Sharma
7995e3288f Add until filter to docker image ls
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

add handling for multiple filters

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Update integration/image/list_test.go

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Add documentation of filter

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Update integration/image/list_test.go

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Fix bug with CommitOptions

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

add wrapping of text to 80 chars

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>
2023-11-08 12:12:23 +01:00
Sebastiaan van Stijn
0c0943bcdf Merge pull request #46783 from thaJeztah/fix_test
integration/networking: fix TestBridgeICC
2023-11-08 12:11:40 +01:00
Sebastiaan van Stijn
58785c2932 integration/networking: fix TestBridgeICC
This test broke in 98323ac114.

This commit renamed WithMacAddress into WithContainerWideMacAddress.
This helper sets the MacAddress field in container.Config. However, API
v1.44 now ignores this field if the NetworkMode has no matching entry in
EndpointsConfig.

This fix uses the helper WithMacAddress and specify for which
EndpointConfig the MacAddress is specified.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-08 10:23:24 +01:00
Sebastiaan van Stijn
c14694a424 Merge pull request #46779 from dmcgowan/c8d-default-auth-domain
Default the auth config domain to the target image domain
2023-11-07 16:44:31 +01:00
Sebastiaan van Stijn
49cea49cfa Merge pull request #45905 from akerouanton/endpoint-specific-mac-address
api: Add a field MacAddress to EndpointSettings
2023-11-07 16:37:27 +01:00
Ken Bannister
6979503a85 Include port in URL for locally built Swagger docs
Signed-off-by: Ken Bannister <kb2ma@runbox.com>
2023-11-07 08:07:32 -05:00
Derek McGowan
755f008c1e Default the auth config domain to the target image domain
When server address is not provided with the auth configuration,
use the domain from the image provided with the auth.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-11-06 17:02:18 -08:00
Sebastiaan van Stijn
796da163f9 Merge pull request #46778 from corhere/libc8d/revert-serialize-exec-starts-workaround
Revert "libcontainerd: work around exec start bug in c8d"
2023-11-06 21:12:39 +01:00
Cory Snider
7d9d601e6d project: document supported containerd versions
We only support containerd versions which have fixes for
https://github.com/containerd/containerd/issues/8557.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-06 13:43:18 -05:00
Cory Snider
29ac09ee9d Revert "libcontainerd: work around exec start bug in c8d"
The workaround is no longer required. The bug has been fixed in stable
versions of all supported containerd branches.

This reverts commit fb7ec1555c.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-06 13:26:44 -05:00
Sebastiaan van Stijn
c1a289e05a docs/api: update redirect metadata for hugo
docs.docker.com switched from Jekyll to Hugo, which uses "aliases"
instead of "redirect_from".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-06 18:16:47 +01:00
Sebastiaan van Stijn
02011af7b0 Merge pull request #46774 from elezar/bump-cdi-dependency
Update container-device-interface to v0.6.2
2023-11-04 13:27:17 +01:00
Evan Lezar
49e04102c1 Update container-device-interface to v0.6.2
This includes migrating from the github.com/container-orchestrated-devices
repo to tags.cncf.io.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-11-04 01:00:19 +01:00
Paweł Gronowski
3cb8e9526e Merge pull request #46769 from vvoland/c8d-shared-mounter-impl
daemon/snapshotter: Align mounter implementations
2023-11-03 18:45:36 +01:00
Paweł Gronowski
26f63600c1 daemon/snapshotter: Align mounter implementations
Change the non-refcounted implementation to perform the mount using the
same identity and access right. They should be the same regardless if
we're refcounting or not.

This also allows to refactor refCountMounter into a mounter decorator.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-03 18:13:14 +01:00
Albin Kerouanton
6fd893bd3a Merge pull request #46768 from akerouanton/dedicated-default-nw-config-struct
daemon/config: Put params for the default network into a dedicated struct
2023-11-03 17:03:05 +01:00
Albin Kerouanton
d5d41c2849 daemon/config: Put params for the default network into a dedicated struct
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 14:10:41 +01:00
Sebastiaan van Stijn
ed1a61dcb7 Merge pull request #46663 from akerouanton/ci-otel-windows
ci: Setup otel tracing for windows integration tests
2023-11-03 13:51:39 +01:00
Sebastiaan van Stijn
5b19725de2 Merge pull request #46668 from corhere/libn/svc-record-update-without-store
libnetwork: svc record update without store
2023-11-03 13:47:12 +01:00
Sebastiaan van Stijn
0ac748a340 Merge pull request #46628 from sgehrig/46621-container_wait
#46621 ensure that errors reading the response body are returned to the caller
2023-11-03 13:20:59 +01:00
Sebastiaan van Stijn
26c054edbf Merge pull request #46531 from akerouanton/networking-suite-bridge-tests
integration: Add a new networking integration test suite
2023-11-03 12:33:44 +01:00
Sebastiaan van Stijn
39393f651e Merge pull request #46766 from thaJeztah/fix_TestInfoAPI
integration/system: update TestInfoAPI to not use string-matching
2023-11-03 11:43:28 +01:00
Sebastiaan van Stijn
587ad8845a integration/system: update TestInfoAPI to not use string-matching
This test was rewritten from an integration-cli test in commit
68d9beedbe, and originally implemented in
f4942ed864, which rewrote it from a unit-
test to an integration test.

Originally, it would check for the raw JSON response from the daemon, and
check for individual fields to be present in the output, but after commit
0fd5a65428, `client.Info()` was used, and
now the response is unmarshalled into a `system.Info`.

The remainder of the test remained the same in that rewrite, and as a
result were were now effectively testing if a `system.Info` struct,
when marshalled as JSON would show all the fields (surprise: it does).

TL;DR; the test would even pass with an empty `system.Info{}` struct,
which didn't provide much coverage, as it passed without a daemon:

    func TestInfoAPI(t *testing.T) {
        // always shown fields
        stringsToCheck := []string{
            "ID",
            "Containers",
            "ContainersRunning",
            "ContainersPaused",
            "ContainersStopped",
            "Images",
            "LoggingDriver",
            "OperatingSystem",
            "NCPU",
            "OSType",
            "Architecture",
            "MemTotal",
            "KernelVersion",
            "Driver",
            "ServerVersion",
            "SecurityOptions",
        }

        out := fmt.Sprintf("%+v", system.Info{})
        for _, linePrefix := range stringsToCheck {
            assert.Check(t, is.Contains(out, linePrefix))
        }
    }

This patch makes the test _slightly_ better by checking if the fields
are non-empty. More work is needed on this test though; currently it
uses the (already running) daemon, so it's hard to check for specific
fields to be correct (withouth knowing state of the daemon), but it's
not unlikely that other tests (partially) cover some of that. A TODO
comment was added to look into that (we should probably combine some
tests to prevent overlap, and make it easier to spot "gaps" as well).

While working on this, also moving the `SystemTime` into this test,
because that field is (no longer) dependent on "debug" state

(It is was actually this change that led me down this rabbit-hole)

                         ()_()
                         (-.-)
                        '(")(")'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-03 10:17:15 +01:00
Albin Kerouanton
c1ab6eda4b integration/networking: Test bridge ICC and INC
Following tests are implemented in this specific commit:

- Inter-container communications for internal and non-internal
  bridge networks, over IPv4 and IPv6.
- Inter-container communications using IPv6 link-local addresses for
  internal and non-internal bridge networks.
- Inter-network communications for internal and non-internal bridge
  networks, over IPv4 and IPv6, are disallowed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 09:58:50 +01:00
Albin Kerouanton
409ea700c7 integration: Add a new networking integration test suite
This commit introduces a new integration test suite aimed at testing
networking features like inter-container communication, network
isolation, port mapping, etc... and how they interact with daemon-level
and network-level parameters.

So far, there's pretty much no tests making sure our networks are well
configured: 1. there're a few tests for port mapping, but they don't
cover all use cases ; 2. there're a few tests that check if a specific
iptables rule exist, but that doesn't prevent that specific iptables
rule to be wrong in the first place.

As we're planning to refactor how iptables rules are written, and change
some of them to fix known security issues, we need a way to test all
combinations of parameters. So far, this was done by hand, which is
particularly painful and time consuming. As such, this new test suite is
foundational to upcoming work.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 09:58:50 +01:00
Sebastiaan van Stijn
050e6066af Merge pull request #46517 from rumpl/c8d-windows-pull-linux
c8d: test a backend dependent error on pull
2023-11-03 09:28:16 +01:00
Sebastiaan van Stijn
8068038a60 Merge pull request #46758 from rumpl/c8d-image-list-digest
Skip the busybox digest check when containerd is enabled
2023-11-03 09:16:48 +01:00
Sebastiaan van Stijn
90cc178baa Merge pull request #46759 from rumpl/c8d-index-pull-test
Skip TestPullManifestList when using containerd
2023-11-03 09:14:42 +01:00
Sebastiaan van Stijn
5baf486545 Merge pull request #46764 from cpuguy83/fix_probe_interval
Fix case where health start interval is 0 uses default
2023-11-03 08:32:55 +01:00
Sebastiaan van Stijn
ec32f0db82 Merge pull request #46762 from akerouanton/seccomp-io_uring
seccomp: block io_uring_* syscalls in default profile
2023-11-02 21:10:29 +01:00
Brian Goff
02a932d63f Fix case where health start interval is 0 uses default
When the start interval is 0 we should treat that as unset.
This is especially important for older API versions where we reset the
value to 0.

Instead of using the default probe value we should be using the
configured `interval` value (which may be a default as well) which gives
us back the old behavior before support for start interval was added.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-11-02 20:02:16 +00:00
Brian Goff
2f138d860e Merge pull request #46685 from rumpl/c8d-tag-does-not-exist
c8d: Return the "tag does not exist error"
2023-11-02 12:33:36 -07:00
Brian Goff
d3f4876f41 Merge pull request #46751 from thaJeztah/fix_TestSaveRepoWithMultipleImages_flaky
integration: make TestSaveRepoWithMultipleImages less flaky
2023-11-02 12:32:07 -07:00
Albin Kerouanton
891241e7e7 seccomp: block io_uring_* syscalls in default profile
This syncs the seccomp profile with changes made to containerd's default
profile in [1].

The original containerd issue and PR mention:

> Security experts generally believe io_uring to be unsafe. In fact
> Google ChromeOS and Android have turned it off, plus all Google
> production servers turn it off. Based on the blog published by Google
> below it seems like a bunch of vulnerabilities related to io_uring can
> be exploited to breakout of the container.
>
> [2]
>
> Other security reaserchers also hold this opinion: see [3] for a
> blackhat presentation on io_uring exploits.

For the record, these syscalls were added to the allowlist in [4].

[1]: a48ddf4a20
[2]: https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html
[3]: https://i.blackhat.com/BH-US-23/Presentations/US-23-Lin-bad_io_uring.pdf
[4]: https://github.com/moby/moby/pull/39415

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-02 19:05:47 +01:00
Djordje Lukic
3c2b06099d Skip TestPullManifestList when using containerd
This test is very weird, the Size in the manifests that it creates is
wrong, graph drivers only print a warning in that case but containerd
fails because it verifies more things. The media types are also wrong in
the containerd case, the manifest list forces the media type to be
"schema2.MediaTypeManifest" but in the containerd case the media type is
an OCI one.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-02 16:09:35 +01:00
Djordje Lukic
6ca1b9e4ce Skip the busybox digest check when containerd is enabled
We always have a digest with containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-02 14:07:49 +01:00
Sebastiaan van Stijn
fb3cc5e716 Merge pull request #46755 from corhere/libn/netip-overlaps
libnetwork/ipam: refactor prefix-overlap checks
2023-11-02 13:33:40 +01:00
Sebastiaan van Stijn
217054ddea Merge pull request #46754 from Frankkkkk/master
builder/dockerfile: errmsg: quote build target
2023-11-02 12:40:22 +01:00
Frank Villaro-Dixon
123ade3763 builder/dockerfile: errmsg: quote build target
The build target is not quoted and it makes it difficult for some
persons to see what the problem is.

By quoting it we emphasize that the target name is variable.

Signed-off-by: Frank Villaro-Dixon <frank.villarodixon@merkle.com>
2023-11-02 09:38:14 +01:00
Cory Snider
7257c77e19 libnetwork/ipam: refactor prefix-overlap checks
I am finally convinced that, given two netip.Prefix values a and b, the
expression

    a.Contains(b.Addr()) || b.Contains(a.Addr())

is functionally equivalent to

    a.Overlaps(b)

The (netip.Prefix).Contains method works by masking the address with the
prefix's mask and testing whether the remaining most-significant bits
are equal to the same bits in the prefix. The (netip.Prefix).Overlaps
method works by masking the longer prefix to the length of the shorter
prefix and testing whether the remaining most-significant bits are
equal. This is equivalent to
shorterPrefix.Contains(longerPrefix.Addr()), therefore applying Contains
symmetrically to two prefixes will always yield the same result as
applying Overlaps to the two prefixes in either order.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-01 11:44:24 -04:00
Sebastiaan van Stijn
60e88c873e Merge pull request #46736 from corhere/robust-tarsum
pkg/tarsum: handle xattrs like archive/tar does
2023-10-31 19:52:47 +01:00
Sebastiaan van Stijn
d9dce8e0d0 Merge pull request #46618 from vvoland/c8d-pull-all-tags-2
c8d/pull: Handle pull all tags (2nd approach)
2023-10-31 16:45:18 +01:00
Sebastiaan van Stijn
4be5b4147d Merge pull request #46748 from tonistiigi/containerd-platforms-normalize
builder-next: make sure worker platforms normalized for containerd
2023-10-31 12:38:09 +01:00
Sebastiaan van Stijn
4e8ba395f2 integration: TestSaveRepoWithMultipleImages: minor cleanup
- use consts for fixed values
- remove redundant `cmp.Nil(err)`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:54:57 +01:00
Sebastiaan van Stijn
0dae0f2b5e integration: TestSaveRepoWithMultipleImages remove redundant remove
This delete was originally added in b37fdc5dd1
and migrated from `deleteImages(repoName)` in commit 1e55ace875,
however, deleting `foobar-save-multi-images-test` (`foobar-save-multi-images-test:latest`)
always resulted in an error;

    Error response from daemon: No such image: foobar-save-multi-images-test:latest

This patch removes the redundant image delete.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:52:51 +01:00
Sebastiaan van Stijn
30cd8b8fca integration: make TestSaveRepoWithMultipleImages less flaky
Shutting down containers on Windows can take a long time (with hyper-v),
causing this test to be flaky; seen failing on windows 2022;

    === FAIL: github.com/docker/docker/integration/image TestSaveRepoWithMultipleImages (23.16s)
        save_test.go:104: timeout waiting for container to exit

Looking at the test, we run a container only to commit it, and the test
does not make changes to the container's filesystem; it only runs a container
with a custom command (`true`).

Instead of running the container, we can _create_ a container and commit it;
this simplifies the tests, and prevents having to wait for the container to
exit (before committing).

To verify:

    make BIND_DIR=. DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestSaveRepoWithMultipleImages test-integration

    INFO: Testing against a local daemon
    === RUN   TestSaveRepoWithMultipleImages
    --- PASS: TestSaveRepoWithMultipleImages (1.20s)
    PASS

    DONE 1 tests in 2.668s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:32:02 +01:00
Sebastiaan van Stijn
1c50526092 daemon: improve daemon start informational log message
When starting a daemon in debug mode (such as used in CI), many log-messages
are printed during startup. As a result, the log message indicating whether
graph-drivers or snapshotters are used may appear far separate from the
informational log about the daemon (and selected storage-driver).

The existing log-driver also unconditionally uses the legacy "graph-driver"
terminology, instead of the more generic "storage-driver".

This patch changes the log message shown during startup to use the generic
"graph-driver" as field, and adds a new field that indicates wheter we're
using snapshotters or graph-drivers.

Given that snapshotters will be the default at some point, an alternative
could be to include the _type_ of driver used, for example;
`io.containerd.snapshotter.v1`, which may continue to be relevant after
snapshotters become the default, and at which point (potentially) the
type of snapshotter becomes more relevant.

Before this change:

    TEST_INTEGRATION_USE_SNAPSHOTTER=1 DOCKER_GRAPHDRIVER=overlayfs dockerd
    ...
    INFO[2023-10-31T09:12:33.586269801Z] Starting daemon with containerd snapshotter integration enabled
    INFO[2023-10-31T09:12:33.586322176Z] Loading containers: start.
    INFO[2023-10-31T09:12:33.640514759Z] Loading containers: done.
    INFO[2023-10-31T09:12:33.646498134Z] Docker daemon                                 commit=dcf7287d647bcb515015e389df46ccf1e09855b7 graphdriver=overlayfs version=dev
    INFO[2023-10-31T09:12:33.646706551Z] Daemon has completed initialization
    INFO[2023-10-31T09:12:33.658840592Z] API listen on /var/run/docker.sock

With this change;

    TEST_INTEGRATION_USE_SNAPSHOTTER=1 DOCKER_GRAPHDRIVER=overlayfs dockerd
    ...
    INFO[2023-10-31T08:41:38.841155928Z] Starting daemon with containerd snapshotter integration enabled
    INFO[2023-10-31T08:41:38.841207512Z] Loading containers: start.
    INFO[2023-10-31T08:41:38.902461053Z] Loading containers: done.
    INFO[2023-10-31T08:41:38.910535137Z] Docker daemon                                 commit=dcf7287d647bcb515015e389df46ccf1e09855b7 containerd-snapshotter=true storage-driver=overlayfs version=dev
    INFO[2023-10-31T08:41:38.910936803Z] Daemon has completed initialization

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:20:27 +01:00
Tonis Tiigi
a99bb24ea8 builder-next: make sure worker platforms normalized for containerd
These platforms are filled by default from containerd
introspection API and may not be normalized. Initializing
wrong platform in here results in incorrect platform
for BUILDPLATFORM and TARGETPLATFORM build-args for
Dockerfile frontend (and probably other side effects).

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-10-30 23:14:34 -07:00
Sebastiaan van Stijn
e9efc0a361 Merge pull request #46741 from vvoland/c8d-push-notags
c8d/push: Return error when repository has no tags
2023-10-30 20:08:31 +01:00
Sebastiaan van Stijn
171f153e2c Merge pull request #46728 from rumpl/golangci-lint
Move the golangci-lint configuration file to the root dir
2023-10-30 18:55:17 +01:00
Scott Moser
29977d5058 contrib/check-config: conditionally disable color output.
Redirecting check-config.sh output to a file puts control character
output into that file, which isn't helpful for reading.

Disable colorized output if either
1. NO_COLOR environment is set to "1"
2. stdout is not a terminal.

Signed-off-by: Scott Moser <smoser@brickies.net>
2023-10-30 10:59:23 -04:00
Djordje Lukic
d60de4cafd Move the golangci-lint configuration file to the root dir
This will help other tools and IDEs find our lint configuration

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-30 13:48:58 +01:00
Sebastiaan van Stijn
dcf7287d64 Merge pull request #46662 from vvoland/c8d-pull-access-denied-msg-2
c8d/pull: Don't wrap `no basic auth` error
2023-10-30 12:43:41 +01:00
Paweł Gronowski
cd95cd0671 c8d/push: Return error when repository has no tags
In case of `docker push -a`, we need to return an error if there is no
image for the given repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-30 10:55:08 +01:00
Paweł Gronowski
df34db1158 c8d/pull: Don't wrap no basic auth error
Don't wrap the `no basic auth credentials` error from containerd and
return it as-is.

The error will look like:
```
failed to resolve reference "docker.io/library/aodkoakds:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-30 09:40:08 +01:00
Albin Kerouanton
5a83bebf76 ci: Setup otel tracing for windows integration tests
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-28 14:45:30 +02:00
Philipp Fruck
29ddf2214d docs(API): Add reason to network connect 403
The 403 error might not only be raised in swarm operations. It is
also returned when the given container is already connected to the
network and is currently running. I noticed this when during the
following PR: https://github.com/containers/podman/pull/20365

Signed-off-by: Philipp Fruck <dev@p-fruck.de>
2023-10-28 14:10:19 +02:00
Sebastiaan van Stijn
f44b085da6 Merge pull request #46732 from laurazard/add-exec-group-add-user-test
exec/tests: add test for `--group-add` with `--user`
2023-10-27 15:14:52 +02:00
Sebastiaan van Stijn
53ebecea7c Merge pull request #46734 from thaJeztah/vendor_containerd_1.7.8
vendor: github.com/containerd/containerd v1.7.8
2023-10-27 15:13:09 +02:00
Sebastiaan van Stijn
ae75dafe47 Merge pull request #46735 from thaJeztah/containerd_binary_1.7.8
update containerd binary to v1.7.8
2023-10-27 14:50:04 +02:00
Cory Snider
63a9d72ee8 pkg/tarsum: handle xattrs like archive/tar does
When writing a tar file with archive/tar, extended attributes in the
deprecated (tar.Header).Xattrs map take precedence over conflicting
'SCHILY.xattr' records in the (tar.Header).PAXRecords map. Update
package tarsum to follow the same precedence rules as archive/tar.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-26 19:23:10 -04:00
Sebastiaan van Stijn
4356962c69 update containerd binary to v1.7.8
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.8
full diff: https://github.com/containerd/containerd/compare/v1.7.6...v1.7.8

Notable Updates

- Fix ambiguous TLS fallback
- Update Go to 1.20.10
- Add a new image label on converted schema 1 images
- Fix handling for missing basic auth credentials
- Fix potential deadlock in create handler for containerd-shim-runc-v2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-27 00:48:26 +02:00
Sebastiaan van Stijn
38c4ceb00a vendor: github.com/containerd/containerd v1.7.8
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.8
full diff: https://github.com/containerd/containerd/compare/v1.7.7...v1.7.8

Notable Updates

- Fix ambiguous TLS fallback
- Update Go to 1.20.10
- Add a new image label on converted schema 1 images
- Fix handling for missing basic auth credentials
- Fix potential deadlock in create handler for containerd-shim-runc-v2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-27 00:44:49 +02:00
Sebastiaan van Stijn
34f5793521 vendor: google.golang.org/grpc v1.58.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-27 00:43:19 +02:00
Sebastiaan van Stijn
bd23c7729e vendor: golang.org/x/tools v0.10.0
full diff:  https://github.com/golang/tools/compare/v0.8.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-27 00:36:59 +02:00
Sebastiaan van Stijn
205e4de3fd vendor: golang.org/x/mod v0.11.0
no changes in vendored files

full diff: https://github.com/golang/mod/compare/v0.10.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-27 00:35:25 +02:00
Sebastiaan van Stijn
7cabe08399 Merge pull request #46707 from corhere/replace-xattrs-with-paxrecords
pkg/archive, pkg/tarsum: replace use of Xattrs with PAXRecords
2023-10-26 21:37:44 +02:00
Laura Brehm
153d7e4038 exec/tests: add test for --group-add with --user
Adds test ensuring that additional groups set with `--group-add`
are kept on exec when container had `--user` set on run.

Regression test for https://github.com/moby/moby/issues/46712

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-26 19:33:13 +01:00
Sebastiaan van Stijn
987f76ea78 Merge pull request #46731 from vvoland/c8d-history-fix-nil-deref
c8d/history: Fix nil dereference
2023-10-26 17:45:08 +02:00
Paweł Gronowski
27e064e7e9 c8d/history: Fix nil dereference
Check if `Created` is not nil before dereferencing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-26 15:37:45 +02:00
Sebastiaan van Stijn
1b9411ef77 Merge pull request #46723 from laurazard/cleanup-redundant-calls
daemon/exec_linux: assorted cleanups
2023-10-26 09:54:51 +02:00
Sebastiaan van Stijn
e0476beb78 Merge pull request #46688 from thaJeztah/restart_nocancel
daemon: daemon.containerRestart: don't cancel restart on context cancel
2023-10-26 09:50:20 +02:00
Richard Hansen
808120e5b8 New host_ipv6 bridge option to SNAT IPv6 connections
Add a new `com.docker.network.host_ipv6` bridge option to compliment
the existing `com.docker.network.host_ipv4` option. When set to an
IPv6 address, this causes the bridge to insert `SNAT` rules instead of
`MASQUERADE` rules (assuming `ip6tables` is enabled).  `SNAT` makes it
possible for users to control the source IP address used for outgoing
connections.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-25 20:11:49 -04:00
Laura Brehm
9d578a2e6d Cleanup duplicate imports
Kept `coci` import alias since we use it elsewhere,
maybe to prevent confusion with our own `oci` package.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-26 00:10:23 +01:00
Sebastiaan van Stijn
fc4d035e7a Merge pull request #46677 from rhansen/nat-test
bridge: Add unit tests for outgoing NAT rules
2023-10-26 00:15:48 +02:00
Sebastiaan van Stijn
c9ccbfad11 Merge pull request from GHSA-jq35-85cj-fj4p
[master] deny /sys/devices/virtual/powercap
2023-10-25 23:57:51 +02:00
Albin Kerouanton
59c5f6e35b integration-cli: mark dockerCmdWithFail as an helper function
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 22:55:59 +02:00
Albin Kerouanton
61c78d9438 daemon: Validate MacAddress on container create
This was previously checked only on ContainerStart.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 22:55:59 +02:00
Albin Kerouanton
ee9f0ed895 api: Deprecate ContainerConfig.MacAddress
Having a sandbox/container-wide MacAddress field makes little sense
since a container can be connected to multiple networks at the same
time. This field is an artefact of old times where a container could be
connected to a single network only.

As we now have a way to specify per-endpoint mac address, this field is
now deprecated.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-25 22:55:59 +02:00
Albin Kerouanton
052562ffd5 api: Add a field MacAddress to EndpointSettings
Prior to this commit, only container.Config had a MacAddress field and
it's used only for the first network the container connects to. It's a
relic of old times where custom networks were not supported.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 22:52:26 +02:00
Albin Kerouanton
98323ac114 integration/container: target v1.43 in a soon-to-break test
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 22:52:26 +02:00
Sebastiaan van Stijn
4776a9e6fb Merge pull request #46722 from thaJeztah/dedup_additionalgroups
daemon: remove redundant withResetAdditionalGIDs option
2023-10-25 22:48:19 +02:00
Sebastiaan van Stijn
460e1b3600 Merge pull request #46720 from akerouanton/container-create-init-structs
api: ContainerCreate: clean up BC conditions
2023-10-25 22:47:29 +02:00
Albin Kerouanton
4f0cab3b74 api: ContainerCreate: re-organize BC conditions
- Merge BC conds for API < v1.42 together
- Merge BC conds for API < v1.44 together
- Re-order BC conds by API version
- Move pids-limit normalization after BC conds

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:18 +02:00
Albin Kerouanton
79589354df api: ContainerCreate: remove repetitive nil checks
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:17 +02:00
Albin Kerouanton
44a25f31a8 api: ContainerCreate: lowercase ErrEmptyConfig
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:17 +02:00
Albin Kerouanton
91eee33f62 api: ContainerCreate: return an error when config is nil
The same error is already returned by `(*Daemon).containerCreate()` but
since this function is also called by the cluster executor, the error
has to be duplicated.

Doing that allows to remove a nil check on container config in
`postContainersCreate`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:17 +02:00
Sebastiaan van Stijn
1f096174d6 Merge pull request #46717 from rumpl/c8d-image-children
c8d: Use the labels to get the children of an image
2023-10-25 18:00:46 +02:00
Sebastiaan van Stijn
a54a34ef0e Merge pull request #46718 from rumpl/c8d-tag-label
c8d: Preserve image labels when tagging
2023-10-25 17:55:43 +02:00
Djordje Lukic
7d5445e312 Merge pull request #46719 from rumpl/c8d-image-created-date
c8d: show the real image creation date when listing images
2023-10-25 17:38:01 +02:00
Sebastiaan van Stijn
417018659b daemon: remove redundant withResetAdditionalGIDs option
containerd's `WithUser` function now resets this property, starting with
[3eda46af12b1deedab3d0802adb2e81cb3521950][1] (v1.7.0-beta.4), so we no
longer need this function.

[1]: 3eda46af12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-25 17:15:05 +02:00
Sebastiaan van Stijn
e36260f39a Merge pull request #46715 from laurazard/c8d-fix-swarm-failures
c8d/exec: Add additional groups on exec
2023-10-25 17:10:45 +02:00
Djordje Lukic
8f756fe679 c8d: show the real image creation date when listing images
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-25 16:43:24 +02:00
Albin Kerouanton
a73dfe68d3 api: ContainerCreate: init hostConfig and networkingConfig when nil
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 15:40:40 +02:00
Djordje Lukic
7e0cb4c46f c8d: Use the labels to get the children of an image
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-25 15:12:42 +02:00
Djordje Lukic
f5535d516f c8d: Preserve image labels when tagging
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-25 14:56:36 +02:00
Paweł Gronowski
dcc80204bc Merge pull request #46716 from akerouanton/fix-bad-condition
api: fix a nil check on the wrong var
2023-10-25 10:57:27 +02:00
Albin Kerouanton
70158284b7 api: fix a nil check on the wrong var
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 09:40:18 +02:00
Akihiro Suda
a66ddb7c59 Merge pull request #46711 from thaJeztah/switch_user
migrate to github.com/moby/sys/user
2023-10-25 14:07:01 +09:00
Laura Brehm
d7266afc7e c8d/exec: Add additional groups on exec
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-25 02:28:39 +01:00
Sebastiaan van Stijn
df3a321164 migrate to github.com/moby/sys/user
The github.com/opencontainers/runc/libcontainer/user package was moved
to a separate module. While there's still uses of the old module in
our code-base, runc itself is migrating to the new module, and deprecated
the old package (for runc 1.2).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 15:45:02 +02:00
Sebastiaan van Stijn
aeb8972281 daemon: daemon.containerRestart: don't cancel restart on context cancel
commit def549c8f6 passed through the context
to the daemon.ContainerStart function. As a result, restarting containers
no longer is an atomic operation, because a context cancellation could
interrupt the restart (between "stopping" and "(re)starting"), resulting
in the container being stopped, but not restarted.

Restarting a container, or more factually; making a successful request on
the `/containers/{id]/restart` endpoint, should be an atomic operation.

This patch uses a context.WithoutCancel for restart requests.

It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).

Before this patch, starting a container that bind-mounts the docker socket,
then restarting itself from within the container would cancel the restart
operation. The container would be stopped, but not started after that:

    docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
    docker exec myself sh -c 'docker restart myself'

    docker ps -a
    CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS                       PORTS     NAMES
    3a2a741c65ff   docker:cli    "docker-entrypoint.s…"   26 seconds ago   Exited (128) 7 seconds ago             myself

With this patch: the stop still cancels the exec, but does not cancel the
restart operation, and the container is started again:

    docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
    docker exec myself sh -c 'docker restart myself'
    docker ps
    CONTAINER ID   IMAGE        COMMAND                  CREATED              STATUS         PORTS     NAMES
    4393a01f7c75   docker:cli   "docker-entrypoint.s…"   About a minute ago   Up 4 seconds             myself

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 12:56:45 +02:00
Sebastiaan van Stijn
b4a08b3b7c Merge pull request #46700 from thaJeztah/bump_compress
vendor: github.com/klauspost/compress v1.17.2
2023-10-24 12:48:31 +02:00
Cory Snider
52da88201c hack/validate: stop suppressing Xattrs deprecation
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Cory Snider
c44c9dfa79 pkg/tarsum: migrate to (tar.Header).PAXRecords
Fix a silly bug in the implementation which had the effect of
len(h.Xattrs) blank entries being inserted in the middle of
orderedHeaders. Luckily this is not a load-bearing bug: empty headers
are ignored as the tarsum digest is computed by concatenating header
keys and values without any intervening delimiter.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Cory Snider
3cf409aa9e pkg/archive: migrate to (tar.Header).PAXRecords
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Cory Snider
6a8a792019 pkg/archive: test tar headers are interoperable
The existing pkg/archive unit tests are primarily round-trip tests which
assert that pkg/archive produces tarballs which pkg/archive can unpack.
While these tests are effective at catching regressions in archiving or
unarchiving, they have a blind spot for regressions in compatibility
with the rest of the ecosystem. For example, a typo in the capabilities
extended attribute constant would result in subtly broken image layer
tarballs, but the existing tests would not catch the bug if both the
archiving and unarchiving implementations have the same typo.

Extend the test for archiving an overlay filesystem layer to assert that
the overlayfs style whiteouts (extended attributes and device files) are
transformed into AUFS-style whiteouts (magic file names).

Extend the test for archiving files with extended attributes to assert
that the extended attribute is encoded into the file's tar header in the
standard, interoperable format compatible with the rest of the
ecosystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Sebastiaan van Stijn
452ca90fe5 Merge pull request #46698 from thaJeztah/update_gowinres
Dockerfile: update github.com/tc-hib/go-winres v0.3.1
2023-10-23 16:34:50 +02:00
Sebastiaan van Stijn
f2c67ea82f vendor: github.com/klauspost/compress v1.17.2
fixes data corruption with zstd output in "best"

- 1.17.2 diff: https://github.com/klauspost/compress/compare/v1.17.1...v1.17.2
- full diff: https://github.com/klauspost/compress/compare/v1.16.5...v1.17.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-23 16:32:24 +02:00
Sebastiaan van Stijn
a7b44ea10f Dockerfile: update github.com/tc-hib/go-winres v0.3.1
Update the GOWINRES_VERSION to v0.3.1;

full diff: https://github.com/tc-hib/go-winres/compare/v0.3.0...v0.3.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-23 12:12:43 +02:00
Richard Hansen
0cf113e250 Add unit tests for outgoing NAT rules
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-21 13:53:58 -04:00
Bjorn Neergaard
777e9f2710 Merge pull request #46690 from neersighted/buildkit_0.12.3
vendor: github.com/moby/buildkit v0.12.3
2023-10-20 12:58:51 -06:00
Bjorn Neergaard
619d98ab49 vendor: github.com/moby/buildkit v0.12.3
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-10-20 12:01:52 -06:00
Sebastiaan van Stijn
164167ea91 Merge pull request #46687 from thaJeztah/more_nocancel
daemon: use context.WithoutCancel in more places
2023-10-20 19:36:44 +02:00
Sebastiaan van Stijn
aad51c0b4e daemon: daemon.shutdownContainer: use context.WithoutCancel
Use context.WithoutCancel so that both the containerStop and
container.Wait can share the same parent context. This context is still
a "TODO", but can be wired up in future.

It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-20 17:50:06 +02:00
Sebastiaan van Stijn
dca89c67ef daemon: daemon.containerStop: use context.WithoutCancel
Follow-up to fc94ed0a86. Now that
f6e44bc0e8 added the compatcontext
package, we can start using context.WithoutCancel.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-20 17:44:30 +02:00
Djordje Lukic
f800215946 Merge pull request #46570 from dmcgowan/fix-disk-usage-test
integration: fix disk usage test for c8d
2023-10-20 17:11:31 +02:00
Djordje Lukic
8166818791 c8d: Return the "tag does not exist error"
In the tagged case the error message when the image/tag is not found
should be "tag does not exist: ref"

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-20 16:55:20 +02:00
Sebastiaan van Stijn
c3ca4f5de0 Merge pull request #46088 from thaJeztah/remove_deprecated_utils
integration-cli: remove deprecated `dockerCmd` and `waitRun` utilities
2023-10-20 15:44:46 +02:00
Sebastiaan van Stijn
74641d2006 Merge pull request #46680 from vvoland/c8d-history-off-by-one
c8d/history: Fill ID for parents without the label
2023-10-19 22:59:29 +02:00
Cory Snider
4af420f978 libnetwork/internal/kvstore: prune unused method
The datastore never calls Get() due to how the cache is implemented.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 12:57:42 -04:00
Cory Snider
4039b9c9c4 libnetwork/datastore: drop (KVObject).DataScope()
It wasn't being used for anything meaningful.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 12:38:39 -04:00
Cory Snider
4f4a897dda libnetwork/datastore: drop (*Store).Scope() method
It unconditionally returned scope.Local.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 12:38:37 -04:00
Cory Snider
4b40d82233 libnetwork/datastore: un-embed mutex from cache
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 12:37:12 -04:00
Cory Snider
9536fabaa8 libnetwork/datastore: minor code cleanup
While there is nothing inherently wrong with goto statements, their use
here is not helping with readability.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 12:37:12 -04:00
Cory Snider
43dccc6c1a libnetwork/datastore: unconditionally use ds.cache
ds.cache is never nil so the uncached code paths are unreachable in
practice. And given how many KVObject deep-copy implementations shallow
copy pointers and other reference-typed values, there is the distinct
possibility that disabling the datastore cache could break things.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 12:37:10 -04:00
Sebastiaan van Stijn
5a72ed3406 integration-cli: remove deprecated dockerCmd and waitRun utilities
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:02:30 +02:00
Sebastiaan van Stijn
06115bf82c integration-cli: DockerCLIPruneSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:02:16 +02:00
Sebastiaan van Stijn
285ab8c861 integration-cli: DockerCLIPluginsSuite: replace dockerCmd
Also fixed some variables that shadowed package-level vars, and
used consts for fixed values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:02:04 +02:00
Sebastiaan van Stijn
e28da7686e integration-cli: DockerCLIPluginLogDriverSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:01:46 +02:00
Sebastiaan van Stijn
a837cb444c integration-cli: DockerExternalVolumeSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:01:37 +02:00
Sebastiaan van Stijn
5f884478e5 integration-cli: DockerCLIImportSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:01:24 +02:00
Sebastiaan van Stijn
885eceba1d integration-cli: DockerCLITopSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:01:09 +02:00
Sebastiaan van Stijn
6ce6b63482 integration-cli: DockerCLIHistorySuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:00:56 +02:00
Sebastiaan van Stijn
268e72f10f integration-cli: DockerCLIRestartSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:00:39 +02:00
Sebastiaan van Stijn
c388416aac integration-cli: DockerCLICreateSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:00:27 +02:00
Sebastiaan van Stijn
5c297fcc09 integration-cli: DockerCLIStatsSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 18:00:00 +02:00
Sebastiaan van Stijn
7d2a6f2030 integration-cli: DockerCLICommitSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:59:48 +02:00
Sebastiaan van Stijn
a4522b6a8f integration-cli: DockerCLIStartSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:59:34 +02:00
Sebastiaan van Stijn
f7ddfdbae6 integration-cli: DockerCLIUpdateSuite: replace dockerCmd and waitRun
Also adding some consts for fixed values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:59:24 +02:00
Sebastiaan van Stijn
d1108b8a06 integration-cli: DockerDaemonSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:59:10 +02:00
Sebastiaan van Stijn
9c161cc65a integration-cli: DockerCLIVolumeSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:58:57 +02:00
Sebastiaan van Stijn
f482857959 integration-cli: DockerCLIImagesSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:58:12 +02:00
Sebastiaan van Stijn
f1102e98bb integration-cli: DockerCLIPortSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:57:41 +02:00
Sebastiaan van Stijn
ba4d491562 integration-cli: DockerCLIRmiSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:57:31 +02:00
Cory Snider
5b3086db1f libnetwork/datastore: prevent accidental recursion
The datastore cache only uses the reference to its datastore to get a
reference to the backing store. Modify the cache to take the backing
store reference directly so that methods on the datastore can't get
called, as that might result in infinite recursion between datastore and
cache methods.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-19 11:56:08 -04:00
Sebastiaan van Stijn
16e4bc9827 integration-cli: DockerCLISearchSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:54:11 +02:00
Sebastiaan van Stijn
89d7b77022 integration-cli: DockerRegistrySuite: replace dockerCmd
Also renaming vars that collided with package-level vars
and using consts for fixed values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:53:43 +02:00
Sebastiaan van Stijn
c867ac4527 integration-cli: DockerCLICpSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:52:44 +02:00
Sebastiaan van Stijn
980a5bc72a integration-cli: DockerCLIBuildSuite: replace dockerCmd
Also adding some consts for fixed values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:52:39 +02:00
Sebastiaan van Stijn
4292f42308 integration-cli: DockerCLISaveLoadSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:42:48 +02:00
Sebastiaan van Stijn
50062b413c integration-cli: DockerCLIRunSuite: replace dockerCmd and waitRun
Also adding some consts for fixed values

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:40:35 +02:00
Sebastiaan van Stijn
d63cab5193 integration-cli: DockerCLILogsSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:40:15 +02:00
Sebastiaan van Stijn
8e2e1f3d28 integration-cli: DockerRegistryAuthHtpasswdSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:39:53 +02:00
Sebastiaan van Stijn
d3f4580519 integration-cli: DockerCLILinksSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:39:17 +02:00
Sebastiaan van Stijn
58c5986f8b integration-cli: DockerCLIHealthSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:39:00 +02:00
Sebastiaan van Stijn
707bb9b9ed integration-cli: DockerBenchmarkSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:38:54 +02:00
Sebastiaan van Stijn
eafb618594 integration-cli: findContainerIP: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:38:45 +02:00
Sebastiaan van Stijn
ba32bebd05 integration-cli: DockerCLIInfoSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:38:33 +02:00
Sebastiaan van Stijn
4562f07887 integration-cli: DockerCLIExecSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:36:46 +02:00
Sebastiaan van Stijn
8c661b5e9a integration-cli: DockerCLIEventSuite: replace dockerCmd and waitRun
Also removing some waitRun call, as they were not actually checked for
results, and the tests depended on that behavior (to get events about
the container starting etc).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:36:41 +02:00
Sebastiaan van Stijn
cf95278122 integration-cli: DockerCLIAttachSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:35:46 +02:00
Sebastiaan van Stijn
1baec48367 integration-cli: DockerAPISuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:34:55 +02:00
Sebastiaan van Stijn
908821d48a integration-cli: dockerCmdWithFail: remove unused return
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:20:56 +02:00
Sebastiaan van Stijn
02fd8485fa integration-cli: DockerCLINetmodeSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:20:56 +02:00
Sebastiaan van Stijn
dbb2bd727d integration-cli: DockerNetworkSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:20:56 +02:00
Sebastiaan van Stijn
5cc978c6a0 integration-cli: DockerCLIInspectSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:20:56 +02:00
Sebastiaan van Stijn
86fb048e59 integration-cli: DockerCLIPsSuite: replace dockerCmd and waitRun
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:20:56 +02:00
Sebastiaan van Stijn
b4ff5234bd integration-cli: remove transformCmd utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:20:53 +02:00
Paweł Gronowski
cdb44010c9 c8d/history: Fill ID for parents without the label
When choosing the next image, don't reject images without the classic
builder parent label. The intention was to *prefer* images them instead
of making that a condition.
This fixes the ID not being filled for parent images that weren't built
with the classic builder.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-19 16:41:45 +02:00
Djordje Lukic
29d863d451 Merge pull request #46653 from vvoland/c8d-history-fix-tags
c8d/history: Fix Tags assigned from parents
2023-10-19 14:53:09 +02:00
Djordje Lukic
0253fedf03 Merge pull request #46655 from vvoland/c8d-push-propagate-source-children
c8d/push: Set distribution source recursively
2023-10-19 10:30:47 +02:00
Paweł Gronowski
d28bd54f2a c8d/history: Fix Tags assigned from parents
The `Tags` slice of each history entry was filled with tags of parent
image. Change it to correctly assign the current image tags.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-19 10:19:04 +02:00
Derek McGowan
e97716a95a Fix disk usage test
Check for accurate values that may contain content sizes unknown to the
usage test in the calculation. Avoid asserting using deep equals when
only the expected value range is known to the test.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-18 21:20:54 -07:00
Bjorn Neergaard
5338c06c2a Merge pull request #46667 from stephan-henningsen/patch-2
Add IP_NF_MANGLE to check-config.sh
2023-10-18 14:55:22 -07:00
Stephan Henningsen
cf9073397c Update check-config.sh
Add IP_NF_MANGLE to "Generally Required" kernel features, since it appears to be necessary for Docker Swarm to work.

Closes https://github.com/moby/moby/issues/46636

Signed-off-by: Stephan Henningsen <stephan-henningsen@users.noreply.github.com>
2023-10-18 21:59:04 +02:00
Sebastiaan van Stijn
46d11bddbb Merge pull request #46670 from thaJeztah/bump_gotest_tools
vendor: gotest.tools/v3 v3.5.1
2023-10-18 17:43:55 +02:00
Sebastiaan van Stijn
7bb54c3dc5 vendor: gotest.tools/v3 v3.5.1
full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.5.0..v3.5.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-18 14:37:07 +02:00
Sebastiaan van Stijn
70589b8548 Merge pull request #46665 from thaJeztah/update_image_spec
vendor: github.com/opencontainers/image-spec v1.1.0-rc5 and use OCI consts
2023-10-18 10:23:22 +02:00
Sebastiaan van Stijn
1f965317b6 Merge pull request #46457 from thaJeztah/bump_swarmkit
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230911190601-f082dd7a0cee
2023-10-18 10:20:38 +02:00
Cory Snider
bcca214e36 libnetwork: open-code updating svc records
Inline the tortured logic for deciding when to skip updating the svc
records to give us a fighting chance at deciphering the logic behind the
logic and spotting logic bugs.

Update the service records synchronously. The only potential for issues
is if this change introduces deadlocks, which should be fixed by
restrucuting the mutexes rather than papering over the issue with
sketchy hacks like deferring the operation to a goroutine.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 19:51:21 -04:00
Cory Snider
33cf73f699 libnetwork: drop (*Controller).nmap
Its only remaining purpose is to elide removing the endpoint from the
service records if it was not previously added. Deleting the service
records is an idempotent operation so it is harmless to delete service
records which do not exist.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 19:46:18 -04:00
Cory Snider
804ef16822 libnetwork: only delete svc db entry on network rm
The service db entry for each network is deleted by
(*Controller).cleanupServiceDiscovery() when the network is deleted.
There is no need to also eagerly delete it whenever the network's
endpoint count drops to zero.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 19:46:18 -04:00
Cory Snider
c85398b020 libnetwork: drop vestigial endpoint-rename logic
The logic to rename an endpoint includes code which would synchronize
the renamed service records to peers through the distributed datastore.
It would trigger the remote peers to pick up the rename by touching a
datastore object which remote peers would have subscribed to events on.
The code also asserts that the local peer is subscribed to updates on
the network associated with the endpoint, presumably as a proxy for
asserting that the remote peers would also be subscribed.
https://github.com/moby/libnetwork/pull/712

Libnetwork no longer has support for distributed datastores or
subscribing to datastore object updates, so this logic can be deleted.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 19:46:18 -04:00
Cory Snider
29da565133 libnetwork: change netWatch map to a set
The map keys are only tested for presence. The value stored at the keys
is unused.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 18:26:34 -04:00
Cory Snider
0456c0db87 libnetwork: refactor isDistributedControl()
The meaning of the (*Controller).isDistributedControl() method is not
immediately clear from the name, and it does not have any doc comment.
It returns true if and only if the controller is neither a manager node
nor an agent node -- that is, if the daemon is _not_ participating in a
Swarm cluster. The method name likely comes from the old abandoned
datastore-as-IPC control plane architecture for libnetwork. Refactor

    c.isDistributedControl() -> !c.isSwarmNode()

to make it easier to understand code which consumes the method.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 17:59:19 -04:00
Cory Snider
749d4abd41 libnetwork: get rid of watchLoop goroutine
Replace with roughly equivalent code which relies upon the existing
mutexes for synchronization.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-17 17:06:52 -04:00
Sebastiaan van Stijn
f05dce8d3a replace some strings with OCI consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-17 16:34:52 +02:00
Sebastiaan van Stijn
e3feb05175 vendor: github.com/opencontainers/image-spec v1.1.0-rc5
full diff: https://github.com/opencontainers/image-spec/compare/v1.1.0-rc4...v1.1.0-rc5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-17 14:03:41 +02:00
Sebastiaan van Stijn
b85185e659 Merge pull request #46652 from laurazard/fix-test-import-custom-image-custom-plat
c8d integration: skip TestImportWithCustomPlatform
2023-10-17 13:56:57 +02:00
Stefan Gehrig
0d27579fc7 Ensure that non-JSON-parsing errors are returned to the caller
Signed-off-by: Stefan Gehrig <stefan.gehrig.hn@googlemail.com>
Co-authored-by: Cory Snider <corhere@gmail.com>
2023-10-17 10:32:42 +02:00
Sebastiaan van Stijn
f6576e9a74 Merge pull request #46657 from thaJeztah/bump_grpc
vendor: google.golang.org/grpc v1.56.3
2023-10-16 23:32:13 +02:00
Sebastiaan van Stijn
26730c891a vendor: google.golang.org/grpc v1.56.3
server: prohibit more than MaxConcurrentStreams handlers from running at once
(CVE-2023-44487).

In addition to this change, applications should ensure they do not leave running
tasks behind related to the RPC before returning from method handlers, or should
enforce appropriate limits on any such work.

- https://github.com/grpc/grpc-go/compare/v1.56.2...v1.56.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 20:37:51 +02:00
Sebastiaan van Stijn
91cb91a304 Merge pull request #46645 from thaJeztah/otel_semconv
switch to go.opentelemetry.io/otel/semconv/v1.17.0
2023-10-16 20:10:15 +02:00
Bjorn Neergaard
fd3066c168 Merge pull request #46646 from thaJeztah/start_interval_validation
HealthCheck: add validation for minimum accepted start-interval (1ms)
2023-10-16 11:07:14 -07:00
Sebastiaan van Stijn
a7dd561c4a Merge pull request #46651 from akerouanton/releaseNetwork-NetworkDisabled
daemon: release sandbox even when NetworkDisabled
2023-10-16 19:33:08 +02:00
Sebastiaan van Stijn
79521da5cc Merge pull request #46656 from vvoland/c8d-pull-access-denied-msg
c8d/pull: Return same access denied error as graphdrivers
2023-10-16 19:12:47 +02:00
Laura Brehm
6f625ae65a c8d integration: skip TestImportWithCustomPlatform
We support importing images for other platforms when
using the containerd image store, so we shouldn't validate
the image OS on import.

This commit also splits the test into two, so that we can
keep running the "success" import with a custom platform tests
running w/ c8d while skipping the "error/rejection" test cases.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-16 18:08:36 +01:00
Cory Snider
af22957b94 Merge pull request #46493 from rhansen/bridge-cleanups
bridge driver: various code quality improvements
2023-10-16 13:08:06 -04:00
Paweł Gronowski
9357bcb582 c8d/pull: Return same access denied error as graphdrivers
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 17:54:01 +02:00
Paweł Gronowski
488559a330 c8d/push: Set distribution source recursively
After a successful push, all pushed blobs should have a
distribution.source label pointing to the new registry.

Before this commit, the label was only appended to the top-level blob
(manifest or manifest list). Adjust this to also do that recursively to
its children.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 17:07:39 +02:00
Paweł Gronowski
d9b5445f39 c8d/pull: Handle pull all tags
Use the distribution code to query the remote repository for tags and
pull them sequentially just like the non-c8d pull.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 14:19:29 +02:00
Paweł Gronowski
095d2a29a3 distribution: Add Tags
Add a function to return tags for the given repository reference. This
is needed to implement the `pull -a` (pull all tags) for containerd
which doesn't directly use distribution, but we need to somehow make an
API call to the registry to obtain the available tags.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 14:19:27 +02:00
Paweł Gronowski
6040283f23 Merge pull request #46581 from vvoland/c8d-push-hide-jsons
c8d/push: Show progress only on blobs
2023-10-16 13:39:25 +02:00
Sebastiaan van Stijn
5c3d0fbc5e Merge pull request #46572 from crazy-max/ci-snapshotter
ci: split and use matrix to test containerd backed image store
2023-10-16 13:06:51 +02:00
payall4u
9664f33e0d daemon: release sandbox even when NetworkDisabled
When the default bridge is disabled by setting dockerd's `--bridge=none`
option, the daemon still creates a sandbox for containers with no
network attachment specified. In that case `NetworkDisabled` will be set
to true.

However, currently the `releaseNetwork` call will early return if
NetworkDisabled is true. Thus, these sandboxes won't be deleted until
the daemon is restarted. If a high number of such containers are
created, the daemon would then take few minutes to start.

See https://github.com/moby/moby/issues/42461.

Signed-off-by: payall4u <payall4u@qq.com>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-16 12:08:01 +02:00
Sebastiaan van Stijn
2df698025c HealthCheck: add validation for minimum accepted start-interval
This is a follow-up to 2216d3ca8d, which
implemented the StartInterval for health-checks, but did not add validation
for the minimum accepted interval;

> The time to wait between checks in nanoseconds during the start period.
> It should be 0 or at least 1000000 (1 ms). 0 means inherit.

This patch adds validation for the minimum accepted interval (1ms).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 10:46:18 +02:00
Sebastiaan van Stijn
25fb4dd50d switch to go.opentelemetry.io/otel/semconv/v1.17.0
While updating the docker/docker dependency in BuildKit, I noticed that the
dependency tree showed _two_ separate versions of the semconv package;
BuildKit and containerd were using the v1.17.0 version and docker/docker was
using v1.7.0.

This patch updates the version we use to align with BuildKit and containerd.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 10:28:54 +02:00
Akihiro Suda
670bc0a46c Merge pull request #46644 from thaJeztah/bump_buildkit_v0.12.3
vendor: github.com/moby/buildkit v0.12.3-dev
2023-10-16 05:14:19 +09:00
Richard Hansen
96f85def5b s/HostIP/HostIPv4/ for com.docker.network.host_ipv4 setting
Rename all variables/fields/map keys associated with the
`com.docker.network.host_ipv4` option from `HostIP` to `HostIPv4`.
Rationale:

  * This makes the variable/field name consistent with the option
    name.
  * This makes the code more readable because it is clear that the
    variable/field does not hold an IPv6 address.  This will hopefully
    avoid bugs like <https://github.com/moby/moby/issues/46445> in the
    future.
  * If IPv6 SNAT support is ever added, the names will be symmetric.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 02:47:14 -04:00
Richard Hansen
2a14b6cf60 Use iptRule to simplify setIcc (code health)
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 02:47:14 -04:00
Richard Hansen
d7c6fd2f80 Move programChainRule logic to iptRule methods (code health)
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 02:47:13 -04:00
Richard Hansen
e260808a57 Move duplicate logic to iptRule.Exists method (code health)
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 01:41:09 -04:00
Richard Hansen
14d2535f13 Move iptables.IPVersion into iptRule struct (code health)
Rather than pass an `iptables.IPVersion` value alongside every
`iptRule` parameter, embed the IP version in the `iptRule` struct.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 01:41:09 -04:00
Richard Hansen
4e219ebafb Eliminate unnecessary iptRule.preArgs field (code health)
That field was only used to pass `-t nat` for NAT rules.  Now `-t
<tableName>` (where `<tableName>` is one of the `iptables.Table`
values) is always passed, eliminating the need for `preArgs`.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 01:41:09 -04:00
Richard Hansen
4662e9889c Simplify setupIPTablesInternal parameters (code health)
Pass the entire `*networkConfiguration` struct to
`setupIPTablesInternal` to simplify the function signature and improve
code readability.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-10-14 01:41:09 -04:00
Sebastiaan van Stijn
919fe72f60 vendor: github.com/moby/buildkit v0.12.3-dev
update buildkit to the latest code in the v0.12 branch:

full diff: f94ed7cec3...6560bb937e

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-10-13 17:05:56 -07:00
Sebastiaan van Stijn
80a9fc6d36 Merge pull request #46565 from vvoland/c8d-mirrors-fix
daemon/RegistryHosts: Don't lose mirrors
2023-10-13 22:31:24 +02:00
Sebastiaan van Stijn
cdb3f9fb8d Merge pull request #46643 from neersighted/revert_46631
Revert "Return no basic auth creds error when needed"
2023-10-13 20:36:48 +02:00
Bjorn Neergaard
40fde69be9 Revert "Return no basic auth creds error when needed"
This reverts commit 8777592397, which
turns out to break other test cases/the registry flow.

The correct place to handle missing credentials is instead
15bf23df09/remotes/docker/authorizer.go (L200).

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-10-13 11:39:48 -06:00
Sebastiaan van Stijn
c2aa895908 Merge pull request #46638 from vvoland/c8d-diff-singleflight
c8d/changes: Fix concurrents diffs
2023-10-13 18:12:24 +02:00
CrazyMax
e1bacd18a3 ci: continue-on-error on non-PR with snapshotter enabled
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-13 16:40:18 +02:00
CrazyMax
70892220c8 ci: use matrix to test containerd backed image store
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-13 16:40:13 +02:00
CrazyMax
03decbc1e6 ci: split tests in a reusable workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-13 16:38:07 +02:00
Sebastiaan van Stijn
bb1fb9aaad Merge pull request #46637 from neersighted/libnetwork_ctstate
libnetwork: use conntrack and --ctstate for all rules
2023-10-13 16:03:47 +02:00
Paweł Gronowski
5a34c7c245 Merge pull request #46533 from vvoland/c8d-save-multiple-repo
c8d/save-load: Reimplement non-c8d idiosyncrasies
2023-10-13 14:41:33 +02:00
Paweł Gronowski
bb7408e851 c8d/changes: Fix concurrents diffs
Use a unique parent view snapshot key for each diff request.

I considered using singleflight at first, but I realized it wouldn't
really be correct.
The diff can take some time, so there's a window of time between the
diff start and finish, where the file system can change.
These changes not always will be reflected in the running diff.
With singleflight, the second diff request which happened before the
previous diff was finished, would not include changes made to the
container filesystem after the first diff request has started.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-13 13:15:50 +02:00
Sebastiaan van Stijn
6c1c76d02a Merge pull request #46631 from rumpl/c8d-no-basic-auth
Return no basic auth creds error when needed
2023-10-13 13:03:26 +02:00
Paweł Gronowski
e3d8e979de integration-cli/TestSaveAndLoadWithProgressBar: Skip with c8d
Not implemented yet, but we have an open ticket.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-13 11:52:13 +02:00
Paweł Gronowski
f51c5e1a5c c8d/save: Don't save name if exported by digest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-13 11:52:11 +02:00
Paweł Gronowski
b5f5987959 c8d/load: Fix Loaded image message
It should also contain the image tag, not just repository name.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-13 11:52:09 +02:00
Paweł Gronowski
42af8795a3 c8d/save: Implement exporting all tags
Implement a behavior from the graphdriver's export where `docker save
something` (untagged reference) would export all images matching the
specified repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-13 11:52:07 +02:00
Bjorn Neergaard
8047b69ba3 Merge pull request #46591 from vvoland/c8d-adjust-TestPsListContainersSize
integration-cli: Make TestPsListContainersSize work with c8d
2023-10-13 00:00:54 -07:00
Bjorn Neergaard
f20abbc96c libnetwork: use conntrack and --ctstate for all rules
On modern kernels this is an alias; however newer code has preferred
ctstate while older code has preferred the deprecated 'state' name.

Prefer the newer name for uniformity in the rules libnetwork creates,
and because some implementations/distributions of the xtables userland
tools may not support the legacy alias.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-10-13 00:56:30 -06:00
Bjorn Neergaard
796c05ca32 Merge pull request #46094 from slonopotamus/darwin-build-fix
pkg/chrootarchive: fix Darwin build
2023-10-12 15:15:18 -07:00
Sebastiaan van Stijn
3e43f8e9f4 Merge pull request #46594 from regisb/36598-regisb/registry-mirror-path
registry: allow mirror paths in config
2023-10-12 20:37:52 +02:00
Sebastiaan van Stijn
4cbac66746 Merge pull request #46633 from rumpl/c8d-skip-pull-all-tags
Skip TestPullAllTagsFromCentralRegistry
2023-10-12 18:40:23 +02:00
Djordje Lukic
c279c48a0e Skip TestPullAllTagsFromCentralRegistry
The image that this test pulls contains an error in the linux/amd64
manifest description, the reported size is 424 but the actual size is
524, making this test fail with containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-12 15:47:48 +02:00
Sebastiaan van Stijn
0a82696d70 Merge pull request #46483 from thaJeztah/api_move_image_types
api/types: move various types to api/types/(images|containers|swarm)
2023-10-12 15:29:11 +02:00
Djordje Lukic
8777592397 Return no basic auth creds error when needed
If the auth config contains an empty username/password we should return
an error

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-12 15:02:09 +02:00
Sebastiaan van Stijn
adea457841 Merge pull request #46553 from thaJeztah/no_panic
libnetwork: Controller: getKeys, getPrimaryKeyTag: prevent panic and small refactor
2023-10-12 14:19:06 +02:00
Paweł Gronowski
3205ace201 Merge pull request #46589 from vvoland/c8d-push-mounted-exists
c8d/push: Show Mounted/Already exists status
2023-10-12 12:52:24 +02:00
Paweł Gronowski
c5389354ac Merge pull request #46542 from thaJeztah/update_distribution_2.8.3
vendor: github.com/docker/distribution v2.8.3
2023-10-12 12:13:15 +02:00
Sebastiaan van Stijn
ebef4efb88 api/types: move ContainerLogsOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:30:12 +02:00
Sebastiaan van Stijn
9670d9364d api/types: move ContainerListOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:24 +02:00
Sebastiaan van Stijn
7bce33eb0f api/types: move ContainerStartOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:24 +02:00
Sebastiaan van Stijn
0f77875220 api/types: move ContainerRemoveOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
9498d897ab api/types: move ContainerCommitOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
30f09b4a1a api/types: move ContainerAttachOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
95b92b1f97 api/types: move ResizeOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
ec69501e94 api/types: move ServiceCreateResponse, and generate from swagger
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
5b3e6555a3 api/types: move ServiceUpdateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:22 +02:00
Sebastiaan van Stijn
48cacbca24 api/types: move image-types to api/types/image
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:20 +02:00
Sebastiaan van Stijn
aacd100a70 Merge pull request #46623 from thaJeztah/bump_x_net
vendor: golang.org/x/net v0.17.0
2023-10-12 11:15:22 +02:00
Paweł Gronowski
44dbbeb196 c8d/progress: Remove unused mountable
It's no longer needed as we get this information from containerd
directly.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-12 11:08:11 +02:00
Paweł Gronowski
bcbbd9538f c8d/progress: Show Mounted/Exists status
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-12 11:08:08 +02:00
Sebastiaan van Stijn
056be8b7b0 Merge pull request #46473 from thaJeztah/c8d_logs
migrate to github.com/containerd/log v0.1.0
2023-10-12 10:35:02 +02:00
Akihiro Suda
b33ad6fff6 Merge pull request #46622 from thaJeztah/bump_golang_1.21.3
update to go1.21.3
2023-10-12 07:20:21 +09:00
Sebastiaan van Stijn
6f6ef7906a Merge pull request #46578 from rumpl/skip-max-concurrency-tests
c8d: Skip tests that check max concurrency
2023-10-11 22:41:41 +02:00
Bjorn Neergaard
bea1462f4f Merge pull request #46564 from AkihiroSuda/fix-46563
Limit OOMScoreAdj when running in UserNS ("Rootful-in-Rootless")
2023-10-11 12:06:18 -07:00
Sebastiaan van Stijn
bb8bc1ffc8 update to go1.21.3
go1.21.3 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.21.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.3+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.21.2...go1.21.3

From the security mailing:

[security] Go 1.21.3 and Go 1.20.10 are released

Hello gophers,

We have just released Go versions 1.21.3 and 1.20.10, minor point releases.

These minor releases include 1 security fixes following the security policy:

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 20:01:17 +02:00
Sebastiaan van Stijn
3bc45d78c9 update to go1.21.2
go1.21.2 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the compiler, the go command, the linker, the runtime,
and the runtime/metrics package. See the Go 1.21.2 milestone on our issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.2+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.21.1...go1.21.2

From the security mailing:

[security] Go 1.21.2 and Go 1.20.9 are released

Hello gophers,

We have just released Go versions 1.21.2 and 1.20.9, minor point releases.

These minor releases include 1 security fixes following the security policy:

- cmd/go: line directives allows arbitrary execution during build

  "//line" directives can be used to bypass the restrictions on "//go:cgo_"
  directives, allowing blocked linker and compiler flags to be passed during
  compliation. This can result in unexpected execution of arbitrary code when
  running "go build". The line directive requires the absolute path of the file in
  which the directive lives, which makes exploting this issue significantly more
  complex.

  This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 20:01:05 +02:00
Sebastiaan van Stijn
949f8c9f2a Merge pull request #46602 from vvoland/integration-TestPullNonExistingImage
TestPullNonExistingImage: Simplify and move to integration
2023-10-11 19:18:16 +02:00
Sebastiaan van Stijn
1800dd0876 vendor: golang.org/x/net v0.17.0
full diff: https://github.com/golang/net/compare/v0.13.0...v0.17.0

This fixes the same CVE as go1.21.3 and go1.20.10;

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 18:58:01 +02:00
Sebastiaan van Stijn
1946aead21 vendor: golang.org/x/crypto v0.14.0
full diff: https://github.com/golang/crypto/compare/v0.11.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 18:57:10 +02:00
Sebastiaan van Stijn
0679ae984b vendor: golang.org/x/text v0.13.0
full diff: https://github.com/golang/text/compare/v0.11.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 18:56:07 +02:00
Sebastiaan van Stijn
bc33c27796 vendor: golang.org/x/sys v0.13.0
full diff: https://github.com/golang/sys/compare/v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 18:55:24 +02:00
Sebastiaan van Stijn
f2e163b75d Merge pull request #46620 from vvoland/c8d-test-ignore-squash
integration/c8d: Skip TestBuildSquashParent
2023-10-11 18:35:55 +02:00
Sebastiaan van Stijn
db806533e6 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230911190601-f082dd7a0cee
full diff: 12f0c246fe...f082dd7a0c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:54:24 +02:00
Sebastiaan van Stijn
b81b27c21f vendor: github.com/go-logr/logr v1.2.4
full diff: https://github.com/go-logr/logr/compare/v1.2.3...v1.2.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:54:24 +02:00
Sebastiaan van Stijn
cff4f20c44 migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Sebastiaan van Stijn
e1d0aacb6a vendor: github.com/docker/distribution v2.8.3
- Fix storageDriver gcs not registered in binaries
- reference: replace uses of deprecated function SplitHostname
- Dont parse errors as JSON unless Content-Type is set to JSON
- update to go1.20.8
- Set Content-Type header in registry client ReadFrom
- deprecate reference package, migrate to github.com/distribution/reference
- digestset: deprecate package in favor of go-digest/digestset
- Do not close HTTP request body in HTTP handler

full diff: https://github.com/distribution/distribution/compare/v2.8.2...v2.8.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:11:25 +02:00
Sebastiaan van Stijn
d7caea206e Merge pull request #46585 from cpuguy83/fix_etwlogs
Revert "daemon/logger/etwlogs: rewrite to use go-winio/pkg/etw"
2023-10-11 17:07:50 +02:00
Sebastiaan van Stijn
2835d1f7b2 Merge pull request #46603 from akerouanton/libnet-bridge-internal
libnet/d/bridge: Don't set container's gateway when network is internal
2023-10-11 17:07:02 +02:00
Sebastiaan van Stijn
26c5d1ea0d Merge pull request #46551 from akerouanton/libnet-resolver-otel
libnet: add OTEL tracing to the embedded DNS
2023-10-11 17:03:30 +02:00
Sebastiaan van Stijn
f0698dae14 Merge pull request #46590 from vvoland/c8d-tag-ctx-withoutcancel
daemon/c8d: Use WithoutCancel instead of context.Background
2023-10-11 16:55:16 +02:00
Paweł Gronowski
0cd02ab1b4 integration/c8d: Skip TestBuildSquashParent
`docker build --squash` is an experimental feature which is not
implemented for containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-11 15:45:43 +02:00
Paweł Gronowski
e78152aecb Merge pull request #46605 from vvoland/c8d-load-compressed
c8d/load: Handle compressed archives
2023-10-11 13:48:30 +02:00
Paweł Gronowski
aca9ea42bd vendor: github.com/moby/buildkit 0.12.3-dev
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-11 11:44:03 +02:00
Paweł Gronowski
d871a665de daemon/RegistryHosts: Don't lose mirrors
`docker.io` is present in the `IndexConfigs` so the `Mirrors` property
would get lost because a fresh `RegistryConfig` object was created.

Instead of creating a new object, reuse the existing one and just
mutate its fields.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-11 11:43:54 +02:00
Sebastiaan van Stijn
a67b7554c8 Merge pull request #46595 from vvoland/c8d-rmi-untaggeddangling
c8d/rmi: Don't emit Untagged for dangling images
2023-10-10 18:45:21 +02:00
Sebastiaan van Stijn
8d21af5be3 Merge pull request #46612 from rumpl/c8d-push-distib-source-label
c8d: Simplify error handling and distribution source label
2023-10-10 18:41:58 +02:00
Sebastiaan van Stijn
fe30e5730f Merge pull request #46611 from thaJeztah/bump_containerd_1.7.7
vendor: github.com/containerd/containerd v1.7.7
2023-10-10 16:55:21 +02:00
Sebastiaan van Stijn
a293cd6ab9 vendor: github.com/containerd/containerd v1.7.7
full diff: https://github.com/containerd/containerd/compare/v1.7.6...v1.7.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-10 14:47:44 +02:00
Djordje Lukic
f3aa9e151d c8d: Simplify error handling and distribution source label
Extract the distribution source label append into its own function and
make it not fail on any error, we do still log the error.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-10 14:13:34 +02:00
Paweł Gronowski
5f062032f7 TestPullNonExistingImage: Move to integration
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-10 13:36:29 +02:00
Paweł Gronowski
53345fe357 TestPullNonExistingImage: Use subtests
Rewrite the test to replace manual goroutines handling with paralell
subtests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-10 13:35:35 +02:00
Sebastiaan van Stijn
2edb858b18 vendor: golang.org/x/net v0.13.0, golang.org/x/crypto v0.11.0
full diff: https://github.com/golang/net/compare/v0.10.0...v0.13.0
full diff: https://github.com/golang/crypto/compare/v0.9.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-10 12:35:40 +02:00
Sebastiaan van Stijn
87de23a6cd vendor: golang.org/x/text v0.11.0
full diff: https://github.com/golang/text/compare/v0.9.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-10 12:31:52 +02:00
Djordje Lukic
23f2e0fea7 Skip tests that check max concurrency
These are not yet implemented with containerd snapshotters. We skip them
now because implementing this is not trivial with containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-10 11:36:36 +02:00
Albin Kerouanton
37ca57e9d5 libnet/d/bridge: inline error checks
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-10 10:46:44 +02:00
Sebastiaan van Stijn
3149c3efe6 Merge pull request #46588 from vvoland/c8d-history-ids
c8d/history: Fill ID and Tags for all entries
2023-10-10 10:05:01 +02:00
Paweł Gronowski
0a3fc95be6 Merge pull request #46604 from akerouanton/upgrade-delve
Dockerfile: upgrade Delve
2023-10-09 14:23:14 +02:00
Paweł Gronowski
fcabc48ee4 c8d/load: Handle compressed archives
Allow to load images from compressed tars.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-09 14:03:22 +02:00
Albin Kerouanton
cbc2a71c27 libnet/d/bridge: Don't set container's gateway when network is internal
So far, internal networks were only isolated from the host by iptables
DROP rules. As a consequence, outbound connections from containers would
timeout instead of being "rejected" through an immediate ICMP dest/port
unreachable, a TCP RST or a failing `connect` syscall.

This was visible when internal containers were trying to resolve a
domain that don't match any container on the same network (be it a truly
"external" domain, or a container that don't exist/is dead). In that
case, the embedded resolver would try to forward DNS queries for the
different values of resolv.conf `search` option, making DNS resolution
slow to return an error, and the slowness being exacerbated by some libc
implementations.

This change makes `connect` syscall to return ENETUNREACH, and thus
solves the broader issue of failing fast when external connections are
attempted.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-09 13:57:54 +02:00
Albin Kerouanton
491758a446 Dockerfile: upgrade Delve
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-09 13:13:43 +02:00
Régis Behmo
c587ba3422 registry: allow mirror path prefix in config
Path prefixes were originally disallowed in the `--registry-mirrors`
option because the /v1 endpoint was assumed to be at the root of the
URI. This is no longer the case in v2.

Close #36598

Signed-off-by: Régis Behmo <regis@behmo.com>
2023-10-07 17:33:12 +02:00
Albin Kerouanton
2c4551d86d libnet: resolver: remove direct use of logrus
This causes logs written through `r.log(ctx)` to not end in OTEL traces.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-06 19:14:48 +02:00
Albin Kerouanton
4de8459265 libnet: add OTEL tracing to the embedded DNS
This change creates a few OTEL spans and plumb context through the DNS
resolver and DNS backends (ie. Sandbox and Network). This should help
better understand how much lock contention impacts performance, and
help debug issues related to DNS queries (we basically have no
visibility into what's happening here right now).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-06 19:14:48 +02:00
Paweł Gronowski
b8ba263099 c8d/rmi: Handle explicit dangling name
This isn't something that user should do, but technically the dangling
images exist in the image store and user can pass its name (`moby-dangling@digest`).
Change it so rmi now recognizes that it's actually a dangling image and
doesn't handle it like a regular tagged image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-06 10:20:37 +02:00
Paweł Gronowski
25a813e924 c8d/rmi: Don't emit Untagged for dangling images
A dangling image isn't really a tag.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-06 10:20:35 +02:00
Brian Goff
f6fa56194f Merge pull request #46448 from xinfengliu/improve-stats-collector
Make one-shot stats faster
2023-10-05 08:10:29 -07:00
Paweł Gronowski
ee77ba5e49 Merge pull request #46593 from vvoland/integration-cli-TestDaemonNoSpaceLeftOnDeviceError-new-containerd
TestDaemonNoSpaceLeftOnDeviceError: Adjust to snapshotters
2023-10-05 11:52:22 +02:00
Paweł Gronowski
f99c4ec798 c8d/history: Fill ID and Tags for all entries
Find the parent images by image set by the legacy builder.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 16:01:17 +02:00
Paweł Gronowski
1f6bacc486 TestDaemonNoSpaceLeftOnDeviceError: Adjust to snapshotters
- Pass empty containerd socket which forces the daemon to create a new
  supervised containerd. Otherwise a global containerd daemon will be
  used and the pulled image data will be stored in its data directory,
  instead of the the newly specified `data-root` that has a limited
  storage capacity.

- Don't try to use `vfs` snapshotter, instead use `native` which is
  containerd's equivalent for `vfs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 15:59:30 +02:00
Paweł Gronowski
28d057cb0e c8d/TestPsListContainersSize: Only check if size increased
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 14:29:51 +02:00
Paweł Gronowski
34400dd4a2 TestPsListContainersSize: Fix size parsing/formatting
Use go-units to parse/format.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 14:29:47 +02:00
Paweł Gronowski
e0e6ae741e daemon/c8d: Use WithoutCancel instead of context.Background
Instead of passing a completely fresh context without any values, just
discard the cancellation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 13:17:51 +02:00
Brian Goff
c47cdd5f39 Revert "daemon/logger/etwlogs: rewrite to use go-winio/pkg/etw"
This reverts commit a9fa147a92.
The commit is unfortunately broken as it is still using `providerHandle`
to write events but that handle is never actually set, so it is always
invalid. All logging fails.

Note: This is note a straight revert due to the change to
containerd/log.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-10-03 22:42:36 +00:00
Sebastiaan van Stijn
bc2c8279de Merge pull request #46582 from vvoland/c8d-push-defer-noncancellable-context
daemon/c8d: Use non cancellable context in defers
2023-10-03 07:55:12 -07:00
Sebastiaan van Stijn
77a1411f9b Merge pull request #46575 from thaJeztah/dockerfile_distribution
Dockerfile: update test-registry to v2.8.3
2023-10-03 07:51:11 -07:00
Paweł Gronowski
6f27bef9fc daemon/c8d: Use non cancellable context in defers
Fixes leases not being released when operation was cancelled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-03 15:27:40 +02:00
Paweł Gronowski
77f7c83114 c8d/push: Show progress only on blobs
To match the graphdriver's push behavior which only shows the progress
for layers.
Exclude indexes, manifests and image configs from the push progress.
Don't explicitly check for `IsLayerType` to also handle other
potentially big blobs (like buildkit attestations).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-03 14:59:19 +02:00
Sebastiaan van Stijn
e8a7a9435f Dockerfile: update test-registry to v2.8.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-02 11:33:14 -07:00
Sebastiaan van Stijn
591c98cdbe Merge pull request #46573 from cpuguy83/double_trace_logs
Remove duplicated trace logs
2023-10-01 06:50:29 +02:00
Brian Goff
5b16dd6469 Remove duplicated trace logs
This was mistakenly added to bklog.
Since this is getting attached to the standard logger, and bklog is
using the standard logger, we only need this added once.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-30 22:59:47 +00:00
Akihiro Suda
ad877271f3 Limit OOMScoreAdj when running in UserNS ("Rootful-in-Rootless")
Fix issue 46563 "Rootful-in-Rootless dind doesn't work since systemd v250 (due to oom score adj)"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-09-29 21:50:08 +09:00
Djordje Lukic
7d30dafd69 Merge pull request #46543 from rumpl/c8d-fix-manifest-platform 2023-09-29 13:58:57 +02:00
Djordje Lukic
4a276afeb5 Merge pull request #46519 from rumpl/c8d-skip-config-image 2023-09-29 13:57:48 +02:00
Xinfeng Liu
3d70af4ee4 adjust stats sampling order
Signed-off-by: Xinfeng Liu <XinfengLiu@icloud.com>
2023-09-29 15:33:48 +08:00
Sebastiaan van Stijn
e2c23ed57d Merge pull request #46069 from thaJeztah/update_to_go1.21
update to go1.21.1, default to GOTOOLCHAIN=local
2023-09-28 20:24:21 +02:00
Sebastiaan van Stijn
1239c96413 Merge pull request #46562 from neersighted/go_mod_bump
vendor.mod: bump go line to 1.20
2023-09-28 20:18:42 +02:00
Sebastiaan van Stijn
c4f55b53b0 Merge pull request #46561 from vvoland/c8d-integrationcli-skipsome
c8d: Adjust some integration-cli tests
2023-09-28 18:46:50 +02:00
Bjorn Neergaard
16063c7456 vendor.mod: bump go line to 1.20
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-28 10:01:25 -06:00
Paweł Gronowski
a4822abe37 c8d: Skip TestPullFailsWithAlteredManifest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-28 17:13:38 +02:00
Paweł Gronowski
f9f439763a c8d/TestPullFailsWithAlteredManifest: Adjust error message
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-28 17:13:36 +02:00
Paweł Gronowski
4001241ccf c8d/TestPullByDigestNoFallback: Adjust error message
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-28 17:13:33 +02:00
Marat Radchenko
9e3ed7b429 pkg/chrootarchive: fix Darwin build
Before this commit, `doPack`, `doUnpack` and `doUnpackLayer` were not implemented for Darwin, causing build failure.

This change allows all non-Linux Unixes to use FreeBSD reexec-based pack/unpack implementation

See also: moby/buildkit#4059
See also: 8b843732b3

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2023-09-28 16:23:33 +03:00
Sebastiaan van Stijn
e465ebf2f3 update to go1.21.1, and fix download-URL
This required changes to the download-URL, as downloads are now provided
using the full version (including the `.0` patch version);

    curl -sI https://go.dev/dl/go1.21.windows-amd64.zip | grep 'location'
    location: https://dl.google.com/go/go1.21.windows-amd64.zip

    curl -sI https://dl.google.com/go/go1.21.windows-amd64.zip
    HTTP/2 404
    # ...

    curl -sI https://dl.google.com/go/go1.21.0.windows-amd64.zip
    HTTP/2 200
    # ...

Unfortunately this also means that the GO_VERSION can no longer be set to
versions lower than 1.21.0 (without additional changes), because older
versions do NOT provide the `.0` version, and Go 1.21.0 and up, no longer
provides URLs _without_ the `.0` version.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-28 00:51:34 +02:00
Sebastiaan van Stijn
aa282973d4 Dockerfile: use GOTOOLCHAIN=local
Related discussion in https://github.com/docker-library/golang/issues/472

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-28 00:51:18 +02:00
Djordje Lukic
f66c000229 c8d: Fix getting the platform of a single-platform image
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-28 00:16:51 +02:00
Sebastiaan van Stijn
a1d966c492 Merge pull request #46555 from thaJeztah/distribution_test_fixes
fix faulty tests for distribution-client
2023-09-27 17:06:01 +02:00
Paweł Gronowski
7c0c525764 Merge pull request #46552 from vvoland/internal-compatcontext
internal: Add compatcontext.WithoutCancel
2023-09-27 16:50:18 +02:00
Sebastiaan van Stijn
2c0ad62b24 Merge pull request #46554 from thaJeztah/remove_intermediates
remove some intermediate vars, and small refactor for error-handling
2023-09-27 16:06:46 +02:00
Sebastiaan van Stijn
2c89640ab9 distribution: TestPullSchema2Config fix test response
The test was depending on the client constructing an error based on the
http-status code, and the client not reading the response body if the
response was not a JSON response.

This fix;

- adds the correct content-type headers in the response
- includes error-messages in the response
- adds additional tests to cover both the plain (non-JSON) and JSON
  error responses, as well as an empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 14:03:59 +02:00
Sebastiaan van Stijn
a9fcb775e5 integration-cli: fix getTestTokenService not sending header
This utility was setting the content-type header after WriteHeader was
called, and the header was not sent because of that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 13:57:48 +02:00
Sebastiaan van Stijn
3197160114 daemon: Daemon.SetNetworkBootstrapKeys: make error-handling idiomatic
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:08:28 +02:00
Sebastiaan van Stijn
f91c3cfda6 daemon: Daemon.initNetworkController: remove intermediate var
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:08:28 +02:00
Sebastiaan van Stijn
dcc75e1563 libnetwork: Controller: agentInit, agentDriverNotify rm intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:08:28 +02:00
Sebastiaan van Stijn
a384102fdf libnetwork/datastore: Store.Map, Store.List: remove intermediate vars
Inline the closures, and rename a var to be more descriptive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:07:31 +02:00
Sebastiaan van Stijn
bb5402e6fb libnetwork: Controller: getKeys, getPrimaryKeyTag: slight refactor
- use named return variables to make the function more self-describing
- rename variable for readability
- slightly optimize slice initialization, and keep linters happy

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:01:54 +02:00
Sebastiaan van Stijn
603f49706e libnetwork: Controller: getKeys, getPrimaryKeyTag: prevent panic
Prevent potential panics if we don't have the expected number of keys
for the subsystem.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:01:54 +02:00
Paweł Gronowski
f6e44bc0e8 internal: Add compatcontext.WithoutCancel
Copy the implementation of `context.WithoutCancel` introduced in Go 1.21
to be able to use it when building with older versions.
This will use the stdlib directly when building with Go 1.21+.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-27 11:46:30 +02:00
Sebastiaan van Stijn
605c8fb75d Merge pull request #46546 from thaJeztah/libnetwork_return_errs
libnetwork: Controller.cleanupLocalEndpoints, sandboxCleanup: return errors
2023-09-27 10:31:56 +02:00
Sebastiaan van Stijn
324cb3d08f Merge pull request #46545 from thaJeztah/libnetwork_NetworkByID_simplify
libnetwork: Controller.NetworkByID: remove redundant error-handling
2023-09-27 10:30:47 +02:00
Sebastiaan van Stijn
f3143745b2 Merge pull request #46547 from thaJeztah/libnetwork_store_nolock
libnetwork: Controller: remove mutex for "store"
2023-09-27 10:23:32 +02:00
Sebastiaan van Stijn
b1855bb4af Merge pull request #46548 from thaJeztah/libnetwork_inline_populateSpecial
libnetwork: inline populateSpecial NetworkWalker
2023-09-27 10:13:15 +02:00
Sebastiaan van Stijn
479079c6b1 Merge pull request #46549 from thaJeztah/nwAgent_unexport_mutex
libnetwork: nwAgent: un-export mutex
2023-09-27 10:12:23 +02:00
Xinfeng Liu
95aea39348 Make one-shot stats faster
This commit moves one-shot stats processing out of the publishing
channels, i.e. collect stats directly.

Also changes the method of getSystemCPUUsage() on Linux to return
number of online CPUs also.

Signed-off-by: Xinfeng Liu <XinfengLiu@icloud.com>
2023-09-27 14:10:23 +08:00
Bjorn Neergaard
97e28de7e2 Merge pull request #46544 from akerouanton/api-otel-operation
api: Add method and path to trace operation string
2023-09-26 13:07:03 -06:00
Sebastiaan van Stijn
618d9b5d54 libnetwork: nwAgent: un-export mutex
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:46:27 +02:00
Sebastiaan van Stijn
7cda3fb7b5 libnetwork: inline populateSpecial NetworkWalker
It was only used in a single place, and it was defined far away from
where it was used.

Move the code inline, so that it's clear at a glance what it's doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:41:50 +02:00
Sebastiaan van Stijn
ca1307c56e libnetwork: Controller: remove mutex for "store"
The store field is only mutated by Controller.initStores(), which is
only called inside the cosntructor (libnetwork.New), so there should be
no need to protect the field with a mutex in non-exported functions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:34:12 +02:00
Sebastiaan van Stijn
a8ea752a93 libnetwork: Controller.cleanupLocalEndpoints: return errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:28:18 +02:00
Sebastiaan van Stijn
2e60051c92 libnetwork: Controller.sandboxCleanup: return errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:28:18 +02:00
Sebastiaan van Stijn
642cf261a8 libnetwork: Controller.NetworkByID: remove redundant error-handling
Controller.getNetworkFromStore() already returns a ErrNoSuchNetwork if
no network was found, so we don't need to convert the existing error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:22:52 +02:00
Sebastiaan van Stijn
e3975fba84 Merge pull request #46530 from thaJeztah/libnetwork_walkless_step1
libnetwork: assorted cleanups in Sandbox
2023-09-26 19:19:31 +02:00
Sebastiaan van Stijn
7f9cce2b50 Merge pull request #46541 from akerouanton/delve-integration-tests
hack: Run integration tests through Delve
2023-09-26 19:05:48 +02:00
Albin Kerouanton
430a5f8581 api: Add method and path to trace operation string
Currently, all traces coming from the API have an empty operation
string, which make them indistinguishable from each other without looking
at the logs of the root span, and prevent proper filtering on Jaeger UI.

With this change, traces get the route pattern as the operation string.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-26 18:56:25 +02:00
Albin Kerouanton
41c186c344 hack: Run integration tests through Delve
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-26 15:42:11 +02:00
Sebastiaan van Stijn
b94e88c1e2 Merge pull request #46386 from dperny/add-swarm-seccomp-apparmor
Add support for swarm seccomp and apparmor
2023-09-26 09:45:16 +02:00
Drew Erny
42a51cb285 Add support for swarm seccomp and apparmor
And also no-new-privileges

Signed-off-by: Drew Erny <derny@mirantis.com>
2023-09-25 12:38:26 -05:00
Sebastiaan van Stijn
bd7b27b5c8 Merge pull request #46535 from vvoland/integration-cli-saveload-dont-use-repositories
integration-cli/SaveLoad: Don't check `repositories` file
2023-09-25 17:22:28 +02:00
Paweł Gronowski
3d62ab9585 Merge pull request #46536 from vvoland/integration-cli-saveload-repoflags
integration-cli/TestSaveAndLoadRepoFlags: Ignore LastTagTime difference
2023-09-25 11:59:37 +02:00
Paweł Gronowski
7f6bd671fe Merge pull request #46484 from vchiranjeeviak/vchiranjeeviak/container-list-code-refactor
daemon: containers list code refactor
2023-09-25 11:34:51 +02:00
Chiranjeevi Tirunagari
0c66111c40 daemon: containers list code refactor
Signed-off-by: Chiranjeevi Tirunagari <vchiranjeeviak.tirunagari@gmail.com>
2023-09-25 10:28:31 +05:30
Djordje Lukic
51f0e7b0e4 Merge pull request #46534 from vvoland/c8d-integration-cli-saveload-skip-parent 2023-09-22 16:49:23 +02:00
Paweł Gronowski
13b775a96e integration-cli/TestSaveAndLoadRepoFlags: Ignore LastTagTime difference
The reason it doesn't change with the graphdrivers is caused by an
implementation detail and the fact that the image is loaded into the
same daemon it was saved from.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 16:35:07 +02:00
Paweł Gronowski
7c202d2fa4 integration-cli/SaveLoad: Don't check repositories file
Rewrite TestSaveMultipleNames and TestSaveSingleTag  so that they don't
use legacy `repositories` file (which isn't present in the OCI
archives).
`docker save` output is now OCI compatible, so we don't need
to use the legacy file.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:42:50 +02:00
Paweł Gronowski
724db8650c integration-cli: Skip TestLoadZeroSizeLayer with c8d
The input archive is in the old Docker format that's not OCI compatible
and is not supported by the containerd archive import:
```
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/VERSION
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/json
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/layer.tar
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/VERSION
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/json
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/layer.tar
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/VERSION
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/json
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/layer.tar
repositories
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:34:02 +02:00
Paweł Gronowski
1337a66130 integration-cli: Skip Parent tests with c8d
Parent property is not available in the OCI config.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:33:47 +02:00
Sebastiaan van Stijn
3614749b55 Merge pull request #45966 from neersighted/buildkit_0.12
Update to BuildKit 0.12
2023-09-22 02:13:15 +02:00
Sebastiaan van Stijn
90b756b2a1 Merge pull request #46525 from thaJeztah/daemon_typed_logformat
daemon: strongly type containerd log.OutputFormat
2023-09-21 22:27:02 +02:00
CrazyMax
d5b067e04a ci(buildkit): remove regex skipping tests with digest inconsistency
Skipping digest-related tests is no longer necessary after 4065831652

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:25:28 -06:00
CrazyMax
286704bf6f ci(buildkit): expose github runtime for gha tests
This exposes `ACTIONS_RUNTIME_TOKEN` and `ACTIONS_CACHE_URL`, which are
used to skip cache exporter tests, when combined with
a8789cbd4a

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:25:10 -06:00
Bjorn Neergaard
34e71b374b builder-next/a/ci/pull: strongly type flightcontrol.Group
Now that this is a generic, we can define a struct type at the package
level, and remove the casting logic necessary when we had to use
interface{}.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:57 -06:00
Bjorn Neergaard
c8773e10b1 builder-next/wrapper: use new exptypes constants
Introduced in 4fc2d7b5e7

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:55 -06:00
Bjorn Neergaard
a728bd0ee9 builder-next/pull: use ResolveMode.String() helper
Introduced years ago in 6644e1b0da

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:52 -06:00
Bjorn Neergaard
221b73e23d builder-next: apply source policies on config resolve
SourcePolicy was accounted for in 330cf7ae7d

TODO: replace applySourcePolicies with BuildKit's implementation, which
is currently unexported.

Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:50 -06:00
Bjorn Neergaard
2246297ae6 builder-next: adopt new wrapped types
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:

* containerd's LeaseManager and ContentStore have to be a
  (namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
  bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:48 -06:00
Bjorn Neergaard
c217e3c87a vendor: github.com/moby/buildkit v0.12.2
The following changes were required:
* integration/build: progressui's signature changed in 6b8fbed01e
* builder-next: flightcontrol.Group has become a generic type in 8ffc03b8f0
* builder-next/executor: add github.com/moby/buildkit/executor/resources types, necessitated by 6e87e4b455
* builder-next: stub util/network/Namespace.Sample(), necessitated by 963f16179f

Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:45 -06:00
Bjorn Neergaard
8cd5f04ea3 daemon/c8d: use new containerd LabelDistributionSource constant
Introduced in dd3eedf3c3

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:42 -06:00
Bjorn Neergaard
fd6dd6935b vendor: github.com/containerd/containerd v1.7.6
The DeepEqual ignore required in the daemon tests is a bit ugly, but it
works given the new protoc output.

We also have to ignore lints related to schema1 deprecations; these do
not apply as we must continue to support this schema version.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:40 -06:00
Bjorn Neergaard
79a4cbbec9 builder-next: make stub executor generic
The current executor is only tested on Linux, so let's be honest about
that. Stubbing this correctly helps avoid incorrectly trying to call
into Linux-only code in e.g. libnetwork.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:27 -06:00
Sebastiaan van Stijn
d7a31cfb2d libnetwork: Sandbox.resolveName: slightly simplify locking
Simplify the lock/unlock cycle, and make the "lookupAlias" branch
more similar to the non-lookupAlias variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:36 +02:00
Sebastiaan van Stijn
f549aaa205 libnetwork: Sandbox.resolveName: add fast-path for alias lookups
Skip faster when we're looking for aliases. Also check for the list
of aliases to be empty, not just `nil` (although in practice it should
be equivalent).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
9249b34be8 libnetwork: Sandbox.resolveName: rename vars for clarity
- use `nameOrAlias` for the name (or alias) to resolve
- use `lookupAlias` to indicate what the intent is; this function
  is either looking up aliases or "regular" names. Ideally we would
  split the function, but let's keep that for a future exercise.
- name the `ipv6Miss` output variable. The "ipv6 miss" logic is rather
  confusing, and should probably be revisited, but let's start with
  giving the variable a name to make it more apparent what it is.
- use `nw` for networks, which is the more common local name

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
4401ccac22 libnetwork: Sandbox: remove some intermediate vars
- remove some intermediate vars, or move them closer to where they're used.
- ResolveService: use strings.SplitN to limit number of elements. This
  code is only used to validate the input, results are not used.
- ResolveService: return early instead of breaking the loop. This makes
  it clearer from the code that were not returning anything (nil, nil).
- Controller.sandboxCleanup(): rename a var, and slight refactor of
  error-handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
d3afa80b96 Merge pull request #46348 from thaJeztah/refactor_isClusterEligible
libnetwork: rewrite Network.isClusterEligible to return agent
2023-09-21 14:31:31 +02:00
Bjorn Neergaard
0e80073e01 daemon: strongly type containerd log.OutputFormat
This type was introduced in
0a79e67e4f

Make use of it throughout our log-format handling code, and convert back
to a string before we pass it to the containerd client.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 05:40:17 -06:00
Sebastiaan van Stijn
0d9da7367d Merge pull request #46522 from thaJeztah/update_hcsshim
vendor: github.com/Microsoft/hcsshim v0.11.1
2023-09-21 10:25:18 +02:00
Sebastiaan van Stijn
4ff252456b libnetwork: rewrite Network.isClusterEligible to return agent
This function was used to check if the network is a multi-host, swarm-scoped
network. Part of this check involved a check whether the cluster-agent was
present.

In all places where this function was used, the next step after checking if
the network was "cluster eligible", was to get the agent, and (again) check
if it was not nil.

This patch rewrites the isClusterEligible utility into a clusterAgent utility,
which both checks if the network is cluster-eligible, and returns the agent
(if set). For convenience, an "ok" bool is added, which callers can use to
return early (although just checking for nilness would likely have been
sufficient).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:21 +02:00
Sebastiaan van Stijn
6203e3660d libnetwork: Endpoint: return early if no agent was found
This removes redundant nil-checks in Endpoint.deleteServiceInfoFromCluster
and Endpoint.addServiceInfoToCluster.

These functions return early if the network is not ["cluster eligible"][1],
and the function used for that (`Network.isClusterEligible`) requires the
[agent to not be `nil`][2].

This check moved around a few times ([3][3], [4][4]), but was originally
added in [libnetwork 1570][5] which, among others, tried to avoid a nil-pointer
exception reported in [moby 28712][6], which accessed the `Controller.agent`
[without locking][7]. That issue was addressed by adding locks, adding a
`Controller.getAgent` accessor, and updating deleteServiceInfoFromCluster
to use a local var. It also sprinkled this `nil` check to be on the safe
side, but as `Network.isClusterEligible` already checks for the agent
to not be `nil`, this should not be redundant.

[1]: 5b53ddfcdd/libnetwork/agent.go (L529-L534)
[2]: 5b53ddfcdd/libnetwork/agent.go (L688-L696)
[3]: f2307265c7
[4]: 6426d1e66f
[5]: 8dcf9960aa
[6]: https://github.com/moby/moby/issues/28712
[7]: 75fd88ba89/vendor/github.com/docker/libnetwork/agent.go (L452)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:21 +02:00
Sebastiaan van Stijn
6eeef51c6a libnetwork: Controller.agentSetup: use structured logs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:15 +02:00
Sebastiaan van Stijn
8b95ea4a35 libnetwork: Controller.agentSetup: remove redundant condition
The function returns at the start if there agent is non-nil.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:15:06 +02:00
Sebastiaan van Stijn
3a520c1c0f Merge pull request #46521 from dmcgowan/fix-makefile-graphdriver
Fix graphdriver lookup in makefile
2023-09-21 08:43:43 +02:00
Sebastiaan van Stijn
e852ba55a5 vendor: github.com/Microsoft/hcsshim v0.11.1
full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.10...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 08:37:39 +02:00
Derek McGowan
06a640084f Fix graphdriver lookup in makefile
When graphdriver is not provided the graphdriver is looked up
from docker info, but without quotes it may fail and set the
graphdriver to an incorrect value.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-20 14:46:43 -07:00
Sebastiaan van Stijn
1ed5d91555 Merge pull request #46365 from thaJeztah/libnetwork_endpoint_nits
libnetwork: Endpoint: fixing some nits
2023-09-20 22:01:28 +02:00
Sebastiaan van Stijn
4dbfe7e17e Merge pull request #46502 from rumpl/c8d-fix-diff
c8d: Fix `docker diff`
2023-09-20 21:16:08 +02:00
Sebastiaan van Stijn
d8a51d2887 Merge pull request #46463 from thaJeztah/fix_version_checks
client: negotiate api version before handling version-specific code
2023-09-20 19:48:32 +02:00
Djordje Lukic
093481618d c8d: Skip the test that checks Config.Image
It's not set when containerd is used as an image store and buildkit
never sets it either, so let's skip this test if snapshotters are used

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-20 17:01:42 +02:00
Djordje Lukic
7444b88f5d Merge pull request #46495 from rumpl/c8d-fix-create-platform 2023-09-20 16:39:38 +02:00
Paweł Gronowski
7c865c156f Merge pull request #46515 from vvoland/c8d-pull-fix-progress-orderd
c8d/pull: Fix stale Downloading progress
2023-09-20 16:38:39 +02:00
Sebastiaan van Stijn
e061793212 Merge pull request #46315 from thaJeztah/libnetwork_remove_sandbox_interface_step2
libnetwork/osl: Namespace: assorted cleanups
2023-09-20 16:13:15 +02:00
Sebastiaan van Stijn
e6907243af client: negotiate api version before handling version-specific code
We try to perform API-version negotiation as lazy as possible (and only execute
when we are about to make an API request). However, some code requires API-version
dependent handling (to set options, or remove options based on the version of the
API we're using).

Currently this code depended on the caller code to perform API negotiation (or
to configure the API version) first, which may not happen, and because of that
we may be missing options (or set options that are not supported on older API
versions).

This patch:

- splits the code that triggered API-version negotiation to a separate
  Client.checkVersion() function.
- updates NewVersionError to accept a context
- updates NewVersionError to perform API-version negotiation (if enabled)
- updates various Client functions to manually trigger API-version negotiation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 15:23:27 +02:00
Djordje Lukic
3936c57294 c8d: test a backend dependent error on pull
moby and containerd have slightly different error messages when someone
tries to pull an image that doesn't contain the current platform,
instead of looking inside the error returned by containerd we match the
errors in the test related to what image backend we are using

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-20 14:47:51 +02:00
Djordje Lukic
207c4d537c c8d: Fix docker diff
Diffing a container yielded some extra changes that come from the
files/directories that we mount inside the container (/etc/resolv.conf
for example). To avoid that we create an intermediate snapshot that has
these files, with this we can now diff the container fs with its parent
and only get the differences that were made inside the container.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-20 14:16:22 +02:00
Paweł Gronowski
601c9b29e6 c8d/pull: Fix stale Downloading progress
Final progress messages were sent after the progress updater finished
which made the "Downloading" progress not being updated into "Download
complete".

Fix by sending the final messages after the progress has finished.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-20 14:10:58 +02:00
Sebastiaan van Stijn
313a090c0e libnetwork/osl: add some TODOs
These came up during review of a refactor, and need further investigating.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:45:45 +02:00
Sebastiaan van Stijn
9d3b1f9419 libnetwork/osl: make constructing Interfaces more atomic
It's still not "great", but implement a `newInterface()` constructor
to create a new Interface instance, instead of creating a partial
instance and applying "options" after the fact.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:45:40 +02:00
Sebastiaan van Stijn
47f9e70385 libnetwork/osl: Namespace.Restore: conditionally fetch IPs
We're only using the results if the interface doesn't have an address
yet, so skip this step if we don't use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:27 +02:00
Sebastiaan van Stijn
ee5a91e663 libnetwork/osl: Namespace.Restore: flatten nested conditions
Flatten some nested "if"-statements, and improve error.

Errors returned by this function are not handled, and only logged, so
make them more informative if debugging is needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:27 +02:00
Sebastiaan van Stijn
299bd58c5a libnetwork/osl: Namespace.Restore: rename vars for readability
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:27 +02:00
Sebastiaan van Stijn
7b96663082 libnetwork/osl: Namespace: inline setGateway and setGatewayIPv6
They were not consistently used, and the locations where they were
used were already "setters", so we may as well inline the code.

Also updating Namespace.Restore to keep the lock slightly longer,
instead of locking/unlocking for each property individually, although
we should consider to keep the long for the duration of the whole
function to make it more atomic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
bd17d27658 libnetwork/osl: Namespace: make error-handling more idiomatic
Check for non-nil errors (and return early) instead of the reverse.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
0b4a70ca2c libnetwork/osl: Namespace: programRoute, removeRoute rm path arg
Remove the argument, because it was not used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
542fe0da40 libnetwork/osl: Namespace: make mutex private
Make the mutex internal to the Namespace; locking/unlocking should not
be done externally, and this makes it easier to see where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
338fc49060 libnetwork/osl: implement Namespace.RemoveInterface
Interface.Remove() was directly accessing Namespace "internals", such
as locking/unlocking. Move the code from Interface.Remove() into the
Namespace instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:34:47 +02:00
Sebastiaan van Stijn
7cfb81ba04 Merge pull request #46342 from thaJeztah/libnetwork_nwAgent_ip
libnetwork: nwAgent.bindAddr: change to net.IP
2023-09-20 10:27:06 +02:00
Sebastiaan van Stijn
34dc2f10b4 Merge pull request #46479 from vvoland/pull-check-reserved-name-in-api
api/pull: Move reference parsing from imageService, validate repo
2023-09-19 17:17:31 +02:00
Sebastiaan van Stijn
54d84229ce Merge pull request #46511 from rumpl/c8d-system-df-image-containers
c8d: Number of containers using an image
2023-09-19 17:08:01 +02:00
Sebastiaan van Stijn
1c34831291 Merge pull request #46513 from vvoland/c8d-pull-pretty-jws
c8d/pull: Support legacy schema1 prettyjws manifests
2023-09-19 17:03:50 +02:00
Paweł Gronowski
3ff75aeb07 c8d/pull: Deprecation notice for schema1 manifests
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-19 15:46:42 +02:00
Paweł Gronowski
2cb94ade6c distribution: Extract schema1 deprecation notice
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-19 15:39:05 +02:00
Sebastiaan van Stijn
3350f815c7 Merge pull request #46480 from thaJeztah/remove_remote_endpoints
libnetwork: remove some dead code around netWatch
2023-09-19 14:00:12 +02:00
Djordje Lukic
39fe25b69e c8d: Fix getting image with platform
We weren't checking for the asked platform in the case the image was a
manifest, only if it was a manifest list.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-19 13:54:24 +02:00
Paweł Gronowski
68b67b3230 c8d/pull: Support legacy schema1 prettyjws manifests
Makes it possible to pull `application/vnd.docker.distribution.manifest.v1+prettyjws`
legacy manifests.

They are not stored in their original form but are converted to the OCI
manifests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-19 13:17:12 +02:00
Paweł Gronowski
cf5657d4bb Merge pull request #46512 from rumpl/run-once
Don't run tests twice
2023-09-19 12:56:22 +02:00
Djordje Lukic
d5425f28ae Don't run tests twice
It's useless

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-19 11:37:55 +02:00
Bjorn Neergaard
bddd826d7a profiles/apparmor: deny /sys/devices/virtual/powercap
While this is not strictly necessary as the default OCI config masks this
path, it is possible that the user disabled path masking, passed their
own list, or is using a forked (or future) daemon version that has a
modified default config/allows changing the default config.

Add some defense-in-depth by also masking out this problematic hardware
device with the AppArmor LSM.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-18 16:41:03 -06:00
Bjorn Neergaard
83cac3c3e3 oci/defaults: deny /sys/devices/virtual/powercap
The ability to read these files may offer a power-based sidechannel
attack against any workloads running on the same kernel.

This was originally [CVE-2020-8694][1], which was fixed in
[949dd0104c496fa7c14991a23c03c62e44637e71][2] by restricting read access
to root. However, since many containers run as root, this is not
sufficient for our use case.

While untrusted code should ideally never be run, we can add some
defense in depth here by masking out the device class by default.

[Other mechanisms][3] to access this hardware exist, but they should not
be accessible to a container due to other safeguards in the
kernel/container stack (e.g. capabilities, perf paranoia).

[1]: https://nvd.nist.gov/vuln/detail/CVE-2020-8694
[2]: 949dd0104c
[3]: https://web.eece.maine.edu/~vweaver/projects/rapl/

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-18 16:40:03 -06:00
Sebastiaan van Stijn
5d87dc98ea Merge pull request #46507 from rumpl/c8d-fix-tests
c8d: Skip TestRemoveImageGarbageCollector
2023-09-18 23:06:00 +02:00
Sebastiaan van Stijn
f13065b1f7 Merge pull request #46183 from akerouanton/validate-NetworkingConfig
daemon: Improve NetworkingConfig & EndpointSettings validation
2023-09-18 22:09:29 +02:00
Sebastiaan van Stijn
a24d254397 Merge pull request #46436 from thaJeztah/search_remove_unused_bits
registry: assorted fixes for search
2023-09-18 21:28:33 +02:00
Sebastiaan van Stijn
07e28a968b Merge pull request #46509 from thaJeztah/search_no_jsonstream
registry: remove jsonmessage dependency
2023-09-18 21:26:57 +02:00
Djordje Lukic
2394b07bce c8d: Number of containers using an image
Return the number of containers that use an image if it was asked,
during a `docker system df` call for example.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-18 19:36:49 +02:00
Sebastiaan van Stijn
563bb58c8d Merge pull request #46510 from thaJeztah/update_buildkit_deps
update some dependencies in preparation of containerd v1.7.x and buildkit v0.12
2023-09-18 19:24:43 +02:00
Albin Kerouanton
e19e541e2c libnet: add comment to ipamType
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 18:26:56 +02:00
Albin Kerouanton
acf825def2 api/t/net: test EndpointIPAMConfig.Validate()
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 18:26:56 +02:00
Albin Kerouanton
3092b261e2 daemon: move most of validateEndpointSettings into api/t/net
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 18:26:56 +02:00
Sebastiaan van Stijn
554036040b vendor: github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24
full diff: 43070de90f...ced1acdcaa

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:18:49 +02:00
Sebastiaan van Stijn
6fc3ead321 vendor: google.golang.org/grpc v1.56.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:30 +02:00
Sebastiaan van Stijn
73041028b6 vendor: google.golang.org/protobuf v1.31.0
full diff: https://github.com/protocolbuffers/protobuf-go/compare/v1.30.0...v1.31.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:30 +02:00
Sebastiaan van Stijn
54dc8d2968 vendor: github.com/cenkalti/backoff/v4 v4.2.1
no local changes

full diff: https://github.com/cenkalti/backoff/compare/v4.2.0...v4.2.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:30 +02:00
Sebastiaan van Stijn
d97e4300ee vendor: google.golang.org/api v0.114.0
full diff:

- https://github.com/googleapis/google-api-go-client/compare/v0.110.0...v0.114.0
- https://github.com/googleapis/gax-go/compare/v2.7.0...v2.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:30 +02:00
Sebastiaan van Stijn
d50585b05b vendor: golang.org/x/tools v0.8.0
full diff: https://github.com/golang/tools/compare/v0.6.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:29 +02:00
Sebastiaan van Stijn
51d8a975ab vendor: golang.org/x/oauth2 v0.7.0
full diff: https://github.com/golang/oauth2/compare/v0.6.0...v0.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:29 +02:00
Sebastiaan van Stijn
9670db3904 vendor: golang.org/x/crypto v0.9.0
full diff: https://github.com/golang/crypto/compare/v0.3.0...v0.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:29 +02:00
Sebastiaan van Stijn
a046616249 vendor: golang.org/x/sys v0.10.0
full diff: https://github.com/golang/sys/compare/v0.8.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:29 +02:00
Sebastiaan van Stijn
f29155950f vendor: golang.org/x/sync v0.3.0
full diff: https://github.com/golang/sync/compare/v0.1.0...v0.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:29 +02:00
Sebastiaan van Stijn
0354791147 vendor: github.com/klauspost/compress v1.16.5
full diff: https://github.com/klauspost/compress/compare/v1.16.3...v1.16.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:28 +02:00
Sebastiaan van Stijn
87dd1de046 vendor: github.com/tonistiigi/vt100 v0.0.0-20230623042737-f9a4f7ef6531
fix possible panix on fast resizing

full diff: 8066bb9726...f9a4f7ef65

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:17:01 +02:00
Sebastiaan van Stijn
bd6cadffb0 vendor: github.com/containerd/continuity v0.4.2
full diff: https://github.com/containerd/continuity/compare/v0.4.1...v0.4.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:05:35 +02:00
Sebastiaan van Stijn
6d93db06db vendor: github.com/vbatts/tar-split v0.11.3
full diff: https://github.com/vbatts/tar-split/compare/v0.11.2...v0.11.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:04:26 +02:00
Sebastiaan van Stijn
171ecf8cf2 vendor: github.com/vishvananda/netns v0.0.4
full diff: https://github.com/vishvananda/netns/compare/v0.0.2...v0.0.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 18:03:23 +02:00
Albin Kerouanton
81ab8db1c3 api/t/net: add missing comment to ValidateIPAM
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:28:08 +02:00
Albin Kerouanton
04a47e88d2 api/t/net: move endpoint structs into endpoint.go
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:28:08 +02:00
Albin Kerouanton
19c07198b6 daemon: Check if endpoint address is in allowed range
This issue wasn't caught on ContainerCreate or NetworkConnect (when
container wasn't started yet).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:28:07 +02:00
Albin Kerouanton
bfd8c6deb7 daemon: Validate EndpointSettings.IPAMConfig.LinkLocalIPs
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:28:07 +02:00
Albin Kerouanton
4bd0553274 daemon: Return all validation errors for NetworkingConfig and EndpointSettings
Thus far, validation code would stop as soon as a bad value was found.
Now, we try to validate as much as we can, to return all errors to the
API client.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:25:06 +02:00
Albin Kerouanton
ff503882f7 daemon: Improve NetworkingConfig & EndpointSettings validation
So far, only a subset of NetworkingConfig was validated when calling
ContainerCreate. Other parameters would be validated when the container
was started. And the same goes for EndpointSettings on NetworkConnect.

This commit adds two validation steps:

1. Check if the IP addresses set in endpoint's IPAMConfig are valid,
   when ContainerCreate and ConnectToNetwork is called ;
2. Check if the network allows static IP addresses, only on
   ConnectToNetwork as we need the libnetwork's Network for that and it
   might not exist until NetworkAttachment requests are sent to the
   Swarm leader (which happens only when starting the container) ;

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:21:06 +02:00
Sebastiaan van Stijn
4c03618fab registry: remove jsonmessage dependency
Just return a regular error, because the API converts the error to
the expected ErrorResponse. Before/After produce the same API response:

    curl -v --unix-socket /var/run/docker.sock 'http://localhost/v1.43/images/search?term=hello'
    *   Trying /var/run/docker.sock:0...
    * Connected to localhost (/var/run/docker.sock) port 80 (#0)
    > GET /v1.43/images/search?term=hello HTTP/1.1
    > Host: localhost
    > User-Agent: curl/7.74.0
    > Accept: */*
    >
    * Mark bundle as not supporting multiuse
    < HTTP/1.1 500 Internal Server Error
    < Api-Version: 1.44
    < Content-Type: application/json
    < Docker-Experimental: false
    < Ostype: linux
    < Server: Docker/dev (linux)
    < Traceparent: 00-c38c2da5cf30305fcb66836a28e227bf-d16f4f7d2c7002a1-01
    < Date: Mon, 18 Sep 2023 14:30:18 GMT
    < Content-Length: 41
    <
    {"message":"Unexpected status code 409"}
    * Connection #0 to host localhost left intact

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 16:35:45 +02:00
Djordje Lukic
6c563cda82 c8d: fix TestRemoveContainerAfterLiveRestore
There is no need to pass the storage driver to the daemon the test
starts

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-18 15:34:20 +02:00
Paweł Gronowski
0c4397deaf api/pull: Validate repo name
Copy the check for "scratch" image pull attempt from the distribution
to the API.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-18 15:23:13 +02:00
Paweł Gronowski
ccb37fa567 imageService/PullImage: Move reference parse to api
Make `PullImage` accept `reference.Named` directly instead of
duplicating the parsing code for both graphdriver and containerd image
service implementations.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-18 15:23:12 +02:00
Bjorn Neergaard
05c60724ed Merge pull request #46506 from thaJeztah/deprecate_pkg_loopback
deprecate pkg/loopback (utility package for devicemapper)
2023-09-18 07:18:57 -06:00
Sebastiaan van Stijn
3bd3cdd82e Merge pull request #46476 from vvoland/libcontainerd-windows-reap-fix
libcontainerd/windows: Fix cleanup on `newIOFromProcess` error
2023-09-18 15:06:56 +02:00
Sebastiaan van Stijn
b94f928be6 Merge pull request #46499 from thaJeztah/libcontainer_local_vars
libcontainer: client.processEventStream: use locally scoped variables
2023-09-18 15:04:41 +02:00
Sebastiaan van Stijn
85769732c2 Merge pull request #46437 from thaJeztah/distribution_no_named
distribution: newRepository: remove naked return and intermediate vars
2023-09-18 15:02:51 +02:00
Djordje Lukic
24405e8044 Merge pull request #46494 from rumpl/c8d-fix-daemon-proxy 2023-09-18 14:43:05 +02:00
Djordje Lukic
28861e0c47 c8d: Skip TestRemoveImageGarbageCollector
This test checks how the layer store works, so we don't need it when we
use containerd as image store

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-18 14:35:57 +02:00
Sebastiaan van Stijn
bf692d47fb deprecate pkg/loopback (utility package for devicemapper)
This package was introduced in af59752712
as a utility package for devicemapper, which was removed in commit
dc11d2a2d8 (v25.0.0).

It looks like there's no external consumers of this package, so we should
consider removing it, but deprecating it first, just in case.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 13:44:44 +02:00
Sebastiaan van Stijn
a1833d8b4d Merge pull request #46492 from rumpl/c8d-tag-digest-name
c8d: Refuse images with digest algo when tagging
2023-09-17 15:58:57 +02:00
Sebastiaan van Stijn
96faee9762 libcontainer: client.processEventStream: use locally scoped variables
- use local variables and remove some intermediate variables
- handle the events inside the switch itself; this makes all the
  switch branches use the same logic, instead of "some" using
  a `continue`, and others falling through to have the event handled
  outside of the switch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-17 14:29:54 +02:00
Sebastiaan van Stijn
e598ffcdf8 Merge pull request #46486 from thaJeztah/update_containerd_binary_1.7.6
update containerd binary to v1.7.6
2023-09-16 13:38:31 +02:00
Djordje Lukic
32e2c53467 c8d: Fix pull progress update
We were sending the "Pulling from ..." message too early, if the pull
progress wasn't able to resolve the image we wouldn't sent the error
back. Sending that first message would have flushed the output stream
and image_routes.go would return a nil error.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-16 13:07:13 +02:00
Djordje Lukic
5a4962f80d c8d: Refuse images with digest algo when tagging
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-16 00:13:44 +02:00
Bjorn Neergaard
30a57db97c Merge pull request #46489 from thaJeztah/less_logrus
remove more direct uses of logrus
2023-09-15 13:55:01 -06:00
Sebastiaan van Stijn
bd523abd44 remove more direct uses of logrus
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 20:12:27 +02:00
Sebastiaan van Stijn
a07f6470b7 Merge pull request #46488 from thaJeztah/bump_containerd_1.6.24
vendor: github.com/containerd/containerd v1.6.24
2023-09-15 20:11:53 +02:00
Paweł Gronowski
a33043f0e0 Merge pull request #46485 from vvoland/c8d-push-all-tags
c8d/push: Support pushing all tags
2023-09-15 17:38:05 +02:00
Sebastiaan van Stijn
1ffbbe798a vendor: github.com/containerd/containerd v1.6.24
full diff: https://github.com/containerd/containerd/compare/v1.6.22...v1.6.24

v1.6.24 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.6.23...v1.6.24

The twenty-fourth patch release for containerd 1.6 contains various fixes
and updates.

Notable Updates

- CRI: fix leaked shim caused by high IO pressure
- Update to go1.20.8
- Update runc to v1.1.9
- Backport: add configurable mount options to overlay snapshotter
- log: cleanups and improvements to decouple more from logrus

v1.6.23 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.6.22...v1.6.23

The twenty-third patch release for containerd 1.6 contains various fixes
and updates.

Notable Updates

- Add stable ABI support in windows platform matcher + update hcsshim tag
- cri: Don't use rel path for image volumes
- Upgrade GitHub actions packages in release workflow
- update to go1.19.12
- backport: ro option for userxattr mount check + cherry-pick: Fix ro mount option being passed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 17:34:58 +02:00
Albin Kerouanton
336e1e9b7e Merge pull request #45906 from akerouanton/create-with-several-networks
api: Allow ContainerCreate to take several EndpointsConfig for >= API 1.44
2023-09-15 16:36:59 +02:00
Paweł Gronowski
7f281907dd c8d/push: Add missing message about repository
Add "The push referers to repository X" message which is present in the
push output when using the graphdrivers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-15 16:27:45 +02:00
Paweł Gronowski
babf907bfd c8d/push: Support pushing all tags
Implement missing feature that pushes all tags from the provided local
repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-15 16:27:43 +02:00
Albin Kerouanton
7ec9f304e9 daemon/cluster: create managed ctr with multiple EndpointsConfig
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Albin Kerouanton
5e15ed314b api: Improve error on ContainerCreate with multiple endpoints
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Sebastiaan van Stijn
cc9c930e29 vendor: github.com/Microsoft/hcsshim v0.9.10
Add support for platform compatibility check for windows + add windows builds

full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.8...v0.9.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 12:52:02 +02:00
Sebastiaan van Stijn
24102aa8ca update containerd binary to v1.7.6
Update the version used in testing;

full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.6

v1.7.6 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.5...v1.7.6

The sixth patch release for containerd 1.7 contains various fixes and updates.

- Fix log package for clients overwriting the global logger
- Fix blockfile snapshotter copy on Darwin
- Add support for Linux usernames on non-Linux platforms
- Update Windows platform matcher to invoke stable ABI compability function
- Update Golang to 1.20.8
- Update push to inherit distribution sources from parent

v1.7.5 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.4...v1.7.5

The fifth patch release for containerd 1.7 fixes a versioning issue from
the previous release and includes some internal logging API changes.

v1.7.4 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.4

The fourth patch release for containerd 1.7 contains remote differ plugin support,
a new block file based snapshotter, and various fixes and updates.

Notable Updates

- Add blockfile snapshotter
- Add remote/proxy differ
- Update runc binary to v1.1.9
- Cri: Don't use rel path for image volumes
- Allow attaching to any combination of stdin/out/err
- Fix ro mount option being passed
- Fix leaked shim caused by high IO pressure
- Add configurable mount options to overlay snapshotter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 12:21:07 +02:00
Albin Kerouanton
bbcd662532 api: Allow ContainerCreate to take several EndpointsConfig for >= 1.44
The API endpoint `/containers/create` accepts several EndpointsConfig
since v1.22 but the daemon would error out in such case. This check is
moved from the daemon to the api and is now applied only for API < 1.44,
effectively allowing the daemon to create containers connected to
several networks.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 10:07:29 +02:00
Sebastiaan van Stijn
20f9635469 Merge pull request #46475 from dmcgowan/c8d-registry-token-support
Add support for registry token in containerd pull logic
2023-09-14 17:18:07 +02:00
Sebastiaan van Stijn
35e9d688ba libnetwork: remove netWatch.remoteEps, as it's unused
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-14 15:58:48 +02:00
Sebastiaan van Stijn
8ae5dc4aae libnetwork: Network.updateSvcRecord: remove unused localEps arg
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-14 15:58:48 +02:00
Sebastiaan van Stijn
37b2ea9a8d libnetwork: Controller.processEndpointCreate, Delete: remove nmap arg
It was used to pass the controller's own field; let's just use the
field directly instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-14 15:58:44 +02:00
Derek McGowan
62e55fd58a Add support for registry token in containerd pull logic
When registry token is provided, the authorization header can be
directly applied to the registry request. No other type of
authorization will be attempted when the registry token is provided.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-14 06:35:23 -07:00
Paweł Gronowski
00e7ef5c3c Merge pull request #46477 from thaJeztah/freeze_frozen
contrib: outline purpose of download-frozen-images-v2.sh
2023-09-14 14:38:26 +02:00
Sebastiaan van Stijn
01cc1cc923 Merge pull request #46471 from foundriesio/atomic-layer-data-write
daemon: overlay2: Write layer metadata atomically
2023-09-14 12:10:17 +02:00
Sebastiaan van Stijn
14c5f7bf1d contrib: outline purpose of download-frozen-images-v2.sh
We occassionally receive contributions to this script that are outside
its intended scope. Let's add a comment to the script that outlines
what it's meant for, and a link to a GitHub ticket with alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-14 11:36:22 +02:00
Paweł Gronowski
0937aef261 libcontainerd/windows: Don't reap on failure
Synchronize the code to do the same thing as Exec.
reap doesn't need to be called before the start event was sent.
There's already a defer block which cleans up the process in case where
an error occurs.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:11:33 +02:00
Paweł Gronowski
b805599ef6 libcontainer/windows: Remove unneeded var declaration
The cleanup defer uses an `outErr` now, so we don't need to worry about
shadowing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:10:40 +02:00
Paweł Gronowski
55b664046c libcontainer/windows: Fix process not being killed after stdio attach failure
Error check in defer block used wrong error variable which is always nil
if the flow reaches the defer. This caused the `newProcess.Kill` to be
never called if the subsequent attemp to attach to the stdio failed.
Although this only happens in Exec (as Start does overwrite the error),
this also adjusts the Start to also use the returned error to avoid this
kind of mistake in future changes.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:10:11 +02:00
Sebastiaan van Stijn
39b2bf51ca Merge pull request #46406 from akerouanton/issue-46404
daemon: fix under what conditions container's mac-address is applied
2023-09-13 23:35:07 +02:00
Sebastiaan van Stijn
a232f9463c Merge pull request #46474 from thaJeztah/remove_unused_arg
libnetwork/portmapper: Remove unused arg from New(), and un-export PortMapper.Allocator
2023-09-13 23:32:43 +02:00
Sebastiaan van Stijn
982e5afd1c Merge pull request #46472 from thaJeztah/dmcg_triage
Add dmcgowan as curator
2023-09-13 20:54:11 +02:00
Sebastiaan van Stijn
9c84994830 libnetwork/portmapper: remove unused PortMapper.checkIP
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:54:11 +02:00
Sebastiaan van Stijn
f5d6af13d0 libnetwork/portmapper: un-export PortMapper.Allocator
It was only accessed through methods on PortMapper, and in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:38:53 +02:00
Sebastiaan van Stijn
863909a749 libnetwork/portmapper: New(): remove unused argument
None of the code using this function was setting the value, so let's
simplify and remove the argument.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:12:53 +02:00
Sebastiaan van Stijn
0a8bd82a37 Merge pull request #46446 from rhansen/host_ipv4
Fix host_ipv4 bridge option when IPv6 and ip6tables are enabled
2023-09-13 18:08:03 +02:00
Sebastiaan van Stijn
5ab8d41d9f Add dmcgowan as curator
Adding Derek as curator, so that he's able to perform triage tasks
in this repository :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 17:22:24 +02:00
Mike Sul
de2447c2ab daemon: overlay2: Write layer metadata atomically
When the daemon process or the host running it is abruptly terminated,
the layer metadata file can become inconsistent on the file system.
Specifically, `link` and `lower` files may exist but be empty, leading
to overlay mounting errors during layer extraction, such as:
"failed to register layer: error creating overlay mount to <path>:
too many levels of symbolic links."

This commit introduces the use of `AtomicWriteFile` to ensure that the
layer metadata files contain correct data when they exist on the file system.

Signed-off-by: Mike <mike.sul@foundries.io>
2023-09-13 15:07:32 +02:00
Sebastiaan van Stijn
76915b16e7 Merge pull request #46347 from thaJeztah/libnetwork_early_returns
libnetwork: Network: add some early returns
2023-09-13 12:36:34 +02:00
Sebastiaan van Stijn
3b04fd10e8 Merge pull request #46251 from akerouanton/libnet-forbid-duplicated-network-names
libnet: Make sure network names are unique
2023-09-12 16:53:14 +02:00
Sebastiaan van Stijn
9641c90eaf Merge pull request #46458 from vvoland/vendor-resenje-singleflight-4.0.0
vendor: resenje.org/singleflight v0.4.0
2023-09-12 14:40:05 +02:00
Sebastiaan van Stijn
7fbc3a9bc0 Merge pull request #46453 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-12 10:43:25 +02:00
Sebastiaan van Stijn
ff85c363a8 Merge pull request #46440 from thaJeztah/update_cdi
vendor: github.com/cncf-tags/container-device-interface v0.6.1
2023-09-12 10:43:02 +02:00
Sebastiaan van Stijn
0b69c2c42f Merge pull request #46443 from thaJeztah/mod_tidy
go mod tidy
2023-09-12 10:42:40 +02:00
Albin Kerouanton
78479b1915 libnet: Make sure network names are unique
Fixes #18864, #20648, #33561, #40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-12 10:40:13 +02:00
Sebastiaan van Stijn
ff328e748d Merge pull request #46397 from thaJeztah/fix_neighbor_delete
libnetwork/osl: Namespace.DeleteNeighbor: assorted cleanups, and ignore "non-exist" warnings
2023-09-12 10:38:43 +02:00
Sebastiaan van Stijn
4f28802f09 Merge pull request #46413 from thaJeztah/builder_diffid_type
daemon/containerd: use "DiffID" type for diff fields
2023-09-12 10:08:46 +02:00
Sebastiaan van Stijn
75308e471c Merge pull request #46411 from thaJeztah/simplify_scratch
daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
2023-09-12 09:36:38 +02:00
Paweł Gronowski
5fa011dc0c vendor: resenje.org/singleflight v0.4.0
Fixes the context aware singleflight not preserving context values.

full diff: https://github.com/janos/singleflight/compare/v0.3.0...v0.4.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-12 09:34:03 +02:00
Sebastiaan van Stijn
fd15ddbc30 daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
Windows doesn't support "FROM scratch", and the platform was only used
for validation on other platforms if a platform was provided, so no need
to set defaults.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:18:13 +02:00
Sebastiaan van Stijn
4229032676 daemon/containerd: use "DiffID" type for diff fields
strong-type the fields with the expected type, to make it more explicit
what we're expecting here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:15:04 +02:00
Sebastiaan van Stijn
cdb5947316 Merge pull request #46455 from akerouanton/enable-ipv6-automatically
daemon: automatically set network EnableIPv6 if needed
2023-09-11 22:18:22 +02:00
Albin Kerouanton
5d5eeac310 daemon: automatically set network EnableIPv6 if needed
PR 4f47013feb added a validation step to `NetworkCreate` to ensure
no IPv6 subnet could be set on a network if its `EnableIPv6` parameter
is false.

Before that, the daemon was accepting such request but was doing nothing
with the IPv6 subnet.

This validation step is now deleted, and we automatically set
`EnableIPv6` if an IPv6 subnet was specified.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-11 20:53:29 +02:00
Sebastiaan van Stijn
51d647122a Merge pull request #46383 from vvoland/c8d-legacybuilder-fix-layer-parent-snapshot
c8d/legacybuilder: Assorted fixes
2023-09-11 20:41:59 +02:00
Djordje Lukic
29dcf646a3 Merge pull request #46454 from rumpl/c8d-disable-schema1 2023-09-11 19:55:43 +02:00
Djordje Lukic
d0d3ddd045 Merge pull request #46375 from rumpl/c8d-userns-remap
c8d: Handle userns properly
2023-09-11 19:04:35 +02:00
Djordje Lukic
0313544f4a c8d: Handle userns properly
If the daemon is run with --userns-remap we need to chown the prepared
snapshot

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 16:39:29 +02:00
Paweł Gronowski
c63a952dc9 c8d/builder: Don't append empty tar layer to manifest
To match the number of layers in config created in
`images.CreateChildImage`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:44 +02:00
Paweł Gronowski
9127285985 c8d/builder: Lease layer snapshots
Create a lease for the snapshot and hold it until the layer is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:39 +02:00
Paweł Gronowski
f22b112005 c8d/commit: Unpack committed image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:14 +02:00
Paweł Gronowski
8832cdf6e1 c8d: Common unpack code for specific manifest
Use `ImageService.unpackImage` when we want to unpack an image and we
know the exact platform-manifest to be unpacked beforehand.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:10 +02:00
Paweł Gronowski
c6bded3475 c8d/builder: Don't use diffID as snapshot parent
DiffID is only a digest of the one tar layer and matches the snapshot ID
only for the first layer (DiffID = ChainID).

Instead of generating random ID as a key for rolayer, just use the
snapshot ID of the unpacked image content and use it later as a parent
for creating a new RWLayer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:04 +02:00
Paweł Gronowski
8c7e19c5ff c8d/builder: Set empty diffID for rolayer
diffID is the digest of a tar archive containing changes to the parent
layer - rolayer doesn't have any changes to the parent.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:02 +02:00
Paweł Gronowski
6ebfa57364 c8d/builder: Don't mount the rolayer snapshot
The view snapshot and its mounts are not used.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:19:59 +02:00
Sebastiaan van Stijn
c41121cc48 update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 15:47:00 +02:00
Djordje Lukic
4acbf7d8c3 c8d: disable schema1 registry integration tests
schema1 was deprecated a while ago, containerd fails to push to a
schema1 registry, let's just skip these tests for the containerd
integration

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 15:30:44 +02:00
Sebastiaan van Stijn
ccae064209 Merge pull request #46423 from vvoland/c8d-images-all
c8d/builder: Store parent in c8d image label
2023-09-11 14:51:20 +02:00
Djordje Lukic
833b514d98 c8d/mount: Create same directory owners as graphdrivers
Graph drivers create the parent directory with
rootPair().GID:CurrentIdentity().UID owner. This change brings these in
line

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 13:24:42 +02:00
Sebastiaan van Stijn
f8a4b6755d Merge pull request #46442 from thaJeztah/less_polling
integration: don't poll for containers to be running
2023-09-11 13:15:38 +02:00
Albin Kerouanton
a8975c9042 api: Add consts for predefined networks
Constants for both platform-specific and platform-independent networks
are added to the api/network package.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-10 15:39:54 +02:00
Richard Hansen
12e27dfd8f Fix host_ipv4 bridge option when IPv6 and ip6tables are enabled
Before this commit, setting the `com.docker.network.host_ipv4` bridge
option when `enable_ipv6` is true and the experimental `ip6tables`
option is enabled would cause Docker to fail to create the network:

> failed to create network `test-network`: Error response from daemon:
> Failed to Setup IP tables: Unable to enable NAT rule: (iptables
> failed: `ip6tables --wait -t nat -I POSTROUTING -s fd01::/64 ! -o
> br-test -j SNAT --to-source 192.168.0.2`: ip6tables
> v1.8.7 (nf_tables): Bad IP address "192.168.0.2"
>
> Try `ip6tables -h` or `ip6tables --help` for more information.
>  (exit status 2))

Fix this error by passing nil -- not the `host_ipv4` address -- when
creating the IPv6 rules.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-09-10 04:03:07 -04:00
Sebastiaan van Stijn
7c0f6741eb go mod tidy
Looks like some dependencies ended up on in the wrong list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-09 00:53:46 +02:00
Sebastiaan van Stijn
cd204f1118 libnetwork/osl: Namespace.DeleteNeighbor: remove intermediate vars
- store linkIndex in a local variable so that it can be reused
- remove / rename some intermediate vars that shadowed existing declaration

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:14 +02:00
Sebastiaan van Stijn
4215a1542b libnetwork/osl: Namespace.DeleteNeighbor: remove osDelete argument
This argument was originally added in libnetwork:
03f440667f

At the time, this argument was conditional, but currently it's always set
to "true", so let's remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:14 +02:00
Sebastiaan van Stijn
bc77104fed libnetwork/osl: Namespace.DeleteNeighbor: don't warn on non-existing neighbor
The code ignores these errors, but will unconditionally print a warning;

> If the kernel deletion fails for the neighbor entry still remote it
> from the namespace cache. Otherwise if the neighbor moves back to the
> same host again, kernel update can fail.

Let's reduce noise if the neighbor wasn't found, to prevent logs like:

    Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.186662370+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
    Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.366585939+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
    Aug 16 13:26:42 master1.local dockerd[4019880]: time="2023-08-16T13:26:42.366658513+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"

While changing this code, also slightly rephrase the code-comment, and
fix a typo ("remote -> remove").

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

libnetwork/osl: Namespace.DeleteNeighbor: rephrase code-comment

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:11 +02:00
Sebastiaan van Stijn
6ce5aa1cd5 Merge pull request #46434 from vvoland/test-ignore-cgroup-check
hack: Add TEST_IGNORE_CGROUP_CHECK
2023-09-08 23:23:18 +02:00
Sebastiaan van Stijn
c3ab3f1dde Merge pull request #46412 from rumpl/c8d-pull-progress
c8d: Better pull progress
2023-09-08 23:10:23 +02:00
Sebastiaan van Stijn
d582a26107 Merge pull request #46439 from akerouanton/no-optimizations
hack: compile w/o optimizations & inlining when debugging
2023-09-08 23:09:33 +02:00
Sebastiaan van Stijn
5e7eade1f7 integration: don't poll for containers to be running
container.Run() should be a synchronous operation in normal circumstances;
the container is created and started, so polling after that for the
container to be in the "running" state should not be needed.

This should also prevent issues when a container (for whatever reason)
exited immediately after starting; in that case we would continue
polling for it to be running (which likely would never happen).

Let's skip the polling; if the container is not in the expected state
(i.e. exited), tests should fail as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:05:43 +02:00
Sebastiaan van Stijn
b7d29c1e23 registry: un-skip some tests that don't require root
These tests seem to be running fine without being root, so let's not skip them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 18:35:09 +02:00
Sebastiaan van Stijn
de27973538 registry: merge validateEndpoint into newV1Endpoint
validateEndpoint was doing more than just validating; it was also implicitly
mutating the endpoint that was passed to it (by reference).

Given that validation only happend when constructing a new v1Endpoint, let's
merge these functions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 18:31:17 +02:00
Albin Kerouanton
6cc6682f5f daemon: fix under what conditions container's mac-address is applied
The daemon would pass an EndpointCreateOption to set the interface MAC
address if the network name and the provided network mode were matching.
Obviously, if the network mode is a network ID, it won't work. To make
things worse, the network mode is never normalized if it's a partial ID.

To fix that: 1. the condition under what the container's mac-address is
applied is updated to also match the full ID; 2. the network mode is
normalized to a full ID when it's only a partial one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 18:15:00 +02:00
Sebastiaan van Stijn
97e782ee2b Merge pull request #46422 from thaJeztah/fix_resize_tests
integration/container: fix flaky resize tests, and some cleaning up
2023-09-08 18:14:18 +02:00
Sebastiaan van Stijn
e6d9d216dd Merge pull request #46432 from vvoland/c8d-skip-image-rm-test
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
2023-09-08 17:44:55 +02:00
Albin Kerouanton
820f37b1a6 hack: compile w/o optimizations & inlining when debugging
Without these compile flags, Delve is unable to report the value of some
variables and it's not possible to jump into inlined code.

As the contributing docs already mention that `DOCKER_DEBUG` should
disable "build optimizations", the env var is reused here instead of
introducing a new one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 17:16:58 +02:00
Sebastiaan van Stijn
c44f8958f6 vendor: github.com/cncf-tags/container-device-interface v0.6.1
Removes uses of the github.com/opencontainers/runc/libcontainer/devices
package.

full diff: https://github.com/cncf-tags/container-device-interface/compare/v0.6.0...v0.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 16:01:56 +02:00
Sebastiaan van Stijn
0434b653c8 Merge pull request #46435 from vvoland/c8d-inspect-prefix
c8d/image: Allow truncated id to have sha256: prefix
2023-09-08 15:59:24 +02:00
Sebastiaan van Stijn
fa3ddc5116 distribution: newRepository: remove naked return and intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 15:44:58 +02:00
Sebastiaan van Stijn
6dcefa3739 Merge pull request #46431 from rumpl/fix-ci-timeout
Use process substitution to redirect to tee
2023-09-08 15:28:09 +02:00
Sebastiaan van Stijn
78ccd5d07c Merge pull request #46433 from rumpl/c8d-remove-panic
c8d: Remove the panic from UpdateConfig
2023-09-08 15:17:53 +02:00
Sebastiaan van Stijn
43740ab642 registry: remove intermediate vars in mock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:22:21 +02:00
Sebastiaan van Stijn
550e628f05 registry: newV1Endpoint: make it clear we skip validation for Docker Hub
validateEndpoint uses `v1Endpoint.ping` to verify if the search API can
use a secure connection, and to fall back to basic auth. For Docker Hub,
we don't allow insecure connections, and `v1Endpoint.ping` will not connect
to Docker Hub (Docker Hub also does not implement the `_ping` endpoint,
so doing so would always fail).

Let's make it more clear that we don't do any validation, and return
early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:19:33 +02:00
Paweł Gronowski
5728516371 c8d/image: Allow truncated id to have sha256: prefix
Fixes TestInspectByPrefix when running with c8d snapshotters enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 14:10:10 +02:00
Paweł Gronowski
c2004fb8c4 hack: Add TEST_IGNORE_CGROUP_CHECK
Add env variable that allows to run integration-cli tests with cgroup v2 enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 13:41:03 +02:00
Sebastiaan van Stijn
7e506f71f3 registry: v1Endpoint.ping: include URL in debug log
Also remove log from `validateEndpoint`, because we don't actually
ping the default (Docker Hub).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:21:28 +02:00
Sebastiaan van Stijn
0c6f8f9290 registry: v1Endpoint.ping: don't io.Readall the response
We have the response available, which is an io.Reader, so we don't have
to read the entire response into memory before decoding.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:17:22 +02:00
Sebastiaan van Stijn
e9ad878df6 registry: v1Endpoint.ping: add fast-path for X-Docker-Registry-Standalone
This function was making a request to the `_ping` endpoint, which (if
implemented) would return a JSON response, which we unmarshal (the only
field we use from the response is the `Standalone` field).

However, if the response had a `X-Docker-Registry-Standalone`, that header
took precedence, and would overwrite the earlier `Standalone` value we
obtained from the JSON response.

This patch adds a fast-path for situations where the header is present,
in which case we can skip handling the JSON response altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:17:16 +02:00
Djordje Lukic
9e797a6db9 c8d: Remove the panic from UpdateConfig
We don't really want the daemon to panic for this so let's log a warning
about max downloads and uploads

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 12:21:16 +02:00
Paweł Gronowski
4982c70db4 integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
With containerd image store the images don't depend on each other even
if they share the same content and it's totally fine to delete the
"parent" image.

The skip is necessary because deleting the "parent" image does not
produce an error with the c8d image store and deleting the `busybox`
image breaks other tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 12:08:35 +02:00
Sebastiaan van Stijn
0f7a65e59b registry: remove use of v1 Version field and X-Docker-Registry-Version
- The `Version` field was not used for any purpose, other than a debug log
- The `X-Docker-Registry-Version` header was part of the registry v1 spec,
  however, as we're not using the `Version` field, we don't need the
  header for anything.
- The `X-Docker-Registry-Config` header was only set by the mock registry;
  there's no code consuming it, so we don't need to mock it (even if an
  actual v1 registry / search API would return it).

It's also worth noting that we never call the `_ping` endpoint when using
Docker Hub's search API, and Docker Hub does not even implement the `_ping`
endpoint;

    curl -fsSL https://index.docker.io/_ping | head -n 4
    <!DOCTYPE html>
    <html lang="en">
    <head>
    <title>Docker</title>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 11:55:53 +02:00
Djordje Lukic
3d8b8dc09a Use process substitution to redirect to tee
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.

Using process substitution makes the integration tests quit.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 10:56:21 +02:00
Paweł Gronowski
c0ece8d4f3 integration-cli: Use different label in TestBuildMultiStageGlobalArg
To avoid collision with TestBuildMultiStageArg.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 09:47:56 +02:00
Paweł Gronowski
081b74b97d c8d/builder: Store parent in c8d image label
Images built by classic builder will have an additional label (in the
containerd image object, not image config) pointing to a parent of that
image.

This allows to differentiate intermediate images (dangling
images created as a result of a each Dockerfile instruction) from the
final images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 09:47:53 +02:00
CrazyMax
152036f0aa Merge pull request #46429 from cpuguy83/swap_to_otel_action
CI: Switch to use tracing action
2023-09-08 08:22:20 +02:00
Sebastiaan van Stijn
5f59f7bb49 integration/container: combine TestResize tests into subtests
Reduce some of the boiler-plating, and by combining the tests, we skip
the testenv.Clean() in between each of the tests. Performance gain isn't
really measurable, but every bit should help :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:36:12 +02:00
Sebastiaan van Stijn
a4ceb0e4ac integration/container: TestResize, TestResizeWithInvalidSize: rm poll.WaitOn
container.Run should be an synchronous operation; the container should
be running after the request was made (or produce an error). Simplify
these tests, and remove the redundant polling.

These were added as part of 8f800c9415,
but no such polls were in place before the refactor, and there's no
mention of these during review of the PR, so I assume these were just
added either as a "precaution", or a result of "copy/paste" from another
test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:34:45 +02:00
Sebastiaan van Stijn
ee7ca6822a integration/container: fix flaky TestResizeWhenContainerNotStarted
This test was failing frequently on Windows, where the test was waiting
for the container to exit before continuing;

    === FAIL: github.com/docker/docker/integration/container TestResizeWhenContainerNotStarted (18.69s)
    resize_test.go:58: timeout hit after 10s: waiting for container to be one of (exited), currently running

It looks like this test is merely validating that a container in any non-
running state should produce an error, so there's no need to run a container
(waiting for it to stop), and just "creating" a container (which would be
in `created` state) should work for this purpose.

Looking at 8f800c9415, I see `createSimpleContainer`
and `runSimpleContainer` utilities were added, so I'm even wondering if the
original intent was to use `createSimpleContainer` for  this test.

While updating, also check if we get the expected error-type, instead of
only checking for the error-message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:33:22 +02:00
Sebastiaan van Stijn
06499c52e2 Merge pull request #46416 from thaJeztah/move_IsOSSupported
pkg/system: deprecate IsOSSupported() and ErrNotSupportedOperatingSystem, and implement image.CheckOS
2023-09-08 00:25:36 +02:00
Brian Goff
da5ed8b35b CI: Switch to use tracing action
This takes care of the TODO item now that the action is merged.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 22:23:05 +00:00
Bjorn Neergaard
ce4e325504 Merge pull request #45652 from cpuguy83/otel
Add otel tracing
2023-09-07 15:42:08 -06:00
Sebastiaan van Stijn
a3c97beee0 image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
150b657bad pkg/system: move ErrNotSupportedOperatingSystem to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:04:33 +02:00
Brian Goff
9b7784781d Add test for buildkit history trace propagation
This test ensures that we are able to propagate traces into buildkit's
history API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
3b4ccb2eca CI: Setup otel tracing for integration tests
This wires up the integration tests to export spans to a jager instance.
After tests are finished it exports the data out of jaeger and uploads
as an artifact to the action run.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
de391bc48b Merge pull request #46418 from vvoland/c8d-import-fix-double-close
c8d/import: Don't close compressed stream twice
2023-09-07 16:30:01 +02:00
Djordje Lukic
776c376227 c8d: Better pull progress
- check if we have to download layers and print the approriate message
- show the digest of the pulled manifest(list)
- skip pulling if we already have the right manifest

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-07 15:34:20 +02:00
Sebastiaan van Stijn
efea287b44 Merge pull request #46415 from thaJeztah/distribution_simplify_filterManifests
distribution: filterManifests split complicated condition
2023-09-07 13:59:27 +02:00
Paweł Gronowski
1cb45e582c c8d/import: Don't close compressed stream twice
The compressor is already closed a few lines below and there's no error
returns between so the defer is not needed.

Calling Close twice on a writerCloserWrapper is unsafe as it causes it
to put the same buffer to the pool multiple times.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-07 10:33:25 +02:00
Sebastiaan van Stijn
075a2d89b9 Merge pull request #46417 from thaJeztah/idtools_preserve_error
pkg/idtools: remove sync.Once, and include lookup error
2023-09-06 23:08:43 +02:00
Sebastiaan van Stijn
b5376c7cec pkg/idtools: remove sync.Once, and include lookup error
When running a `docker cp` to copy files to/from a container, the
lookup of the `getent` executable happens within the container's
filesystem, so we cannot re-use the results.

Unfortunately, that also means we can't preserve the results for
any other uses of these functions, but probably the lookup should not
be "too" costly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:34:26 +02:00
Sebastiaan van Stijn
62725ab277 Merge pull request #46414 from thaJeztah/less_logrus
daemon: daemon.createSpec: remove uses of logrus
2023-09-06 15:32:01 +02:00
Sebastiaan van Stijn
0be416dfdc distribution: filterManifests split complicated condition
Break up the complicated "if" condition into multiple checks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 13:34:10 +02:00
Sebastiaan van Stijn
150b1c8c73 daemon: daemon.createSpec: remove uses of logrus
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 13:30:33 +02:00
Djordje Lukic
2c95ddf4f3 Merge pull request #46405 from rumpl/image-events 2023-09-06 12:31:35 +02:00
Sebastiaan van Stijn
791549508a Merge pull request #46084 from rumpl/fix-test-arch
test: Remove DOCKER_ENGINE_GOARCH from the tests
2023-09-05 18:08:57 +02:00
Djordje Lukic
dbb4d54e01 c8d: Add image save events
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-05 17:29:15 +02:00
Djordje Lukic
09470db089 c8d: Add image pull events
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-05 17:29:13 +02:00
Djordje Lukic
1847dc4b7f c8d: Add image push events
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-05 17:28:57 +02:00
Sebastiaan van Stijn
032797ea4b Merge pull request #46376 from thaJeztah/migrate_distribution_reference
migrate to new github.com/distribution/reference module
2023-09-05 15:04:51 +02:00
Sebastiaan van Stijn
7abd7fa739 Merge pull request #46398 from vvoland/tests-integration-cli-build-multline-images
integration-cli/build: Parse multiline images -q output
2023-09-05 12:22:34 +02:00
Sebastiaan van Stijn
1148a24e64 migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
9c4e82435e Merge pull request #46351 from thaJeztah/api_events_actions_enum
api/types/events: define "Action" type and consts
2023-09-05 11:11:42 +02:00
Sebastiaan van Stijn
2243046bf8 Merge pull request #46384 from rumpl/push-print-digest
c8d: Print the tag, digest and size after push
2023-09-04 16:47:02 +02:00
Paweł Gronowski
dffccfd0f1 integration-cli/build: Alias cmp as is
To make it consistent with other tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-04 16:40:08 +02:00
Paweł Gronowski
5dbd198b53 integration-cli/build: Parse multiline images -q output
This causes the test to have a saner error message when the `images
-q` returns multiple images separated by newline.

Before this the test would fail with `invalid reference format` when
parsing the multiline string as an image reference.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-04 16:40:05 +02:00
Sebastiaan van Stijn
4bf1a946f9 Merge pull request #46396 from thaJeztah/fix_duplicate_definition
libnetwork/osl: remove dead code
2023-09-04 15:06:58 +02:00
Djordje Lukic
96516002cc c8d: Print the tag, digest and size after push
After a push is completed we must return this information back to the
caller.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-04 13:43:19 +02:00
Sebastiaan van Stijn
417328e44e libnetwork/osl: remove dead code
The test-file had a duplicate definition for ErrNotImplemented, which
caused an error in this package, and was not used otherwise, so we can
remove this file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 10:37:25 +02:00
Sebastiaan van Stijn
97206b7cba Merge pull request #46360 from thaJeztah/search_split_search_service_step1
registry: split search-related code to separate files
2023-09-01 18:31:16 +02:00
Bjorn Neergaard
74fafbda51 Merge pull request #46265 from dvdksn/docs-rephrase-images-create
docs(api): rephrase description for /images/create
2023-09-01 07:04:27 -06:00
David Karlsson
237eb3b5f1 docs(api): rephrase description for /images/create
Updated the description to clarify that this is the endpoint to use if
you want to pull an image.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-08-31 21:05:43 +02:00
Paweł Gronowski
8dfaf0c780 Merge pull request #46313 from vvoland/c8d-image-ociwrapper
containerd integration: Persist Docker-specific ImageConfig fields
2023-08-31 19:51:47 +02:00
Paweł Gronowski
0ffa3dd870 daemon/c8d: Use Docker imagespec
This makes the c8d code which creates/reads OCI types not lose
Docker-specific features like ONBUILD or Healthcheck.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 17:15:05 +02:00
Paweł Gronowski
14af90b868 c8d/integration/TestBuildOnBuildCache skip parent check
Parent is a graph-driver only field which is stored in the ImageStore.
It's not available when using containerd snapshotters.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 16:23:17 +02:00
Paweł Gronowski
2004f60bd6 Merge pull request #46363 from vvoland/image-spec-specs-go
image/spec: Add Go structs
2023-08-31 16:22:46 +02:00
Paweł Gronowski
366a5f1d74 image/spec: Add Go structs
Add Go structs describing the image spec which extend the OCI types.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 12:29:45 +02:00
Sebastiaan van Stijn
e9b3a16cdc Merge pull request #46381 from vvoland/image-spec-healthcheck-startinterval
image/spec: Add Healthcheck.StartInterval (v1.3)
2023-08-31 12:23:32 +02:00
Sebastiaan van Stijn
aa9eb76b0d Merge pull request #46379 from vvoland/image-spec-fix-superseded-typo
image/spec: Fix a typo in "superseded"
2023-08-31 12:22:03 +02:00
Sebastiaan van Stijn
02db973460 Merge pull request #46380 from rumpl/test-platform-timeout
test: remove platform related timeout values
2023-08-31 11:57:09 +02:00
Paweł Gronowski
536826d66f image/spec: Add Healthcheck.StartInterval (v1.3)
This field was added in 2216d3ca8d.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 11:22:54 +02:00
Sebastiaan van Stijn
6fc6b9bf31 Merge pull request #46378 from thaJeztah/client_fix_test
client: TestImageTagInvalidSourceImageName remove invalid test-case
2023-08-31 11:02:20 +02:00
Sebastiaan van Stijn
47a3b31ff6 Merge pull request #46377 from thaJeztah/distribution_simplify_mocks
distribution: simplify mocks
2023-08-31 11:01:45 +02:00
Djordje Lukic
75097cf09a test: remove platform related timeout values
These were dependent on the DOCKER_ENGINE_GOARCH environment variable
but this var was no longer set. There was also some weird check to see
if the architecture is "windows" which doesn't make sense. Seeing how
nothing failed ever since the TIMEOUT was no longer platform-dependent
we can safely remove this check.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-31 10:39:28 +02:00
Paweł Gronowski
6dd78a9ac2 image/spec: Fix typo in "superseded"
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 10:35:14 +02:00
Djordje Lukic
84a4f37cf7 test: use info from the version endpoint for arch checks
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-31 09:36:48 +02:00
Djordje Lukic
159b168eea test/integration: Remove checks for "not arm" in tests
We no longer have any arm (not 64) CI.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-31 09:36:48 +02:00
Djordje Lukic
ebb9fade23 makefile: use info -f to get the current storage driver
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-31 09:36:48 +02:00
Sebastiaan van Stijn
c243efb0cd client: TestImageTagInvalidSourceImageName remove invalid test-case
The test considered `Foo/bar` to be an invalid name, with the assumption
that it was `[docker.io]/Foo/bar`. However, this was incorrect, and the
test passed because the reference parsing had a bug; if the first element
(`Foo`) is not lowercase (so not a valid namespace /  "path element"), then
it *should* be considered a domain (as uppercase domain names are valid).

The reference parser did not account for this, and running the test with
a version of the parser with a fix caused the test to fail:

    === Failed
    === FAIL: client TestImageTagInvalidSourceImageName/invalidRepo/FOO/bar (0.00s)
        image_tag_test.go:54: assertion failed: expected error to contain "not a valid repository/tag", got "Error response from daemon: client should not have made an API call"
            Error response from daemon: client should not have made an API call

    === FAIL: client TestImageTagInvalidSourceImageName (0.00s)

This patch removes the faulty test-case.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 21:43:02 +02:00
Sebastiaan van Stijn
3a4a04e462 distribution: simplify mocks
embed the interface, so that we don't have to implement stubs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 19:18:59 +02:00
Sebastiaan van Stijn
d390f86080 Merge pull request #46369 from thaJeztah/killWithSignal_dont_discard_errs
daemon: Daemon.killWithSignal(): don't fully discard errors, but log them
2023-08-30 18:22:23 +02:00
Sebastiaan van Stijn
a74f1ab423 Merge pull request #46374 from thaJeztah/image_spec_combined
image/spec: merge history of the v1, v1.1 and v1.2 specification
2023-08-30 17:56:46 +02:00
Sebastiaan van Stijn
d864d1eb97 image/spec: add back versioned files
Add back files at the old locations, as there may be external links
referencing the specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:17:17 +02:00
Sebastiaan van Stijn
7aa0bc5790 merge image-spec v1.2 into specs.md
This merges the v1.2 specs to provide a single history of the
specification.

To view the combined history:

    git log --follow image/spec/spec.md

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:11:14 +02:00
Sebastiaan van Stijn
823d0b3765 merge image-spec v1.1 into specs.md
This merges the v1.1 specs to provide a single history of the
specification.

To view the combined history:

    git log --follow image/spec/spec.md
2023-08-30 16:09:30 +02:00
Sebastiaan van Stijn
b8de6c01b5 image/spec: rename v1.2.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:06:36 +02:00
Sebastiaan van Stijn
c3f2bd1009 image/spec: rename v1.1.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:06:07 +02:00
Sebastiaan van Stijn
7ff38506ab image/spec: rename v1.0.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:05:34 +02:00
Sebastiaan van Stijn
32d5e6312d image/spec: use permalink to versioned documents
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:04:08 +02:00
Sebastiaan van Stijn
daa4618da8 Merge pull request #46371 from thaJeztah/image_specs_amend
image/specs: add missing OnBuild (v1.0), ArgsEscaped (v1.1), and Shell (v1.2) fields
2023-08-30 15:54:44 +02:00
Sebastiaan van Stijn
7b7019a4da image/spec: add missing "Shell" field (v1.2)
This field was added in b18ae8c9cc, which
was part of v1.12.0-rc1 and later, which used image spec v1.2.0.

This patch amends the v1.2 spec to include the missing field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 14:33:11 +02:00
Sebastiaan van Stijn
5380f3f0c4 image/spec: add missing "ArgsEscaped" field (v1.1, v1.2)
This field was added in 9db5db1b94, which
was part of v1.10.0-rc1 and later, which used image spec v1.1.0.

It's worth noting that documentation for the v1.1.0 image spec was not
yet available until commit 4fa0eccd10,
which was included in v1.12.0-rc1 and up. The `ArgsEscaped` field was
also adopted by the OCI image spec since [v1.1.0-rc3][1], but considered
deprecated, and not recommended to be used.

This patch amends the v1.1 and v1.2 specifications to describe the field.

[1]: 59780aa569

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 13:25:00 +02:00
Sebastiaan van Stijn
f33370219b image/spec: add missing "OnBuild" field
This field was added in commit 9f994c9646,
which was merged before the image-spec v1.0.0 was released (which happened
in commit 79910625f0).

This patch backfills the specifications to describe the property.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 13:04:26 +02:00
Sebastiaan van Stijn
b526ce268f Merge pull request #46370 from thaJeztah/image_spec_touch_ups
image/spec: minor fixes and touch-ups in markdown files
2023-08-30 12:46:38 +02:00
Sebastiaan van Stijn
e19edd3f25 image/spec: minor fixes and touch-ups in markdown files
- remove some trailing commas, which made the JSON invalid (some of these
  were fixed in the 1.2 spec, but not in older versions).
- synchronise some formatting / phrasing between versions, to make them
  easier to compare.
- remove non-breaking spaces (`NBSP`) in example outputs, and replace
  them with regular spaces.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 11:18:20 +02:00
Sebastiaan van Stijn
b9f9e20f01 daemon: Daemon.killWithSignal(): don't discard snapshotting errors
While there's not much we can do if we failed to store a snapshot of the
container's state, let's log the error in case it happens in stad of discarding.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 23:11:06 +02:00
Sebastiaan van Stijn
af535233cc daemon: Daemon.killWithSignal(): don't discard handleContainerExit error
Daemon.handleContainerExit() returns an error if snapshotting the container's
state to disk fails. There's not much we can do with the error if it occurs,
but let's log the error if that happens, instead of discarding it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 23:11:06 +02:00
Sebastiaan van Stijn
9af5484fcc Merge pull request #43998 from thaJeztah/docker_py_6
testing: update docker-py 6.1.3
2023-08-29 22:59:27 +02:00
Sebastiaan van Stijn
6cf939fcc0 Merge pull request #46320 from thaJeztah/c8d_log_release_fails
daemon/containerd: some fixes and enhancements
2023-08-29 22:04:38 +02:00
Sebastiaan van Stijn
afee36d07d Merge pull request #46253 from vvoland/volume-local-restore-mounted-status
volume/local: Don't unmount, restore mounted status
2023-08-29 19:57:41 +02:00
Sebastiaan van Stijn
8c6a46f643 libnetwork: Endpoint.AddStaticRoute don't create StaticRoute if unused
This function either had to create a new StaticRoute, or add the destination
to the list of routes. Skip creating a StaticRoute struct if we're not
gonna use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 19:51:41 +02:00
Sebastiaan van Stijn
1dcb11f81d libnetwork: Endpoint.InterfaceName: remove redundant nil check
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 19:50:55 +02:00
Sebastiaan van Stijn
a49b71b481 Merge pull request #46311 from thaJeztah/libnetwork_register
libnetwork: fix missing imports, code comment, and minor comment change (dupwords)
2023-08-29 18:34:20 +02:00
Sebastiaan van Stijn
726dbc0f29 Merge pull request #46364 from corhere/libn/controller-is-a-registerer
libnetwork: notify another driver registerer
2023-08-29 17:49:14 +02:00
Sebastiaan van Stijn
a9a0ffaf51 testing: update docker-py 6.1.3
full diff: https://github.com/docker/docker-py/compare/6.0.1...6.1.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:31:33 +02:00
Sebastiaan van Stijn
f6959bc597 test-docker-py: skip AttachContainerTest::test_run_container_reading_socket_ws
Tests are failing with this error:

    E   ValueError: scheme http+docker is invalid

Which is reported in docker-py in https://github.com/docker/docker-py/issues/1478.
Not sure what changed in the tests, but could be due to updated Python
version or dependencies, but let's skip it for now.

Test failure:

    ___________ AttachContainerTest.test_run_container_reading_socket_ws ___________
    tests/integration/api_container_test.py:1245: in test_run_container_reading_socket_ws
        pty_stdout = self.client.attach_socket(container, opts, ws=True)
    docker/utils/decorators.py:19: in wrapped
        return f(self, resource_id, *args, **kwargs)
    docker/api/container.py:98: in attach_socket
        return self._attach_websocket(container, params)
    docker/utils/decorators.py:19: in wrapped
        return f(self, resource_id, *args, **kwargs)
    docker/api/client.py:312: in _attach_websocket
        return self._create_websocket_connection(full_url)
    docker/api/client.py:315: in _create_websocket_connection
        return websocket.create_connection(url)
    /usr/local/lib/python3.7/site-packages/websocket/_core.py:601: in create_connection
        websock.connect(url, **options)
    /usr/local/lib/python3.7/site-packages/websocket/_core.py:245: in connect
        options.pop('socket', None))
    /usr/local/lib/python3.7/site-packages/websocket/_http.py:117: in connect
        hostname, port, resource, is_secure = parse_url(url)
    /usr/local/lib/python3.7/site-packages/websocket/_url.py:62: in parse_url
        raise ValueError("scheme %s is invalid" % scheme)
    E   ValueError: scheme http+docker is invalid
    ------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:31:09 +02:00
Sebastiaan van Stijn
e0f171f342 testing: update docker-py 6.0.1
release notes: https://github.com/docker/docker-py/releases/tag/6.0.1

full diff: https://github.com/docker/docker-py/compare/5.0.3...6.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:31:07 +02:00
Sebastiaan van Stijn
79a0f82ca1 test-docker-py: remove comment about docker 17.06 limitations
Our dev-container now has buildx installed, so we're no longer
limited to 17.06.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:29:49 +02:00
Sebastiaan van Stijn
be90e5e1d4 libnetwork: Controller.addServiceBinding: fix duplicate word in comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:55:46 +02:00
Sebastiaan van Stijn
77c2613d3a libnetwork: fix some missing imports on macOS and FreeBSD
This was introduced in 1980deffae, which
changed the implementation, but forgot to update imports in these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:55:44 +02:00
Sebastiaan van Stijn
5da2dd98e9 registry: move search-related code to separate files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:43 +02:00
Sebastiaan van Stijn
f1a7028900 registry: rename v1-endpoint tests to have a common prefix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:43 +02:00
Sebastiaan van Stijn
19f6f0b3db registry: move v1 endpoint tests to endpoint_test.go
Moves the TestPingRegistryEndpoint and TestEndpoint tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:43 +02:00
Sebastiaan van Stijn
47bf587f84 registry: ResolveAuthConfig: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:18 +02:00
Cory Snider
a0a8d9d057 libnetwork: notify another driver registerer
There is no meaningful distinction between driverapi.Registerer and
drvregistry.DriverNotifyFunc. They are both used to register a network
driver with an interested party. They have the same function signature.
The only difference is that the latter could be satisfied by an
anonymous closure. However, in practice the only implementation of
drvregistry.DriverNotifyFunc is the
(*libnetwork.Controller).RegisterDriver method. This same method also
makes the libnetwork.Controller type satisfy the Registerer interface,
therefore the DriverNotifyFunc type is redundant. Change
drvregistry.Networks to notify a Registerer and drop the
DriverNotifyFunc type.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-08-29 10:32:18 -04:00
Sebastiaan van Stijn
cc1627ae10 Merge pull request #46362 from thaJeztah/errdefs_no_deps
errdefs: remove gotest.tools dependency and remove some redundant import comments
2023-08-29 14:50:49 +02:00
Sebastiaan van Stijn
2832886761 Merge pull request #46361 from thaJeztah/registry_update_godoc
registry: update some godoc
2023-08-29 14:45:51 +02:00
Paweł Gronowski
2689484402 volume/local: Don't unmount, restore mounted status
On startup all local volumes were unmounted as a cleanup mechanism for
the non-clean exit of the last engine process.

This caused live-restored volumes that used special volume opt mount
flags to be broken. While the refcount was restored, the _data directory
was just unmounted, so all new containers mounting this volume would
just have the access to the empty _data directory instead of the real
volume.

With this patch, the mountpoint isn't unmounted. Instead, if the volume
is already mounted, just mark it as mounted, so the next time Mount is
called only the ref count is incremented, but no second attempt to mount
it is performed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-29 14:16:37 +02:00
Sebastiaan van Stijn
acdad37160 Merge pull request #46359 from thaJeztah/deprecate_endpoint_version
registry: deprecate APIEndpoint.Version and APIVersion type
2023-08-29 14:01:30 +02:00
Sebastiaan van Stijn
f1a6e16258 errdefs: remove redundant import comments
A package only needs one "import" comment to enforce, so keeping
one in the go.doc.

It should be noted that even with that; in most cases, go will ignore
these comments (if go modules are used, even in "vendor" mode).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 13:56:55 +02:00
Sebastiaan van Stijn
b6ba2a4f75 errdefs: remove gotest.tools dependency
It was only used in a single test, and was not using any of
the gotest.tools features, so let's remove it as dependency.

With this, the package has no external dependencies (only stdlib).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 13:55:10 +02:00
Sebastiaan van Stijn
3f8cbd8ec9 registry: update some godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 13:22:14 +02:00
Sebastiaan van Stijn
e10eca3d1a daemon/containerd: rename some vars that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 12:54:36 +02:00
Sebastiaan van Stijn
aefbd49038 daemon/containerd: newROLayerForImage: remove unused args
Also rename variables that collided with imports.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 12:54:19 +02:00
Sebastiaan van Stijn
28d201feb7 daemon/containerd: log errors when releasing leases
Log a warning if we encounter an error when releasing leases. While it
may not have direct consequences, failing to release the lease should be
unexpected, so let's make them visible.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 12:54:14 +02:00
Sebastiaan van Stijn
d43e61758a registry: deprecate APIEndpoint.Version and APIVersion type
This field was used when the code supported both "v1" and "v2" registries.
We no longer support v1 registries, and the only v1 endpoint that's still
used is for the legacy "search" endpoint, which does not use the APIEndpoint
type.

As no code is using this field, and the value will always be set to "v2",
we can deprecated the Version field.

I'm keeping this field for 1 release, to give notice to any potential
external consumer, after which we can delete it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 12:49:46 +02:00
Sebastiaan van Stijn
69c19cf0b0 Merge pull request #46356 from thaJeztah/registry_cleanup_v1strip
registry: simplify `trimV1Address`
2023-08-29 12:47:54 +02:00
Sebastiaan van Stijn
1f5322a94e Merge pull request #46357 from thaJeztah/distribution_use_the_const
distribution: use registry consts for API version in tests
2023-08-29 12:29:06 +02:00
Paweł Gronowski
aef703fa1b integration/liveRestore: Check volume content
Make sure that the content in the live-restored volume mounted in a new
container is the same as the content in the old container.
This checks if volume's _data directory doesn't get unmounted on
startup.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-29 11:46:29 +02:00
Sebastiaan van Stijn
bea39a49bc Merge pull request #46344 from thaJeztah/libnetwork_drivers_remote_cleanup_tests
libnetwork/drivers/remote: tests: cleanup dead code, and return concrete types
2023-08-29 11:34:00 +02:00
Sebastiaan van Stijn
bb7b7522b1 Merge pull request #46343 from thaJeztah/libnetwork_drivers_bridge_cleanup_tests
libnetwork/drivers/bridge: tests: remove some dead-code and return concrete type
2023-08-29 11:33:32 +02:00
Sebastiaan van Stijn
94e924906c Merge pull request #46345 from thaJeztah/libnetwork_remove_Endpoint_Interface
libnetwork: remove Endpoint.Interface()
2023-08-29 11:33:10 +02:00
Sebastiaan van Stijn
4e4f62b6db distribution: use registry consts for API version in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:58:38 +02:00
Sebastiaan van Stijn
aa59b0f5a2 registry: improve error for invalid search endpoints
Explain that search is not supported on v2 endpoints, and include the
offending endpoint in the error-message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:47:29 +02:00
Sebastiaan van Stijn
14b53c6318 registry: simplify trimV1Address
First, remove the loop over `apiVersions`. The `apiVersions` map has two
entries (`APIVersion1 => "v1"`, and `APIVersion2 => "v2"`), and `APIVersion1`
is skipped, which means that the loop effectively translates to;

    if apiVersionStr == "v2" {
        return "", invalidParamf("unsupported V1 version path %s", apiVersionStr)
    }

Which leaves us with "anything else" being returned as-is.

This patch removes the loop, and replaces the remaining handling to check
for the "v2" suffix to produce an error, or to strip the "v1" suffix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:45:39 +02:00
Sebastiaan van Stijn
062c80199f registry: combine TestEndpointParse and TestEndpointParseInvalid
Combine the two tests into a TestV1EndpointParse function, and rewrite
them to use gotest.tools for asserting.

Also changing the test-cases to use "https://", as the scheme doesn't
matter for this test, but using "http://" may trip-up some linters,
so let's avoid that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:36:46 +02:00
Sebastiaan van Stijn
0f871f8cb7 api/types/events: define "Action" type and consts
Define consts for the Actions we use for events, instead of "ad-hoc" strings.
Having these consts makes it easier to find where specific events are triggered,
makes the events less error-prone, and allows documenting each Action (if needed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 00:38:08 +02:00
Sebastiaan van Stijn
10a3a3bc49 daemon: inline some variables when emitting events
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 00:38:08 +02:00
Bjorn Neergaard
8d404ac408 Merge pull request #46354 from thaJeztah/fix_TestLoadBufferedEventsOnlyFromPast
daemon/events; fix TestLoadBufferedEventsOnlyFromPast
2023-08-28 16:05:40 -06:00
Sebastiaan van Stijn
9331d6de1e daemon/events; fix TestLoadBufferedEventsOnlyFromPast
commit 70ad5b818f changed event.Type
to be a strong type, no longer an alias for string. for some reason,
this test passed on the PR, but failed later on;

    === Failed
    === FAIL: daemon/events TestLoadBufferedEventsOnlyFromPast (0.00s)
        events_test.go:203: assertion failed: network (messages[0].Type events.Type) != network (string)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 22:11:00 +02:00
Sebastiaan van Stijn
a65c948e7e Merge pull request #46335 from thaJeztah/api_move_checkpoint_types
api/types: move checkpoint-types to api/types/checkpoint
2023-08-28 19:02:19 +02:00
Sebastiaan van Stijn
8309206160 Merge pull request #46350 from thaJeztah/strongtype_eventstype
api/types/events: make events.Type an actual type
2023-08-28 16:44:26 +02:00
Sebastiaan van Stijn
51a1c5f3b4 Merge pull request #46336 from thaJeztah/events_deprecated_fields
Remove uses of deprecated "Status", "ID", and "From" fields for events
2023-08-28 16:35:06 +02:00
Sebastiaan van Stijn
7f6dda359b Merge pull request #46352 from thaJeztah/restart_policy_improve_error
api: ValidateRestartPolicy: improve errors for invalid policies
2023-08-28 15:14:11 +02:00
Sebastiaan van Stijn
328c6159f2 Merge pull request #46334 from thaJeztah/remove_empty_file
api/server/router/network: remove empty file
2023-08-28 14:40:52 +02:00
Sebastiaan van Stijn
f6f6c32138 api: ValidateRestartPolicy: improve errors for invalid policies
Make the error message slightly clearer on "what" part is not valid,
and provide suggestions on what are acceptable values.

Before this change:

    docker create --restart=always:3 busybox
    Error response from daemon: invalid restart policy: maximum retry count cannot be used with restart policy 'always'

    docker create --restart=always:-1 busybox
    Error response from daemon: invalid restart policy: maximum retry count cannot be used with restart policy 'always'

    docker create --restart=unknown busybox
    Error response from daemon: invalid restart policy 'unknown'

After this change:

    docker create --restart=always:3 busybox
    Error response from daemon: invalid restart policy: maximum retry count can only be used with 'on-failure'

    docker create --restart=always:-1 busybox
    Error response from daemon: invalid restart policy: maximum retry count can only be used with 'on-failure' and cannot be negative

    docker create --restart=unknown busybox
    Error response from daemon: invalid restart policy: unknown policy 'unknown'; use one of 'no', 'always', 'on-failure', or 'unless-stopped'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 14:00:58 +02:00
Sebastiaan van Stijn
333e04e84e integration-cli: TestEventsFormat: rewrite test to not use deprecated fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:17:01 +02:00
Sebastiaan van Stijn
fa79b5d59f integration/container: TestPause: don't depend on deprecated fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:17:01 +02:00
Sebastiaan van Stijn
0161aad462 client: TestEvents: don't depend on deprecated fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:17:01 +02:00
Sebastiaan van Stijn
5123ae4352 daemon: TestLogContainerEventCopyLabels: inline vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:17:01 +02:00
Sebastiaan van Stijn
2928c5ae92 daemon: TestHealthStates: don't depend on deprecated fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:17:00 +02:00
Sebastiaan van Stijn
62ec14d9ec daemon/events: verify non-deprecated fields
Some tests were testing the deprecated fields, instead of their non-deprecated
alternatives.

This patch adds a utility to verify that they match, and rewrites the tests
to check the non-deprecated fields instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:17:00 +02:00
Sebastiaan van Stijn
f443006fa2 daemon/events: rewrite test-assertions with gotest.tools
- clean up "//import" comment, as test-files cannot be imported, and only
  one "//import" comment is needed per package.
- remove some intermediate variables
- rewrite assertions to use gotest.tools
- use assert.Check()) (non-fatal) where possible

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:16:57 +02:00
Sebastiaan van Stijn
70ad5b818f api/types/events: make events.Type an actual type
This type was added in 247f4796d2, and
at the time was added as an alias for string;

> api/types/events: add "Type" type for event-type enum
>
> Currently just an alias for string, but we can change it to be an
> actual type.

Now that all code uses the defined types, we should be able to make
this an actual type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:12:38 +02:00
Sebastiaan van Stijn
8569e8684f Merge pull request #46338 from thaJeztah/daemon_events_cleanup
daemon: clean up event handling-code, and remove some dead code
2023-08-28 13:12:10 +02:00
Sebastiaan van Stijn
3fcb3d94c7 Merge pull request #46339 from thaJeztah/no_min_max
rename uses of "max", "min", which are now builtins in go1.21
2023-08-28 13:08:28 +02:00
Sebastiaan van Stijn
26a9d6474a Merge pull request #46349 from thaJeztah/update_crun
Dockerfile: update crun binary to v1.8.7
2023-08-28 11:36:53 +02:00
Sebastiaan van Stijn
5a02ed5e84 integration: use events-consts in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 23:44:25 +02:00
Sebastiaan van Stijn
9ea50365d6 daemon/events: use events-consts in tests, and fix vars that collided
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 22:23:29 +02:00
Sebastiaan van Stijn
2f8e957713 Dockerfile: update crun binary to v1.8.7
Updating the version of crun that we use in our tests to a version that
supports the "features" command (crun v1.8.6 and up). This should prevent
some warnings in our logs:

    WARN[2023-08-26T17:05:35.042978552Z] Failed to run [/usr/local/bin/crun features]: "unknown command features\n"  error="exit status 1"

full diff: https://github.com/containers/crun/compare/1.4.5...1.8.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 21:43:00 +02:00
Sebastiaan van Stijn
32e79c464a libnetwork: Network: return early where possible
Add a fast-patch to some functions, to prevent locking/unlocking,
or other operations that would not be needed;

- Network.addDriverInfoToCluster
- Network.deleteDriverInfoFromCluster
- Network.addServiceInfoToCluster
- Network.deleteServiceInfoFromCluster
- Network.addDriverWatches

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:47:18 +02:00
Sebastiaan van Stijn
5512c7e838 libnetwork: Network.Services: remove some intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:40:31 +02:00
Sebastiaan van Stijn
d6b8298cfd libnetwork: Network.Services: return early and don't lock in a loop
- return early when failing to fetch the driver
- store network-ID and controller in a variable to prevent repeatedly
  locking/unlocking. We don't expect the network's ID to change
  during this operation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:40:31 +02:00
Sebastiaan van Stijn
8b21609654 libnetwork: remove Endpoint.Interface
This method is not part of any interface, and identical to Endpoint.Iface,
but one returns an Interface-type (driverapi.InterfaceInfo) and the other
returns a concrete type (EndpointInterface).

Interface-matching should generally happen on the receiver side, and this
function was only used in a single location, and passed as argument to
Driver.CreateEndpoint, which already matches the interface by accepting
a driverapi.InterfaceInfo.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:24:32 +02:00
Sebastiaan van Stijn
f8181b8875 libnetwork: Endpoint.Iface, Endpoint.Interface remove redundant "if"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:24:32 +02:00
Sebastiaan van Stijn
6a61bacafe libnetwork/drivers/remote: rollbackEndpoint.Interface: return concrete type
Interface-matching should generally happen on the receiver side, and this
function was only used in a single location, and passed as argument to
Driver.CreateEndpoint, which already matches the interface by accepting
a driverapi.InterfaceInfo.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:18:31 +02:00
Sebastiaan van Stijn
dc061d2ed3 libnetwork/drivers/remote: driver.DeleteNetwork: remove var that collided
Remove the var, because `delete` is a builtin.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:18:30 +02:00
Sebastiaan van Stijn
40b1389917 libnetwork/drivers/remote: remove unused testEndpoint.Interface
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:18:30 +02:00
Sebastiaan van Stijn
15435f7293 libnetwork/drivers/bridge: testEndpoint.Interface: return concrete type
Interface-matching should generally happen on the receiver side, and this
function was only used in a single location, and passed as argument to
Driver.CreateEndpoint, which already matches the interface by accepting
a driverapi.InterfaceInfo.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:13:27 +02:00
Sebastiaan van Stijn
9afb688f5f libnetwork/drivers/bridge: getIPv4Data: remove unused argument
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:13:27 +02:00
Sebastiaan van Stijn
639449f84e libnetwork: nwAgent.bindAddr: change to net.IP
Store the IP-address as a net.IP instead of a string.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:08:42 +02:00
Sebastiaan van Stijn
b75e831567 libnetwork: resolveAddr: add GoDoc
Describe the behavior of this function, as it was not documented.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-27 20:08:39 +02:00
Sebastiaan van Stijn
5b53ddfcdd Merge pull request #46340 from thaJeztah/carry_46292_update_readme
Update example in README.md to close cli object (carry 46292)
2023-08-27 00:01:05 +02:00
Sebastiaan van Stijn
ebe2347ac6 daemon: remove LogPluginEventWithAttributes as it's not used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:04:43 +02:00
Sebastiaan van Stijn
aa764e6009 daemon: logNetworkEvent, logSecretEvent, logConfigEvent rm unused args
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:04:43 +02:00
Sebastiaan van Stijn
fa13b0715f integration/internal/swarm: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
79495c5b6a pkg/tailfile: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
df2f25a977 pkg/plugins: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
6c036f267f pkg/sysinfo: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
55192de9e3 pkg/archive: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
a3867992b7 daemon: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:21 +02:00
msadiq058
f6b2e34268 Update example in README.md to close cli object
Signed-off-by: Mohd Sadiq <mohdsadiq058@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 19:55:45 +02:00
Sebastiaan van Stijn
318b3d4fe5 api/types/versions: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 19:38:47 +02:00
Sebastiaan van Stijn
cb394a62e5 restartmanager: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 19:37:19 +02:00
Sebastiaan van Stijn
8f64e2e925 daemon: daemon.logClusterEvent: use events.Type for event-types
Also swapping the order of arguments; putting the "attributes" arguments
last, so that variables can be more cleanly inlined.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 19:20:42 +02:00
Sebastiaan van Stijn
3dea2f230a daemon: inline some vars when producing events
Also moves the clusterEventAction closer to where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 17:45:21 +02:00
Sebastiaan van Stijn
634f6ec1d4 Merge pull request #46333 from thaJeztah/remove_devicemapper_leftovers
hack, Dockerfile: remove devicemapper leftover,  build-tags, and libdevmapper-dev dependency
2023-08-26 16:42:58 +02:00
Sebastiaan van Stijn
385f6429be api/server/router/network: remove empty file
The content of this file was removed in c0bc14e8dd,
and all it container since was the package name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:51:19 +02:00
Sebastiaan van Stijn
350223201e integration/container: TestCheckpoint: remove intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:44:16 +02:00
Sebastiaan van Stijn
b688af2226 api/types: move checkpoint-types to api/types/checkpoint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:37:41 +02:00
Sebastiaan van Stijn
70baaec6a5 Dockerfile: remove libdevmapper-dev
The devicemapper graphdriver has been removed in commit
dc11d2a2d8, and we should
no longer need this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 16:46:16 +02:00
Sebastiaan van Stijn
12d1f4f385 hack: remove devicemapper leftovers and build-tags
This check was added in 98fe4bd8f1, to check
whether dm_task_deferred_remove could be removed, and to conditionally set
the corresponding build-tags. Now that the devicemapper graphdriver has been
removed in dc11d2a2d8, we no longer need this.

This patch:

- removes uses of the (no longer used) `libdm`, `dlsym_deferred_remove`,
  and `libdm_no_deferred_remove` build-tags.
- removes the `add_buildtag` utility, which is now unused.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 16:39:33 +02:00
Sebastiaan van Stijn
3e5b2a6ef6 Merge pull request #46323 from thaJeztah/api_docs_update_urls
docs: update links to Go documentation
2023-08-25 14:14:39 +02:00
Sebastiaan van Stijn
4824c60893 Merge pull request #46324 from thaJeztah/hack_less_redirects
Dockerfile: Windows: update Golang download domains to cut down redirects
2023-08-25 13:49:32 +02:00
Sebastiaan van Stijn
f6a5318f94 Dockerfile: Windows: update Golang download domains to cut down redirects
The `golang.org` domain moved to `go.dev`, and the download-URLs we were
using resulted in 2 redirects;

    curl -sI https://golang.org/dl/go1.20.windows-amd64.zip | grep 'location'
    location: https://go.dev/dl/go1.20.windows-amd64.zip

    curl -sI https://go.dev/dl/go1.20.windows-amd64.zip | grep 'location'
    location: https://dl.google.com/go/go1.20.windows-amd64.zip

    curl -sI https://dl.google.com/go/go1.20.windows-amd64.zip
    HTTP/2 200
    # ...

This patch cuts it down to one redirects. I decided not to use the "final"
(`dl.google.com`) URL, because that URL is not documented in the Golang docs,
and visiting the domain itself (https://dl.google.com/) redirects to a marketing
page for "Google Chrome".

Trying the `/go/` path (https://dl.google.com/go/) also does not show a landing
page that lists downloads, so I'm considering those URLs to be "unstable".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 12:53:19 +02:00
Sebastiaan van Stijn
2aabd64477 hack: update link to GOPATH documentation
This documentation moved to a different page, and the Go documentation
moved to the https://go.dev/ domain.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 12:53:19 +02:00
Sebastiaan van Stijn
b18e170631 CONTRIBUTING.md: update links to golang docs and blog
- docs moved to https://go.dev/doc/
- blog moved to https://go.dev/blog/

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 12:43:31 +02:00
Sebastiaan van Stijn
136e86bb5c api: swagger: update link to Go documentation
Go documentation moved to the `go.dev` domain;

    curl -sI https://golang.org/doc/install/source#environment | grep 'location'
    location: https://go.dev/doc/install/source

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 12:43:30 +02:00
Sebastiaan van Stijn
4862d39144 docs/api: update links to Go documentation
Go documentation moved to the `go.dev` domain;

    curl -sI https://golang.org/doc/install/source#environment | grep 'location'
    location: https://go.dev/doc/install/source

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 12:43:30 +02:00
Sebastiaan van Stijn
b249376bb0 Merge pull request #46321 from thaJeztah/update_golangci_lint
update golangci-lint to v1.54.2
2023-08-25 12:43:07 +02:00
Sebastiaan van Stijn
cd49f9affd update golangci-lint to v1.54.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 02:19:35 +02:00
Sebastiaan van Stijn
6d5204b63e Merge pull request #46316 from thaJeztah/rm_more_buildtags
remove some remaining pre-go1.17 build-tags
2023-08-25 01:06:39 +02:00
Sebastiaan van Stijn
f10e182ca7 Merge pull request #46317 from thaJeztah/gofumpt_all_the_things
Format code with gofumpt
2023-08-25 01:06:05 +02:00
Sebastiaan van Stijn
cfd431d4d5 Merge pull request #46211 from thaJeztah/daemon_error_cleanup
daemon: WithNamespaces(): improve error-handling, and assorted cleanups
2023-08-24 19:23:18 +02:00
Sebastiaan van Stijn
d52f20048d api/server: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 18:00:19 +02:00
Sebastiaan van Stijn
e9b09b49fd builder/builder-next: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:50 +02:00
Sebastiaan van Stijn
73894af9ff contrib/apparmor: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:35 +02:00
Sebastiaan van Stijn
ce01d779df daemon: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:22 +02:00
Sebastiaan van Stijn
211d3a8c32 pkg/stack: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:01 +02:00
Sebastiaan van Stijn
95e99e629c pkg/fileutils: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:58:46 +02:00
Sebastiaan van Stijn
6c6e1fb13d pkg/containerfs: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:58:32 +02:00
Sebastiaan van Stijn
1e579a7a2a pkg/sysinfo: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:58:12 +02:00
Sebastiaan van Stijn
263c00f27c testutil: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:57:54 +02:00
Sebastiaan van Stijn
b71951c70c container: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:57:34 +02:00
Sebastiaan van Stijn
87cedbe443 image: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:57:22 +02:00
Sebastiaan van Stijn
976da91fb7 pkg/loopback: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:56:50 +02:00
Sebastiaan van Stijn
b4d6eca9b8 pkg/chrootarchive: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:56:24 +02:00
Sebastiaan van Stijn
07e6b0ac70 integration: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:55:14 +02:00
Sebastiaan van Stijn
498da44aab remove some remaining pre-go1.17 build-tags
commit ab35df454d removed most of the pre-go1.17
build-tags, but for some reason, "go fix" doesn't remove these, so removing
the remaining ones manually

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:51:07 +02:00
Sebastiaan van Stijn
79f5aac2d0 Merge pull request #46314 from thaJeztah/libnetwork_endpointinfo_godoc
libnetwork: add godoc to Endpoint
2023-08-24 17:33:11 +02:00
Sebastiaan van Stijn
713066accc Merge pull request #46304 from thaJeztah/libnetwork_add_del_checkfirewalld
libnetwork/iptables: (Add|Del)InterfaceFirewalld: check firewalld status
2023-08-24 17:06:39 +02:00
Sebastiaan van Stijn
210abfaef6 libnetwork: add godoc to Endpoint
Copying relevant documentation from the EndpointInfo interface. We should
remove this interface, and the related Info() function, but it's currently
acting as a "gate" to prevent accessing the Endpoint's accessors without
making sure it's fully hydrated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:36:33 +02:00
Sebastiaan van Stijn
53a6661684 Merge pull request #46305 from thaJeztah/libnetwork_remove_sandbox_interface
libnetwork/osl: remove Sandbox and Info interfaces
2023-08-24 16:23:32 +02:00
Sebastiaan van Stijn
04422f5ec1 daemon: WithNamespaces(): add notes about user-namespaces
While working on this code, I noticed that there's currently an issue
with userns enabled. When userns is enabled, joining another container's
namespace must also join its user-namespace.

However, a container can only be in a single user namespace, so if a
container joins namespaces from multiple containers, latter user-namespaces
overwrite former ones.

We must add validation for this, but in the meantime, add notes / todo's.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:21:26 +02:00
Sebastiaan van Stijn
dd26e6b15e daemon: Daemon.getIpcContainer: make errors less repetitive
- Most error-message returned would already include "container" and the
  container ID in the error-message (e.g. "container %s is not running"),
  so there's no need to add a custom prefix for that.
- os.Stat returns a PathError, which already includes the operation ("stat"),
  the path, and the underlying error that occurred.

And while updating, let's also fix the name to be proper camelCase :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:20:42 +02:00
Sebastiaan van Stijn
3d94eb9bcd daemon: Daemon.getPidContainer: change to accept "id" argument
This function didn't need the whole container, only its ID, so let's
use that as argument. This also makes it consistent with getIpcContainer.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:20:42 +02:00
Sebastiaan van Stijn
bc7f341f29 daemon: WithNamespaces(): fix incorrect error for PID, IPC namespace
`Daemon.getPidContainer()` was wrapping the error-message with a message
("cannot join PID of a non running container") that did not reflect the
actual reason for the error; `Daemon.GetContainer()` could either return
an invalid parameter (invalid / empty identifier), or a "not found" error
if the specified container-ID could not be found.

In the latter case, we don't want to return a "not found" error through
the API, as this would indicate that the container we're _starting_ was
not found (which is not the case), so we need to convert the error into
an `errdefs.ErrInvalidParameter` (the container-ID specified for the PID
namespace is invalid if the container doesn't exist).

This logic is similar to what we do for IPC namespaces. which received
a similar fix in c3d7a0c603.

This patch updates the error-types, and moves them into the getIpcContainer
and getPidContainer container functions, both of which should return
an "invalid parameter" if the container was not found.

It's worth noting that, while `WithNamespaces()` may return an "invalid
parameter" error, the `start` endpoint itself may _not_ be. as outlined
in commit bf1fb97575, starting a container
that has an invalid configuration should be considered an internal server
error, and is not an invalid _request_. However, for uses other than
container "start", `WithNamespaces()` should return the correct error
to allow code to handle it accordingly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:19:07 +02:00
Sebastiaan van Stijn
bd5d6480e7 daemon: WithNamespaces(): remove redundant "if"
This check was originally used to only validate the mode if it was set to
a non-empty value (see commit 072400fc4b), but
validation was made unconditional in c3d7a0c603.

Given that a `CgroupnsMode` can't be both [`CgroupnsMode.IsEmpty()`][1]
and [`CgroupnsMode.IsPrivate`][2], we can remove the extra check.

[1]: e0da5cb929/api/types/container/hostconfig.go (L33-L36)
[2]: e0da5cb929/api/types/container/hostconfig.go (L23-L26)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:12:22 +02:00
Sebastiaan van Stijn
6eaefe5bf5 daemon: WithNamespaces(): use switch instead of if/else if/else
We were using a mixture of approaches for these; aligning them a bit
to all use switch statements.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:12:22 +02:00
Sebastiaan van Stijn
43aa67a982 daemon: WithNamespaces(): use OCI-spec consts for namespaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:12:21 +02:00
Sebastiaan van Stijn
759698581e daemon: WithNamespaces(): inline variables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:12:21 +02:00
Sebastiaan van Stijn
13648a0e21 daemon: remove Daemon.checkContainer and related utils
This was added in 12485d62ee to save some
duplication, but was really over-engineered to save a few lines of code,
at the cost of hiding away what it does and also potentially returning
inconsistent errors (not addressed in this patch). Let's start with
inlining these.

This removes;

- Daemon.checkContainer
- daemon.containerIsRunning
- daemon.containerIsNotRestarting

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 16:12:18 +02:00
Sebastiaan van Stijn
56597db804 Merge pull request #46312 from rumpl/test-download-distribution
test/integration: Download the registry binary
2023-08-24 16:11:11 +02:00
Djordje Lukic
78828f9b58 test/integration: Download the registry binary
We used to have to clone and build the registry v2 but now that we have
updated the version we can directtly copy the binary from the official
distribution/distribution image.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-24 15:06:43 +02:00
Sebastiaan van Stijn
0d9cf9e66b Merge pull request #46309 from oss-qm/submit/fix-src-perms
Fix some broken executable flags
2023-08-24 15:04:45 +02:00
Sebastiaan van Stijn
64f5d9b119 Merge pull request #46213 from thaJeztah/daemon_remove_errors
daemon: cleanupContainer: don't fail if container is already stopped
2023-08-24 13:34:43 +02:00
Sebastiaan van Stijn
1fa586de52 Merge pull request #46307 from rumpl/test-update-registry
test: update registry version to latest
2023-08-24 13:34:10 +02:00
Sebastiaan van Stijn
b4c59d25d6 Merge pull request #46301 from vvoland/c8d-legacybuilder-fix-layermismatch
c8d/legacybuilder: Fix `mismatched image rootfs` errors
2023-08-24 12:45:06 +02:00
Enrico Weigelt, metux IT consult
bd2c553870 Fix some broken executable flags
Some non-executable files had executable mode flag.

Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
2023-08-24 12:23:59 +02:00
Djordje Lukic
143b3b2ef3 test: update registry version to latest
The one used in the integration tests was 6 years old.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-24 12:08:32 +02:00
Paweł Gronowski
01214bafd2 c8d/builder: Don't drop fields from created image
Previous image created a new partially filled image.
This caused child images to lose their parent's layers.

Instead of creating a new object and trying to replace its fields, just
clone the original passed image and change its ID to the manifest
digest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-24 09:44:12 +02:00
Sebastiaan van Stijn
b92cf68add libnetwork/osl: remove Sandbox and Info interfaces
It only has a single implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 23:29:09 +02:00
Sebastiaan van Stijn
b63400fa4a Merge pull request #46149 from thaJeztah/libnetwork_remove_interface_interface
libnetwork/osl: remove Interface Interface and other funny stories
2023-08-23 23:28:41 +02:00
Sebastiaan van Stijn
9668613541 Merge pull request #46303 from thaJeztah/TestDiskUsage_no_panic
TestDiskUsage: don't panic if results don't match
2023-08-23 21:07:02 +02:00
Sebastiaan van Stijn
c881360e88 libnetwork/osl: remove Interface.Master() as it's unused
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:15 +02:00
Sebastiaan van Stijn
b9bf407929 libnetwork/osl: remove Interface Interface
There's only one implementation; let's use that.
Also fixing a linting issue;

    libnetwork/osl/interface_linux.go:91:2: S1001: should use copy(to, from) instead of a loop (gosimple)
        for i, iface := range n.iFaces {
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:15 +02:00
Sebastiaan van Stijn
8631e69cdf libnetwork/osl: remove Sandbox.InterfaceOptions() etc.
InterfaceOptions() returned an IfaceOptionSetter interface, which contained
"methods" that returned functional options. Such a construct could have made
sense if the functional options returned would (e.g.) be pre-propagated with
information from the Sandbox (network namespace), but none of that was the case.

There was only one implementation of IfaceOptionSetter (networkNamespace),
which happened to be the same as the only implementation of Sandbox, so remove
the interface as well, to help networkNamespace with its multi-personality
disorder.

This patch:

- removes Sandbox.Bridge() and makes it a regular function (WithIsBridge)
- removes Sandbox.Master() and makes it a regular function (WithMaster)
- removes Sandbox.MacAddress() and makes it a regular function (WithMACAddress)
- removes Sandbox.Address() and makes it a regular function (WithIPv4Address)
- removes Sandbox.AddressIPv6() and makes it a regular function (WithIPv6Address)
- removes Sandbox.LinkLocalAddresses() and makes it a regular function (WithLinkLocalAddresses)
- removes Sandbox.Routes() and makes it a regular function (WithRoutes)
- removes Sandbox.InterfaceOptions().
- removes the IfaceOptionSetter interface.

Note that the IfaceOption signature was changes as well to allow returning
an error. This is not currently used, but will be used for some options
in the near future, so adding that in preparation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:15 +02:00
Sebastiaan van Stijn
f3d29db6a2 libnetwork/osl: remove Sandbox.NeighborOptions() etc.
NeighborOptions() returned an NeighborOptionSetter interface, which
contained "methods" that returned functional options. Such a construct
could have made sense if the functional options returned would (e.g.)
be pre-propagated with information from the Sandbox (network namespace),
but none of that was the case.

There was only one implementation of NeighborOptionSetter (networkNamespace),
which happened to be the same as the only implementation of Sandbox, so
remove the interface as well, to help networkNamespace with its multi-personality
disorder.

This patch:

- removes Sandbox.LinkName() and makes it a regular function (WithLinkName)
- removes Sandbox.Family() and makes it a regular function (WithFamily)
- removes Sandbox.NeighborOptions().
- removes the NeighborOptionSetter interface

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:15 +02:00
Sebastiaan van Stijn
a365fb0e9d libnetwork: move more osl.Sandbox related code to Linux-only files
osl.NewSandbox() always returns a nil interface on Windows (and other non-Linux
platforms). This means that any code that these fields are always nil, and
any code using these fields must be considered Linux-only.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:15 +02:00
Sebastiaan van Stijn
95abde479a libnetwork: implement Controller.setupOSLSandbox
osl.NewSandbox() always returns a nil interface on Windows (and other non-Linux
platforms). This means that any code that these fields are always nil, and
any code using these fields must be considered Linux-only;

- libnetwork/Controller.defOsSbox
- libnetwork/Sandbox.osSbox

Ideally, these fields would live in Linux-only files, but they're referenced
in various platform-neutral parts of the code, so let's start with moving
the initialization code to Linux-only files.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:15 +02:00
Sebastiaan van Stijn
35456d2eb1 libnetwork/osl: add godoc to networkNamespace
Copying the descriptions from the Sandbox, Info, NeighborOptionSetter,
and IfaceOptionSetter interfaces that it implements.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 20:13:14 +02:00
Sebastiaan van Stijn
aecfa55c4c libnetwork/iptables: (Add|Del)InterfaceFirewalld: check firewalld status
Check if firewalld is running before running the function, so that consumers
of the function don't have to check for the status.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 19:58:54 +02:00
Sebastiaan van Stijn
0e3b2ec267 Merge pull request #46245 from thaJeztah/firewalld_dont_fail_on_removal
libnetwork/iptables: ProgramChain: don't fail if interface not found
2023-08-23 19:58:18 +02:00
Sebastiaan van Stijn
53afd2ae9f Merge pull request #46201 from thaJeztah/daemon_updateSandboxNetworkSettings_err
libnetwork: remove unused err-return, and minor refactor around Sandbox creating
2023-08-23 19:56:58 +02:00
Sebastiaan van Stijn
d74f0a47de Merge pull request #46207 from thaJeztah/daemon_cleanup_withLibnetwork
daemon: withLibnetwork(): return early if networking is disabled
2023-08-23 19:56:01 +02:00
Sebastiaan van Stijn
71f4916357 Merge pull request #46246 from thaJeztah/fix_TestFirewalldInit
libnetwork/iptables: fix TestFirewalldInit
2023-08-23 19:38:08 +02:00
Sebastiaan van Stijn
4b0d38de06 TestDiskUsage: don't panic if results don't match
This test is currently failing with containerd-integration, which should
be looked into, but let's start with preventing it from panicking, to make
the test-failures less noisy;

        --- FAIL: TestDiskUsage/after_container.Run (0.26s)
    panic: runtime error: index out of range [0] with length 0 [recovered]
        panic: runtime error: index out of range [0] with length 0

    goroutine 280 [running]:
    testing.tRunner.func1.2({0xb07a00, 0x40002006a8})
        /usr/local/go/src/testing/testing.go:1526 +0x1c8
    testing.tRunner.func1()
        /usr/local/go/src/testing/testing.go:1529 +0x364
    panic({0xb07a00, 0x40002006a8})
        /usr/local/go/src/runtime/panic.go:884 +0x1f4
    github.com/docker/docker/integration/system.TestDiskUsage.func3(0x0?, {0x0, {0x14ea4a8, 0x0, 0x0}, {0x14ea4a8, 0x0, 0x0}, {0x14ea4a8, 0x0, ...}, ...})
        /go/src/github.com/docker/docker/integration/system/disk_usage_test.go:82 +0x7e4
    github.com/docker/docker/integration/system.TestDiskUsage.func4(0x4000235c80?)
        /go/src/github.com/docker/docker/integration/system/disk_usage_test.go:118 +0x8c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 19:15:21 +02:00
Sebastiaan van Stijn
9b9348ce86 Merge pull request #46284 from vvoland/c8d-legacybuilder-fix-from-scratch
c8d: Fix building Dockerfiles that have `FROM scratch`
2023-08-23 18:57:53 +02:00
Sebastiaan van Stijn
e2fc596de4 Merge pull request #46293 from rumpl/fix-platform-check
Don't return lease not found errors on platform mismatch
2023-08-23 15:55:04 +02:00
Sebastiaan van Stijn
c0568a95d8 integration/container: check some error-types in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 15:50:46 +02:00
Sebastiaan van Stijn
2b583c0923 daemon: cleanupContainer: slightly cleanup error messages
Also remove integration-cli: `DockerAPISuite.TestContainerAPIDeleteConflict`,
which was testing the same conditions as `TestRemoveContainerRunning` in
integration/container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 15:50:43 +02:00
Sebastiaan van Stijn
69cf2ad6a5 daemon: cleanupContainer: don't fail if container is already stopped
Saw this failure in a flaky test, and I wondered why we consider this an
error condition;

    === RUN   TestKillWithStopSignalAndRestartPolicies
        main_test.go:32: assertion failed: error is not nil: Error response from daemon: Could not kill running container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7, cannot remove - Container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7 is not running: failed to remove 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7
    --- FAIL: TestKillWithStopSignalAndRestartPolicies (0.84s)
    === RUN   TestKillWithStopSignalAndRestartPolicies/same-signal-disables-restart-policy
        --- PASS: TestKillWithStopSignalAndRestartPolicies/same-signal-disables-restart-policy (0.42s)
    === RUN   TestKillWithStopSignalAndRestartPolicies/different-signal-keep-restart-policy
        --- PASS: TestKillWithStopSignalAndRestartPolicies/different-signal-keep-restart-policy (0.23s)

In the above;

1. `Error response from daemon: Could not kill running container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7`
2. `cannot remove - Container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7 is not running`
3. `failed to remove 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7`

So it looks like the removal fails because we couldn't kill the container
because it was already stopped, which may be a race condition where the first
check shows the container to be running (but may already be in process to be
removed or killed. In either case, we probably shouldn't fail the removal if
the container is already stopped.

This patch adds a `isNotRunning()` utility, so that we can ignore this case,
and proceed with the removal.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 15:47:43 +02:00
Sebastiaan van Stijn
20b770237c daemon/network: Settings: move sandbox fields together
Just a minor nit; let's move these fields together.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 14:26:58 +02:00
Sebastiaan van Stijn
405f571ae8 daemon: updateSandboxNetworkSettings: remove unused error-return
This function never returns an error, so let's remove the error-return,
and give it a slightly more to-the-point name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 14:26:58 +02:00
Sebastiaan van Stijn
2eccf0e8d1 libnetwork: Controller.NewSandbox: don't generate ID if not used
Windows uses the container-iD as ID for sandboxes, so it's not needed to
generate an ID  when running on Windows.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 14:26:53 +02:00
Sebastiaan van Stijn
389b21a341 Merge pull request #46262 from thaJeztah/libnetwork_resolv_cleanups
libnetwork: resolve: assorted cleanups
2023-08-23 14:22:33 +02:00
Sebastiaan van Stijn
13c4eaea92 Merge pull request #46205 from thaJeztah/libnetwork_noexecroot
libnetwork: cleanup SetBasePath, un-export SetExternalKey and other cleanups
2023-08-23 14:21:30 +02:00
Sebastiaan van Stijn
8f0b62e4d9 Merge pull request #46226 from thaJeztah/conntrack_reduce_noise
libnetwork/iptables: reduce DeleteConntrackEntriesXX debug logs
2023-08-23 14:19:53 +02:00
Sebastiaan van Stijn
bb5813a93d Merge pull request #46240 from thaJeztah/libnetwork_named_returns
libnetwork: Controller.NewNetwork, Controller.NewSandbox: use named error-return
2023-08-23 14:19:27 +02:00
Sebastiaan van Stijn
cec49ba976 Merge pull request #46247 from thaJeztah/libnetwork_unexport_ZoneSettings
libnetwork/iptables: un-export ZoneSettings, and slight refactor
2023-08-23 14:17:38 +02:00
Sebastiaan van Stijn
0e7186af95 Merge pull request #46278 from thaJeztah/libnetwork_remove_InterfaceInfo_interface
libnetwork: remove InterfaceInfo interface
2023-08-23 14:16:23 +02:00
Sebastiaan van Stijn
8c3b14ee85 Merge pull request #46264 from thaJeztah/libnetwork_sandbox_split_options
libnetwork: move all SandboxOptions to a separate file
2023-08-23 14:13:52 +02:00
Sebastiaan van Stijn
77539e7bc7 Merge pull request #46279 from thaJeztah/libnetwork_remove_sandbox_info
libnetwork/osl: remove Sandbox.Info()
2023-08-23 14:13:26 +02:00
Sebastiaan van Stijn
79fa17e181 Merge pull request #46298 from rumpl/c8d-test-skip-graphdriver
test: Skip graph driver tests when using containerd snapshotters
2023-08-23 14:00:58 +02:00
Sebastiaan van Stijn
18c5d863d6 Merge pull request #46296 from thaJeztah/replace_dockerignore
replace dockerfile/dockerignore with patternmatcher/ignorefile
2023-08-23 10:55:14 +02:00
Djordje Lukic
6cbe06ff3f test: Skip graph driver tests when using containerd snapshotters
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-23 09:48:27 +02:00
Sebastiaan van Stijn
3553b4c684 replace dockerfile/dockerignore with patternmatcher/ignorefile
The BuildKit dockerignore package was integrated in the patternmatcher
repository / module. This patch updates our uses of the BuildKit package
with its new location.

A small local change was made to keep the format of the existing error message,
because the "ignorefile" package is slightly more agnostic in that respect
and doesn't include ".dockerignore" in the error message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 00:53:18 +02:00
Sebastiaan van Stijn
a479b287c7 vendor: github.com/moby/patternmatcher v0.6.0
- integrate frontend/dockerfile/dockerignore from buildkit

full diff: https://github.com/moby/patternmatcher/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 00:50:05 +02:00
Sebastiaan van Stijn
70ae5c13ea Merge pull request #44704 from thaJeztah/api_hostconfig_RestartPolicyMode_enum
api/types/container: add RestartPolicyMode type and enum
2023-08-22 22:31:51 +02:00
Bjorn Neergaard
8383430946 Merge pull request #45759 from akerouanton/validate-ipam-config
Validate IPAM config before handing it over to libnetwork
2023-08-22 13:58:28 -06:00
Paweł Gronowski
bedcc94de4 daemon: Handle NotFound when deleting container lease
If the lease doesn't exit (for example when creating the container
failed), just ignore the not found error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:22 +02:00
Paweł Gronowski
dfaff9598c c8d/run: Allow running container without image
This allows the legacy builder to apply changes to the `FROM scratch`
layer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:20 +02:00
Paweł Gronowski
eb56493f4e c8d/commit: Don't produce an empty layer
If the diff is empty and don't produce an empty layer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:18 +02:00
Albin Kerouanton
3e8af0817a ipam: Replace ChildSubnet with parent Subnet when its mask is bigger
Prior to moby/moby#44968, libnetwork would happily accept a ChildSubnet
with a bigger mask than its parent subnet. In such case, it was
producing IP addresses based on the parent subnet, and the child subnet
was not allocated from the address pool.

This commit automatically fixes invalid ChildSubnet for networks stored
in libnetwork's datastore.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-22 17:12:06 +02:00
Albin Kerouanton
4f47013feb api: Validate IPAM config before creating a network
Currently, IPAM config is never validated by the API. Some checks
are done by the CLI, but they're not exhaustive. And some of these
misconfigurations might be caught early by libnetwork (ie. when the
network is created), and others only surface when connecting a container
to a misconfigured network. In both cases, the API would return a 500.

Although the `NetworkCreate` endpoint might already return warnings,
these are never displayed by the CLI. As such, it was decided during a
maintainer's call to return validation errors _for all API versions_.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-22 17:11:54 +02:00
Sebastiaan van Stijn
2be118379e api/types/container: add RestartPolicyMode type and enum
Also move the validation function to live with the type definition,
which allows it to be used outside of the daemon as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-22 16:40:57 +02:00
Djordje Lukic
b8ff8ea58e Don't return an error if the lease is not found
If the image for the wanted platform doesn't exist then the lease
doesn't exist either. Returning this error hides the real error, so
let's not return it.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-22 14:52:41 +02:00
Sebastiaan van Stijn
331854a126 Merge pull request #46239 from thaJeztah/vars_collide
libnetwork:  rename vars that collided
2023-08-22 10:50:35 +02:00
Sebastiaan van Stijn
fa517bb420 Merge pull request #46271 from neersighted/bin_image_meta
bin-image: metadata cleanup, take two
2023-08-21 21:43:04 +02:00
Sebastiaan van Stijn
5d15da8290 Merge pull request #46206 from thaJeztah/libnetwork_no_walk
libnetwork: implement Controller.GetSandbox, remove Controller.WalkSandboxes and related code
2023-08-21 21:21:16 +02:00
Sebastiaan van Stijn
3ffa00909d Merge pull request #46285 from thaJeztah/gha_report_timeout
gha: set 10-minute timeout on "report" actions
2023-08-21 20:19:39 +02:00
Bjorn Neergaard
764419ed4e Merge pull request #45534 from polarathene/fix/config-init-limits
fix: Normalize `RLIMIT_NOFILE` to sensible defaults
2023-08-21 12:09:38 -06:00
Brian Goff
fc15e08a6c Merge pull request #46137 from thaJeztah/v1_deprecation 2023-08-21 10:51:23 -07:00
Sebastiaan van Stijn
dff53a02ef libnetwork: remove Controller.Sandboxes as it's no longer used
The Controller.Sandboxes method was used by some SandboxWalkers. Now
that those have been removed, there are no longer any consumers of this
method, so let's remove it for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 19:08:43 +02:00
Sebastiaan van Stijn
d6f340e784 gha: set 10-minute timeout on "report" actions
I had a CI run fail to "Upload reports":

    Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
    Finished backoff for retry #1, continuing with upload
    Total file count: 211 ---- Processed file #160 (75.8%)
    ...
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    A 503 status code has been received, will attempt to retry the upload
    ##### Begin Diagnostic HTTP information #####
    Status Code: 503
    Status Message: Service Unavailable
    Header Information: {
      "content-length": "592",
      "content-type": "application/json; charset=utf-8",
      "date": "Mon, 21 Aug 2023 14:08:10 GMT",
      "server": "Kestrel",
      "cache-control": "no-store,no-cache",
      "pragma": "no-cache",
      "strict-transport-security": "max-age=2592000",
      "x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
      "activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
      "x-frame-options": "SAMEORIGIN"
    }
    ###### End Diagnostic HTTP information ######
    Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
    Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
    Error: aborting artifact upload
    Total file count: 211 ---- Processed file #165 (78.1%)
    A 503 status code has been received, will attempt to retry the upload
    Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0

As a result, the "Download reports" continued retrying:

    ...
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    An error occurred while attempting to download a file
    Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
        at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
        at Object.onceWrapper (node:events:627:28)
        at ClientRequest.emit (node:events:513:28)
        at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
        at Object.onceWrapper (node:events:627:28)
        at TLSSocket.emit (node:events:525:35)
        at TLSSocket.Socket._onTimeout (node:net:550:8)
        at listOnTimeout (node:internal/timers:559:17)
        at processTimers (node:internal/timers:502:7)
    Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
    Total file count: 1004 ---- Processed file #436 (43.4%)

And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 18:52:52 +02:00
Sebastiaan van Stijn
f44fbab3ec libnetwork: remove Controller.WalkSandboxes and related code
This functionality has been replaced with Controller.GetSandbox, and is
no longer used anywhere.

This patch removes:

- the Controller.WalkSandboxes method
- the SandboxContainerWalker SandboxWalker
- the SandboxWalker type

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:06:28 +02:00
Sebastiaan van Stijn
6dba98cf38 libnetwork: implement Controller.GetSandbox(containerID)
Various parts of the code were using "walkers" to iterate over the
controller's sandboxes, and the only condition for all of them was
to find the sandbox for a given container-ID. Iterating over all
sandboxes was also sub-optimal, because on Windows, the ContainerID
is used as Sandbox-ID, which can be used to lookup the sandbox from
the "sandboxes" map on the controller.

This patch implements a GetSandbox method on the controller that
looks up the sandbox for a given container-ID, using the most optimal
approach (depending on the platform).

The new method can return errors for invalid (empty) container-IDs, and
a "not found" error to allow consumers to detect non-existing sandboxes,
or potentially invalid IDs.

This new method replaces the (non-exported) Daemon.getNetworkSandbox(),
which was only used internally, in favor of directly accessing the
controller's method.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:06:26 +02:00
Sebastiaan van Stijn
6c4153f348 libnetwork: parallelTester: move vars closer to where they're used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:14 +02:00
Sebastiaan van Stijn
e690cdb6a3 libnetwork: remove redundant sandboxTable type
It was not exported so let's remove the abstraction to not make it look
like something more than it is.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:14 +02:00
Sebastiaan van Stijn
3c10db669e libnetwork: remove unused SandboxKeyWalker
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:08 +02:00
Sebastiaan van Stijn
a01bcf9767 Merge pull request #46270 from thaJeztah/classic_build_remove_logs
classic builder: align "removing intermediate container" output
2023-08-21 12:30:55 +02:00
Sebastiaan van Stijn
cc414a2012 libnetwork/osl: remove Sandbox.Info()
"Pay no attention to the implementation behind the curtain!"

There's only one implementation of the Sandbox interface, and only one implementation
of the Info interface, and they both happens to be implemented by the same type:
networkNamespace. Let's merge these interfaces.

And now that we know that there's one, and only one Info, we can drop the charade,
and relieve the Sandbox from its dual personality.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-20 19:26:39 +02:00
Sebastiaan van Stijn
3b9f4395cf libnetwork: remove InterfaceInfo interface
Use the only implementation (EndpointInterface) instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-20 19:08:21 +02:00
Albin Kerouanton
d146e592d8 api/t/network: Move IPAM types to their own file
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-19 11:56:28 +02:00
Sebastiaan van Stijn
cf15460a3b Merge pull request #46273 from crazy-max/ci-bin-image-repo-origin
ci(bin-image): check repo origin
2023-08-19 11:19:13 +02:00
CrazyMax
219d4d9db9 ci(bin-image): check repo origin
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-19 07:19:10 +02:00
Bjorn Neergaard
d125823d3f hack: use long SHA for DOCKER_GITCOMMIT
This better aligns to GHA/CI settings, and is in general a better
practice in the year 2023.

We also drop the 'unsupported' fallback for `git rev-parse` in the
Makefile; we have a better fallback behavior for an empty
DOCKER_GITCOMMIT in `hack/make.sh`.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:42:51 -06:00
Bjorn Neergaard
2010f4338e ci(bin-image): clean up metadata
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:32:59 -06:00
Bjorn Neergaard
ad91fc1b00 ci(bin-image): clean up env var handling
There are still messy special cases (e.g. DOCKER_GITCOMMIT vs VERSION),
but this makes things a little easier to follow, as we keep
GHA-specifics in the GHA files.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:30:20 -06:00
Sebastiaan van Stijn
5dc44f03f0 classic builder: align "removing intermediate container" output
This is something that stood out to me: removing the intermediate
container is part of a build step, but unlike the other output from
the build, wasn't indented (and prefixed with `--->`) to be shown
as part of the build.

This patch adds the `--->` prefix, to make it clearer what step the
removal was part of.

While at it, I also updated the message itself: this output is printed
_after_ the intermediate container has been removed, so we may as well
make it match reality, so I changed "removing" to "removed".

Before:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> a416a98b71e2
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in a1a65b9365ac
    Removing intermediate container a1a65b9365ac
     ---> 8c6b57ebebdd
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 9fa977b763a5
    Removing intermediate container 9fa977b763a5
     ---> 795c1f2fc7b9
    Successfully built 795c1f2fc7b9

After:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> fc9db2894f4e
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in 38d7c34c2178
     ---> Removed intermediate container 38d7c34c2178
     ---> 7c0dbc45111d
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 629620285d4c
     ---> Removed intermediate container 629620285d4c
     ---> b92f70f2e57d
    Successfully built b92f70f2e57d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 21:27:56 +02:00
Sebastiaan van Stijn
fc302d7b7b Merge pull request #46146 from akerouanton/libnet-errors-into-http-status-code
api: Convert libnet's errors into HTTP status code
2023-08-18 17:21:03 +02:00
Sebastiaan van Stijn
a866e50e6e Merge pull request #46238 from vvoland/c8d-inspect-handle-missing-config
c8d/inspect: Ignore manifest with missing config
2023-08-18 15:35:44 +02:00
Sebastiaan van Stijn
1c937c5816 Merge pull request #46189 from vvoland/c8d-more-mount-refcount
c8d integration: Use refcount mounter for diff and export
2023-08-18 15:29:13 +02:00
Sebastiaan van Stijn
6cef3c2b77 Merge pull request #46256 from crazy-max/ci-win-baseimg
windows: update default base image for dev container
2023-08-18 14:15:00 +02:00
Sebastiaan van Stijn
06e540c271 libnetwork: move all SandboxOptions to a separate file
Just a minor reorganisation; move options together into a dedicated file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:12:25 +02:00
Sebastiaan van Stijn
2afa4eba43 libnetwork: resolver: Resolver.dialExtDNS use joinHostPort and cleanup
Slightly refactor Resolver.dialExtDNS:

- use net.JoinHostPort to properly format IPv6 addresses
- define a const for the default port, and avoid int ->  string
  conversion if no custom port is defined
- slightly simplify logic if the HostLoopback is used (at the cost of
  duplicating one line); in that case we don't need to define the closure

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:06:42 +02:00
Sebastiaan van Stijn
3218e26a22 libnetwork: resolver: remove some intermediate variables
Use struct-literals where possible for slightly more readable code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:06:42 +02:00
Sebastiaan van Stijn
986de11464 libnetwork: resolver: remove setCommonFlags, use createRespMsg
This function was added in 36fd9d02be
(libnetwork: ce6c6e8c35),
because there were multiple places where a DNS response was created,
which had to use the same options. However, new "common" options were
added since, and having it in a function separate from the other (also
common) options was just hiding logic, so let's remove it.

What the above probably _should_ have done was to create a common utility
to create a DNS response (as all other options are shared as well). This
was actually done in 0c22e1bd07 (libnetwork:
be3531759b),
which added a `createRespMsg` utility, but missed that it could be used
for both cases.

This patch:

- removes the setCommonFlags function
- uses createRespMsg instead to share common options

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:06:39 +02:00
Sebastiaan van Stijn
9efe6501bb Merge pull request #46261 from vvoland/volume-decrement-refcount-typo
volume/local: Fix debug log typo
2023-08-18 12:50:38 +02:00
Paweł Gronowski
7f965d55c7 volume/local: Fix debug log typo
Active count is incremented, but message claimed the opposite.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-18 11:38:50 +02:00
Kevin Alvarez
678ce73907 windows: update default base image for dev container
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-18 10:19:10 +02:00
Sebastiaan van Stijn
01ac4892e0 Merge pull request #46243 from thaJeztah/remove_datascope_consts
libnetwork/datastore: remove deprecated scope consts
2023-08-17 23:49:56 +02:00
Sebastiaan van Stijn
f7ce59f7dc Merge pull request #46259 from neersighted/bin_image_sha
ci(bin-image): populate DOCKER_GITCOMMIT, take 2
2023-08-17 22:50:57 +02:00
Sebastiaan van Stijn
0c522c6bbf libnetwork/datastore: remove deprecated scope consts
Removes the deprecated consts, which moved to a separate "scope" package
in commit 6ec03d6745, and are no longer used;

- datastore.LocalScope
- datastore.GlobalScope
- datastore.SwarmScope

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-17 22:42:15 +02:00
Sebastiaan van Stijn
16f152b1ec Merge pull request #46242 from thaJeztah/bump_swarmkit
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230815220644-3f2e40b3ed51
2023-08-17 21:35:45 +02:00
Bjorn Neergaard
73ffb48bfb ci(bin-image): populate DOCKER_GITCOMMIT, take 2
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-17 13:17:33 -06:00
Bjorn Neergaard
73e47ee39c Merge pull request #46257 from neersighted/bin_image_sha
ci(bin-image): populate DOCKER_GITCOMMIT
2023-08-17 13:12:22 -06:00
Bjorn Neergaard
9aed6308d4 ci(bin-image): populate DOCKER_GITCOMMIT
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-17 11:53:27 -06:00
Albin Kerouanton
c22ec82477 libnet: Fix error capitalization
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-17 16:48:09 +02:00
Albin Kerouanton
bd0111c1f4 libnet: Replace NoServiceError with UnavailableError
UnavailableError is now compatible with errdefs.UnavailableError. These
errors will now return a 503 instead of a 500.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-17 16:46:53 +02:00
Albin Kerouanton
42d34e40f9 libnet: Replace BadRequest with InvalidParameter
InvalidParameter is now compatible with errdefs.InvalidParameter. Thus,
these errors will now return a 400 status code instead of a 500.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-17 16:45:04 +02:00
Paweł Gronowski
a64adda4e7 c8d/inspect: Ignore manifest with missing config
Fix a failure to inspect image if any of its present manifest references
an image config which isn't present locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-17 09:48:29 +02:00
Sebastiaan van Stijn
c85a33d74b libnetwork/iptables: un-export ZoneSettings, and slight refactor
- un-export ZoneSettings, because it's only used internally
- make conversion to a "interface" slice a method on the struct
- remove the getDockerZoneSettings() function, and move the type-definition
  close to where it's used, as it was only used in a single location

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 17:41:23 +02:00
Sebastiaan van Stijn
d979d2af45 libnetwork/iptables: fix TestFirewalldInit
This test didn't make a lot of sense, because `checkRunning()` depends on
the `connection` package-var being set, which is done by `firewalldInit()`,
so would never be true on its own.

Add a small utility that opens its own D-Bus connection to verify if
firewalld is running, and otherwise skips the tests (preserving any
error in the process).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 17:28:07 +02:00
Sebastiaan van Stijn
513063bcf9 libnetwork/iptables: ProgramChain: don't fail if interface not found
DelInterfaceFirewalld returns an error if the interface to delete was
not found. Let's ignore cases where we were successfully able to get
the list of interfaces in the zone, but the interface was not part of
the zone.

This patch changes the error for these cases to an errdefs.ErrNotFound,
and updates IPTable.ProgramChain to ignore those errors.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 17:25:10 +02:00
Sebastiaan van Stijn
d83ead8434 Merge pull request #46188 from akerouanton/custom-multierror
Add a temporary drop-in replacement for errors.Join
2023-08-16 17:16:09 +02:00
Albin Kerouanton
64de635626 Add a temporary drop-in replacement for errors.Join
As we have a hard time figuring out what moby/moby#46099 should look
like, this drop-in replacement will solve the initial formatting problem
we have. It's made internal such that we can remove it whenever we want
and unlike moby/moby#46099 doesn't require thoughtful API changes.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 16:18:41 +02:00
Sebastiaan van Stijn
cff341e5e6 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230815220644-3f2e40b3ed51
Remove uses of deprecated datastore.LocalScope const

full diff: 60421a63a7...3f2e40b3ed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 14:49:13 +02:00
Sebastiaan van Stijn
56b62640a2 libnetwork: Controller.NewSandbox: use named error-return
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 13:25:10 +02:00
Sebastiaan van Stijn
cbe692ffd1 libnetwork: Controller.NewNetwork: use named error-return
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.

This patch:

- introduces a `retErr` output variable, to be used in defer statements.
- explicitly changes some `err` uses to locally-scoped variables.
- moves some variable definitions closer to where they're used (where possible).

While working on this change, there was one point in the code where
error handling was ambiguous. I added a note for that, in case this
was not a bug:

> This code was previously assigning the error to the global "err"
> variable (before it was renamed to "retErr"), but in case of a
> "MaskableError" did not *return* the error:
> b325dcbff6/libnetwork/controller.go (L566-L573)
>
> Depending on code paths further down, that meant that this error
> was either overwritten by other errors (and thus not handled in
> defer statements) or handled (if no other code was overwriting it.
>
> I suspect this was a bug (but possible without effect), but it could
> have been intentional. This logic is confusing at least, and even
> more so combined with the handling in defer statements that check for
> both the "err" return AND "skipCfgEpCount":
> b325dcbff6/libnetwork/controller.go (L586-L602)
>
> To save future visitors some time to dig up history:
>
> - config-only networks were added in 25082206df
> - the special error-handling and "skipCfgEpcoung" was added in ddd22a8198
> - and updated in 87b082f365 to don't use string-matching

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 13:25:10 +02:00
Sebastiaan van Stijn
e2f9d6c4c3 libnetwork: rename vars that collided with builtins
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 12:34:25 +02:00
Sebastiaan van Stijn
b325dcbff6 Merge pull request #46195 from thaJeztah/daemon_cleanup_start
daemon: Daemon.ContainerStart(): make validateState a regular function, and remove containerNotModifiedError
2023-08-16 12:18:40 +02:00
Sebastiaan van Stijn
9889585a79 Merge pull request #46223 from thaJeztah/cleanup_handleContainerExit
daemon: Daemon.handleContainerExit(): reduce ambiguity in error handling
2023-08-16 12:17:08 +02:00
Sebastiaan van Stijn
e8f0f5a4ce libnetwork: rename agent type to reduce collisions
There were quite some places where the type collided with variables
named `agent`. Let's rename the type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 12:12:39 +02:00
Sebastiaan van Stijn
b3e249f401 Merge pull request #46236 from thaJeztah/libnetwork_bridge_outputvars
libnetwork/drivers/bridge: setupIPChains(): name output variables
2023-08-16 11:24:44 +02:00
Sebastiaan van Stijn
0503cf2510 libnetwork/drivers/bridge: setupIPChains(): name output variables
This function has _four_ output variables of the same type, and several
defer statements that checked the error returned (but using the `err`
variable).

This patch names the return variables to make it clearer what's being
returned, and renames the error-return to `retErr` to make it clearer
where we're dealing with the returned error (and not any local err), to
prevent accidentally shadowing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 00:26:35 +02:00
Bjorn Neergaard
fc5702b284 Merge pull request #46184 from thaJeztah/bump_swarmkit
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230808164555-1983e41a9fff
2023-08-15 16:11:58 -06:00
Bjorn Neergaard
a443c2b18d Merge pull request #46197 from thaJeztah/bridge_nowindows
libnetwork/drivers/bridge: rename some linux-only files
2023-08-15 16:11:38 -06:00
Bjorn Neergaard
dca7544320 Merge pull request #46198 from thaJeztah/libnetwork_nowindows_testfiles
libnetwork: rename and merge unix-only testfiles
2023-08-15 16:11:21 -06:00
Sebastiaan van Stijn
52b5b5be98 Merge pull request #46224 from sam-thibault/remove-ibm-jenkins-jobs
Remove s390x and ppc64le Jenkins pipelines
2023-08-14 18:45:35 +02:00
Sebastiaan van Stijn
b9d9504df5 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230808164555-1983e41a9fff
notable changes:

- Free unused volumes in more cases
- Convert BasicKeyRequest to KeyRequest to use cloudflare/cfssl 1.x.x
- reduce direct imports of logrus

diffs:

- github.com/cloudflare/cfssl v1.6.4; full diff: https://github.com/cloudflare/cfssl/compare/5d63dbd981b5...v1.6.4
- github.com/moby/swarmkit/v2; full diff: https://github.com/moby/swarmkit/compare/bc71908479e5...60421a63a7f148ba1ec7d35c55e4cf9ac03d6b78

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 18:39:57 +02:00
Sebastiaan van Stijn
7b66ae1531 vendor: golang.org/x/crypto v0.3.0
full diff: https://github.com/golang/crypto/compare/v0.2.0...v0.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 18:38:35 +02:00
Sebastiaan van Stijn
16cda4138d Merge pull request #46192 from AkihiroSuda/runc-1.1.9
update runc to v1.1.9
2023-08-14 17:33:19 +02:00
Sam Thibault
59aa3dce8a remove s390x and ppc64ls pipelines
Signed-off-by: Sam Thibault <sam.thibault@docker.com>
2023-08-14 16:54:44 +02:00
Sebastiaan van Stijn
ea2d686468 libnetwork/iptables: DeleteConntrackEntries: remove "totals" returns
There's nothing handling these results, and they're logged as debug-logs,
so we may as well remove the returned variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 16:53:36 +02:00
Sebastiaan van Stijn
f1240393d9 libnetwork/iptables: reduce DeleteConntrackEntriesXX debug logs
Both functions were generating debug logs if there was nothing to log.
The function already produces logs if things failed while deleting entries,
so these logs would only be printed if there was nothing to delete, so can
safely be discarded.

Before this change:

    DEBU[2023-08-14T12:33:23.082052638Z] Revoking external connectivity on endpoint sweet_swirles (1519f9376a3abe7a1c981600c25e8df6bbd0a3bc3a074f1c2b3bcbad0438443b)
    DEBU[2023-08-14T12:33:23.085782847Z] DeleteConntrackEntries purged ipv4:0, ipv6:0
    DEBU[2023-08-14T12:33:23.085793847Z] DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 16:51:43 +02:00
Sebastiaan van Stijn
18a0ff2b2b daemon: Daemon.handleContainerExit(): rename err-var for clarity
The "cpErr" naming was a bit confusing; give it a more descriptive name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 15:17:43 +02:00
Sebastiaan van Stijn
178125ae39 libcontainerd/supervisor: explicitly ignore process kill errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 14:02:27 +02:00
Sebastiaan van Stijn
7b28bc51e6 Merge pull request #46209 from thaJeztah/testutils_WithPIDMode
integration/internal/container: add WithPIDMode option
2023-08-14 13:55:30 +02:00
Sebastiaan van Stijn
f8363690ca daemon: Daemon.handleContainerExit(): reduce ambiguity in error handling
This goroutine was added in c458bca6dc, and
looks for errors from the wait channel. If no error is returned, it attempts
to start the container, and *updates* the error if a failure happened while
doing so, so that the code below it can update the container's status, and
perform auto-remove (if set for the container).

However, due to the formatting of the code, it was easy to overlook that
the "err" variable was not local to the "if" statement.

This patch breaks up the if-statement in an attempt to make it clearer that
this is not a local "err" variable, and adds a code-comment explaining the
logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 13:05:16 +02:00
Sebastiaan van Stijn
cb36f57299 Merge pull request #46208 from thaJeztah/oci_DefaultLinuxSpec_consts
oci: DefaultLinuxSpec: use OCI-spec consts for namespaces
2023-08-14 10:41:41 +02:00
Sebastiaan van Stijn
72648f0ba6 oci: DefaultLinuxSpec: use OCI-spec consts for namespaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 19:06:25 +02:00
Sebastiaan van Stijn
17571ff199 integration/internal/container: add WithPIDMode option
Some files used aliases, others didn't, and they didn't appear to be
required.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 19:06:01 +02:00
Sebastiaan van Stijn
917dae58e1 integration/internal/container: remove import aliases
Some files used aliases, others didn't, and they didn't appear to be
required.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 19:06:01 +02:00
Sebastiaan van Stijn
e0f5bb4820 daemon: withLibnetwork(): return early if networking is disabled
The function was checking in a loop if networking for the container was
disabled. Change the function to return early, and to only set hooks
if one needs to be set.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 19:03:32 +02:00
Sebastiaan van Stijn
225691e258 daemon: withLibnetwork(): use OCI-spec consts for namespaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 19:03:32 +02:00
Sebastiaan van Stijn
8376595621 libnetwork: un-export SetExternalKey
It's only called as part of the "libnetwork-setkey" re-exec, so un-exporting
it to make clear it's not for external use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 15:29:20 +02:00
Sebastiaan van Stijn
91a3a95385 libnetwork/options: OptionExecRoot: skip osl.SetBasePath on non-Linux
The basepath is only used on Linux, so no need to call it on other
platforms. SetBasePath was already stubbed out on other platforms,
but "osl" was still imported in various places where it was not actually
used, so trying to reduce imports to get a better picture of what parts
are used (and not used).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 15:29:20 +02:00
Sebastiaan van Stijn
48ea7ec970 libnetwork/osl: use filepath.Join() only when changing basedir
Use filepath.Join() only when the base-path is updated, instead of every
time it is accessed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 15:29:20 +02:00
Sebastiaan van Stijn
8bf62010a4 libnetwork: merge linux-only test-files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 01:27:38 +02:00
Sebastiaan van Stijn
c0562d4eed libnetwork: rename unix-only testfiles
Some tests were implicitly skipped through the `getTestEnv()` utility,
which made it hard to discover they were not ran on Windows.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 01:27:38 +02:00
Sebastiaan van Stijn
8070f15966 libnetwork/drivers/bridge: rename some linux-only files
This makes it easier to spot if code is only used on Linux. Note that "all of"
the bridge driver is Linux-only.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 00:37:43 +02:00
Sebastiaan van Stijn
014fefee1d libnetwork/drivers/bridge: minor formatting fixes
My IDE kept on re-formatting, so let's do so.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-12 00:37:43 +02:00
Sebastiaan van Stijn
a9e8110fe3 Merge pull request #46187 from thaJeztah/libnetwork_move_windows_things
libnetwork: move some code to platform-specific files
2023-08-12 00:31:49 +02:00
Sebastiaan van Stijn
80d158e0de daemon: remove containerNotModifiedError
Removing this type, because:

- containerNotModifiedError is not an actual error, and abstracting it away
  was hiding some of these details. It also wasn't used as a sentinel error
  anywhere, so doesn't have to be its own type.
- Defining a type just to toggle the error-message between "not running"
  and "not stopped" felt a bit over-the-top, as each variant was only used once.
- So  "it only had one job", and it didn't even do that right; it produced
  capitalized error messages, which makes linters unhappy.

So, let's just inline what it does in the two places it was used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 21:28:06 +02:00
Sebastiaan van Stijn
dffe634c19 daemon: Daemon.ContainerStart(): make validateState a regular function
There's no need for this to be a closure; let's just make it a regular
function. While moving it out, also make some minor code-changes and
add some code-comments to describe the flow / intent, which may not
be trivial for people that are not familiar with these details.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 21:28:06 +02:00
Sebastiaan van Stijn
896f246888 Merge pull request #46191 from thaJeztah/integration_testutil_improvements
integration/container: refactor some test-utilities and fix var-names shadowing imports
2023-08-11 17:49:42 +02:00
Sebastiaan van Stijn
5fe81ef1bb Merge pull request #46193 from thaJeztah/daemon_error_comment
daemon: Daemon.containerStart(): add comment to clarify error-type
2023-08-11 16:18:49 +02:00
Paweł Gronowski
6da42ca830 c8d/diff: Reuse mount, mount parent as read-only
The container rw layer may already be mounted, so it's not safe to use
it in another overlay mount. Use the ref counted mounter (which will
reuse the existing mount if it exists) to avoid that.

Also, mount the parent mounts (layers of the base image) in a read-only
mode.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-11 15:30:38 +02:00
Paweł Gronowski
051d51b222 c8d/export: Use ref counted mounter
To prevent mounting the container rootfs in a rw mode if it's already
mounted.  This can't use `mount.WithReadonlyTempMount` because the
archive code does a chroot with a pivot_root, which creates a new
directory in the rootfs.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-11 15:30:36 +02:00
Paweł Gronowski
303e2b124e integration: Add test for not breaking overlayfs
Check that operations that could potentially perform overlayfs mounts
that could cause undefined behaviors.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-11 15:30:29 +02:00
Sebastiaan van Stijn
05deecaa45 internal/testutils/netnsutils: move utils that were not used on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 15:00:30 +02:00
Sebastiaan van Stijn
d4e1c072e2 libnetwork: move resolverIPSandbox closer to where it's used
It's only used on non-Windows platforms, so let's move it there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 15:00:30 +02:00
Sebastiaan van Stijn
f661bd8ee5 libnetwork: Resolved.SetupFunc() minor cleanup
Remove intermediate variables in favor of struct-literals.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 15:00:30 +02:00
Sebastiaan van Stijn
de4ba13400 libnetwork: move resolver tests that were skipped on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 15:00:30 +02:00
Sebastiaan van Stijn
1e4e9161c5 libnetwork: move TestDNSOptions to a non-windows file
It was only testing stub implementations on Windows that are not
used in production code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 14:59:30 +02:00
Sebastiaan van Stijn
bf1fb97575 daemon: Daemon.containerStart(): add comment to clarify error-type
Any error that occurs while creating the spec, even if it's the
result of an invalid container config, must be considered a System
error (internal server error), as it's not an error with the request
to start the container.

Invalid configuration in the config itself must be validated when
creating the container (creating its config), but some errors are
dependent on the current state, for example when starting a container
that shares a namespace with another container, and that container
is not running (or missing).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 14:47:22 +02:00
Akihiro Suda
b039bbc678 update runc binary to v1.1.9
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-08-11 21:29:53 +09:00
Akihiro Suda
79b467808e vendor.mod: github.com/opencontainers/runc v1.1.9
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-08-11 21:29:16 +09:00
Sebastiaan van Stijn
74feadacf8 integration/internal/container: refactor CreateExpectingErr
This utility was only used for a single test, and it was very limited
in functionality as it only allowed for a certain error-string to be
matched.

Let's change it into a more generic function; a helper that allows a
container to be created from a `TestContainerConfig` (which can be
constructed using `NewTestConfig`) and that returns the response from
client.ContainerCreate(), so that any result from that can be tested,
leaving it up to the test to check the results.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 14:03:08 +02:00
Sebastiaan van Stijn
0899ba4a3f integration/internal/container: add NewTestConfig utility
Introduce a NewTestConfig utility, to allow using the available utilities
for constructing a config, and use them with the regular API client.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 14:03:08 +02:00
Sebastiaan van Stijn
3cb52a6359 integration/internal/container: use consistent name for api-client
The `client` variable was colliding with the `client` import. In some cases
the confusing `cli` name (it's not the "cli") was used. Given that such names
can easily start spreading (through copy/paste, or "code by example"), let's
make a one-time pass through all of them in this package to use the same name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 14:02:17 +02:00
Sebastiaan van Stijn
26be2bc6b9 integration/container: use consistent name for api-client
The `client` variable was colliding with the `client` import in various
files. While it didn't conflict in all files, there was inconsistency
in the naming, sometimes using the confusing `cli` name (it's not the
"cli"), and such names can easily start spreading (through copy/paste,
or "code by example").

Let's make a one-time pass through all of them in this package to use
the same name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-11 13:51:57 +02:00
Sebastiaan van Stijn
e0da5cb929 Merge pull request #46171 from thaJeztah/client_context
client: Client.buildRequest: use http.NewRequestWithContext
2023-08-10 21:00:13 +02:00
Paweł Gronowski
220fba06e7 Merge pull request #46182 from akerouanton/daemon-create-replace-pkg-errors
daemon/create.go: Supersede github.com/pkg/errors
2023-08-10 13:26:52 +02:00
Sebastiaan van Stijn
3c8b68c636 Merge pull request #46181 from vvoland/tests-volumemounts-nilerror
test/volume: Replace Check with NilError where suitable
2023-08-10 13:12:28 +02:00
Albin Kerouanton
38e26c4717 daemon/create.go: Fix error capitalization
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-10 12:15:34 +02:00
Sebastiaan van Stijn
8b69cb4482 Merge pull request #46176 from vvoland/refactor-integration-tag-subtests
integration/tag: Use subtests, make parallel and move to client unit test
2023-08-10 12:13:17 +02:00
Paweł Gronowski
435ecfe6e0 test/volume: Replace Check with NilError where suitable
In these cases, continuing after a non nil error will result in a nil
dereference in panic.
Change the `assert.Check` to `assert.NilError` to avoid that.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-10 11:18:56 +02:00
Sebastiaan van Stijn
3d3ce9812f integration/tag: Move to client unit test
This test was testing the client-side validation, so might as well
move it there, and validate that the client invalidates before
trying to make an API call.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-10 10:37:04 +02:00
Albin Kerouanton
742475bc8d daemon/create.go: Supersede github.com/pkg/errors
Will make it possible to use `errors.Join()` in that file.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-10 01:30:10 +02:00
Sebastiaan van Stijn
4cc796ab93 client: Client.buildRequest: use http.NewRequestWithContext
Attach the context to the request while we're creating it, instead of
creating the context first, and adding the context later.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-09 20:30:25 +02:00
Sebastiaan van Stijn
58dc0fcd1e client: Client.Ping: re-use request when falling back to GET
Re-use the request, and change the method to GET instead of building
a new request "from scratch".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-09 20:30:25 +02:00
Sebastiaan van Stijn
87fff769f4 client: Client.checkResponseErr: change errorMessage to an error
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-09 20:30:25 +02:00
Paweł Gronowski
71da8c13e1 integration/tag: Use subtests and make parallel
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 14:11:18 +02:00
Sebastiaan van Stijn
89b542b421 Merge pull request #46175 from vvoland/pkg-plugins-gotest
pkg/plugins: Rewrite with assert.Check
2023-08-09 13:44:05 +02:00
Sebastiaan van Stijn
cfc117826a Merge pull request #46168 from vvoland/hack-integrationcli-dont-failfast
hack/test: Don't fail-fast before integration-cli
2023-08-09 13:43:43 +02:00
Sebastiaan van Stijn
1614b534fe Merge pull request #46169 from vvoland/hack-integration-testfilter-fixes
hack/test: Fix `TEST_FILTER` with regex matching integration and integration-cli tests
2023-08-09 13:29:46 +02:00
Paweł Gronowski
11a0c2779b pkg/plugins: Rewrite with assert.Check
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 12:22:25 +02:00
Sebastiaan van Stijn
6598cba32f Merge pull request #46174 from thaJeztah/libnetwork_osl_cleanups
libnetwork/osl: remove redundant locks, and assorted cleanups
2023-08-09 12:17:54 +02:00
Paweł Gronowski
43956e1b71 hack/test: Don't exit early when all tests are filtered out
Don't exit immediately (due to `set -e` bash behavior) when grep returns
with a non-zero exit code. Use empty dirs instead and let it print
messages about all tests being filtered out.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 09:47:02 +02:00
Paweł Gronowski
ae008570ff hack/test: Split -test.run for integration and integration-cli
To avoid passing the `/` prefix in the -test.run to the integration test
suite, which for some reason executes all tests, but works fine with
integration-cli.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 09:47:01 +02:00
Paweł Gronowski
510cac5f5b hack/test: Fix checking if integration-cli are filtered out
Previous check checked if ANY of the test directories isn't
integration-cli. This means it was true if TEST_FILTER matched multiple
tests from both integration and integration-cli suite.

Remove the grep `-v` inversion and replace it with a bash negation, so
it actually checks if there is no `integration-cli` in test dirs.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 09:47:00 +02:00
Sebastiaan van Stijn
2cccb1f02c Merge pull request #46075 from thaJeztah/libnetwork_remove_NetworkInfo
libnetwork: remove Network.Info() and remove NetworkInfo interface
2023-08-09 01:15:44 +02:00
Sebastiaan van Stijn
8a1ca49657 libnetwork/osl: nwIface: add godoc
Copy the godoc from the interface to the implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 23:05:42 +02:00
Sebastiaan van Stijn
16785b9b7b libnetwork/osl: move all networkNamespace methods together
These methods were sprinkled throughout the code; let's move
them together.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 23:04:19 +02:00
Sebastiaan van Stijn
5b0fa7aaca libnetwork/osl: some minor nits
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:33:29 +02:00
Sebastiaan van Stijn
972d80b596 libnetwork/osl: clean up newInfo() a bit
Use struct-literals in some places to make it slightly more visible
what we're creating where.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:33:28 +02:00
Sebastiaan van Stijn
0da721ec38 libnetwork/osl: make newKey and newInfo a t.Helper()
Both were passed testing.T, but it was not used, so let's make use of it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:33:28 +02:00
Sebastiaan van Stijn
d9442aab88 libnetwork/osl: nwIface: remove mutex altogether
The mutex is only used on reads, but there's nothing protecting writes,
and it looks like nothing is mutating fields after creation, so let's
remove this altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:33:07 +02:00
Sebastiaan van Stijn
2afe18d2ce libnetwork/osl: nwIface: unexport sync.Mutex
Don't make the mutex public. This also gives a better clue
if the mutex is used externally.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:32:26 +02:00
Sebastiaan van Stijn
8b989ac665 libnetwork/osl: let's not do this, etc.
No context in the commit that added it, but PR discussion shows that
the API was mostly exploratory, and it was 8 Years go, so let's not
head in that direction :) b646784859

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:15:13 +02:00
Sebastiaan van Stijn
3d0a7d819c libnetwork: remove Network.Info() and remove NetworkInfo interface
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:05:32 +02:00
Sebastiaan van Stijn
74354043ff remove uses of libnetwork/Network.Info()
Now that we removed the interface, there's no need to cast the Network
to a NetworkInfo interface, so we can remove uses of the `Info()` method.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 22:05:30 +02:00
Sebastiaan van Stijn
2b449e0e65 Merge pull request #46173 from akerouanton/remove-unused-ipam-errors
libnet/ipamapi: Remove unused errors
2023-08-08 21:56:08 +02:00
Albin Kerouanton
36a0946aa9 libnet/ipamapi: Remove unused errors
These errors aren't used in our repo and seem unused by the OSS
community (this was checked with Sourcegraph).

- ErrIpamInternalError has never been used
- ErrInvalidRequest is unused since moby/libnetwork@c85356efa
- ErrPoolNotFound has never been used
- ErrOverlapPool has never been used
- ErrNoAvailablePool has never been used

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-08 19:55:47 +02:00
Sebastiaan van Stijn
2884018e7f Merge pull request #46051 from thaJeztah/ipam_clean
libnetwork/ipam: assorted cleanup and refactor
2023-08-08 18:39:00 +02:00
Sebastiaan van Stijn
2ef7b479a5 Merge pull request #46166 from thaJeztah/pkg_plugin_cleanup_STEP3
pkg/plugins: override timeouts during tests
2023-08-08 18:38:08 +02:00
Sebastiaan van Stijn
0485b8f0fb Merge pull request #46170 from akerouanton/daemon-network-replace-pkg-errors
daemon/network.go: Remove github.com/pkg/errors pkg
2023-08-08 18:37:31 +02:00
Albin Kerouanton
fd4ec26313 daemon/network.go: Fix error capitalization
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-08 17:40:19 +02:00
Albin Kerouanton
64e01c627f daemon/network.go: Remove github.com/pkg/errors pkg
PR moby/moby#45759 is going to use the new `errors.Join` function  to
return a list of validation errors.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-08 17:39:53 +02:00
Sebastiaan van Stijn
94dc10378d libnetwork: network.requestPoolHelper: slightly optimize order of checks
Check the preferredPool first, as other checks could be doing more
(such as locking, or validating / parsing). Also adding a note, as
it's unclear why we're ignoring invalid pools here.

The "invalid" conditions was added in [libnetwork#1095][1], which
moved code to reduce os-specific dependencies in the ipam package,
but also introduced a types.IsIPNetValid() function, which considers
"0.0.0.0/0" invalid, and added it to the condition to return early.

Unfortunately review does not mention this change, so there's no
context why. Possibly this was done to prevent errors further down
the line (when checking for overlaps), but returning an error here
instead would likely have avoided that as well, so we can only guess.

To make this code slightly more transparent, this patch also inlines
the "types.IsIPNetValid" function, as it's not used anywhere else,
and inlining it makes it more visible.

[1]: 5ca79d6b87 (diff-bdcd879439d041827d334846f9aba01de6e3683ed8fdd01e63917dae6df23846)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:24 +02:00
Sebastiaan van Stijn
67e2c1d482 libnetwork: network.requestPoolHelper: remove dead code
This code was only run if no preferred pool was specified, however,
since [libnetwork#1162][2], the function would already return early
if a preferred pools was set (and the overlap check to be skipped),
so this was now just dead code.

[2]: 9cc3385f44

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:24 +02:00
Sebastiaan van Stijn
ad68883c5a libnetwork: network.requestPoolHelper: don't defer in a loop
This function intentionally holds a lock / lease on address-pools to
prevent trying the same pool repeatedly.

Let's try to make this logic slightly more transparent, and prevent
defining defers in a loop. Releasing all the pools in a singe defer
also allows us to get the network-name once, which prevents locking
and unlocking the network for each iteration.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:24 +02:00
Sebastiaan van Stijn
32fcde6d9e libnetwork: network.IpamConfig, network.IpamInfo: name output vars
Both functions have multiple output vars with generic types, which made
it hard to grasp what's what.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:24 +02:00
Sebastiaan van Stijn
df03357d19 libnetwork/ipam: move PoolID.FromString() to a PoolIDFromString() func
This makes it easier to consume, without first having to create an empty
PoolID.

Performance is the same:

    BenchmarkPoolIDFromString-10   6100345   196.5 ns/op  112 B/op   3 allocs/op
    BenchmarkPoolIDFromString-10   6252750   192.0 ns/op  112 B/op   3 allocs/op

Note that I opted not to change the return-type to a pointer, as that seems
to perform less;

    BenchmarkPoolIDFromString-10   6252750   192.0 ns/op  112 B/op   3 allocs/op
    BenchmarkPoolIDFromString-10   5288682   226.6 ns/op  192 B/op   4 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:24 +02:00
Sebastiaan van Stijn
808fed550d libnetwork/ipam: PoolID.String(): don't use fmt.Sprintf
As this function may be called repeatedly to convert to/from a string,
it may be worth optimizing it a bit. Adding a minimal Benchmark for
it as well.

Before/after:

    BenchmarkPoolIDToString-10   2842830   424.3 ns/op   232 B/op  12 allocs/op
    BenchmarkPoolIDToString-10   7176738   166.8 ns/op   112 B/op   7 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:24 +02:00
Sebastiaan van Stijn
87fc8c772b libnetwork/ipam: Allocator.RequestPool: name args, output vars
network.requestPoolHelper and Allocator.RequestPool have many args and
output vars with generic types. Add names for them to make it easier to
grasp what's what.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:57:20 +02:00
Sebastiaan van Stijn
6dbc9c1c53 libnetwork/ipam: Allocator.RequestPool: mark options arg as unused
The options are unused, other than for debug-logging, which made it look
as if they were actually consumed anywhere, but they aren't.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:56:24 +02:00
Sebastiaan van Stijn
7047964bd6 libnetwork/ipam: Allocator.RequestPool: make parseErr only handle errors
This makes it slightly more readable to see what's returned in each of
the code-paths. Also move validation of pool/subpool earlier in the
function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:56:24 +02:00
Sebastiaan van Stijn
821ef5cbaf libnetwork/ipams/null: use consts for fixed values
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:56:22 +02:00
Sebastiaan van Stijn
05ef5559c3 pkg/plugins: override timeouts during tests
Some tests were testing non-existing plugins, but therefore triggered
the retry-loop, which times out after 15-30 seconds. Add some options
to allow overriding this timeout during tests.

Before:

    go test -v -run '^(TestGet|TestNewClientWithTimeout)$'
    === RUN   TestGet
    === RUN   TestGet/success
    === RUN   TestGet/not_implemented
    === RUN   TestGet/not_exists
    WARN[0000] Unable to locate plugin: vegetable, retrying in 1s
    WARN[0001] Unable to locate plugin: vegetable, retrying in 2s
    WARN[0003] Unable to locate plugin: vegetable, retrying in 4s
    WARN[0007] Unable to locate plugin: vegetable, retrying in 8s
    --- PASS: TestGet (15.02s)
        --- PASS: TestGet/success (0.00s)
        --- PASS: TestGet/not_implemented (0.00s)
        --- PASS: TestGet/not_exists (15.02s)
    === RUN   TestNewClientWithTimeout
        client_test.go:166: started remote plugin server listening on: http://127.0.0.1:36275
    WARN[0015] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": context deadline exceeded (Client.Timeout exceeded while awaiting headers), retrying in 1s
    WARN[0017] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": context deadline exceeded (Client.Timeout exceeded while awaiting headers), retrying in 2s
    WARN[0019] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": net/http: request canceled (Client.Timeout exceeded while awaiting headers), retrying in 4s
    WARN[0024] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": net/http: request canceled (Client.Timeout exceeded while awaiting headers), retrying in 8s
    --- PASS: TestNewClientWithTimeout (17.64s)
    PASS
    ok  	github.com/docker/docker/pkg/plugins	32.664s

After:

    go test -v -run '^(TestGet|TestNewClientWithTimeout)$'
    === RUN   TestGet
    === RUN   TestGet/success
    === RUN   TestGet/not_implemented
    === RUN   TestGet/not_exists
    WARN[0000] Unable to locate plugin: this-plugin-does-not-exist, retrying in 1s
    --- PASS: TestGet (1.00s)
        --- PASS: TestGet/success (0.00s)
        --- PASS: TestGet/not_implemented (0.00s)
        --- PASS: TestGet/not_exists (1.00s)
    === RUN   TestNewClientWithTimeout
        client_test.go:167: started remote plugin server listening on: http://127.0.0.1:45973
    --- PASS: TestNewClientWithTimeout (0.04s)
    PASS
    ok  	github.com/docker/docker/pkg/plugins	1.050s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:41:11 +02:00
Sebastiaan van Stijn
63d477b20e Merge pull request #46039 from thaJeztah/cleanup_bridge
libnetwork/drivers/bridge: assorted cleanups
2023-08-08 14:06:50 +02:00
Sebastiaan van Stijn
f9cae2acbe Merge pull request #46165 from akerouanton/remove-api-CheckDuplicate-warning
api: Remove duplicated check on CheckDuplicate
2023-08-08 13:42:17 +02:00
Paweł Gronowski
6841a53d17 hack/test: Don't fail-fast before integration-cli
If TEST_INTEGRATION_FAIL_FAST is not set, run the integration-cli tests
even if integration tests failed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-08 13:29:54 +02:00
Sebastiaan van Stijn
4ab4330677 Merge pull request #46080 from thaJeztah/pkg_plugin_cleanup_STEP2
pkg/plugins: some cleaning up (step 2)
2023-08-08 12:28:16 +02:00
Sebastiaan van Stijn
2aa24519da ibnetwork/drivers/bridge: newLink: validate before creating
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:50:40 +02:00
Sebastiaan van Stijn
5d722b35d9 libnetwork/drivers/bridge: bridgeNetwork.getEndpoint(): move lock
Don't lock if there's no need to.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:50:39 +02:00
Sebastiaan van Stijn
eba15fe905 libnetwork/drivers/bridge: driver.link: don't defer in a loop
Collect a list of all the links we successfully enabled (if any), and
use a single defer to disable them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:50:39 +02:00
Sebastiaan van Stijn
76b736c242 libnetwork/drivers/bridge: driver.link: name return var for defer handling
Name the return variable to prevent accidental shadowing of the error,
which is used in defers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:50:39 +02:00
Sebastiaan van Stijn
ea5f21ceac libnetwork/drivers/bridge: don't convert IP to string and back again
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:50:39 +02:00
Sebastiaan van Stijn
8b6203b613 libnetwork/drivers/bridge: link.Enable: don't register reload on error
Only register a reload function if we actually managed to enable the link.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:50:34 +02:00
Sebastiaan van Stijn
0f4ba145ee libnetwork/drivers/bridge: link.Enable, link.Disable use iptables.Action
The iptables package has types defined for these actions; use them directly
instead of creating a string only to convert it to a known value.

As the linkContainers() function is only used internally, and with fixed
values, we can also remove the validation, and InvalidIPTablesCfgError
error, which is now unused.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 11:47:31 +02:00
Sebastiaan van Stijn
f2be77a5ab Merge pull request #46058 from thaJeztah/cleanup_buildSandboxOptions
daemon: cleanup Daemon.buildSandboxOptions
2023-08-08 11:38:24 +02:00
Albin Kerouanton
40adb4317b api: Remove duplicated check on CheckDuplicate
Partially revert commit 94b880f.

The CheckDuplicate field has been introduced in commit 2ab94e1. At that
time, this check was done in the network router. It was then moved to
the daemon package in commit 3ca2982. However, commit 94b880f duplicated
the logic into the network router for no apparent reason. Finally,
commit ab18718 made sure a 409 would be returned instead of a 500.

As this logic is first done by the daemon, the error -> warning
conversion can't happen because CheckDuplicate has to be true for the
daemon package to return an error. If it's false, the daemon proceed
with the network creation, set the Warning field of its return value and
return no error.

Thus, the CheckDuplicate logic in the api is removed and
libnetwork.NetworkNameError now implements the ErrConflict interface.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-08 10:00:02 +02:00
Sebastiaan van Stijn
3e52785aae volume/drivers: use plugin Content-Type headers v1.2
The MediaType was changed twice in;

- b3b7eb2723 ("application/vnd.docker.plugins.v1+json"   -> "application/vnd.docker.plugins.v1.1+json")
- 54587d861d ("application/vnd.docker.plugins.v1.1+json" -> "application/vnd.docker.plugins.v1.2+json")

But the (integration) tests were still using the old version, so let's
use the VersionMimeType const that's defined, and use the updated version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-07 20:38:54 +02:00
Sebastiaan van Stijn
481dde8b70 libnetwork: use plugin Content-Type headers v1.2
The MediaType was changed twice in;

- b3b7eb2723 ("application/vnd.docker.plugins.v1+json"   -> "application/vnd.docker.plugins.v1.1+json")
- 54587d861d ("application/vnd.docker.plugins.v1.1+json" -> "application/vnd.docker.plugins.v1.2+json")

But the (integration) tests were still using the old version, so let's
use the VersionMimeType const that's defined, and use the updated version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-07 20:38:54 +02:00
Sebastiaan van Stijn
93b5c31541 integration-cli: use plugin Content-Type headers v1.2
The MediaType was changed twice in;

- b3b7eb2723 ("application/vnd.docker.plugins.v1+json"   -> "application/vnd.docker.plugins.v1.1+json")
- 54587d861d ("application/vnd.docker.plugins.v1.1+json" -> "application/vnd.docker.plugins.v1.2+json")

But the (integration) tests were still using the old version, so let's
use the VersionMimeType const that's defined, and use the updated version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-07 20:38:54 +02:00
Sebastiaan van Stijn
b663c7c5c3 pkg/plugins: add convenience alias for VersionMimeType
Add an alias in the pkg/plugins package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-07 20:38:51 +02:00
Sebastiaan van Stijn
4b19b2f4ba Merge pull request #46004 from elezar/add-cdi-spec-dirs-to-info
Add CDISpecDirs to Info output
2023-08-07 20:14:06 +02:00
Sebastiaan van Stijn
b32c7594a5 Merge pull request #46056 from thaJeztah/daemon_cleanup_assorted
daemon: assorted cleanups
2023-08-07 10:51:25 +02:00
Brennan Kinney
c8930105bc fix: Normalize RLIMIT_NOFILE to sensible defaults
During review, it was decided to remove `LimitNOFILE` from `docker.service` to rely on the systemd v240 implicit default of `1024:524288`. On supported platforms with systemd prior to v240, packagers will patch the service with an explicit `LimitNOFILE=1024:524288`.

- `1024` soft limit is an implicit default, avoiding unexpected breakage. Software that needs a higher limit should request to raise the soft limit for its process.
- `524288` hard limit is an implicit default since systemd v240 and is adequate for most processes (_half of the historical limit from `fs.nr_open` of `1048576`_), while 4096 is the implicit default from the kernel (often too low). Individual containers can be started with `--ulimit` when a larger hard limit is required.
- The hard limit may not exceed `fs.nr_open` (_which a value of `infinity` will resolve to_). On most systems with systemd v240 or newer, this will resolve to an excessive size of 2^30 (over 1 billion).
- When set to `infinity` (usually as the soft limit) software may experience significantly increased resource usage, resulting in a performance regression or runtime failures that are difficult to troubleshoot.
- OpenRC current config approach lacks support for different soft/hard limits being set as it adjusts additional limits and `ulimit` does not support mixed usage of `-H` + `-S`. A soft limit of `524288` is not ideal, but 2^19 is much less overhead than 2^30, whilst a hard limit of 4096 would be problematic for Docker.

Signed-off-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-08-07 12:37:14 +12:00
Brian Goff
b1c112d35e Merge pull request #46157 from vvoland/pkg-plugins-racy 2023-08-05 10:46:00 -07:00
Sebastiaan van Stijn
4f9540f19f daemon: remove some intermediate vars and some cleanup
Remove some intermediate vars, move vars closer to where they're used,
and introduce local var for `nw.Name()` to reduce some locking/unlocking in:

- `Daemon.allocateNetwork()`
- `Daemon.releaseNetwork()`
- `Daemon.connectToNetwork()`
- `Daemon.disconnectFromNetwork()`
- `Daemon.findAndAttachNetwork()`

Also un-wrapping some lines to make it slightly easier to read the conditions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-05 10:44:21 +02:00
Sebastiaan van Stijn
8153a7760e daemon: Daemon.tryDetachContainerFromClusterNetwork: cleanup
- Remove intermediate variable
- Optimize the order of checks in the condition; check for unmanaged containers
  first, before getting information about cluster state and network information.
- Simplify the log messages, as the error would already contain the same
  information about the network (name or ID) and container (ID), so would
  print the network ID twice:

    error detaching from network <ID>: could not find network attachment for container <ID> to network <name or ID>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-05 10:44:21 +02:00
Sebastiaan van Stijn
a018bb1da7 daemon: Daemon.initializeNetworking: remove var declaration
The function was declaring an err variable which was shadowed. It was
intended for directly assigning to a struct field, but as this function
is directly mutating an existing object, and the err variable was declared
far away from its use, let's use an intermediate var for that to make it
slightly more atomic.

While at it, also combined two "if" branches.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-05 10:44:21 +02:00
Sebastiaan van Stijn
12d8029d56 daemon: Daemon.getNetworkedContainer: remove intermediate var
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-05 10:44:21 +02:00
Sebastiaan van Stijn
d1aa1979b4 daemon: buildEndpointInfo: minor refactor
store network.Name() in a variable to reduce repeatedly locking/unlocking
of the network (although this is very, very minimal in the grand scheme
of things).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-05 10:43:45 +02:00
Evan Lezar
7a59913b1a Add CDISpecDirs to Info output
This change adds the configured CDI spec directories to the
system info output.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-08-04 11:46:34 +02:00
Paweł Gronowski
0034a98eb1 pkg/plugins: Guard storage and unparallel racy tests
These tests were made parallel to speed up the execution, but this
turned out to be flaky, because they mutate some shared state.

The tests use shared `storage` variable without any synchronization.
However, adding synchronization is not enough in all cases, some tests
register the same plugin, so they can't be run in parallel to each
other.

This commit adds the synchronization around `storage` variable
modification and removes parallel from the tests where it's not enough.

Before:
```
$ go test -race -v . -count 1
...
--- FAIL: TestGet (15.02s)
    --- FAIL: TestGet/not_implemented (0.00s)
        testing.go:1446: race detected during execution of test
    testing.go:1446: race detected during execution of test
FAIL
FAIL    github.com/docker/docker/pkg/plugins    17.655s
FAIL
```

After:
```
$ go test -race -v . -count 1
ok      github.com/docker/docker/pkg/plugins    32.702s
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-04 11:22:41 +02:00
Brian Goff
eb3ace962d Merge pull request #46130 from vvoland/ci-mobybin-sha 2023-08-03 11:25:49 -07:00
Brian Goff
25be5c9cfc Merge pull request #46133 from thaJeztah/client_cleanup 2023-08-03 09:38:22 -07:00
Sebastiaan van Stijn
01a9c5b70e Merge pull request #46144 from akerouanton/verifyNetworkingConfig-unneeded-error-wrapping
daemon: Remove unneeded error wrapping in verifyNetworkingConfig
2023-08-03 12:54:39 +02:00
Evan Lezar
bbb9255562 Skip CDI driver registration if CDISpecDirs is empty
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-08-03 12:21:44 +02:00
Evan Lezar
3b71197fb8 Set default CDI spec dirs after parsing args
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-08-03 12:21:44 +02:00
Albin Kerouanton
f3e62199ea daemon: Remove unneeded error wrapping in verifyNetworkingConfig
This function is called by `daemon.containerCreate()` which is already
wrapping errors coming from `verifyNetworkingConfig()` with
`errdefs.InvalidParameter()`. So `verifyNetworkingConfig()` should only
return standard errors.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-03 11:21:52 +02:00
Paweł Gronowski
d7a9f15775 bakefile: Remove default value of DOCKER_GITCOMMIT
"HEAD" will still be used as a version if no DOCKER_COMMIT is provided
(for example when not running via `make`), but it won't prevent it being
set to the GITHUB_SHA variable when it's present.

This should fix `Git commit` reported by `docker version` for the
binaries generated by `moby-bin`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-03 10:23:05 +02:00
Sebastiaan van Stijn
f576f568a3 Merge pull request #46055 from thaJeztah/daemon_cleanup_buildCreateEndpointOptions
daemon: cleanup buildCreateEndpointOptions
2023-08-02 19:49:26 +02:00
Sebastiaan van Stijn
9249e0b3f8 Merge pull request #46143 from thaJeztah/nonderscore
docs/api: remove stray underscore in version-history.md
2023-08-02 16:34:52 +02:00
Sebastiaan van Stijn
2b028dc55b daemon: Daemon.buildSandboxOptions: use range when looping
Makes the code slightly more readable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:16 +02:00
Sebastiaan van Stijn
33b4285eb1 daemon: Daemon.buildSandboxOptions: don't use PortBinding.GetCopy()
This code was initializing a new PortBinding, and creating a deep copy
for each binding. It's unclear what the intent was here, but at least
PortBinding.GetCopy() wasn't adding much value, as it created a new
PortBinding, [copying all values from the original][1], which includes
a [copy of IPAddresses in it][2]. Our original "template" did not have any
of that, so let's forego that, and just create new PortBindings as we go.

[1]: 454b6a7cf5/libnetwork/types/types.go (L110-L120)
[2]: 454b6a7cf5/libnetwork/types/types.go (L236-L244)

Benchmarking before/after;

    BenchmarkPortBindingCopy-10    166752   6230 ns/op  1600 B/op  100 allocs/op
    BenchmarkPortBindingNoCopy-10  226989   5056 ns/op  1600 B/op  100 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:16 +02:00
Sebastiaan van Stijn
ddb3d46533 daemon: Daemon.buildSandboxOptions: remove intermediate vars
These were not adding much, so just getting rid of them. Also added a
TODO to move this code to the type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:16 +02:00
Sebastiaan van Stijn
27a27db73f daemon: Daemon.buildSandboxOptions: move vars to where they're used
Move variables closer to where they're used instead of defining them all
at the start of the function.

Also removing some intermediate variables, unwrapped some lines, and combined
some checks to a single check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:16 +02:00
Sebastiaan van Stijn
91cab53a3e libnetwork: make OptionDNS, OptionDNSOptions, OptionDNSSearch take a slice
Outside of some tests, these options are the only code setting these fields,
so we can update them to set the value, instead of appending.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:15 +02:00
Sebastiaan van Stijn
2cbed9d2a8 daemon: inline Daemon.getDNSSearchSettings
This function was created as a "method", but didn't use the Daemon in any
way, and all other options were checked inline, so let's not pretend this
function is more "special" than the other checks, and inline the code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:15 +02:00
Sebastiaan van Stijn
603547fa19 daemon: change Daemon.setupPathsAndSandboxOptions to a regular func
It's not using the daemon in any way, so let's change it to a regular
function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:14:15 +02:00
Sebastiaan van Stijn
07f2df69c7 daemon: buildCreateEndpointOptions: minor nits
- store network.Name() in a variable to reduce repeatedly locking/unlocking
  of the network (although this is very, very minimal in the grand scheme
  of things).
- un-wrap long conditions
- ever so slightly optimise some conditions by changeing the order of checks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 16:12:36 +02:00
Sebastiaan van Stijn
5158a33f15 daemon: buildCreateEndpointOptions: use range when looping
Makes the code slightly more readable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:44 +02:00
Sebastiaan van Stijn
1c6dae1291 daemon: buildCreateEndpointOptions: don't use PortBinding.GetCopy()
This code was initializing a new PortBinding, and creating a deep copy
for each binding. It's unclear what the intent was here, but at least
PortBinding.GetCopy() wasn't adding much value, as it created a new
PortBinding, [copying all values from the original][1], which includes
a [copy of IPAddresses in it][2]. Our original "template" did not have any
of that, so let's forego that, and just create new PortBindings as we go.

[1]: 454b6a7cf5/libnetwork/types/types.go (L110-L120)
[2]: 454b6a7cf5/libnetwork/types/types.go (L236-L244)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:44 +02:00
Sebastiaan van Stijn
cc79024761 daemon: buildCreateEndpointOptions: remove intermediate vars
These were not adding much, so just getting rid of them. Also added a
TODO to move this code to the type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:43 +02:00
Sebastiaan van Stijn
45de99aa06 daemon: buildCreateEndpointOptions: don't parse empty vip
Also keep network.ID() in a local variable to prevent locking the network
twice.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:43 +02:00
Sebastiaan van Stijn
7d429125d2 daemon: buildCreateEndpointOptions: move vars to where they're used
Move variables closer to where they're used instead of defining them all
at the start of the function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:43 +02:00
Sebastiaan van Stijn
6ce92aa523 daemon: buildCreateEndpointOptions: skip getPortMapInfo() if not needed
`getPortMapInfo` does many things; it creates a copy of all the sandbox
endpoints, gets the driver, endpoints, and network from store, and creates
port-bindings for all exposed and mapped ports.

We should look if we can create a more minimal implementation for this
purpose, but in the meantime, let's prevent it being called if we don't
need it by making it the second condition in the check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:43 +02:00
Sebastiaan van Stijn
9e9a17950a daemon: FindNetwork: minor cleanups
- don't initialize slices; it's not needed to append to them
- store network-ID in a var to prevent repeated lock/unlocking in nw.ID()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:40:43 +02:00
Sebastiaan van Stijn
0eea8d69b2 Merge pull request #46052 from thaJeztah/refactor_buildNetworkResource
daemon: refactor buildNetworkResource
2023-08-02 14:40:16 +02:00
Sebastiaan van Stijn
8c4af5dacb distribution: show image schema deprecation on all registries
When we added this deprecation warning, some registries had not yet
moved away from the deprecated specification, so we made the warning
conditional for pulling from Docker Hub.

That condition was added in 647dfe99a5,
which is over 4 Years ago, which should be time enough for images
and registries to have moved to current specifications.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:15:34 +02:00
Sebastiaan van Stijn
982bc0e228 distribution: update warning for deprecated image formats
- Use the same warning for both "v1 in manifest-index" and bare "v1" images.
- Update URL to use a "/go/" redirect, which allows the docs team to more
  easily redirect the URL to relevant docs (if things move).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 14:14:47 +02:00
Sebastiaan van Stijn
9bd2b7e7af Merge pull request #46138 from akerouanton/integration-run-attach
integration: Add RunAttach helper
2023-08-02 13:45:28 +02:00
Albin Kerouanton
5bd8aa5246 integration: Add RunAttach helper
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-02 10:23:32 +02:00
Albin Kerouanton
d7fb4dd170 integration: Extract stream demultiplexing from container.Exec
The original code in container.Exec was potentially leaking the copy
goroutine when the context was cancelled or timed out. The new
`demultiplexStreams()` function won't return until the goroutine has
finished its work, and to ensure that it takes care of closing the
hijacked connection.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-02 10:23:31 +02:00
Sebastiaan van Stijn
584f016e99 docs/api: remove stray underscore in version-history.md
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 09:12:19 +02:00
Akihiro Suda
0ac039f979 Merge pull request #46139 from thaJeztah/update_go1.20.7
update to go1.20.7
2023-08-02 14:49:29 +09:00
Sebastiaan van Stijn
d5cb7cdeae update to go1.20.7
Includes a fix for CVE-2023-29409

go1.20.7 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.20.7 milestone on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.6...go1.20.7

From the mailing list announcement:

[security] Go 1.20.7 and Go 1.19.12 are released

Hello gophers,

We have just released Go versions 1.20.7 and 1.19.12, minor point releases.

These minor releases include 1 security fixes following the security policy:

- crypto/tls: restrict RSA keys in certificates to <= 8192 bits

  Extremely large RSA keys in certificate chains can cause a client/server
  to expend significant CPU time verifying signatures. Limit this by
  restricting the size of RSA keys transmitted during handshakes to <=
  8192 bits.

  Based on a survey of publicly trusted RSA keys, there are currently only
  three certificates in circulation with keys larger than this, and all
  three appear to be test certificates that are not actively deployed. It
  is possible there are larger keys in use in private PKIs, but we target
  the web PKI, so causing breakage here in the interests of increasing the
  default safety of users of crypto/tls seems reasonable.

  Thanks to Mateusz Poliwczak for reporting this issue.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 23:46:01 +02:00
Sebastiaan van Stijn
69c9adb7d3 Merge pull request #45925 from thaJeztah/deprecate_is_automated
api: search: deprecate is_automated field, and is-automated filter
2023-08-01 16:27:00 +02:00
Sebastiaan van Stijn
fced566714 client: move resolveTLSConfig to a Client.tlsConfig()
This makes it slightly clearer what it does, as "resolve" may give the
impression it's doing more than just returning the TLS config configured
for the client.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 16:20:22 +02:00
Sebastiaan van Stijn
47af265234 client: Dialer: inline fallbackDial
fallbackDial was only used in a single place, and it was defined far away
from where it's used, so let's inline it, so that it's clear at a glance
what we're doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 16:20:18 +02:00
Sebastiaan van Stijn
74d9850bb9 client/buildkit: ClientOpts: update docs to use doc-links, and inline
inline the closures, and update the GoDoc to use doc-links to the related
buildkit function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 16:19:36 +02:00
Sebastiaan van Stijn
a1202648ff Merge pull request #46100 from thaJeztah/libnetwork_var_collide
libnetwork: rename vars that collided with builtin and type
2023-08-01 15:59:04 +02:00
Sebastiaan van Stijn
8d482accd1 Merge pull request #46123 from thaJeztah/log_the_log
swap logrus types for their containerd/logs aliases
2023-08-01 14:38:39 +02:00
Sebastiaan van Stijn
971083d419 api: search: deprecate is_automated field, and is-automated filter
The is-automated field is being deprecated by Docker Hub's search API,
and will always be set to "false" in future.

This patch deprecates the field and related filter for the Engine's API.

In future, the `is-automated` filter will no longer yield any results
when searching for `is-automated=true`, and will be ignored when
searching for `is-automated=false`.

Given that this field is deprecated by an external API, the deprecation
will not be versioned, and will apply to any API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:45:44 +02:00
Sebastiaan van Stijn
5e2a1195d7 swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00
Sebastiaan van Stijn
39a13456c1 Merge pull request #46109 from thaJeztah/windows_remove_discoverapi
libnetwork/driver: remove discoverAPI from Windows and Windows overlay
2023-08-01 10:51:56 +02:00
Sebastiaan van Stijn
2f17d5fa8f Merge pull request #46122 from thaJeztah/fix_daemon_integration_test
Fix TestDaemonProxy integration tests
2023-08-01 10:22:02 +02:00
Sebastiaan van Stijn
95bbbc0418 Merge pull request #46110 from thaJeztah/libnetwork_dead_code
libnetwork: remove some dead code, and un-export internal functions
2023-08-01 00:57:34 +02:00
Sebastiaan van Stijn
97daeb0020 Merge pull request #46131 from akerouanton/libnet-bridge-errors
Improve libnet/d/bridge errors
2023-08-01 00:15:21 +02:00
Brian Goff
1a51898d2e TestDaemonProxy: use new scanners to check logs
Also fixes up some cleanup issues.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 00:13:50 +02:00
Brian Goff
476e788090 Fix daemon proxy test for "reload sanitized"
I noticed this was always being skipped because of race conditions
checking the logs.

This change adds a log scanner which will look through the logs line by
line rather than allocating a big buffer.
Additionally it adds a `poll.Check` which we can use to actually wait
for the desired log entry.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 00:11:28 +02:00
Albin Kerouanton
e2d400bea9 libnet/d/bridge: Inline InvalidLinkIPAddrError
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-31 21:33:02 +02:00
Albin Kerouanton
6df4a00bb9 libnet/d/bridge: Remove pointless BadRequestError impl
IPv4AddrNoMatchError and IPv6AddrNoMatchError are currently implementing
BadRequestError. They are returned in two cases, and none are due to a
bad user request:

- When calling daemon's CreateNetwork route, if the bridge's IPv4
  address or none of the bridge's IPv6 addresses match what's requested.
  If that happens, there's a big issue somewhere in libnetwork or the
  kernel.
- When restoring a network, for the same reason. In that case, the
  on-disk state drifted from the interface state.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-31 21:32:00 +02:00
Albin Kerouanton
7c13985fa1 libnet/d/bridge: Inline InvalidIPTablesCfgError
This error can only be reached because of an error in our code, so it's
not a "bad user request". As it's never type asserted, no need to keep
it around.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-31 21:30:46 +02:00
Albin Kerouanton
d8f42ee21a libnet/d/bridge: Inline IPTableCfgError
This error is only used in defensive checks whereas the precondition is
already checked by caller. If we reach it, we messed something else. So
it's definitely not a BadRequest. Also, it's not type asserted anywhere,
so just inline it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-31 21:30:20 +02:00
Sebastiaan van Stijn
a40d00c421 libnetwork: remove ErrDataStoreNotInitialized
If was not used as a sentinel error, so inline the error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:33 +02:00
Sebastiaan van Stijn
66b379785e libnetwork: remove unused InvalidContainerIDError
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:33 +02:00
Sebastiaan van Stijn
d8cd4f6421 libnetwork: remove unused UnknownEndpointError
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:33 +02:00
Sebastiaan van Stijn
fc5c16f491 libnetwork: remove unused NetworkTypeError
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:33 +02:00
Sebastiaan van Stijn
e9f1888560 libnetwork: remove unused ErrInvalidConfigFile
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:32 +02:00
Sebastiaan van Stijn
eadaaa7f79 libnetwork: remove unused ErrNoContainer
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:32 +02:00
Sebastiaan van Stijn
99ab11bb90 libnetwork: remove unused ErrInvalidJoin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:32 +02:00
Sebastiaan van Stijn
9174eb00e2 libnetwork: remove unused ErrInvalidNetworkDriver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:32 +02:00
Sebastiaan van Stijn
7c2665298a libnetwork/types: move GetMinimalIP to overlay-driver and un-export
It was only used in drivers/overlay, and was not a function for any
"type" defined by libnetwork.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:32 +02:00
Sebastiaan van Stijn
d9b7a5f01a libnetwork/types: remove ErrInvalidProtocolBinding
It was not used as a sentinel error, and didn't carry a specific type,
which made it a rather complex way to create an error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:32 +02:00
Sebastiaan van Stijn
4445169cb5 libnetwork/types: remove PortBinding.Equal
It was only used in tests, so move it to a utility in the tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 21:00:26 +02:00
Sebastiaan van Stijn
4269712d06 libnetwork/types: remove unused UUID type
This type was added moved to the types package as part of a refactor
in 778e2a72b3
but the introduction of the sandbox API changed the existing API to
weak types (not using a plain string);
9a47be244a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 20:57:48 +02:00
Sebastiaan van Stijn
c203171ef6 libnetwork/types: remove unused RetryError and TimeoutError
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-31 20:57:45 +02:00
Albin Kerouanton
6d37ab0a83 libnet/d/bridge: Change interface implemented by error types
- InvalidIPTablesCfgError: implement InternalError instead of
  BadRequestError. This error is returned when an invalid iptables
  action is passed as argument (ie. none of -A, -I, or -D).
- ErrInvalidDriverConfig: don't implement BadRequestError. This is
  returned when libnetwork controller initialization pass bad driver
  config -- there's no call from an HTTP route.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-31 19:17:09 +02:00
Albin Kerouanton
03db2f8c3d libnet/d/bridge: Remove unused error types
These error types are removed:

- ErrInvalidContainerConfig
- ErrInvalidPort
- ErrInvalidAddressBinding
- InvalidSandboxIDError
- IPv4AddrRangeError

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-31 19:17:09 +02:00
Brian Goff
8197752d68 TestDaemonProxy: check proxy settings early
Allows tests to report their proxy settings for easier troubleshooting
on failures.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-31 13:33:32 +02:00
Sebastiaan van Stijn
7baab8bd6c Merge pull request #46118 from thaJeztah/bump_containerd_1.6.22
vendor: github.com/containerd/containerd v1.6.22
2023-07-31 13:13:51 +02:00
Sebastiaan van Stijn
4d674897f3 vendor: github.com/containerd/containerd v1.6.22
- full diff: https://github.com/containerd/containerd/compare/v1.6.21...v1.6.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.22

---

Notable Updates

- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
  when `securityContext.RunAsUser`, `RunAsUsername` are empty
- CRI: Write generated CNI config atomically
- Fix concurrent writes for `UpdateContainerStats`
- Make `checkContainerTimestamps` less strict on Windows
- Port-Forward: Correctly handle known errors
- Resolve `docker.NewResolver` race condition
- SecComp: Always allow `name_to_handle_at`
- Adding support to run hcsshim from local clone
- Pinned image support
- Runtime/V2/RunC: Handle early exits w/o big locks
- CRITool: Move up to CRI-TOOLS v1.27.0
- Fix cpu architecture detection issue on emulated ARM platform
- Task: Don't `close()` io before `cancel()`
- Fix panic when remote differ returns empty result
- Plugins: Notify readiness when registered plugins are ready
- Unwrap io errors in server connection receive error handling

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-30 15:59:52 +02:00
Sebastiaan van Stijn
935c04f33a Merge pull request #46117 from thaJeztah/bump_gotest_tools
vendor: gotest.tools/v3 v3.5.0
2023-07-30 15:59:33 +02:00
Akihiro Suda
975031bfcb Merge pull request #46107 from thaJeztah/update_image_spec
vendor: github.com/opencontainers/image-spec v1.1.0-rc4
2023-07-30 11:07:00 +09:00
Akihiro Suda
5919a2944a Merge pull request #46102 from thaJeztah/update_containerd_binary_1.7.3
update containerd binary to v1.7.3
2023-07-30 11:05:07 +09:00
Sebastiaan van Stijn
5438356348 libnetwork: rename vars that collided with builtin and type
- cap is a builtin
- agent is a type defined in the package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-29 22:28:03 +02:00
Sebastiaan van Stijn
a908460adb Merge pull request #46043 from thaJeztah/cleanup_iptables_the_firewall_strikes_back
libnetwork/iptables: cleaning up: "there's more where that came from"
2023-07-29 22:12:48 +02:00
Sebastiaan van Stijn
ce053a14aa vendor: gotest.tools/v3 v3.5.0
- go.mod: update dependencies and go version by
- Use Go1.20
- Fix couple of typos
- Added `WithStdout` and `WithStderr` helpers
- Moved `cmdOperators` handling from `RunCmd` to `StartCmd`
- Deprecate `assert.ErrorType`
- Remove outdated Dockerfile
- add godoc links

full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.4.0...v3.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-29 19:51:21 +02:00
Sebastiaan van Stijn
077208287b Merge pull request #46106 from thaJeztah/fix_filter_errors
api/types/filters: fix errors not being matched by errors.Is()
2023-07-29 03:20:01 +02:00
Sebastiaan van Stijn
be0f4a4737 libnetwork/driver: remove discoverAPI from Windows and Windows overlay
Follow-up to fca38bcd0a, which made the
Discover API optional for drivers to implement, but forgot to remove the
stubs from the Windows drivers, which didn't implement this API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-29 00:33:32 +02:00
Sebastiaan van Stijn
9200c6be90 Merge pull request #46108 from thaJeztah/driver_scopes
libnetwork: move datastore Scope consts to libnetwork/scope
2023-07-28 23:40:32 +02:00
Sebastiaan van Stijn
6ec03d6745 libnetwork: move datastore Scope consts to libnetwork/scope
The "Capability" type defines DataScope and ConnectivityScope fields,
but their value was set from consts in the datastore package, which
required importing that package and its dependencies for the consts
only.

This patch:

- Moves the consts to a separate "scope" package
- Adds aliases for the consts in the datastore package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 21:56:48 +02:00
Sebastiaan van Stijn
6f2e0fac19 Merge pull request #46086 from thaJeztah/split_discoverapi
libnetwork/driverapi: make discoverAPI an optional part of the interface
2023-07-28 18:09:52 +02:00
Sebastiaan van Stijn
86299bbd35 vendor: github.com/opencontainers/image-spec v1.1.0-rc4
changes:

- specs-go: remove artifact prefixed annotations
- Switch from scratch to empty
- Remove artifact media type reference
- image-index: add artifactType to specs and schema
- Add artifactType to image index
- Apply version change from #1050
- Specify the content of the scratch blob
- Add language from artifacttype field to forbid allowlists of media types
- spec: clarify descriptor, align with de facto artifact usage
- Remove special guidance around wasm
- Update descriptor.go
- releases: use +dev as in-development suffix
- version: bump HEAD back to -dev
- image-index: add the subject field

full diff: https://github.com/opencontainers/image-spec/compare/v1.1.0-rc3...v1.1.0-rc4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 18:04:08 +02:00
Sebastiaan van Stijn
fca38bcd0a libnetwork/driverapi: make discoverAPI an optional part of the interface
Most drivers do not implement this, so detect if a driver implements
the discoverAPI, and remove the implementation from drivers that do
not support it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 17:16:00 +02:00
Sebastiaan van Stijn
70e620f791 Merge pull request #46101 from thaJeztah/libnetwork_remote_concrete_type
libnetwork/remote: newDriver(): return concrete type
2023-07-28 17:14:52 +02:00
Sebastiaan van Stijn
490fee7d45 api/types/filters: fix errors not being matched by errors.Is()
I found that the errors returned weren't matched with `errors.Is()` when
wrapped.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 17:07:41 +02:00
Sebastiaan van Stijn
bf48d3ec29 update containerd binary to v1.7.3
- full diff: https://github.com/containerd/containerd/compare/v1.7.2...v1.7.3
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.3

----

Welcome to the v1.7.3 release of containerd!

The third patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
  when `securityContext.RunAsUser`,`RunAsUsername` are empty
- CRI: write generated CNI config atomically
- Port-Forward: Correctly handle known errors
- Resolve docker.NewResolver race condition
- Fix `net.ipv4.ping_group_range` with userns
- Runtime/V2/RunC: handle early exits w/o big locks
- SecComp: always allow `name_to_handle_at`
- CRI: Windows Pod Stats: Add a check to skip stats for containers that
  are not running
- Task: don't `close()` io before cancel()
- Remove CNI conf_template deprecation
- Fix issue for HPC pod metrics

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 13:21:20 +02:00
Sebastiaan van Stijn
a78381c399 update containerd binary to v1.7.2
- full diff: https://github.com/containerd/containerd/compare/v1.7.1...v1.7.2
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.2

----

Welcome to the v1.7.2 release of containerd!

The second patch release for containerd 1.7 includes enhancements to CRI
sandbox mode, Windows snapshot mounting support, and CRI and container IO
bug fixes.

CRI/Sandbox Updates

- Publish sandbox events
- Make stats respect sandbox's platform

Other Notable Updates

- Mount snapshots on Windows
- Notify readiness when registered plugins are ready
- Fix `cio.Cancel()` should close pipes
- CDI: Use CRI `Config.CDIDevices` field for CDI injection

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 13:20:34 +02:00
Sebastiaan van Stijn
0d76fc431a libnetwork/remote: newDriver(): return concrete type
Interface matching will happen on the receiver side.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 12:52:38 +02:00
Sebastiaan van Stijn
b2bde4a7d8 Merge pull request #46090 from thaJeztah/cleanup_TestLinuxParseMountRawSplit
volume/mounts: cleanup tests
2023-07-28 11:12:04 +02:00
Sebastiaan van Stijn
0e0c725224 Merge pull request #45511 from xpivarc/capabilites
Do not drop effective&permitted set
2023-07-28 01:17:43 +02:00
Sebastiaan van Stijn
ed9cd96bdb Merge pull request #46083 from akerouanton/move-libnet-testutils
tests: Move libnetwork/testutils to internal/testutils/netnsutils
2023-07-27 20:35:42 +02:00
Bjorn Neergaard
1bbc9d1ae4 Merge pull request #46091 from thaJeztah/windows_fix_service_register
windows: fix --register-service when executed from within binary directory
2023-07-27 12:04:02 -06:00
Albin Kerouanton
2870559b2c Merge pull request #45649 from akerouanton/allow-all-ipv6-icc
libnet/d/bridge: Allow IPv6 ICC from any IP address
2023-07-27 19:41:55 +02:00
Sebastiaan van Stijn
597ab901b9 Merge pull request #46079 from vvoland/c8d-missing-config
c8d/container: Follow snapshot parents for size calculation
2023-07-27 18:08:56 +02:00
Sebastiaan van Stijn
3e8fda0a70 windows: fix --register-service when executed from within binary directory
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows.

This issue was not limited to the go command itself, and could also affect binaries
that use `os.Command`, `os.LookPath`, etc.

From the related blogpost (https://blog.golang.org/path-security):

> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing

At time of the go1.15 release, the Go team considered changing the behavior of
`os.LookPath()` and `exec.LookPath()` to be a breaking change, and made the
behavior "opt-in" by providing the `golang.org/x/sys/execabs` package as a
replacement.

However, for the go1.19 release, this changed, and the default behavior of
`os.LookPath()` and `exec.LookPath()` was changed. From the release notes:
https://go.dev/doc/go1.19#os-exec-path

> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.

A result of this change was that registering the daemon as a Windows service
no longer worked when done from within the directory of the binary itself:

    C:\> cd "Program Files\Docker\Docker\resources"
    C:\Program Files\Docker\Docker\resources> dockerd --register-service
    exec: "dockerd": cannot run executable found relative to current directory

Note that using an absolute path would work around the issue:

    C:\Program Files\Docker\Docker>resources\dockerd.exe --register-service

This patch changes `registerService()` to use `os.Executable()`, instead of
depending on `os.Args[0]` and `exec.LookPath()` for resolving the absolute
path of the binary.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-27 16:20:49 +02:00
Sebastiaan van Stijn
d69b1fdb72 volume/mounts: cleanup tests
- don't use un-keyed structs
- user assert.Check where possible
- use consts for fixed values

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-27 15:00:13 +02:00
Sebastiaan van Stijn
10aff57b26 Merge pull request #46037 from AkihiroSuda/rro-remove-short
volume: remove the short RRO forms in favor of the long forms
2023-07-27 14:11:30 +02:00
Sebastiaan van Stijn
198ba185c7 Merge pull request #46081 from vvoland/c8d-refreshimage-refactor
daemon/list: Refactor refreshImage and make `readConfig` return errdefs
2023-07-27 13:52:51 +02:00
Albin Kerouanton
da9e44a620 libnet/d/bridge: Allow IPv6 ICC from any IP address
IPv6 ipt rules are exactly the same as IPv4 rules, although both
protocol don't use the same networking model. This has bad consequences,
for instance: 1. the current v6 rules disallow Neighbor
Solication/Advertisement ; 2. multicast addresses can't be used ; 3.
link-local addresses are blocked too.

To solve this, this commit changes the following rules:

```
-A DOCKER-ISOLATION-STAGE-1 ! -s fdf1:a844:380c:b247::/64 -o br-21502e5b2c6c -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fdf1:a844:380c:b247::/64 -i br-21502e5b2c6c -j DROP
```

into:

```
-A DOCKER-ISOLATION-STAGE-1 ! -s fdf1:a844:380c:b247::/64 ! -i br-21502e5b2c6c   -o br-21502e5b2c6c -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fdf1:a844:380c:b247::/64   -i br-21502e5b2c6c ! -o br-21502e5b2c6c -j DROP
```

These rules only limit the traffic ingressing/egressing the bridge, but
not traffic between veth on the same bridge.

Note that, the Kernel takes care of dropping invalid IPv6 packets, eg.
loopback spoofing, thus these rules don't need to be more specific.

Solve #45460.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-27 10:56:08 +02:00
Sebastiaan van Stijn
c30c3f94c7 Merge pull request #46041 from thaJeztah/datastore_remove_libkv
libnetwork: remove most of kvstore
2023-07-26 22:27:28 +02:00
Albin Kerouanton
31d09f6ee9 testutils: move Logger interface to testutils
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-26 21:16:09 +02:00
Albin Kerouanton
492c09276d tests: Move libnetwork/testutils to internal/testutils/netnsutils
We don't want to maintain backward compatibility for this package, so
better make it an internal.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-26 21:06:36 +02:00
Paweł Gronowski
13180c1c49 daemon/list: Drop unused arg from containerReducer
refreshImage is the only function used as a reducer and it doesn't use
the `filter *listContext`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-26 17:28:12 +02:00
Paweł Gronowski
68991ae240 daemon/list: Refactor refreshImage
Add context comments and make it a bit more readable.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-26 17:28:10 +02:00
Sebastiaan van Stijn
7a44e4cde0 Merge pull request #46034 from rumpl/c8d-image-list
c8d: Various images/json API fixes
2023-07-26 16:50:27 +02:00
Djordje Lukic
e5cffc07b3 c8d: Fix TestImagesFilterMultiReference
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-26 14:34:42 +02:00
Djordje Lukic
1d8108b009 c8d: images/json, add image labels to the response
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-26 14:33:22 +02:00
Djordje Lukic
a0bba26919 c8d: Populate RepoTags instead of adding a new image to the list
Aggregate same images into one object and add the list of tags pointing
to it to the RepoTags array

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-26 14:33:20 +02:00
Djordje Lukic
ad2760ec80 integration: Move image filter tests to integration
- use assert.Check to continue the test even if a check fails
- assert the total number of images returned, not only their RepoTags
- use subtests

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-26 14:33:16 +02:00
Paweł Gronowski
4d8e3f54cc c8d/container: Follow snapshot parents for size calculation
Refactor GetContainerLayerSize to calculate unpacked image size only by
following the snapshot parent tree directly instead of following it by
using diff ids from image config.

This works even if the original manifest/config used to create that
container is no longer present in the content store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-26 14:15:15 +02:00
Bjorn Neergaard
427f95c4e6 Merge pull request #45978 from thaJeztah/client_docs_touch_up
client: touch-up docs, cleanup some tests, and minor refactoring
2023-07-26 05:55:46 -06:00
Paweł Gronowski
5a39bee635 daemon/list: Replace ErrImageDoesNotExist check
Check for generic `errdefs.NotFound` rather than specific error helper
struct when checking if the error is caused by the image not being
present.
It still works for `ErrImageDoesNotExist` because it
implements the NotFound errdefs interface too.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-26 13:40:04 +02:00
Paweł Gronowski
7379d18018 c8d/readConfig: Translate c8d NotFound to errdefs
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-26 13:40:02 +02:00
Sebastiaan van Stijn
f3b76bc1da daemon: refactor buildNetworkResource to use a struct-literal
Now that all helper functions are updated, we can use a struct-literal
for this function, which makes it slightly easier to read.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-26 13:03:22 +02:00
Sebastiaan van Stijn
a8c0b05052 daemon: split buildDetailedNetworkResources into two functions
Split the buildDetailedNetworkResources function into separate functions for
collecting container attachments (`buildContainerAttachments`) and service
attachments (`buildServiceAttachments`). This allows us to get rid of the
"verbose" bool, and makes the logic slightly more transparent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-26 13:03:12 +02:00
Sebastiaan van Stijn
8caf974dcd daemon: refactor buildEndpointResource
- Pass the endpoint and endpoint-info, instead of individual fields from the
  endpoint.
- Remove redundant nil-check, as it's already checked on the call-side
  in `buildDetailedNetworkResources`, which skips endpoints without info.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-26 12:06:19 +02:00
Sebastiaan van Stijn
437ced91ec daemon: refactor buildIpamResources
Make the function return the constructed network.IPAM instead of applying
it to a network struct, and rename it to "buildIPAMResources".

Rewrite the function itself:

- Use struct-literals where possible to make it slightly more readable.
- Use a boolean (hasIPv4Config, hasIPv6Config) for both IPv4 and IPv6 to
  check whether the IPAM-info needs to be added. This makes the logic the
  same for both, and makes the processing order-independent. This also
  allows for the `network.IpamInfo()` call to be skipped if it's not needed.
- Change order of "ipv4 config / ipv4 info" and "ipv6 config / ipv4 info"
  blocks to make it slightly clearer (and to allow skipping the forementioned
  call to `network.IpamInfo()`).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-26 12:06:19 +02:00
Sebastiaan van Stijn
ca5ac19ea4 daemon: refactor buildPeerInfoResources
Move the length-check into the function, and change the code to
be a basic type-case, as networkdb.PeerInfo and network.PeerInfo
are identical types.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-26 12:06:17 +02:00
Sebastiaan van Stijn
536e3692c6 Merge pull request #46017 from thaJeztah/pkg_plugin_cleanup
pkg/plugins: some cleaning up (step 1)
2023-07-26 02:10:36 +02:00
Sebastiaan van Stijn
5fe05d3eeb Merge pull request #46053 from thaJeztah/network_backend_remove_FindNetwork
api/server/router/network: remove unused Backend.FindNetwork
2023-07-25 23:16:02 +02:00
Sebastiaan van Stijn
02dc5ea720 libnetwork/datastore: remove Store.KVStore()
It's no longer used, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:42:17 +02:00
Sebastiaan van Stijn
0f7bf67f83 pkg/plugins: TestGet(): use sub-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:38 +02:00
Sebastiaan van Stijn
e1ad4aa002 pkg/plugins: make package-level socketsPath var a LocalRegistry field
This variable was only accessed from within LocalRegistry methods, but
due to being a package-level variable, tests had to deal with setting
and resetting it.

Move it to be a field scoped to the LocalRegistry. This simplifies the
tests, and to make this more transparent, also removing the "Setup()"
helper (which, wasn't marked as a t.Helper()).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
5bd44cf3c4 pkg/plugins: remove "load()" function
It was used in a single place and was abstracting "loadWithRetry";
let's just inline it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
77c03221c9 pkg/plugins/transport: remove RequestFactory interface
The client's transport can only be set by newClientWithTransport, which
is not exported, and always uses a transport.HTTPTransport.

However, requestFactory is mocked in one of the tests, so keep the interface,
but make it a local, non-exported one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
d12ec5f796 pkg/plugins/transport: remove unused Transport interface
The interface is not consumed anywhere, and only non-exported functions
produced one, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
dfd331b2c8 pkg/plugins/transport: export httpTransport, and return concrete type
Make NewHTTPTransport return a concrete type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
77103c7c03 pkg/plugins/transport: inline newHTTPRequest
It was only used in a single location; just inline the code

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
4af4adc7ba pkg/plugins: remove LocalRegistry.SpecsPaths()
This field was exported, but never mutated outside of the package, and
effectively a rather "creative" way to define a method on LocalRegistry.

While un-exporting also store these paths in a field, instead of constructing
them on every call, as the results won't change during the lifecycle of the
LocalRegistry.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:36 +02:00
Sebastiaan van Stijn
a44c25c2f1 pkg/plugins: split exported from implementation
Split the exported SpecsPaths from the platform-specific implementations,
so that documentation can be maintained in a single location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:11 +02:00
Sebastiaan van Stijn
954d50b88a plg/plugins: rename vars that collided, or poorly cased
Reduce some noise while reading the code in my IDE :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:35:43 +02:00
Sebastiaan van Stijn
20889dbac6 pkg/plugins: move Plugin.ScopedPath to platform-agnostic file
Since 0e5eaf8ee3, these implementations
were fully identical, so removing the duplicate, and move it to a
platform-agnostic file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:35:40 +02:00
Sebastiaan van Stijn
a5ee0d6af8 libnetwork: don't access KVStore directly in tests
Test the datastore, not the KVStore backing it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:21:03 +02:00
Sebastiaan van Stijn
b378669836 libnetwork/datastore: don't access KVStore directly in tests
Test the datastore, not the KVStore backing it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:20:21 +02:00
Sebastiaan van Stijn
65978e6982 libnetwork/datastore: cleanup tests
- use gotest.tools assertions
- use consts and struct-literals where possible
- use assert.Check instead of t.Fatal() where possible
- fix some unhandled errors

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:20:21 +02:00
Sebastiaan van Stijn
478f4aed63 libnetwork/datastore: ScopeCfg.IsValid(): un-wrap conditions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:20:21 +02:00
Sebastiaan van Stijn
37cbdeb1f2 libnetwork: remove most of kvstore
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:20:18 +02:00
Sebastiaan van Stijn
e4134d5c0d Merge pull request #46048 from thaJeztah/libnetwork_test
libnetwork: refactor TestErrorInterfaces into a test
2023-07-25 17:12:59 +02:00
Sebastiaan van Stijn
d713b7ed45 api/server/router/network: remove unused Backend.FindNetwork
This function was used by the postNetworkConnect() handler, but is handled
by the backend itself, starting with d63a5a1ff5.
Since that commit, this function was no longer used, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 16:33:57 +02:00
Akihiro Suda
8291893239 Merge pull request #46070 from thaJeztah/bump_runtime_spec
vendor: github.com/opencontainers/runtime-spec v1.1.0
2023-07-25 23:33:48 +09:00
Sebastiaan van Stijn
7e4ffa3fa9 Merge pull request #46050 from thaJeztah/libnetwork_remove_interface
libnetwork: remove Network interface
2023-07-25 16:02:19 +02:00
Sebastiaan van Stijn
b3ed92d88b Merge pull request #46040 from thaJeztah/datastore_no_interfaces
libnetwork/datastore: remove DataStore interface, and rename constructor
2023-07-25 10:41:43 +02:00
Sebastiaan van Stijn
08815cb1b9 Merge pull request #46071 from thaJeztah/fix_comment
daemon/containerd: ImageService. LoadImage fix stray slash in comment
2023-07-25 10:39:53 +02:00
Sebastiaan van Stijn
1bc7a2aacd daemon/containerd: ImageService. LoadImage fix stray slash in comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:24:31 +02:00
Sebastiaan van Stijn
126a089382 vendor: github.com/opencontainers/runtime-spec v1.1.0
- Add I/O Priority Configuration for Process Group in Linux Containers
- spec: add scheduler entity (based on the Linux sched_setattr(2) syscall).

full diff: https://github.com/opencontainers/runtime-spec/compare/v1.1.0-rc.2...v1.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:19:44 +02:00
Luboslav Pivarc
42fa7a1951 Integration test for capabilities
Verify non-root containers are able to use file
capabilities.

Signed-off-by: Luboslav Pivarc <lpivarc@redhat.com>
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-24 17:10:50 -04:00
Sebastiaan van Stijn
d4d645d32a Merge pull request #46060 from vvoland/integration-no-fail-fast
ci/integration: Don't fail fast
2023-07-24 20:53:34 +02:00
Sebastiaan van Stijn
7ced56b356 Merge pull request #46065 from corhere/libn/defuse-datastore-key-latent-race
libnetwork/datastore: prevent data races in Key()
2023-07-24 19:53:24 +02:00
Cory Snider
5ef9e2632f libnetwork/datastore: prevent data races in Key()
The rootChain variable that the Key function references is a
package-global slice. As the append() built-in may append to the slice's
backing array in place, it is theoretically possible for the temporary
slices in concurrent Key() calls to share the same backing array, which
would be a data race. Thankfully in my tests (on Go 1.20.6)

    cap(rootChain) == len(rootChain)

held true, so in practice a new slice is always allocated and there is
no race. But that is a very brittle assumption to depend upon, which
could blow up in our faces at any time without warning. Rewrite the
implementation in a way which cannot lead to data races.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-24 12:18:04 -04:00
Paweł Gronowski
48cc28e4ef hack/integration: Add TEST_INTEGRATION_FAIL_FAST
Before this change, integration test would fail fast and not execute all
test suites when one suite fails.
Change this behavior into opt-in enabled by TEST_INTEGRATION_FAIL_FAST
variable.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-24 16:54:31 +02:00
Sebastiaan van Stijn
96b473a0bd Merge pull request #45992 from jedevc/registry-hosts-use-service-config
daemon: use the registry service config for getting registry hosts
2023-07-24 16:45:09 +02:00
Sebastiaan van Stijn
f70e1b315f libnetwork/datastore: remove unused ParseKey() utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-24 16:01:43 +02:00
Sebastiaan van Stijn
704a19b25d libnetwork/datastore: remove DataStore interface
It only had a single implementation, so let's remove the interface.

While changing, also renaming;

- datastore.DataStore -> datastore.Store
- datastore.NewDataStore -> datastore.New
- datastore.NewDataStoreFromConfig -> datastore.FromConfig

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-24 15:54:05 +02:00
Sebastiaan van Stijn
d5b1e43b8f libnetwork/datastore: move MockData to a _test file
It's only used in tests, and only within this package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-24 15:28:07 +02:00
Bjorn Neergaard
d16579971e Merge pull request #45981 from thaJeztah/bump_buildkit_0.11
vendor: github.com/moby/buildkit v0.11.7-dev
2023-07-24 07:13:40 -06:00
Sebastiaan van Stijn
0ec73a7892 vendor: github.com/moby/buildkit v0.11.7-dev
full diff: 0a15675913...616c3f613b

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-24 10:03:53 +02:00
Sebastiaan van Stijn
afd4805278 Merge pull request #45399 from vvoland/winddows-unskip-kill-tests
integration/windows: Unskip some kill tests
2023-07-23 16:29:19 +02:00
Sebastiaan van Stijn
24fef11e44 Merge pull request #45724 from akhramov/feature/chroot-archive-freebsd-build
pkg/chrootarchive: fix FreeBSD build
2023-07-23 16:26:29 +02:00
Sebastiaan van Stijn
64c6f72988 libnetwork: remove Network interface
There's only one implementation; drop the interface and use the
concrete type instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-22 11:56:41 +02:00
Sebastiaan van Stijn
edafcb2c39 libnetwork/iptables: un-export ErrConntrackNotConfigurable, IsConntrackProgrammable
These were only used internally, and ErrConntrackNotConfigurable was not used
as a sentinel error anywhere. Remove ErrConntrackNotConfigurable, and change
IsConntrackProgrammable to return an error instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-21 21:08:12 +02:00
Sebastiaan van Stijn
e57b807a42 libnetwork: Controller.NewNetwork: inline arrangeUserFilterRule()
arrangeUserFilterRule uses the package-level [`ctrl` variable][1], which
holds a reference to a controller instance. This variable is set by
[`setupArrangeUserFilterRule()`][2], which is called when initialization
a controller ([`libnetwork.New`][3]).

In normal circumstances, there would only be one controller, created during
daemon startup, and the instance of the controller would be the same as
the controller that `NewNetwork` is called from, but there's no protection
for the `ctrl` variable, and various integration tests create their own
controller instance.

The global `ctrl` var was introduced in [54e7900fb89b1aeeb188d935f29cf05514fd419b][4],
with the assumption that [only one controller could ever exist][5].

This patch tries to reduce uses of the `ctrl` variable, and as we're calling
this code from inside a method on a specific controller, we inline the code
and use that specific controller instead.

[1]: 37b908aa62/libnetwork/firewall_linux.go (L12)
[2]: 37b908aa62/libnetwork/firewall_linux.go (L14-L17)
[3]: 37b908aa62/libnetwork/controller.go (L163)
[4]: 54e7900fb8
[5]: https://github.com/moby/libnetwork/pull/2471#discussion_r343457183

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-21 21:08:03 +02:00
Sebastiaan van Stijn
be8ff186d6 libnetwork: refactor TestErrorInterfaces into a test
This function was added in libnetwork through 50964c9948
and, based on the name of the function and its signature, I think it
was meant to be a test. This patch refactors it to be one.

Changing it into a test made it slightly broken:

    go test -v -run TestErrorInterfaces
    === RUN   TestErrorInterfaces
        errors_test.go:15: Failed to detect err network  not found is of type BadRequestError. Got type: libnetwork.ErrNoSuchNetwork
        errors_test.go:15: Failed to detect err endpoint  not found is of type BadRequestError. Got type: libnetwork.ErrNoSuchEndpoint
        errors_test.go:42: Failed to detect err unknown driver "" is of type ForbiddenError. Got type: libnetwork.NetworkTypeError
        errors_test.go:42: Failed to detect err unknown network  id  is of type ForbiddenError. Got type: *libnetwork.UnknownNetworkError
        errors_test.go:42: Failed to detect err unknown endpoint  id  is of type ForbiddenError. Got type: *libnetwork.UnknownEndpointError
    --- FAIL: TestErrorInterfaces (0.00s)
    FAIL

This was because some errors were tested twice, but for the wrong type
(`NetworkTypeError`, `UnknownNetworkError`, `UnknownEndpointError`).

Moving them to the right test left no test-cases for `types.ForbiddenError`,
so I added `ActiveContainerError` to not make that part of the code feel lonely.

Other failures were because some errors were changed from `types.BadRequestError`
to a `types.NotFoundError` error in commit ba012a703a,
so I moved those to the right part.

Before this patch:

    go test -v -run TestErrorInterfaces
    === RUN   TestErrorInterfaces
    --- PASS: TestErrorInterfaces (0.00s)
    PASS
    ok  	github.com/docker/docker/libnetwork	0.013s

After this patch:

    go test -v -run TestErrorInterfaces
    === RUN   TestErrorInterfaces
    --- PASS: TestErrorInterfaces (0.00s)
    PASS
    ok  	github.com/docker/docker/libnetwork	0.013s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-21 20:52:48 +02:00
Sebastiaan van Stijn
9484520327 libnetwork: arrangeUserFilterRule: don't return early
commit ffd75c2e0c updated this function to
set up the DOCKER-USER chain for both iptables and ip6tables, however the
function would return early if a failure happened (instead of continuing
with the next iptables version).

This patch extracts setting up the chain to a separate function, and updates
arrangeUserFilterRule to log the failure as a warning, but continue with
the next iptables version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-21 20:08:58 +02:00
Sebastiaan van Stijn
a5f45b47a3 libnetwork: Controller: combine iptablesEnabled and ip6tablesEnabled
These functions were mostly identical, except for iptables being enabled
by default (unless explicitly disabled by config).

Rewrite the function to a enabledIptablesVersions, which returns the list
of iptables-versions that are enabled for the controller. This prevents
having to acquire a lock twice, and simplifies arrangeUserFilterRule, which
can now just iterate over the enabled versions.

Also moving this function to a linux-only file, as other platforms don't have
the iptables types defined.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-21 20:08:53 +02:00
Sebastiaan van Stijn
454b6a7cf5 Merge pull request #45954 from thaJeztah/cleanup_ioutils
pkg/ioutils: some cleanups in tests
2023-07-20 19:44:21 +02:00
Akihiro Suda
483a1933a2 volume: remove the short RRO forms in favor of the long forms
"ro-non-recursive", "ro-force-recursive", and "rro" are
now removed from the legacy mount API.

CLI may still support them via the new mount API (if we want).

Follow-up to PR 45278

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-07-20 22:34:09 +09:00
Sebastiaan van Stijn
49dce504bf Merge pull request #46032 from thaJeztah/bk_deps_part2
vendor: update dependencies in preparation of BuildKit v0.12 and containerd 1.7 update (part 2)
2023-07-20 11:01:38 +02:00
Sebastiaan van Stijn
5da84141c4 vendor: github.com/prometheus/common v0.42.0
full diff: https://github.com/prometheus/common/compare/v0.37.0...v0.42.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-20 08:02:55 +02:00
Sebastiaan van Stijn
44ab4d44e0 vendor: github.com/prometheus/procfs v0.9.0
full diff: https://github.com/prometheus/procfs/compare/v0.8.0...v0.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-20 08:01:34 +02:00
Sebastiaan van Stijn
6791b8051a vendor: github.com/felixge/httpsnoop v1.0.3
full diff: https://github.com/felixge/httpsnoop/compare/v1.0.2...v1.0.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-20 07:58:41 +02:00
Sebastiaan van Stijn
9d3590f375 vendor: github.com/Microsoft/go-winio v0.6.1
Unfortunately also brings in golang.org/x/tools as a dependency, due to
go-winio using a "tools.go" file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-20 07:54:31 +02:00
Sebastiaan van Stijn
35d180126a vendor: google.golang.org/protobuf v1.30.0
no changes in vendored files

full diff: https://github.com/protocolbuffers/protobuf-go/compare/v1.29.1...v1.30.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-20 07:50:35 +02:00
Sebastiaan van Stijn
b835c28383 vendor: golang.org/x/oauth2 v0.6.0
full diff: https://github.com/golang/oauth2/compare/v0.1.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-20 07:49:00 +02:00
Sebastiaan van Stijn
51fad48097 Merge pull request #45937 from thaJeztah/testattach_table
integration/container: use subtests for TestAttach
2023-07-19 19:24:33 +02:00
Sebastiaan van Stijn
6b2ee07a26 Merge pull request #45929 from neersighted/with-go-mod_ROOTDIR
hack: use Git-free ROOTDIR
2023-07-19 19:23:00 +02:00
Sebastiaan van Stijn
203c683dff Merge pull request #46028 from thaJeztah/bump_runc_binary_1.1.8
Dockerfile: update runc binary to v1.1.8
2023-07-19 19:18:21 +02:00
Sebastiaan van Stijn
3cfc1ffb0a pkg/ioutils: some cleanups in tests
- remove gotest.tools dependency as it was only used in one test,
  and only for a trivial check
- use t.TempDir()
- rename vars that collided with package types
- don't use un-keyed structs
- explicitly ignore some errors to please linters
- use iotest.ErrReader
- TestReadCloserWrapperClose: verify reading works before closing :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 19:17:22 +02:00
Akihiro Suda
3b77e38b8e Merge pull request #46027 from thaJeztah/bk_deps
vendor: update dependencies in preparation of BuildKit v0.12 and containerd 1.7 update
2023-07-20 02:03:13 +09:00
Sebastiaan van Stijn
df86d855f5 Dockerfile: update runc binary to v1.1.8
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.8
full diff: https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.9

This is the eighth patch release of the 1.1.z release branch of runc.
The most notable change is the addition of RISC-V support, along with a
few bug fixes.

- Support riscv64.
- init: do not print environment variable value.
- libct: fix a race with systemd removal.
- tests/int: increase num retries for oom tests.
- man/runc: fixes.
- Fix tmpfs mode opts when dir already exists.
- docs/systemd: fix a broken link.
- ci/cirrus: enable some rootless tests on cs9.
- runc delete: call systemd's reset-failed.
- libct/cg/sd/v1: do not update non-frozen cgroup after frozen failed.
- CI: bump Fedora, Vagrant, bats.
- .codespellrc: update for 2.2.5.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:22:03 +02:00
Sebastiaan van Stijn
f5e64a9eaa vendor: github.com/containerd/stargz-snapshotter/estargz v0.14.3
no changes in vendored files

full diff: https://github.com/containerd/stargz-snapshotter/estargz/compare/v0.13.0...v0.14.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:02 +02:00
Sebastiaan van Stijn
d431345bf3 vendor: google.golang.org/protobuf v1.29.1
full diff: https://github.com/protocolbuffers/protobuf-go/compare/v1.28.1...v1.29.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:02 +02:00
Sebastiaan van Stijn
30916df3d3 vendor: github.com/containerd/ttrpc v1.2.2
full diff: https://github.com/containerd/ttrpc/compare/v1.1.1...v1.2.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:01 +02:00
Sebastiaan van Stijn
4f4dd2d995 vendor: github.com/containerd/go-cni v1.1.9
full diff: https://github.com/containerd/go-cni/compare/v1.1.6...v1.1.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:01 +02:00
Sebastiaan van Stijn
287130a254 vendor: github.com/cenkalti/backoff/v4 v4.2.0
full diff: https://github.com/cenkalti/backoff/compare/v4.1.2...v4.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:01 +02:00
Sebastiaan van Stijn
7145129f27 vendor: k8s.io/klog/v2 v2.90.1
full diff: https://github.com/kubernetes/klog/compare/v2.80.1...v2.90.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:01 +02:00
Sebastiaan van Stijn
410a52b763 vendor: github.com/fsnotify/fsnotify v1.6.0
full diff: https://github.com/fsnotify/fsnotify/compare/v1.5.1...v1.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:01 +02:00
Sebastiaan van Stijn
741c28ae9f vendor: github.com/containerd/cgroups v1.1.0
no changes in vendored files

full diff: https://github.com/containerd/cgroups/compare/v1.0.4...v1.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:01 +02:00
Sebastiaan van Stijn
e84b72710b vendor: github.com/aws/aws-sdk-go-v2/config v1.18.16
full diff: https://github.com/aws/aws-sdk-go-v2/compare/config/v1.17.4...config/v1.18.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:00 +02:00
Sebastiaan van Stijn
e235763a38 vendor: github.com/aws/aws-sdk-go-v2/credentials v1.13.16
full diff: https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.12.17...credentials/v1.13.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:00 +02:00
Sebastiaan van Stijn
4c30bf1667 vendor: github.com/aws/aws-sdk-go-v2 v1.17.6
full diff: https://github.com/aws/aws-sdk-go-v2/compare/v1.16.13...v1.17.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:00 +02:00
Sebastiaan van Stijn
cb53ce9f89 vendor: github.com/aws/smithy-go v1.13.5
full diff: https://github.com/aws/smithy-go/compare/v1.13.1...v1.13.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:06:00 +02:00
Sebastiaan van Stijn
f463e50719 vendor: github.com/AdaLogics/go-fuzz-headers v0.0.0-20230106234847-43070de90fa1
full diff: 3345c89a7c...43070de90f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:59 +02:00
Sebastiaan van Stijn
359ab384fd vendor: google.golang.org/genproto v0.0.0-20230306155012-7f2fa6fef1f4
full diff: 10f96fb3db...7f2fa6fef1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:59 +02:00
Sebastiaan van Stijn
b11bdb9212 vendor: cloud.google.com/go/logging v1.7.0
full diff: https://github.com/googleapis/google-cloud-go/compare/logging/v1.4.2...logging/v1.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:59 +02:00
Sebastiaan van Stijn
64609120d8 vendor: github.com/googleapis/gax-go/v2 v2.7.0
full diff: https://github.com/googleapis/gax-go/compare/v2.4.0...v2.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:59 +02:00
Sebastiaan van Stijn
2990defea7 vendor: go.opencensus.io v0.24.0
full diff: https://github.com/census-instrumentation/opencensus-go/compare/v0.23.0...v0.24.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:58 +02:00
Sebastiaan van Stijn
d7aa47ddb8 vendor: github.com/googleapis/enterprise-certificate-proxy v0.2.3
full diff: https://github.com/googleapis/enterprise-certificate-proxy/compare/v0.1.0...v0.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:58 +02:00
Sebastiaan van Stijn
296040b1f4 vendor: github.com/cespare/xxhash/v2 v2.2.0
full diff: https://github.com/cespare/xxhash/compare/v2.1.2...v2.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:58 +02:00
Sebastiaan van Stijn
033f882314 vendor: golang.org/x/oauth2 v0.5.0
full diff: https://github.com/golang/oauth2/compare/v0.1.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:58 +02:00
Sebastiaan van Stijn
b615d195b2 vendor: github.com/golang/protobuf v1.5.3
- jsonpb: accept 'null' as a valid representation of NullValue in unmarshal
  The canonical JSON representation for NullValue is JSON "null".

full diff: https://github.com/golang/protobuf/compare/v1.5.2...v1.5.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:57 +02:00
Sebastiaan van Stijn
c5c8f433e6 vendor: github.com/containerd/typeurl/v2 v2.1.1
- unmarshal does not return nil object when value is nil
- fixes "ctr tasks checkpoint returns invalid task checkpoint option for io.containerd.runc.v2: unknown"

full diff: https://github.com/containerd/typeurl/compare/v2.1.0...v2.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:05:57 +02:00
Sebastiaan van Stijn
82190c7296 Merge pull request #46022 from thaJeztah/bump_buildx_compose
Dockerfile: update buildx v0.11.2, compose v2.20.1
2023-07-19 16:58:35 +02:00
Sebastiaan van Stijn
6f2a242193 Merge pull request #46024 from vvoland/c8d-inspect-variant
c8d/inspect: Include platform Variant
2023-07-19 15:33:36 +02:00
Sebastiaan van Stijn
6025938ee9 Merge pull request #45987 from thaJeztah/cleanup_iptables_the_sequel
libnetwork/iptables: some cleanups and refactoring: the sequel
2023-07-19 14:38:12 +02:00
Paweł Gronowski
2659f7f740 c8d/inspect: Include platform Variant
Variant was mistakenly omitted in the returned V1Image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-19 13:03:40 +02:00
Sebastiaan van Stijn
a8fd1e3135 Dockerfile: update compose to v2.20.1
Update the COMPOSE_VERSION :)

- release notes: https://github.com/docker/compose/releases/tag/v2.20.1
- full diff: https://github.com/docker/compose/compare/v2.20.0...v2.20.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 13:01:52 +02:00
Sebastiaan van Stijn
d78893921a Dockerfile: update buildx to v0.11.2
Update the BUILDX_VERSION :)

release notes:

- https://github.com/docker/buildx/releases/tag/v0.11.1
- https://github.com/docker/buildx/releases/tag/v0.11.2

full diff: https://github.com/docker/buildx/compare/v0.11.0...v0.11.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 12:59:01 +02:00
Sebastiaan van Stijn
e942499c66 Merge pull request #45991 from thaJeztah/client_cleanups
client: return explicit "nil" for no errors, and remove nil check
2023-07-19 12:17:32 +02:00
Sebastiaan van Stijn
9ba41b791e Merge pull request #45963 from rumpl/c8d-image-save-lease
c8d: Make sure the content isn't removed while we export
2023-07-19 11:53:31 +02:00
Djordje Lukic
f3a6b0fd08 c8d: Make sure the content isn't removed while we export
This change add leases for all the content that will be exported, once
the image(s) are exported the lease is removed, thus letting
containerd's GC to do its job if needed. This fixes the case where
someone would remove an image that is still being exported.

This fixes the TestAPIImagesSaveAndLoad cli integration test.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-19 09:59:11 +02:00
Sebastiaan van Stijn
f061f39145 Merge pull request #46009 from vvoland/c8d-resolve-fix-digested-and-named
c8d/resolveImage: Fix Digested and Named reference
2023-07-19 00:12:03 +02:00
Bjorn Neergaard
124b1141d7 Merge pull request #46016 from neersighted/AUTHORS_.mailmap
AUTHORS,.mailmap: update with recent contributors
2023-07-18 15:29:58 -06:00
Bjorn Neergaard
6cba44277f AUTHORS,.mailmap: update with recent contributors
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 13:55:16 -06:00
Sebastiaan van Stijn
f329397077 Merge pull request #46007 from vvoland/c8d-inspect-fix-duplicate-digested
c8d/inspect: Don't duplicate digested ref
2023-07-18 20:19:40 +02:00
Bjorn Neergaard
db7fef6e50 Merge pull request #46012 from neersighted/buildkit-ref_nits
ci(buildkit): remove misleading code from buildkit-ref
2023-07-18 11:33:38 -06:00
Artem Khramov
8b843732b3 pkg/chrootarchive: fix FreeBSD build
For unix targets, `goInChroot()` is only implemented for `Linux`,
hence FreeBSD build fails.

This change

- Adds FreeBSD-specific chrooted tar/untar implementation
- Fixes statUnix() to accomodate to FreeBSD devminor/devmajor
- quirk. See also: https://github.com/containerd/containerd/pull/5991

Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Cory Snider <corhere@gmail.com>
2023-07-18 18:42:08 +02:00
Paweł Gronowski
eaf141c86a c8d/resolveImage: Fix Digested and Named reference
When resolving a reference that is both a Named and Digested, it could
be resolved to an image that has the same digest, but completely
different repository name.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 17:55:46 +02:00
Paweł Gronowski
028eab9ebb c8d/inspect: Don't duplicate digested ref
If image name is already an untagged digested reference, don't produce
additional digested ref.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 17:47:53 +02:00
Bjorn Neergaard
7310a7cd0c ci(buildkit): remove early-return from buildkit-ref
This doesn't really make sense now that this script returns a
$GITHUB_ENV snippet.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:42 -06:00
Bjorn Neergaard
4ecc01f3ad ci(buildkit): remove misleading code from buildkit-ref
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.

We also better document the script, and follow some best practices for
both POSIX sh and Bash.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:40 -06:00
Bjorn Neergaard
48ff8a95cc hack/with-go-mod.sh: warn on stderr
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:39 -06:00
Sebastiaan van Stijn
8e51b8b59c Merge pull request #46008 from thaJeztah/c8d_nits
c8d: some minor cleanups
2023-07-18 15:52:04 +02:00
Sebastiaan van Stijn
058a6e94a1 Merge pull request #45994 from jedevc/buildkit-ref-replace
ci: extract buildkit version correctly with replace-d modules
2023-07-18 15:22:38 +02:00
Sebastiaan van Stijn
750c441dfd daemon: remove intermediate var
The imgSvcConfig is defined locally, and discarded if an error occurs,
so no need to use the intermediate vars here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 14:25:41 +02:00
Sebastiaan van Stijn
468d6616bf daemon: configureLocalContentStore: return concrete types
The interface is defined on the receiver-side, and returning concrete
types makes it more transparent what we're creating.

As these namespaced wrappers were not exported, let's inline them, so
that it's clear at a glance what it's doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 14:18:42 +02:00
Sebastiaan van Stijn
375c4eb31c daemon: rename vars that shadowed package-level types
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 14:06:33 +02:00
Sebastiaan van Stijn
034e7e153f daemon: rename containerdCli to containerdClient
The containerdCli was somewhat confusing (is it the CLI?); let's rename
to make it match what it is :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 13:57:27 +02:00
Bjorn Neergaard
0c6b616656 Merge pull request #45946 from thaJeztah/plugin_refactor_setupRemotePluginServer
pkg/plugins: don't share plugin server between tests and t.Parallel()
2023-07-17 17:51:04 -06:00
Sebastiaan van Stijn
79a198bc08 Merge pull request #46000 from thaJeztah/notestyourself
quota: remove gotest.tools from testhelpers
2023-07-18 00:29:48 +02:00
Sebastiaan van Stijn
7799f23627 Merge pull request #45979 from thaJeztah/hijack_share_request_builder
client: Client.postHijacked: use Client.buildRequest
2023-07-17 23:50:34 +02:00
Sebastiaan van Stijn
1aa17222e7 quota: remove gotest.tools from testhelpers
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)

The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.

This patch removes the gotest.tools code from this file.

Before this patch:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")
      -update
            update golden values

With this patch applied:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 23:05:09 +02:00
Akihiro Suda
29eb0be134 Merge pull request #45993 from thaJeztah/update_fsutils
vendor: github.com/containerd/continuity v0.4.1, tonistiigi/fsutil v0.0.0-20230629203738-36ef4d8c0dbb
2023-07-18 04:12:07 +09:00
Bjorn Neergaard
5491f9a9a9 Merge pull request #45997 from neersighted/build_tweaks
Build system grab-bag
2023-07-17 11:16:59 -06:00
Bjorn Neergaard
a972dbd682 hack/make/.binary: use with-go-mod.sh
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:38:12 -06:00
Bjorn Neergaard
5563b09ac2 hack: use Git-free ROOTDIR convention
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:38:10 -06:00
Bjorn Neergaard
780e8b2332 hack/d/cli.sh: properly handle errors in curl
Add `-f` to output nothing to tar if the curl fails, and `-S` to report
errors if they happen.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:19 -06:00
Bjorn Neergaard
12a19dcd84 Dockerfile: improve CLI/rootlesskit caching
Use bind-mounts instead of a `COPY` for cli.sh, and use `COPY --link`
for rootlesskit's build stage.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:19 -06:00
Bjorn Neergaard
235cd6c6b2 Dockerfile(.simple): align APT_MIRROR support
Use a non-slash escape sequence to support mirrors with a path
component, and do not unconditionally replace the mirror in
Dockerfile.simple.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:18 -06:00
Bjorn Neergaard
bcea83ab9b Makefile: pass through APT_MIRROR
This aligns `docker build` as invoked by the Makefile with both `docker
buildx bake` as invoked by the Makefile and directly by the user.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:14 -06:00
Bjorn Neergaard
37dcdac218 Merge pull request #40751 from cpuguy83/tmpfs_for_main_test_daemon
Put integration daemon socket in /run
2023-07-17 10:17:54 -06:00
Sebastiaan van Stijn
ba513805d0 Merge pull request #45983 from thaJeztah/libnetwork_bridge_error
libnetwork/drivers/bridge: setupBridgeNetFiltering: improve error handling
2023-07-17 16:23:49 +02:00
Justin Chadwell
f8c0d92a22 ci: extract buildkit version correctly with replace-d modules
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 14:17:49 +01:00
Bjorn Neergaard
75d7e053dc Merge pull request #45984 from thaJeztah/libnetwork_cleanup_options
libnetwork/options: remove unused NewGeneric, and use map[string]any
2023-07-17 07:05:11 -06:00
Sebastiaan van Stijn
010c3718e3 vendor: github.com/tonistiigi/fsutil v0.0.0-20230629203738-36ef4d8c0dbb
- Fix copy on windows plus tests
- Fix follow symlinkResolver on Windows
- Implement proper renameFile on Windows
- Fix potential nil pointer dereference
- Use RunWithPrivileges
- Fix leaking file handle
- handle mkdir race for diskwriter
- walk: avoid stat()'ing files unnecessarily
- ci: fix freebsd workflow
- update to Go 1.20

full diff: fb433841cb...36ef4d8c0d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 14:38:11 +02:00
Sebastiaan van Stijn
531dbd7af5 vendor: github.com/containerd/continuity v0.4.1
full diff: https://github.com/containerd/continuity/compare/v0.3.0...v0.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 14:37:20 +02:00
Bjorn Neergaard
3f8ca3553d Merge pull request #45986 from thaJeztah/libnetwork_cleanup_config
libnetwork/config: add Config.DriverConfig() and un-export DriverCfg
2023-07-17 06:22:24 -06:00
Justin Chadwell
eb68c5e747 daemon: support using CIDR notation for getting registry hosts
insecure-registries supports using CIDR notation, however, buildkit in
moby was not respecting these. We can update the RegistryHosts function
to support this by inserting the correct host into the lookup map if
it's explicitly marked as insecure.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 12:34:54 +01:00
Justin Chadwell
e0b065cc33 daemon: use the registry service config for getting registry hosts
The RegistryHosts lookup function is used by both BuildKit and by the
containerd snapshotter. However, this function differs in behaviour from
the config parser for the RegistryConfig:

- The protocol for insecure registries is treated as significant by
  RegistryHosts, while the RegistryConfig strips this information.
- RegistryConfig validates and deduplicates mirrors.
- RegistryConfig does not parse the insecure-registries as URLs, which
  can lead to parsing opaque URLs as was possible by the RegistryHosts
  function.

This patch updates the lookup function to ensure consistency.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 12:30:34 +01:00
Sebastiaan van Stijn
2da589d454 client: return explicit "nil" for no errors, and remove nil check
- remove some intermediate variables
- explicitly return "nil" if there's no error
- remove redundant check for response-headers being nil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 12:24:28 +02:00
Sebastiaan van Stijn
738b16d873 libnetwork/config: add Config.DriverConfig() and un-export DriverCfg
The driver-configurations are only set when creating a new controller,
using the `config.OptionDriverConfig()` option that can be passed to
`New()`, and used as "read-only" after that.

Taking away any other paths that set these options, the only type used
for per-driver options are a `map[string]interface{}`, so we can change
the type from `map[string]interface{}` to a `map[string]map[string]interface{}`,
(or its "modern" variant: `map[string]map[string]any`), so that it's
no longer needed to cast the type before use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:57:14 +02:00
Sebastiaan van Stijn
82bb3d8d2b libnetwork: TestUserChain: use assert.Check and is.ErrorContains
Don't fail early if we can still test more, and be slightly more strict
in what error we're looking for.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:53:13 +02:00
Sebastiaan van Stijn
4e709e75da libnetwork: TestUserChain: re-use IPTables instances
The test already creates instances for each ip-version, so let's
re-use them. While changing, also use assert.Check to not fail early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:53:13 +02:00
Sebastiaan van Stijn
c471255153 libnetwork: TestUserChain: don't manually manipulate Controller.cfg.DriverCfg
New() allows for driver-options to be passed using the config.OptionDriverConfig.
Update the test to not manually mutate the controller's configuration after
creating.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:53:10 +02:00
Sebastiaan van Stijn
ee79423124 Merge pull request #45985 from thaJeztah/libnetwork_remove_IsValidName
libnetwork/config: remove IsValidName utility
2023-07-17 09:33:45 +02:00
Sebastiaan van Stijn
0761240c43 Merge pull request #45988 from thaJeztah/libnetwork_drivers_clean
libnetwork, libnetwork/drivers: some minor cleanups
2023-07-17 09:32:33 +02:00
Sebastiaan van Stijn
ddd33c6bbd libnetwork/iptables: move IPTable.LoopbackByVersion() to a utility
Not critical, but when used from ChainInfo, we had to construct an IPTable
based on the version of the ChainInfo, which then only used the version
we passed to get the right loopback.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 21:53:36 +02:00
Sebastiaan van Stijn
aad2dbb93d libnetwork/iptables: GetIptable: validate provided IPversion
For backward-compatibility, continue to accept empty strings
as default (IPv4).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 21:53:29 +02:00
Sebastiaan van Stijn
47063ca3ae libnetwork/iptables: un-export IPTable.Version
We have the GetIptable "constructor". Let's make that the canonical way
to initialize.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:47:00 +02:00
Sebastiaan van Stijn
9c2cd65c0d libnetwork/iptables: make some vars local, and move bestEffortLock lock
Make some variables local to the if-branches to be slightly more iodiomatic,
and to make clear it's only used in that branch.

Move the bestEffortLock locking later in IPtable.raw(), because that function'
could return before the lock was even needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:46:59 +02:00
Sebastiaan van Stijn
93d050f504 libnetwork/iptables: NewChain, RemoveExistingChain: validate chain, table
Now that all consumers of these functions are passing non-empty values,
let's validate that no empty strings for either chain or table are passed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:46:54 +02:00
Sebastiaan van Stijn
ad0c928ab5 libnetwork/iptables: resetIptables(): don't pass empty table name
Don't depend on a default being set, but explicitly pass the table.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:37:05 +02:00
Sebastiaan van Stijn
1198fe8f6b libnetwork/iptables: un-export FirewalldInit
It's only used internally, and it was last used in commit:
0220b06cd6

But moved into the iptables package in this commit:
998f3ce22c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:37:04 +02:00
Sebastiaan van Stijn
20900b76f9 libnetwork/iptables: make xLockWaitMsg a const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:37:04 +02:00
Sebastiaan van Stijn
995da1e51d libnetwork/iptables: group "enum" consts per type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:37:04 +02:00
Sebastiaan van Stijn
c74a083672 libnetwork/iptables: IPTable.ExistChain(): remove redundant if/else
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:37:04 +02:00
Sebastiaan van Stijn
9f9d57590b libnetwork: getTestEnv(): use literals for options
Contructing these options was a bit convoluted; let's use literals
for these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:29:50 +02:00
Sebastiaan van Stijn
534858aaed libnetwork/drivers: rewrite some strings to reduce quote-escaping
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:05:36 +02:00
Sebastiaan van Stijn
7c360778bb libnetwork/drivers/bridge: driver.configure: remove redundant err-check
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:05:36 +02:00
Sebastiaan van Stijn
e21ff6c0c9 libnetwork/config: remove IsValidName utility
This utility was not used for "Config", but for Networks and Endpoints.
Having this utility made it look like more than it was, and the related
test was effectively testing stdlib.

Abstracting the validation also was hiding that, while validation does
not allow "empty" names, it happily allows leading/trailing whitespace,
and does not remove that before creating networks or endpoints;

    docker network create "bridge "
    docker network create "bridge  "
    docker network create "bridge   "
    docker network create " bridge  "
    docker network create "  bridge "
    docker network create "   bridge"

    docker network ls --filter driver=bridge
    NETWORK ID     NAME        DRIVER    SCOPE
    d4d53210f185      bridge   bridge    local
    e9afba0d99de     bridge    bridge    local
    69fb7a7ba67c    bridge     bridge    local
    a452bf065403   bridge      bridge    local
    49d96c59061d   bridge      bridge    local
    8eae1c4be12c   bridge      bridge    local
    86dd65b881b9   bridge      bridge    local

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 19:42:44 +02:00
Sebastiaan van Stijn
948f55d0b7 libnetwork/options: remove unused NewGeneric, and use map[string]any
Remove the NewGeneric utility as it was not used anywhere, except for
in tests.

Also "modernize" the type, and use `any` instead of `interface{}`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 19:39:59 +02:00
Sebastiaan van Stijn
fa5c13542c libnetwork/drivers/bridge: setupBridgeNetFiltering: improve error handling
- Use a more modern approach to check error-types
- Touch-up grammar of the error-message
- Remove redundant "nil" check for errors, as it's never nil at that point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 12:50:03 +02:00
Sebastiaan van Stijn
077049d0b9 client: WithTLSClientConfig: return early if no transport is set
tlsconfig.Client() does various things, including reading certs and
checking them. So we may as well return early if we're not gonna be
able to use the config.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 18:31:57 +02:00
Sebastiaan van Stijn
6846c63252 client: WithTLSClientConfig, WithTLSClientConfigFromEnv: inline vars
Remove intermediate variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 18:31:57 +02:00
Sebastiaan van Stijn
9d1e690751 client: TestGetAPIPath: don't use un-keyed structs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 18:31:56 +02:00
Sebastiaan van Stijn
6552441b46 client: TestClientRedirect: cleanup
- don't use un-keyed structs
- use http consts where possible
- use errors.As instead of manually checking the error-type

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 18:31:56 +02:00
Sebastiaan van Stijn
8d514f4e2e client: touch-up docs, and user doc-links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 18:31:51 +02:00
Sebastiaan van Stijn
c219b09d4a client: Client.postHijacked: use Client.buildRequest
Use Client.buildRequest instead of a local copy of the same logic so
that we're using the same logic, and there's less chance of diverging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 02:39:08 +02:00
Sebastiaan van Stijn
e11555218b client: Client.setupHijackConn: explicitly ignore errors
Just making my IDE and some linters slightly happier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 02:37:03 +02:00
Sebastiaan van Stijn
37b908aa62 Merge pull request #45965 from cpuguy83/swarm_health_start_interval
Add health start interval support to swarm mode
2023-07-15 00:36:06 +02:00
Sebastiaan van Stijn
96635e9e18 Merge pull request #45941 from thaJeztah/update_go_1.20.6
update go to go1.20.6
2023-07-14 23:59:48 +02:00
Sebastiaan van Stijn
41f235a2f8 gha: add note about buildkit using older go version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:39:33 +02:00
Sebastiaan van Stijn
1ead2dd35d update go to go1.20.6
go1.20.6 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to the compiler, cgo, the cover tool, the go command,
the runtime, and the crypto/ecdsa, go/build, go/printer, net/mail, and text/template
packages. See the Go 1.20.6 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.20.5...go1.20.6

These minor releases include 1 security fixes following the security policy:

net/http: insufficient sanitization of Host header

The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.

Thanks to Bartek Nowotarski for reporting this issue.

Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:36:46 +02:00
Sebastiaan van Stijn
dab9ffb252 Merge pull request #45942 from thaJeztah/fix_host_header
client: define a "dummy" hostname to use for local connections
2023-07-14 21:58:12 +02:00
Sebastiaan van Stijn
e1db9e9848 testutil: use dummyhost for non-tcp connections
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 18:57:56 +02:00
Sebastiaan van Stijn
6b7705d5b2 pkg/plugins: use a dummy hostname for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

Before this patch, tests would fail on go1.20.6:

    === FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
    time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
    time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
    time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
    time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
        authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 18:57:55 +02:00
Sebastiaan van Stijn
92975f0c11 client: define a "dummy" hostname to use for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.

Before this patch (using go1.20.6):

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    === RUN   TestAttachWithTTY
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithTTY (0.11s)
    === RUN   TestAttachWithoutTTy
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithoutTTy (0.02s)
    FAIL

With this patch applied:

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    INFO: Testing against a local daemon
    === RUN   TestAttachWithTTY
    --- PASS: TestAttachWithTTY (0.12s)
    === RUN   TestAttachWithoutTTy
    --- PASS: TestAttachWithoutTTy (0.02s)
    PASS

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 18:57:53 +02:00
Sebastiaan van Stijn
2a59188760 client: TestSetHostHeader: don't use un-keyed literals
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 18:56:47 +02:00
Brian Goff
f93cfb2e31 Support for health start interval to swarm mode
Adds conversions for health start interval to/from grpc for swarmkit.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-14 16:47:10 +00:00
Brian Goff
16e6d8af55 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230713153928-bc71908479e5
This brings in changes needed to support health start intervals in
swarm.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-14 16:46:55 +00:00
Cory Snider
0c2699da27 Merge pull request #45737 from pkwarren/pkw/issue-44940-dockerd-json-logs
Update dockerd to support JSON logging format
2023-07-13 19:00:31 -04:00
Sebastiaan van Stijn
79c72390b9 integration/container: use subtests for TestAttach
- Combine TestAttachWithTTY and TestAttachWithoutTTy to a single test using sub-tests
- Set up and tear-down the test-environment once
- Remove redundant client.ContainerRemove, as it's taken care of by testEnv.Clean()
- Run both tests in parallel

      make TEST_FILTER=TestAttach DOCKER_GRAPHDRIVER=overlay2 TESTDEBUG=1 test-integration
      Loaded image: busybox:latest
      Loaded image: busybox:glibc
      Loaded image: debian:bullseye-slim
      Loaded image: hello-world:latest
      Loaded image: arm32v7/hello-world:latest
      INFO: Testing against a local daemon
      === RUN   TestAttach
      === RUN   TestAttach/without_TTY
      === PAUSE TestAttach/without_TTY
      === RUN   TestAttach/with_TTY
      === PAUSE TestAttach/with_TTY
      === CONT  TestAttach/without_TTY
      === CONT  TestAttach/with_TTY
      --- PASS: TestAttach (0.00s)
          --- PASS: TestAttach/without_TTY (0.03s)
          --- PASS: TestAttach/with_TTY (0.03s)
      PASS

      DONE 3 tests in 1.347s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-13 18:24:14 +02:00
Sebastiaan van Stijn
193f16238b Merge pull request #45957 from vvoland/tests-fix-setuptest
integration: Don't env cleanup before parallel subtests
2023-07-13 15:28:06 +02:00
Bjorn Neergaard
f2c0df5cd6 Merge pull request #45949 from thaJeztah/rename_var
testutil: rename variable that collided with import
2023-07-13 07:01:47 -06:00
Sebastiaan van Stijn
81e7cd9339 Merge pull request #45955 from thaJeztah/client_table_test
client: some cleanup in request tests
2023-07-13 14:50:22 +02:00
Bjorn Neergaard
509015b498 Merge pull request #45956 from rumpl/cli-test-helper
integration-cli: Add t.Helper() to the cli test helper functions
2023-07-13 06:40:58 -06:00
Paweł Gronowski
f9e2eed55d integration: Don't env cleanup before parallel subtests
Calling function returned from setupTest (which calls testEnv.Clean) in
a defer block inside a test that spawns parallel subtests caused the
cleanup function to be called before any of the subtest did anything.

Change the defer expressions to use `t.Cleanup` instead to call it only
after all subtests have also finished.
This only changes tests which have parallel subtests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-13 13:41:00 +02:00
Djordje Lukic
cdac084fb4 Add t.Helper() to the cli test helper functions
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-13 13:37:26 +02:00
Sebastiaan van Stijn
c395496e06 testutil: rename variable that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-13 13:05:10 +02:00
Sebastiaan van Stijn
1370b3c679 client: minor test improvements for requests
- use assert.Check() where possible to not fail early
- improve checks for error-types
- rename "testURL" var to be more descriptive, and use a const

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-13 11:40:00 +02:00
Sebastiaan van Stijn
f7f0a17ea2 client: TestSetHostHeader: use sub-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-13 11:28:13 +02:00
Bjorn Neergaard
d93d3e2d75 Merge pull request #45933 from elezar/bump-cdi-dependency
Bump container-device-interface dependency to v0.6.0
2023-07-12 16:51:31 -06:00
Sebastiaan van Stijn
2cb982b506 pkg/plugins: run tests with t.Parallel()
Some tests are testing timeouts and take a long time to run. Run the tests
in parallel, so that the test-suite takes shorter to run.

Before:

    ok  github.com/docker/docker/pkg/plugins	34.013s

After:

    ok  github.com/docker/docker/pkg/plugins	17.945s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-12 22:30:46 +02:00
Sebastiaan van Stijn
b39362295a pkg/plugins: don't share plugin server between tests
Refactor setupRemotePluginServer() to be a helper, and to spin up a test-
server for each test instead of sharing the same instance between tests.

This allows the tests to be run in parallel without stepping on each-other's
toes (tearing down the server).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-12 22:30:43 +02:00
Brian Goff
1d0bb72c5b Merge pull request #45947 from crazy-max/ci-buildkit-goversion 2023-07-12 12:22:10 -07:00
Cory Snider
b1ead679f3 Merge pull request #45931 from corhere/install-compose-plugin
Dockerfile/shell: install compose cli plugin
2023-07-12 13:24:12 -04:00
CrazyMax
ee9fe2c838 ci(buildkit): match moby go version for buildkit tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-12 19:17:25 +02:00
CrazyMax
3379414c76 vendor: github.com/moby/buildkit@v0.11 (0a15675)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-12 19:17:24 +02:00
Cory Snider
5e4878ed2b Dockerfile/shell: install compose cli plugin
It's convenient to have in the dev container when debugging issues which
reproduce consistently when deploying containers through compose.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-12 12:19:32 -04:00
Bjorn Neergaard
06b991f48f Merge pull request #45944 from neersighted/moby-bin-tags
ci(bin-image): add SHA-based tags
2023-07-12 08:03:48 -06:00
Sebastiaan van Stijn
e57479dc30 Merge pull request #45888 from thaJeztah/cleanup_iptables
libnetwork/iptables: some cleanups and refactoring
2023-07-12 15:14:47 +02:00
Bjorn Neergaard
ecfa4f5866 ci(bin-image): add SHA-based tags
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-12 07:04:41 -06:00
Sebastiaan van Stijn
93e65a2a6c Merge pull request #45918 from akerouanton/hack-make-run-disable-tls
Disable tls when launching dockerd through hack/make.sh
2023-07-12 14:37:56 +02:00
Bjorn Neergaard
daa3166cb6 Merge pull request #45932 from crazy-max/fix-moby-bin2
ci(bin-image): fix meta step
2023-07-12 06:11:56 -06:00
Evan Lezar
ce2e6b5cda Bump container-device-interface dependency to v0.6.0
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-07-11 23:21:12 +02:00
Kevin Alvarez
0a126a85a4 ci(bin-image): fix meta step
We can't upload the same file in a matrix so generate
metadata in prepare job instead. Also fixes wrong bake meta
file in merge job.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-11 22:54:27 +02:00
Bjorn Neergaard
ebcb230cff Merge pull request #45923 from thaJeztah/client_header
client: remove custom "headers" type (use http.Header), and omit "version" header on API >= 1.30
2023-07-11 14:30:20 -06:00
CrazyMax
749d7449f9 ci(bin-image): fix typo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-11 22:01:55 +02:00
CrazyMax
8b7ec57cb4 Merge pull request #45930 from crazy-max/fix-moby-bin
ci(bin-image): don't set tags when pushing by digest
2023-07-11 21:31:32 +02:00
Sebastiaan van Stijn
6fee7c8f8b Merge pull request #45861 from thaJeztah/fileutils_deprecate_GetTotalUsedFds
pkg/fileutils: GetTotalUsedFds(): slight optimization for macOS
2023-07-11 20:06:24 +02:00
CrazyMax
1686540594 ci(bin-image): don't set tags when pushing by digest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-11 19:57:53 +02:00
Bjorn Neergaard
080844b9b3 Merge pull request #44740 from crazy-max/moby-bin
ci: push bin image to Docker Hub
2023-07-11 11:44:36 -06:00
Bjorn Neergaard
8c301fa933 Merge pull request #45915 from akerouanton/create-overlay-on-non-manager
libnet: Return proper error when overlay network can't be created
2023-07-11 10:13:08 -06:00
Sebastiaan van Stijn
43007108e6 Merge pull request #45917 from josegonzalez/patch-2
Sort unconsumed build arguments before usage
2023-07-11 13:25:58 +02:00
Sebastiaan van Stijn
e3f20f5088 Merge pull request #45924 from thaJeztah/registry_minor_fixes
registry: minor fixes and cleanups in search code
2023-07-11 13:19:31 +02:00
Sebastiaan van Stijn
83477ce8d0 client: remove custom "headers" type, and use "http.Header" instead
Use http.Header, which is more descriptive on intent, and we're already
importing the package in the client. Removing the "header" type also fixes
various locations where the type was shadowed by local variables named
"headers".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 13:14:28 +02:00
Sebastiaan van Stijn
e4ce676d3d client: ServiceCreate, ServiceUpdate: omit "version" header on API >= 1.30
The "version" header was added in c0afd9c873,
but used the wrong information to get the API version.

This issue was fixed in a9d20916c3, which switched
the API handler code to get the API version from the context. That change is part
of Docker Engine 20.10 (API v1.30 and up)

This patch updates the code to only set the header on APi v1.29 and older, as it's
not used by newer API versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 13:00:00 +02:00
Albin Kerouanton
d29240d9eb libnet: Return a 403 when overlay network isn't allowed
With this change, the API will now return a 403 instead of a 500 when
trying to create an overlay network on a non-manager node.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-11 12:41:24 +02:00
Albin Kerouanton
21dcbada2d libnet: Return proper error when overlay network can't be created
The commit befff0e13f inadvertendly
disabled the error returned when trying to create an overlay network on
a node which is not part of a Swarm cluster.

Since commit e3708a89cc the overlay
netdriver returns the error: `no VNI provided`.

This commit reinstate the original error message by checking if the node
is a manager before calling libnetwork's `controller.NewNetwork()`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-11 12:40:55 +02:00
Sebastiaan van Stijn
93853eca94 pkg/fileutils: GetTotalUsedFds(): slight optimization for macOS
This patch contains some optimizations I still had stashed when working
on eaa9494b71.

- Use the bytes package for handling the output of "lsof", instead of
  converting to a string.
- Count the number of newlines in the output, instead of splitting the
  output into a slice of strings. We're only interested in the number
  of lines in the output.
- Use lsof's -F option to only print the file-descriptor for each line,
  as we don't need other information.
- Use the -l, -n, and -P options to omit converting usernames, host names,
  and port numbers.

From the [LSOF(8)][1] man-page:

   -l    This option inhibits the conversion of user ID numbers to
         login names. It is also useful when login name lookup is
         working improperly or slowly.

   -n    This option inhibits the conversion of network numbers to host
         names for network files. Inhibiting conversion can make lsof run faster.
         It is also useful when host name lookup is not working properly.

   -P    This option inhibits the conversion of port numbers to port names for network files.
         Inhibiting the conversion can make lsof run a little faster.
         It is also useful when host name lookup is not working properly.

Output looks something like;

    lsof -lnP -Ff -p 39849
    p39849
    fcwd
    ftxt
    ftxt
    f0
    f1
    f2
    f3
    f4
    f5
    f6
    f7
    f8
    f9
    f10
    f11

Before/After:

    BenchmarkGetTotalUsedFds-10  122  9479384 ns/op   10816 B/op  63 allocs/op
    BenchmarkGetTotalUsedFds-10  154  7814697 ns/op    7257 B/op  60 allocs/op

[1]: https://opensource.apple.com/source/lsof/lsof-49/lsof/lsof.man.auto.html

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 10:50:31 +02:00
Sebastiaan van Stijn
c5c977855d registry: session.searchRepositories(): return typed error, and small cleanup
- return a errdefs.System if we fail to decode the registry's response
- use strconv.Itoa instead of fmt.Sprintf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 09:47:02 +02:00
Sebastiaan van Stijn
68ebfa2f18 registry: session.searchRepositories(): log actual search URL
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 09:41:52 +02:00
Sebastiaan van Stijn
27698f1d13 registry: ParseSearchIndexInfo: remove redundant error-handling
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-10 17:48:10 +02:00
Sebastiaan van Stijn
462d6ef826 Merge pull request #45901 from thaJeztah/api_move_system_info
api/types: move system info types to api/types/system
2023-07-10 17:45:13 +02:00
Sebastiaan van Stijn
e477a57cc5 Merge pull request #45920 from thaJeztah/fix_expose_npe
daemon/containerd: fix assignment to entry in nil map during commit
2023-07-10 13:53:16 +02:00
Sebastiaan van Stijn
a0e1155b28 daemon/containerd: fix assignment to entry in nil map during commit
A panic would happen when converting an config that had ports exposed, because
the ExposedPorts map in the OCI-spec was not initialized. This could happen
when committing a container, or when using the classic builder and the
parent image had ports exposed, for example

    FROM busybox AS stage0
    EXPOSE 80

    FROM stage0 AS stage1
    RUN echo hello

Example of the panic:

    2023/07/07 15:13:02 http: panic serving @: assignment to entry in nil map
    goroutine 1944 [running]:
    net/http.(*conn).serve.func1()
    	/usr/local/go/src/net/http/server.go:1854 +0xbf
    panic({0x45f660, 0xb6a8d0})
    	/usr/local/go/src/runtime/panic.go:890 +0x263
    github.com/docker/docker/daemon/containerd.containerConfigToOciImageConfig(...)
    	/go/src/github.com/docker/docker/daemon/containerd/image_import.go:397
    github.com/docker/docker/daemon/containerd.generateCommitImageConfig({0xc001470498, {0x0, 0x0}, {0xc000c437d8, 0x5}, {0x0, 0x0}, {0xc000c43b27, 0x5}, {0x0, ...}, ...}, ...)
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:138 +0x40e
    github.com/docker/docker/daemon/containerd.(*ImageService).CommitImage(0xc0008853e0, {0xb8f660, 0xc000c4f7c0}, {{0x0, 0x0}, {0x0, 0x0}, 0xc00104b900, 0xc00104b180, {0xc0011a7640, ...}, ...})
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:82 +0x73b
    github.com/docker/docker/daemon/containerd.(*ImageService).CommitBuildStep(0xc0008853e0, {0xb8f660, 0xc000c4f7c0}, {{0x0, 0x0}, {0x0, 0x0}, 0xc00104b900, 0xc00104b180, {0xc0011a7640, ...}, ...})
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:308 +0x110
    github.com/docker/docker/builder/dockerfile.(*Builder).commitContainer(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, 0xc0010b2b60, {0xc0011a7640, 0x40}, 0xc00104b180)
    	/go/src/github.com/docker/docker/builder/dockerfile/internals.go:61 +0x168
    github.com/docker/docker/builder/dockerfile.(*Builder).commit(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, 0xc0010b2b60, {0xc0012a7d80?, 0xc001340060?})
    	/go/src/github.com/docker/docker/builder/dockerfile/internals.go:45 +0x1aa
    github.com/docker/docker/builder/dockerfile.dispatchLabel({0xb8f660, 0xc000c4f7c0}, {0xc0010b2b60, 0xc000c6b628, 0xc0012b8cc0, {0xb80f60, 0xc0011a46c0}, 0xc000bc2560}, 0x1e24a85?)
    	/go/src/github.com/docker/docker/builder/dockerfile/dispatchers.go:83 +0x258
    github.com/docker/docker/builder/dockerfile.dispatch({0xb8f660, 0xc000c4f7c0}, {0xc0010b2b60, 0xc000c6b628, 0xc0012b8cc0, {0xb80f60, 0xc0011a46c0}, 0xc000bc2560}, {0xb7be40, 0xc00111cde0})
    	/go/src/github.com/docker/docker/builder/dockerfile/evaluator.go:74 +0x529
    github.com/docker/docker/builder/dockerfile.(*Builder).dispatchDockerfileWithCancellation(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, {0xc000b1d380, 0x1, 0xc0011a4660?}, {0x0, 0x0, 0x0?}, 0x5c, ...)
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:296 +0x8f2
    github.com/docker/docker/builder/dockerfile.(*Builder).build(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, {0xb80f60, 0xc0011a46c0}, 0xc0011a49f0)
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:211 +0x2e5
    github.com/docker/docker/builder/dockerfile.(*BuildManager).Build(0xc0008868c0, {0xb8f708, 0xc0011a44b0}, {{0xb789c0, 0xc0011a4540}, {{0xb6b940, 0xc000c22a50}, {0xb6c5e0, 0xc000c22a68}, {0xb6c5e0, ...}, ...}, ...})
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:98 +0x358
    github.com/docker/docker/api/server/backend/build.(*Backend).Build(0xc0007d0870, {0xb8f708, 0xc0011a44b0}, {{0xb789c0, 0xc0011a4540}, {{0xb6b940, 0xc000c22a50}, {0xb6c5e0, 0xc000c22a68}, {0xb6c5e0, ...}, ...}, ...})
    	/go/src/github.com/docker/docker/api/server/backend/build/backend.go:69 +0x186
    github.com/docker/docker/api/server/router/build.(*buildRouter).postBuild(0xc0008333c0, {0xb8f708, 0xc0011a44b0}, {0xb8e130, 0xc0000ed500}, 0xc0010d4800, 0xc0012df760?)
    	/go/src/github.com/docker/docker/api/server/router/build/build_routes.go:280 +0x7a6
    github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1({0xb8f708, 0xc0011a44b0}, {0xb8e130?, 0xc0000ed500?}, 0x36cf80?, 0xc0010ab550?)
    	/go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0x15b
    github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1({0xb8f708, 0xc0011a4480}, {0xb8e130, 0xc0000ed500}, 0xc000d787e8?, 0xc000d787a0?)
    	/go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x4d7
    github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1({0xb8f708, 0xc0011a4480}, {0xb8e130?, 0xc0000ed500?}, 0xc0010d4800, 0xc0010ab500?)
    	/go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x649
    github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1({0xb8e130, 0xc0000ed500}, 0xc0010d4700)
    	/go/src/github.com/docker/docker/api/server/server.go:53 +0x1ce
    net/http.HandlerFunc.ServeHTTP(0xc0010d4600?, {0xb8e130?, 0xc0000ed500?}, 0xc000d789e8?)
    	/usr/local/go/src/net/http/server.go:2122 +0x2f
    github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001a7e00, {0xb8e130, 0xc0000ed500}, 0xc000d37600)
    	/go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:210 +0x1cf
    net/http.serverHandler.ServeHTTP({0xb7ec58?}, {0xb8e130, 0xc0000ed500}, 0xc000d37600)
    	/usr/local/go/src/net/http/server.go:2936 +0x316
    net/http.(*conn).serve(0xc0012661b0, {0xb8f708, 0xc000fd0360})
    	/usr/local/go/src/net/http/server.go:1995 +0x612
    created by net/http.(*Server).Serve
    	/usr/local/go/src/net/http/server.go:3089 +0x5ed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-10 10:58:06 +02:00
Albin Kerouanton
6b1b71ced4 Disable tls when launching dockerd through hack/make.sh
The daemon sleeps for 15 seconds at start up when the API binds to a TCP
socket with no TLS certificate set. That's what the hack/make/run script
does, but it doesn't explicitly disable tls, thus we're experiencing
this annoying delay every time we use this script.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-09 22:26:07 +02:00
Sebastiaan van Stijn
49b55b220e Merge pull request #42138 from cpuguy83/test_daemon_logging
Improve test daemon logging
2023-07-09 10:02:43 +02:00
Jose Diaz-Gonzalez
079a9d4562 Sort unconsumed build arguments before usage
Golang map iteration order is not guaranteed, so in some cases the built slice has it's output of order as well. This means that testing for exact warning messages in docker build output would result in random test failures, making it more annoying for end-users to test against this functionality.

Signed-off-by: Jose Diaz-Gonzalez <email@josediazgonzalez.com>
2023-07-09 03:23:06 -04:00
CrazyMax
41261ea4ec ci: push bin image to Docker Hub
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-09 02:59:03 +02:00
Bjorn Neergaard
c57097bcd4 Merge pull request #45908 from corhere/libn/drop-swarmkit-cruft
libnetwork: drop cruft formerly needed by Swarmkit
2023-07-07 15:26:00 -06:00
Cory Snider
5b3acc15d1 libnetwork: drop legacy driver cruft
...that Swarmkit no longer needs now that it has been migrated to use
the new-style driver registration APIs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:02:58 -04:00
Cory Snider
f9a83daebb libnetwork: delete package bitseq
Package idm was the last remaining user of bitseq.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 14:53:44 -04:00
Cory Snider
8b167535db libnetwork: delete package idm
The only remaining user is Swarmkit, which now has its own private copy
of the package tailored to its needs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 14:52:08 -04:00
Cory Snider
7366e44019 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230707182847-6f78b8199b05
36334ed187...6f78b8199b

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 14:48:12 -04:00
Brian Goff
7d548c6c00 Merge pull request #45902 from thaJeztah/fix_volume_npe 2023-07-07 07:38:21 -07:00
Sebastiaan van Stijn
a490248f4d daemon: daemon.prepareMountPoints(): fix panic if mount is not a volume
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)

However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)

This would panic if the MountPoint was not a volume;

    github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
    github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
    created by github.com/docker/docker/daemon.(*Daemon).restore
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]

This issue was introduced in 647c2a6cdd

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 15:31:00 +02:00
Sebastiaan van Stijn
c90229ed9a api/types: move system info types to api/types/system
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 13:01:36 +02:00
Luboslav Pivarc
3aef732e61 Do not drop effective&permitted set
Currently moby drops ep sets before the entrypoint is executed.
This does mean that with combination of no-new-privileges the
file capabilities stops working with non-root containers.
This is undesired as the usability of such containers is harmed
comparing to running root containers.

This commit therefore sets the effective/permitted set in order
to allow use of file capabilities or libcap(3)/prctl(2) respectively
with combination of no-new-privileges and without respectively.

For no-new-privileges the container will be able to obtain capabilities
that are requested.

Signed-off-by: Luboslav Pivarc <lpivarc@redhat.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-06 13:23:30 -06:00
Bjorn Neergaard
98d3da79ef Merge pull request #45881 from corhere/libn/register-configless-drivers
libnetwork: stop passing config to drivers which ignore the config argument
2023-07-06 12:17:17 -06:00
Cory Snider
d519bde5da libnet/drivers: stop passing config to drivers...
...which ignore the config argument. Notably, none of the network
drivers referenced by Swarmkit use config, which is good as Swarmkit
unconditionally passes nil for the config when registering drivers.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-06 12:57:00 -04:00
Cory Snider
1980deffae libn: refactor platform driver registration
Hide knowledge of the network driver initializer functions from
controller.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-06 12:56:09 -04:00
Bjorn Neergaard
b60c02b065 Merge pull request #45887 from thaJeztah/move_mtu
daemon/config: move MTU to BridgeConfig, and warn when using on Windows
2023-07-06 09:41:06 -06:00
Sebastiaan van Stijn
a2e9ae0055 Merge pull request #45895 from thaJeztah/albin_maintainer 2023-07-06 17:14:01 +02:00
Sebastiaan van Stijn
065b00b787 Merge pull request #45894 from thaJeztah/laura_maintainer 2023-07-06 17:12:33 +02:00
Sebastiaan van Stijn
bf42ccfbec Merge pull request #45893 from thaJeztah/kevin_maintainer 2023-07-06 17:08:50 +02:00
Bjorn Neergaard
c9397ec8d4 Merge pull request #45892 from corhere/libn/overlay-cut-vni-list
libnetwork/drivers/overlay: parse VNI list option in O(1) memory
2023-07-06 08:38:19 -06:00
Sebastiaan van Stijn
c87c4b5bbf Add Albin Kerouanton (@akerouanton) as maintainer
Albin is currently a curator, has been contributing for various years prior
to that, and has taken on the daunting task to work on Moby's networking stack.

Albin would be a great addition to our list of maintainers and to allow him
to perform his work in these areas in a more official capacity.

I nominated Albin as maintainer, and votes passed, so opening a PR to
make it official.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-06 14:47:20 +02:00
Sebastiaan van Stijn
0066138379 Add Kevin Alvarez (@crazy-max) as maintainer
Kevin is a maintainer for BuildKit, Buildx, and Docker's official GitHub
actions (among others), has been our "in-house GitHub actions expert"
for a long time, and has made significant contributions to the integration
with BuildKit, and to improve our build pipeline(s).

Kevin would be a great addition to our list of maintainers and to allow him
to perform his work in these areas in a more official capacity.

I nominated Kevin as maintainer, and votes passed, so opening a PR to
make it official.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-06 14:43:27 +02:00
Sebastiaan van Stijn
d8056ddecd Add Laura Brehm (@laurazard) as maintainer
Laura has done significant work on the containerd integration, helping
triage and fixing bugs, both in this repository, containerd, and the
docker CLI, and would make a great addition to our list of maintainers.

I nominated Laura as maintainer, and votes passed, so opening a PR to
make it official.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-06 14:42:39 +02:00
Bjorn Neergaard
e4c866f387 Merge pull request #40894 from cpuguy83/health_start_interval
Add health start interval
2023-07-05 19:47:41 -06:00
Brian Goff
2216d3ca8d Add health start interval
This adds an additional interval to be used by healthchecks during the
start period.
Typically when a container is just starting you want to check if it is
ready more quickly than a typical healthcheck might run. Without this
users have to balance between running healthchecks to frequently vs
taking a very long time to mark a container as healthy for the first
time.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:44:17 +00:00
Brian Goff
2ab2503b6c Put integration daemon socket in /run
This fixes a case where on Docker For Mac if you need to bind mount the
bundles dir (e.g. to get test results back).
The unix socket does not work over oxsfs, so instead we put it in a
tmpfs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-05 23:13:08 +00:00
Brian Goff
914888cf8b Improve test daemon logging
1. On failed start tail the daemon logs
2. Exposes generic tailing functions to make test debugging simpler

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-05 22:23:25 +00:00
Cory Snider
51bcf5954f libnet/d/overlay: parse VNI list w/o allocating
Parse in O(1) memory by using strings.Cut to split the string
iteratively.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 18:12:06 -04:00
Cory Snider
96ca669fa4 libnet/d/overlay: extract VNI option parsing...
...to a shared utility function.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 18:12:04 -04:00
Bjorn Neergaard
e77f9d2327 Merge pull request #45877 from thaJeztah/imports_and_vars
fix some minor linting issues
2023-07-05 15:54:57 -06:00
Bjorn Neergaard
ab9ff592c5 Merge pull request #45882 from corhere/libn/ditch-idm
libnetwork/d/o/ovmanager: switch to using bitmap package directly
2023-07-05 15:49:20 -06:00
Sebastiaan van Stijn
a940462fc7 libnetwork/drivers/bridge: minor cleanups for iptables funcs
setupBridgeNetFiltering:
- Indicate that the bridgeInterface argument is unused (but it's needed
  to satisfy the signature).
- Return instead of nullifying the err. Still not great, but I thought it
  was very slightly more logical thing to do.

checkBridgeNetFiltering:
- Remove unused argument, and scope ipVerName to the branch where it's
  used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:27:08 +02:00
Sebastiaan van Stijn
6817b3697b libnetwork/iptables: signalHandler(): use s switch
It felt ever-so-slightly more readable than if/else if/(else if...)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:27:08 +02:00
Sebastiaan van Stijn
51bbcdb3c5 libnetwork/iptables: merge Conn.initConnection into newConnection
initConnection was effectively just part of the constructor; ot was not
used elsewhere. Merge the two functions to simplify things a bit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:27:08 +02:00
Sebastiaan van Stijn
0921360133 libnetwork/iptables: checkRunning(): use early return
Remove redundant variable declarations, and use an early return instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:27:08 +02:00
Sebastiaan van Stijn
98592608d4 libnetwork/iptables: remove unused Ebtables const
This const was added in 8301dcc6d7, before
being moved to libnetwork, and moved back, but it was never used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:27:07 +02:00
Sebastiaan van Stijn
126525c03e libnetwork/iptables: TestReloaded(): minor cleanup
- remove local bridgeName variable that shadowed the const, but
  used the same value
- remove some redundant `var` declarations, and changed fixed
  values to a const

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:27:00 +02:00
Sebastiaan van Stijn
b216669a02 libnetwork/iptables: TestPassthrough(): skip without firewalld
The test was not doing anything without firewalld running, but did
not skip either.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:46 +02:00
Sebastiaan van Stijn
753c190ef9 libnetwork/drivers/bridge: rename vars that collided with type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:46 +02:00
Sebastiaan van Stijn
d1ebe6689f libnetwork/iptables: errors should not be capitalized
None of these errors were string-matched anywhere, so let's change them
to be non-capitalized, as they should.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:46 +02:00
Sebastiaan van Stijn
afe8d3076f libnetwork/iptables: remove ErrIptablesNotFound
looks like this error was added in 1cbdaebaa1,
and later moved to libnetwork in 44c96449c2
which also updated the description to something that doesn't match what
it means.

In either case, this error was never used as a special / sentinel error,
so we can just use a regular error return.

While at it, I also lower-cased the error-message; it's not string-matched
anywhere, so we can update it to make linters more happy.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
16f80f649b libnetwork/iptables: ChainInfo: don't pass whole IPTable as value
It only needed the IPVersion, so let's pass that instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
42653787ea libnetwork/iptables: ChainInfo.Output(): explicitly suppress errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
ea4baa24b1 libnetwork/iptables: IPTable.RemoveExistingChain() slight refactor
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
04e54c6bb0 libnetwork/iptables: IPTable.exists(): return early on error
Also remove a redundant string cast for the Table value.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
829374337f libnetwork/iptables: don't use err.Error() if not needed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
9717734d1c libnetwork/iptables: IPTable.NewChain() minor cleanups
- validate input variables before constructing the ChainInfo
- only construct the ChainInfo if things were successful

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:45 +02:00
Sebastiaan van Stijn
9bb0e7a5ee libnetwork/iptables: inline some args
Just inline the args if they're not dynamically constructed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:26:44 +02:00
Cory Snider
564a13285f libnetwork/bitmap: improve documentation
Clarify that the argument to New is an exclusive upper bound.

Correct the documentation for SetAnyInRange: the end argument is
inclusive rather than exclusive.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 16:10:32 -04:00
Cory Snider
0fc6bf9a6e libn/d/o/ovmanager: assign vxlans using bitmap pkg
The idm package wraps bitseq.Handle to provide an offset and
synchronization. bitseq.Handle wraps bitmap.Bitmap to provide
persistence in a datastore. As no datastore is passed and the offset is
zero, the idm.Idm instance is nothing more than a concurrency-safe
wrapper around a bitmap.Bitmap with differently-named methods. Switch
over to using bitmap.Bitmap directly, using the ovmanager driver's mutex
for concurrency control.

Hold the driver mutex for the entire duration that VXLANs are being
assigned to the new network. This makes allocating VXLANs for a network
an atomic operation.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 16:10:30 -04:00
Cory Snider
f0127f1617 libn/d/o/ovmanager: inline obtainVxlanID method
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 16:10:12 -04:00
Cory Snider
e97492e579 libn/d/o/ovmanager: drop mutex from network type
In the network.obtainVxlanID() method, the mutex only guards a local
variable and a function argument. Locking is therefore unnecessary.

The network.releaseVxlanID() method is only called in two contexts:
driver.NetworkAllocate(), where the network struct is a local variable
and network.releaseVxlanID() is only called in failure code-paths in
which the network does not escape; and driver.NetworkFree(), while the
driver mutex is held. Locking is therefore unnecessary.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 16:10:08 -04:00
Sebastiaan van Stijn
11c6ec6eb2 Merge pull request #45891 from corhere/fix-daemon-restart-iptables-collision
integration: disable iptables in parallel tests
2023-07-05 21:48:20 +02:00
Cory Snider
6fb6635ceb libn/d/o/ovmanager: make mutexes private fields
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 15:36:03 -04:00
Cory Snider
cdcb7c28c5 integration: disable iptables in parallel tests
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.

Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 13:49:53 -04:00
Sebastiaan van Stijn
ce00af8590 Merge pull request #45889 from corhere/fix-flaky-plugins-client-test
pkg/plugins: make unit test less time sensitive
2023-07-05 19:17:10 +02:00
Cory Snider
9cee34bc94 pkg/plugins: make unit test less time sensitive
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 12:09:37 -04:00
Bjorn Neergaard
ba02bbb3b4 Merge pull request #45886 from thaJeztah/more_grepable
use string-literals for easier grep'ing
2023-07-05 07:02:14 -06:00
Bjorn Neergaard
569e1a4b77 Merge pull request #45870 from thaJeztah/libnetwork_cleanup_kvstore
libnetwork: remove dead / unused code from datastore and kvstore
2023-07-05 07:00:55 -06:00
Sebastiaan van Stijn
3721a525ce daemon: initBridgeDriver(): pass BridgeConfig, instead of daemon config
Now that the MTU field was moved, this function only needs the BridgeConfig,
which contains all options for the default "bridge" network.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 14:43:36 +02:00
Sebastiaan van Stijn
b8220f5d0d daemon/config: move MTU to BridgeConfig
This option is only used for the default bridge network; let's move the
field to that struct to make it clearer what it's used for.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 14:43:35 +02:00
Sebastiaan van Stijn
11abd0742e dockerd: "--mtu": update description, hide on Windows and warn if set
The --mtu option is only used for the default "bridge" network on Linux.
On Windows, the flag is available, but ignored. As this option has been
available for a long time, and was always silently ignored, deprecating
or removing it would be a breaking change (and perhaps it's possible to
support it in future).

This patch:

- hides the option on Windows binaries
- logs a warning if the option is set to any non-zero value other than
  the default on a Windows binary

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 14:43:27 +02:00
Sebastiaan van Stijn
6944d2dddb libnetwork: TestBoltdbBackend(): use t.TempDir()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:21 +02:00
Sebastiaan van Stijn
2fd88c7ca4 libnetwork: inline store config options
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:21 +02:00
Sebastiaan van Stijn
cda187222e libnetwork/config: remove options that were only used in tests
The OptionLocalKVProvider, OptionLocalKVProviderURL, and OptionLocalKVProviderConfig
options were only used in tests, so un-export them, and move them to the
test-files.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:21 +02:00
Sebastiaan van Stijn
bc80c5d067 libnetwork: rename vars that shadowed with pkg vars and imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:21 +02:00
Sebastiaan van Stijn
332ffe8d74 libnetwork/networkdb: NetworkDB.Watch(): remove unused "key" argument
This function was implemented in dd4950f36d
which added a "key" field, but that field was never used anywhere, and
still appears unused.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:20 +02:00
Sebastiaan van Stijn
4c4149a09c libnetwork/internal/kvstore: remove unused Delete()
All code is using the atomic alternatives (AtomicDelete)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:20 +02:00
Sebastiaan van Stijn
4d09e60f5b libnetwork/internal/kvstore: remove unused Watch() method
The BoltDB store is not Watchable, and the Watch function was never used,
so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:20 +02:00
Sebastiaan van Stijn
c14a9f5b3d libnetwork/datastore: un-export Mutex
Keep the mutex internal to the DataStore.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:20 +02:00
Sebastiaan van Stijn
e21e802fc6 libnetwork/datastore: remove unused DeleteTree() method
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:20 +02:00
Sebastiaan van Stijn
58d2f21dae libnetwork/datastore: remove unused PutObject(), DeleteObject()
all code is using the atomic alternatives for these (PutObjectAtomic,
DeleteObjectAtomic)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:20 +02:00
Sebastiaan van Stijn
a3b0181503 libnetwork/datastore: remove Watch(), Watchable(), RestartWatch()
The `store.Watch()` was only used in `Controller.processEndpointCreate()`,
and skipped if the store was not "watchable" (`store.Watchable()`).

Whether a store is watchable depends on the store's datastore.scope;
local stores are not watchable;

    func (ds *datastore) Watchable() bool {
        return ds.scope != LocalScope
    }

datastore is only initialized in two locations, and both locations set the
scope field to LocalScope:

datastore.newClient() (also called by datastore.NewDataStore()):
3e4c9d90cf/libnetwork/datastore/datastore.go (L213)

datastore.NewTestDataStore() (used in tests);
3e4c9d90cf/libnetwork/datastore/datastore_test.go (L14-L17)

Furthermore, the backing BoltDB kvstore does not implement the Watch()
method;

3e4c9d90cf/libnetwork/internal/kvstore/boltdb/boltdb.go (L464-L467)

Based on the above;

- our datastore is never Watchable()
- so datastore.Watch() is never used

This patch removes the Watchable(), Watch(), and RestartWatch() functions,
as well as the code handling watching.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
2409a36e29 libnetwork/datastore: cache.get(): remove unused "key" argument
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
824abbf8d9 libnetwork/datastore: remove redundant datastore.sequential
The sequential field determined whether a lock was needed when storing
and retrieving data. This field was always set to true, with the exception
of NewTestDataStore() in the tests.

This field was added in a18e2f9965
to make locking optional for non-local scoped stores. Such stores are no
longer used, so we can remove this field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
e9b6965079 libnetwork/datastore: remove unused datastore.Active()
The value was set, and updated, but never used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
b32e41f016 libnetwork/internal/kvstore/boltdb: un-export Mutex
Keep the mutex internal to BoltDB.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
7e7c7bbc17 libnetwork/internal/kvstore/boltdb: BoltDB.List(): minor cleanup
cleanup the code to be slightly more idiomatic

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
95b96eebdd libnetwork/internal/kvstore/boltdb: BoltDB.Get(): don't shadow error
Don't shadow the original error if we got one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:19 +02:00
Sebastiaan van Stijn
fb61b07bcf libnetwork/internal/kvstore/boltdb: BoltDB.Exists(): fix error handling
This function could potentially return "true" even if an error was returned.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:18 +02:00
Sebastiaan van Stijn
c94ccd4207 libnetwork/internal/kvstore/boltdb: minor cleanup/refactor
Make the code slightly more idiomatic; remove some "var" declarations,
remove some intermediate variables and redundant error-checks, and remove
the filePerm const.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:18 +02:00
Sebastiaan van Stijn
acfd3934a7 libnetwork/internal/kvstore: AtomicDelete(): remove unused "deleted" return
This boolean was not used anywhere, so we can remove it. Also cleaning up
the implementation a bit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:18 +02:00
Sebastiaan van Stijn
b576682bdc libnetwork/internal/kvstore: AtomicPut(): remove unused "created" return
This boolean was not used anywhere, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:18 +02:00
Sebastiaan van Stijn
c37b58bbc3 libnetwork/internal/kvstore: remove unused WriteOptions
The WriteOptions struct was only used to set the "IsDir" option. This option
was added in d635a8e32b
and was only supported by the etcd libkv store.

The BoltDB store does not support this option, making the WriteOptions
struct fully redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:18 +02:00
Sebastiaan van Stijn
e515bef423 libnetwork/internal/kvstore: remove unused WatchTree and NewLock methods
These were not used, and not implemented by the BoltDB store.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:18 +02:00
Sebastiaan van Stijn
a373983a86 libnetwork/internal/kvstore: fix some linting issues
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:17 +02:00
Sebastiaan van Stijn
05988f88b7 libnetwork/internal/kvstore: remove unused Config options
The only remaining kvstore is BoltDB, which doesn't use TLS connections
or authentication, so we can remove these options.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:30:17 +02:00
Sebastiaan van Stijn
ec11aea880 pkg/jsonmessage: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:02 +02:00
Sebastiaan van Stijn
fded42c3bd pkg/ioutils: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:02 +02:00
Sebastiaan van Stijn
1da079f211 pkg/idtools: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
84000190d3 opts: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
ddec605aef integration: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
96a1c444cc libnetwork: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
0db4a32b9c daemon/cluster: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
4e69e16fde client: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
c3d533f37f cli/debug: use string-literals for easier grep'ing
Also removed some newlines from t.Fatal() as they shouldn't be needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:00 +02:00
Sebastiaan van Stijn
02815416bb daemon: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:00 +02:00
Sebastiaan van Stijn
6331a3a346 integration-cli: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:00 +02:00
Sebastiaan van Stijn
2f61620339 builder/builder-next: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:26:51 +02:00
Sebastiaan van Stijn
202907b14c builder/dockerfile: use string-literals for easier grep'ing
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:25:59 +02:00
Sebastiaan van Stijn
4ee0cf6878 Merge pull request #45847 from thaJeztah/sysinfo_singleflight
api: use singleflight for /info endpoint
2023-07-04 13:45:49 +02:00
Sebastiaan van Stijn
a1c9a686b0 api: use singleflight for /info endpoint
Prevent potential suggestion when many concurrent requests happen on
the /info endpoint. It's worth noting that with this change,
requests to the endpoint while another request is still in flight
will share the results, hence might be slightly incorrect (for example,
the output includes SystemTime, which may now be incorrect).

Assuming that under normal circumstances, requests will still
happen fast enough to not be shared, this may not be a problem,
but we could decide to update specific fields to not be shared.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-04 12:22:32 +02:00
Sebastiaan van Stijn
20364bd658 Merge pull request #45873 from thaJeztah/systeminfo_no_ad_hoc_type
api: info: don't use ad-hoc type for compatibility with old api versions
2023-07-04 12:21:51 +02:00
Sebastiaan van Stijn
8846c7e0ae daemon/cluster/executor/container: fix mixed pointer/value receiver
Got a linter warning on this one, and I don't think eventFilter() was
intentionally using a value (not pointer).

> Struct containerConfig has methods on both value and pointer receivers.
> Such usage is not recommended by the Go Documentation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-04 00:04:24 +02:00
Sebastiaan van Stijn
99a8e53573 Merge pull request #45875 from thaJeztah/pkg_system_no_windows
pkg/system: remove windows stubs for MkDev/MkNod
2023-07-03 23:56:22 +02:00
Sebastiaan van Stijn
6dff1b8c82 Merge pull request #45874 from thaJeztah/integration_no_sys
integration: don't use pkg/system MkNod/mkDev
2023-07-03 23:53:21 +02:00
Sebastiaan van Stijn
8ea78b34ab rename some variables that shadowed imports or package types
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 23:49:57 +02:00
Sebastiaan van Stijn
9f4acceb6a remove redundant alias for libnetwork/datastore imports
These aliases were not needed, and only used in a couple of places,
which made it inconsistent, so let's use the import without aliasing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 23:47:40 +02:00
Sebastiaan van Stijn
cffe563d4d pkg/system: remove windows stubs for MkDev/MkNod
They're not used anywhere, so let's remove them; better to have
a compile error than a panic at runtime.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 18:50:30 +02:00
Sebastiaan van Stijn
6b6fed06b1 integration: don't use pkg/system MkNod/mkDev
These tests are Linux-only, so we don't need the wrapper.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 18:48:32 +02:00
Sebastiaan van Stijn
46c7319f02 Merge pull request #45858 from thaJeztah/update_cgroups
vendor: github.com/containerd/cgroups/v3 v3.0.2
2023-07-03 15:55:33 +02:00
Bjorn Neergaard
c3020d8cdb Merge pull request #45872 from thaJeztah/libnetwork_remote_fix_varnames
libnetwork/drivers/remote: rename vars that collided
2023-07-03 07:30:50 -06:00
Sebastiaan van Stijn
94c975e25a api: info: don't use ad-hoc type for compatibility with old api versions
- Add the field as a "deprecated" field in the API type.
- Don't error when failing to parse the options, but produce a warning
  instead, because the client won't be able to fix issues in the daemon
  configuration. This was unlikely to happen, as the daemon probably
  would fail to start with an invalid config, but just in case.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 13:45:12 +02:00
Sebastiaan van Stijn
9ffca1fedd libnetwork/drivers/remote: rename vars that collided
rename variables that collided with types and pre-defined funcs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 10:37:15 +02:00
Sebastiaan van Stijn
754bc52a79 Merge pull request #45868 from thaJeztah/docs_plugin_disable_force_carry
docs: api:  add missing "force" query arg on plugin disable
2023-07-02 21:57:42 +02:00
Sebastiaan van Stijn
42690b6d57 Merge pull request #45866 from thaJeztah/api_remove_deprecated_swarm
docs: api: remove outdated information from ServerVersion
2023-07-02 21:56:55 +02:00
Brian Goff
2ea6794c97 Merge pull request #45863 from thaJeztah/update_msgp
vendor: github.com/tinylib/msgp v1.1.8
2023-07-02 11:39:19 -07:00
Sebastiaan van Stijn
f6258f70cb docs: api v1.28 - v1.40: add missing "force" query arg on plugin disable
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 13:04:06 +02:00
Sebastiaan van Stijn
892e9f2c23 docs: api v1.41: add missing "force" query arg on plugin disable
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 13:03:33 +02:00
Sebastiaan van Stijn
a4bdfb963f docs: api v1.42: add missing "force" query arg on plugin disable
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 13:03:24 +02:00
Sebastiaan van Stijn
85ccb25eb8 docs: api v1.43: add missing "force" query arg on plugin disable
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 13:02:32 +02:00
Sebastiaan van Stijn
aba8e04ab1 docs: api: amend changelog for API 1.28 for "force" option
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 12:53:52 +02:00
Sebastiaan van Stijn
3e4c9d90cf Merge pull request #43897 from milas/docs-plugin-disable-force
api: swagger: add missing "force" query arg on plugin disable
2023-07-02 12:53:39 +02:00
Sebastiaan van Stijn
7055244547 docs: api v1.32 - v1.40: remove outdated information from ServerVersion
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 12:38:33 +02:00
Sebastiaan van Stijn
ed0dbb8518 docs: api v1.41: remove outdated information from ServerVersion
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 12:33:43 +02:00
Sebastiaan van Stijn
b770a50dee docs: api v1.42: remove outdated information from ServerVersion
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 12:33:35 +02:00
Sebastiaan van Stijn
18d77ff455 docs: api v1.43: remove outdated information from ServerVersion
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 12:33:20 +02:00
Sebastiaan van Stijn
92f1ddaf0a api: remove outdated information from ServerVersion
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-02 12:33:20 +02:00
Bjorn Neergaard
70f57409c3 Merge pull request #45864 from thaJeztah/api_remove_deprecated
docs: api: remove "ClusterStore" and "ClusterAdvertise" fields
2023-07-01 13:03:11 -06:00
Sebastiaan van Stijn
e8f206972a docs: api v1.42: remove "ClusterStore" and "ClusterAdvertise" fields
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece

This patch removes the fields from the swagger file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-01 16:37:51 +02:00
Sebastiaan van Stijn
e58a60902c docs: api v1.43: remove "ClusterStore" and "ClusterAdvertise" fields
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece

This patch removes the fields from the swagger file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-01 16:36:52 +02:00
Sebastiaan van Stijn
3c905d0db9 api: remove "ClusterStore" and "ClusterAdvertise" fields
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece

This patch removes the fields from the swagger file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-01 16:34:46 +02:00
Sebastiaan van Stijn
e4d809b2b0 vendor: github.com/tinylib/msgp v1.1.8
This is an indirect dependency for github.com/fluent/fluent-logger-golang,
which does not yet use a go.mod. Update the dependency to the latest patch
release, which contains some fixes, and updates for newer go versions;

full diff: https://github.com/tinylib/msgp/compare/v1.1.6...v1.1.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-01 15:29:36 +02:00
Sebastiaan van Stijn
a61494e634 Merge pull request #45839 from vvoland/c8d-dont-prune-used
c8d/prune: Fix images being deleted when they're still used with a different reference
2023-06-30 21:12:42 +02:00
Sebastiaan van Stijn
f379af6d17 vendor: github.com/containerd/cgroups/v3 v3.0.2
full diff: https://github.com/containerd/cgroups/compare/v3.0.1...v3.0.2

relevant changes:

- cgroup2: only enable the cpuset controller if cpus or mems is specified
- cgroup1 delete: proceed to the next subsystem when a cgroup is not found
- Cgroup2: Reduce allocations for manager.Stat
- Improve performance by for pid stats (cgroups1) re-using readuint
- Reduce allocs in ReadUint64 by pre-allocating byte buffer
- cgroup2: rm/simplify some code

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 19:32:26 +02:00
Bjorn Neergaard
75ee002347 Merge pull request #45854 from thaJeztah/daemon_config_remove_contextfield
daemon/config: remove unused CommonConfig.Context
2023-06-30 10:19:49 -06:00
Paweł Gronowski
e638351ef9 c8d/prune: Handle containers started from image id
If an image is only by id instead of its name, don't prune it
completely. but only untag it and create a dangling image for it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:19:03 +02:00
Paweł Gronowski
a93298d4db c8d/prune: Exclude dangling tag of the images used by containers
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:18:56 +02:00
Paweł Gronowski
a6d5db3f9b c8d/softDelete: Deep copy Labels
So we don't override the original Labels in the passed image object.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:18:50 +02:00
Paweł Gronowski
2b0655a71a c8d/softDelete: Extract ensureDanglingImage
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:18:47 +02:00
Paweł Gronowski
a96e6044cc testutil/environment: Add GetTestDanglingImageId
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:18:45 +02:00
Sebastiaan van Stijn
647ed1d205 Merge pull request #45855 from thaJeztah/remove_overlay_cfg
libnetwork/drivers: remove unused "config" parameters and fields
2023-06-30 17:30:48 +02:00
Sebastiaan van Stijn
13472c6cf1 daemon/config: remove unused CommonConfig.Context
This field was added in f0e6e135a8, and
from that change I suspect it was intended to store the default SELinux
mount-labels to be set on containers.

However, it was never used, so let's remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 16:44:53 +02:00
Sebastiaan van Stijn
b6ad25bf5e Merge pull request #45848 from thaJeztah/improve_fd_count
pkg/fileutils: GetTotalUsedFds: reduce allocations
2023-06-30 14:54:57 +02:00
Sebastiaan van Stijn
40908c5fcd libnetwork/drivers: inline capabilities options
Remove the intermediate variable, and move the option closer
to where it's used, as in some cases we created the variable,
but could return with an error before it was used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 14:36:01 +02:00
Sebastiaan van Stijn
97285711f3 libnetwork/drivers/overlay: Register does not require DriverCallback
This function was not using the DriverCallback interface, and only
required the Registerer interface.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 14:29:30 +02:00
Sebastiaan van Stijn
a718ccd0c5 libnetwork/drivers: remove unused "config" parameters and fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 14:26:32 +02:00
Sebastiaan van Stijn
d547477bc2 Merge pull request #45851 from thaJeztah/gha_fix_missing_daemonjson
gha: don't fail if no daemon.json is present
2023-06-30 11:01:59 +02:00
Sebastiaan van Stijn
264dbad43a gha: don't fail if no daemon.json is present
CI failed sometimes if no daemon.json was present:

    Run sudo rm /etc/docker/daemon.json
    sudo rm /etc/docker/daemon.json
    sudo service docker restart
    docker version
    docker info
    shell: /usr/bin/bash -e {0}
    env:
    DESTDIR: ./build
    BUILDKIT_REPO: moby/buildkit
    BUILDKIT_TEST_DISABLE_FEATURES: cache_backend_azblob,cache_backend_s3,merge_diff
    BUILDKIT_REF: 798ad6b0ce9f2fe86dfb2b0277e6770d0b545871
    rm: cannot remove '/etc/docker/daemon.json': No such file or directory
    Error: Process completed with exit code 1.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:50:42 +02:00
Sebastiaan van Stijn
ec79d0fc05 pkg/fileutils: GetTotalUsedFds(): use fast-path for Kernel 6.2 and up
Linux 6.2 and up (commit [f1f1f2569901ec5b9d425f2e91c09a0e320768f3][1])
provides a fast path for the number of open files for the process.

From the [Linux docs][2]:

> The number of open files for the process is stored in 'size' member of
> `stat()` output for /proc/<pid>/fd for fast access.

[1]: f1f1f25699
[2]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files

This patch adds a fast-path for Kernels that support this, and falls back
to the slow path if the Size fields is zero.

Comparing on a Fedora 38 (kernel 6.2.9-300.fc38.x86_64):

Before/After:

    go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
    BenchmarkGetTotalUsedFds        57264     18595 ns/op     408 B/op      10 allocs/op
    BenchmarkGetTotalUsedFds       370392      3271 ns/op      40 B/op       3 allocs/op

Note that the slow path has 1 more file-descriptor, due to the open
file-handle for /proc/<pid>/fd during the calculation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:58 +02:00
Sebastiaan van Stijn
eaa9494b71 pkg/fileutils: GetTotalUsedFds: reduce allocations
Use File.Readdirnames instead of os.ReadDir, as we're only interested in
the number of files, and results don't have to be sorted.

Before:

    BenchmarkGetTotalUsedFds-5   	  149272	      7896 ns/op	     945 B/op	      20 allocs/op

After:

    BenchmarkGetTotalUsedFds-5   	  153517	      7644 ns/op	     408 B/op	      10 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:58 +02:00
Sebastiaan van Stijn
03390be5fa pkg/fileutils: add BenchmarkGetTotalUsedFds
go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
    goos: linux
    goarch: arm64
    pkg: github.com/docker/docker/pkg/fileutils
    BenchmarkGetTotalUsedFds-5   	  149272	      7896 ns/op	     945 B/op	      20 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:57 +02:00
Sebastiaan van Stijn
252e94f499 pkg/fileutils: GetTotalUsedFds(): don't pretend to support FreeBSD
Commit 8d56108ffb moved this function from
the generic (no build-tags) fileutils.go to a unix file, adding "freebsd"
to the build-tags.

This likely was a wrong assumption (as other files had freebsd build-tags).
FreeBSD's procfs does not mention `/proc/<pid>/fd` in the manpage, and
we don't test FreeBSD in CI, so let's drop it, and make this a Linux-only
file.

While updating also dropping the import-tag, as we're planning to move
this file internal to the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:51 +02:00
Sebastiaan van Stijn
3229517b2c Merge pull request #45816 from laurazard/fix-non-default-platform-images
c8d/images: handle images without manifests for default platform
2023-06-29 20:54:06 +02:00
Sebastiaan van Stijn
b8ac06d9d1 Merge pull request #45844 from moby/maintainers_email
MAINTAINERS: update my email; AUTHORS,.mailmap: update with recent contributors
2023-06-29 18:05:26 +02:00
Bjorn Neergaard
7747327561 AUTHORS,.mailmap: update with recent contributors
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-29 08:22:14 -06:00
Bjorn Neergaard
60d2eeaa5b MAINTAINERS: update my email
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-29 08:13:41 -06:00
Brian Goff
14de8ba4ea Merge pull request #44741 from crazy-max/rm-apt-mirror 2023-06-29 07:11:41 -07:00
Bjorn Neergaard
46cf8cecc8 Merge pull request #43169 from thaJeztah/use_gofumpt
Format code with gofumpt
2023-06-29 06:23:52 -06:00
Akihiro Suda
fc7a3befda Merge pull request #45823 from thaJeztah/bump_swarmkit
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230627115642-ad0f3ae162fa
2023-06-29 08:33:29 +09:00
Sebastiaan van Stijn
a2154e1249 Merge pull request #45836 from neersighted/ctn-build-extra
docs: document more buildx targets
2023-06-29 00:38:59 +02:00
Sebastiaan van Stijn
a764cd5a40 pkg/chrootarchive: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:33:03 +02:00
Sebastiaan van Stijn
210932b3bf daemon: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:33:03 +02:00
Sebastiaan van Stijn
e236994b76 daemon/snapshotter: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:32:38 +02:00
Sebastiaan van Stijn
9213e35011 daemon/containerd: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:54 +02:00
Sebastiaan van Stijn
4b3900a257 daemon/logger: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:53 +02:00
Sebastiaan van Stijn
050de1e4f6 daemon/logger/journald: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:53 +02:00
Sebastiaan van Stijn
a083677af2 daemon/logger/local: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:53 +02:00
Sebastiaan van Stijn
74db0e9339 daemon/logger/loggertest: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:53 +02:00
Sebastiaan van Stijn
48124bac18 daemon/logger/loggerutils: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:53 +02:00
Sebastiaan van Stijn
011a32a4ad daemon/logger/gelf: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
7ed5a30b5c daemon/logger/splunk: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
bb5a1eb99d daemon/logger/jsonfilelog: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
1437037ed2 daemon/logger/awslogs: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
f930559076 pkg/stdcopy: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
1216328c2d pkg/plugins: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
651086d1d9 pkg/jsonmessage: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
1291bbfa50 pkg/capabilities: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
79af23e1ea pkg/ioutils: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
ac1514abf4 reference: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
99968eec3f restartmanager: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
f18ac2d0bc quota: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:50 +02:00
Sebastiaan van Stijn
b2663a0f73 layer: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:50 +02:00
Sebastiaan van Stijn
83b2daca6a plugin: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:50 +02:00
Sebastiaan van Stijn
169fab5146 profiles/seccomp: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:50 +02:00
Sebastiaan van Stijn
4175a550fd libcontainerd: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:50 +02:00
Sebastiaan van Stijn
091991606a opts: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:49 +02:00
Sebastiaan van Stijn
784ba59a08 registry: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:49 +02:00
Sebastiaan van Stijn
dd5ea7e996 libnetwork: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:49 +02:00
Sebastiaan van Stijn
bba21735bf libnetwork/ipamutils: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:49 +02:00
Sebastiaan van Stijn
0b75c02276 libnetwork/resolvconf: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
801cd50744 libnetwork/portallocator: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
6187ada21f libnetwork/options: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
882f7bbf1f libnetwork/osl: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
32e716e848 libnetwork/networkdb: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
65e2149b3e libnetwork/netutils: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
1cd937a867 libnetwork/etchosts: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:48 +02:00
Sebastiaan van Stijn
540e150e4e libnetwork/cmd: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:47 +02:00
Sebastiaan van Stijn
fffcbdae4c libnetwork/iptables: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:47 +02:00
Sebastiaan van Stijn
6f3fcbcfe1 libnetwork/ipam(s): format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:47 +02:00
Sebastiaan van Stijn
eb6437b4db libnetwork/datastore: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:47 +02:00
Sebastiaan van Stijn
defa8ba7b4 ibnetwork/bitmap: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:47 +02:00
Sebastiaan van Stijn
f349754b55 libnetwork/bitseq: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:47 +02:00
Sebastiaan van Stijn
3af2963c74 libnetwork/drvregistry: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:46 +02:00
Sebastiaan van Stijn
dc17f5e613 libnetwork/drivers/remote: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:46 +02:00
Sebastiaan van Stijn
485977de57 libnetwork/drivers/windows: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:46 +02:00
Sebastiaan van Stijn
2cc5c2d2e6 libnetwork/drivers/overlay: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:46 +02:00
Sebastiaan van Stijn
e74028554e libnetwork/drivers/macvlan: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:46 +02:00
Sebastiaan van Stijn
7b02ccda86 libnetwork/drivers/ipvlan: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:46 +02:00
Sebastiaan van Stijn
17a35bc645 libnetwork/drivers/bridge: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:45 +02:00
Sebastiaan van Stijn
d2a6956afb daemon/graphdriver: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:34 +02:00
Sebastiaan van Stijn
8d923c60b9 daemon/config: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Sebastiaan van Stijn
425ccb1241 runconfig: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Sebastiaan van Stijn
c0edf0861b integration/system: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Sebastiaan van Stijn
5cc1c6eca0 integration/service: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Sebastiaan van Stijn
bb5ace2efb integration/plugin: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Sebastiaan van Stijn
9bd3e011e1 integration/internal: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Sebastiaan van Stijn
89a4e6b889 integration/container: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:10 +02:00
Sebastiaan van Stijn
846bcd7dd1 distribution: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:30:52 +02:00
Sebastiaan van Stijn
b7f5db432f daemon/images: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:26:42 +02:00
Sebastiaan van Stijn
2d12dc3a58 daemon/cluster: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:26:32 +02:00
Sebastiaan van Stijn
316c7331eb pkg/parsers: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:26:03 +02:00
Sebastiaan van Stijn
fb017754e1 pkg/system: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:26:02 +02:00
Sebastiaan van Stijn
ee2ac6c205 cmd/dockerd: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:45 +02:00
Sebastiaan van Stijn
47276bdb28 cmd/docker-proxy: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:22 +02:00
Sebastiaan van Stijn
0af6d9e291 client: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
3146ecbae6 api/types: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
b9833a2059 builder: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
540d6539be builder/remotecontext: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
3e0565e17c builder/dockerfile: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
9d4abbbc30 builder/builder-next: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
6592646632 api/server: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Sebastiaan van Stijn
5e9a354ff1 pkg/tarsum: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Sebastiaan van Stijn
2aac054f52 pkg/authorization: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Sebastiaan van Stijn
f7552f1de4 pkg/archive: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:09 +02:00
CrazyMax
7c697f58f2 Makefile: remove unused BUILD_APT_MIRROR
BUILD_APT_MIRROR added in https://github.com/moby/moby/pull/26375
is not used anymore.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-29 00:11:45 +02:00
CrazyMax
a1d2132bf6 Dockerfile: use default apt mirrors
Use default apt mirrors and also check APT_MIRROR
is set before updating mirrors.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-29 00:10:27 +02:00
Sebastiaan van Stijn
225e2562c9 integration-cli: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:08:27 +02:00
Sebastiaan van Stijn
a31411c679 testutil: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:05:27 +02:00
Sebastiaan van Stijn
4c281fb29a volume: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:05:23 +02:00
Philip K. Warren
a08abec9f8 Update dockerd to support JSON logging format
Update docker to support a '--log-format' option, which accepts either
'text' (default) or 'json'. Propagate the log format to containerd as
well, to ensure that everything will be logged consistently.

Signed-off-by: Philip K. Warren <pkwarren@gmail.com>
2023-06-28 12:46:28 -05:00
Bjorn Neergaard
d2452c2102 Merge pull request #45837 from neersighted/allow_name_to_handle_at
seccomp: add name_to_handle_at to allowlist
2023-06-28 10:27:55 -06:00
Bjorn Neergaard
b335e3d305 seccomp: add name_to_handle_at to allowlist
Based on the analysis on [the previous PR][1].

  [1]: https://github.com/moby/moby/pull/45766#pullrequestreview-1493908145

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-28 05:44:48 -06:00
Laura Brehm
6d3bcd8017 c8d/images: handle images without manifests for default platform
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-06-28 10:53:43 +02:00
Bjorn Neergaard
8caf63896c docs: document more buildx targets
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-27 14:03:02 -06:00
Sebastiaan van Stijn
a78c06e3f0 Merge pull request #45754 from cpuguy83/fix_live_restore_local_vol_mounts
Restore active mount counts on live-restore
2023-06-27 21:40:09 +02:00
Bjorn Neergaard
5f48fc36e1 Merge pull request #45766 from bartier/45518-fix-disagreement
remove name_to_handle_at(2) from filtered syscalls
2023-06-27 11:33:47 -06:00
Brian Goff
647c2a6cdd Restore active mount counts on live-restore
When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-27 16:33:23 +00:00
Sebastiaan van Stijn
7f8b1cd7f9 Merge pull request #45763 from thaJeztah/dockerfile_more_resilient
Dockerfile: make cli stages more resilient against unclean termination
2023-06-27 16:08:57 +02:00
Sebastiaan van Stijn
33565f1533 Merge pull request #45828 from thaJeztah/sudo_tee
gha: Setup Runner: add missing sudo
2023-06-27 16:06:43 +02:00
Sebastiaan van Stijn
d8bc5828cd gha: Setup Runner: add missing sudo
I think this may be missing a sudo (as all other operations do use
sudo to access daemon.json);

    Run if [ ! -e /etc/docker/daemon.json ]; then
      if [ ! -e /etc/docker/daemon.json ]; then
       echo '{}' | tee /etc/docker/daemon.json >/dev/null
      fi
      DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
      sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null
      sudo service docker restart
      shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
      env:
        GO_VERSION: 1.20.5
        GOTESTLIST_VERSION: v0.3.1
        TESTSTAT_VERSION: v0.1.3
        ITG_CLI_MATRIX_SIZE: 6
        DOCKER_EXPERIMENTAL: 1
        DOCKER_GRAPHDRIVER: overlay2
    tee: /etc/docker/daemon.json: Permission denied
    Error: Process completed with exit code 1.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 14:53:54 +02:00
Vitor Anjos
fdc9b7cceb remove name_to_handle_at(2) from filtered syscalls
Signed-off-by: Vitor Anjos <bartier@users.noreply.github.com>
2023-06-27 09:49:38 -03:00
Sebastiaan van Stijn
9f6dbbc7ea Dockerfile: make cli stages more resilient against unclean termination
The Dockerfile in this repository performs many stages in parallel. If any of
those stages fails to build (which could be due to networking congestion),
other stages are also (forcibly?) terminated, which can cause an unclean
shutdown.

In some case, this can cause `git` to be terminated, leaving a `.lock` file
behind in the cache mount. Retrying the build now will fail, and the only
workaround is to clean the build-cache (which causes many stages to be
built again, potentially triggering the problem again).

     > [dockercli-integration 3/3] RUN --mount=type=cache,id=dockercli-integration-git-linux/arm64/v8,target=./.git     --mount=type=cache,target=/root/.cache/go-build,id=dockercli-integration-build-linux/arm64/v8     /download-or-build-cli.sh v17.06.2-ce https://github.com/docker/cli.git /build:
    #0 1.575 fatal: Unable to create '/go/src/github.com/docker/cli/.git/shallow.lock': File exists.
    #0 1.575
    #0 1.575 Another git process seems to be running in this repository, e.g.
    #0 1.575 an editor opened by 'git commit'. Please make sure all processes
    #0 1.575 are terminated then try again. If it still fails, a git process
    #0 1.575 may have crashed in this repository earlier:
    #0 1.575 remove the file manually to continue.

This patch:

- Updates the Dockerfile to remove `.lock` files (`shallow.lock`, `index.lock`)
  that may have been left behind from previous builds. I put this code in the
  Dockerfile itself (not the script), as the script may be used in other
  situations outside of the Dockerfile (for which we cannot guarantee no other
  git session is active).
- Adds a `docker --version` step to the stage; this is mostly to verify the
  build was successful (and to be consistent with other stages).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 14:46:21 +02:00
Bjorn Neergaard
24ad2e5db6 Merge pull request #45818 from tianon/containerd-from-scratch
Skip cache lookup for "FROM scratch" in containerd
2023-06-27 06:13:09 -06:00
Sebastiaan van Stijn
19bd0730ef vendor: github.com/moby/swarmkit/v2 v2.0.0-20230627115642-ad0f3ae162fa
no changes to vendored files

full diff: 36334ed187...ad0f3ae162

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 14:01:20 +02:00
Sebastiaan van Stijn
b1569b9c18 Merge pull request #45820 from thaJeztah/re_enable_dco
Revert "Temporarily skip DCO check"
2023-06-27 11:00:17 +02:00
Sebastiaan van Stijn
e2dad6c3ff Revert "Temporarily skip DCO check"
re-enable the DCO check, which was temporarily disabled to migrate
old commits from github.com/docker/libkv

This reverts commit 7d7225fae6.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 10:30:34 +02:00
Sebastiaan van Stijn
8d4982ed33 Merge pull request #45817 from thaJeztah/integrate_libkv_2
Integrate github.com/docker/libkv
2023-06-27 10:27:26 +02:00
Tianon Gravi
1741771b67 Skip cache lookup for "FROM scratch" in containerd
Ideally, this should actually do a lookup across images that have no parent, but I wasn't 100% sure how to accomplish that so I opted for the smaller change of having `FROM scratch` builds not be cached for now.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-26 16:02:32 -07:00
Sebastiaan van Stijn
7d7225fae6 Temporarily skip DCO check
The migrated history has some commits that missed a DCO:

These commits do not have a proper 'Signed-off-by:' marker:

 - 3fa22634a617e2c52d2c5f061826e5107e27985f
 - 9b11053e9147884c43c9a9d8ebfcd7bb9470e8b5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:52:28 +02:00
Sebastiaan van Stijn
e60cda7051 libnetwork/internal/kvstore/boltdb: fix linting issues
libnetwork/internal/kvstore/boltdb/boltdb.go:452:28: unnecessary conversion (unconvert)
                _ = bucket.Delete([]byte(key))
                                        ^
    libnetwork/internal/kvstore/boltdb/boltdb.go:425:2: S1023: redundant `return` statement (gosimple)
        return
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:52:04 +02:00
Sebastiaan van Stijn
d18b89ced6 libnetwork/internal/kvstore: remove some unused code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:51:53 +02:00
Sebastiaan van Stijn
b873d70369 replace libkv with local fork
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:51:42 +02:00
Sebastiaan van Stijn
5d25143ef3 libnetwork/kvstore: rewrite code for new location
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:49:52 +02:00
Sebastiaan van Stijn
3887475971 Integrate github.com/docker/libkv
A reduced set of the dependency, only taking the parts that are used. Taken from
upstream commit: dfacc563de

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    cd ~/projects

    # create a temporary clone of docker
    git clone https://github.com/docker/libkv.git temp_libkv
    cd temp_libkv

    # create branch to work with
    git checkout -b migrate_libkv

    # remove all code, except for the files we need; rename the remaining ones to their new target location
    git filter-repo --force \
        --path libkv.go \
        --path store/store.go \
        --path store/boltdb/boltdb.go \
        --path-rename libkv.go:libnetwork/internal/kvstore/kvstore_manage.go \
        --path-rename store/store.go:libnetwork/internal/kvstore/kvstore.go \
        --path-rename store/boltdb/:libnetwork/internal/kvstore/boltdb/

    # go to the target github.com/moby/moby repository
    cd ~/projects/docker

    # create a branch to work with
    git checkout -b integrate_libkv

    # add the temporary repository as an upstream and make sure it's up-to-date
    git remote add temp_libkv ~/projects/temp_libkv
    git fetch temp_libkv

    # merge the upstream code, rewriting "pkg/symlink" to "symlink"
    git merge --allow-unrelated-histories --signoff -S temp_libkv/migrate_libkv

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:47:08 +02:00
Bjorn Neergaard
8805e38398 Merge pull request #45799 from cpuguy83/containerd_logrus
Switch all logging to use containerd log pkg
2023-06-26 11:51:44 -06:00
Bjorn Neergaard
ddae599cab Merge pull request #45814 from thaJeztah/cleanup_homedir
pkg/homedir: use os.UserHomeDir(), deprecate GetShortcutString(), Key()
2023-06-26 08:11:45 -06:00
Paweł Gronowski
dd1c95edcd integration/windows: Unskip some kill tests
Unskip:
- TestKillWithStopSignalAndRestartPolicies
- TestKillStoppedContainer

integration tests on Windows.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-26 15:20:36 +02:00
Sebastiaan van Stijn
3c1de2e667 pkg/homedir: deprecate Key() utility
This utility was only used in tests, and internally, and no longer
used since we switch to using os.UserHomeDir() from stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:46:03 +02:00
Sebastiaan van Stijn
ddd9665289 pkg/homedir: deprecate GetShortcutString() utility
This function was last used in the pkg/mflag package, which was removed
in 14712f9ff0, and is no longer used in
libnetwork code since e6de8aec2f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:45:21 +02:00
Sebastiaan van Stijn
6876e45f9e pkg/homedir: unify implementations
Unify some of the exported functions instead of maintaining separate functions
per platform.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:43:26 +02:00
Sebastiaan van Stijn
0aca7f2a0d pkg/homedir: use os.UserHomeDir()
Use os.UserHomeDir() instead of writing our own, but keep the fallback
on Linux.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:30:42 +02:00
Sebastiaan van Stijn
db381fb21b Merge pull request #45812 from thaJeztah/no_homedir
integration-cli: don't use pkg/homedir in test
2023-06-26 14:28:41 +02:00
Sebastiaan van Stijn
0215a62d5b integration-cli: don't use pkg/homedir in test
I'm considering deprecating the "Key()" utility, as it was only
used in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 13:39:43 +02:00
Bjorn Neergaard
56261fabcb Merge pull request #45743 from thaJeztah/testutil_nits
testutil: remove unused TEST_OSTYPE env-var, and remove Execution.OSType field
2023-06-26 05:04:12 -06:00
Sebastiaan van Stijn
76d8bfdff4 testutil/environment: remove Execution.OSType field
This field was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info, and had to be manually
set using the TEST_OSTYPE env-var.

This patch removes the field in favor of the equivalent in DaemonInfo. It's
more verbose, but also less ambiguous what information we're using (i.e.,
the platform the daemon is running on, not the local platform).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:18:09 +02:00
Sebastiaan van Stijn
325786430b testutil/environment: remove unused TEST_OSTYPE env-var
This env-var was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:17:12 +02:00
Sebastiaan van Stijn
0fe2ac0437 testutil/environment: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:17:09 +02:00
Brian Goff
7dd547c5ff Merge pull request #45802 from dperny/fix-missing-csi-topology
Fix missing Topology in NodeCSIInfo
2023-06-24 08:38:40 -07:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
da05a2ef7f Merge pull request #45800 from corhere/fix-45788-restore-exit-status
daemon: fix restoring container with missing task
2023-06-23 20:21:07 +02:00
Drew Erny
cdb1293eea Fix missing Topology in NodeCSIInfo
Added code to correctly retrieve and convert the Topology from the gRPC
Swarm Node.

Signed-off-by: Drew Erny <derny@mirantis.com>
2023-06-23 11:45:50 -05:00
Cory Snider
165dfd6c3e daemon: fix restoring container with missing task
Before 4bafaa00aa, if the daemon was
killed while a container was running and the container shim is killed
before the daemon is restarted, such as if the host system is
hard-rebooted, the daemon would restore the container to the stopped
state and set the exit code to 255. The aforementioned commit introduced
a regression where the container's exit code would instead be set to 0.
Fix the regression so that the exit code is once against set to 255 on
restore.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 11:28:45 -04:00
Bjorn Neergaard
8d070e30f5 Merge pull request #45797 from corhere/fix-health-probe-double-unlock
daemon: fix double-unlock in health check probe
2023-06-22 18:17:09 -06:00
Cory Snider
5dbb5cc3e5 Merge pull request #45792 from corhere/fix-45770-processevent-nil-check
daemon: fix panic on failed exec start
2023-06-22 18:05:10 -04:00
Cory Snider
786c9adaa2 daemon: fix double-unlock in health check probe
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-22 17:48:21 -04:00
Cory Snider
3b28a24e97 daemon: fix panic on failed exec start
If an exec fails to start in such a way that containerd publishes an
exit event for it, daemon.ProcessEvent will race
daemon.ContainerExecStart in handling the failure. This race has been a
long-standing bug, which was mostly harmless until
4bafaa00aa. After that change, the daemon
would dereference a nil pointer and crash if ProcessEvent won the race.
Restore the status quo buggy behaviour by adding a check to skip the
dereference if execConfig.Process is nil.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-22 17:04:51 -04:00
Sebastiaan van Stijn
eb76c93098 Merge pull request #45767 from crazy-max/fix-host-gateway
builder: pass host-gateway IP as worker label
2023-06-22 22:56:57 +02:00
Sebastiaan van Stijn
b3843992fc Merge pull request #45781 from neersighted/c8d_stargz_refcount 2023-06-21 16:46:51 +02:00
Sebastiaan van Stijn
5ff7be32f0 Merge pull request #45784 from vvoland/busybox-5007 2023-06-21 16:29:48 +02:00
Paweł Gronowski
e010223186 contrib/busybox: Update to FRP-5007-g82accfc19
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-21 14:15:05 +02:00
CrazyMax
21e50b89c9 builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-21 13:21:38 +02:00
Sebastiaan van Stijn
404160ad87 Merge pull request #45579 from tonistiigi/otlp-traces-for-history
builder-next: enable OTLP tracing for history records
2023-06-21 01:05:49 +02:00
Sebastiaan van Stijn
ab60412cb4 Merge pull request #45736 from thaJeztah/reserve_once
daemon: registerName(): don't reserve name twice
2023-06-20 23:47:40 +02:00
Bjorn Neergaard
21c0a54a6b c8d: mark stargz as requiring reference-counted mounts
The stargz snapshotter cannot be re-mounted, so the reference-counted
path must be used.

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-20 12:59:13 -06:00
Sebastiaan van Stijn
66c497c5f1 Merge pull request #45738 from thaJeztah/dont_cancel_stop
don't cancel container stop when cancelling context
2023-06-20 17:01:53 +02:00
Akihiro Suda
3ecceafa32 Merge pull request #45776 from thaJeztah/update_buildx_0.11
Dockerfile: update buildx to v0.11.0
2023-06-20 22:49:33 +09:00
Sebastiaan van Stijn
4d831949a7 Dockerfile: update buildx to v0.11.0
Update the version of buildx we use in the dev-container to v0.11.0;
https://github.com/docker/buildx/releases/tag/v0.11.0

Full diff: https://github.com/docker/buildx/compare/v0.10.5..v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 12:34:21 +02:00
Sebastiaan van Stijn
fc94ed0a86 don't cancel container stop when cancelling context
Commit 90de570cfa passed through the request
context to daemon.ContainerStop(). As a result, cancelling the context would
cancel the "graceful" stop of the container, and would proceed with forcefully
killing the container.

This patch partially reverts the changes from 90de570cfa
and breaks the context to prevent cancelling the context from cancelling the stop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 11:53:23 +02:00
Sebastiaan van Stijn
3fae7ea16f Merge pull request #45769 from thaJeztah/fix_docker_py
testing: temporarily pin docker-py tests to use "bullseye"
2023-06-20 11:51:23 +02:00
Sebastiaan van Stijn
19d860fa9d testing: temporarily pin docker-py tests to use "bullseye"
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.

However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;

    Loaded image: emptyfs:latest
    Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
    INFO: Building docker-sdk-python3:5.0.3...
    tests/Dockerfile:6
    --------------------
       5 |     ARG APT_MIRROR
       6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
       7 | >>>     && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
       8 |
    --------------------
    ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list     && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2

This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 10:48:42 +02:00
Tonis Tiigi
cfa08f8366 builder-next: enable OTLP tracing for history records
This enables picking up OTLP tracing context for the gRPC
requests.

Also sets up the in-memory recorder that BuildKit History API
can use to store the traces associated with specific build
in a database after build completes.

This doesn't enable Jaeger tracing endpoints from env
but this can be easily enabled by adding another import if
maintainers want it.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-15 18:40:24 -07:00
Sebastiaan van Stijn
97455cc31f Merge pull request #45504 from tianon/oci-history-master
Use OCI "History" type instead of inventing our own copy
2023-06-15 15:27:43 +02:00
Sebastiaan van Stijn
d2e7a6e799 Merge pull request #45744 from thaJeztah/update_go_1.20.5
update go to go1.20.5
2023-06-14 22:19:55 +02:00
Sebastiaan van Stijn
98a44bb18e update go to go1.20.5
go1.20.5 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, the
runtime, and the crypto/rsa, net, and os packages. See the Go 1.20.5 milestone
on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.4...go1.20.5

These minor releases include 3 security fixes following the security policy:

- cmd/go: cgo code injection
  The go command may generate unexpected code at build time when using cgo. This
  may result in unexpected behavior when running a go program which uses cgo.

  This may occur when running an untrusted module which contains directories with
  newline characters in their names. Modules which are retrieved using the go command,
  i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
  GO111MODULE=off, may be affected).

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.

- runtime: unexpected behavior of setuid/setgid binaries

  The Go runtime didn't act any differently when a binary had the setuid/setgid
  bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
  I/O file descriptors closed, opening any files could result in unexpected
  content being read/written with elevated prilieges. Similarly if a setuid/setgid
  program was terminated, either via panic or signal, it could leak the contents
  of its registers.

  Thanks to Vincent Dehors from Synacktiv for reporting this issue.

  This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.

- cmd/go: improper sanitization of LDFLAGS

  The go command may execute arbitrary code at build time when using cgo. This may
  occur when running "go get" on a malicious module, or when running any other
  command which builds untrusted code. This is can by triggered by linker flags,
  specified via a "#cgo LDFLAGS" directive.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-14 12:47:05 +02:00
Sebastiaan van Stijn
a9788886e6 Merge pull request #45720 from ndeloof/copy_uidgid
fix `docker cp -a` failing to access `/` in container
2023-06-14 01:11:36 +02:00
Sebastiaan van Stijn
3ba67ee214 daemon: registerName(): don't reserve name twice
daemon.generateNewName() already reserves the generated name, but its name
did not indicate it did. The daemon.registerName() assumed that the generated
name still had to be reserved, which could mean it would try to reserve the
same name again.

This patch renames daemon.generateNewName to daemon.generateAndReserveName
to make it clearer what it does, and updates registerName() to return early
if it successfully generated (and registered) the container name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-13 13:33:33 +02:00
Tianon Gravi
2a6ff3c24f Use OCI "History" type instead of inventing our own copy
The most notable change here is that the OCI's type uses a pointer for `Created`, which we probably should've been too, so most of these changes are accounting for that (and embedding our `Equal` implementation in the one single place it was used).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-12 13:47:17 -07:00
Sebastiaan van Stijn
4bef3e9ed3 Merge pull request #45729 from thaJeztah/remove_rootlesskit_dep
pkg/rootless: remove GetRootlessKitClient, and move to daemon
2023-06-12 14:58:42 +02:00
Sebastiaan van Stijn
59b5c6075f pkg/rootless: remove GetRootlessKitClient, and move to daemon
This utility was only used in a single location (as part of `docker info`),
but the `pkg/rootless` package is imported in various locations, causing
rootlesskit to be a dependency for consumers of that package.

Move GetRootlessKitClient to the daemon code, which is the only location
it was used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-12 13:44:30 +02:00
Sebastiaan van Stijn
ed798d651a Merge pull request #45704 from corhere/fix-zeroes-in-linux-resources
daemon: stop setting container resources to zero
2023-06-12 09:44:07 +02:00
Nicolas De Loof
3cc5d62f8a run getent with a noop stdin
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-12 09:35:40 +02:00
Brian Goff
e1c92184f0 Merge pull request #44526 from crazy-max/rm-dockerfile-e2e
remove Dockerfile.e2e
2023-06-10 10:53:51 -07:00
Akihiro Suda
78cf11575d Merge pull request #45677 from thaJeztah/client_useragent
client: add WithUserAgent() option
2023-06-10 16:29:13 +09:00
Sebastiaan van Stijn
32acecbf5e Merge pull request #45721 from corhere/test-runtimeoptions-config
daemon: test runtimeoptions runtime options
2023-06-09 18:05:53 +02:00
Cory Snider
71589848a0 daemon: test runtimeoptions runtime options
For configured runtimes with a runtimeType other than
io.containerd.runc.v1, io.containerd.runc.v2 and
io.containerd.runhcs.v1, the only supported way to pass configuration is
through the generic containerd "runtimeoptions/v1".Options type. Add a
unit test case which verifies that the options set in the daemon config
are correctly unmarshaled into the daemon's in-memory runtime config,
and that the map keys for the daemon config align with the ones used
when configuring cri-containerd (PascalCase, not camelCase or
snake_case).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-09 11:16:31 -04:00
Sebastiaan van Stijn
2b402ff8b7 Merge pull request #45706 from jg-public/rootlesskit-v1.1.1
Rootlesskit v1.1.1
2023-06-09 15:17:39 +02:00
Sebastiaan van Stijn
f139017bd0 Merge pull request #44598 from cpuguy83/save_tar_oci
image save: make output tarball OCI compliant
2023-06-09 14:50:51 +02:00
Sebastiaan van Stijn
a6048fc792 client: add WithUserAgent() option
When constructing the client, and setting the User-Agent, care must be
taken to apply the header in the right location, as custom headers can
be set in the CLI configuration, and merging these custom headers should
not override the User-Agent header.

This patch adds a dedicated `WithUserAgent()` option, which stores the
user-agent separate from other headers, centralizing the merging of
other headers, so that other parts of the (CLI) code don't have to be
concerned with merging them in the right order.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 09:59:29 +02:00
Jan Garcia
197b0b16e3 vendor: github.com/sirupsen/logrus v1.9.3
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-06-08 22:24:43 +02:00
Brian Goff
1f9eb9ab07 Merge pull request #45713 from AkihiroSuda/rro-services
daemon/cluster: convert new BindOptions
2023-06-08 11:30:34 -07:00
Brian Goff
c7d4f2afdb Merge pull request #45698 from rumpl/feat-dockerd-mounts-rootfs
c8d: Use reference counting while mounting a snapshot
2023-06-08 10:46:16 -07:00
Akihiro Suda
038a361a91 daemon/cluster: convert new BindOptions
Convert CreateMountpoint, ReadOnlyNonRecursive, and ReadOnlyForceRecursive.

See moby/swarmkit PR 3134

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-06-08 10:17:04 +09:00
Akihiro Suda
d41d2c3751 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230607145746-36334ed1876
01bb7a4139...36334ed187

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-06-08 10:14:24 +09:00
Djordje Lukic
32d58144fd c8d: Use reference counting while mounting a snapshot
Some snapshotters (like overlayfs or zfs) can't mount the same
directories twice. For example if the same directroy is used as an upper
directory in two mounts the kernel will output this warning:

    overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.

And indeed accessing the files from both mounts will result in an "No
such file or directory" error.

This change introduces reference counts for the mounts, if a directory
is already mounted the mount interface will only increment the mount
counter and return the mount target effectively making sure that the
filesystem doesn't end up in an undefined behavior.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-06-07 15:50:01 +02:00
Cory Snider
8a094fe609 daemon: ensure OCI options play nicely together
Audit the OCI spec options used for Linux containers to ensure they are
less order-dependent. Ensure they don't assume that any pointer fields
are non-nil and that they don't unintentionally clobber mutations to the
spec applied by other options.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-06 13:10:05 -04:00
Cory Snider
dea870f4ea daemon: stop setting container resources to zero
Many of the fields in LinuxResources struct are pointers to scalars for
some reason, presumably to differentiate between set-to-zero and unset
when unmarshaling from JSON, despite zero being outside the acceptable
range for the corresponding kernel tunables. When creating the OCI spec
for a container, the daemon sets the container's OCI spec CPUShares and
BlkioWeight parameters to zero when the corresponding Docker container
configuration values are zero, signifying unset, despite the minimum
acceptable value for CPUShares being two, and BlkioWeight ten. This has
gone unnoticed as runC does not distingiush set-to-zero from unset as it
also uses zero internally to represent unset for those fields. However,
kata-containers v3.2.0-alpha.3 tries to apply the explicit-zero resource
parameters to the container, exactly as instructed, and fails loudly.
The OCI runtime-spec is silent on how the runtime should handle the case
when those parameters are explicitly set to out-of-range values and
kata's behaviour is not unreasonable, so the daemon must therefore be in
the wrong.

Translate unset values in the Docker container's resources HostConfig to
omit the corresponding fields in the container's OCI spec when starting
and updating a container in order to maximize compatibility with
runtimes.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-06 12:13:05 -04:00
Jan Garcia
8c4dfc9e6a vendor: github.com/rootless-containers/rootlesskit v1.1.1
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-06-06 09:26:32 +02:00
Jan Garcia
0b1c1877c5 update RootlessKit to v1.1.1
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-06-06 09:26:27 +02:00
Cory Snider
9ff169ccf4 daemon: modernize oci_linux_test.go
Switch to using t.TempDir() instead of rolling our own.

Clean up mounts leaked by the tests as otherwise the tests fail due to
the leaked mounts because unlike the old cleanup code, t.TempDir()
cleanup does not ignore errors from os.RemoveAll.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-05 18:30:30 -04:00
Bjorn Neergaard
1aae94074e Merge pull request #45696 from neersighted/check_config_bpf_2
contrib/check-config: move xt_bpf check to overlay section
2023-06-05 14:03:03 -06:00
Sebastiaan van Stijn
ccfc2e5d8d Merge pull request #45697 from vvoland/dockerfile-separate-cli-2
Dockerfile: Separate cli follow-up
2023-06-05 18:23:09 +02:00
Paweł Gronowski
0f9c8e684a Dockerfile: Move dockercli to base-dev
Avoids invalidation of dev-systemd-true and dev-base when changing the
CLI version/repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-05 17:02:48 +02:00
Paweł Gronowski
5433b88e2d hack/cli.sh: Quiet origin cleanup
Don't show `error: No such remote: 'origin'` error when building for the
first time and the cached git repository doesn't a remote yet.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-05 17:02:46 +02:00
Bjorn Neergaard
9aecb906fb Merge pull request #45358 from vvoland/dockerfile-separate-cli
Dockerfile: Use separate cli for shell and integration-cli
2023-06-05 08:17:34 -06:00
Bjorn Neergaard
800ea039ec contrib/check-config: move xt_bpf check to overlay section
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-05 08:11:05 -06:00
Paweł Gronowski
49f76a34b5 Dockerfile/shell: Install buildx cli plugin
Installs the buildx cli plugin in the container shell by default.
Previously user had to manually download the buildx binary to use
buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-05 15:25:23 +02:00
Paweł Gronowski
17c99f7164 Dockerfile: Use separate cli for shell and integration-cli
Use separate cli for integration-cli to allow use newer CLI for
interactive dev shell usage.

Both versions can be overriden with DOCKERCLI_VERSION or
DOCKERCLI_INTEGRATION_VERSION. Binary is downloaded from
download.docker.com if it's available, otherwise it's built from the
source.

For backwards compatibility DOCKER_CLI_PATH overrides BOTH clis.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-05 15:25:21 +02:00
Sebastiaan van Stijn
717cd0aaa8 Merge pull request #45694 from neersighted/check-config_xt_bpf
contrib/check-config: check for xt_bpf
2023-06-03 23:56:25 +02:00
Sebastiaan van Stijn
1916c996d6 Merge pull request #45664 from thaJeztah/bump_swarmkit
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230531205928-01bb7a41396b
2023-06-03 18:05:03 +02:00
Bjorn Neergaard
1910fdde81 contrib/check-config: check for xt_bpf
We omit xt_u32 as it's optional; since we will remove support for this
module in the future, it's simpler to check for xt_bpf, which will
become the new baseline.

Related issues:
* https://github.com/microsoft/WSL/issues/10029#issuecomment-1574440255
* https://github.com/docker/for-win/issues/13450#issuecomment-1574443139

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-02 19:31:13 -06:00
Sebastiaan van Stijn
4d34c1cd68 Merge pull request #45690 from vvoland/builder-use-moby-exporter-by-default
builder-next: Set moby exporter as default
2023-06-02 17:04:28 +02:00
Paweł Gronowski
d63569c73d builder-next: Set moby exporter as default
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-02 15:38:53 +02:00
Bjorn Neergaard
7bdeb1dfc1 Merge pull request #45644 from vvoland/c8d-load-unpack-attestation
c8d/load: Don't unpack pseudo images
2023-06-02 07:20:33 -06:00
Bjorn Neergaard
ff7ca32b79 Merge pull request #45676 from thaJeztah/dockerfile_more_link
Dockerfile: use COPY --link for source code as well
2023-06-02 06:45:25 -06:00
Sebastiaan van Stijn
79c7d26495 Merge pull request #45670 from thaJeztah/c8d_useragent_more_details
containerd: add c8d version and storage-driver to User-Agent
2023-06-02 11:24:02 +02:00
Paweł Gronowski
4295806736 c8d/handlers: Handle error in walkPresentChildren
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-02 10:23:03 +02:00
Sebastiaan van Stijn
ff2342154b Dockerfile: use COPY --link for source code as well
I missed the most important COPY in 637ca59375

Copying the source code into the dev-container does not depend on the parent
layers, so can use the --link option as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 22:17:09 +02:00
Cory Snider
cc51f0b3d3 Merge pull request #43980 from corhere/rcu-daemon-config
daemon: read-copy-update the daemon config
2023-06-01 21:34:14 +02:00
Cory Snider
0f6eeecac0 daemon: consolidate runtimes config validation
The daemon has made a habit of mutating the DefaultRuntime and Runtimes
values in the Config struct to merge defaults. This would be fine if it
was a part of the regular configuration loading and merging process,
as is done with other config options. The trouble is it does so in
surprising places, such as in functions with 'verify' or 'validate' in
their name. It has been necessary in order to validate that the user has
not defined a custom runtime named "runc" which would shadow the
built-in runtime of the same name. Other daemon code depends on the
runtime named "runc" always being defined in the config, but merging it
with the user config at the same time as the other defaults are merged
would trip the validation. The root of the issue is that the daemon has
used the same config values for both validating the daemon runtime
configuration as supplied by the user and for keeping track of which
runtimes have been set up by the daemon. Now that a completely separate
value is used for the latter purpose, surprising contortions are no
longer required to make the validation work as intended.

Consolidate the validation of the runtimes config and merging of the
built-in runtimes into the daemon.setupRuntimes() function. Set the
result of merging the built-in runtimes config and default default
runtime on the returned runtimes struct, without back-propagating it
onto the config.Config argument.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
Cory Snider
d222bf097c daemon: reload runtimes w/o breaking containers
The existing runtimes reload logic went to great lengths to replace the
directory containing runtime wrapper scripts as atomically as possible
within the limitations of the Linux filesystem ABI. Trouble is,
atomically swapping the wrapper scripts directory solves the wrong
problem! The runtime configuration is "locked in" when a container is
started, including the path to the runC binary. If a container is
started with a runtime which requires a daemon-managed wrapper script
and then the daemon is reloaded with a config which no longer requires
the wrapper script (i.e. some args -> no args, or the runtime is dropped
from the config), that container would become unmanageable. Any attempts
to stop, exec or otherwise perform lifecycle management operations on
the container are likely to fail due to the wrapper script no longer
existing at its original path.

Atomically swapping the wrapper scripts is also incompatible with the
read-copy-update paradigm for reloading configuration. A handler in the
daemon could retain a reference to the pre-reload configuration for an
indeterminate amount of time after the daemon configuration has been
reloaded and updated. It is possible for the daemon to attempt to start
a container using a deleted wrapper script if a request to run a
container races a reload.

Solve the problem of deleting referenced wrapper scripts by ensuring
that all wrapper scripts are *immutable* for the lifetime of the daemon
process. Any given runtime wrapper script must always exist with the
same contents, no matter how many times the daemon config is reloaded,
or what changes are made to the config. This is accomplished by using
everyone's favourite design pattern: content-addressable storage. Each
wrapper script file name is suffixed with the SHA-256 digest of its
contents to (probabilistically) guarantee immutability without needing
any concurrency control. Stale runtime wrapper scripts are only cleaned
up on the next daemon restart.

Split the derived runtimes configuration from the user-supplied
configuration to have a place to store derived state without mutating
the user-supplied configuration or exposing daemon internals in API
struct types. Hold the derived state and the user-supplied configuration
in a single struct value so that they can be updated as an atomic unit.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
Cory Snider
0b592467d9 daemon: read-copy-update the daemon config
Ensure data-race-free access to the daemon configuration without
locking by mutating a deep copy of the config and atomically storing
a pointer to the copy into the daemon-wide configStore value. Any
operations which need to read from the daemon config must capture the
configStore value only once and pass it around to guarantee a consistent
view of the config.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:24 -04:00
Cory Snider
742ac6e275 daemon: make config reloading more transactional
Config reloading has interleaved validations and other fallible
operations with mutating the live daemon configuration. The daemon
configuration could be left in a partially-reloaded state if any of the
operations returns an error. Mutating a copy of the configuration and
atomically swapping the config struct on success is not currently an
option as config values are not copyable due to the presence of
sync.Mutex fields. Introduce a two-phase commit protocol to defer any
mutations of the daemon state until after all fallible operations have
succeeded.

Reload transactions are not yet entirely hermetic. The platform
reloading logic for custom runtimes on *nix could still leave the
directory of generated runtime wrapper scripts in an indeterminate state
if an error is encountered.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:24 -04:00
Cory Snider
038449467e Update BuildKit registry config on daemon reload
Historically, daemon.RegistryHosts() has returned a docker.RegistryHosts
callback function which closes over a point-in-time snapshot of the
daemon configuration. When constructing the BuildKit builder at daemon
startup, the return value of daemon.RegistryHosts() has been used.
Therefore the BuildKit builder would use the registry configuration as
it was at daemon startup for the life of the process, even if the
registry configuration is changed and the configuration reloaded.
Provide BuildKit with a RegistryHosts callback which reflects the
live daemon configuration after reloads so that registry operations
performed by BuildKit always use the same configuration as the rest of
the daemon.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:21 -04:00
Cory Snider
982e4fb448 api/server: get features from a callback fn
Passing around a bare pointer to the map of configured features in order
to propagate to consumers changes to the configuration across reloads is
dangerous. Map operations are not atomic, so concurrently reading from
the map while it is being updated is a data race as there is no
synchronization. Use a getter function to retrieve the current features
map so the features can be retrieved race-free.

Remove the unused features argument from the build router.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:43:27 -04:00
Sebastiaan van Stijn
d099e47e00 containerd: add c8d version and storage-driver to User-Agent
With this patch, the user-agent has information about the containerd-client
version and the storage-driver that's used when using the containerd-integration;

    time="2023-06-01T11:27:07.959822887Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=53590f34-096a-4fd1-9c58-d3b8eb7e5092 service=registry version=2.8.2
    ...
    172.18.0.1 - - [01/Jun/2023:11:30:12 +0000] "HEAD /v2/multifoo/blobs/sha256:c7ec7661263e5e597156f2281d97b160b91af56fa1fd2cc045061c7adac4babd HTTP/1.1" 404 157 "" "docker/dev go/go1.20.4 git-commit/8d67d0c1a8 kernel/5.15.49-linuxkit-pr os/linux arch/arm64 containerd-client/1.6.21+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 18:21:58 +02:00
Sebastiaan van Stijn
a6da1480b5 dockerversion: DockerUserAgent(): allow custom versions to be passed
Allow additional metadata to be passed as part of the generated User-Agent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 18:21:58 +02:00
Sebastiaan van Stijn
9a1f2e6d7c dockerversion: remove insertUpstreamUserAgent()
It was not really "inserting" anything, just formatting and appending.
Simplify this by changing this in to a `getUpstreamUserAgent()` function
which returns the upstream User-Agent (if any) into a `UpstreamClient()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 18:21:58 +02:00
Sebastiaan van Stijn
ff40d2d787 dockerversion: simplify escapeStr()
Use a const for the characters to escape, instead of implementing
this as a generic escaping function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 18:21:58 +02:00
Sebastiaan van Stijn
eb9a5392bc dockerversion: add a basic unit-test
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 18:21:55 +02:00
Sebastiaan van Stijn
c679da9ae1 Merge pull request #45669 from thaJeztah/c8d_useragent
containerd: set user-agent when pushing/pulling images
2023-06-01 18:10:24 +02:00
Sebastiaan van Stijn
90e87c4753 Merge pull request #45660 from thaJeztah/dockerfile_copy_link
Dockerfile: use COPY --link to copy artifacts from build-stages
2023-06-01 16:26:01 +02:00
Sebastiaan van Stijn
66137ae429 containerd: set user-agent when pushing/pulling images
Before this, the client would report itself as containerd, and the containerd
version from the containerd go module:

    time="2023-06-01T09:43:21.907359755Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=67b89d83-eac0-4f85-b36b-b1b18e80bde1 service=registry version=2.8.2
    ...
    172.18.0.1 - - [01/Jun/2023:09:43:33 +0000] "HEAD /v2/multifoo/blobs/sha256:cb269d7c0c1ca22fb5a70342c3ed2196c57a825f94b3f0e5ce3aa8c55baee829 HTTP/1.1" 404 157 "" "containerd/1.6.21+unknown"

With this patch, the user-agent has the docker daemon information;

    time="2023-06-01T11:27:07.959822887Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=53590f34-096a-4fd1-9c58-d3b8eb7e5092 service=registry version=2.8.2
    ...
    172.18.0.1 - - [01/Jun/2023:11:27:20 +0000] "HEAD /v2/multifoo/blobs/sha256:c7ec7661263e5e597156f2281d97b160b91af56fa1fd2cc045061c7adac4babd HTTP/1.1" 404 157 "" "docker/dev go/go1.20.4 git-commit/8d67d0c1a8 kernel/5.15.49-linuxkit-pr os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 14:20:45 +02:00
Sebastiaan van Stijn
06aaf87aab vendor: github.com/moby/swarmkit/v2 v2.0.0-20230531205928-01bb7a41396b
- Fix timeouts from very long raft messages
- fix: code optimization
- update dependencies

full diff: 75e92ce14f...01bb7a4139

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-31 23:04:47 +02:00
Sebastiaan van Stijn
637ca59375 Dockerfile: use COPY --link to copy artifacts from build-stages
Build-cache for the build-stages themselves are already invalidated if the
base-images they're using is updated, and the COPY operations don't depend
on previous steps (as there's no overlap between artifacts copied).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-31 11:52:18 +02:00
Sebastiaan van Stijn
8d67d0c1a8 Merge pull request #45437 from thaJeztah/vendor_image_spec
vendor: github.com/opencontainers/image-spec v1.1.0-rc3
2023-05-31 11:12:51 +02:00
Paweł Gronowski
4d3238dc0b c8d/load: Don't unpack pseudo images
Don't unpack image manifests which are not a real images that can't be
unpacked.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-31 10:47:26 +02:00
Paweł Gronowski
b08bff8ba3 c8d/load: Use walkImageManifests
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-31 10:47:25 +02:00
Paweł Gronowski
5210f48bfc c8d/list: Use walkImageManifests
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-31 10:47:23 +02:00
Paweł Gronowski
fabc1d5bef c8d: Add walkImageManifests and ImageManifest wrapper
The default implementation of the containerd.Image interface provided by
the containerd operates on the parent index/manifest list of the image
and the platform matcher.

This isn't convenient when a specific manifest is already known and it's
redundant to search the whole index for a manifest that matches the
given platform matcher. It can also result in a different manifest
picked up than expected when multiple manifests with the same platform
are present.

This introduces a walkImageManifests which walks the provided image and
calls a handler with a ImageManifest, which is a simple wrapper that
implements containerd.Image interfaces and performs all containerd.Image
operations against a platform specific manifest instead of the root
manifest list/index.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-31 10:47:22 +02:00
Bjorn Neergaard
abc05cf335 Merge pull request #45645 from sebthom/patch-1
Update blogpost URL
2023-05-30 15:36:37 -06:00
Bjorn Neergaard
988f5ac342 Merge pull request #45647 from rumpl/fix-snapshotter-change
c8d: Fix re-pull of an image when the snapshotter is changed
2023-05-30 15:32:55 -06:00
Cory Snider
d43b398746 Merge pull request #45657 from corhere/libn/setup-resolver-with-verbose-iptables
libnetwork: fix resolver restore w/ chatty 'iptables -C'
2023-05-30 21:44:14 +02:00
Cory Snider
a25434654e Merge pull request #45654 from corhere/libn/fix-embedded-resolver-live-reload
libnetwork: fix sandbox restore
2023-05-30 21:43:46 +02:00
Cory Snider
1178319313 libn: fix resolver restore w/ chatty 'iptables -C'
Resolver.setupIPTable() checks whether it needs to flush or create the
user chains used for NATing container DNS requests by testing for the
existence of the rules which jump to said user chains. Unfortunately it
does so using the IPTable.RawCombinedOutputNative() method, which
returns a non-nil error if the iptables command returns any output even
if the command exits with a zero status code. While that is fine with
iptables-legacy as it prints no output if the rule exists, iptables-nft
v1.8.7 prints some information about the rule. Consequently,
Resolver.setupIPTable() would incorrectly think that the rule does not
exist during container restore and attempt to create it. This happened
work work by coincidence before 8f5a9a741b
because the failure to create the already-existing table would be
ignored and the new NAT rules would be inserted before the stale rules
left in the table from when the container was last started/restored. Now
that failing to create the table is treated as a fatal error, the
incompatibility with iptables-nft is no longer hidden.

Switch to using IPTable.ExistsNative() to test for the existence of the
jump rules as it correctly only checks the iptables command's exit
status without regard for whether it outputs anything.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-30 14:32:27 -04:00
Cory Snider
50eb2d2782 libnetwork: fix sandbox restore
The method to restore a network namespace takes a collection of
interfaces to restore with the options to apply. The interface names are
structured data, tuples of (SrcName, DstPrefix) but for whatever reason
are being passed into Restore() serialized to strings. A refactor,
f0be4d126d, accidentally broke the
serialization by dropping the delimiter. Rather than fix the
serialization and leave the time-bomb for someone else to trip over,
pass the interface names as structured data.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-30 12:27:59 -04:00
Cory Snider
18bf3aa442 libnetwork: log why osl sandbox restore failed
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-30 12:17:44 -04:00
Djordje Lukic
ed32f5e241 Make sure the image is unpacked for the current snapshotter
Switching snapshotter implementations would result in an error when
preparing a snapshot, check that the image is indeed unpacked for the
current snapshot before trying to prepare a snapshot.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-05-30 14:45:30 +02:00
Sebastiaan van Stijn
2cd23ffeec Merge pull request #45628 from thaJeztah/context_simplify
builder/remotecontext: remove mimeTypes struct, use consts
2023-05-30 13:40:42 +02:00
sebthom
d58df1fc6c Update blogpost URL
Signed-off-by: sebthom <sebthom@users.noreply.github.com>
2023-05-29 22:37:09 +02:00
Sebastiaan van Stijn
098b0fd1a0 Merge pull request #45627 from thaJeztah/remove_builder_streaming
builder/remotecontext: remove CachableSource, NewCachableSource
2023-05-29 19:04:32 +02:00
Sebastiaan van Stijn
44124ab6b0 builder/remotecontext: remove CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-29 16:35:42 +02:00
Sebastiaan van Stijn
3a643154be Merge pull request #44697 from crazy-max/generate-files
Update and validation of generated files
2023-05-29 16:34:54 +02:00
CrazyMax
5efe72415d remove Dockerfile.e2e
Dockerfile.e2e is not used anymore. Integration tests run
through the main Dockerfile.

Also removes the daemon OS/Arch detection script that is not
necessary anymore. It was used to select the Dockerfile based
on the arch like Dockerfile.arm64 but we don't have those
anymore. Was also used to check referenced frozen images
in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 11:03:48 +02:00
CrazyMax
fd72b134d5 update generated files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
735537d6b1 replace gogofast with gogofaster extension
gogofaster is identical as gogofast but removes XXX_unrecognized

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
1eaea43581 fix protos and "go generate" commands
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
Kevin Alvarez
7daaa00120 hack: generated files update and validation
Adds a Dockerfile and make targets to update and validate
generated files (proto, seccomp default profile)

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
f1ca793980 use tools build constraint for proto dependencies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:13:15 +02:00
Brian Goff
1e55ace875 Add oci index and layout files to exported tars
This makes the output of `docker save` fully OCI compliant.

When using the containerd image store, this code is not used. That
exporter will just use containerd's export method and should give us the
output we want for multi-arch images.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 20:35:14 +00:00
Akihiro Suda
2ebd97dec1 Merge pull request #45641 from cpuguy83/exec_npe
Fix npe in exec resize when exec errored
2023-05-28 19:44:23 +09:00
Brian Goff
487ea81316 Fix npe in exec resize when exec errored
In cases where an exec start failed the exec process will be nil even
though the channel to signal that the exec started was closed.

Ideally ExecConfig would get a nice refactor to handle this case better
(ie. it's not started so don't close that channel).
This is a minimal fix to prevent NPE. Luckilly this would only get
called by a client and only the http request goroutine gets the panic
(http lib recovers the panic).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 00:14:47 +00:00
Cory Snider
8f7bbc39a4 Merge pull request #45636 from corhere/libn/fix-encrypted-overlay-nonstandard-port
libnetwork/d/overlay: support encryption on any port
2023-05-26 22:40:56 +02:00
Cory Snider
9a692a3802 libn/d/overlay: support encryption on any port
While the VXLAN interface and the iptables rules to mark outgoing VXLAN
packets for encryption are configured to use the Swarm data path port,
the XFRM policies for actually applying the encryption are hardcoded to
match packets with destination port 4789/udp. Consequently, encrypted
overlay networks do not pass traffic when the Swarm is configured with
any other data path port: encryption is not applied to the outgoing
VXLAN packets and the destination host drops the received cleartext
packets. Use the configured data path port instead of hardcoding port
4789 in the XFRM policies.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-26 14:36:34 -04:00
Sebastiaan van Stijn
e410e27547 builder/remotecontext: remove mimeTypes struct, use consts
This struct was never modified; let's just use consts for these.

Also remove the args return from detectContentType(), as it was
not used anywhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 15:21:15 +02:00
Sebastiaan van Stijn
13fb24458c Merge pull request #45626 from thaJeztah/deprecate_builder_streaming
builder/remotecontext: deprecate CachableSource, NewCachableSource
2023-05-26 15:12:49 +02:00
Sebastiaan van Stijn
b42e367045 vendor: github.com/opencontainers/image-spec v1.1.0-rc3
full diff: https://github.com/opencontainers/image-spec/compare/3a7f492d3f1b...v1.1.0-rc3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 02:34:50 +02:00
Sebastiaan van Stijn
0db4174513 Merge pull request #45278 from AkihiroSuda/rro
Support recursively read-only (RRO) mounts
2023-05-26 02:24:43 +02:00
Sebastiaan van Stijn
37d4b0bee9 builder/remotecontext: deprecate CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 00:05:08 +02:00
Sebastiaan van Stijn
88f6a92d22 Merge pull request #45624 from corhere/libc8d/serialize-exec-starts-workaround
libcontainerd: work around exec start bug in c8d
2023-05-25 23:02:34 +02:00
Sebastiaan van Stijn
a4c54362c3 Merge pull request #45581 from thaJeztah/vendor_buildkit_0.11.7_dev
vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
2023-05-25 22:27:06 +02:00
Cory Snider
fb7ec1555c libcontainerd: work around exec start bug in c8d
It turns out that the unnecessary serialization removed in
b75246202a happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-25 16:00:29 -04:00
Sebastiaan van Stijn
79ca6630d4 vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
full diff: https://github.com/moby/buildkit/compare/v0.11.6...798ad6b0ce9f2fe86dfb2b0277e6770d0b545871

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 21:35:53 +02:00
Sebastiaan van Stijn
d5dc675d37 Merge pull request #45280 from corhere/libnet/no-overlay-accept-rule
libnetwork/drivers/overlay: stop programming INPUT ACCEPT rule
2023-05-25 21:03:32 +02:00
Sebastiaan van Stijn
10505cac52 Merge pull request #45619 from thaJeztah/update_go_runc_v1.1.0
vendor: github.com/containerd/go-runc v1.1.0
2023-05-25 20:19:05 +02:00
Akihiro Suda
5045a2de24 Support recursively read-only (RRO) mounts
`docker run -v /foo:/foo:ro` is now recursively read-only on kernel >= 5.12.

Automatically falls back to the legacy non-recursively read-only mount mode on kernel < 5.12.

Use `ro-non-recursive` to disable RRO.
Use `ro-force-recursive` or `rro` to explicitly enable RRO. (Fails on kernel < 5.12)

Fix issue 44978
Fix docker/for-linux issue 788

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-26 01:58:24 +09:00
Sebastiaan van Stijn
3512b04093 vendor: github.com/containerd/go-runc v1.1.0
full diff: https://github.com/containerd/go-runc/compare/v1.0.0...v1.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 18:56:52 +02:00
Cory Snider
1b28b0ed5a Merge pull request #45134 from elezar/add-cdi-support
Add support for CDI devices under Linux
2023-05-25 18:06:31 +02:00
Sebastiaan van Stijn
02c9f038b3 Merge pull request #45618 from vvoland/c8d-inspect-created-time
c8d/inspect: Fill `Created` time if available
2023-05-25 17:16:56 +02:00
Paweł Gronowski
b9b8b6597a c8d/inspect: Fill Created time if available
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-25 14:59:49 +02:00
Sebastiaan van Stijn
f1d5385515 Merge pull request #45609 from thaJeztah/constantly_numb
c8d: ImageService.softImageDelete: use OCI and containerd constants
2023-05-25 09:52:31 +02:00
Sebastiaan van Stijn
91b976ba46 Merge pull request #45607 from corhere/lock-in-snapshotter-setting
daemon: lock in snapshotter setting at daemon init
2023-05-25 01:47:19 +02:00
Sebastiaan van Stijn
f17c9e4aeb c8d: ImageService.softImageDelete: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 01:44:36 +02:00
Sebastiaan van Stijn
df5deab20b c8d: ImageService.softImageDelete: use OCI and containerd constants
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 01:44:31 +02:00
Cory Snider
9b9c5242eb daemon: lock in snapshotter setting at daemon init
Feature flags are one of the configuration items which can be reloaded
without restarting the daemon. Whether the daemon uses the containerd
snapshotter service or the legacy graph drivers is controlled by a
feature flag. However, much of the code which checks the snapshotter
feature flag assumes that the flag cannot change at runtime. Make it so
that the snapshotter setting can only be changed by restarting the
daemon, even if the flag state changes after a live configuration
reload.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-24 16:56:17 -04:00
Sebastiaan van Stijn
7e01865fb6 Merge pull request #45592 from vvoland/dangling-image-repotagsdigests-test
integration: Add TestImageInspectEmptyTagsAndDigests
2023-05-24 19:18:56 +02:00
Sebastiaan van Stijn
1e3cdd3512 Merge pull request #45603 from thaJeztah/windows_notgo
hack/make/.binary: don't use "netgo" when building Windows binaries
2023-05-24 18:41:38 +02:00
Sebastiaan van Stijn
53d1b12bc0 hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 09:16:41 -06:00
Sebastiaan van Stijn
c5126d1435 Merge pull request #45601 from vvoland/c8d-exists
c8d/pull: Use same progress action as distribution
2023-05-24 12:48:12 +02:00
Paweł Gronowski
a7bc65fbd8 c8d/pull: Use same progress action as distribution
Docker with containerd integration emits "Exists" progress action when a
layer of the currently pulled image already exists. This is different
from the non-c8d Docker which emits "Already exists".

This makes both implementations consistent by emitting backwards
compatible "Already exists" action.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-24 11:16:57 +02:00
Akihiro Suda
1d68544fbf Merge pull request #45596 from thaJeztah/bump_zfs
vendor: github.com/mistifyio/go-zfs/v3 v3.0.1
2023-05-24 16:10:12 +09:00
Sebastiaan van Stijn
cf4df9d8ae Merge pull request #45597 from thaJeztah/skip_criu
Dockerfile: temporarily skip CRIU stage
2023-05-23 17:53:06 +02:00
Paweł Gronowski
6506579e18 integration: Add TestImageInspectEmptyTagsAndDigests
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:36 +02:00
Paweł Gronowski
a93aadc2e6 hack: Rename .ensure-emptyfs to .build-empty-images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:35 +02:00
Paweł Gronowski
3a31f81838 hack/ensure-emptyfs: Create dangling image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:33 +02:00
Sebastiaan van Stijn
4373547857 Add testenv.UsingSnapshotter utility
To allow skipping integration tests that don't apply to the
containerd snapshotter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:30 +02:00
Sebastiaan van Stijn
d3d2823edf Dockerfile: temporarily skip CRIU stage
The package repository currently has issues;

    => ERROR https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/Release.key

The only test currently using this binary is currently skipped, as the test
was broken;
6e98a7f2c9/integration/container/checkpoint_test.go (L32-L33)

So let's disable this stage for the time being.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-23 12:58:22 +02:00
Sebastiaan van Stijn
86108812b8 vendor: github.com/mistifyio/go-zfs/v3 v3.0.1
Switching to the v3 version, which was renamed to be compatible with
go modules.

Full diff: https://github.com/mistifyio/go-zfs/compare/f784269be439...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-23 12:52:16 +02:00
Brian Goff
ddd67b2535 Make image save more like the OCI layout for blobs
This moves the blobs around so they follow the OCI spec.
Note that because docker reads paths from the manifest.json inside the
tar this is not a breaking change.

This does, however, remove the old layer "VERSION" file which had a big
"why is this even here" in the code comments. I suspect it does not
matter at all even for really old versions of Docker. In any case it is
a useless file for any even relatively modern version of Docker.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-22 22:29:29 +00:00
Sebastiaan van Stijn
6e98a7f2c9 Merge pull request #45548 from tianon/sysvinit-debian
Remove Upstart and cgroups bits from Debian sysvinit script
2023-05-22 16:08:55 +02:00
Bjorn Neergaard
ecbd126d6a Merge pull request #45586 from corhere/fix-flaky-resolver-test
libnetwork/osl: restore the right thread's netns
2023-05-19 20:45:38 -06:00
Sebastiaan van Stijn
7796891381 Merge pull request #45475 from thaJeztah/remove_old_buildtags 2023-05-20 02:10:19 +02:00
Cory Snider
871cf72363 libnetwork: check for netns leaks from prior tests
TestProxyNXDOMAIN has proven to be susceptible to failing as a
consequence of unlocked threads being set to the wrong network
namespace. As the failure mode looks a lot like a bug in the test
itself, it seems prudent to add a check for mismatched namespaces to the
test so we will know for next time that the root cause lies elsewhere.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 19:36:18 -04:00
Sebastiaan van Stijn
49f8e4f1e3 Merge pull request #45574 from thaJeztah/remove_driverwarnings 2023-05-20 01:04:32 +02:00
Cory Snider
6d79864135 libnetwork/osl: restore the right thread's netns
osl.setIPv6 mistakenly captured the calling goroutine's thread's network
namespace instead of the network namespace of the thread getting its
namespace temporarily changed. As this function appears to only be
called from contexts in the process's initial network namespace, this
mistake would be of little consequence at runtime. The libnetwork unit
tests, on the other hand, unshare network namespaces so as not to
interfere with each other or the host's network namespace. But due to
this bug, the isolation backfires and the network namespace of
goroutines used by a test which are expected to be in the initial
network namespace can randomly become the isolated network namespace of
some other test. Symptoms include a loopback network server running in
one goroutine being inexplicably and randomly being unreachable by a
client in another goroutine.

Capture the original network namespace of the thread from the thread to
be tampered with, after locking the goroutine to the thread.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 18:35:59 -04:00
Cory Snider
d4f3858a40 libnetwork: leave global logger alone in tests
Swapping out the global logger on the fly is causing tests to flake out
by logging to a test's log output after the test function has returned.
Refactor Resolver to use a dependency-injected logger and the resolver
unit tests to inject a private logger instance into the Resolver under
test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 18:35:58 -04:00
Cory Snider
0cc6e445d7 libnetwork: make resolver tests less confusing
tstwriter mocks the server-side connection between the resolver and the
container, not the resolver and the external DNS server, so returning
the external DNS server's address as w.LocalAddr() is technically
incorrect and misleading. Only the protocols need to match as the
resolver uses the client's choice of protocol to determine which
protocol to use when forwarding the query to the external DNS server.
While this change has no material impact on the tests, it makes the
tests slightly more comprehensible for the next person.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 18:35:58 -04:00
Sebastiaan van Stijn
411a9e1b86 daemon: remove devicemapper driver-warnings
commit dc11d2a2d8 removed the devicemapper
storage-driver, so these warnings are no longer relevant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:45:31 +02:00
Sebastiaan van Stijn
424a1c5d21 daemon: remove warning for overlay/overlay2 without d_type
commit 0abb8dec3f removed support for
running overlay/overlay2 on top of a backing filesystem without d_type
support, and  turned it into a fatal error when starting the daemon,
so there's no need to generate warnings for this situation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:43:51 +02:00
Sebastiaan van Stijn
ab35df454d remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Bjorn Neergaard
87f8be8110 Merge pull request #45587 from vvoland/c8d-layerless
c8d/list: Show layerless images
2023-05-19 11:17:58 -06:00
Paweł Gronowski
34964c2454 c8d/list: Show layerless images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-19 18:12:14 +02:00
Bjorn Neergaard
cb14534d78 Merge pull request #45584 from corhere/fix-vfs-xattrs-unit-test
d/graphdriver/vfs: fix unit test
2023-05-19 14:50:58 +01:00
Akihiro Suda
2d3a1418c2 Merge pull request #45577 from thaJeztah/update_cobra
vendor: github.com/spf13/cobra v1.7.0
2023-05-19 21:55:51 +09:00
Cory Snider
61757032da d/graphdriver/vfs: fix unit test
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 08:26:15 -04:00
Bjorn Neergaard
1545693255 Merge pull request #45464 from corhere/xattrs-matter-in-images
Fail unpacking images with xattrs to filesystems without xattr support
2023-05-19 13:21:09 +01:00
Sebastiaan van Stijn
6e4bd3bbc2 vendor: github.com/spf13/cobra v1.7.0
release notes: https://github.com/spf13/cobra/releases/tag/v1.7.0

Features

- Allow to preserve ordering of completions in bash, zsh, pwsh, & fish
- Add support for PowerShell 7.2+ in completions
- Allow sourcing zsh completion script

Bug fixes

- Don't remove flag values that match sub-command name
- Fix powershell completions not returning single word
- Remove masked template import variable name
- Correctly detect completions with dash in argument

Testing & CI/CD

- Deprecate Go 1.15 in CI
- Deprecate Go 1.16 in CI
- Add testing for Go 1.20 in CI
- Add tests to illustrate unknown flag bug

Maintenance

- Update main image to better handle dark backgrounds
- Fix stale.yaml mispellings
- Remove stale bot from GitHub actions
- Add makefile target for installing dependencies
- Add Sia to projects using Cobra
- Add Vitess and Arewefastyet to projects using cobra
- Fixup for Kubescape github org
- Fix route for GitHub workflows badge
- Fixup for GoDoc style documentation
- Various bash scripting improvements for completion
- Add Constellation to projects using Cobra

Documentation

- Add documentation about disabling completion descriptions
- Improve MarkFlagsMutuallyExclusive example in user guide
- Update shell_completions.md
- Update copywrite year
- Document suggested layout of subcommands
- Replace deprecated ExactValidArgs with MatchAll in doc

full diff: https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 01:52:30 +02:00
Sebastiaan van Stijn
f8dec8f1c6 Merge pull request #45565 from corhere/fix-dns-servfail
libnetwork: just forward the external DNS response
2023-05-18 23:30:45 +02:00
Cory Snider
6690d2969c pkg/archive: bail if setting xattrs is unsupported
Extended attributes are set on files in container images for a reason.
Fail to unpack if extended attributes are present in a layer and setting
the attributes on the unpacked files fails for any reason.

Add an option to the vfs graph driver to opt into the old behaviour
where ENOTSUPP and EPERM errors encountered when setting extended
attributes are ignored. Make it abundantly clear to users and anyone
triaging their bug reports that they are shooting themselves in the
foot by enabling this option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 17:21:12 -04:00
Cory Snider
0cdfd5f275 pkg/system: return rich errors from L(g|s)etxattr
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 16:31:01 -04:00
Bjorn Neergaard
ccd834ea25 Merge pull request #45557 from crazy-max/fix-worker-id
build: use daemon id as worker id for the graph driver controller
2023-05-18 21:17:43 +01:00
Cory Snider
41356227f2 libnetwork: just forward the external DNS response
Our resolver is just a forwarder for external DNS so it should act like
it. Unless it's a server failure or refusal, take the response at face
value and forward it along to the client. RFC 8020 is only applicable to
caching recursive name servers and our resolver is neither caching nor
recursive.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 16:04:19 -04:00
Sebastiaan van Stijn
1b3c2743cc Merge pull request #45550 from corhere/fix-empty-container-decode
Allow empty body in `POST /commit` again
2023-05-18 21:46:00 +02:00
Sebastiaan van Stijn
50d2c94bd6 Merge pull request #43637 from thaJeztah/remove_deprecated_storage_drivers
Remove deprecated devicemapper storage driver
2023-05-18 21:23:41 +02:00
Kevin Alvarez
6d139e5e95 build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-18 21:17:29 +02:00
Sebastiaan van Stijn
054add40a1 Merge pull request #45552 from miles-to-go/fix-insecure-registries-reload
Fix insecure registries reload
2023-05-18 20:56:55 +02:00
Sebastiaan van Stijn
7a821d4c99 Merge pull request #45567 from crazy-max/ci-bin-image-distribute
ci(bin-image): distribute build across runners
2023-05-18 20:55:47 +02:00
Kevin Alvarez
668af4be82 ci(bin-image): distribute build across runners
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-18 18:31:07 +02:00
Brian Goff
de9f12ea96 Merge pull request #45563 from vvoland/fix-45556
api/inspect: Fix nil RepoTags and RepoDigests
2023-05-18 07:59:56 -07:00
Paweł Gronowski
1be26e9f0c api/inspect: Fix nil RepoTags and RepoDigests
Make RepoTags and RepoDigests empty arrays instead of nil.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-18 15:57:18 +02:00
Bjorn Neergaard
b42c65d796 Merge pull request #45561 from laurazard/fix-pull-platform
fix: `docker pull` with platform checks wrong image tag
2023-05-18 14:14:12 +01:00
Laura Brehm
f450ea64e6 fix: docker pull with platform checks wrong image tag
This fixes a bug where, if a user pulls an image with a tag != `latest` and
a specific platform, we return an NotFound error for the wrong (`latest`) tag.
see: https://github.com/moby/moby/issues/45558

This bug was introduced in 779a5b3029
in the changes to `daemon/images/image_pull.go`, when we started returning the error from the call to
`GetImage` after the pull. We do this call, if pulling with a specified platform, to check if the platform
of the pulled image matches the requested platform (for cases with single-arch images).
However, when we call `GetImage` we're not passing the image tag, only name, so `GetImage` assumes `latest`
which breaks when the user has requested a different tag, since there might not be such an image in the store.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-18 13:14:59 +01:00
Nolan Miles
f3645a2aa3 reorder load funcs to match newServiceConfig()'s order
Signed-off-by: Nolan Miles <nolanpmiles@gmail.com>
2023-05-17 21:34:14 -04:00
Nolan Miles
3b15156e4d add mirror to daemon reload test for insecure registries
Signed-off-by: Nolan Miles <nolanpmiles@gmail.com>
2023-05-17 21:34:14 -04:00
Bjorn Neergaard
2ea5c1af57 Merge pull request #45551 from tianon/rm-upstart
Remove Upstart scripts
2023-05-18 01:21:36 +01:00
Tianon Gravi
0d8087fbbc Remove Upstart scripts
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-05-17 14:20:14 -07:00
Tianon Gravi
405f4589a6 Merge pull request #45505 from neersighted/mailmap_update
mailmap update and regenerate AUTHORS
2023-05-17 20:43:48 +00:00
Cory Snider
967c7bc5d3 api/server: allow empty body for POST /commit again
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-17 16:01:11 -04:00
Cory Snider
3ceb3810d7 client: don't marshal typed nils in request body
The internal Client request methods which accept an object as a body use
nil to signal that the request should not have a body. But it is easy to
accidentally pass a typed-nil value as the object, e.g. if the object
comes from a function argument or struct field of a concrete type. The
result is that these requests will, surprisingly, have a JSON body of
`null`. Treat typed-nil pointers the same as untyped nils for the
purposes of determining whether or not the request should include a
body.

Stop assuming that POST requests should always have a body. POST /commit
does not require a body, for example.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-17 14:58:12 -04:00
Tianon Gravi
ae737656f9 Remove Upstart and cgroups bits from Debian sysvinit script
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Additionally, this removes the "cgroups v1" setup code because it's more reasonable now for us to expect something _else_ to have set up cgroups appropriately (especially cgroups v2).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-05-16 15:39:33 -07:00
Evan Lezar
c2630c93f2 Update vendoring
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:11:09 +02:00
Evan Lezar
dedf5747cc Remove need to set Capabilities for cdi driver
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:58 +02:00
Evan Lezar
7ec9561a77 Add support for CDI devices to docker daemon under linux
These changes add basic CDI integration to the docker daemon.

A cdi driver is added to handle cdi device requests. This
is gated by an experimental feature flag and is only supported on linux

This change also adds a CDISpecDirs (cdi-spec-dirs) option to the config.
This allows the default values of `/etc/cdi`, /var/run/cdi` to be overridden
which is useful for testing.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:57 +02:00
Evan Lezar
5a200ade7c Rename daemon/devices_liunx.go to daemom/devices.go
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:57 +02:00
Akihiro Suda
1371aee3cc Merge pull request #45469 from thaJeztah/deprecate_virtualsize_STEP2
API: omit deprecated VirtualSize field for API v1.44 and up
2023-05-16 15:10:07 +09:00
Sebastiaan van Stijn
9548916aa8 Merge pull request #45512 from thaJeztah/libnetwork_fix_readme_example
libnetwork: update example in README.md
2023-05-15 21:48:06 +01:00
Sebastiaan van Stijn
72b2469be0 Merge pull request #45536 from thaJeztah/containerd_binary_1.7.1
update containerd binary to v1.7.1
2023-05-15 15:25:01 +01:00
Sebastiaan van Stijn
484785456c update containerd binary to v1.7.1
full diff: https://github.com/containerd/containerd/compare/v1.7.0...v1.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-15 12:52:50 +01:00
Bjorn Neergaard
be220af9fb Merge pull request #44395 from jmrouet/master
Fixing dockerd-rootless-setuptools.sh when user name contains a backslash
2023-05-12 09:39:29 -06:00
Sebastiaan van Stijn
bafcfdf8c5 Merge pull request #45484 from thaJeztah/remove_deprecated_stubs
remove deprecated types, fields, and functions
2023-05-12 14:03:26 +01:00
Sebastiaan van Stijn
5d5f2e3a93 Merge pull request #45530 from rumpl/fix-empty-auth-pull
c8d: The authorizer needs to be set even if AuthConfig is empty
2023-05-12 10:48:27 +01:00
Djordje Lukic
348a79958d c8d: The authorizer needs to be set even if AuthConfig is empty
Without the authorizer pulling will fail if the user is not logged-in

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-05-12 09:41:34 +02:00
Jean-Michel Rouet
2f0ba0a7e5 more robust dockerd-rootless-setuptools.sh
Fixing case where username may contain a backslash.
This case can happen for winbind/samba active directory domain users.

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Use more meaningful variable name

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Update contrib/dockerd-rootless-setuptool.sh

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Use more meaningful variable name

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Update contrib/dockerd-rootless-setuptool.sh

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>
2023-05-12 09:22:14 +02:00
Akihiro Suda
9e6370819b Merge pull request #45473 from zhangguanzhang/master-commentStr
[builder] make chownComment easy to read
2023-05-12 09:35:10 +09:00
Tianon Gravi
46ce4ec8cf Merge pull request #45322 from laurazard/c8d-upstream-classic-builder
c8d: implement classic builder
2023-05-11 18:10:21 +00:00
Sebastiaan van Stijn
9983d896de Merge pull request #45519 from crazy-max/bin-image
bin-image bake target
2023-05-11 18:07:35 +02:00
Sebastiaan van Stijn
7c43e8c45a Merge pull request #45513 from thaJeztah/client_remove_uses_of_IsErrNotFound
client: make IsErrNotFound an alias for errdefs.IsNotFound, clean-up assertions on tests
2023-05-11 17:42:15 +02:00
Sebastiaan van Stijn
f0791afade Merge pull request #45517 from vvoland/c8d-authconfig-default
c8d: Better handling of partially filled `AuthConfig`
2023-05-11 17:38:17 +02:00
CrazyMax
135d8f04f9 ci: bin-image workflow
This workflow will just build the bin-image bake target.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-11 15:52:41 +02:00
Kevin Alvarez
ae1ca67178 bin-image bake target
Allows to build a non-runnable image that contains bundles.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-11 15:52:41 +02:00
Sebastiaan van Stijn
d922f94d93 Merge pull request #45516 from thaJeztah/vendor_distribution_v2.8.2-beta.2
vendor: github.com/docker/distribution v2.8.2-beta.2
2023-05-11 14:59:44 +02:00
Paweł Gronowski
3309e45ca1 c8d: Don't create authorizer for empty AuthConfig
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-11 14:32:46 +02:00
Paweł Gronowski
2ad499f93e c8d/authorizer: Default to docker.io
When the `ServerAddress` in the `AuthConfig` provided by the client is
empty, default to the default registry (registry-1.docker.io).

This makes the behaviour the same as with the containerd image store
integration disabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-11 14:17:57 +02:00
Laura Brehm
bd6868557d c8d/builder: implement cache
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 12:02:09 +01:00
Sebastiaan van Stijn
7821d2d788 vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

bump to release/2.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 12:58:52 +02:00
Laura Brehm
e46674b6a7 c8d: implement classic builder
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 11:57:01 +01:00
Brian Goff
e58c267d66 Merge pull request #45304 from crazy-max/buildkit-version-buildinfo
Set BuildKit version using buildinfo
2023-05-10 16:58:18 -07:00
Sebastiaan van Stijn
3b443c8cfc Merge pull request #45508 from laurazard/c8d-dangling-image-history
c8d: image history – handle dangling images
2023-05-10 22:30:02 +02:00
Sebastiaan van Stijn
8d76acfe6c testutil: replace uses of client.IsErrNotFound
The client no longer returns the old error-type, so we can use errdefs
instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 22:13:50 +02:00
Sebastiaan van Stijn
56fb56ccf0 integration: update error-assertions in tests
- use is.ErrorType
- replace uses of client.IsErrNotFound for errdefs.IsNotFound, as
  the client no longer returns the old error-type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 22:13:50 +02:00
Sebastiaan van Stijn
0538cdd226 integration-cli: update error-assertions in tests
- use is.ErrorType
- replace uses of client.IsErrNotFound for errdefs.IsNotFound, as
  the client no longer returns the old error-type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 22:13:50 +02:00
Sebastiaan van Stijn
66ff1e063e client: update error-assertions in tests
- use is.ErrorType
- replace uses of client.IsErrNotFound for errdefs.IsNotFound, as
  the client no longer returns the old error-type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 22:13:38 +02:00
Sebastiaan van Stijn
685b3d820a client: make IsErrNotFound an alias for errdefs.IsNotFound
None of the client will return the old error-types, so there's no need
to keep the compatibility code. We can consider deprecating this function
in favor of the errdefs equivalent this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 21:52:43 +02:00
Sebastiaan van Stijn
f70d9933d1 Merge pull request #45427 from laurazard/c8d-fix-incomplete-images
c8d: implement missing image delete logic
2023-05-10 18:01:45 +02:00
Sebastiaan van Stijn
9e817251a8 libnetwork/docs: fix broken link
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 12:05:05 +02:00
Sebastiaan van Stijn
17882ed614 libnetwork: update example in README.md
Align the example with the code updated in 4e0319c878.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 12:01:06 +02:00
Sebastiaan van Stijn
3f7fde76c2 Merge pull request #45507 from corhere/cleanup-reexec-inits
[chore] clean up reexec.Init() calls
2023-05-10 11:49:11 +02:00
Laura Brehm
4603b6d6b6 c8d: fix image history for dangling images
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-10 01:33:15 +01:00
Brian Goff
03885e1b00 Merge pull request #45490 from Jeyanthinath/consistent_alias
fixing consistent aliases for OCI spec imports
2023-05-09 16:33:32 -07:00
Cory Snider
4e0319c878 [chore] clean up reexec.Init() calls
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)

Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-09 19:13:17 -04:00
Sebastiaan van Stijn
b07d2a3827 Merge pull request #45486 from thaJeztah/daemon_monitor_logclean
daemon: handleContainerExit(): use logrus.WithFields
2023-05-10 00:51:50 +02:00
Bjorn Neergaard
9e43c11480 AUTHORS: regenerate
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-09 15:14:50 -06:00
Bjorn Neergaard
5fee4439a0 .mailmap: update with my new email
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-09 15:14:40 -06:00
Sebastiaan van Stijn
563fc92dd4 Merge pull request #45496 from thaJeztah/pkg_aaparser_deprecate_version
pkg/aaparser: deprecate GetVersion, as it's no longer used
2023-05-09 19:18:39 +02:00
Sebastiaan van Stijn
b587c5a2f8 Merge pull request #45501 from laurazard/c8d-image-history
c8d: add image history
2023-05-09 18:46:44 +02:00
Laura Brehm
e8be792130 c8d: fix missing image history
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-09 16:19:05 +01:00
Sebastiaan van Stijn
e3e715666f pkg/aaparser: deprecate GetVersion, as it's no longer used
Our templates no longer contain version-specific rules, so this function
is no longer used. This patch deprecates it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:47:03 +02:00
Sebastiaan van Stijn
06221297bf Merge pull request #45495 from thaJeztah/apparmor_remove_version_code
profiles/apparmor: remove use of aaparser.GetVersion()
2023-05-09 16:46:32 +02:00
Sebastiaan van Stijn
a8b1ce2447 Merge pull request #45492 from thaJeztah/contrib_apparmor_remove_version
contrib/apparmor: remove remaining version-conditionals (< 2.9) from template
2023-05-09 16:46:08 +02:00
Jeyanthinath Muthuram
71d7908656 added alias validation
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 19:03:44 +05:30
Sebastiaan van Stijn
ecaab085db profiles/apparmor: remove use of aaparser.GetVersion()
commit 7008a51449 removed version-conditional
rules from the template, so we no longer need the apparmor_parser Version.

This patch removes the call to `aaparser.GetVersion()`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-08 13:50:13 +02:00
Sebastiaan van Stijn
f445ee1e6c contrib/apparmor: remove remaining version-conditionals (< 2.9) from template
Commit 2e19a4d56b removed all other version-
conditional statements from the AppArmor template, but left this one in place.

These conditions were added in 8cf89245f5
to account for old versions of debian/ubuntu (apparmor_parser < 2.9)
that lacked some options;

> This allows us to use the apparmor profile we have in contrib/apparmor/
> and solves the problems where certain functions are not apparent on older
> versions of apparmor_parser on debian/ubuntu.

Those patches were from 2015/2016, and all currently supported distro
versions should now have more current versions than that. Looking at the
oldest supported versions;

Ubuntu 18.04 "Bionic":

    apparmor_parser --version
    AppArmor parser version 2.12
    Copyright (C) 1999-2008 Novell Inc.
    Copyright 2009-2012 Canonical Ltd.

Debian 10 "Buster"

    apparmor_parser --version
    AppArmor parser version 2.13.2
    Copyright (C) 1999-2008 Novell Inc.
    Copyright 2009-2018 Canonical Ltd.

This patch removes the remaining conditionals.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-08 13:17:59 +02:00
Jeyanthinath Muthuram
307b09e7eb fixing consistent aliases for OCI spec imports
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 15:27:52 +05:30
Sebastiaan van Stijn
c651a53558 Merge pull request #45488 from doringeman/dorin.geman/execDuration-in-containerExit
daemon: handleContainerExit(): add execDuration in attributes
2023-05-08 11:14:28 +02:00
Sebastiaan van Stijn
d9d9950d8d Merge pull request #45482 from thaJeztah/docs_api_fix_missing_endpoint
docs/api: version-history: also mention /system/df for VirtualSize
2023-05-08 11:03:40 +02:00
Dorin Geman
2ad37e1832 daemon: handleContainerExit(): add execDuration in attributes
Add `execDuration` field to the event attributes map. This is useful for tracking how long the container ran.

Signed-off-by: Dorin Geman <dorin.geman@docker.com>
2023-05-08 10:26:33 +03:00
Laura Brehm
cad97135b3 c8d: implement missing image delete logic
Ports over all the previous image delete logic, such as:
- Introduce `prune` and `force` flags
- Introduce the concept of hard and soft image delete conflics, which represent:
  - image referenced in multiple tags (soft conflict)
  - image being used by a stopped container (soft conflict)
  - image being used by a running container (hard conflict)
- Implement delete logic such as:
  - if deleting by reference, and there are other references to the same image, just
    delete the passed reference
  - if deleting by reference, and there is only 1 reference and the image is being used
    by a running container, throw an error if !force, or delete the reference and create
    a dangling reference otherwise
  - if deleting by imageID, and force is true, remove all tags (otherwise soft conflict)
  - if imageID, check if stopped container is using the image (soft conflict), and
    delete anyway if force
  - if imageID was passed in, check if running container is using the image (hard conflict)
  - if `prune` is true, and the image being deleted has dangling parents, remove them

This commit also implements logic to get image parents in c8d by comparing shared layers.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-08 02:32:03 +01:00
Sebastiaan van Stijn
de363f1404 daemon: handleContainerExit(): use logrus.WithFields
Use `WithFields()` instead of chaining multiple `WithField()` calls.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-07 21:24:34 +02:00
Sebastiaan van Stijn
fb96b94ed0 daemon: remove handling for deprecated "oom-score-adjust", and produce error
This option was deprecated in 5a922dc162, which
is part of the v24.0.0 release, so we can remove it from master.

This patch;

- adds a check to ValidatePlatformConfig, and produces a fatal error
  if oom-score-adjust is set
- removes the deprecated libcontainerd/supervisor.WithOOMScore
- removes the warning from docker info

With this patch:

    dockerd --oom-score-adjust=-500 --validate
    Flag --oom-score-adjust has been deprecated, and will be removed in the next release.
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.

And when using `daemon.json`:

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
316781be48 api/types/container: remove deprecated ContainerChangeResponseItem
This was deprecated in dbb48e4b29, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
9c14f98583 api/types: remove deprecated AuthConfig
This was deprecated in 818ee96219, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
55608d8381 pkg/system: remove deprecated IsProcessAlive, IsProcessZombie, KillProcess
These were deprecated in 9d5e754caa, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
a3b12ec420 pkg/system: remove deprecated DefaultPathEnv()
This was deprecated in 9f3e5eead5, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
dc76b90705 pkg/system: remove deprecated MemInfo, ReadMemInfo()
These were deprecated in 2d49080056, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
7d0488b11e pkg/ioutils: remove deprecated TempDir() alias
This function was deprecated in c63ea32a17, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:16 +02:00
Sebastiaan van Stijn
152c482fee pkg/platform: remove deprecated OSType const
This const was deprecated in 5c78cbd3be, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:16 +02:00
Sebastiaan van Stijn
913b0f51ca API: omit deprecated VirtualSize field for API v1.44 and up
This field is deprecated since 1261fe69a3,
and will now be omitted on API v1.44 and up for the `GET /images/json`,
`GET /images/{id}/json`, and `GET /system/df`  endpoints.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:35:00 +02:00
Sebastiaan van Stijn
fdc7a78652 docs/api: version-history: also mention /system/df for VirtualSize
Commit 1261fe69a3 deprecated the VirtualSize
field, but forgot to mention that it's also included in the /system/df
endpoint.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:28:37 +02:00
Sebastiaan van Stijn
88f4bf4ae4 Merge pull request #45468 from thaJeztah/bump_api_1.44
API: bump version to 1.44
2023-05-06 15:10:59 +02:00
Sebastiaan van Stijn
fd0b109bf7 API: bump version to 1.44
The 24.0 branch was created, so changes in master/main should now be
targeting the next version of the API (1.44).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 01:41:49 +02:00
zhangguanzhang
43b1afb2cf make chownComment easy to read
Signed-off-by: zhangguanzhang <zhangguanzhang@qq.com>
2023-05-05 19:22:49 +08:00
Kevin Alvarez
7665feeb52 hack: enable Go modules when building dockerd and docker-proxy
This is a workaround to have buildinfo with deps embedded in the
binary. We need to create a go.mod file before building with
-modfile=vendor.mod, otherwise it fails with:
"-modfile cannot be used to set the module root directory."

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 12:30:52 +02:00
CrazyMax
5400366b90 Set BuildKit version using buildinfo
This sets BuildKit version from the build information embedded
in running binary so we are aligned with the expected vendoring.

We iterate over all dependencies and find the BuildKit one
and set the right version. We also check if the module is
replaced and use it this case.

There is also additional checks if a pseudo version is
detected. See comments in code for more info.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 12:30:52 +02:00
Sebastiaan van Stijn
c33b31116c daemon/graphdriver: remove isDeprecated() utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-20 23:51:54 +02:00
Sebastiaan van Stijn
dc11d2a2d8 remove deprecated devicemapper storage-driver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-20 23:51:54 +02:00
Cory Snider
1e195acee4 libn/d/overlay: stop programming INPUT ACCEPT rule
Encrypted overlay networks are unique in that they are the only kind of
network for which libnetwork programs an iptables rule to explicitly
accept incoming packets. No other network driver does this. The overlay
driver doesn't even do this for unencrypted networks!

Because the ACCEPT rule is appended to the end of INPUT table rather
than inserted at the front, the rule can be entirely inert on many
common configurations. For example, FirewallD programs an unconditional
REJECT rule at the end of the INPUT table, so any ACCEPT rules appended
after it have no effect. And on systems where the rule is effective, its
presence may subvert the administrator's intentions. In particular,
automatically appending the ACCEPT rule could allow incoming traffic
which the administrator was expecting to be dropped implicitly with a
default-DROP policy.

Let the administrator always have the final say in how incoming
encrypted overlay packets are filtered by no longer automatically
programming INPUT ACCEPT iptables rules for them.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-05 11:35:19 -04:00
Milas Bowman
eb0edeafdd api: swagger: add missing "force" query arg on plugin disable
This has been around for a long time - since v17.04 (API v1.28)
but was never documented.

It allows removing a plugin even if it's still in use.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-08-01 16:06:08 -04:00
Sebastiaan van Stijn
4981b04d10 Merge pull request #202 from jhowardmsft/etcd-io-bbolt
boltdb/bolt is not maintained. Use bbolt instead.
2018-09-12 22:54:06 +02:00
John Howard
ea4dd1bbf6 boltdb/bolt is not maintained. Used bbolt
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-11 12:42:39 -07:00
Santhosh Manohar
ec5fbce415 Merge pull request #140 from aboch/to
Allow client to set boltdb timeout
2016-11-08 17:06:21 -08:00
Alessandro Boch
6c0e0d6250 Allow client to set boltdb timeout
- also in case of no persistent connection

Signed-off-by: Alessandro Boch <aboch@docker.com>
2016-11-08 16:51:51 -08:00
Santhosh Manohar
b4c828e6a0 Merge pull request #127 from mrjana/boltdb
Do not return boltdb bucket not found error
2016-04-15 14:08:06 -07:00
Jana Radhakrishnan
a0f6d0798a Do not return boltdb bucket not found error
While doing a boltdb operation and if the bucket is not found
we should not return a boltdb specific bucket not found error
because this causes leaky abstraction where in the user of libkv
needs to know about boltdb and import boltdb dependencies
neither of which is desirable. Replaced all the bucket not found
errors with the more generic `store.ErrKeyNotFound` error which
is more appropriate.

Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
2016-04-15 13:50:43 -07:00
Alexandre Beslic
2d3c380c0f Merge pull request #121 from Jimdo/etcd_auth
Add support for etcd authentication
2016-04-04 14:49:35 -07:00
Paul Seiffert
deb21faad0 Add support for etcd authentication 2016-03-31 14:55:59 +02:00
Alexandre Beslic
2478cfa966 Merge pull request #115 from allencloud/fix-typo
fix typo
2016-03-07 11:07:56 -08:00
allencloud
cc0ed694ee fix typo
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-03-05 22:59:26 +08:00
Alexandre Beslic
0dfad687a0 Merge pull request #114 from stweil/master
Fix typo in error text
2016-02-29 10:50:14 -08:00
Stefan Weil
fbdb24ec6a Fix typo in error text
The typo was found and fixed by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2016-02-23 22:40:39 +01:00
Alexandre Beslic
dcf8828165 Return store.ErrKeyExists on AtomicPut
When using AtomicPut with 'previous' set at nil, it interprets
that the Key should be created with the AtomicPut. Instead of
returning a generic error, we return store.ErrKeyExists if the
key exists in the store during the operation.

Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-10-14 09:51:18 -07:00
Alexandre Beslic
99db1f6a07 Merge pull request #86 from sanimej/bugfix
boltDB AtomicDelete fails for non-existent key
2015-10-14 14:55:08 -07:00
Alexandre Beslic
bf2f48aec0 Merge pull request #84 from mavenugo/b-ts
Cherry-pick 2b31300cef639e8c54aaaf52a6f57d1cf11883a3 into master (Synchronize boltbd libkv apis)
2015-10-12 12:09:29 -07:00
Jana Radhakrishnan
002f9c37ca Synchronize boltbd libkv apis
Currently boltdb uses a handle which can be accessed
concurrently from multiple go routines and all of them
trying to open and close the boldb db handle which can
cause havoc. Use a mutex to serialize db access and
handle access.

Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
2015-10-12 11:51:36 -07:00
Alexandre Beslic
2a86b2bdb9 Merge pull request #79 from sanimej/bolt
Add transient connection option to bolt db
2015-10-08 13:24:52 -07:00
Alexandre Beslic
1f587cf86c Merge pull request #77 from abronan/enhance_documentation
Update README.md and add more documentation under the 'docs' folder
2015-10-07 18:06:06 -07:00
Santhosh Manohar
3e2529e881 Merge pull request #78 from abronan/change_unsupported_errors
Change unsupported call error in boltdb backend to use general store error
2015-10-06 15:33:28 -07:00
Alexandre Beslic
15e5d4067a Change unsupported call error in boltdb backend to use general store error
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-10-06 06:05:48 -07:00
Alexandre Beslic
3b5fb3ec87 Remove docs in libkv.go
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-10-05 14:32:40 -07:00
Alexandre Beslic
2603af65e5 Migration to the new Etcd client
This commit migrates the old 'go-etcd' client, which is deprecated
to the new 'coreos/etcd/client'.

One notable change is the ability to specify an 'IsDir' parameter
to the 'Put' call. This allows to circumvent the limitations of etcd
regarding the key/directory distinction while setting up Watches on
a directory. A conservative measure to set up a watch that should be
used the same way for etcd/consul/zookeeper is to enforce the 'IsDir'
parameter with 'WriteOptions' on 'Put' to avoid the 'NotANode' error
thrown by etcd on Watch call. Consul and zookeeper are not using the
'IsDir' parameter.

Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-10-05 05:57:23 -07:00
Santhosh Manohar
f06f78e638 boltDB AtomicDelete fails for non-existent key
Signed-off-by: Santhosh Manohar <santhosh@docker.com>
2015-10-03 02:25:19 -07:00
Santhosh Manohar
0b81042dc3 Add transient connection option to bolt db
Signed-off-by: Santhosh Manohar <santhosh@docker.com>
2015-09-30 18:37:17 -07:00
Victor Vieux
3c89e44f3e Merge pull request #63 from abronan/consul_ttl_lock
Add TTL to Consul lock using store.LockOptions
2015-09-23 10:46:28 -07:00
Alexandre Beslic
7225aba3af add TTL to consul lock using store.LockOptions and add tests for the Lock ttl behavior
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-09-23 02:22:02 -07:00
Santhosh Manohar
1de5aa45ce Merge pull request #62 from chenchun/fix
Fix AtomicPut panic if previous KVPair is not nil and key not exists in boltdb
2015-09-22 05:56:07 -07:00
Chun Chen
1f9cf17713 Fix AtomicPut panic if previous KVPair is not nil and key not exists in boltdb
Signed-off-by: Chun Chen <ramichen@tencent.com>
2015-09-22 14:23:59 +08:00
Alexandre Beslic
79b8613c95 Merge pull request #55 from abronan/etcd_tls
Carry #45: Add etcd TLS client code and update README.md.
2015-09-18 11:11:43 -07:00
Alexandre Beslic
599deba8c7 nits and refactor for etcd tls config
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-09-17 09:13:45 -07:00
Adam Thornton
cc9e627f8e Add etcd TLS client code and update README.md.
Signed-off-by: Adam Thornton <adam.thornton@gonkulator.io>
2015-09-17 09:08:53 -07:00
Alexandre Beslic
81a45f92c5 Merge pull request #56 from mavenugo/bolt_to
Honoring ConnectionTimeout in boltdb
2015-09-16 11:15:51 -07:00
Madhu Venugopal
64eb25dff9 Honoring ConnectionTimeout in boltdb
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2015-09-16 10:52:11 -07:00
Alexandre Beslic
29a75f0428 Merge pull request #51 from abronan/fix_vendor_import
Refactor libkv to not directly import storage backends
2015-09-08 10:18:50 -07:00
Alexandre Beslic
3ec6dfa346 Refactor libkv to not directly import storage backends
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-09-07 10:43:01 -07:00
Alexandre Beslic
b1d5d7edeb Merge pull request #47 from chenchun/fix_boltdb
Create dir first before opening data file of boltdb
2015-08-24 11:04:49 -07:00
Chun Chen
f817826408 Create dir first before opening data file of boltdb
Signed-off-by: Chun Chen <ramichen@tencent.com>
2015-08-25 01:41:57 +08:00
Alexandre Beslic
6c6315a1b3 Merge pull request #46 from sanimej/bolt
BoltDB backend support for libkv
2015-08-20 23:54:03 -07:00
Santhosh Manohar
29af4e0f27 BoltDB backend support for libkv
Signed-off-by: Santhosh Manohar <santhosh@docker.com>
2015-08-13 21:39:53 -07:00
Alexandre Beslic
e908876506 Merge pull request #33 from abronan/drop_ephemeral
Change Heartbeat/Ephemeral combination to use a single TTL field
2015-08-13 17:09:49 -07:00
Alexandre Beslic
c01507da42 Merge pull request #40 from chenchun/fix_zk_list
Fix zookeeper List function not exist key error
2015-08-12 18:09:27 -07:00
Alexandre Beslic
06f01227c5 change heartbeat/ephemeral combination to use a single ttl field
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-08-12 11:12:38 -07:00
Chun Chen
f26e4f4c52 Minor fix of store comment
Signed-off-by: Chun Chen <ramichen@tencent.com>
2015-08-12 18:32:56 +08:00
Alexandre Beslic
518ab82942 Merge pull request #39 from chenchun/better_error_msg
Notify user of supported backend storage
2015-08-11 11:59:53 -07:00
Chun Chen
6b43181cfa Notify user of supported backend storage
Signed-off-by: Chun Chen <ramichen@tencent.com>
2015-08-11 14:37:01 +08:00
Alexandre Beslic
ebeec0dd96 Merge pull request #30 from ahmetalpbalkan/rm-WatchCallback
Remove unused type WatchCallback
2015-07-21 13:50:23 -07:00
Ahmet Alp Balkan
a961b70b2b Remove unused type WatchCallback
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-07-21 13:48:32 -07:00
Alexandre Beslic
0dda8605e6 Merge pull request #19 from erikh/doc-fixes
Doc fixes
2015-06-26 09:39:42 -07:00
Erik Hollensbe
06151b8a4d add top-level godoc: sample some parts of the README
Signed-off-by: Erik Hollensbe <github@hollensbe.org>
2015-06-25 00:05:57 +00:00
Alexandre Beslic
638eb1907e Merge pull request #14 from spikecurtis/etcd-atomic-create
Support "Atomic Create" in etcd
2015-06-18 20:37:29 -07:00
Spike Curtis
a2d4e1c246 API Change: AtomicPut with previous = nil is create.
AtomicPut can now be used to Compare-and-Swap against the state
where the key doesn't yet exist.  E.g. a race where two clients
create the same key: one succeeds, the other fails.

Pass nil for the previous argument of AtomicPut for this
behavior.  Before, this would cause an error.

Implements this change for all three backends.
2015-06-18 11:21:11 -07:00
Alexandre Beslic
7d01dbf6fa Merge pull request #13 from ahmetalpbalkan/master
Use Backend type for stores
2015-06-15 15:23:07 -07:00
Ahmet Alp Balkan
fcb98254b1 Use Backend type for stores
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-15 15:05:46 -07:00
Andrea Luzzardi
e2061c9708 Merge pull request #10 from abronan/fix_mock_dependency
Remove dependency to testify through the mock package
2015-06-12 16:08:09 -07:00
Alexandre Beslic
79c320c1cf remove testify dependency through mock package
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-06-12 15:54:16 -07:00
Antonio Murdaca
48f5dca2ce Remove logging in library
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-06-13 00:08:33 +02:00
Alexandre Beslic
dbc1a6fa9a refactor and move store backends in sub-packages
Signed-off-by: Alexandre Beslic <abronan@docker.com>
2015-06-09 21:38:14 -07:00
6812 changed files with 705340 additions and 319178 deletions

View File

@@ -0,0 +1,21 @@
{
"name": "moby",
"build": {
"context": "..",
"dockerfile": "../Dockerfile",
"target": "devcontainer"
},
"workspaceFolder": "/go/src/github.com/docker/docker",
"workspaceMount": "source=${localWorkspaceFolder},target=/go/src/github.com/docker/docker,type=bind,consistency=cached",
"remoteUser": "root",
"runArgs": ["--privileged"],
"customizations": {
"vscode": {
"extensions": [
"golang.go"
]
}
}
}

1
.github/CODEOWNERS vendored
View File

@@ -5,7 +5,6 @@
builder/** @tonistiigi
contrib/mkimage/** @tianon
daemon/graphdriver/devmapper/** @rhvgoyal
daemon/graphdriver/overlay2/** @dmcgowan
daemon/graphdriver/windows/** @johnstep
daemon/logger/awslogs/** @samuelkarp

View File

@@ -19,12 +19,18 @@ Please provide the following information:
**- How to verify it**
**- Description for the changelog**
**- Human readable description for the release notes**
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
pull request for inclusion in the changelog.
It must be placed inside the below triple backticks section.
NOTE: Only fill this section if changes introduced in this PR are user-facing.
The PR must have a relevant impact/ label.
-->
```markdown changelog
```
**- A picture of a cute animal (not mandatory but encouraged)**

View File

@@ -13,7 +13,7 @@ runs:
shell: bash
- run: |
if [ ! -e /etc/docker/daemon.json ]; then
echo '{}' | tee /etc/docker/daemon.json >/dev/null
echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
fi
DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null

View File

@@ -0,0 +1,14 @@
name: 'Setup Tracing'
description: 'Composite action to set up the tracing for test jobs'
runs:
using: composite
steps:
- run: |
set -e
# Jaeger is set up on Windows through an inline run step. If you update Jaeger here, don't forget to update
# the version set in .github/workflows/.windows.yml.
docker run -d --net=host --name jaeger -e COLLECTOR_OTLP_ENABLED=true jaegertracing/all-in-one:1.46
docker0_ip="$(ip -f inet addr show docker0 | grep -Po 'inet \K[\d.]+')"
echo "OTEL_EXPORTER_OTLP_ENDPOINT=http://${docker0_ip}:4318" >> "${GITHUB_ENV}"
shell: bash

View File

@@ -3,31 +3,41 @@ name: .dco
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
env:
ALPINE_VERSION: 3.16
ALPINE_VERSION: "3.20"
jobs:
run:
runs-on: ubuntu-20.04
timeout-minutes: 10 # guardrails timeout for the whole job
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Dump context
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
script: |
console.log(JSON.stringify(context, null, 2));
-
name: Get base ref
id: base-ref
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
result-encoding: string
script: |
@@ -39,10 +49,12 @@ jobs:
name: Validate
run: |
docker run --rm \
-v "$(pwd):/workspace" \
--quiet \
-v ./:/workspace \
-w /workspace \
-e VALIDATE_REPO \
-e VALIDATE_BRANCH \
alpine:${{ env.ALPINE_VERSION }} sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && cd /workspace && hack/validate/dco'
alpine:${{ env.ALPINE_VERSION }} sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && hack/validate/dco'
env:
VALIDATE_REPO: ${{ github.server_url }}/${{ github.repository }}.git
VALIDATE_BRANCH: ${{ steps.base-ref.outputs.result }}

45
.github/workflows/.test-prepare.yml vendored Normal file
View File

@@ -0,0 +1,45 @@
# reusable workflow
name: .test-prepare
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
outputs:
matrix:
description: Test matrix
value: ${{ jobs.run.outputs.matrix }}
jobs:
run:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
outputs:
matrix: ${{ steps.set.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Create matrix
id: set
uses: actions/github-script@v7
with:
script: |
let matrix = ['graphdriver'];
if ("${{ contains(github.event.pull_request.labels.*.name, 'containerd-integration') || github.event_name != 'pull_request' }}" == "true") {
matrix.push('snapshotter');
}
await core.group(`Set matrix`, async () => {
core.info(`matrix: ${JSON.stringify(matrix)}`);
core.setOutput('matrix', JSON.stringify(matrix));
});

477
.github/workflows/.test.yml vendored Normal file
View File

@@ -0,0 +1,477 @@
# reusable workflow
name: .test
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
inputs:
storage:
required: true
type: string
default: "graphdriver"
env:
GO_VERSION: "1.22.12"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
ITG_CLI_MATRIX_SIZE: 6
DOCKER_EXPERIMENTAL: 1
DOCKER_GRAPHDRIVER: ${{ inputs.storage == 'snapshotter' && 'overlayfs' || 'overlay2' }}
TEST_INTEGRATION_USE_SNAPSHOTTER: ${{ inputs.storage == 'snapshotter' && '1' || '' }}
SETUP_BUILDX_VERSION: latest
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
jobs:
unit:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-unit
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-unit-${{ inputs.storage }}
path: /tmp/reports/*
retention-days: 1
unit-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- unit
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
name: test-reports-unit-${{ inputs.storage }}
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
docker-py:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-docker-py
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
curl -sSLf localhost:16686/api/traces?service=integration-test-client > /tmp/reports/jaeger-trace.json
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-docker-py/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-docker-py-${{ inputs.storage }}
path: /tmp/reports/*
retention-days: 1
integration-flaky:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration-flaky
env:
TEST_SKIP_INTEGRATION_CLI: 1
integration:
runs-on: ${{ matrix.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
strategy:
fail-fast: false
matrix:
os:
- ubuntu-20.04
- ubuntu-22.04
mode:
- ""
- rootless
- systemd
#- rootless-systemd FIXME: https://github.com/moby/moby/issues/44084
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Prepare
run: |
CACHE_DEV_SCOPE=dev
if [[ "${{ matrix.mode }}" == *"rootless"* ]]; then
echo "DOCKER_ROOTLESS=1" >> $GITHUB_ENV
fi
if [[ "${{ matrix.mode }}" == *"systemd"* ]]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
CACHE_DEV_SCOPE="${CACHE_DEV_SCOPE}systemd"
fi
echo "CACHE_DEV_SCOPE=${CACHE_DEV_SCOPE}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=${{ env.CACHE_DEV_SCOPE }}
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION_CLI: 1
TESTCOVERAGE: 1
-
name: Prepare reports
if: always()
run: |
reportsName=${{ matrix.os }}
if [ -n "${{ matrix.mode }}" ]; then
reportsName="$reportsName-${{ matrix.mode }}"
fi
reportsPath="/tmp/reports/$reportsName"
echo "TESTREPORTS_NAME=$reportsName" >> $GITHUB_ENV
mkdir -p bundles $reportsPath
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
curl -sSLf localhost:16686/api/traces?service=integration-test-client > $reportsPath/jaeger-trace.json
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration,${{ matrix.mode }}
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-integration-${{ inputs.storage }}-${{ env.TESTREPORTS_NAME }}
path: /tmp/reports/*
retention-days: 1
integration-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- integration
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: test-reports-integration-${{ inputs.storage }}-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
integration-cli-prepare:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Install gotestlist
run:
go install github.com/crazy-max/gotestlist/cmd/gotestlist@${{ env.GOTESTLIST_VERSION }}
-
name: Create matrix
id: tests
working-directory: ./integration-cli
run: |
# This step creates a matrix for integration-cli tests. Tests suites
# are distributed in integration-cli job through a matrix. There is
# also overrides being added to the matrix like "./..." to run
# "Test integration" step exclusively and specific tests suites that
# take a long time to run.
matrix="$(gotestlist -d ${{ env.ITG_CLI_MATRIX_SIZE }} -o "./..." -o "DockerSwarmSuite" -o "DockerNetworkSuite|DockerExternalVolumeSuite" ./...)"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.tests.outputs.matrix }}
integration-cli:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
needs:
- integration-cli-prepare
strategy:
fail-fast: false
matrix:
test: ${{ fromJson(needs.integration-cli-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION: 1
TESTCOVERAGE: 1
TESTFLAGS: "-test.run (${{ matrix.test }})/"
-
name: Prepare reports
if: always()
run: |
reportsName=$(echo -n "${{ matrix.test }}" | sha256sum | cut -d " " -f 1)
reportsPath=/tmp/reports/$reportsName
echo "TESTREPORTS_NAME=$reportsName" >> $GITHUB_ENV
mkdir -p bundles $reportsPath
echo "${{ matrix.test }}" | tr -s '|' '\n' | tee -a "$reportsPath/tests.txt"
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
curl -sSLf localhost:16686/api/traces?service=integration-test-client > $reportsPath/jaeger-trace.json
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration-cli
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-integration-cli-${{ inputs.storage }}-${{ env.TESTREPORTS_NAME }}
path: /tmp/reports/*
retention-days: 1
integration-cli-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- integration-cli
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: test-reports-integration-cli-${{ inputs.storage }}-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY

View File

@@ -3,21 +3,35 @@ name: .windows
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_call:
inputs:
os:
required: true
type: string
storage:
required: true
type: string
default: "graphdriver"
send_coverage:
required: false
type: boolean
default: false
env:
GO_VERSION: "1.20.4"
GO_VERSION: "1.22.12"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
TESTSTAT_VERSION: v0.1.25
WINDOWS_BASE_IMAGE: mcr.microsoft.com/windows/servercore
WINDOWS_BASE_TAG_2019: ltsc2019
WINDOWS_BASE_TAG_2022: ltsc2022
@@ -29,6 +43,7 @@ env:
jobs:
build:
runs-on: ${{ inputs.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -39,7 +54,7 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: ${{ env.GOPATH }}/src/github.com/docker/docker
-
@@ -58,7 +73,7 @@ jobs:
}
-
name: Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: |
~\AppData\Local\go-build
@@ -75,9 +90,12 @@ jobs:
-
name: Build base image
run: |
docker pull ${{ env.WINDOWS_BASE_IMAGE }}:${{ env.WINDOWS_BASE_IMAGE_TAG }}
docker tag ${{ env.WINDOWS_BASE_IMAGE }}:${{ env.WINDOWS_BASE_IMAGE_TAG }} microsoft/windowsservercore
docker build --build-arg GO_VERSION -t ${{ env.TEST_IMAGE_NAME }} -f Dockerfile.windows .
& docker build `
--build-arg WINDOWS_BASE_IMAGE `
--build-arg WINDOWS_BASE_IMAGE_TAG `
--build-arg GO_VERSION `
-t ${{ env.TEST_IMAGE_NAME }} `
-f Dockerfile.windows .
-
name: Build binaries
run: |
@@ -96,16 +114,16 @@ jobs:
docker cp "${{ env.TEST_CTN_NAME }}`:c`:\containerd\bin\containerd-shim-runhcs-v1.exe" ${{ env.BIN_OUT }}\
-
name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: build-${{ inputs.os }}
name: build-${{ inputs.storage }}-${{ inputs.os }}
path: ${{ env.BIN_OUT }}/*
if-no-files-found: error
retention-days: 2
unit-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120
timeout-minutes: 120 # guardrails timeout for the whole job
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -115,7 +133,7 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: ${{ env.GOPATH }}/src/github.com/docker/docker
-
@@ -135,7 +153,7 @@ jobs:
}
-
name: Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: |
~\AppData\Local\go-build
@@ -152,9 +170,12 @@ jobs:
-
name: Build base image
run: |
docker pull ${{ env.WINDOWS_BASE_IMAGE }}:${{ env.WINDOWS_BASE_IMAGE_TAG }}
docker tag ${{ env.WINDOWS_BASE_IMAGE }}:${{ env.WINDOWS_BASE_IMAGE_TAG }} microsoft/windowsservercore
docker build --build-arg GO_VERSION -t ${{ env.TEST_IMAGE_NAME }} -f Dockerfile.windows .
& docker build `
--build-arg WINDOWS_BASE_IMAGE `
--build-arg WINDOWS_BASE_IMAGE_TAG `
--build-arg GO_VERSION `
-t ${{ env.TEST_IMAGE_NAME }} `
-f Dockerfile.windows .
-
name: Test
run: |
@@ -166,36 +187,39 @@ jobs:
-
name: Send to Codecov
if: inputs.send_coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
working-directory: ${{ env.GOPATH }}\src\github.com\docker\docker
directory: bundles
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.os }}-unit-reports
name: ${{ inputs.os }}-${{ inputs.storage }}-unit-reports
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
retention-days: 1
unit-test-report:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
if: always()
needs:
- unit-test
steps:
-
name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download artifacts
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: ${{ inputs.os }}-unit-reports
name: ${{ inputs.os }}-${{ inputs.storage }}-unit-reports
path: /tmp/artifacts
-
name: Install teststat
@@ -204,19 +228,20 @@ jobs:
-
name: Create summary
run: |
teststat -markdown $(find /tmp/artifacts -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
find /tmp/artifacts -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
integration-test-prepare:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
@@ -241,17 +266,23 @@ jobs:
integration-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
needs:
- build
- integration-test-prepare
strategy:
fail-fast: false
matrix:
storage:
- ${{ inputs.storage }}
runtime:
- builtin
- containerd
test: ${{ fromJson(needs.integration-test-prepare.outputs.matrix) }}
exclude:
- storage: snapshotter
runtime: builtin
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -262,18 +293,28 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: ${{ env.GOPATH }}/src/github.com/docker/docker
-
name: Set up Jaeger
run: |
# Jaeger is set up on Linux through the setup-tracing action. If you update Jaeger here, don't forget to
# update the version set in .github/actions/setup-tracing/action.yml.
Invoke-WebRequest -Uri "https://github.com/jaegertracing/jaeger/releases/download/v1.46.0/jaeger-1.46.0-windows-amd64.tar.gz" -OutFile ".\jaeger-1.46.0-windows-amd64.tar.gz"
tar -zxvf ".\jaeger-1.46.0-windows-amd64.tar.gz"
Start-Process '.\jaeger-1.46.0-windows-amd64\jaeger-all-in-one.exe'
echo "OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
shell: pwsh
-
name: Env
run: |
Get-ChildItem Env: | Out-String
-
name: Download artifacts
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: build-${{ inputs.os }}
name: build-${{ inputs.storage }}-${{ inputs.os }}
path: ${{ env.BIN_OUT }}
-
name: Init
@@ -285,6 +326,9 @@ jobs:
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
Write-Output "${{ env.BIN_OUT }}" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
$testName = ([System.BitConverter]::ToString((New-Object System.Security.Cryptography.SHA256Managed).ComputeHash([System.Text.Encoding]::UTF8.GetBytes("${{ matrix.test }}"))) -replace '-').ToLower()
echo "TESTREPORTS_NAME=$testName" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
-
# removes docker service that is currently installed on the runner. we
# could use Uninstall-Package but not yet available on Windows runners.
@@ -342,6 +386,11 @@ jobs:
"--exec-root=$env:TEMP\moby-exec", `
"--pidfile=$env:TEMP\docker.pid", `
"--register-service"
If ("${{ inputs.storage }}" -eq "snapshotter") {
# Make the env-var visible to the service-managed dockerd, as there's no CLI flag for this option.
& reg add "HKLM\SYSTEM\CurrentControlSet\Services\docker" /v Environment /t REG_MULTI_SZ /s '@' /d TEST_INTEGRATION_USE_SNAPSHOTTER=1
echo "TEST_INTEGRATION_USE_SNAPSHOTTER=1" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
Write-Host "Starting service"
Start-Service -Name docker
Write-Host "Service started successfully!"
@@ -390,7 +439,7 @@ jobs:
DOCKER_HOST: npipe:////./pipe/docker_engine
-
name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
@@ -415,12 +464,13 @@ jobs:
-
name: Send to Codecov
if: inputs.send_coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
working-directory: ${{ env.GOPATH }}\src\github.com\docker\docker
directory: bundles
env_vars: RUNNER_OS
flags: integration,${{ matrix.runtime }}
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Docker info
run: |
@@ -458,37 +508,53 @@ jobs:
Sort-Object @{Expression="TimeCreated";Descending=$false} |
ForEach-Object {"$($_.TimeCreated.ToUniversalTime().ToString("o")) [$($_.LevelDisplayName)] $($_.Message)"} |
Tee-Object -file ".\bundles\daemon.log"
-
name: Download Jaeger traces
if: always()
run: |
Invoke-WebRequest `
-Uri "http://127.0.0.1:16686/api/traces?service=integration-test-client" `
-OutFile ".\bundles\jaeger-trace.json"
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.os }}-integration-reports-${{ matrix.runtime }}
name: ${{ inputs.os }}-${{ inputs.storage }}-integration-reports-${{ matrix.runtime }}-${{ env.TESTREPORTS_NAME }}
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
retention-days: 1
integration-test-report:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
if: always()
needs:
- integration-test
strategy:
fail-fast: false
matrix:
storage:
- ${{ inputs.storage }}
runtime:
- builtin
- containerd
exclude:
- storage: snapshotter
runtime: builtin
steps:
-
name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download artifacts
uses: actions/download-artifact@v3
name: Download reports
uses: actions/download-artifact@v4
with:
name: ${{ inputs.os }}-integration-reports-${{ matrix.runtime }}
path: /tmp/artifacts
path: /tmp/reports
pattern: ${{ inputs.os }}-${{ inputs.storage }}-integration-reports-${{ matrix.runtime }}-*
merge-multiple: true
-
name: Install teststat
run: |
@@ -496,4 +562,4 @@ jobs:
-
name: Create summary
run: |
teststat -markdown $(find /tmp/artifacts -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY

276
.github/workflows/arm64.yml vendored Normal file
View File

@@ -0,0 +1,276 @@
name: arm64
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
env:
GO_VERSION: "1.22.12"
TESTSTAT_VERSION: v0.1.25
DESTDIR: ./build
SETUP_BUILDX_VERSION: edge
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
DOCKER_EXPERIMENTAL: 1
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build:
runs-on: ubuntu-22.04-arm
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
strategy:
fail-fast: false
matrix:
target:
- binary
- dynbinary
steps:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v6
with:
targets: ${{ matrix.target }}
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
build-dev:
runs-on: ubuntu-22.04-arm
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
steps:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
*.cache-from=type=gha,scope=dev-arm64
*.cache-to=type=gha,scope=dev-arm64,mode=max
*.output=type=cacheonly
test-unit:
runs-on: ubuntu-22.04-arm
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev-arm64
-
name: Test
run: |
make -o build test-unit
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-unit-arm64-graphdriver
path: /tmp/reports/*
retention-days: 1
test-unit-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- test-unit
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: vendor.sum
-
name: Download reports
uses: actions/download-artifact@v4
with:
pattern: test-reports-unit-arm64-*
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
test-integration:
runs-on: ubuntu-22.04-arm
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ github.event_name != 'pull_request' }}
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v6
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev-arm64
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION_CLI: 1
TESTCOVERAGE: 1
-
name: Prepare reports
if: always()
run: |
reportsPath="/tmp/reports/arm64-graphdriver"
mkdir -p bundles $reportsPath
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
curl -sSLf localhost:16686/api/traces?service=integration-test-client > $reportsPath/jaeger-trace.json
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-integration-arm64-graphdriver
path: /tmp/reports/*
retention-days: 1
test-integration-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
continue-on-error: ${{ github.event_name != 'pull_request' }}
if: always()
needs:
- test-integration
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: vendor.sum
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: test-reports-integration-arm64-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY

215
.github/workflows/bin-image.yml vendored Normal file
View File

@@ -0,0 +1,215 @@
name: bin-image
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
tags:
- 'v*'
pull_request:
env:
MOBYBIN_REPO_SLUG: moby/moby-bin
DOCKER_GITCOMMIT: ${{ github.sha }}
VERSION: ${{ github.ref }}
PLATFORM: Moby Engine - Nightly
PRODUCT: moby-bin
PACKAGER_NAME: The Moby Project
SETUP_BUILDX_VERSION: latest
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
jobs:
validate-dco:
if: ${{ !startsWith(github.ref, 'refs/tags/v') }}
uses: ./.github/workflows/.dco.yml
prepare:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
outputs:
platforms: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.MOBYBIN_REPO_SLUG }}
### versioning strategy
## push semver tag v23.0.0
# moby/moby-bin:23.0.0
# moby/moby-bin:latest
## push semver prerelease tag v23.0.0-beta.1
# moby/moby-bin:23.0.0-beta.1
## push on master
# moby/moby-bin:master
## push on 23.0 branch
# moby/moby-bin:23.0
## any push
# moby/moby-bin:sha-ad132f5
tags: |
type=semver,pattern={{version}}
type=ref,event=branch
type=ref,event=pr
type=sha
-
name: Rename meta bake definition file
# see https://github.com/docker/metadata-action/issues/381#issuecomment-1918607161
run: |
bakeFile="${{ steps.meta.outputs.bake-file }}"
mv "${bakeFile#cwd://}" "/tmp/bake-meta.json"
-
name: Upload meta bake definition
uses: actions/upload-artifact@v4
with:
name: bake-meta
path: /tmp/bake-meta.json
if-no-files-found: error
retention-days: 1
-
name: Create platforms matrix
id: platforms
run: |
echo "matrix=$(docker buildx bake bin-image-cross --print | jq -cr '.target."bin-image-cross".platforms')" >>${GITHUB_OUTPUT}
build:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
- prepare
if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled')
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.platforms) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Download meta bake definition
uses: actions/download-artifact@v4
with:
name: bake-meta
path: /tmp
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Login to Docker Hub
if: github.event_name != 'pull_request' && github.repository == 'moby/moby'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_MOBYBIN_USERNAME }}
password: ${{ secrets.DOCKERHUB_MOBYBIN_TOKEN }}
-
name: Build
id: bake
uses: docker/bake-action@v6
with:
source: .
files: |
./docker-bake.hcl
/tmp/bake-meta.json
targets: bin-image
set: |
*.platform=${{ matrix.platform }}
*.output=type=image,name=${{ env.MOBYBIN_REPO_SLUG }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' && github.repository == 'moby/moby' }}
*.tags=
-
name: Export digest
if: github.event_name != 'pull_request' && github.repository == 'moby/moby'
run: |
mkdir -p /tmp/digests
digest="${{ fromJSON(steps.bake.outputs.metadata)['bin-image']['containerimage.digest'] }}"
touch "/tmp/digests/${digest#sha256:}"
-
name: Upload digest
if: github.event_name != 'pull_request' && github.repository == 'moby/moby'
uses: actions/upload-artifact@v4
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- build
if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') && github.event_name != 'pull_request' && github.repository == 'moby/moby'
steps:
-
name: Download meta bake definition
uses: actions/download-artifact@v4
with:
name: bake-meta
path: /tmp
-
name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_MOBYBIN_USERNAME }}
password: ${{ secrets.DOCKERHUB_MOBYBIN_TOKEN }}
-
name: Create manifest list and push
working-directory: /tmp/digests
run: |
set -x
docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map("-t " + .) | join(" ")' /tmp/bake-meta.json) \
$(printf '${{ env.MOBYBIN_REPO_SLUG }}@sha256:%s ' *)
-
name: Inspect image
run: |
set -x
docker buildx imagetools inspect ${{ env.MOBYBIN_REPO_SLUG }}:$(jq -cr '.target."docker-metadata-action".args.DOCKER_META_VERSION' /tmp/bake-meta.json)

View File

@@ -1,5 +1,14 @@
name: buildkit
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,10 +19,14 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
env:
GO_VERSION: "1.22.12"
DESTDIR: ./build
SETUP_BUILDX_VERSION: latest
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
jobs:
validate-dco:
@@ -21,23 +34,25 @@ jobs:
build:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v6
with:
targets: binary
-
name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: binary
path: ${{ env.DESTDIR }}
@@ -46,11 +61,12 @@ jobs:
test:
runs-on: ubuntu-20.04
timeout-minutes: 120
env:
BUILDKIT_REPO: moby/buildkit
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- build
env:
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildkit-tests"
strategy:
fail-fast: false
matrix:
@@ -74,42 +90,66 @@ jobs:
disabledFeatures="${disabledFeatures},merge_diff"
fi
echo "BUILDKIT_TEST_DISABLE_FEATURES=${disabledFeatures}" >> $GITHUB_ENV
# Expose `ACTIONS_RUNTIME_TOKEN` and `ACTIONS_CACHE_URL`, which is used
# in BuildKit's test suite to skip/unskip cache exporters:
# https://github.com/moby/buildkit/blob/567a99433ca23402d5e9b9f9124005d2e59b8861/client/client_test.go#L5407-L5411
-
name: Expose GitHub Runtime
uses: crazy-max/ghaction-github-runtime@v3
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: moby
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: BuildKit ref
run: |
echo "BUILDKIT_REF=$(./hack/buildkit-ref)" >> $GITHUB_ENV
echo "$(./hack/buildkit-ref)" >> $GITHUB_ENV
working-directory: moby
-
name: Checkout BuildKit ${{ env.BUILDKIT_REF }}
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: ${{ env.BUILDKIT_REPO }}
ref: ${{ env.BUILDKIT_REF }}
path: buildkit
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Download binary artifacts
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: binary
path: ./buildkit/build/moby/
-
name: Update daemon.json
run: |
sudo rm /etc/docker/daemon.json
sudo rm -f /etc/docker/daemon.json
sudo service docker restart
docker version
docker info
-
name: Build test image
uses: docker/bake-action@v6
with:
source: .
workdir: ./buildkit
targets: integration-tests
set: |
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
run: |
@@ -119,6 +159,5 @@ jobs:
TEST_DOCKERD: "1"
TEST_DOCKERD_BINARY: "./build/moby/dockerd"
TESTPKGS: "./${{ matrix.pkg }}"
# Skip buildkit tests checking the digest (see https://github.com/moby/buildkit/pull/3736)
TESTFLAGS: "-v --parallel=1 --timeout=30m --run=/^Test([^R]|.[^e]|..[^p]|...[^r]|....[^o]|.....[^S])/worker=${{ matrix.worker }}$"
TESTFLAGS: "-v --parallel=1 --timeout=30m --run=//worker=${{ matrix.worker }}$"
working-directory: buildkit

View File

@@ -1,5 +1,14 @@
name: ci
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,12 +19,13 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
tags:
- 'v*'
- '[0-9]+.x'
pull_request:
env:
DESTDIR: ./build
SETUP_BUILDX_VERSION: latest
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
jobs:
validate-dco:
@@ -23,6 +33,7 @@ jobs:
build:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
strategy:
@@ -32,17 +43,16 @@ jobs:
- binary
- dynbinary
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v6
with:
targets: ${{ matrix.target }}
-
@@ -53,17 +63,10 @@ jobs:
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.target }}
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 7
prepare-cross:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
outputs:
@@ -71,7 +74,7 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
@@ -85,6 +88,7 @@ jobs:
cross:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- validate-dco
- prepare-cross
@@ -93,11 +97,6 @@ jobs:
matrix:
platform: ${{ fromJson(needs.prepare-cross.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
-
name: Prepare
run: |
@@ -105,10 +104,14 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v6
with:
targets: all
set: |
@@ -121,11 +124,33 @@ jobs:
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
govulncheck:
runs-on: ubuntu-24.04
timeout-minutes: 120 # guardrails timeout for the whole job
permissions:
# required to write sarif report
security-events: write
# required to check out the repository
contents: read
steps:
-
name: Upload artifacts
uses: actions/upload-artifact@v3
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
name: cross-${{ env.PLATFORM_PAIR }}
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 7
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Run
uses: docker/bake-action@v6
with:
targets: govulncheck
env:
GOVULNCHECK_FORMAT: sarif
-
name: Upload SARIF report
if: ${{ github.event_name != 'pull_request' && github.repository == 'moby/moby' }}
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ${{ env.DESTDIR }}/govulncheck.out

71
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,71 @@
name: codeql
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
tags:
- 'v*'
pull_request:
# The branches below must be a subset of the branches above
branches: ["master"]
schedule:
# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday)
# │ │ │ │ │
# │ │ │ │ │
# │ │ │ │ │
# * * * * *
- cron: '0 9 * * 4'
jobs:
codeql:
runs-on: ubuntu-24.04
timeout-minutes: 10
permissions:
actions: read
contents: read
security-events: write
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 2
# CodeQL 2.16.4's auto-build added support for multi-module repositories,
# and is trying to be smart by searching for modules in every directory,
# including vendor directories. If no module is found, it's creating one
# which is ... not what we want, so let's give it a "go.mod".
# see: https://github.com/docker/cli/pull/4944#issuecomment-2002034698
- name: Create go.mod
run: |
ln -s vendor.mod go.mod
ln -s vendor.sum go.sum
- name: Update Go
uses: actions/setup-go@v5
with:
go-version: "1.22.12"
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: go
- name: Autobuild
uses: github/codeql-action/autobuild@v3
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:go"

View File

@@ -1,5 +1,14 @@
name: test
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,17 +19,15 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
tags:
- 'v*'
- '[0-9]+.x'
pull_request:
env:
GO_VERSION: "1.20.4"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
ITG_CLI_MATRIX_SIZE: 6
DOCKER_EXPERIMENTAL: 1
DOCKER_GRAPHDRIVER: overlay2
GO_VERSION: "1.22.12"
GIT_PAGER: "cat"
PAGER: "cat"
SETUP_BUILDX_VERSION: latest
SETUP_BUILDKIT_IMAGE: moby/buildkit:latest
jobs:
validate-dco:
@@ -28,6 +35,7 @@ jobs:
build-dev:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
needs:
- validate-dco
strategy:
@@ -43,15 +51,16 @@ jobs:
if [ "${{ matrix.mode }}" = "systemd" ]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
fi
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v2
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -59,8 +68,24 @@ jobs:
*.cache-to=type=gha,scope=dev${{ matrix.mode }},mode=max
*.output=type=cacheonly
test:
needs:
- build-dev
- validate-dco
uses: ./.github/workflows/.test.yml
secrets: inherit
strategy:
fail-fast: false
matrix:
storage:
- graphdriver
- snapshotter
with:
storage: ${{ matrix.storage }}
validate-prepare:
runs-on: ubuntu-20.04
timeout-minutes: 10 # guardrails timeout for the whole job
needs:
- validate-dco
outputs:
@@ -68,12 +93,12 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Create matrix
id: scripts
run: |
scripts=$(jq -ncR '[inputs]' <<< "$(ls -I .validate -I all -I default -I dco -I golangci-lint.yml -I yamllint.yaml -A ./hack/validate/)")
scripts=$(cd ./hack/validate && jq -nc '$ARGS.positional - ["all", "default", "dco"] | map(select(test("[.]")|not)) + ["generate-files"]' --args *)
echo "matrix=$scripts" >> $GITHUB_OUTPUT
-
name: Show matrix
@@ -82,7 +107,7 @@ jobs:
validate:
runs-on: ubuntu-20.04
timeout-minutes: 120
timeout-minutes: 30 # guardrails timeout for the whole job
needs:
- validate-prepare
- build-dev
@@ -93,7 +118,7 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: 0
-
@@ -101,10 +126,14 @@ jobs:
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build dev image
uses: docker/bake-action@v2
uses: docker/bake-action@v6
with:
targets: dev
set: |
@@ -114,400 +143,9 @@ jobs:
run: |
make -o build validate-${{ matrix.script }}
unit:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-unit
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Send to Codecov
uses: codecov/codecov-action@v3
with:
directory: ./bundles
env_vars: RUNNER_OS
flags: unit
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: unit-reports
path: /tmp/reports/*
unit-report:
runs-on: ubuntu-20.04
if: always()
needs:
- unit
steps:
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v3
with:
name: unit-reports
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
docker-py:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-docker-py
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-docker-py/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: docker-py-reports
path: /tmp/reports/*
integration-flaky:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration-flaky
env:
TEST_SKIP_INTEGRATION_CLI: 1
integration:
runs-on: ${{ matrix.os }}
timeout-minutes: 120
needs:
- build-dev
strategy:
fail-fast: false
matrix:
os:
- ubuntu-20.04
- ubuntu-22.04
mode:
- ""
- rootless
- systemd
#- rootless-systemd FIXME: https://github.com/moby/moby/issues/44084
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Prepare
run: |
CACHE_DEV_SCOPE=dev
if [[ "${{ matrix.mode }}" == *"rootless"* ]]; then
echo "DOCKER_ROOTLESS=1" >> $GITHUB_ENV
fi
if [[ "${{ matrix.mode }}" == *"systemd"* ]]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
CACHE_DEV_SCOPE="${CACHE_DEV_SCOPE}systemd"
fi
echo "CACHE_DEV_SCOPE=${CACHE_DEV_SCOPE}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=${{ env.CACHE_DEV_SCOPE }}
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION_CLI: 1
TESTCOVERAGE: 1
-
name: Prepare reports
if: always()
run: |
reportsPath="/tmp/reports/${{ matrix.os }}"
if [ -n "${{ matrix.mode }}" ]; then
reportsPath="$reportsPath-${{ matrix.mode }}"
fi
mkdir -p bundles $reportsPath
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
-
name: Send to Codecov
uses: codecov/codecov-action@v3
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration,${{ matrix.mode }}
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: integration-reports
path: /tmp/reports/*
integration-report:
runs-on: ubuntu-20.04
if: always()
needs:
- integration
steps:
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v3
with:
name: integration-reports
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
integration-cli-prepare:
runs-on: ubuntu-20.04
needs:
- validate-dco
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Install gotestlist
run:
go install github.com/crazy-max/gotestlist/cmd/gotestlist@${{ env.GOTESTLIST_VERSION }}
-
name: Create matrix
id: tests
working-directory: ./integration-cli
run: |
# This step creates a matrix for integration-cli tests. Tests suites
# are distributed in integration-cli job through a matrix. There is
# also overrides being added to the matrix like "./..." to run
# "Test integration" step exclusively and specific tests suites that
# take a long time to run.
matrix="$(gotestlist -d ${{ env.ITG_CLI_MATRIX_SIZE }} -o "./..." -o "DockerSwarmSuite" -o "DockerNetworkSuite|DockerExternalVolumeSuite" ./...)"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.tests.outputs.matrix }}
integration-cli:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
- integration-cli-prepare
strategy:
fail-fast: false
matrix:
test: ${{ fromJson(needs.integration-cli-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION: 1
TESTCOVERAGE: 1
TESTFLAGS: "-test.run (${{ matrix.test }})/"
-
name: Prepare reports
if: always()
run: |
reportsPath=/tmp/reports/$(echo -n "${{ matrix.test }}" | sha256sum | cut -d " " -f 1)
mkdir -p bundles $reportsPath
echo "${{ matrix.test }}" | tr -s '|' '\n' | tee -a "$reportsPath/tests.txt"
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
-
name: Send to Codecov
uses: codecov/codecov-action@v3
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration-cli
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: integration-cli-reports
path: /tmp/reports/*
integration-cli-report:
runs-on: ubuntu-20.04
if: always()
needs:
- integration-cli
steps:
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v3
with:
name: integration-cli-reports
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
prepare-smoke:
smoke-prepare:
runs-on: ubuntu-20.04
timeout-minutes: 10 # guardrails timeout for the whole job
needs:
- validate-dco
outputs:
@@ -515,7 +153,7 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
@@ -529,16 +167,14 @@ jobs:
smoke:
runs-on: ubuntu-20.04
timeout-minutes: 20 # guardrails timeout for the whole job
needs:
- prepare-smoke
- smoke-prepare
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare-smoke.outputs.matrix) }}
platform: ${{ fromJson(needs.smoke-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Prepare
run: |
@@ -546,13 +182,17 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Test
uses: docker/bake-action@v2
uses: docker/bake-action@v6
with:
targets: binary-smoketest
set: |

88
.github/workflows/validate-pr.yml vendored Normal file
View File

@@ -0,0 +1,88 @@
name: validate-pr
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
pull_request:
types: [opened, edited, labeled, unlabeled, synchronize]
jobs:
check-area-label:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
steps:
- name: Missing `area/` label
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/') && !contains(join(github.event.pull_request.labels.*.name, ','), 'area/')
run: |
echo "::error::Every PR with an 'impact/*' label should also have an 'area/*' label"
exit 1
- name: OK
run: exit 0
check-changelog:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
env:
HAS_IMPACT_LABEL: ${{ contains(join(github.event.pull_request.labels.*.name, ','), 'impact/') }}
PR_BODY: |
${{ github.event.pull_request.body }}
steps:
- name: Check changelog description
run: |
# Extract the `markdown changelog` note code block
block=$(echo -n "$PR_BODY" | tr -d '\r' | awk '/^```markdown changelog$/{flag=1;next}/^```$/{flag=0}flag')
# Strip empty lines
desc=$(echo "$block" | awk NF)
if [ "$HAS_IMPACT_LABEL" = "true" ]; then
if [ -z "$desc" ]; then
echo "::error::Changelog section is empty. Please provide a description for the changelog."
exit 1
fi
len=$(echo -n "$desc" | wc -c)
if [[ $len -le 6 ]]; then
echo "::error::Description looks too short: $desc"
exit 1
fi
else
if [ -n "$desc" ]; then
echo "::error::PR has a changelog description, but no changelog label"
echo "::error::Please add the relevant 'impact/' label to the PR or remove the changelog description"
exit 1
fi
fi
echo "This PR will be included in the release notes with the following note:"
echo "$desc"
check-pr-branch:
runs-on: ubuntu-20.04
timeout-minutes: 120 # guardrails timeout for the whole job
env:
PR_TITLE: ${{ github.event.pull_request.title }}
steps:
# Backports or PR that target a release branch directly should mention the target branch in the title, for example:
# [X.Y backport] Some change that needs backporting to X.Y
# [X.Y] Change directly targeting the X.Y branch
- name: Check release branch
id: title_branch
run: |
# get the intended major version prefix ("[27.1 backport]" -> "27.") from the PR title.
[[ "$PR_TITLE" =~ ^\[([0-9]*\.)[^]]*\] ]] && branch="${BASH_REMATCH[1]}"
# get major version prefix from the release branch ("27.x -> "27.")
[[ "$GITHUB_BASE_REF" =~ ^([0-9]*\.) ]] && target_branch="${BASH_REMATCH[1]}" || target_branch="$GITHUB_BASE_REF"
if [[ "$target_branch" != "$branch" ]] && ! [[ "$GITHUB_BASE_REF" == "master" && "$branch" == "" ]]; then
echo "::error::PR is opened against the $GITHUB_BASE_REF branch, but its title suggests otherwise."
exit 1
fi

View File

@@ -1,5 +1,14 @@
name: windows-2019
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -13,10 +22,21 @@ jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
run:
test-prepare:
uses: ./.github/workflows/.test-prepare.yml
needs:
- validate-dco
run:
needs:
- test-prepare
uses: ./.github/workflows/.windows.yml
secrets: inherit
strategy:
fail-fast: false
matrix:
storage: ${{ fromJson(needs.test-prepare.outputs.matrix) }}
with:
os: windows-2019
storage: ${{ matrix.storage }}
send_coverage: false

View File

@@ -1,5 +1,14 @@
name: windows-2022
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -10,16 +19,28 @@ on:
branches:
- 'master'
- '[0-9]+.[0-9]+'
- '[0-9]+.x'
pull_request:
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
run:
test-prepare:
uses: ./.github/workflows/.test-prepare.yml
needs:
- validate-dco
run:
needs:
- test-prepare
uses: ./.github/workflows/.windows.yml
secrets: inherit
strategy:
fail-fast: false
matrix:
storage: ${{ fromJson(needs.test-prepare.outputs.matrix) }}
with:
os: windows-2022
storage: ${{ matrix.storage }}
send_coverage: true

188
.golangci.yml Normal file
View File

@@ -0,0 +1,188 @@
linters:
enable:
- depguard
- dupword # Checks for duplicate words in the source code.
- goimports
- gosec
- gosimple
- govet
- forbidigo
- importas
- ineffassign
- misspell
- revive
- staticcheck
- typecheck
- unconvert
- unused
disable:
- errcheck
run:
concurrency: 2
modules-download-mode: vendor
skip-dirs:
- docs
linters-settings:
dupword:
ignore:
- "true" # some tests use this as expected output
- "false" # some tests use this as expected output
- "root" # for tests using "ls" output with files owned by "root:root"
forbidigo:
forbid:
- pkg: github.com/vishvananda/netlink$
p: ^netlink\.(Handle\.)?(AddrList|BridgeVlanList|ChainList|ClassList|ConntrackTableList|ConntrackDeleteFilter$|ConntrackDeleteFilters|DevLinkGetDeviceList|DevLinkGetAllPortList|DevlinkGetDeviceParams|FilterList|FouList|GenlFamilyList|GTPPDPList|LinkByName|LinkByAlias|LinkList|LinkSubscribeWithOptions|NeighList$|NeighProxyList|NeighListExecute|NeighSubscribeWithOptions|LinkGetProtinfo|QdiscList|RdmaLinkList|RdmaLinkByName|RdmaLinkDel|RouteList|RouteListFilteredIter|RuleListFiltered$|RouteSubscribeWithOptions|RuleList$|RuleListFiltered|SocketGet|SocketDiagTCPInfo|SocketDiagTCP|SocketDiagUDPInfo|SocketDiagUDP|UnixSocketDiagInfo|UnixSocketDiag|VDPAGetDevConfigList|VDPAGetDevList|VDPAGetMGMTDevList|XfrmPolicyList|XfrmStateList)
msg: Use internal nlwrap package for EINTR handling.
- pkg: github.com/docker/docker/internal/nlwrap$
p: ^nlwrap.Handle.(BridgeVlanList|ChainList|ClassList|ConntrackDeleteFilter$|DevLinkGetDeviceList|DevLinkGetAllPortList|DevlinkGetDeviceParams|FilterList|FouList|GenlFamilyList|GTPPDPList|LinkByAlias|LinkSubscribeWithOptions|NeighList$|NeighProxyList|NeighListExecute|NeighSubscribeWithOptions|LinkGetProtinfo|QdiscList|RdmaLinkList|RdmaLinkByName|RdmaLinkDel|RouteListFilteredIter|RuleListFiltered$|RouteSubscribeWithOptions|RuleList$|RuleListFiltered|SocketGet|SocketDiagTCPInfo|SocketDiagTCP|SocketDiagUDPInfo|SocketDiagUDP|UnixSocketDiagInfo|UnixSocketDiag|VDPAGetDevConfigList|VDPAGetDevList|VDPAGetMGMTDevList)
msg: Add a wrapper to nlwrap.Handle for EINTR handling and update the list in .golangci.yml.
analyze-types: true
importas:
# Do not allow unaliased imports of aliased packages.
no-unaliased: true
alias:
# Enforce alias to prevent it accidentally being used instead of our
# own errdefs package (or vice-versa).
- pkg: github.com/containerd/errdefs
alias: cerrdefs
- pkg: github.com/opencontainers/image-spec/specs-go/v1
alias: ocispec
govet:
check-shadowing: false
gosec:
excludes:
- G115 # FIXME temporarily suppress 'G115: integer overflow conversion': it produces many hits, some of which may be false positives, and need to be looked at; see https://github.com/moby/moby/issues/48358
depguard:
rules:
main:
deny:
- pkg: io/ioutil
desc: The io/ioutil package has been deprecated, see https://go.dev/doc/go1.16#ioutil
- pkg: "github.com/stretchr/testify/assert"
desc: Use "gotest.tools/v3/assert" instead
- pkg: "github.com/stretchr/testify/require"
desc: Use "gotest.tools/v3/assert" instead
- pkg: "github.com/stretchr/testify/suite"
desc: Do not use
- pkg: "github.com/containerd/containerd/errdefs"
desc: The errdefs package has moved to a separate module, https://github.com/containerd/errdefs
- pkg: "github.com/containerd/containerd/log"
desc: The logs package has moved to a separate module, https://github.com/containerd/log
- pkg: "github.com/containerd/containerd/pkg/userns"
desc: Use github.com/moby/sys/userns instead.
- pkg: "github.com/opencontainers/runc/libcontainer/userns"
desc: Use github.com/moby/sys/userns instead.
- pkg: "github.com/tonistiigi/fsutil"
desc: The fsutil module does not have a stable API, so we should not have a direct dependency unless necessary.
revive:
rules:
# FIXME make sure all packages have a description. Currently, there's many packages without.
- name: package-comments
disabled: true
issues:
# The default exclusion rules are a bit too permissive, so copying the relevant ones below
exclude-use-default: false
exclude-rules:
# We prefer to use an "exclude-list" so that new "default" exclusions are not
# automatically inherited. We can decide whether or not to follow upstream
# defaults when updating golang-ci-lint versions.
# Unfortunately, this means we have to copy the whole exclusion pattern, as
# (unlike the "include" option), the "exclude" option does not take exclusion
# ID's.
#
# These exclusion patterns are copied from the default excludes at:
# https://github.com/golangci/golangci-lint/blob/v1.46.2/pkg/config/issues.go#L10-L104
# EXC0001
- text: "Error return value of .((os\\.)?std(out|err)\\..*|.*Close|.*Flush|os\\.Remove(All)?|.*print(f|ln)?|os\\.(Un)?Setenv). is not checked"
linters:
- errcheck
# EXC0006
- text: "Use of unsafe calls should be audited"
linters:
- gosec
# EXC0007
- text: "Subprocess launch(ed with variable|ing should be audited)"
linters:
- gosec
# EXC0008
# TODO: evaluate these and fix where needed: G307: Deferring unsafe method "*os.File" on type "Close" (gosec)
- text: "(G104|G307)"
linters:
- gosec
# EXC0009
- text: "(Expect directory permissions to be 0750 or less|Expect file permissions to be 0600 or less)"
linters:
- gosec
# EXC0010
- text: "Potential file inclusion via variable"
linters:
- gosec
# Looks like the match in "EXC0007" above doesn't catch this one
# TODO: consider upstreaming this to golangci-lint's default exclusion rules
- text: "G204: Subprocess launched with a potential tainted input or cmd arguments"
linters:
- gosec
# Looks like the match in "EXC0009" above doesn't catch this one
# TODO: consider upstreaming this to golangci-lint's default exclusion rules
- text: "G306: Expect WriteFile permissions to be 0600 or less"
linters:
- gosec
# Exclude some linters from running on tests files.
- path: _test\.go
linters:
- errcheck
- gosec
# Suppress golint complaining about generated types in api/types/
- text: "type name will be used as (container|volume)\\.(Container|Volume).* by other packages, and that stutters; consider calling this"
path: "api/types/(volume|container)/"
linters:
- revive
# FIXME temporarily suppress these until we migrated these to internal.
- text: "SA1019: fileutils\\.GetTotalUsedFds"
linters:
- staticcheck
# FIXME temporarily suppress these (see https://github.com/gotestyourself/gotest.tools/issues/272)
- text: "SA1019: (assert|cmp|is)\\.ErrorType is deprecated"
linters:
- staticcheck
# FIXME temporarily suppress these until https://github.com/moby/moby/pull/49072 is merged, which removes their use.
- text: "SA1019: system\\.(FromStatT|Mkdev|Mknod|StatT)"
path: "pkg/archive/"
linters:
- staticcheck
# FIXME temporarily suppress these until they are moved internal to container/streams.
- text: "SA1019: ioutils\\.(ErrClosed|BytesPipe|NewBytesPipe)"
path: "container/stream/"
linters:
- staticcheck
- text: "ineffectual assignment to ctx"
source: "ctx[, ].*=.*\\(ctx[,)]"
linters:
- ineffassign
- text: "SA4006: this value of `ctx` is never used"
source: "ctx[, ].*=.*\\(ctx[,)]"
linters:
- staticcheck
# Maximum issues count per one linter. Set to 0 to disable. Default is 50.
max-issues-per-linter: 0
# Maximum count of issues with the same text. Set to 0 to disable. Default is 3.
max-same-issues: 0

View File

@@ -7,6 +7,7 @@
#
# For an explanation of this file format, consult gitmailmap(5).
Aaron Yoshitake <airandfingers@gmail.com>
Aaron L. Xu <liker.xu@foxmail.com>
Aaron L. Xu <liker.xu@foxmail.com> <likexu@harmonycloud.cn>
Aaron Lehmann <alehmann@netflix.com>
@@ -30,7 +31,10 @@ Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp> <suda.akihiro@lab.ntt.co.jp>
Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp> <suda.kyoto@gmail.com>
Akshay Moghe <akshay.moghe@gmail.com>
Alano Terblanche <alano.terblanche@docker.com>
Alano Terblanche <alano.terblanche@docker.com> <18033717+Benehiko@users.noreply.github.com>
Albin Kerouanton <albinker@gmail.com>
Albin Kerouanton <albinker@gmail.com> <557933+akerouanton@users.noreply.github.com>
Albin Kerouanton <albinker@gmail.com> <albin@akerouanton.name>
Aleksa Sarai <asarai@suse.de>
Aleksa Sarai <asarai@suse.de> <asarai@suse.com>
@@ -58,6 +62,8 @@ Allen Sun <allensun.shl@alibaba-inc.com> <allen.sun@daocloud.io>
Allen Sun <allensun.shl@alibaba-inc.com> <shlallen1990@gmail.com>
Anca Iordache <anca.iordache@docker.com>
Andrea Denisse Gómez <crypto.andrea@protonmail.ch>
Andrew Baxter <423qpsxzhh8k3h@s.rendaw.me>
Andrew Baxter <423qpsxzhh8k3h@s.rendaw.me> andrew <>
Andrew Kim <taeyeonkim90@gmail.com>
Andrew Kim <taeyeonkim90@gmail.com> <akim01@fortinet.com>
Andrew Weiss <andrew.weiss@docker.com> <andrew.weiss@microsoft.com>
@@ -103,6 +109,9 @@ Bily Zhang <xcoder@tenxcloud.com>
Bin Liu <liubin0329@gmail.com>
Bin Liu <liubin0329@gmail.com> <liubin0329@users.noreply.github.com>
Bingshen Wang <bingshen.wbs@alibaba-inc.com>
Bjorn Neergaard <bjorn@neersighted.com>
Bjorn Neergaard <bjorn@neersighted.com> <bjorn.neergaard@docker.com>
Bjorn Neergaard <bjorn@neersighted.com> <bneergaard@mirantis.com>
Boaz Shuster <ripcurld.github@gmail.com>
Bojun Zhu <bojun.zhu@foxmail.com>
Boqin Qin <bobbqqin@gmail.com>
@@ -115,6 +124,7 @@ Brian Goff <cpuguy83@gmail.com> <bgoff@cpuguy83-mbp.home>
Brian Goff <cpuguy83@gmail.com> <bgoff@cpuguy83-mbp.local>
Brian Goff <cpuguy83@gmail.com> <brian.goff@microsoft.com>
Brian Goff <cpuguy83@gmail.com> <cpuguy@hey.com>
Calvin Liu <flycalvin@qq.com>
Cameron Sparr <gh@sparr.email>
Carlos de Paula <me@carlosedp.com>
Chander Govindarajan <chandergovind@gmail.com>
@@ -126,6 +136,7 @@ Chen Mingjie <chenmingjie0828@163.com>
Chen Qiu <cheney-90@hotmail.com>
Chen Qiu <cheney-90@hotmail.com> <21321229@zju.edu.cn>
Chengfei Shang <cfshang@alauda.io>
Chentianze <cmoman@126.com>
Chris Dias <cdias@microsoft.com>
Chris McKinnel <chris.mckinnel@tangentlabs.co.uk>
Chris Price <cprice@mirantis.com>
@@ -134,6 +145,8 @@ Chris Telfer <ctelfer@docker.com>
Chris Telfer <ctelfer@docker.com> <ctelfer@users.noreply.github.com>
Christopher Biscardi <biscarch@sketcht.com>
Christopher Latham <sudosurootdev@gmail.com>
Christopher Petito <chrisjpetito@gmail.com>
Christopher Petito <chrisjpetito@gmail.com> <47751006+krissetto@users.noreply.github.com>
Christy Norman <christy@linux.vnet.ibm.com>
Chun Chen <ramichen@tencent.com> <chenchun.feed@gmail.com>
Corbin Coleman <corbin.coleman@docker.com>
@@ -141,6 +154,8 @@ Cristian Ariza <dev@cristianrz.com>
Cristian Staretu <cristian.staretu@gmail.com>
Cristian Staretu <cristian.staretu@gmail.com> <unclejack@users.noreply.github.com>
Cristian Staretu <cristian.staretu@gmail.com> <unclejacksons@gmail.com>
cui fliter <imcusg@gmail.com>
cui fliter <imcusg@gmail.com> cuishuang <imcusg@gmail.com>
CUI Wei <ghostplant@qq.com> cuiwei13 <cuiwei13@pku.edu.cn>
Daehyeok Mun <daehyeok@gmail.com>
Daehyeok Mun <daehyeok@gmail.com> <daehyeok@daehyeok-ui-MacBook-Air.local>
@@ -167,6 +182,8 @@ Dattatraya Kumbhar <dattatraya.kumbhar@gslab.com>
Dave Goodchild <buddhamagnet@gmail.com>
Dave Henderson <dhenderson@gmail.com> <Dave.Henderson@ca.ibm.com>
Dave Tucker <dt@docker.com> <dave@dtucker.co.uk>
David Dooling <dooling@gmail.com>
David Dooling <dooling@gmail.com> <david.dooling@docker.com>
David M. Karr <davidmichaelkarr@gmail.com>
David Sheets <dsheets@docker.com> <sheets@alum.mit.edu>
David Sissitka <me@dsissitka.com>
@@ -213,6 +230,8 @@ Felix Hupfeld <felix@quobyte.com> <quofelix@users.noreply.github.com>
Felix Ruess <felix.ruess@gmail.com> <felix.ruess@roboception.de>
Feng Yan <fy2462@gmail.com>
Fengtu Wang <wangfengtu@huawei.com> <wangfengtu@huawei.com>
Filipe Pina <hzlu1ot0@duck.com>
Filipe Pina <hzlu1ot0@duck.com> <636320+fopina@users.noreply.github.com>
Francisco Carriedo <fcarriedo@gmail.com>
Frank Rosquin <frank.rosquin+github@gmail.com> <frank.rosquin@gmail.com>
Frank Yang <yyb196@gmail.com>
@@ -264,6 +283,7 @@ Hollie Teal <hollie@docker.com> <hollie.teal@docker.com>
Hollie Teal <hollie@docker.com> <hollietealok@users.noreply.github.com>
hsinko <21551195@zju.edu.cn> <hsinko@users.noreply.github.com>
Hu Keping <hukeping@huawei.com>
Huajin Tong <fliterdashen@gmail.com>
Hui Kang <hkang.sunysb@gmail.com>
Hui Kang <hkang.sunysb@gmail.com> <kangh@us.ibm.com>
Huu Nguyen <huu@prismskylabs.com> <whoshuu@gmail.com>
@@ -330,6 +350,8 @@ John Howard <github@lowenna.com> <john.howard@microsoft.com>
John Howard <github@lowenna.com> <john@lowenna.com>
John Stephens <johnstep@docker.com> <johnstep@users.noreply.github.com>
Jon Surrell <jon.surrell@gmail.com> <jon.surrell@automattic.com>
Jonathan A. Sternberg <jonathansternberg@gmail.com>
Jonathan A. Sternberg <jonathansternberg@gmail.com> <jonathan.sternberg@docker.com>
Jonathan Choy <jonathan.j.choy@gmail.com>
Jonathan Choy <jonathan.j.choy@gmail.com> <oni@tetsujinlabs.com>
Jordan Arentsen <blissdev@gmail.com>
@@ -369,7 +391,9 @@ Ken Cochrane <kencochrane@gmail.com> <KenCochrane@gmail.com>
Ken Herner <kherner@progress.com> <chosenken@gmail.com>
Ken Reese <krrgithub@gmail.com>
Kenfe-Mickaël Laventure <mickael.laventure@gmail.com>
Kevin Alvarez <crazy-max@users.noreply.github.com>
Kevin Alvarez <github@crazymax.dev>
Kevin Alvarez <github@crazymax.dev> <1951866+crazy-max@users.noreply.github.com>
Kevin Alvarez <github@crazymax.dev> <crazy-max@users.noreply.github.com>
Kevin Feyrer <kevin.feyrer@btinternet.com> <kevinfeyrer@users.noreply.github.com>
Kevin Kern <kaiwentan@harmonycloud.cn>
Kevin Meredith <kevin.m.meredith@gmail.com>
@@ -469,10 +493,15 @@ Mikael Davranche <mikael.davranche@corp.ovh.com>
Mikael Davranche <mikael.davranche@corp.ovh.com> <mikael.davranche@corp.ovh.net>
Mike Casas <mkcsas0@gmail.com> <mikecasas@users.noreply.github.com>
Mike Goelzer <mike.goelzer@docker.com> <mgoelzer@docker.com>
Milas Bowman <devnull@milas.dev>
Milas Bowman <devnull@milas.dev> <milas.bowman@docker.com>
Milas Bowman <devnull@milas.dev> <milasb@gmail.com>
Milind Chawre <milindchawre@gmail.com>
Misty Stanley-Jones <misty@docker.com> <misty@apache.org>
Mohammad Banikazemi <MBanikazemi@gmail.com>
Mohammad Banikazemi <MBanikazemi@gmail.com> <mb@us.ibm.com>
Mohd Sadiq <mohdsadiq058@gmail.com> <42430865+msadiq058@users.noreply.github.com>
Mohd Sadiq <mohdsadiq058@gmail.com> <mohdsadiq058@gmail.com>
Mohit Soni <mosoni@ebay.com> <mohitsoni1989@gmail.com>
Moorthy RS <rsmoorthy@gmail.com> <rsmoorthy@users.noreply.github.com>
Moysés Borges <moysesb@gmail.com>
@@ -497,6 +526,7 @@ Olli Janatuinen <olli.janatuinen@gmail.com> <olljanat@users.noreply.github.com>
Onur Filiz <onur.filiz@microsoft.com>
Onur Filiz <onur.filiz@microsoft.com> <ofiliz@users.noreply.github.com>
Ouyang Liduo <oyld0210@163.com>
Patrick St. laurent <patrick@saint-laurent.us>
Patrick Stapleton <github@gdi2290.com>
Paul Liljenberg <liljenberg.paul@gmail.com> <letters@paulnotcom.se>
Pavel Tikhomirov <ptikhomirov@virtuozzo.com> <ptikhomirov@parallels.com>
@@ -520,6 +550,8 @@ Qin TianHuan <tianhuan@bingotree.cn>
Ray Tsang <rayt@google.com> <saturnism@users.noreply.github.com>
Renaud Gaubert <rgaubert@nvidia.com> <renaud.gaubert@gmail.com>
Richard Scothern <richard.scothern@gmail.com>
Rob Murray <rob.murray@docker.com>
Rob Murray <rob.murray@docker.com> <148866618+robmry@users.noreply.github.com>
Robert Terhaar <rterhaar@atlanticdynamic.com> <robbyt@users.noreply.github.com>
Roberto G. Hashioka <roberto.hashioka@docker.com> <roberto_hashioka@hotmail.com>
Roberto Muñoz Fernández <robertomf@gmail.com> <roberto.munoz.fernandez.contractor@bbva.com>
@@ -530,6 +562,7 @@ Rongxiang Song <tinysong1226@gmail.com>
Rony Weng <ronyweng@synology.com>
Ross Boucher <rboucher@gmail.com>
Rui Cao <ruicao@alauda.io>
Rui JingAn <quiterace@gmail.com>
Runshen Zhu <runshen.zhu@gmail.com>
Ryan Stelly <ryan.stelly@live.com>
Ryoga Saito <contact@proelbtn.com>
@@ -548,7 +581,9 @@ Sebastiaan van Stijn <github@gone.nl>
Sebastiaan van Stijn <github@gone.nl> <moby@example.com>
Sebastiaan van Stijn <github@gone.nl> <sebastiaan@ws-key-sebas3.dpi1.dpi>
Sebastiaan van Stijn <github@gone.nl> <thaJeztah@users.noreply.github.com>
Sebastian Thomschke <sebthom@users.noreply.github.com>
Seongyeol Lim <seongyeol37@gmail.com>
Serhii Nakon <serhii.n@thescimus.com>
Shaun Kaasten <shaunk@gmail.com>
Shawn Landden <shawn@churchofgit.com> <shawnlandden@gmail.com>
Shengbo Song <thomassong@tencent.com>
@@ -695,6 +730,8 @@ Xiaodong Liu <liuxiaodong@loongson.cn>
Xiaodong Zhang <a4012017@sina.com>
Xiaohua Ding <xiao_hua_ding@sina.cn>
Xiaoyu Zhang <zhang.xiaoyu33@zte.com.cn>
Xinfeng Liu <XinfengLiu@icloud.com>
Xinfeng Liu <XinfengLiu@icloud.com> <xinfeng.liu@gmail.com>
Xuecong Liao <satorulogic@gmail.com>
Yamasaki Masahide <masahide.y@gmail.com>
Yao Zaiyong <yaozaiyong@hotmail.com>

78
AUTHORS
View File

@@ -10,6 +10,7 @@ Aaron Huslage <huslage@gmail.com>
Aaron L. Xu <liker.xu@foxmail.com>
Aaron Lehmann <alehmann@netflix.com>
Aaron Welch <welch@packet.net>
Aaron Yoshitake <airandfingers@gmail.com>
Abel Muiño <amuino@gmail.com>
Abhijeet Kasurde <akasurde@redhat.com>
Abhinandan Prativadi <aprativadi@gmail.com>
@@ -27,6 +28,7 @@ Adam Miller <admiller@redhat.com>
Adam Mills <adam@armills.info>
Adam Pointer <adam.pointer@skybettingandgaming.com>
Adam Singer <financeCoding@gmail.com>
Adam Thornton <adam.thornton@maryville.com>
Adam Walz <adam@adamwalz.net>
Adam Williams <awilliams@mirantis.com>
AdamKorcz <adam@adalogics.com>
@@ -61,6 +63,7 @@ alambike <alambike@gmail.com>
Alan Hoyle <alan@alanhoyle.com>
Alan Scherger <flyinprogrammer@gmail.com>
Alan Thompson <cloojure@gmail.com>
Alano Terblanche <alano.terblanche@docker.com>
Albert Callarisa <shark234@gmail.com>
Albert Zhang <zhgwenming@gmail.com>
Albin Kerouanton <albinker@gmail.com>
@@ -140,6 +143,7 @@ Andreas Tiefenthaler <at@an-ti.eu>
Andrei Gherzan <andrei@resin.io>
Andrei Ushakov <aushakov@netflix.com>
Andrei Vagin <avagin@gmail.com>
Andrew Baxter <423qpsxzhh8k3h@s.rendaw.me>
Andrew C. Bodine <acbodine@us.ibm.com>
Andrew Clay Shafer <andrewcshafer@gmail.com>
Andrew Duckworth <grillopress@gmail.com>
@@ -173,6 +177,7 @@ Andy Rothfusz <github@developersupport.net>
Andy Smith <github@anarkystic.com>
Andy Wilson <wilson.andrew.j+github@gmail.com>
Andy Zhang <andy.zhangtao@hotmail.com>
Aneesh Kulkarni <askthefactorcamera@gmail.com>
Anes Hasicic <anes.hasicic@gmail.com>
Angel Velazquez <angelcar@amazon.com>
Anil Belur <askb23@gmail.com>
@@ -191,6 +196,7 @@ Anton Löfgren <anton.lofgren@gmail.com>
Anton Nikitin <anton.k.nikitin@gmail.com>
Anton Polonskiy <anton.polonskiy@gmail.com>
Anton Tiurin <noxiouz@yandex.ru>
Antonio Aguilar <antonio@zoftko.com>
Antonio Murdaca <antonio.murdaca@gmail.com>
Antonis Kalipetis <akalipetis@gmail.com>
Antony Messerli <amesserl@rackspace.com>
@@ -219,7 +225,6 @@ Avi Das <andas222@gmail.com>
Avi Kivity <avi@scylladb.com>
Avi Miller <avi.miller@oracle.com>
Avi Vaid <avaid1996@gmail.com>
ayoshitake <airandfingers@gmail.com>
Azat Khuyiyakhmetov <shadow_uz@mail.ru>
Bao Yonglei <baoyonglei@huawei.com>
Bardia Keyoumarsi <bkeyouma@ucsc.edu>
@@ -236,6 +241,7 @@ Ben Golub <ben.golub@dotcloud.com>
Ben Gould <ben@bengould.co.uk>
Ben Hall <ben@benhall.me.uk>
Ben Langfeld <ben@langfeld.me>
Ben Lovy <ben@deciduously.com>
Ben Sargent <ben@brokendigits.com>
Ben Severson <BenSeverson@users.noreply.github.com>
Ben Toews <mastahyeti@gmail.com>
@@ -262,7 +268,7 @@ Billy Ridgway <wrridgwa@us.ibm.com>
Bily Zhang <xcoder@tenxcloud.com>
Bin Liu <liubin0329@gmail.com>
Bingshen Wang <bingshen.wbs@alibaba-inc.com>
Bjorn Neergaard <bneergaard@mirantis.com>
Bjorn Neergaard <bjorn@neersighted.com>
Blake Geno <blakegeno@gmail.com>
Boaz Shuster <ripcurld.github@gmail.com>
bobby abbott <ttobbaybbob@gmail.com>
@@ -279,6 +285,7 @@ Brandon Liu <bdon@bdon.org>
Brandon Philips <brandon.philips@coreos.com>
Brandon Rhodes <brandon@rhodesmill.org>
Brendan Dixon <brendand@microsoft.com>
Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Brent Salisbury <brent.salisbury@docker.com>
Brett Higgins <brhiggins@arbor.net>
Brett Kochendorfer <brett.kochendorfer@gmail.com>
@@ -312,6 +319,7 @@ Burke Libbey <burke@libbey.me>
Byung Kang <byung.kang.ctr@amrdec.army.mil>
Caleb Spare <cespare@gmail.com>
Calen Pennington <cale@edx.org>
Calvin Liu <flycalvin@qq.com>
Cameron Boehmer <cameron.boehmer@gmail.com>
Cameron Sparr <gh@sparr.email>
Cameron Spear <cameronspear@gmail.com>
@@ -358,11 +366,13 @@ Chen Qiu <cheney-90@hotmail.com>
Cheng-mean Liu <soccerl@microsoft.com>
Chengfei Shang <cfshang@alauda.io>
Chengguang Xu <cgxu519@gmx.com>
Chentianze <cmoman@126.com>
Chenyang Yan <memory.yancy@gmail.com>
chenyuzhu <chenyuzhi@oschina.cn>
Chetan Birajdar <birajdar.chetan@gmail.com>
Chewey <prosto-chewey@users.noreply.github.com>
Chia-liang Kao <clkao@clkao.org>
Chiranjeevi Tirunagari <vchiranjeeviak.tirunagari@gmail.com>
chli <chli@freewheel.tv>
Cholerae Hu <choleraehyq@gmail.com>
Chris Alfonso <calfonso@redhat.com>
@@ -404,6 +414,7 @@ Christopher Crone <christopher.crone@docker.com>
Christopher Currie <codemonkey+github@gmail.com>
Christopher Jones <tophj@linux.vnet.ibm.com>
Christopher Latham <sudosurootdev@gmail.com>
Christopher Petito <chrisjpetito@gmail.com>
Christopher Rigor <crigor@gmail.com>
Christy Norman <christy@linux.vnet.ibm.com>
Chun Chen <ramichen@tencent.com>
@@ -433,8 +444,8 @@ Cristian Staretu <cristian.staretu@gmail.com>
cristiano balducci <cristiano.balducci@gmail.com>
Cristina Yenyxe Gonzalez Garcia <cristina.yenyxe@gmail.com>
Cruceru Calin-Cristian <crucerucalincristian@gmail.com>
cui fliter <imcusg@gmail.com>
CUI Wei <ghostplant@qq.com>
cuishuang <imcusg@gmail.com>
Cuong Manh Le <cuong.manhle.vn@gmail.com>
Cyprian Gracz <cyprian.gracz@micro-jumbo.eu>
Cyril F <cyrilf7x@gmail.com>
@@ -513,6 +524,7 @@ David Dooling <dooling@gmail.com>
David Gageot <david@gageot.net>
David Gebler <davidgebler@gmail.com>
David Glasser <glasser@davidglasser.net>
David Karlsson <35727626+dvdksn@users.noreply.github.com>
David Lawrence <david.lawrence@docker.com>
David Lechner <david@lechnology.com>
David M. Karr <davidmichaelkarr@gmail.com>
@@ -602,6 +614,7 @@ Donald Huang <don.hcd@gmail.com>
Dong Chen <dongluo.chen@docker.com>
Donghwa Kim <shanytt@gmail.com>
Donovan Jones <git@gamma.net.nz>
Dorin Geman <dorin.geman@docker.com>
Doron Podoleanu <doronp@il.ibm.com>
Doug Davis <dug@us.ibm.com>
Doug MacEachern <dougm@vmware.com>
@@ -636,6 +649,7 @@ Emily Rose <emily@contactvibe.com>
Emir Ozer <emirozer@yandex.com>
Eng Zer Jun <engzerjun@gmail.com>
Enguerran <engcolson@gmail.com>
Enrico Weigelt, metux IT consult <info@metux.net>
Eohyung Lee <liquidnuker@gmail.com>
epeterso <epeterson@breakpoint-labs.com>
er0k <er0k@er0k.net>
@@ -661,6 +675,7 @@ Erik Hollensbe <github@hollensbe.org>
Erik Inge Bolsø <knan@redpill-linpro.com>
Erik Kristensen <erik@erikkristensen.com>
Erik Sipsma <erik@sipsma.dev>
Erik Sjölund <erik.sjolund@gmail.com>
Erik St. Martin <alakriti@gmail.com>
Erik Weathers <erikdw@gmail.com>
Erno Hopearuoho <erno.hopearuoho@gmail.com>
@@ -676,6 +691,7 @@ Evan Allrich <evan@unguku.com>
Evan Carmi <carmi@users.noreply.github.com>
Evan Hazlett <ejhazlett@gmail.com>
Evan Krall <krall@yelp.com>
Evan Lezar <elezar@nvidia.com>
Evan Phoenix <evan@fallingsnow.net>
Evan Wies <evan@neomantra.net>
Evelyn Xu <evelynhsu21@gmail.com>
@@ -722,6 +738,7 @@ Feroz Salam <feroz.salam@sourcegraph.com>
Ferran Rodenas <frodenas@gmail.com>
Filipe Brandenburger <filbranden@google.com>
Filipe Oliveira <contato@fmoliveira.com.br>
Filipe Pina <hzlu1ot0@duck.com>
Flavio Castelli <fcastelli@suse.com>
Flavio Crisciani <flavio.crisciani@docker.com>
Florian <FWirtz@users.noreply.github.com>
@@ -744,6 +761,7 @@ Frank Groeneveld <frank@ivaldi.nl>
Frank Herrmann <fgh@4gh.tv>
Frank Macreery <frank@macreery.com>
Frank Rosquin <frank.rosquin+github@gmail.com>
Frank Villaro-Dixon <frank.villarodixon@merkle.com>
Frank Yang <yyb196@gmail.com>
Fred Lifton <fred.lifton@docker.com>
Frederick F. Kautz IV <fkautz@redhat.com>
@@ -765,6 +783,7 @@ Gabriel L. Somlo <gsomlo@gmail.com>
Gabriel Linder <linder.gabriel@gmail.com>
Gabriel Monroy <gabriel@opdemand.com>
Gabriel Nicolas Avellaneda <avellaneda.gabriel@gmail.com>
Gabriel Tomitsuka <gabriel@tomitsuka.com>
Gaetan de Villele <gdevillele@gmail.com>
Galen Sampson <galen.sampson@gmail.com>
Gang Qiao <qiaohai8866@gmail.com>
@@ -780,6 +799,7 @@ Geoff Levand <geoff@infradead.org>
Geoffrey Bachelet <grosfrais@gmail.com>
Geon Kim <geon0250@gmail.com>
George Kontridze <george@bugsnag.com>
George Ma <mayangang@outlook.com>
George MacRorie <gmacr31@gmail.com>
George Xie <georgexsh@gmail.com>
Georgi Hristozov <georgi@forkbomb.nl>
@@ -865,6 +885,8 @@ Hsing-Yu (David) Chen <davidhsingyuchen@gmail.com>
hsinko <21551195@zju.edu.cn>
Hu Keping <hukeping@huawei.com>
Hu Tao <hutao@cn.fujitsu.com>
Huajin Tong <fliterdashen@gmail.com>
huang-jl <1046678590@qq.com>
HuanHuan Ye <logindaveye@gmail.com>
Huanzhong Zhang <zhanghuanzhong90@gmail.com>
Huayi Zhang <irachex@gmail.com>
@@ -899,6 +921,7 @@ Illo Abdulrahim <abdulrahim.illo@nokia.com>
Ilya Dmitrichenko <errordeveloper@gmail.com>
Ilya Gusev <mail@igusev.ru>
Ilya Khlopotov <ilya.khlopotov@gmail.com>
imalasong <2879499479@qq.com>
imre Fitos <imre.fitos+github@gmail.com>
inglesp <peter.inglesby@gmail.com>
Ingo Gottwald <in.gottwald@gmail.com>
@@ -916,6 +939,7 @@ J Bruni <joaohbruni@yahoo.com.br>
J. Nunn <jbnunn@gmail.com>
Jack Danger Canty <jackdanger@squareup.com>
Jack Laxson <jackjrabbit@gmail.com>
Jack Walker <90711509+j2walker@users.noreply.github.com>
Jacob Atzen <jacob@jacobatzen.dk>
Jacob Edelman <edelman.jd@gmail.com>
Jacob Tomlinson <jacob@tom.linson.uk>
@@ -959,6 +983,7 @@ Jannick Fahlbusch <git@jf-projects.de>
Januar Wayong <januar@gmail.com>
Jared Biel <jared.biel@bolderthinking.com>
Jared Hocutt <jaredh@netapp.com>
Jaroslav Jindrak <dzejrou@gmail.com>
Jaroslaw Zabiello <hipertracker@gmail.com>
Jasmine Hegman <jasmine@jhegman.com>
Jason A. Donenfeld <Jason@zx2c4.com>
@@ -974,6 +999,7 @@ Jason Shepherd <jason@jasonshepherd.net>
Jason Smith <jasonrichardsmith@gmail.com>
Jason Sommer <jsdirv@gmail.com>
Jason Stangroome <jason@codeassassin.com>
Jasper Siepkes <siepkes@serviceplanet.nl>
Javier Bassi <javierbassi@gmail.com>
jaxgeller <jacksongeller@gmail.com>
Jay <teguhwpurwanto@gmail.com>
@@ -983,6 +1009,7 @@ Jean Rouge <rougej+github@gmail.com>
Jean-Baptiste Barth <jeanbaptiste.barth@gmail.com>
Jean-Baptiste Dalido <jeanbaptiste@appgratis.com>
Jean-Christophe Berthon <huygens@berthon.eu>
Jean-Michel Rouet <jm.rouet@gmail.com>
Jean-Paul Calderone <exarkun@twistedmatrix.com>
Jean-Pierre Huynh <jean-pierre.huynh@ounet.fr>
Jean-Tiare Le Bigot <jt@yadutaf.fr>
@@ -1001,6 +1028,7 @@ Jeffrey Bolle <jeffreybolle@gmail.com>
Jeffrey Morgan <jmorganca@gmail.com>
Jeffrey van Gogh <jvg@google.com>
Jenny Gebske <jennifer@gebske.de>
Jeongseok Kang <piono623@naver.com>
Jeremy Chambers <jeremy@thehipbot.com>
Jeremy Grosser <jeremy@synack.me>
Jeremy Huntwork <jhuntwork@lightcubesolutions.com>
@@ -1013,10 +1041,12 @@ Jeroen Jacobs <github@jeroenj.be>
Jesse Dearing <jesse.dearing@gmail.com>
Jesse Dubay <jesse@thefortytwo.net>
Jessica Frazelle <jess@oxide.computer>
Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
Jezeniel Zapanta <jpzapanta22@gmail.com>
Jhon Honce <jhonce@redhat.com>
Ji.Zhilong <zhilongji@gmail.com>
Jian Liao <jliao@alauda.io>
Jian Zeng <anonymousknight96@gmail.com>
Jian Zhang <zhangjian.fnst@cn.fujitsu.com>
Jiang Jinyang <jjyruby@gmail.com>
Jianyong Wu <jianyong.wu@arm.com>
@@ -1081,6 +1111,7 @@ Jon Johnson <jonjohnson@google.com>
Jon Surrell <jon.surrell@gmail.com>
Jon Wedaman <jweede@gmail.com>
Jonas Dohse <jonas@dohse.ch>
Jonas Geiler <git@jonasgeiler.com>
Jonas Heinrich <Jonas@JonasHeinrich.com>
Jonas Pfenniger <jonas@pfenniger.name>
Jonathan A. Schweder <jonathanschweder@gmail.com>
@@ -1141,6 +1172,7 @@ junxu <xujun@cmss.chinamobile.com>
Jussi Nummelin <jussi.nummelin@gmail.com>
Justas Brazauskas <brazauskasjustas@gmail.com>
Justen Martin <jmart@the-coder.com>
Justin Chadwell <me@jedevc.com>
Justin Cormack <justin.cormack@docker.com>
Justin Force <justin.force@gmail.com>
Justin Keller <85903732+jk-vb@users.noreply.github.com>
@@ -1183,6 +1215,7 @@ Ke Xu <leonhartx.k@gmail.com>
Kei Ohmura <ohmura.kei@gmail.com>
Keith Hudgins <greenman@greenman.org>
Keli Hu <dev@keli.hu>
Ken Bannister <kb2ma@runbox.com>
Ken Cochrane <kencochrane@gmail.com>
Ken Herner <kherner@progress.com>
Ken ICHIKAWA <ichikawa.ken@jp.fujitsu.com>
@@ -1192,7 +1225,7 @@ Kenjiro Nakayama <nakayamakenjiro@gmail.com>
Kent Johnson <kentoj@gmail.com>
Kenta Tada <Kenta.Tada@sony.com>
Kevin "qwazerty" Houdebert <kevin.houdebert@gmail.com>
Kevin Alvarez <crazy-max@users.noreply.github.com>
Kevin Alvarez <github@crazymax.dev>
Kevin Burke <kev@inburke.com>
Kevin Clark <kevin.clark@gmail.com>
Kevin Feyrer <kevin.feyrer@btinternet.com>
@@ -1225,6 +1258,7 @@ Konstantin Gribov <grossws@gmail.com>
Konstantin L <sw.double@gmail.com>
Konstantin Pelykh <kpelykh@zettaset.com>
Kostadin Plachkov <k.n.plachkov@gmail.com>
kpcyrd <git@rxv.cc>
Krasi Georgiev <krasi@vip-consult.solutions>
Krasimir Georgiev <support@vip-consult.co.uk>
Kris-Mikael Krister <krismikael@protonmail.com>
@@ -1245,6 +1279,7 @@ Lakshan Perera <lakshan@laktek.com>
Lalatendu Mohanty <lmohanty@redhat.com>
Lance Chen <cyen0312@gmail.com>
Lance Kinley <lkinley@loyaltymethods.com>
Lars Andringa <l.s.andringa@rug.nl>
Lars Butler <Lars.Butler@gmail.com>
Lars Kellogg-Stedman <lars@redhat.com>
Lars R. Damerow <lars@pixar.com>
@@ -1306,6 +1341,7 @@ Lorenzo Fontana <fontanalorenz@gmail.com>
Lotus Fenn <fenn.lotus@gmail.com>
Louis Delossantos <ldelossa.ld@gmail.com>
Louis Opter <kalessin@kalessin.fr>
Luboslav Pivarc <lpivarc@redhat.com>
Luca Favatella <luca.favatella@erlang-solutions.com>
Luca Marturana <lucamarturana@gmail.com>
Luca Orlandi <luca.orlandi@gmail.com>
@@ -1344,6 +1380,7 @@ Manuel Meurer <manuel@krautcomputing.com>
Manuel Rüger <manuel@rueg.eu>
Manuel Woelker <github@manuel.woelker.org>
mapk0y <mapk0y@gmail.com>
Marat Radchenko <marat@slonopotamus.org>
Marc Abramowitz <marc@marc-abramowitz.com>
Marc Kuo <kuomarc2@gmail.com>
Marc Tamsky <mtamsky@gmail.com>
@@ -1383,6 +1420,7 @@ Martijn van Oosterhout <kleptog@svana.org>
Martin Braun <braun@neuroforge.de>
Martin Dojcak <martin.dojcak@lablabs.io>
Martin Honermeyer <maze@strahlungsfrei.de>
Martin Jirku <martin@jirku.sk>
Martin Kelly <martin@surround.io>
Martin Mosegaard Amdisen <martin.amdisen@praqma.com>
Martin Muzatko <martin@happy-css.com>
@@ -1461,6 +1499,7 @@ Michael Holzheu <holzheu@linux.vnet.ibm.com>
Michael Hudson-Doyle <michael.hudson@canonical.com>
Michael Huettermann <michael@huettermann.net>
Michael Irwin <mikesir87@gmail.com>
Michael Kebe <michael.kebe@hkm.de>
Michael Kuehn <micha@kuehn.io>
Michael Käufl <docker@c.michael-kaeufl.de>
Michael Neale <michael.neale@gmail.com>
@@ -1509,10 +1548,11 @@ Mike Lundy <mike@fluffypenguin.org>
Mike MacCana <mike.maccana@gmail.com>
Mike Naberezny <mike@naberezny.com>
Mike Snitzer <snitzer@redhat.com>
Mike Sul <mike.sul@foundries.io>
mikelinjie <294893458@qq.com>
Mikhail Sobolev <mss@mawhrin.net>
Miklos Szegedi <miklos.szegedi@cloudera.com>
Milas Bowman <milasb@gmail.com>
Milas Bowman <devnull@milas.dev>
Milind Chawre <milindchawre@gmail.com>
Miloslav Trmač <mitr@redhat.com>
mingqing <limingqing@cyou-inc.com>
@@ -1524,6 +1564,7 @@ mlarcher <github@ringabell.org>
Mohammad Banikazemi <MBanikazemi@gmail.com>
Mohammad Nasirifar <farnasirim@gmail.com>
Mohammed Aaqib Ansari <maaquib@gmail.com>
Mohd Sadiq <mohdsadiq058@gmail.com>
Mohit Soni <mosoni@ebay.com>
Moorthy RS <rsmoorthy@gmail.com>
Morgan Bauer <mbauer@us.ibm.com>
@@ -1606,6 +1647,7 @@ Noah Treuhaft <noah.treuhaft@docker.com>
NobodyOnSE <ich@sektor.selfip.com>
noducks <onemannoducks@gmail.com>
Nolan Darilek <nolan@thewordnerd.info>
Nolan Miles <nolanpmiles@gmail.com>
Noriki Nakamura <noriki.nakamura@miraclelinux.com>
nponeccop <andy.melnikov@gmail.com>
Nurahmadie <nurahmadie@gmail.com>
@@ -1644,6 +1686,7 @@ Patrick Böänziger <patrick.baenziger@bsi-software.com>
Patrick Devine <patrick.devine@docker.com>
Patrick Haas <patrickhaas@google.com>
Patrick Hemmer <patrick.hemmer@gmail.com>
Patrick St. laurent <patrick@saint-laurent.us>
Patrick Stapleton <github@gdi2290.com>
Patrik Cyvoct <patrik@ptrk.io>
pattichen <craftsbear@gmail.com>
@@ -1661,6 +1704,7 @@ Paul Lietar <paul@lietar.net>
Paul Liljenberg <liljenberg.paul@gmail.com>
Paul Morie <pmorie@gmail.com>
Paul Nasrat <pnasrat@gmail.com>
Paul Seiffert <paul.seiffert@jimdo.com>
Paul Weaver <pauweave@cisco.com>
Paulo Gomes <pjbgf@linux.com>
Paulo Ribeiro <paigr.io@gmail.com>
@@ -1674,6 +1718,7 @@ Pavlos Ratis <dastergon@gentoo.org>
Pavol Vargovcik <pallly.vargovcik@gmail.com>
Pawel Konczalski <mail@konczalski.de>
Paweł Gronowski <pawel.gronowski@docker.com>
payall4u <payall4u@qq.com>
Peeyush Gupta <gpeeyush@linux.vnet.ibm.com>
Peggy Li <peggyli.224@gmail.com>
Pei Su <sillyousu@gmail.com>
@@ -1703,7 +1748,9 @@ Phil Estes <estesp@gmail.com>
Phil Sphicas <phil.sphicas@att.com>
Phil Spitler <pspitler@gmail.com>
Philip Alexander Etling <paetling@gmail.com>
Philip K. Warren <pkwarren@gmail.com>
Philip Monroe <phil@philmonroe.com>
Philipp Fruck <dev@p-fruck.de>
Philipp Gillé <philipp.gille@gmail.com>
Philipp Wahala <philipp.wahala@gmail.com>
Philipp Weissensteiner <mail@philippweissensteiner.com>
@@ -1741,6 +1788,7 @@ Quentin Brossard <qbrossard@gmail.com>
Quentin Perez <qperez@ocs.online.net>
Quentin Tayssier <qtayssier@gmail.com>
r0n22 <cameron.regan@gmail.com>
Rachit Sharma <rachitsharma613@gmail.com>
Radostin Stoyanov <rstoyanov1@gmail.com>
Rafal Jeczalik <rjeczalik@gmail.com>
Rafe Colton <rafael.colton@gmail.com>
@@ -1773,6 +1821,7 @@ Rich Horwood <rjhorwood@apple.com>
Rich Moyse <rich@moyse.us>
Rich Seymour <rseymour@gmail.com>
Richard Burnison <rburnison@ebay.com>
Richard Hansen <rhansen@rhansen.org>
Richard Harvey <richard@squarecows.com>
Richard Mathie <richard.mathie@amey.co.uk>
Richard Metzler <richard@paadee.com>
@@ -1788,6 +1837,7 @@ Ritesh H Shukla <sritesh@vmware.com>
Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
Rob Cowsill <42620235+rcowsill@users.noreply.github.com>
Rob Gulewich <rgulewich@netflix.com>
Rob Murray <rob.murray@docker.com>
Rob Vesse <rvesse@dotnetrdf.org>
Robert Bachmann <rb@robertbachmann.at>
Robert Bittle <guywithnose@gmail.com>
@@ -1842,6 +1892,7 @@ Royce Remer <royceremer@gmail.com>
Rozhnov Alexandr <nox73@ya.ru>
Rudolph Gottesheim <r.gottesheim@loot.at>
Rui Cao <ruicao@alauda.io>
Rui JingAn <quiterace@gmail.com>
Rui Lopes <rgl@ruilopes.com>
Ruilin Li <liruilin4@huawei.com>
Runshen Zhu <runshen.zhu@gmail.com>
@@ -1869,6 +1920,7 @@ ryancooper7 <ryan.cooper7@gmail.com>
RyanDeng <sheldon.d1018@gmail.com>
Ryo Nakao <nakabonne@gmail.com>
Ryoga Saito <contact@proelbtn.com>
Régis Behmo <regis@behmo.com>
Rémy Greinhofer <remy.greinhofer@livelovely.com>
s. rannou <mxs@sbrk.org>
Sabin Basyal <sabin.basyal@gmail.com>
@@ -1885,6 +1937,7 @@ Sam J Sharpe <sam.sharpe@digital.cabinet-office.gov.uk>
Sam Neirinck <sam@samneirinck.com>
Sam Reis <sreis@atlassian.com>
Sam Rijs <srijs@airpost.net>
Sam Thibault <sam.thibault@docker.com>
Sam Whited <sam@samwhited.com>
Sambuddha Basu <sambuddhabasu1@gmail.com>
Sami Wagiaalla <swagiaal@redhat.com>
@@ -1908,6 +1961,7 @@ Satoshi Tagomori <tagomoris@gmail.com>
Scott Bessler <scottbessler@gmail.com>
Scott Collier <emailscottcollier@gmail.com>
Scott Johnston <scott@docker.com>
Scott Moser <smoser@brickies.net>
Scott Percival <scottp@lastyard.com>
Scott Stamp <scottstamp851@gmail.com>
Scott Walls <sawalls@umich.edu>
@@ -1923,6 +1977,7 @@ Sebastiaan van Steenis <mail@superseb.nl>
Sebastiaan van Stijn <github@gone.nl>
Sebastian Höffner <sebastian.hoeffner@mevis.fraunhofer.de>
Sebastian Radloff <sradloff23@gmail.com>
Sebastian Thomschke <sebthom@users.noreply.github.com>
Sebastien Goasguen <runseb@gmail.com>
Senthil Kumar Selvaraj <senthil.thecoder@gmail.com>
Senthil Kumaran <senthil@uthcode.com>
@@ -1934,6 +1989,7 @@ Sergey Evstifeev <sergey.evstifeev@gmail.com>
Sergii Kabashniuk <skabashnyuk@codenvy.com>
Sergio Lopez <slp@redhat.com>
Serhat Gülçiçek <serhat25@gmail.com>
Serhii Nakon <serhii.n@thescimus.com>
SeungUkLee <lsy931106@gmail.com>
Sevki Hasirci <s@sevki.org>
Shane Canon <scanon@lbl.gov>
@@ -1996,6 +2052,7 @@ Stanislav Bondarenko <stanislav.bondarenko@gmail.com>
Stanislav Levin <slev@altlinux.org>
Steeve Morin <steeve.morin@gmail.com>
Stefan Berger <stefanb@linux.vnet.ibm.com>
Stefan Gehrig <stefan.gehrig.hn@googlemail.com>
Stefan J. Wernli <swernli@microsoft.com>
Stefan Praszalowicz <stefan@greplin.com>
Stefan S. <tronicum@user.github.com>
@@ -2003,6 +2060,7 @@ Stefan Scherer <stefan.scherer@docker.com>
Stefan Staudenmeyer <doerte@instana.com>
Stefan Weil <sw@weilnetz.de>
Steffen Butzer <steffen.butzer@outlook.com>
Stephan Henningsen <stephan-henningsen@users.noreply.github.com>
Stephan Spindler <shutefan@gmail.com>
Stephen Benjamin <stephen@redhat.com>
Stephen Crosby <stevecrozz@gmail.com>
@@ -2141,6 +2199,7 @@ Tomek Mańko <tomek.manko@railgun-solutions.com>
Tommaso Visconti <tommaso.visconti@gmail.com>
Tomoya Tabuchi <t@tomoyat1.com>
Tomáš Hrčka <thrcka@redhat.com>
Tomáš Virtus <nechtom@gmail.com>
tonic <tonicbupt@gmail.com>
Tonny Xu <tonny.xu@gmail.com>
Tony Abboud <tdabboud@hotmail.com>
@@ -2185,6 +2244,7 @@ Victor I. Wood <viw@t2am.com>
Victor Lyuboslavsky <victor@victoreda.com>
Victor Marmol <vmarmol@google.com>
Victor Palma <palma.victor@gmail.com>
Victor Toni <victor.toni@gmail.com>
Victor Vieux <victor.vieux@docker.com>
Victoria Bialas <victoria.bialas@docker.com>
Vijaya Kumar K <vijayak@caviumnetworks.com>
@@ -2204,6 +2264,7 @@ Vinod Kulkarni <vinod.kulkarni@gmail.com>
Vishal Doshi <vishal.doshi@gmail.com>
Vishnu Kannan <vishnuk@google.com>
Vitaly Ostrosablin <vostrosablin@virtuozzo.com>
Vitor Anjos <bartier@users.noreply.github.com>
Vitor Monteiro <vmrmonteiro@gmail.com>
Vivek Agarwal <me@vivek.im>
Vivek Dasgupta <vdasgupt@redhat.com>
@@ -2217,6 +2278,7 @@ VladimirAus <v_roudakov@yahoo.com>
Vladislav Kolesnikov <vkolesnikov@beget.ru>
Vlastimil Zeman <vlastimil.zeman@diffblue.com>
Vojtech Vitek (V-Teq) <vvitek@redhat.com>
voloder <110066198+voloder@users.noreply.github.com>
Walter Leibbrandt <github@wrl.co.za>
Walter Stanish <walter@pratyeka.org>
Wang Chao <chao.wang@ucloud.cn>
@@ -2234,6 +2296,7 @@ Wassim Dhif <wassimdhif@gmail.com>
Wataru Ishida <ishida.wataru@lab.ntt.co.jp>
Wayne Chang <wayne@neverfear.org>
Wayne Song <wsong@docker.com>
weebney <weebney@gmail.com>
Weerasak Chongnguluam <singpor@gmail.com>
Wei Fu <fuweid89@gmail.com>
Wei Wu <wuwei4455@gmail.com>
@@ -2250,6 +2313,7 @@ Wenxuan Zhao <viz@linux.com>
Wenyu You <21551128@zju.edu.cn>
Wenzhi Liang <wenzhi.liang@gmail.com>
Wes Morgan <cap10morgan@gmail.com>
Wesley Pettit <wppttt@amazon.com>
Wewang Xiaorenfine <wang.xiaoren@zte.com.cn>
Wiktor Kwapisiewicz <wiktor@metacode.biz>
Will Dietz <w@wdtz.org>
@@ -2289,7 +2353,7 @@ xiekeyang <xiekeyang@huawei.com>
Ximo Guanter Gonzálbez <joaquin.guantergonzalbez@telefonica.com>
xin.li <xin.li@daocloud.io>
Xinbo Weng <xihuanbo_0521@zju.edu.cn>
Xinfeng Liu <xinfeng.liu@gmail.com>
Xinfeng Liu <XinfengLiu@icloud.com>
Xinzi Zhou <imdreamrunner@gmail.com>
Xiuming Chen <cc@cxm.cc>
Xuecong Liao <satorulogic@gmail.com>
@@ -2355,6 +2419,7 @@ Zen Lin(Zhinan Lin) <linzhinan@huawei.com>
Zhang Kun <zkazure@gmail.com>
Zhang Wei <zhangwei555@huawei.com>
Zhang Wentao <zhangwentao234@huawei.com>
zhangguanzhang <zhangguanzhang@qq.com>
ZhangHang <stevezhang2014@gmail.com>
zhangxianwei <xianwei.zw@alibaba-inc.com>
Zhenan Ye <21551168@zju.edu.cn>
@@ -2381,6 +2446,7 @@ Zuhayr Elahi <zuhayr.elahi@docker.com>
Zunayed Ali <zunayed@gmail.com>
Álvaro Lázaro <alvaro.lazaro.g@gmail.com>
Átila Camurça Alves <camurca.home@gmail.com>
吴小白 <296015668@qq.com>
尹吉峰 <jifeng.yin@gmail.com>
屈骏 <qujun@tiduyun.com>
徐俊杰 <paco.xu@daocloud.io>

View File

@@ -101,7 +101,7 @@ the contributors guide.
<td>
<p>
Register for the Docker Community Slack at
<a href="https://dockr.ly/slack" target="_blank">https://dockr.ly/slack</a>.
<a href="https://dockr.ly/comm-slack" target="_blank">https://dockr.ly/comm-slack</a>.
We use the #moby-project channel for general discussion, and there are separate channels for other Moby projects such as #containerd.
</p>
</td>
@@ -422,6 +422,6 @@ The rules:
guidelines. Since you've read all the rules, you now know that.
If you are having trouble getting into the mood of idiomatic Go, we recommend
reading through [Effective Go](https://golang.org/doc/effective_go.html). The
[Go Blog](https://blog.golang.org) is also a great resource. Drinking the
reading through [Effective Go](https://go.dev/doc/effective_go). The
[Go Blog](https://go.dev/blog/) is also a great resource. Drinking the
kool-aid is a lot easier than going thirsty.

View File

@@ -1,17 +1,35 @@
# syntax=docker/dockerfile:1
# syntax=docker/dockerfile:1.7
ARG GO_VERSION=1.20.4
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG GO_VERSION=1.22.12
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
ARG XX_VERSION=1.2.1
ARG XX_VERSION=1.6.1
ARG VPNKIT_VERSION=0.5.0
ARG DOCKERCLI_VERSION=v17.06.2-ce
ARG DOCKERCLI_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_VERSION=v27.5.0
# cli version used for integration-cli tests
ARG DOCKERCLI_INTEGRATION_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_INTEGRATION_VERSION=v17.06.2-ce
ARG BUILDX_VERSION=0.20.0
ARG COMPOSE_VERSION=v2.32.4
ARG SYSTEMD="false"
ARG DEBIAN_FRONTEND=noninteractive
ARG DOCKER_STATIC=1
# REGISTRY_VERSION specifies the version of the registry to download from
# https://hub.docker.com/r/distribution/distribution. This version of
# the registry is used to test schema 2 manifests. Generally, the version
# specified here should match a current release.
ARG REGISTRY_VERSION=2.8.3
# delve is currently only supported on linux/amd64 and linux/arm64;
# https://github.com/go-delve/delve/blob/v1.8.1/pkg/proc/native/support_sentinel.go#L1-L6
ARG DELVE_SUPPORTED=${TARGETPLATFORM#linux/amd64} DELVE_SUPPORTED=${DELVE_SUPPORTED#linux/arm64}
ARG DELVE_SUPPORTED=${DELVE_SUPPORTED:+"unsupported"}
ARG DELVE_SUPPORTED=${DELVE_SUPPORTED:-"supported"}
# cross compilation helper
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
@@ -26,22 +44,19 @@ COPY --from=build-dummy /build /build
FROM --platform=$BUILDPLATFORM ${GOLANG_IMAGE} AS base
COPY --from=xx / /
RUN echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
ARG APT_MIRROR
RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
&& sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
ARG DEBIAN_FRONTEND
RUN apt-get update && apt-get install --no-install-recommends -y file
ENV GO111MODULE=off
ENV GOTOOLCHAIN=local
FROM base AS criu
ARG DEBIAN_FRONTEND
ADD --chmod=0644 https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/Release.key /etc/apt/trusted.gpg.d/criu.gpg.asc
RUN --mount=type=cache,sharing=locked,id=moby-criu-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-criu-aptcache,target=/var/cache/apt \
echo 'deb https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/ /' > /etc/apt/sources.list.d/criu.list \
echo 'deb https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_12/ /' > /etc/apt/sources.list.d/criu.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends criu \
&& install -D /usr/sbin/criu /build/criu
&& install -D /usr/sbin/criu /build/criu \
&& /build/criu --version
# registry
FROM base AS registry-src
@@ -50,11 +65,7 @@ RUN git init . && git remote add origin "https://github.com/distribution/distrib
FROM base AS registry
WORKDIR /go/src/github.com/docker/distribution
# REGISTRY_VERSION specifies the version of the registry to build and install
# from the https://github.com/docker/distribution repository. This version of
# the registry is used to test both schema 1 and schema 2 manifests. Generally,
# the version specified here should match a current release.
ARG REGISTRY_VERSION=v2.3.0
# REGISTRY_VERSION_SCHEMA1 specifies the version of the registry to build and
# install from the https://github.com/docker/distribution repository. This is
# an older (pre v2.3.0) version of the registry that only supports schema1
@@ -67,11 +78,10 @@ RUN --mount=from=registry-src,src=/usr/src/registry,rw \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=tmpfs,target=/go/src <<EOT
set -ex
git fetch -q --depth 1 origin "${REGISTRY_VERSION}" +refs/tags/*:refs/tags/*
git checkout -q FETCH_HEAD
export GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH"
CGO_ENABLED=0 xx-go build -o /build/registry-v2 -v ./cmd/registry
xx-verify /build/registry-v2
# Make the /build directory no matter what so that it doesn't fail on arm64 or
# any other platform where we don't build this registry
mkdir /build
case $TARGETPLATFORM in
linux/amd64|linux/arm/v7|linux/ppc64le|linux/s390x)
git fetch -q --depth 1 origin "${REGISTRY_VERSION_SCHEMA1}" +refs/tags/*:refs/tags/*
@@ -82,6 +92,9 @@ RUN --mount=from=registry-src,src=/usr/src/registry,rw \
esac
EOT
FROM distribution/distribution:$REGISTRY_VERSION AS registry-v2
RUN mkdir /build && mv /bin/registry /build/registry-v2
# go-swagger
FROM base AS swagger-src
WORKDIR /usr/src/swagger
@@ -109,7 +122,6 @@ EOT
# See also frozenImages in "testutil/environment/protect.go" (which needs to
# be updated when adding images to this list)
FROM debian:${BASE_DEBIAN_DISTRO} AS frozen-images
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-frozen-images-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-frozen-images-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
@@ -123,7 +135,7 @@ ARG TARGETVARIANT
RUN /download-frozen-image-v2.sh /build \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
busybox:glibc@sha256:1f81263701cddf6402afe9f33fca0266d9fff379e59b1748f33d3072da71ee85 \
debian:bullseye-slim@sha256:dacf278785a4daa9de07596ec739dbc07131e189942772210709c5c0777e8437 \
debian:bookworm-slim@sha256:2bc5c236e9b262645a323e9088dfa3bb1ecb16cc75811daf40a23a824d665be9 \
hello-world:latest@sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9 \
arm32v7/hello-world:latest@sha256:50b8560ad574c779908da71f7ce370c0a2471c098d44d1c8f6b513c5a55eeeb1
@@ -135,10 +147,10 @@ RUN git init . && git remote add origin "https://github.com/go-delve/delve.git"
# from the https://github.com/go-delve/delve repository.
# It can be used to run Docker with a possibility of
# attaching debugger to it.
ARG DELVE_VERSION=v1.20.1
ARG DELVE_VERSION=v1.23.0
RUN git fetch -q --depth 1 origin "${DELVE_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS delve-build
FROM base AS delve-supported
WORKDIR /usr/src/delve
ARG TARGETPLATFORM
RUN --mount=from=delve-src,src=/usr/src/delve,rw \
@@ -149,16 +161,8 @@ RUN --mount=from=delve-src,src=/usr/src/delve,rw \
xx-verify /build/dlv
EOT
# delve is currently only supported on linux/amd64 and linux/arm64;
# https://github.com/go-delve/delve/blob/v1.8.1/pkg/proc/native/support_sentinel.go#L1-L6
FROM binary-dummy AS delve-windows
FROM binary-dummy AS delve-linux-arm
FROM binary-dummy AS delve-linux-ppc64le
FROM binary-dummy AS delve-linux-s390x
FROM delve-build AS delve-linux-amd64
FROM delve-build AS delve-linux-arm64
FROM delve-linux-${TARGETARCH} AS delve-linux
FROM delve-${TARGETOS} AS delve
FROM binary-dummy AS delve-unsupported
FROM delve-${DELVE_SUPPORTED} AS delve
FROM base AS tomll
# GOTOML_VERSION specifies the version of the tomll binary to build and install
@@ -175,7 +179,7 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
FROM base AS gowinres
# GOWINRES_VERSION defines go-winres tool version
ARG GOWINRES_VERSION=v0.3.0
ARG GOWINRES_VERSION=v0.3.1
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/tc-hib/go-winres@${GOWINRES_VERSION}" \
@@ -192,17 +196,19 @@ RUN git init . && git remote add origin "https://github.com/containerd/container
# When updating the binary version you may also need to update the vendor
# version to pick up bug fixes or new APIs, however, usually the Go packages
# are built from a commit from the master branch.
ARG CONTAINERD_VERSION=v1.7.0
ARG CONTAINERD_VERSION=v1.7.25
RUN git fetch -q --depth 1 origin "${CONTAINERD_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerd-build
WORKDIR /go/src/github.com/containerd/containerd
ARG DEBIAN_FRONTEND
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-containerd-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-containerd-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
gcc libbtrfs-dev libsecret-1-dev
gcc \
libbtrfs-dev \
libsecret-1-dev \
pkg-config
ARG DOCKER_STATIC
RUN --mount=from=containerd-src,src=/usr/src/containerd,rw \
--mount=type=cache,target=/root/.cache/go-build,id=containerd-build-$TARGETPLATFORM <<EOT
@@ -223,7 +229,7 @@ FROM binary-dummy AS containerd-windows
FROM containerd-${TARGETOS} AS containerd
FROM base AS golangci_lint
ARG GOLANGCI_LINT_VERSION=v1.51.2
ARG GOLANGCI_LINT_VERSION=v1.60.2
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/golangci/golangci-lint/cmd/golangci-lint@${GOLANGCI_LINT_VERSION}" \
@@ -237,40 +243,41 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
&& /build/gotestsum --version
FROM base AS shfmt
ARG SHFMT_VERSION=v3.6.0
ARG SHFMT_VERSION=v3.8.0
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "mvdan.cc/sh/v3/cmd/shfmt@${SHFMT_VERSION}" \
&& /build/shfmt --version
# dockercli
FROM base AS dockercli-src
WORKDIR /tmp/dockercli
RUN git init . && git remote add origin "https://github.com/docker/cli.git"
ARG DOCKERCLI_VERSION
RUN git fetch -q --depth 1 origin "${DOCKERCLI_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
RUN [ -d ./components/cli ] && mv ./components/cli /usr/src/dockercli || mv /tmp/dockercli /usr/src/dockercli
WORKDIR /usr/src/dockercli
FROM base AS gopls
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "golang.org/x/tools/gopls@latest" \
&& /build/gopls version
FROM base AS dockercli
WORKDIR /go/src/github.com/docker/cli
ARG DOCKERCLI_REPOSITORY
ARG DOCKERCLI_VERSION
ARG DOCKERCLI_CHANNEL=stable
ARG TARGETPLATFORM
RUN xx-apt-get install -y --no-install-recommends gcc libc6-dev
RUN --mount=from=dockercli-src,src=/usr/src/dockercli,rw \
--mount=type=cache,target=/root/.cache/go-build,id=dockercli-build-$TARGETPLATFORM <<EOT
set -e
DOWNLOAD_URL="https://download.docker.com/linux/static/${DOCKERCLI_CHANNEL}/$(xx-info march)/docker-${DOCKERCLI_VERSION#v}.tgz"
if curl --head --silent --fail "${DOWNLOAD_URL}" 1>/dev/null 2>&1; then
mkdir /build
curl -Ls "${DOWNLOAD_URL}" | tar -xz docker/docker
mv docker/docker /build/docker
else
CGO_ENABLED=0 xx-go build -o /build/docker ./cmd/docker
fi
xx-verify /build/docker
EOT
RUN --mount=source=hack/dockerfile/cli.sh,target=/download-or-build-cli.sh \
--mount=type=cache,id=dockercli-git-$TARGETPLATFORM,sharing=locked,target=./.git \
--mount=type=cache,target=/root/.cache/go-build,id=dockercli-build-$TARGETPLATFORM \
rm -f ./.git/*.lock \
&& /download-or-build-cli.sh ${DOCKERCLI_VERSION} ${DOCKERCLI_REPOSITORY} /build \
&& /build/docker --version
FROM base AS dockercli-integration
WORKDIR /go/src/github.com/docker/cli
ARG DOCKERCLI_INTEGRATION_REPOSITORY
ARG DOCKERCLI_INTEGRATION_VERSION
ARG TARGETPLATFORM
RUN --mount=source=hack/dockerfile/cli.sh,target=/download-or-build-cli.sh \
--mount=type=cache,id=dockercli-git-$TARGETPLATFORM,sharing=locked,target=./.git \
--mount=type=cache,target=/root/.cache/go-build,id=dockercli-build-$TARGETPLATFORM \
rm -f ./.git/*.lock \
&& /download-or-build-cli.sh ${DOCKERCLI_INTEGRATION_VERSION} ${DOCKERCLI_INTEGRATION_REPOSITORY} /build \
&& /build/docker --version
# runc
FROM base AS runc-src
@@ -280,17 +287,20 @@ RUN git init . && git remote add origin "https://github.com/opencontainers/runc.
# that is used. If you need to update runc, open a pull request in the containerd
# project first, and update both after that is merged. When updating RUNC_VERSION,
# consider updating runc in vendor.mod accordingly.
ARG RUNC_VERSION=v1.1.7
ARG RUNC_VERSION=v1.2.5
RUN git fetch -q --depth 1 origin "${RUNC_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS runc-build
WORKDIR /go/src/github.com/opencontainers/runc
ARG DEBIAN_FRONTEND
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-runc-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-runc-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
dpkg-dev gcc libc6-dev libseccomp-dev
dpkg-dev \
gcc \
libc6-dev \
libseccomp-dev \
pkg-config
ARG DOCKER_STATIC
RUN --mount=from=runc-src,src=/usr/src/runc,rw \
--mount=type=cache,target=/root/.cache/go-build,id=runc-build-$TARGETPLATFORM <<EOT
@@ -317,7 +327,6 @@ RUN git fetch -q --depth 1 origin "${TINI_VERSION}" +refs/tags/*:refs/tags/* &&
FROM base AS tini-build
WORKDIR /go/src/github.com/krallin/tini
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends cmake
@@ -325,7 +334,9 @@ ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
xx-apt-get install -y --no-install-recommends \
gcc libc6-dev
gcc \
libc6-dev \
pkg-config
RUN --mount=from=tini-src,src=/usr/src/tini,rw \
--mount=type=cache,target=/root/.cache/go-build,id=tini-build-$TARGETPLATFORM <<EOT
set -e
@@ -344,18 +355,19 @@ FROM tini-${TARGETOS} AS tini
FROM base AS rootlesskit-src
WORKDIR /usr/src/rootlesskit
RUN git init . && git remote add origin "https://github.com/rootless-containers/rootlesskit.git"
# When updating, also update rootlesskit commit in vendor.mod accordingly.
ARG ROOTLESSKIT_VERSION=v1.1.0
# When updating, also update vendor.mod and hack/dockerfile/install/rootlesskit.installer accordingly.
ARG ROOTLESSKIT_VERSION=v2.3.2
RUN git fetch -q --depth 1 origin "${ROOTLESSKIT_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS rootlesskit-build
WORKDIR /go/src/github.com/rootless-containers/rootlesskit
ARG DEBIAN_FRONTEND
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-rootlesskit-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-rootlesskit-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
gcc libc6-dev
gcc \
libc6-dev \
pkg-config
ENV GO111MODULE=on
ARG DOCKER_STATIC
RUN --mount=from=rootlesskit-src,src=/usr/src/rootlesskit,rw \
@@ -368,15 +380,15 @@ RUN --mount=from=rootlesskit-src,src=/usr/src/rootlesskit,rw \
xx-go build -o /build/rootlesskit-docker-proxy -ldflags="$([ "$DOCKER_STATIC" != "1" ] && echo "-linkmode=external")" ./cmd/rootlesskit-docker-proxy
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit-docker-proxy
EOT
COPY ./contrib/dockerd-rootless.sh /build/
COPY ./contrib/dockerd-rootless-setuptool.sh /build/
COPY --link ./contrib/dockerd-rootless.sh /build/
COPY --link ./contrib/dockerd-rootless-setuptool.sh /build/
FROM rootlesskit-build AS rootlesskit-linux
FROM binary-dummy AS rootlesskit-windows
FROM rootlesskit-${TARGETOS} AS rootlesskit
FROM base AS crun
ARG CRUN_VERSION=1.4.5
ARG CRUN_VERSION=1.12
RUN --mount=type=cache,sharing=locked,id=moby-crun-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-crun-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
@@ -423,7 +435,11 @@ RUN git fetch -q --depth 1 origin "${CONTAINERUTILITY_VERSION}" +refs/tags/*:ref
FROM base AS containerutil-build
WORKDIR /usr/src/containerutil
ARG TARGETPLATFORM
RUN xx-apt-get install -y --no-install-recommends gcc g++ libc6-dev
RUN xx-apt-get install -y --no-install-recommends \
gcc \
g++ \
libc6-dev \
pkg-config
RUN --mount=from=containerutil-src,src=/usr/src/containerutil,rw \
--mount=type=cache,target=/root/.cache/go-build,id=containerutil-build-$TARGETPLATFORM <<EOT
set -e
@@ -437,28 +453,39 @@ FROM binary-dummy AS containerutil-linux
FROM containerutil-build AS containerutil-windows-amd64
FROM containerutil-windows-${TARGETARCH} AS containerutil-windows
FROM containerutil-${TARGETOS} AS containerutil
FROM docker/buildx-bin:${BUILDX_VERSION} as buildx
FROM docker/compose-bin:${COMPOSE_VERSION} as compose
FROM base AS dev-systemd-false
COPY --from=dockercli /build/ /usr/local/cli
COPY --from=frozen-images /build/ /docker-frozen-images
COPY --from=swagger /build/ /usr/local/bin/
COPY --from=delve /build/ /usr/local/bin/
COPY --from=tomll /build/ /usr/local/bin/
COPY --from=gowinres /build/ /usr/local/bin/
COPY --from=tini /build/ /usr/local/bin/
COPY --from=registry /build/ /usr/local/bin/
COPY --from=criu /build/ /usr/local/bin/
COPY --from=gotestsum /build/ /usr/local/bin/
COPY --from=golangci_lint /build/ /usr/local/bin/
COPY --from=shfmt /build/ /usr/local/bin/
COPY --from=runc /build/ /usr/local/bin/
COPY --from=containerd /build/ /usr/local/bin/
COPY --from=rootlesskit /build/ /usr/local/bin/
COPY --from=vpnkit / /usr/local/bin/
COPY --from=containerutil /build/ /usr/local/bin/
COPY --from=crun /build/ /usr/local/bin/
COPY hack/dockerfile/etc/docker/ /etc/docker/
COPY --link --from=frozen-images /build/ /docker-frozen-images
COPY --link --from=swagger /build/ /usr/local/bin/
COPY --link --from=delve /build/ /usr/local/bin/
COPY --link --from=tomll /build/ /usr/local/bin/
COPY --link --from=gowinres /build/ /usr/local/bin/
COPY --link --from=tini /build/ /usr/local/bin/
COPY --link --from=registry /build/ /usr/local/bin/
COPY --link --from=registry-v2 /build/ /usr/local/bin/
# Skip the CRIU stage for now, as the opensuse package repository is sometimes
# unstable, and we're currently not using it in CI.
#
# FIXME(thaJeztah): re-enable this stage when https://github.com/moby/moby/issues/38963 is resolved (see https://github.com/moby/moby/pull/38984)
# COPY --link --from=criu /build/ /usr/local/bin/
COPY --link --from=gotestsum /build/ /usr/local/bin/
COPY --link --from=golangci_lint /build/ /usr/local/bin/
COPY --link --from=shfmt /build/ /usr/local/bin/
COPY --link --from=runc /build/ /usr/local/bin/
COPY --link --from=containerd /build/ /usr/local/bin/
COPY --link --from=rootlesskit /build/ /usr/local/bin/
COPY --link --from=vpnkit / /usr/local/bin/
COPY --link --from=containerutil /build/ /usr/local/bin/
COPY --link --from=crun /build/ /usr/local/bin/
COPY --link hack/dockerfile/etc/docker/ /etc/docker/
COPY --link --from=buildx /buildx /usr/local/libexec/docker/cli-plugins/docker-buildx
COPY --link --from=compose /docker-compose /usr/libexec/docker/cli-plugins/docker-compose
ENV PATH=/usr/local/cli:$PATH
ENV TEST_CLIENT_BINARY=/usr/local/cli-integration/docker
ENV CONTAINERD_ADDRESS=/run/docker/containerd/containerd.sock
ENV CONTAINERD_NAMESPACE=moby
WORKDIR /go/src/github.com/docker/docker
@@ -478,7 +505,6 @@ RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
ENTRYPOINT ["hack/dind-systemd"]
FROM dev-systemd-${SYSTEMD} AS dev-base
ARG DEBIAN_FRONTEND
RUN groupadd -r docker
RUN useradd --create-home --gid docker unprivilegeduser \
&& mkdir -p /home/unprivilegeduser/.local/share/docker \
@@ -512,9 +538,6 @@ RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
net-tools \
patch \
pigz \
python3-pip \
python3-setuptools \
python3-wheel \
sudo \
systemd-journal-remote \
thin-provisioning-tools \
@@ -530,8 +553,6 @@ RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
RUN update-alternatives --set iptables /usr/sbin/iptables-legacy || true \
&& update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true \
&& update-alternatives --set arptables /usr/sbin/arptables-legacy || true
ARG YAMLLINT_VERSION=1.27.1
RUN pip3 install yamllint==${YAMLLINT_VERSION}
RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
apt-get update && apt-get install --no-install-recommends -y \
@@ -539,18 +560,19 @@ RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
pkg-config \
dpkg-dev \
libapparmor-dev \
libdevmapper-dev \
libseccomp-dev \
libsecret-1-dev \
libsystemd-dev \
libudev-dev
libudev-dev \
yamllint
COPY --link --from=dockercli /build/ /usr/local/cli
COPY --link --from=dockercli-integration /build/ /usr/local/cli-integration
FROM base AS build
COPY --from=gowinres /build/ /usr/local/bin/
WORKDIR /go/src/github.com/docker/docker
ENV GO111MODULE=off
ENV CGO_ENABLED=1
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-build-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-build-aptcache,target=/var/cache/apt \
apt-get update && apt-get install --no-install-recommends -y \
@@ -565,11 +587,11 @@ RUN --mount=type=cache,sharing=locked,id=moby-build-aptlib,target=/var/lib/apt \
gcc \
libapparmor-dev \
libc6-dev \
libdevmapper-dev \
libseccomp-dev \
libsecret-1-dev \
libsystemd-dev \
libudev-dev
libudev-dev \
pkg-config
ARG DOCKER_BUILDTAGS
ARG DOCKER_DEBUG
ARG DOCKER_GITCOMMIT=HEAD
@@ -589,7 +611,7 @@ RUN <<EOT
XX_CC_PREFER_LINKER=ld xx-clang --setup-target-triple
fi
EOT
RUN --mount=type=bind,target=. \
RUN --mount=type=bind,target=.,rw \
--mount=type=tmpfs,target=cli/winresources/dockerd \
--mount=type=tmpfs,target=cli/winresources/docker-proxy \
--mount=type=cache,target=/root/.cache/go-build,id=moby-build-$TARGETPLATFORM <<EOT
@@ -615,13 +637,13 @@ COPY --from=build /build/ /
# usage:
# > docker buildx bake all
FROM scratch AS all
COPY --from=tini /build/ /
COPY --from=runc /build/ /
COPY --from=containerd /build/ /
COPY --from=rootlesskit /build/ /
COPY --from=containerutil /build/ /
COPY --from=vpnkit / /
COPY --from=build /build /
COPY --link --from=tini /build/ /
COPY --link --from=runc /build/ /
COPY --link --from=containerd /build/ /
COPY --link --from=rootlesskit /build/ /
COPY --link --from=containerutil /build/ /
COPY --link --from=vpnkit / /
COPY --link --from=build /build /
# smoke tests
# usage:
@@ -637,8 +659,13 @@ RUN <<EOT
docker-proxy --version
EOT
# devcontainer is a stage used by .devcontainer/devcontainer.json
FROM dev-base AS devcontainer
COPY --link . .
COPY --link --from=gopls /build/ /usr/local/bin/
# usage:
# > make shell
# > SYSTEMD=true make shell
FROM dev-base AS dev
COPY . .
COPY --link . .

View File

@@ -1,84 +0,0 @@
ARG GO_VERSION=1.20.4
FROM golang:${GO_VERSION}-alpine AS base
ENV GO111MODULE=off
RUN apk --no-cache add \
bash \
build-base \
curl \
lvm2-dev \
jq
RUN mkdir -p /build/
RUN mkdir -p /go/src/github.com/docker/docker/
WORKDIR /go/src/github.com/docker/docker/
FROM base AS frozen-images
# Get useful and necessary Hub images so we can "docker load" locally instead of pulling
COPY contrib/download-frozen-image-v2.sh /
RUN /download-frozen-image-v2.sh /build \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
debian:bullseye-slim@sha256:dacf278785a4daa9de07596ec739dbc07131e189942772210709c5c0777e8437 \
hello-world:latest@sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9 \
arm32v7/hello-world:latest@sha256:50b8560ad574c779908da71f7ce370c0a2471c098d44d1c8f6b513c5a55eeeb1
# See also frozenImages in "testutil/environment/protect.go" (which needs to be updated when adding images to this list)
FROM base AS dockercli
COPY hack/dockerfile/install/install.sh ./install.sh
COPY hack/dockerfile/install/dockercli.installer ./
RUN PREFIX=/build ./install.sh dockercli
# TestDockerCLIBuildSuite dependency
FROM base AS contrib
COPY contrib/syscall-test /build/syscall-test
COPY contrib/httpserver/Dockerfile /build/httpserver/Dockerfile
COPY contrib/httpserver contrib/httpserver
RUN CGO_ENABLED=0 go build -buildmode=pie -o /build/httpserver/httpserver github.com/docker/docker/contrib/httpserver
# Build the integration tests and copy the resulting binaries to /build/tests
FROM base AS builder
# Set tag and add sources
COPY . .
# Copy test sources tests that use assert can print errors
RUN mkdir -p /build${PWD} && find integration integration-cli -name \*_test.go -exec cp --parents '{}' /build${PWD} \;
# Build and install test binaries
ARG DOCKER_GITCOMMIT=undefined
RUN hack/make.sh build-integration-test-binary
RUN mkdir -p /build/tests && find . -name test.main -exec cp --parents '{}' /build/tests \;
## Generate testing image
FROM alpine:3.10 as runner
ENV DOCKER_REMOTE_DAEMON=1
ENV DOCKER_INTEGRATION_DAEMON_DEST=/
ENTRYPOINT ["/scripts/run.sh"]
# Add an unprivileged user to be used for tests which need it
RUN addgroup docker && adduser -D -G docker unprivilegeduser -s /bin/ash
# GNU tar is used for generating the emptyfs image
RUN apk --no-cache add \
bash \
ca-certificates \
g++ \
git \
inetutils-ping \
iptables \
libcap2-bin \
pigz \
tar \
xz
COPY hack/test/e2e-run.sh /scripts/run.sh
COPY hack/make/.ensure-emptyfs /scripts/ensure-emptyfs.sh
COPY integration/testdata /tests/integration/testdata
COPY integration/build/testdata /tests/integration/build/testdata
COPY integration-cli/fixtures /tests/integration-cli/fixtures
COPY --from=frozen-images /build/ /docker-frozen-images
COPY --from=dockercli /build/ /usr/bin/
COPY --from=contrib /build/ /tests/contrib/
COPY --from=builder /build/ /

View File

@@ -5,17 +5,14 @@
# This represents the bare minimum required to build and test Docker.
ARG GO_VERSION=1.20.4
ARG GO_VERSION=1.22.12
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
FROM ${GOLANG_IMAGE}
ENV GO111MODULE=off
# allow replacing httpredir or deb mirror
ARG APT_MIRROR=deb.debian.org
RUN sed -ri "s/(httpredir|deb).debian.org/$APT_MIRROR/g" /etc/apt/sources.list
ENV GOTOOLCHAIN=local
# Compile and runtime deps
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#build-dependencies
@@ -26,7 +23,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
cmake \
git \
libapparmor-dev \
libdevmapper-dev \
libseccomp-dev \
ca-certificates \
e2fsprogs \

View File

@@ -154,21 +154,17 @@
# The number of build steps below are explicitly minimised to improve performance.
# Extremely important - do not change the following line to reference a "specific" image,
# such as `mcr.microsoft.com/windows/servercore:ltsc2022`. If using this Dockerfile in process
# isolated containers, the kernel of the host must match the container image, and hence
# would fail between Windows Server 2016 (aka RS1) and Windows Server 2019 (aka RS5).
# It is expected that the image `microsoft/windowsservercore:latest` is present, and matches
# the hosts kernel version before doing a build.
FROM microsoft/windowsservercore
ARG WINDOWS_BASE_IMAGE=mcr.microsoft.com/windows/servercore
ARG WINDOWS_BASE_IMAGE_TAG=ltsc2022
FROM ${WINDOWS_BASE_IMAGE}:${WINDOWS_BASE_IMAGE_TAG}
# Use PowerShell as the default shell
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG GO_VERSION=1.20.4
ARG GO_VERSION=1.22.12
ARG GOTESTSUM_VERSION=v1.8.2
ARG GOWINRES_VERSION=v0.3.0
ARG CONTAINERD_VERSION=v1.7.0
ARG GOWINRES_VERSION=v0.3.1
ARG CONTAINERD_VERSION=v1.7.25
# Environment variable notes:
# - GO_VERSION must be consistent with 'Dockerfile' used by Linux.
@@ -179,6 +175,7 @@ ENV GO_VERSION=${GO_VERSION} `
GIT_VERSION=2.11.1 `
GOPATH=C:\gopath `
GO111MODULE=off `
GOTOOLCHAIN=local `
FROM_DOCKERFILE=1 `
GOTESTSUM_VERSION=${GOTESTSUM_VERSION} `
GOWINRES_VERSION=${GOWINRES_VERSION}
@@ -223,8 +220,8 @@ RUN `
Download-File $location C:\gitsetup.zip; `
`
Write-Host INFO: Downloading go...; `
$dlGoVersion=$Env:GO_VERSION -replace '\.0$',''; `
Download-File "https://golang.org/dl/go${dlGoVersion}.windows-amd64.zip" C:\go.zip; `
$dlGoVersion=$Env:GO_VERSION; `
Download-File "https://go.dev/dl/go${dlGoVersion}.windows-amd64.zip" C:\go.zip; `
`
Write-Host INFO: Downloading compiler 1 of 3...; `
Download-File https://raw.githubusercontent.com/moby/docker-tdmgcc/master/gcc.zip C:\gcc.zip; `

568
Jenkinsfile vendored
View File

@@ -1,568 +0,0 @@
#!groovy
pipeline {
agent none
options {
buildDiscarder(logRotator(daysToKeepStr: '30'))
timeout(time: 2, unit: 'HOURS')
timestamps()
}
parameters {
booleanParam(name: 'arm64', defaultValue: true, description: 'ARM (arm64) Build/Test')
booleanParam(name: 's390x', defaultValue: false, description: 'IBM Z (s390x) Build/Test')
booleanParam(name: 'ppc64le', defaultValue: false, description: 'PowerPC (ppc64le) Build/Test')
booleanParam(name: 'dco', defaultValue: true, description: 'Run the DCO check')
}
environment {
DOCKER_BUILDKIT = '1'
DOCKER_EXPERIMENTAL = '1'
DOCKER_GRAPHDRIVER = 'overlay2'
APT_MIRROR = 'cdn-fastly.deb.debian.org'
CHECK_CONFIG_COMMIT = '33a3680e08d1007e72c3b3f1454f823d8e9948ee'
TESTDEBUG = '0'
TIMEOUT = '120m'
}
stages {
stage('pr-hack') {
when { changeRequest() }
steps {
script {
echo "Workaround for PR auto-cancel feature. Borrowed from https://issues.jenkins-ci.org/browse/JENKINS-43353"
def buildNumber = env.BUILD_NUMBER as int
if (buildNumber > 1) milestone(buildNumber - 1)
milestone(buildNumber)
}
}
}
stage('DCO-check') {
when {
beforeAgent true
expression { params.dco }
}
agent { label 'arm64 && ubuntu-2004' }
steps {
sh '''
docker run --rm \
-v "$WORKSPACE:/workspace" \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
alpine sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && cd /workspace && hack/validate/dco'
'''
}
}
stage('Build') {
parallel {
stage('s390x') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.s390x }
}
}
agent { label 's390x-ubuntu-2004' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Integration tests") {
environment { TEST_SKIP_INTEGRATION_CLI = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=s390x-integration
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('s390x integration-cli') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.s390x }
}
}
agent { label 's390x-ubuntu-2004' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration-cli tests") {
environment { TEST_SKIP_INTEGRATION = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=s390x-integration-cli
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('ppc64le') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.ppc64le }
}
}
agent { label 'ppc64le-ubuntu-1604' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker buildx build --load --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Integration tests") {
environment { TEST_SKIP_INTEGRATION_CLI = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=ppc64le-integration
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('ppc64le integration-cli') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.ppc64le }
}
}
agent { label 'ppc64le-ubuntu-1604' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker buildx build --load --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration-cli tests") {
environment { TEST_SKIP_INTEGRATION = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=ppc64le-integration-cli
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('arm64') {
when {
beforeAgent true
expression { params.arm64 }
}
agent { label 'arm64 && ubuntu-2004' }
environment {
TEST_SKIP_INTEGRATION_CLI = '1'
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh 'docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .'
}
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Integration tests") {
environment { TEST_SKIP_INTEGRATION_CLI = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=arm64-integration
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
}
}
}
}

View File

@@ -24,17 +24,21 @@
# subsystem maintainers accountable. If ownership is unclear, they are the de facto owners.
people = [
"akerouanton",
"akihirosuda",
"anusha",
"coolljt0725",
"corhere",
"cpuguy83",
"crazy-max",
"estesp",
"johnstep",
"justincormack",
"kolyshkin",
"laurazard",
"mhbauer",
"neersighted",
"robmry",
"rumpl",
"runcom",
"samuelkarp",
@@ -63,14 +67,12 @@
# - close an issue or pull request when it's inappropriate or off-topic
people = [
"akerouanton",
"alexellis",
"andrewhsu",
"bsousaa",
"crazy-max",
"dmcgowan",
"fntlnz",
"gianarb",
"laurazard",
"olljanat",
"programmerq",
"ripcurld",
@@ -357,6 +359,11 @@
Email = "dnephin@gmail.com"
GitHub = "dnephin"
[people.dmcgowan]
Name = "Derek McGowan"
Email = "derek@mcgstyle.net"
GitHub = "dmcgowan"
[people.duglin]
Name = "Doug Davis"
Email = "dug@us.ibm.com"
@@ -459,7 +466,7 @@
[people.neersighted]
Name = "Bjorn Neergaard"
Email = "bneergaard@mirantis.com"
Email = "bjorn@neersighted.com"
GitHub = "neersighted"
[people.olljanat]
@@ -472,6 +479,11 @@
Email = "jeff@docker.com"
GitHub = "programmerq"
[people.robmry]
Name = "Rob Murray"
Email = "rob.murray@docker.com"
GitHub = "robmry"
[people.ripcurld]
Name = "Boaz Shuster"
Email = "ripcurld.github@gmail.com"

View File

@@ -1,17 +1,11 @@
.PHONY: all binary dynbinary build cross help install manpages run shell test test-docker-py test-integration test-unit validate validate-% win
DOCKER ?= docker
BUILDX ?= $(DOCKER) buildx
# set the graph driver as the current graphdriver if not set
DOCKER_GRAPHDRIVER := $(if $(DOCKER_GRAPHDRIVER),$(DOCKER_GRAPHDRIVER),$(shell docker info 2>&1 | grep "Storage Driver" | sed 's/.*: //'))
DOCKER_GRAPHDRIVER := $(if $(DOCKER_GRAPHDRIVER),$(DOCKER_GRAPHDRIVER),$(shell docker info -f '{{ .Driver }}' 2>&1))
export DOCKER_GRAPHDRIVER
# get OS/Arch of docker engine
DOCKER_OSARCH := $(shell bash -c 'source hack/make/.detect-daemon-osarch && echo $${DOCKER_ENGINE_OSARCH}')
DOCKERFILE := $(shell bash -c 'source hack/make/.detect-daemon-osarch && echo $${DOCKERFILE}')
DOCKER_GITCOMMIT := $(shell git rev-parse --short HEAD || echo unsupported)
DOCKER_GITCOMMIT := $(shell git rev-parse HEAD)
export DOCKER_GITCOMMIT
# allow overriding the repository and branch that validation scripts are running
@@ -20,6 +14,9 @@ export VALIDATE_REPO
export VALIDATE_BRANCH
export VALIDATE_ORIGIN_BRANCH
export PAGER
export GIT_PAGER
# env vars passed through directly to Docker's build scripts
# to allow things like `make KEEPBUNDLE=1 binary` easily
# `project/PACKAGERS.md` have some limited documentation of some of these
@@ -28,10 +25,9 @@ export VALIDATE_ORIGIN_BRANCH
# option of "go build". For example, a built-in graphdriver priority list
# can be changed during build time like this:
#
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper" dynbinary
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,zfs" dynbinary
#
DOCKER_ENVS := \
-e BUILD_APT_MIRROR \
-e BUILDFLAGS \
-e KEEPBUNDLE \
-e DOCKER_BUILD_ARGS \
@@ -41,6 +37,10 @@ DOCKER_ENVS := \
-e DOCKER_BUILDKIT \
-e DOCKER_BASH_COMPLETION_PATH \
-e DOCKER_CLI_PATH \
-e DOCKERCLI_VERSION \
-e DOCKERCLI_REPOSITORY \
-e DOCKERCLI_INTEGRATION_VERSION \
-e DOCKERCLI_INTEGRATION_REPOSITORY \
-e DOCKER_DEBUG \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT \
@@ -58,8 +58,10 @@ DOCKER_ENVS := \
-e TEST_FORCE_VALIDATE \
-e TEST_INTEGRATION_DIR \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_INTEGRATION_FAIL_FAST \
-e TEST_SKIP_INTEGRATION \
-e TEST_SKIP_INTEGRATION_CLI \
-e TEST_IGNORE_CGROUP_CHECK \
-e TESTCOVERAGE \
-e TESTDEBUG \
-e TESTDIRS \
@@ -75,7 +77,12 @@ DOCKER_ENVS := \
-e PLATFORM \
-e DEFAULT_PRODUCT_LICENSE \
-e PRODUCT \
-e PACKAGER_NAME
-e PACKAGER_NAME \
-e PAGER \
-e GIT_PAGER \
-e OTEL_EXPORTER_OTLP_ENDPOINT \
-e OTEL_EXPORTER_OTLP_PROTOCOL \
-e OTEL_SERVICE_NAME
# note: we _cannot_ add "-e DOCKER_BUILDTAGS" here because even if it's unset in the shell, that would shadow the "ENV DOCKER_BUILDTAGS" set in our Dockerfile, which is very important for our official builds
# to allow `make BIND_DIR=. shell` or `make BIND_DIR= test`
@@ -83,7 +90,7 @@ DOCKER_ENVS := \
# note: BINDDIR is supported for backwards-compatibility here
BIND_DIR := $(if $(BINDDIR),$(BINDDIR),$(if $(DOCKER_HOST),,bundles))
# DOCKER_MOUNT can be overriden, but use at your own risk!
# DOCKER_MOUNT can be overridden, but use at your own risk!
ifndef DOCKER_MOUNT
DOCKER_MOUNT := $(if $(BIND_DIR),-v "$(CURDIR)/$(BIND_DIR):/go/src/github.com/docker/docker/$(BIND_DIR)")
DOCKER_MOUNT := $(if $(DOCKER_BINDDIR_MOUNT_OPTS),$(DOCKER_MOUNT):$(DOCKER_BINDDIR_MOUNT_OPTS),$(DOCKER_MOUNT))
@@ -107,8 +114,6 @@ DOCKER_PORT_FORWARD := $(if $(DOCKER_PORT),-p "$(DOCKER_PORT)",)
DELVE_PORT_FORWARD := $(if $(DELVE_PORT),-p "$(DELVE_PORT)",)
DOCKER_FLAGS := $(DOCKER) run --rm --privileged $(DOCKER_CONTAINER_NAME) $(DOCKER_ENVS) $(DOCKER_MOUNT) $(DOCKER_PORT_FORWARD) $(DELVE_PORT_FORWARD)
BUILD_APT_MIRROR := $(if $(DOCKER_BUILD_APT_MIRROR),--build-arg APT_MIRROR=$(DOCKER_BUILD_APT_MIRROR))
export BUILD_APT_MIRROR
SWAGGER_DOCS_PORT ?= 9000
@@ -136,25 +141,33 @@ endif
DOCKER_RUN_DOCKER := $(DOCKER_FLAGS) "$(DOCKER_IMAGE)"
DOCKER_BUILD_ARGS += --build-arg=GO_VERSION
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_VERSION
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_REPOSITORY
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_INTEGRATION_VERSION
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_INTEGRATION_REPOSITORY
ifdef DOCKER_SYSTEMD
DOCKER_BUILD_ARGS += --build-arg=SYSTEMD=true
endif
BUILD_OPTS := ${BUILD_APT_MIRROR} ${DOCKER_BUILD_ARGS} ${DOCKER_BUILD_OPTS} -f "$(DOCKERFILE)"
BUILD_OPTS := ${DOCKER_BUILD_ARGS} ${DOCKER_BUILD_OPTS}
BUILD_CMD := $(BUILDX) build
BAKE_CMD := $(BUILDX) bake
default: binary
.PHONY: all
all: build ## validate all checks, build linux binaries, run all tests,\ncross build non-linux binaries, and generate archives
$(DOCKER_RUN_DOCKER) bash -c 'hack/validate/default && hack/make.sh'
.PHONY: binary
binary: bundles ## build statically linked linux binaries
$(BAKE_CMD) binary
.PHONY: dynbinary
dynbinary: bundles ## build dynamically linked linux binaries
$(BAKE_CMD) dynbinary
.PHONY: cross
cross: bundles ## cross build the binaries
$(BAKE_CMD) binary-cross
@@ -168,12 +181,15 @@ clean: clean-cache
clean-cache: ## remove the docker volumes that are used for caching in the dev-container
docker volume rm -f docker-dev-cache docker-mod-cache
.PHONY: help
help: ## this help
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z0-9_-]+:.*?## / {gsub("\\\\n",sprintf("\n%22c",""), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
.PHONY: install
install: ## install the linux binaries
KEEPBUNDLE=1 hack/make.sh install-binary
.PHONY: run
run: build ## run the docker daemon in a container
$(DOCKER_RUN_DOCKER) sh -c "KEEPBUNDLE=1 hack/make.sh install-binary run"
@@ -186,17 +202,22 @@ endif
build: bundles
$(BUILD_CMD) $(BUILD_OPTS) $(shell_target) --load -t "$(DOCKER_IMAGE)" .
.PHONY: shell
shell: build ## start a shell inside the build env
$(DOCKER_RUN_DOCKER) bash
.PHONY: test
test: build test-unit ## run the unit, integration and docker-py tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-integration test-docker-py
.PHONY: test-docker-py
test-docker-py: build ## run the docker-py tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-docker-py
.PHONY: test-integration-cli
test-integration-cli: test-integration ## (DEPRECATED) use test-integration
.PHONY: test-integration
ifneq ($(and $(TEST_SKIP_INTEGRATION),$(TEST_SKIP_INTEGRATION_CLI)),)
test-integration:
@echo Both integrations suites skipped per environment variables
@@ -205,18 +226,29 @@ test-integration: build ## run the integration tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-integration
endif
.PHONY: test-integration-flaky
test-integration-flaky: build ## run the stress test for all new integration tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-integration-flaky
.PHONY: test-unit
test-unit: build ## run the unit tests
$(DOCKER_RUN_DOCKER) hack/test/unit
.PHONY: validate
validate: build ## validate DCO, Seccomp profile generation, gofmt,\n./pkg/ isolation, golint, tests, tomls, go vet and vendor
$(DOCKER_RUN_DOCKER) hack/validate/all
.PHONY: validate-generate-files
validate-generate-files:
$(BUILD_CMD) --target "validate" \
--output "type=cacheonly" \
--file "./hack/dockerfiles/generate-files.Dockerfile" .
.PHONY: validate-%
validate-%: build ## validate specific check
$(DOCKER_RUN_DOCKER) hack/validate/$*
.PHONY: win
win: bundles ## cross build the binary for windows
$(BAKE_CMD) --set *.platform=windows/amd64 binary
@@ -235,3 +267,16 @@ swagger-docs: ## preview the API documentation
-e 'REDOC_OPTIONS=hide-hostname="true" lazy-rendering' \
-p $(SWAGGER_DOCS_PORT):80 \
bfirsh/redoc:1.14.0
.PHONY: generate-files
generate-files:
$(eval $@_TMP_OUT := $(shell mktemp -d -t moby-output.XXXXXXXXXX))
@if [ -z "$($@_TMP_OUT)" ]; then \
echo "Temp dir is not set"; \
exit 1; \
fi
$(BUILD_CMD) --target "update" \
--output "type=local,dest=$($@_TMP_OUT)" \
--file "./hack/dockerfiles/generate-files.Dockerfile" .
cp -R "$($@_TMP_OUT)"/. .
rm -rf "$($@_TMP_OUT)"/*

View File

@@ -32,7 +32,7 @@ New projects can be added if they fit with the community goals. Docker is commit
However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.
The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful.
The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.
The releases are supported by the maintainers, community and users, on a best efforts basis only. For customers who want enterprise or commercial support, [Docker Desktop](https://www.docker.com/products/docker-desktop/) and [Mirantis Container Runtime](https://www.mirantis.com/software/mirantis-container-runtime/) are the appropriate products for these use cases.
-----

View File

@@ -37,6 +37,6 @@ There is hopefully enough example material in the file for you to copy a similar
When you make edits to `swagger.yaml`, you may want to check the generated API documentation to ensure it renders correctly.
Run `make swagger-docs` and a preview will be running at `http://localhost`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation.
Run `make swagger-docs` and a preview will be running at `http://localhost:9000`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation.
The production documentation is generated by vendoring `swagger.yaml` into [docker/docker.github.io](https://github.com/docker/docker.github.io).

View File

@@ -2,8 +2,17 @@ package api // import "github.com/docker/docker/api"
// Common constants for daemon and client.
const (
// DefaultVersion of Current REST API
DefaultVersion = "1.43"
// DefaultVersion of the current REST API.
DefaultVersion = "1.47"
// MinSupportedAPIVersion is the minimum API version that can be supported
// by the API server, specified as "major.minor". Note that the daemon
// may be configured with a different minimum API version, as returned
// in [github.com/docker/docker/api/types.Version.MinAPIVersion].
//
// API requests for API versions lower than the configured version produce
// an error.
MinSupportedAPIVersion = "1.24"
// NoBaseImageSpecifier is the symbol used by the FROM
// command to specify that no base image is to be used.

View File

@@ -1,7 +0,0 @@
//go:build !windows
// +build !windows
package api // import "github.com/docker/docker/api"
// MinVersion represents Minimum REST API version supported
const MinVersion = "1.12"

View File

@@ -1,8 +0,0 @@
package api // import "github.com/docker/docker/api"
// MinVersion represents Minimum REST API version supported
// Technically the first daemon API version released on Windows is v1.25 in
// engine version 1.13. However, some clients are explicitly using downlevel
// APIs (e.g. docker-compose v2.1 file format) and that is just too restrictive.
// Hence also allowing 1.24 on Windows.
const MinVersion string = "1.24"

View File

@@ -5,7 +5,7 @@ import (
"fmt"
"strconv"
"github.com/docker/distribution/reference"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/events"
@@ -76,7 +76,7 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
return "", nil
}
var imageID = build.ImageID
imageID := build.ImageID
if options.Squash {
if imageID, err = squashBuild(build, b.imageComponent); err != nil {
return "", err
@@ -88,11 +88,9 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
}
}
if !useBuildKit {
stdout := config.ProgressWriter.StdoutFormatter
fmt.Fprintf(stdout, "Successfully built %s\n", stringid.TruncateID(imageID))
}
if imageID != "" && !useBuildKit {
stdout := config.ProgressWriter.StdoutFormatter
_, _ = fmt.Fprintf(stdout, "Successfully built %s\n", stringid.TruncateID(imageID))
err = tagImages(ctx, b.imageComponent, config.ProgressWriter.StdoutFormatter, image.ID(imageID), tags)
}
return imageID, err
@@ -104,7 +102,7 @@ func (b *Backend) PruneCache(ctx context.Context, opts types.BuildCachePruneOpti
if err != nil {
return nil, errors.Wrap(err, "failed to prune build cache")
}
b.eventsService.Log("prune", events.BuilderEventType, events.Actor{
b.eventsService.Log(events.ActionPrune, events.BuilderEventType, events.Actor{
Attributes: map[string]string{
"reclaimed": strconv.FormatInt(buildCacheSize, 10),
},

View File

@@ -5,7 +5,7 @@ import (
"fmt"
"io"
"github.com/docker/distribution/reference"
"github.com/distribution/reference"
"github.com/docker/docker/image"
"github.com/pkg/errors"
)

View File

@@ -1,34 +0,0 @@
package server
import (
"net/http"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/versions"
"github.com/gorilla/mux"
"google.golang.org/grpc/status"
)
// makeErrorHandler makes an HTTP handler that decodes a Docker error and
// returns it in the response.
func makeErrorHandler(err error) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
statusCode := httpstatus.FromError(err)
vars := mux.Vars(r)
if apiVersionSupportsJSONErrors(vars["version"]) {
response := &types.ErrorResponse{
Message: err.Error(),
}
_ = httputils.WriteJSON(w, statusCode, response)
} else {
http.Error(w, status.Convert(err).Message(), statusCode)
}
}
}
func apiVersionSupportsJSONErrors(version string) bool {
const firstAPIVersionWithJSONErrors = "1.23"
return version == "" || versions.GreaterThan(version, firstAPIVersionWithJSONErrors)
}

View File

@@ -1,13 +1,14 @@
package httpstatus // import "github.com/docker/docker/api/server/httpstatus"
import (
"context"
"fmt"
"net/http"
cerrdefs "github.com/containerd/containerd/errdefs"
cerrdefs "github.com/containerd/errdefs"
"github.com/containerd/log"
"github.com/docker/distribution/registry/api/errcode"
"github.com/docker/docker/errdefs"
"github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
@@ -19,63 +20,55 @@ type causer interface {
// FromError retrieves status code from error message.
func FromError(err error) int {
if err == nil {
logrus.WithFields(logrus.Fields{"error": err}).Error("unexpected HTTP error handling")
log.G(context.TODO()).WithError(err).Error("unexpected HTTP error handling")
return http.StatusInternalServerError
}
var statusCode int
// Stop right there
// Are you sure you should be adding a new error class here? Do one of the existing ones work?
// Note that the below functions are already checking the error causal chain for matches.
switch {
case errdefs.IsNotFound(err):
statusCode = http.StatusNotFound
return http.StatusNotFound
case errdefs.IsInvalidParameter(err):
statusCode = http.StatusBadRequest
return http.StatusBadRequest
case errdefs.IsConflict(err):
statusCode = http.StatusConflict
return http.StatusConflict
case errdefs.IsUnauthorized(err):
statusCode = http.StatusUnauthorized
return http.StatusUnauthorized
case errdefs.IsUnavailable(err):
statusCode = http.StatusServiceUnavailable
return http.StatusServiceUnavailable
case errdefs.IsForbidden(err):
statusCode = http.StatusForbidden
return http.StatusForbidden
case errdefs.IsNotModified(err):
statusCode = http.StatusNotModified
return http.StatusNotModified
case errdefs.IsNotImplemented(err):
statusCode = http.StatusNotImplemented
return http.StatusNotImplemented
case errdefs.IsSystem(err) || errdefs.IsUnknown(err) || errdefs.IsDataLoss(err) || errdefs.IsDeadline(err) || errdefs.IsCancelled(err):
statusCode = http.StatusInternalServerError
return http.StatusInternalServerError
default:
statusCode = statusCodeFromGRPCError(err)
if statusCode != http.StatusInternalServerError {
if statusCode := statusCodeFromGRPCError(err); statusCode != http.StatusInternalServerError {
return statusCode
}
statusCode = statusCodeFromContainerdError(err)
if statusCode != http.StatusInternalServerError {
if statusCode := statusCodeFromContainerdError(err); statusCode != http.StatusInternalServerError {
return statusCode
}
statusCode = statusCodeFromDistributionError(err)
if statusCode != http.StatusInternalServerError {
if statusCode := statusCodeFromDistributionError(err); statusCode != http.StatusInternalServerError {
return statusCode
}
if e, ok := err.(causer); ok {
return FromError(e.Cause())
}
logrus.WithFields(logrus.Fields{
log.G(context.TODO()).WithFields(log.Fields{
"module": "api",
"error": err,
"error_type": fmt.Sprintf("%T", err),
}).Debugf("FIXME: Got an API for which error does not match any expected type!!!: %+v", err)
}
}).Debug("FIXME: Got an API for which error does not match any expected type!!!")
if statusCode == 0 {
statusCode = http.StatusInternalServerError
return http.StatusInternalServerError
}
return statusCode
}
// statusCodeFromGRPCError returns status code according to gRPC error

View File

@@ -12,5 +12,4 @@ import (
// container configuration.
type ContainerDecoder interface {
DecodeConfig(src io.Reader) (*container.Config, *container.HostConfig, *network.NetworkingConfig, error)
DecodeHostConfig(src io.Reader) (*container.HostConfig, error)
}

View File

@@ -1,12 +1,17 @@
package httputils // import "github.com/docker/docker/api/server/httputils"
import (
"encoding/json"
"fmt"
"net/http"
"strconv"
"strings"
"github.com/docker/distribution/reference"
"github.com/distribution/reference"
"github.com/docker/docker/errdefs"
"github.com/pkg/errors"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// BoolValue transforms a form value in different formats into a boolean type.
@@ -109,3 +114,24 @@ func ArchiveFormValues(r *http.Request, vars map[string]string) (ArchiveOptions,
}
return ArchiveOptions{name, path}, nil
}
// DecodePlatform decodes the OCI platform JSON string into a Platform struct.
func DecodePlatform(platformJSON string) (*ocispec.Platform, error) {
var p ocispec.Platform
if err := json.Unmarshal([]byte(platformJSON), &p); err != nil {
return nil, errdefs.InvalidParameter(errors.Wrap(err, "failed to parse platform"))
}
hasAnyOptional := (p.Variant != "" || p.OSVersion != "" || len(p.OSFeatures) > 0)
if p.OS == "" && p.Architecture == "" && hasAnyOptional {
return nil, errdefs.InvalidParameter(errors.New("optional platform fields provided, but OS and Architecture are missing"))
}
if p.OS == "" || p.Architecture == "" {
return nil, errdefs.InvalidParameter(errors.New("both OS and Architecture must be provided"))
}
return &p, nil
}

View File

@@ -1,9 +1,16 @@
package httputils // import "github.com/docker/docker/api/server/httputils"
import (
"encoding/json"
"net/http"
"net/url"
"testing"
"github.com/containerd/platforms"
"github.com/docker/docker/errdefs"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"gotest.tools/v3/assert"
)
func TestBoolValue(t *testing.T) {
@@ -103,3 +110,23 @@ func TestInt64ValueOrDefaultWithError(t *testing.T) {
t.Fatal("Expected an error.")
}
}
func TestParsePlatformInvalid(t *testing.T) {
for _, tc := range []ocispec.Platform{
{
OSVersion: "1.2.3",
OSFeatures: []string{"a", "b"},
},
{OSVersion: "12.0"},
{OS: "linux"},
{Architecture: "amd64"},
} {
t.Run(platforms.Format(tc), func(t *testing.T) {
js, err := json.Marshal(tc)
assert.NilError(t, err)
_, err = DecodePlatform(string(js))
assert.Check(t, errdefs.IsInvalidParameter(err))
})
}
}

View File

@@ -4,11 +4,12 @@ import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"sort"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/stdcopy"
@@ -16,7 +17,11 @@ import (
// WriteLogStream writes an encoded byte stream of log messages from the
// messages channel, multiplexing them with a stdcopy.Writer if mux is true
func WriteLogStream(_ context.Context, w io.Writer, msgs <-chan *backend.LogMessage, config *types.ContainerLogsOptions, mux bool) {
func WriteLogStream(_ context.Context, w http.ResponseWriter, msgs <-chan *backend.LogMessage, config *container.LogsOptions, mux bool) {
// See https://github.com/moby/moby/issues/47448
// Trigger headers to be written immediately.
w.WriteHeader(http.StatusOK)
wf := ioutils.NewWriteFlusher(w)
defer wf.Close()

View File

@@ -1,9 +1,9 @@
package server // import "github.com/docker/docker/api/server"
import (
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/middleware"
"github.com/sirupsen/logrus"
)
// handlerWithGlobalMiddlewares wraps the handler function for a request with
@@ -16,7 +16,7 @@ func (s *Server) handlerWithGlobalMiddlewares(handler httputils.APIFunc) httputi
next = m.WrapHandler(next)
}
if logrus.GetLevel() == logrus.DebugLevel {
if log.GetLevel() == log.DebugLevel {
next = middleware.DebugRequestMiddleware(next)
}

View File

@@ -4,17 +4,21 @@ import (
"context"
"net/http"
"github.com/containerd/log"
"github.com/docker/docker/api/types/registry"
"github.com/sirupsen/logrus"
)
// CORSMiddleware injects CORS headers to each request
// when it's configured.
//
// Deprecated: CORS headers should not be set on the API. This feature will be removed in the next release.
type CORSMiddleware struct {
defaultHeaders string
}
// NewCORSMiddleware creates a new CORSMiddleware with default headers.
//
// Deprecated: CORS headers should not be set on the API. This feature will be removed in the next release.
func NewCORSMiddleware(d string) CORSMiddleware {
return CORSMiddleware{defaultHeaders: d}
}
@@ -29,7 +33,7 @@ func (c CORSMiddleware) WrapHandler(handler func(ctx context.Context, w http.Res
corsHeaders = "*"
}
logrus.Debugf("CORS header is enabled and set to: %s", corsHeaders)
log.G(ctx).Debugf("CORS header is enabled and set to: %s", corsHeaders)
w.Header().Add("Access-Control-Allow-Origin", corsHeaders)
w.Header().Add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, "+registry.AuthHeader)
w.Header().Add("Access-Control-Allow-Methods", "HEAD, GET, POST, DELETE, PUT, OPTIONS")

View File

@@ -8,15 +8,15 @@ import (
"net/http"
"strings"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/pkg/ioutils"
"github.com/sirupsen/logrus"
)
// DebugRequestMiddleware dumps the request to logger
func DebugRequestMiddleware(handler func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error) func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
logrus.Debugf("Calling %s %s", r.Method, r.RequestURI)
log.G(ctx).Debugf("Calling %s %s", r.Method, r.RequestURI)
if r.Method != http.MethodPost {
return handler(ctx, w, r, vars)
@@ -44,9 +44,9 @@ func DebugRequestMiddleware(handler func(ctx context.Context, w http.ResponseWri
maskSecretKeys(postForm)
formStr, errMarshal := json.Marshal(postForm)
if errMarshal == nil {
logrus.Debugf("form data: %s", string(formStr))
log.G(ctx).Debugf("form data: %s", string(formStr))
} else {
logrus.Debugf("form data: %q", postForm)
log.G(ctx).Debugf("form data: %q", postForm)
}
}

View File

@@ -6,6 +6,7 @@ import (
"net/http"
"runtime"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/versions"
)
@@ -13,19 +14,40 @@ import (
// VersionMiddleware is a middleware that
// validates the client and server versions.
type VersionMiddleware struct {
serverVersion string
defaultVersion string
minVersion string
serverVersion string
// defaultAPIVersion is the default API version provided by the API server,
// specified as "major.minor". It is usually configured to the latest API
// version [github.com/docker/docker/api.DefaultVersion].
//
// API requests for API versions greater than this version are rejected by
// the server and produce a [versionUnsupportedError].
defaultAPIVersion string
// minAPIVersion is the minimum API version provided by the API server,
// specified as "major.minor".
//
// API requests for API versions lower than this version are rejected by
// the server and produce a [versionUnsupportedError].
minAPIVersion string
}
// NewVersionMiddleware creates a new VersionMiddleware
// with the default versions.
func NewVersionMiddleware(s, d, m string) VersionMiddleware {
return VersionMiddleware{
serverVersion: s,
defaultVersion: d,
minVersion: m,
// NewVersionMiddleware creates a VersionMiddleware with the given versions.
func NewVersionMiddleware(serverVersion, defaultAPIVersion, minAPIVersion string) (*VersionMiddleware, error) {
if versions.LessThan(defaultAPIVersion, api.MinSupportedAPIVersion) || versions.GreaterThan(defaultAPIVersion, api.DefaultVersion) {
return nil, fmt.Errorf("invalid default API version (%s): must be between %s and %s", defaultAPIVersion, api.MinSupportedAPIVersion, api.DefaultVersion)
}
if versions.LessThan(minAPIVersion, api.MinSupportedAPIVersion) || versions.GreaterThan(minAPIVersion, api.DefaultVersion) {
return nil, fmt.Errorf("invalid minimum API version (%s): must be between %s and %s", minAPIVersion, api.MinSupportedAPIVersion, api.DefaultVersion)
}
if versions.GreaterThan(minAPIVersion, defaultAPIVersion) {
return nil, fmt.Errorf("invalid API version: the minimum API version (%s) is higher than the default version (%s)", minAPIVersion, defaultAPIVersion)
}
return &VersionMiddleware{
serverVersion: serverVersion,
defaultAPIVersion: defaultAPIVersion,
minAPIVersion: minAPIVersion,
}, nil
}
type versionUnsupportedError struct {
@@ -45,18 +67,18 @@ func (e versionUnsupportedError) InvalidParameter() {}
func (v VersionMiddleware) WrapHandler(handler func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error) func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
w.Header().Set("Server", fmt.Sprintf("Docker/%s (%s)", v.serverVersion, runtime.GOOS))
w.Header().Set("API-Version", v.defaultVersion)
w.Header().Set("OSType", runtime.GOOS)
w.Header().Set("Api-Version", v.defaultAPIVersion)
w.Header().Set("Ostype", runtime.GOOS)
apiVersion := vars["version"]
if apiVersion == "" {
apiVersion = v.defaultVersion
apiVersion = v.defaultAPIVersion
}
if versions.LessThan(apiVersion, v.minVersion) {
return versionUnsupportedError{version: apiVersion, minVersion: v.minVersion}
if versions.LessThan(apiVersion, v.minAPIVersion) {
return versionUnsupportedError{version: apiVersion, minVersion: v.minAPIVersion}
}
if versions.GreaterThan(apiVersion, v.defaultVersion) {
return versionUnsupportedError{version: apiVersion, maxVersion: v.defaultVersion}
if versions.GreaterThan(apiVersion, v.defaultAPIVersion) {
return versionUnsupportedError{version: apiVersion, maxVersion: v.defaultAPIVersion}
}
ctx = context.WithValue(ctx, httputils.APIVersionKey{}, apiVersion)
return handler(ctx, w, r, vars)

View File

@@ -2,27 +2,82 @@ package middleware // import "github.com/docker/docker/api/server/middleware"
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"runtime"
"testing"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestNewVersionMiddlewareValidation(t *testing.T) {
tests := []struct {
doc, defaultVersion, minVersion, expectedErr string
}{
{
doc: "defaults",
defaultVersion: api.DefaultVersion,
minVersion: api.MinSupportedAPIVersion,
},
{
doc: "invalid default lower than min",
defaultVersion: api.MinSupportedAPIVersion,
minVersion: api.DefaultVersion,
expectedErr: fmt.Sprintf("invalid API version: the minimum API version (%s) is higher than the default version (%s)", api.DefaultVersion, api.MinSupportedAPIVersion),
},
{
doc: "invalid default too low",
defaultVersion: "0.1",
minVersion: api.MinSupportedAPIVersion,
expectedErr: fmt.Sprintf("invalid default API version (0.1): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid default too high",
defaultVersion: "9999.9999",
minVersion: api.DefaultVersion,
expectedErr: fmt.Sprintf("invalid default API version (9999.9999): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid minimum too low",
defaultVersion: api.MinSupportedAPIVersion,
minVersion: "0.1",
expectedErr: fmt.Sprintf("invalid minimum API version (0.1): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid minimum too high",
defaultVersion: api.DefaultVersion,
minVersion: "9999.9999",
expectedErr: fmt.Sprintf("invalid minimum API version (9999.9999): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.doc, func(t *testing.T) {
_, err := NewVersionMiddleware("1.2.3", tc.defaultVersion, tc.minVersion)
if tc.expectedErr == "" {
assert.Check(t, err)
} else {
assert.Check(t, is.Error(err, tc.expectedErr))
}
})
}
}
func TestVersionMiddlewareVersion(t *testing.T) {
defaultVersion := "1.10.0"
minVersion := "1.2.0"
expectedVersion := defaultVersion
expectedVersion := "<not set>"
handler := func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
v := httputils.VersionFromContext(ctx)
assert.Check(t, is.Equal(expectedVersion, v))
return nil
}
m := NewVersionMiddleware(defaultVersion, defaultVersion, minVersion)
m, err := NewVersionMiddleware("1.2.3", api.DefaultVersion, api.MinSupportedAPIVersion)
assert.NilError(t, err)
h := m.WrapHandler(handler)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
@@ -35,19 +90,19 @@ func TestVersionMiddlewareVersion(t *testing.T) {
errString string
}{
{
expectedVersion: "1.10.0",
expectedVersion: api.DefaultVersion,
},
{
reqVersion: "1.9.0",
expectedVersion: "1.9.0",
reqVersion: api.MinSupportedAPIVersion,
expectedVersion: api.MinSupportedAPIVersion,
},
{
reqVersion: "0.1",
errString: "client version 0.1 is too old. Minimum supported API version is 1.2.0, please upgrade your client to a newer version",
errString: fmt.Sprintf("client version 0.1 is too old. Minimum supported API version is %s, please upgrade your client to a newer version", api.MinSupportedAPIVersion),
},
{
reqVersion: "9999.9999",
errString: "client version 9999.9999 is too new. Maximum supported API version is 1.10.0",
errString: fmt.Sprintf("client version 9999.9999 is too new. Maximum supported API version is %s", api.DefaultVersion),
},
}
@@ -71,9 +126,8 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
return nil
}
defaultVersion := "1.10.0"
minVersion := "1.2.0"
m := NewVersionMiddleware(defaultVersion, defaultVersion, minVersion)
m, err := NewVersionMiddleware("1.2.3", api.DefaultVersion, api.MinSupportedAPIVersion)
assert.NilError(t, err)
h := m.WrapHandler(handler)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
@@ -81,12 +135,12 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
ctx := context.Background()
vars := map[string]string{"version": "0.1"}
err := h(ctx, resp, req, vars)
err = h(ctx, resp, req, vars)
assert.Check(t, is.ErrorContains(err, ""))
hdr := resp.Result().Header
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/"+defaultVersion))
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/1.2.3"))
assert.Check(t, is.Contains(hdr.Get("Server"), runtime.GOOS))
assert.Check(t, is.Equal(hdr.Get("API-Version"), defaultVersion))
assert.Check(t, is.Equal(hdr.Get("OSType"), runtime.GOOS))
assert.Check(t, is.Equal(hdr.Get("Api-Version"), api.DefaultVersion))
assert.Check(t, is.Equal(hdr.Get("Ostype"), runtime.GOOS))
}

View File

@@ -15,7 +15,6 @@ type Backend interface {
// Prune build cache
PruneCache(context.Context, types.BuildCachePruneOptions) (*types.BuildCachePruneReport, error)
Cancel(context.Context, string) error
}

View File

@@ -9,18 +9,16 @@ import (
// buildRouter is a router to talk with the build controller
type buildRouter struct {
backend Backend
daemon experimentalProvider
routes []router.Route
features *map[string]bool
backend Backend
daemon experimentalProvider
routes []router.Route
}
// NewRouter initializes a new build router
func NewRouter(b Backend, d experimentalProvider, features *map[string]bool) router.Router {
func NewRouter(b Backend, d experimentalProvider) router.Router {
r := &buildRouter{
backend: b,
daemon: d,
features: features,
backend: b,
daemon: d,
}
r.initRoutes()
return r

View File

@@ -14,6 +14,7 @@ import (
"strings"
"sync"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
@@ -24,9 +25,7 @@ import (
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/progress"
"github.com/docker/docker/pkg/streamformatter"
units "github.com/docker/go-units"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type invalidParam struct {
@@ -42,6 +41,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
SuppressOutput: httputils.BoolValue(r, "q"),
NoCache: httputils.BoolValue(r, "nocache"),
ForceRemove: httputils.BoolValue(r, "forcerm"),
PullParent: httputils.BoolValue(r, "pull"),
MemorySwap: httputils.Int64ValueOrZero(r, "memswap"),
Memory: httputils.Int64ValueOrZero(r, "memory"),
CPUShares: httputils.Int64ValueOrZero(r, "cpushares"),
@@ -66,17 +66,14 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
return nil, invalidParam{errors.New("security options are not supported on " + runtime.GOOS)}
}
version := httputils.VersionFromContext(ctx)
if httputils.BoolValue(r, "forcerm") && versions.GreaterThanOrEqualTo(version, "1.12") {
if httputils.BoolValue(r, "forcerm") {
options.Remove = true
} else if r.FormValue("rm") == "" && versions.GreaterThanOrEqualTo(version, "1.12") {
} else if r.FormValue("rm") == "" {
options.Remove = true
} else {
options.Remove = httputils.BoolValue(r, "rm")
}
if httputils.BoolValue(r, "pull") && versions.GreaterThanOrEqualTo(version, "1.16") {
options.PullParent = true
}
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.32") {
options.Platform = r.FormValue("platform")
}
@@ -107,7 +104,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
}
if ulimitsJSON := r.FormValue("ulimits"); ulimitsJSON != "" {
var buildUlimits = []*units.Ulimit{}
buildUlimits := []*container.Ulimit{}
if err := json.Unmarshal([]byte(ulimitsJSON), &buildUlimits); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading ulimit settings")}
}
@@ -127,7 +124,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
// so that it can print a warning about "foo" being unused if there is
// no "ARG foo" in the Dockerfile.
if buildArgsJSON := r.FormValue("buildargs"); buildArgsJSON != "" {
var buildArgs = map[string]*string{}
buildArgs := map[string]*string{}
if err := json.Unmarshal([]byte(buildArgsJSON), &buildArgs); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading build args")}
}
@@ -135,7 +132,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
}
if labelsJSON := r.FormValue("labels"); labelsJSON != "" {
var labels = map[string]string{}
labels := map[string]string{}
if err := json.Unmarshal([]byte(labelsJSON), &labels); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading labels")}
}
@@ -143,7 +140,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
}
if cacheFromJSON := r.FormValue("cachefrom"); cacheFromJSON != "" {
var cacheFrom = []string{}
cacheFrom := []string{}
if err := json.Unmarshal([]byte(cacheFromJSON), &cacheFrom); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading cache-from")}
}
@@ -248,7 +245,7 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
}
_, err = output.Write(streamformatter.FormatError(err))
if err != nil {
logrus.Warnf("could not write error response: %v", err)
log.G(ctx).Warnf("could not write error response: %v", err)
}
return nil
}
@@ -342,8 +339,12 @@ type flusher interface {
Flush()
}
type nopFlusher struct{}
func (f *nopFlusher) Flush() {}
func wrapOutputBufferedUntilRequestRead(rc io.ReadCloser, out io.Writer) (io.ReadCloser, io.Writer) {
var fl flusher = &ioutils.NopFlusher{}
var fl flusher = &nopFlusher{}
if f, ok := out.(flusher); ok {
fl = f
}

View File

@@ -1,10 +1,10 @@
package checkpoint // import "github.com/docker/docker/api/server/router/checkpoint"
import "github.com/docker/docker/api/types"
import "github.com/docker/docker/api/types/checkpoint"
// Backend for Checkpoint
type Backend interface {
CheckpointCreate(container string, config types.CheckpointCreateOptions) error
CheckpointDelete(container string, config types.CheckpointDeleteOptions) error
CheckpointList(container string, config types.CheckpointListOptions) ([]types.Checkpoint, error)
CheckpointCreate(container string, config checkpoint.CreateOptions) error
CheckpointDelete(container string, config checkpoint.DeleteOptions) error
CheckpointList(container string, config checkpoint.ListOptions) ([]checkpoint.Summary, error)
}

View File

@@ -5,7 +5,7 @@ import (
"net/http"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/checkpoint"
)
func (s *checkpointRouter) postContainerCheckpoint(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -13,7 +13,7 @@ func (s *checkpointRouter) postContainerCheckpoint(ctx context.Context, w http.R
return err
}
var options types.CheckpointCreateOptions
var options checkpoint.CreateOptions
if err := httputils.ReadJSON(r, &options); err != nil {
return err
}
@@ -32,10 +32,9 @@ func (s *checkpointRouter) getContainerCheckpoints(ctx context.Context, w http.R
return err
}
checkpoints, err := s.backend.CheckpointList(vars["name"], types.CheckpointListOptions{
checkpoints, err := s.backend.CheckpointList(vars["name"], checkpoint.ListOptions{
CheckpointDir: r.Form.Get("dir"),
})
if err != nil {
return err
}
@@ -48,11 +47,10 @@ func (s *checkpointRouter) deleteContainerCheckpoint(ctx context.Context, w http
return err
}
err := s.backend.CheckpointDelete(vars["name"], types.CheckpointDeleteOptions{
err := s.backend.CheckpointDelete(vars["name"], checkpoint.DeleteOptions{
CheckpointDir: r.Form.Get("dir"),
CheckpointID: vars["checkpoint"],
})
if err != nil {
return err
}

View File

@@ -14,32 +14,31 @@ import (
// execBackend includes functions to implement to provide exec functionality.
type execBackend interface {
ContainerExecCreate(name string, config *types.ExecConfig) (string, error)
ContainerExecCreate(name string, options *container.ExecOptions) (string, error)
ContainerExecInspect(id string) (*backend.ExecInspect, error)
ContainerExecResize(name string, height, width int) error
ContainerExecStart(ctx context.Context, name string, options container.ExecStartOptions) error
ContainerExecStart(ctx context.Context, name string, options backend.ExecStartConfig) error
ExecExists(name string) (bool, error)
}
// copyBackend includes functions to implement to provide container copy functionality.
type copyBackend interface {
ContainerArchivePath(name string, path string) (content io.ReadCloser, stat *types.ContainerPathStat, err error)
ContainerCopy(name string, res string) (io.ReadCloser, error)
ContainerArchivePath(name string, path string) (content io.ReadCloser, stat *container.PathStat, err error)
ContainerExport(ctx context.Context, name string, out io.Writer) error
ContainerExtractToDir(name, path string, copyUIDGID, noOverwriteDirNonDir bool, content io.Reader) error
ContainerStatPath(name string, path string) (stat *types.ContainerPathStat, err error)
ContainerStatPath(name string, path string) (stat *container.PathStat, err error)
}
// stateBackend includes functions to implement to provide container state lifecycle functionality.
type stateBackend interface {
ContainerCreate(ctx context.Context, config types.ContainerCreateConfig) (container.CreateResponse, error)
ContainerCreate(ctx context.Context, config backend.ContainerCreateConfig) (container.CreateResponse, error)
ContainerKill(name string, signal string) error
ContainerPause(name string) error
ContainerRename(oldName, newName string) error
ContainerResize(name string, height, width int) error
ContainerRestart(ctx context.Context, name string, options container.StopOptions) error
ContainerRm(name string, config *types.ContainerRmConfig) error
ContainerStart(ctx context.Context, name string, hostConfig *container.HostConfig, checkpoint string, checkpointDir string) error
ContainerRm(name string, config *backend.ContainerRmConfig) error
ContainerStart(ctx context.Context, name string, checkpoint string, checkpointDir string) error
ContainerStop(ctx context.Context, name string, options container.StopOptions) error
ContainerUnpause(name string) error
ContainerUpdate(name string, hostConfig *container.HostConfig) (container.ContainerUpdateOKBody, error)
@@ -50,11 +49,10 @@ type stateBackend interface {
type monitorBackend interface {
ContainerChanges(ctx context.Context, name string) ([]archive.Change, error)
ContainerInspect(ctx context.Context, name string, size bool, version string) (interface{}, error)
ContainerLogs(ctx context.Context, name string, config *types.ContainerLogsOptions) (msgs <-chan *backend.LogMessage, tty bool, err error)
ContainerLogs(ctx context.Context, name string, config *container.LogsOptions) (msgs <-chan *backend.LogMessage, tty bool, err error)
ContainerStats(ctx context.Context, name string, config *backend.ContainerStatsConfig) error
ContainerTop(name string, psArgs string) (*container.ContainerTopOKBody, error)
Containers(ctx context.Context, config *types.ContainerListOptions) ([]*types.Container, error)
Containers(ctx context.Context, config *container.ListOptions) ([]*types.Container, error)
}
// attachBackend includes function to implement to provide container attaching functionality.
@@ -64,7 +62,7 @@ type attachBackend interface {
// systemBackend includes functions to implement to provide system wide containers functionality
type systemBackend interface {
ContainersPrune(ctx context.Context, pruneFilters filters.Args) (*types.ContainersPruneReport, error)
ContainersPrune(ctx context.Context, pruneFilters filters.Args) (*container.PruneReport, error)
}
type commitBackend interface {

View File

@@ -56,7 +56,6 @@ func (r *containerRouter) initRoutes() {
router.NewPostRoute("/containers/{name:.*}/wait", r.postContainersWait),
router.NewPostRoute("/containers/{name:.*}/resize", r.postContainersResize),
router.NewPostRoute("/containers/{name:.*}/attach", r.postContainersAttach),
router.NewPostRoute("/containers/{name:.*}/copy", r.postContainersCopy), // Deprecated since 1.8 (API v1.20), errors out since 1.12 (API v1.24)
router.NewPostRoute("/containers/{name:.*}/exec", r.postContainerExecCreate),
router.NewPostRoute("/exec/{name:.*}/start", r.postContainerExecStart),
router.NewPostRoute("/exec/{name:.*}/resize", r.postContainerExecResize),

View File

@@ -8,8 +8,10 @@ import (
"net/http"
"runtime"
"strconv"
"strings"
"github.com/containerd/containerd/platforms"
"github.com/containerd/log"
"github.com/containerd/platforms"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
@@ -17,13 +19,17 @@ import (
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/versions"
containerpkg "github.com/docker/docker/container"
networkSettings "github.com/docker/docker/daemon/network"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/libnetwork/netlabel"
"github.com/docker/docker/pkg/ioutils"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/docker/docker/runconfig"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"go.opentelemetry.io/otel"
"golang.org/x/net/websocket"
)
@@ -36,15 +42,15 @@ func (s *containerRouter) postCommit(ctx context.Context, w http.ResponseWriter,
return err
}
// TODO: remove pause arg, and always pause in backend
pause := httputils.BoolValue(r, "pause")
version := httputils.VersionFromContext(ctx)
if r.FormValue("pause") == "" && versions.GreaterThanOrEqualTo(version, "1.13") {
pause = true
}
// FIXME(thaJeztah): change this to unmarshal just [container.Config]:
// The commit endpoint accepts a [container.Config], but the decoder uses a
// [container.CreateRequest], which is a superset, and also contains
// [container.HostConfig] and [network.NetworkConfig]. Those structs
// are discarded here, but decoder.DecodeConfig also performs validation,
// so a request containing those additional fields would result in a
// validation error.
config, _, _, err := s.decoder.DecodeConfig(r.Body)
if err != nil && err != io.EOF { // Do not fail if body is empty.
if err != nil && !errors.Is(err, io.EOF) { // Do not fail if body is empty.
return err
}
@@ -54,7 +60,7 @@ func (s *containerRouter) postCommit(ctx context.Context, w http.ResponseWriter,
}
imgID, err := s.backend.CreateImageFromContainer(ctx, r.Form.Get("container"), &backend.CreateImageConfig{
Pause: pause,
Pause: httputils.BoolValueOrDefault(r, "pause", true), // TODO(dnephin): remove pause arg, and always pause in backend
Tag: ref,
Author: r.Form.Get("author"),
Comment: r.Form.Get("comment"),
@@ -77,7 +83,7 @@ func (s *containerRouter) getContainersJSON(ctx context.Context, w http.Response
return err
}
config := &types.ContainerListOptions{
config := &container.ListOptions{
All: httputils.BoolValue(r, "all"),
Size: httputils.BoolValue(r, "size"),
Since: r.Form.Get("since"),
@@ -98,6 +104,15 @@ func (s *containerRouter) getContainersJSON(ctx context.Context, w http.Response
return err
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.46") {
for _, c := range containers {
// Ignore HostConfig.Annotations because it was added in API v1.46.
c.HostConfig.Annotations = nil
}
}
return httputils.WriteJSON(w, http.StatusOK, containers)
}
@@ -115,14 +130,20 @@ func (s *containerRouter) getContainersStats(ctx context.Context, w http.Respons
oneShot = httputils.BoolValueOrDefault(r, "one-shot", false)
}
config := &backend.ContainerStatsConfig{
Stream: stream,
OneShot: oneShot,
OutStream: w,
Version: httputils.VersionFromContext(ctx),
}
return s.backend.ContainerStats(ctx, vars["name"], config)
return s.backend.ContainerStats(ctx, vars["name"], &backend.ContainerStatsConfig{
Stream: stream,
OneShot: oneShot,
OutStream: func() io.Writer {
// Assume that when this is called the request is OK.
w.WriteHeader(http.StatusOK)
if !stream {
return w
}
wf := ioutils.NewWriteFlusher(w)
wf.Flush()
return wf
},
})
}
func (s *containerRouter) getContainersLogs(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -141,7 +162,7 @@ func (s *containerRouter) getContainersLogs(ctx context.Context, w http.Response
}
containerName := vars["name"]
logsConfig := &types.ContainerLogsOptions{
logsConfig := &container.LogsOptions{
Follow: httputils.BoolValue(r, "follow"),
Timestamps: httputils.BoolValue(r, "timestamps"),
Since: r.Form.Get("since"),
@@ -175,48 +196,27 @@ func (s *containerRouter) getContainersExport(ctx context.Context, w http.Respon
return s.backend.ContainerExport(ctx, vars["name"], w)
}
type bodyOnStartError struct{}
func (bodyOnStartError) Error() string {
return "starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"
}
func (bodyOnStartError) InvalidParameter() {}
func (s *containerRouter) postContainersStart(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
ctx, span := otel.Tracer("").Start(ctx, "containerRouter.postContainersStart")
defer span.End()
// If contentLength is -1, we can assumed chunked encoding
// or more technically that the length is unknown
// https://golang.org/src/pkg/net/http/request.go#L139
// net/http otherwise seems to swallow any headers related to chunked encoding
// including r.TransferEncoding
// allow a nil body for backwards compatibility
version := httputils.VersionFromContext(ctx)
var hostConfig *container.HostConfig
//
// A non-nil json object is at least 7 characters.
if r.ContentLength > 7 || r.ContentLength == -1 {
if versions.GreaterThanOrEqualTo(version, "1.24") {
return bodyOnStartError{}
}
if err := httputils.CheckForJSON(r); err != nil {
return err
}
c, err := s.decoder.DecodeHostConfig(r.Body)
if err != nil {
return err
}
hostConfig = c
return errdefs.InvalidParameter(errors.New("starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"))
}
if err := httputils.ParseForm(r); err != nil {
return err
}
checkpoint := r.Form.Get("checkpoint")
checkpointDir := r.Form.Get("checkpoint-dir")
if err := s.backend.ContainerStart(ctx, vars["name"], hostConfig, checkpoint, checkpointDir); err != nil {
if err := s.backend.ContainerStart(ctx, vars["name"], r.Form.Get("checkpoint"), r.Form.Get("checkpoint-dir")); err != nil {
return err
}
@@ -252,25 +252,14 @@ func (s *containerRouter) postContainersStop(ctx context.Context, w http.Respons
return nil
}
func (s *containerRouter) postContainersKill(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *containerRouter) postContainersKill(_ context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
name := vars["name"]
if err := s.backend.ContainerKill(name, r.Form.Get("signal")); err != nil {
var isStopped bool
if errdefs.IsConflict(err) {
isStopped = true
}
// Return error that's not caused because the container is stopped.
// Return error if the container is not running and the api is >= 1.20
// to keep backwards compatibility.
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.20") || !isStopped {
return errors.Wrapf(err, "Cannot kill container: %s", name)
}
return errors.Wrapf(err, "cannot kill container: %s", name)
}
w.WriteHeader(http.StatusNoContent)
@@ -486,23 +475,55 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
config, hostConfig, networkingConfig, err := s.decoder.DecodeConfig(r.Body)
if err != nil {
if errors.Is(err, io.EOF) {
return errdefs.InvalidParameter(errors.New("invalid JSON: got EOF while reading request body"))
}
return err
}
if config == nil {
return errdefs.InvalidParameter(runconfig.ErrEmptyConfig)
}
if hostConfig == nil {
hostConfig = &container.HostConfig{}
}
if networkingConfig == nil {
networkingConfig = &network.NetworkingConfig{}
}
if networkingConfig.EndpointsConfig == nil {
networkingConfig.EndpointsConfig = make(map[string]*network.EndpointSettings)
}
// The NetworkMode "default" is used as a way to express a container should
// be attached to the OS-dependant default network, in an OS-independent
// way. Doing this conversion as soon as possible ensures we have less
// NetworkMode to handle down the path (including in the
// backward-compatibility layer we have just below).
//
// Note that this is not the only place where this conversion has to be
// done (as there are various other places where containers get created).
if hostConfig.NetworkMode == "" || hostConfig.NetworkMode.IsDefault() {
hostConfig.NetworkMode = networkSettings.DefaultNetwork
if nw, ok := networkingConfig.EndpointsConfig[network.NetworkDefault]; ok {
networkingConfig.EndpointsConfig[hostConfig.NetworkMode.NetworkName()] = nw
delete(networkingConfig.EndpointsConfig, network.NetworkDefault)
}
}
version := httputils.VersionFromContext(ctx)
adjustCPUShares := versions.LessThan(version, "1.19")
// When using API 1.24 and under, the client is responsible for removing the container
if hostConfig != nil && versions.LessThan(version, "1.25") {
if versions.LessThan(version, "1.25") {
hostConfig.AutoRemove = false
}
if hostConfig != nil && versions.LessThan(version, "1.40") {
if versions.LessThan(version, "1.40") {
// Ignore BindOptions.NonRecursive because it was added in API 1.40.
for _, m := range hostConfig.Mounts {
if bo := m.BindOptions; bo != nil {
bo.NonRecursive = false
}
}
// Ignore KernelMemoryTCP because it was added in API 1.40.
hostConfig.KernelMemoryTCP = 0
@@ -511,14 +532,26 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
hostConfig.IpcMode = container.IPCModeShareable
}
}
if hostConfig != nil && versions.LessThan(version, "1.41") && !s.cgroup2 {
if versions.LessThan(version, "1.41") {
// Older clients expect the default to be "host" on cgroup v1 hosts
if hostConfig.CgroupnsMode.IsEmpty() {
if !s.cgroup2 && hostConfig.CgroupnsMode.IsEmpty() {
hostConfig.CgroupnsMode = container.CgroupnsModeHost
}
}
if hostConfig != nil && versions.LessThan(version, "1.42") {
var platform *ocispec.Platform
if versions.GreaterThanOrEqualTo(version, "1.41") {
if v := r.Form.Get("platform"); v != "" {
p, err := platforms.Parse(v)
if err != nil {
return errdefs.InvalidParameter(err)
}
platform = &p
}
}
if versions.LessThan(version, "1.42") {
for _, m := range hostConfig.Mounts {
// Ignore BindOptions.CreateMountpoint because it was added in API 1.42.
if bo := m.BindOptions; bo != nil {
@@ -538,9 +571,14 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
bo.CreateMountpoint = false
}
}
if runtime.GOOS == "linux" {
// ConsoleSize is not respected by Linux daemon before API 1.42
hostConfig.ConsoleSize = [2]uint{0, 0}
}
}
if hostConfig != nil && versions.GreaterThanOrEqualTo(version, "1.42") {
if versions.GreaterThanOrEqualTo(version, "1.42") {
// Ignore KernelMemory removed in API 1.42.
hostConfig.KernelMemory = 0
for _, m := range hostConfig.Mounts {
@@ -556,28 +594,69 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
}
}
if hostConfig != nil && runtime.GOOS == "linux" && versions.LessThan(version, "1.42") {
// ConsoleSize is not respected by Linux daemon before API 1.42
hostConfig.ConsoleSize = [2]uint{0, 0}
}
if hostConfig != nil && versions.LessThan(version, "1.43") {
if versions.LessThan(version, "1.43") {
// Ignore Annotations because it was added in API v1.43.
hostConfig.Annotations = nil
}
var platform *specs.Platform
if versions.GreaterThanOrEqualTo(version, "1.41") {
if v := r.Form.Get("platform"); v != "" {
p, err := platforms.Parse(v)
if err != nil {
return errdefs.InvalidParameter(err)
defaultReadOnlyNonRecursive := false
if versions.LessThan(version, "1.44") {
if config.Healthcheck != nil {
// StartInterval was added in API 1.44
config.Healthcheck.StartInterval = 0
}
// Set ReadOnlyNonRecursive to true because it was added in API 1.44
// Before that all read-only mounts were non-recursive.
// Keep that behavior for clients on older APIs.
defaultReadOnlyNonRecursive = true
for _, m := range hostConfig.Mounts {
if m.Type == mount.TypeBind {
if m.BindOptions != nil && m.BindOptions.ReadOnlyForceRecursive {
// NOTE: that technically this is a breaking change for older
// API versions, and we should ignore the new field.
// However, this option may be incorrectly set by a client with
// the expectation that the failing to apply recursive read-only
// is enforced, so we decided to produce an error instead,
// instead of silently ignoring.
return errdefs.InvalidParameter(errors.New("BindOptions.ReadOnlyForceRecursive needs API v1.44 or newer"))
}
}
platform = &p
}
// Creating a container connected to several networks is not supported until v1.44.
if len(networkingConfig.EndpointsConfig) > 1 {
l := make([]string, 0, len(networkingConfig.EndpointsConfig))
for k := range networkingConfig.EndpointsConfig {
l = append(l, k)
}
return errdefs.InvalidParameter(errors.Errorf("Container cannot be created with multiple network endpoints: %s", strings.Join(l, ", ")))
}
}
if hostConfig != nil && hostConfig.PidsLimit != nil && *hostConfig.PidsLimit <= 0 {
if versions.LessThan(version, "1.45") {
for _, m := range hostConfig.Mounts {
if m.VolumeOptions != nil && m.VolumeOptions.Subpath != "" {
return errdefs.InvalidParameter(errors.New("VolumeOptions.Subpath needs API v1.45 or newer"))
}
}
}
var warnings []string
if warn, err := handleMACAddressBC(config, hostConfig, networkingConfig, version); err != nil {
return err
} else if warn != "" {
warnings = append(warnings, warn)
}
if warn, err := handleSysctlBC(hostConfig, networkingConfig, version); err != nil {
return err
} else if warn != "" {
warnings = append(warnings, warn)
}
if hostConfig.PidsLimit != nil && *hostConfig.PidsLimit <= 0 {
// Don't set a limit if either no limit was specified, or "unlimited" was
// explicitly set.
// Both `0` and `-1` are accepted as "unlimited", and historically any
@@ -585,28 +664,222 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
hostConfig.PidsLimit = nil
}
ccr, err := s.backend.ContainerCreate(ctx, types.ContainerCreateConfig{
Name: name,
Config: config,
HostConfig: hostConfig,
NetworkingConfig: networkingConfig,
AdjustCPUShares: adjustCPUShares,
Platform: platform,
ccr, err := s.backend.ContainerCreate(ctx, backend.ContainerCreateConfig{
Name: name,
Config: config,
HostConfig: hostConfig,
NetworkingConfig: networkingConfig,
Platform: platform,
DefaultReadOnlyNonRecursive: defaultReadOnlyNonRecursive,
})
if err != nil {
return err
}
ccr.Warnings = append(ccr.Warnings, warnings...)
return httputils.WriteJSON(w, http.StatusCreated, ccr)
}
// handleMACAddressBC takes care of backward-compatibility for the container-wide MAC address by mutating the
// networkingConfig to set the endpoint-specific MACAddress field introduced in API v1.44. It returns a warning message
// or an error if the container-wide field was specified for API >= v1.44.
func handleMACAddressBC(config *container.Config, hostConfig *container.HostConfig, networkingConfig *network.NetworkingConfig, version string) (string, error) {
deprecatedMacAddress := config.MacAddress //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
// For older versions of the API, migrate the container-wide MAC address to EndpointsConfig.
if versions.LessThan(version, "1.44") {
if deprecatedMacAddress == "" {
// If a MAC address is supplied in EndpointsConfig, discard it because the old API
// would have ignored it.
for _, ep := range networkingConfig.EndpointsConfig {
ep.MacAddress = ""
}
return "", nil
}
if !hostConfig.NetworkMode.IsBridge() && !hostConfig.NetworkMode.IsUserDefined() {
return "", runconfig.ErrConflictContainerNetworkAndMac
}
epConfig, err := epConfigForNetMode(version, hostConfig.NetworkMode, networkingConfig)
if err != nil {
return "", err
}
epConfig.MacAddress = deprecatedMacAddress
return "", nil
}
// The container-wide MacAddress parameter is deprecated and should now be specified in EndpointsConfig.
if deprecatedMacAddress == "" {
return "", nil
}
var warning string
if hostConfig.NetworkMode.IsBridge() || hostConfig.NetworkMode.IsUserDefined() {
ep, err := epConfigForNetMode(version, hostConfig.NetworkMode, networkingConfig)
if err != nil {
return "", errors.Wrap(err, "unable to migrate container-wide MAC address to a specific network")
}
// ep is the endpoint that needs the container-wide MAC address; migrate the address
// to it, or bail out if there's a mismatch.
if ep.MacAddress == "" {
ep.MacAddress = deprecatedMacAddress
} else if ep.MacAddress != deprecatedMacAddress {
return "", errdefs.InvalidParameter(errors.New("the container-wide MAC address must match the endpoint-specific MAC address for the main network, or be left empty"))
}
}
warning = "The container-wide MacAddress field is now deprecated. It should be specified in EndpointsConfig instead."
config.MacAddress = "" //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
return warning, nil
}
// handleSysctlBC migrates top level network endpoint-specific '--sysctl'
// settings to an DriverOpts for an endpoint. This is necessary because sysctls
// are applied during container task creation, but sysctls that name an interface
// (for example 'net.ipv6.conf.eth0.forwarding') cannot be applied until the
// interface has been created. So, these settings are removed from hostConfig.Sysctls
// and added to DriverOpts[netlabel.EndpointSysctls].
//
// Because interface names ('ethN') are allocated sequentially, and the order of
// network connections is not deterministic on container restart, only 'eth0'
// would work reliably in a top-level '--sysctl' option, and then only when
// there's a single initial network connection. So, settings for 'eth0' are
// migrated to the primary interface, identified by 'hostConfig.NetworkMode'.
// Settings for other interfaces are treated as errors.
//
// In the DriverOpts, because the interface name cannot be determined in advance, the
// interface name is replaced by "IFNAME". For example, 'net.ipv6.conf.eth0.forwarding'
// becomes 'net.ipv6.conf.IFNAME.forwarding'. The value in DriverOpts is a
// comma-separated list.
//
// A warning is generated when settings are migrated.
func handleSysctlBC(
hostConfig *container.HostConfig,
netConfig *network.NetworkingConfig,
version string,
) (string, error) {
if !hostConfig.NetworkMode.IsPrivate() {
return "", nil
}
var ep *network.EndpointSettings
var toDelete []string
var netIfSysctls []string
for k, v := range hostConfig.Sysctls {
// If the sysctl name matches "net.*.*.eth0.*" ...
if spl := strings.SplitN(k, ".", 5); len(spl) == 5 && spl[0] == "net" && strings.HasPrefix(spl[3], "eth") {
netIfSysctl := fmt.Sprintf("net.%s.%s.IFNAME.%s=%s", spl[1], spl[2], spl[4], v)
// Find the EndpointConfig to migrate settings to, if not already found.
if ep == nil {
/* TODO(robmry) - apply this to the API version used in 28.0.0
// Per-endpoint sysctls were introduced in API version 1.46. Migration is
// needed, but refuse to do it automatically for newer versions of the API.
if versions.GreaterThan(version, "1.??") {
return "", fmt.Errorf("interface specific sysctl setting %q must be supplied using driver option '%s'",
k, netlabel.EndpointSysctls)
}
*/
var err error
ep, err = epConfigForNetMode(version, hostConfig.NetworkMode, netConfig)
if err != nil {
return "", fmt.Errorf("unable to find a network for sysctl %s: %w", k, err)
}
}
// Only try to migrate settings for "eth0", anything else would always
// have behaved unpredictably.
if spl[3] != "eth0" {
return "", fmt.Errorf(`unable to determine network endpoint for sysctl %s, use driver option '%s' to set per-interface sysctls`,
k, netlabel.EndpointSysctls)
}
// Prepare the migration.
toDelete = append(toDelete, k)
netIfSysctls = append(netIfSysctls, netIfSysctl)
}
}
if ep == nil {
return "", nil
}
newDriverOpt := strings.Join(netIfSysctls, ",")
warning := fmt.Sprintf(`Migrated sysctl %q to DriverOpts{%q:%q}.`,
strings.Join(toDelete, ","),
netlabel.EndpointSysctls, newDriverOpt)
// Append existing per-endpoint sysctls to the migrated sysctls (give priority
// to per-endpoint settings).
if ep.DriverOpts == nil {
ep.DriverOpts = map[string]string{}
}
if oldDriverOpt, ok := ep.DriverOpts[netlabel.EndpointSysctls]; ok {
newDriverOpt += "," + oldDriverOpt
}
ep.DriverOpts[netlabel.EndpointSysctls] = newDriverOpt
// Delete migrated settings from the top-level sysctls.
for _, k := range toDelete {
delete(hostConfig.Sysctls, k)
}
return warning, nil
}
// epConfigForNetMode finds, or creates, an entry in netConfig.EndpointsConfig
// corresponding to nwMode.
//
// nwMode.NetworkName() may be the network's name, its id, or its short-id.
//
// The corresponding endpoint in netConfig.EndpointsConfig may be keyed on a
// different one of name/id/short-id. If there's any ambiguity (there are
// endpoints but the names don't match), return an error and do not create a new
// endpoint, because it might be a duplicate.
func epConfigForNetMode(
version string,
nwMode container.NetworkMode,
netConfig *network.NetworkingConfig,
) (*network.EndpointSettings, error) {
nwName := nwMode.NetworkName()
// It's always safe to create an EndpointsConfig entry under nwName if there are
// no entries already (because there can't be an entry for this network nwName
// refers to under any other name/short-id/id).
if len(netConfig.EndpointsConfig) == 0 {
es := &network.EndpointSettings{}
netConfig.EndpointsConfig = map[string]*network.EndpointSettings{
nwName: es,
}
return es, nil
}
// There cannot be more than one entry in EndpointsConfig with API < 1.44.
if versions.LessThan(version, "1.44") {
// No need to check for a match between NetworkMode and the names/ids in EndpointsConfig,
// the old version of the API would pick this network anyway.
for _, ep := range netConfig.EndpointsConfig {
return ep, nil
}
}
// There is existing endpoint config - if it's not indexed by NetworkMode.Name(), we
// can't tell which network the container-wide settings are intended for. NetworkMode,
// the keys in EndpointsConfig and the NetworkID in EndpointsConfig may mix network
// name/id/short-id. It's not safe to create EndpointsConfig under the NetworkMode
// name to store the container-wide setting, because that may result in two sets
// of EndpointsConfig for the same network and one set will be discarded later. So,
// reject the request ...
ep, ok := netConfig.EndpointsConfig[nwName]
if !ok {
return nil, errdefs.InvalidParameter(
errors.New("HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks"))
}
return ep, nil
}
func (s *containerRouter) deleteContainers(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
name := vars["name"]
config := &types.ContainerRmConfig{
config := &backend.ContainerRmConfig{
ForceRemove: httputils.BoolValue(r, "force"),
RemoveVolume: httputils.BoolValue(r, "v"),
RemoveLink: httputils.BoolValue(r, "link"),
@@ -654,7 +927,7 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
}
contentType := types.MediaTypeRawStream
setupStreams := func(multiplexed bool) (io.ReadCloser, io.Writer, io.Writer, error) {
setupStreams := func(multiplexed bool, cancel func()) (io.ReadCloser, io.Writer, io.Writer, error) {
conn, _, err := hijacker.Hijack()
if err != nil {
return nil, nil, nil, err
@@ -667,11 +940,15 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
if multiplexed && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = types.MediaTypeMultiplexedStream
}
fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
// FIXME(thaJeztah): we should not ignore errors here; see https://github.com/moby/moby/pull/48359#discussion_r1725562802
fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: %v\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n", contentType)
} else {
fmt.Fprintf(conn, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n")
// FIXME(thaJeztah): we should not ignore errors here; see https://github.com/moby/moby/pull/48359#discussion_r1725562802
fmt.Fprint(conn, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n")
}
go notifyClosed(ctx, conn, cancel)
closer := func() error {
httputils.CloseStreams(conn)
return nil
@@ -691,11 +968,11 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
}
if err = s.backend.ContainerAttach(containerName, attachConfig); err != nil {
logrus.WithError(err).Errorf("Handler for %s %s returned error", r.Method, r.URL.Path)
log.G(ctx).WithError(err).Errorf("Handler for %s %s returned error", r.Method, r.URL.Path)
// Remember to close stream if error happens
conn, _, errHijack := hijacker.Hijack()
if errHijack != nil {
logrus.WithError(err).Errorf("Handler for %s %s: unable to close stream; error when hijacking connection", r.Method, r.URL.Path)
log.G(ctx).WithError(err).Errorf("Handler for %s %s: unable to close stream; error when hijacking connection", r.Method, r.URL.Path)
} else {
statusCode := httpstatus.FromError(err)
statusText := http.StatusText(statusCode)
@@ -720,7 +997,7 @@ func (s *containerRouter) wsContainersAttach(ctx context.Context, w http.Respons
version := httputils.VersionFromContext(ctx)
setupStreams := func(multiplexed bool) (io.ReadCloser, io.Writer, io.Writer, error) {
setupStreams := func(multiplexed bool, cancel func()) (io.ReadCloser, io.Writer, io.Writer, error) {
wsChan := make(chan *websocket.Conn)
h := func(conn *websocket.Conn) {
wsChan <- conn
@@ -739,6 +1016,8 @@ func (s *containerRouter) wsContainersAttach(ctx context.Context, w http.Respons
if versions.GreaterThanOrEqualTo(version, "1.28") {
conn.PayloadType = websocket.BinaryFrame
}
// TODO: Close notifications
return conn, conn, conn, nil
}
@@ -765,9 +1044,9 @@ func (s *containerRouter) wsContainersAttach(ctx context.Context, w http.Respons
select {
case <-started:
if err != nil {
logrus.Errorf("Error attaching websocket: %s", err)
log.G(ctx).Errorf("Error attaching websocket: %s", err)
} else {
logrus.Debug("websocket connection was closed by client")
log.G(ctx).Debug("websocket connection was closed by client")
}
return nil
default:

View File

@@ -0,0 +1,352 @@
package container
import (
"strings"
"testing"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/libnetwork/netlabel"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestHandleMACAddressBC(t *testing.T) {
testcases := []struct {
name string
apiVersion string
ctrWideMAC string
networkMode container.NetworkMode
epConfig map[string]*network.EndpointSettings
expEpWithCtrWideMAC string
expEpWithNoMAC string
expCtrWideMAC string
expWarning string
expError string
}{
{
name: "old api ctr-wide mac mix id and name",
apiVersion: "1.43",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expEpWithCtrWideMAC: "aNetName",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "old api clear ep mac",
apiVersion: "1.43",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {MacAddress: "11:22:33:44:55:66"}},
expEpWithNoMAC: "aNetName",
},
{
name: "old api no-network ctr-wide mac",
apiVersion: "1.43",
networkMode: "none",
ctrWideMAC: "11:22:33:44:55:66",
expError: "conflicting options: mac-address and the network mode",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "old api create ep",
apiVersion: "1.43",
networkMode: "aNetId",
ctrWideMAC: "11:22:33:44:55:66",
epConfig: map[string]*network.EndpointSettings{},
expEpWithCtrWideMAC: "aNetId",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "old api migrate ctr-wide mac",
apiVersion: "1.43",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expEpWithCtrWideMAC: "aNetName",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "new api no macs",
apiVersion: "1.44",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
},
{
name: "new api ep specific mac",
apiVersion: "1.44",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {MacAddress: "11:22:33:44:55:66"}},
},
{
name: "new api migrate ctr-wide mac to new ep",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{},
expEpWithCtrWideMAC: "aNetName",
expWarning: "The container-wide MacAddress field is now deprecated",
expCtrWideMAC: "",
},
{
name: "new api migrate ctr-wide mac to existing ep",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expEpWithCtrWideMAC: "aNetName",
expWarning: "The container-wide MacAddress field is now deprecated",
expCtrWideMAC: "",
},
{
name: "new api mode vs name mismatch",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expError: "unable to migrate container-wide MAC address to a specific network: HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "new api mac mismatch",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {MacAddress: "00:11:22:33:44:55"}},
expError: "the container-wide MAC address must match the endpoint-specific MAC address",
expCtrWideMAC: "11:22:33:44:55:66",
},
}
for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
cfg := &container.Config{
MacAddress: tc.ctrWideMAC, //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
}
hostCfg := &container.HostConfig{
NetworkMode: tc.networkMode,
}
epConfig := make(map[string]*network.EndpointSettings, len(tc.epConfig))
for k, v := range tc.epConfig {
v := *v
epConfig[k] = &v
}
netCfg := &network.NetworkingConfig{
EndpointsConfig: epConfig,
}
warning, err := handleMACAddressBC(cfg, hostCfg, netCfg, tc.apiVersion)
if tc.expError == "" {
assert.Check(t, err)
} else {
assert.Check(t, is.ErrorContains(err, tc.expError))
}
if tc.expWarning == "" {
assert.Check(t, is.Equal(warning, ""))
} else {
assert.Check(t, is.Contains(warning, tc.expWarning))
}
if tc.expEpWithCtrWideMAC != "" {
got := netCfg.EndpointsConfig[tc.expEpWithCtrWideMAC].MacAddress
assert.Check(t, is.Equal(got, tc.ctrWideMAC))
}
if tc.expEpWithNoMAC != "" {
got := netCfg.EndpointsConfig[tc.expEpWithNoMAC].MacAddress
assert.Check(t, is.Equal(got, ""))
}
gotCtrWideMAC := cfg.MacAddress //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
assert.Check(t, is.Equal(gotCtrWideMAC, tc.expCtrWideMAC))
})
}
}
func TestEpConfigForNetMode(t *testing.T) {
testcases := []struct {
name string
apiVersion string
networkMode string
epConfig map[string]*network.EndpointSettings
expEpId string
expNumEps int
expError bool
}{
{
name: "old api no eps",
apiVersion: "1.43",
networkMode: "mynet",
expNumEps: 1,
},
{
name: "new api no eps",
apiVersion: "1.44",
networkMode: "mynet",
expNumEps: 1,
},
{
name: "old api with ep",
apiVersion: "1.43",
networkMode: "mynet",
epConfig: map[string]*network.EndpointSettings{
"anything": {EndpointID: "epone"},
},
expEpId: "epone",
expNumEps: 1,
},
{
name: "new api with matching ep",
apiVersion: "1.44",
networkMode: "mynet",
epConfig: map[string]*network.EndpointSettings{
"mynet": {EndpointID: "epone"},
},
expEpId: "epone",
expNumEps: 1,
},
{
name: "new api with mismatched ep",
apiVersion: "1.44",
networkMode: "mynet",
epConfig: map[string]*network.EndpointSettings{
"shortid": {EndpointID: "epone"},
},
expError: true,
},
}
for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
netConfig := &network.NetworkingConfig{
EndpointsConfig: tc.epConfig,
}
ep, err := epConfigForNetMode(tc.apiVersion, container.NetworkMode(tc.networkMode), netConfig)
if tc.expError {
assert.Check(t, is.ErrorContains(err, "HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks"))
} else {
assert.Assert(t, err)
assert.Check(t, is.Equal(ep.EndpointID, tc.expEpId))
assert.Check(t, is.Len(netConfig.EndpointsConfig, tc.expNumEps))
}
})
}
}
func TestHandleSysctlBC(t *testing.T) {
testcases := []struct {
name string
apiVersion string
networkMode string
sysctls map[string]string
epConfig map[string]*network.EndpointSettings
expEpSysctls []string
expSysctls map[string]string
expWarningContains []string
expError string
}{
{
name: "migrate to new ep",
apiVersion: "1.46",
networkMode: "mynet",
sysctls: map[string]string{
"net.ipv6.conf.all.disable_ipv6": "0",
"net.ipv6.conf.eth0.accept_ra": "2",
"net.ipv6.conf.eth0.forwarding": "1",
},
expSysctls: map[string]string{
"net.ipv6.conf.all.disable_ipv6": "0",
},
expEpSysctls: []string{"net.ipv6.conf.IFNAME.forwarding=1", "net.ipv6.conf.IFNAME.accept_ra=2"},
expWarningContains: []string{
"Migrated",
"net.ipv6.conf.eth0.accept_ra", "net.ipv6.conf.IFNAME.accept_ra=2",
"net.ipv6.conf.eth0.forwarding", "net.ipv6.conf.IFNAME.forwarding=1",
},
},
{
name: "migrate nothing",
apiVersion: "1.46",
networkMode: "mynet",
sysctls: map[string]string{
"net.ipv6.conf.all.disable_ipv6": "0",
},
expSysctls: map[string]string{
"net.ipv6.conf.all.disable_ipv6": "0",
},
},
/* TODO(robmry) - enable this test for the API version used in 28.0.0
{
name: "migration disabled for newer api",
apiVersion: "1.??",
networkMode: "mynet",
sysctls: map[string]string{
"net.ipv6.conf.eth0.accept_ra": "2",
},
expError: "must be supplied using driver option 'com.docker.network.endpoint.sysctls'",
},
*/
{
name: "only migrate eth0",
apiVersion: "1.46",
networkMode: "mynet",
sysctls: map[string]string{
"net.ipv6.conf.eth1.accept_ra": "2",
},
expError: "unable to determine network endpoint",
},
{
name: "net name mismatch",
apiVersion: "1.46",
networkMode: "mynet",
epConfig: map[string]*network.EndpointSettings{
"shortid": {EndpointID: "epone"},
},
sysctls: map[string]string{
"net.ipv6.conf.eth1.accept_ra": "2",
},
expError: "unable to find a network for sysctl",
},
}
for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
hostCfg := &container.HostConfig{
NetworkMode: container.NetworkMode(tc.networkMode),
Sysctls: map[string]string{},
}
for k, v := range tc.sysctls {
hostCfg.Sysctls[k] = v
}
netCfg := &network.NetworkingConfig{
EndpointsConfig: tc.epConfig,
}
warnings, err := handleSysctlBC(hostCfg, netCfg, tc.apiVersion)
for _, s := range tc.expWarningContains {
assert.Check(t, is.Contains(warnings, s))
}
if tc.expError != "" {
assert.Check(t, is.ErrorContains(err, tc.expError))
} else {
assert.Check(t, err)
assert.Check(t, is.DeepEqual(hostCfg.Sysctls, tc.expSysctls))
ep := netCfg.EndpointsConfig[tc.networkMode]
if ep == nil {
assert.Check(t, is.Nil(tc.expEpSysctls))
} else {
got, ok := ep.DriverOpts[netlabel.EndpointSysctls]
assert.Check(t, ok)
// Check for expected ep-sysctls.
for _, want := range tc.expEpSysctls {
assert.Check(t, is.Contains(got, want))
}
// Check for unexpected ep-sysctls.
assert.Check(t, is.Len(got, len(strings.Join(tc.expEpSysctls, ","))))
}
}
})
}
}

View File

@@ -10,51 +10,12 @@ import (
"net/http"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/api/types/container"
gddohttputil "github.com/golang/gddo/httputil"
)
type pathError struct{}
func (pathError) Error() string {
return "Path cannot be empty"
}
func (pathError) InvalidParameter() {}
// postContainersCopy is deprecated in favor of getContainersArchive.
//
// Deprecated since 1.8 (API v1.20), errors out since 1.12 (API v1.24)
func (s *containerRouter) postContainersCopy(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.24") {
w.WriteHeader(http.StatusNotFound)
return nil
}
cfg := types.CopyConfig{}
if err := httputils.ReadJSON(r, &cfg); err != nil {
return err
}
if cfg.Resource == "" {
return pathError{}
}
data, err := s.backend.ContainerCopy(vars["name"], cfg.Resource)
if err != nil {
return err
}
defer data.Close()
w.Header().Set("Content-Type", "application/x-tar")
_, err = io.Copy(w, data)
return err
}
// // Encode the stat to JSON, base64 encode, and place in a header.
func setContainerPathStatHeader(stat *types.ContainerPathStat, header http.Header) error {
// setContainerPathStatHeader encodes the stat to JSON, base64 encode, and place in a header.
func setContainerPathStatHeader(stat *container.PathStat, header http.Header) error {
statJSON, err := json.Marshal(stat)
if err != nil {
return err

View File

@@ -7,13 +7,14 @@ import (
"net/http"
"strconv"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/stdcopy"
"github.com/sirupsen/logrus"
)
func (s *containerRouter) getExecByID(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -38,7 +39,7 @@ func (s *containerRouter) postContainerExecCreate(ctx context.Context, w http.Re
return err
}
execConfig := &types.ExecConfig{}
execConfig := &container.ExecOptions{}
if err := httputils.ReadJSON(r, execConfig); err != nil {
return err
}
@@ -56,7 +57,7 @@ func (s *containerRouter) postContainerExecCreate(ctx context.Context, w http.Re
// Register an instance of Exec in container.
id, err := s.backend.ContainerExecCreate(vars["name"], execConfig)
if err != nil {
logrus.Errorf("Error setting up exec command in container %s: %v", vars["name"], err)
log.G(ctx).Errorf("Error setting up exec command in container %s: %v", vars["name"], err)
return err
}
@@ -71,23 +72,14 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
return err
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.22") {
// API versions before 1.22 did not enforce application/json content-type.
// Allow older clients to work by patching the content-type.
if r.Header.Get("Content-Type") != "application/json" {
r.Header.Set("Content-Type", "application/json")
}
}
var (
execName = vars["name"]
stdin, inStream io.ReadCloser
stdout, stderr, outStream io.Writer
)
execStartCheck := &types.ExecStartCheck{}
if err := httputils.ReadJSON(r, execStartCheck); err != nil {
options := &container.ExecStartOptions{}
if err := httputils.ReadJSON(r, options); err != nil {
return err
}
@@ -95,19 +87,21 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
return err
}
if execStartCheck.ConsoleSize != nil {
if options.ConsoleSize != nil {
version := httputils.VersionFromContext(ctx)
// Not supported before 1.42
if versions.LessThan(version, "1.42") {
execStartCheck.ConsoleSize = nil
options.ConsoleSize = nil
}
// No console without tty
if !execStartCheck.Tty {
execStartCheck.ConsoleSize = nil
if !options.Tty {
options.ConsoleSize = nil
}
}
if !execStartCheck.Detach {
if !options.Detach {
var err error
// Setting up the streaming http interface.
inStream, outStream, err = httputils.HijackConnection(w)
@@ -118,43 +112,44 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
if _, ok := r.Header["Upgrade"]; ok {
contentType := types.MediaTypeRawStream
if !execStartCheck.Tty && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
if !options.Tty && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = types.MediaTypeMultiplexedStream
}
fmt.Fprint(outStream, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n")
_, _ = fmt.Fprint(outStream, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n")
} else {
fmt.Fprint(outStream, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n")
_, _ = fmt.Fprint(outStream, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n")
}
// copy headers that were removed as part of hijack
if err := w.Header().WriteSubset(outStream, nil); err != nil {
return err
}
fmt.Fprint(outStream, "\r\n")
_, _ = fmt.Fprint(outStream, "\r\n")
stdin = inStream
stdout = outStream
if !execStartCheck.Tty {
if options.Tty {
stdout = outStream
} else {
stderr = stdcopy.NewStdWriter(outStream, stdcopy.Stderr)
stdout = stdcopy.NewStdWriter(outStream, stdcopy.Stdout)
}
}
options := container.ExecStartOptions{
// Now run the user process in container.
//
// TODO: Maybe we should we pass ctx here if we're not detaching?
err := s.backend.ContainerExecStart(context.Background(), execName, backend.ExecStartConfig{
Stdin: stdin,
Stdout: stdout,
Stderr: stderr,
ConsoleSize: execStartCheck.ConsoleSize,
}
// Now run the user process in container.
// Maybe we should we pass ctx here if we're not detaching?
if err := s.backend.ContainerExecStart(context.Background(), execName, options); err != nil {
if execStartCheck.Detach {
ConsoleSize: options.ConsoleSize,
})
if err != nil {
if options.Detach {
return err
}
stdout.Write([]byte(err.Error() + "\r\n"))
logrus.Errorf("Error running exec %s in container: %v", execName, err)
_, _ = fmt.Fprintf(stdout, "%v\r\n", err)
log.G(ctx).Errorf("Error running exec %s in container: %v", execName, err)
}
return nil
}

View File

@@ -0,0 +1,54 @@
package container
import (
"context"
"net"
"syscall"
"github.com/containerd/log"
"github.com/docker/docker/internal/unix_noeintr"
"golang.org/x/sys/unix"
)
func notifyClosed(ctx context.Context, conn net.Conn, notify func()) {
sc, ok := conn.(syscall.Conn)
if !ok {
log.G(ctx).Debug("notifyClosed: conn does not support close notifications")
return
}
rc, err := sc.SyscallConn()
if err != nil {
log.G(ctx).WithError(err).Warn("notifyClosed: failed get raw conn for close notifications")
return
}
epFd, err := unix_noeintr.EpollCreate()
if err != nil {
log.G(ctx).WithError(err).Warn("notifyClosed: failed to create epoll fd")
return
}
defer unix.Close(epFd)
err = rc.Control(func(fd uintptr) {
err := unix_noeintr.EpollCtl(epFd, unix.EPOLL_CTL_ADD, int(fd), &unix.EpollEvent{
Events: unix.EPOLLHUP,
Fd: int32(fd),
})
if err != nil {
log.G(ctx).WithError(err).Warn("notifyClosed: failed to register fd for close notifications")
return
}
events := make([]unix.EpollEvent, 1)
if _, err := unix_noeintr.EpollWait(epFd, events, -1); err != nil {
log.G(ctx).WithError(err).Warn("notifyClosed: failed to wait for close notifications")
return
}
notify()
})
if err != nil {
log.G(ctx).WithError(err).Warn("notifyClosed: failed to register for close notifications")
return
}
}

View File

@@ -0,0 +1,10 @@
//go:build !linux
package container
import (
"context"
"net"
)
func notifyClosed(ctx context.Context, conn net.Conn, notify func()) {}

View File

@@ -3,13 +3,13 @@ package distribution // import "github.com/docker/docker/api/server/router/distr
import (
"context"
"github.com/distribution/reference"
"github.com/docker/distribution"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types/registry"
)
// Backend is all the methods that need to be implemented
// to provide image specific functionality.
type Backend interface {
GetRepository(context.Context, reference.Named, *registry.AuthConfig) (distribution.Repository, error)
GetRepositories(context.Context, reference.Named, *registry.AuthConfig) ([]distribution.Repository, error)
}

View File

@@ -4,15 +4,18 @@ import (
"context"
"encoding/json"
"net/http"
"os"
"github.com/distribution/reference"
"github.com/docker/distribution"
"github.com/docker/distribution/manifest/manifestlist"
"github.com/docker/distribution/manifest/schema1"
"github.com/docker/distribution/manifest/schema2"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/registry"
distributionpkg "github.com/docker/docker/distribution"
"github.com/docker/docker/errdefs"
v1 "github.com/opencontainers/image-spec/specs-go/v1"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
@@ -23,10 +26,10 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
w.Header().Set("Content-Type", "application/json")
image := vars["name"]
imgName := vars["name"]
// TODO why is reference.ParseAnyReference() / reference.ParseNormalizedNamed() not using the reference.ErrTagInvalidFormat (and so on) errors?
ref, err := reference.ParseAnyReference(image)
ref, err := reference.ParseAnyReference(imgName)
if err != nil {
return errdefs.InvalidParameter(err)
}
@@ -36,32 +39,58 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
// full image ID
return errors.Errorf("no manifest found for full image ID")
}
return errdefs.InvalidParameter(errors.Errorf("unknown image reference format: %s", image))
return errdefs.InvalidParameter(errors.Errorf("unknown image reference format: %s", imgName))
}
// For a search it is not an error if no auth was given. Ignore invalid
// AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(r.Header.Get(registry.AuthHeader))
distrepo, err := s.backend.GetRepository(ctx, namedRef, authConfig)
repos, err := s.backend.GetRepositories(ctx, namedRef, authConfig)
if err != nil {
return err
}
blobsrvc := distrepo.Blobs(ctx)
// Fetch the manifest; if a mirror is configured, try the mirror first,
// but continue with upstream on failure.
//
// FIXME(thaJeztah): construct "repositories" on-demand;
// GetRepositories() will attempt to connect to all endpoints (registries),
// but we may only need the first one if it contains the manifest we're
// looking for, or if the configured mirror is a pull-through mirror.
//
// Logic for this could be implemented similar to "distribution.Pull()",
// which uses the "pullEndpoints" utility to iterate over the list
// of endpoints;
//
// - https://github.com/moby/moby/blob/12c7411b6b7314bef130cd59f1c7384a7db06d0b/distribution/pull.go#L17-L31
// - https://github.com/moby/moby/blob/12c7411b6b7314bef130cd59f1c7384a7db06d0b/distribution/pull.go#L76-L152
var lastErr error
for _, repo := range repos {
distributionInspect, err := s.fetchManifest(ctx, repo, namedRef)
if err != nil {
lastErr = err
continue
}
return httputils.WriteJSON(w, http.StatusOK, distributionInspect)
}
return lastErr
}
func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distribution.Repository, namedRef reference.Named) (registry.DistributionInspect, error) {
var distributionInspect registry.DistributionInspect
if canonicalRef, ok := namedRef.(reference.Canonical); !ok {
namedRef = reference.TagNameOnly(namedRef)
taggedRef, ok := namedRef.(reference.NamedTagged)
if !ok {
return errdefs.InvalidParameter(errors.Errorf("image reference not tagged: %s", image))
return registry.DistributionInspect{}, errdefs.InvalidParameter(errors.Errorf("image reference not tagged: %s", namedRef))
}
descriptor, err := distrepo.Tags(ctx).Get(ctx, taggedRef.Tag())
if err != nil {
return err
return registry.DistributionInspect{}, err
}
distributionInspect.Descriptor = v1.Descriptor{
distributionInspect.Descriptor = ocispec.Descriptor{
MediaType: descriptor.MediaType,
Digest: descriptor.Digest,
Size: descriptor.Size,
@@ -76,7 +105,7 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
// we have a digest, so we can retrieve the manifest
mnfstsrvc, err := distrepo.Manifests(ctx)
if err != nil {
return err
return registry.DistributionInspect{}, err
}
mnfst, err := mnfstsrvc.Get(ctx, distributionInspect.Descriptor.Digest)
if err != nil {
@@ -88,14 +117,14 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
reference.ErrNameEmpty,
reference.ErrNameTooLong,
reference.ErrNameNotCanonical:
return errdefs.InvalidParameter(err)
return registry.DistributionInspect{}, errdefs.InvalidParameter(err)
}
return err
return registry.DistributionInspect{}, err
}
mediaType, payload, err := mnfst.Payload()
if err != nil {
return err
return registry.DistributionInspect{}, err
}
// update MediaType because registry might return something incorrect
distributionInspect.Descriptor.MediaType = mediaType
@@ -107,7 +136,7 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
switch mnfstObj := mnfst.(type) {
case *manifestlist.DeserializedManifestList:
for _, m := range mnfstObj.Manifests {
distributionInspect.Platforms = append(distributionInspect.Platforms, v1.Platform{
distributionInspect.Platforms = append(distributionInspect.Platforms, ocispec.Platform{
Architecture: m.Platform.Architecture,
OS: m.Platform.OS,
OSVersion: m.Platform.OSVersion,
@@ -116,8 +145,9 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
})
}
case *schema2.DeserializedManifest:
configJSON, err := blobsrvc.Get(ctx, mnfstObj.Config.Digest)
var platform v1.Platform
blobStore := distrepo.Blobs(ctx)
configJSON, err := blobStore.Get(ctx, mnfstObj.Config.Digest)
var platform ocispec.Platform
if err == nil {
err := json.Unmarshal(configJSON, &platform)
if err == nil && (platform.OS != "" || platform.Architecture != "") {
@@ -125,12 +155,14 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
}
}
case *schema1.SignedManifest:
platform := v1.Platform{
if os.Getenv("DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE") == "" {
return registry.DistributionInspect{}, distributionpkg.DeprecatedSchema1ImageError(namedRef)
}
platform := ocispec.Platform{
Architecture: mnfstObj.Architecture,
OS: "linux",
}
distributionInspect.Platforms = append(distributionInspect.Platforms, platform)
}
return httputils.WriteJSON(w, http.StatusOK, distributionInspect)
return distributionInspect, nil
}

View File

@@ -1,8 +1,22 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.22
package grpc // import "github.com/docker/docker/api/server/router/grpc"
import (
"context"
"fmt"
"os"
"strings"
"github.com/containerd/containerd/defaults"
"github.com/containerd/log"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/internal/otelutil"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/moby/buildkit/util/stack"
"github.com/moby/buildkit/util/tracing"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"golang.org/x/net/http2"
"google.golang.org/grpc"
)
@@ -15,12 +29,18 @@ type grpcRouter struct {
// NewRouter initializes a new grpc http router
func NewRouter(backends ...Backend) router.Router {
tp, _ := otelutil.NewTracerProvider(context.Background(), false)
opts := []grpc.ServerOption{
grpc.StatsHandler(tracing.ServerStatsHandler(otelgrpc.WithTracerProvider(tp))),
grpc.ChainUnaryInterceptor(unaryInterceptor, grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
grpc.MaxRecvMsgSize(defaults.DefaultMaxRecvMsgSize),
grpc.MaxSendMsgSize(defaults.DefaultMaxSendMsgSize),
}
r := &grpcRouter{
h2Server: &http2.Server{},
grpcServer: grpc.NewServer(
grpc.UnaryInterceptor(grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
),
h2Server: &http2.Server{},
grpcServer: grpc.NewServer(opts...),
}
for _, b := range backends {
b.RegisterGRPC(r.grpcServer)
@@ -39,3 +59,21 @@ func (gr *grpcRouter) initRoutes() {
router.NewPostRoute("/grpc", gr.serveGRPC),
}
}
func unaryInterceptor(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp any, err error) {
// This method is used by the clients to send their traces to buildkit so they can be included
// in the daemon trace and stored in the build history record. This method can not be traced because
// it would cause an infinite loop.
if strings.HasSuffix(info.FullMethod, "opentelemetry.proto.collector.trace.v1.TraceService/Export") {
return handler(ctx, req)
}
resp, err = handler(ctx, req)
if err != nil {
log.G(ctx).WithError(err).Error(info.FullMethod)
if log.GetLevel() >= log.DebugLevel {
fmt.Fprintf(os.Stderr, "%+v", stack.Formatter(grpcerrors.FromGRPC(err)))
}
}
return resp, err
}

View File

@@ -4,13 +4,13 @@ import (
"context"
"io"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/distribution/reference"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
dockerimage "github.com/docker/docker/image"
specs "github.com/opencontainers/image-spec/specs-go/v1"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// Backend is all the methods that need to be implemented
@@ -22,23 +22,23 @@ type Backend interface {
}
type imageBackend interface {
ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]types.ImageDeleteResponseItem, error)
ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]image.DeleteResponse, error)
ImageHistory(ctx context.Context, imageName string) ([]*image.HistoryResponseItem, error)
Images(ctx context.Context, opts types.ImageListOptions) ([]*types.ImageSummary, error)
GetImage(ctx context.Context, refOrID string, options image.GetImageOpts) (*dockerimage.Image, error)
Images(ctx context.Context, opts image.ListOptions) ([]*image.Summary, error)
GetImage(ctx context.Context, refOrID string, options backend.GetImageOpts) (*dockerimage.Image, error)
TagImage(ctx context.Context, id dockerimage.ID, newRef reference.Named) error
ImagesPrune(ctx context.Context, pruneFilters filters.Args) (*types.ImagesPruneReport, error)
ImagesPrune(ctx context.Context, pruneFilters filters.Args) (*image.PruneReport, error)
}
type importExportBackend interface {
LoadImage(ctx context.Context, inTar io.ReadCloser, outStream io.Writer, quiet bool) error
ImportImage(ctx context.Context, ref reference.Named, platform *specs.Platform, msg string, layerReader io.Reader, changes []string) (dockerimage.ID, error)
ImportImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, msg string, layerReader io.Reader, changes []string) (dockerimage.ID, error)
ExportImage(ctx context.Context, names []string, outStream io.Writer) error
}
type registryBackend interface {
PullImage(ctx context.Context, image, tag string, platform *specs.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
PushImage(ctx context.Context, ref reference.Named, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
PullImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
PushImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
}
type Searcher interface {

View File

@@ -2,6 +2,7 @@ package image // import "github.com/docker/docker/api/server/router/image"
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
@@ -9,12 +10,14 @@ import (
"strings"
"time"
"github.com/containerd/containerd/platforms"
"github.com/docker/distribution/reference"
"github.com/containerd/platforms"
"github.com/distribution/reference"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
opts "github.com/docker/docker/api/types/image"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/builder/remotecontext"
@@ -24,7 +27,8 @@ import (
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/progress"
"github.com/docker/docker/pkg/streamformatter"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
@@ -41,7 +45,7 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
comment = r.Form.Get("message")
progressErr error
output = ioutils.NewWriteFlusher(w)
platform *specs.Platform
platform *ocispec.Platform
)
defer output.Close()
@@ -52,7 +56,7 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
if p := r.FormValue("platform"); p != "" {
sp, err := platforms.Parse(p)
if err != nil {
return err
return errdefs.InvalidParameter(err)
}
platform = &sp
}
@@ -66,10 +70,39 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
}
}
// Special case: "pull -a" may send an image name with a
// trailing :. This is ugly, but let's not break API
// compatibility.
imgName := strings.TrimSuffix(img, ":")
ref, err := reference.ParseNormalizedNamed(imgName)
if err != nil {
return errdefs.InvalidParameter(err)
}
// TODO(thaJeztah) this could use a WithTagOrDigest() utility
if tag != "" {
// The "tag" could actually be a digest.
var dgst digest.Digest
dgst, err = digest.Parse(tag)
if err == nil {
ref, err = reference.WithDigest(reference.TrimNamed(ref), dgst)
} else {
ref, err = reference.WithTag(ref, tag)
}
if err != nil {
return errdefs.InvalidParameter(err)
}
}
if err := validateRepoName(ref); err != nil {
return errdefs.Forbidden(err)
}
// For a pull it is not an error if no auth was given. Ignore invalid
// AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(r.Header.Get(registry.AuthHeader))
progressErr = ir.backend.PullImage(ctx, img, tag, platform, metaHeaders, authConfig, output)
progressErr = ir.backend.PullImage(ctx, ref, platform, metaHeaders, authConfig, output)
} else { // import
src := r.Form.Get("fromSrc")
@@ -109,7 +142,7 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
id, progressErr = ir.backend.ImportImage(ctx, tagRef, platform, comment, layerReader, r.Form["changes"])
if progressErr == nil {
output.Write(streamformatter.FormatStatus("", id.String()))
_, _ = output.Write(streamformatter.FormatStatus("", "%v", id.String()))
}
}
if progressErr != nil {
@@ -157,7 +190,7 @@ func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter
var ref reference.Named
// Tag is empty only in case ImagePushOptions.All is true.
// Tag is empty only in case PushOptions.All is true.
if tag != "" {
r, err := httputils.RepoTagReference(img, tag)
if err != nil {
@@ -172,7 +205,24 @@ func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter
ref = r
}
if err := ir.backend.PushImage(ctx, ref, metaHeaders, authConfig, output); err != nil {
var platform *ocispec.Platform
// Platform is optional, and only supported in API version 1.46 and later.
// However the PushOptions struct previously was an alias for the PullOptions struct
// which also contained a Platform field.
// This means that older clients may be sending a platform field, even
// though it wasn't really supported by the server.
// Don't break these clients and just ignore the platform field on older APIs.
if versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.46") {
if formPlatform := r.Form.Get("platform"); formPlatform != "" {
p, err := httputils.DecodePlatform(formPlatform)
if err != nil {
return err
}
platform = p
}
}
if err := ir.backend.PushImage(ctx, ref, platform, metaHeaders, authConfig, output); err != nil {
if !output.Flushed() {
return err
}
@@ -253,7 +303,7 @@ func (ir *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter,
}
func (ir *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
img, err := ir.backend.GetImage(ctx, vars["name"], opts.GetImageOpts{Details: true})
img, err := ir.backend.GetImage(ctx, vars["name"], backend.GetImageOpts{Details: true})
if err != nil {
return err
}
@@ -263,6 +313,20 @@ func (ir *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWrite
return err
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.44") {
imageInspect.VirtualSize = imageInspect.Size //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
if imageInspect.Created == "" {
// backwards compatibility for Created not existing returning "0001-01-01T00:00:00Z"
// https://github.com/moby/moby/issues/47368
imageInspect.Created = time.Time{}.Format(time.RFC3339Nano)
}
}
if versions.GreaterThanOrEqualTo(version, "1.45") {
imageInspect.Container = "" //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
imageInspect.ContainerConfig = nil //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
}
return httputils.WriteJSON(w, http.StatusOK, imageInspect)
}
@@ -282,15 +346,28 @@ func (ir *imageRouter) toImageInspect(img *image.Image) (*types.ImageInspect, er
comment = img.History[len(img.History)-1].Comment
}
// Make sure we output empty arrays instead of nil.
if repoTags == nil {
repoTags = []string{}
}
if repoDigests == nil {
repoDigests = []string{}
}
var created string
if img.Created != nil {
created = img.Created.Format(time.RFC3339Nano)
}
return &types.ImageInspect{
ID: img.ID().String(),
RepoTags: repoTags,
RepoDigests: repoDigests,
Parent: img.Parent.String(),
Comment: comment,
Created: img.Created.Format(time.RFC3339Nano),
Container: img.Container,
ContainerConfig: &img.ContainerConfig,
Created: created,
Container: img.Container, //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
ContainerConfig: &img.ContainerConfig, //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
DockerVersion: img.DockerVersion,
Author: img.Author,
Config: img.Config,
@@ -299,13 +376,12 @@ func (ir *imageRouter) toImageInspect(img *image.Image) (*types.ImageInspect, er
Os: img.OperatingSystem(),
OsVersion: img.OSVersion,
Size: img.Details.Size,
VirtualSize: img.Details.Size, //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
GraphDriver: types.GraphDriverData{
Name: img.Details.Driver,
Data: img.Details.Metadata,
},
RootFS: rootFSToAPIType(img.RootFS),
Metadata: types.ImageMetadata{
Metadata: imagetypes.Metadata{
LastTagTime: img.Details.LastUpdated,
},
}, nil
@@ -347,16 +423,23 @@ func (ir *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
sharedSize = httputils.BoolValue(r, "shared-size")
}
images, err := ir.backend.Images(ctx, types.ImageListOptions{
var manifests bool
if versions.GreaterThanOrEqualTo(version, "1.47") {
manifests = httputils.BoolValue(r, "manifests")
}
images, err := ir.backend.Images(ctx, imagetypes.ListOptions{
All: httputils.BoolValue(r, "all"),
Filters: imageFilters,
SharedSize: sharedSize,
Manifests: manifests,
})
if err != nil {
return err
}
useNone := versions.LessThan(version, "1.43")
withVirtualSize := versions.LessThan(version, "1.44")
for _, img := range images {
if useNone {
if len(img.RepoTags) == 0 && len(img.RepoDigests) == 0 {
@@ -371,6 +454,9 @@ func (ir *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
img.RepoDigests = []string{}
}
}
if withVirtualSize {
img.VirtualSize = img.Size //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
}
}
return httputils.WriteJSON(w, http.StatusOK, images)
@@ -395,7 +481,12 @@ func (ir *imageRouter) postImagesTag(ctx context.Context, w http.ResponseWriter,
return errdefs.InvalidParameter(err)
}
img, err := ir.backend.GetImage(ctx, vars["name"], opts.GetImageOpts{})
refName := reference.FamiliarName(ref)
if refName == string(digest.Canonical) {
return errdefs.InvalidParameter(errors.New("refusing to create an ambiguous tag using digest algorithm as name"))
}
img, err := ir.backend.GetImage(ctx, vars["name"], backend.GetImageOpts{})
if err != nil {
return errdefs.NotFound(err)
}
@@ -429,7 +520,7 @@ func (ir *imageRouter) getImagesSearch(ctx context.Context, w http.ResponseWrite
// AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(r.Header.Get(registry.AuthHeader))
var headers = http.Header{}
headers := http.Header{}
for k, v := range r.Header {
k = http.CanonicalHeaderKey(k)
if strings.HasPrefix(k, "X-Meta-") {
@@ -460,3 +551,12 @@ func (ir *imageRouter) postImagesPrune(ctx context.Context, w http.ResponseWrite
}
return httputils.WriteJSON(w, http.StatusOK, pruneReport)
}
// validateRepoName validates the name of a repository.
func validateRepoName(name reference.Named) error {
familiarName := reference.FamiliarName(name)
if familiarName == api.NoBaseImageSpecifier {
return fmt.Errorf("'%s' is a reserved name", familiarName)
}
return nil
}

View File

@@ -3,30 +3,28 @@ package network // import "github.com/docker/docker/api/server/router/network"
import (
"context"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/libnetwork"
)
// Backend is all the methods that need to be implemented
// to provide network specific functionality.
type Backend interface {
FindNetwork(idName string) (libnetwork.Network, error)
GetNetworks(filters.Args, types.NetworkListConfig) ([]types.NetworkResource, error)
CreateNetwork(nc types.NetworkCreateRequest) (*types.NetworkCreateResponse, error)
ConnectContainerToNetwork(containerName, networkName string, endpointConfig *network.EndpointSettings) error
GetNetworks(filters.Args, backend.NetworkListConfig) ([]network.Inspect, error)
CreateNetwork(nc network.CreateRequest) (*network.CreateResponse, error)
ConnectContainerToNetwork(ctx context.Context, containerName, networkName string, endpointConfig *network.EndpointSettings) error
DisconnectContainerFromNetwork(containerName string, networkName string, force bool) error
DeleteNetwork(networkID string) error
NetworksPrune(ctx context.Context, pruneFilters filters.Args) (*types.NetworksPruneReport, error)
NetworksPrune(ctx context.Context, pruneFilters filters.Args) (*network.PruneReport, error)
}
// ClusterBackend is all the methods that need to be implemented
// to provide cluster network specific functionality.
type ClusterBackend interface {
GetNetworks(filters.Args) ([]types.NetworkResource, error)
GetNetwork(name string) (types.NetworkResource, error)
GetNetworksByName(name string) ([]types.NetworkResource, error)
CreateNetwork(nc types.NetworkCreateRequest) (string, error)
GetNetworks(filters.Args) ([]network.Inspect, error)
GetNetwork(name string) (network.Inspect, error)
GetNetworksByName(name string) ([]network.Inspect, error)
CreateNetwork(nc network.CreateRequest) (string, error)
RemoveNetwork(name string) error
}

View File

@@ -1 +0,0 @@
package network // import "github.com/docker/docker/api/server/router/network"

View File

@@ -7,13 +7,13 @@ import (
"strings"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/libnetwork"
netconst "github.com/docker/docker/libnetwork/datastore"
"github.com/docker/docker/libnetwork/scope"
"github.com/pkg/errors"
)
@@ -31,7 +31,7 @@ func (n *networkRouter) getNetworksList(ctx context.Context, w http.ResponseWrit
return err
}
var list []types.NetworkResource
var list []network.Summary
nr, err := n.cluster.GetNetworks(filter)
if err == nil {
list = nr
@@ -39,7 +39,7 @@ func (n *networkRouter) getNetworksList(ctx context.Context, w http.ResponseWrit
// Combine the network list returned by Docker daemon if it is not already
// returned by the cluster manager
localNetworks, err := n.backend.GetNetworks(filter, types.NetworkListConfig{Detailed: versions.LessThan(httputils.VersionFromContext(ctx), "1.28")})
localNetworks, err := n.backend.GetNetworks(filter, backend.NetworkListConfig{Detailed: versions.LessThan(httputils.VersionFromContext(ctx), "1.28")})
if err != nil {
return err
}
@@ -59,7 +59,7 @@ func (n *networkRouter) getNetworksList(ctx context.Context, w http.ResponseWrit
}
if list == nil {
list = []types.NetworkResource{}
list = []network.Summary{}
}
return httputils.WriteJSON(w, http.StatusOK, list)
@@ -75,17 +75,13 @@ func (e invalidRequestError) Error() string {
func (e invalidRequestError) InvalidParameter() {}
type ambigousResultsError string
type ambiguousResultsError string
func (e ambigousResultsError) Error() string {
func (e ambiguousResultsError) Error() string {
return "network " + string(e) + " is ambiguous"
}
func (ambigousResultsError) InvalidParameter() {}
func nameConflict(name string) error {
return errdefs.Conflict(libnetwork.NetworkNameError(name))
}
func (ambiguousResultsError) InvalidParameter() {}
func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
@@ -102,7 +98,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
return errors.Wrapf(invalidRequestError{err}, "invalid value for verbose: %s", v)
}
}
scope := r.URL.Query().Get("scope")
networkScope := r.URL.Query().Get("scope")
// In case multiple networks have duplicate names, return error.
// TODO (yongtang): should we wrap with version here for backward compatibility?
@@ -112,29 +108,29 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
// For full name and partial ID, save the result first, and process later
// in case multiple records was found based on the same term
listByFullName := map[string]types.NetworkResource{}
listByPartialID := map[string]types.NetworkResource{}
listByFullName := map[string]network.Inspect{}
listByPartialID := map[string]network.Inspect{}
// TODO(@cpuguy83): All this logic for figuring out which network to return does not belong here
// Instead there should be a backend function to just get one network.
filter := filters.NewArgs(filters.Arg("idOrName", term))
if scope != "" {
filter.Add("scope", scope)
if networkScope != "" {
filter.Add("scope", networkScope)
}
nw, _ := n.backend.GetNetworks(filter, types.NetworkListConfig{Detailed: true, Verbose: verbose})
for _, network := range nw {
if network.ID == term {
return httputils.WriteJSON(w, http.StatusOK, network)
networks, _ := n.backend.GetNetworks(filter, backend.NetworkListConfig{Detailed: true, Verbose: verbose})
for _, nw := range networks {
if nw.ID == term {
return httputils.WriteJSON(w, http.StatusOK, nw)
}
if network.Name == term {
if nw.Name == term {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByFullName[network.ID] = network
listByFullName[nw.ID] = nw
}
if strings.HasPrefix(network.ID, term) {
if strings.HasPrefix(nw.ID, term) {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByPartialID[network.ID] = network
listByPartialID[nw.ID] = nw
}
}
@@ -144,7 +140,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
// or if the get network was passed with a network name and scope as swarm
// return the network. Skipped using isMatchingScope because it is true if the scope
// is not set which would be case if the client API v1.30
if strings.HasPrefix(nwk.ID, term) || (netconst.SwarmScope == scope) {
if strings.HasPrefix(nwk.ID, term) || networkScope == scope.Swarm {
// If we have a previous match "backend", return it, we need verbose when enabled
// ex: overlay/partial_ID or name/swarm_scope
if nwv, ok := listByPartialID[nwk.ID]; ok {
@@ -156,25 +152,25 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
}
nr, _ := n.cluster.GetNetworks(filter)
for _, network := range nr {
if network.ID == term {
return httputils.WriteJSON(w, http.StatusOK, network)
networks, _ = n.cluster.GetNetworks(filter)
for _, nw := range networks {
if nw.ID == term {
return httputils.WriteJSON(w, http.StatusOK, nw)
}
if network.Name == term {
if nw.Name == term {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByFullName) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByFullName[network.ID]; !ok {
listByFullName[network.ID] = network
if _, ok := listByFullName[nw.ID]; !ok {
listByFullName[nw.ID] = nw
}
}
if strings.HasPrefix(network.ID, term) {
if strings.HasPrefix(nw.ID, term) {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByPartialID) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByPartialID[network.ID]; !ok {
listByPartialID[network.ID] = network
if _, ok := listByPartialID[nw.ID]; !ok {
listByPartialID[nw.ID] = nw
}
}
}
@@ -186,7 +182,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
}
if len(listByFullName) > 1 {
return errors.Wrapf(ambigousResultsError(term), "%d matches found based on name", len(listByFullName))
return errors.Wrapf(ambiguousResultsError(term), "%d matches found based on name", len(listByFullName))
}
// Find based on partial ID, returns true only if no duplicates
@@ -196,7 +192,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
}
if len(listByPartialID) > 1 {
return errors.Wrapf(ambigousResultsError(term), "%d matches found based on ID prefix", len(listByPartialID))
return errors.Wrapf(ambiguousResultsError(term), "%d matches found based on ID prefix", len(listByPartialID))
}
return libnetwork.ErrNoSuchNetwork(term)
@@ -207,27 +203,21 @@ func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWr
return err
}
var create types.NetworkCreateRequest
var create network.CreateRequest
if err := httputils.ReadJSON(r, &create); err != nil {
return err
}
if nws, err := n.cluster.GetNetworksByName(create.Name); err == nil && len(nws) > 0 {
return nameConflict(create.Name)
return libnetwork.NetworkNameError(create.Name)
}
// For a Swarm-scoped network, this call to backend.CreateNetwork is used to
// validate the configuration. The network will not be created but, if the
// configuration is valid, ManagerRedirectError will be returned and handled
// below.
nw, err := n.backend.CreateNetwork(create)
if err != nil {
var warning string
if _, ok := err.(libnetwork.NetworkNameError); ok {
// check if user defined CheckDuplicate, if set true, return err
// otherwise prepare a warning message
if create.CheckDuplicate {
return nameConflict(create.Name)
}
warning = libnetwork.NetworkNameError(create.Name).Error()
}
if _, ok := err.(libnetwork.ManagerRedirectError); !ok {
return err
}
@@ -235,9 +225,8 @@ func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWr
if err != nil {
return err
}
nw = &types.NetworkCreateResponse{
ID: id,
Warning: warning,
nw = &network.CreateResponse{
ID: id,
}
}
@@ -249,7 +238,7 @@ func (n *networkRouter) postNetworkConnect(ctx context.Context, w http.ResponseW
return err
}
var connect types.NetworkConnect
var connect network.ConnectOptions
if err := httputils.ReadJSON(r, &connect); err != nil {
return err
}
@@ -258,7 +247,7 @@ func (n *networkRouter) postNetworkConnect(ctx context.Context, w http.ResponseW
// The reason is that, In case of attachable network in swarm scope, the actual local network
// may not be available at the time. At the same time, inside daemon `ConnectContainerToNetwork`
// does the ambiguity check anyway. Therefore, passing the name to daemon would be enough.
return n.backend.ConnectContainerToNetwork(connect.Container, vars["id"], connect.EndpointConfig)
return n.backend.ConnectContainerToNetwork(ctx, connect.Container, vars["id"], connect.EndpointConfig)
}
func (n *networkRouter) postNetworkDisconnect(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -266,7 +255,7 @@ func (n *networkRouter) postNetworkDisconnect(ctx context.Context, w http.Respon
return err
}
var disconnect types.NetworkDisconnect
var disconnect network.DisconnectOptions
if err := httputils.ReadJSON(r, &disconnect); err != nil {
return err
}
@@ -321,47 +310,47 @@ func (n *networkRouter) postNetworksPrune(ctx context.Context, w http.ResponseWr
// For full name and partial ID, save the result first, and process later
// in case multiple records was found based on the same term
// TODO (yongtang): should we wrap with version here for backward compatibility?
func (n *networkRouter) findUniqueNetwork(term string) (types.NetworkResource, error) {
listByFullName := map[string]types.NetworkResource{}
listByPartialID := map[string]types.NetworkResource{}
func (n *networkRouter) findUniqueNetwork(term string) (network.Inspect, error) {
listByFullName := map[string]network.Inspect{}
listByPartialID := map[string]network.Inspect{}
filter := filters.NewArgs(filters.Arg("idOrName", term))
nw, _ := n.backend.GetNetworks(filter, types.NetworkListConfig{Detailed: true})
for _, network := range nw {
if network.ID == term {
return network, nil
networks, _ := n.backend.GetNetworks(filter, backend.NetworkListConfig{Detailed: true})
for _, nw := range networks {
if nw.ID == term {
return nw, nil
}
if network.Name == term && !network.Ingress {
if nw.Name == term && !nw.Ingress {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByFullName[network.ID] = network
listByFullName[nw.ID] = nw
}
if strings.HasPrefix(network.ID, term) {
if strings.HasPrefix(nw.ID, term) {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByPartialID[network.ID] = network
listByPartialID[nw.ID] = nw
}
}
nr, _ := n.cluster.GetNetworks(filter)
for _, network := range nr {
if network.ID == term {
return network, nil
networks, _ = n.cluster.GetNetworks(filter)
for _, nw := range networks {
if nw.ID == term {
return nw, nil
}
if network.Name == term {
if nw.Name == term {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByFullName) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByFullName[network.ID]; !ok {
listByFullName[network.ID] = network
if _, ok := listByFullName[nw.ID]; !ok {
listByFullName[nw.ID] = nw
}
}
if strings.HasPrefix(network.ID, term) {
if strings.HasPrefix(nw.ID, term) {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByPartialID) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByPartialID[network.ID]; !ok {
listByPartialID[network.ID] = network
if _, ok := listByPartialID[nw.ID]; !ok {
listByPartialID[nw.ID] = nw
}
}
}
@@ -373,7 +362,7 @@ func (n *networkRouter) findUniqueNetwork(term string) (types.NetworkResource, e
}
}
if len(listByFullName) > 1 {
return types.NetworkResource{}, errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on name)", term, len(listByFullName)))
return network.Inspect{}, errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on name)", term, len(listByFullName)))
}
// Find based on partial ID, returns true only if no duplicates
@@ -383,8 +372,8 @@ func (n *networkRouter) findUniqueNetwork(term string) (types.NetworkResource, e
}
}
if len(listByPartialID) > 1 {
return types.NetworkResource{}, errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on ID prefix)", term, len(listByPartialID)))
return network.Inspect{}, errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on ID prefix)", term, len(listByPartialID)))
}
return types.NetworkResource{}, errdefs.NotFound(libnetwork.ErrNoSuchNetwork(term))
return network.Inspect{}, errdefs.NotFound(libnetwork.ErrNoSuchNetwork(term))
}

View File

@@ -5,8 +5,9 @@ import (
"io"
"net/http"
"github.com/docker/distribution/reference"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/plugin"
@@ -14,11 +15,11 @@ import (
// Backend for Plugin
type Backend interface {
Disable(name string, config *types.PluginDisableConfig) error
Enable(name string, config *types.PluginEnableConfig) error
Disable(name string, config *backend.PluginDisableConfig) error
Enable(name string, config *backend.PluginEnableConfig) error
List(filters.Args) ([]types.Plugin, error)
Inspect(name string) (*types.Plugin, error)
Remove(name string, config *types.PluginRmConfig) error
Remove(name string, config *backend.PluginRmConfig) error
Set(name string, args []string) error
Privileges(ctx context.Context, ref reference.Named, metaHeaders http.Header, authConfig *registry.AuthConfig) (types.PluginPrivileges, error)
Pull(ctx context.Context, ref reference.Named, name string, metaHeaders http.Header, authConfig *registry.AuthConfig, privileges types.PluginPrivileges, outStream io.Writer, opts ...plugin.CreateOpt) error

View File

@@ -6,9 +6,10 @@ import (
"strconv"
"strings"
"github.com/docker/distribution/reference"
"github.com/distribution/reference"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/pkg/ioutils"
@@ -186,7 +187,8 @@ func (pr *pluginRouter) createPlugin(ctx context.Context, w http.ResponseWriter,
}
options := &types.PluginCreateOptions{
RepoName: r.FormValue("name")}
RepoName: r.FormValue("name"),
}
if err := pr.backend.CreateFromContext(ctx, r.Body, options); err != nil {
return err
@@ -206,7 +208,7 @@ func (pr *pluginRouter) enablePlugin(ctx context.Context, w http.ResponseWriter,
if err != nil {
return err
}
config := &types.PluginEnableConfig{Timeout: timeout}
config := &backend.PluginEnableConfig{Timeout: timeout}
return pr.backend.Enable(name, config)
}
@@ -217,7 +219,7 @@ func (pr *pluginRouter) disablePlugin(ctx context.Context, w http.ResponseWriter
}
name := vars["name"]
config := &types.PluginDisableConfig{
config := &backend.PluginDisableConfig{
ForceDisable: httputils.BoolValue(r, "force"),
}
@@ -230,7 +232,7 @@ func (pr *pluginRouter) removePlugin(ctx context.Context, w http.ResponseWriter,
}
name := vars["name"]
config := &types.PluginRmConfig{
config := &backend.PluginRmConfig{
ForceRemove: httputils.BoolValue(r, "force"),
}
return pr.backend.Remove(name, config)

View File

@@ -3,46 +3,41 @@ package swarm // import "github.com/docker/docker/api/server/router/swarm"
import (
"context"
basictypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
types "github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/swarm"
)
// Backend abstracts a swarm manager.
type Backend interface {
Init(req types.InitRequest) (string, error)
Join(req types.JoinRequest) error
Init(req swarm.InitRequest) (string, error)
Join(req swarm.JoinRequest) error
Leave(ctx context.Context, force bool) error
Inspect() (types.Swarm, error)
Update(uint64, types.Spec, types.UpdateFlags) error
Inspect() (swarm.Swarm, error)
Update(uint64, swarm.Spec, swarm.UpdateFlags) error
GetUnlockKey() (string, error)
UnlockSwarm(req types.UnlockRequest) error
GetServices(basictypes.ServiceListOptions) ([]types.Service, error)
GetService(idOrName string, insertDefaults bool) (types.Service, error)
CreateService(types.ServiceSpec, string, bool) (*basictypes.ServiceCreateResponse, error)
UpdateService(string, uint64, types.ServiceSpec, basictypes.ServiceUpdateOptions, bool) (*basictypes.ServiceUpdateResponse, error)
UnlockSwarm(req swarm.UnlockRequest) error
GetServices(types.ServiceListOptions) ([]swarm.Service, error)
GetService(idOrName string, insertDefaults bool) (swarm.Service, error)
CreateService(swarm.ServiceSpec, string, bool) (*swarm.ServiceCreateResponse, error)
UpdateService(string, uint64, swarm.ServiceSpec, types.ServiceUpdateOptions, bool) (*swarm.ServiceUpdateResponse, error)
RemoveService(string) error
ServiceLogs(context.Context, *backend.LogSelector, *basictypes.ContainerLogsOptions) (<-chan *backend.LogMessage, error)
GetNodes(basictypes.NodeListOptions) ([]types.Node, error)
GetNode(string) (types.Node, error)
UpdateNode(string, uint64, types.NodeSpec) error
ServiceLogs(context.Context, *backend.LogSelector, *container.LogsOptions) (<-chan *backend.LogMessage, error)
GetNodes(types.NodeListOptions) ([]swarm.Node, error)
GetNode(string) (swarm.Node, error)
UpdateNode(string, uint64, swarm.NodeSpec) error
RemoveNode(string, bool) error
GetTasks(basictypes.TaskListOptions) ([]types.Task, error)
GetTask(string) (types.Task, error)
GetSecrets(opts basictypes.SecretListOptions) ([]types.Secret, error)
CreateSecret(s types.SecretSpec) (string, error)
GetTasks(types.TaskListOptions) ([]swarm.Task, error)
GetTask(string) (swarm.Task, error)
GetSecrets(opts types.SecretListOptions) ([]swarm.Secret, error)
CreateSecret(s swarm.SecretSpec) (string, error)
RemoveSecret(idOrName string) error
GetSecret(id string) (types.Secret, error)
UpdateSecret(idOrName string, version uint64, spec types.SecretSpec) error
GetConfigs(opts basictypes.ConfigListOptions) ([]types.Config, error)
CreateConfig(s types.ConfigSpec) (string, error)
GetSecret(id string) (swarm.Secret, error)
UpdateSecret(idOrName string, version uint64, spec swarm.SecretSpec) error
GetConfigs(opts types.ConfigListOptions) ([]swarm.Config, error)
CreateConfig(s swarm.ConfigSpec) (string, error)
RemoveConfig(id string) error
GetConfig(id string) (types.Config, error)
UpdateConfig(idOrName string, version uint64, spec types.ConfigSpec) error
GetConfig(id string) (swarm.Config, error)
UpdateConfig(idOrName string, version uint64, spec swarm.ConfigSpec) error
}

View File

@@ -6,6 +6,7 @@ import (
"net/http"
"strconv"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
basictypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
@@ -15,7 +16,6 @@ import (
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
func (sr *swarmRouter) initCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -36,7 +36,7 @@ func (sr *swarmRouter) initCluster(ctx context.Context, w http.ResponseWriter, r
}
nodeID, err := sr.backend.Init(req)
if err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error initializing swarm")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error initializing swarm")
return err
}
return httputils.WriteJSON(w, http.StatusOK, nodeID)
@@ -62,7 +62,7 @@ func (sr *swarmRouter) leaveCluster(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) inspectCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
swarm, err := sr.backend.Inspect()
if err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error getting swarm")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting swarm")
return err
}
@@ -114,7 +114,7 @@ func (sr *swarmRouter) updateCluster(ctx context.Context, w http.ResponseWriter,
}
if err := sr.backend.Update(version, swarm, flags); err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error configuring swarm")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error configuring swarm")
return err
}
return nil
@@ -127,7 +127,7 @@ func (sr *swarmRouter) unlockCluster(ctx context.Context, w http.ResponseWriter,
}
if err := sr.backend.UnlockSwarm(req); err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error unlocking swarm")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error unlocking swarm")
return err
}
return nil
@@ -136,7 +136,7 @@ func (sr *swarmRouter) unlockCluster(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) getUnlockKey(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
unlockKey, err := sr.backend.GetUnlockKey()
if err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error retrieving swarm unlock key")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error retrieving swarm unlock key")
return err
}
@@ -168,7 +168,7 @@ func (sr *swarmRouter) getServices(ctx context.Context, w http.ResponseWriter, r
services, err := sr.backend.GetServices(basictypes.ServiceListOptions{Filters: filter, Status: status})
if err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error getting services")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting services")
return err
}
@@ -194,7 +194,7 @@ func (sr *swarmRouter) getService(ctx context.Context, w http.ResponseWriter, r
service, err := sr.backend.GetService(vars["id"], insertDefaults)
if err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error getting service")
@@ -209,6 +209,10 @@ func (sr *swarmRouter) createService(ctx context.Context, w http.ResponseWriter,
if err := httputils.ReadJSON(r, &service); err != nil {
return err
}
// TODO(thaJeztah): remove logentries check and migration code in release v26.0.0.
if service.TaskTemplate.LogDriver != nil && service.TaskTemplate.LogDriver.Name == "logentries" {
return errdefs.InvalidParameter(errors.New("the logentries logging driver has been deprecated and removed"))
}
// Get returns "" if the header does not exist
encodedAuth := r.Header.Get(registry.AuthHeader)
@@ -219,9 +223,10 @@ func (sr *swarmRouter) createService(ctx context.Context, w http.ResponseWriter,
}
adjustForAPIVersion(v, &service)
}
resp, err := sr.backend.CreateService(service, encodedAuth, queryRegistry)
if err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithFields(log.Fields{
"error": err,
"service-name": service.Name,
}).Debug("Error creating service")
@@ -236,6 +241,10 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
if err := httputils.ReadJSON(r, &service); err != nil {
return err
}
// TODO(thaJeztah): remove logentries check and migration code in release v26.0.0.
if service.TaskTemplate.LogDriver != nil && service.TaskTemplate.LogDriver.Name == "logentries" {
return errdefs.InvalidParameter(errors.New("the logentries logging driver has been deprecated and removed"))
}
rawVersion := r.URL.Query().Get("version")
version, err := strconv.ParseUint(rawVersion, 10, 64)
@@ -260,7 +269,7 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
resp, err := sr.backend.UpdateService(vars["id"], version, service, flags, queryRegistry)
if err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error updating service")
@@ -271,7 +280,7 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) removeService(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := sr.backend.RemoveService(vars["id"]); err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error removing service")
@@ -315,7 +324,7 @@ func (sr *swarmRouter) getNodes(ctx context.Context, w http.ResponseWriter, r *h
nodes, err := sr.backend.GetNodes(basictypes.NodeListOptions{Filters: filter})
if err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error getting nodes")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting nodes")
return err
}
@@ -325,7 +334,7 @@ func (sr *swarmRouter) getNodes(ctx context.Context, w http.ResponseWriter, r *h
func (sr *swarmRouter) getNode(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
node, err := sr.backend.GetNode(vars["id"])
if err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error getting node")
@@ -349,7 +358,7 @@ func (sr *swarmRouter) updateNode(ctx context.Context, w http.ResponseWriter, r
}
if err := sr.backend.UpdateNode(vars["id"], version, node); err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error updating node")
@@ -366,7 +375,7 @@ func (sr *swarmRouter) removeNode(ctx context.Context, w http.ResponseWriter, r
force := httputils.BoolValue(r, "force")
if err := sr.backend.RemoveNode(vars["id"], force); err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error removing node")
@@ -386,7 +395,7 @@ func (sr *swarmRouter) getTasks(ctx context.Context, w http.ResponseWriter, r *h
tasks, err := sr.backend.GetTasks(basictypes.TaskListOptions{Filters: filter})
if err != nil {
logrus.WithContext(ctx).WithError(err).Debug("Error getting tasks")
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting tasks")
return err
}
@@ -396,7 +405,7 @@ func (sr *swarmRouter) getTasks(ctx context.Context, w http.ResponseWriter, r *h
func (sr *swarmRouter) getTask(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
task, err := sr.backend.GetTask(vars["id"])
if err != nil {
logrus.WithContext(ctx).WithFields(logrus.Fields{
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"task-id": vars["id"],
}).Debug("Error getting task")

View File

@@ -8,6 +8,7 @@ import (
"github.com/docker/docker/api/server/httputils"
basictypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/versions"
)
@@ -25,9 +26,9 @@ func (sr *swarmRouter) swarmLogs(ctx context.Context, w http.ResponseWriter, r *
return fmt.Errorf("Bad parameters: you must choose at least one stream")
}
// there is probably a neater way to manufacture the ContainerLogsOptions
// there is probably a neater way to manufacture the LogsOptions
// struct, probably in the caller, to eliminate the dependency on net/http
logsConfig := &basictypes.ContainerLogsOptions{
logsConfig := &container.LogsOptions{
Follow: httputils.BoolValue(r, "follow"),
Timestamps: httputils.BoolValue(r, "timestamps"),
Since: r.Form.Get("since"),
@@ -77,6 +78,16 @@ func adjustForAPIVersion(cliVersion string, service *swarm.ServiceSpec) {
if cliVersion == "" {
return
}
if versions.LessThan(cliVersion, "1.46") {
if service.TaskTemplate.ContainerSpec != nil {
for i, mount := range service.TaskTemplate.ContainerSpec.Mounts {
if mount.TmpfsOptions != nil {
mount.TmpfsOptions.Options = nil
service.TaskTemplate.ContainerSpec.Mounts[i] = mount
}
}
}
}
if versions.LessThan(cliVersion, "1.40") {
if service.TaskTemplate.ContainerSpec != nil {
// Sysctls for docker swarm services weren't supported before
@@ -118,4 +129,26 @@ func adjustForAPIVersion(cliVersion string, service *swarm.ServiceSpec) {
service.Mode.ReplicatedJob = nil
service.Mode.GlobalJob = nil
}
if versions.LessThan(cliVersion, "1.44") {
if service.TaskTemplate.ContainerSpec != nil {
// seccomp, apparmor, and no_new_privs were added in 1.44.
if service.TaskTemplate.ContainerSpec.Privileges != nil {
service.TaskTemplate.ContainerSpec.Privileges.Seccomp = nil
service.TaskTemplate.ContainerSpec.Privileges.AppArmor = nil
service.TaskTemplate.ContainerSpec.Privileges.NoNewPrivileges = false
}
if service.TaskTemplate.ContainerSpec.Healthcheck != nil {
// StartInterval was added in API 1.44
service.TaskTemplate.ContainerSpec.Healthcheck.StartInterval = 0
}
}
}
if versions.LessThan(cliVersion, "1.46") {
if service.TaskTemplate.ContainerSpec != nil && service.TaskTemplate.ContainerSpec.OomScoreAdj != 0 {
// OomScoreAdj was added in API 1.46
service.TaskTemplate.ContainerSpec.OomScoreAdj = 0
}
}
}

View File

@@ -4,14 +4,13 @@ import (
"reflect"
"testing"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/go-units"
)
func TestAdjustForAPIVersion(t *testing.T) {
var (
expectedSysctls = map[string]string{"foo": "bar"}
)
expectedSysctls := map[string]string{"foo": "bar"}
// testing the negative -- does this leave everything else alone? -- is
// prohibitively time-consuming to write, because it would need an object
// with literally every field filled in.
@@ -40,13 +39,25 @@ func TestAdjustForAPIVersion(t *testing.T) {
ConfigName: "configRuntime",
},
},
Ulimits: []*units.Ulimit{
Ulimits: []*container.Ulimit{
{
Name: "nofile",
Soft: 100,
Hard: 200,
},
},
Mounts: []mount.Mount{
{
Type: mount.TypeTmpfs,
Source: "/foo",
Target: "/bar",
TmpfsOptions: &mount.TmpfsOptions{
Options: [][]string{
{"exec"},
},
},
},
},
},
Placement: &swarm.Placement{
MaxReplicas: 222,
@@ -59,6 +70,19 @@ func TestAdjustForAPIVersion(t *testing.T) {
},
}
adjustForAPIVersion("1.46", spec)
if !reflect.DeepEqual(
spec.TaskTemplate.ContainerSpec.Mounts[0].TmpfsOptions.Options,
[][]string{{"exec"}},
) {
t.Error("TmpfsOptions.Options was stripped from spec")
}
adjustForAPIVersion("1.45", spec)
if len(spec.TaskTemplate.ContainerSpec.Mounts[0].TmpfsOptions.Options) != 0 {
t.Error("TmpfsOptions.Options not stripped from spec")
}
// first, does calling this with a later version correctly NOT strip
// fields? do the later version first, so we can reuse this spec in the
// next test.

View File

@@ -9,6 +9,7 @@ import (
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/system"
)
// DiskUsageOptions holds parameters for system disk usage query.
@@ -26,8 +27,8 @@ type DiskUsageOptions struct {
// Backend is the methods that need to be implemented to provide
// system specific functionality.
type Backend interface {
SystemInfo() *types.Info
SystemVersion() types.Version
SystemInfo(context.Context) (*system.Info, error)
SystemVersion(context.Context) (types.Version, error)
SystemDiskUsage(ctx context.Context, opts DiskUsageOptions) (*types.DiskUsage, error)
SubscribeToEvents(since, until time.Time, ef filters.Args) ([]events.Message, chan interface{})
UnsubscribeFromEvents(chan interface{})
@@ -37,7 +38,7 @@ type Backend interface {
// ClusterBackend is all the methods that need to be implemented
// to provide cluster system specific functionality.
type ClusterBackend interface {
Info() swarm.Info
Info(context.Context) swarm.Info
}
// StatusProvider provides methods to get the swarm status of the current node.

View File

@@ -1,8 +1,13 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.22
package system // import "github.com/docker/docker/api/server/router/system"
import (
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/types/system"
buildkit "github.com/docker/docker/builder/builder-next"
"resenje.org/singleflight"
)
// systemRouter provides information about the Docker system overall.
@@ -12,11 +17,16 @@ type systemRouter struct {
cluster ClusterBackend
routes []router.Route
builder *buildkit.Builder
features *map[string]bool
features func() map[string]bool
// collectSystemInfo is a single-flight for the /info endpoint,
// unique per API version (as different API versions may return
// a different API response).
collectSystemInfo singleflight.Group[string, *system.Info]
}
// NewRouter initializes a new system router
func NewRouter(b Backend, c ClusterBackend, builder *buildkit.Builder, features *map[string]bool) router.Router {
func NewRouter(b Backend, c ClusterBackend, builder *buildkit.Builder, features func() map[string]bool) router.Router {
r := &systemRouter{
backend: b,
cluster: c,

View File

@@ -7,6 +7,7 @@ import (
"net/http"
"time"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/router/build"
"github.com/docker/docker/api/types"
@@ -14,11 +15,11 @@ import (
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/system"
timetypes "github.com/docker/docker/api/types/time"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/pkg/ioutils"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
@@ -31,7 +32,7 @@ func (s *systemRouter) pingHandler(ctx context.Context, w http.ResponseWriter, r
w.Header().Add("Cache-Control", "no-cache, no-store, must-revalidate")
w.Header().Add("Pragma", "no-cache")
builderVersion := build.BuilderVersion(*s.features)
builderVersion := build.BuilderVersion(s.features())
if bv := builderVersion; bv != "" {
w.Header().Set("Builder-Version", string(bv))
}
@@ -57,51 +58,61 @@ func (s *systemRouter) swarmStatus() string {
}
func (s *systemRouter) getInfo(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
info := s.backend.SystemInfo()
if s.cluster != nil {
info.Swarm = s.cluster.Info()
info.Warnings = append(info.Warnings, info.Swarm.Warnings...)
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.25") {
// TODO: handle this conversion in engine-api
type oldInfo struct {
*types.Info
ExecutionDriver string
}
old := &oldInfo{
Info: info,
ExecutionDriver: "<not supported>",
}
nameOnlySecurityOptions := []string{}
kvSecOpts, err := types.DecodeSecurityOptions(old.SecurityOptions)
info, _, _ := s.collectSystemInfo.Do(ctx, version, func(ctx context.Context) (*system.Info, error) {
info, err := s.backend.SystemInfo(ctx)
if err != nil {
return err
return nil, err
}
for _, s := range kvSecOpts {
nameOnlySecurityOptions = append(nameOnlySecurityOptions, s.Name)
if s.cluster != nil {
info.Swarm = s.cluster.Info(ctx)
info.Warnings = append(info.Warnings, info.Swarm.Warnings...)
}
old.SecurityOptions = nameOnlySecurityOptions
return httputils.WriteJSON(w, http.StatusOK, old)
}
if versions.LessThan(version, "1.39") {
if info.KernelVersion == "" {
info.KernelVersion = "<unknown>"
if versions.LessThan(version, "1.25") {
// TODO: handle this conversion in engine-api
kvSecOpts, err := system.DecodeSecurityOptions(info.SecurityOptions)
if err != nil {
info.Warnings = append(info.Warnings, err.Error())
}
var nameOnly []string
for _, so := range kvSecOpts {
nameOnly = append(nameOnly, so.Name)
}
info.SecurityOptions = nameOnly
}
if info.OperatingSystem == "" {
info.OperatingSystem = "<unknown>"
if versions.LessThan(version, "1.39") {
if info.KernelVersion == "" {
info.KernelVersion = "<unknown>"
}
if info.OperatingSystem == "" {
info.OperatingSystem = "<unknown>"
}
}
}
if versions.GreaterThanOrEqualTo(version, "1.42") {
info.KernelMemory = false
}
if versions.LessThan(version, "1.44") {
for k, rt := range info.Runtimes {
// Status field introduced in API v1.44.
info.Runtimes[k] = system.RuntimeWithStatus{Runtime: rt.Runtime}
}
}
if versions.LessThan(version, "1.46") {
// Containerd field introduced in API v1.46.
info.Containerd = nil
}
if versions.GreaterThanOrEqualTo(version, "1.42") {
info.KernelMemory = false
}
return info, nil
})
return httputils.WriteJSON(w, http.StatusOK, info)
}
func (s *systemRouter) getVersion(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
info := s.backend.SystemVersion()
info, err := s.backend.SystemVersion(ctx)
if err != nil {
return err
}
return httputils.WriteJSON(w, http.StatusOK, info)
}
@@ -185,6 +196,11 @@ func (s *systemRouter) getDiskUsage(ctx context.Context, w http.ResponseWriter,
b.Parent = "" //nolint:staticcheck // ignore SA1019 (Parent field is deprecated)
}
}
if versions.LessThan(version, "1.44") {
for _, b := range systemDiskUsage.Images {
b.VirtualSize = b.Size //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
}
}
du := types.DiskUsage{
BuildCache: buildCache,
@@ -250,6 +266,7 @@ func (s *systemRouter) getEvents(ctx context.Context, w http.ResponseWriter, r *
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
output := ioutils.NewWriteFlusher(w)
defer output.Close()
output.Flush()
@@ -259,7 +276,18 @@ func (s *systemRouter) getEvents(ctx context.Context, w http.ResponseWriter, r *
buffered, l := s.backend.SubscribeToEvents(since, until, ef)
defer s.backend.UnsubscribeFromEvents(l)
shouldSkip := func(ev events.Message) bool { return false }
if versions.LessThan(httputils.VersionFromContext(ctx), "1.46") {
// Image create events were added in API 1.46
shouldSkip = func(ev events.Message) bool {
return ev.Type == "image" && ev.Action == "create"
}
}
for _, ev := range buffered {
if shouldSkip(ev) {
continue
}
if err := enc.Encode(ev); err != nil {
return err
}
@@ -274,7 +302,10 @@ func (s *systemRouter) getEvents(ctx context.Context, w http.ResponseWriter, r *
case ev := <-l:
jev, ok := ev.(events.Message)
if !ok {
logrus.Warnf("unexpected event message: %q", ev)
log.G(ctx).Warnf("unexpected event message: %q", ev)
continue
}
if shouldSkip(jev) {
continue
}
if err := enc.Encode(jev); err != nil {
@@ -283,7 +314,7 @@ func (s *systemRouter) getEvents(ctx context.Context, w http.ResponseWriter, r *
case <-timeout:
return nil
case <-ctx.Done():
logrus.Debug("Client context cancelled, stop sending events")
log.G(ctx).Debug("Client context cancelled, stop sending events")
return nil
}
}

View File

@@ -3,11 +3,9 @@ package volume // import "github.com/docker/docker/api/server/router/volume"
import (
"context"
"github.com/docker/docker/volume/service/opts"
// TODO return types need to be refactored into pkg
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/volume"
"github.com/docker/docker/volume/service/opts"
)
// Backend is the methods that need to be implemented to provide
@@ -17,7 +15,7 @@ type Backend interface {
Get(ctx context.Context, name string, opts ...opts.GetOption) (*volume.Volume, error)
Create(ctx context.Context, name, driverName string, opts ...opts.CreateOption) (*volume.Volume, error)
Remove(ctx context.Context, name string, opts ...opts.RemoveOption) error
Prune(ctx context.Context, pruneFilters filters.Args) (*types.VolumesPruneReport, error)
Prune(ctx context.Context, pruneFilters filters.Args) (*volume.PruneReport, error)
}
// ClusterBackend is the backend used for Swarm Cluster Volumes. Regular

View File

@@ -6,6 +6,7 @@ import (
"net/http"
"strconv"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/versions"
@@ -13,7 +14,6 @@ import (
"github.com/docker/docker/errdefs"
"github.com/docker/docker/volume/service/opts"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
const (
@@ -116,10 +116,10 @@ func (v *volumeRouter) postVolumesCreate(ctx context.Context, w http.ResponseWri
// Instead, we will allow creating a volume with a duplicate name, which
// should not break anything.
if req.ClusterVolumeSpec != nil && versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) {
logrus.Debug("using cluster volume")
log.G(ctx).Debug("using cluster volume")
vol, err = v.cluster.CreateVolume(req)
} else {
logrus.Debug("using regular volume")
log.G(ctx).Debug("using regular volume")
vol, err = v.backend.Create(ctx, req.Name, req.Driver, opts.WithCreateOptions(req.DriverOpts), opts.WithCreateLabels(req.Labels))
}

View File

@@ -11,7 +11,6 @@ import (
"gotest.tools/v3/assert"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/volume"
"github.com/docker/docker/errdefs"
@@ -78,7 +77,6 @@ func TestGetVolumeByNameFoundRegular(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"volume1": {
Name: "volume1",
},
@@ -108,6 +106,7 @@ func TestGetVolumeByNameFoundSwarm(t *testing.T) {
_, err := callGetVolume(v, "volume1")
assert.NilError(t, err)
}
func TestListVolumes(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
@@ -636,7 +635,7 @@ func (b *fakeVolumeBackend) Remove(_ context.Context, name string, o ...opts.Rem
return nil
}
func (b *fakeVolumeBackend) Prune(_ context.Context, _ filters.Args) (*types.VolumesPruneReport, error) {
func (b *fakeVolumeBackend) Prune(_ context.Context, _ filters.Args) (*volume.PruneReport, error) {
return nil, nil
}

View File

@@ -4,14 +4,16 @@ import (
"context"
"net/http"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/middleware"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/server/router/debug"
"github.com/docker/docker/api/types"
"github.com/docker/docker/dockerversion"
"github.com/gorilla/mux"
"github.com/sirupsen/logrus"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)
// versionMatcher defines a variable matcher to be parsed by the router
@@ -29,8 +31,8 @@ func (s *Server) UseMiddleware(m middleware.Middleware) {
s.middlewares = append(s.middlewares, m)
}
func (s *Server) makeHTTPHandler(handler httputils.APIFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
func (s *Server) makeHTTPHandler(handler httputils.APIFunc, operation string) http.HandlerFunc {
return otelhttp.NewHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Define the context that we'll pass around to share info
// like the docker-request-id.
//
@@ -42,6 +44,7 @@ func (s *Server) makeHTTPHandler(handler httputils.APIFunc) http.HandlerFunc {
// use intermediate variable to prevent "should not use basic type
// string as key in context.WithValue" golint errors
ctx := context.WithValue(r.Context(), dockerversion.UAStringKey{}, r.Header.Get("User-Agent"))
r = r.WithContext(ctx)
handlerFunc := s.handlerWithGlobalMiddlewares(handler)
@@ -53,31 +56,25 @@ func (s *Server) makeHTTPHandler(handler httputils.APIFunc) http.HandlerFunc {
if err := handlerFunc(ctx, w, r, vars); err != nil {
statusCode := httpstatus.FromError(err)
if statusCode >= 500 {
logrus.Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
log.G(ctx).Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
}
makeErrorHandler(err)(w, r)
_ = httputils.WriteJSON(w, statusCode, &types.ErrorResponse{
Message: err.Error(),
})
}
}
}), operation).ServeHTTP
}
type pageNotFoundError struct{}
func (pageNotFoundError) Error() string {
return "page not found"
}
func (pageNotFoundError) NotFound() {}
// CreateMux returns a new mux with all the routers registered.
func (s *Server) CreateMux(routers ...router.Router) *mux.Router {
m := mux.NewRouter()
logrus.Debug("Registering routers")
log.G(context.TODO()).Debug("Registering routers")
for _, apiRouter := range routers {
for _, r := range apiRouter.Routes() {
f := s.makeHTTPHandler(r.Handler())
f := s.makeHTTPHandler(r.Handler(), r.Method()+" "+r.Path())
logrus.Debugf("Registering %s, %s", r.Method(), r.Path())
log.G(context.TODO()).Debugf("Registering %s, %s", r.Method(), r.Path())
m.Path(versionMatcher + r.Path()).Methods(r.Method()).Handler(f)
m.Path(r.Path()).Methods(r.Method()).Handler(f)
}
@@ -85,11 +82,16 @@ func (s *Server) CreateMux(routers ...router.Router) *mux.Router {
debugRouter := debug.NewRouter()
for _, r := range debugRouter.Routes() {
f := s.makeHTTPHandler(r.Handler())
f := s.makeHTTPHandler(r.Handler(), r.Method()+" "+r.Path())
m.Path("/debug" + r.Path()).Handler(f)
}
notFoundHandler := makeErrorHandler(pageNotFoundError{})
notFoundHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = httputils.WriteJSON(w, http.StatusNotFound, &types.ErrorResponse{
Message: "page not found",
})
})
m.HandleFunc(versionMatcher+"/{path:.*}", notFoundHandler)
m.NotFoundHandler = notFoundHandler
m.MethodNotAllowedHandler = notFoundHandler

View File

@@ -15,7 +15,11 @@ import (
func TestMiddlewares(t *testing.T) {
srv := &Server{}
srv.UseMiddleware(middleware.NewVersionMiddleware("0.1omega2", api.DefaultVersion, api.MinVersion))
m, err := middleware.NewVersionMiddleware("0.1omega2", api.DefaultVersion, api.MinSupportedAPIVersion)
if err != nil {
t.Fatal(err)
}
srv.UseMiddleware(*m)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
resp := httptest.NewRecorder()

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +0,0 @@
package types // import "github.com/docker/docker/api/types"
import "github.com/docker/docker/api/types/registry"
// AuthConfig contains authorization information for connecting to a Registry.
//
// Deprecated: use github.com/docker/docker/api/types/registry.AuthConfig
type AuthConfig = registry.AuthConfig

View File

@@ -0,0 +1,26 @@
package auxprogress
import (
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// ManifestPushedInsteadOfIndex is a note that is sent when a manifest is pushed
// instead of an index. It is sent when the pushed image is an multi-platform
// index, but the whole index couldn't be pushed.
type ManifestPushedInsteadOfIndex struct {
ManifestPushedInsteadOfIndex bool `json:"manifestPushedInsteadOfIndex"` // Always true
// OriginalIndex is the descriptor of the original image index.
OriginalIndex ocispec.Descriptor `json:"originalIndex"`
// SelectedManifest is the descriptor of the manifest that was pushed instead.
SelectedManifest ocispec.Descriptor `json:"selectedManifest"`
}
// ContentMissing is a note that is sent when push fails because the content is missing.
type ContentMissing struct {
ContentMissing bool `json:"contentMissing"` // Always true
// Desc is the descriptor of the root object that was attempted to be pushed.
Desc ocispec.Descriptor `json:"desc"`
}

View File

@@ -5,13 +5,32 @@ import (
"io"
"time"
"github.com/docker/distribution/reference"
"github.com/distribution/reference"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// ContainerCreateConfig is the parameter set to ContainerCreate()
type ContainerCreateConfig struct {
Name string
Config *container.Config
HostConfig *container.HostConfig
NetworkingConfig *network.NetworkingConfig
Platform *ocispec.Platform
DefaultReadOnlyNonRecursive bool
}
// ContainerRmConfig holds arguments for the container remove
// operation. This struct is used to tell the backend what operations
// to perform.
type ContainerRmConfig struct {
ForceRemove, RemoveVolume, RemoveLink bool
}
// ContainerAttachConfig holds the streams to use when connecting to a container to view logs.
type ContainerAttachConfig struct {
GetStreams func(multiplexed bool) (io.ReadCloser, io.Writer, io.Writer, error)
GetStreams func(multiplexed bool, cancel func()) (io.ReadCloser, io.Writer, io.Writer, error)
UseStdin bool
UseStdout bool
UseStderr bool
@@ -70,8 +89,15 @@ type LogSelector struct {
type ContainerStatsConfig struct {
Stream bool
OneShot bool
OutStream io.Writer
Version string
OutStream func() io.Writer
}
// ExecStartConfig holds the options to start container's exec.
type ExecStartConfig struct {
Stdin io.Reader
Stdout io.Writer
Stderr io.Writer
ConsoleSize *[2]uint `json:",omitempty"`
}
// ExecInspect holds information about a running process started
@@ -111,6 +137,13 @@ type CreateImageConfig struct {
Changes []string
}
// GetImageOpts holds parameters to retrieve image information
// from the backend.
type GetImageOpts struct {
Platform *ocispec.Platform
Details bool
}
// CommitConfig is the configuration for creating an image as part of a build.
type CommitConfig struct {
Author string
@@ -122,3 +155,25 @@ type CommitConfig struct {
ContainerOS string
ParentImageID string
}
// PluginRmConfig holds arguments for plugin remove.
type PluginRmConfig struct {
ForceRemove bool
}
// PluginEnableConfig holds arguments for plugin enable
type PluginEnableConfig struct {
Timeout int
}
// PluginDisableConfig holds arguments for plugin disable.
type PluginDisableConfig struct {
ForceDisable bool
}
// NetworkListConfig stores the options available for listing networks
type NetworkListConfig struct {
// TODO(@cpuguy83): naming is hard, this is pulled from what was being used in the router before moving here
Detailed bool
Verbose bool
}

View File

@@ -6,7 +6,7 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/pkg/streamformatter"
specs "github.com/opencontainers/image-spec/specs-go/v1"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// PullOption defines different modes for accessing images
@@ -42,5 +42,5 @@ type GetImageAndLayerOptions struct {
PullOption PullOption
AuthConfig map[string]registry.AuthConfig
Output io.Writer
Platform *specs.Platform
Platform *ocispec.Platform
}

View File

@@ -0,0 +1,7 @@
package checkpoint
// Summary represents the details of a checkpoint when listing endpoints.
type Summary struct {
// Name is the name of the checkpoint.
Name string
}

View File

@@ -0,0 +1,19 @@
package checkpoint
// CreateOptions holds parameters to create a checkpoint from a container.
type CreateOptions struct {
CheckpointID string
CheckpointDir string
Exit bool
}
// ListOptions holds parameters to list checkpoints for a container.
type ListOptions struct {
CheckpointDir string
}
// DeleteOptions holds parameters to delete a checkpoint from a container.
type DeleteOptions struct {
CheckpointID string
CheckpointDir string
}

View File

@@ -2,117 +2,15 @@ package types // import "github.com/docker/docker/api/types"
import (
"bufio"
"context"
"io"
"net"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
units "github.com/docker/go-units"
)
// CheckpointCreateOptions holds parameters to create a checkpoint from a container
type CheckpointCreateOptions struct {
CheckpointID string
CheckpointDir string
Exit bool
}
// CheckpointListOptions holds parameters to list checkpoints for a container
type CheckpointListOptions struct {
CheckpointDir string
}
// CheckpointDeleteOptions holds parameters to delete a checkpoint from a container
type CheckpointDeleteOptions struct {
CheckpointID string
CheckpointDir string
}
// ContainerAttachOptions holds parameters to attach to a container.
type ContainerAttachOptions struct {
Stream bool
Stdin bool
Stdout bool
Stderr bool
DetachKeys string
Logs bool
}
// ContainerCommitOptions holds parameters to commit changes into a container.
type ContainerCommitOptions struct {
Reference string
Comment string
Author string
Changes []string
Pause bool
Config *container.Config
}
// ContainerExecInspect holds information returned by exec inspect.
type ContainerExecInspect struct {
ExecID string `json:"ID"`
ContainerID string
Running bool
ExitCode int
Pid int
}
// ContainerListOptions holds parameters to list containers with.
type ContainerListOptions struct {
Size bool
All bool
Latest bool
Since string
Before string
Limit int
Filters filters.Args
}
// ContainerLogsOptions holds parameters to filter logs with.
type ContainerLogsOptions struct {
ShowStdout bool
ShowStderr bool
Since string
Until string
Timestamps bool
Follow bool
Tail string
Details bool
}
// ContainerRemoveOptions holds parameters to remove containers.
type ContainerRemoveOptions struct {
RemoveVolumes bool
RemoveLinks bool
Force bool
}
// ContainerStartOptions holds parameters to start containers.
type ContainerStartOptions struct {
CheckpointID string
CheckpointDir string
}
// CopyToContainerOptions holds information
// about files to copy into a container
type CopyToContainerOptions struct {
AllowOverwriteDirWithFile bool
CopyUIDGID bool
}
// EventsOptions holds parameters to filter events with.
type EventsOptions struct {
Since string
Until string
Filters filters.Args
}
// NetworkListOptions holds parameters to filter the list of networks with.
type NetworkListOptions struct {
Filters filters.Args
}
// NewHijackedResponse intializes a HijackedResponse type
func NewHijackedResponse(conn net.Conn, mediaType string) HijackedResponse {
return HijackedResponse{Conn: conn, Reader: bufio.NewReader(conn), mediaType: mediaType}
@@ -175,7 +73,7 @@ type ImageBuildOptions struct {
NetworkMode string
ShmSize int64
Dockerfile string
Ulimits []*units.Ulimit
Ulimits []*container.Ulimit
// BuildArgs needs to be a *string instead of just a string so that
// we can tell the difference between "" (empty string) and no value
// at all (nil). See the parsing of buildArgs in
@@ -196,7 +94,7 @@ type ImageBuildOptions struct {
Target string
SessionID string
Platform string
// Version specifies the version of the unerlying builder to use
// Version specifies the version of the underlying builder to use
Version BuilderVersion
// BuildID is an optional identifier that can be passed together with the
// build request. The same identifier can be used to gracefully cancel the
@@ -231,89 +129,13 @@ type ImageBuildResponse struct {
OSType string
}
// ImageCreateOptions holds information to create images.
type ImageCreateOptions struct {
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry.
Platform string // Platform is the target platform of the image if it needs to be pulled from the registry.
}
// ImageImportSource holds source information for ImageImport
type ImageImportSource struct {
Source io.Reader // Source is the data to send to the server to create this image from. You must set SourceName to "-" to leverage this.
SourceName string // SourceName is the name of the image to pull. Set to "-" to leverage the Source attribute.
}
// ImageImportOptions holds information to import images from the client host.
type ImageImportOptions struct {
Tag string // Tag is the name to tag this image with. This attribute is deprecated.
Message string // Message is the message to tag the image with
Changes []string // Changes are the raw changes to apply to this image
Platform string // Platform is the target platform of the image
}
// ImageListOptions holds parameters to list images with.
type ImageListOptions struct {
// All controls whether all images in the graph are filtered, or just
// the heads.
All bool
// Filters is a JSON-encoded set of filter arguments.
Filters filters.Args
// SharedSize indicates whether the shared size of images should be computed.
SharedSize bool
// ContainerCount indicates whether container count should be computed.
ContainerCount bool
}
// ImageLoadResponse returns information to the client about a load process.
type ImageLoadResponse struct {
// Body must be closed to avoid a resource leak
Body io.ReadCloser
JSON bool
}
// ImagePullOptions holds information to pull images.
type ImagePullOptions struct {
All bool
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry
PrivilegeFunc RequestPrivilegeFunc
Platform string
}
// RequestPrivilegeFunc is a function interface that
// clients can supply to retry operations after
// getting an authorization error.
// This function returns the registry authentication
// header value in base 64 format, or an error
// if the privilege request fails.
type RequestPrivilegeFunc func() (string, error)
// ImagePushOptions holds information to push images.
type ImagePushOptions ImagePullOptions
// ImageRemoveOptions holds parameters to remove images.
type ImageRemoveOptions struct {
Force bool
PruneChildren bool
}
// ImageSearchOptions holds parameters to search images with.
type ImageSearchOptions struct {
RegistryAuth string
PrivilegeFunc RequestPrivilegeFunc
Filters filters.Args
Limit int
}
// ResizeOptions holds parameters to resize a tty.
// It can be used to resize container ttys and
// exec process ttys too.
type ResizeOptions struct {
Height uint
Width uint
}
type RequestPrivilegeFunc func(context.Context) (string, error)
// NodeListOptions holds parameters to list nodes with.
type NodeListOptions struct {
@@ -340,15 +162,6 @@ type ServiceCreateOptions struct {
QueryRegistry bool
}
// ServiceCreateResponse contains the information returned to a client
// on the creation of a new service.
type ServiceCreateResponse struct {
// ID is the ID of the created service.
ID string
// Warnings is a set of non-fatal warning messages to pass on to the user.
Warnings []string `json:",omitempty"`
}
// Values for RegistryAuthFrom in ServiceUpdateOptions
const (
RegistryAuthFromSpec = "spec"
@@ -427,7 +240,7 @@ type PluginInstallOptions struct {
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry
RemoteRef string // RemoteRef is the plugin name on the registry
PrivilegeFunc RequestPrivilegeFunc
AcceptPermissionsFunc func(PluginPrivileges) (bool, error)
AcceptPermissionsFunc func(context.Context, PluginPrivileges) (bool, error)
Args []string
}

View File

@@ -1,67 +0,0 @@
package types // import "github.com/docker/docker/api/types"
import (
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// configs holds structs used for internal communication between the
// frontend (such as an http server) and the backend (such as the
// docker daemon).
// ContainerCreateConfig is the parameter set to ContainerCreate()
type ContainerCreateConfig struct {
Name string
Config *container.Config
HostConfig *container.HostConfig
NetworkingConfig *network.NetworkingConfig
Platform *specs.Platform
AdjustCPUShares bool
}
// ContainerRmConfig holds arguments for the container remove
// operation. This struct is used to tell the backend what operations
// to perform.
type ContainerRmConfig struct {
ForceRemove, RemoveVolume, RemoveLink bool
}
// ExecConfig is a small subset of the Config struct that holds the configuration
// for the exec feature of docker.
type ExecConfig struct {
User string // User that will run the command
Privileged bool // Is the container in privileged mode
Tty bool // Attach standard streams to a tty.
ConsoleSize *[2]uint `json:",omitempty"` // Initial console size [height, width]
AttachStdin bool // Attach the standard input, makes possible user interaction
AttachStderr bool // Attach the standard error
AttachStdout bool // Attach the standard output
Detach bool // Execute in detach mode
DetachKeys string // Escape keys for detach
Env []string // Environment variables
WorkingDir string // Working directory
Cmd []string // Execution commands and args
}
// PluginRmConfig holds arguments for plugin remove.
type PluginRmConfig struct {
ForceRemove bool
}
// PluginEnableConfig holds arguments for plugin enable
type PluginEnableConfig struct {
Timeout int
}
// PluginDisableConfig holds arguments for plugin disable.
type PluginDisableConfig struct {
ForceDisable bool
}
// NetworkListConfig stores the options available for listing networks
type NetworkListConfig struct {
// TODO(@cpuguy83): naming is hard, this is pulled from what was being used in the router before moving here
Detailed bool
Verbose bool
}

View File

@@ -1,6 +0,0 @@
package container
// ContainerChangeResponseItem change item in response to ContainerChanges operation
//
// Deprecated: use [FilesystemChange].
type ContainerChangeResponseItem = FilesystemChange

View File

@@ -1,11 +1,11 @@
package container // import "github.com/docker/docker/api/types/container"
import (
"io"
"time"
"github.com/docker/docker/api/types/strslice"
"github.com/docker/go-connections/nat"
dockerspec "github.com/moby/docker-image-spec/specs-go/v1"
)
// MinimumDuration puts a minimum on user configured duration.
@@ -33,33 +33,7 @@ type StopOptions struct {
}
// HealthConfig holds configuration settings for the HEALTHCHECK feature.
type HealthConfig struct {
// Test is the test to perform to check that the container is healthy.
// An empty slice means to inherit the default.
// The options are:
// {} : inherit healthcheck
// {"NONE"} : disable healthcheck
// {"CMD", args...} : exec arguments directly
// {"CMD-SHELL", command} : run command with system's default shell
Test []string `json:",omitempty"`
// Zero means to inherit. Durations are expressed as integer nanoseconds.
Interval time.Duration `json:",omitempty"` // Interval is the time to wait between checks.
Timeout time.Duration `json:",omitempty"` // Timeout is the time to wait before considering the check to have hung.
StartPeriod time.Duration `json:",omitempty"` // The start period for the container to initialize before the retries starts to count down.
// Retries is the number of consecutive failures needed to consider a container as unhealthy.
// Zero means inherit.
Retries int `json:",omitempty"`
}
// ExecStartOptions holds the options to start container's exec.
type ExecStartOptions struct {
Stdin io.Reader
Stdout io.Writer
Stderr io.Writer
ConsoleSize *[2]uint `json:",omitempty"`
}
type HealthConfig = dockerspec.HealthcheckConfig
// Config contains the configuration data about a container.
// It should hold only portable information about the container.
@@ -87,10 +61,13 @@ type Config struct {
WorkingDir string // Current directory (PWD) in the command will be launched
Entrypoint strslice.StrSlice // Entrypoint to run when starting the container
NetworkDisabled bool `json:",omitempty"` // Is network disabled
MacAddress string `json:",omitempty"` // Mac Address of the container
OnBuild []string // ONBUILD metadata that were defined on the image Dockerfile
Labels map[string]string // List of labels set to this container
StopSignal string `json:",omitempty"` // Signal to stop a container
StopTimeout *int `json:",omitempty"` // Timeout (in seconds) to stop a container
Shell strslice.StrSlice `json:",omitempty"` // Shell for shell-form of RUN, CMD, ENTRYPOINT
// Mac Address of the container.
//
// Deprecated: this field is deprecated since API v1.44. Use EndpointSettings.MacAddress instead.
MacAddress string `json:",omitempty"`
OnBuild []string // ONBUILD metadata that were defined on the image Dockerfile
Labels map[string]string // List of labels set to this container
StopSignal string `json:",omitempty"` // Signal to stop a container
StopTimeout *int `json:",omitempty"` // Timeout (in seconds) to stop a container
Shell strslice.StrSlice `json:",omitempty"` // Shell for shell-form of RUN, CMD, ENTRYPOINT
}

View File

@@ -0,0 +1,44 @@
package container
import (
"io"
"os"
"time"
)
// PruneReport contains the response for Engine API:
// POST "/containers/prune"
type PruneReport struct {
ContainersDeleted []string
SpaceReclaimed uint64
}
// PathStat is used to encode the header from
// GET "/containers/{name:.*}/archive"
// "Name" is the file or directory name.
type PathStat struct {
Name string `json:"name"`
Size int64 `json:"size"`
Mode os.FileMode `json:"mode"`
Mtime time.Time `json:"mtime"`
LinkTarget string `json:"linkTarget"`
}
// CopyToContainerOptions holds information
// about files to copy into a container
type CopyToContainerOptions struct {
AllowOverwriteDirWithFile bool
CopyUIDGID bool
}
// StatsResponseReader wraps an io.ReadCloser to read (a stream of) stats
// for a container, as produced by the GET "/stats" endpoint.
//
// The OSType field is set to the server's platform to allow
// platform-specific handling of the response.
//
// TODO(thaJeztah): remove this wrapper, and make OSType part of [StatsResponse].
type StatsResponseReader struct {
Body io.ReadCloser `json:"body"`
OSType string `json:"ostype"`
}

View File

@@ -0,0 +1,13 @@
package container
import "github.com/docker/docker/api/types/network"
// CreateRequest is the request message sent to the server for container
// create calls. It is a config wrapper that holds the container [Config]
// (portable) and the corresponding [HostConfig] (non-portable) and
// [network.NetworkingConfig].
type CreateRequest struct {
*Config
HostConfig *HostConfig `json:"HostConfig,omitempty"`
NetworkingConfig *network.NetworkingConfig `json:"NetworkingConfig,omitempty"`
}

View File

@@ -0,0 +1,9 @@
package container
type errInvalidParameter struct{ error }
func (e *errInvalidParameter) InvalidParameter() {}
func (e *errInvalidParameter) Unwrap() error {
return e.error
}

View File

@@ -0,0 +1,43 @@
package container
// ExecOptions is a small subset of the Config struct that holds the configuration
// for the exec feature of docker.
type ExecOptions struct {
User string // User that will run the command
Privileged bool // Is the container in privileged mode
Tty bool // Attach standard streams to a tty.
ConsoleSize *[2]uint `json:",omitempty"` // Initial console size [height, width]
AttachStdin bool // Attach the standard input, makes possible user interaction
AttachStderr bool // Attach the standard error
AttachStdout bool // Attach the standard output
Detach bool // Execute in detach mode
DetachKeys string // Escape keys for detach
Env []string // Environment variables
WorkingDir string // Working directory
Cmd []string // Execution commands and args
}
// ExecStartOptions is a temp struct used by execStart
// Config fields is part of ExecConfig in runconfig package
type ExecStartOptions struct {
// ExecStart will first check if it's detached
Detach bool
// Check if there's a tty
Tty bool
// Terminal size [height, width], unused if Tty == false
ConsoleSize *[2]uint `json:",omitempty"`
}
// ExecAttachOptions is a temp struct used by execAttach.
//
// TODO(thaJeztah): make this a separate type; ContainerExecAttach does not use the Detach option, and cannot run detached.
type ExecAttachOptions = ExecStartOptions
// ExecInspect holds information returned by exec inspect.
type ExecInspect struct {
ExecID string `json:"ID"`
ContainerID string
Running bool
ExitCode int
Pid int
}

View File

@@ -1,13 +1,16 @@
package container // import "github.com/docker/docker/api/types/container"
import (
"errors"
"fmt"
"strings"
"github.com/docker/docker/api/types/blkiodev"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/strslice"
"github.com/docker/go-connections/nat"
units "github.com/docker/go-units"
"github.com/docker/go-units"
)
// CgroupnsMode represents the cgroup namespace mode of the container
@@ -132,12 +135,12 @@ type NetworkMode string
// IsNone indicates whether container isn't using a network stack.
func (n NetworkMode) IsNone() bool {
return n == "none"
return n == network.NetworkNone
}
// IsDefault indicates whether container uses the default network stack.
func (n NetworkMode) IsDefault() bool {
return n == "default"
return n == network.NetworkDefault
}
// IsPrivate indicates whether container uses its private network stack.
@@ -271,33 +274,42 @@ type DeviceMapping struct {
// RestartPolicy represents the restart policies of the container.
type RestartPolicy struct {
Name string
Name RestartPolicyMode
MaximumRetryCount int
}
type RestartPolicyMode string
const (
RestartPolicyDisabled RestartPolicyMode = "no"
RestartPolicyAlways RestartPolicyMode = "always"
RestartPolicyOnFailure RestartPolicyMode = "on-failure"
RestartPolicyUnlessStopped RestartPolicyMode = "unless-stopped"
)
// IsNone indicates whether the container has the "no" restart policy.
// This means the container will not automatically restart when exiting.
func (rp *RestartPolicy) IsNone() bool {
return rp.Name == "no" || rp.Name == ""
return rp.Name == RestartPolicyDisabled || rp.Name == ""
}
// IsAlways indicates whether the container has the "always" restart policy.
// This means the container will automatically restart regardless of the exit status.
func (rp *RestartPolicy) IsAlways() bool {
return rp.Name == "always"
return rp.Name == RestartPolicyAlways
}
// IsOnFailure indicates whether the container has the "on-failure" restart policy.
// This means the container will automatically restart of exiting with a non-zero exit status.
func (rp *RestartPolicy) IsOnFailure() bool {
return rp.Name == "on-failure"
return rp.Name == RestartPolicyOnFailure
}
// IsUnlessStopped indicates whether the container has the
// "unless-stopped" restart policy. This means the container will
// automatically restart unless user has put it to stopped state.
func (rp *RestartPolicy) IsUnlessStopped() bool {
return rp.Name == "unless-stopped"
return rp.Name == RestartPolicyUnlessStopped
}
// IsSame compares two RestartPolicy to see if they are the same
@@ -305,6 +317,33 @@ func (rp *RestartPolicy) IsSame(tp *RestartPolicy) bool {
return rp.Name == tp.Name && rp.MaximumRetryCount == tp.MaximumRetryCount
}
// ValidateRestartPolicy validates the given RestartPolicy.
func ValidateRestartPolicy(policy RestartPolicy) error {
switch policy.Name {
case RestartPolicyAlways, RestartPolicyUnlessStopped, RestartPolicyDisabled:
if policy.MaximumRetryCount != 0 {
msg := "invalid restart policy: maximum retry count can only be used with 'on-failure'"
if policy.MaximumRetryCount < 0 {
msg += " and cannot be negative"
}
return &errInvalidParameter{errors.New(msg)}
}
return nil
case RestartPolicyOnFailure:
if policy.MaximumRetryCount < 0 {
return &errInvalidParameter{errors.New("invalid restart policy: maximum retry count cannot be negative")}
}
return nil
case "":
// Versions before v25.0.0 created an empty restart-policy "name" as
// default. Allow an empty name with "any" MaximumRetryCount for
// backward-compatibility.
return nil
default:
return &errInvalidParameter{fmt.Errorf("invalid restart policy: unknown policy '%s'; use one of '%s', '%s', '%s', or '%s'", policy.Name, RestartPolicyDisabled, RestartPolicyAlways, RestartPolicyOnFailure, RestartPolicyUnlessStopped)}
}
}
// LogMode is a type to define the available modes for logging
// These modes affect how logs are handled when log messages start piling up.
type LogMode string
@@ -322,6 +361,12 @@ type LogConfig struct {
Config map[string]string
}
// Ulimit is an alias for [units.Ulimit], which may be moving to a different
// location or become a local type. This alias is to help transitioning.
//
// Users are recommended to use this alias instead of using [units.Ulimit] directly.
type Ulimit = units.Ulimit
// Resources contains container's resources (cgroups config, ulimits...)
type Resources struct {
// Applicable to all platforms
@@ -349,14 +394,14 @@ type Resources struct {
// KernelMemory specifies the kernel memory limit (in bytes) for the container.
// Deprecated: kernel 5.4 deprecated kmem.limit_in_bytes.
KernelMemory int64 `json:",omitempty"`
KernelMemoryTCP int64 `json:",omitempty"` // Hard limit for kernel TCP buffer memory (in bytes)
MemoryReservation int64 // Memory soft limit (in bytes)
MemorySwap int64 // Total memory usage (memory + swap); set `-1` to enable unlimited swap
MemorySwappiness *int64 // Tuning container memory swappiness behaviour
OomKillDisable *bool // Whether to disable OOM Killer or not
PidsLimit *int64 // Setting PIDs limit for a container; Set `0` or `-1` for unlimited, or `null` to not change.
Ulimits []*units.Ulimit // List of ulimits to be set in the container
KernelMemory int64 `json:",omitempty"`
KernelMemoryTCP int64 `json:",omitempty"` // Hard limit for kernel TCP buffer memory (in bytes)
MemoryReservation int64 // Memory soft limit (in bytes)
MemorySwap int64 // Total memory usage (memory + swap); set `-1` to enable unlimited swap
MemorySwappiness *int64 // Tuning container memory swappiness behaviour
OomKillDisable *bool // Whether to disable OOM Killer or not
PidsLimit *int64 // Setting PIDs limit for a container; Set `0` or `-1` for unlimited, or `null` to not change.
Ulimits []*Ulimit // List of ulimits to be set in the container
// Applicable to Windows
CPUCount int64 `json:"CpuCount"` // CPU count

View File

@@ -0,0 +1,105 @@
package container
import (
"testing"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestValidateRestartPolicy(t *testing.T) {
tests := []struct {
name string
input RestartPolicy
expectedErr string
}{
{
name: "empty",
input: RestartPolicy{},
},
{
name: "empty with invalid MaxRestartCount (for backward compatibility)",
input: RestartPolicy{MaximumRetryCount: 123},
expectedErr: "", // Allowed for backward compatibility
},
{
name: "empty with negative MaxRestartCount)",
input: RestartPolicy{MaximumRetryCount: -123},
expectedErr: "", // Allowed for backward compatibility
},
{
name: "always",
input: RestartPolicy{Name: RestartPolicyAlways},
},
{
name: "always with MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyAlways, MaximumRetryCount: 123},
expectedErr: "invalid restart policy: maximum retry count can only be used with 'on-failure'",
},
{
name: "always with negative MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyAlways, MaximumRetryCount: -123},
expectedErr: "invalid restart policy: maximum retry count can only be used with 'on-failure' and cannot be negative",
},
{
name: "unless-stopped",
input: RestartPolicy{Name: RestartPolicyUnlessStopped},
},
{
name: "unless-stopped with MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyUnlessStopped, MaximumRetryCount: 123},
expectedErr: "invalid restart policy: maximum retry count can only be used with 'on-failure'",
},
{
name: "unless-stopped with negative MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyUnlessStopped, MaximumRetryCount: -123},
expectedErr: "invalid restart policy: maximum retry count can only be used with 'on-failure' and cannot be negative",
},
{
name: "disabled",
input: RestartPolicy{Name: RestartPolicyDisabled},
},
{
name: "disabled with MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyDisabled, MaximumRetryCount: 123},
expectedErr: "invalid restart policy: maximum retry count can only be used with 'on-failure'",
},
{
name: "disabled with negative MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyDisabled, MaximumRetryCount: -123},
expectedErr: "invalid restart policy: maximum retry count can only be used with 'on-failure' and cannot be negative",
},
{
name: "on-failure",
input: RestartPolicy{Name: RestartPolicyOnFailure},
},
{
name: "on-failure with MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyOnFailure, MaximumRetryCount: 123},
},
{
name: "on-failure with negative MaxRestartCount",
input: RestartPolicy{Name: RestartPolicyOnFailure, MaximumRetryCount: -123},
expectedErr: "invalid restart policy: maximum retry count cannot be negative",
},
{
name: "unknown policy",
input: RestartPolicy{Name: "unknown"},
expectedErr: "invalid restart policy: unknown policy 'unknown'; use one of 'no', 'always', 'on-failure', or 'unless-stopped'",
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
err := ValidateRestartPolicy(tc.input)
if tc.expectedErr == "" {
assert.Check(t, err)
} else {
assert.Check(t, is.ErrorType(err, errdefs.IsInvalidParameter))
assert.Check(t, is.Error(err, tc.expectedErr))
}
})
}
}

View File

@@ -1,42 +1,45 @@
//go:build !windows
// +build !windows
package container // import "github.com/docker/docker/api/types/container"
import "github.com/docker/docker/api/types/network"
// IsValid indicates if an isolation technology is valid
func (i Isolation) IsValid() bool {
return i.IsDefault()
}
// NetworkName returns the name of the network stack.
func (n NetworkMode) NetworkName() string {
if n.IsBridge() {
return "bridge"
} else if n.IsHost() {
return "host"
} else if n.IsContainer() {
return "container"
} else if n.IsNone() {
return "none"
} else if n.IsDefault() {
return "default"
} else if n.IsUserDefined() {
return n.UserDefined()
}
return ""
}
// IsBridge indicates whether container uses the bridge network stack
func (n NetworkMode) IsBridge() bool {
return n == "bridge"
return n == network.NetworkBridge
}
// IsHost indicates whether container uses the host network stack.
func (n NetworkMode) IsHost() bool {
return n == "host"
return n == network.NetworkHost
}
// IsUserDefined indicates user-created network
func (n NetworkMode) IsUserDefined() bool {
return !n.IsDefault() && !n.IsBridge() && !n.IsHost() && !n.IsNone() && !n.IsContainer()
}
// NetworkName returns the name of the network stack.
func (n NetworkMode) NetworkName() string {
switch {
case n.IsDefault():
return network.NetworkDefault
case n.IsBridge():
return network.NetworkBridge
case n.IsHost():
return network.NetworkHost
case n.IsNone():
return network.NetworkNone
case n.IsContainer():
return "container"
case n.IsUserDefined():
return n.UserDefined()
default:
return ""
}
}

Some files were not shown because too many files have changed in this diff Show More