Compare commits

..

536 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
a312606256 Merge commit from fork
[26.0] AuthZ plugin security fixes
2024-07-23 21:36:28 +02:00
Jameson Hyde
754d8f168e If url includes scheme, urlPath will drop hostname, which would not match the auth check
Signed-off-by: Jameson Hyde <jameson.hyde@docker.com>
(cherry picked from commit 754fb8d9d03895ae3ab60d2ad778152b0d835206)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5282cb25d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-17 13:09:10 +02:00
Jameson Hyde
4c31251093 Authz plugin security fixes for 0-length content and path validation
Signed-off-by: Jameson Hyde <jameson.hyde@docker.com>

fix comments

(cherry picked from commit 9659c3a52bac57e615b5fb49b0652baca448643e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2ac8a479c5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-07-17 13:09:08 +02:00
Sebastiaan van Stijn
8ce163b7dc Merge pull request #47807 from vvoland/v26.0-47805
[26.0 backport] update to go1.21.10
2024-06-08 17:25:26 +02:00
Paweł Gronowski
ac366556b7 update to go1.21.10
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.9...go1.21.10

These minor releases include 2 security fixes following the security policy:

- cmd/go: arbitrary code execution during build on darwin
On Darwin, building a Go module which contains CGO can trigger arbitrary code execution when using the Apple version of ld, due to usage of the -lto_library flag in a "#cgo LDFLAGS" directive.
Thanks to Juho Forsén of Mattermost for reporting this issue.
This is CVE-2024-24787 and Go issue https://go.dev/issue/67119.

- net: malformed DNS message can cause infinite loop
A malformed DNS message in response to a query can cause the Lookup functions to get stuck in an infinite loop.
Thanks to long-name-let-people-remember-you on GitHub for reporting this issue, and to Mateusz Poliwczak for bringing the issue to our attention.
This is CVE-2024-24788 and Go issue https://go.dev/issue/66754.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.3

**- Description for the changelog**

```markdown changelog
Update Go runtime to 1.21.10
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6c97e0e0b5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-05-08 09:43:51 +02:00
Paweł Gronowski
54aee20458 Merge pull request #47758 from austinvazquez/cherry-pick-ab570ab3d62038b3d26f96a9bb585d0b6095b9b4-to-26.0
[26.0 backport] fix: avoid nil dereference on image history Created value
2024-04-25 17:42:59 +02:00
Christopher Petito
bd6b5b6d37 nil dereference fix on image history Created value
Issue was caused by the changes here https://github.com/moby/moby/pull/45504
First released in v25.0.0-beta.1

Signed-off-by: Christopher Petito <47751006+krissetto@users.noreply.github.com>
(cherry picked from commit ab570ab3d6)
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-04-23 23:19:52 +00:00
Sebastiaan van Stijn
7cef0d9cd1 Merge pull request from GHSA-x84c-p2g9-rqv9
[26.0 backport] Disable IPv6 for endpoints in '--ipv6=false' networks.
2024-04-18 17:50:34 +02:00
Rob Murray
841c4c8057 Disable IPv6 for endpoints in '--ipv6=false' networks.
No IPAM IPv6 address is given to an interface in a network with
'--ipv6=false', but the kernel would assign a link-local address and,
in a macvlan/ipvlan network, the interface may get a SLAAC-assigned
address.

So, disable IPv6 on the interface to avoid that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-16 14:58:04 +01:00
Paweł Gronowski
60b9add796 Merge pull request #47705 from robmry/backport-26.0/47662_ipvlan_l3_dns
[26.0 backport] Enable external DNS for ipvlan-l3, and disable it for macvlan/ipvlan with no parent interface
2024-04-10 14:59:53 +02:00
Rob Murray
8ad7f863b3 Run ipvlan tests even if 'modprobe ipvlan' fails
This reverts commit a77e147d32.

The ipvlan integration tests have been skipped in CI because of a check
intended to ensure the kernel has ipvlan support - which failed, but
seems to be unnecessary (probably because kernels have moved on).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 11:48:58 +01:00
Rob Murray
dc2755273e Stop macvlan with no parent from using ext-dns
We document that an macvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an macvlan
network was still configured with a gateway. So, DNS proxying would
be enabled in the internal resolver (and, if the host's resolver
was on a localhost address, requests to external resolvers from the
host's network namespace would succeed).

This change disables configuration of a gateway for a macvlan Endpoint
if no parent interface is specified.

(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway will still be set up. Documentation
will need to be updated to note that '--internal' should be used to
prevent DNS request forwarding in this case.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 11:48:58 +01:00
Rob Murray
7b570f00ba Enable DNS proxying for ipvlan-l3
The internal DNS resolver should only forward requests to external
resolvers if the libnetwork.Sandbox served by the resolver has external
network access (so, no forwarding for '--internal' networks).

The test for external network access was whether the Sandbox had an
Endpoint with a gateway configured.

However, an ipvlan-l3 networks with external network access does not
have a gateway, it has a default route bound to an interface.

Also, we document that an ipvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an ipvlan-l2
network was configured with a gateway. So, DNS proxying would be enabled
in the internal resolver (and, if the host's resolver was on a localhost
address, requests to external resolvers from the host's network
namespace would succeed).

So, this change adjusts the test for enabling DNS proxying to include
a check for '--internal' (as a shortcut) and, for non-internal networks,
checks for a default route as well as a gateway. It also disables
configuration of a gateway or a default route for an ipvlan Endpoint if
no parent interface is specified.

(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway/default route will still be set up
and external DNS proxying will be enabled. The network must be
configured as '--internal' to prevent that from happening.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 11:48:58 +01:00
Rob Murray
8cdcc4f3c7 Move dummy DNS server to integration/internal/network
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 11:48:58 +01:00
Paweł Gronowski
ed752f6544 Merge pull request #47701 from vvoland/v26.0-47691
[26.0 backport] vendor: github.com/containerd/containerd v1.7.15
2024-04-09 14:15:36 +02:00
Paweł Gronowski
9db1b6f28c Merge pull request #47702 from vvoland/v26.0-47647
[26.0 backport] github/ci: Check if backport is opened against the expected branch
2024-04-09 13:50:28 +02:00
Sebastiaan van Stijn
6261281247 Merge pull request #47700 from vvoland/v26.0-47673
[26.0 backport] vendor: golang.org/x/net v0.23.0
2024-04-09 13:33:11 +02:00
Sebastiaan van Stijn
90355e5166 Merge pull request #47696 from vvoland/v26.0-47658
[26.0 backport] Fix cases where we are wrapping a nil error
2024-04-09 13:29:51 +02:00
Paweł Gronowski
72615b19db github/ci: Check if backport is opened against the expected branch
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 61269e718f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 12:13:00 +02:00
Paweł Gronowski
23e7919e0a Merge pull request #47694 from cpuguy83/26_oci_tar_no_platform
[26.0] save: Remove platform from config descriptor
2024-04-09 12:11:11 +02:00
Paweł Gronowski
c943936458 vendor: github.com/containerd/containerd v1.7.15
full diff: https://github.com/containerd/containerd/compare/v1.7.14...v1.7.15

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5ae5969739)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 12:10:09 +02:00
Sebastiaan van Stijn
8b7940f037 vendor: golang.org/x/net v0.23.0
full diff: https://github.com/golang/net/compare/v0.22.0...v0.23.0

Includes a fix for CVE-2023-45288, which is also addressed in go1.22.2
and go1.21.9;

> http2: close connections when receiving too many headers
>
> Maintaining HPACK state requires that we parse and process
> all HEADERS and CONTINUATION frames on a connection.
> When a request's headers exceed MaxHeaderBytes, we don't
> allocate memory to store the excess headers but we do
> parse them. This permits an attacker to cause an HTTP/2
> endpoint to read arbitrary amounts of data, all associated
> with a request which is going to be rejected.
>
> Set a limit on the amount of excess header frames we
> will process before closing a connection.
>
> Thanks to Bartek Nowotarski for reporting this issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d66589496e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 11:25:28 +02:00
Sebastiaan van Stijn
55207ea637 vendor: golang.org/x/net v0.22.0, golang.org/x/crypto v0.21.0
full diffs changes relevant to vendored code:

- https://github.com/golang/net/compare/v0.18.0...v0.22.0
    - websocket: add support for dialing with context
    - http2: remove suspicious uint32->v conversion in frame code
    - http2: send an error of FLOW_CONTROL_ERROR when exceed the maximum octets
- https://github.com/golang/crypto/compare/v0.17.0...v0.21.0
    - internal/poly1305: drop Go 1.12 compatibility
    - internal/poly1305: improve sum_ppc64le.s
    - ocsp: don't use iota for externally defined constants

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1ca74361b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 11:25:24 +02:00
Brian Goff
a7fa5e1deb Fix cases where we are wrapping a nil error
This was using `errors.Wrap` when there was no error to wrap, meanwhile
we are supposed to be creating a new error.

Found this while investigating some log corruption issues and
unexpectedly getting a nil reader and a nil error from `getTailReader`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 0a48d26fbc)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 10:19:58 +02:00
Sebastiaan van Stijn
8730cccee2 Merge pull request #47692 from vvoland/v26.0-47689
[26.0 backport] update containerd binary to v1.7.15
2024-04-09 09:42:11 +02:00
Brian Goff
61d547bf00 save: Remove platform from config descriptor
This was brought up by bmitch that its not expected to have a platform
object in the config descriptor.
Also checked with tianon who agreed, its not _wrong_ but is unexpected
and doesn't neccessarily make sense to have it there.

Also, while technically incorrect, ECR is throwing an error when it sees
this.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 9160b9fda6)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-04-08 17:11:50 +00:00
Sebastiaan van Stijn
0c9ff4ca23 Merge pull request #47690 from vvoland/v26.0-47682
[26.0 backport] ci/validate-pr: Use `::error::` command to print errors
2024-04-08 19:07:59 +02:00
Paweł Gronowski
46ca4a74b4 update containerd binary to v1.7.15
Update the containerd binary that's used in CI

- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.15
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.15

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3485cfbb1e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 18:09:01 +02:00
Paweł Gronowski
dea47c0810 ci/validate-pr: Use ::error:: command to print errors
This will make Github render the log line as an error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fb92caf2aa)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 14:21:21 +02:00
Sebastiaan van Stijn
f3842ab533 Merge pull request #47671 from vvoland/v26.0-47670
[26.0 backport] update to go1.21.9
2024-04-04 14:30:13 +02:00
Paweł Gronowski
e0815819de update to go1.21.9
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.

These minor releases include 1 security fixes following the security policy:

- http2: close connections when receiving too many headers

Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.

Set a limit on the amount of excess header frames we will process before
closing a connection.

Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.

This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 329d403e20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-04 10:15:00 +02:00
Paweł Gronowski
07c797281e Merge pull request #47637 from vvoland/v26.0-47610
[26.0 backport] Dockerfile: update docker CLI to v26.0.0
2024-04-02 10:39:01 +02:00
Albin Kerouanton
f2550b3c09 Merge pull request #47646 from vvoland/v26.0-47621
[26.0 backport] Restore the SetKey prestart hook.
2024-03-28 10:15:52 +00:00
Rob Murray
fc14d8f932 Restore the SetKey prestart hook.
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"

Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.

But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.

This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit fde80fe2e7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-28 10:05:08 +01:00
Paweł Gronowski
c035ef2283 Merge pull request #47638 from vvoland/v26.0-47636
[26.0 backport] ci: update workflow artifacts retention
2024-03-27 16:58:28 +01:00
CrazyMax
703f14793e ci: update workflow artifacts retention
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit aff003139c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-27 13:21:09 +01:00
Paweł Gronowski
30e94dadbc Dockerfile: update docker CLI to v26.0.0
Update the CLI that's used in the dev-container

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc2...v26.0.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ea72f9f72c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-27 13:17:05 +01:00
Bjorn Neergaard
8b79278316 Merge pull request #47599 from neersighted/short_id_aliases_removal
api: document changed behavior of the `Aliases` field in v1.45
2024-03-20 08:33:39 -06:00
Bjorn Neergaard
22726fb63b api: document changed behavior of the Aliases field in v1.45
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-03-20 08:23:48 -06:00
Bjorn Neergaard
963e1f3eed Merge pull request #47597 from vvoland/c8d-list-fix-shared-size
c8d/list: Fix shared size calculation
2024-03-20 07:26:09 -06:00
Paweł Gronowski
3312b82515 c8d/list: Add a test case for images sharing a top layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 13:17:56 +01:00
Paweł Gronowski
ad8a5a5732 c8d/list: Fix diffIDs being outputted instead of chainIDs
The `identity.ChainIDs` call was accidentally removed in
b37ced2551.

This broke the shared size calculation for images with more than one
layer that were sharing the same compressed layer.

This was could be reproduced with:
```
$ docker pull docker.io/docker/desktop-kubernetes-coredns:v1.11.1
$ docker pull docker.io/docker/desktop-kubernetes-etcd:3.5.10-0
$ docker system df
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:50 +01:00
Paweł Gronowski
0c2d83b5fb c8d/list: Handle unpacked layers when calculating shared size
After a535a65c4b the size reported by the
image list was changed to include all platforms of that image.

This made the "shared size" calculation consider all diff ids of all the
platforms available in the image which caused "snapshot not found"
errors when multiple images were sharing the same layer which wasn't
unpacked.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:28 +01:00
Paweł Gronowski
330d777c53 Merge pull request #47591 from vvoland/api-1.45
docs/api: add documentation for API v1.45
2024-03-19 14:27:45 +01:00
Paweł Gronowski
3d2a56e7cf docs/api: add documentation for API v1.45
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v26.0.0 release.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 13:37:05 +01:00
Paweł Gronowski
4531a371f2 Merge pull request #47580 from vvoland/c8d-list-slow
c8d/list: Generate image summary concurrently
2024-03-19 13:32:52 +01:00
Paweł Gronowski
731a64069f c8d/list: Generate image summary concurrently
Run `imageSummary` concurrently to avoid being IO blocked on the
containerd gRPC.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:58 +01:00
Paweł Gronowski
dade279565 c8d/list: Add Images benchmark
Benchmark the `Images` implementation (image list) against an image
store with 10, 100 and 1000 random images. Currently the images are
single-platform only.

The images are generated randomly, but a fixed seed is used so the
actual testing data will be the same across different executions.

Because the content store is not a real containerd image store but a
local implementation, a small delay (500us) is added to each content
store method call. This is to simulate a real-world usage where each
containerd client call requires a gRPC call.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:56 +01:00
Sebastiaan van Stijn
23e1af45c6 Merge pull request #47582 from vvoland/vendor-buildkit-0.13.1
vendor: github.com/moby/buildkit v0.13.1
2024-03-18 21:53:15 +01:00
Paweł Gronowski
e7c60a30e6 vendor: github.com/moby/buildkit v0.13.1
full diff: https://github.com/moby/buildkit/compare/v0.13.0...v0.13.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 20:16:09 +01:00
Bjorn Neergaard
641e341eed Merge pull request #47538 from robmry/libnet-resolver-nxdomain
libnet: Don't forward to upstream resolvers on internal nw
2024-03-18 11:22:59 -06:00
Sebastiaan van Stijn
dd146571ea Merge pull request #47568 from vvoland/c8d-list-fix
c8d/list: Fix premature `Images` return
2024-03-18 15:28:09 +01:00
Paweł Gronowski
fe70ee9477 Merge pull request #47577 from vvoland/c8d-list-labels-filter
c8d/list: Don't setup label filter if it's not specified
2024-03-18 15:13:40 +01:00
Sebastiaan van Stijn
307962dbd5 Merge pull request #47578 from thaJeztah/fix_resolvconf_go_version
resolvconf: add //go:build directives to prevent downgrading to go1.16 language
2024-03-18 14:00:03 +01:00
Sebastiaan van Stijn
7e56442cee Merge pull request #47574 from thaJeztah/bump_tools
Dockerfile: update docker CLI to v26.0.0-rc2, docker compose v2.25.0
2024-03-18 13:59:42 +01:00
Sebastiaan van Stijn
ebf300c165 Merge pull request #47579 from vvoland/flaky-testdiskusage
integration: Remove Parallel from TestDiskUsage
2024-03-18 13:59:28 +01:00
Paweł Gronowski
2e4ebf032a c8d/list: Pass ctx to setupLabelFilter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 13:50:45 +01:00
Paweł Gronowski
153de36b3f c8d/list: Add empty index test case
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:50:10 +01:00
Sebastiaan van Stijn
4ff655f4b8 resolvconf: add //go:build directives to prevent downgrading to go1.16 language
Commit 8921897e3b introduced the uses of `clear()`,
which requires go1.21, but Go is downgrading this file to go1.16 when used in
other projects (due to us not yet being a go module);

    0.175 + xx-go build '-gcflags=' -ldflags '-X github.com/moby/buildkit/version.Version=b53a13e -X github.com/moby/buildkit/version.Revision=b53a13e4f5c8d7e82716615e0f23656893df89af -X github.com/moby/buildkit/version.Package=github.com/moby/buildkit -extldflags '"'"'-static'"'" -tags 'osusergo netgo static_build seccomp ' -o /usr/bin/buildkitd ./cmd/buildkitd
    181.8 # github.com/docker/docker/libnetwork/internal/resolvconf
    181.8 vendor/github.com/docker/docker/libnetwork/internal/resolvconf/resolvconf.go:509:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-18 12:28:21 +01:00
Paweł Gronowski
1c03312378 integration: Remove Parallel from TestDiskUsage
Check if removing the Parallel execution from that test fixes its
flakiness.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:11:34 +01:00
Paweł Gronowski
f512dba037 c8d/list: Fix premature Images return
52a80b40e2 extracted the `imageSummary`
function but introduced a bug causing the whole caller function to
return if the image should be skipped.

`imageSummary` returns a nil error and nil image when the image doesn't
have any platform or all its platforms are not available locally.
In this case that particular image should be skipped, instead of failing
the whole image list operation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:43:12 +01:00
Paweł Gronowski
89dc2860ba c8d/list: Handle missing configs in label filter
Don't error out the filter if an image config is missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:29:16 +01:00
Paweł Gronowski
6f3892dc99 c8d/list: Don't setup label filter if it's not specified
Don't run filter function which would only run through the images
reading theirs config without checking any label anyway.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:23:31 +01:00
Paweł Gronowski
a9bca45e92 Merge pull request #47575 from thaJeztah/bump_shfmt
Dockerfile: update mvdan/shfmt to v3.8.0
2024-03-18 09:26:35 +01:00
Sebastiaan van Stijn
fe8fb9b9a1 Dockerfile: update mvdan/shfmt to v3.8.0
- full diff: https://github.com/mvdan/sh/compare/v3.7.0...v3.8.0
- 3.7.0 release notes: https://github.com/mvdan/sh/releases/tag/v3.7.0
- 3.8.0 release notes: https://github.com/mvdan/sh/releases/tag/v3.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-17 13:36:43 +01:00
Sebastiaan van Stijn
70e46f2c7c Merge pull request #47559 from AkihiroSuda/fix-47436
rootless: fix `open /etc/docker/plugins: permission denied`
2024-03-16 15:54:09 +01:00
Sebastiaan van Stijn
23339a6147 Merge pull request #47570 from thaJeztah/bump_xx_1.4
Dockerfile: update xx to v1.4.0
2024-03-16 15:53:49 +01:00
Sebastiaan van Stijn
4bd30829d1 Dockerfile: update docker compose to v2.25.0
Update the version of compose that's used in the dev-container.

- full diff: https://github.com/docker/compose/compare/v2.24.7...v2.25.0
- release notes: https://github.com/docker/compose/releases/tag/v2.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-16 14:22:42 +01:00
Sebastiaan van Stijn
971562b005 Dockerfile: update docker CLI to v26.0.0-rc2
Update the CLI that's used in the dev-container to the latest rc

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc1...v26.0.0-rc2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-16 14:22:30 +01:00
Akihiro Suda
d742659877 rootless: fix open /etc/docker/plugins: permission denied
Fix issue 47436

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-16 22:03:34 +09:00
Sebastiaan van Stijn
4f46c44725 Dockerfile: update xx to v1.4.0
full diff: https://github.com/tonistiigi/xx/compare/v1.2.1...v1.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 19:59:48 +01:00
Bjorn Neergaard
1f539a6e85 Merge pull request #47569 from thaJeztah/make_fix_empty_check
Makefile: generate-files: fix check for empty TMP_OUT
2024-03-15 12:07:00 -06:00
Bjorn Neergaard
959c2ee6cf Merge pull request #47558 from AkihiroSuda/fix-47248
plugin: fix mounting /etc/hosts when running in UserNS
2024-03-15 12:06:48 -06:00
Sebastiaan van Stijn
25c9e6e8df Makefile: generate-files: fix check for empty TMP_OUT
commit c655b7dc78 added a check to make sure
the TMP_OUT variable was not set to an empty value, as such a situation would
perform an `rm -rf /**` during cleanup.

However, it was a bit too eager, because Makefile conditionals (`ifeq`) are
evaluated when parsing the Makefile, which happens _before_ the make target
is executed.

As a result `$@_TMP_OUT` was always empty when the `ifeq` was evaluated,
making it not possible to execute the `generate-files` target.

This patch changes the check to use a shell command to evaluate if the var
is set to an empty value.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 17:54:54 +01:00
Akihiro Suda
762ec4b60c plugin: fix mounting /etc/hosts when running in UserNS
Fix `error mounting "/etc/hosts" to rootfs at "/etc/hosts": mount
/etc/hosts:/etc/hosts (via /proc/self/fd/6), flags: 0x5021: operation
not permitted`.

This error was introduced in 7d08d84b03
(`dockerd-rootless.sh: set rootlesskit --state-dir=DIR`) that changed
the filesystem of the state dir from /tmp to /run (in a typical setup).

Fix issue 47248

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-15 22:16:34 +09:00
Sebastiaan van Stijn
979f03f9f6 Merge pull request #47567 from thaJeztah/move_rootless_mountopts
daemon: move getUnprivilegedMountFlags to internal package
2024-03-15 14:13:23 +01:00
Sebastiaan van Stijn
7b414f5703 daemon: move getUnprivilegedMountFlags to internal package
This code is currently only used in the daemon, but is also needed in other
places. We should consider moving this code to github.com/moby/sys, so that
BuildKit can also use the same implementation instead of maintaining a fork;
moving it to internal allows us to reuse this code inside the repository, but
does not allow external consumers to depend on it (which we don't want as
it's not a permanent location).

As our code only uses this in linux files, I did not add a stub for other
platforms (but we may decide to do that in the moby/sys repository).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 12:55:09 +01:00
Sebastiaan van Stijn
ff05850e7e Merge pull request #47563 from vvoland/buildkit-runc-override
builder-next: Add env-var to override runc used by buildkit
2024-03-14 20:17:01 +01:00
Sebastiaan van Stijn
cdf70c0a51 Merge pull request #47430 from vvoland/inspect-remove-container
api/image-inspect: Remove Container and ContainerConfig
2024-03-14 19:27:43 +01:00
Sebastiaan van Stijn
40c681355e Merge pull request #47562 from thaJeztah/update_protobuf
vendor: google.golang.org/protobuf v1.33.0, github.com/golang/protobuf v1.5.4
2024-03-14 19:14:00 +01:00
Albin Kerouanton
790c3039d0 libnet: Don't forward to upstream resolvers on internal nw
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.

This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-14 17:46:48 +00:00
Paweł Gronowski
10bdc7136c builder-next: Add env-var to override runc used by buildkit
Adds an experimental `DOCKER_BUILDKIT_RUNC_COMMAND` variable that allows
to specify different runc-compatible binary to be used by the buildkit's
runc executor.

This allows runtimes like sysbox be used for the containers spawned by
buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-14 17:35:26 +01:00
Paweł Gronowski
a8abb67c5e Merge pull request #47561 from thaJeztah/bump_tools
Dockerfile: update buildx to v0.13.1,  compose v2.24.7
2024-03-14 13:46:24 +01:00
Sebastiaan van Stijn
1ca89d7eae vendor: google.golang.org/protobuf v1.33.0, github.com/golang/protobuf v1.5.4
full diffs:

- https://github.com/protocolbuffers/protobuf-go/compare/v1.31.0...v1.33.0
- https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4

From the Go security announcement list;

> Version v1.33.0 of the google.golang.org/protobuf module fixes a bug in
> the google.golang.org/protobuf/encoding/protojson package which could cause
> the Unmarshal function to enter an infinite loop when handling some invalid
> inputs.
>
> This condition could only occur when unmarshaling into a message which contains
> a google.protobuf.Any value, or when the UnmarshalOptions.UnmarshalUnknown
> option is set. Unmarshal now correctly returns an error when handling these
> inputs.
>
> This is CVE-2024-24786.

In a follow-up post;

> A small correction: This vulnerability applies when the UnmarshalOptions.DiscardUnknown
> option is set (as well as when unmarshaling into any message which contains a
> google.protobuf.Any). There is no UnmarshalUnknown option.
>
> In addition, version 1.33.0 of google.golang.org/protobuf inadvertently
> introduced an incompatibility with the older github.com/golang/protobuf
> module. (https://github.com/golang/protobuf/issues/1596) Users of the older
> module should update to github.com/golang/protobuf@v1.5.4.

govulncheck results in our code:

    govulncheck ./...
    Scanning your code and 1221 packages across 204 dependent modules for known vulnerabilities...

    === Symbol Results ===

    Vulnerability #1: GO-2024-2611
        Infinite loop in JSON unmarshaling in google.golang.org/protobuf
      More info: https://pkg.go.dev/vuln/GO-2024-2611
      Module: google.golang.org/protobuf
        Found in: google.golang.org/protobuf@v1.31.0
        Fixed in: google.golang.org/protobuf@v1.33.0
        Example traces found:
          #1: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Peek
          #2: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Read
          #3: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls protojson.Unmarshal

    Your code is affected by 1 vulnerability from 1 module.
    This scan found no other vulnerabilities in packages you import or modules you
    require.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 13:12:54 +01:00
Sebastiaan van Stijn
f40bdf5f63 Dockerfile: update compose to v2.24.7
full diff: https://github.com/docker/compose/compare/v2.24.5...v2.24.7

release notes:

- https://github.com/docker/compose/releases/tag/v2.24.6
- https://github.com/docker/compose/releases/tag/v2.24.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 12:40:26 +01:00
Sebastiaan van Stijn
3f73d23ea0 Dockerfile: update buildx to v0.13.1
release notes:

- https://github.com/docker/buildx/releases/tag/v0.13.1
- https://github.com/docker/buildx/releases/tag/v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 12:38:06 +01:00
Paweł Gronowski
77b05b97f4 Merge pull request #47556 from vvoland/deprecate-notls
Be more explicit about non-TLS TCP access deprecation
2024-03-14 12:07:42 +01:00
Lei Jitang
e3bc82f7d4 Merge pull request #47542 from eriksjolund/47407-clarify-git-clone
set-up-git.md: clarify URL in git clone command
2024-03-14 17:24:02 +08:00
Sebastiaan van Stijn
342923b01c Merge pull request #47555 from rumpl/feat-c8d-prom
c8d: Prometheus metrics
2024-03-13 17:35:14 +01:00
Sebastiaan van Stijn
15122b3b1c Merge pull request #47350 from vvoland/cache-refactor
c8d/cache: Use the same cache logic as graphdrivers
2024-03-13 17:19:36 +01:00
Djordje Lukic
388ecf65bc c8d: Send push metrics to prom
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
bb3ab1edb7 c8d: Send pull metrics to prom
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
da245cab15 c8d: Send history metrics to prometheus
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
1cfd763214 c8d: Send image delete metrics to prometheus
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
0ce714a085 images: Export the image actions prometheus counter
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:36 +01:00
Paweł Gronowski
bcb4794eea Be more explicit about non-TLS TCP access deprecation
Turn warnings into a deprecation notice and highlight that it will
prevent daemon startup in future releases.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 14:22:10 +01:00
Paweł Gronowski
0d5ef431a1 docker-py: Temporarily skip test_commit and test_commit_with_changes
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 13:03:48 +01:00
Paweł Gronowski
03cddc62f4 api/image-inspect: Remove Container and ContainerConfig
Don't include these fields starting from API v1.45.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 13:02:29 +01:00
Akihiro Suda
825635a5bf Merge pull request #47552 from thaJeztah/vendor_containerd_1.7.14
vendor: github.com/containerd/containerd v1.7.14
2024-03-13 11:57:52 +09:00
Sebastiaan van Stijn
ec19fd6fed vendor: github.com/containerd/containerd v1.7.14
- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.14
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.14

Welcome to the v1.7.14 release of containerd!

The fourteenth patch release for containerd 1.7 contains various fixes and updates.

Highlights

- Update builds to use go 1.21.8
- Fix various timing issues with docker pusher
- Register imagePullThroughput and count with MiB
- Move high volume event logs to Trace level

Container Runtime Interface (CRI)

- Handle pod transition states gracefully while listing pod stats

Runtime

- Update runc-shim to process exec exits before init

Dependency Changes

- github.com/containerd/nri v0.4.0 -> v0.6.0
- github.com/containerd/ttrpc v1.2.2 -> v1.2.3
- google.golang.org/genproto/googleapis/rpc 782d3b101e98 -> cbb8c96f2d6d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-12 12:46:19 +01:00
Sebastiaan van Stijn
d19f6d4b6d vendor: github.com/containerd/ttrpc v1.2.3
full diff: https://github.com/containerd/ttrpc/compare/v1.2.2..v1.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-12 12:37:44 +01:00
Sebastiaan van Stijn
b8165a9cd1 Merge pull request #47494 from vvoland/devcontainer-golang
devcontainer: Add Golang extension
2024-03-11 17:50:13 +01:00
Erik Sjölund
a6a445d86b set-up-git.md: clarify URL in git clone command
Fixes https://github.com/moby/moby/issues/47407

Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
2024-03-09 16:42:44 +01:00
Sebastiaan van Stijn
0fb845858d Merge pull request #47505 from akerouanton/fix-TestBridgeICC-ipv6
inte/networking:  ping with -6 specified when needed
2024-03-08 18:33:46 +01:00
Paweł Gronowski
db2263749b Merge pull request #47530 from vvoland/flaky-liverestore
volume: Don't decrement refcount below 0
2024-03-08 12:28:10 +01:00
Sebastiaan van Stijn
1abf17c779 Merge pull request #47512 from robmry/46329_internal_resolver_ipv6_upstream
Add IPv6 nameserver to the internal DNS's upstreams.
2024-03-07 21:21:12 +01:00
Paweł Gronowski
294fc9762e volume: Don't decrement refcount below 0
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.

This makes sure we don't underflow the refcount (uint64) when
decrementing it.

The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 20:42:20 +01:00
Paweł Gronowski
eef352b565 devcontainer: Use a separate devcontainer target
Use a separate `devcontainer` Dockerfile target, this allows to include
the `gopls` in the devcontainer so it doesn't have to be installed by
the Go vscode extension.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 20:39:56 +01:00
Paweł Gronowski
f4c696eef1 Merge pull request #47449 from vvoland/c8d-list-single
c8d/list: Add test and combine size
2024-03-07 18:49:19 +01:00
Albin Kerouanton
5a009cdd5b inte/networking: add isIPv6 flag
Make sure the `ping` command used by `TestBridgeICC` actually has
the `-6` flag when it runs IPv6 test cases. Without this flag,
IPv6 connectivity isn't tested properly.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-07 17:55:53 +01:00
Paweł Gronowski
2f1a32e3e5 c8d/list: Skip images with non matching platform
Currently this won't have any real effect because the platform matcher
matches all platform and is only used for sorting.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:12 +01:00
Paweł Gronowski
72f1f82f28 c8d/list: Remove outdated TODO
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:10 +01:00
Paweł Gronowski
52a80b40e2 c8d/list: Extract imageSummary function
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:09 +01:00
Paweł Gronowski
288a14e264 c8d/list: Simplify "best" image selection
Don't save all present images,  inline the sorting into the loop
instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:07 +01:00
Paweł Gronowski
b37ced2551 c8d/list: Count containers by their manifest
Move containers counting out of `singlePlatformImage` and count them
based on the `ImageManifest` property.

(also remove ChainIDs calculation as they're no longer used)

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:26:53 +01:00
Paweł Gronowski
a535a65c4b c8d/list: Combine size
Multi-platform images are coalesced into one entry now.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:32 +01:00
Paweł Gronowski
582de4bc3c c8d/list: Add TestImageList
Add unit test for `Images` implementation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:31 +01:00
Paweł Gronowski
a6e7e67d3a specialimage: Return optional ocispec.Index
To ease accessing image descriptors in tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:30 +01:00
Paweł Gronowski
1b108bdfeb daemon/c8d: Cache SnapshotService
Avoid fetching `SnapshotService` from client every time. Fetch it once
and then store when creating the image service.

This also allows to pass custom snapshotter implementation for unit
testing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:29 +01:00
Paweł Gronowski
74e2f23e1a daemon/c8d: Use i.images and i.content
Use `image.Store` and `content.Store` stored in the ImageService struct
instead of fetching it every time from containerd client.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:27 +01:00
Paweł Gronowski
e8496b1ee4 imageService: Extract common code from MakeImageCache
Both containerd and graphdriver image service use the same code to
create the cache - they only supply their own `cacheAdaptor` struct.

Extract the shared code to `cache.New`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 15:39:42 +01:00
Paweł Gronowski
d66177591e c8d/cache: Use the same cache logic as graphdrivers
Implement the cache adaptor for containerd image store and use the same
cache logic.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:06 +01:00
Paweł Gronowski
bf30fee58a image/cache: Refactor backend specific code
Move image store backend specific code out of the cache code and move it
to a separate interface to allow using the same cache code with
containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:04 +01:00
Paweł Gronowski
608d77d740 Merge pull request #47497 from robmry/resolvconf_fixes
Fix 'resolv.conf' parsing issues
2024-03-07 13:05:10 +01:00
Paweł Gronowski
66adfc729a Merge pull request #47521 from robmry/no_ipv6_addr_when_ipv6_disabled
Don't configure IPv6 addr/gw when IPv6 disabled.
2024-03-07 12:57:27 +01:00
Sebastiaan van Stijn
ab4b5a4890 Merge pull request #47519 from thaJeztah/dupword
golangci-lint: enable dupword linter
2024-03-07 12:41:54 +01:00
Sebastiaan van Stijn
f5a5e3f203 golangci-lint: enable dupword linter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 11:44:27 +01:00
Sebastiaan van Stijn
773f792b88 Merge pull request #47523 from tonistiigi/snapshot-lock-fix
builder-next: fix missing lock in ensurelayer
2024-03-07 11:17:25 +01:00
Sebastiaan van Stijn
4adc40ac40 fix duplicate words (dupwords)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 10:57:03 +01:00
Rob Murray
8921897e3b Ignore bad ndots in host resolv.conf
Rather than error out if the host's resolv.conf has a bad ndots option,
just ignore it. Still validate ndots supplied via '--dns-option' and
treat failure as an error.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-07 09:27:34 +00:00
Tonis Tiigi
37545cc644 builder-next: fix missing lock in ensurelayer
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.

In that case the reference count got mixed up and on
release only one of these layers was actually released.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-03-06 23:11:32 -08:00
Bjorn Neergaard
6c10086976 Merge pull request #47520 from vvoland/buildkit-v13-leaseutil
builder-next/export: Use leaseutil for descref lease
2024-03-06 11:55:02 -07:00
Rob Murray
ef5295cda4 Don't configure IPv6 addr/gw when IPv6 disabled.
When IPv6 is disabled in a container by, for example, using the --sysctl
option - an IPv6 address/gateway is still allocated. Don't attempt to
apply that config because doing so enables IPv6 on the interface.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-06 18:32:31 +00:00
Paweł Gronowski
49b77753cb builder-next/export: Use leaseutil for descref lease
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 17:47:37 +01:00
Paweł Gronowski
d91665a2ac Merge pull request #47511 from vvoland/buildkit-v13.0
vendor: github.com/moby/buildkit v0.13.0
2024-03-06 17:41:55 +01:00
Sebastiaan van Stijn
4e53936f0a Merge pull request #47509 from thirdkeyword/master
remove repetitive words
2024-03-06 13:52:16 +01:00
Sebastiaan van Stijn
cb8c8e9631 Merge pull request #47498 from Dzejrou/lower-perm-fix
daemon: overlay2: remove world writable permission from the lower file
2024-03-06 13:09:30 +01:00
Paweł Gronowski
c4fc6c3371 builder-next/executor: Replace removed network.Sample
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 12:02:12 +01:00
Paweł Gronowski
0f30791a0d vendor: github.com/moby/buildkit v0.13.0
full diff: https://github.com/moby/buildkit/compare/v0.13.0-rc3...v0.13.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 11:54:43 +01:00
Rob Murray
4e8d9a4522 Add IPv6 nameserver to the internal DNS's upstreams.
When configuring the internal DNS resolver - rather than keep IPv6
nameservers read from the host's resolv.conf in the container's
resolv.conf, treat them like IPv4 addresses and use them as upstream
resolvers.

For IPv6 nameservers, if there's a zone identifier in the address or
the container itself doesn't have IPv6 support, mark the upstream
addresses for use in the host's network namespace.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-06 10:47:18 +00:00
Albin Kerouanton
7c7e453255 Merge pull request #47474 from robmry/47441_mac_addr_config_migration
Don't create endpoint config for MAC addr config migration
2024-03-06 11:04:17 +01:00
thirdkeyword
06628e383a remove repetitive words
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-06 18:03:51 +08:00
Sebastiaan van Stijn
4046928978 Merge pull request #47504 from AkihiroSuda/rootlesskit-2.0.2
update RootlessKit to 2.0.2
2024-03-06 10:12:32 +01:00
Albin Kerouanton
21835a5696 inte/networking: rename linkLocal flag into isLinkLocal
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-06 00:16:08 +01:00
Akihiro Suda
b32cfc3b3a dockerd-rootless-setuptool.sh: check RootlessKit functionality
RootlessKit will print hints if something is still unsatisfied.

e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint
rootless-containers/rootlesskit@33c3e7ca6c

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:43:00 +09:00
Akihiro Suda
49fd8df9b9 Dockerfile: update RootlessKit to v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:38:55 +09:00
Akihiro Suda
72ec187dfe go.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:38:00 +09:00
Akihiro Suda
83cda67f73 go.mod: golang.org/x/sys v0.18.0
https://github.com/golang/sys/compare/v0.16.0...v0.18.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-06 07:37:37 +09:00
Paweł Gronowski
460b4aebdf Merge pull request #47502 from vvoland/go-1.21.8
update to go1.21.8
2024-03-05 21:58:11 +01:00
Paweł Gronowski
57b7ffa7f6 update to go1.21.8
go1.21.8 (released 2024-03-05) includes 5 security fixes

- crypto/x509: Verify panics on certificates with an unknown public key algorithm (CVE-2024-24783, https://go.dev/issue/65390)
- net/http: memory exhaustion in Request.ParseMultipartForm (CVE-2023-45290, https://go.dev/issue/65383)
- net/http, net/http/cookiejar: incorrect forwarding of sensitive headers and cookies on HTTP redirect (CVE-2023-45289, https://go.dev/issue/65065)
- html/template: errors returned from MarshalJSON methods may break template escaping (CVE-2024-24785, https://go.dev/issue/65697)
- net/mail: comments in display names are incorrectly handled (CVE-2024-24784, https://go.dev/issue/65083)

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.1

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.7...go1.21.8

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-05 19:17:18 +01:00
Jaroslav Jindrak
cadb124ab6 daemon: overlay2: remove world writable permission from the lower file
In de2447c, the creation of the 'lower' file was changed from using
os.Create to using ioutils.AtomicWriteFile, which ignores the system's
umask. This means that even though the requested permission in the
source code was always 0666, it was 0644 on systems with default
umask of 0022 prior to de2447c, so the move to AtomicFile potentially
increased the file's permissions.

This is not a security issue because the parent directory does not
allow writes into the file, but it can confuse security scanners on
Linux-based systems into giving false positives.

Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
2024-03-05 14:25:50 +01:00
Sebastiaan van Stijn
046827c657 Merge pull request #47485 from vvoland/vendor-dns
vendor: github.com/miekg/dns v1.1.57
2024-03-04 11:55:01 +01:00
Sebastiaan van Stijn
04c9d7f6a3 Merge pull request #47465 from vvoland/v26-remove-deprecated
api/search: Reset `is_automated` to false
2024-03-04 11:27:24 +01:00
Paweł Gronowski
b2921509e5 api/search: Reset is_automated field to false
The field will still be present in the response, but will always be
`false`.
Searching for `is-automated=true` will yield no results, while
`is-automated=false` will effectively be a no-op.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-04 10:15:59 +01:00
Rob Murray
f04f69e366 Accumulate resolv.conf options
If there are multiple "options" lines, keep the options from all of
them.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-01 16:59:28 +00:00
Rob Murray
7f69142aa0 resolv.conf comments have '#' or ';' in the first column
When a '#' or ';' appears anywhere else, it's not a comment marker.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-01 16:58:04 +00:00
Sebastiaan van Stijn
97a5435d33 Merge pull request #47477 from robmry/resolvconf_gocompat
Remove slices.Clone() calls to avoid Go bug
2024-03-01 17:28:01 +01:00
Rob Murray
91d9307738 Replace uses of slices.Clone()
Avoid https://github.com/golang/go/issues/64759

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-01 15:27:29 +00:00
Paweł Gronowski
12dea3fa9e devcontainer: Add Golang extension automatically
When using devcontainers in VSCode, install the Go extension
automatically in the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-01 14:54:35 +01:00
Sebastiaan van Stijn
137a9d6a4c Merge pull request #47395 from robmry/47370_windows_natnw_dns_test
Test DNS on Windows 'nat' networks
2024-03-01 13:02:52 +01:00
Paweł Gronowski
9f4e824a6e vendor: github.com/miekg/dns v1.1.57
full diff: https://github.com/github.com/miekg/dns/compare/v1.1.43...v1.1.57

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-01 10:14:22 +01:00
Albin Kerouanton
f8e6801533 Merge pull request #47478 from fopina/patch-1
fix typo in error message
2024-03-01 08:53:43 +01:00
Filipe Pina
ef681124ca fix typo in error message
Signed-off-by: Filipe Pina <hzlu1ot0@duck.com>
2024-02-29 23:27:00 +00:00
Rob Murray
a580544d82 Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-29 17:02:19 +00:00
Sebastiaan van Stijn
b8aa8579ca Merge pull request #47352 from serhii-nakon/allow_host_loopback
Allow to enable host loopback and use 10.0.2.2 to connect to the host (OPTIONALLY)
2024-02-29 17:58:28 +01:00
Paweł Gronowski
225ccc0cfd Merge pull request #47473 from vvoland/cli-v26
Dockerfile: Update dev cli to v26.0.0-rc1
2024-02-29 16:02:16 +01:00
Paweł Gronowski
d19d98b136 Merge pull request #47475 from thaJeztah/nothing_to_see_here_move_along_move_along
distribution/xfer: fix pull progress message
2024-02-29 14:46:41 +01:00
Sebastiaan van Stijn
ebf3f8c7fe distribution/xfer: fix pull progress message
This message accidentally changed in ac2a028dcc
because my IDE's "refactor tool" was a bit over-enthusiastic. It also went and
updated the tests accordingly, so CI didn't catch this :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-29 14:02:55 +01:00
Sebastiaan van Stijn
a242208be3 Merge pull request #47457 from vvoland/ci-report-timeout
ci: Update `teststat` to v0.1.25
2024-02-29 13:39:09 +01:00
Paweł Gronowski
2af2496c8c Dockerfile: Update dev cli to v26.0.0-rc1
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 12:45:17 +01:00
Paweł Gronowski
fc0e5401f2 ci: Update teststat to v0.1.25
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 10:06:07 +01:00
Bjorn Neergaard
8517c3386c Merge pull request #47458 from vvoland/ci-reports-better-find
ci: Make `find` for test reports more specific
2024-02-29 01:47:07 -07:00
Sebastiaan van Stijn
6c3b3523c9 Merge pull request #47041 from robmry/46968_refactor_resolvconf
Refactor 'resolv.conf' generation.
2024-02-29 09:33:55 +01:00
Bjorn Neergaard
d40b140c08 Merge pull request #47440 from thaJeztah/fix_ping_connection_errs
client: fix connection-errors being shadowed by API version errors
2024-02-28 13:33:49 -07:00
Sebastiaan van Stijn
81428bf11b Merge pull request #47459 from thaJeztah/disable_schema1
disable pulling legacy image formats by default
2024-02-28 17:12:31 +01:00
Sebastiaan van Stijn
230cb53d3b Merge pull request #47462 from vvoland/integration-testdaemonproxy-reset-otel
integration: Reset `OTEL_EXPORTER_OTLP_ENDPOINT` for sub-daemons
2024-02-28 17:11:54 +01:00
Sebastiaan van Stijn
3ca1d751e5 Merge pull request #47461 from vvoland/vendor-buildkit-0.13.0-rc3
vendor: github.com/moby/buildkit v0.13.0-rc3
2024-02-28 14:12:43 +01:00
Sebastiaan van Stijn
589dc5e647 Merge pull request #47456 from huang-jl/fix_restore_digest
libcontainerd: change the digest used when restoring
2024-02-28 14:05:40 +01:00
Sebastiaan van Stijn
62b33a2604 disable pulling legacy image formats by default
This patch disables pulling legacy (schema1 and schema 2, version 1) images by
default.

A `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE` environment-variable is
introduced to allow re-enabling this feature, aligning with the environment
variable used in containerd 2.0 (`CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`).

With this patch, attempts to pull a legacy image produces an error:

With graphdrivers:

    docker pull docker:1.0
    1.0: Pulling from library/docker
    [DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

With the containerd image store enabled, output is slightly different
as it returns the error before printing the `1.0: pulling ...`:

    docker pull docker:1.0
    Error response from daemon: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

Using the "distribution" endpoint to resolve the digest for an image also
produces an error:

    curl -v --unix-socket /var/run/docker.sock http://foo/distribution/docker.io/library/docker:1.0/json
    *   Trying /var/run/docker.sock:0...
    * Connected to foo (/var/run/docker.sock) port 80 (#0)
    > GET /distribution/docker.io/library/docker:1.0/json HTTP/1.1
    > Host: foo
    > User-Agent: curl/7.88.1
    > Accept: */*
    >
    < HTTP/1.1 400 Bad Request
    < Api-Version: 1.45
    < Content-Type: application/json
    < Docker-Experimental: false
    < Ostype: linux
    < Server: Docker/dev (linux)
    < Date: Tue, 27 Feb 2024 16:09:42 GMT
    < Content-Length: 354
    <
    {"message":"[DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"}
    * Connection #0 to host foo left intact

Starting the daemon with the `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`
env-var set to a non-empty value allows pulling the image;

    docker pull docker:1.0
    [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
    b0a0e6710d13: Already exists
    d193ad713811: Already exists
    ba7268c3149b: Already exists
    c862d82a67a2: Already exists
    Digest: sha256:5e7081837926c7a40e58881bbebc52044a95a62a2ea52fb240db3fc539212fe5
    Status: Image is up to date for docker:1.0
    docker.io/library/docker:1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-28 13:57:43 +01:00
Paweł Gronowski
5fe96e234d integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:07 +01:00
Paweł Gronowski
84eecc4a30 Revert "integration/TestDaemonProxy: Remove OTEL span"
This reverts commit 56aeb548b2.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:03 +01:00
Paweł Gronowski
261dccc98a builder-next: Add Info to emptyProvider
To satisfy the `content.InfoReaderProvider` interface.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:20:55 +01:00
Paweł Gronowski
2c9c5e1c03 vendor: github.com/moby/buildkit v0.13.0-rc3
full diff: https://github.com/moby/buildkit/compare/v0.13.0-rc2...v0.13.0-rc3

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:05:02 +01:00
serhii.n
b649e272bb Allow to enable host loopback and use 10.0.2.2 to connect to the host (OPTIONALLY)
This should allow to enable host loopback by setting
DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK to false,
defaults true.

Signed-off-by: serhii.n <serhii.n@thescimus.com>
2024-02-28 00:52:35 +02:00
Paweł Gronowski
e4de4dea5c ci: Make find for test reports more specific
Don't use all `*.json` files blindly, take only these that are likely to
be reports from go test.
Also, use `find ... -exec` instead of piping results to `xargs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 23:38:03 +01:00
Sebastiaan van Stijn
b37f8c8070 Merge pull request #47460 from thaJeztah/bump_bolt
vendor: go.etcd.io/bbolt v1.3.9
2024-02-27 20:01:52 +01:00
Sebastiaan van Stijn
9be820d8ca vendor: go.etcd.io/bbolt v1.3.9
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.7...v1.3.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-27 18:24:01 +01:00
Sebastiaan van Stijn
f6fa6ff9ed Merge pull request #47391 from vvoland/rro-backwards-compatible
api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-02-27 18:04:46 +01:00
Sebastiaan van Stijn
220835106b Merge pull request #47364 from vvoland/buildkit-v13
vendor: github.com/moby/buildkit v0.13.0-rc2
2024-02-27 16:38:04 +01:00
Paweł Gronowski
2c25ca9dba Merge pull request #47455 from vvoland/c8d-skip-last-windows-tests
c8d/windows: Temporarily skip two failing tests
2024-02-27 14:01:31 +01:00
Paweł Gronowski
94f9f39b24 Merge pull request #47454 from vvoland/c8d-pull-pullingfslayer-truncated
c8d/pull: Output truncated id for `Pulling fs layer`
2024-02-27 13:28:38 +01:00
huang-jl
da643c0b8a libcontainerd: change the digest used when restoring
For current implementation of Checkpoint Restore (C/R) in docker, it
will write the checkpoint to content store. However, when restoring
libcontainerd uses .Digest().Encoded(), which will remove the info
of alg, leading to error.

Signed-off-by: huang-jl <1046678590@qq.com>
2024-02-27 20:17:31 +08:00
Rob Murray
9083c2f10d Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-27 11:40:11 +00:00
Paweł Gronowski
44167988c3 c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 12:04:21 +01:00
Paweł Gronowski
2d31532a00 otel: Default metrics protocol to http/protobuf
Buildkit added support for exporting metrics in:
7de2e4fb32

Explicitly set the protocol for exporting metrics like we do for the
traces. We need that because Buildkit defaults to grpc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:12 +01:00
CrazyMax
60358bfcab ci(buildkit): dedicated step to build test image
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:11 +01:00
Paweł Gronowski
f5722da5e0 mobyexporter: Store temporary config descriptor
Temporarily store the produced config descriptor for the buildkit
history to work.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:09 +01:00
Paweł Gronowski
951e42cd60 builder-next: Replace ResolveImageConfig with ResolveSourceMetadata
30c069cb03
removed the `ResolveImageConfig` method in favor of more generic
`ResolveSourceMetadata` that can also support other things than images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:37 +01:00
Paweł Gronowski
e01a1c5d09 builder/mobyexporter: Set image.name response key
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:35 +01:00
Paweł Gronowski
fa467caf4d builder-next/mobyexporter: Use OptKeyName const
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:33 +01:00
Paweł Gronowski
59ad1690f7 builder-next: Adjust to source changes
Adjust to cache sources changes from:
6b27487fec

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:17 +01:00
Paweł Gronowski
b04a2dad6b builder/controller: Adjust NewWorkerOpt call
8bfd280ab7
added a new argument that allows to specify different runtime.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:15 +01:00
Paweł Gronowski
bc6d88c09a cmd/dockerd: Fix overriding OTEL resource
e358792815
changed that field to a function and added an `OverrideResource`
function that allows to override it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:14 +01:00
Paweł Gronowski
a79bb1e832 builder-next/exporter: Sync with new signature
1c1777b7c0
added an explicit id argument to the Resolve method.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:12 +01:00
Paweł Gronowski
e68f71259a integration/build: Use fsutil.NewFS
StaticDirSource definition changed and can no longer be initialized from
the composite literal.

a80b48544c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:10 +01:00
Paweł Gronowski
dd6992617e integration/build: Use new buildkit progressui
Introduced in: 37131781d7

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:09 +01:00
Paweł Gronowski
31545c3b67 vendor: github.com/moby/buildkit v0.13.0-rc2
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:07 +01:00
CrazyMax
f90b03ee5d go.mod: bump to go 1.21 and use local toolchain when vendoring
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:25:20 +01:00
Paweł Gronowski
16aa7dd67f c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:09:16 +01:00
Sebastiaan van Stijn
8cdb5a9070 Merge pull request #47450 from neersighted/image_created_omitempty
api: omit missing Created field from ImageInspect response
2024-02-26 20:06:52 +01:00
Sebastiaan van Stijn
ffd294ebcc Merge pull request #45967 from tianon/c8d-image-list
c8d: Adjust "image list" to return only a single item for each image store entry
2024-02-26 20:05:29 +01:00
Bjorn Neergaard
881260148f api: omit missing Created field from ImageInspect response
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-26 10:26:15 -07:00
Paweł Gronowski
432390320e api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 11:37:30 +01:00
Sebastiaan van Stijn
c70d7905fb Merge pull request #47432 from vvoland/c8d-pull-fslayer
c8d/pull: Progress fixes
2024-02-26 10:38:00 +01:00
Sebastiaan van Stijn
0eecd59153 Merge pull request #47245 from thaJeztah/bump_otel
vendor: OTEL v0.46.1 / v1.21.0
2024-02-23 17:47:27 +01:00
Sebastiaan van Stijn
6aea26b431 client: fix connection-errors being shadowed by API version mismatch errors
Commit e6907243af applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

    docker -H unix:///no/such/socket.sock secret ls
    "secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

    docker -H unix:///no/such/socket.sock secret ls
    Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 15:17:10 +01:00
Sebastiaan van Stijn
913478b428 client: doRequest: make sure we return a connection-error
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 15:13:22 +01:00
Sebastiaan van Stijn
901b90593d client: NegotiateAPIVersion: do not ignore (connection) errors from Ping
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 14:31:55 +01:00
Sebastiaan van Stijn
349abc64ed client: fix TestPingWithError
This test was added in 27ef09a46f, which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.

The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.

Running the test also shows that returning a response is unexpected, and
ignored:

    === RUN   TestPingWithError
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    --- PASS: TestPingWithError (0.00s)
    PASS

This patch updates the test to remove the response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 14:25:52 +01:00
Sebastiaan van Stijn
24fe934a7b Merge pull request #47423 from vvoland/ci-check-changelog
ci: Require changelog description
2024-02-23 12:24:13 +01:00
Paweł Gronowski
05b883bdc8 mounts/validate: Don't check source exists with CreateMountpoint
Don't error out when mount source doesn't exist and mounts has
`CreateMountpoint` option enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-23 11:20:55 +01:00
Sebastiaan van Stijn
c516804d6f vendor: OTEL v0.46.1 / v1.21.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 10:11:07 +01:00
Sebastiaan van Stijn
445d500aeb Merge pull request #47410 from vvoland/test-daemonproxy-no-otel
integration/TestDaemonProxy: Remove OTEL span
2024-02-22 22:19:53 +01:00
Sebastiaan van Stijn
1ffaf469ba Merge pull request #47175 from corhere/best-effort-xattrs-classic-builder
builder/dockerfile: ADD with best-effort xattrs
2024-02-22 20:14:22 +01:00
Albin Kerouanton
842d1b3c12 Merge pull request #47433 from akerouanton/libnet-ds-extra-space-in-err
libnet/ds: remove extra space in error msg
2024-02-22 19:38:26 +01:00
Albin Kerouanton
83c02f7a11 libnet/ds: remove extra space in error msg
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-22 18:49:28 +01:00
Paweł Gronowski
14df52b709 c8d/pull: Don't emit Downloading with 0 progress
To align with the graphdrivers behavior and don't send unnecessary
progress messages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:16 +01:00
Paweł Gronowski
ff5f780f2b c8d/pull: Emit Pulling fs layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:15 +01:00
Paweł Gronowski
5689dabfb3 pkg/streamformatter: Make progressOutput concurrency safe
Sync access to the underlying `io.Writer` with a mutex.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:13 +01:00
Sebastiaan van Stijn
7d081179e9 Merge pull request #47422 from akerouanton/libnet-ds-DeleteIdempotent
libnet: Replace DeleteAtomic in retry loops with Delete
2024-02-22 17:24:05 +01:00
Paweł Gronowski
3865c63d45 Merge pull request #47426 from vvoland/vendor-continuity
vendor: github.com/containerd/continuity v0.4.3
2024-02-22 14:28:41 +01:00
Paweł Gronowski
1d473549e8 ci: Require changelog description
Any PR that is labeled with any `impact/*` label should have a
description for the changelog and an `area/*` label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 13:40:23 +01:00
Paweł Gronowski
b2aaf5c2b0 vendor: github.com/containerd/continuity v0.4.3
full diff: https://github.com/containerd/continuity/compare/v0.4.3...v0.4.2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 10:24:52 +01:00
Albin Kerouanton
cbd45e83cf libnet: Replace DeleteAtomic in retry loops with DeleteIdempotent
A common pattern in libnetwork is to delete an object using
`DeleteAtomic`, ie. to check the optimistic lock, but put in a retry
loop to refresh the data and the version index used by the optimistic
lock.

This commit introduces a new `Delete` method to delete without
checking the optimistic lock. It focuses only on the few places where
it's obvious the calling code doesn't rely on the side-effects of the
retry loop (ie. refreshing the object to be deleted).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-22 08:22:09 +01:00
Sebastiaan van Stijn
cba87125b2 Merge pull request #47405 from vvoland/validate-vendor-nopager
validate/vendor: Disable pager for git diff
2024-02-21 17:16:11 +01:00
CrazyMax
2a41ce93fe Merge pull request #47409 from crazy-max/ci-codecov-token
ci: set codecov token
2024-02-21 16:32:15 +01:00
Sebastiaan van Stijn
c42ae61e62 Merge pull request #47417 from thaJeztah/resolver_improve_logs
libnetwork: resolve: use structured logs for DNS error
2024-02-21 10:41:06 +01:00
Sebastiaan van Stijn
d9e082ff54 libnetwork: resolve: use structured logs for DNS error
I noticed that this log didn't use structured logs;

    [resolver] failed to query DNS server: 10.115.11.146:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:46361->10.115.11.146:53: i/o timeout
    [resolver] failed to query DNS server: 10.44.139.225:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:53991->10.44.139.225:53: i/o timeout

But other logs did;

    DEBU[2024-02-20T15:48:51.026704088Z] [resolver] forwarding query                   client-addr="udp:172.19.0.2:39661" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t A"
    DEBU[2024-02-20T15:48:51.028331088Z] [resolver] forwarding query                   client-addr="udp:172.19.0.2:35163" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t AAAA"
    DEBU[2024-02-20T15:48:51.057329755Z] [resolver] received AAAA record "2a00:1450:400e:801::200e" for "google.com." from udp:192.168.65.7
    DEBU[2024-02-20T15:48:51.057666880Z] [resolver] received A record "142.251.36.14" for "google.com." from udp:192.168.65.7

As we're already constructing a logger with these fields, we may as well use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-20 17:01:06 +01:00
Paweł Gronowski
8761bffcaf Makefile: Pass PAGER/GIT_PAGER variable
Allow to override the PAGER/GIT_PAGER variables inside the container.
Use `cat` as pager when running in Github Actions (to avoid things like
`git diff` stalling the CI).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 11:52:58 +01:00
Paweł Gronowski
56aeb548b2 integration/TestDaemonProxy: Remove OTEL span
Don't use OTEL tracing in this test because we're testing the HTTP proxy
behavior here and we don't want OTEL to interfere.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 10:21:53 +01:00
CrazyMax
38827ba290 ci: set codecov token
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-20 08:58:27 +01:00
Sebastiaan van Stijn
9d1541526c Merge pull request #47361 from robmry/47331_swarm_ipam_validation
Don't enforce new validation rules for existing networks
2024-02-16 16:58:33 +01:00
Sebastiaan van Stijn
7bf8d2606e Merge pull request #47356 from robmry/47329_restore_internal_bridge_addr
Make 'internal' bridge networks accessible from host
2024-02-16 13:23:38 +01:00
Sebastiaan van Stijn
bf053be997 Merge pull request #47373 from vvoland/aws-v1.24.1
vendor: bump github.com/aws/aws-sdk-go-v2 to v1.24.1
2024-02-15 12:00:47 +01:00
Sebastiaan van Stijn
101241c804 Merge pull request #47382 from robmry/run_macvlan_ipvlan_tests
Run the macvlan/ipvlan integration tests
2024-02-14 23:29:38 +01:00
Paweł Gronowski
bddd892e91 c8d: Adjust "image list" to return only a single item for each image store entry
This will return a single entry for each name/value pair, and for now
all the "image specific" metadata (labels, config, size) should be
either "default platform" or "first platform we have locally" (which
then matches the logic for commands like `docker image inspect`, etc)
with everything else (just ID, maybe?) coming from the manifest
list/index.

That leaves room for the longer-term implementation to add new fields to
describe the _other_ images that are part of the manifest list/index.

Co-authored-by: Tianon Gravi <admwiggin@gmail.com>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 18:44:37 +01:00
Paweł Gronowski
2aa13e950d awslogs: Replace depreacted WithEndpointResolver usage
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 18:38:05 +01:00
Paweł Gronowski
70a4a9c969 vendor: bump github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to v1.32.0
v1.33.0 is also available, but it would also cause
`github.com/aws/aws-sdk-go-v2` change from v1.24.1 to v1.25.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 17:23:55 +01:00
Sebastiaan van Stijn
a0f12f96eb Merge pull request #47374 from tianon/api-inspect-created
Set `Created` to `0001-01-01T00:00:00Z` on older API versions
2024-02-14 17:09:07 +01:00
Akihiro Suda
3d354593c9 Merge pull request #47385 from thaJeztah/update_go_1.21.7
update to go1.21.7
2024-02-15 00:26:10 +09:00
Rob Murray
9faf4855d5 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:09:45 +00:00
Rob Murray
4eb95d01bc Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:08:10 +00:00
Sebastiaan van Stijn
7c2975d2df update to go1.21.7
go1.21.7 (released 2024-02-06) includes fixes to the compiler, the go command,
the runtime, and the crypto/x509 package. See the Go 1.21.7 milestone on our
issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.6...go1.21.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-14 12:56:06 +01:00
Paweł Gronowski
903412d0fc api/history: Mention empty Created
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-14 11:00:17 +01:00
Sebastiaan van Stijn
b8a71a1c44 Merge pull request #47375 from robmry/47370_windows_nat_network_dns
Set up DNS names for Windows default network
2024-02-13 13:47:28 +01:00
Rob Murray
443f56efb0 Set up DNS names for Windows default network
DNS names were only set up for user-defined networks. On Linux, none
of the built-in networks (bridge/host/none) have built-in DNS, so they
don't need DNS names.

But, on Windows, the default network is "nat" and it does need the DNS
names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-12 21:11:44 +00:00
Tianon Gravi
b4fbe226e8 Set Created to 0001-01-01T00:00:00Z on older API versions
This matches the prior behavior before 2a6ff3c24f.

This also updates the Swagger documentation for the current version to note that the field might be the empty string and what that means.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2024-02-12 12:39:16 -08:00
Cory Snider
5bcd2f6860 builder/dockerfile: ADD with best-effort xattrs
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.

The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-12 13:31:44 -05:00
Paweł Gronowski
999f90ac1c vendor: bump github.com/aws/aws-sdk-go-v2 to v1.24.1
In preparation for buildkit v0.13

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-12 18:22:32 +01:00
Akihiro Suda
a60546b084 Merge pull request #47371 from thaJeztah/bump_nydus
vendor: github.com/containerd/nydus-snapshotter v0.13.7
2024-02-12 18:53:05 +09:00
Sebastiaan van Stijn
ef7766304c vendor: github.com/containerd/nydus-snapshotter v0.13.7
Update to the latest patch release, which contains changes from v0.13.5 to
remove the reference package from "github.com/docker/distribution", which
is now a separate module.

full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.8.2...v0.13.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:25:28 +01:00
Sebastiaan van Stijn
6932939326 vendor: google.golang.org/genproto/googleapis/rpc 49dd2c1f3d0b
manually aligned the indirect dependencies to be on the same commit

diff: b8732ec382...49dd2c1f3d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:25:26 +01:00
Sebastiaan van Stijn
10a72f2504 vendor: cloud.google.com/go/logging v1.8.1
full diff: https://github.com/googleapis/google-cloud-go/compare/logging/v1.7.0...logging/v1.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:24:51 +01:00
Sebastiaan van Stijn
a60fef0c41 vendor: golang.org/x/exp v0.0.0-20231006140011-7918f672742d
full diff: c95f2b4c22...7918f67274

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-12 09:13:32 +01:00
Bjorn Neergaard
86b86412a1 Merge pull request #47362 from thaJeztah/migrate_image_spec
migrate image spec to github.com/moby/docker-image-spec
2024-02-09 14:22:42 -07:00
Sebastiaan van Stijn
03a17a2887 migrate image spec to github.com/moby/docker-image-spec
The specification was migrated to a separate module:
https://github.com/moby/docker-image-spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-09 19:12:18 +01:00
Rob Murray
a26c953b94 Add comment explaining network-create flow for Swarm
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-09 11:56:46 +00:00
Rob Murray
571af915d5 Don't enforce new validation rules for existing networks
Non-swarm networks created before network-creation-time validation
was added in 25.0.0 continued working, because the checks are not
re-run.

But, swarm creates networks when needed (with 'agent=true'), to
ensure they exist on each agent - ignoring the NetworkNameError
that says the network already existed.

By ignoring validation errors on creation of a network with
agent=true, pre-existing swarm networks with IPAM config that would
fail the new checks will continue to work too.

New swarm (overlay) networks are still validated, because they are
initially created with 'agent=false'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-09 11:56:46 +00:00
Sebastiaan van Stijn
97478c99f8 Merge pull request #47360 from thaJeztah/image_spec_clean
image/spec: remove link to docs.docker.com "registry" specification
2024-02-08 18:50:18 +01:00
Sebastiaan van Stijn
b71c2792d2 image/spec: remove link to docs.docker.com "registry" specification
This spec is not directly relevant for the image spec, and the Docker
documentation no longer includes the actual specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-08 17:39:18 +01:00
Sebastiaan van Stijn
57e8352c9e Merge pull request #47359 from vvoland/c8d-1.7.13
vendor: github.com/containerd/containerd v1.7.13
2024-02-08 16:25:16 +01:00
Paweł Gronowski
4ab11a1148 vendor: github.com/containerd/containerd v1.7.13
No major changes, it just adds `content.InfoReaderProvider` interface.

full diff: https://github.com/containerd/containerd/compare/v1.7.12...v1.7.13

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-08 15:04:04 +01:00
Sebastiaan van Stijn
23d80f729e Merge pull request #46981 from thaJeztah/bump_prometheus
vendor: github.com/prometheus/client_golang v1.17.0
2024-02-07 23:06:06 +01:00
Rob Murray
419f5a6372 Make 'internal' bridge networks accessible from host
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).

This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-07 19:12:10 +00:00
Sebastiaan van Stijn
475019d70a vendor: github.com/prometheus/procfs v0.12.0
- https://github.com/prometheus/procfs/compare/v0.11.1...v0.12.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-07 02:40:09 +01:00
Sebastiaan van Stijn
63c354aae2 vendor: github.com/prometheus/client_golang v1.17.0
full diffs:

- https://github.com/prometheus/client_golang/compare/v1.14.0...v1.17.0
- https://github.com/prometheus/client_model/compare/v0.3.0...v0.5.0
- https://github.com/prometheus/common/compare/v0.42.0...v0.44.0
- https://github.com/prometheus/procfs/compare/v0.9.0...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-07 02:40:07 +01:00
Sebastiaan van Stijn
9e075f3808 Merge pull request #47155 from thaJeztah/remove_deprecated_api_versions
api: remove deprecated API versions (API < v1.24)
2024-02-07 01:43:04 +01:00
Rob Murray
beb97f7fdf Refactor 'resolv.conf' generation.
Replace regex matching/replacement and re-reading of generated files
with a simple parser, and struct to remember and manipulate the file
content.

Annotate the generated file with a header comment saying the file is
generated, but can be modified, and a trailing comment describing how
the file was generated and listing external nameservers.

Always start with the host's resolv.conf file, whether generating config
for host networking, or with/without an internal resolver - rather than
editing a file previously generated for a different use-case.

Resolves an issue where rewrites of the generated file resulted in
default IPv6 nameservers being unnecessarily added to the config.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-06 22:26:12 +00:00
Sebastiaan van Stijn
d2f12e6d51 Merge pull request #47336 from rumpl/history-config
c8d: Use the same logic to get the present images
2024-02-06 19:42:51 +01:00
Sebastiaan van Stijn
14503ccebd api/server/middleware: NewVersionMiddleware: add validation
Make sure the middleware cannot be initialized with out of range versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:45 +01:00
Sebastiaan van Stijn
e1897cbde4 api/server/middleware:use API-consts in tests
Use the API consts to have more realistic values in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:45 +01:00
Sebastiaan van Stijn
0fef6e1c99 api/server/middleware: VersionMiddleware: improve docs
Improve documentation and rename fields and variables to be more descriptive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:45 +01:00
Sebastiaan van Stijn
6b01719ffb api: add MinSupportedAPIVersion const
This const contains the minimum API version that can be supported by the
API server. The daemon is currently configured to use the same version,
but we may increment the _configured_ minimum version when deprecating
old API versions in future.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
19a04efa2f api: remove API < v1.24
Commit 08e4e88482 (Docker Engine v25.0.0)
deprecated API version v1.23 and lower, but older API versions could be
enabled through the DOCKER_MIN_API_VERSION environment variable.

This patch removes all support for API versions < v1.24.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
8758d08bb4 api: remove handling of HostConfig on POST /containers/{id}/start (api < v1.24)
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
ffd877f948 api: remove plain-text error-responses (api < v1.24)
Commit 322e2a7d05 changed the format of errors
returned by the API to be in JSON format for API v1.24. Older versions of
the API returned errors in plain-text format.

API v1.23 and older are deprecated, so we can remove support for plain-text
error responses.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
b3a0ff9944 api: remove POST /containers/{id}/copy endpoint (api < v1.23)
This endpoint was deprecated in API v1.20 (Docker Engine v1.8.0) in
commit db9cc91a9e, in favor of the
`PUT /containers/{id}/archive` and `HEAD /containers/{id}/archive`
endpoints, and disabled in API v1.24 (Docker Engine v1.12.0) through
commit 428328908d.

This patch removes the endpoint, and the associated `daemon.ContainerCopy`
method in the backend.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
83f790cccc api: POST /exec/{id}/start: remove support for API < v1.21
API v1.21 (Docker Engine v1.9.0) enforces the request to have a JSON
content-type on exec start (see 45dc57f229).
An exception was added in 0b5e628e14 to
make this check conditional (supporting API < 1.21).

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
d1974aa492 api: remove code for container stats on api < v1.21
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.20 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
ed93110e11 api: update test to reflect reality on Windows
The TestInspectAPIContainerResponse mentioned that Windows does not
support API versions before v1.25.

While technically, no stable release existed for Windows with API versions
before that (see f811d5b128), API version
v1.24 was enabled in e4af39aeb3, to have
a consistend fallback version for API version negotiation.

This patch updates the test to reflect that change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
570d5a9645 api: remove code for ContainerInspect on api v1.20
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.20.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
f0dd554e3c api: remove code for ContainerInspect on api < v1.20
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.19 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
dfdf2adf0c api: POST /containers/{id}/kill: remove handling for api < 1.20
API v1.20 and up produces an error when signalling / killing a non-running
container (see c92377e300). Older API versions
allowed this, and an exception was added in 621e3d8587.

API v1.23 and older are deprecated, so we can remove this handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:42 +01:00
Sebastiaan van Stijn
2970b320aa api: remove code for adjusting CPU shares (api < v1.19)
API versions before 1.19 allowed CpuShares that were greater than the maximum
or less than the minimum supported by the kernel, and relied on the kernel to
do the right thing.

Commit ed39fbeb2a introduced code to adjust the
CPU shares to be within the accepted range when using API version 1.18 or
lower.

API v1.23 and older are deprecated, so we can remove support for this
functionality.

Currently, there's no validation for CPU shares to be within an acceptable
range; a TODO was added to add validation for this option, and to use the
`linuxMinCPUShares` and `linuxMaxCPUShares` consts for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:33 +01:00
Sebastiaan van Stijn
ef25f0aa52 api: POST /build: remove version-gate for "pull" (api < v1.16)
The "pull" option was added in API v1.16 (Docker Engine v1.4.0) in commit
054e57a622, which gated the option by API
version.

API v1.23 and older are deprecated, so we can remove the gate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Sebastiaan van Stijn
7fa116830b api: POST /build: remove version-gate for "rm", "force-rm" (api < v1.16)
The "rm" option was made the default in API v1.12 (Docker Engine v1.0.0)
in commit b60d647172, and "force-rm" was
added in 667e2bd4ea.

API v1.23 and older are deprecated, so we can remove these gates.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Sebastiaan van Stijn
1b1147e46b api: POST /commit: remove version-gate for "pause" (api < v1.16)
The "pause" flag was added in API v1.13 (Docker Engine v1.1.0), and is
enabled by default (see 17d870bed5).

API v1.23 and older are deprecated, so we can remove the version-gate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:27 +01:00
Sebastiaan van Stijn
d26bdfe226 runconfig: remove fixtures for api < v1.19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:42:24 +01:00
Djordje Lukic
f1e6958295 c8d: Use the same logic to get the present images
Inspect and history used two different ways to find the present images.
This made history fail in some cases where image inspect would work (if
a configuration of a manifest wasn't found in the content store).

With this change we now use the same logic for both inspect and history.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-02-06 16:35:53 +01:00
Sebastiaan van Stijn
27ac2beca0 Merge pull request #47342 from vvoland/cache-ocispec-platforms
image/cache: Use Platform from ocispec
2024-02-06 15:45:49 +01:00
Sebastiaan van Stijn
9e10605e77 Merge pull request #47341 from thaJeztah/seccomp_updates
profiles/seccomp: add syscalls for kernel v5.17 - v6.6, match containerd's profile
2024-02-06 15:22:16 +01:00
Paweł Gronowski
2c01d53d96 image/cache: Use Platform from ocispec
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 14:26:51 +01:00
Sebastiaan van Stijn
d69729e053 seccomp: add futex_wake syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 9f6c532f59

    futex: Add sys_futex_wake()

    To complement sys_futex_waitv() add sys_futex_wake(). This syscall
    implements what was previously known as FUTEX_WAKE_BITSET except it
    uses 'unsigned long' for the bitmask and takes FUTEX2 flags.

    The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:12:40 +01:00
Sebastiaan van Stijn
10d344d176 seccomp: add futex_wait syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cb8c4312af

    futex: Add sys_futex_wait()

    To complement sys_futex_waitv()/wake(), add sys_futex_wait(). This
    syscall implements what was previously known as FUTEX_WAIT_BITSET
    except it uses 'unsigned long' for the value and bitmask arguments,
    takes timespec and clockid_t arguments for the absolute timeout and
    uses FUTEX2 flags.

    The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:12:40 +01:00
Sebastiaan van Stijn
df57a080b6 seccomp: add futex_requeue syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 0f4b5f9722

    futex: Add sys_futex_requeue()

    Finish off the 'simple' futex2 syscall group by adding
    sys_futex_requeue(). Unlike sys_futex_{wait,wake}() its arguments are
    too numerous to fit into a regular syscall. As such, use struct
    futex_waitv to pass the 'source' and 'destination' futexes to the
    syscall.

    This syscall implements what was previously known as FUTEX_CMP_REQUEUE
    and uses {val, uaddr, flags} for source and {uaddr, flags} for
    destination.

    This design explicitly allows requeueing between different types of
    futex by having a different flags word per uaddr.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:12:31 +01:00
Sebastiaan van Stijn
8826f402f9 seccomp: add map_shadow_stack syscall (kernel v6.6, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: c35559f94e

    x86/shstk: Introduce map_shadow_stack syscall

    When operating with shadow stacks enabled, the kernel will automatically
    allocate shadow stacks for new threads, however in some cases userspace
    will need additional shadow stacks. The main example of this is the
    ucontext family of functions, which require userspace allocating and
    pivoting to userspace managed stacks.

    Unlike most other user memory permissions, shadow stacks need to be
    provisioned with special data in order to be useful. They need to be setup
    with a restore token so that userspace can pivot to them via the RSTORSSP
    instruction. But, the security design of shadow stacks is that they
    should not be written to except in limited circumstances. This presents a
    problem for userspace, as to how userspace can provision this special
    data, without allowing for the shadow stack to be generally writable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 14:02:33 +01:00
Sebastiaan van Stijn
6f242f1a28 seccomp: add fchmodat2 syscall (kernel v6.6, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 09da082b07

    fs: Add fchmodat2()

    On the userspace side fchmodat(3) is implemented as a wrapper
    function which implements the POSIX-specified interface. This
    interface differs from the underlying kernel system call, which does not
    have a flags argument. Most implementations require procfs [1][2].

    There doesn't appear to be a good userspace workaround for this issue
    but the implementation in the kernel is pretty straight-forward.

    The new fchmodat2() syscall allows to pass the AT_SYMLINK_NOFOLLOW flag,
    unlike existing fchmodat.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 13:59:04 +01:00
Sebastiaan van Stijn
4d0d5ee10d seccomp: add cachestat syscall (kernel v6.5, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cf264e1329

    NAME
        cachestat - query the page cache statistics of a file.

    SYNOPSIS
        #include <sys/mman.h>

        struct cachestat_range {
            __u64 off;
            __u64 len;
        };

        struct cachestat {
            __u64 nr_cache;
            __u64 nr_dirty;
            __u64 nr_writeback;
            __u64 nr_evicted;
            __u64 nr_recently_evicted;
        };

        int cachestat(unsigned int fd, struct cachestat_range *cstat_range,
            struct cachestat *cstat, unsigned int flags);

    DESCRIPTION
        cachestat() queries the number of cached pages, number of dirty
        pages, number of pages marked for writeback, number of evicted
        pages, number of recently evicted pages, in the bytes range given by
        `off` and `len`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 13:57:00 +01:00
Sebastiaan van Stijn
1251982cf7 seccomp: add set_mempolicy_home_node syscall (kernel v5.17, libseccomp v2.5.4)
This syscall is gated by CAP_SYS_NICE, matching the profile in containerd.

containerd: a6e52c74fa
libseccomp: d83cb7ac25
kernel: c6018b4b25

    mm/mempolicy: add set_mempolicy_home_node syscall
    This syscall can be used to set a home node for the MPOL_BIND and
    MPOL_PREFERRED_MANY memory policy.  Users should use this syscall after
    setting up a memory policy for the specified range as shown below.

      mbind(p, nr_pages * page_size, MPOL_BIND, new_nodes->maskp,
            new_nodes->size + 1, 0);
      sys_set_mempolicy_home_node((unsigned long)p, nr_pages * page_size,
                    home_node, 0);

    The syscall allows specifying a home node/preferred node from which
    kernel will fulfill memory allocation requests first.
    ...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 13:53:15 +01:00
Sebastiaan van Stijn
203ffb1c09 Merge pull request #47330 from vvoland/cache-fix-older-windows
image/cache: Ignore Build and Revision on Windows
2024-02-06 13:02:00 +01:00
Sebastiaan van Stijn
cae5d323e1 Merge pull request #47332 from AkihiroSuda/rootlesskit-2.0.1
Update Rootlesskit to v2.0.1
2024-02-06 10:38:19 +01:00
Akihiro Suda
7f1b700227 Dockerfile: update RootlessKit to v2.0.1
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.1

Fix issue 47327 (`rootless lxc-user-nic: /etc/resolv.conf missing ip`)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-06 11:51:47 +09:00
Akihiro Suda
f1730a6512 go.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.1
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-06 11:51:00 +09:00
Akihiro Suda
f7192bb0b4 vendor.mod: github.com/google/uuid v1.6.0
https://github.com/google/uuid/releases/tag/v1.6.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-06 11:50:00 +09:00
Sebastiaan van Stijn
2156635843 Merge pull request #47232 from vvoland/fix-save-manifests
image/save: Fix untagged images not present in index.json
2024-02-05 19:06:54 +01:00
Paweł Gronowski
91ea04089b image/cache: Ignore Build and Revision on Windows
The compatibility depends on whether `hyperv` or `process` container
isolation is used.
This fixes cache not being used when building images based on older
Windows versions on a newer Windows host.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 17:41:21 +01:00
Paweł Gronowski
2ef0b53e51 integration/save: Add tests checking OCI archive output
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 11:17:58 +01:00
Sebastiaan van Stijn
6b83319773 Merge pull request #47299 from laurazard/plugin-install-digest
plugins: Fix panic when fetching by digest
2024-02-05 09:39:05 +01:00
Sebastiaan van Stijn
ee3c710f22 Merge pull request #47319 from coolljt0725/fix_broken_links
Fix broken links in project
2024-02-05 09:31:17 +01:00
Laura Brehm
74d51e8553 plugins: fix panic installing from repo w/ digest
Only print the tag when the received reference has a tag, if
we can't cast the received tag to a `reference.Tagged` then
skip printing the tag as it's likely a digest.

Fixes panic when trying to install a plugin from a reference
with a digest such as
`vieux/sshfs@sha256:1d3c3e42c12138da5ef7873b97f7f32cf99fb6edde75fa4f0bcf9ed277855811`

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-02-04 20:23:06 +00:00
Lei Jitang
e910a79e2b Remove 'VERSION' from the text
We removed `VERSION` file in this PR:
https://github.com/moby/moby/pull/35368

Signed-off-by: Lei Jitang <leijitang@outlook.com>
2024-02-04 11:59:03 +08:00
Lei Jitang
9fcea5b933 Fix broken links in project/README.md
Related to https://github.com/moby/moby/pull/37104,
`RELEASE-CHECKLIST.md` has been removed by this PR

Signed-off-by: Lei Jitang <leijitang@outlook.com>
2024-02-04 11:57:53 +08:00
Sebastiaan van Stijn
e61c425cc2 Merge pull request #47315 from thaJeztah/update_dev_cli_compose
Dockerfile: update docker-cli to v25.0.2, docker compose v2.24.5
2024-02-03 15:23:01 +01:00
Sebastiaan van Stijn
10d6f5213a Dockerfile: update docker compose to v2.24.5
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.24.3...v2.24.5
- release notes: https://github.com/docker/compose/releases/tag/v2.24.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-03 13:51:49 +01:00
Sebastiaan van Stijn
9c92c07acf Dockerfile: update dev-shell version of the cli to v25.0.2
Update the docker CLI that's available for debugging in the dev-shell
to the v25 release.

full diff: https://github.com/docker/cli/compare/v25.0.1...v25.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-03 13:50:18 +01:00
Sebastiaan van Stijn
0616b4190e Merge pull request #47309 from akerouanton/libnet-bridge-mtu-ignore-einval
libnet: bridge: ignore EINVAL when configuring bridge MTU
2024-02-03 11:36:19 +01:00
Albin Kerouanton
89470a7114 libnet: bridge: ignore EINVAL when configuring bridge MTU
Since 964ab7158c, we explicitly set the bridge MTU if it was specified.
Unfortunately, kernel <v4.17 have a check preventing us to manually set
the MTU to anything greater than 1500 if no links is attached to the
bridge, which is how we do things -- create the bridge, set its MTU and
later on, attach veths to it.

Relevant kernel commit: 804b854d37

As we still have to support CentOS/RHEL 7 (and their old v3.10 kernels)
for a few more months, we need to ignore EINVAL if the MTU is > 1500
(but <= 65535).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 19:32:45 +01:00
Sebastiaan van Stijn
701dd989f1 Merge pull request #47302 from akerouanton/libnet-ds-PersistConnection
libnet: boltdb: remove PersistConnection
2024-02-02 19:05:09 +01:00
Brian Goff
7c4828f8fb Merge pull request #47256 from corhere/journald/quit-when-youre-ahead
daemon/logger/journald: quit waiting when the logger closes
2024-02-02 09:10:42 -08:00
Sebastiaan van Stijn
7964cae9e8 Merge pull request #47306 from akerouanton/revert-automatically-enable-ipv6
Revert "daemon: automatically set network EnableIPv6 if needed"
2024-02-02 16:29:49 +01:00
Albin Kerouanton
e37172c613 api/t/network: ValidateIPAM: ignore v6 subnet when IPv6 is disabled
Commit 4f47013feb introduced a new validation step to make sure no
IPv6 subnet is configured on a network which has EnableIPv6=false.

Commit 5d5eeac310 then removed that validation step and automatically
enabled IPv6 for networks with a v6 subnet. But this specific commit
was reverted in c59e93a67b and now the error introduced by 4f47013feb
is re-introduced.

But it turns out some users expect a network created with an IPv6
subnet and EnableIPv6=false to actually have no IPv6 connectivity.
This restores that behavior.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 10:34:39 +01:00
Albin Kerouanton
c59e93a67b Revert "daemon: automatically set network EnableIPv6 if needed"
This reverts commit 5d5eeac310.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 10:34:26 +01:00
Albin Kerouanton
83af50aee3 libnet: boltdb: inline getDBhandle()
Previous commit made getDBhandle a one-liner returning a struct
member -- making it useless. Inline it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:19:07 +01:00
Albin Kerouanton
4d7c11c208 libnet: boltdb: remove PersistConnection
This parameter was used to tell the boltdb kvstore not to open/close
the underlying boltdb db file before/after each get/put operation.

Since d21d0884ae, we've a single datastore instance shared by all
components that need it. That commit set `PersistConnection=true`.
We can now safely remove this param altogether, and remove all the
code that was opening and closing the db file before and after each
operation -- it's dead code!

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:19:07 +01:00
Albin Kerouanton
8070a9aa66 libnet: drop TestMultipleControllersWithSameStore
This test is non-representative of what we now do in libnetwork.
Since the ability of opening the same boltdb database multiple
times in parallel will be dropped in the next commit, just remove
this test.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:19:07 +01:00
Albin Kerouanton
ca683c1c77 Merge pull request #47233 from robmry/47146-duplicate_mac_addrs2
Only restore a configured MAC addr on restart.
2024-02-02 09:08:17 +01:00
Albin Kerouanton
025967efd0 Merge pull request #47293 from robmry/47229-internal-bridge-firewalld
Add internal n/w bridge to firewalld docker zone
2024-02-02 08:36:27 +01:00
Albin Kerouanton
add2c4c79b Merge pull request #47285 from corhere/libn/one-datastore-to-rule-them-all
libnetwork: share a single datastore with drivers
2024-02-02 08:03:01 +01:00
Sebastiaan van Stijn
8604cc400d Merge pull request #47242 from robmry/remove_etchosts_build_unused_params
Remove unused params from etchosts.Build()
2024-02-02 01:09:10 +01:00
Brian Goff
e240ba44b7 Merge pull request #47300 from corhere/libc8d/fix-startup-data-race
libcontainerd/supervisor: fix data race
2024-02-01 15:02:00 -08:00
Laura Brehm
82dda18898 tests: add plugin install test w/ digest
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:

```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-02-01 23:00:38 +00:00
Cory Snider
dd20bf4862 libcontainerd/supervisor: fix data race
The monitorDaemon() goroutine calls startContainerd() then blocks on
<-daemonWaitCh to wait for it to exit. The startContainerd() function
would (re)initialize the daemonWaitCh so a restarted containerd could be
waited on. This implementation was race-free because startContainerd()
would synchronously initialize the daemonWaitCh before returning. When
the call to start the managed containerd process was moved into the
waiter goroutine, the code to initialize the daemonWaitCh struct field
was also moved into the goroutine. This introduced a race condition.

Move the daemonWaitCh initialization to guarantee that it happens before
the startContainerd() call returns.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-01 15:53:18 -05:00
Sebastiaan van Stijn
f5cf22ca99 Merge pull request #47259 from vvoland/api-build-version
api: Document `version` in `/build`
2024-02-01 19:16:25 +01:00
Paweł Gronowski
0c3b8ccda7 api: Document version in /build
It was introduced in API v1.38 but wasn't documented.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-01 17:00:07 +01:00
Sebastiaan van Stijn
810ef4d0b6 Merge pull request #47244 from thaJeztah/bump_grpc
vendor: google.golang.org/grpc v1.59.0
2024-02-01 15:44:15 +01:00
Rob Murray
2cc627932a Add internal n/w bridge to firewalld docker zone
Containers attached to an 'internal' bridge network are unable to
communicate when the host is running firewalld.

Non-internal bridges are added to a trusted 'docker' firewalld zone, but
internal bridges were not.

DOCKER-ISOLATION iptables rules are still configured for an internal
network, they block traffic to/from addresses outside the network's subnet.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 11:49:53 +00:00
Sebastiaan van Stijn
968f05910a Merge pull request #47263 from crazy-max/bump-actions
ci: bump remaining gha to latest stable
2024-02-01 11:39:29 +01:00
Rob Murray
8c64b85fb9 No inspect 'Config.MacAddress' unless configured.
Do not set 'Config.MacAddress' in inspect output unless the MAC address
is configured.

Also, make sure it is filled in for a configured address on the default
network before the container is started (by translating the network name
from 'default' to 'config' so that the address lookup works).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:57:35 +00:00
Rob Murray
dae33031e0 Only restore a configured MAC addr on restart.
The API's EndpointConfig struct has a MacAddress field that's used for
both the configured address, and the current address (which may be generated).

A configured address must be restored when a container is restarted, but a
generated address must not.

The previous attempt to differentiate between the two, without adding a field
to the API's EndpointConfig that would show up in 'inspect' output, was a
field in the daemon's version of EndpointSettings, MACOperational. It did
not work, MACOperational was set to true when a configured address was
used. So, while it ensured addresses were regenerated, it failed to preserve
a configured address.

So, this change removes that code, and adds DesiredMacAddress to the wrapped
version of EndpointSettings, where it is persisted but does not appear in
'inspect' results. Its value is copied from MacAddress (the API field) when
a container is created.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:55:54 +00:00
CrazyMax
a2026ee442 ci: update to docker/bake-action@v4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 09:33:02 +01:00
CrazyMax
5a3c463a37 ci: update to codecov/codecov-action@v4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 09:33:02 +01:00
CrazyMax
9babc02283 ci: update to actions/download-artifact@v4 and actions/upload-artifact@v4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 09:33:02 +01:00
CrazyMax
a83557d747 ci: update to actions/cache@v3
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-01 08:29:54 +01:00
Cory Snider
2200c0137f libnetwork/datastore: don't parse file path
File paths can contain commas, particularly paths returned from
t.TempDir() in subtests which include commas in their names. There is
only one datastore provider and it only supports a single address, so
the only use of parsing the address is to break tests in mysterious
ways.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-31 21:26:28 -05:00
Cory Snider
d21d0884ae libnetwork: share a single datastore with drivers
The bbolt library wants exclusive access to the boltdb file and uses
file locking to assure that is the case. The controller and each network
driver that needs persistent storage instantiates its own unique
datastore instance, backed by the same boltdb file. The boltdb kvstore
implementation works around multiple access to the same boltdb file by
aggressively closing the boltdb file between each transaction. This is
very inefficient. Have the controller pass its datastore instance into
the drivers and enable the PersistConnection option to disable closing
the boltdb between transactions.

Set data-dir in unit tests which instantiate libnetwork controllers so
they don't hang trying to lock the default boltdb database file.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-31 21:08:34 -05:00
Sebastiaan van Stijn
3e230cfdcc Merge pull request from GHSA-xw73-rw38-6vjc
image/cache: Restrict cache candidates to locally built images
2024-02-01 01:12:23 +01:00
Sebastiaan van Stijn
f42b8ae8db Merge pull request #47278 from thaJeztah/bump_containerd_binary_1.7.13
update containerd binary to v1.7.13
2024-02-01 00:03:58 +01:00
Sebastiaan van Stijn
7a920fd275 Merge pull request #47268 from thaJeztah/bump_runc_binary_1.1.12
update runc binary to v1.1.12
2024-01-31 22:50:35 +01:00
Sebastiaan van Stijn
584e60e820 Merge pull request #47273 from vvoland/vendor-bk-0.12.5
vendor: github.com/moby/buildkit v0.12.5
2024-01-31 22:24:49 +01:00
Sebastiaan van Stijn
8ee908e47c Merge pull request #47272 from thaJeztah/bump_runc_1.1.12
vendor: github.com/opencontainers/runc v1.1.12
2024-01-31 22:17:42 +01:00
Sebastiaan van Stijn
835cdcac95 update containerd binary to v1.7.13
Update the containerd binary that's used in CI

- full diff: https://github.com/containerd/containerd/compare/v1.7.12...v1.7.13
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 22:01:50 +01:00
Paweł Gronowski
f4a93b6993 vendor: github.com/moby/buildkit v0.12.5
full diff: https://github.com/moby/buildkit/compare/v0.12.4...v0.12.5

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-31 21:24:50 +01:00
Sebastiaan van Stijn
b20dccba5e vendor: github.com/opencontainers/runc v1.1.12
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.12
- full diff: https://github.com/opencontainers/runc/compare/v1.1.11...v1.1.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 21:17:56 +01:00
Sebastiaan van Stijn
44bf407d4d update runc binary to v1.1.12
Update the runc binary that's used in CI and for the static packages, which
includes a fix for [CVE-2024-21626].

- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.12
- full diff: https://github.com/opencontainers/runc/compare/v1.1.11...v1.1.12

[CVE-2024-21626]: https://github.com/opencontainers/runc/security/advisories/GHSA-xr7r-f8xq-vfvv

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 21:05:31 +01:00
Sebastiaan van Stijn
8a81b9d35f Merge pull request #47264 from vvoland/ci-fix-makeps1-templatefail
hack/make.ps1: Fix go list pattern
2024-01-31 21:01:08 +01:00
Paweł Gronowski
ecb217cf69 hack/make.ps1: Fix go list pattern
The double quotes inside a single quoted string don't need to be
escaped.
Looks like different Powershell versions are treating this differently
and it started failing unexpectedly without any changes on our side.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-31 19:54:27 +01:00
Sebastiaan van Stijn
2df4755725 Merge pull request #47250 from thaJeztah/update_actions
gha: update actions to account for node 16 deprecation
2024-01-31 12:45:29 +01:00
Sebastiaan van Stijn
3a8191225a gha: update to crazy-max/ghaction-github-runtime@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff: https://github.com/crazy-max/ghaction-github-runtime/compare/v2.2.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
08251978a8 gha: update to docker/login-action@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff https://github.com/docker/login-action/compare/v2.2.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
5d396e0533 gha: update to docker/setup-qemu-action@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff https://github.com/docker/setup-qemu-action/compare/v2.2.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
4a1839ef1d gha: update to docker/bake-action@v4
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff https://github.com/docker/bake-action/compare/v2.3.0...v4.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:17 +01:00
Sebastiaan van Stijn
b7fd571b0a gha: update to docker/setup-buildx-action@v3
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff: https://github.com/docker/setup-buildx-action/compare/v2.10.0...v3.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:16 +01:00
Sebastiaan van Stijn
00a2626b56 gha: update to docker/metadata-action@v5
- Node 20 as default runtime (requires Actions Runner v2.308.0 or later)
- full diff: https://github.com/docker/metadata-action/compare/v4.6.0...v5.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:50:11 +01:00
Sebastiaan van Stijn
e27a785f43 gha: update to actions/setup-go@v5
- full diff: https://github.com/actions/setup-go/compare/v3.5.0...v5.0.0

v5

In scope of this release, we change Nodejs runtime from node16 to node20.
Moreover, we update some dependencies to the latest versions.

Besides, this release contains such changes as:

- Fix hosted tool cache usage on windows
- Improve documentation regarding dependencies caching

V4

The V4 edition of the action offers:

- Enabled caching by default
- The action will try to enable caching unless the cache input is explicitly
  set to false.

Please see "Caching dependency files and build outputs" for more information:
https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:46:17 +01:00
Sebastiaan van Stijn
fb53ee6ba3 gha: update to actions/github-script@v7
- full diff: https://github.com/actions/github-script/compare/v6.4.1...v7.0.1

breaking changes: https://github.com/actions/github-script?tab=readme-ov-file#v7

> Version 7 of this action updated the runtime to Node 20
> https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions
>
> All scripts are now run with Node 20 instead of Node 16 and are affected
> by any breaking changes between Node 16 and 20
>
> The previews input now only applies to GraphQL API calls as REST API previews
> are no longer necessary
> https://github.blog/changelog/2021-10-14-rest-api-preview-promotions/.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:46:17 +01:00
Sebastiaan van Stijn
0ffddc6bb8 gha: update to actions/checkout@v4
Release notes:

- https://github.com/actions/checkout/compare/v3.6.0...v4.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 10:46:14 +01:00
Cory Snider
987fe37ed1 d/logger/journald: quit waiting when logger closes
If a reader has caught up to the logger and is waiting for the next
message, it should stop waiting when the logger is closed. Otherwise
the reader will unnecessarily wait the full closedDrainTimeout for no
log messages to arrive.

This case was overlooked when the journald reader was recently
overhauled to be compatible with systemd 255, and the reader tests only
failed when a logical race happened to settle in such a way to exercise
the bugged code path. It was only after implicit flushing on close was
added to the journald test harness that the Follow tests would
repeatably fail due to this bug. (No new regression tests are needed.)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 17:57:12 -05:00
Cory Snider
d53b7d7e46 d/logger/journald: sync logger on close in tests
The journald reader test harness injects an artificial asynchronous
delay into the logging pipeline: a logged message won't be written to
the journal until at least 150ms after the Log() call returns. If a test
returns while log messages are still in flight to be written, the logs
may attempt to be written after the TempDir has been cleaned up, leading
to spurious errors.

The logger read tests which interleave writing and reading have to
include explicit synchronization points to work reliably with this delay
in place. On the other hand, tests should not be required to sync the
logger explicitly before returning. Override the Close() method in the
test harness wrapper to wait for in-flight logs to be flushed to disk.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 17:18:46 -05:00
Cory Snider
39c5c16521 d/logger/loggertest: improve TestConcurrent
- Check the return value when logging messages
- Log the stream (stdout/stderr) and list of messages that were not read
- Wait until the logger is closed before returning early (panic/fatal)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 17:18:12 -05:00
Cory Snider
5792bf7ab3 d/logger/journald: log journal-remote cmd output
Writing the systemd-journal-remote command output directly to os.Stdout
and os.Stderr makes it nearly impossible to tell which test case the
output is related to when the tests are not run in verbose mode. Extend
the journald sender fake to redirect output to the test log so they
interleave with the rest of the test output.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 14:50:57 -05:00
Cory Snider
982e777d49 d/logger/journald: fix data race in test harness
The Go race detector was detecting a data race when running the
TestLogRead/Follow/Concurrent test against the journald logging driver.
The race was in the test harness, specifically syncLogger. The waitOn
field would be reassigned each time a log entry is sent to the journal,
which is not concurrency-safe. Make it concurrency-safe using the same
patterns that are used in the log follower implementation to synchronize
with the logger.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-30 14:34:15 -05:00
Sebastiaan van Stijn
f472dda2e9 Merge pull request #47236 from akerouanton/remove-sb-leave-options-param
libnet: remove arg `options` from (*Endpoint).Leave()
2024-01-30 16:57:36 +01:00
Sebastiaan van Stijn
ca40ac030c vendor: google.golang.org/grpc v1.59.0
full diff:

- https://github.com/grpc/grpc-go/compare/v1.58.3...v1.59.0
- 782d3b101e...b8732ec382
- https://github.com/googleapis/google-cloud-go/compare/v0.110.4...v0.110.7
- https://github.com/googleapis/google-cloud-go/compare/compute/v1.21.0...compute/v1.23.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-29 18:59:21 +01:00
Sebastiaan van Stijn
0818a476e5 vendor: github.com/go-logr/logr v1.3.0
full diff: https:// github.com/go-logr/logr/compare/v1.2.4...v1.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-29 18:51:54 +01:00
Sebastiaan van Stijn
a0b53f6fd2 vendor: golang.org/x/net v0.18.0
full diff: https://github.com/golang/net/compare/v0.17.0...v0.18.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-29 18:49:40 +01:00
Rob Murray
2ddec74d59 Remove unused params from etchosts.Build()
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-29 15:37:08 +00:00
Albin Kerouanton
794f7127ef Merge pull request #47062 from robmry/35954-default_ipv6_enabled
Detect IPv6 support in containers, generate '/etc/hosts' accordingly.
2024-01-29 16:31:35 +01:00
Paweł Gronowski
5e13f54f57 c8d/save: Handle digested reference same as ID
When saving an image treat `image@sha256:abcdef...` the same as
`abcdef...`, this makes it:

- Not export the digested tag as the image name
- Not try to export all tags from the image repository

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-29 16:29:05 +01:00
Paweł Gronowski
d131f00fff image/save: Fix untagged images not present in index.json
Saving an image via digested reference, ID or truncated ID doesn't store
the image reference in the archive. This also causes the save code to
not add the image's manifest to the index.json.
This commit explicitly adds the untagged manifests to the index.json if
no tagged manifests were added.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-29 16:26:37 +01:00
Albin Kerouanton
7d4ee14147 Merge pull request #47234 from corhere/libn/overlay-peerdb-unused-flags
libnetwork/d/overlay: drop unused `miss` flags from peerAdd
2024-01-29 13:02:08 +01:00
Brian Goff
0f507ef624 Merge pull request #47019 from corhere/fix-journald-logs-systemd-255
logger/journald: fix tailing logs with systemd 255
2024-01-28 12:22:16 -08:00
Albin Kerouanton
21136865ac libnet: remove arg options from (*Endpoint).Leave()
This arg is never set by any caller. Better remove it

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-27 09:26:36 +01:00
Cory Snider
a8e8a4cdad libn/d/overlay: drop miss flags from peerAddOp
as all callers unconditionally set them to false.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 15:43:57 -05:00
Cory Snider
6ee58c2d29 libnetwork/d/overlay: drop miss flags from peerAdd
as all callers unconditionally set them to false.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 15:38:13 -05:00
Cory Snider
905477c8ae logger/journald: drop errDrainDone sentinel
errDrainDone is a sentinel error which is never supposed to escape the
package. Consequently, it needs to be filtered out of returns all over
the place, adding boilerplate. Forgetting to filter out these errors
would be a logic bug which the compiler would not help us catch. Replace
it with boolean multi-valued returns as they can't be accidentally
ignored or propagated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 12:42:09 -05:00
Cory Snider
d70fe8803c logger/journald: wait no longer than the deadline
While it doesn't really matter if the reader waits for an extra
arbitrary period beyond an arbitrary hardcoded timeout, it's also
trivial and cheap to implement, and nice to have.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 12:42:04 -05:00
Cory Snider
e94ec8068d logger/journald: use deadline for drain timeout
The journald reader uses a timer to set an upper bound on how long to
wait for the final log message of a stopped container. However, the
timer channel is only received from in non-blocking select statements!
There isn't enough benefit of using a timer to offset the cost of having
to manage the timer resource. Setting a deadline and comparing the
current time is just as effective, without having to manage the
lifecycle of any runtime resources.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 11:47:02 -05:00
Cory Snider
71bfffdad1 l/journald: make tests compatible with systemd 255
Synthesize a boot ID for journal entries fed into
systemd-journal-remote, as required by systemd 255.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 11:47:02 -05:00
Cory Snider
931568032a daemon/logger/loggertest: expand log-follow tests
Following logs with a non-negative tail when the container log is empty
is broken on the journald driver when used with systemd 255. Add tests
which cover this edge case to our loggertest suite.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-26 11:47:02 -05:00
Sebastiaan van Stijn
ee6cbc540e Merge pull request #47188 from thaJeztah/cleanup_newRouterOptions
cmd/dockerd: newRouterOptions: pass cluster as argument, and slight cleanup
2024-01-26 16:49:19 +01:00
Sebastiaan van Stijn
6d34bb71a0 Merge pull request #47230 from thaJeztah/update_dev_cli_compose
Dockerfile: update docker-cli to v25.0.1, docker compose v2.24.3
2024-01-26 10:18:58 +01:00
Sebastiaan van Stijn
388ba9a69c Dockerfile: update docker compose to v2.24.3
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.24.2...v2.24.3
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 09:30:17 +01:00
Sebastiaan van Stijn
3eb1527fdb Dockerfile: update dev-shell version of the cli to v25.0.1
Update the docker CLI that's available for debugging in the dev-shell
to the v25 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 09:23:14 +01:00
Sebastiaan van Stijn
93861bb5e6 Merge pull request #47227 from dvdksn/docs-api-remove-dead-links
docs: remove dead links from api version history
2024-01-25 22:59:25 +01:00
David Karlsson
7f94acb6ab docs: remove dead links from api verison history
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-01-25 20:15:24 +01:00
Sebastiaan van Stijn
97a5614d23 Merge pull request #47224 from s4ke/bump-swarmkit-generic-resources#main
Fix HasResource inverted boolean error - vendor swarmkit v2.0.0-20240125134710-dcda100a8261
2024-01-25 18:49:52 +01:00
Martin Braun
5c2eda6f71 vendor swarmkit v2.0.0-20240125134710-dcda100a8261
Signed-off-by: Martin Braun <braun@neuroforge.de>
2024-01-25 16:26:04 +01:00
Paweł Gronowski
96d461d27e builder/windows: Don't set ArgsEscaped for RUN cache probe
Previously this was done indirectly - the `compare` function didn't
check the `ArgsEscaped`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:07 +01:00
Paweł Gronowski
877ebbe038 image/cache: Check image platform
Make sure the cache candidate platform matches the requested.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:05 +01:00
Paweł Gronowski
96ac22768a image/cache: Restrict cache candidates to locally built images
Restrict cache candidates only to images that were built locally.
This doesn't affect builds using `--cache-from`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:03 +01:00
Paweł Gronowski
c6156dc51b daemon/imageStore: Mark images built locally
Store additional image property which makes it possible to distinguish
if image was built locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:00 +01:00
Paweł Gronowski
537348763f image/cache: Compare all config fields
Add checks for some image config fields that were missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:03:58 +01:00
Sebastiaan van Stijn
c7b3cb101b Merge pull request #47213 from thaJeztah/more_gocompat
add more //go:build directives to prevent downgrading to go1.16 language
2024-01-25 14:37:29 +01:00
Sebastiaan van Stijn
69d2923e4e Merge pull request #46419 from vvoland/pkg-pools-close-noop
pkg/ioutils: Make subsequent Close calls a no-op
2024-01-25 14:24:43 +01:00
Sebastiaan van Stijn
86198815a2 Merge pull request #47209 from corhere/sliceutil-map
internal/sliceutil: add utilities to map values
2024-01-25 11:51:50 +01:00
Sebastiaan van Stijn
d864df5a1d Merge pull request #47208 from akerouanton/libnet-ds-remove-unused-key-params
libnet/ds: remove unused param `key` from `GetObject` and `List`
2024-01-25 11:46:20 +01:00
Sebastiaan van Stijn
bd4ff31775 add more //go:build directives to prevent downgrading to go1.16 language
This is a follow-up to 2cf230951f, adding
more directives to adjust for some new code added since:

Before this patch:

    make -C ./internal/gocompat/
    GO111MODULE=off go generate .
    GO111MODULE=on go mod tidy
    GO111MODULE=on go test -v

    # github.com/docker/docker/internal/sliceutil
    internal/sliceutil/sliceutil.go:3:12: type parameter requires go1.18 or later (-lang was set to go1.16; check go.mod)
    internal/sliceutil/sliceutil.go:3:14: predeclared comparable requires go1.18 or later (-lang was set to go1.16; check go.mod)
    internal/sliceutil/sliceutil.go:4:19: invalid map key type T (missing comparable constraint)

    # github.com/docker/docker/libnetwork
    libnetwork/endpoint.go:252:17: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

    # github.com/docker/docker/daemon
    daemon/container_operations.go:682:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    daemon/inspect.go:42:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

With this patch:

    make -C ./internal/gocompat/
    GO111MODULE=off go generate .
    GO111MODULE=on go mod tidy
    GO111MODULE=on go test -v
    === RUN   TestModuleCompatibllity
        main_test.go:321: all packages have the correct go version specified through //go:build
    --- PASS: TestModuleCompatibllity (0.00s)
    PASS
    ok  	gocompat	0.031s
    make: Leaving directory '/go/src/github.com/docker/docker/internal/gocompat'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-25 11:18:44 +01:00
Cory Snider
e245fb76de internal/sliceutil: add utilities to map values
Functional programming for the win! Add a utility function to map the
values of a slice, along with a curried variant, to tide us over until
equivalent functionality gets added to the standard library
(https://go.dev/issue/61898)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-24 17:56:29 -05:00
Albin Kerouanton
3147a013fb libnet/ds: remove unused param key from List
Since 43dccc6 the `key` param is never used and can be safely
removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-24 22:42:18 +01:00
Albin Kerouanton
f7ef0e9fc7 libnet/ds: remove unused param key from GetObject
Since 43dccc6 the `key` param is never used and can be safely
removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-24 22:42:18 +01:00
Sebastiaan van Stijn
e8346c53d9 Merge pull request #46786 from rumpl/c8d-userns-namespace
c8d: Use a specific containerd namespace when userns are remapped
2024-01-24 20:36:40 +01:00
Djordje Lukic
3a617e5463 c8d: Use a specific containerd namespace when userns are remapped
We need to isolate the images that we are remapping to a userns, we
can't mix them with "normal" images. In the graph driver case this means
we create a new root directory where we store the images and everything
else, in the containerd case we can use a new namespace.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-01-24 15:46:16 +01:00
Sebastiaan van Stijn
43ffb1ee9d Merge pull request #47148 from thaJeztah/api_remove_deprecated_types
api/types: remove deprecated type-aliases
2024-01-24 12:40:27 +01:00
Sebastiaan van Stijn
115c7673dc Merge pull request #47198 from thaJeztah/image_remove_IDFromDigest
image: remove deprecated IDFromDigest
2024-01-24 12:36:42 +01:00
Sebastiaan van Stijn
f7e2357745 image: remove deprecated IDFromDigest
This function was deprecated in 456ea1bb1d
(Docker v24.0), and is no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:54:59 +01:00
Sebastiaan van Stijn
314ea05f8d Merge pull request #47197 from corhere/libn/carry-typo-fixes
libnetwork: carry typo fixes from moby/libnetwork repo
2024-01-24 01:31:08 +01:00
Sebastiaan van Stijn
13f46948dd api/types: remove deprecated container-types
These types were deprecated in v25.0, and moved to api/types/container;

This patch removes the aliases for;

- api/types.ResizeOptions (deprecated in 95b92b1f97)
- api/types.ContainerAttachOptions (deprecated in 30f09b4a1a)
- api/types.ContainerCommitOptions (deprecated in 9498d897ab)
- api/types.ContainerRemoveOptions (deprecated in 0f77875220)
- api/types.ContainerStartOptions (deprecated in 7bce33eb0f)
- api/types.ContainerListOptions (deprecated in 9670d9364d)
- api/types.ContainerLogsOptions (deprecated in ebef4efb88)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
4b09bc2145 api/types: remove deprecated service-types
These types were deprecated in v25.0, and moved to api/types/swarm;

This patch removes the aliases for;

- api/types.ServiceUpdateResponse (deprecated in 5b3e6555a3)
- api/types.ServiceCreateResponse (deprecated in ec69501e94)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
49637d0206 api/types: remove deprecated image-types
These types were deprecated in 48cacbca24
(v25.0), and moved to api/types/image.

This patch removes the aliases for;

- api/types.ImageDeleteResponseItem
- api/types.ImageSummary
- api/types.ImageMetadata

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
eccb1a3eb8 api/types: remove deprecated checkpoint-types
These types were deprecated in b688af2226
(v25.0), and moved to api/types/checkpoint.

This patch removes the aliases for;

- api/types.CheckpointCreateOptions
- api/types.CheckpointListOptions
- api/types.CheckpointDeleteOptions
- api/types.Checkpoint

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
0b1921649f api/types: remove deprecated system info types and functions
These types were deprecated in c90229ed9a
(v25.0), and moved to api/types/system.

This patch removes the aliases for;

- api/types.Info
- api/types.Commit
- api/types.PluginsInfo
- api/types.NetworkAddressPool
- api/types.Runtime
- api/types.SecurityOpt
- api/types.KeyValue
- api/types.DecodeSecurityOptions

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:24 +01:00
Sebastiaan van Stijn
4544bedea3 Merge pull request #47139 from thaJeztah/api_move_image_options
api/types: move image options to api/types/image
2024-01-24 01:26:55 +01:00
Cory Snider
6f44138269 libnetwork: fix tiny grammar mistake on design.md
Co-authored-by: Farhim Ferdous <37705070+AluBhorta@users.noreply.github.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-23 18:26:45 -05:00
Cory Snider
9a41cc58d9 libnetwork: fix typo in iptables.go
Co-authored-by: Ikko Ashimine <eltociear@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-23 18:25:08 -05:00
Sebastiaan van Stijn
ac2a028dcc api/types: move image options to api/types/image
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 00:10:33 +01:00
Sebastiaan van Stijn
8906adc8d4 Merge pull request #47138 from thaJeztah/move_image_backend_opt
api/types/image: move GetImageOpts to api/types/backend
2024-01-23 23:41:38 +01:00
Sebastiaan van Stijn
0bb84f5cef Merge pull request #47195 from akerouanton/fix-multiple-rename-error
daemon: rename: don't reload endpoint from datastore
2024-01-23 23:41:07 +01:00
Albin Kerouanton
80c44b4b2e daemon: rename: don't reload endpoint from datastore
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.

This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.

As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).

In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 22:53:21 +01:00
Sebastiaan van Stijn
2da96d6c12 Merge pull request #47121 from voloder/master
Make sure that make doesn't rm -rf the system out of existence
2024-01-23 19:03:06 +01:00
Paweł Gronowski
0b64499a24 Merge pull request #47194 from vvoland/volume-cifs-resolve-optout-2
volume/local: Fix CIFS urls with spaces, add tests
2024-01-23 18:58:52 +01:00
Sebastiaan van Stijn
9763709c05 Merge pull request #47181 from akerouanton/fix-aliases-on-default-bridge
daemon: only add short cid to aliases for custom networks
2024-01-23 18:28:33 +01:00
Paweł Gronowski
250886741b volume/local: Fix cifs url containing spaces
Unescapes the URL to avoid passing an URL encoded address to the kernel.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:11 +01:00
Paweł Gronowski
f4beb130b0 volume/local: Add tests for parsing nfs/cifs mounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:09 +01:00
Paweł Gronowski
df43311f3d volume/local: Break early if addr was specified
I made a mistake in the last commit - after resolving the IP from the
passed `addr` for CIFS it would still resolve the `device` part.

Apply only one name resolution

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:17:53 +01:00
Albin Kerouanton
9f37672ca8 daemon: only add short cid to aliases for custom networks
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.

This commit reintroduces the old logic.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 17:07:40 +01:00
Sebastiaan van Stijn
c25773ecbf cmd/dockerd: newRouterOptions: pass cluster as argument, and slight cleanup
- pass the cluster as an argument instead of manually setting it after
  creating the router-options
- remove the "opts" variable, to prevent it accidentally being used (with
  the assumption that's the value returned)
- use a struct-literal for the returned options.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 16:54:54 +01:00
Sebastiaan van Stijn
f19f233ca5 Merge pull request #47187 from thaJeztah/fix_gateway_ip
fix "host-gateway-ip" label not set for builder workers
2024-01-23 16:52:35 +01:00
Sebastiaan van Stijn
ca67dbd12c Merge pull request #47185 from vvoland/volume-cifs-resolve-optout
volume/local: Make host resolution backwards compatible
2024-01-23 16:23:40 +01:00
Paweł Gronowski
cac52f7173 Merge pull request #47167 from vvoland/c8d-prefer-default-platform-snapshot
c8d/snapshot: Create any platform if not specified
2024-01-23 15:25:17 +01:00
Sebastiaan van Stijn
00c9785e2e fix "host-gateway-ip" label not set for builder workers
Commit 21e50b89c9 added a label on the buildkit
worker to advertise the host-gateway-ip. This option can be either set by the
user in the daemon config, or otherwise defaults to the gateway-ip.

If no value is set by the user, discovery of the gateway-ip happens when
initializing the network-controller (`NewDaemon`, `daemon.restore()`).

However d222bf097c changed how we handle the
daemon config. As a result, the `cli.Config` used when initializing the
builder only holds configuration information form the daemon config
(user-specified or defaults), but is not updated with information set
by `NewDaemon`.

This patch adds an accessor on the daemon to get the current daemon config.
An alternative could be to return the config by `NewDaemon` (which should
likely be a _copy_ of the config).

Before this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: <nil>

After this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.18.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 14:58:01 +01:00
Paweł Gronowski
0d51cf9db8 volume/local: Make host resolution backwards compatible
Commit 8ae94cafa5 added a DNS resolution
of the `device` part of the volume option.

The previous way to resolve the passed hostname was to use `addr`
option, which was handled by the same code path as the `nfs` mount type.

The issue is that `addr` is also an SMB module option handled by kernel
and passing a hostname as `addr` produces an invalid argument error.

To fix that, restore the old behavior to handle `addr` the same way as
before, and only perform the new DNS resolution of `device` if there is
no `addr` passed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 14:49:05 +01:00
Sebastiaan van Stijn
1786517338 Merge pull request #47179 from thaJeztah/update_compose
Dockerfile: update docker compose to v2.24.2
2024-01-23 11:25:58 +01:00
Sebastiaan van Stijn
22a504935f Merge pull request #45474 from thaJeztah/testing_cleanups
assorted test fixes and cleanups
2024-01-23 10:01:27 +01:00
Sebastiaan van Stijn
05d952b246 Dockerfile: update docker compose to v2.24.2
Update the version of compose used in CI to the latest version.

- full diff: docker/compose@v2.24.1...v2.24.2
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 09:51:41 +01:00
Sebastiaan van Stijn
d86d24de35 Merge pull request #47174 from corhere/richer-xattr-errors
pkg/system: return even richer xattr errors
2024-01-23 09:46:12 +01:00
Sebastiaan van Stijn
20bd690844 integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
34668a5945 pkg/archive: fixe some unclosed file-handles in tests
Also fixing a "defer in loop" warning, instead changing to use
sub-tests, and simplifying some code, using os.WriteFile() instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
1090aaaedd libnetwork: fix some unclosed file-handles in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
c482383458 fix some leaking mounts in tests
This should help with errors such as:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride3702360633/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

    === RUN   TestSysctlOverrideHost
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverrideHost226485533/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverrideHost (0.00s)

    === RUN   TestDockerSuite/TestRunWithVolumesIsRecursive
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestDockerSuiteTestRunWithVolumesIsRecursive1156692230/001/tmpfs: device or resource busy
        --- FAIL: TestDockerSuite/TestRunWithVolumesIsRecursive (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:23 +01:00
Cory Snider
43bf65c174 pkg/system: return even richer xattr errors
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-22 15:25:10 -05:00
Sebastiaan van Stijn
a3a42c459e api/types/image: move GetImageOpts to api/types/backend
The `GetImageOpts` struct is used for options to be passed to the backend,
and are not used in client code. This struct currently is intended for internal
use only.

This patch moves the `GetImageOpts` struct to the backend package to prevent
it being imported in the client, and to make it more clear that this is part
of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 20:45:21 +01:00
Sebastiaan van Stijn
c87e0ad209 Merge pull request #47168 from robmry/47146-duplicate_mac_addrs
Remove generated MAC addresses on restart.
2024-01-22 19:48:24 +01:00
Rob Murray
cd53b7380c Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-22 17:52:20 +00:00
Paweł Gronowski
fb19f1fc20 c8d/snapshot: Create any platform if not specified
With containerd snapshotters enabled `docker run` currently fails when
creating a container from an image that doesn't have the default host
platform without an explicit `--platform` selection:

```
$ docker run image:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

This is confusing and the graphdriver behavior is much better here,
because it runs whatever platform the image has, but prints a warning:

```
$ docker run image:amd64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
```

This commits changes the containerd snapshotter behavior to be the same
as the graphdriver. This doesn't affect container creation when platform
is specified explicitly.

```
$ docker run --rm --platform linux/arm64 asdf:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 16:15:07 +01:00
Sebastiaan van Stijn
3602ba0afd Merge pull request #47162 from vvoland/25-fix-swarm-startinterval
daemon/cluster/executer: Add missing `StartInterval`
2024-01-22 15:51:37 +01:00
Sebastiaan van Stijn
f0eef50273 Merge pull request #47159 from akerouanton/fix-bad-http-code
daemon: return an InvalidParameter error when ep settings are wrong
2024-01-22 15:06:06 +01:00
Sebastiaan van Stijn
b6a5c2968b Merge pull request #47160 from vvoland/save-fix-oci-diffids
image/save: Fix layers order in OCI manifest
2024-01-22 15:05:06 +01:00
Paweł Gronowski
6100190e5c daemon/cluster/executer: Add missing StartInterval
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 14:42:17 +01:00
Paweł Gronowski
17fd6562bf image/save: Fix layers order in OCI manifest
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:48:12 +01:00
Paweł Gronowski
4979605212 image/save: Change layers type to DiffID
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:47:57 +01:00
Albin Kerouanton
fcc651972e daemon: return an InvalidParameter error when ep settings are wrong
Since v25.0 (commit ff50388), we validate endpoint settings when
containers are created, instead of doing so when containers are started.
However, a container created prior to that release would still trigger
validation error at start-time. In such case, the API returns a 500
status code because the Go error isn't wrapped into an InvalidParameter
error. This is now fixed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-22 12:48:23 +01:00
Sebastiaan van Stijn
42a07883f7 Merge pull request #47154 from thaJeztah/integration_cli_api_versions
integration-cli: adjust inspect tests to use current API version
2024-01-22 11:07:45 +01:00
Sebastiaan van Stijn
5a3a101af2 Merge pull request #47151 from thaJeztah/fix_TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs
integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
2024-01-22 10:23:50 +01:00
Sebastiaan van Stijn
6803dd8c90 Merge pull request #47124 from thaJeztah/remove_deprecated_api_docs
docs: remove documentation for deprecated API versions (v1.23 and before)
2024-01-22 10:23:31 +01:00
Sebastiaan van Stijn
82acb922cb Merge pull request #47147 from thaJeztah/integration_minor_refactor
integration: improve some asserts, and add asserts for unhandled errs
2024-01-22 10:23:03 +01:00
Akihiro Suda
e5dbe10e65 Merge pull request #47123 from thaJeztah/cli_25
Dockerfile: update docker-cli to v25.0.0, docker compose v2.24.1
2024-01-22 11:05:43 +09:00
Akihiro Suda
8528ed3409 Merge pull request #47128 from thaJeztah/remove_pkg_loopback
remove deprecated pkg/loopback (utility package for devicemapper)
2024-01-22 11:05:15 +09:00
Akihiro Suda
570b8a794c Merge pull request #47129 from thaJeztah/pkg_system_deprecated
pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
2024-01-22 11:04:25 +09:00
Akihiro Suda
5ad5334c2e Merge pull request #47130 from thaJeztah/pkg_homedir_deprecated
pkg/homedir: remove deprecated Key() and GetShortcutString()
2024-01-22 11:03:54 +09:00
Akihiro Suda
417826376f Merge pull request #47131 from thaJeztah/pkg_containerfs_deprecated
pkg/containerfs: remove deprecated ResolveScopedPath
2024-01-22 11:03:23 +09:00
Akihiro Suda
e30610aaa3 Merge pull request #47143 from thaJeztah/golang_x_updates
vendor: assorted golang.org/x/... updates
2024-01-22 11:02:48 +09:00
Sebastiaan van Stijn
a0466ca8e1 integration-cli: TestInspectAPIMultipleNetworks: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
13a384a6fa integration-cli: TestInspectAPIBridgeNetworkSettings121: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
52e3fff828 integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
This test was added in 75f6929b44, but pinned
to the API version that was current at the time (v1.20), which is now
deprecated.

Update the test to use the current API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:37:31 +01:00
Sebastiaan van Stijn
521123944a docs/api: remove version matrices from swagger files
These tables linked to deprecated API versions, and an up-to-date version of
the matrix is already included at https://docs.docker.com/engine/api/#api-version-matrix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:17:53 +01:00
Sebastiaan van Stijn
d54be2ee6d docs: remove documentation for deprecated API versions < v1.23
These versions are deprecated in v25.0.0, and disabled by default,
see 08e4e88482.

Users that need to refer to documentation for older API versions,
can use archived versions of the documentation on GitHub:

- API v1.23 and before: https://github.com/moby/moby/tree/v25.0.0/docs/api
- API v1.17 and before: https://github.com/moby/moby/tree/v1.9.1/docs/reference/api

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:12:42 +01:00
Sebastiaan van Stijn
64a6cc3afd integration/build: improve some asserts, and add asserts for unhandled errs
- add some asserts for unhandled errors
- use consts for fixed values, and slightly re-format Dockerfile contentt
- inline one-line Dockerfiles
- fix some vars to be properly camel-cased
- improve assert for error-types;

Before:

    === RUN   TestBuildPlatformInvalid
        build_test.go:685: assertion failed: expression is false: errdefs.IsInvalidParameter(err)
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

After:

    === RUN   TestBuildPlatformInvalid
        build_test.go:689: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
a88cd68d3e integration/images: improve some asserts, and add asserts for unhandled errs
Before:

    === FAIL: amd64.integration.image TestImagePullPlatformInvalid (0.01s)
        pull_test.go:37: assertion failed: expression is false: errdefs.IsInvalidParameter(err)

After:

    === RUN   TestImagePullPlatformInvalid
        pull_test.go:37: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
27e85c7b68 Merge pull request #47141 from thaJeztah/internalize_pkg_platforms
pkg/platforms: internalize in daemon/containerd
2024-01-20 23:47:48 +01:00
Sebastiaan van Stijn
a404017a86 vendor: golang.org/x/tools v0.14.0
full diff: https://github.com/golang/tools/comopare/v0.13.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:31 +01:00
Sebastiaan van Stijn
41a2aa2ee2 vendor: golang.org/x/oauth2 v0.11.0
full diff: https://github.com/golang/oauth2/comopare/v0.10.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:31 +01:00
Sebastiaan van Stijn
2799417da1 vendor: golang.org/x/mod v0.13.0, golang.org/x/tools v0.13.0
full diff:

- https://github.com/golang/mod/comopare/v0.11.0...v0.13.0
- https://github.com/golang/tools/comopare/v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:30 +01:00
Sebastiaan van Stijn
407ad89ff0 vendor: golang.org/x/sync v0.5.0
full diff: https://github.com/golang/sync/comopare/v0.3.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:30 +01:00
Sebastiaan van Stijn
94b4765363 pkg/platforms: internalize in daemon/containerd
This matcher was only used internally in the containerd implementation of
the image store. Un-export it, and make it a local utility in that package
to prevent external use.

This package was introduced in 1616a09b61
(v24.0), and there are no known external consumers of this package, so there
should be no need to deprecate / alias the old location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:28:56 +01:00
Sebastiaan van Stijn
c5187380d8 Merge pull request #46252 from thaJeztah/libnetwork_sandbox_sorting
libnetwork: Sandbox.ResolveName: refactor ordering of endpoints
2024-01-20 17:41:35 +01:00
Sebastiaan van Stijn
0a9bc3b507 libnetwork: Sandbox.ResolveName: refactor ordering of endpoints
When resolving names in swarm mode, services with exposed ports are
connected to user overlay network, ingress network, and local (docker_gwbridge)
networks. Name resolution should prioritize returning the VIP/IPs on user
overlay network over ingress and local networks.

Sandbox.ResolveName implemented this by taking the list of endpoints,
splitting the list into 3 separate lists based on the type of network
that the endpoint was attached to (dynamic, ingress, local), and then
creating a new list, applying the networks in that order.

This patch refactors that logic to use a custom sorter (sort.Interface),
which makes the code more transparent, and prevents iterating over the
list of endpoints multiple times.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 12:41:33 +01:00
Sebastiaan van Stijn
17c3829528 Merge pull request #47132 from corhere/allow-container-ip-outside-subpool
libnetwork: loosen container IPAM validation
2024-01-20 11:29:51 +01:00
Cory Snider
058b30023f libnetwork: loosen container IPAM validation
Permit container network attachments to set any static IP address within
the network's IPAM master pool, including when a subpool is configured.
Users have come to depend on being able to statically assign container
IP addresses which are guaranteed not to collide with automatically-
assigned container addresses.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:18:15 -05:00
Sebastiaan van Stijn
dfdd5169a2 Merge pull request #46113 from akerouanton/remove-deprecated-oom-score-adjust
daemon: remove --oom-score-adjust flag
2024-01-20 01:35:37 +01:00
Sebastiaan van Stijn
844ca49743 pkg/containerfs: remove deprecated ResolveScopedPath
This function was deprecated in b8f2caa80a
(v25.0), and is no longer in use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:27:07 +01:00
Sebastiaan van Stijn
2767d9ba05 pkg/homedir: remove deprecated Key() and GetShortcutString()
These were deprecated in ddd9665289 (v25.0),
and 3c1de2e667, and are no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:11:44 +01:00
Sebastiaan van Stijn
f16a2179a6 pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
These were deprecated in a3c97beee0 (v25.0.0),
and are no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:01:11 +01:00
Sebastiaan van Stijn
e2086b941f remove deprecated pkg/loopback (utility package for devicemapper)
This package was introduced in af59752712
as a utility package for devicemapper, which was removed in commit
dc11d2a2d8 (v25.0.0), and the package
was deprecated in bf692d47fb.

This patch removes the package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 00:55:09 +01:00
Albin Kerouanton
f07c45e4f2 daemon: remove --oom-score-adjust flag
This flag was marked deprecated in commit 5a922dc16 (released in v24.0)
and to be removed in the next release.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-20 00:40:28 +01:00
Sebastiaan van Stijn
307fe9c716 Dockerfile: update docker compose to v2.24.1
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.24.0...v2.24.1
- release notes: https://github.com/docker/compose/releases/tag/v2.24.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 23:27:59 +01:00
Sebastiaan van Stijn
dfced4b557 Dockerfile: update dev-shell version of the cli to v25.0.0
Update the docker CLI that's available for debugging in the dev-shell
to the v25 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 23:25:12 +01:00
Sebastiaan van Stijn
75fd873fe6 Merge pull request #47009 from cpuguy83/swarm_rotate_key_flake
De-flake TestSwarmClusterRotateUnlockKey... again... maybe?
2024-01-19 23:13:34 +01:00
voloder
c655b7dc78 Assert temp output directory is not an empty string
Signed-off-by: voloder <110066198+voloder@users.noreply.github.com>
2024-01-19 22:45:54 +01:00
Rob Murray
a8f7c5ee48 Detect IPv6 support in containers.
Some configuration in a container depends on whether it has support for
IPv6 (including default entries for '::1' etc in '/etc/hosts').

Before this change, the container's support for IPv6 was determined by
whether it was connected to any IPv6-enabled networks. But, that can
change over time, it isn't a property of the container itself.

So, instead, detect IPv6 support by looking for '::1' on the container's
loopback interface. It will not be present if the kernel does not have
IPv6 support, or the user has disabled it in new namespaces by other
means.

Once IPv6 support has been determined for the container, its '/etc/hosts'
is re-generated accordingly.

The daemon no longer disables IPv6 on all interfaces during initialisation.
It now disables IPv6 only for interfaces that have not been assigned an
IPv6 address. (But, even if IPv6 is disabled for the container using the
sysctl 'net.ipv6.conf.all.disable_ipv6=1', interfaces connected to IPv6
networks still get IPv6 addresses that appear in the internal DNS. There's
more to-do!)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-19 20:24:07 +00:00
Cory Snider
0046b16d87 daemon: set libnetwork sandbox key w/o OCI hook
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:23:12 +00:00
Sebastiaan van Stijn
31ccdbb7a8 Merge pull request #45687 from vvoland/volume-mount-subpath
volumes: Implement subpath mount
2024-01-19 18:41:12 +01:00
Paweł Gronowski
5bbcc41c20 volumes/subpath: Plumb context
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:21 +01:00
Paweł Gronowski
cb1af229f2 daemon/populateVolumes: Support volume subpath
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:20 +01:00
Paweł Gronowski
349a52b279 container: Change comment into debug log
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:18 +01:00
Paweł Gronowski
42afac91d7 internal/safepath: Add windows implementation
All components of the path are locked before the check, and
released once the path is already mounted.
This makes it impossible to replace the mounted directory until it's
actually mounted in the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:17 +01:00
Paweł Gronowski
5841ed4e5e internal/safepath: Adapt k8s openat2 fallback
Adapts the function source code to the Moby codebase.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:16 +01:00
Paweł Gronowski
56bb143a4d internal/safepath: Import k8s safeopen function
For use as a soft fallback if Openat2 is not available.
Source: 55fb1805a1/pkg/volume/util/subpath/subpath_linux.go

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:14 +01:00
Paweł Gronowski
3784316d46 internal/safepath: Handle EINTR in unix syscalls
Handle EINTR by retrying the syscall.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:13 +01:00
Paweł Gronowski
9a0cde66ba internal/safepath: Add linux implementation
All subpath components are opened with openat, relative to the base
volume directory and checked against the volume escape.
The final file descriptor is mounted from the /proc/self/fd/<fd> to a
temporary mount point owned by the daemon and then passed to the
underlying container runtime.
Temporary mountpoint is removed after the container is started.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:12 +01:00
Paweł Gronowski
bfb810445c volumes: Implement subpath mount
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume
directory.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:10 +01:00
Paweł Gronowski
f2e1105056 Introduce a helper that collects cleanup functions
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:17 +01:00
Paweł Gronowski
f07387466a daemon/oci: Extract side effects from withMounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:16 +01:00
Paweł Gronowski
9c8752505f volume/mounts: Rename errors in defer block
To make it easier to distinguish if an output variable is modified.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:15 +01:00
Sebastiaan van Stijn
17d80442cc Merge pull request #47111 from robmry/fix_TestAddRemoveInterface
Fix libnetwork/osl test TestAddRemoveInterface
2024-01-19 16:15:49 +01:00
Sebastiaan van Stijn
d35e923a4c Merge pull request #47116 from thaJeztah/minor_nits
assorted minor linting issues
2024-01-19 16:09:12 +01:00
Sebastiaan van Stijn
67de5c84d3 Merge pull request #47118 from vvoland/api-1.45
API: bump version to 1.45
2024-01-19 15:35:47 +01:00
Paweł Gronowski
5bcbedb7ee API: bump version to 1.45
Docker 25.0 was released with API v1.44, so any change in the API should
now target v1.45.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 14:42:19 +01:00
Sebastiaan van Stijn
35789fce99 daemon.images: ImageService.getImage: use named fields in struct literals
Prevent things from breaking if additional fields are added to this struct.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 13:11:40 +01:00
Sebastiaan van Stijn
7c1914411f daemon/images: ImageService.manifestMatchesPlatform: optimize logger
We constructed a "function level" logger, which was used once "as-is", but
also added additional Fields in a loop (for each resource), effectively
overwriting the previous one for each iteration. Adding additional
fields can result in some overhead, so let's construct a "logger" only for
inside the loop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 13:08:30 +01:00
Sebastiaan van Stijn
5581efe7cd rename "ociimage" var to be proper camelCase
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:50:13 +01:00
Sebastiaan van Stijn
66cf6e3a7a rename "image" vars to prevent conflicts with imports
We have many "image" packages, so these vars easily conflict/shadow
imports. Let's rename them (and in some cases use a const) to
prevent that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:49:53 +01:00
Tianon Gravi
4a40d10b60 Merge pull request #47109 from whalelines/git-url-regex
Fix isGitURL regular expression
2024-01-18 14:02:57 -08:00
Rob Murray
c72e458a72 Fix libnetwork/osl test TestAddRemoveInterface
For some time, when adding an interface with no IPv6 address (an
interface to a network that does not have IPv6 enabled), we've been
disabling IPv6 on that interface.

As part of a separate change, I'm removing that logic - there's nothing
wrong with having IPv6 enabled on an interface with no routable address.
The difference is that the kernel will assign a link-local address.

TestAddRemoveInterface does this...
- Assign an IPv6 link-local address to one end of a veth interface, and
  add it to a namespace.
- Add a bridge with no assigned IPv6 address to the namespace.
- Remove the veth interface from the namespace.
- Put the veth interface back into the namespace, still with an
  explicitly assigned IPv6 link local address.

When IPv6 is disabled on the bridge interface, the test passes.

But, when IPv6 is enabled, the bridge gets a kernel assigned link-local
address.

Then, when re-adding the veth interface, the test generates an error in
'osl/interface_linux.go:checkRouteConflict()'. The conflict is between
the explicitly assigned fe80::2 on the veth, and a route for fe80::/64
belonging to the bridge.

So, in preparation for not-disabling IPv6 on these interfaces, use a
unique-local address in the test instead of link-local.

I don't think that changes the intent of the test.

With the change to not-always disable IPv6, it is possible to repro the
problem with a real container, disconnect and re-connect a user-defined
network with '--subnet fe80::/64' while the container's connected to an
IPv4 network. So, strictly speaking, that will be a regression.

But, it's also possible to repro the problem in master, by disconnecting
and re-connecting the fe80::/64 network while another IPv6 network is
connected. So, I don't think it's a problem we need to address, perhaps
other than by prohibiting '--subnet fe80::/64'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-18 21:01:41 +00:00
David Dooling
768146b1b0 Fix isGitURL regular expression
Escape period (.) so regular expression does not match any character before "git".

Signed-off-by: David Dooling <david.dooling@docker.com>
2024-01-18 14:14:08 -06:00
Paweł Gronowski
585d74bad1 pkg/ioutils: Make subsequent Close attempts noop
Turn subsequent `Close` calls into a no-op and produce a warning with an
optional stack trace (if debug mode is enabled).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-04 11:21:04 +01:00
Brian Goff
fbdc02534a De-flake TestSwarmClusterRotateUnlockKey... again... maybe?
This hopefully makes the test less flakey (or removes any flake that
would be caused by the test itself).

1. Adds tail of cluster daemon logs when there is a test failure so we
   can more easily see what may be happening
2. Scans the daemon logs to check if the key is rotated before
   restarting the daemon. This is a little hacky but a little better
   than assuming it is done after a hard-coded 3 seconds.
3. Cleans up the `node ls` check such that it uses a poll function

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-01-04 00:18:58 +00:00
1604 changed files with 100909 additions and 34711 deletions

View File

@@ -3,11 +3,19 @@
"build": {
"context": "..",
"dockerfile": "../Dockerfile",
"target": "dev"
"target": "devcontainer"
},
"workspaceFolder": "/go/src/github.com/docker/docker",
"workspaceMount": "source=${localWorkspaceFolder},target=/go/src/github.com/docker/docker,type=bind,consistency=cached",
"remoteUser": "root",
"runArgs": ["--privileged"]
"runArgs": ["--privileged"],
"customizations": {
"vscode": {
"extensions": [
"golang.go"
]
}
}
}

View File

@@ -22,9 +22,12 @@ Please provide the following information:
**- Description for the changelog**
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
pull request for inclusion in the changelog.
It must be placed inside the below triple backticks section:
-->
```markdown changelog
```
**- A picture of a cute animal (not mandatory but encouraged)**

View File

@@ -12,7 +12,7 @@ on:
default: "graphdriver"
env:
GO_VERSION: "1.21.8"
GO_VERSION: "1.21.9"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
ITG_CLI_MATRIX_SIZE: 6
@@ -70,6 +70,7 @@ jobs:
with:
name: test-reports-unit-${{ inputs.storage }}
path: /tmp/reports/*
retention-days: 1
unit-report:
runs-on: ubuntu-20.04
@@ -150,6 +151,7 @@ jobs:
with:
name: test-reports-docker-py-${{ inputs.storage }}
path: /tmp/reports/*
retention-days: 1
integration-flaky:
runs-on: ubuntu-20.04
@@ -271,6 +273,7 @@ jobs:
with:
name: test-reports-integration-${{ inputs.storage }}-${{ env.TESTREPORTS_NAME }}
path: /tmp/reports/*
retention-days: 1
integration-report:
runs-on: ubuntu-20.04
@@ -410,6 +413,7 @@ jobs:
with:
name: test-reports-integration-cli-${{ inputs.storage }}-${{ env.TESTREPORTS_NAME }}
path: /tmp/reports/*
retention-days: 1
integration-cli-report:
runs-on: ubuntu-20.04

View File

@@ -19,7 +19,7 @@ on:
default: false
env:
GO_VERSION: "1.21.8"
GO_VERSION: "1.21.10"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
WINDOWS_BASE_IMAGE: mcr.microsoft.com/windows/servercore
@@ -190,6 +190,7 @@ jobs:
with:
name: ${{ inputs.os }}-${{ inputs.storage }}-unit-reports
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
retention-days: 1
unit-test-report:
runs-on: ubuntu-latest
@@ -508,6 +509,7 @@ jobs:
with:
name: ${{ inputs.os }}-${{ inputs.storage }}-integration-reports-${{ matrix.runtime }}-${{ env.TESTREPORTS_NAME }}
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
retention-days: 1
integration-test-report:
runs-on: ubuntu-latest

View File

@@ -13,7 +13,7 @@ on:
pull_request:
env:
GO_VERSION: "1.21.8"
GO_VERSION: "1.21.10"
DESTDIR: ./build
jobs:
@@ -50,6 +50,9 @@ jobs:
timeout-minutes: 120
needs:
- build
env:
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildkit-tests"
strategy:
fail-fast: false
matrix:
@@ -115,6 +118,14 @@ jobs:
sudo service docker restart
docker version
docker info
-
name: Build test image
uses: docker/bake-action@v4
with:
workdir: ./buildkit
targets: integration-tests
set: |
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
run: |

View File

@@ -51,14 +51,6 @@ jobs:
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.target }}
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 7
prepare-cross:
runs-on: ubuntu-latest
@@ -119,11 +111,3 @@ jobs:
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: cross-${{ env.PLATFORM_PAIR }}
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 7

View File

@@ -13,7 +13,9 @@ on:
pull_request:
env:
GO_VERSION: "1.21.8"
GO_VERSION: "1.21.10"
GIT_PAGER: "cat"
PAGER: "cat"
jobs:
validate-dco:

62
.github/workflows/validate-pr.yml vendored Normal file
View File

@@ -0,0 +1,62 @@
name: validate-pr
on:
pull_request:
types: [opened, edited, labeled, unlabeled]
jobs:
check-area-label:
runs-on: ubuntu-20.04
steps:
- name: Missing `area/` label
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/') && !contains(join(github.event.pull_request.labels.*.name, ','), 'area/')
run: |
echo "::error::Every PR with an 'impact/*' label should also have an 'area/*' label"
exit 1
- name: OK
run: exit 0
check-changelog:
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/')
runs-on: ubuntu-20.04
env:
PR_BODY: |
${{ github.event.pull_request.body }}
steps:
- name: Check changelog description
run: |
# Extract the `markdown changelog` note code block
block=$(echo -n "$PR_BODY" | tr -d '\r' | awk '/^```markdown changelog$/{flag=1;next}/^```$/{flag=0}flag')
# Strip empty lines
desc=$(echo "$block" | awk NF)
if [ -z "$desc" ]; then
echo "::error::Changelog section is empty. Please provide a description for the changelog."
exit 1
fi
len=$(echo -n "$desc" | wc -c)
if [[ $len -le 6 ]]; then
echo "::error::Description looks too short: $desc"
exit 1
fi
echo "This PR will be included in the release notes with the following note:"
echo "$desc"
check-pr-branch:
runs-on: ubuntu-20.04
env:
PR_TITLE: ${{ github.event.pull_request.title }}
steps:
# Backports or PR that target a release branch directly should mention the target branch in the title, for example:
# [X.Y backport] Some change that needs backporting to X.Y
# [X.Y] Change directly targeting the X.Y branch
- name: Get branch from PR title
id: title_branch
run: echo "$PR_TITLE" | sed -n 's/^\[\([0-9]*\.[0-9]*\)[^]]*\].*/branch=\1/p' >> $GITHUB_OUTPUT
- name: Check release branch
if: github.event.pull_request.base.ref != steps.title_branch.outputs.branch && !(github.event.pull_request.base.ref == 'master' && steps.title_branch.outputs.branch == '')
run: echo "::error::PR title suggests targetting the ${{ steps.title_branch.outputs.branch }} branch, but is opened against ${{ github.event.pull_request.base.ref }}" && exit 1

View File

@@ -1,6 +1,7 @@
linters:
enable:
- depguard
- dupword # Checks for duplicate words in the source code.
- goimports
- gosec
- gosimple
@@ -25,6 +26,11 @@ linters:
- docs
linters-settings:
dupword:
ignore:
- "true" # some tests use this as expected output
- "false" # some tests use this as expected output
- "root" # for tests using "ls" output with files owned by "root:root"
importas:
# Do not allow unaliased imports of aliased packages.
no-unaliased: true

View File

@@ -1,19 +1,19 @@
# syntax=docker/dockerfile:1
ARG GO_VERSION=1.21.8
ARG GO_VERSION=1.21.10
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
ARG XX_VERSION=1.2.1
ARG XX_VERSION=1.4.0
ARG VPNKIT_VERSION=0.5.0
ARG DOCKERCLI_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_VERSION=v25.0.2
ARG DOCKERCLI_VERSION=v26.0.0
# cli version used for integration-cli tests
ARG DOCKERCLI_INTEGRATION_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_INTEGRATION_VERSION=v17.06.2-ce
ARG BUILDX_VERSION=0.12.1
ARG COMPOSE_VERSION=v2.24.5
ARG BUILDX_VERSION=0.13.1
ARG COMPOSE_VERSION=v2.25.0
ARG SYSTEMD="false"
ARG DOCKER_STATIC=1
@@ -198,7 +198,7 @@ RUN git init . && git remote add origin "https://github.com/containerd/container
# When updating the binary version you may also need to update the vendor
# version to pick up bug fixes or new APIs, however, usually the Go packages
# are built from a commit from the master branch.
ARG CONTAINERD_VERSION=v1.7.13
ARG CONTAINERD_VERSION=v1.7.15
RUN git fetch -q --depth 1 origin "${CONTAINERD_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerd-build
@@ -245,12 +245,18 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
&& /build/gotestsum --version
FROM base AS shfmt
ARG SHFMT_VERSION=v3.6.0
ARG SHFMT_VERSION=v3.8.0
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "mvdan.cc/sh/v3/cmd/shfmt@${SHFMT_VERSION}" \
&& /build/shfmt --version
FROM base AS gopls
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "golang.org/x/tools/gopls@latest" \
&& /build/gopls version
FROM base AS dockercli
WORKDIR /go/src/github.com/docker/cli
ARG DOCKERCLI_REPOSITORY
@@ -655,6 +661,11 @@ RUN <<EOT
docker-proxy --version
EOT
# devcontainer is a stage used by .devcontainer/devcontainer.json
FROM dev-base AS devcontainer
COPY --link . .
COPY --link --from=gopls /build/ /usr/local/bin/
# usage:
# > make shell
# > SYSTEMD=true make shell

View File

@@ -5,7 +5,7 @@
# This represents the bare minimum required to build and test Docker.
ARG GO_VERSION=1.21.8
ARG GO_VERSION=1.21.10
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"

View File

@@ -161,10 +161,10 @@ FROM ${WINDOWS_BASE_IMAGE}:${WINDOWS_BASE_IMAGE_TAG}
# Use PowerShell as the default shell
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG GO_VERSION=1.21.8
ARG GO_VERSION=1.21.10
ARG GOTESTSUM_VERSION=v1.8.2
ARG GOWINRES_VERSION=v0.3.1
ARG CONTAINERD_VERSION=v1.7.13
ARG CONTAINERD_VERSION=v1.7.15
# Environment variable notes:
# - GO_VERSION must be consistent with 'Dockerfile' used by Linux.

View File

@@ -16,6 +16,9 @@ export VALIDATE_REPO
export VALIDATE_BRANCH
export VALIDATE_ORIGIN_BRANCH
export PAGER
export GIT_PAGER
# env vars passed through directly to Docker's build scripts
# to allow things like `make KEEPBUNDLE=1 binary` easily
# `project/PACKAGERS.md` have some limited documentation of some of these
@@ -77,6 +80,8 @@ DOCKER_ENVS := \
-e DEFAULT_PRODUCT_LICENSE \
-e PRODUCT \
-e PACKAGER_NAME \
-e PAGER \
-e GIT_PAGER \
-e OTEL_EXPORTER_OTLP_ENDPOINT \
-e OTEL_EXPORTER_OTLP_PROTOCOL \
-e OTEL_SERVICE_NAME
@@ -250,9 +255,10 @@ swagger-docs: ## preview the API documentation
.PHONY: generate-files
generate-files:
$(eval $@_TMP_OUT := $(shell mktemp -d -t moby-output.XXXXXXXXXX))
ifeq ($($@_TMP_OUT),)
$(error Could not create temp directory.)
endif
@if [ -z "$($@_TMP_OUT)" ]; then \
echo "Temp dir is not set"; \
exit 1; \
fi
$(BUILD_CMD) --target "update" \
--output "type=local,dest=$($@_TMP_OUT)" \
--file "./hack/dockerfiles/generate-files.Dockerfile" .

View File

@@ -2,8 +2,17 @@ package api // import "github.com/docker/docker/api"
// Common constants for daemon and client.
const (
// DefaultVersion of Current REST API
DefaultVersion = "1.44"
// DefaultVersion of the current REST API.
DefaultVersion = "1.45"
// MinSupportedAPIVersion is the minimum API version that can be supported
// by the API server, specified as "major.minor". Note that the daemon
// may be configured with a different minimum API version, as returned
// in [github.com/docker/docker/api/types.Version.MinAPIVersion].
//
// API requests for API versions lower than the configured version produce
// an error.
MinSupportedAPIVersion = "1.24"
// NoBaseImageSpecifier is the symbol used by the FROM
// command to specify that no base image is to be used.

View File

@@ -1,34 +0,0 @@
package server
import (
"net/http"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/versions"
"github.com/gorilla/mux"
"google.golang.org/grpc/status"
)
// makeErrorHandler makes an HTTP handler that decodes a Docker error and
// returns it in the response.
func makeErrorHandler(err error) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
statusCode := httpstatus.FromError(err)
vars := mux.Vars(r)
if apiVersionSupportsJSONErrors(vars["version"]) {
response := &types.ErrorResponse{
Message: err.Error(),
}
_ = httputils.WriteJSON(w, statusCode, response)
} else {
http.Error(w, status.Convert(err).Message(), statusCode)
}
}
}
func apiVersionSupportsJSONErrors(version string) bool {
const firstAPIVersionWithJSONErrors = "1.23"
return version == "" || versions.GreaterThan(version, firstAPIVersionWithJSONErrors)
}

View File

@@ -12,5 +12,4 @@ import (
// container configuration.
type ContainerDecoder interface {
DecodeConfig(src io.Reader) (*container.Config, *container.HostConfig, *network.NetworkingConfig, error)
DecodeHostConfig(src io.Reader) (*container.HostConfig, error)
}

View File

@@ -6,6 +6,7 @@ import (
"net/http"
"runtime"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/versions"
)
@@ -13,19 +14,40 @@ import (
// VersionMiddleware is a middleware that
// validates the client and server versions.
type VersionMiddleware struct {
serverVersion string
defaultVersion string
minVersion string
serverVersion string
// defaultAPIVersion is the default API version provided by the API server,
// specified as "major.minor". It is usually configured to the latest API
// version [github.com/docker/docker/api.DefaultVersion].
//
// API requests for API versions greater than this version are rejected by
// the server and produce a [versionUnsupportedError].
defaultAPIVersion string
// minAPIVersion is the minimum API version provided by the API server,
// specified as "major.minor".
//
// API requests for API versions lower than this version are rejected by
// the server and produce a [versionUnsupportedError].
minAPIVersion string
}
// NewVersionMiddleware creates a new VersionMiddleware
// with the default versions.
func NewVersionMiddleware(s, d, m string) VersionMiddleware {
return VersionMiddleware{
serverVersion: s,
defaultVersion: d,
minVersion: m,
// NewVersionMiddleware creates a VersionMiddleware with the given versions.
func NewVersionMiddleware(serverVersion, defaultAPIVersion, minAPIVersion string) (*VersionMiddleware, error) {
if versions.LessThan(defaultAPIVersion, api.MinSupportedAPIVersion) || versions.GreaterThan(defaultAPIVersion, api.DefaultVersion) {
return nil, fmt.Errorf("invalid default API version (%s): must be between %s and %s", defaultAPIVersion, api.MinSupportedAPIVersion, api.DefaultVersion)
}
if versions.LessThan(minAPIVersion, api.MinSupportedAPIVersion) || versions.GreaterThan(minAPIVersion, api.DefaultVersion) {
return nil, fmt.Errorf("invalid minimum API version (%s): must be between %s and %s", minAPIVersion, api.MinSupportedAPIVersion, api.DefaultVersion)
}
if versions.GreaterThan(minAPIVersion, defaultAPIVersion) {
return nil, fmt.Errorf("invalid API version: the minimum API version (%s) is higher than the default version (%s)", minAPIVersion, defaultAPIVersion)
}
return &VersionMiddleware{
serverVersion: serverVersion,
defaultAPIVersion: defaultAPIVersion,
minAPIVersion: minAPIVersion,
}, nil
}
type versionUnsupportedError struct {
@@ -45,18 +67,18 @@ func (e versionUnsupportedError) InvalidParameter() {}
func (v VersionMiddleware) WrapHandler(handler func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error) func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
w.Header().Set("Server", fmt.Sprintf("Docker/%s (%s)", v.serverVersion, runtime.GOOS))
w.Header().Set("API-Version", v.defaultVersion)
w.Header().Set("API-Version", v.defaultAPIVersion)
w.Header().Set("OSType", runtime.GOOS)
apiVersion := vars["version"]
if apiVersion == "" {
apiVersion = v.defaultVersion
apiVersion = v.defaultAPIVersion
}
if versions.LessThan(apiVersion, v.minVersion) {
return versionUnsupportedError{version: apiVersion, minVersion: v.minVersion}
if versions.LessThan(apiVersion, v.minAPIVersion) {
return versionUnsupportedError{version: apiVersion, minVersion: v.minAPIVersion}
}
if versions.GreaterThan(apiVersion, v.defaultVersion) {
return versionUnsupportedError{version: apiVersion, maxVersion: v.defaultVersion}
if versions.GreaterThan(apiVersion, v.defaultAPIVersion) {
return versionUnsupportedError{version: apiVersion, maxVersion: v.defaultAPIVersion}
}
ctx = context.WithValue(ctx, httputils.APIVersionKey{}, apiVersion)
return handler(ctx, w, r, vars)

View File

@@ -2,27 +2,82 @@ package middleware // import "github.com/docker/docker/api/server/middleware"
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"runtime"
"testing"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestNewVersionMiddlewareValidation(t *testing.T) {
tests := []struct {
doc, defaultVersion, minVersion, expectedErr string
}{
{
doc: "defaults",
defaultVersion: api.DefaultVersion,
minVersion: api.MinSupportedAPIVersion,
},
{
doc: "invalid default lower than min",
defaultVersion: api.MinSupportedAPIVersion,
minVersion: api.DefaultVersion,
expectedErr: fmt.Sprintf("invalid API version: the minimum API version (%s) is higher than the default version (%s)", api.DefaultVersion, api.MinSupportedAPIVersion),
},
{
doc: "invalid default too low",
defaultVersion: "0.1",
minVersion: api.MinSupportedAPIVersion,
expectedErr: fmt.Sprintf("invalid default API version (0.1): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid default too high",
defaultVersion: "9999.9999",
minVersion: api.DefaultVersion,
expectedErr: fmt.Sprintf("invalid default API version (9999.9999): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid minimum too low",
defaultVersion: api.MinSupportedAPIVersion,
minVersion: "0.1",
expectedErr: fmt.Sprintf("invalid minimum API version (0.1): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid minimum too high",
defaultVersion: api.DefaultVersion,
minVersion: "9999.9999",
expectedErr: fmt.Sprintf("invalid minimum API version (9999.9999): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.doc, func(t *testing.T) {
_, err := NewVersionMiddleware("1.2.3", tc.defaultVersion, tc.minVersion)
if tc.expectedErr == "" {
assert.Check(t, err)
} else {
assert.Check(t, is.Error(err, tc.expectedErr))
}
})
}
}
func TestVersionMiddlewareVersion(t *testing.T) {
defaultVersion := "1.10.0"
minVersion := "1.2.0"
expectedVersion := defaultVersion
expectedVersion := "<not set>"
handler := func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
v := httputils.VersionFromContext(ctx)
assert.Check(t, is.Equal(expectedVersion, v))
return nil
}
m := NewVersionMiddleware(defaultVersion, defaultVersion, minVersion)
m, err := NewVersionMiddleware("1.2.3", api.DefaultVersion, api.MinSupportedAPIVersion)
assert.NilError(t, err)
h := m.WrapHandler(handler)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
@@ -35,19 +90,19 @@ func TestVersionMiddlewareVersion(t *testing.T) {
errString string
}{
{
expectedVersion: "1.10.0",
expectedVersion: api.DefaultVersion,
},
{
reqVersion: "1.9.0",
expectedVersion: "1.9.0",
reqVersion: api.MinSupportedAPIVersion,
expectedVersion: api.MinSupportedAPIVersion,
},
{
reqVersion: "0.1",
errString: "client version 0.1 is too old. Minimum supported API version is 1.2.0, please upgrade your client to a newer version",
errString: fmt.Sprintf("client version 0.1 is too old. Minimum supported API version is %s, please upgrade your client to a newer version", api.MinSupportedAPIVersion),
},
{
reqVersion: "9999.9999",
errString: "client version 9999.9999 is too new. Maximum supported API version is 1.10.0",
errString: fmt.Sprintf("client version 9999.9999 is too new. Maximum supported API version is %s", api.DefaultVersion),
},
}
@@ -71,9 +126,8 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
return nil
}
defaultVersion := "1.10.0"
minVersion := "1.2.0"
m := NewVersionMiddleware(defaultVersion, defaultVersion, minVersion)
m, err := NewVersionMiddleware("1.2.3", api.DefaultVersion, api.MinSupportedAPIVersion)
assert.NilError(t, err)
h := m.WrapHandler(handler)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
@@ -81,12 +135,12 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
ctx := context.Background()
vars := map[string]string{"version": "0.1"}
err := h(ctx, resp, req, vars)
err = h(ctx, resp, req, vars)
assert.Check(t, is.ErrorContains(err, ""))
hdr := resp.Result().Header
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/"+defaultVersion))
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/1.2.3"))
assert.Check(t, is.Contains(hdr.Get("Server"), runtime.GOOS))
assert.Check(t, is.Equal(hdr.Get("API-Version"), defaultVersion))
assert.Check(t, is.Equal(hdr.Get("API-Version"), api.DefaultVersion))
assert.Check(t, is.Equal(hdr.Get("OSType"), runtime.GOOS))
}

View File

@@ -42,6 +42,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
SuppressOutput: httputils.BoolValue(r, "q"),
NoCache: httputils.BoolValue(r, "nocache"),
ForceRemove: httputils.BoolValue(r, "forcerm"),
PullParent: httputils.BoolValue(r, "pull"),
MemorySwap: httputils.Int64ValueOrZero(r, "memswap"),
Memory: httputils.Int64ValueOrZero(r, "memory"),
CPUShares: httputils.Int64ValueOrZero(r, "cpushares"),
@@ -66,17 +67,14 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
return nil, invalidParam{errors.New("security options are not supported on " + runtime.GOOS)}
}
version := httputils.VersionFromContext(ctx)
if httputils.BoolValue(r, "forcerm") && versions.GreaterThanOrEqualTo(version, "1.12") {
if httputils.BoolValue(r, "forcerm") {
options.Remove = true
} else if r.FormValue("rm") == "" && versions.GreaterThanOrEqualTo(version, "1.12") {
} else if r.FormValue("rm") == "" {
options.Remove = true
} else {
options.Remove = httputils.BoolValue(r, "rm")
}
if httputils.BoolValue(r, "pull") && versions.GreaterThanOrEqualTo(version, "1.16") {
options.PullParent = true
}
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.32") {
options.Platform = r.FormValue("platform")
}

View File

@@ -24,7 +24,6 @@ type execBackend interface {
// copyBackend includes functions to implement to provide container copy functionality.
type copyBackend interface {
ContainerArchivePath(name string, path string) (content io.ReadCloser, stat *types.ContainerPathStat, err error)
ContainerCopy(name string, res string) (io.ReadCloser, error)
ContainerExport(ctx context.Context, name string, out io.Writer) error
ContainerExtractToDir(name, path string, copyUIDGID, noOverwriteDirNonDir bool, content io.Reader) error
ContainerStatPath(name string, path string) (stat *types.ContainerPathStat, err error)
@@ -39,7 +38,7 @@ type stateBackend interface {
ContainerResize(name string, height, width int) error
ContainerRestart(ctx context.Context, name string, options container.StopOptions) error
ContainerRm(name string, config *backend.ContainerRmConfig) error
ContainerStart(ctx context.Context, name string, hostConfig *container.HostConfig, checkpoint string, checkpointDir string) error
ContainerStart(ctx context.Context, name string, checkpoint string, checkpointDir string) error
ContainerStop(ctx context.Context, name string, options container.StopOptions) error
ContainerUnpause(name string) error
ContainerUpdate(name string, hostConfig *container.HostConfig) (container.ContainerUpdateOKBody, error)

View File

@@ -56,7 +56,6 @@ func (r *containerRouter) initRoutes() {
router.NewPostRoute("/containers/{name:.*}/wait", r.postContainersWait),
router.NewPostRoute("/containers/{name:.*}/resize", r.postContainersResize),
router.NewPostRoute("/containers/{name:.*}/attach", r.postContainersAttach),
router.NewPostRoute("/containers/{name:.*}/copy", r.postContainersCopy), // Deprecated since 1.8 (API v1.20), errors out since 1.12 (API v1.24)
router.NewPostRoute("/containers/{name:.*}/exec", r.postContainerExecCreate),
router.NewPostRoute("/exec/{name:.*}/start", r.postContainerExecStart),
router.NewPostRoute("/exec/{name:.*}/resize", r.postContainerExecResize),

View File

@@ -39,13 +39,6 @@ func (s *containerRouter) postCommit(ctx context.Context, w http.ResponseWriter,
return err
}
// TODO: remove pause arg, and always pause in backend
pause := httputils.BoolValue(r, "pause")
version := httputils.VersionFromContext(ctx)
if r.FormValue("pause") == "" && versions.GreaterThanOrEqualTo(version, "1.13") {
pause = true
}
config, _, _, err := s.decoder.DecodeConfig(r.Body)
if err != nil && !errors.Is(err, io.EOF) { // Do not fail if body is empty.
return err
@@ -57,7 +50,7 @@ func (s *containerRouter) postCommit(ctx context.Context, w http.ResponseWriter,
}
imgID, err := s.backend.CreateImageFromContainer(ctx, r.Form.Get("container"), &backend.CreateImageConfig{
Pause: pause,
Pause: httputils.BoolValueOrDefault(r, "pause", true), // TODO(dnephin): remove pause arg, and always pause in backend
Tag: ref,
Author: r.Form.Get("author"),
Comment: r.Form.Get("comment"),
@@ -118,14 +111,11 @@ func (s *containerRouter) getContainersStats(ctx context.Context, w http.Respons
oneShot = httputils.BoolValueOrDefault(r, "one-shot", false)
}
config := &backend.ContainerStatsConfig{
return s.backend.ContainerStats(ctx, vars["name"], &backend.ContainerStatsConfig{
Stream: stream,
OneShot: oneShot,
OutStream: w,
Version: httputils.VersionFromContext(ctx),
}
return s.backend.ContainerStats(ctx, vars["name"], config)
})
}
func (s *containerRouter) getContainersLogs(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -178,14 +168,6 @@ func (s *containerRouter) getContainersExport(ctx context.Context, w http.Respon
return s.backend.ContainerExport(ctx, vars["name"], w)
}
type bodyOnStartError struct{}
func (bodyOnStartError) Error() string {
return "starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"
}
func (bodyOnStartError) InvalidParameter() {}
func (s *containerRouter) postContainersStart(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
// If contentLength is -1, we can assumed chunked encoding
// or more technically that the length is unknown
@@ -193,33 +175,17 @@ func (s *containerRouter) postContainersStart(ctx context.Context, w http.Respon
// net/http otherwise seems to swallow any headers related to chunked encoding
// including r.TransferEncoding
// allow a nil body for backwards compatibility
version := httputils.VersionFromContext(ctx)
var hostConfig *container.HostConfig
//
// A non-nil json object is at least 7 characters.
if r.ContentLength > 7 || r.ContentLength == -1 {
if versions.GreaterThanOrEqualTo(version, "1.24") {
return bodyOnStartError{}
}
if err := httputils.CheckForJSON(r); err != nil {
return err
}
c, err := s.decoder.DecodeHostConfig(r.Body)
if err != nil {
return err
}
hostConfig = c
return errdefs.InvalidParameter(errors.New("starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"))
}
if err := httputils.ParseForm(r); err != nil {
return err
}
checkpoint := r.Form.Get("checkpoint")
checkpointDir := r.Form.Get("checkpoint-dir")
if err := s.backend.ContainerStart(ctx, vars["name"], hostConfig, checkpoint, checkpointDir); err != nil {
if err := s.backend.ContainerStart(ctx, vars["name"], r.Form.Get("checkpoint"), r.Form.Get("checkpoint-dir")); err != nil {
return err
}
@@ -255,25 +221,14 @@ func (s *containerRouter) postContainersStop(ctx context.Context, w http.Respons
return nil
}
func (s *containerRouter) postContainersKill(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *containerRouter) postContainersKill(_ context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
name := vars["name"]
if err := s.backend.ContainerKill(name, r.Form.Get("signal")); err != nil {
var isStopped bool
if errdefs.IsConflict(err) {
isStopped = true
}
// Return error that's not caused because the container is stopped.
// Return error if the container is not running and the api is >= 1.20
// to keep backwards compatibility.
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.20") || !isStopped {
return errors.Wrapf(err, "Cannot kill container: %s", name)
}
return errors.Wrapf(err, "cannot kill container: %s", name)
}
w.WriteHeader(http.StatusNoContent)
@@ -512,7 +467,6 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
}
version := httputils.VersionFromContext(ctx)
adjustCPUShares := versions.LessThan(version, "1.19")
// When using API 1.24 and under, the client is responsible for removing the container
if versions.LessThan(version, "1.25") {
@@ -638,6 +592,14 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
}
}
if versions.LessThan(version, "1.45") {
for _, m := range hostConfig.Mounts {
if m.VolumeOptions != nil && m.VolumeOptions.Subpath != "" {
return errdefs.InvalidParameter(errors.New("VolumeOptions.Subpath needs API v1.45 or newer"))
}
}
}
var warnings []string
if warn, err := handleMACAddressBC(config, hostConfig, networkingConfig, version); err != nil {
return err
@@ -658,7 +620,6 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
Config: config,
HostConfig: hostConfig,
NetworkingConfig: networkingConfig,
AdjustCPUShares: adjustCPUShares,
Platform: platform,
DefaultReadOnlyNonRecursive: defaultReadOnlyNonRecursive,
})

View File

@@ -11,49 +11,10 @@ import (
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/versions"
gddohttputil "github.com/golang/gddo/httputil"
)
type pathError struct{}
func (pathError) Error() string {
return "Path cannot be empty"
}
func (pathError) InvalidParameter() {}
// postContainersCopy is deprecated in favor of getContainersArchive.
//
// Deprecated since 1.8 (API v1.20), errors out since 1.12 (API v1.24)
func (s *containerRouter) postContainersCopy(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.24") {
w.WriteHeader(http.StatusNotFound)
return nil
}
cfg := types.CopyConfig{}
if err := httputils.ReadJSON(r, &cfg); err != nil {
return err
}
if cfg.Resource == "" {
return pathError{}
}
data, err := s.backend.ContainerCopy(vars["name"], cfg.Resource)
if err != nil {
return err
}
defer data.Close()
w.Header().Set("Content-Type", "application/x-tar")
_, err = io.Copy(w, data)
return err
}
// // Encode the stat to JSON, base64 encode, and place in a header.
// setContainerPathStatHeader encodes the stat to JSON, base64 encode, and place in a header.
func setContainerPathStatHeader(stat *types.ContainerPathStat, header http.Header) error {
statJSON, err := json.Marshal(stat)
if err != nil {

View File

@@ -71,15 +71,6 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
return err
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.22") {
// API versions before 1.22 did not enforce application/json content-type.
// Allow older clients to work by patching the content-type.
if r.Header.Get("Content-Type") != "application/json" {
r.Header.Set("Content-Type", "application/json")
}
}
var (
execName = vars["name"]
stdin, inStream io.ReadCloser
@@ -96,6 +87,8 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
}
if execStartCheck.ConsoleSize != nil {
version := httputils.VersionFromContext(ctx)
// Not supported before 1.42
if versions.LessThan(version, "1.42") {
execStartCheck.ConsoleSize = nil

View File

@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"net/http"
"os"
"github.com/distribution/reference"
"github.com/docker/distribution"
@@ -12,6 +13,7 @@ import (
"github.com/docker/distribution/manifest/schema2"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/registry"
distributionpkg "github.com/docker/docker/distribution"
"github.com/docker/docker/errdefs"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -24,10 +26,10 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
w.Header().Set("Content-Type", "application/json")
image := vars["name"]
imgName := vars["name"]
// TODO why is reference.ParseAnyReference() / reference.ParseNormalizedNamed() not using the reference.ErrTagInvalidFormat (and so on) errors?
ref, err := reference.ParseAnyReference(image)
ref, err := reference.ParseAnyReference(imgName)
if err != nil {
return errdefs.InvalidParameter(err)
}
@@ -37,7 +39,7 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
// full image ID
return errors.Errorf("no manifest found for full image ID")
}
return errdefs.InvalidParameter(errors.Errorf("unknown image reference format: %s", image))
return errdefs.InvalidParameter(errors.Errorf("unknown image reference format: %s", imgName))
}
// For a search it is not an error if no auth was given. Ignore invalid
@@ -153,6 +155,9 @@ func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distrib
}
}
case *schema1.SignedManifest:
if os.Getenv("DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE") == "" {
return registry.DistributionInspect{}, distributionpkg.DeprecatedSchema1ImageError(namedRef)
}
platform := ocispec.Platform{
Architecture: mnfstObj.Architecture,
OS: "linux",

View File

@@ -21,7 +21,7 @@ type grpcRouter struct {
// NewRouter initializes a new grpc http router
func NewRouter(backends ...Backend) router.Router {
unary := grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(unaryInterceptor(), grpcerrors.UnaryServerInterceptor))
stream := grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(otelgrpc.StreamServerInterceptor(), grpcerrors.StreamServerInterceptor))
stream := grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(otelgrpc.StreamServerInterceptor(), grpcerrors.StreamServerInterceptor)) //nolint:staticcheck // TODO(thaJeztah): ignore SA1019 for deprecated options: see https://github.com/moby/moby/issues/47437
r := &grpcRouter{
h2Server: &http2.Server{},
@@ -46,7 +46,7 @@ func (gr *grpcRouter) initRoutes() {
}
func unaryInterceptor() grpc.UnaryServerInterceptor {
withTrace := otelgrpc.UnaryServerInterceptor()
withTrace := otelgrpc.UnaryServerInterceptor() //nolint:staticcheck // TODO(thaJeztah): ignore SA1019 for deprecated options: see https://github.com/moby/moby/issues/47437
return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
// This method is used by the clients to send their traces to buildkit so they can be included

View File

@@ -6,6 +6,7 @@ import (
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
@@ -24,8 +25,8 @@ type Backend interface {
type imageBackend interface {
ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]image.DeleteResponse, error)
ImageHistory(ctx context.Context, imageName string) ([]*image.HistoryResponseItem, error)
Images(ctx context.Context, opts types.ImageListOptions) ([]*image.Summary, error)
GetImage(ctx context.Context, refOrID string, options image.GetImageOpts) (*dockerimage.Image, error)
Images(ctx context.Context, opts image.ListOptions) ([]*image.Summary, error)
GetImage(ctx context.Context, refOrID string, options backend.GetImageOpts) (*dockerimage.Image, error)
TagImage(ctx context.Context, id dockerimage.ID, newRef reference.Named) error
ImagesPrune(ctx context.Context, pruneFilters filters.Args) (*types.ImagesPruneReport, error)
}

View File

@@ -15,8 +15,9 @@ import (
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
opts "github.com/docker/docker/api/types/image"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/builder/remotecontext"
@@ -72,9 +73,9 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
// Special case: "pull -a" may send an image name with a
// trailing :. This is ugly, but let's not break API
// compatibility.
image := strings.TrimSuffix(img, ":")
imgName := strings.TrimSuffix(img, ":")
ref, err := reference.ParseNormalizedNamed(image)
ref, err := reference.ParseNormalizedNamed(imgName)
if err != nil {
return errdefs.InvalidParameter(err)
}
@@ -189,7 +190,7 @@ func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter
var ref reference.Named
// Tag is empty only in case ImagePushOptions.All is true.
// Tag is empty only in case PushOptions.All is true.
if tag != "" {
r, err := httputils.RepoTagReference(img, tag)
if err != nil {
@@ -285,7 +286,7 @@ func (ir *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter,
}
func (ir *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
img, err := ir.backend.GetImage(ctx, vars["name"], opts.GetImageOpts{Details: true})
img, err := ir.backend.GetImage(ctx, vars["name"], backend.GetImageOpts{Details: true})
if err != nil {
return err
}
@@ -305,6 +306,10 @@ func (ir *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWrite
imageInspect.Created = time.Time{}.Format(time.RFC3339Nano)
}
}
if versions.GreaterThanOrEqualTo(version, "1.45") {
imageInspect.Container = "" //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
imageInspect.ContainerConfig = nil //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
}
return httputils.WriteJSON(w, http.StatusOK, imageInspect)
}
@@ -359,7 +364,7 @@ func (ir *imageRouter) toImageInspect(img *image.Image) (*types.ImageInspect, er
Data: img.Details.Metadata,
},
RootFS: rootFSToAPIType(img.RootFS),
Metadata: opts.Metadata{
Metadata: imagetypes.Metadata{
LastTagTime: img.Details.LastUpdated,
},
}, nil
@@ -401,7 +406,7 @@ func (ir *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
sharedSize = httputils.BoolValue(r, "shared-size")
}
images, err := ir.backend.Images(ctx, types.ImageListOptions{
images, err := ir.backend.Images(ctx, imagetypes.ListOptions{
All: httputils.BoolValue(r, "all"),
Filters: imageFilters,
SharedSize: sharedSize,
@@ -458,7 +463,7 @@ func (ir *imageRouter) postImagesTag(ctx context.Context, w http.ResponseWriter,
return errdefs.InvalidParameter(errors.New("refusing to create an ambiguous tag using digest algorithm as name"))
}
img, err := ir.backend.GetImage(ctx, vars["name"], opts.GetImageOpts{})
img, err := ir.backend.GetImage(ctx, vars["name"], backend.GetImageOpts{})
if err != nil {
return errdefs.NotFound(err)
}

View File

@@ -213,6 +213,10 @@ func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWr
return libnetwork.NetworkNameError(create.Name)
}
// For a Swarm-scoped network, this call to backend.CreateNetwork is used to
// validate the configuration. The network will not be created but, if the
// configuration is valid, ManagerRedirectError will be returned and handled
// below.
nw, err := n.backend.CreateNetwork(create)
if err != nil {
if _, ok := err.(libnetwork.ManagerRedirectError); !ok {

View File

@@ -10,6 +10,7 @@ import (
"github.com/docker/docker/api/server/middleware"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/server/router/debug"
"github.com/docker/docker/api/types"
"github.com/docker/docker/dockerversion"
"github.com/gorilla/mux"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
@@ -57,19 +58,13 @@ func (s *Server) makeHTTPHandler(handler httputils.APIFunc, operation string) ht
if statusCode >= 500 {
log.G(ctx).Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
}
makeErrorHandler(err)(w, r)
_ = httputils.WriteJSON(w, statusCode, &types.ErrorResponse{
Message: err.Error(),
})
}
}), operation).ServeHTTP
}
type pageNotFoundError struct{}
func (pageNotFoundError) Error() string {
return "page not found"
}
func (pageNotFoundError) NotFound() {}
// CreateMux returns a new mux with all the routers registered.
func (s *Server) CreateMux(routers ...router.Router) *mux.Router {
m := mux.NewRouter()
@@ -91,7 +86,12 @@ func (s *Server) CreateMux(routers ...router.Router) *mux.Router {
m.Path("/debug" + r.Path()).Handler(f)
}
notFoundHandler := makeErrorHandler(pageNotFoundError{})
notFoundHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = httputils.WriteJSON(w, http.StatusNotFound, &types.ErrorResponse{
Message: "page not found",
})
})
m.HandleFunc(versionMatcher+"/{path:.*}", notFoundHandler)
m.NotFoundHandler = notFoundHandler
m.MethodNotAllowedHandler = notFoundHandler

View File

@@ -15,8 +15,11 @@ import (
func TestMiddlewares(t *testing.T) {
srv := &Server{}
const apiMinVersion = "1.12"
srv.UseMiddleware(middleware.NewVersionMiddleware("0.1omega2", api.DefaultVersion, apiMinVersion))
m, err := middleware.NewVersionMiddleware("0.1omega2", api.DefaultVersion, api.MinSupportedAPIVersion)
if err != nil {
t.Fatal(err)
}
srv.UseMiddleware(*m)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
resp := httptest.NewRecorder()

View File

@@ -19,10 +19,10 @@ produces:
consumes:
- "application/json"
- "text/plain"
basePath: "/v1.44"
basePath: "/v1.45"
info:
title: "Docker Engine API"
version: "1.44"
version: "1.45"
x-logo:
url: "https://docs.docker.com/assets/images/logo-docker-main.png"
description: |
@@ -55,8 +55,8 @@ info:
the URL is not supported by the daemon, a HTTP `400 Bad Request` error message
is returned.
If you omit the version-prefix, the current version of the API (v1.44) is used.
For example, calling `/info` is the same as calling `/v1.44/info`. Using the
If you omit the version-prefix, the current version of the API (v1.45) is used.
For example, calling `/info` is the same as calling `/v1.45/info`. Using the
API without a version-prefix is deprecated and will be removed in a future release.
Engine releases in the near future should support this version of the API,
@@ -427,6 +427,10 @@ definitions:
type: "object"
additionalProperties:
type: "string"
Subpath:
description: "Source path inside the volume. Must be relative without any back traversals."
type: "string"
example: "dir-inside-volume/subdirectory"
TmpfsOptions:
description: "Optional configuration for the `tmpfs` type."
type: "object"
@@ -8770,8 +8774,7 @@ paths:
<p><br /></p>
> **Deprecated**: This field is deprecated and will always
> be "false" in future.
> **Deprecated**: This field is deprecated and will always be "false".
type: "boolean"
example: false
name:
@@ -8814,13 +8817,8 @@ paths:
description: |
A JSON encoded value of the filters (a `map[string][]string`) to process on the images list. Available filters:
- `is-automated=(true|false)` (deprecated, see below)
- `is-official=(true|false)`
- `stars=<number>` Matches images that has at least 'number' stars.
The `is-automated` filter is deprecated. The `is_automated` field has
been deprecated by Docker Hub's search API. Consequently, searching
for `is-automated=true` will yield no results.
type: "string"
tags: ["Image"]
/images/prune:

View File

@@ -18,7 +18,6 @@ type ContainerCreateConfig struct {
HostConfig *container.HostConfig
NetworkingConfig *network.NetworkingConfig
Platform *ocispec.Platform
AdjustCPUShares bool
DefaultReadOnlyNonRecursive bool
}
@@ -91,7 +90,6 @@ type ContainerStatsConfig struct {
Stream bool
OneShot bool
OutStream io.Writer
Version string
}
// ExecInspect holds information about a running process started
@@ -131,6 +129,13 @@ type CreateImageConfig struct {
Changes []string
}
// GetImageOpts holds parameters to retrieve image information
// from the backend.
type GetImageOpts struct {
Platform *ocispec.Platform
Details bool
}
// CommitConfig is the configuration for creating an image as part of a build.
type CommitConfig struct {
Author string

View File

@@ -157,42 +157,12 @@ type ImageBuildResponse struct {
OSType string
}
// ImageCreateOptions holds information to create images.
type ImageCreateOptions struct {
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry.
Platform string // Platform is the target platform of the image if it needs to be pulled from the registry.
}
// ImageImportSource holds source information for ImageImport
type ImageImportSource struct {
Source io.Reader // Source is the data to send to the server to create this image from. You must set SourceName to "-" to leverage this.
SourceName string // SourceName is the name of the image to pull. Set to "-" to leverage the Source attribute.
}
// ImageImportOptions holds information to import images from the client host.
type ImageImportOptions struct {
Tag string // Tag is the name to tag this image with. This attribute is deprecated.
Message string // Message is the message to tag the image with
Changes []string // Changes are the raw changes to apply to this image
Platform string // Platform is the target platform of the image
}
// ImageListOptions holds parameters to list images with.
type ImageListOptions struct {
// All controls whether all images in the graph are filtered, or just
// the heads.
All bool
// Filters is a JSON-encoded set of filter arguments.
Filters filters.Args
// SharedSize indicates whether the shared size of images should be computed.
SharedSize bool
// ContainerCount indicates whether container count should be computed.
ContainerCount bool
}
// ImageLoadResponse returns information to the client about a load process.
type ImageLoadResponse struct {
// Body must be closed to avoid a resource leak
@@ -200,14 +170,6 @@ type ImageLoadResponse struct {
JSON bool
}
// ImagePullOptions holds information to pull images.
type ImagePullOptions struct {
All bool
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry
PrivilegeFunc RequestPrivilegeFunc
Platform string
}
// RequestPrivilegeFunc is a function interface that
// clients can supply to retry operations after
// getting an authorization error.
@@ -216,15 +178,6 @@ type ImagePullOptions struct {
// if the privilege request fails.
type RequestPrivilegeFunc func() (string, error)
// ImagePushOptions holds information to push images.
type ImagePushOptions ImagePullOptions
// ImageRemoveOptions holds parameters to remove images.
type ImageRemoveOptions struct {
Force bool
PruneChildren bool
}
// ImageSearchOptions holds parameters to search images with.
type ImageSearchOptions struct {
RegistryAuth string

View File

@@ -5,8 +5,8 @@ import (
"time"
"github.com/docker/docker/api/types/strslice"
dockerspec "github.com/docker/docker/image/spec/specs-go/v1"
"github.com/docker/go-connections/nat"
dockerspec "github.com/moby/docker-image-spec/specs-go/v1"
)
// MinimumDuration puts a minimum on user configured duration.

View File

@@ -1,9 +1,57 @@
package image
import ocispec "github.com/opencontainers/image-spec/specs-go/v1"
import "github.com/docker/docker/api/types/filters"
// GetImageOpts holds parameters to inspect an image.
type GetImageOpts struct {
Platform *ocispec.Platform
Details bool
// ImportOptions holds information to import images from the client host.
type ImportOptions struct {
Tag string // Tag is the name to tag this image with. This attribute is deprecated.
Message string // Message is the message to tag the image with
Changes []string // Changes are the raw changes to apply to this image
Platform string // Platform is the target platform of the image
}
// CreateOptions holds information to create images.
type CreateOptions struct {
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry.
Platform string // Platform is the target platform of the image if it needs to be pulled from the registry.
}
// PullOptions holds information to pull images.
type PullOptions struct {
All bool
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry
// PrivilegeFunc is a function that clients can supply to retry operations
// after getting an authorization error. This function returns the registry
// authentication header value in base64 encoded format, or an error if the
// privilege request fails.
//
// Also see [github.com/docker/docker/api/types.RequestPrivilegeFunc].
PrivilegeFunc func() (string, error)
Platform string
}
// PushOptions holds information to push images.
type PushOptions PullOptions
// ListOptions holds parameters to list images with.
type ListOptions struct {
// All controls whether all images in the graph are filtered, or just
// the heads.
All bool
// Filters is a JSON-encoded set of filter arguments.
Filters filters.Args
// SharedSize indicates whether the shared size of images should be computed.
SharedSize bool
// ContainerCount indicates whether container count should be computed.
ContainerCount bool
}
// RemoveOptions holds parameters to remove images.
type RemoveOptions struct {
Force bool
PruneChildren bool
}

View File

@@ -96,6 +96,7 @@ type BindOptions struct {
type VolumeOptions struct {
NoCopy bool `json:",omitempty"`
Labels map[string]string `json:",omitempty"`
Subpath string `json:",omitempty"`
DriverConfig *Driver `json:",omitempty"`
}

View File

@@ -94,7 +94,7 @@ type SearchResult struct {
Name string `json:"name"`
// IsAutomated indicates whether the result is automated.
//
// Deprecated: the "is_automated" field is deprecated and will always be "false" in the future.
// Deprecated: the "is_automated" field is deprecated and will always be "false".
IsAutomated bool `json:"is_automated"`
// Description is a textual description of the repository
Description string `json:"description"`

View File

@@ -82,7 +82,7 @@ type ImageInspect struct {
// Depending on how the image was created, this field may be empty.
//
// Deprecated: this field is omitted in API v1.45, but kept for backward compatibility.
Container string
Container string `json:",omitempty"`
// ContainerConfig is an optional field containing the configuration of the
// container that was last committed when creating the image.
@@ -91,7 +91,7 @@ type ImageInspect struct {
// and it is not in active use anymore.
//
// Deprecated: this field is omitted in API v1.45, but kept for backward compatibility.
ContainerConfig *container.Config
ContainerConfig *container.Config `json:",omitempty"`
// DockerVersion is the version of Docker that was used to build the image.
//

View File

@@ -1,138 +1,35 @@
package types
import (
"github.com/docker/docker/api/types/checkpoint"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/system"
)
// CheckpointCreateOptions holds parameters to create a checkpoint from a container.
// ImageImportOptions holds information to import images from the client host.
//
// Deprecated: use [checkpoint.CreateOptions].
type CheckpointCreateOptions = checkpoint.CreateOptions
// Deprecated: use [image.ImportOptions].
type ImageImportOptions = image.ImportOptions
// CheckpointListOptions holds parameters to list checkpoints for a container
// ImageCreateOptions holds information to create images.
//
// Deprecated: use [checkpoint.ListOptions].
type CheckpointListOptions = checkpoint.ListOptions
// Deprecated: use [image.CreateOptions].
type ImageCreateOptions = image.CreateOptions
// CheckpointDeleteOptions holds parameters to delete a checkpoint from a container
// ImagePullOptions holds information to pull images.
//
// Deprecated: use [checkpoint.DeleteOptions].
type CheckpointDeleteOptions = checkpoint.DeleteOptions
// Deprecated: use [image.PullOptions].
type ImagePullOptions = image.PullOptions
// Checkpoint represents the details of a checkpoint when listing endpoints.
// ImagePushOptions holds information to push images.
//
// Deprecated: use [checkpoint.Summary].
type Checkpoint = checkpoint.Summary
// Deprecated: use [image.PushOptions].
type ImagePushOptions = image.PushOptions
// Info contains response of Engine API:
// GET "/info"
// ImageListOptions holds parameters to list images with.
//
// Deprecated: use [system.Info].
type Info = system.Info
// Deprecated: use [image.ListOptions].
type ImageListOptions = image.ListOptions
// Commit holds the Git-commit (SHA1) that a binary was built from, as reported
// in the version-string of external tools, such as containerd, or runC.
// ImageRemoveOptions holds parameters to remove images.
//
// Deprecated: use [system.Commit].
type Commit = system.Commit
// PluginsInfo is a temp struct holding Plugins name
// registered with docker daemon. It is used by [system.Info] struct
//
// Deprecated: use [system.PluginsInfo].
type PluginsInfo = system.PluginsInfo
// NetworkAddressPool is a temp struct used by [system.Info] struct.
//
// Deprecated: use [system.NetworkAddressPool].
type NetworkAddressPool = system.NetworkAddressPool
// Runtime describes an OCI runtime.
//
// Deprecated: use [system.Runtime].
type Runtime = system.Runtime
// SecurityOpt contains the name and options of a security option.
//
// Deprecated: use [system.SecurityOpt].
type SecurityOpt = system.SecurityOpt
// KeyValue holds a key/value pair.
//
// Deprecated: use [system.KeyValue].
type KeyValue = system.KeyValue
// ImageDeleteResponseItem image delete response item.
//
// Deprecated: use [image.DeleteResponse].
type ImageDeleteResponseItem = image.DeleteResponse
// ImageSummary image summary.
//
// Deprecated: use [image.Summary].
type ImageSummary = image.Summary
// ImageMetadata contains engine-local data about the image.
//
// Deprecated: use [image.Metadata].
type ImageMetadata = image.Metadata
// ServiceCreateResponse contains the information returned to a client
// on the creation of a new service.
//
// Deprecated: use [swarm.ServiceCreateResponse].
type ServiceCreateResponse = swarm.ServiceCreateResponse
// ServiceUpdateResponse service update response.
//
// Deprecated: use [swarm.ServiceUpdateResponse].
type ServiceUpdateResponse = swarm.ServiceUpdateResponse
// ContainerStartOptions holds parameters to start containers.
//
// Deprecated: use [container.StartOptions].
type ContainerStartOptions = container.StartOptions
// ResizeOptions holds parameters to resize a TTY.
// It can be used to resize container TTYs and
// exec process TTYs too.
//
// Deprecated: use [container.ResizeOptions].
type ResizeOptions = container.ResizeOptions
// ContainerAttachOptions holds parameters to attach to a container.
//
// Deprecated: use [container.AttachOptions].
type ContainerAttachOptions = container.AttachOptions
// ContainerCommitOptions holds parameters to commit changes into a container.
//
// Deprecated: use [container.CommitOptions].
type ContainerCommitOptions = container.CommitOptions
// ContainerListOptions holds parameters to list containers with.
//
// Deprecated: use [container.ListOptions].
type ContainerListOptions = container.ListOptions
// ContainerLogsOptions holds parameters to filter logs with.
//
// Deprecated: use [container.LogsOptions].
type ContainerLogsOptions = container.LogsOptions
// ContainerRemoveOptions holds parameters to remove containers.
//
// Deprecated: use [container.RemoveOptions].
type ContainerRemoveOptions = container.RemoveOptions
// DecodeSecurityOptions decodes a security options string slice to a type safe
// [system.SecurityOpt].
//
// Deprecated: use [system.DecodeSecurityOptions].
func DecodeSecurityOptions(opts []string) ([]system.SecurityOpt, error) {
return system.DecodeSecurityOptions(opts)
}
// Deprecated: use [image.RemoveOptions].
type ImageRemoveOptions = image.RemoveOptions

View File

@@ -1,14 +0,0 @@
# Legacy API type versions
This package includes types for legacy API versions. The stable version of the API types live in `api/types/*.go`.
Consider moving a type here when you need to keep backwards compatibility in the API. This legacy types are organized by the latest API version they appear in. For instance, types in the `v1p19` package are valid for API versions below or equal `1.19`. Types in the `v1p20` package are valid for the API version `1.20`, since the versions below that will use the legacy types in `v1p19`.
## Package name conventions
The package name convention is to use `v` as a prefix for the version number and `p`(patch) as a separator. We use this nomenclature due to a few restrictions in the Go package name convention:
1. We cannot use `.` because it's interpreted by the language, think of `v1.20.CallFunction`.
2. We cannot use `_` because golint complains about it. The code is actually valid, but it looks probably more weird: `v1_20.CallFunction`.
For instance, if you want to modify a type that was available in the version `1.21` of the API but it will have different fields in the version `1.22`, you want to create a new package under `api/types/versions/v1p21`.

View File

@@ -1,35 +0,0 @@
// Package v1p19 provides specific API types for the API version 1, patch 19.
package v1p19 // import "github.com/docker/docker/api/types/versions/v1p19"
import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/versions/v1p20"
"github.com/docker/go-connections/nat"
)
// ContainerJSON is a backcompatibility struct for APIs prior to 1.20.
// Note this is not used by the Windows daemon.
type ContainerJSON struct {
*types.ContainerJSONBase
Volumes map[string]string
VolumesRW map[string]bool
Config *ContainerConfig
NetworkSettings *v1p20.NetworkSettings
}
// ContainerConfig is a backcompatibility struct for APIs prior to 1.20.
type ContainerConfig struct {
*container.Config
MacAddress string
NetworkDisabled bool
ExposedPorts map[nat.Port]struct{}
// backward compatibility, they now live in HostConfig
VolumeDriver string
Memory int64
MemorySwap int64
CPUShares int64 `json:"CpuShares"`
CPUSet string `json:"Cpuset"`
}

View File

@@ -1,40 +0,0 @@
// Package v1p20 provides specific API types for the API version 1, patch 20.
package v1p20 // import "github.com/docker/docker/api/types/versions/v1p20"
import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/go-connections/nat"
)
// ContainerJSON is a backcompatibility struct for the API 1.20
type ContainerJSON struct {
*types.ContainerJSONBase
Mounts []types.MountPoint
Config *ContainerConfig
NetworkSettings *NetworkSettings
}
// ContainerConfig is a backcompatibility struct used in ContainerJSON for the API 1.20
type ContainerConfig struct {
*container.Config
MacAddress string
NetworkDisabled bool
ExposedPorts map[nat.Port]struct{}
// backward compatibility, they now live in HostConfig
VolumeDriver string
}
// StatsJSON is a backcompatibility struct used in Stats for APIs prior to 1.21
type StatsJSON struct {
types.Stats
Network types.NetworkStats `json:"network,omitempty"`
}
// NetworkSettings is a backward compatible struct for APIs prior to 1.21
type NetworkSettings struct {
types.NetworkSettingsBase
types.DefaultNetworkSettings
}

View File

@@ -238,13 +238,13 @@ type TopologyRequirement struct {
// If requisite is specified, all topologies in preferred list MUST
// also be present in the list of requisite topologies.
//
// If the SP is unable to to make the provisioned volume available
// If the SP is unable to make the provisioned volume available
// from any of the preferred topologies, the SP MAY choose a topology
// from the list of requisite topologies.
// If the list of requisite topologies is not specified, then the SP
// MAY choose from the list of all possible topologies.
// If the list of requisite topologies is specified and the SP is
// unable to to make the provisioned volume available from any of the
// unable to make the provisioned volume available from any of the
// requisite topologies it MUST fail the CreateVolume call.
//
// Example 1:
@@ -254,7 +254,7 @@ type TopologyRequirement struct {
// {"region": "R1", "zone": "Z3"}
// preferred =
// {"region": "R1", "zone": "Z3"}
// then the the SP SHOULD first attempt to make the provisioned volume
// then the SP SHOULD first attempt to make the provisioned volume
// available from "zone" "Z3" in the "region" "R1" and fall back to
// "zone" "Z2" in the "region" "R1" if that is not possible.
//
@@ -268,7 +268,7 @@ type TopologyRequirement struct {
// preferred =
// {"region": "R1", "zone": "Z4"},
// {"region": "R1", "zone": "Z2"}
// then the the SP SHOULD first attempt to make the provisioned volume
// then the SP SHOULD first attempt to make the provisioned volume
// accessible from "zone" "Z4" in the "region" "R1" and fall back to
// "zone" "Z2" in the "region" "R1" if that is not possible. If that
// is not possible, the SP may choose between either the "zone"
@@ -287,7 +287,7 @@ type TopologyRequirement struct {
// preferred =
// {"region": "R1", "zone": "Z5"},
// {"region": "R1", "zone": "Z3"}
// then the the SP SHOULD first attempt to make the provisioned volume
// then the SP SHOULD first attempt to make the provisioned volume
// accessible from the combination of the two "zones" "Z5" and "Z3" in
// the "region" "R1". If that's not possible, it should fall back to
// a combination of "Z5" and other possibilities from the list of

View File

@@ -9,6 +9,7 @@ import (
"fmt"
"io"
"path"
"strconv"
"strings"
"sync"
"time"
@@ -34,14 +35,15 @@ import (
pkgprogress "github.com/docker/docker/pkg/progress"
"github.com/docker/docker/reference"
"github.com/moby/buildkit/cache"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb/sourceresolver"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/solver"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/source"
"github.com/moby/buildkit/source/containerimage"
srctypes "github.com/moby/buildkit/source/types"
"github.com/moby/buildkit/sourcepolicy"
policy "github.com/moby/buildkit/sourcepolicy/pb"
spb "github.com/moby/buildkit/sourcepolicy/pb"
"github.com/moby/buildkit/util/flightcontrol"
"github.com/moby/buildkit/util/imageutil"
@@ -80,9 +82,77 @@ func NewSource(opt SourceOpt) (*Source, error) {
return &Source{SourceOpt: opt}, nil
}
// ID returns image scheme identifier
func (is *Source) ID() string {
return srctypes.DockerImageScheme
// Schemes returns a list of SourceOp identifier schemes that this source
// should match.
func (is *Source) Schemes() []string {
return []string{srctypes.DockerImageScheme}
}
// Identifier constructs an Identifier from the given scheme, ref, and attrs,
// all of which come from a SourceOp.
func (is *Source) Identifier(scheme, ref string, attrs map[string]string, platform *pb.Platform) (source.Identifier, error) {
return is.registryIdentifier(ref, attrs, platform)
}
// Copied from github.com/moby/buildkit/source/containerimage/source.go
func (is *Source) registryIdentifier(ref string, attrs map[string]string, platform *pb.Platform) (source.Identifier, error) {
id, err := containerimage.NewImageIdentifier(ref)
if err != nil {
return nil, err
}
if platform != nil {
id.Platform = &ocispec.Platform{
OS: platform.OS,
Architecture: platform.Architecture,
Variant: platform.Variant,
OSVersion: platform.OSVersion,
}
if platform.OSFeatures != nil {
id.Platform.OSFeatures = append([]string{}, platform.OSFeatures...)
}
}
for k, v := range attrs {
switch k {
case pb.AttrImageResolveMode:
rm, err := resolver.ParseImageResolveMode(v)
if err != nil {
return nil, err
}
id.ResolveMode = rm
case pb.AttrImageRecordType:
rt, err := parseImageRecordType(v)
if err != nil {
return nil, err
}
id.RecordType = rt
case pb.AttrImageLayerLimit:
l, err := strconv.Atoi(v)
if err != nil {
return nil, errors.Wrapf(err, "invalid layer limit %s", v)
}
if l <= 0 {
return nil, errors.Errorf("invalid layer limit %s", v)
}
id.LayerLimit = &l
}
}
return id, nil
}
func parseImageRecordType(v string) (client.UsageRecordType, error) {
switch client.UsageRecordType(v) {
case "", client.UsageRecordTypeRegular:
return client.UsageRecordTypeRegular, nil
case client.UsageRecordTypeInternal:
return client.UsageRecordTypeInternal, nil
case client.UsageRecordTypeFrontend:
return client.UsageRecordTypeFrontend, nil
default:
return "", errors.Errorf("invalid record type %s", v)
}
}
func (is *Source) resolveLocal(refStr string) (*image.Image, error) {
@@ -107,7 +177,7 @@ type resolveRemoteResult struct {
dt []byte
}
func (is *Source) resolveRemote(ctx context.Context, ref string, platform *ocispec.Platform, sm *session.Manager, g session.Group) (string, digest.Digest, []byte, error) {
func (is *Source) resolveRemote(ctx context.Context, ref string, platform *ocispec.Platform, sm *session.Manager, g session.Group) (digest.Digest, []byte, error) {
p := platforms.DefaultSpec()
if platform != nil {
p = *platform
@@ -116,34 +186,36 @@ func (is *Source) resolveRemote(ctx context.Context, ref string, platform *ocisp
key := "getconfig::" + ref + "::" + platforms.Format(p)
res, err := is.g.Do(ctx, key, func(ctx context.Context) (*resolveRemoteResult, error) {
res := resolver.DefaultPool.GetResolver(is.RegistryHosts, ref, "pull", sm, g)
ref, dgst, dt, err := imageutil.Config(ctx, ref, res, is.ContentStore, is.LeaseManager, platform, []*policy.Policy{})
dgst, dt, err := imageutil.Config(ctx, ref, res, is.ContentStore, is.LeaseManager, platform)
if err != nil {
return nil, err
}
return &resolveRemoteResult{ref: ref, dgst: dgst, dt: dt}, nil
})
if err != nil {
return ref, "", nil, err
return "", nil, err
}
return res.ref, res.dgst, res.dt, nil
return res.dgst, res.dt, nil
}
// ResolveImageConfig returns image config for an image
func (is *Source) ResolveImageConfig(ctx context.Context, ref string, opt llb.ResolveImageConfigOpt, sm *session.Manager, g session.Group) (string, digest.Digest, []byte, error) {
func (is *Source) ResolveImageConfig(ctx context.Context, ref string, opt sourceresolver.Opt, sm *session.Manager, g session.Group) (digest.Digest, []byte, error) {
if opt.ImageOpt == nil {
return "", nil, fmt.Errorf("can only resolve an image: %v, opt: %v", ref, opt)
}
ref, err := applySourcePolicies(ctx, ref, opt.SourcePolicies)
if err != nil {
return "", "", nil, err
return "", nil, err
}
resolveMode, err := source.ParseImageResolveMode(opt.ResolveMode)
resolveMode, err := resolver.ParseImageResolveMode(opt.ImageOpt.ResolveMode)
if err != nil {
return ref, "", nil, err
return "", nil, err
}
switch resolveMode {
case source.ResolveModeForcePull:
ref, dgst, dt, err := is.resolveRemote(ctx, ref, opt.Platform, sm, g)
case resolver.ResolveModeForcePull:
return is.resolveRemote(ctx, ref, opt.Platform, sm, g)
// TODO: pull should fallback to local in case of failure to allow offline behavior
// the fallback doesn't work currently
return ref, dgst, dt, err
/*
if err == nil {
return dgst, dt, err
@@ -153,10 +225,10 @@ func (is *Source) ResolveImageConfig(ctx context.Context, ref string, opt llb.Re
return "", dt, err
*/
case source.ResolveModeDefault:
case resolver.ResolveModeDefault:
// default == prefer local, but in the future could be smarter
fallthrough
case source.ResolveModePreferLocal:
case resolver.ResolveModePreferLocal:
img, err := is.resolveLocal(ref)
if err == nil {
if opt.Platform != nil && !platformMatches(img, opt.Platform) {
@@ -165,19 +237,19 @@ func (is *Source) ResolveImageConfig(ctx context.Context, ref string, opt llb.Re
path.Join(img.OS, img.Architecture, img.Variant),
)
} else {
return ref, "", img.RawJSON(), err
return "", img.RawJSON(), err
}
}
// fallback to remote
return is.resolveRemote(ctx, ref, opt.Platform, sm, g)
}
// should never happen
return ref, "", nil, fmt.Errorf("builder cannot resolve image %s: invalid mode %q", ref, opt.ResolveMode)
return "", nil, fmt.Errorf("builder cannot resolve image %s: invalid mode %q", ref, opt.ImageOpt.ResolveMode)
}
// Resolve returns access to pulling for an identifier
func (is *Source) Resolve(ctx context.Context, id source.Identifier, sm *session.Manager, vtx solver.Vertex) (source.SourceInstance, error) {
imageIdentifier, ok := id.(*source.ImageIdentifier)
imageIdentifier, ok := id.(*containerimage.ImageIdentifier)
if !ok {
return nil, errors.Errorf("invalid image identifier %v", id)
}
@@ -201,7 +273,7 @@ type puller struct {
is *Source
resolveLocalOnce sync.Once
g flightcontrol.Group[struct{}]
src *source.ImageIdentifier
src *containerimage.ImageIdentifier
desc ocispec.Descriptor
ref string
config []byte
@@ -253,7 +325,7 @@ func (p *puller) resolveLocal() {
}
}
if p.src.ResolveMode == source.ResolveModeDefault || p.src.ResolveMode == source.ResolveModePreferLocal {
if p.src.ResolveMode == resolver.ResolveModeDefault || p.src.ResolveMode == resolver.ResolveModePreferLocal {
ref := p.src.Reference.String()
img, err := p.is.resolveLocal(ref)
if err == nil {
@@ -302,12 +374,17 @@ func (p *puller) resolve(ctx context.Context, g session.Group) error {
if err != nil {
return struct{}{}, err
}
newRef, _, dt, err := p.is.ResolveImageConfig(ctx, ref.String(), llb.ResolveImageConfigOpt{Platform: &p.platform, ResolveMode: p.src.ResolveMode.String()}, p.sm, g)
_, dt, err := p.is.ResolveImageConfig(ctx, ref.String(), sourceresolver.Opt{
Platform: &p.platform,
ImageOpt: &sourceresolver.ResolveImageOpt{
ResolveMode: p.src.ResolveMode.String(),
},
}, p.sm, g)
if err != nil {
return struct{}{}, err
}
p.ref = newRef
p.ref = ref.String()
p.config = dt
}
return struct{}{}, nil
@@ -866,12 +943,8 @@ func applySourcePolicies(ctx context.Context, str string, spls []*spb.Policy) (s
if err != nil {
return "", errors.WithStack(err)
}
op := &pb.Op{
Op: &pb.Op_Source{
Source: &pb.SourceOp{
Identifier: srctypes.DockerImageScheme + "://" + ref.String(),
},
},
op := &pb.SourceOp{
Identifier: srctypes.DockerImageScheme + "://" + ref.String(),
}
mut, err := sourcepolicy.NewEngine(spls).Evaluate(ctx, op)
@@ -884,9 +957,9 @@ func applySourcePolicies(ctx context.Context, str string, spls []*spb.Policy) (s
t string
ok bool
)
t, newRef, ok := strings.Cut(op.GetSource().GetIdentifier(), "://")
t, newRef, ok := strings.Cut(op.GetIdentifier(), "://")
if !ok {
return "", errors.Errorf("could not parse ref: %s", op.GetSource().GetIdentifier())
return "", errors.Errorf("could not parse ref: %s", op.GetIdentifier())
}
if ok && t != srctypes.DockerImageScheme {
return "", &imageutil.ResolveToNonImageError{Ref: str, Updated: newRef}

View File

@@ -22,6 +22,9 @@ func (s *snapshotter) GetDiffIDs(ctx context.Context, key string) ([]layer.DiffI
}
func (s *snapshotter) EnsureLayer(ctx context.Context, key string) ([]layer.DiffID, error) {
s.layerCreateLocker.Lock(key)
defer s.layerCreateLocker.Unlock(key)
diffIDs, err := s.GetDiffIDs(ctx, key)
if err != nil {
return nil, err

View File

@@ -17,6 +17,7 @@ import (
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/snapshot"
"github.com/moby/buildkit/util/leaseutil"
"github.com/moby/locker"
"github.com/opencontainers/go-digest"
"github.com/pkg/errors"
bolt "go.etcd.io/bbolt"
@@ -51,10 +52,11 @@ type checksumCalculator interface {
type snapshotter struct {
opt Opt
refs map[string]layer.Layer
db *bolt.DB
mu sync.Mutex
reg graphIDRegistrar
refs map[string]layer.Layer
db *bolt.DB
mu sync.Mutex
reg graphIDRegistrar
layerCreateLocker *locker.Locker
}
// NewSnapshotter creates a new snapshotter
@@ -71,10 +73,11 @@ func NewSnapshotter(opt Opt, prevLM leases.Manager, ns string) (snapshot.Snapsho
}
s := &snapshotter{
opt: opt,
db: db,
refs: map[string]layer.Layer{},
reg: reg,
opt: opt,
db: db,
refs: map[string]layer.Layer{},
reg: reg,
layerCreateLocker: locker.New(),
}
slm := newLeaseManager(s, prevLM)

View File

@@ -389,9 +389,10 @@ func (b *Builder) Build(ctx context.Context, opt backend.BuildConfig) (*builder.
}
req := &controlapi.SolveRequest{
Ref: id,
Exporter: exporterName,
ExporterAttrs: exporterAttrs,
Ref: id,
Exporters: []*controlapi.Exporter{
&controlapi.Exporter{Type: exporterName, Attrs: exporterAttrs},
},
Frontend: "dockerfile.v0",
FrontendAttrs: frontendAttrs,
Session: opt.Options.SessionID,

View File

@@ -67,11 +67,11 @@ func newController(ctx context.Context, rt http.RoundTripper, opt Opt) (*control
}
func getTraceExporter(ctx context.Context) trace.SpanExporter {
exp, err := detect.Exporter()
span, _, err := detect.Exporter()
if err != nil {
log.G(ctx).WithError(err).Error("Failed to detect trace exporter for buildkit controller")
}
return exp
return span
}
func newSnapshotterController(ctx context.Context, rt http.RoundTripper, opt Opt) (*control.Controller, error) {
@@ -105,7 +105,8 @@ func newSnapshotterController(ctx context.Context, rt http.RoundTripper, opt Opt
wo, err := containerd.NewWorkerOpt(opt.Root, opt.ContainerdAddress, opt.Snapshotter, opt.ContainerdNamespace,
opt.Rootless, map[string]string{
label.Snapshotter: opt.Snapshotter,
}, dns, nc, opt.ApparmorProfile, false, nil, "", ctd.WithTimeout(60*time.Second))
}, dns, nc, opt.ApparmorProfile, false, nil, "", nil, ctd.WithTimeout(60*time.Second),
)
if err != nil {
return nil, err
}
@@ -302,9 +303,11 @@ func newGraphDriverController(ctx context.Context, rt http.RoundTripper, opt Opt
}
exp, err := mobyexporter.New(mobyexporter.Opt{
ImageStore: dist.ImageStore,
Differ: differ,
ImageTagger: opt.ImageTagger,
ImageStore: dist.ImageStore,
ContentStore: store,
Differ: differ,
ImageTagger: opt.ImageTagger,
LeaseManager: lm,
})
if err != nil {
return nil, err

View File

@@ -16,6 +16,7 @@ import (
"github.com/moby/buildkit/executor"
"github.com/moby/buildkit/executor/oci"
"github.com/moby/buildkit/executor/resources"
resourcestypes "github.com/moby/buildkit/executor/resources/types"
"github.com/moby/buildkit/executor/runcexecutor"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/solver/pb"
@@ -56,9 +57,16 @@ func newExecutor(root, cgroupParent string, net *libnetwork.Controller, dnsConfi
return nil, err
}
runcCmds := []string{"runc"}
// TODO: FIXME: testing env var, replace with something better or remove in a major version or two
if runcOverride := os.Getenv("DOCKER_BUILDKIT_RUNC_COMMAND"); runcOverride != "" {
runcCmds = []string{runcOverride}
}
return runcexecutor.New(runcexecutor.Opt{
Root: filepath.Join(root, "executor"),
CommandCandidates: []string{"runc"},
CommandCandidates: runcCmds,
DefaultCgroupParent: cgroupParent,
Rootless: rootless,
NoPivot: os.Getenv("DOCKER_RAMDISK") != "",
@@ -128,8 +136,8 @@ func (iface *lnInterface) init(c *libnetwork.Controller, n *libnetwork.Network)
}
// TODO(neersighted): Unstub Sample(), and collect data from the libnetwork Endpoint.
func (iface *lnInterface) Sample() (*network.Sample, error) {
return &network.Sample{}, nil
func (iface *lnInterface) Sample() (*resourcestypes.NetworkSample, error) {
return &resourcestypes.NetworkSample{}, nil
}
func (iface *lnInterface) Set(s *specs.Spec) error {

View File

@@ -1,24 +1,27 @@
package mobyexporter
import (
"bytes"
"context"
"encoding/json"
"fmt"
"strings"
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/leases"
"github.com/containerd/log"
distref "github.com/distribution/reference"
"github.com/docker/docker/image"
"github.com/docker/docker/internal/compatcontext"
"github.com/docker/docker/layer"
"github.com/moby/buildkit/exporter"
"github.com/moby/buildkit/exporter/containerimage"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
"github.com/moby/buildkit/util/leaseutil"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
const (
keyImageName = "name"
)
// Differ can make a moby layer from a snapshot
type Differ interface {
EnsureLayer(ctx context.Context, key string) ([]layer.DiffID, error)
@@ -30,9 +33,11 @@ type ImageTagger interface {
// Opt defines a struct for creating new exporter
type Opt struct {
ImageStore image.Store
Differ Differ
ImageTagger ImageTagger
ImageStore image.Store
Differ Differ
ImageTagger ImageTagger
ContentStore content.Store
LeaseManager leases.Manager
}
type imageExporter struct {
@@ -45,13 +50,14 @@ func New(opt Opt) (exporter.Exporter, error) {
return im, nil
}
func (e *imageExporter) Resolve(ctx context.Context, opt map[string]string) (exporter.ExporterInstance, error) {
func (e *imageExporter) Resolve(ctx context.Context, id int, opt map[string]string) (exporter.ExporterInstance, error) {
i := &imageExporterInstance{
imageExporter: e,
id: id,
}
for k, v := range opt {
switch k {
case keyImageName:
switch exptypes.ImageExporterOptKey(k) {
case exptypes.OptKeyName:
for _, v := range strings.Split(v, ",") {
ref, err := distref.ParseNormalizedNamed(v)
if err != nil {
@@ -71,10 +77,15 @@ func (e *imageExporter) Resolve(ctx context.Context, opt map[string]string) (exp
type imageExporterInstance struct {
*imageExporter
id int
targetNames []distref.Named
meta map[string][]byte
}
func (e *imageExporterInstance) ID() int {
return e.id
}
func (e *imageExporterInstance) Name() string {
return "exporting to image"
}
@@ -83,7 +94,7 @@ func (e *imageExporterInstance) Config() *exporter.Config {
return exporter.NewConfig()
}
func (e *imageExporterInstance) Export(ctx context.Context, inp *exporter.Source, sessionID string) (map[string]string, exporter.DescriptorReference, error) {
func (e *imageExporterInstance) Export(ctx context.Context, inp *exporter.Source, inlineCache exptypes.InlineCache, sessionID string) (map[string]string, exporter.DescriptorReference, error) {
if len(inp.Refs) > 1 {
return nil, nil, fmt.Errorf("exporting multiple references to image store is currently unsupported")
}
@@ -103,18 +114,14 @@ func (e *imageExporterInstance) Export(ctx context.Context, inp *exporter.Source
case 0:
config = inp.Metadata[exptypes.ExporterImageConfigKey]
case 1:
platformsBytes, ok := inp.Metadata[exptypes.ExporterPlatformsKey]
if !ok {
return nil, nil, fmt.Errorf("cannot export image, missing platforms mapping")
ps, err := exptypes.ParsePlatforms(inp.Metadata)
if err != nil {
return nil, nil, fmt.Errorf("cannot export image, failed to parse platforms: %w", err)
}
var p exptypes.Platforms
if err := json.Unmarshal(platformsBytes, &p); err != nil {
return nil, nil, errors.Wrapf(err, "failed to parse platforms passed to exporter")
if len(ps.Platforms) != len(inp.Refs) {
return nil, nil, errors.Errorf("number of platforms does not match references %d %d", len(ps.Platforms), len(inp.Refs))
}
if len(p.Platforms) != len(inp.Refs) {
return nil, nil, errors.Errorf("number of platforms does not match references %d %d", len(p.Platforms), len(inp.Refs))
}
config = inp.Metadata[fmt.Sprintf("%s/%s", exptypes.ExporterImageConfigKey, p.Platforms[0].ID)]
config = inp.Metadata[fmt.Sprintf("%s/%s", exptypes.ExporterImageConfigKey, ps.Platforms[0].ID)]
}
var diffs []digest.Digest
@@ -157,7 +164,21 @@ func (e *imageExporterInstance) Export(ctx context.Context, inp *exporter.Source
diffs, history = normalizeLayersAndHistory(diffs, history, ref)
config, err = patchImageConfig(config, diffs, history, inp.Metadata[exptypes.ExporterInlineCache])
var inlineCacheEntry *exptypes.InlineCacheEntry
if inlineCache != nil {
inlineCacheResult, err := inlineCache(ctx)
if err != nil {
return nil, nil, err
}
if inlineCacheResult != nil {
if ref != nil {
inlineCacheEntry, _ = inlineCacheResult.FindRef(ref.ID())
} else {
inlineCacheEntry = inlineCacheResult.Ref
}
}
}
config, err = patchImageConfig(config, diffs, history, inlineCacheEntry)
if err != nil {
return nil, nil, err
}
@@ -171,8 +192,10 @@ func (e *imageExporterInstance) Export(ctx context.Context, inp *exporter.Source
}
_ = configDone(nil)
if e.opt.ImageTagger != nil {
for _, targetName := range e.targetNames {
var names []string
for _, targetName := range e.targetNames {
names = append(names, targetName.String())
if e.opt.ImageTagger != nil {
tagDone := oneOffProgress(ctx, "naming to "+targetName.String())
if err := e.opt.ImageTagger.TagImage(ctx, image.ID(digest.Digest(id)), targetName); err != nil {
return nil, nil, tagDone(err)
@@ -181,8 +204,49 @@ func (e *imageExporterInstance) Export(ctx context.Context, inp *exporter.Source
}
}
return map[string]string{
resp := map[string]string{
exptypes.ExporterImageConfigDigestKey: configDigest.String(),
exptypes.ExporterImageDigestKey: id.String(),
}, nil, nil
}
if len(names) > 0 {
resp["image.name"] = strings.Join(names, ",")
}
descRef, err := e.newTempReference(ctx, config)
if err != nil {
return nil, nil, fmt.Errorf("failed to create a temporary descriptor reference: %w", err)
}
return resp, descRef, nil
}
func (e *imageExporterInstance) newTempReference(ctx context.Context, config []byte) (exporter.DescriptorReference, error) {
lm := e.opt.LeaseManager
dgst := digest.FromBytes(config)
leaseCtx, done, err := leaseutil.WithLease(ctx, lm, leaseutil.MakeTemporary)
if err != nil {
return nil, err
}
unlease := func(ctx context.Context) error {
err := done(compatcontext.WithoutCancel(ctx))
if err != nil {
log.G(ctx).WithError(err).Error("failed to delete descriptor reference lease")
}
return err
}
desc := ocispec.Descriptor{
Digest: dgst,
MediaType: "application/vnd.docker.container.image.v1+json",
Size: int64(len(config)),
}
if err := content.WriteBlob(leaseCtx, e.opt.ContentStore, desc.Digest.String(), bytes.NewReader(config), desc); err != nil {
unlease(leaseCtx)
return nil, fmt.Errorf("failed to save temporary image config: %w", err)
}
return containerimage.NewDescriptorReference(desc, unlease), nil
}

View File

@@ -8,6 +8,7 @@ import (
"github.com/containerd/containerd/platforms"
"github.com/containerd/log"
"github.com/moby/buildkit/cache"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
"github.com/moby/buildkit/util/progress"
"github.com/moby/buildkit/util/system"
"github.com/opencontainers/go-digest"
@@ -38,7 +39,7 @@ func parseHistoryFromConfig(dt []byte) ([]ocispec.History, error) {
return config.History, nil
}
func patchImageConfig(dt []byte, dps []digest.Digest, history []ocispec.History, cache []byte) ([]byte, error) {
func patchImageConfig(dt []byte, dps []digest.Digest, history []ocispec.History, cache *exptypes.InlineCacheEntry) ([]byte, error) {
m := map[string]json.RawMessage{}
if err := json.Unmarshal(dt, &m); err != nil {
return nil, errors.Wrap(err, "failed to parse image config for patch")
@@ -75,7 +76,7 @@ func patchImageConfig(dt []byte, dps []digest.Digest, history []ocispec.History,
}
if cache != nil {
dt, err := json.Marshal(cache)
dt, err := json.Marshal(cache.Data)
if err != nil {
return nil, err
}

View File

@@ -19,7 +19,7 @@ func NewExporterWrapper(exp exporter.Exporter) (exporter.Exporter, error) {
}
// Resolve applies moby specific attributes to the request.
func (e *imageExporterMobyWrapper) Resolve(ctx context.Context, exporterAttrs map[string]string) (exporter.ExporterInstance, error) {
func (e *imageExporterMobyWrapper) Resolve(ctx context.Context, id int, exporterAttrs map[string]string) (exporter.ExporterInstance, error) {
if exporterAttrs == nil {
exporterAttrs = make(map[string]string)
}
@@ -33,5 +33,5 @@ func (e *imageExporterMobyWrapper) Resolve(ctx context.Context, exporterAttrs ma
exporterAttrs[string(exptypes.OptKeyDanglingPrefix)] = "moby-dangling"
}
return e.exp.Resolve(ctx, exporterAttrs)
return e.exp.Resolve(ctx, id, exporterAttrs)
}

View File

@@ -12,7 +12,7 @@ import (
"github.com/containerd/containerd/platforms"
"github.com/containerd/containerd/rootfs"
"github.com/containerd/log"
"github.com/docker/docker/builder/builder-next/adapters/containerimage"
imageadapter "github.com/docker/docker/builder/builder-next/adapters/containerimage"
mobyexporter "github.com/docker/docker/builder/builder-next/exporter"
distmetadata "github.com/docker/docker/distribution/metadata"
"github.com/docker/docker/distribution/xfer"
@@ -23,7 +23,7 @@ import (
"github.com/moby/buildkit/cache"
cacheconfig "github.com/moby/buildkit/cache/config"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/client/llb/sourceresolver"
"github.com/moby/buildkit/executor"
"github.com/moby/buildkit/exporter"
localexporter "github.com/moby/buildkit/exporter/local"
@@ -37,6 +37,7 @@ import (
"github.com/moby/buildkit/solver/llbsolver/ops"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/source"
"github.com/moby/buildkit/source/containerimage"
"github.com/moby/buildkit/source/git"
"github.com/moby/buildkit/source/http"
"github.com/moby/buildkit/source/local"
@@ -75,7 +76,7 @@ type Opt struct {
ContentStore *containerdsnapshot.Store
CacheManager cache.Manager
LeaseManager *leaseutil.Manager
ImageSource *containerimage.Source
ImageSource *imageadapter.Source
DownloadManager *xfer.LayerDownloadManager
V2MetadataService distmetadata.V2MetadataService
Transport nethttp.RoundTripper
@@ -212,6 +213,49 @@ func (w *Worker) LoadRef(ctx context.Context, id string, hidden bool) (cache.Imm
return w.CacheManager().Get(ctx, id, nil, opts...)
}
func (w *Worker) ResolveSourceMetadata(ctx context.Context, op *pb.SourceOp, opt sourceresolver.Opt, sm *session.Manager, g session.Group) (*sourceresolver.MetaResponse, error) {
if opt.SourcePolicies != nil {
return nil, errors.New("source policies can not be set for worker")
}
var platform *pb.Platform
if p := opt.Platform; p != nil {
platform = &pb.Platform{
Architecture: p.Architecture,
OS: p.OS,
Variant: p.Variant,
OSVersion: p.OSVersion,
}
}
id, err := w.SourceManager.Identifier(&pb.Op_Source{Source: op}, platform)
if err != nil {
return nil, err
}
switch idt := id.(type) {
case *containerimage.ImageIdentifier:
if opt.ImageOpt == nil {
opt.ImageOpt = &sourceresolver.ResolveImageOpt{}
}
dgst, config, err := w.ImageSource.ResolveImageConfig(ctx, idt.Reference.String(), opt, sm, g)
if err != nil {
return nil, err
}
return &sourceresolver.MetaResponse{
Op: op,
Image: &sourceresolver.ResolveImageResponse{
Digest: dgst,
Config: config,
},
}, nil
}
return &sourceresolver.MetaResponse{
Op: op,
}, nil
}
// ResolveOp converts a LLB vertex into a LLB operation
func (w *Worker) ResolveOp(v solver.Vertex, s frontend.FrontendLLBBridge, sm *session.Manager) (solver.Op, error) {
if baseOp, ok := v.Sys().(*pb.Op); ok {
@@ -236,7 +280,7 @@ func (w *Worker) ResolveOp(v solver.Vertex, s frontend.FrontendLLBBridge, sm *se
}
// ResolveImageConfig returns image config for an image
func (w *Worker) ResolveImageConfig(ctx context.Context, ref string, opt llb.ResolveImageConfigOpt, sm *session.Manager, g session.Group) (string, digest.Digest, []byte, error) {
func (w *Worker) ResolveImageConfig(ctx context.Context, ref string, opt sourceresolver.Opt, sm *session.Manager, g session.Group) (digest.Digest, []byte, error) {
return w.ImageSource.ResolveImageConfig(ctx, ref, opt, sm, g)
}
@@ -526,3 +570,7 @@ type emptyProvider struct{}
func (p *emptyProvider) ReaderAt(ctx context.Context, dec ocispec.Descriptor) (content.ReaderAt, error) {
return nil, errors.Errorf("ReaderAt not implemented for empty provider")
}
func (p *emptyProvider) Info(ctx context.Context, d digest.Digest) (content.Info, error) {
return content.Info{}, errors.Errorf("Info not implemented for empty provider")
}

View File

@@ -64,7 +64,7 @@ type ExecBackend interface {
// ContainerRm removes a container specified by `id`.
ContainerRm(name string, config *backend.ContainerRmConfig) error
// ContainerStart starts a new container
ContainerStart(ctx context.Context, containerID string, hostConfig *container.HostConfig, checkpoint string, checkpointDir string) error
ContainerStart(ctx context.Context, containerID string, checkpoint string, checkpointDir string) error
// ContainerWait stops processing until the given container is stopped.
ContainerWait(ctx context.Context, name string, condition containerpkg.WaitCondition) (<-chan containerpkg.StateStatus, error)
}

View File

@@ -72,7 +72,7 @@ func (c *containerManager) Run(ctx context.Context, cID string, stdout, stderr i
}
}()
if err := c.backend.ContainerStart(ctx, cID, nil, "", ""); err != nil {
if err := c.backend.ContainerStart(ctx, cID, "", ""); err != nil {
close(finished)
logCancellationError(cancelErrCh, "error from ContainerStart: "+err.Error())
return err

View File

@@ -46,7 +46,7 @@ func (m *MockBackend) CommitBuildStep(ctx context.Context, c backend.CommitConfi
return "", nil
}
func (m *MockBackend) ContainerStart(ctx context.Context, containerID string, hostConfig *container.HostConfig, checkpoint string, checkpointDir string) error {
func (m *MockBackend) ContainerStart(ctx context.Context, containerID string, checkpoint string, checkpointDir string) error {
return nil
}

View File

@@ -10,11 +10,11 @@ import (
)
// DistributionInspect returns the image digest with the full manifest.
func (cli *Client) DistributionInspect(ctx context.Context, image, encodedRegistryAuth string) (registry.DistributionInspect, error) {
func (cli *Client) DistributionInspect(ctx context.Context, imageRef, encodedRegistryAuth string) (registry.DistributionInspect, error) {
// Contact the registry to retrieve digest and platform information
var distributionInspect registry.DistributionInspect
if image == "" {
return distributionInspect, objectNotFoundError{object: "distribution", id: image}
if imageRef == "" {
return distributionInspect, objectNotFoundError{object: "distribution", id: imageRef}
}
if err := cli.NewVersionError(ctx, "1.30", "distribution inspect"); err != nil {
@@ -28,7 +28,7 @@ func (cli *Client) DistributionInspect(ctx context.Context, image, encodedRegist
}
}
resp, err := cli.get(ctx, "/distribution/"+image+"/json", url.Values{}, headers)
resp, err := cli.get(ctx, "/distribution/"+imageRef+"/json", url.Values{}, headers)
defer ensureReaderClosed(resp)
if err != nil {
return distributionInspect, err

View File

@@ -8,13 +8,13 @@ import (
"strings"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
)
// ImageCreate creates a new image based on the parent options.
// It returns the JSON content in the response body.
func (cli *Client) ImageCreate(ctx context.Context, parentReference string, options types.ImageCreateOptions) (io.ReadCloser, error) {
func (cli *Client) ImageCreate(ctx context.Context, parentReference string, options image.CreateOptions) (io.ReadCloser, error) {
ref, err := reference.ParseNormalizedNamed(parentReference)
if err != nil {
return nil, err

View File

@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
@@ -20,7 +20,7 @@ func TestImageCreateError(t *testing.T) {
client := &Client{
client: newMockClient(errorMock(http.StatusInternalServerError, "Server error")),
}
_, err := client.ImageCreate(context.Background(), "reference", types.ImageCreateOptions{})
_, err := client.ImageCreate(context.Background(), "reference", image.CreateOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsSystem))
}
@@ -58,7 +58,7 @@ func TestImageCreate(t *testing.T) {
}),
}
createResponse, err := client.ImageCreate(context.Background(), expectedReference, types.ImageCreateOptions{
createResponse, err := client.ImageCreate(context.Background(), expectedReference, image.CreateOptions{
RegistryAuth: expectedRegistryAuth,
})
if err != nil {

View File

@@ -8,11 +8,12 @@ import (
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
)
// ImageImport creates a new image based on the source options.
// It returns the JSON content in the response body.
func (cli *Client) ImageImport(ctx context.Context, source types.ImageImportSource, ref string, options types.ImageImportOptions) (io.ReadCloser, error) {
func (cli *Client) ImageImport(ctx context.Context, source types.ImageImportSource, ref string, options image.ImportOptions) (io.ReadCloser, error) {
if ref != "" {
// Check if the given image name can be resolved
if _, err := reference.ParseNormalizedNamed(ref); err != nil {

View File

@@ -11,6 +11,7 @@ import (
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
@@ -20,7 +21,7 @@ func TestImageImportError(t *testing.T) {
client := &Client{
client: newMockClient(errorMock(http.StatusInternalServerError, "Server error")),
}
_, err := client.ImageImport(context.Background(), types.ImageImportSource{}, "image:tag", types.ImageImportOptions{})
_, err := client.ImageImport(context.Background(), types.ImageImportSource{}, "image:tag", image.ImportOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsSystem))
}
@@ -63,7 +64,7 @@ func TestImageImport(t *testing.T) {
importResponse, err := client.ImageImport(context.Background(), types.ImageImportSource{
Source: strings.NewReader("source"),
SourceName: "image_source",
}, "repository_name:imported", types.ImageImportOptions{
}, "repository_name:imported", image.ImportOptions{
Tag: "imported",
Message: "A message",
Changes: []string{"change1", "change2"},

View File

@@ -5,14 +5,13 @@ import (
"encoding/json"
"net/url"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/versions"
)
// ImageList returns a list of images in the docker host.
func (cli *Client) ImageList(ctx context.Context, options types.ImageListOptions) ([]image.Summary, error) {
func (cli *Client) ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error) {
var images []image.Summary
// Make sure we negotiated (if the client is configured to do so),

View File

@@ -11,7 +11,6 @@ import (
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/errdefs"
@@ -24,7 +23,7 @@ func TestImageListError(t *testing.T) {
client: newMockClient(errorMock(http.StatusInternalServerError, "Server error")),
}
_, err := client.ImageList(context.Background(), types.ImageListOptions{})
_, err := client.ImageList(context.Background(), image.ListOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsSystem))
}
@@ -36,7 +35,7 @@ func TestImageListConnectionError(t *testing.T) {
client, err := NewClientWithOpts(WithAPIVersionNegotiation(), WithHost("tcp://no-such-host.invalid"))
assert.NilError(t, err)
_, err = client.ImageList(context.Background(), types.ImageListOptions{})
_, err = client.ImageList(context.Background(), image.ListOptions{})
assert.Check(t, is.ErrorType(err, IsErrConnectionFailed))
}
@@ -44,11 +43,11 @@ func TestImageList(t *testing.T) {
const expectedURL = "/images/json"
listCases := []struct {
options types.ImageListOptions
options image.ListOptions
expectedQueryParams map[string]string
}{
{
options: types.ImageListOptions{},
options: image.ListOptions{},
expectedQueryParams: map[string]string{
"all": "",
"filter": "",
@@ -56,7 +55,7 @@ func TestImageList(t *testing.T) {
},
},
{
options: types.ImageListOptions{
options: image.ListOptions{
Filters: filters.NewArgs(
filters.Arg("label", "label1"),
filters.Arg("label", "label2"),
@@ -70,7 +69,7 @@ func TestImageList(t *testing.T) {
},
},
{
options: types.ImageListOptions{
options: image.ListOptions{
Filters: filters.NewArgs(filters.Arg("dangling", "false")),
},
expectedQueryParams: map[string]string{
@@ -153,7 +152,7 @@ func TestImageListApiBefore125(t *testing.T) {
version: "1.24",
}
options := types.ImageListOptions{
options := image.ListOptions{
Filters: filters.NewArgs(filters.Arg("reference", "image:tag")),
}
@@ -174,12 +173,12 @@ func TestImageListWithSharedSize(t *testing.T) {
for _, tc := range []struct {
name string
version string
options types.ImageListOptions
options image.ListOptions
sharedSize string // expected value for the shared-size query param, or empty if it should not be set.
}{
{name: "unset after 1.42, no options set", version: "1.42"},
{name: "set after 1.42, if requested", version: "1.42", options: types.ImageListOptions{SharedSize: true}, sharedSize: "1"},
{name: "unset before 1.42, even if requested", version: "1.41", options: types.ImageListOptions{SharedSize: true}},
{name: "set after 1.42, if requested", version: "1.42", options: image.ListOptions{SharedSize: true}, sharedSize: "1"},
{name: "unset before 1.42, even if requested", version: "1.41", options: image.ListOptions{SharedSize: true}},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/errdefs"
)
@@ -19,7 +19,7 @@ import (
// FIXME(vdemeester): there is currently used in a few way in docker/docker
// - if not in trusted content, ref is used to pass the whole reference, and tag is empty
// - if in trusted content, ref is used to pass the reference name, and tag for the digest
func (cli *Client) ImagePull(ctx context.Context, refStr string, options types.ImagePullOptions) (io.ReadCloser, error) {
func (cli *Client) ImagePull(ctx context.Context, refStr string, options image.PullOptions) (io.ReadCloser, error) {
ref, err := reference.ParseNormalizedNamed(refStr)
if err != nil {
return nil, err

View File

@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
@@ -23,7 +23,7 @@ func TestImagePullReferenceParseError(t *testing.T) {
}),
}
// An empty reference is an invalid reference
_, err := client.ImagePull(context.Background(), "", types.ImagePullOptions{})
_, err := client.ImagePull(context.Background(), "", image.PullOptions{})
if err == nil || !strings.Contains(err.Error(), "invalid reference format") {
t.Fatalf("expected an error, got %v", err)
}
@@ -33,7 +33,7 @@ func TestImagePullAnyError(t *testing.T) {
client := &Client{
client: newMockClient(errorMock(http.StatusInternalServerError, "Server error")),
}
_, err := client.ImagePull(context.Background(), "myimage", types.ImagePullOptions{})
_, err := client.ImagePull(context.Background(), "myimage", image.PullOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsSystem))
}
@@ -41,7 +41,7 @@ func TestImagePullStatusUnauthorizedError(t *testing.T) {
client := &Client{
client: newMockClient(errorMock(http.StatusUnauthorized, "Unauthorized error")),
}
_, err := client.ImagePull(context.Background(), "myimage", types.ImagePullOptions{})
_, err := client.ImagePull(context.Background(), "myimage", image.PullOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsUnauthorized))
}
@@ -52,7 +52,7 @@ func TestImagePullWithUnauthorizedErrorAndPrivilegeFuncError(t *testing.T) {
privilegeFunc := func() (string, error) {
return "", fmt.Errorf("Error requesting privilege")
}
_, err := client.ImagePull(context.Background(), "myimage", types.ImagePullOptions{
_, err := client.ImagePull(context.Background(), "myimage", image.PullOptions{
PrivilegeFunc: privilegeFunc,
})
if err == nil || err.Error() != "Error requesting privilege" {
@@ -67,7 +67,7 @@ func TestImagePullWithUnauthorizedErrorAndAnotherUnauthorizedError(t *testing.T)
privilegeFunc := func() (string, error) {
return "a-auth-header", nil
}
_, err := client.ImagePull(context.Background(), "myimage", types.ImagePullOptions{
_, err := client.ImagePull(context.Background(), "myimage", image.PullOptions{
PrivilegeFunc: privilegeFunc,
})
assert.Check(t, is.ErrorType(err, errdefs.IsUnauthorized))
@@ -108,7 +108,7 @@ func TestImagePullWithPrivilegedFuncNoError(t *testing.T) {
privilegeFunc := func() (string, error) {
return "IAmValid", nil
}
resp, err := client.ImagePull(context.Background(), "myimage", types.ImagePullOptions{
resp, err := client.ImagePull(context.Background(), "myimage", image.PullOptions{
RegistryAuth: "NotValid",
PrivilegeFunc: privilegeFunc,
})
@@ -179,7 +179,7 @@ func TestImagePullWithoutErrors(t *testing.T) {
}, nil
}),
}
resp, err := client.ImagePull(context.Background(), pullCase.reference, types.ImagePullOptions{
resp, err := client.ImagePull(context.Background(), pullCase.reference, image.PullOptions{
All: pullCase.all,
})
if err != nil {

View File

@@ -8,7 +8,7 @@ import (
"net/url"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/errdefs"
)
@@ -17,7 +17,7 @@ import (
// It executes the privileged function if the operation is unauthorized
// and it tries one more time.
// It's up to the caller to handle the io.ReadCloser and close it properly.
func (cli *Client) ImagePush(ctx context.Context, image string, options types.ImagePushOptions) (io.ReadCloser, error) {
func (cli *Client) ImagePush(ctx context.Context, image string, options image.PushOptions) (io.ReadCloser, error) {
ref, err := reference.ParseNormalizedNamed(image)
if err != nil {
return nil, err

View File

@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
@@ -23,12 +23,12 @@ func TestImagePushReferenceError(t *testing.T) {
}),
}
// An empty reference is an invalid reference
_, err := client.ImagePush(context.Background(), "", types.ImagePushOptions{})
_, err := client.ImagePush(context.Background(), "", image.PushOptions{})
if err == nil || !strings.Contains(err.Error(), "invalid reference format") {
t.Fatalf("expected an error, got %v", err)
}
// An canonical reference cannot be pushed
_, err = client.ImagePush(context.Background(), "repo@sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff", types.ImagePushOptions{})
_, err = client.ImagePush(context.Background(), "repo@sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff", image.PushOptions{})
if err == nil || err.Error() != "cannot push a digest reference" {
t.Fatalf("expected an error, got %v", err)
}
@@ -38,7 +38,7 @@ func TestImagePushAnyError(t *testing.T) {
client := &Client{
client: newMockClient(errorMock(http.StatusInternalServerError, "Server error")),
}
_, err := client.ImagePush(context.Background(), "myimage", types.ImagePushOptions{})
_, err := client.ImagePush(context.Background(), "myimage", image.PushOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsSystem))
}
@@ -46,7 +46,7 @@ func TestImagePushStatusUnauthorizedError(t *testing.T) {
client := &Client{
client: newMockClient(errorMock(http.StatusUnauthorized, "Unauthorized error")),
}
_, err := client.ImagePush(context.Background(), "myimage", types.ImagePushOptions{})
_, err := client.ImagePush(context.Background(), "myimage", image.PushOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsUnauthorized))
}
@@ -57,7 +57,7 @@ func TestImagePushWithUnauthorizedErrorAndPrivilegeFuncError(t *testing.T) {
privilegeFunc := func() (string, error) {
return "", fmt.Errorf("Error requesting privilege")
}
_, err := client.ImagePush(context.Background(), "myimage", types.ImagePushOptions{
_, err := client.ImagePush(context.Background(), "myimage", image.PushOptions{
PrivilegeFunc: privilegeFunc,
})
if err == nil || err.Error() != "Error requesting privilege" {
@@ -72,7 +72,7 @@ func TestImagePushWithUnauthorizedErrorAndAnotherUnauthorizedError(t *testing.T)
privilegeFunc := func() (string, error) {
return "a-auth-header", nil
}
_, err := client.ImagePush(context.Background(), "myimage", types.ImagePushOptions{
_, err := client.ImagePush(context.Background(), "myimage", image.PushOptions{
PrivilegeFunc: privilegeFunc,
})
assert.Check(t, is.ErrorType(err, errdefs.IsUnauthorized))
@@ -109,7 +109,7 @@ func TestImagePushWithPrivilegedFuncNoError(t *testing.T) {
privilegeFunc := func() (string, error) {
return "IAmValid", nil
}
resp, err := client.ImagePush(context.Background(), "myimage:tag", types.ImagePushOptions{
resp, err := client.ImagePush(context.Background(), "myimage:tag", image.PushOptions{
RegistryAuth: "NotValid",
PrivilegeFunc: privilegeFunc,
})
@@ -179,7 +179,7 @@ func TestImagePushWithoutErrors(t *testing.T) {
}, nil
}),
}
resp, err := client.ImagePush(context.Background(), tc.reference, types.ImagePushOptions{
resp, err := client.ImagePush(context.Background(), tc.reference, image.PushOptions{
All: tc.all,
})
if err != nil {

View File

@@ -5,12 +5,11 @@ import (
"encoding/json"
"net/url"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
)
// ImageRemove removes an image from the docker host.
func (cli *Client) ImageRemove(ctx context.Context, imageID string, options types.ImageRemoveOptions) ([]image.DeleteResponse, error) {
func (cli *Client) ImageRemove(ctx context.Context, imageID string, options image.RemoveOptions) ([]image.DeleteResponse, error) {
query := url.Values{}
if options.Force {

View File

@@ -10,7 +10,6 @@ import (
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
@@ -22,7 +21,7 @@ func TestImageRemoveError(t *testing.T) {
client: newMockClient(errorMock(http.StatusInternalServerError, "Server error")),
}
_, err := client.ImageRemove(context.Background(), "image_id", types.ImageRemoveOptions{})
_, err := client.ImageRemove(context.Background(), "image_id", image.RemoveOptions{})
assert.Check(t, is.ErrorType(err, errdefs.IsSystem))
}
@@ -31,7 +30,7 @@ func TestImageRemoveImageNotFound(t *testing.T) {
client: newMockClient(errorMock(http.StatusNotFound, "no such image: unknown")),
}
_, err := client.ImageRemove(context.Background(), "unknown", types.ImageRemoveOptions{})
_, err := client.ImageRemove(context.Background(), "unknown", image.RemoveOptions{})
assert.Check(t, is.ErrorContains(err, "no such image: unknown"))
assert.Check(t, is.ErrorType(err, errdefs.IsNotFound))
}
@@ -93,7 +92,7 @@ func TestImageRemove(t *testing.T) {
}, nil
}),
}
imageDeletes, err := client.ImageRemove(context.Background(), "image_id", types.ImageRemoveOptions{
imageDeletes, err := client.ImageRemove(context.Background(), "image_id", image.RemoveOptions{
Force: removeCase.force,
PruneChildren: removeCase.pruneChildren,
})

View File

@@ -90,15 +90,15 @@ type ImageAPIClient interface {
ImageBuild(ctx context.Context, context io.Reader, options types.ImageBuildOptions) (types.ImageBuildResponse, error)
BuildCachePrune(ctx context.Context, opts types.BuildCachePruneOptions) (*types.BuildCachePruneReport, error)
BuildCancel(ctx context.Context, id string) error
ImageCreate(ctx context.Context, parentReference string, options types.ImageCreateOptions) (io.ReadCloser, error)
ImageCreate(ctx context.Context, parentReference string, options image.CreateOptions) (io.ReadCloser, error)
ImageHistory(ctx context.Context, image string) ([]image.HistoryResponseItem, error)
ImageImport(ctx context.Context, source types.ImageImportSource, ref string, options types.ImageImportOptions) (io.ReadCloser, error)
ImageImport(ctx context.Context, source types.ImageImportSource, ref string, options image.ImportOptions) (io.ReadCloser, error)
ImageInspectWithRaw(ctx context.Context, image string) (types.ImageInspect, []byte, error)
ImageList(ctx context.Context, options types.ImageListOptions) ([]image.Summary, error)
ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error)
ImageLoad(ctx context.Context, input io.Reader, quiet bool) (types.ImageLoadResponse, error)
ImagePull(ctx context.Context, ref string, options types.ImagePullOptions) (io.ReadCloser, error)
ImagePush(ctx context.Context, ref string, options types.ImagePushOptions) (io.ReadCloser, error)
ImageRemove(ctx context.Context, image string, options types.ImageRemoveOptions) ([]image.DeleteResponse, error)
ImagePull(ctx context.Context, ref string, options image.PullOptions) (io.ReadCloser, error)
ImagePush(ctx context.Context, ref string, options image.PushOptions) (io.ReadCloser, error)
ImageRemove(ctx context.Context, image string, options image.RemoveOptions) ([]image.DeleteResponse, error)
ImageSearch(ctx context.Context, term string, options types.ImageSearchOptions) ([]registry.SearchResult, error)
ImageSave(ctx context.Context, images []string) (io.ReadCloser, error)
ImageTag(ctx context.Context, image, ref string) error

View File

@@ -15,7 +15,7 @@ import (
)
var (
testBuf = []byte("Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo")
testBuf = []byte("Buffalo1 buffalo2 Buffalo3 buffalo4 buffalo5 buffalo6 Buffalo7 buffalo8")
testBufSize = len(testBuf)
)

View File

@@ -45,9 +45,6 @@ func installConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
flags.StringVar(&conf.CgroupParent, "cgroup-parent", "", "Set parent cgroup for all containers")
flags.StringVar(&conf.RemappedRoot, "userns-remap", "", "User/Group setting for user namespaces")
flags.BoolVar(&conf.LiveRestoreEnabled, "live-restore", false, "Enable live restore of docker when containers are still running")
// TODO(thaJeztah): Used to produce a deprecation error; remove the flag and the OOMScoreAdjust field for the next release after v25.0.0.
flags.IntVar(&conf.OOMScoreAdjust, "oom-score-adjust", 0, "Set the oom_score_adj for the daemon (deprecated)")
_ = flags.MarkDeprecated("oom-score-adjust", "and will be removed in the next release.")
flags.BoolVar(&conf.Init, "init", false, "Run an init in the container to forward signals and reap processes")
flags.StringVar(&conf.InitPath, "init-path", "", "Path to the docker-init binary")
flags.Int64Var(&conf.CPURealtimePeriod, "cpu-rt-period", 0, "Limit the CPU real-time period in microseconds for the parent cgroup for all containers (not supported with cgroups v2)")

View File

@@ -242,7 +242,7 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
// Override BuildKit's default Resource so that it matches the semconv
// version that is used in our code.
detect.Resource = resource.Default()
detect.OverrideResource(resource.Default())
detect.Recorder = detect.NewTraceRecorder()
tp, err := detect.TracerProvider()
@@ -256,7 +256,10 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
pluginStore := plugin.NewStore()
var apiServer apiserver.Server
cli.authzMiddleware = initMiddlewares(&apiServer, cli.Config, pluginStore)
cli.authzMiddleware, err = initMiddlewares(&apiServer, cli.Config, pluginStore)
if err != nil {
return errors.Wrap(err, "failed to start API server")
}
d, err := daemon.NewDaemon(ctx, cli.Config, pluginStore, cli.authzMiddleware)
if err != nil {
@@ -307,14 +310,12 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
//
// FIXME(thaJeztah): better separate runtime and config data?
daemonCfg := d.Config()
routerOptions, err := newRouterOptions(routerCtx, &daemonCfg, d)
routerOpts, err := newRouterOptions(routerCtx, &daemonCfg, d, c)
if err != nil {
return err
}
routerOptions.cluster = c
httpServer.Handler = apiServer.CreateMux(routerOptions.Build()...)
httpServer.Handler = apiServer.CreateMux(routerOpts.Build()...)
go d.ProcessClusterNotifications(ctx, c.GetWatchStream())
@@ -356,7 +357,7 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
notifyStopping()
shutdownDaemon(ctx, d)
if err := routerOptions.buildkit.Close(); err != nil {
if err := routerOpts.buildkit.Close(); err != nil {
log.G(ctx).WithError(err).Error("Failed to close buildkit")
}
@@ -379,12 +380,20 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
// TODO: This can be removed after buildkit is updated to use http/protobuf as the default.
func setOTLPProtoDefault() {
const (
tracesEnv = "OTEL_EXPORTER_OTLP_TRACES_PROTOCOL"
protoEnv = "OTEL_EXPORTER_OTLP_PROTOCOL"
tracesEnv = "OTEL_EXPORTER_OTLP_TRACES_PROTOCOL"
metricsEnv = "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL"
protoEnv = "OTEL_EXPORTER_OTLP_PROTOCOL"
defaultProto = "http/protobuf"
)
if os.Getenv(tracesEnv) == "" && os.Getenv(protoEnv) == "" {
os.Setenv(tracesEnv, "http/protobuf")
if os.Getenv(protoEnv) == "" {
if os.Getenv(tracesEnv) == "" {
os.Setenv(tracesEnv, defaultProto)
}
if os.Getenv(metricsEnv) == "" {
os.Setenv(metricsEnv, defaultProto)
}
}
}
@@ -397,23 +406,17 @@ type routerOptions struct {
cluster *cluster.Cluster
}
func newRouterOptions(ctx context.Context, config *config.Config, d *daemon.Daemon) (routerOptions, error) {
opts := routerOptions{}
func newRouterOptions(ctx context.Context, config *config.Config, d *daemon.Daemon, c *cluster.Cluster) (routerOptions, error) {
sm, err := session.NewManager()
if err != nil {
return opts, errors.Wrap(err, "failed to create sessionmanager")
return routerOptions{}, errors.Wrap(err, "failed to create sessionmanager")
}
manager, err := dockerfile.NewBuildManager(d.BuilderBackend(), d.IdentityMapping())
if err != nil {
return opts, err
return routerOptions{}, err
}
cgroupParent := newCgroupParent(config)
ro := routerOptions{
sessionManager: sm,
features: d.Features,
daemon: d,
}
bk, err := buildkit.New(ctx, buildkit.Opt{
SessionManager: sm,
@@ -435,18 +438,22 @@ func newRouterOptions(ctx context.Context, config *config.Config, d *daemon.Daem
ContainerdNamespace: config.ContainerdNamespace,
})
if err != nil {
return opts, err
return routerOptions{}, err
}
bb, err := buildbackend.NewBackend(d.ImageService(), manager, bk, d.EventsService)
if err != nil {
return opts, errors.Wrap(err, "failed to create buildmanager")
return routerOptions{}, errors.Wrap(err, "failed to create buildmanager")
}
ro.buildBackend = bb
ro.buildkit = bk
return ro, nil
return routerOptions{
sessionManager: sm,
buildBackend: bb,
features: d.Features,
buildkit: bk,
daemon: d,
cluster: c,
}, nil
}
func (cli *DaemonCli) reloadConfig() {
@@ -622,6 +629,10 @@ func loadDaemonCliConfig(opts *daemonOptions) (*config.Config, error) {
conf.CDISpecDirs = nil
}
if err := loadCLIPlatformConfig(conf); err != nil {
return nil, err
}
return conf, nil
}
@@ -708,14 +719,15 @@ func (opts routerOptions) Build() []router.Router {
return routers
}
func initMiddlewares(s *apiserver.Server, cfg *config.Config, pluginStore plugingetter.PluginGetter) *authorization.Middleware {
v := dockerversion.Version
func initMiddlewares(s *apiserver.Server, cfg *config.Config, pluginStore plugingetter.PluginGetter) (*authorization.Middleware, error) {
exp := middleware.NewExperimentalMiddleware(cfg.Experimental)
s.UseMiddleware(exp)
vm := middleware.NewVersionMiddleware(v, api.DefaultVersion, cfg.MinAPIVersion)
s.UseMiddleware(vm)
vm, err := middleware.NewVersionMiddleware(dockerversion.Version, api.DefaultVersion, cfg.MinAPIVersion)
if err != nil {
return nil, err
}
s.UseMiddleware(*vm)
if cfg.CorsHeaders != "" {
c := middleware.NewCORSMiddleware(cfg.CorsHeaders)
@@ -724,7 +736,7 @@ func initMiddlewares(s *apiserver.Server, cfg *config.Config, pluginStore plugin
authzMiddleware := authorization.NewMiddleware(cfg.AuthorizationPlugins, pluginStore)
s.UseMiddleware(authzMiddleware)
return authzMiddleware
return authzMiddleware, nil
}
func (cli *DaemonCli) getContainerdDaemonOpts() ([]supervisor.DaemonOpt, error) {
@@ -832,6 +844,7 @@ func loadListeners(cfg *config.Config, tlsConfig *tls.Config) ([]net.Listener, [
if proto == "tcp" && !authEnabled {
log.G(ctx).WithField("host", protoAddr).Warn("Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network.")
log.G(ctx).WithField("host", protoAddr).Warn("Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!")
log.G(ctx).WithField("host", protoAddr).Warn("[DEPRECATION NOTICE] In future versions this will be a hard failure preventing the daemon from starting! Learn more at: https://docs.docker.com/go/api-security/")
time.Sleep(time.Second)
// If TLSVerify is explicitly set to false we'll take that as "Please let me shoot myself in the foot"

View File

@@ -3,11 +3,28 @@ package main
import (
cdcgroups "github.com/containerd/cgroups/v3"
systemdDaemon "github.com/coreos/go-systemd/v22/daemon"
"github.com/docker/docker/daemon"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/pkg/sysinfo"
"github.com/pkg/errors"
)
// loadCLIPlatformConfig loads the platform specific CLI configuration
func loadCLIPlatformConfig(conf *config.Config) error {
if conf.RemappedRoot == "" {
return nil
}
containerdNamespace, containerdPluginNamespace, err := daemon.RemapContainerdNamespaces(conf)
if err != nil {
return err
}
conf.ContainerdNamespace = containerdNamespace
conf.ContainerdPluginNamespace = containerdPluginNamespace
return nil
}
// preNotifyReady sends a message to the host when the API is active, but before the daemon is
func preNotifyReady() {
}

View File

@@ -16,6 +16,12 @@ func getDefaultDaemonConfigFile() (string, error) {
return "", nil
}
// loadCLIPlatformConfig loads the platform specific CLI configuration
// there is none on windows, so this is a no-op
func loadCLIPlatformConfig(conf *config.Config) error {
return nil
}
// setDefaultUmask doesn't do anything on windows
func setDefaultUmask() error {
return nil

View File

@@ -5,7 +5,7 @@ import (
"sync"
)
// attachContext is the context used for for attach calls.
// attachContext is the context used for attach calls.
type attachContext struct {
mu sync.Mutex
ctx context.Context

View File

@@ -514,14 +514,14 @@ func (container *Container) AddMountPointWithVolume(destination string, vol volu
}
// UnmountVolumes unmounts all volumes
func (container *Container) UnmountVolumes(volumeEventLog func(name string, action events.Action, attributes map[string]string)) error {
func (container *Container) UnmountVolumes(ctx context.Context, volumeEventLog func(name string, action events.Action, attributes map[string]string)) error {
var errs []string
for _, volumeMount := range container.MountPoints {
if volumeMount.Volume == nil {
continue
}
if err := volumeMount.Cleanup(); err != nil {
if err := volumeMount.Cleanup(ctx); err != nil {
errs = append(errs, err.Error())
continue
}

View File

@@ -15,8 +15,6 @@ import (
"github.com/docker/docker/api/types/events"
mounttypes "github.com/docker/docker/api/types/mount"
swarmtypes "github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/pkg/stringid"
"github.com/docker/docker/volume"
volumemounts "github.com/docker/docker/volume/mounts"
"github.com/moby/sys/mount"
"github.com/opencontainers/selinux/go-selinux/label"
@@ -129,34 +127,11 @@ func (container *Container) NetworkMounts() []Mount {
}
// CopyImagePathContent copies files in destination to the volume.
func (container *Container) CopyImagePathContent(v volume.Volume, destination string) error {
rootfs, err := container.GetResourcePath(destination)
if err != nil {
func (container *Container) CopyImagePathContent(volumePath, destination string) error {
if err := label.Relabel(volumePath, container.MountLabel, true); err != nil && !errors.Is(err, syscall.ENOTSUP) {
return err
}
if _, err := os.Stat(rootfs); err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
id := stringid.GenerateRandomID()
path, err := v.Mount(id)
if err != nil {
return err
}
defer func() {
if err := v.Unmount(id); err != nil {
log.G(context.TODO()).Warnf("error while unmounting volume %s: %v", v.Name(), err)
}
}()
if err := label.Relabel(path, container.MountLabel, true); err != nil && !errors.Is(err, syscall.ENOTSUP) {
return err
}
return copyExistingContents(rootfs, path)
return copyExistingContents(destination, volumePath)
}
// ShmResourcePath returns path to shm
@@ -396,7 +371,7 @@ func (container *Container) DetachAndUnmount(volumeEventLog func(name string, ac
Warn("Unable to unmount")
}
}
return container.UnmountVolumes(volumeEventLog)
return container.UnmountVolumes(ctx, volumeEventLog)
}
// ignoreUnsupportedXAttrs ignores errors when extended attributes
@@ -419,9 +394,13 @@ func copyExistingContents(source, destination string) error {
return err
}
if len(dstList) != 0 {
// destination is not empty, do not copy
log.G(context.TODO()).WithFields(log.Fields{
"source": source,
"destination": destination,
}).Debug("destination is not empty, do not copy")
return nil
}
return fs.CopyDir(destination, source, ignoreUnsupportedXAttrs())
}

View File

@@ -1,6 +1,7 @@
package container // import "github.com/docker/docker/container"
import (
"context"
"fmt"
"os"
"path/filepath"
@@ -128,7 +129,7 @@ func (container *Container) ConfigMounts() []Mount {
// On Windows it only delegates to `UnmountVolumes` since there is nothing to
// force unmount.
func (container *Container) DetachAndUnmount(volumeEventLog func(name string, action events.Action, attributes map[string]string)) error {
return container.UnmountVolumes(volumeEventLog)
return container.UnmountVolumes(context.TODO(), volumeEventLog)
}
// TmpfsMounts returns the list of tmpfs mounts

View File

@@ -15,6 +15,7 @@
# * DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=(builtin|slirp4netns|implicit): the rootlesskit port driver. Defaults to "builtin".
# * DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX=(auto|true|false): whether to protect slirp4netns with a dedicated mount namespace. Defaults to "auto".
# * DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP=(auto|true|false): whether to protect slirp4netns with seccomp. Defaults to "auto".
# * DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK=(true|false): prohibit connections to 127.0.0.1 on the host (including via 10.0.2.2, in the case of slirp4netns). Defaults to "true".
# To apply an environment variable via systemd, create ~/.config/systemd/user/docker.service.d/override.conf as follows,
# and run `systemctl --user daemon-reload && systemctl --user restart docker`:
@@ -71,6 +72,7 @@ fi
: "${DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER:=builtin}"
: "${DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX:=auto}"
: "${DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP:=auto}"
: "${DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK:=}"
net=$DOCKERD_ROOTLESS_ROOTLESSKIT_NET
mtu=$DOCKERD_ROOTLESS_ROOTLESSKIT_MTU
if [ -z "$net" ]; then
@@ -98,6 +100,11 @@ if [ -z "$mtu" ]; then
mtu=1500
fi
host_loopback="--disable-host-loopback"
if [ "$DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK" = "false" ]; then
host_loopback=""
fi
dockerd="${DOCKERD:-dockerd}"
if [ -z "$_DOCKERD_ROOTLESS_CHILD" ]; then
@@ -125,7 +132,7 @@ if [ -z "$_DOCKERD_ROOTLESS_CHILD" ]; then
--net=$net --mtu=$mtu \
--slirp4netns-sandbox=$DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX \
--slirp4netns-seccomp=$DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP \
--disable-host-loopback --port-driver=$DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER \
$host_loopback --port-driver=$DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER \
--copy-up=/etc --copy-up=/run \
--propagation=rslave \
$DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS \

View File

@@ -8,25 +8,6 @@ import (
"github.com/docker/docker/errdefs"
)
// ContainerCopy performs a deprecated operation of archiving the resource at
// the specified path in the container identified by the given name.
func (daemon *Daemon) ContainerCopy(name string, res string) (io.ReadCloser, error) {
ctr, err := daemon.GetContainer(name)
if err != nil {
return nil, err
}
data, err := daemon.containerCopy(ctr, res)
if err == nil {
return data, nil
}
if os.IsNotExist(err) {
return nil, containerFileNotFound{res, name}
}
return nil, errdefs.System(err)
}
// ContainerStatPath stats the filesystem resource at the specified path in the
// container identified by the given name.
func (daemon *Daemon) ContainerStatPath(name string, path string) (stat *types.ContainerPathStat, err error) {

View File

@@ -161,55 +161,6 @@ func (daemon *Daemon) containerExtractToDir(container *container.Container, path
return nil
}
func (daemon *Daemon) containerCopy(container *container.Container, resource string) (rc io.ReadCloser, err error) {
container.Lock()
defer func() {
if err != nil {
// Wait to unlock the container until the archive is fully read
// (see the ReadCloseWrapper func below) or if there is an error
// before that occurs.
container.Unlock()
}
}()
cfs, err := daemon.openContainerFS(container)
if err != nil {
return nil, err
}
defer func() {
if err != nil {
cfs.Close()
}
}()
err = cfs.RunInFS(context.TODO(), func() error {
_, err := os.Stat(resource)
return err
})
if err != nil {
return nil, err
}
tb, err := archive.NewTarballer(resource, &archive.TarOptions{
Compression: archive.Uncompressed,
})
if err != nil {
return nil, err
}
cfs.GoInFS(context.TODO(), tb.Do)
archv := tb.Reader()
reader := ioutils.NewReadCloserWrapper(archv, func() error {
err := archv.Close()
_ = cfs.Close()
container.Unlock()
return err
})
daemon.LogContainerEvent(container, events.ActionCopy)
return reader, nil
}
// checkIfPathIsInAVolume checks if the path is in a volume. If it is, it
// cannot be in a read-only volume. If it is not in a volume, the container
// cannot be configured with a read-only rootfs.

View File

@@ -109,11 +109,11 @@ func TestServiceConvertToGRPCGenericRuntimePlugin(t *testing.T) {
}
func TestServiceConvertToGRPCContainerRuntime(t *testing.T) {
image := "alpine:latest"
const imgName = "alpine:latest"
s := swarmtypes.ServiceSpec{
TaskTemplate: swarmtypes.TaskSpec{
ContainerSpec: &swarmtypes.ContainerSpec{
Image: image,
Image: imgName,
},
},
Mode: swarmtypes.ServiceMode{
@@ -131,8 +131,8 @@ func TestServiceConvertToGRPCContainerRuntime(t *testing.T) {
t.Fatal("expected type swarmapi.TaskSpec_Container")
}
if v.Container.Image != image {
t.Fatalf("expected image %s; received %s", image, v.Container.Image)
if v.Container.Image != imgName {
t.Fatalf("expected image %s; received %s", imgName, v.Container.Image)
}
}

View File

@@ -12,7 +12,6 @@ import (
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/events"
"github.com/docker/docker/api/types/filters"
opts "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/swarm"
@@ -39,7 +38,7 @@ type Backend interface {
SetupIngress(clustertypes.NetworkCreateRequest, string) (<-chan struct{}, error)
ReleaseIngress() (<-chan struct{}, error)
CreateManagedContainer(ctx context.Context, config backend.ContainerCreateConfig) (container.CreateResponse, error)
ContainerStart(ctx context.Context, name string, hostConfig *container.HostConfig, checkpoint string, checkpointDir string) error
ContainerStart(ctx context.Context, name string, checkpoint string, checkpointDir string) error
ContainerStop(ctx context.Context, name string, config container.StopOptions) error
ContainerLogs(ctx context.Context, name string, config *container.LogsOptions) (msgs <-chan *backend.LogMessage, tty bool, err error)
ConnectContainerToNetwork(containerName, networkName string, endpointConfig *network.EndpointSettings) error
@@ -78,5 +77,5 @@ type VolumeBackend interface {
type ImageBackend interface {
PullImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
GetRepositories(context.Context, reference.Named, *registry.AuthConfig) ([]distribution.Repository, error)
GetImage(ctx context.Context, refOrID string, options opts.GetImageOpts) (*image.Image, error)
GetImage(ctx context.Context, refOrID string, options backend.GetImageOpts) (*image.Image, error)
}

View File

@@ -17,7 +17,6 @@ import (
"github.com/docker/docker/api/types/backend"
containertypes "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/events"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
containerpkg "github.com/docker/docker/container"
"github.com/docker/docker/daemon"
@@ -76,7 +75,7 @@ func (c *containerAdapter) pullImage(ctx context.Context) error {
named, err := reference.ParseNormalizedNamed(spec.Image)
if err == nil {
if _, ok := named.(reference.Canonical); ok {
_, err := c.imageBackend.GetImage(ctx, spec.Image, imagetypes.GetImageOpts{})
_, err := c.imageBackend.GetImage(ctx, spec.Image, backend.GetImageOpts{})
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return err
}
@@ -348,7 +347,7 @@ func (c *containerAdapter) start(ctx context.Context) error {
return err
}
return c.backend.ContainerStart(ctx, c.container.name(), nil, "", "")
return c.backend.ContainerStart(ctx, c.container.name(), "", "")
}
func (c *containerAdapter) inspect(ctx context.Context) (types.ContainerJSON, error) {

View File

@@ -56,9 +56,9 @@ const (
DefaultPluginNamespace = "plugins.moby"
// defaultMinAPIVersion is the minimum API version supported by the API.
// This version can be overridden through the "DOCKER_MIN_API_VERSION"
// environment variable. The minimum allowed version is determined
// by [minAPIVersion].
defaultMinAPIVersion = "1.24"
// environment variable. It currently defaults to the minimum API version
// supported by the API server.
defaultMinAPIVersion = api.MinSupportedAPIVersion
// SeccompProfileDefault is the built-in default seccomp profile.
SeccompProfileDefault = "builtin"
// SeccompProfileUnconfined is a special profile name for seccomp to use an
@@ -610,8 +610,8 @@ func ValidateMinAPIVersion(ver string) error {
if strings.EqualFold(ver[0:1], "v") {
return errors.New(`API version must be provided without "v" prefix`)
}
if versions.LessThan(ver, minAPIVersion) {
return errors.Errorf(`minimum supported API version is %s: %s`, minAPIVersion, ver)
if versions.LessThan(ver, defaultMinAPIVersion) {
return errors.Errorf(`minimum supported API version is %s: %s`, defaultMinAPIVersion, ver)
}
if versions.GreaterThan(ver, api.DefaultVersion) {
return errors.Errorf(`maximum supported API version is %s: %s`, api.DefaultVersion, ver)

View File

@@ -33,9 +33,6 @@ const (
// OCI runtime being shipped with the docker daemon package.
StockRuntimeName = "runc"
// minAPIVersion represents Minimum REST API version supported
minAPIVersion = "1.12"
// userlandProxyBinary is the name of the userland-proxy binary.
// In rootless-mode, [rootless.RootlessKitDockerProxyBinary] is used instead.
userlandProxyBinary = "docker-proxy"
@@ -83,7 +80,6 @@ type Config struct {
Ulimits map[string]*units.Ulimit `json:"default-ulimits,omitempty"`
CPURealtimePeriod int64 `json:"cpu-rt-period,omitempty"`
CPURealtimeRuntime int64 `json:"cpu-rt-runtime,omitempty"`
OOMScoreAdjust int `json:"oom-score-adjust,omitempty"` // Deprecated: configure the daemon's oom-score-adjust using a process manager instead.
Init bool `json:"init,omitempty"`
InitPath string `json:"init-path,omitempty"`
SeccompProfile string `json:"seccomp-profile,omitempty"`
@@ -178,10 +174,6 @@ func verifyDefaultCgroupNsMode(mode string) error {
// ValidatePlatformConfig checks if any platform-specific configuration settings are invalid.
func (conf *Config) ValidatePlatformConfig() error {
if conf.OOMScoreAdjust != 0 {
return errors.New(`DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.`)
}
if conf.EnableUserlandProxy {
if conf.UserlandProxyPath == "" {
return errors.New("invalid userland-proxy-path: userland-proxy is enabled, but userland-proxy-path is not set")

View File

@@ -13,13 +13,6 @@ const (
// default value. On Windows keep this empty so the value is auto-detected
// based on other options.
StockRuntimeName = ""
// minAPIVersion represents Minimum REST API version supported
// Technically the first daemon API version released on Windows is v1.25 in
// engine version 1.13. However, some clients are explicitly using downlevel
// APIs (e.g. docker-compose v2.1 file format) and that is just too restrictive.
// Hence also allowing 1.24 on Windows.
minAPIVersion string = "1.24"
)
// BridgeConfig is meant to store all the parameters for both the bridge driver and the default bridge network. On

View File

@@ -99,6 +99,45 @@ func (daemon *Daemon) getPIDContainer(id string) (*container.Container, error) {
return ctr, nil
}
// setupContainerDirs sets up base container directories (root, ipc, tmpfs and secrets).
func (daemon *Daemon) setupContainerDirs(c *container.Container) (_ []container.Mount, err error) {
if err := daemon.setupContainerMountsRoot(c); err != nil {
return nil, err
}
if err := daemon.setupIPCDirs(c); err != nil {
return nil, err
}
if err := daemon.setupSecretDir(c); err != nil {
return nil, err
}
defer func() {
if err != nil {
daemon.cleanupSecretDir(c)
}
}()
var ms []container.Mount
if !c.HostConfig.IpcMode.IsPrivate() && !c.HostConfig.IpcMode.IsEmpty() {
ms = append(ms, c.IpcMounts()...)
}
tmpfsMounts, err := c.TmpfsMounts()
if err != nil {
return nil, err
}
ms = append(ms, tmpfsMounts...)
secretMounts, err := c.SecretMounts()
if err != nil {
return nil, err
}
ms = append(ms, secretMounts...)
return ms, nil
}
func (daemon *Daemon) setupIPCDirs(c *container.Container) error {
ipcMode := c.HostConfig.IpcMode
@@ -380,6 +419,7 @@ func serviceDiscoveryOnDefaultNetwork() bool {
func setupPathsAndSandboxOptions(container *container.Container, cfg *config.Config, sboxOptions *[]libnetwork.SandboxOption) error {
var err error
var originResolvConfPath string
// Set the correct paths for /etc/hosts and /etc/resolv.conf, based on the
// networking-mode of the container. Note that containers with "container"
@@ -393,8 +433,8 @@ func setupPathsAndSandboxOptions(container *container.Container, cfg *config.Con
*sboxOptions = append(
*sboxOptions,
libnetwork.OptionOriginHostsPath("/etc/hosts"),
libnetwork.OptionOriginResolvConfPath("/etc/resolv.conf"),
)
originResolvConfPath = "/etc/resolv.conf"
case container.HostConfig.NetworkMode.IsUserDefined():
// The container uses a user-defined network. We use the embedded DNS
// server for container name resolution and to act as a DNS forwarder
@@ -407,10 +447,7 @@ func setupPathsAndSandboxOptions(container *container.Container, cfg *config.Con
// If systemd-resolvd is used, the "upstream" DNS servers can be found in
// /run/systemd/resolve/resolv.conf. We do not query those DNS servers
// directly, as they can be dynamically reconfigured.
*sboxOptions = append(
*sboxOptions,
libnetwork.OptionOriginResolvConfPath("/etc/resolv.conf"),
)
originResolvConfPath = "/etc/resolv.conf"
default:
// For other situations, such as the default bridge network, container
// discovery / name resolution is handled through /etc/hosts, and no
@@ -423,12 +460,16 @@ func setupPathsAndSandboxOptions(container *container.Container, cfg *config.Con
// DNS servers on the host can be dynamically updated.
//
// Copy the host's resolv.conf for the container (/run/systemd/resolve/resolv.conf or /etc/resolv.conf)
*sboxOptions = append(
*sboxOptions,
libnetwork.OptionOriginResolvConfPath(cfg.GetResolvConf()),
)
originResolvConfPath = cfg.GetResolvConf()
}
// Allow tests to point at their own resolv.conf file.
if envPath := os.Getenv("DOCKER_TEST_RESOLV_CONF_PATH"); envPath != "" {
log.G(context.TODO()).Infof("Using OriginResolvConfPath from env: %s", envPath)
originResolvConfPath = envPath
}
*sboxOptions = append(*sboxOptions, libnetwork.OptionOriginResolvConfPath(originResolvConfPath))
container.HostsPath, err = container.GetRootResourcePath("hosts")
if err != nil {
return err

View File

@@ -2,211 +2,226 @@ package containerd
import (
"context"
"reflect"
"strings"
"encoding/json"
"errors"
"fmt"
"github.com/containerd/containerd/content"
cerrdefs "github.com/containerd/containerd/errdefs"
"github.com/containerd/log"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
imagetype "github.com/docker/docker/api/types/image"
"github.com/docker/docker/builder"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/image"
"github.com/docker/docker/image/cache"
"github.com/docker/docker/internal/multierror"
"github.com/docker/docker/layer"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// MakeImageCache creates a stateful image cache.
func (i *ImageService) MakeImageCache(ctx context.Context, cacheFrom []string) (builder.ImageCache, error) {
images := []*image.Image{}
if len(cacheFrom) == 0 {
return &localCache{
imageService: i,
}, nil
}
for _, c := range cacheFrom {
h, err := i.ImageHistory(ctx, c)
if err != nil {
continue
}
for _, hi := range h {
if hi.ID != "<missing>" {
im, err := i.GetImage(ctx, hi.ID, imagetype.GetImageOpts{})
if err != nil {
return nil, err
}
images = append(images, im)
}
}
}
return &imageCache{
lc: &localCache{
imageService: i,
},
images: images,
imageService: i,
}, nil
func (i *ImageService) MakeImageCache(ctx context.Context, sourceRefs []string) (builder.ImageCache, error) {
return cache.New(ctx, cacheAdaptor{i}, sourceRefs)
}
type localCache struct {
imageService *ImageService
type cacheAdaptor struct {
is *ImageService
}
func (ic *localCache) GetCache(parentID string, cfg *container.Config, platform ocispec.Platform) (imageID string, err error) {
func (c cacheAdaptor) Get(id image.ID) (*image.Image, error) {
ctx := context.TODO()
ref := id.String()
var children []image.ID
// FROM scratch
if parentID == "" {
c, err := ic.imageService.getImagesWithLabel(ctx, imageLabelClassicBuilderFromScratch, "1")
if err != nil {
return "", err
}
children = c
} else {
c, err := ic.imageService.Children(ctx, image.ID(parentID))
if err != nil {
return "", err
}
children = c
outImg, err := c.is.GetImage(ctx, id.String(), backend.GetImageOpts{})
if err != nil {
return nil, fmt.Errorf("GetImage: %w", err)
}
var match *image.Image
for _, child := range children {
ccDigestStr, err := ic.imageService.getImageLabelByDigest(ctx, child.Digest(), imageLabelClassicBuilderContainerConfig)
c8dImg, err := c.is.resolveImage(ctx, ref)
if err != nil {
return nil, fmt.Errorf("resolveImage: %w", err)
}
var errFound = errors.New("success")
err = c.is.walkImageManifests(ctx, c8dImg, func(img *ImageManifest) error {
desc, err := img.Config(ctx)
if err != nil {
return "", err
}
if ccDigestStr == "" {
continue
log.G(ctx).WithFields(log.Fields{
"image": img,
"error": err,
}).Warn("failed to get config descriptor for image")
return nil
}
dgst, err := digest.Parse(ccDigestStr)
info, err := c.is.content.Info(ctx, desc.Digest)
if err != nil {
log.G(ctx).WithError(err).Warnf("invalid container config digest: %q", ccDigestStr)
continue
}
var cc container.Config
if err := readConfig(ctx, ic.imageService.content, ocispec.Descriptor{Digest: dgst}, &cc); err != nil {
if errdefs.IsNotFound(err) {
log.G(ctx).WithError(err).WithField("image", child).Warnf("missing container config: %q", ccDigestStr)
continue
if !cerrdefs.IsNotFound(err) {
log.G(ctx).WithFields(log.Fields{
"image": img,
"desc": desc,
"error": err,
}).Warn("failed to get info of image config")
}
return "", err
return nil
}
if cache.CompareConfig(&cc, cfg) {
childImage, err := ic.imageService.GetImage(ctx, child.String(), imagetype.GetImageOpts{Platform: &platform})
if dgstStr, ok := info.Labels[contentLabelGcRefContainerConfig]; ok {
dgst, err := digest.Parse(dgstStr)
if err != nil {
if errdefs.IsNotFound(err) {
continue
}
return "", err
log.G(ctx).WithFields(log.Fields{
"label": contentLabelClassicBuilderImage,
"value": dgstStr,
"content": desc.Digest,
"error": err,
}).Warn("invalid digest in label")
return nil
}
if childImage.Created != nil && (match == nil || match.Created.Before(*childImage.Created)) {
match = childImage
configDesc := ocispec.Descriptor{
Digest: dgst,
}
var config container.Config
if err := readConfig(ctx, c.is.content, configDesc, &config); err != nil {
if !errdefs.IsNotFound(err) {
log.G(ctx).WithFields(log.Fields{
"configDigest": dgst,
"error": err,
}).Warn("failed to read container config")
}
return nil
}
outImg.ContainerConfig = config
// We already have the config we looked for, so return an error to
// stop walking the image further. This error will be ignored.
return errFound
}
return nil
})
if err != nil && err != errFound {
return nil, err
}
return outImg, nil
}
func (c cacheAdaptor) GetByRef(ctx context.Context, refOrId string) (*image.Image, error) {
return c.is.GetImage(ctx, refOrId, backend.GetImageOpts{})
}
func (c cacheAdaptor) SetParent(target, parent image.ID) error {
ctx := context.TODO()
_, imgs, err := c.is.resolveAllReferences(ctx, target.String())
if err != nil {
return fmt.Errorf("failed to list images with digest %q", target)
}
var errs []error
is := c.is.images
for _, img := range imgs {
if img.Labels == nil {
img.Labels = make(map[string]string)
}
img.Labels[imageLabelClassicBuilderParent] = parent.String()
if _, err := is.Update(ctx, img, "labels."+imageLabelClassicBuilderParent); err != nil {
errs = append(errs, fmt.Errorf("failed to update parent label on image %v: %w", img, err))
}
}
if match == nil {
return "", nil
return multierror.Join(errs...)
}
func (c cacheAdaptor) GetParent(target image.ID) (image.ID, error) {
ctx := context.TODO()
value, err := c.is.getImageLabelByDigest(ctx, target.Digest(), imageLabelClassicBuilderParent)
if err != nil {
return "", fmt.Errorf("failed to read parent image: %w", err)
}
return match.ID().String(), nil
dgst, err := digest.Parse(value)
if err != nil {
return "", fmt.Errorf("invalid parent value: %q", value)
}
return image.ID(dgst), nil
}
type imageCache struct {
images []*image.Image
imageService *ImageService
lc *localCache
func (c cacheAdaptor) Create(parent *image.Image, target image.Image, extraLayer layer.DiffID) (image.ID, error) {
ctx := context.TODO()
data, err := json.Marshal(target)
if err != nil {
return "", fmt.Errorf("failed to marshal image config: %w", err)
}
var layerDigest digest.Digest
if extraLayer != "" {
info, err := findContentByUncompressedDigest(ctx, c.is.client.ContentStore(), digest.Digest(extraLayer))
if err != nil {
return "", fmt.Errorf("failed to find content for diff ID %q: %w", extraLayer, err)
}
layerDigest = info.Digest
}
var parentRef string
if parent != nil {
parentRef = parent.ID().String()
}
img, err := c.is.CreateImage(ctx, data, parentRef, layerDigest)
if err != nil {
return "", fmt.Errorf("failed to created cached image: %w", err)
}
return image.ID(img.ImageID()), nil
}
func (ic *imageCache) GetCache(parentID string, cfg *container.Config, platform ocispec.Platform) (imageID string, err error) {
func (c cacheAdaptor) IsBuiltLocally(target image.ID) (bool, error) {
ctx := context.TODO()
value, err := c.is.getImageLabelByDigest(ctx, target.Digest(), imageLabelClassicBuilderContainerConfig)
if err != nil {
return false, fmt.Errorf("failed to read container config label: %w", err)
}
return value != "", nil
}
func (c cacheAdaptor) Children(id image.ID) []image.ID {
ctx := context.TODO()
imgID, err := ic.lc.GetCache(parentID, cfg, platform)
if err != nil {
return "", err
}
if imgID != "" {
for _, s := range ic.images {
if ic.isParent(ctx, s, image.ID(imgID)) {
return imgID, nil
}
}
}
var parent *image.Image
lenHistory := 0
if parentID != "" {
parent, err = ic.imageService.GetImage(ctx, parentID, imagetype.GetImageOpts{Platform: &platform})
if id.String() == "" {
imgs, err := c.is.getImagesWithLabel(ctx, imageLabelClassicBuilderFromScratch, "1")
if err != nil {
return "", err
log.G(ctx).WithError(err).Error("failed to get from scratch images")
return nil
}
lenHistory = len(parent.History)
}
for _, target := range ic.images {
if !isValidParent(target, parent) || !isValidConfig(cfg, target.History[lenHistory]) {
continue
}
return target.ID().String(), nil
return imgs
}
return "", nil
}
func isValidConfig(cfg *container.Config, h image.History) bool {
// todo: make this format better than join that loses data
return strings.Join(cfg.Cmd, " ") == h.CreatedBy
}
func isValidParent(img, parent *image.Image) bool {
if len(img.History) == 0 {
return false
}
if parent == nil || len(parent.History) == 0 && len(parent.RootFS.DiffIDs) == 0 {
return true
}
if len(parent.History) >= len(img.History) {
return false
}
if len(parent.RootFS.DiffIDs) > len(img.RootFS.DiffIDs) {
return false
}
for i, h := range parent.History {
if !reflect.DeepEqual(h, img.History[i]) {
return false
}
}
for i, d := range parent.RootFS.DiffIDs {
if d != img.RootFS.DiffIDs[i] {
return false
}
}
return true
}
func (ic *imageCache) isParent(ctx context.Context, img *image.Image, parentID image.ID) bool {
ii, err := ic.imageService.resolveImage(ctx, img.ImageID())
imgs, err := c.is.Children(ctx, id)
if err != nil {
return false
}
parent, ok := ii.Labels[imageLabelClassicBuilderParent]
if ok {
return parent == parentID.String()
log.G(ctx).WithError(err).Error("failed to get image children")
return nil
}
p, err := ic.imageService.GetImage(ctx, parentID.String(), imagetype.GetImageOpts{})
if err != nil {
return false
}
return ic.isParent(ctx, p, parentID)
return imgs
}
func findContentByUncompressedDigest(ctx context.Context, cs content.Manager, uncompressed digest.Digest) (content.Info, error) {
var out content.Info
errStopWalk := errors.New("success")
err := cs.Walk(ctx, func(i content.Info) error {
out = i
return errStopWalk
}, `labels."containerd.io/uncompressed"==`+uncompressed.String())
if err != nil && err != errStopWalk {
return out, err
}
if out.Digest == "" {
return out, errdefs.NotFound(errors.New("no content matches this uncompressed digest"))
}
return out, nil
}

View File

@@ -15,11 +15,11 @@ import (
"github.com/containerd/containerd/platforms"
"github.com/containerd/log"
"github.com/distribution/reference"
imagetype "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/daemon/images"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/image"
imagespec "github.com/docker/docker/image/spec/specs-go/v1"
imagespec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -31,7 +31,7 @@ var truncatedID = regexp.MustCompile(`^(sha256:)?([a-f0-9]{4,64})$`)
var errInconsistentData error = errors.New("consistency error: data changed during operation, retry")
// GetImage returns an image corresponding to the image referred to by refOrID.
func (i *ImageService) GetImage(ctx context.Context, refOrID string, options imagetype.GetImageOpts) (*image.Image, error) {
func (i *ImageService) GetImage(ctx context.Context, refOrID string, options backend.GetImageOpts) (*image.Image, error) {
desc, err := i.resolveImage(ctx, refOrID)
if err != nil {
return nil, err
@@ -46,9 +46,9 @@ func (i *ImageService) GetImage(ctx context.Context, refOrID string, options ima
if err != nil {
return nil, err
}
ociimage := presentImages[0]
ociImage := presentImages[0]
img := dockerOciImageToDockerImagePartial(image.ID(desc.Target.Digest), ociimage)
img := dockerOciImageToDockerImagePartial(image.ID(desc.Target.Digest), ociImage)
parent, err := i.getImageLabelByDigest(ctx, desc.Target.Digest, imageLabelClassicBuilderParent)
if err != nil {
@@ -172,13 +172,13 @@ func (i *ImageService) presentImages(ctx context.Context, desc containerdimages.
return presentImages, nil
}
func (i *ImageService) GetImageManifest(ctx context.Context, refOrID string, options imagetype.GetImageOpts) (*ocispec.Descriptor, error) {
func (i *ImageService) GetImageManifest(ctx context.Context, refOrID string, options backend.GetImageOpts) (*ocispec.Descriptor, error) {
platform := matchAllWithPreference(platforms.Default())
if options.Platform != nil {
platform = platforms.Only(*options.Platform)
}
cs := i.client.ContentStore()
cs := i.content
img, err := i.resolveImage(ctx, refOrID)
if err != nil {
@@ -237,7 +237,7 @@ func (i *ImageService) GetImageManifest(ctx context.Context, refOrID string, opt
func (i *ImageService) size(ctx context.Context, desc ocispec.Descriptor, platform platforms.MatchComparer) (int64, error) {
var size int64
cs := i.client.ContentStore()
cs := i.content
handler := containerdimages.LimitManifests(containerdimages.ChildrenHandler(cs), platform, 1)
var wh containerdimages.HandlerFunc = func(ctx context.Context, desc ocispec.Descriptor) ([]ocispec.Descriptor, error) {
@@ -360,7 +360,7 @@ func (i *ImageService) resolveImage(ctx context.Context, refOrID string) (contai
// pointing to the same repository as the given reference.
func (i *ImageService) getAllImagesWithRepository(ctx context.Context, ref reference.Named) ([]containerdimages.Image, error) {
nameFilter := "^" + regexp.QuoteMeta(ref.Name()) + ":" + reference.TagRegexp.String() + "$"
return i.client.ImageService().List(ctx, "name~="+strconv.Quote(nameFilter))
return i.images.List(ctx, "name~="+strconv.Quote(nameFilter))
}
func imageFamiliarName(img containerdimages.Image) string {
@@ -378,7 +378,7 @@ func imageFamiliarName(img containerdimages.Image) string {
// targeting the specified digest.
// If images have different values, an errdefs.Conflict error will be returned.
func (i *ImageService) getImageLabelByDigest(ctx context.Context, target digest.Digest, labelKey string) (string, error) {
imgs, err := i.client.ImageService().List(ctx, "target.digest=="+target.String()+",labels."+labelKey)
imgs, err := i.images.List(ctx, "target.digest=="+target.String()+",labels."+labelKey)
if err != nil {
return "", errdefs.System(err)
}

View File

@@ -23,12 +23,10 @@ import (
"github.com/distribution/reference"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/builder"
"github.com/docker/docker/errdefs"
dimage "github.com/docker/docker/image"
imagespec "github.com/docker/docker/image/spec/specs-go/v1"
"github.com/docker/docker/internal/compatcontext"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/archive"
@@ -36,6 +34,7 @@ import (
"github.com/docker/docker/pkg/streamformatter"
"github.com/docker/docker/pkg/stringid"
registrypkg "github.com/docker/docker/registry"
imagespec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/identity"
"github.com/opencontainers/image-spec/specs-go"
@@ -84,7 +83,7 @@ func (i *ImageService) GetImageAndReleasableLayer(ctx context.Context, refOrID s
if opts.PullOption != backend.PullOptionForcePull {
// TODO(laurazard): same as below
img, err := i.GetImage(ctx, refOrID, imagetypes.GetImageOpts{Platform: opts.Platform})
img, err := i.GetImage(ctx, refOrID, backend.GetImageOpts{Platform: opts.Platform})
if err != nil && opts.PullOption == backend.PullOptionNoPull {
return nil, nil, err
}
@@ -119,7 +118,7 @@ func (i *ImageService) GetImageAndReleasableLayer(ctx context.Context, refOrID s
// TODO(laurazard): pullForBuilder should return whatever we
// need here instead of having to go and get it again
img, err := i.GetImage(ctx, refOrID, imagetypes.GetImageOpts{
img, err := i.GetImage(ctx, refOrID, backend.GetImageOpts{
Platform: opts.Platform,
})
if err != nil {
@@ -156,7 +155,7 @@ func (i *ImageService) pullForBuilder(ctx context.Context, name string, authConf
return nil, err
}
img, err := i.GetImage(ctx, name, imagetypes.GetImageOpts{Platform: platform})
img, err := i.GetImage(ctx, name, backend.GetImageOpts{Platform: platform})
if err != nil {
if errdefs.IsNotFound(err) && img != nil && platform != nil {
imgPlat := ocispec.Platform{
@@ -203,12 +202,12 @@ func newROLayerForImage(ctx context.Context, imgDesc *ocispec.Descriptor, i *Ima
platMatcher = platforms.Only(*platform)
}
confDesc, err := containerdimages.Config(ctx, i.client.ContentStore(), *imgDesc, platMatcher)
confDesc, err := containerdimages.Config(ctx, i.content, *imgDesc, platMatcher)
if err != nil {
return nil, err
}
diffIDs, err := containerdimages.RootFS(ctx, i.client.ContentStore(), confDesc)
diffIDs, err := containerdimages.RootFS(ctx, i.content, confDesc)
if err != nil {
return nil, err
}
@@ -447,7 +446,7 @@ func (i *ImageService) CreateImage(ctx context.Context, config []byte, parent st
if err != nil {
return nil, err
}
parentImageManifest, err := containerdimages.Manifest(ctx, i.client.ContentStore(), parentDesc, platforms.Default())
parentImageManifest, err := containerdimages.Manifest(ctx, i.content, parentDesc, platforms.Default())
if err != nil {
return nil, err
}
@@ -456,7 +455,7 @@ func (i *ImageService) CreateImage(ctx context.Context, config []byte, parent st
parentDigest = parentDesc.Digest
}
cs := i.client.ContentStore()
cs := i.content
ra, err := cs.ReaderAt(ctx, ocispec.Descriptor{Digest: layerDigest})
if err != nil {
@@ -505,7 +504,7 @@ func (i *ImageService) createImageOCI(ctx context.Context, imgToCreate imagespec
}
}()
manifestDesc, ccDesc, err := writeContentsForImage(ctx, i.snapshotter, i.client.ContentStore(), imgToCreate, layers, containerConfig)
manifestDesc, ccDesc, err := writeContentsForImage(ctx, i.snapshotter, i.content, imgToCreate, layers, containerConfig)
if err != nil {
return "", err
}
@@ -524,13 +523,13 @@ func (i *ImageService) createImageOCI(ctx context.Context, imgToCreate imagespec
img.Labels[imageLabelClassicBuilderFromScratch] = "1"
}
createdImage, err := i.client.ImageService().Update(ctx, img)
createdImage, err := i.images.Update(ctx, img)
if err != nil {
if !cerrdefs.IsNotFound(err) {
return "", err
}
if createdImage, err = i.client.ImageService().Create(ctx, img); err != nil {
if createdImage, err = i.images.Create(ctx, img); err != nil {
return "", fmt.Errorf("failed to create new image: %w", err)
}
}

View File

@@ -20,9 +20,9 @@ import (
"github.com/containerd/log"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/image"
imagespec "github.com/docker/docker/image/spec/specs-go/v1"
"github.com/docker/docker/internal/compatcontext"
"github.com/docker/docker/pkg/archive"
imagespec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/identity"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
@@ -38,7 +38,7 @@ with adaptations to match the Moby data model and services.
// CommitImage creates a new image from a commit config.
func (i *ImageService) CommitImage(ctx context.Context, cc backend.CommitConfig) (image.ID, error) {
container := i.containers.Get(cc.ContainerID)
cs := i.client.ContentStore()
cs := i.content
var parentManifest ocispec.Manifest
var parentImage imagespec.DockerOCIImage

View File

@@ -55,7 +55,14 @@ import (
// conflict will not be reported.
//
// TODO(thaJeztah): image delete should send prometheus counters; see https://github.com/moby/moby/issues/45268
func (i *ImageService) ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]imagetypes.DeleteResponse, error) {
func (i *ImageService) ImageDelete(ctx context.Context, imageRef string, force, prune bool) (response []imagetypes.DeleteResponse, retErr error) {
start := time.Now()
defer func() {
if retErr == nil {
dimages.ImageActions.WithValues("delete").UpdateSince(start)
}
}()
var c conflictType
if !force {
c |= conflictSoft
@@ -375,7 +382,7 @@ func (i *ImageService) imageDeleteHelper(ctx context.Context, img images.Image,
CreatedAt: time.Now(),
Labels: img.Labels,
}
if _, err = i.client.ImageService().Create(ctx, img); err != nil && !cerrdefs.IsAlreadyExists(err) {
if _, err = i.images.Create(ctx, img); err != nil && !cerrdefs.IsAlreadyExists(err) {
return fmt.Errorf("failed to create dangling image: %w", err)
}
}

View File

@@ -4,8 +4,8 @@ import (
"context"
"github.com/containerd/containerd/images"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/events"
imagetypes "github.com/docker/docker/api/types/image"
)
// LogImageEvent generates an event related to an image with only the default attributes.
@@ -13,7 +13,7 @@ func (i *ImageService) LogImageEvent(imageID, refName string, action events.Acti
ctx := context.TODO()
attributes := map[string]string{}
img, err := i.GetImage(ctx, imageID, imagetypes.GetImageOpts{})
img, err := i.GetImage(ctx, imageID, backend.GetImageOpts{})
if err == nil && img.Config != nil {
// image has not been removed yet.
// it could be missing if the event is `delete`.

View File

@@ -2,12 +2,14 @@ package containerd
import (
"context"
"time"
containerdimages "github.com/containerd/containerd/images"
"github.com/containerd/containerd/platforms"
"github.com/containerd/log"
"github.com/distribution/reference"
imagetype "github.com/docker/docker/api/types/image"
dimages "github.com/docker/docker/daemon/images"
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/identity"
"github.com/pkg/errors"
@@ -16,6 +18,7 @@ import (
// ImageHistory returns a slice of HistoryResponseItem structures for the
// specified image name by walking the image lineage.
func (i *ImageService) ImageHistory(ctx context.Context, name string) ([]*imagetype.HistoryResponseItem, error) {
start := time.Now()
img, err := i.resolveImage(ctx, name)
if err != nil {
return nil, err
@@ -28,7 +31,7 @@ func (i *ImageService) ImageHistory(ctx context.Context, name string) ([]*imaget
if err != nil {
return nil, err
}
ociimage := presentImages[0]
ociImage := presentImages[0]
var (
history []*imagetype.HistoryResponseItem
@@ -36,7 +39,7 @@ func (i *ImageService) ImageHistory(ctx context.Context, name string) ([]*imaget
)
s := i.client.SnapshotService(i.snapshotter)
diffIDs := ociimage.RootFS.DiffIDs
diffIDs := ociImage.RootFS.DiffIDs
for i := range diffIDs {
chainID := identity.ChainID(diffIDs[0 : i+1]).String()
@@ -48,7 +51,7 @@ func (i *ImageService) ImageHistory(ctx context.Context, name string) ([]*imaget
sizes = append(sizes, use.Size)
}
for _, h := range ociimage.History {
for _, h := range ociImage.History {
size := int64(0)
if !h.EmptyLayer {
if len(sizes) == 0 {
@@ -84,13 +87,12 @@ func (i *ImageService) ImageHistory(ctx context.Context, name string) ([]*imaget
return imgs
}
is := i.client.ImageService()
currentImg := img
for _, h := range history {
dgst := currentImg.Target.Digest.String()
h.ID = dgst
imgs, err := is.List(ctx, "target.digest=="+dgst)
imgs, err := i.images.List(ctx, "target.digest=="+dgst)
if err != nil {
return nil, err
}
@@ -114,6 +116,7 @@ func (i *ImageService) ImageHistory(ctx context.Context, name string) ([]*imaget
}
}
dimages.ImageActions.WithValues("history").UpdateSince(start)
return history, nil
}
@@ -157,5 +160,5 @@ func (i *ImageService) getParentsByBuilderLabel(ctx context.Context, img contain
return nil, nil
}
return i.client.ImageService().List(ctx, "target.digest=="+dgst.String())
return i.images.List(ctx, "target.digest=="+dgst.String())
}

View File

@@ -20,11 +20,11 @@ import (
"github.com/docker/docker/builder/dockerfile"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/image"
imagespec "github.com/docker/docker/image/spec/specs-go/v1"
"github.com/docker/docker/internal/compatcontext"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/pools"
"github.com/google/uuid"
imagespec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/specs-go"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
@@ -66,7 +66,7 @@ func (i *ImageService) ImportImage(ctx context.Context, ref reference.Named, pla
return "", errdefs.InvalidParameter(err)
}
cs := i.client.ContentStore()
cs := i.content
compressedDigest, uncompressedDigest, mt, err := saveArchive(ctx, cs, layerReader)
if err != nil {
@@ -299,11 +299,9 @@ func writeBlobAndReturnDigest(ctx context.Context, cs content.Store, mt string,
// saveImage creates an image in the ImageService or updates it if it exists.
func (i *ImageService) saveImage(ctx context.Context, img images.Image) error {
is := i.client.ImageService()
if _, err := is.Update(ctx, img); err != nil {
if _, err := i.images.Update(ctx, img); err != nil {
if cerrdefs.IsNotFound(err) {
if _, err := is.Create(ctx, img); err != nil {
if _, err := i.images.Create(ctx, img); err != nil {
return errdefs.Unknown(err)
}
} else {

View File

@@ -3,28 +3,33 @@ package containerd
import (
"context"
"encoding/json"
"runtime"
"sort"
"strings"
"sync"
"time"
"github.com/containerd/containerd/content"
cerrdefs "github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/labels"
"github.com/containerd/containerd/platforms"
cplatforms "github.com/containerd/containerd/platforms"
"github.com/containerd/containerd/snapshots"
"github.com/containerd/log"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
imagetypes "github.com/docker/docker/api/types/image"
timetypes "github.com/docker/docker/api/types/time"
"github.com/docker/docker/container"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/image"
dockerspec "github.com/moby/docker-image-spec/specs-go/v1"
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/identity"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
// Subset of ocispec.Image that only contains Labels
@@ -57,10 +62,9 @@ func (r byCreated) Less(i, j int) bool { return r[i].Created < r[j].Created }
// Images returns a filtered list of images.
//
// TODO(thaJeztah): implement opts.ContainerCount (used for docker system df); see https://github.com/moby/moby/issues/43853
// TODO(thaJeztah): verify behavior of `RepoDigests` and `RepoTags` for images without (untagged) or multiple tags; see https://github.com/moby/moby/issues/43861
// TODO(thaJeztah): verify "Size" vs "VirtualSize" in images; see https://github.com/moby/moby/issues/43862
func (i *ImageService) Images(ctx context.Context, opts types.ImageListOptions) ([]*imagetypes.Summary, error) {
func (i *ImageService) Images(ctx context.Context, opts imagetypes.ListOptions) ([]*imagetypes.Summary, error) {
if err := opts.Filters.Validate(acceptedImageFilterTags); err != nil {
return nil, err
}
@@ -70,13 +74,13 @@ func (i *ImageService) Images(ctx context.Context, opts types.ImageListOptions)
return nil, err
}
imgs, err := i.client.ImageService().List(ctx)
imgs, err := i.images.List(ctx)
if err != nil {
return nil, err
}
// TODO(thaJeztah): do we need to take multiple snapshotters into account? See https://github.com/moby/moby/issues/45273
snapshotter := i.client.SnapshotService(i.snapshotter)
snapshotter := i.snapshotterService(i.snapshotter)
sizeCache := make(map[digest.Digest]int64)
snapshotSizeFn := func(d digest.Digest) (int64, error) {
if s, ok := sizeCache[d]; ok {
@@ -90,18 +94,6 @@ func (i *ImageService) Images(ctx context.Context, opts types.ImageListOptions)
return usage.Size, nil
}
var (
allContainers []*container.Container
summaries = make([]*imagetypes.Summary, 0, len(imgs))
root []*[]digest.Digest
layers map[digest.Digest]int
)
if opts.SharedSize {
root = make([]*[]digest.Digest, 0, len(imgs))
layers = make(map[digest.Digest]int)
}
contentStore := i.client.ContentStore()
uniqueImages := map[digest.Digest]images.Image{}
tagsByDigest := map[digest.Digest][]string{}
intermediateImages := map[digest.Digest]struct{}{}
@@ -123,6 +115,9 @@ func (i *ImageService) Images(ctx context.Context, opts types.ImageListOptions)
}
}
// TODO: Allow platform override?
platformMatcher := matchAllWithPreference(cplatforms.Default())
for _, img := range imgs {
isDangling := isDanglingImage(img)
@@ -150,50 +145,48 @@ func (i *ImageService) Images(ctx context.Context, opts types.ImageListOptions)
tagsByDigest[dgst] = append(tagsByDigest[dgst], reference.FamiliarString(ref))
}
if opts.ContainerCount {
allContainers = i.containers.List()
resultsMut := sync.Mutex{}
eg, egCtx := errgroup.WithContext(ctx)
eg.SetLimit(runtime.NumCPU() * 2)
var (
summaries = make([]*imagetypes.Summary, 0, len(imgs))
root []*[]digest.Digest
layers map[digest.Digest]int
)
if opts.SharedSize {
root = make([]*[]digest.Digest, 0, len(imgs))
layers = make(map[digest.Digest]int)
}
for _, img := range uniqueImages {
err := i.walkImageManifests(ctx, img, func(img *ImageManifest) error {
if isPseudo, err := img.IsPseudoImage(ctx); isPseudo || err != nil {
return err
}
available, err := img.CheckContentAvailable(ctx)
if err != nil {
log.G(ctx).WithFields(log.Fields{
"error": err,
"manifest": img.Target(),
"image": img.Name(),
}).Warn("checking availability of platform specific manifest failed")
return nil
}
if !available {
return nil
}
image, chainIDs, err := i.singlePlatformImage(ctx, contentStore, tagsByDigest[img.RealTarget.Digest], img, opts, allContainers)
img := img
eg.Go(func() error {
image, allChainsIDs, err := i.imageSummary(egCtx, img, platformMatcher, opts, tagsByDigest)
if err != nil {
return err
}
// No error, but image should be skipped.
if image == nil {
return nil
}
resultsMut.Lock()
summaries = append(summaries, image)
if opts.SharedSize {
root = append(root, &chainIDs)
for _, id := range chainIDs {
root = append(root, &allChainsIDs)
for _, id := range allChainsIDs {
layers[id] = layers[id] + 1
}
}
resultsMut.Unlock()
return nil
})
if err != nil {
return nil, err
}
}
if err := eg.Wait(); err != nil {
return nil, err
}
if opts.SharedSize {
@@ -211,36 +204,159 @@ func (i *ImageService) Images(ctx context.Context, opts types.ImageListOptions)
return summaries, nil
}
func (i *ImageService) singlePlatformImage(ctx context.Context, contentStore content.Store, repoTags []string, imageManifest *ImageManifest, opts types.ImageListOptions, allContainers []*container.Container) (*imagetypes.Summary, []digest.Digest, error) {
diffIDs, err := imageManifest.RootFS(ctx)
// imageSummary returns a summary of the image, including the total size of the image and all its platforms.
// It also returns the chainIDs of all the layers of the image (including all its platforms).
// All return values will be nil if the image should be skipped.
func (i *ImageService) imageSummary(ctx context.Context, img images.Image, platformMatcher platforms.MatchComparer,
opts imagetypes.ListOptions, tagsByDigest map[digest.Digest][]string,
) (_ *imagetypes.Summary, allChainIDs []digest.Digest, _ error) {
// Total size of the image including all its platform
var totalSize int64
// ChainIDs of all the layers of the image (including all its platform)
var allChainsIDs []digest.Digest
// Count of containers using the image
var containersCount int64
// Single platform image manifest preferred by the platform matcher
var best *ImageManifest
var bestPlatform ocispec.Platform
err := i.walkImageManifests(ctx, img, func(img *ImageManifest) error {
if isPseudo, err := img.IsPseudoImage(ctx); isPseudo || err != nil {
return nil
}
available, err := img.CheckContentAvailable(ctx)
if err != nil {
log.G(ctx).WithFields(log.Fields{
"error": err,
"manifest": img.Target(),
"image": img.Name(),
}).Warn("checking availability of platform specific manifest failed")
return nil
}
if !available {
return nil
}
conf, err := img.Config(ctx)
if err != nil {
return err
}
var dockerImage dockerspec.DockerOCIImage
if err := readConfig(ctx, i.content, conf, &dockerImage); err != nil {
return err
}
target := img.Target()
diffIDs, err := img.RootFS(ctx)
if err != nil {
return err
}
chainIDs := identity.ChainIDs(diffIDs)
ts, _, err := i.singlePlatformSize(ctx, img)
if err != nil {
return err
}
totalSize += ts
allChainsIDs = append(allChainsIDs, chainIDs...)
if opts.ContainerCount {
i.containers.ApplyAll(func(c *container.Container) {
if c.ImageManifest != nil && c.ImageManifest.Digest == target.Digest {
containersCount++
}
})
}
var platform ocispec.Platform
if target.Platform != nil {
platform = *target.Platform
} else {
platform = dockerImage.Platform
}
// Filter out platforms that don't match the requested platform. Do it
// after the size, container count and chainIDs are summed up to have
// the single combined entry still represent the whole multi-platform
// image.
if !platformMatcher.Match(platform) {
return nil
}
if best == nil || platformMatcher.Less(platform, bestPlatform) {
best = img
bestPlatform = platform
}
return nil
})
if err != nil {
return nil, nil, errors.Wrapf(err, "failed to get rootfs of image %s", imageManifest.Name())
return nil, nil, err
}
if best == nil {
// TODO we should probably show *something* for images we've pulled
// but are 100% shallow or an empty manifest list/index
// ("tianon/scratch:index" is an empty example image index and
// "tianon/scratch:list" is an empty example manifest list)
return nil, nil, nil
}
image, err := i.singlePlatformImage(ctx, i.content, tagsByDigest[best.RealTarget.Digest], best)
if err != nil {
return nil, nil, err
}
image.Size = totalSize
if opts.ContainerCount {
image.Containers = containersCount
}
return image, allChainsIDs, nil
}
func (i *ImageService) singlePlatformSize(ctx context.Context, imgMfst *ImageManifest) (totalSize int64, contentSize int64, _ error) {
// TODO(thaJeztah): do we need to take multiple snapshotters into account? See https://github.com/moby/moby/issues/45273
snapshotter := i.client.SnapshotService(i.snapshotter)
snapshotter := i.snapshotterService(i.snapshotter)
diffIDs, err := imgMfst.RootFS(ctx)
if err != nil {
return -1, -1, errors.Wrapf(err, "failed to get rootfs of image %s", imgMfst.Name())
}
imageSnapshotID := identity.ChainID(diffIDs).String()
unpackedUsage, err := calculateSnapshotTotalUsage(ctx, snapshotter, imageSnapshotID)
if err != nil {
if !cerrdefs.IsNotFound(err) {
log.G(ctx).WithError(err).WithFields(log.Fields{
"image": imageManifest.Name(),
"image": imgMfst.Name(),
"snapshotID": imageSnapshotID,
}).Warn("failed to calculate unpacked size of image")
}
unpackedUsage = snapshots.Usage{Size: 0}
}
contentSize, err := imageManifest.Size(ctx)
contentSize, err = imgMfst.Size(ctx)
if err != nil {
return nil, nil, err
return -1, -1, err
}
// totalSize is the size of the image's packed layers and snapshots
// (unpacked layers) combined.
totalSize := contentSize + unpackedUsage.Size
totalSize = contentSize + unpackedUsage.Size
return totalSize, contentSize, nil
}
func (i *ImageService) singlePlatformImage(ctx context.Context, contentStore content.Store, repoTags []string, imageManifest *ImageManifest) (*imagetypes.Summary, error) {
var repoDigests []string
rawImg := imageManifest.Metadata()
target := rawImg.Target.Digest
@@ -272,11 +388,16 @@ func (i *ImageService) singlePlatformImage(ctx context.Context, contentStore con
cfgDesc, err := imageManifest.Image.Config(ctx)
if err != nil {
return nil, nil, err
return nil, err
}
var cfg configLabels
if err := readConfig(ctx, contentStore, cfgDesc, &cfg); err != nil {
return nil, nil, err
return nil, err
}
totalSize, _, err := i.singlePlatformSize(ctx, imageManifest)
if err != nil {
return nil, errors.Wrapf(err, "failed to calculate size of image %s", imageManifest.Name())
}
summary := &imagetypes.Summary{
@@ -297,18 +418,7 @@ func (i *ImageService) singlePlatformImage(ctx context.Context, contentStore con
summary.Created = cfg.Created.Unix()
}
if opts.ContainerCount {
// Get container count
var containers int64
for _, c := range allContainers {
if c.ImageID == image.ID(target.String()) {
containers++
}
}
summary.Containers = containers
}
return summary, identity.ChainIDs(diffIDs), nil
return summary, nil
}
type imageFilterFunc func(image images.Image) bool
@@ -320,13 +430,13 @@ type imageFilterFunc func(image images.Image) bool
func (i *ImageService) setupFilters(ctx context.Context, imageFilters filters.Args) (filterFunc imageFilterFunc, outErr error) {
var fltrs []imageFilterFunc
err := imageFilters.WalkValues("before", func(value string) error {
img, err := i.GetImage(ctx, value, imagetypes.GetImageOpts{})
img, err := i.GetImage(ctx, value, backend.GetImageOpts{})
if err != nil {
return err
}
if img != nil && img.Created != nil {
fltrs = append(fltrs, func(candidate images.Image) bool {
cand, err := i.GetImage(ctx, candidate.Name, imagetypes.GetImageOpts{})
cand, err := i.GetImage(ctx, candidate.Name, backend.GetImageOpts{})
if err != nil {
return false
}
@@ -340,13 +450,13 @@ func (i *ImageService) setupFilters(ctx context.Context, imageFilters filters.Ar
}
err = imageFilters.WalkValues("since", func(value string) error {
img, err := i.GetImage(ctx, value, imagetypes.GetImageOpts{})
img, err := i.GetImage(ctx, value, backend.GetImageOpts{})
if err != nil {
return err
}
if img != nil && img.Created != nil {
fltrs = append(fltrs, func(candidate images.Image) bool {
cand, err := i.GetImage(ctx, candidate.Name, imagetypes.GetImageOpts{})
cand, err := i.GetImage(ctx, candidate.Name, backend.GetImageOpts{})
if err != nil {
return false
}
@@ -380,7 +490,7 @@ func (i *ImageService) setupFilters(ctx context.Context, imageFilters filters.Ar
return nil, err
}
labelFn, err := setupLabelFilter(i.client.ContentStore(), imageFilters)
labelFn, err := setupLabelFilter(ctx, i.content, imageFilters)
if err != nil {
return nil, err
}
@@ -430,7 +540,7 @@ func (i *ImageService) setupFilters(ctx context.Context, imageFilters filters.Ar
// setupLabelFilter parses filter args for "label" and "label!" and returns a
// filter func which will check if any image config from the given image has
// labels that match given predicates.
func setupLabelFilter(store content.Store, fltrs filters.Args) (func(image images.Image) bool, error) {
func setupLabelFilter(ctx context.Context, store content.Store, fltrs filters.Args) (func(image images.Image) bool, error) {
type labelCheck struct {
key string
value string
@@ -464,19 +574,25 @@ func setupLabelFilter(store content.Store, fltrs filters.Args) (func(image image
}
}
return func(image images.Image) bool {
ctx := context.TODO()
if len(checks) == 0 {
return nil, nil
}
return func(image images.Image) bool {
// This is not an error, but a signal to Dispatch that it should stop
// processing more content (otherwise it will run for all children).
// It will be returned once a matching config is found.
errFoundConfig := errors.New("success, found matching config")
err := images.Dispatch(ctx, presentChildrenHandler(store, images.HandlerFunc(func(ctx context.Context, desc ocispec.Descriptor) (subdescs []ocispec.Descriptor, err error) {
if !images.IsConfigType(desc.MediaType) {
return nil, nil
}
var cfg configLabels
if err := readConfig(ctx, store, desc, &cfg); err != nil {
if errdefs.IsNotFound(err) {
return nil, nil
}
return nil, err
}
@@ -536,6 +652,11 @@ func computeSharedSize(chainIDs []digest.Digest, layers map[digest.Digest]int, s
}
size, err := sizeFn(chainID)
if err != nil {
// Several images might share the same layer and neither of them
// might be unpacked (for example if it's a non-host platform).
if cerrdefs.IsNotFound(err) {
continue
}
return 0, err
}
sharedSize += size

View File

@@ -0,0 +1,368 @@
package containerd
import (
"context"
"fmt"
"math/rand"
"os"
"path/filepath"
"sort"
"strconv"
"testing"
"time"
"github.com/containerd/containerd"
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/metadata"
"github.com/containerd/containerd/namespaces"
"github.com/containerd/containerd/platforms"
"github.com/containerd/containerd/snapshots"
"github.com/containerd/log/logtest"
imagetypes "github.com/docker/docker/api/types/image"
daemonevents "github.com/docker/docker/daemon/events"
"github.com/docker/docker/internal/testutils/specialimage"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func imagesFromIndex(index ...*ocispec.Index) []images.Image {
var imgs []images.Image
for _, idx := range index {
for _, desc := range idx.Manifests {
imgs = append(imgs, images.Image{
Name: desc.Annotations["io.containerd.image.name"],
Target: desc,
})
}
}
return imgs
}
func BenchmarkImageList(b *testing.B) {
populateStore := func(ctx context.Context, is *ImageService, dir string, count int) {
// Use constant seed for reproducibility
src := rand.NewSource(1982731263716)
for i := 0; i < count; i++ {
platform := platforms.DefaultSpec()
// 20% is other architecture than the host
if i%5 == 0 {
platform.Architecture = "other"
}
idx, err := specialimage.RandomSinglePlatform(dir, platform, src)
assert.NilError(b, err)
imgs := imagesFromIndex(idx)
for _, desc := range imgs {
_, err := is.images.Create(ctx, desc)
assert.NilError(b, err)
}
}
}
for _, count := range []int{10, 100, 1000} {
csDir := b.TempDir()
ctx := namespaces.WithNamespace(context.TODO(), "testing-"+strconv.Itoa(count))
cs := &delayedStore{
store: &blobsDirContentStore{blobs: filepath.Join(csDir, "blobs/sha256")},
overhead: 500 * time.Microsecond,
}
is := fakeImageService(b, ctx, cs)
populateStore(ctx, is, csDir, count)
b.Run(strconv.Itoa(count)+"-images", func(b *testing.B) {
for i := 0; i < b.N; i++ {
_, err := is.Images(ctx, imagetypes.ListOptions{All: true})
assert.NilError(b, err)
}
})
}
}
func TestImageList(t *testing.T) {
ctx := namespaces.WithNamespace(context.TODO(), "testing")
blobsDir := t.TempDir()
multilayer, err := specialimage.MultiLayer(blobsDir)
assert.NilError(t, err)
twoplatform, err := specialimage.TwoPlatform(blobsDir)
assert.NilError(t, err)
emptyIndex, err := specialimage.EmptyIndex(blobsDir)
assert.NilError(t, err)
cs := &blobsDirContentStore{blobs: filepath.Join(blobsDir, "blobs/sha256")}
for _, tc := range []struct {
name string
images []images.Image
opts imagetypes.ListOptions
check func(*testing.T, []*imagetypes.Summary)
}{
{
name: "one multi-layer image",
images: imagesFromIndex(multilayer),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 1))
assert.Check(t, is.Equal(all[0].ID, multilayer.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[0].RepoTags, []string{"multilayer:latest"}))
},
},
{
name: "one image with two platforms is still one entry",
images: imagesFromIndex(twoplatform),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 1))
assert.Check(t, is.Equal(all[0].ID, twoplatform.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[0].RepoTags, []string{"twoplatform:latest"}))
},
},
{
name: "two images are two entries",
images: imagesFromIndex(multilayer, twoplatform),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 2))
assert.Check(t, is.Equal(all[0].ID, multilayer.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[0].RepoTags, []string{"multilayer:latest"}))
assert.Check(t, is.Equal(all[1].ID, twoplatform.Manifests[0].Digest.String()))
assert.Check(t, is.DeepEqual(all[1].RepoTags, []string{"twoplatform:latest"}))
},
},
{
name: "three images, one is an empty index",
images: imagesFromIndex(multilayer, emptyIndex, twoplatform),
check: func(t *testing.T, all []*imagetypes.Summary) {
assert.Check(t, is.Len(all, 2))
},
},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
ctx := logtest.WithT(ctx, t)
service := fakeImageService(t, ctx, cs)
for _, img := range tc.images {
_, err := service.images.Create(ctx, img)
assert.NilError(t, err)
}
all, err := service.Images(ctx, tc.opts)
assert.NilError(t, err)
sort.Slice(all, func(i, j int) bool {
firstTag := func(idx int) string {
if len(all[idx].RepoTags) > 0 {
return all[idx].RepoTags[0]
}
return ""
}
return firstTag(i) < firstTag(j)
})
tc.check(t, all)
})
}
}
func fakeImageService(t testing.TB, ctx context.Context, cs content.Store) *ImageService {
snapshotter := &testSnapshotterService{}
mdb := newTestDB(ctx, t)
snapshotters := map[string]snapshots.Snapshotter{
containerd.DefaultSnapshotter: snapshotter,
}
service := &ImageService{
images: metadata.NewImageStore(mdb),
containers: emptyTestContainerStore(),
content: cs,
eventsService: daemonevents.New(),
snapshotterServices: snapshotters,
snapshotter: containerd.DefaultSnapshotter,
}
// containerd.Image gets the services directly from containerd.Client
// so we need to create a "fake" containerd.Client with the test services.
c8dCli, err := containerd.New("", containerd.WithServices(
containerd.WithImageStore(service.images),
containerd.WithContentStore(cs),
containerd.WithSnapshotters(snapshotters),
))
assert.NilError(t, err)
service.client = c8dCli
return service
}
type blobsDirContentStore struct {
blobs string
}
type fileReaderAt struct {
*os.File
}
func (f *fileReaderAt) Size() int64 {
fi, err := f.Stat()
if err != nil {
return -1
}
return fi.Size()
}
func (s *blobsDirContentStore) ReaderAt(ctx context.Context, desc ocispec.Descriptor) (content.ReaderAt, error) {
p := filepath.Join(s.blobs, desc.Digest.Encoded())
r, err := os.Open(p)
if err != nil {
return nil, err
}
return &fileReaderAt{r}, nil
}
func (s *blobsDirContentStore) Writer(ctx context.Context, opts ...content.WriterOpt) (content.Writer, error) {
return nil, fmt.Errorf("read-only")
}
func (s *blobsDirContentStore) Status(ctx context.Context, _ string) (content.Status, error) {
return content.Status{}, fmt.Errorf("not implemented")
}
func (s *blobsDirContentStore) Delete(ctx context.Context, dgst digest.Digest) error {
return fmt.Errorf("read-only")
}
func (s *blobsDirContentStore) ListStatuses(ctx context.Context, filters ...string) ([]content.Status, error) {
return nil, nil
}
func (s *blobsDirContentStore) Abort(ctx context.Context, ref string) error {
return fmt.Errorf("not implemented")
}
func (s *blobsDirContentStore) Walk(ctx context.Context, fn content.WalkFunc, filters ...string) error {
entries, err := os.ReadDir(s.blobs)
if err != nil {
return err
}
for _, e := range entries {
if e.IsDir() {
continue
}
d := digest.FromString(e.Name())
if d == "" {
continue
}
stat, err := e.Info()
if err != nil {
return err
}
if err := fn(content.Info{Digest: d, Size: stat.Size()}); err != nil {
return err
}
}
return nil
}
func (s *blobsDirContentStore) Info(ctx context.Context, dgst digest.Digest) (content.Info, error) {
f, err := os.Open(filepath.Join(s.blobs, dgst.Encoded()))
if err != nil {
return content.Info{}, err
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
return content.Info{}, err
}
return content.Info{
Digest: dgst,
Size: stat.Size(),
}, nil
}
func (s *blobsDirContentStore) Update(ctx context.Context, info content.Info, fieldpaths ...string) (content.Info, error) {
return content.Info{}, fmt.Errorf("read-only")
}
// delayedStore is a content store wrapper that adds a constant delay to all
// operations in order to imitate gRPC overhead.
//
// The delay is constant to make the benchmark results more reproducible
// Since content store may be accessed concurrently random delay would be
// order-dependent.
type delayedStore struct {
store content.Store
overhead time.Duration
}
func (s *delayedStore) delay() {
time.Sleep(s.overhead)
}
func (s *delayedStore) ReaderAt(ctx context.Context, desc ocispec.Descriptor) (content.ReaderAt, error) {
s.delay()
return s.store.ReaderAt(ctx, desc)
}
func (s *delayedStore) Writer(ctx context.Context, opts ...content.WriterOpt) (content.Writer, error) {
s.delay()
return s.store.Writer(ctx, opts...)
}
func (s *delayedStore) Status(ctx context.Context, st string) (content.Status, error) {
s.delay()
return s.store.Status(ctx, st)
}
func (s *delayedStore) Delete(ctx context.Context, dgst digest.Digest) error {
s.delay()
return s.store.Delete(ctx, dgst)
}
func (s *delayedStore) ListStatuses(ctx context.Context, filters ...string) ([]content.Status, error) {
s.delay()
return s.store.ListStatuses(ctx, filters...)
}
func (s *delayedStore) Abort(ctx context.Context, ref string) error {
s.delay()
return s.store.Abort(ctx, ref)
}
func (s *delayedStore) Walk(ctx context.Context, fn content.WalkFunc, filters ...string) error {
s.delay()
return s.store.Walk(ctx, fn, filters...)
}
func (s *delayedStore) Info(ctx context.Context, dgst digest.Digest) (content.Info, error) {
s.delay()
return s.store.Info(ctx, dgst)
}
func (s *delayedStore) Update(ctx context.Context, info content.Info, fieldpaths ...string) (content.Info, error) {
s.delay()
return s.store.Update(ctx, info, fieldpaths...)
}

Some files were not shown because too many files have changed in this diff Show More