The "backend" types in API were designed to decouple the API server
implementation from the daemon, or other parts of the code that
back the API server. This would allow the daemon to evolve (e.g.
functionality moved to different subsystems) without that impacting
the API server's implementation.
Now that the API server is no longer part of the API package (module),
there is no benefit to having it in the API module. The API server
may evolve (and require changes in the backend), which has no direct
relation with the API module (types, responses); the backend definition
is, however, coupled to the API server implementation.
It's worth noting that, while "technically" possible to use the API
server package, and implement an alternative backend implementation,
this has never been a prime objective. The backend definition was
never considered "stable", and we don't expect external users to
(attempt) to use it as such.
This patch moves the backend types to the daemon/server package,
so that they can evolve with the daemon and API server implementation
without that impacting the API module (which we intend to be stable,
following SemVer).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Merge the matchAllWithPreference matcher into the matchAnyWithPreference
matcher to reduce code duplication and simplify the code.
Signed-off-by: Cesar Talledo <cesar.talledo@docker.com>
This parameter was already supported for some time in the backend (for
purposes related to docker system prune). It was also already present in
the imagetypes.ListOptions but was never actually handled by the client.
Make it available by default in the response.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When listing images, prioritize the tagged version. When computing the
unique images, the last returned image would be used as a base and repo
tags would be merged into it.
This makes it so `RepoDigests` has the correct value when there is both
a dangling and non-dangling version of the same image.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Split singlePlatformSize into a separate methods of the `ImageManifest`
that calculate the corresponding usage.
This also allows us to simplify a part of the `multiPlatformSummary`
because we don't calculate the content size twice now.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Fix empty `Manifests` field for multi-platform images that have no
platform blobs available locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
imageSummary erroneously returned a nil multi-platform summary in case
of a multi-platform image, which the OCI index is available locally, but
none of its platform manifest is.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Manifests` field to image inspect (`/images/{name}/json`) response.
This is the same as in `/images/json`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
While the endpoint returns a detailed information about its children, it
doesn't actually expose the descriptor of the root OCI index/manifest
list.
This commits adds the target description to the returned JSON.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
daemon/containerd/image_list.go:162:3: The copy of the 'for' variable "img" can be deleted (Go 1.22+) (copyloopvar)
img := img
^
daemon/containerd/image_delete_test.go:222:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/image_list_test.go:89:3: The copy of the 'for' variable "count" can be deleted (Go 1.22+) (copyloopvar)
count := count
^
daemon/containerd/image_list_test.go:306:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/image_test.go:145:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/platform_matchers_test.go:131:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/platform_matchers_test.go:134:5: The copy of the 'for' variable "strict" can be deleted (Go 1.22+) (copyloopvar)
strict := strict
^
daemon/containerd/platform_matchers_test.go:159:7: The copy of the 'for' variable "p" can be deleted (Go 1.22+) (copyloopvar)
p := p
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `manifests` option, as used for the `--tree` option on `docker image ls`
currently sorts manifests to put those that are present first. The intent was
to present "available" images at the top of each tree, followed by images that
were not pulled.
However, there's some limitations to this. First of all, the current approach
makes the output non-deterministic as the order in which variants are pulled
determines the order in which they're presented, i.e., the last pulled variant
is returned first (I omitted some variants in the example for brevity);
Here's the result of pulling `linux/riscv64`, then pulling `linux/arm64`;
docker pull --platform=linux/riscv64 alpine:latest
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 10.6MB 3.37MB
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
├─ linux/amd64 33735bd63cf8 0B 0B
└─ linux/arm64/v8 9cee2b382fe2 0B 0B
docker pull --platform=linux/arm64 alpine:latest
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 24.2MB 7.46MB
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
└─ linux/amd64 33735bd63cf8 0B 0B
Repeating the steps but in reverse order results in the output to be reversed;
docker image rm alpine:latest
docker pull --platform=linux/arm64 alpine:latest
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 13.6MB 4.09MB
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
├─ linux/amd64 33735bd63cf8 0B 0B
└─ linux/riscv64 80cde017a105 0B 0B
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 24.2MB 7.46MB
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
└─ linux/amd64 33735bd63cf8 0B 0B
The second limitation is that order sometimes matters; when matching a
platform from a manifest-index, implementations may find multiple suitable
candidates. In most cases the _most_ suitable candidate can be selected
(e.g., prefer `linux/arm/v7` over `linux/arm/v6`), but manifest-indices do
allow multiple entries for the same platform, in which case implementations
match the first entry found.
While these situations will be less common (and usually due to incorect use
of tooling such as `docker manifest`), being able to observe the order in
which manifests appeared in the index can help debugging or help the user
understand why a specific variant was selected.
We should therefore not re-order these manifests, and return them in the
order in which they appeared. If we decide to present "present" variants
before "non-present" variants, we can do this ordering on the client side.
With this patch applied;
docker pull --quiet --platform=linux/riscv64 alpine:latest
docker pull --quiet --platform=linux/arm64 alpine:latest
docker image ls --tree alpine
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 24.2MB 7.46MB
├─ linux/amd64 33735bd63cf8 0B 0B
├─ linux/arm/v6 50f635c8b04d 0B 0B
├─ linux/arm/v7 f2f82d424957 0B 0B
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
├─ linux/386 b3e87f642f5c 0B 0B
├─ linux/ppc64le c7a6800e3dc5 0B 0B
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
└─ linux/s390x 2b5b26e09ca2 0B 0B
Which matches the order of the manifests in the index:
docker buildx imagetools inspect --raw alpine:latest | jq -c .manifests[].platform
{"architecture":"amd64","os":"linux"}
{"architecture":"arm","os":"linux","variant":"v6"}
{"architecture":"arm","os":"linux","variant":"v7"}
{"architecture":"arm64","os":"linux","variant":"v8"}
{"architecture":"386","os":"linux"}
{"architecture":"ppc64le","os":"linux"}
{"architecture":"riscv64","os":"linux"}
{"architecture":"s390x","os":"linux"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this, the image list would not show images that are not a valid
container image, but could be a valid artifact.
While they're not directly usable by docker, we should still show them
so the user can still discover them and at least be able to delete them.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use a regular for loop instead of ApplyAll which spawns a separate
goroutine for each separate container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `GET /images/json` requires an optional `container-count` parameter
which set the `Containers` property of in the ImageSummary to a number
of containers using that image.
This was also propagated to the new manifest list property which
includes a list of all the container IDs that are using this specific
image manifest.
Disconnect the `ImageData.Containers` property from this option and
always include it by default without an explicit opt-in.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).
There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.
This patch:
- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to prevent using the "log" package, which was also
migrated to a separate module.
There are still some uses of the old package in (indirect) dependencies,
which should go away over time.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't fail-fast when encountering an image that targets an unexpected
descriptor (neither a manifest nor index). Log a warning instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `identity.ChainIDs` call was accidentally removed in
b37ced2551.
This broke the shared size calculation for images with more than one
layer that were sharing the same compressed layer.
This was could be reproduced with:
```
$ docker pull docker.io/docker/desktop-kubernetes-coredns:v1.11.1
$ docker pull docker.io/docker/desktop-kubernetes-etcd:3.5.10-0
$ docker system df
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
After a535a65c4b the size reported by the
image list was changed to include all platforms of that image.
This made the "shared size" calculation consider all diff ids of all the
platforms available in the image which caused "snapshot not found"
errors when multiple images were sharing the same layer which wasn't
unpacked.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
52a80b40e2 extracted the `imageSummary`
function but introduced a bug causing the whole caller function to
return if the image should be skipped.
`imageSummary` returns a nil error and nil image when the image doesn't
have any platform or all its platforms are not available locally.
In this case that particular image should be skipped, instead of failing
the whole image list operation.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't run filter function which would only run through the images
reading theirs config without checking any label anyway.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Currently this won't have any real effect because the platform matcher
matches all platform and is only used for sorting.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Move containers counting out of `singlePlatformImage` and count them
based on the `ImageManifest` property.
(also remove ChainIDs calculation as they're no longer used)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
