When logging an unmarshal failure in the registry error handling code,
the function was incorrectly logging the uninitialized `derrs` variable
instead of the actual JSON unmarshal error `jerr`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When inspecting multi-platform images where some layer blobs were
missing from the content store, the image inspect operation would return
too early causing some data (like config details or unpacked size) to be
omitted even though are available.
This ensures that `docker image inspect` returns as much information as
possible.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Updates docker pull to pull related attestation manifest and
any signatures for that manifest in cosign referrer objects.
These objects are transferred with the image when running
docker save and docker load and can be used to identify
the image in future updates.
Push is not updated atm as the currect push semantics
in containerd mode do not have correct immutability
guaranteed and don't work with image indexes.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
All methods are singular; while pruning will impact multiple items,
it's more consistent to use singular for all operations.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the progress package up into the client as a temporary shared location for
common clients like CLI and compose.
The progress package is used by the daemon to write progress updates to
some sink, typically a streamformatter. This package is of little use to
API clients as this package does not provide any facilities to consume
the progress updates.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
Move the streamformatter package up into the client for a temporary
shared location between common clients like CLI and compose.
The streamformatter package is used by the daemon to write streams of
status and progress messages to API clients. It is completely out of
scope of the api module and not used outside the daemon. Remove the
unused rawSteamFormatter, whose purpose is to render the progress as a
TUI.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
For API < v1.52:
- In container inspect:
- Restore GraphDriver when a snapshotter is used.
- Remove field Storage
- Related to commit efa077848f
- In image inspect:
- Restore GraphDriver when a snapshotter is used.
- Related to commit c441b2ef19
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Rob Murray <rob.murray@docker.com>
Most of the code in the filters package relates to the unmarshaling,
validation and application of filters from client requests. None of this
is necessary or particularly useful for Go SDK users. Move the full-fat
filters package into daemon/internal and switch all the daemon code to
import that package so we are free to iterate upon the code without
worrying about source-code interface compatibility.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The InspectResponse type contains various fields that are deprecated
and removed from current API versions, but that were kept for the API
server to produce the fields when downgrading to older API versions.
- The `Parent` field is only used for the legacy builder, and only set for
images that are built locally (i.e., not persisted when pulling an image).
- The `DockerVersion` field is only set when building images with the legacy
builder, and empty in most cases.
Both fields were implicitly deprecated with the deprecation of the legacy
builder, and deprecated for the API in [moby@bd8a99b], which was backported
to the 28.x release.
This patch:
- Removes the deprecated fields from the `InspectResposne` struct; this
means that [`client.ImageInspect`] won't unmarshal those fields, but
the [`docker image inspect`] CLI command defaults to printing the raw
output as returned by the API, so can continue to show any field returned
in the API response. As a side-note; we should change the CLI to default
to show the unmarshalled response, and introduce a `--format=jsonraw`
(or `--raw`) option to make printing the raw response opt-in.
- Updates the API server to backfill the fields if they are set.
[moby@bd8a99b]: bd8a99b400
[`client.ImageInspect`]: f739c61c69/client/image_inspect.go (L14-L64)
[`docker image inspect`]: 74e3520724/cli/command/image/inspect.go (L59-L81)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The InspectResponse type contains various fields that are deprecated
and removed from current API versions, but that were kept for the API
server to produce the fields when downgrading to older API versions.
This patch:
- Introduces a `imagebackend.InspectData` type for the daemon to use for
returning the data needed to propagate the API response. It currently
embeds the API response type and combines it with the legacy fields,
but this could be changed to return the internal Image type, and
mapping the fields to the API response type in the router.
- Removes the deprecated fields from the `InspectResposne` struct; this
means that [`client.ImageInspect`] won't unmarshal those fields, but
the [`docker image inspect`] CLI command defaults to printing the raw
output as returned by the API, so can continue to show any field returned
in the API response. As a side-note; we should change the CLI to default
to show the unmarshalled response, and introduce a `--format=jsonraw`
(or `--raw`) option to make printing the raw response opt-in.
This patch removes the following fields from the `InspectResponse` type;
- `VirtualSize`: this field became obsolete with the migration to content-
addressable images in docker v1.10 ([moby@4352da7]), but was still returned
with a copy of the `Size` field. It was deprecated in API v1.43 ([moby@1261fe6]),
and removed in API v1.44 ([moby@913b0f5]).
- `Container` and `ContainerConfig`: both fields were deprecated in
API v1.44 ([moby@1602e2f]), and removed in API v1.45 ([moby@03cddc6]).
remove deprecated Container, ContainerConfig, VirtualSize
[moby@4352da7]: 4352da7803
[moby@1261fe6]: 1261fe69a3
[moby@913b0f5]: 913b0f51ca
[moby@1602e2f]: 1602e2f4f1
[moby@03cddc6]: 03cddc62f4
[`client.ImageInspect`]: f739c61c69/client/image_inspect.go (L14-L64)
[`docker image inspect`]: 74e3520724/cli/command/image/inspect.go (L59-L81)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The DockerVersion field was used by the legacy builder, and set when
importing an image; when importing an image, this would potentially
result in less reproducible images, as the docker version used to import
the image would be encoded in the image's "v1" fields.
For the legacy builder, including the version of docker used to build
the image could still be useful information (but could be set as comment,
similar to what BuildKit does), however, many code paths were also shared
with other parts of the code; e.g., when listing images or inspecting images,
the `DockerVersion` field would always be set to the current version of
the docker daemon, and not taken from the information available in the
image (if any).
This patch removes locations where the `DockerVersion` field was set to
the current version of the daemon binary. When inspecting an image, the
field is still set with the information in the image itself (which may
be empty in most cases).
This also reduces the number of places where the `dockerversion` package
is used, which still needs a new home.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package was aliased as "imagespec" in some places, and "dockerspec"
in other places, which made it easy to confuse.
Change all uses of this package to be aliased as "dockerspec" and configure
an "importas" linting check to enforce it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The image inspect response has various fields that were deprecated as
part of the legacy builder, or Dockerfile syntax;
- The `Parent` field is only used for the legacy builder, and only set for
images that are built locally (i.e., not persisted when pulling an image).
- The `DockerVersion` field is only set when building images with the legacy
builder, and empty in most cases.
This patch deprecates the fields in the `InspectResponse` go struct, as
these fields will no longer be set in future once the legacy builder is
removed (`Parent`, `DockerVersion`). The legacy builder's deprecation in
[cli@4d8e457] / [cli@fd22746] (docker 23.0, API v1.42), however the related
API fields were kept so that information of legacy images would not be
discarded.
The API continues to return these fields if set, allowing the client to
print the fields for informational purposes when printing the raw response,
but these fields should be considered "transitional", and not be depended
on; deprecating the fields helps raise awareness.
[cli@4d8e457]: 4d8e45782b
[cli@fd22746]: fd2274692f
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These utilities are very handy to use in integration tests, too. Move
the package so it can be imported by them.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The PushImage method for the ImageService used positional arguments for its
options, which made it more difficult to introduce new options. This patch
introduces a `PushOptions` struct to specify the options. As part of these
changes, the `platform` option was already adjusted to accept a slice of
platforms, which currently is not supported, but may be in the near future.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The PullImage method for the ImageService used positional arguments for its
options, which made it more difficult to introduce new options. This patch
introduces a `PullOptions` struct to specify the options. As part of these
changes, the `platform` option was already adjusted to accept a slice of
platforms, which currently is not supported, but may be in the near future.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Container.State struct holds the container's state, and most of
its fields are expected to change dynamically. Some o these state-changes
are explicit, for example, setting the container to be "stopped". Other
state changes can be more explicit, for example due to the containers'
process exiting or being "OOM" killed by the kernel.
The distinction between explicit ("desired") state changes and "state"
("actual state") is sometimes vague; for some properties, we clearly
separated them, for example if a user requested the container to be
stopped or restarted, we store state in the Container object itself;
HasBeenManuallyStopped bool // used for unless-stopped restart policy
HasBeenManuallyRestarted bool `json:"-"` // used to distinguish restart caused by restart policy from the manual one
Other properties are more ambiguous. such as "HasBeenStartedBefore" and
"RestartCount", which are stored on the Container (and persisted to
disk), but may be more related to "actual" state, and likely should
not be persisted;
RestartCount int
HasBeenStartedBefore bool
Given that (per the above) concurrency must be taken into account, most
changes to the `container.State` struct should be protected; here's where
things get blurry. While the `State` type provides various accessor methods,
only some of them take concurrency into account; for example, [State.IsRunning]
and [State.GetPID] acquire a lock, whereas [State.ExitCodeValue] does not.
Even the (commonly used) [State.StateString] has no locking at all.
The way to handle this is error-prone; [container.State] contains a mutex,
and it's exported. Given that its embedded in the [container.Container]
struct, it's also exposed as an exported mutex for the container. The
assumption here is that by "merging" the two, the caller to acquire a lock
when either the container _or_ its state must be mutated. However, because
some methods on `container.State` handle their own locking, consumers must
be deeply familiar with the internals; if both changes to the `Container`
AND `Container.State` must be made. This gets amplified more as some
(exported!) methods, such as [container.SetRunning] mutate multiple fields,
but don't acquire a lock (so expect the caller to hold one), but their
(also exported) counterpart (e.g. [State.IsRunning]) do.
It should be clear from the above, that this needs some architectural
changes; a clearer separation between "desired" and "actual" state (opening
the potential to update the container's config without manually touching
its `State`), possibly a method to obtain a read-only copy of the current
state (for those querying state), and reviewing which fields belong where
(and should be persisted to disk, or only remain in memory).
This PR preserves the status quo; it makes no structural changes, other
than exposing where we access the container's state. Where previously the
State fields and methods were referred to as "part of the container"
(e.g. `ctr.IsRunning()` or `ctr.Running`), we now explicitly reference
the embedded `State` (`ctr.State.IsRunning`, `ctr.State.Running`).
The exception (for now) is the mutex, which is still referenced through
the embedded struct (`ctr.Lock()` instead of `ctr.State.Lock()`), as this
is (mostly) by design to protect the container, and what's in it (including
its `State`).
[State.IsRunning]: c4afa77157/daemon/container/state.go (L205-L209)
[State.GetPID]: c4afa77157/daemon/container/state.go (L211-L216)
[State.ExitCodeValue]: c4afa77157/daemon/container/state.go (L218-L228)
[State.StateString]: c4afa77157/daemon/container/state.go (L102-L131)
[container.State]: c4afa77157/daemon/container/state.go (L15-L23)
[container.Container]: c4afa77157/daemon/container/container.go (L67-L75)
[container.SetRunning]: c4afa77157/daemon/container/state.go (L230-L277)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This change makes the `GraphDriver` field in `image.InspectResponse` optional. This field will only be returned when using moby engine graph drivers as a backend storage implementation. It will be omitted when using the containerd image backend.
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
When building a non-native platform, it's not unpacked by default.
History tries to read the disk usage of all the layer and it doesn't
handle missing snapshots gracefully.
This patch fixes this.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The containerd errors and related containerd/errdefs should now
be interchangeable with moby/errdefs, so there's no longer a need
to convert these errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ensure events for containerd backend are properly sent when deleted via
image prune. Fix prune output to only show deleted images rather than
the deletion of each blob.
Signed-off-by: Derek McGowan <derek@mcg.dev>
