Add cleanup for the init layer directory if any operation fails after
driver.CreateReadWrite() succeeds in initMount(). Previously, failures
in driver.Get(), initFunc(), or driver.Put() would leave an orphaned
overlay2 directory.
Related to moby/moby#45939
Signed-off-by: Jan Scheffler <jan.scheffler@qodev.ai>
(cherry picked from commit 3fdde529e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add cleanup for the RW layer directory if saveMount() fails after
driver.CreateReadWrite() succeeds. Previously, this failure path would
leave an orphaned overlay2 directory with no corresponding metadata.
Related to moby/moby#45939
Signed-off-by: Jan Scheffler <jan.scheffler@qodev.ai>
(cherry picked from commit d7a6250b91)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Start the metadata transaction before creating the overlay2 directory.
This ensures that if driver.Create() fails, we can properly cancel the
transaction. Previously, if StartTransaction() failed after driver.Create()
succeeded, the defer cleanup would not run (not registered yet), leaving
an orphaned overlay2 directory.
The fix reorders operations so that:
1. Transaction is started first (no filesystem changes yet)
2. Overlay2 directory is created second (transaction ready for cleanup)
3. Defer is registered after both succeed (tx is guaranteed non-nil)
If driver.Create() fails, the transaction is explicitly cancelled before
returning. The nil check for tx in the defer is no longer needed since
tx is guaranteed to exist when the defer runs.
Related to moby/moby#45939
Signed-off-by: Jan Scheffler <jan.scheffler@qodev.ai>
(cherry picked from commit 70004549fb)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The image exporter wrapper was unconditionally setting `unpack=true` for
all build exports, preventing users from controlling this behavior
through buildkit's output image exporter option.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Currently requires replace rule for swarmkit etcd.
BuildKit itself doesn't use etcd but version gets bumped
via unused dependency.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This workaround was added in df519e9e1a, pending
a fix in containerd;
> daemon: Fix giving up too early while connecting to containerd socket
>
> Explicitly set the gRPC connection params to take the timeout into
> account to workaround the containerd v2 client not passing down the
> stack.
>
> containerd v2 replaced usages of deprecated gRPC functions but didn't
> pass the timeout to the actual dial connection options.
A fix for this was merged in [containerd@ee574e7], which is part of containerd
v2.1.0-beta.0, and backported to containerd v2.0.4 through [containerd@6b5efba].
[containerd@ee574e7]: ee574e76e7
[containerd@6b5efba]: 6b5efba83b
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Introduced DefaultIsolation method in the Daemon to return the daemon configured isolation mode for Windows.
Signed-off-by: Vigilans <vigilans@foxmail.com>
Use a singleflight for collecting the build-cache usage; collecting this
information is potentially heavy-weight, and we already do this for the
other disk-usage methods.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move calculation of the data to the builder backend, to align with
the other type of objects. This also allows us to skip the verbose
data if it's not used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Writing the OCI manifest file to the blobs/digest dir will update the
directory mtime, producing a tar file containing a member with a
contemporary mtime. Exported tars for the same image will therefore have
different checksums.
Although this was previously addressed by overriding the mtime manually
to 0, this was done before the OCI manifest file was written. This
change simply moves the call to system.Chtimes to set the mtime of the
blobs/digest directory to 0 after writing the OCI manifest file.
This commit also updates the TestSaveCheckTimes integration test to
check the mtime of all members in the exported tar to ensure that all
mtime are not newer than img.Created or 0 (depending on whether the
containerd-snapshotter is disabled or enabled, respectively).
Signed-off-by: Sam Nicholls <sam.nicholls@nanoporetech.com>
This type was used as Aux message for docker push, was not documented,
and only present for Docker Content Trust (which is deprecated).
This patch removes it from the API module, and moves the type internal.
We can stop sending this Aux message once DCT is fully phased out.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The schema of a JSON-stream message is very pertinent to the api module.
Provide a canonical definition in the api module and refactor the daemon
code to use it. Drop the long-deprecated ErrorMessage field from the API
definition, but have the daemon continue to emit it for compatibility
with docker-py v7.1.0.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
Move the progress package up into the client as a temporary shared location for
common clients like CLI and compose.
The progress package is used by the daemon to write progress updates to
some sink, typically a streamformatter. This package is of little use to
API clients as this package does not provide any facilities to consume
the progress updates.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
Move the streamformatter package up into the client for a temporary
shared location between common clients like CLI and compose.
The streamformatter package is used by the daemon to write streams of
status and progress messages to API clients. It is completely out of
scope of the api module and not used outside the daemon. Remove the
unused rawSteamFormatter, whose purpose is to render the progress as a
TUI.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
This change moves the api/types/versions package out into client and daemon versions.
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
The BuildCache.Parent field was removed in API v1.42 in [moby@e0db820].
While we had to keep the Go struct field around to backfill the field for
older API versions, it's no longer part of API v1.42 and up (using the
"omitempty" is just an implementation detail).
Older clients unconditionally use the `Parents` field if set, and usage
of this field is very limited, so let's remove the field without back-
filling, and have clients use the replacement field;
https://github.com/docker/cli/blob/v28.5.1/cli/command/formatter/buildcache.go
[moby@e0db820]: e0db8207f3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Most of the code in the filters package relates to the unmarshaling,
validation and application of filters from client requests. None of this
is necessary or particularly useful for Go SDK users. Move the full-fat
filters package into daemon/internal and switch all the daemon code to
import that package so we are free to iterate upon the code without
worrying about source-code interface compatibility.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The default gc policy calculations based on percentage were calculated
improperly. These were calculated correctly in buildkit, but the
calculation method was not copied over correctly when updating the
values.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
The DockerVersion field was used by the legacy builder, and set when
importing an image; when importing an image, this would potentially
result in less reproducible images, as the docker version used to import
the image would be encoded in the image's "v1" fields.
For the legacy builder, including the version of docker used to build
the image could still be useful information (but could be set as comment,
similar to what BuildKit does), however, many code paths were also shared
with other parts of the code; e.g., when listing images or inspecting images,
the `DockerVersion` field would always be set to the current version of
the docker daemon, and not taken from the information available in the
image (if any).
This patch removes locations where the `DockerVersion` field was set to
the current version of the daemon binary. When inspecting an image, the
field is still set with the information in the image itself (which may
be empty in most cases).
This also reduces the number of places where the `dockerversion` package
is used, which still needs a new home.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was a bit of an oversight; when setting additional fields to add,
the compat package currently replaces fields unconditionally. This may
have sounded like an OK idea, but it makes it more complicated to augment
responses where current versions use an "omitempty", but older API versions
should return default / zero-values.
This patch:
- Changes the meaning of "extra fields"; extra fields are only used if
the field is not present in the response.
- Makes the merging of "extra fields" recursive; this makes it easier
to patch responses where extra fields must be added to nested structs.
Previously, this would require the nested struct to be wrapped with a
`compat.Wrap` and replaced as a whole; lacking a "replace" option made
that more complicated, so making the extra fields recursive.
- Comment-out a test that tested the old behavior of replacing fields;
we currently have no cases where we must _replace_ fields or structs,
so I did not yet implement such an option, but we can implement a
`WithReplaceFields` (e.g.) once there's a need.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These utilities are very handy to use in integration tests, too. Move
the package so it can be imported by them.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Change the types for IP address and prefix struct fields to netip.Addr
and netip.Prefix for convenience. Fields such as
swarm.InitRequest.ListenAddr which may encode non-numeric values such as
a network interface name have not been modified.
Signed-off-by: Cory Snider <csnider@mirantis.com>
These utilities are going to be needed elsewhere in the daemon to handle
netip values from API requests.
Signed-off-by: Cory Snider <csnider@mirantis.com>
As there is a meaningful distinction between a nil slice and an empty
but non-nil slice in some contexts, having sliceutil.Map return an empty
slice when passed a nil slice can introduce subtle bugs. Modify Map to
return a nil slice when passed a nil slice.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The compat package marshals the base value into JSON to convert it into
a map[string]any, then applies the compat options, and finally
re-encodes the map into JSON.
`json.Marshal` is used for both marshalling operations. However, this
function encodes special HTML characters by default. The API responses
aren't meant to be embedded into HTML, so this behavior is not desired.
Replace `json.Marshal` with a `json.Encoder` with HTML escaping
disabled. Extra care is taken to drop the trailing LF added by
`(*json.Encoder).Encode()` (see [1]).
[1]: https://pkg.go.dev/encoding/json#Encoder.Encode
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Add a package to help augmenting API responses with additional fields,
replacing fields, or to remove fields from the response to allow for
legacy API responses to be produced without having to keep deprecated
fields in the API type definitions.
Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
