The Container.State struct holds the container's state, and most of
its fields are expected to change dynamically. Some o these state-changes
are explicit, for example, setting the container to be "stopped". Other
state changes can be more explicit, for example due to the containers'
process exiting or being "OOM" killed by the kernel.
The distinction between explicit ("desired") state changes and "state"
("actual state") is sometimes vague; for some properties, we clearly
separated them, for example if a user requested the container to be
stopped or restarted, we store state in the Container object itself;
HasBeenManuallyStopped bool // used for unless-stopped restart policy
HasBeenManuallyRestarted bool `json:"-"` // used to distinguish restart caused by restart policy from the manual one
Other properties are more ambiguous. such as "HasBeenStartedBefore" and
"RestartCount", which are stored on the Container (and persisted to
disk), but may be more related to "actual" state, and likely should
not be persisted;
RestartCount int
HasBeenStartedBefore bool
Given that (per the above) concurrency must be taken into account, most
changes to the `container.State` struct should be protected; here's where
things get blurry. While the `State` type provides various accessor methods,
only some of them take concurrency into account; for example, [State.IsRunning]
and [State.GetPID] acquire a lock, whereas [State.ExitCodeValue] does not.
Even the (commonly used) [State.StateString] has no locking at all.
The way to handle this is error-prone; [container.State] contains a mutex,
and it's exported. Given that its embedded in the [container.Container]
struct, it's also exposed as an exported mutex for the container. The
assumption here is that by "merging" the two, the caller to acquire a lock
when either the container _or_ its state must be mutated. However, because
some methods on `container.State` handle their own locking, consumers must
be deeply familiar with the internals; if both changes to the `Container`
AND `Container.State` must be made. This gets amplified more as some
(exported!) methods, such as [container.SetRunning] mutate multiple fields,
but don't acquire a lock (so expect the caller to hold one), but their
(also exported) counterpart (e.g. [State.IsRunning]) do.
It should be clear from the above, that this needs some architectural
changes; a clearer separation between "desired" and "actual" state (opening
the potential to update the container's config without manually touching
its `State`), possibly a method to obtain a read-only copy of the current
state (for those querying state), and reviewing which fields belong where
(and should be persisted to disk, or only remain in memory).
This PR preserves the status quo; it makes no structural changes, other
than exposing where we access the container's state. Where previously the
State fields and methods were referred to as "part of the container"
(e.g. `ctr.IsRunning()` or `ctr.Running`), we now explicitly reference
the embedded `State` (`ctr.State.IsRunning`, `ctr.State.Running`).
The exception (for now) is the mutex, which is still referenced through
the embedded struct (`ctr.Lock()` instead of `ctr.State.Lock()`), as this
is (mostly) by design to protect the container, and what's in it (including
its `State`).
[State.IsRunning]: c4afa77157/daemon/container/state.go (L205-L209)
[State.GetPID]: c4afa77157/daemon/container/state.go (L211-L216)
[State.ExitCodeValue]: c4afa77157/daemon/container/state.go (L218-L228)
[State.StateString]: c4afa77157/daemon/container/state.go (L102-L131)
[container.State]: c4afa77157/daemon/container/state.go (L15-L23)
[container.Container]: c4afa77157/daemon/container/container.go (L67-L75)
[container.SetRunning]: c4afa77157/daemon/container/state.go (L230-L277)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was deprecated in [moby@cfcbfab] when this struct still lived
in the API. The field is no longer used, and we don't have to carry it
forward as part of the new client module.
[moby@cfcbfab]: cfcbfabb0f
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since moving from `net.IP` to `netip.Addr`, we can support more proper
nameserver values. This commit adds some tests related to IPv6 scoping.
Signed-off-by: Nam Nguyen <namnguyen@google.com>
the "reference" filter was introduced in [moby@820b809] (docker 1.13.0-rc1)
to replace the "filter" query argument. That commit initially included a
version-gate anticipating the API version to be used for v17.12, but as
this was yet unknown, the version-gate was removed in [moby@0f9d22c].
A later PR re-introduced a version-gate in [moby@4a19009], reflecting the
API version in which the deprecation was (finally) completed.
For the client, [moby@c6e3145] added a fallback was added for older daemons
(docker 1.12.0 and older, using API < v1.25) that did not support the new
filter.
Looking at the above, any version of docker 1.13.0 or above handles the
"reference" filter, but (depending on the docker version) may also handle
the old filter on API < 1.28 or API < 1.41. Removing this option will only
impact daemon versions older than 1.13.0, which are long obsolete.
Given that current clients forcibly remove the "reference" filter and replace
it with the old "filter" when using API v1.24, we keep support on the daemon
side, but update the version to v1.24, and only if no reference filter is
set.
[moby@820b809]: 820b809e70
[moby@c6e3145]: c6e31454ba
[moby@0f9d22c]: 0f9d22cd66
[moby@4a19009]: 4a1900915a
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit [moby@5d6b566] migrated this validation from the CLI to the client,
but for some reason picked the wrong API version inside ServiceCreate.
The CLI code was added to an existing validation, which only handled
validation when creating a service, but not when updating, which meant
that adding this option to an existing service would not invalidate it.
This patch:
- moves the version-gate to the validation code
- merges validateServiceSpecForAPIVersion into validateServiceSpec, to
keep the validation combined, and to make sure validation happens both
on create and update.
[moby@5d6b566]: 5d6b56699d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The version header is no longer used since [moby@a9d2091] (v20.10.0-beta1)
which was not gated by API version, as handling of the header was broken
(using the client version, instead of the API version used for the request).
Given that any current version of the daemon, regardless of API version will
ignore the header, this code was only in place to allow connecting to a
daemon older than (v20.10.0-beta1), which would be long EOL now.
[moby@a9d2091]: a9d20916c3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `force` option on volume remove was added in [moby@6c5c34d] (docker
1.13.0-rc1, API v1.25), but did not gate the feature to API version, so
effectively introduced it to all existing API versions. After this,
[moby@e98e4a7] enabled experimental features by default, and added API
version gates, but only did so on the client side, so the daemon / API
server would continue to accept the `force` option on any API version.
Let's remove this code, given that:
- API v1.24 is the oldest API version we still handle, and only as fallback.
- This code silently discards the user's option (no warning / error)
- Every current version of the daemon handles the option, regardless
of API version (only a 9+ year old daemon wouldn't handle it).
[moby@6c5c34d]: 6c5c34d50d
[moby@e98e4a7]: e98e4a7111
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The WithMockClient option was explicitly resetting the client's API
version (see [1]), which differs from the regular client, which is
initialized with the current API version used by the client (see [2]).
This patch:
- reduces the `WithMockClient` to only set the custom HTTP client, leaving
other fields un-touched.
- adds a test utility and updates tests to handle the API-version prefix
- removes redundant uses of `WithVersion()` in tests; for most test-cases
it was used to make sure a current API version is used that supports the
feature being tested, but there was no test to verify the behavior for
lower API versions, so we may as well test against "latest".
[1]: 5a582729d8/client/client_mock_test.go (L22-L36)
[2]: 5a582729d8/client/client.go (L167-L190)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function is used to validate a service-spec for a specific API
version; renaming it to be less ambiguous.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The config is a required argument (to create a container, at least
an image is needed), but the function was missing a check for this,
which would result in a panic if the client was using API v1.44 or
up due to the changes from ee9f0ed895
attempting to [reset the deprecated `MacAddress` field][1].
In practice, this would unlikely be hit, and we didn't hit this in
unit-tests, due to a bug in `WithMockClient`, which initializes the
client with an [empty API version][2], which is different from the
actual client, which [initializes the client with the MaxAPIVersion][3]
This patch updates the function to return an error if a nil config is
passed.
[1]: 5a582729d8/client/container_create.go (L72-L75)
[2]: 5a582729d8/client/client_mock_test.go (L22-L36)
[3]: 5a582729d8/client/client.go (L167-L190)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
CheckDuplicate is removed in API v1.44, and no longer used by
daemons supporting that API version (v25.0.0-beta.1 and up)
regardless of the API version used, but it must be set to true
when sent to older daemons (see [moby@78479b1]).
This patch moves adding the field to the client through an ad-hoc struct
so that we don't have to carry the field in the API module.
We can remove this once daemon versions v24.0 and lower are no longer
expected to be used (when Mirantis Container Runtime v23 is EOL).
https://github.com/moby/moby/blob/v2.0.0-beta.0/project/BRANCHES-AND-TAGS.md.
This field was removed from API v1.44 and no longer used by daemons supporting
that API version (v25.0.0-beta.1 and up) regardless of the API version used,
but for older version of the daemon required this option to be set.
[moby@78479b1]: 78479b1915
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pass the Route as a whole, instead of some of its properties; this
allows the method to act on additional information provided by the
route.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was used in the CLI to produce a warning added in [moby@4a8b3ca]
to print a warning when building Linux images from a Windows client.
Window's filesystem does not have an "executable" bit, which mean that,
for example, copying a shell script to an image during build would lose
the executable bit. So for Windows clients, the executable bit would be
set on all files, unconditionally.
Originally this was detected in the client, which had direct access to
the API response headers, but when refactoring the client to use a common
library in [moby@535c4c9], this was refactored into a `ImageBuildResponse`
wrapper, deconstructing the API response into an `io.Reader` and a string
field containing only the `OSType` header.
The warning was removed in [cli@af65ee4], so we don't have to carry this
field in the new client module going forward.
With the field removed, we can consider the client to return the full
HTTP response again, but leaving that for a follow-up, as we may want
to rewrite these streaming functions altogether.
[moby@4a8b3ca]: 4a8b3cad60
[moby@535c4c9]: 535c4c9a59
[cli@af65ee4]: af65ee4584
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Type type was defined before the API had a definition fro the exec-inspect
response. When a type definition was added in [moby@2a34207], the definition
was moved from the backend to the API, and the backend type implemented as
an alias.
Technically, we could keep a _concrete_ type for the backend, and handle
conversion to the corresponding API type in the router, but currently,
this would likely only add extra complexity.
We could still opt for doing so when the backend requires additional fields
or changes that should not be reflected in the API response.
[moby@2a34207]: 2a342079c6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type was introduced in [moby@3f9f231], at which type no API response
types were defined, and the [`containerRouter.getExecByID`] would return
the daemon's internal [`exec.Config`] type from [`backend.ContainerExecInspect`].
Tracing back history about the discrepancy between the type used by the client
and the actual response type; commit [moby@2a34207] added the missing type in
the API, which was documented as part of the API swagger definition since the
start ([moby@0243936]), and updated in [moby@74cb739], so we can't use the
reduced struct as response type.
[moby@3f9f231]: 3f9f23114f
[moby@2a34207]: 2a342079c6
[`containerRouter.getExecByID`]: 3f9f23114f/api/server/router/container/exec.go (L18-L25)
[`backend.ContainerExecInspect`]: 3f9f23114f/api/server/router/container/backend.go (L18)
[`exec.Config`]: 3f9f23114f/daemon/exec/exec.go (L13-L31)
[moby@0243936]: 0243936d92
[moby@74cb739]: 74cb739766
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option is no longer supported by runc, and is deprecated in the kernel.
We removed support for this feature from all API versions, so it's better
to also amend the docs for older API versions.
[kernel v5.4]: https://github.com/torvalds/linux/commit/0158115f702b0ba208ab0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
kernel-memory limits are not supported in cgroups v2, and were obsoleted in
[kernel v5.4], producing a `ENOTSUP` in kernel v5.16. Support for this option
was removed in runc and other runtimes, as various LTS kernels contained a
broken implementation, resulting in unpredictable behavior.
We deprecated this option in [moby@b8ca7de], producing a warning when used,
and actively ignore the option since [moby@0798f5f].
Given that setting this option had no effect in most situations, we should
just remove this option instead of continuing to handle it with the expectation
that a runtime may still support it.
Note that we still support RHEL 8 (kernel 4.18) and RHEL 9 (kernel 5.14). We
no longer build packages for Ubuntu 20.04 (kernel 5.4) and Debian Bullseye 11
(kernel 5.10), which still have an LTS / ESM programme, but for those it would
only impact situations where a runtime is used that still supports it, and
an old API version was used.
[kernel v5.4]: https://github.com/torvalds/linux/commit/0158115f702b0ba208ab0
[moby@b8ca7de]: b8ca7de823
[moby@0798f5f]: 0798f5f5cf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This introduces a `WithMinimumAPIVersion` RouteWrapper to configure the
minimum API version required for a route. It produces a 400 (Invalid Request)
error when accessing the endpoint on API versions lower than the given version.
Note that technically, it should produce a 404 ("not found") error,
as the endpoint should be considered "non-existing" on such API versions,
but 404 status-codes are used in business logic for various endpoints.
This patch allows removal of corresponding API-version checks from the client,
and other implementation of clients for the API. While the produced error message
is slightly more "technical", these situations should be rare and only happen
when the API version of the client is explicitly overridden, or a client was
implemented with a fixed API version (potentially missing version checks).
Before this patch, these errors were produced by the client:
DOCKER_API_VERSION=v1.24 docker container prune -f
docker container prune requires API version 1.25, but the Docker daemon API version is 1.24
With this patch applied, the error is returned by the daemon:
DOCKER_API_VERSION=v1.24 docker container prune -f
Error response from daemon: POST /containers/prune requires minimum API version 1.25
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Windows test workflow jobs were missing the dependency on the
`validate-dco` job so they ran regardless whether the DCO check passed
or not.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The format for filters changed in 93d1dd8036
(docker v1.10 / API v1.22). As part of that implementation, the daemon
would parse the new format, and fall back to parsing the old format if
this failed. This fallback was not based on API version, so any version
of the API released since would continue to accept both the legacy and
curent format.
For the client, the change in format caused a regression when connecting
to an older daemon; a `ToParamWithVersion` utility was introduced in
[docker/engine-api@81388f0] to produce the old format when the client was
connected to a docker v1.9 or older daemon, using an old API version.
Given that any version of docker 1.10 or above would support both formats,
regardless of the API version used, and API v1.22 is no longer supported,
it should be safe to assume we can drop the version-specific format in the
client. Even if the client would be using API v1.22 (or older), the format
would only be necessary for an actual docker v1.9 daemon, which would be
very unlikely, and a daemon that's 9 Years old.
[docker/engine-api@81388f0]: 81388f00dd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On docker 1.12 (API v1.24) and older, the `SecurityOptions` field of the
`/info` response would only list names of the security options that are
enabled in the daemon. API v1.25 added additional information to this
information. Initially, this included a change to return the information
in structured format (b237189e6c), which
was a backward-incompatible change, so an alternative format was introduced
in 514ca09426 to used a string-slice, but
prefixing options with `name=`, followed by the name of the security-options
and any config options related to it as `key[=<value>]` pairs.
On current API versions:
curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.51/info' | jq .SecurityOptions
[
"name=seccomp,profile=builtin",
"name=cgroupns"
]
On API version v1.24:
curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.24/info' | jq .SecurityOptions
[
"seccomp",
"cgroupns"
]
The Docker CLI unconditionally handles either format when presenting the
information; for backward-compatibility, it contains fallback code to handle
cases where no `name=` prefix is present, but this logic is not based on
API version.
Given that any current version of the CLI is handling either format, and
versions of the CLI that did not have this handling are at least 9 Years
old (and long EOL), removing the old format is unlikely to be causing
issues and we can remove this special handling, and return the information
in the current format.
If we consider this information to be relevant for clients, we should
ultimately consider making it available in a more structured format as
was the original intent of b237189e6c.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
