Set the GO_MD2MAN make variable to elide building go-md2man from
vendored sources and use the specified command instead.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit edfde78355)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Vendor the go-md2man tool used to generate the man pages so that the
only dependency is a Go toolchain.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 05d7008419)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Mostly, this makes sure that literals (such as true, false, host,
private, examples of options usage etc.) are typeset in bold, except for
filenames, which are typeset in italic.
While at it,
- remove some default values from synopsis as it should not
be there;
- fix man pages references (page name in bold, volume number in
regular).
This is not a complete fix, but a step in the right direction.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 690d166632)
Signed-off-by: Cory Snider <csnider@mirantis.com>
1. Escape asterisks and underscores, that have special meaning in
Markdown. While most markdown processors are smart enough to
distinguish whether it's a literal * or _ or a formatting directive,
escaping makes things more explicit.
2. Fix using wrong level of headings in some dm options (most are ####,
but some were #####).
3. Do not use sub-heading for examples in some dm options (this is how
it's done in the rest of the man page).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 374b779dd1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
This fix tries to address issues raised in moby/moby#44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.
Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
(cherry picked from commit a8b8f9b288)
Signed-off-by: Cory Snider <csnider@mirantis.com>
This removes documentation related to legacy overlay networks using
an external k/v store.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68e9223289)
Signed-off-by: Cory Snider <csnider@mirantis.com>
- the `--disable-legacy-registry` daemon flag was removed
- duplicate keys with conflicting values for engine labels
now produce an error instead of a warning.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 13ff896b38)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Fix 19 typos, grammatical errors and duplicated words.
These fixes have minimal impact on the code as these are either in the
doc files or in comments inside the code files.
Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
(cherry picked from commit 20f8455562)
Signed-off-by: Cory Snider <csnider@mirantis.com>
This builds (and depends) on https://github.com/moby/moby/pull/34087
Version 2:
- remove --ipc argument validation (it is now done by daemon)
- add/document 'none' value
- docs/reference/run.md: add a table with better modes description
- dockerd(8) typesetting fixes
Version 3:
- remove ipc mode tests from cli/command/container/opts_test.go
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c23d4b017a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
This is a new option added specifically to allow for debugging of bugs
in Docker's storage drivers or libdm itself.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit 25baee8ab9)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Prepare to move the dockerd man page back to this repository from
docker/cli, retaining history.
This partially reverts commit b5579a4ce3.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 7d3f09a9c3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
With graphdrivers, the old image was still kept as a dangling image.
Keep the same behavior with containerd.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit db40a6132b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When loading a multi-platform image, it's not necessary to unpack all
platforms, especially those which have a completely different OS.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6ebe6a7353)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Log the error to the progress output instead.
The image is still loaded into the content store and image service even
if the unpacking failed, so don't error out the whole operation to avoid
missing the load events for other image names loaded from the same
archive.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 728894b7d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use a regular for loop instead of ApplyAll which spawns a separate
goroutine for each separate container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a5d75f6d27)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `GET /images/json` requires an optional `container-count` parameter
which set the `Containers` property of in the ImageSummary to a number
of containers using that image.
This was also propagated to the new manifest list property which
includes a list of all the container IDs that are using this specific
image manifest.
Disconnect the `ImageData.Containers` property from this option and
always include it by default without an explicit opt-in.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b93cf37dcd)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).
There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 050afe1e1a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
These must have a whitespace before them, otherwise they are ignored.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 218c08b283)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit 0071832226 introduced
per-endpoint sysctls, and migration to them from the top-level
'--sysctl' option.
The migration was intended to be short-term, disabled in the
next major release and code was added to check for the next
API version. But now, the API version will be bumped in a
minor release - this breaking change needs to wait until the
next major release, and we don't yet know the API version
number for that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 17adc1478b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When defaultipam.newPoolData is asked for a pool of 64-bits
or more, it ends up with an overflowed u64 - so, it just
subtracts one to get a nearly-big-enough range (for a 64-bit
subnet).
When defaultipam.getAddress is called with an ipr (sub-pool
range), the range it calls bitmask.SetAnyInRange with is
exclusive of end. So, its end param can't be MaxUint64,
because that's the max value for the top end of the range
and, when checking the range, SetAnyInRange fails.
When fixed-cidr-v6 behaves more like fixed-cidr, it will ask
for a 64-bit range if that's what fixed-cidr-v6 needs. So,
it hits the bug when allocating an address for, for example:
docker network create --ipv6 --subnet fddd::/64 --ip-range fddd::/64 b46
The additional check for "ipr == base" avoids the issue in
this case, by ignoring the ipr/sub-pool range if ipr is the
same as the pool itself (not really a sub-pool).
But, it still fails when ipr!=base. For example:
docker network create --ipv6 --subnet fddd::/56 --ip-range fddd::/64 b46
So, also subtract one from 'end' if it's going to hit the max
value allowed by the Bitmap.
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 496b457ad8)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Replace the regexp check with a function.
Keep the use of regexp.QuoteMeta.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 508939821b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Replace the regexp checking ID validity with a for loop.
The benefits are:
- faster (up to 10x faster with less allocations);
- no init overhead to compile the regexp.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b66d4b567a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Replace the regexp checking ID validity with a function. The benefits
are:
- function is faster (up to 10x faster with less allocations);
- no init overhead to compile the regexp;
Add a test case.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1c0dc8a94f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.
Rendering the docs (`make swagger-docs`) showed a warning:
> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7dec1c67a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.
Rendering the docs (`make swagger-docs`) showed a warning:
> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 66b5b8bfa8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit 2ce811e632 migrated the use of the
userns package to the github.com/moby/sys/user module.
After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).
This patch migrates our code to use the new module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b0ef10a9a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
[buildkit@29b4b1a537][1] applied changes to `buildkitd` to set the correct
defaults, which should be 16MB, but used the library defaults. Without that
change, builds using large Dockerfiles would fail with a `ResourceExhausted`
error;
=> [internal] load build definition from Dockerfile
=> => transferring dockerfile: 896.44kB
ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (44865299 vs. 16777216)
However those changes were applied to the `buildkitd` code, which is the
daemon when running BuildKit standalone (or in a container through the
`container` driver). When running a build with the BuildKit builder compiled
into the Docker Engine, that code is not used, so the BuildKit changes did
not fix the issue.
This patch applies the same changes as were made in [buildkit@29b4b1a537][1]
to the gRPC endpoint provided by the dockerd daemon.
[1]: 29b4b1a537
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cdbfae1d3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].
The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.
[1]: 3778ae603c
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2ce811e632)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`dockerd-rootless-setuptool.sh check` now skips the smoke test for
running RootlessKit.
Fix docker/docker-install issue 417
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e2237240f5)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Without this, the dependency on systemd is said to be "automagic", which
can lead to breakage, for example, if a binary package of docker is
built on a system that has systemd installed then installed on a system
that does not have systemd installed.
for example: https://bugs.gentoo.org/914076
Signed-off-by: William Hubbs <w.d.hubbs@gmail.com>
(cherry picked from commit 499c842c52)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Docker EE is no more. Point users looking for commercial support at the
currently-maintained commercial products based on the Moby project:
Docker Desktop and Mirantis Container Runtime.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit b37c983d31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This check was updated in f460110ef5, but
introduced some bugs;
- the regular expressions were meant to define a capturing group, but
the braces (`(`, `)`) were escaped (they previously were used by
`sed`, which requires different escaping), so no value was captured.
- the check itself was not updated to use the resulting `$target_branch`
env-var, so was comparing against the `$GITHUB_BASE_REF` (which is
the branch name before stripping minor versions).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e0b98a3222)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We'll be using release branches for minor version updates, so instead
of (e.g.) a 27.0 branch, we'll be using 27.x and continue using the
branch for minor version updates.
This patch changes the validation step to only compare against the
major version.
Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f460110ef5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package is only used by the daemon, so move it to the internal
rootless package instead.
Note that technically this could be in daemon/internal, but as there's
already an existing internal/rootless package (which needs to be in the
top-level internal package because it's also used by /plugin), I'm moving
it there.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit efdaca2792)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The only external consumer are the `graphdriver` and `graphdriver/shim`
packages in github.com/docker/go-plugins-helpers, which depended on
[ContainerFS][1], which was removed in 9ce2b30b81.
graphdriver-plugins were deprecated in 6da604aa6a,
and support for them removed in 555dac5e14,
so removing this should not be an issue.
Ideally this package would've been moved inside `daemon/internal`, but it's used
by the `daemon` (cleanupContainer), `plugin` package, and by `graphdrivers`,
so needs to be in the top-level `internal/` package.
[1]: 6eecb7beb6/graphdriver/api.go (L218)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2970e5358)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Improve some GoDoc to use docs links
- Change the Windows stub to an actual wrapper function, as we don't
want it to be updateable, and it currently shows as "variable" on
pkg.go.dev, which is confusing.
- Remove "import" comments in preparation of moving this package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a3e6ce95c4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The execution-driver was replaced with containerd since docker 1.11 (API
v1.23) in 9c4570a958, after which the value
was no longer set. The field was left in the type definition.
Commit 1fb1136fec removed its use from the
CLI and [docker/engine-api@39c7d7e] removed it from the API type, followed
by an update to the API docs in 3c6ef4c29d.
Changes to the API types were not pulled into the engine until v1.13, and
probably because of that gated it on API version < 1.25 instead of < 1.24
(see 6d98e344c7); setting a "not supported"
value for older versions.
Based on the above; this field was deprecated in API v1.23, and empty
since then. Given that the minimum API version supported by the engine
is not v1.24, we can safely remove it.
[docker/engine-api@39c7d7e]: 39c7d7ec19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e4d792a06d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.
Add a callback that is called by the exporter wrapper.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 1506bbcfe8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This is only a callback that notifies about event so there is no way to
react to the error.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit bce76d486e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The test checks that it's possible to set a per-interface sysctl
using '--sysctl' - but, after API v1.46, it's not (and driver option
'com.docker.network.endpoint.sysctls' must be used instead).
Test added in commit fde80fe2
Per-interface sysctls added, with API changes, in commit 00718322
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit f649fd0c97)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Update the containerd binary that's used in CI and for the static packages.
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.19
- full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19
Welcome to the v1.7.19 release of containerd!
The nineteenth patch release for containerd 1.7 contains various updates and
splits the main module from the api module in preparation for the same change
in containerd 2.0. Splitting the modules will allow 1.7 and 2.x to both exist
as transitive dependencies without running into API registration errors.
Projects should use this version as the minimum 1.7 version in preparing to
use containerd 2.0 or to be imported alongside it.
Highlights
- Fix support for OTLP config
- Add API go module
- Remove overlayfs volatile option on temp mounts
- Update runc binary to v1.1.13
- Migrate platforms package to github.com/containerd/platforms
- Migrate reference/docker package to github.com/distribution/reference
Container Runtime Interface (CRI)
- Fix panic in NRI from nil CRI reference
- Fix Windows HPC working directory
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 398e15b7de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d0aa3eaccf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Highlights
- Fix support for OTLP config
- Add API go module
- Remove overlayfs volatile option on temp mounts
- Update runc binary to v1.1.13
- Migrate platforms package to github.com/containerd/platforms
- Migrate reference/docker package to github.com/distribution/reference
Container Runtime Interface (CRI)
- Fix panic in NRI from nil CRI reference
- Fix Windows HPC working directory
full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8983957ac5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Fix process handle leak when launching a job container
- Add EndpointState attribute to the HNSEndpoint struct to support
hyperv containers for k8s
- Add support for loadbalancer policy update in hns
- Changes for checking the global version for modify policy version support
- OutBoundNATPolicy Schema changes (add MaxPortPoolUsage to OutboundNatPolicySetting)
full diff: https://github.com/microsoft/hcsshim/compare/v0.11.5...v0.11.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a2fe103f0d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.12+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.11...go1.21.12
These minor releases include 1 security fixes following the security policy:
net/http: denial of service due to improper 100-continue handling
The net/http HTTP/1.1 client mishandled the case where a server responds to a request with an "Expect: 100-continue" header with a non-informational (200 or higher) status. This mishandling could leave a client connection in an invalid state, where the next request sent on the connection will fail.
An attacker sending a request to a net/http/httputil.ReverseProxy proxy can exploit this mishandling to cause a denial of service by sending "Expect: 100-continue" requests which elicit a non-informational response from the backend. Each such request leaves the proxy with an invalid connection, and causes one subsequent request using that connection to fail.
Thanks to Geoff Franks for reporting this issue.
This is CVE-2024-24791 and Go issue https://go.dev/issue/67555.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.21.12
**- Description for the changelog**
```markdown changelog
Update Go runtime to 1.21.12
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4d1d7c3ebe)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Regression introduced in 01eecb6.
A port mapping from a specific IPv6 host address can be used
by a container on an IPv4-only network, docker-proxy makes the
connection.
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit dfbcddb9f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Keep allocated subnets in-order, so that they're not mistakenly
reallocated due to a gap in the list where misplaced subnets should
have been.
Introduced in 9d288b5.
The iterator over allocated subnets was incremented too early, this
change moves it past three clauses in addrSpace.allocatePredefinedPool().
The three new unit tests correspond to a separate failure caused by
incrementing before each of them.
(cherry picked from commit 4de54ee14c)
Signed-off-by: Rob Murray <rob.murray@docker.com>
createTarFile accepts a opts (TarOptions) argument to specify whether
userns is enabled; whe should consider always detecting locally, but
at least make sure we're consistently working with the same value within
this function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 969993a729)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was discarding EPERM errors if it detected that userns was
enabled; move such checks to the caller-site, so that they can decide
how to handle the error (which, in case of userns may be to log and ignore).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 379ce56cd8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since 838047a1f5, the overlayWhiteoutConverter
is supported with userns enabled, so we no longer need this check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af85e47343)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like the way it picks up #nosec comments changed, causing the
linter error to re-appear;
pkg/archive/archive_linux.go:57:17: G305: File traversal when extracting zip/tar archive (gosec)
Name: filepath.Join(hdr.Name, WhiteoutOpaqueDir),
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d4160d5aa7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like the way it picks up #nosec comments changed, causing the
linter error to re-appear;
builder/remotecontext/remote.go:48:17: G107: Potential HTTP request made with variable url (gosec)
if resp, err = http.Get(address); err != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 04bf0e3d69)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `Node` field and related `ContainerNode` type were used by the classic
(standalone) Swarm API. API documentation for this field was already removed
in 234d5a78fe (API 1.41 / docker 20.10), and
as the Docker Engine didn't implement these fields for the Swarm API, it
would always have been unset / nil.
Let's do a quick deprecation, and remove it on the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1fc9236119)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Graphdriver plugins] are an experimental feature that allow extending the
Docker Engine with custom storage drivers for storing images and containers.
This feature was not maintained since its inception, and will no longer be
supported in upcoming releases.
Users of this feature are recommended to instead configure the Docker Engine
to use the [containerd image store], and a custom [snapshotter].
This patch:
- Disables loading graphdriver plugins by default, producing an error instead.
- Introduces a temporary `DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS` environment
variable to re-enable the deprecated features; this allows users to still
use the feature on a v27.0 daemon, but disabling it by default will give a
strong message that it will no longer be supported.
[Graphdriver plugins]: https://github.com/docker/cli/blob/v26.1.4/docs/extend/plugins_graphdriver.md
[containerd image store]: https://docs.docker.com/storage/containerd/
[snapshotter]: https://github.com/containerd/containerd/tree/v1.7.18/docs/snapshotters
Before this patch (ignore the "Unable to load plugin" errors, as there's no plugin);
dockerd --experimental -s my-driver
...
INFO[2024-06-21T10:42:49.574901255Z] containerd successfully booted in 0.011384s
INFO[2024-06-21T10:42:50.575891922Z] [graphdriver] trying configured driver: my-driver
WARN[2024-06-21T10:42:50.576121547Z] Unable to locate plugin: my-driver, retrying in 1s
WARN[2024-06-21T10:42:51.577131506Z] Unable to locate plugin: my-driver, retrying in 2s
WARN[2024-06-21T10:42:53.582637715Z] Unable to locate plugin: my-driver, retrying in 4s
With this patch:
dockerd --experimental -s my-driver
...
INFO[2024-06-21T10:32:35.123078845Z] [graphdriver] trying configured driver: my-driver
ERRO[2024-06-21T10:32:35.123127012Z] Failed to GetDriver graph driver=my-driver error="DEPRECATED: Experimental graphdriver plugins are deprecated, and disabled by default. This feature will be removed in the next release. See https://docs.docker.com/go/deprecated/" home-dir=/var/lib/docker
INFO[2024-06-21T10:32:35.124735595Z] stopping healthcheck following graceful shutdown module=libcontainerd
INFO[2024-06-21T10:32:35.124743137Z] stopping event stream following graceful shutdown error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: error initializing graphdriver: driver not supported: my-driver
With the `DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS` env-var set:
DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS=1 dockerd --experimental -s my-driver
...
INFO[2024-06-21T10:35:04.149901970Z] containerd successfully booted in 0.013614s
INFO[2024-06-21T10:35:05.148195845Z] [graphdriver] trying configured driver: my-driver
WARN[2024-06-21T10:35:05.150647679Z] Unable to locate plugin: my-driver, retrying in 1s
WARN[2024-06-21T10:35:06.152531221Z] Unable to locate plugin: my-driver, retrying in 2s
WARN[2024-06-21T10:35:08.158452389Z] Unable to locate plugin: my-driver, retrying in 4s
WARN[2024-06-21T10:35:12.163699293Z] Unable to locate plugin: my-driver, retrying in 8s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since it's a [][]string, there should only be two levels of array
in the OpenAPI spec. Also, the outermost level array shouldn't have
properties: (it should have items: instead).
Co-authored-by: Mark Yen <mark.yen@suse.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows distros to put docker-proxy under libexec paths as is done
for docker-init.
Also expands the lookup to to not require a `docker/` subdir in libexec
subdir.
Since it is a generic helper that may be used for something else in the
future, this is only done for binaries with a `docker-`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Move the function internal to the daemon, where it's used. Deliberately
not mentioning the new location, as this function should not be used
externally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 17c3269a37 moved the ContainerStats
type to the container package, and renamed it to StatsResponse. However,
this name is chosen poorly, as it documents it to be the response of
the API endpoint, but is more accurately a wrapper around a reader,
used to read a (stream of) StatsJSON. We want to change StatsJSON
to StatsResponse, as it's more consistent with other response types.
As 17c3269a37 did not make it into a
non-pre-release, we can still change this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- inline the auth.TokenHandlerOptions in the auth.NewTokenHandlerWithOptions call
- construct a authHandlers slice to make it more clear that this is a variadic
list of authentication-handlers.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Found back the PR related to this contributors, and they addressed
their name in an intermediate rebase, but it got lost in a later one.
While at it, also fixed an entry next to it :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the system (or Docker) crashes while saivng the volume options, on
restart the daemon will error out when trying to read the options file
because it doesn't contain valid JSON.
In such a crash scenario, the new volume will be treated as though it
has the default options configuration. This is not ideal, but volumes
created on very old Docker versions (pre-1.11[1], circa 2016) do not
have opts.json and so doing some kind of cleanup when loading the volume
store (even if we take care to only delete empty volumes) could delete
existing volumes carried over from very old Docker versions that users
would not expect to disappear.
Ultimately, if a user creates a volume and the system crashes, a volume
that has the wrong config is better than Docker not being able to start.
[1]: commit b05b237075 ("Support mount opts for `local` volume driver")
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Update daemon versions, and minimum supported API version to be more
representative to what the API would return.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update daemon versions, and minimum supported API version to be more
representative to what the API would return.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I noticed some duplicates made their way in, in
084219a5f9 and some authors
didn't have git configured properly to include the name
they used for the sign-off
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The stringid.TruncateID utility is used to provide a consistent length
for "short IDs" (containers, networks). While the dummy interfaces need
a short identifier, they use their own format and don't have to follow
the same length as is used for "short IDs" elsewhere.
In addition, stringid.TruncateID has an additional check for the given
ID to contain colons (":"), which won't be the case for network-IDs that
are passed to it, so this check is redundant.
This patch moves the truncating local to the getDummyName function, so
that it can define its own semantics, independent of changes elsewhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function description mentions that the returned value will contain
a truncated ID, but the function was only prepending the prefix, which
meant that callers had to be aware that truncating is necessary.
This patch moves truncating the ID into the utility to make its use
less error-prone, and to make the code a bite more DRY.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The stringid.TruncateID utility is used to provide a consistent length
for "short IDs" (containers, networks). While the dummy interfaces need
a short identifier, they use their own format and don't have to follow
the same length as is used for "short IDs" elsewhere.
In addition, stringid.TruncateID has an additional check for the given
ID to contain colons (":"), which won't be the case for network-IDs that
are passed to it, so this check is redundant.
This patch moves the truncating local to the getDummyName function, so
that it can define its own semantics, independent of changes elsewhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function description mentions that the returned value will contain
a truncated ID, but the function was only prepending the prefix, which
meant that callers had to be aware that truncating is necessary.
This patch moves truncating the ID into the utility to make its use
less error-prone, and to make the code a bite more DRY.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Following changes to the port mapping code, the DNAT iptables rule was
inserted into the nat table rather than appended.
This meant DNAT was applied before the rule that should have skipped
it when a packet was from a bridge network.
So, packets sent from a container on one network to a mapped port on
the host's address were DNAT'd before docker-proxy could pick them up,
then they were dropped by a rule intended to isolate the networks.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This is the API version to be released with v27.0, and the API
is now frozen for this release, so we can create the documentation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type is included in various types used in the API, but comes from
a separate module. The go-units module may be moving to the moby org,
and it is yet to be decided if the Ulimit type is a good fit for that
module (which deals with more generic units, such as "size" and "duration"
otherwise).
This patch introduces an alias to help during the transition of this type
to it's new location. The alias makes sure that existing code continues
to work (at least for now), but we need to start updating such code after
this PR is merged.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- rename variables that shadowed imports
- remove some intermediate vars
- slight reformating for readability
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 18f4f775ed.
Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).
Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.
By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This reverts commit d365702dbd.
Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).
Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.
By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.
Signed-off-by: Rob Murray <rob.murray@docker.com>
While working on this file, I noticed the `isContainerCreateNameConflict`,
`isUnknownContainer`, and `isStoppedContainer` utilities, which are used
to perform error-type detection through string-matching.
These utilities were added in 534a90a993,
as part of the initial implementation of the Swarm executor in Docker.
At that time, the Docker API client did not return typed errors, and
various part of the code depended on string matching, which is brittle,
and it looks like `isContainerCreateNameConflict` at least is already
broken since c9d0a77657, which changed
the error-message.
Starting with ebcb7d6b40, we use typed
errors through the errdefs package, so we can replace these utilities:
The `isUnknownContainer` utility is replace by `errdefs.IsNotFound`,
which is returned if the object is not found. Interestingly, this utility
was checking for containers only (`No such container`), but was also
used for an `removeNetworks` call. Tracking back history of that use to
verify if it was _intentionally_ checking for a "container not found"
error;
- This check added in the initial implementation 534a90a993
- Moved from `controller.Remove` to `container.Shutdown` to make sure the
sandbox was removed in 680d0ba4ab
- And finally touched again in 70fa7b6a3f,
which was a follow-up to the previous one, and fixed the conditions
to prevent returning early before the network was removed.
None of those patches mention that these errors are related to containers,
and checking the codepath that's executed, we can only expect a
`libmetwork.ErrNoSuchNetwork` to be returned, so this looks to have been
a bug.
The `isStoppedContainer` utility is replaced by `errdefs.IsNotModified`,
which is the error (status) returned in situations where the container
is already stopped; caf502a0bc/daemon/stop.go (L30-L35)
This is the only
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function returns the default network to use for the daemon platform;
moving this to a location separate from runconfig, which is planned to
be dismantled and moved to the API.
While it might be convenient to move this utility inside api/types/container,
we don't want to advertise this function too widely, as the default returned
can ONLY be considered correct when ran on the daemon-side. An alternative
would be to introduce an argument (daemonPlatform), which isn't very convenient
to use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility is only used in two places, and simple enough to duplicate.
There's no external consumers, and a copy of this utility exists in docker/cli
for use on the client side, so we could consider skipping deprecation,
but just to be on the safe side ':)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These used aliases that weren't used elsewhere, so renaming / removing
to keep some consistency. Some local variables were renamed to prevent
shadowing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This populates the "Image" field on containerd containers, but only when
using the containerd image store.
This allows containerd clients to look up the image information.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This will be used in the next commit to test that changes are propagated
to the containerd store.
It is also just generally useful for debugging purposes.
- docs/api: update version history
- daemon: add fillContainerdInfo utility
- api: update swagger file with new types
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Buildkit doesn't call the engine API when it builds an image without
tag. Wrap the exporter returned by the worker that calls a callback when
a new image is exported from buildkit.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This const was exported because it was in use by pkg/symlink. This
dependency was removed in a48c6e3005,
after which this const was only used internally.
This patch deprecates the const and introduces a non-exported const
to use.
There are no known external consumers of this const, so we may skip
deprecating it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use a switch instead of if/else for readability and to reduce
the risk of duplicates in the checks.
- Align order between Windows and Linux implementation for easier
comparing of differences in the implementation.
- Add a check for `IsHost()` in the Windows implementation which
would never occur currently, but is implemented.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not a full list yet, but renaming to prevent shadowing, and to use a more
consistent short form (ctr for container).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The commit endpoint accepts a container.Config, but uses the decoder to
unmarshal the request. The decoder uses a CreateRequest, which is a superset,
and also contains HostConfig and network.NetworkConfig. Those structs are
discarded in the router, but decoder.DecodeConfig also performs validation,
so a request containing those additional fields would result in a validation
error.
We should rewrite this code to only unmarshal what's expected.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the type to api/types/container.CreateRequest, together with other
types used by the container API endpoints.
The Decoder, and related validation code is kept in the runconfig package
for now, but should likely be moved elsewhere (inside the API).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove uses of this function and mark it deprecated. There's no known
consumers of this function, but let's stay on the safe side, and mark
it deprected for 1 release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility used to be responsible for backward compatibility with old
API versions, but was reduced to a single line. Inline the code, and
inline the SetDefaultNetModeIfBlank code, which in itself also was
just 3 lines of code.
A platform check was added to only set the default network conditionally,
but other paths in the codebase don't perform this conditionally, so a
TODO was added, to verify if this behavior is needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that the backward-compatibility code has been removed, the Linux and
Windows implementations of this struct are identical, so the platform-
specific code can be removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The runconfig package contained compatibility code to unmarshal API
requests on API < v1.18, and to convert them to current API versions.
These fields were marked as deprecated, but didn't mention relevant API
versions, so some digging was needed to track back history;
API versions before 1.18 accepted top-level `Memory`, `MemorySwap`,
`CpuShares`, and `Cpuset` fields as part of the container create requests.
These fields were not considered "portable", and therefore moved to the
`HostConfig` struct in 837eec064d. The
API version at that time was [v1.18]. For backward-compatibility, the
existing top-level fields were kept, and conversion code was added in
[ContainerHostConfigFromJob] to copy their values to `HostConfig` if
present.
A refactor in 767df67e31 introduced a new
`ContainerConfigWrapper` struct, which embedded the container-config and
a (non-exported) `hostConfigWrapper`. This resulted in an incompatibility
when compiling with gccgo, sn eb97de7dee
removed the non-exported `hostConfigWrapper`, instead embedding the
`HostConfig` and adding a `CpuSet` field. The API version at that time
was [v1.19].
With the introduction of Windows containers, which did not need conversion
code as it never supported previous API versions, the `ContainerConfigWrapper`
was split to Linux and Windows implementation in f6ed590596.
This change introduced a `SetDefaultNetModeIfBlank` function to set the
default network-mode on Linux. Windows did not have a default network,
but did require a separate `ValidateNetMode` implemenation.
The `ContainerConfigWrapper` was expanded to include `NetworkingConfig`
in 2bb3fc1bc5 for API [v1.22], but did
not involve backward-compatiblity / conversion code.
Based on the above, all conversion code present in runconfig is related
to API versions [v1.18] or before. 19a04efa2f,
and other commits in [moby PR 47155] removed support for API < v1.24, so
this conversion code is no longer needed.
This patch removes the legacy fields from the `ContainerConfigWrapper`,
and removes the corresponding conversion code. The `InnerHostConfig` field
is also renamed, as it is no longer shadowed by the `container.HostConfig`
that was embedded for backward-compatibility.
[v1.18]: 837eec064d/api/common.go (L18)
[v1.19]: 767df67e31/api/common.go (L20)
[v1.22]: 2bb3fc1bc5/api/common.go (L21)
[moby PR 47155]: https://github.com/moby/moby/pull/47155
[ContainerHostConfigFromJob]: 837eec064d/runconfig/hostconfig.go (L149-L162)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was depending on top-level fields that were deprecated since
API v1.18. These fields are no longer sent by current clients.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This wrapper is now a plain alias for hcsshim.HNSEndpointRequest, so let's
remove the extra abstraction.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This synchronisation was added in [libnetwork@0a61693]:
> Adding synchronization around peerAdd and peerDelete to prevent network
> connectivity issue
>
> When multiple networks are present in a Swarm Cluster, multiple peerAdd
> or peerDelete calls are an issue for different remote endpoints. These
> threads are updating the remote endpoint to HNS parallelly. In 2016 HNS
> code base, we don't have synchronization around remoteEndpoint addition
> and deletion. So serializing the peerAdd and peerDelete calls from docker
> network driver.
We no longer support and test Windows 2016, as it reached EOL / end of
[standard support][1], so we can remove this special condition.
[libnetwork@0a61693]: c90114ce7c
[1]: https://en.wikipedia.org/wiki/Windows_10,_version_1607
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
TestChangesDirsEmpty and TestChangesDirsMutated fail on Windows V19H1 (1903)
and up, possibly due to changes in the kernel:
=== FAIL: github.com/docker/docker/pkg/archive TestChangesDirsEmpty (0.21s)
changes_test.go:261: Reported changes for identical dirs: [{\dirSymlink C}]
=== FAIL: github.com/docker/docker/pkg/archive TestChangesDirsMutated (0.14s)
changes_test.go:391: unexpected change "C \\dirSymlink" "\\dirnew"
commit 8f4b3b0ad4 added a version-dependent
skip for those tests, but as we no longer run CI on versions before V19H1,
we can remove the kernel-version check, and skip it on Windows unconditionally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While using iota can be convenient, it can also make it harder to grasp
what value is assigned. Use of iota also makes changing values implicit;
changing the order of these consts implicitly means their value changes.
This can be problematic, as some of these consts are a plain `int` and
while golang is strong-typed, it does allow plain `int` values to be
used for such values.
For example, `archive.Tar` accepts a `Compression` as second argument,
but allows a plain int to be passed, so both of these are equivalent;
archive.Tar(contextDir, archive.Uncompressed)
archive.Tar(contextDir, 0)
This patch removes the use of `iota`, and instead explicitly setting a
value for each to prevent accidental changes in their value, which can
be hard to discover.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
[pkg/system.Lgetxattr] is only implemented on Linux, and always produces
an ErrNotSupportedPlatform on other platforms.
This patch removes the call to this function, but intentionally leaves
it commented-out as a reminder to include this code if this would ever
be refactored and implemented on other platforms.
[pkg/system.Lgetxattr]: d1273b2b4a/pkg/system/xattrs_unsupported.go (L1-L8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were added in baacae8345, but are
currently only used in tests inside pkg/archive. There are no external
users of this function, so we should deprecated them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit d59758450b changed this function to
be a wrapper for `filepath.ToSlash`. It was used in the CLI for the classic
builder, but is no longer used in our codebase.
However, there may still be some consumers that copied the CLI code for the
classic builder that didn't synchronise their implementation yet, so let's
deprecate this function to give them a warning that they should no longer
use this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gosec uses a non-standard format for "automated" comments to suppress
false positives (such comments should not have a leading space, but
are not allowed to start with a non-alphabetical character). However,
current versions of gosec do allow a leading space.
This patch reformats the comments to prevent them from being changed
by IDEs when reformating code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gosec uses a non-standard format for "automated" comments to suppress
false positives (such comments should not have a leading space, but
are not allowed to start with a non-alphabetical character). However,
current versions of gosec do allow a leading space.
This patch reformats the comments to prevent them from being changed
by IDEs when reformating code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package was originally added in 46833ee1c3
for use in the devicemapper graphdriver. The devicemapper graphdriver was
deprecated and has been removed. The only remaining consumer is an integration
test.
Deprecate the package and mark it for removal in the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This value was originally added in 46833ee1c3,
at which time golang.org/x/sys/unix didn't have utilities for this syscall.
A later patch switched the implementation to use the golang/x/sys/unix
implementation in 2841b05b71, but kept the
local variable.
golang.org/x/sys now has a const for this, so let's use it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The only case where macvlan interfaces are unable to share a parent is
when the macvlan mode is passthru. This change tightens the check to
that situation.
It also makes the error message more specific to avoid suggesting that
sharing parents is never correct.
Signed-off-by: Andrew Baxter <423qpsxzhh8k3h@s.rendaw.me>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function is no longer used, and has no external users. Deprecated
the function and mark if for removal for the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function is only used for the legacy v1 image format.
Deprecate the function, and make image/v1 self-contained.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in 7a7357dae1 as part of the
LCOW implementation. LCOW has been removed, and this option was no longer
in use because of that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- explicitly suppress some errors
- use fmt.Fprintln instead of manually appending a newline
- remove an outdated TODO; looking at the suggestion, it's not a
realistic option
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- imageMount was shadowing the imageMount type
- copy was shadowing the copy builtin
- container was shadowing the container import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only used in a single location, and other locations were shadowing
the function through local variables. As it's a one-liner, inlining the
code may be just as transparent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The health-check start interval added in API v1.44, and the start
interval option is ignored when creating a Swarm service using an older
API version. However, due to an oversight, the option is not ignored
when older API clients _update_ a Swarm service. Fix this oversight by
moving the adjustment code into the adjustForAPIVersion function used by
both the createService and updateService handler functions.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Includes two commits from Arash Deshmeh:
add exec option to API TmpfsOptions and the related volume functions
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
feature: daemon handles tmpfs mounts exec option
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
Updated by Drew Erny
Signed-off-by: Drew Erny <derny@mirantis.com>
`Daemon.Kill()` was sending a SIGKILL to the daemon process but wasn't
waiting until the process was really killed. While the race window is
really small, better safe than sorry.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When the container stops or during `restore`, `daemon.releaseNetwork` is
used to clear all net-related state carried by a container. However, the
fields `SandboxID` and `SandboxKey` are never cleared. On the next start,
these fields will be replaced with new values. There's no point in
preserving these data since they became invalid as soon as the container
stopped.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When the daemon crashes, the host unexpectedly reboot, or the daemon
restarts with live-restore enabled, running containers might stop and the
on-disk state for containers might diverge from reality. All these
situations are currently handled by the daemon's `restore` method.
That method calls `daemon.Cleanup()` for all the dead containers. In
turn, `Cleanup` calls `daemon.releaseNetwork()`. However, this last
method won't do anything because it expects the `netController` to be
initialized when it's called. That's not the case in the `restore` code
path -- the `netController` is initialized _after_ cleaning up dead
containers.
There's a chicken-egg problem here, and fixing that would require some
important architectural changes (eg. change the way libnet's controller
is initialized).
Since `releaseNetwork()` early exits, dead containers won't ever have
their networking state cleaned. This led to bugs in Docker Desktop,
among other things.
Fix that by calling `releaseNetwork` after initializing the
`netController`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
setupBridgeIPv6 attempts to add a route to a new network while
the bridge device is 'down', so it always fails (and the route
is added anyway when the bridge is set 'up').
I'm almost sure the RouteAdd can be removed but, this close to
the moby 27.0 release, only sure-enough to demote the log message
from error to debug.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The `Sandbox.SetKey()` method is called through an OCI prestart hook
which then calls back the daemon through a UNIX socket. This method is
responsible for provisioning interfaces, etc... into the sandbox.
A new EnvironCarrier is used to propagate the trace context to the
prestart hook, which then marhsals an OTel MapCarrier into the JSON
payload sent back to the daemon. That way, every spans created from
`SetKey()` are correctly parented to the original `ContainerStart` API
call.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Add a nil check to handle a case where the image config JSON would
deserialize into a nil map.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Update the runc binary that's used in CI and for the static packages.
full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13
Release notes:
* If building with Go 1.22.x, make sure to use 1.22.4 or a later version.
* Support go 1.22.4+.
* runc list: fix race with runc delete.
* Fix set nofile rlimit error.
* libct/cg/fs: fix setting rt_period vs rt_runtime.
* Fix a debug msg for user ns in nsexec.
* script/*: fix gpg usage wrt keyboxd.
* CI fixes and misc backports.
* Fix codespell warnings.
* Silence security false positives from golang/net.
* libcontainer: allow containers to make apps think fips is enabled/disabled for testing.
* allow overriding VERSION value in Makefile.
* Vagrantfile.fedora: bump Fedora to 39.
* ci/cirrus: rm centos stream 8.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13
Release notes:
* If building with Go 1.22.x, make sure to use 1.22.4 or a later version.
* Support go 1.22.4+.
* runc list: fix race with runc delete.
* Fix set nofile rlimit error.
* libct/cg/fs: fix setting rt_period vs rt_runtime.
* Fix a debug msg for user ns in nsexec.
* script/*: fix gpg usage wrt keyboxd.
* CI fixes and misc backports.
* Fix codespell warnings.
* Silence security false positives from golang/net.
* libcontainer: allow containers to make apps think fips is enabled/disabled for testing.
* allow overriding VERSION value in Makefile.
* Vagrantfile.fedora: bump Fedora to 39.
* ci/cirrus: rm centos stream 8.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like some packages fail in go module mode, because they require
recent Go versions:
GO111MODULE=on go test -v
# github.com/docker/docker/libnetwork/ipamutils
../../libnetwork/ipamutils/utils.go:46:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipamutils/utils.go:51:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/portallocator
../../libnetwork/portallocator/portallocator.go:179:7: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/netutils
../../libnetwork/netutils/utils_linux.go:66:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/netutils/utils_linux.go:75:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/api/server/router/grpc
../../api/server/router/grpc/grpc.go:56:48: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/container
../../container/view.go:335:47: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/ipams/defaultipam
../../libnetwork/ipams/defaultipam/address_space.go:33:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:53:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:124:10: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:125:21: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:146:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:310:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:311:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/drivers/bridge
../../libnetwork/drivers/bridge/port_mapping_linux.go:76:15: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/drivers/bridge/port_mapping_linux.go:201:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This internal package was added in f6e44bc0e8
to preserve compatibility with go1.20 and older. At the time, our vendor.mod
still had go1.18 as minimum version requirement (see [1]), which got updated to go1.20
in 16063c7456, and go1.21 in f90b03ee5d
The version of BuildKit we use already started using context.WithoutCancel,
without a fallback, so we no longer can provide compatibility with older
versions of Go, which makes our compatiblity package redundant.
This patch removes the package, and updates our code to use stdlib's context
instead.
[1]: f6e44bc0e8/vendor.mod (L7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added to use a specific format for the vendor.mod/go.mod
file, but we should no longer need this, as go1.21 is now the
minimum.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
26.1.1 added env var DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE to make it
possible to create an IPv4-only network, even with a read-only
"/proc/sys/net" that meant IPv6 could not be disabled on an
interface.
In 27.0 it's easier to enable IPv6, just '--ipv6' when creating the
network - in particular, there's no need to allocate a subnet, because
a unique-local prefix will be assigned by default).
So, this change removes the env-var workaround. Now, the workarounds
are to enable IPv6, mount "/proc/sys/net" read-write, disable IPv6
by default in OS configuration, or remove support for IPv6 from the
kernel.
Signed-off-by: Rob Murray <rob.murray@docker.com>
On an nftables host, the ip6_tables kernel module may not be loaded,
but it needs to be for dockerd to run (with ip6tables now enabled by
default).
If ip6tables doesn't work, try the dind official image's trick for
loading the module using "ip link show".
Signed-off-by: Rob Murray <rob.murray@docker.com>
Small cleanup of this function;
- change to a regular function, as it does not depend on the daemon
- use an early return
- explicitly refer to EndpointSettings.EndpointSettings, not the top-
level EndpointSettings.
- use a struct-literal.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When bridge driver opt com.docker.network.bridge.gatway_mode_ipv[46]
is set to "routed", there is no NAT.
When there's no NAT, there's no meaning to the HostPort field in a
port mapping (all the port mapping does is open the container's port),
and the HostIP field is only used to determine the address family.
So, check port bindings, and raise errors if fields are unexpectedly
set when the mapping only applies to a gateway_mode=routed network.
Zero-addresses are allowed, to say the mapping/open-port should be
IPv4-only or IPv6-only, and host ports are not allowed.
A mapping with no host address, so it applies to IPv4 and IPv6 when
the default binding is 0.0.0.0, may include a host port if either
uses NAT. The port number is ignored for the directly-routed family.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Display a PortBinding in a format that's more like the one
used in the CLI, but includes the container IP if known.
Signed-off-by: Rob Murray <rob.murray@docker.com>
It was added so that tests could replace it before it was picked
up and used by a new network's PortMapper, so that tests were isolated
from each other. Now the PortMapper is not used by the bridge driver,
neither is driver's portAllocator.
Instead of replacing the driver.portAllocator in tests, reset the
singleton instance using its ReleaseAll().
Un-export portallocator.NewInstance, now the tests aren't using it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Without this change, if a port mapping did not specify a host address
and the network was IPv6-enabled, the same port would be allocated for
mappings from '0.0.0.0' and '::'. But, if the port mapping was specified
with explicit addresses even, for example:
-p 0.0.0.0:8080-8083:80 -p '[::]:8083-8080:80'
This change looks for port mappings that only differ in the host IP
address, and makes sure it allocates the same port for all of them. If
it can't, it fails with an error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Add bridge driver options...
com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>
com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>
If set to "routed", no NAT or masquerade rules are set up for port
mappings.
When NAT is disabled, the mapping is shown in 'inspect' output with
no host port number. For example, for "-p 80" with NAT disabled for
IPv6 but not IPv4:
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "32768"
},
{
"HostIp": "::",
"HostPort": ""
}
Signed-off-by: Rob Murray <rob.murray@docker.com>
Use the bridge driver's iptables types to set up portmapping related
iptables rules - instead of using iptables.Forward, which is bridge
specific code in the iptables package.
Remove iptables.Forward() and its unit test, the bridge driver's
version is covered by TestAddPortMappings.
Remove hairpinMode from iptables.ChainInfo hairpinMode relates to bridge
driver specific behaviour, that is now implemented in the bridge driver.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The bridge driver now does its own port-mapping, rather than using the
portmapper module (which ran as two completely separate instances, for
IPv4 and IPv6).
When asked for a mapping from any host address (0.0.0.0/0) with a range
of host ports, the same port will be allocated for IPv4 and IPv6, or the
mapping will fail with an error if that's not possible.
The bridge driver now manages its own port mappings. So, remove
linux-specific PortMapper code and make what's left Windows-only.
Also, replace the portmapper.userlandProxy interface with StartProxy().
Signed-off-by: Rob Murray <rob.murray@docker.com>
For a port mapping like '-p 8080-8083:80', when some non-docker process
is using a port in the range, try other ports in the range. And, don't
do that on live-restore.
Because the port mapping may fail on live-restore, leaving no ports
mapped for the endpoint - update the view of mapped ports shown in
'inspect' output. (The wrong mappings will still be shown in 'docker ps',
the container will be left running and connected to the network, it just
won't work. There's plenty of scope for better error handling here.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Similar to portallocator.RequestPortInRange(), but it attempts to
allocate the same port for multiple IP addresses.
Signed-off-by: Rob Murray <rob.murray@docker.com>
It's an alias for the OCI-spec type, which was only there for
convenience, but will be deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
They might still change to "Mounted from" or "Already exists" when
containerd updates the status in tracker.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The target variable was already overwritten with the new value. Use the
original value instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `Config` field returned by this endpoint (used for "image inspect") returns
additional fields that are not part of the image's configuration and not part of
the [Docker Image Spec] and the [OCI Image Spec].
These additional fields are included in the response, due to an
implementation detail, where the [api/types.ImageInspec] type used
for the response is using the [container.Config] type.
The [container.Config] type is a superset of the image config, and while the
image's Config is used as a _template_ for containers created from the image,
the additional fields are set at runtime (from options passed when creating
the container) and not taken from the image Config.
These fields are never set (and always return the default value for the type),
but are not omitted in the response when left empty. As these fields were not
intended to be part of the image configuration response, they are deprecated,
and will be removed from the API.
The following fields are currently included in the API response, but
are not part of the underlying image's Config, and deprecated:
- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)
[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
[OCI Image Spec]: https://github.com/opencontainers/image-spec/blob/v1.1.0/specs-go/v1/config.go#L24-L62
[api/types.ImageInspec]: https://github.com/moby/moby/blob/v26.1.4/api/types/types.go#L87-L104
[container.Config]: https://github.com/moby/moby/blob/v26.1.4/api/types/container/config.go#L47-L82
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Image.Config field currently reuses the ContainerConfig definition,
matching the Go implementation, which also uses that type.
However, the ContainerConfig type contains various fields that are not
part of the image config, and would never be set. The Image.Config is
used as template / default values for containers started from the image,
but will only use the fields that are part of the [Docker image spec].
This patch updates the swagger files used in the documentation to use a
separate `ImageConfig` definition for the Image.Config field. The new
definition is a copy of the existing `ContainerConfig` type, but with
updated descriptions for fields, and with an example response that omits
the fields that should not be used.
The following fields are currently included in the `Config` field of the API
response, but are not part of the underlying image's config:
- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)
[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Image.Config field currently reuses the ContainerConfig definition,
matching the Go implementation, which also uses that type.
However, the ContainerConfig type contains various fields that are not
part of the image config, and would never be set. The Image.Config is
used as template / default values for containers started from the image,
but will only use the fields that are part of the [Docker image spec].
This patch updates the swagger files used in the documentation to use a
separate `ImageConfig` definition for the Image.Config field. The new
definition is a copy of the existing `ContainerConfig` type, but with
updated descriptions for fields, and with an example response that omits
the fields that should not be used.
The following fields are currently included in the `Config` field of the API
response, but are not part of the underlying image's config:
- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)
[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add a OCI platform fields as parameters to the `POST /images/{id}/push`
that allow to specify a specific-platform manifest to be pushed instead
of the whole image index.
When no platform was requested and pushing whole index failed, fallback
to pushing a platform-specific manifest with a best candidate (if it's
possible to choose one).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This moves the type, but we should consider removing this type, and just
returning an io.ReadCloser
This type was added in 9fd2c0feb0c131d01d727d50baa7183b976c7bdc;
> Make docker load to output json when the response content type is json
> Swarm hijacks the response from docker load and returns JSON rather
> than plain text like the Engine does. This makes the API library to return
> information to figure that out.
However the "load" endpoint unconditionally returns JSON;
7b9d2ef6e5/api/server/router/image/image_routes.go (L248-L255)
Commit 96d7db665b made the response-type depend
on whether "quiet" was set, but this logic got changed in a follow-up
2f27632cde, which made the JSON response-type
unconditionally, but the output produced depend on whether"quiet" was set.
We should deprecated the "quiet" option, as it's really a client
responsibility.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Note that RequestPrivilegeFunc could not be referenced, as it would
introduce a circular import, so copying the definition instead.
Also combining the other search-related types in the package to be in
the same file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the response type; other types related to stats are left
for now, but should be moved (as well as utilities ported from
the CLI repository).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the type to api/types/container and creates an alias for
exec attach; ContainerExecAttach currently uses the same type as
ContainerExecStart, but does not all the same options (and some
options cannot be used).
We need to split the actual types, but lets start with aliasing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Container and ContainerConfig fields have been deprecated, and removed
since API v1.45 in commit 03cddc62f4.
This patch fixes the swagger and documentation to no longer mention them
as they are no longer returned by API v1.45 and higher.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was added in 428328908dc529b1678fb3d8b033fb0591a294e3;
> Deprecate /containers/(id or name)/copy endpoint
> This endpoint has been deprecated since 1.8. Return an error starting
> from this API version (1.24) in order to make sure it's not used for the
> next API version and so that we can remove it sometimes later.
We deprecated and removed those older API versions, and the test was
effectively only verifying that a non-existing endpoint returns a 404,
so let's remove it.
This also removes api/types.CopyConfig, which was only used in this
test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove intermediate variable
- format a "todo" comment as an actual todo ':)
- explicitly suppress some unhandled errors to keep linters happy
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove deprecated UnsafeSlice and use unsafe.Slice
- Stabilize the behaviour of Prev when the cursor already points to
the first element
- Fix Cursor.Prev() out of range issues in v1.3.9
- Relates to boltdb/bolt/issues/357 (Cursor inconsistent when mixing
cursor.Delete() with Put() in same transaction)
- Bump go version to 1.21.9
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.9...v1.3.10
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Touch-up some GoDoc in the package, and remove "import" comments.
This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. The "import" comments are ignored when
used in go module mode so have little benefit. Let's remove them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. While it's not too problematic to have
this dependency, the tests only used basic assertions from gotest.tools,
which could be easily re-implemented without the dependency.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Windows, Darwin, and FreeBSD implementations were identical, other
than their GoDoc to be different. Unify them so that we don't have to
maintain separate GoDoc for each.
It's worth noting that FreeBSD also supports Pdeathsig, so could be
using the same implementation as Linux. However, we don't test/maintain
the FreeBSD implementation, and it would require updating to GoDoc to
be more specific about the use of `/proc/self/exe`, so keeping the
status quo for now.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This combines the implementations of the Self function, to allow having
a single GoDoc to document the behavior. The naiveSelf function is kept,
because it's used in unit-tests.
There is a minor change in behavior, as this patch removes the stub for
unsupported platforms (non-linux, windows, freebsd or darwin), which will
now use `os.Args[0]`. The stub was added in 21537b818d
to fix compilation of https://github.com/ethereum/go-ethereum on OpenBSD,
which had docker/docker as dependency. It looks like that repository no
longer has this dependency, and as this was only to make the code
compilable, is unlikely to be a problem.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 069fdc8a08 changed most uses of
the syscall package to switch utsname from unsigned to signed (see
069fdc8a08). Those don't seem to be
impacting the code used here, so either stdlib or golang.org/x/sys/unix
should work for this case.
I chose stdlib's syscall package for this case, in case we'd decide to
move this package to a separate module (and want to limit its dependencies).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function largely identical to GetFSMagic, except for suppressing
ENOENT errors. The only consumer of this function was fsChecker.IsMounted,
which would ignore errors either way, and only use the "success" case to
check if the detected filesystem-type was the expected one.
This patch;
- rewrites fsChecker.IsMounted to use GetFSMagic instead
- removes the now unused Mounted function
As we consider daemon/graphdriver to be "internal", and as there are no
public consumers of this, we can remove this function without deprecating
first.
The freebsd implementation also seemed to be broken, as it mixed syscall
with golang.org/x/sys/unix, which used incompatible types. I left the file
in place for now, but we can consider removing it altogether as there's no
active development on making freebsd functional.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before "ip6tables" was enabled by default, dockerd would start normally
when:
- the kernel had no IPv6 support, or
- docker is running as docker-in-docker, and the host doesn't have kernel
module 'ip6_tables' loaded.
Now, the bridge driver will try to set up its ip6tables chains and it'll
fail. By not treating that as an error, the daemon will start and IPv4
will work normally.
A subsequent attempt to create an IPv6 network will fail with an error
about ip6tables. At that point, the user's options are:
- set "ip6tables":false in daemon config
- in the DinD case, "modprobe ip6_tables" on the host, or start dockerd
on the host with ip6tables enabled (causing the kernel module load).
Signed-off-by: Rob Murray <rob.murray@docker.com>
Rob is currently a curator, and has been actively contributing to this
repo for 7 months now.
Beside day-to-day triaging and bug fixing, Rob is an instrumental
contributor to libnetwork, and amongst other things, to the ongoing work
on IPv6 improvements.
I nominated Rob as maintainer, and votes passed, so opening a PR to make
it official.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The CheckDuplicate option is no longer part of the current API; it's
only used by the client when connecting to old API versions, which need
to have this field set.
This patch:
- Removes the CheckDuplicate from the API documentation, as the API
describes the current version of the API (which does not have this
field).
- Moves the CheckDuplicate field to the CreateRequest type; this is
the type used for the network create request. The CheckDuplicate
is not an option that's set by the user, and set internally by
the client, so removing it from the CreateOptions struct moves
it entirely internal.
- Change the CheckDuplicate field to be a pointer; this makes the
"omitempty" become active, and the client will no longer include
the field in the request JSON unless it's set (API < 1.44).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
b5c50afa882e2b34aba880fd5028615e2ef94e07 changed the signature of
NewGatewayFrontend to include a slice of allowed repositories.
Docker does not allow to specify this option, so don't place any
restrictions by passing an empty slice.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
1b1c5bc08ad81add007eb647e66ed0929693f3a0 extended the function signature
with one additional return value.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
eea0b41bf4fb1d69e109ff5ff8045c63f0c0d510 added a new argument to
`instructions.Parse` to support issuing linter warnings.
Classic builder uses it to parse the Dockerfile instructions and its
usage needs adjustment.
The classic builder is deprecated and we won't be adding any new
features to it, so we just pass a nil linter callback.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't fall-back to Google's DNS servers in a network that has an
internal resolver.
Now the default bridge uses the internal resolver, the only reason a
network started by the daemon should end up without any upstream
servers is if the host's resolv.conf doesn't list any. In this case,
the '--dns' option can be used to explicitly configure nameservers
for a container if necessary.
(Note that buildkit's containers do not have an internal resolver, so
they will still set up Google's nameservers if the host has no
resolvers that can be used in the container's namespace.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Until now, containers on the default bridge network have been configured
to talk directly to external DNS servers - their resolv.conf files have
either been populated with nameservers from the host's resolv.conf, or
with servers from '--dns' (or with Google's nameservers as a fallback).
This change makes the internal bridge more like other networks by using
the internal resolver. But, the internal resolver is not populated with
container names or aliases - it's only for external DNS lookups.
Containers on the default network, on a host that has a loopback
resolver (like systemd's on 127.0.0.53) will now use that resolver
via the internal resolver. So, the logic used to find systemd's current
set of resolvers is no longer needed by the daemon.
Legacy links work just as they did before, using '/etc/hosts' and magic.
(Buildkit does not use libnetwork, so it can't use the internal resolver.
But it does use libnetwork/resolvconf's logic to configure resolv.conf.
So, code to set up resolv.conf for a legacy networking without an internal
resolver can't be removed yet.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Update the containerd binary that's used in CI and for the static packages.
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.18
- full diff: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
Welcome to the v1.7.18 release of containerd!
The eighteenth patch release for containerd 1.7 contains various updates along
with an updated version of Go. Go 1.22.4 and 1.21.11 include a fix for a symlink
time of check to time of use race condition during directory removal.
Highlights
- Update Go version to 1.21.11
- Remove uses of platforms.Platform alias
- Migrate log imports to github.com/containerd/log
- Migrate errdefs package to github.com/containerd/errdefs
- Fix usage of "unknown" platform
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.
This patch:
- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to prevent using the "log" package, which was also
migrated to a separate module.
There are still some uses of the old package in (indirect) dependencies,
which should go away over time.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.21.11 (released 2024-06-04) includes security fixes to the archive/zip
and net/netip packages, as well as bug fixes to the compiler, the go command,
the runtime, and the os package. See the Go 1.21.11 milestone on our issue
tracker for details;
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.11+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.10...go1.21.11
From the security announcement;
We have just released Go versions 1.22.4 and 1.21.11, minor point releases.
These minor releases include 2 security fixes following the security policy:
- archive/zip: mishandling of corrupt central directory record
The archive/zip package's handling of certain types of invalid zip files
differed from the behavior of most zip implementations. This misalignment
could be exploited to create an zip file with contents that vary depending
on the implementation reading the file. The archive/zip package now rejects
files containing these errors.
Thanks to Yufan You for reporting this issue.
This is CVE-2024-24789 and Go issue https://go.dev/issue/66869.
- net/netip: unexpected behavior from Is methods for IPv4-mapped IPv6 addresses
The various Is methods (IsPrivate, IsLoopback, etc) did not work as expected
for IPv4-mapped IPv6 addresses, returning false for addresses which would
return true in their traditional IPv4 forms.
Thanks to Enze Wang of Alioth and Jianjun Chen of Zhongguancun Lab
for reporting this issue.
This is CVE-2024-24790 and Go issue https://go.dev/issue/67680.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, starting dockerd with
`--default-network-opt=bridge=com.docker.network.enable_ipv6=true` has
no effect as `NetworkCreateRequest.EnableIPv6` is a basic bool.
This change makes it a `*bool` to make it optional. If clients don't
specify it, the default-network-opt will be applied.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This method was added in 534a90a993 as
part of adding the Swarm cluster backend, and later updated in commit
85b1fdf15c to use a swarmRequestTimeout
const for the timeout.
Nothing in this utility depends on the Cluster struct, and the abstraction
makes it appear as more than it is, which is just a wrapper for
context.WithTimeout().
Let's remove the abstraction to make it less magical.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the type to the api/types/network package, but also introduces
a "Summary" alias; the intent here is to allow diverging the types used
for "list" and "inspect" operations, as list operations may only be
producing a subset of the fields available.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These aliases were added in ac2a028dcc,
which was part of the v26.0 and v26.1 releases. We can remove the
aliases, assuming users that depended on this have migrated to the
new location of these types.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The types.NetworkInspectOptions type was moved to the networks package
in 5bea0c38bc and deprecated, but use of it
was re-introduced in cd3804655a, which was
merged out-of-order.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 1a5dafb31e improved the error messages
produced by adding a check if the client is using as an elevated user. For
this, it attempts to open `\\.\PHYSICALDRIVE0`.
However, it looks like closing the file landed in the wrong branch of the
condition, so the file-handle would not be closed when the os.Open succeeded.
Looking further into this check, it appears the conditions were reversed;
if the check _fails_, it means the user is not running with elevated
permissions, but the check would use elevatedErr == nil.
Fix both by changing the condition to `elevatedErr != nil`.
While at it, also changing the string to use a string-literal, to reduce
the amount of escaping needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make this code slightly more idiomatic, and make it clear in what cases
we don't return an actual response, but an empty / default struct.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove redundant `if statusCode == 0 {` check, which would always be true
- use early returns in the switch
- move all conditions into the switch, and scope the `statusCode` variable
to conditions where it's used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
So far, Moby only had IPv4 prefixes in its 'default-address-pools'. To
get dynamic IPv6 subnet allocations, users had to redefine this
parameter to include IPv6 base network(s). This is needlessly complex
and against Moby's 'batteries-included' principle.
This change generates a ULA base network by deriving a ULA Global ID
from the Engine's Host ID and put that base network into
'default-address-pools'. This Host ID is stable over time (except if
users remove their '/var/lib/docker/engine-id') and thus the GID is
stable too.
This ULA base network won't be put into 'default-address-pools' if users
have manually configured it.
This is loosely based on https://datatracker.ietf.org/doc/html/rfc4193#section-3.2.2.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Until this commit, the default local address pool was initialized by the
defaultipam driver if none was provided by libnet / the daemon.
Now, defaultipam errors out if none is passed and instead the daemon is
made responsible for initializing it with the default values if the user
don'te set the related config parameter.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Until now it's been possible to set per-interface sysctls using, for
example, '--sysctl net.ipv6.conf.eth0.accept_ra=2'. But, the index in
the interface name is allocated serially, and the numbering in a container
with more than one interface may change when a container is restarted.
The change to make it possible to connect a container to more than one
network when it's created increased the ambiguity.
This change adds label "com.docker.network.endpoint.sysctls" to the
DriverOpts in EndpointSettings. This option is explicitly associated
with the interface.
Settings in "--sysctl" for "eth0" are migrated to DriverOpts.
Because using "--sysctl" with any interface apart from "eth0" would have
unpredictable results, it is now an error to use any other interface name
in the top level "--sysctl" option. The error message includes a hint at
how to use the new per-interface setting.
The per-endpoint sysctl name has the interface name replaced by
"IFNAME". For example:
net.ipv6.conf.eth0.accept_ra=2
becomes:
net.ipv6.conf.IFNAME.accept_ra=2
The value of DriverOpts["com.docker.network.endpoint.sysctls"] is a
comma separated list.
Settings from '--sysctl' are applied by the runtime lib during task
creation. So, task creation fails if the endpoint does not exist.
Applying per-endpoint settings during interface configuration means the
endpoint can be created later, which paves the way for removal of the
SetKey OCI prestart hook.
Unlike other DriverOpts, the sysctl label itself is not driver-specific,
but each driver has a chance to check settings/values and raise an error
if a setting would cause it a problem - no such checks have been added
in this initial version. As a future extension, if required, it would be
possible for the driver to echo back valid/extended/modified settings to
libnetwork for it to apply to the interface. (At that point, the syntax
for the options could become driver specific to allow, for example, a
driver to create more than one interface).
Signed-off-by: Rob Murray <rob.murray@docker.com>
- api: swagger: Network: inline examples, and add ConfigOnly, ConfigFrom
These fields were added in 9ee7b4dda9, but
not documented in the API docs / swagger.
Also move the example values per-field to reduce the risk of the example
given from diverging with the actual struct that's used for the request.
- api: swagger: POST /networks/create: document Scope, ConfigOnly, ConfigFrom
Adds missing documentation for Scope, ConfigOnly, and ConfigFrom. The ConfigOnly
and ConfigFrom fields were added in 9ee7b4dda9,
but not documented in the API docs / swagger.
- api: swagger: Network: add Peers
Add documentation for the Peers field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adds missing documentation for Scope, ConfigOnly, and ConfigFrom. The ConfigOnly
and ConfigFrom fields were added in 9ee7b4dda9,
but not documented in the API docs / swagger.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These fields were added in 9ee7b4dda9, but
not documented in the API docs / swagger.
Also move the example values per-field to reduce the risk of the example
given from diverging with the actual struct that's used for the request.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the example values per-field to reduce the risk of the example given
from diverging with the actual struct that's used for the request.
This patch updates older API versions (went back to v1.41).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the example values per-field to reduce the risk of the example given
from diverging with the actual struct that's used for the request.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
GoDoc is mostly copied from NetworkResource, which is the equivalent for
retrieving the information.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We currently depend on the containerd platform-parsing to return typed
errdefs errors; the new containerd platforms module does not return such
errors, and documents that errors returned should not be used as sentinel
errors; c1438e911a/errors.go (L21-L30)
Let's type these errors ourselves, so that we don't depend on the error-types
returned by containerd, and consider that eny platform string that results in
an error is an invalid parameter.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove redundant conversion to strslice.StrSlice
- Use assert.Check where possible to not fail early
- Remove instances of types.ExecStartCheck that used default values
- Minor code-formatting cleanup
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove redundant conversion to strslice.StrSlice
- Use assert.Assert instead of assert.Check to fail early if value is nil
- Minor code-formatting cleanup
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- fix typo in comment
- rename variable that collided with an import
- add log for an unhandled error
- slightly improve error-logs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in ee6959addc to account
for arm (32) requiring a longer timeout at the time, but it was never
used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- rename the client var to not collide with the imported client package
- remove an intermediate startCheck variable
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This var for the incoming attach request.
Just within this one function we also have `cfg`, and `ctr` already, so
`c` just makes things more confusing.
Not to mention `c` is usually referencing a container object in other
parts of the code.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In cases where the client disconnects and there is nothing to read from
a stdio stream after that disconnect, the copy goroutines and file
descriptors are leaked because `io.Copy` is just blocked waiting for
data from the container's I/O stream.
This fix only applies to Linux.
Windows will need a separate fix.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Nothing was validating whether address pools' `base` prefix
were larger than the target subnet `size` they're associated to. As
such invalid address pools would yield no subnet, the error could go
unnoticed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The previous allocator was subnetting address pools eagerly
when the daemon started, and would then just iterate over that
list whenever RequestPool was called. This was leading to high
memory usage whenever IPv6 pools were configured with a target
subnet size too different from the pools prefix size.
For instance: pool = fd00::/8, target size = /64 -- 2 ^ (64-8)
subnets would be generated upfront. This would take approx.
9 * 10^18 bits -- way too much for any human computer in 2024.
Another noteworthy issue, the previous implementation was allocating
a subnet, and then in another layer was checking whether the
allocation was conflicting with some 'reserved networks'. If so,
the allocation would be retried, etc... To make it worse, 'reserved
networks' would be recomputed on every iteration. This is totally
ineffective as there could be 'reserved networks' that fully overlap
a given address pool (or many!).
To fix this issue, a new field `Exclude` is added to `RequestPool`.
It's up to each driver to take it into account. Since we don't know
whether this retry loop is useful for some remote IPAM driver, it's
reimplemented bug-for-bug directly in the remote driver.
The new allocator uses a linear-search algorithm. It takes advantage
of all lists (predefined pools, allocated subnets and reserved
networks) being sorted and logically combines 'allocated' and
'reserved' through a 'double cursor' to iterate on both lists at the
same time while preserving the total order. At the same time, it
iterates over 'predefined' pools and looks for the first empty space
that would be a good fit.
Currently, the size of the allocated subnet is still dictated by
each 'predefined' pools. We should consider hardcoding that size
instead, and let users specify what subnet size they want. This
wasn't possible before as the subnets were generated upfront. This
new allocator should be able to deal with this easily.
The method used for static allocation has been updated to make sure
the ascending order of 'allocated' is preserved. It's bug-for-bug
compatible with the previous implementation.
One consequence of this new algorithm is that we don't keep track
of where the last allocation happened, we just allocate the first
free subnet we find.
Before:
- Allocate: 10.0.1.0/24, 10.0.2.0/24 ; Deallocate: 10.0.1.0/24 ;
Allocate 10.0.3.0/24.
Now, the 3rd allocation would yield 10.0.1.0/24 once again.
As it doesn't change the semantics of the allocator, there's no
reason to worry about that.
Finally, about 'reserved networks'. The heuristics we use are
now properly documented. It was discovered that we don't check
routes for IPv6 allocations -- this can't be changed because
there's no such thing as on-link routes for IPv6.
(Kudos to Rob Murray for coming up with the linear-search idea.)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
If a node is promoted right after another node is demoted, there exists
the possibility of a race, by which the newly promoted manager attempts
to connect to the newly demoted manager for its initial Raft membership.
This connection fails, and the whole swarm Node object exits.
At this point, the daemon nodeRunner sees the exit and restarts the
Node.
However, if the address of the no-longer-manager is recorded in the
nodeRunner's config.joinAddr, the Node again attempts to connect to the
no-longer-manager, and crashes again. This repeats. The solution is to
remove the node entirely and rejoin the Swarm as a new node.
This change erases config.joinAddr from the restart of the nodeRunner,
if the node has previously become Ready. The node becoming Ready
indicates that at some point, it did successfully join the cluster, in
some fashion. If it has successfully joined the cluster, then Swarm has
its own persistent record of known manager addresses. If no joinAddr is
provided, then Swarm will choose from its persisted list of managers to
join, and will join a functioning manager.
Signed-off-by: Drew Erny <derny@mirantis.com>
InvalidParameterErrorf was used whenever an invalid value was found
during PoolID unmarshaling. This error is converted to a 400 HTTP code
by the HTTP server.
However, users never provide PoolIDs directly -- these are constructed
from user-supplied values which are already validated when the PoolID is
marshaled. Hence, if such erroneous value is found, it's an internal
error and should be converted to a 500.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Prior to this change PoolID microformat was using slashes to separate
fields. Those fields include subnet prefixes in CIDR notation, which
also include a slash. This makes future evolution harder than it should
be.
This change introduces a 'v2' microformat based on JSON. This has two
advantages:
1. Fields are clearly named to ensure each value is associated to the
right field.
2. Field values and separators are clearly distinguished to remove any
ambiguity.
The 'v1' encoding will be kept until the next major MCR LTS is released.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This method does nothing more than `return c.store`. It has no value and
adds an unecessary level of indirection. Let's ditch it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Previous commit made it clear that c.store can't be nil. Hence,
`c.store.Close()` can be called without checking if c.store is nil.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This was done in a separate method, called by the ctrler constructor.
This method was returning a nil datastore when c.cfg was nil -- but that
can't happen in practice!
This was giving the impression that the controller could be run without
a datastore properly configured. It's not the case, so make it explicit
by instantiating the datastore before `Controller`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This works around issues with the otel http handler wrapper causing
multiple calls to `WriteHeader` when a `Flush` is called before `Write`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When only stdin is attached the goroutine can only ever exit if:
1. The container pipe is closed while trying to write to it
2. The client closes the stdin read pipe
This is because `io.Copy` does a read on the read side then a write to
the write side.
If reading from the client's stdin pipe blocks, the goroutine will never
get notified that the container pipe is closed.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Pass `context.Context` through `tarexport.Load` and `tarexport.Save`.
Create OTEL spans for the most time consuming operations.
Also, handle context cancellations to actually end saving/loading when
the operation is cancelled - before this PR the daemon would still be
performing the operation even though the user already cancelled it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
In 26.1, we added daemon feature flag "windows-dns-proxy" which could
be set to "true" to make "nslookup" work in Windows containers, by
forwarding requests from the internal resolver to the container's
external DNS servers.
This changes the default to forwarding-enabled - it can be disabled by
via daemon.json using ...
"features": { "windows-dns-proxy": false }
Signed-off-by: Rob Murray <rob.murray@docker.com>
A recent change to prevent containers only connected to --internal
networks from communicating with external DNS servers inadvertently
prevented the daemon's internal DNS server from forwarding requests
within an internal network to a containerised DNS server.
Relax the check, so that only requests that need to be forwarded
from the host's network namespace are dropped.
External DNS servers remain unreachable from the internal network.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Libnet's method `(*Network).createEndpoint()` is already parsing this
netlabel to set the field `ep.iface.mac`. Later on, this same method
invoke the driver's method `CreateEndpoint` with an `InterfaceInfo` arg
and an `options` arg (an opaque map of driver otps).
The `InterfaceInfo` interface contains a `MacAddress()` method that
returns `ep.iface.mac`. And the opaque map may contain the key
`netlabel.MacAddress`.
Prior to this change, the bridge driver was calling `MacAddress()`. If
no value was returned, it'd fall back to the option set in the `options`
map, or generate a MAC address based on the IP address.
However, the expected type of the `options` value is a `net.HardwareAddr`.
This is what's set by the daemon when handing over the endpoint config
to libnet controller. If the value is a string, as is the case if the
MAC address is provided through `EndpointsSettings.DriverOpts`, it
produces an error.
As such, the opaque option and the `MacAddress()` are necessarily the
same -- either nothing or a `net.HardwareAddr`. No need to keep both.
Moreover, the struct `endpointConfiguration` was only used to store that
netlabel value. Drop it too.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
In setupIPv6BridgeNetFiltering(), the bridge should always be named.
Don't fall back to checking the "default" setting for a new bridge.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Check forwarding, then set bridge-nf-call-ip6tables, on a bridge
if IPv6 is enabled - even if no IPv6 address has been assigned.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The code to enable "bridge-nf-call-iptables" or "bridge-nf-call-ip6tables"
was gated on "--iptables=true", it didn't check "--ip6tables=true".
So, split the top level call into IPv4/IPv6 so that the iptables-enable
settings can be checked independently, and simplfied the implementation.
Signed-off-by: Rob Murray <rob.murray@docker.com>
bridgeNetwork.isolateNetwork() checks "--iptables=true" and
"--ip6tables=true" before doing anything with IPv4 and IPv6
respectively. But, it was only called if "--iptables=true".
Now, it's called if "--ip6tables=true", even if "--iptables=false".
Signed-off-by: Rob Murray <rob.murray@docker.com>
The bridge driver's setupIPChains() had an initial sanity check that
"--iptables=true".
But, it's called with "version=IPv6" when "--iptables=false" and
"--ip6tables=true" - the sanity test needed to allow for that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Tests that start a daemon disable iptables, to avoid conflicts with
other tests running in parallel and also creating iptables chains.
Do the same for ip6tables, in prep for them being enabled by-default.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Use defer funcs instead.
For no apparant reasons, a few error cases in the Join method were not
triggering a rollback. This is now fixed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Since commit befff0e1, `(*Controller).getStore()` never returns nil
except if `c.store` isn't initialized yet. This can't happen unless
`New()` returned an error and it wasn't proper caught.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
CORS headers were originally added by 6d5bdff.
These headers could be set without any Authz plugin enabled
beforehand, making this feature quite dangerous.
This commit marks the daemon flag `api-cors-header` as deprecated
and requires the env var `DOCKERD_DEPRECATED_CORS_HEADER` to be
set. When enabled, the daemon will write a deprecation warning to
the logs and the endpoint `GET /info` will return the same
deprecation warning.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.9...go1.21.10
These minor releases include 2 security fixes following the security policy:
- cmd/go: arbitrary code execution during build on darwin
On Darwin, building a Go module which contains CGO can trigger arbitrary code execution when using the Apple version of ld, due to usage of the -lto_library flag in a "#cgo LDFLAGS" directive.
Thanks to Juho Forsén of Mattermost for reporting this issue.
This is CVE-2024-24787 and Go issue https://go.dev/issue/67119.
- net: malformed DNS message can cause infinite loop
A malformed DNS message in response to a query can cause the Lookup functions to get stuck in an infinite loop.
Thanks to long-name-let-people-remember-you on GitHub for reporting this issue, and to Mateusz Poliwczak for bringing the issue to our attention.
This is CVE-2024-24788 and Go issue https://go.dev/issue/66754.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.3
**- Description for the changelog**
```markdown changelog
Update Go runtime to 1.21.10
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This was changed recently so that the bundles target is always run, but
`mkdir bundles` fails when bundles exists...
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
It'd only return true on a host with no IPv6 in its kernel.
So, removed, having fixed the two tests that used it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.
TestDaemonIPv6Enabled tried to run with IPv6 on the default bridge,
but didn't set up a "fixed-cidr-v6" - so the daemon wouldn't start.
It then tried to check the bridge had address "fe80::1", which it
expected to work because it had just used setupV6() to add that
address.
Then it checked that "LinkLocalIPv6Address" was set in container
inspect output, but it wouldn't be (the field is deprecated).
There are working IPv6 tests in the suite (TestDaemonIPv6FixedCIDR,
TestDaemonIPv6FixedCIDRAndMac, TestDaemonIPv6HostMode) - and there's
more coverage in the network integration tests.
So, deleted the test as it didn't seem worth salvaging.
Also deleted now-unused helper functions setupV6(), teardownV6().
Signed-off-by: Rob Murray <rob.murray@docker.com>
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.
If the test had run, it'd have failed because:
- it used "--listen-add", but the option is "--listen-addr"
- so, the daemon wouldn't have started
- it tried to use "--join ::1"
- address "::1" was interpreted as host:port so the Dial() failed,
it needed to be "[::1]".
- it didn't supply a join token
Signed-off-by: Rob Murray <rob.murray@docker.com>
Multicast addresses aren't added by the daemon so, if they're present,
it's because they were explicitly added - possibly to a user-managed
bridge. So, don't remove.
Signed-off-by: Rob Murray <rob.murray@docker.com>
- Remove package variable bridge.bridgeIPv6
- Use netip in more places
- Improve error messages from fixed-cidr-v6 checks
Signed-off-by: Rob Murray <rob.murray@docker.com>
Make the behaviour enabled by env var DOCKER_BRIDGE_PRESERVE_KERNEL_LL
the default...
- don't remove kernel assigned link-local addresses
- or any address in fe80::/64
- don't assign fe80::1 to a bridge
Signed-off-by: Rob Murray <rob.murray@docker.com>
If dockerd runs on a host with a read-only /proc/sys/net filesystem,
it isn't able to enable or disable IPv6 on network interfaces when
attaching a container to a network (including initial networks during
container creation).
In release 26.0.2, a read-only /proc/sys/net meant container creation
failed in all cases.
So, don't attempt to enable/disable IPv6 on an interface if it's already
set appropriately.
If it's not possible to enable IPv6 when it's needed, just log (because
that's what libnetwork has always done if IPv6 is disabled in the
kernel).
If it's not possible to disable IPv6 when it needs to be disabled,
refuse to create the container and raise an error that suggests setting
environment variable "DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE=1", to tell
the daemon it's ok to ignore the problem.
Signed-off-by: Rob Murray <rob.murray@docker.com>
If env var DOCKER_BRIDGE_PRESERVE_KERNEL_LL=1, don't assign fe80::1/64
to a bridge, and don't delete any link local address with prefix fe80::/64.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Prior to this change, cnmallocator would call
`ConfigGlobalScopeDefaultNetworks` right before initializing its
IPAM drivers. This function was mutating some global state used
during drivers init.
This change just remove the global state, and adds an arg to
ipams.Register and defaultipam.Register to pass the global pools
by arguments instead.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
All drivers except the default have a Register function. Before this
change, default's registration was handled by another package. Move
this logic into the driver pkg.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Packages in libnet/ipams are drivers, except builtin -- it's used
to register drivers. Move files one level up and delete this pkg.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
All drivers except the default ipam driver are stored in ipams/.
Since `default` isn't a valid Go pkg name, this package is
renamed to `defaultipam`, following `windowsipam` example.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Prior to this change, daemon's `default-address-pools` param would
be passed to `SetDefaultIPAddressPool()` to set a global var named
`defaultAddressPool`. This var would then be retrieved during the
`default` IPAM driver registration. Both steps were executed in
close succession during libnet's controller initialization.
This change removes the global var and just pass the user-defined
`default-address-pools` to the `default` driver's `Register` fn.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The `RequestPool` method has many args and named returns. This
makes the code hard to follow at times. This commit adds one struct,
`PoolRequest`, to replace these args, and one struct, `AllocatedPool`,
to replace these named returns.
Both structs' fields are properly documented to better define their
semantics, and their relationship with address allocation.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Address spaces are a continuum of addresses that can be used for a
specific purpose (ie. 'local' for unmanaged containers, 'global for
Swarm). v4 and v6 addresses aren't of the same size -- hence
combining them into a single address space doesn't form a continuum.
Better set them apart into two different address spaces.
Also, the upcoming rewrite of `addrSpace` will benefit from that
split.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
`addrSpace` methods are currently scattered in two different files.
As upcoming work will rewrite some of these methods, better put them
into a separate file.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Embedding `sync.Mutex` into a struct is considered a bad practice
as it makes the mutex methods part of the embedding struct's API.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Allow forwarding from any firewalld zone to the 'docker' zone.
This makes it possible to use routable IPv6 addresses on a bridge
network, with masquerading disabled, and have the host forward packets
to it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
/usr/sbin/runc is confined with "runc" profile[1] introduced in AppArmor
v4.0.0. This change breaks stopping of containers, because the profile
assigned to containers doesn't accept signals from the "runc" peer.
AppArmor >= v4.0.0 is currently part of Ubuntu Mantic (23.10) and later.
In the case of Docker, this regression is hidden by the fact that
dockerd itself sends SIGKILL to the running container after runc fails
to stop it. It is still a regression, because graceful shutdowns of
containers via "docker stop" are no longer possible, as SIGTERM from
runc is not delivered to them. This can be seen in logs from dockerd
when run with debug logging enabled and also from tracing signals with
killsnoop utility from bcc[2] (in bpfcc-tools package in Debian/Ubuntu):
Test commands:
root@cloudimg:~# docker run -d --name test redis
ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
root@cloudimg:~# docker stop test
Relevant syslog messages (with wrapped long lines):
Apr 23 20:45:26 cloudimg kernel: audit:
type=1400 audit(1713905126.444:253): apparmor="DENIED"
operation="signal" class="signal" profile="docker-default" pid=9289
comm="runc" requested_mask="receive" denied_mask="receive"
signal=kill peer="runc"
Apr 23 20:45:36 cloudimg dockerd[9030]:
time="2024-04-23T20:45:36.447016467Z"
level=warning msg="Container failed to exit within 10s of kill - trying direct SIGKILL"
container=ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
error="context deadline exceeded"
Killsnoop output after "docker stop ...":
root@cloudimg:~# killsnoop-bpfcc
TIME PID COMM SIG TPID RESULT
20:51:00 9631 runc 3 9581 -13
20:51:02 9637 runc 9 9581 -13
20:51:12 9030 dockerd 9 9581 0
This change extends the docker-default profile with rules that allow
receiving signals from processes that run confined with either runc or
crun profile (crun[4] is an alternative OCI runtime that's also confined
in AppArmor >= v4.0.0, see [1]). It is backward compatible because the
peer value is a regular expression (AARE) so the referenced profile
doesn't have to exist for this profile to successfully compile and load.
Note that the runc profile has an attachment to /usr/sbin/runc. This is
the path where the runc package in Debian/Ubuntu puts the binary. When
the docker-ce package is installed from the upstream repository[3], runc
is installed as part of the containerd.io package at /usr/bin/runc.
Therefore it's still running unconfined and has no issues sending
signals to containers.
[1] https://gitlab.com/apparmor/apparmor/-/commit/2594d936
[2] https://github.com/iovisor/bcc/blob/master/tools/killsnoop.py
[3] https://download.docker.com/linux/ubuntu
[4] https://github.com/containers/crun
Signed-off-by: Tomáš Virtus <nechtom@gmail.com>
If a container is configured with the internal DNS resolver's own
address as an external server, try the next ext server rather than
recursing (return SERVFAIL if there are no other servers).
Signed-off-by: Rob Murray <rob.murray@docker.com>
no changes in vendored codee
full diff: https://github.com/golang/mod/compare/v0.13.0...v0.17.0
- modfile: do not collapse if there are unattached comments within blocks
- modfile: fix crash on AddGoStmt in empty File
- modfile: improve directory path detection and error text consistency
- modfile: use new go version string format in WorkFile.add error
- sumdb: replace globsMatchPath with module.MatchPrefixPatterns
- sumdb/tlog: make NewTiles only generate strictly necessary tiles
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't fail-fast when encountering an image that targets an unexpected
descriptor (neither a manifest nor index). Log a warning instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This error is returned when attempting to walk a descriptor that
*should* be an index or a manifest.
Without this the error is not very helpful sicne there's no way to tell
what triggered it.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Don't mutate the container's `Config.WorkingDir` permanently with a
cleaned path when creating a working directory.
Move the `filepath.Clean` to the `translateWorkingDir` instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `normalizeWorkdir` function has two branches, one that returns a
result of `filepath.Join` which always returns a cleaned path, and
another one where the input string is returned unmodified.
To make these two outputs consistent, also clean the path in the second
branch.
This also makes the cleaning of the container workdir explicit in the
`normalizeWorkdir` function instead of relying on the
`SetupWorkingDirectory` to mutate it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Make the internal DNS resolver for Windows containers forward requests
to upsteam DNS servers when it cannot respond itself, rather than
returning SERVFAIL.
Windows containers are normally configured with the internal resolver
first for service discovery (container name lookup), then external
resolvers from '--dns' or the host's networking configuration.
When a tool like ping gets a SERVFAIL from the internal resolver, it
tries the other nameservers. But, nslookup does not, and with this
change it does not need to.
The internal resolver learns external server addresses from the
container's HNSEndpoint configuration, so it will use the same DNS
servers as processes in the container.
The internal resolver for Windows containers listens on the network's
gateway address, and each container may have a different set of external
DNS servers. So, the resolver uses the source address of the DNS request
to select external resolvers.
On Windows, daemon.json feature option 'windows-no-dns-proxy' can be used
to prevent the internal resolver from forwarding requests (restoring the
old behaviour).
Signed-off-by: Rob Murray <rob.murray@docker.com>
- deprecate Prestart hook
- deprecate kernel memory limits
Additions
- config: add idmap and ridmap mount options
- config.md: allow empty mappings for [r]idmap
- features-linux: Expose idmap information
- mount: Allow relative mount destinations on Linux
- features: add potentiallyUnsafeConfigAnnotations
- config: add support for org.opencontainers.image annotations
Minor fixes:
- config: improve bind mount and propagation doc
full diff: https://github.com/opencontainers/runtime-spec/compare/v1.1.0...v1.2.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds some nolint-comments for the deprecated kernel-memory options; we
deprecated these, but they could technically still be accepted by alternative
runtimes.
daemon/daemon_unix.go:108:3: SA1019: memory.Kernel is deprecated: kernel-memory limits are not supported in cgroups v2, and were obsoleted in [kernel v5.4]. This field should no longer be used, as it may be ignored by runtimes. (staticcheck)
memory.Kernel = &config.KernelMemory
^
daemon/update_linux.go:63:3: SA1019: memory.Kernel is deprecated: kernel-memory limits are not supported in cgroups v2, and were obsoleted in [kernel v5.4]. This field should no longer be used, as it may be ignored by runtimes. (staticcheck)
memory.Kernel = &resources.KernelMemory
^
Prestart hooks are deprecated, and more granular hooks should be used instead.
CreateRuntime are the closest equivalent, and executed in the same locations
as Prestart-hooks, but depending on what these hooks do, possibly one of the
other hooks could be used instead (such as CreateContainer or StartContainer).
As these hooks are still supported, this patch adds nolint comments, but adds
some TODOs to consider migrating to something else;
daemon/nvidia_linux.go:86:2: SA1019: s.Hooks.Prestart is deprecated: use [Hooks.CreateRuntime], [Hooks.CreateContainer], and [Hooks.StartContainer] instead, which allow more granular hook control during the create and start phase. (staticcheck)
s.Hooks.Prestart = append(s.Hooks.Prestart, specs.Hook{
^
daemon/oci_linux.go:76:5: SA1019: s.Hooks.Prestart is deprecated: use [Hooks.CreateRuntime], [Hooks.CreateContainer], and [Hooks.StartContainer] instead, which allow more granular hook control during the create and start phase. (staticcheck)
s.Hooks.Prestart = append(s.Hooks.Prestart, specs.Hook{
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
No IPAM IPv6 address is given to an interface in a network with
'--ipv6=false', but the kernel would assign a link-local address and,
in a macvlan/ipvlan network, the interface may get a SLAAC-assigned
address.
So, disable IPv6 on the interface to avoid that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This reverts commit a77e147d32.
The ipvlan integration tests have been skipped in CI because of a check
intended to ensure the kernel has ipvlan support - which failed, but
seems to be unnecessary (probably because kernels have moved on).
Signed-off-by: Rob Murray <rob.murray@docker.com>
We document that an macvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an macvlan
network was still configured with a gateway. So, DNS proxying would
be enabled in the internal resolver (and, if the host's resolver
was on a localhost address, requests to external resolvers from the
host's network namespace would succeed).
This change disables configuration of a gateway for a macvlan Endpoint
if no parent interface is specified.
(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway will still be set up. Documentation
will need to be updated to note that '--internal' should be used to
prevent DNS request forwarding in this case.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
The internal DNS resolver should only forward requests to external
resolvers if the libnetwork.Sandbox served by the resolver has external
network access (so, no forwarding for '--internal' networks).
The test for external network access was whether the Sandbox had an
Endpoint with a gateway configured.
However, an ipvlan-l3 networks with external network access does not
have a gateway, it has a default route bound to an interface.
Also, we document that an ipvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an ipvlan-l2
network was configured with a gateway. So, DNS proxying would be enabled
in the internal resolver (and, if the host's resolver was on a localhost
address, requests to external resolvers from the host's network
namespace would succeed).
So, this change adjusts the test for enabling DNS proxying to include
a check for '--internal' (as a shortcut) and, for non-internal networks,
checks for a default route as well as a gateway. It also disables
configuration of a gateway or a default route for an ipvlan Endpoint if
no parent interface is specified.
(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway/default route will still be set up
and external DNS proxying will be enabled. The network must be
configured as '--internal' to prevent that from happening.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.
These minor releases include 1 security fixes following the security policy:
- http2: close connections when receiving too many headers
Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.
Set a limit on the amount of excess header frames we will process before
closing a connection.
Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.
This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
full diff: https://github.com/golang/net/compare/v0.22.0...v0.23.0
Includes a fix for CVE-2023-45288, which is also addressed in go1.22.2
and go1.21.9;
> http2: close connections when receiving too many headers
>
> Maintaining HPACK state requires that we parse and process
> all HEADERS and CONTINUATION frames on a connection.
> When a request's headers exceed MaxHeaderBytes, we don't
> allocate memory to store the excess headers but we do
> parse them. This permits an attacker to cause an HTTP/2
> endpoint to read arbitrary amounts of data, all associated
> with a request which is going to be rejected.
>
> Set a limit on the amount of excess header frames we
> will process before closing a connection.
>
> Thanks to Bartek Nowotarski for reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diffs changes relevant to vendored code:
- https://github.com/golang/net/compare/v0.18.0...v0.22.0
- websocket: add support for dialing with context
- http2: remove suspicious uint32->v conversion in frame code
- http2: send an error of FLOW_CONTROL_ERROR when exceed the maximum octets
- https://github.com/golang/crypto/compare/v0.17.0...v0.21.0
- internal/poly1305: drop Go 1.12 compatibility
- internal/poly1305: improve sum_ppc64le.s
- ocsp: don't use iota for externally defined constants
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Unlike its stdlib counterparts, AtomicFileWriter does not take into
consideration umask due to its use of chmod. Failure to recognize this
may cause subtle problems like the one described in #47498.
Therefore the documentation has been updated to let users know that
umask is not taken into consideration when using AtomicFileWriter.
Closes#47516.
Signed-off-by: Antonio Aguilar <antonio@zoftko.com>
This was brought up by bmitch that its not expected to have a platform
object in the config descriptor.
Also checked with tianon who agreed, its not _wrong_ but is unexpected
and doesn't neccessarily make sense to have it there.
Also, while technically incorrect, ECR is throwing an error when it sees
this.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This was using `errors.Wrap` when there was no error to wrap, meanwhile
we are supposed to be creating a new error.
Found this while investigating some log corruption issues and
unexpectedly getting a nil reader and a nil error from `getTailReader`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The NetworkMode "default" is now normalized into the value it
aliases ("bridge" on Linux and "nat" on Windows) by the
ContainerCreate endpoint, the legacy image builder, Swarm's
cluster executor and by the container restore codepath.
builder-next is left untouched as it already uses the normalized
value (ie. bridge).
Going forward, this will make maintenance easier as there's one
less NetworkMode to care about.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"
Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.
But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.
This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"
Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.
But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.
This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This is better because every possible platform combination
does not need to be defined in the Dockerfile. If built
for platform where Delve is not supported then it is just
skipped.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
While github.com/stretchr/testify is not used directly by any of the
repository code, it is a transitive dependency via Swarmkit and
therefore still easy to use without having to revendor. Add lint rules
to ban importing testify packages to make sure nobody does.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Apply command gotest.tools/v3/assert/cmd/gty-migrate-from-testify to the
cnmallocator package to be consistent with the assertion library used
elsewhere in moby.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Moby imports Swarmkit; Swarmkit no longer imports Moby. In order to
accomplish this feat, Swarmkit has introduced a new plugin.Getter
interface so it could stop importing our pkg/plugingetter package. This
new interface is not entirely compatible with our
plugingetter.PluginGetter interface, necessitating a thin adapter.
Swarmkit had to jettison the CNM network allocator to stop having to
import libnetwork as the cnmallocator package is deeply tied to
libnetwork. Move the CNM network allocator into libnetwork, where it
belongs. The package had a short an uninteresting Git history in the
Swarmkit repository so no effort was made to retain history.
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-28 09:46:45 -05:00
1712 changed files with 118770 additions and 42515 deletions
run:echo "::error::PR title suggests targetting the ${{ steps.title_branch.outputs.branch }} branch, but is opened against ${{ github.event.pull_request.base.ref }}" && exit 1
id:title_branch
run:|
# get the intended major version prefix ("[27.1 backport]" -> "27.") from the PR title.
@@ -32,7 +32,7 @@ New projects can be added if they fit with the community goals. Docker is commit
However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.
The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful.
The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.
The releases are supported by the maintainers, community and users, on a best efforts basis only. For customers who want enterprise or commercial support, [Docker Desktop](https://www.docker.com/products/docker-desktop/) and [Mirantis Container Runtime](https://www.mirantis.com/software/mirantis-container-runtime/) are the appropriate products for these use cases.
// There is existing endpoint config - if it's not indexed by NetworkMode.Name(), we
// can't tell which network the container-wide settings was intended for. NetworkMode,
// the keys in EndpointsConfig and the NetworkID in EndpointsConfig may mix network
// name/id/short-id. It's not safe to create EndpointsConfig under the NetworkMode
// name to store the container-wide MAC address, because that may result in two sets
// of EndpointsConfig for the same network and one set will be discarded later. So,
// reject the request ...
ep,ok:=networkingConfig.EndpointsConfig[nwName]
if!ok{
return"",errdefs.InvalidParameter(errors.New("if a container-wide MAC address is supplied, HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks"))
}
// ep is the endpoint that needs the container-wide MAC address; migrate the address
// to it, or bail out if there's a mismatch.
ifep.MacAddress==""{
ep.MacAddress=deprecatedMacAddress
}elseifep.MacAddress!=deprecatedMacAddress{
return"",errdefs.InvalidParameter(errors.New("the container-wide MAC address must match the endpoint-specific MAC address for the main network, or be left empty"))
return"",errors.Wrap(err,"unable to migrate container-wide MAC address to a specific network")
}
// ep is the endpoint that needs the container-wide MAC address; migrate the address
// to it, or bail out if there's a mismatch.
ifep.MacAddress==""{
ep.MacAddress=deprecatedMacAddress
}elseifep.MacAddress!=deprecatedMacAddress{
return"",errdefs.InvalidParameter(errors.New("the container-wide MAC address must match the endpoint-specific MAC address for the main network, or be left empty"))
}
}
warning="The container-wide MacAddress field is now deprecated. It should be specified in EndpointsConfig instead."
expError:"if a container-wide MAC address is supplied, HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks",
expError:"unable to migrate container-wide MAC address to a specific network: HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks",
stream:=grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(otelgrpc.StreamServerInterceptor(),grpcerrors.StreamServerInterceptor))//nolint:staticcheck // TODO(thaJeztah): ignore SA1019 for deprecated options: see https://github.com/moby/moby/issues/47437
withTrace:=otelgrpc.UnaryServerInterceptor()//nolint:staticcheck // TODO(thaJeztah): ignore SA1019 for deprecated options: see https://github.com/moby/moby/issues/47437
returntypes.NetworkResource{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on name)",term,len(listByFullName)))
returnnetwork.Inspect{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on name)",term,len(listByFullName)))
}
// Find based on partial ID, returns true only if no duplicates
returntypes.NetworkResource{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on ID prefix)",term,len(listByPartialID)))
returnnetwork.Inspect{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on ID prefix)",term,len(listByPartialID)))
// TODO(thaJeztah): unify with [Stats]. This wrapper was to account for pre-api v1.21 changes, see https://github.com/moby/moby/commit/d3379946ec96fb6163cb8c4517d7d5a067045801
// CreateRequest is the request message sent to the server for network create call.
typeCreateRequeststruct{
CreateOptions
Namestring// Name is the requested name of the network.
// Deprecated: CheckDuplicate is deprecated since API v1.44, but it defaults to true when sent by the client
// package to older daemons.
CheckDuplicate*bool`json:",omitempty"`
}
// CreateOptions holds options to create a network.
typeCreateOptionsstruct{
Driverstring// Driver is the driver-name used to create the network (e.g. `bridge`, `overlay`)
Scopestring// Scope describes the level at which the network exists (e.g. `swarm` for cluster-wide or `local` for machine level).
EnableIPv6*bool`json:",omitempty"`// EnableIPv6 represents whether to enable IPv6.
IPAM*IPAM// IPAM is the network's IP Address Management.
Internalbool// Internal represents if the network is used internal only.
Attachablebool// Attachable represents if the global scope is manually attachable by regular containers from workers in swarm mode.
Ingressbool// Ingress indicates the network is providing the routing-mesh for the swarm cluster.
ConfigOnlybool// ConfigOnly creates a config-only network. Config-only networks are place-holder networks for network configurations to be used by other networks. ConfigOnly networks cannot be used directly to run containers or services.
ConfigFrom*ConfigReference// ConfigFrom specifies the source which will provide the configuration for this network. The specified network must be a config-only network; see [CreateOptions.ConfigOnly].
Optionsmap[string]string// Options specifies the network-specific options to use for when creating the network.
Labelsmap[string]string// Labels holds metadata specific to the network being created.
}
// ListOptions holds parameters to filter the list of networks with.
typeListOptionsstruct{
Filtersfilters.Args
}
// InspectOptions holds parameters to inspect network.
typeInspectOptionsstruct{
Scopestring
Verbosebool
}
// ConnectOptions represents the data to be used to connect a container to the
// DisconnectOptions represents the data to be used to disconnect a container
// from the network.
typeDisconnectOptionsstruct{
Containerstring
Forcebool
}
// Inspect is the body of the "get network" http response message.
typeInspectstruct{
Namestring// Name is the name of the network
IDstring`json:"Id"`// ID uniquely identifies a network on a single machine
Createdtime.Time// Created is the time the network created
Scopestring// Scope describes the level at which the network exists (e.g. `swarm` for cluster-wide or `local` for machine level)
Driverstring// Driver is the Driver name used to create the network (e.g. `bridge`, `overlay`)
EnableIPv6bool// EnableIPv6 represents whether to enable IPv6
IPAMIPAM// IPAM is the network's IP Address Management
Internalbool// Internal represents if the network is used internal only
Attachablebool// Attachable represents if the global scope is manually attachable by regular containers from workers in swarm mode.
Ingressbool// Ingress indicates the network is providing the routing-mesh for the swarm cluster.
ConfigFromConfigReference// ConfigFrom specifies the source which will provide the configuration for this network.
ConfigOnlybool// ConfigOnly networks are place-holder networks for network configurations to be used by other networks. ConfigOnly networks cannot be used directly to run containers or services.
Containersmap[string]EndpointResource// Containers contains endpoints belonging to the network
Optionsmap[string]string// Options holds the network specific options to use for when creating the network
Labelsmap[string]string// Labels holds metadata specific to the network being created
Peers[]PeerInfo`json:",omitempty"`// List of peer nodes for an overlay network
Servicesmap[string]ServiceInfo`json:",omitempty"`
}
// Summary is used as response when listing networks. It currently is an alias
// for [Inspect], but may diverge in the future, as not all information may
// be included when listing networks.
typeSummary=Inspect
// Address represents an IP address
typeAddressstruct{
Addrstring
@@ -45,6 +123,16 @@ type ServiceInfo struct {
Tasks[]Task
}
// EndpointResource contains network resources allocated and used for a
// container in a network.
typeEndpointResourcestruct{
Namestring
EndpointIDstring
MacAddressstring
IPv4Addressstring
IPv6Addressstring
}
// NetworkingConfig represents the container's networking configuration for each of its interfaces
// Carries the networking configs specified in the `docker run` and `docker network connect` commands
typeNetworkingConfigstruct{
@@ -70,3 +158,9 @@ var acceptedFilters = map[string]bool{
funcValidateFilters(filterfilters.Args)error{
returnfilter.Validate(acceptedFilters)
}
// PruneReport contains the response for Engine API:
// CopyConfig contains request body of Engine API:
// POST "/containers/"+containerID+"/copy"
typeCopyConfigstruct{
Resourcestring
}
// ContainerPathStat is used to encode the header from
// GET "/containers/{name:.*}/archive"
// "Name" is the file or directory name.
typeContainerPathStatstruct{
Namestring`json:"name"`
Sizeint64`json:"size"`
Modeos.FileMode`json:"mode"`
Mtimetime.Time`json:"mtime"`
LinkTargetstring`json:"linkTarget"`
}
// ContainerStats contains response of Engine API:
// GET "/stats"
typeContainerStatsstruct{
Bodyio.ReadCloser`json:"body"`
OSTypestring`json:"ostype"`
}
// Ping contains response of Engine API:
// GET "/_ping"
typePingstruct{
@@ -230,17 +205,6 @@ type Version struct {
BuildTimestring`json:",omitempty"`
}
// ExecStartCheck is a temp struct used by execStart
// Config fields is part of ExecConfig in runconfig package
typeExecStartCheckstruct{
// ExecStart will first check if it's detached
Detachbool
// Check if there's a tty
Ttybool
// Terminal size [height, width], unused if Tty == false
ConsoleSize*[2]uint`json:",omitempty"`
}
// HealthcheckResult stores information about a single run of a healthcheck probe
typeHealthcheckResultstruct{
Starttime.Time// Start is the time this check started
@@ -281,18 +245,6 @@ type ContainerState struct {
Health*Health`json:",omitempty"`
}
// ContainerNode stores information about the node that a container
// is running on. It's only used by the Docker Swarm standalone API
typeContainerNodestruct{
IDstring
IPAddressstring`json:"IP"`
Addrstring
Namestring
Cpusint
Memoryint64
Labelsmap[string]string
}
// ContainerJSONBase contains response of Engine API:
// GET "/containers/{name:.*}/json"
typeContainerJSONBasestruct{
@@ -306,7 +258,7 @@ type ContainerJSONBase struct {
HostnamePathstring
HostsPathstring
LogPathstring
Node*ContainerNode`json:",omitempty"`// Node is only propagated by Docker Swarm standalone API
Node*ContainerNode`json:",omitempty"`// Deprecated: Node was only propagated by Docker Swarm standalone API. It sill be removed in the next release.
Namestring
RestartCountint
Driverstring
@@ -423,84 +375,6 @@ type MountPoint struct {
Propagationmount.Propagation
}
// NetworkResource is the body of the "get network" http response message
typeNetworkResourcestruct{
Namestring// Name is the requested name of the network
IDstring`json:"Id"`// ID uniquely identifies a network on a single machine
Createdtime.Time// Created is the time the network created
Scopestring// Scope describes the level at which the network exists (e.g. `swarm` for cluster-wide or `local` for machine level)
Driverstring// Driver is the Driver name used to create the network (e.g. `bridge`, `overlay`)
EnableIPv6bool// EnableIPv6 represents whether to enable IPv6
IPAMnetwork.IPAM// IPAM is the network's IP Address Management
Internalbool// Internal represents if the network is used internal only
Attachablebool// Attachable represents if the global scope is manually attachable by regular containers from workers in swarm mode.
Ingressbool// Ingress indicates the network is providing the routing-mesh for the swarm cluster.
ConfigFromnetwork.ConfigReference// ConfigFrom specifies the source which will provide the configuration for this network.
ConfigOnlybool// ConfigOnly networks are place-holder networks for network configurations to be used by other networks. ConfigOnly networks cannot be used directly to run containers or services.
Containersmap[string]EndpointResource// Containers contains endpoints belonging to the network
Optionsmap[string]string// Options holds the network specific options to use for when creating the network
Labelsmap[string]string// Labels holds metadata specific to the network being created
Peers[]network.PeerInfo`json:",omitempty"`// List of peer nodes for an overlay network
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.