With improved IPv6 support, a dual-stack container can map a port using
two different networks -- one IPv4-only, the other IPv6-only.
The daemon was updating containers' `EndpointSettings.Ports` by looking
for the first network providing port-mappings. This was incorrect.
Instead, iterate over the whole list of endpoints, and merge everything
together.
The function doing that, ie. `getEndpointPortMapInfo`, is also
considered exposed ports, and nil the PortMap entry if an exposed port
is found. However, exposed ports are always set on a bridge network, so
this was erasing port-mappings found for other networks.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Instead, log the error returned by `nat.NewPort` and move on to the
next port mapping / exposed port.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The function `getEndpointPortMapInfo` is called by `updateJoinInfo` to
update the field `NetworkSettings.Ports` of a container.
However, `updateJoinInfo` is only called by `connectToNetwork` which is
also calling `getPortMapInfo` (which in turn calls
`getEndpointPortMapInfo`).
So, remove the call to `getEndpointPortMapInfo` from `updateJoinInfo`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The UDP proxy is setting a deadline of 90 seconds when reading from the
backend. If no data is received within this interval, it reclaims the
connection.
This means, the backend would see a different connection every 90
seconds if the backend never sends back any reply to a client.
This change prevents the proxy from eagerly GC'ing such connections by
taking into account the last time a datagram was proxyed to the backend.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The UDP proxy used by cmd/docker-proxy is executing Write and Close in
two separate goroutines, such that a Close could interrupt an in-flight
Write.
Introduce a `connTrackEntry` that wraps a `net.Conn` and a `sync.Mutex`
to ensure that Write and Close are serialized.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
It seems to help with the flakiness in the CI.
However, I can't reproduce the flakiness locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Improve performance of function daemon.Containers() (used by docker ps) to
mitigate a latency increase when running large number of containers using the
containerd image store.
We do this by refactoring daemon.Containers() to collect info for containers in
parallel, rather than sequentially, using up to log2(N) worker threads. This
improves the performance from O(N) to O(log2(N)), where N is the number of
containers.
To verify correctness, this commits adds unit and integration tests.
Signed-off-by: Cesar Talledo <cesar.talledo@docker.com>
When a macvlan's parent interface is down it's not possible
to send NA messages. So, ignore the error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The container platform migration tries to deduce the platform data from
the containerd content store if it's available.
However, on Windows we currently default to a non-containerd runtime
setup, so the containerd client is nil and accessing its content store
paniced:
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x11b48e4]
goroutine 87 [running]:
github.com/containerd/containerd/v2/client.(*Client).ContentStore(0xc0003a0008?)
/go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/v2/client/client.go:645 +0x24
github.com/docker/docker/daemon.(*Daemon).load(0xc00026e488, {0xc000c13d40, 0x40})
/go/src/github.com/docker/docker/daemon/container.go:84 +0x289
github.com/docker/docker/daemon.(*Daemon).restore.func1({0xc000c13d40, 0x40})
/go/src/github.com/docker/docker/daemon/daemon.go:236 +0x207
created by github.com/docker/docker/daemon.(*Daemon).restore in goroutine 1
/go/src/github.com/docker/docker/daemon/daemon.go:229 +0x1a7
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x11b48e4]
goroutine 90 [running]:
github.com/containerd/containerd/v2/client.(*Client).ContentStore(0xc000313608?)
/go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/v2/client/client.go:645 +0x24
github.com/docker/docker/daemon.(*Daemon).load(0xc00026e488, {0xc000c13e00, 0x40})
/go/src/github.com/docker/docker/daemon/container.go:84 +0x289
github.com/docker/docker/daemon.(*Daemon).restore.func1({0xc000c13e00, 0x40})
/go/src/github.com/docker/docker/daemon/daemon.go:236 +0x207
created by github.com/docker/docker/daemon.(*Daemon).restore in goroutine 1
/go/src/github.com/docker/docker/daemon/daemon.go:229 +0x1a7
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x11b48e4]
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Split the `ErrConflictHostNetwork` error into two distinct errors:
1. `ErrConflictConnectToHostNetwork` when attempting to change the
network mode of a running container from a different mode to `host`
2. `ErrConflictDisconnectFromHostNetwork` when the network mode of a
running container is `host` and attempting to disconnect from `host`
This commit clarifies error messaging by differentiating between the two
errors, making it clearer which operation failed and how to fix it.
Signed-off-by: Brendon Smith <bws@bws.bio>
On Linux, mask "/proc/interrupts" and "/sys/devices/system/cpu/cpu<x>/thermal_throttle"
inside containers by default. Privileged containers or containers started
with --security-opt="systempaths=unconfined" are not affected.
Mitigates potential Thermal Side-Channel Vulnerability Exploit
(https://github.com/moby/moby/security/advisories/GHSA-6fw5-f8r9-fgfm).
Also: improve integration test TestCreateWithCustomMaskedPaths() to ensure
default masked paths don't apply to privileged containers.
Signed-off-by: Cesar Talledo <cesar.talledo@docker.com>
For kernels that don't have CONFIG_IP_NF_RAW, if the env
var DOCKER_INSECURE_NO_IPTABLES_RAW is set to "1", don't
try to create raw rules.
This means direct routing to published ports is possible
from other hosts on the local network, even if the port
is published to a loopback address.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This function was only used internally, but it still has at least one
external consumer, so adding a "deprecated" comment.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
[homedir.GetConfigHome] only returns an error if the value is empty,
so we can check for a non-empty value instead of an error, which also
means that this value would never be empty.
[homedir.GetConfigHome]: b4bdf12dae/pkg/homedir/homedir_linux.go (L86-L95)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function had to be called both in the daemon startup, as well as
the CLI startup. Which, in case of the cli, meant that the registry
package became a required dependency for all CLI-plugins.
Make the package itself aware of situations where it's running with
rootlessKit enabled. Altogether we should get rid of this package-level
variable, and instead store this in our configuration, and pass through
where it's used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the userland-proxy is running, packets from one bridge network
addressed to the host port are not DNAT'd - so that docker-proxy
can pick them up, and therefore the packet bypasses the network
isolation rules.
Without the userland-proxy, there's no way for a packet from one
bridge network to bypass the network isolation rules. So, in this
case, DNAT is not skipped - and that at-least allows packets
originating from the network that published the port to access
the host port.
Commit 0546d90 improved support for routed mode networks (allowing
nat-mode networks access to containers in routed-mode networks, as
well as just remote access).
That commit changed the "SKIP DNAT" logic, making sure DNAT was
skipped for a routed-mode network if the userland-proxy was enabled
(so, containers in routed mode networks could access ports published
by other networks).
But, it still skipped DNAT for a routed mode network if the userland
proxy was disabled - packets from the routed mode network aimed at
any other network would be dropped by the network isolation rules
anyway, and containers in a routed mode network don't need access to
ports published from that network (because, by definition, there
can't be any).
However, network isolation rules can be worked-around with a rule
in the DOCKER-USER chain, but the SKIP DNAT rule is harder to deal
with.
So, for routed-mode, only skip DNAT if the userland-proxy is
enabled (just like nat-mode networks).
Signed-off-by: Rob Murray <rob.murray@docker.com>
The test had 2 almost identical separate implementations (Linux and
Windows). The Windows one was skipped anyway.
Make one test that covers all test cases.
The test still needs to be fixed for Windows, so don't unskip it yet.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Split singlePlatformSize into a separate methods of the `ImageManifest`
that calculate the corresponding usage.
This also allows us to simplify a part of the `multiPlatformSummary`
because we don't calculate the content size twice now.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
[docker/cli@fc6976d] added support for recursive readonly mounts in the
CLI, adding a ValidateMountWithAPIVersion utility to verify if options
used were supported by the API version.
We usually keep API-version dependent checks in the client, so that
docker/cli (and other users of the client) don't have to implement
their own validation for these.
This patch moves the functionality of ValidateMountWithAPIVersion to
the client.
Once the docker/cli vendoring was updated, we can remove the utility
there.
[docker/cli@fc6976d]: fc6976db45
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was implemented to use various parts of the registry package
that were designed for the daemon code, which was written with the assumption
that it had registry-config available from the daemon's configuration.
However, `ParseSearchIndexInfo` is used by the client / CLI, which does
not have this information.
To work around this problem, the code used a dummy "emptyServiceConfig"
to allow the `Insecure` and `Mirrors` fields to be propagated based on
the same defaults as used by the daemon. The `Mirrors` field would always
be empty, as there are no default mirrors, and (lacking access to the
daemon's config) the `Insecure` field would always default to registries
running on a loopback address (`::1/128`, `127.0.0.1/8`). It's worth noting
that neither the `Mirrors`, nor the `Insecure` field is used by the CLI.
This patch rewrites `ParseSearchIndexInfo` to be self-contained, and not
depend on these constructs (and the `emptyServiceConfig`). For now, the
existing logic for `Insecure` is kept, but replaced by a simplified function
(`isInsecure`) with some optimizations for well-known loopback addresses
(`localhost`, `::1`, `127.0.0.1`) to prevent redundant DNS lookups or
parsing.
Note that similar changes should be made for [ParseRepositoryInfo], which
has a similar fate and is also only used by the client / CLI.
[ResolveRepositoryName]: 11e47996dc/registry/registry.go (L199-L222)
[ParseRepositoryInfo]: d86dd75948/registry/config.go (L375-L381)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The temp-file was created before trying to make the given filename an
absolute path. Reverse the order of code so that we don't create
a temp-file if an error happens.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was introduced in 568f86eb18
to replace [ResolveRepositoryName]. The function was implemented to use
various parts of the registry package that were designed for the daemon
code, which was written with the assumption that it had registry-config
available from the daemon's configuration. However, `ParseRepositoryInfo`
was used in the client / CLI, which does not have this information.
To work around this problem, the code used a dummy "emptyServiceConfig"
to allow the `Insecure` and `Mirrors` fields to be propagated based on
the same defaults as used by the daemon. The `Mirrors` field would always
be empty, as there are no default mirrors, and (lacking access to the
daemon's config) the `Insecure` field would always default to registries
running on a loopback address (`::1/128`, `127.0.0.1/8`). It's worth noting
that neither the `Mirrors`, nor the `Insecure` field is used by the CLI.
This patch rewrites `ParseRepositoryInfo` to be self-contained, and not
depend on these constructs (and the `emptyServiceConfig`). For now, the
existing logic for `Insecure` is kept, but replaced by a simplified function
(`isInsecure`) with some optimizations for well-known loopback addresses
(`localhost`, `::1`, `127.0.0.1`) to prevent redundant DNS lookups or
parsing.
Note that similar changes should be made for [ParseSearchIndexInfo], which
has a similar fate and is also only used by the client / CLI.
[ResolveRepositoryName]: 11e47996dc/registry/registry.go (L199-L222)
[ParseSearchIndexInfo]: d86dd75948/registry/search.go (L153-L162)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was setting some fields to their zero / default-value, which
was redundant, and added additional imports. Remove them as they are
not needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I noticed that the only reason we kept this was so that we could produce
a more targeted error for the deprecated storage-driver plugins, but it's
very unlikely someone used those, and if they did, we already had the
"DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS" added as requirement. Let's
just produce an error if that option is set (and remove that altogether in
a later release, but just that check doesn't add significant complexity).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Explicitly set the gRPC connection params to take the timeout into
account to workaround the containerd v2 client not passing down the
stack.
containerd v2 replaced usages of deprecated gRPC functions but didn't
pass the timeout to the actual dial connection options.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
full diff: https://github.com/golang/net/compare/v0.35.0...v0.36.0
Version v0.36.0 of golang.org/x/net fixes a vulnerability in the
golang.org/x/net/proxy and golang.org/x/net/http/httpproxy packages
which could cause the proxy to be bypassed.
Matching of hosts against proxy patterns could improperly treat an IPv6
zone ID as a hostname component. For example, when the NO_PROXY
environment variable was set to "*.example.com", a request to
"[::1%25.example.com]:80` would incorrectly match and not be proxied.
Thanks to Juho Forsén of Mattermost for reporting this issue.
This is CVE-2025-22870 and Go issue https://go.dev/issue/71984.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- full diff: https://github.com/containerd/containerd/compare/v1.7.25...v1.7.26
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.26
Welcome to the v1.7.26 release of containerd!
The twenty-sixth patch release for containerd 1.7 contains various fixes
and updates.
Highlights
- Add support for syncfs after unpack
- Update runc binary to v1.2.55
- Fix race between serve and immediate shutdown on the server
- Reject oversized messages from the sender
Container Runtime Interface (CRI)
- Fix fatal concurrency error in port forwarding
Node Resource Interface (NRI)
- Fix initial sync race when registering NRI plugins
- Add API support for reading Pod IPs
- Fix plugin sync to use multiple messages if ttrpc max message limit is hit
- Update API to pass configured timeouts to plugins.
- Fix mount removal in adjustments
- Close plugin if initial synchronization fails
- Add support for adjusting OOM score
- Add API support for NRI-native CDI injection
- Add support for pids cgroup
Runtime
- Fix console TTY leak in runc shim
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We return errors in this function, except for this one, which was logged
as Fatal. If we want errors to be logged, we should probably do so in
the function calling daemonCLI.start.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function could produce various logs ("Running in rootless mode") at
the start, but further steps could still fail (such as running with
RootlessKit, but not being configured as rootless).
This patch moves the informational / warning logs further down, so that
we don't produce logs before failing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Explicitly access config field through the Config field, instead
of the top-level "cli". This allows spotting locations where we don't
depend on the CLI, but really just the Config.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method only depended on the CLI config; rewrite it to a
regular function, returning the opts to use for the containerd
daemon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function took the whole daemon CLI as argument, but only needed
the config; change the signature to accept that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method only required the config to be passed; rewrite it to a
regular func that accepts the config.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These tests don't actually run the integration-cli suite, but
the global hack/xxx script errors because it's not set;
---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary /usr/local/cli-integration/docker
# DOCKER_EXPERIMENTAL is set: starting daemon with experimental features enabled!
# cgroup v2 requires TEST_SKIP_INTEGRATION_CLI to be set
make: *** [Makefile:220: test-docker-py] Error 1
Error: Process completed with exit code 2.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fixes unnecessary errors being written to the daemon log after
copying container streams failed due the streams being closed explicitly:
time="2025-03-06T13:20:53.473232423Z" level=error msg="copy stream failed" error="io: read/write on closed pipe" stream=stdin
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
We have tagged version v0.27.0 of golang.org/x/oauth2 in order to address
a security issue.
jws: unexpected memory consumption during token parsing
Version v0.27.0 of golang.org/x/oauth2 fixes a vulnerability in the
golang.org/x/oauth2/jws package which could cause a denial of service.
An attacker can pass a malicious malformed token which causes unexpected
memory to be consumed during parsing.
Thanks to jub0bs for reporting this issue.
This is CVE-2025-22868 and Go issue https://go.dev/issue/71490.
full diff: https://github.com/golang/oauth2/compare/v0.26.0...v0.27.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
No code changes, only comments (warnings) added;
- google: add warning about externally-provided credentials
> Important: If you accept a credential configuration (credential JSON/File/Stream) from an
> external source for authentication to Google Cloud Platform, you must validate it before
> providing it to any Google API or library. Providing an unvalidated credential configuration to
> Google APIs can compromise the security of your systems and data. For more information, refer to
> [Validate credential configurations from external sources](https://cloud.google.com/docs/authentication/external/externally-sourced-credentials).
full diff: https://github.com/golang/oauth2/compare/v0.23.0...v0.26.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We have tagged version v0.35.0 of golang.org/x/crypto in order to address
a security issue. Version v0.35.0 of golang.org/x/crypto fixes a vulnerability
in the golang.org/x/crypto/ssh package which could cause a denial of service.
SSH servers which implement file transfer protocols are vulnerable to a denial
of service attack from clients which complete the key exchange slowly, or not
at all, causing pending content to be read into memory, but never transmitted.
Thanks to Yuichi Watanabe for reporting this issue.
This is CVE-2025-22869 and Go issue https://go.dev/issue/71931.
full diff: https://github.com/golang/crypto/compare/v0.31.0...v0.35.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go maintainers started to unconditionally update the minimum go version
for golang.org/x/ dependencies to go1.23, which means that we'll no longer
be able to support any version below that when updating those dependencies;
> all: upgrade go directive to at least 1.23.0 [generated]
>
> By now Go 1.24.0 has been released, and Go 1.22 is no longer supported
> per the Go Release Policy (https://go.dev/doc/devel/release#policy).
>
> For golang/go#69095.
This updates our minimum version to go1.23, as we won't be able to maintain
compatibility with older versions because of the above.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These chains were added in commit 76417bf ("Don't use ipset"), but
not included in the list of chains that get flushed and deleted
on daemon startup.
Signed-off-by: Rob Murray <rob.murray@docker.com>
api/types/registry/registry.go:60:2: naked return in func `UnmarshalJSON` with 9 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/server/router/build/build_routes.go:359:2: naked return in func `Write` with 5 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
image/tarexport/load.go:429:2: naked return in func `validatedParentLinks` with 12 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/awslogs/cloudwatchlogs.go:684:2: naked return in func `findValidSplit` with 10 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/proxy.go:35:3: naked return in func `StartLogging` with 17 lines of code (nakedret)
return
^
daemon/logger/proxy.go:42:2: naked return in func `StartLogging` with 17 lines of code (nakedret)
return
^
daemon/logger/proxy.go:61:3: naked return in func `StopLogging` with 16 lines of code (nakedret)
return
^
daemon/logger/proxy.go:68:2: naked return in func `StopLogging` with 16 lines of code (nakedret)
return
^
daemon/logger/proxy.go:80:3: naked return in func `Capabilities` with 14 lines of code (nakedret)
return
^
daemon/logger/proxy.go:89:2: naked return in func `Capabilities` with 14 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/graphdriver/fuse-overlayfs/fuseoverlayfs.go:527:2: naked return in func `fusermountU` with 25 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/graphdriver/fsdiff.go:140:3: naked return in func `ApplyDiff` with 20 lines of code (nakedret)
return
^
daemon/graphdriver/fsdiff.go:149:3: naked return in func `ApplyDiff` with 20 lines of code (nakedret)
return
^
daemon/graphdriver/fsdiff.go:153:2: naked return in func `ApplyDiff` with 20 lines of code (nakedret)
return
^
daemon/graphdriver/fsdiff.go:164:3: naked return in func `DiffSize` with 15 lines of code (nakedret)
return
^
daemon/graphdriver/fsdiff.go:169:3: naked return in func `DiffSize` with 15 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/cluster/cluster.go:423:2: naked return in func `managerStats` with 24 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/daemon.go:1570:2: naked return in func `RemapContainerdNamespaces` with 21 lines of code (nakedret)
return
^
daemon/daemon_linux.go:128:2: naked return in func `getCleanPatterns` with 14 lines of code (nakedret)
return
^
daemon/logs.go:180:2: naked return in func `getLogger` with 11 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove output variables, and use explicit returns
daemon/stats_unix.go:359:2: naked return in func `getSystemCPUUsage` with 38 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
registry/service.go:83:4: naked return in func `Auth` with 38 lines of code (nakedret)
return
^
registry/search_session.go:91:2: naked return in func `Read` with 6 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
plugin/backend_linux.go:722:3: naked return in func `CreateFromContext` with 112 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/tarsum/versioning.go:150:2: naked return in func `v1TarHeaderSelect` with 35 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/stdcopy/stdcopy.go:68:2: naked return in func `Write` with 23 lines of code (nakedret)
return
^
pkg/stdcopy/stdcopy_test.go:93:3: naked return in func `getSrcBuffer` with 10 lines of code (nakedret)
return
^
pkg/stdcopy/stdcopy_test.go:97:2: naked return in func `getSrcBuffer` with 10 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/archive/copy_unix_test.go:54:3: naked return in func `fileContentsEqual` with 35 lines of code (nakedret)
return
^
pkg/archive/copy_unix_test.go:60:3: naked return in func `fileContentsEqual` with 35 lines of code (nakedret)
return
^
pkg/archive/copy_unix_test.go:67:3: naked return in func `fileContentsEqual` with 35 lines of code (nakedret)
return
^
pkg/archive/copy_unix_test.go:74:3: naked return in func `fileContentsEqual` with 35 lines of code (nakedret)
return
^
pkg/archive/copy_unix_test.go:83:2: naked return in func `fileContentsEqual` with 35 lines of code (nakedret)
return
^
pkg/archive/diff_test.go:314:3: naked return in func `makeTestLayer` with 35 lines of code (nakedret)
return
^
pkg/archive/diff_test.go:326:5: naked return in func `makeTestLayer` with 35 lines of code (nakedret)
return
^
pkg/archive/diff_test.go:330:5: naked return in func `makeTestLayer` with 35 lines of code (nakedret)
return
^
pkg/archive/diff_test.go:336:3: naked return in func `makeTestLayer` with 35 lines of code (nakedret)
return
^
pkg/archive/copy_unix_test.go:36:2: naked return in func `getTestTempDirs` with 10 lines of code (nakedret)
return
^
pkg/stdcopy/stdcopy_test.go:93:3: naked return in func `getSrcBuffer` with 10 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/archive/archive_linux.go:65:2: naked return in func `ConvertWrite` with 43 lines of code (nakedret)
return
^
pkg/archive/archive.go:265:2: naked return in func `Read` with 11 lines of code (nakedret)
return
^
pkg/archive/copy.go:32:2: naked return in func `copyWithBuffer` with 5 lines of code (nakedret)
return
^
pkg/archive/copy.go:114:3: naked return in func `TarResourceRebase` with 16 lines of code (nakedret)
return
^
pkg/archive/copy.go:449:4: naked return in func `ResolveHostSourcePath` with 26 lines of code (nakedret)
return
^
pkg/archive/copy.go:460:4: naked return in func `ResolveHostSourcePath` with 26 lines of code (nakedret)
return
^
pkg/archive/wrap.go:58:2: naked return in func `parseStringPairs` with 11 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/drivers/remote/driver_test.go:29:2: naked return in func `decodeToMap` with 3 lines of code (nakedret)
return
^
libnetwork/ipams/remote/remote_test.go:23:2: naked return in func `decodeToMap` with 3 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/types/types.go:330:2: naked return in func `ParseCIDR` with 6 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/osl/namespace_linux.go:324:2: naked return in func `DisableARPForVIP` with 28 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/drivers/overlay/encryption.go:370:2: naked return in func `programSA` with 64 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/cmd/networkdb-test/dbclient/ndbClient.go:251:5: naked return in func `checkTable` with 42 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
layer/layer_store.go:524:4: naked return in func `CreateRWLayer` with 72 lines of code (nakedret)
return
^
layer/layer_store.go:534:3: naked return in func `CreateRWLayer` with 72 lines of code (nakedret)
return
^
layer/layer_store.go:537:3: naked return in func `CreateRWLayer` with 72 lines of code (nakedret)
return
^
layer/migration.go:19:3: naked return in func `ChecksumForGraphID` with 28 lines of code (nakedret)
return
^
layer/migration.go:25:3: naked return in func `ChecksumForGraphID` with 28 lines of code (nakedret)
return
^
layer/migration.go:36:3: naked return in func `ChecksumForGraphID` with 28 lines of code (nakedret)
return
^
layer/migration.go:40:3: naked return in func `ChecksumForGraphID` with 28 lines of code (nakedret)
return
^
layer/migration.go:43:2: naked return in func `ChecksumForGraphID` with 28 lines of code (nakedret)
return
^
layer/ro_layer.go:176:2: naked return in func `Read` with 13 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In fairness, these could possibly be an exception to the rule, but adding
explicit returns isn't too bad either, and allows running the nakedret
linter without //nolint tags or exceptions in .golangci-lint.yaml
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
internal/mod/mod.go:22:3: naked return in func `moduleVersion` with 38 lines of code (nakedret)
return
^
internal/mod/mod.go:36:4: naked return in func `moduleVersion` with 38 lines of code (nakedret)
return
^
internal/mod/mod.go:57:2: naked return in func `moduleVersion` with 38 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove output variables, and use explicit returns
integration-cli/check_test.go:86:3: naked return in func `testRun` with 43 lines of code (nakedret)
return
^
integration-cli/check_test.go:97:3: naked return in func `testRun` with 43 lines of code (nakedret)
return
^
integration-cli/docker_cli_cp_utils_test.go:167:2: naked return in func `makeTestContainer` with 44 lines of code (nakedret)
return
^
integration-cli/docker_api_attach_test.go:299:3: naked return in func `readTimeout` with 12 lines of code (nakedret)
return
^
integration-cli/docker_cli_cp_utils_test.go:215:2: naked return in func `startContainerGetOutput` with 11 lines of code (nakedret)
return
^
integration-cli/docker_cli_logs_test.go:276:4: naked return in func `ConsumeWithSpeed` with 18 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove output variables, and use explicit returns
container/stream/bytespipe/bytespipe.go:165:2: naked return in func `Read` with 37 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
builder/remotecontext/detect.go:47:2: naked return in func `Detect` with 17 lines of code (nakedret)
return
^
builder/remotecontext/archive.go:127:2: naked return in func `normalize` with 7 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use explicit returns
- rename error-return that's used in a defer
- move closing the tmpFile on error to a defer
- add debug logs for cases where either closing the file, or failing to
remove the temp-directory
builder/dockerfile/copy.go:369:3: naked return in func `downloadSource` with 67 lines of code (nakedret)
return
^
builder/dockerfile/copy.go:374:3: naked return in func `downloadSource` with 67 lines of code (nakedret)
return
^
builder/dockerfile/copy.go:382:3: naked return in func `downloadSource` with 67 lines of code (nakedret)
return
^
builder/dockerfile/copy.go:398:3: naked return in func `downloadSource` with 67 lines of code (nakedret)
return
^
builder/dockerfile/copy.go:407:3: naked return in func `downloadSource` with 67 lines of code (nakedret)
return
^
builder/dockerfile/copy.go:428:3: naked return in func `downloadSource` with 67 lines of code (nakedret)
return
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Re-implement the Build function using the new implementation from
libnetwork/internal/resolvconf.
BuildKit is the only consumer of the Build function currently, and we
should consider either moving the internal package out of internal, or
to provide a more customized variant of Build that fits BuildKit's
needs (i.e., only patch content, but not write to a file).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The new resolvconf implementation in libnetwork/internal/resolvconf
uses a different order for "search" and puts it after the resolvers.
While this changes the checksum of the file, and thus would be potentially
detected as "user modified", we no longer use this function ourselves,
and BuildKit (which uses this function) only consumes the Content field,
and discards the Hash; https://github.com/moby/buildkit/blob/v0.20.0/executor/oci/resolvconf.go#L99-L103
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The tests were depending on the implementation ignoring invalid
nameservers, as these should always be an IP-address, not a hostname.
Update the tests to use correct values, instead of codifying the
bad behavior.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- rename the "client" argument to "apiClient" to prevent shadowing the client import.
- remove intermediate "mount" var, which shadowed an import
- remove debug logs for stdOut/stdErr
- update the defer to use the container test-utils package to fix unhandled error warnings.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Add test-cases for IPv6 refs
- Add test-cases for validating the insecure-registries passed in the test
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Using DeepEquals showed that the test was missing differences between
nil-mirrors and empty-slice, in addition to mirrors being normalized
(the test only checked for the length).
While we should consider if we need an explicit empty slice (or if a
nil value would be appropriate), at least we now have a test to verify
the behavior.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was a very thin wrapper around newServiceConfig, and didn't save
any code needed; possibly even the reverse, as it was abstracting
what it did under the hood.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We already have been following this convention - add a linter rule that
will help us follow that convention in future changes.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This was not revealed in our tests, which only checked for the length
of the Mirror-slice, but when testing with DeepEquals, tests were failing
when all tests were run (but succeeded on individual tests). The problem
here is that some code can mutate the list of Mirrors and set it to `nil`
or an empty slice, resulting in other tests to fail.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
ValidateIndexName is used by the docker daemon CLI to validate options
passed through CLI flags and daemon.json. However, it also handled
normalizing the registry name ("index.docker.io" -> "docker.io").
This patch splits the normalization code to a separate function. It
is currently not exported, but could be considered in the future;
if we do so, we may want to look for a better place for that function
to not have it in the same package as the registry code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
emptyServiceConfig is a default service-config for situations where
no config-file is available (e.g. when used in the CLI). If won't
have mirrors configured, but does have the default insecure registry
CIDRs for loopback interfaces configured.
Before this patch, this config was constructeed using the same code
that handled constructing the config with a config present, but this
involved parsing CIDR masks, and much more.
With this patch, the service config is constructed as a literal, making
it more transparent that it does not depend on any config or state.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
RepositoryInfo.Official indicates whether the image repository
is an official (docker library official images) repository.
We only need to check this if the image-repository is on Docker Hub.
This patch renames the variable to make it more transparent that this
boolean is for the repository, and not to be confused for IndexInfo.Official,
which indicates if the _registry_ is the "Official" (Docker Hub) registry.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field indicates whether a repository is an official image (Docker
Library / Docker Official Images). This information is deducted from the
image reference, and not used anywhere, other than in tests.
The `RepositoryInfo` and `IndexInfo` types were originally introduced in
568f86eb18, with their fields documented in
4fcb9ac40c.
At the time, the `Official` field was only used for `docker push` to produce
a custom error message if someone would attempt to push an official image
to docker hub (assuming nobody would be able to do so);
6870bde584/api/client/commands.go (L1184-L1194)
Before that commit, the condition for this error message was based on the
given image reference directly; b370acd679b370acd679/commands.go (L421-L428)
This patch deprecates the field, because it's not used, and removes
tests related to it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Relevant changes:
- Update remote content to break up writes to avoid grpc message size limits
- Move CDI device spec out of the OCI package
- Remove deprecated WithCDIDevices in oci spec opts
full diff: https://github.com/containerd/containerd/compare/v2.0.2...v2.0.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's either "not windows" or "windows" (and something else), so the second
condition doesn't have to check if it's windows.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "local" client does not use containerd, but implements the same interface,
many args are not used though, so remove these to make it more clear what's
actually in use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "local" client does not use containerd, but implements the same interface,
many args are not used though, so remove these to make it more clear what's
actually in use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also remove an intermediate var, and remove a "WithError" in favor of
adding the error field to the "WithFields".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
All the tests in it had skips for Windows. The move makes it possible
to add a test that uses L3Segment.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This matches the name used by Swarm in swarmnode.Config. While updating,
also remove code from Cluster.Start that replicated the logic to construct
the path, in favor of using the `stateDir` field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The WatchStream field was set as configuration option in cmd/dockerd,
but not configurable. Move creating the stream to the constructor,
and remove the configuration option. This field was introduced in
59d45c384a, at which time the cmd/dockerd
code needed direct access to the stream, but a later refactor in
05346355db introduced an accessor
(GetWatchStream) for this.
The cluster.Config struct is only used internally, it's unlikely
for any external project to use this, so skipping deprecation.
With this change, the cmd/dockerd package no longer has a direct
import of swarmkit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit ccdfac5328.
Since commit 76417bf, the kernel ip_set module (and friends) are
not required.
Signed-off-by: Rob Murray <rob.murray@docker.com>
A jump to DOCKER-INGRESS chain is only created when Swarm needs
it. That's always after jumps to DOCKER-USER and DOCKER-FORWARD
have been inserted at the top of the FORWARD chain. The
DOCKER-INGRESS rule needs to be between those two other jumps.
Placing the jump to DOCKER-INGRESS at the top of the DOCKER-FORWARD
chain puts it in the right place, without needing to shuffle any
other rules around when it's added.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Noticed this log in CI on Windows, which wasn't clear if it was an error
in Windows or in Docker;
2025-02-25T03:21:35.9273942Z [Error] Handler for POST /v1.48/containers/1713bc845f9bde79aa0017c16613fbfc8810b3272b31dbb2535d3fb1a3550f9c/pause returned error: cannot pause container 1713bc845f9bde79aa0017c16613fbfc8810b3272b31dbb2535d3fb1a3550f9c: Unimplemented: not implemented
Looks like it's a feature that's not implemented when using process-isolation,
so updating the error-message to make it more identifiable as an error
produced by us.
I kept the type to be a "not implemented", which will be converted to a
501 HTTP status (so still logged as error); alternatively, we could make
this a "invalid parameter".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While going through some logs from CI, I noticed this log-entry on Windows,
produced as part of a test;
2025-02-25T03:23:17.6584227Z [Error] Handler for POST /v1.48/containers/b47b1e632188426d6d42a4be04f9a3cc1eca40cfed9536d277011052af0b04f5/update returned error: Cannot update container b47b1e632188426d6d42a4be04f9a3cc1eca40cfed9536d277011052af0b04f5: Restart policy cannot be updated because AutoRemove is enabled for the container
While updating is an error for the user, it's not an error in the daemon,
so we should return the correct error-type (and avoid logging it as an
error in daemon logs).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 0546d90 introduced the use of ipset to reduce the number
of rules that need to be processed per-packet, and make the code
a bit simpler.
But, docker's used on embedded kernels compiled without support
for ipset, so the change is too disruptive.
Replace the two ipset rules with a new chain that writes out the
rule's actions long-hand. So ..
This rule:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst \
-m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
Is transformed into a per-bridge rule in new chain DOCKER-CT:
-A DOCKER-FORWARD -j DOCKER-CT
-A DOCKER-CT -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o bridge1 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
And:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
Is transformed into a per-bridge rule in new chain DOCKER-BRIDGE:
-A DOCKER-FORWARD -j DOCKER-BRIDGE
-A DOCKER-BRIDGE -o docker0 -j DOCKER
-A DOCKER-BRIDGE -o bridge1 -j DOCKER
Signed-off-by: Rob Murray <rob.murray@docker.com>
Creating a swarm network from a config-only network failed
because the new EnableIPv4 wasn't validated/propagated
correctly.
So:
- Always initialise EnableIPv4 to true, including for a config
only network, and on load from the store.
- Treat enableIPv4=true as the no-overridden state when checking
params for a config-from network.
- Propagate settings from the config 'Network' objects attributes
to its generic options, because the network driver only sees
generic options.
- This was happening already for Network.internal, after the
config-only network options were processed. Move that to
'applyConfigurationTo'.
- Add enableIPv4/enableIpv6 - enableIPv6 will normaly be present
anyway. But, for a network created pre-28.x and restored from
the store, there was no entry for 'netlabel.EnableIpv4'.
- Extend TestSwarmScopedNetFromConfig to start a service and
check it's ok.
Signed-off-by: Rob Murray <rob.murray@docker.com>
For swarm networks, Controller.NewNetwork is called to validate
network config ... nothing gets created, but ManagerRedirectError
is returned if the config is ok - then swarm does its own thing.
So, for a --config-from network, merge config before checking
whether it'll have IPv4 enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
With ip6tables enabled (by default), the daemon should start on a
host with IPv6 disabled - then error if IPv6 network creation is
attempted.
That regressed in commit 255fff4 - so, only try to set up network
isolation rules for a network if it's IPv6-enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In 28.0.0, Docker appended to the FORWARD chain - breaking other
applications that had appended their own rules that needed to execute
after Docker's rules.
Move most of Docker's rules out of the filter-FORWARD chain into a
new DOCKER-FORWARD chain, so that Docker can append to DOCKER-FORWARD
without affecting the order of rules in the FORWARD chain.
After daemon startup inserts jumps to DOCKER-USER and DOCKER-FORWARD,
the bridge driver will not touch the FORWARD chain again. DOCKER-INGRESS
is still added to the FORWARD chain, if used, as it was in 27.x and
earlier.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Fix empty `Manifests` field for multi-platform images that have no
platform blobs available locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Fix not being able to inspect images that are OCI-index only (like
`tianon/empty`) or a real multi-platform image which has no platform
blobs available locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
On i386, Linux doesn't provide direct socket syscall but instead
multiplexes them through the socketcall syscall (see `man 2 socketcall`).
This commit fixes compilation for i386 by wrapping the offending syscall
in a new function that uses the socketcall syscall on i386, and
the `setsockopt` syscall on other archs.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When an IPv6 network is first created with no specific IPAM config,
network inspect adds a CIDR range to the gateway address. After the
daemon has been restarted, it's just a plain address.
Once the daaemon's been restated, "info" becomes "config", and the
address is reported correctly from "config".
Make the IPv6 code to report the gateway from "info" use net.IPNet.IP
instead of the whole net.IPNet - like the IPv4 code.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The `ErrPortAlreadyAllocated` error was introduced in ffd68badc0,
and at the time used as sentinel error in the bridge driver. It was later
integrated into libnetwork ([libnetwork@672ced7]), and brought back when
libnetwork was integrated in v1.7.0; 272f8cd4bc
After libnetwork was integrated, the error was unused as sentinel error,
except for locally inside the package as part of a test;
git rev-parse --verify HEAD
496bc46c88
git grep '\.ErrPortAlreadyAllocated'
Which is still the case Today;
git describe --tags --match="v[0-9]*" HEAD
v28.0.0
git rev-parse --verify HEAD
af898abe44
git grep '\.ErrPortAlreadyAllocated'
Same for the `ErrAllPortsAllocated` (added in 739d124480)
and `ErrUnknownProtocol` (added in 303ed3c830)
errors, which were never used as sentinel errors, and still aren't;
git grep '\.ErrAllPortsAllocated'
git grep '\.ErrUnknownProtocol'
vendor/github.com/moby/buildkit/client/llb/source.go: if errors.Is(err, gitutil.ErrUnknownProtocol) {
This patch;
- un-exports these errors as they are not used as sentinel errors
- strips down the `ErrPortAlreadyAllocated`, removing the methods that
were added, but never used.
- removes the `newErrPortAlreadyAllocated` constructor
- renames `ErrPortAlreadyAllocated` to `alreadyAllocatedErr` to follow
go conventions.
[libnetwork@672ced7]: c0474b6438
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Prior to commit fe856b9, containers' network sandbox and interfaces were
created before the containerd task. Now, it's created after.
If this step fails, the containerd task is forcefully deleted, and an
event is sent to the c8d event monitor, which triggers `handleContainerExit`.
Then this method tries to restart the faulty container.
This leads to containers with a published port already in use to be
stuck in a tight restart loop (if they're started with
`--restart=always`) until the port is available. This is needlessly
spamming the daemon logs.
Prior to that commit, a published port already in use wouldn't trigger
the restart process.
This commit adds a check to `handleContainerExit` to ignore exit events
if the latest container error is related to networking setup.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
imageSummary erroneously returned a nil multi-platform summary in case
of a multi-platform image, which the OCI index is available locally, but
none of its platform manifest is.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Create a ContainerInspectResponse definition and Use the example fields
as defined in the definition; add additional examples in that definition
where missing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use the example fields as defined in the ContainerSummary definition,
and add additional examples in that definition where missing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 15fbd67407 updated golangci-lint,
which came with an updated version of the "fatcontext" linter, causing
linting to fail.
We use these to set up spans for our tests; suppress these through the
golangci-lint config (instead of linline "//nolint" comments) so that
we can revisit this approach and see if there's a more idiomatic way.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Mirantis now ships 25.0 for MCR (23.0 reaching end of support)
- Microsoft packages 27.x
- 26.1 is no longer maintained
- Other branches no longer maintained
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The format was validated twice; first just to validate, then to construct
the correct format-wrapper, or error for an unknown format.
This patch combines both switches to a single one.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove intermediate variables or move them closer to where they're used,
as this function has various early returns on errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Move some variables inside the "if" branch to make it clear they're only
used locally, and not outside of the branch.
- Skip handling "env" and "env-regex" options if there are no env-vars
to handle.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Rename variables that collided with imports
- Make capturing interval.Milliseconds conditional as it would trip some
linters for using a potential "zero" value.
- Use WithFields instead of chainging multiple "WithField" calls for logging.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- No need to use `fmt.Errorf` to format the error message
- Use structured logs, and include the message that failed to be sent
in a `message` field.
- When failing to marshal the message, log an error outlining what happened
instead of logging the bare error.
- Move the `messagesLen` variable closer to where it's used, putting the
context handling first as a reminder that we may want to pass a context
through.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is mostly to silences some linters, as we're returning immediately
after, so no looping would happen, but we don't need a defer here either
for the same reason, so let's just remove it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 0e50d946a2 introduced a feature to
allow a custom stop-signal to be set. As part of this, existing code to
parse the signal was extracted to `signal.ParseSignal()`, which accepts
a string either containing a numeric value or a named signal.
When failing to parse the given signal, it returns an error and a magic
"-1" signal. The changes in 0e50d946a2 used
the error when creating a container, but for existing container configs,
it would ignore the error and instead check if the signal was "0", in
which case it would fall back to use the default stop-signal (SIGTERM).
Given that `signal.ParseSignal()` returns "-1" (not "0") for invalid
signals, this would result in the failure going undetected and "-1"
being used instead of the intended default (SIGTERM).
In practice, this issues would unlikely be encountered, as custom signals
are validated when creating the container, but it would be possible for
an image to contain an invalid signal, which would be used by the container
as default.
This patch updates the logic to only use the custom value if no error is
produced and a non-zero, positive signal is returned.
A test-case was added that would fail before this patch:
go test -v -run TestContainerStopSignal
=== RUN TestContainerStopSignal
container_test.go:34: assertion failed: signal -1 (s syscall.Signal) != terminated (defaultStopSignal syscall.Signal)
--- FAIL: TestContainerStopSignal (0.00s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
"SIGTERM" is defined both for Windows and Linux, so we can define the
signal to use as a syscall.Signal, instead of parsing it from a string
whenever we need to use the default.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use t.TempDir() instead of TestMain creating a directory to make
tests self-contained.
- fix some unhandled errors, and missing assertions for error-types
- assert with gotest.tools, but kept the Benchmark tests as-is for now,
to make sure gotest.tools doesn't impact the results.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use t.TempDir()
- use t.Name() instead of hard-coding name
- assert with gotest.tools
- fix some unhandled errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Assert the actual results match the expected one, which should make the
test more complete, and reduces some noise by removing a `t.Log`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pulling some images that share the same content blob but have different chain
IDs caused a panic:
```
panic: runtime error: slice bounds out of range [1:0]
goroutine 318661 [running]:
github.com/docker/docker/daemon/containerd.(*pullProgress).UpdateProgress(0x400fd02d70, {0xaaaada2fda38, 0x400fd02e10}, 0x4019d38810, {0xaaaada2d1640, 0x4018c94600}, {0x0?, 0x0?, 0xaaaadb7c7200?})
/root/build-deb/engine/daemon/containerd/progress.go:232 +0xd84
github.com/docker/docker/daemon/containerd.(*jobs).showProgress.func1()
/root/build-deb/engine/daemon/containerd/progress.go:55 +0x144
created by github.com/docker/docker/daemon/containerd.(*jobs).showProgress in goroutine 318659
/root/build-deb/engine/daemon/containerd/progress.go:48 +0x128
```
The panic was caused by attempting to remove the same committed
layer multiple times from the `p.layers` slice.
This occurred because, in such images, multiple snapshots matched the
same layer by digest rather than by the full layer chain and layer removal
was done by index, leading to repeated deletions at the same index.
This commit:
- Selects a specific snapshot to ensure only one removal per layer.
- Changes snapshot matching to compare the full layer chain instead of
just the layer digest.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
New iptables rules were added by:
- commit 27adcd5 (libnet/d/bridge: port mappings: drop direct-access
when gw_mode=nat)
- commit d216084 (libnet/d/bridge: drop remote connections to port
mapped on lo)
But they're added even when iptables=false or ip6tables=false in daemon
config.
So, group those rules with the other per-port rules, behind a config
check.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Changes in runc code are not impacting code we use;
- libcontainer/utils.MkdirAllInRootOpen is not used
- libcontainer/utils.MkdirAllInRoot is not used
Similarly, while filepath-securejoin is imported, the functions using it
in runc (cgroups.FindCgroupMountpoint, are not used in our codebase, so
these changes don't affect our code; `tryDefaultPath` uses securejoin,
which is used by `FindCgroupMountpoint`, but not used in our codebase.
diffs:
- https://github.com/opencontainers/runc/compare/v1.2.4...v1.2.5
- https://github.com/cyphar/filepath-securejoin/compare/v0.3.5...v0.4.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Move logging out of config.Reload and daemon.Reload itself, as it was not
the right place to know whether it was a "signal" that triggered the reload.
- Use Daemon.Config() to get the new config after reloading. This returns an
immutable copy of the daemon's config, so we can redact fields without having
to use an ad-hoc struct to shadow the underlying fields.
- Use structured logs for logging config reload events.
Before this (plain text):
INFO[2025-02-08T12:13:53.389649297Z] Got signal to reload configuration, reloading from: /etc/docker/daemon.json
INFO[2025-02-08T12:30:34.857691260Z] Reloaded configuration: {"pidfile":"/var/run/docker.pid","data-root":"/var/lib/docker","exec-root":"/var/run/docker","group":"docker","max-concurrent-downloads":3,"max-concurrent-uploads":5,"max-download-attempts":5,"shutdown-timeout":15,"hosts":["unix:///var/run/docker.sock"],"log-level":"info","log-format":"text","swarm-default-advertise-addr":"","swarm-raft-heartbeat-tick":0,"swarm-raft-election-tick":0,"metrics-addr":"","host-gateway-ips":[""],"log-driver":"json-file","mtu":1500,"ip":"0.0.0.0","icc":true,"iptables":true,"ip6tables":true,"ip-forward":true,"ip-masq":true,"userland-proxy":true,"userland-proxy-path":"/usr/local/bin/docker-proxy","default-address-pools":{"Values":null},"network-control-plane-mtu":1500,"experimental":false,"containerd":"/var/run/docker/containerd/containerd.sock","features":{"containerd-snapshotter":false},"builder":{"GC":{},"Entitlements":{}},"containerd-namespace":"moby","containerd-plugin-namespace":"plugins.moby","default-runtime":"runc","runtimes":{"crun":{"path":"/usr/local/bin/crun"}},"seccomp-profile":"builtin","default-shm-size":67108864,"default-ipc-mode":"private","default-cgroupns-mode":"private","resolv-conf":"/etc/resolv.conf","proxies":{}}
Before this (JSON logs):
{"level":"info","msg":"Reloaded configuration: {\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"json\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}","time":"2025-02-08T12:24:38.600761054Z"}
After this (plain text):
INFO[2025-02-08T12:30:34.835953594Z] Got signal to reload configuration config-file=/etc/docker/daemon.json
INFO[2025-02-08T12:30:34.857614135Z] Reloaded configuration config="{\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"text\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}"
After this (JSON logs):
{"config-file":"/etc/docker/daemon.json","level":"info","msg":"Got signal to reload configuration","time":"2025-02-08T12:24:38.589955637Z"}
{"config":"{\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"json\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}","level":"info","msg":"Reloaded configuration","time":"2025-02-08T12:24:38.600736179Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the fifth patch release in the 1.2.z series of runc. It primarily fixes
an issue caused by an upstream systemd bug.
* There was a regression in systemd v230 which made the way we define
device rule restrictions require a systemctl daemon-reload for our
transient units. This caused issues for workloads using NVIDIA GPUs.
Workaround the upstream regression by re-arranging how the unit properties
are defined.
* Dependency github.com/cyphar/filepath-securejoin is updated to v0.4.1,
to allow projects that vendor runc to bump it as well.
* CI: fixed criu-dev compilation.
* Dependency golang.org/x/net is updated to 0.33.0.
full diff: https://github.com/opencontainers/runc/compare/v1.2.4...v1.2.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Checks exhaustiveness of enum switch statements in Go source code;
https://github.com/nishanths/exhaustive
For now allowing "default" to satisfy this, but left TODOs in various
places to make switches actually exhaustive so that we can detect missing
cases when new options are added.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Detects unsupported types passed to json encoding functions and reports
if checks for the returned error can be omitted;
https://github.com/breml/errchkjson
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Detects wasted assignment statements; https://github.com/sanposhiho/wastedassign
For now, ignoring wasted asigns to `ctx` as there were too many hits in
libnetwork in functions that set up spans;
libnetwork/drivers/bridge/bridge_linux.go:1319:2: assigned to ctx, but never used afterwards (wastedassign)
ctx, span := otel.Tracer("").Start(ctx, "libnetwork.drivers.bridge.linkUp", trace.WithAttributes(
^
libnetwork/drivers/bridge/bridge_linux.go:1448:2: assigned to ctx, but never used afterwards (wastedassign)
ctx, span := otel.Tracer("").Start(ctx, "libnetwork.drivers.bridge.Join", trace.WithAttributes(
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This linter has various other options for "correct" use of interfaces,
but those are too disruptive, so only enabling it with the default
settings, which detects duplicate interface definitions withing a
package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
not the latest-latest version, but v1.1.58 is used elsewhere, and I saw
some fixes in v1.1.59 and v1.1.60, and v1.1.61 was docs-only changes.
- Allow use of fs.FS for $INCLUDE and wrap errors
- Add NXT record
- Add ISDN record
- Fix counting of escape sequences when splitting TXT string
- IsDomainName: check for escape as last character
- Add a hook to catch invalid messages
- Fix possible out-of-bounds read in endingToTxtSlice
full diff: https://github.com/miekg/dns/compare/v1.1.57...v1.1.61
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- error (ErrorMessage) was deprecated in 3043c26419
which was part of docker v0.6.0 / API v1.4
- progress (ProgressMessage) was deprecated in 597e0e69b4
which was part of docker v0.7.1 / API v1.8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- ErrorMessage was deprecated in 3043c26419
which was part of docker v0.6.0 / API v1.4
- ProgressMessage was deprecated in 597e0e69b4
which was part of docker v0.7.1 / API v1.8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looking in history to learn why this struct existed, shows that this type
was mostly the result of tech-debt accumulating over time;
- originally ([moby@1aa7f13]) most of the request handling was internal;
the [`call()` function][1] would make a request, read the `response.Body`,
and return it as a `[]byte` (or an error if one happened).
- some features needed the statuscode, so [moby@a4bcf7e] added an extra
output variable to return the `response.StatusCode`.
- some new features required streaming, so [moby@fdd8d4b] changed the
function to return the `response.Body` as a `io.ReadCloser`, instead
of a `[]byte`.
- some features needed access to the content-type header, so a new
`clientRequest` method was introduced in [moby@6b2eeaf] to read the
`Content-Type` header from `response.Headers` and return it as a string.
- of course, `Content-Type` may not be the only header needed, so [moby@0cdc3b7]
changed the signature to return `response.Headers` as a whole as a
`http.Header`
- things became a bit unwieldy now, with the function having four (4) output
variables, so [moby@126529c] chose to refactor this code, introducing a
`serverResponse` struct to wrap them all, not realizing that all these
values were effectively deconstructed from the `url.Response`, so now
re-assembling them into our own "URL response", only preserving a subset
of the information available.
- now that we had a custom struct, it was possible to add more information
to it without changing the signature. When there was a need to know the
URL of the request that initiated the response, [moby@27ef09a] introduced
a `reqURL` field to hold the `request.URL` which notably also is available
in `response.Request.URL`.
In short;
- The original implementation tried to (pre-maturely) abstract the underlying
response to provide a simplified interface.
- While initially not needed, abstracting caused relevant information from
the response (and request) to be unavailable to callers.
- As a result, we ended up in a situation where we are deconstructing the
original `url.Response`, only to re-assemble it into our own, custom struct
(`serverResponsee`) with only a subset of the information preserved.
This patch removes the `serverResponse` struct, instead returning the
`url.Response` as-is, so that all information is preserved, allowing callers
to use the information they need.
There is one follow-up change to consider; commit [moby@589df17] introduced
a `ensureReaderClosed` utility. Before that commit, the response body would
be closed in a more idiomatic way through a [`defer serverResp.body.Close()`][2].
A later change in [docker/engine-api@5dd6452] added an optimization to that
utility, draining the response to allow connections to be reused. While
skipping that utility (and not draining the response) would not be a critical
issue, it may be easy to overlook that utility, and to close the response
body in the "idiomatic" way, resulting in a possible performance regression.
We need to check if that optimization is still relevant or if later changes
in Go itself already take care of this; we should also look if context
cancellation is handled correctly for these. If it's still relevant, we could
- Wrap the the `url.Response` in a custom struct ("drainCloser") to provide
a `Close()` function handling the draining and closing; this would re-
introduce a custom type to be returned, so perhaps not what we want.
- Wrap the `url.Response.Body` in the response returned (so, calling)
`response.Body.Close()` would call the wrapped closer.
- Change the signature of `Client.sendRequest()` (and related) to return
a `close()` func to handle this; doing so would more strongly encourage
callers to close the response body.
[1]: 1aa7f1392d/commands.go (L1008-L1027)
[2]: 589df17a1a/api/client/ps.go (L84-L89)
[moby@1aa7f13]: 1aa7f1392d
[moby@a4bcf7e]: a4bcf7e1ac
[moby@fdd8d4b]: fdd8d4b7d9
[moby@6b2eeaf]: 6b2eeaf896
[moby@0cdc3b7]: 0cdc3b7539
[moby@126529c]: 126529c6d0
[moby@27ef09a]: 27ef09a46f
[moby@589df17]: 589df17a1a
[docker/engine-api@5dd6452]: 5dd6452d4d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Deprecate `api/type.IDResponse` in favor of `container.CommitResponse` and
`container.ExecCreateResponse`, which are currently an alias, but may become
distinct types in a future release. This type will be removed in the next
release.
updates 0df3a0047a
updates 9a20edf7b6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In the bridge network driver, filterPortMappedOnLoopback and
filterDirectAccess each set up some per-port iptables rules - but
they're not called by setPerPortIptables.
So, make sure they're called when firewalld is reloaded.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Introduce a container.ExecCreateResponse type as alias for IDResponse to allow
consumers to use ContainerCommit without having to import the "types" package,
and allows us to differentiate the response for container commit separate from
other endpoints currently using IDResponse.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move api/types.IDResponse to a "common" package (to prevent cyclic import
issues), and introduce a container.CommitResponse type as alias. This allows
consumers to use ContainerCommit without having to import the "types" package,
and allows us to differentiate the response for container commit separate from
other endpoints currently using IDResponse.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The comment was not formatted correctly as it was not the last line,
resulting in some editors / linters not detecting the deprecation.
Updates 639a1214fa
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- un-deprecates `assert.ErrorType`, `cmp.ErrorType`
- assert: ensure message is always displayed and fix under bazel
- poll: Continue(): use format.Message for formatting
- fix TestFromDirSymlink on Windows due to missing drive-letter
- fix various linting issues and minor bugs
full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.5.1...v3.5.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/list.go:126:3: missing cases in switch of type daemon.iterationAction: daemon.includeContainer (exhaustive)
switch includeContainerInList(currentContainer, filter) {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement so that disabling the "default-signifies-exhaustive"
linter option will make it show up.
daemon/monitor.go:158:2: missing cases in switch of type types.EventType: types.EventUnknown, types.EventCreate, types.EventExecAdded, types.EventExecStarted (exhaustive)
switch e {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement so that disabling the "default-signifies-exhaustive"
linter option will make it show up.
daemon/events/events.go:95:2: missing cases in switch of type events.Type: events.BuilderEventType, events.ConfigEventType, events.DaemonEventType, events.NetworkEventType, events.NodeEventType, events.PluginEventType, events.SecretEventType, events.ServiceEventType, events.VolumeEventType (exhaustive)
switch eventType {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement so that disabling the "default-signifies-exhaustive"
linter option will make it show up.
daemon/cluster/convert/service.go:59:3: missing cases in switch of type api.UpdateStatus_UpdateState: api.UpdateStatus_UNKNOWN (exhaustive)
switch s.UpdateStatus.State {
^
daemon/cluster/convert/volume.go:304:2: missing cases in switch of type api.VolumeSpec_VolumeAvailability: api.VolumeAvailabilityDrain (exhaustive)
switch availability {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement so that disabling the "default-signifies-exhaustive"
linter option will make it show up.
daemon/cluster/services.go:560:5: missing cases in switch of type api.LogStream: api.LogStreamUnknown (exhaustive)
switch msg.Stream {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement so that disabling the "default-signifies-exhaustive"
linter option will make it show up.
libnetwork/networkdb/delegate.go:49:2: missing cases in switch of type networkdb.NodeEvent_Type: networkdb.NodeEventTypeInvalid (exhaustive)
switch nEvent.Type {
^
libnetwork/networkdb/delegate.go:223:2: missing cases in switch of type networkdb.TableEvent_Type: networkdb.TableEventTypeInvalid (exhaustive)
switch tEvent.Type {
^
libnetwork/networkdb/nodemgmt.go:51:2: missing cases in switch of type networkdb.nodeState: networkdb.nodeNotFound (exhaustive)
switch newState {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement so that disabling the "default-signifies-exhaustive"
linter option will make it show up.
volume/mounts/linux_parser.go:353:2: missing cases in switch of type mount.Type: mount.TypeNamedPipe, mount.TypeCluster (exhaustive)
switch cfg.Type {
^
volume/mounts/windows_parser.go:392:2: missing cases in switch of type mount.Type: mount.TypeTmpfs, mount.TypeCluster, mount.TypeImage (exhaustive)
switch cfg.Type {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement, and not an explicit "WatchActionKindUnknown",
so that disabling the "default-signifies-exhaustive" linter option will
make it show up.
daemon/events.go:273:2: missing cases in switch of type api.WatchActionKind: api.WatchActionKindUnknown (exhaustive)
switch action {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding a `default` statement, and not an explicit "WaitConditionNextExit",
so that disabling the "default-signifies-exhaustive" linter option will
make it show up.
container/state.go:237:2: missing cases in switch of type container.WaitCondition: container.WaitConditionNextExit (exhaustive)
switch condition {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These need revisiting, similar to similar code in libnetwork/drivers/bridge.
There was already a TODO in place for this, so ignoring these for now.
libnetwork/endpoint.go:148:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
ib, _ := json.Marshal(epMap["ep_iface"])
^
libnetwork/endpoint.go:151:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
jb, _ := json.Marshal(epMap["joinInfo"])
^
libnetwork/endpoint.go:154:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
tb, _ := json.Marshal(epMap["exposed_ports"])
^
libnetwork/endpoint.go:159:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
cb, _ := json.Marshal(epMap["sandbox"])
^
libnetwork/endpoint.go:237:12: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
sal, _ := json.Marshal(epMap["svcAliases"])
^
libnetwork/endpoint.go:242:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
pc, _ := json.Marshal(epMap["ingressPorts"])
^
libnetwork/endpoint.go:247:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
ma, _ := json.Marshal(epMap["myAliases"])
^
libnetwork/endpoint.go:252:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
dn, _ := json.Marshal(epMap["dnsNames"])
^
libnetwork/endpoint_info.go:123:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
rb, _ := json.Marshal(epMap["routes"])
^
libnetwork/endpoint_info.go:490:12: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
tb, _ := json.Marshal(v)
^
libnetwork/network.go:161:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
b, _ := json.Marshal(v)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Log errors, but continue, to keep the existing behavior, but add a TODO
to revisit the logic.
libnetwork/drivers/bridge/bridge_store.go:316:10: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
d, _ := json.Marshal(epMap["ContainerConfig"])
^
libnetwork/drivers/bridge/bridge_store.go:320:9: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
d, _ = json.Marshal(epMap["ExternalConnConfig"])
^
libnetwork/drivers/bridge/bridge_store.go:324:9: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
d, _ = json.Marshal(epMap["PortMapping"])
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/templates/templates.go:17:3: Error return value of `(*encoding/json.Encoder).Encode` is not checked: unsafe type `interface{}` found (errchkjson)
enc.Encode(v)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Print a warning when failing to marshal the JSON for printing
daemon/reload.go:118:19: Error return value of `encoding/json.Marshal` is not checked: unsafe type `net.IP` found (errchkjson)
jsonString, _ := json.Marshal(&struct {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function describes that errors are OK to ignore, so we can suppress
the linter warning.
daemon/cluster/convert/container.go:582:18: Error return value of `encoding/json.Marshal` is not checked (errchkjson)
jsonBytes, _ := json.Marshal(options)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/plugin/logging/cmd/discard/driver.go:84:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(w).Encode(&res)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/docker_cli_external_volume_driver_test.go:116:4: Error return value of `(*encoding/json.Encoder).Encode` is not checked: unsafe type `interface{}` found (errchkjson)
json.NewEncoder(w).Encode(&data)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/driverapi/ipamdata.go:52:11: Error return value of `encoding/json.Marshal` is not checked: unsafe type `interface{}` found (errchkjson)
b, _ := json.Marshal(v)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/diagnostic/server.go:234:19: Error return value of `encoding/json.MarshalIndent` is not checked: unsafe type `github.com/docker/docker/libnetwork/diagnostic.StringInterface` found (errchkjson)
response, _ = json.MarshalIndent(FailCommand(err), "", " ")
^
libnetwork/diagnostic/server.go:239:19: Error return value of `encoding/json.Marshal` is not checked: unsafe type `github.com/docker/docker/libnetwork/diagnostic.StringInterface` found (errchkjson)
response, _ = json.Marshal(FailCommand(err))
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also fixing some minor other linting issues.
api/server/router/volume/volume_routes_test.go:193:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
e.Encode(volumeCreate)
^
api/server/router/volume/volume_routes_test.go:231:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(&buf).Encode(volumeCreate)
^
api/server/router/volume/volume_routes_test.go:260:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(&buf).Encode(volumeCreate)
^
api/server/router/volume/volume_routes_test.go:292:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(&buf).Encode(volumeCreate)
^
api/server/router/volume/volume_routes_test.go:339:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(&buf).Encode(volumeUpdate)
^
api/server/router/volume/volume_routes_test.go:366:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(&buf).Encode(volumeUpdate)
^
api/server/router/volume/volume_routes_test.go:396:2: Error return value of `(*encoding/json.Encoder).Encode` is not checked (errchkjson)
json.NewEncoder(&buf).Encode(volumeUpdate)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Change some variables to a time.Duration to reduce conversions between
integers and durations, which also makes the code slightly more transparent.
pkg/plugins/client_test.go:109:9: Multiplication of durations: `tc.expTimeOff * time.Second` (durationcheck)
s := tc.expTimeOff * time.Second
^
pkg/plugins/client_test.go:132:9: Multiplication of durations: `tc.timeOff * time.Second` (durationcheck)
s := tc.timeOff * time.Second
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/jsonmessage/jsonmessage.go:111:10: Multiplication of durations: `(left / time.Second) * time.Second` (durationcheck)
left = (left / time.Second) * time.Second
^
This code was rounding down time remaining to the nearest second;
- Use go's time.Duration.Round() instead
- Make the calculation conditional, as it was only used if there's enough
space available to print
- Move the declaration of the timeLeftBox var closer to where used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/plugins/pluginrpc-gen/parser_test.go:80:2: assigned to arg, but reassigned without using the value (wastedassign)
arg = f.Args[0]
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/containerd/image_exporter.go:406:2: assigned to msg, but reassigned without using the value (wastedassign)
msg := ""
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add some assertions, and removed a log, instead using it as message
if the assertion fails.
integration/image/remove_test.go:88:2: assigned to inspect, but reassigned without using the value (wastedassign)
inspect, err = client.ImageInspect(ctx, "busybox")
^
integration/image/remove_test.go:91:2: assigned to inspect, but never used afterwards (wastedassign)
inspect, err = client.ImageInspect(ctx, "test-remove-by-digest")
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This needs a better solution, but this allows enabling the "fatcontext"
linter.
plugin/fetch_linux.go:250:6: nested context in loop (fatcontext)
ctx = context.Background()
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
opts/hosts_test.go:110:30: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
"tcp://:5555": fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Changing to use binary.LittleEndian.AppendUint64, which does not require
the slice to have an initial size, and makes the code slightly more
straightforward.
libnetwork/bitmap/sequence.go:296:7: append to slice `ba` with non-zero initialized length (makezero)
ba = append(ba, bm...)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Changing to use binary.LittleEndian.AppendUint64, which does not require
the slice to have an initial size, and makes the code slightly more
straightforward.
libnetwork/internal/kvstore/boltdb/boltdb.go:79:11: append to slice `dbval` with non-zero initialized length (makezero)
dbval = append(dbval, value...)
^
libnetwork/internal/kvstore/boltdb/boltdb.go:228:11: append to slice `dbval` with non-zero initialized length (makezero)
dbval = append(dbval, value...)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
errdefs/helpers_test.go:11:6: identical: interface causal contains identical methods or type constraints from another interface, causing redundancy (iface)
type causal interface {
^
errdefs/is.go:8:6: identical: interface causer contains identical methods or type constraints from another interface, causing redundancy (iface)
type causer interface {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These consts were deprecated in 9c368a93b6, but are
used externally and lack a canonical location. These sids are "special", as they
are available by default in Windows containers, but we need to;
- Reference official documentation / specification for that.
- Add names (not just the sid)
- Consider finding a canonical location for these consts, which could be as part
of the OCI specs, or hcsshim (or otherwise).
Lacking a good place for these, let's un-deprecate them for the time being until
we decided what's the best location for these.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
At first glance, it seemed like the Namespace lock was held to make
local copies of Namespace properties, but all those properties all safe
to access concurrently. So, un-alias those props and reduce the scope
of the mutex lock.
This helps understanding where locking is really needed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
To support this, a new netlabel is added: `com.docker.network.endpoint.ifname`.
It gives the ability to specify the interface name to be set by
netdrivers when the interface is added / moved into the container's
network namespace.
All builtin netdrivers support it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Before this commit, `Interface.dstName` was used to store the dest
ifname prefix, and then the final ifname.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
- Move the "Daemon has completed initialization" log to where it has
actually completed initialization.
- Move buildkit init to its own function.
- Move the builder options to a separate struct, and change initBuildkit
to return it instead of passing the router-options and manipulate it.
Co-authored-by: Brian Goff <cpuguy83@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `Manifests` field to image inspect (`/images/{name}/json`) response.
This is the same as in `/images/json`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Deprecate ImageInspectWithRaw and add a simpler ImageInspect function
which takes optional options.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't use the `GetImage` call which returns a "best-effort" view of the
image that is compatible with the old images.Image response.
Instead, use the multi-platform view of the image to construct the
inspect response.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit aa3a23d (Temporary debug for unsolicited NA) added code to
check for a route from ::1 to ff02::1 if a neighbour advertisement
send failed ... hoping to show that the route appeared fairly quickly
after the first attempt.
The failure didn't happen in CI, perhaps other changes in that PR
changed the timing enough for it not to be an issue. Or, perhaps the
change in commit 9a6e96f (Before sending ARPs/NAs, check the bridge is
ready) was enough to solve the issue.
But, in case it's a timing issue we're just no longer seeing on the CI
hosts - check the route exists before trying to send an NA, and wait
for a short time if not.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Commit 38e76eb (Ask network drivers if they'll use a gateway
address) added an optional driver interface "GwAllocChecker"
to give the driver a chance to say whether, given network
config options, it would use a gateway address if one was
reserved for it in IPAM.
So, implement support for that in the remote network driver.
The driver itself implements the interface, but only tries
to make an HTTP request to the driver plugin if the plugin
has reported support for it in response to an initial
capabilities request.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This minor release include 1 security fix following the security policy:
- crypto/elliptic: timing sidechannel for P-256 on ppc64le
Due to the usage of a variable time instruction in the assembly implementation
of an internal function, a small number of bits of secret scalars are leaked on
the ppc64le architecture. Due to the way this function is used, we do not
believe this leakage is enough to allow recovery of the private key when P-256
is used in any well known protocols.
This is CVE-2025-22866 and Go issue https://go.dev/issue/71383.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.23.6
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This error implemented the Causer interface, but did not implement
the go1.13 unwrapper, which could prevent errors from being matched.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These errors implemented the Causer interface, but did not implement
the go1.13 unwrapper, which could prevent errors from being matched.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This error implemented the Causer interface, but did not implement
the go1.13 unwrapper, which could prevent errors from being matched.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The CommonAPIClient was used to define all the stable interfaces,
and combined with the experimental ones through APIClient. In theory,
this would allow someone to make sure they only depended on non-experimental
methods or to implement an alternative client that only implements the
stable methods.
While there are users currently using this interface, all those uses
depend on the actual client implementation, not a custom one, so they
should be able to switch to use APIClient instead. In the meantime,
start with deprecating, but keeping the interface the same for now,
scheduling it to become an alias, and removed in a future release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Introduce a SwarmManagementAPIClient interface that captures
all swarm-specific methods on the API client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In situations where an empty ID was passed, the client would construct an
invalid API endpoint URL, which either resulted in the "not found" handler
being hit (resulting in a "page not found" error), or even the wrong endpoint
being hit if the client follows redirects.
For example, `/containers/<empty id>/json` (inspect) redirects to `/containers/json`
(docker ps))
Given that empty IDs should never be expected (especially if they're part of
the API URL path), we can validate these and return early.
Its worth noting that a few methods already had an error in place; those
methods were related to the situation mentioned above, where (e.g.) an
"inspect" would redirect to a "list" endpoint. The existing errors, for
convenience, mimicked a "not found" error; this patch changes such errors
to an "Invalid Parameter" instead, which is more correct, but it could be
a breaking change for some edge cases where users parsed the output;
git grep 'objectNotFoundError{'
client/config_inspect.go: return swarm.Config{}, nil, objectNotFoundError{object: "config", id: id}
client/container_inspect.go: return container.InspectResponse{}, nil, objectNotFoundError{object: "container", id: containerID}
client/container_inspect.go: return container.InspectResponse{}, objectNotFoundError{object: "container", id: containerID}
client/distribution_inspect.go: return distributionInspect, objectNotFoundError{object: "distribution", id: imageRef}
client/image_inspect.go: return image.InspectResponse{}, nil, objectNotFoundError{object: "image", id: imageID}
client/network_inspect.go: return network.Inspect{}, nil, objectNotFoundError{object: "network", id: networkID}
client/node_inspect.go: return swarm.Node{}, nil, objectNotFoundError{object: "node", id: nodeID}
client/plugin_inspect.go: return nil, nil, objectNotFoundError{object: "plugin", id: name}
client/secret_inspect.go: return swarm.Secret{}, nil, objectNotFoundError{object: "secret", id: id}
client/service_inspect.go: return swarm.Service{}, nil, objectNotFoundError{object: "service", id: serviceID}
client/task_inspect.go: return swarm.Task{}, nil, objectNotFoundError{object: "task", id: taskID}
client/volume_inspect.go: return volume.Volume{}, nil, objectNotFoundError{object: "volume", id: volumeID}
Two such errors are still left, as "ID or name" would probably be confusing,
but perhaps we can use a more generic error to include those as well (e.g.
"invalid <object> reference: value is empty");
client/distribution_inspect.go: return distributionInspect, objectNotFoundError{object: "distribution", id: imageRef}
client/image_inspect.go: return image.InspectResponse{}, nil, objectNotFoundError{object: "image", id: imageID}
Before this patch:
docker container start ""
Error response from daemon: page not found
Error: failed to start containers:
docker container start " "
Error response from daemon: No such container:
Error: failed to start containers:
With this patch:
docker container start ""
invalid container name or ID: value is empty
Error: failed to start containers:
docker container start " "
invalid container name or ID: value is empty
Error: failed to start containers:
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch, an API response that's valid JSON, but not the right
schema would be silently discarded by the CLI. For example, due to a bug
in Docker Desktop's API proxy, the "normal" (not JSON error) response
would be returned together with a non-200 status code when using an
unsupported API version;
curl -s -w 'STATUS: %{http_code}\n' --unix-socket /var/run/docker.sock 'http://localhost/v1.99/version'
{"Platform":{"Name":"Docker Desktop 4.38.0 (181016)"},"Version":"","ApiVersion":"","GitCommit":"","GoVersion":"","Os":"","Arch":""}
STATUS: 400
Before this patch, this resulted in no output being shown;
DOCKER_API_VERSION=1.99 docker version
Client:
Version: 27.5.1
API version: 1.99 (downgraded from 1.47)
Go version: go1.22.11
Git commit: 9f9e405
Built: Wed Jan 22 13:37:19 2025
OS/Arch: darwin/arm64
Context: desktop-linux
Error response from daemon:
With this patch, an error is generated based on the status:
DOCKER_API_VERSION=1.99 docker version
Client:
Version: 27.5.1
API version: 1.99 (downgraded from 1.47)
Go version: go1.22.11
Git commit: 9f9e405
Built: Wed Jan 22 13:37:19 2025
OS/Arch: darwin/arm64
Context: desktop-linux
Error response from daemon: API returned a 400 (Bad Request) but provided no error-message
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We don't enable STP on bridges belonging to bridge networks,
but bridge ports still need to transition from "disabled" to
"forwarding", after the veth device comes "up". Until then,
the bridge will just drop packets.
So, if a container's network is a veth device, and its other
end is slaved to a bridge - wait for the bridge port to
be "forwarding".
Signed-off-by: Rob Murray <rob.murray@docker.com>
This wires up the new gc types that buildkit exposes in version 0.17.
The previous flag, `KeepBytes`, was renamed to `ReservedBytes` and two
new options, `MaxUsed` and `MinFree` were added.
`MaxUsed` corresponds to the maximum amount of space that buildkit will
use for the build cache and `MinFree` amount of free disk space for the
system to prevent the cache from using that space. This allows greater
configuration of the cache storage usage when used in situations where
docker is not the only service on the system using disk space.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Make it more clear that this loop is for legacy-links, and the timer is
only needed for that purpose.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When failing to resolve the network container, a "not found" error should
not return a 404, but either a "invalid parameter" (400) or "system" (500)
error.
Given that this function is called on container start, not container create,
a 500 (internal server error) is more appropriate, because the API request
(start the container) is valid, but the state of the container isn't.
While working on this, I discovered that no validation happens during container
create; TODO's were added to look into that, but this may be partially
by design (allow a container to be created before the "donor" network
container is created).
Before this patch:
docker container create --name hello --network=container:nosuchcontainer alpine
docker container start hello
Error response from daemon: No such container: nosuchcontainer
Error: failed to start containers: hello
# daemon logs:
DEBU[2025-01-30T11:32:33.595636043Z] error response for POST request error-response="No such container: nosuchcontainer" method=POST module=api request-url=/v1.47/containers/hello/start status=404 vars="map[name:hello version:1.47]"
docker container create --name hello2 --network=container:hello2 alpine
docker container start hello2
Error response from daemon: cannot join own network
Error: failed to start containers: hello2
# daemon logs:
DEBU[2025-01-30T11:33:19.545287551Z] FIXME: Got an API for which error does not match any expected type!!! error="cannot join own network" error_type="*errors.errorString" module=api
DEBU[2025-01-30T11:33:19.545346093Z] error response for POST request error-response="cannot join own network" method=POST module=api request-url=/v1.47/containers/hello2/start status=500 vars="map[name:hello2 version:1.47]"
DEBU[2025-01-30T11:33:19.545369968Z] FIXME: Got an API for which error does not match any expected type!!! error="cannot join own network" error_type="*errors.errorString" module=api
ERRO[2025-01-30T11:33:19.545375426Z] Handler for POST /v1.47/containers/hello2/start returned error: cannot join own network
With this patch:
docker container create --name hello --network=container:nosuchcontainer alpine
docker container start hello
Error response from daemon: joining network of container: No such container: nosuchcontainer
Error: failed to start containers: hello
# daemon logs:
DEBU[2025-01-30T11:35:50.406462760Z] error response for POST request error-response="joining network of container: No such container: nosuchcontainer" method=POST module=api request-url=/v1.47/containers/hello/start status=500 vars="map[name:hello version:1.47]"
ERRO[2025-01-30T11:35:50.406501468Z] Handler for POST /v1.47/containers/hello/start returned error: joining network of container: No such container: nosuchcontainer
docker container create --name hello2 --network=container:hello2 alpine
docker container start hello2
Error response from daemon: cannot join own network namespace
Error: failed to start containers: hello2
# daemon logs:
DEBU[2025-01-30T11:36:15.178475049Z] error response for POST request error-response="cannot join own network" method=POST module=api request-url=/v1.47/containers/hello2/start status=500 vars="map[name:hello2 version:1.47]"
ERRO[2025-01-30T11:36:15.178536507Z] Handler for POST /v1.47/containers/hello2/start returned error: cannot join own network
docker run --name exitedcontainer alpine
docker run --rm --network=container:exitedcontainer alpine
docker: Error response from daemon: cannot join network namespace of a non running container: container exitedcontainer is exited.
# daemon logs:
DEBU[2025-01-30T12:54:28.040637429Z] error response for POST request error-response="cannot join network namespace of a non running container: container exitedcontainer is exited" method=POST module=api request-url=/v1.47/containers/hello2/start status=409 vars="map[name:hello2 version:1.47]"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
rmLink already looked up the parent container's ID, so we should not use
daemon.GetContainer to resolve the container, as that performs fuzzy
matching (name, ID-prefix, or ID).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was the only use of errdefs inside api/types, which is the package
that's imported by external users.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While there may be reasons to keep pkg/errors in production
code, we don't need them for these tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was the only use of github.com/pkg/errors inside api/types,
which is the package that's imported by external users.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It can only be set on a "--internal" network, and the only difference is
that no address is assigned to the bridge. So, there's no route to a
host address.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When overriding the API version through DOCKER_API_VERSION, no validation
happens on the given version. However, some code-paths in the client do
some minor normalizing, and strip the "v" prefix (if present) as part of
[`Client.getAPIPath()`][1].
This resulted in some inconsistent handling of the version that's set. For
example, [`Client.checkResponseErr()`][2] decides whether or not the API
response is expected to support errors in JSON format (`types.ErrorResponse`),
which would fail because `versions.GreaterThan()` does not strip the prefix,
therefore making the first element "zero" (ranking lower than any valid version).
Net result was "mixed" because of this; for example in the following, half
the output is handled correctly ("downgraded from 1.47"), but the response
is handled as < 1.23 (so printed as-is);
DOCKER_API_VERSION=v1.23 docker version
Client: Docker Engine - Community
Version: 27.5.1
API version: v1.23 (downgraded from 1.47)
Go version: go1.22.11
Git commit: 9f9e405
Built: Wed Jan 22 13:41:13 2025
OS/Arch: linux/amd64
Context: default
Error response from daemon: {"message":"client version 1.23 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version"}
Passing the version without v-prefix corrects this problem;
DOCKER_API_VERSION=1.23 docker version
Client: Docker Engine - Community
Version: 27.5.1
API version: 1.99 (downgraded from 1.47)
Go version: go1.22.11
Git commit: 9f9e405
Built: Wed Jan 22 13:41:13 2025
OS/Arch: linux/amd64
Context: default
Error response from daemon: client version 1.99 is too new. Maximum supported API version is 1.47
DOCKER_API_VERSION=v1.99 docker version
Client: Docker Engine - Community
Version: 27.5.1
API version: v1.99 (downgraded from 1.47)
Go version: go1.22.11
Git commit: 9f9e405
Built: Wed Jan 22 13:41:13 2025
OS/Arch: linux/amd64
Context: default
Error response from daemon: {"message":"client version 1.99 is too new. Maximum supported API version is 1.47"}
This patch strips the prefix when setting a custom version, so that
normalization happens consistently. The existing code to strip the
prefix in [`Client.getAPIPath()`][1] is kept for now, in case values
are set through other ways.
[1]: 47dc8d5dd8/client/client.go (L303-L309)
[2]: 47dc8d5dd8/client/request.go (L231-L241)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These errors implement errdefs interfaces, and are only used internally
for convenience. Un-export their implemetations because the types themselves
are not used as sentinel errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only used in a single place, and a generic errdefs.ErrInvalid; the
type itself was not used as sentinel error other than for a unit test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's a generic errdefs.ErrInvalidParameter, and the type itself is not
used as sentinel error anywhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's a generic errdefs.ErrInvalidParameter, and the type itself is not
used as sentinel error anywhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `InternalError()` method was added in [moby/libnetwork@50964c9] to
classify the error. However, the same commit defined interfaces for error
types (in the types package). The [InternalError] interface defined did
not match, as it defines a `Internal()` method instead of `InternalError()`.
In short; these errors were never matching any interface, and the actual
error implementations themselves were also never used as a sentinel error,
so we can safely remove these methods.
[moby/libnetwork@50964c9]: 50964c9948
[InternalError]: 50964c9948/types/types.go (L233-L237)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes the following errors, which were not used anywhere;
- ErrConfigExists
- ErrInvalidNetworkConfig
- ErrInvalidEndpointConfig
- ErrNetworkExists
- ErrIfaceName
- ErrUnsupportedAddressType
- NonDefaultBridgeNeedsIPError
- IPv6AddrNoMatchError
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's a generic error, doesn't implement an errdefs type, is poorly formatted,
and not used as sentinel error anywhere. Let's remove it, and update the error
produced to be slightly more informative (include the invalid type). Worth
noting that there's no need to include the name of the driver in the error,
because the only uses of the error (in `registerNetworkDrivers`) already
decorates it; 5fd7ed2937/libnetwork/drivers_linux.go (L34-L36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only used in a single place, and did not implement an errdef;
the type itself was not used as sentinel error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use Cobra-generated completion scripts for the CLI inside the dev
container shell.
Remove `DOCKER_BASH_COMPLETION_PATH`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The only real purpose of strslice.StrSlice is to provide a custom
json.Unmarshaler implementation for API responses. For all other purposes,
it's a regular string-slice.
This patch removes uses of this type in cases where the custom json.Unmarshaler
is irrelevant; in most cases this was in tests, where results were tested
using "DeepEquals"; for those tests, the type-assertion did not add real
value, so we can cast the values to a []string instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Traditionally when Linux receives remote packets with daddr set to a
loopback address, it reject them as 'martians'. However, when a NAT rule
is applied through iptables this doesn't happen. Our current DNAT rule
used to map host ports to containers is applied unconditionally, even
for such 'martian' packets.
This means a neighbor host (ie. a host connected to the same L2
segment) can send packets to a port mapped on a loopback address. The
purpose of publishing on a loopback address is to make ports
inaccessible to remote hosts -- lack of proper filtering defeats that.
This commit adds an iptables rule to the raw-PREROUTING chain to drop
packets with a loopback dest address and coming from any interface other
than lo.
To accomodate WSL2 mirrored mode, another rule is inserted beforehand to
specifically accept packets coming from the loopback0 interface.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When a NAT-based port mapping is created, the daemon adds a DNAT rule in
nat-DOCKER to replace the dest addr with the container IP. However, the
daemon never sets up rules to filter packets destined directly to the
container port. This allows a rogue neighbor (ie. a host that shares a
L2 segment with the host) to send packets directly to the container on
its container-side exposed port.
For instance, if container port 5000 is mapped to host port 6000, a
neighbor could send packets directly to the container on its port 5000.
Since nat-DOCKER mangles the dest addr, and the nat table forbids DROP
rules, this change adds a new rule in the raw-PREROUTING chain to filter
ingress connections targeting the container's IP address.
This filtering is only done when gw_mode=nat. For the unprotected
variant, no filtering is done.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Commit fc7caf96d reverted 433b1f9b1 as it was introducing a regression,
ie. containers couldn't reach ports published on the host using their
gateway's IP address or the host IP address.
These scenarios are now tested.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
It was only returned in a few places, and not used any different than
a "invalid parameter" error, so let's use a standard errdefs.ErrInvalidParameter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only returned in a few places, and not used any different than
a "invalid parameter" error, so let's use a standard errdefs.ErrInvalidParameter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only returned in 1 place, and not used any different than
a "notfound" error, so let's use a standard errdefs.ErrNotFound
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test has failed a couple of times in CI, but can't repro locally.
Let's find out whether there are any clues in the daemon log.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The Network field is the only field used from the NetworkAttachment within
this code. Remove the NetworkAttachment envelope, and store the Network
field directly instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method was called in a loop, iterating over the container config's
network-attachments. It was defined as a method, but only to lookup
the same attachment we just iterated over existed, and to obtain a copy.
As there were no goroutines involved, dereferencing should not be an issue
and with Go 1.22+, dereferencing loop vars is no longer needed at all,
so we can change this method to a regular constructor; also removing the
redundant error-return in the process.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
All of this function only referenced the Network field in the attachment;
use an intermediate variable to make the code less repetitive.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
c is used as name for the containerConfig receiver; remove the intermediate
variable so that we don't shadow it. There's no bug here, because a new
variable is created; just to prevent confusion and to make linters happier.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 80c44b4b2e removed uses of this
method and added a comment that it should never be used;
> EndpointByID should *never* be called as it's going to create a 2nd instance
> of an Endpoint. The first one lives in the Sandbox the endpoint is attached to.
> Instead, the endpoint should be retrieved by calling [Sandbox.Endpoints()].
Given that the only use of this method is in tests, we can remove if altogether.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only returned in 2 places, and not used any different than
a "notfound" error, so let's use a standard errdefs.NotFound
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This required a minor hack to accommodate Cobra's logic (and a TODO was
added to see if we can improve that logic in upstream). Some changes also
had to be made to our "Usage" template, as our custom template did not
take into account subcommands and long descriptions for commands. We
should review these templates, as some additional features were added
in upstream Cobra that we currently may not be using.
With this patch:
instructions for the (hidden) completion subcommand:
docker completion --help
Usage: docker completion COMMAND
Generate the autocompletion script for docker for the specified shell.
See each sub-command's help for details on how to use the generated script.
Commands:
bash Generate the autocompletion script for bash
fish Generate the autocompletion script for fish
powershell Generate the autocompletion script for powershell
zsh Generate the autocompletion script for zsh
Run 'docker completion COMMAND --help' for more information on a command.
and instructions for installing:
dockerd completion bash --help
Usage: dockerd completion bash
Generate the autocompletion script for the bash shell.
This script depends on the 'bash-completion' package.
If it is not installed already, you can install it via your OS's package manager.
To load completions in your current shell session:
source <(dockerd completion bash)
To load completions for every new session, execute once:
#### Linux:
dockerd completion bash > /etc/bash_completion.d/dockerd
#### macOS:
dockerd completion bash > $(brew --prefix)/etc/bash_completion.d/dockerd
You will need to start a new shell for this setup to take effect.
Options:
--help Print usage
--no-descriptions disable completion descriptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This helper was not marked as t.Helper(), and didn't do a great job at
actually asserting the expected output streams. The existing checks
could've been replaced by a io.Readall to do a fuzzy check for the
response, but let's abstract away what it's doing, and properly check
both STDOUT and STDERR.
make BIND_DIR=. TEST_FILTER='TestTemplated' DOCKER_GRAPHDRIVER=vfs test-integration
...
INFO: Testing against a local daemon
=== RUN TestTemplatedConfig
--- PASS: TestTemplatedConfig (11.73s)
PASS
INFO: Testing against a local daemon
=== RUN TestTemplatedSecret
--- PASS: TestTemplatedSecret (11.04s)
PASS
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function had some LCOW remnants, where the assumption was made
the only on Windows, the image's OS could potentially not match the
host's OS (see 3e6a13ccb8).
While we currently are not able to run a Windows image on Linux (or
vice versa), this function doesn't have to take into account;
- If a shell is configured; use whatever is configured
- otherwise, use "cmd.exe" for Windows images, and "/bin/sh" otherwise
(likely Linux, but the existing code did not account for other platforms).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was not using the daemon, so can be a regular function. While at it,
also changed the parameter type to accept a regular string-slice, as
we don't need strslice.StrSlice's json.Unmarshaler implementation, and
reversed the logic for the early return.
Finally, for uses where the entrypoint was always nil, this patch removes
the use of this utility altogether.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since commit 933fcc9 (Re-remove the SetKey OCI prestart hook),
the network namespace will be set up before endpoints are
created in most cases, apart from build containers.
So, when possible, create the veth with one end in that netns
to save moving it in later. On my host, that saves about 20ms
for each bridge network a container is connected to.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Prefering Check here over NilError so that all defers
wil be executed, instead of potentially failing on the
first one.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now a gratuitous/unsolicted ARP is sent, there's no need to
use an IPv4-based MAC address to preserve arp-cache mappings
between an endpoint's IP addresses and its MAC addresses.
Because a random MAC address is used for the default bridge,
it no longer makes sense to derive container IPv6 addresses
from the MAC address. This "postIPv6" behaviour was needed
before IPv6 addresses could be configured, but not now. So,
IPv6 addresses will now be IPAM-allocated on the default
bridge network, just as they are for user-defined bridges.
Signed-off-by: Rob Murray <rob.murray@docker.com>
With randomly-generated MAC addresses going into wider use in the most
commonly used container network drivers, it is imperative to avoid
address collisions. Increase the number of random bits in randomly-
generated MAC addresses from 32 bits to 46 bits.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The default is to send 3 messages at 1s intervals.
That can be overridden in "docker network create" using:
-o com.docker.network.advertise_addr_nmsgs=3
-o com.docker.network.advertise_addr_ms=1000
Or, in daemon.json for each driver:
"default-network-opts": {
"bridge": {
"com.docker.network.advertise_addr_nmsgs": "3",
"com.docker.network.advertise_addr_ms": "1000"
}
}
The allowed range is 0-3 for the number of messages, and
100-2000ms for the interval. Setting nmsgs to 0 disables the
gratuitous ARP/NA messages.
The default bridge will always use the built-in defaults,
it cannot be configured.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Both net.IP and net.IPMask implement a stringer interface; use it to print
their value. While updating also removed the naked return.
Before this patch:
cannot compute host portion ip address because ip and mask are not compatible: (net.IP{0x20, 0x1, 0xd, 0xb8, 0x20, 0x2, 0x20, 0x1, 0xff, 0xff, 0xab, 0xcd, 0xee, 0xab, 0x0, 0xcd}, net.IPMask{0xff, 0xff, 0xff, 0x0})
cannot compute host portion ip address because ip and mask are not compatible: (net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xad, 0x20, 0x4, 0x5}, net.IPMask{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xff, 0x0})
cannot compute host portion ip address because ip and mask are not compatible: (net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xad, 0x20, 0x4, 0x5}, net.IPMask{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xff, 0xff, 0xff, 0x0})
With this patch:
cannot compute host portion ip address because ip and mask are not compatible: (2001:db8:2002:2001:ffff:abcd:eeab:cd, ffffff00)
cannot compute host portion ip address because ip and mask are not compatible: (173.32.4.5, 000000000000000000000000ffffff00)
cannot compute host portion ip address because ip and mask are not compatible: (173.32.4.5, 00000000000000000000ffffffffff00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rewrite the test using gotest.tools and use the errdefs helpers to assert
the right errdefs definition.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure these errors are properly detected by the errdefs helpers to
implement the right interface / definition.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types were identical to errdefs definitions; make
them an alias and use errdefs utilities to produce them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The default IPAM allocator is unable to represent subnets larger than
64-bits (subnets with a smaller prefix), because it uses a Bitmap
that's limited to 64-bits.
When it's used to represent a 64-bit subnet, the top address can't
be allocated (because bitmap.Bitmap is initialised with the number
of bits it needs to represent in a uint64, so it's one short).
The rest of the daemon doesn't know about these limitations, so
strange things happen when a large IPv6 subnet is used.
No errors are reported, addresses/subnets are just set up incorrectly.
The IPAM code calculates offsets into the bitmap itself, details it
shouldn't need to understand and, because it's working on offsets
into a range it doesn't always notice when it's asked to set a bit
outside the range.
It's unusual to need a big subnet but, for example, it may be useful
for modelling an ISP network, or an ISP's gateway may be in a "/56"
subnet that's outside a 64-bit range used by hosts.
So, use addrset.AddrSet instead of bitmap.Bitmap.
Signed-off-by: Rob Murray <rob.murray@docker.com>
go1.23.5 (released 2025-01-16) includes security fixes to the crypto/x509 and
net/http packages, as well as bug fixes to the compiler, the runtime, and the
net package. See the Go 1.23.5 milestone on our issue tracker for details;
- https://github.com/golang/go/issues?q=milestone%3AGo1.23.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.23.4...go1.23.5
Hello gophers,
We have just released Go versions 1.23.5 and 1.22.11, minor point releases.
These minor releases include 2 security fixes following the security policy:
- crypto/x509: usage of IPv6 zone IDs can bypass URI name constraints
A certificate with a URI which has a IPv6 address with a zone ID may
incorrectly satisfy a URI name constraint that applies to the certificate
chain.
Certificates containing URIs are not permitted in the web PKI, so this
only affects users of private PKIs which make use of URIs.
Thanks to Juho Forsén of Mattermost for reporting this issue.
This is CVE-2024-45341 and Go issue https://go.dev/issue/71156.
- net/http: sensitive headers incorrectly sent after cross-domain redirect
The HTTP client drops sensitive headers after following a cross-domain redirect.
For example, a request to a.com/ containing an Authorization header which is
redirected to b.com/ will not send that header to b.com.
In the event that the client received a subsequent same-domain redirect, however,
the sensitive headers would be restored. For example, a chain of redirects from
a.com/, to b.com/1, and finally to b.com/2 would incorrectly send the Authorization
header to b.com/2.
Thanks to Kyle Seely for reporting this issue.
This is CVE-2024-45336 and Go issue https://go.dev/issue/70530.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If a container was started with
- a numeric uid
- both a user and group (username:groupname)
- uid and gid (uid:gid)
The copy action failed, because the "username:groupname"
was looked up using getent.
This patch;
- splits `user` and `group` before looking up
- if numeric `uid` or `gid` is used and lookup fails,
the `uid` / `gid` is used as-is
The code also looked up the user and group on the host
instead of in the container when using getent; this patch
fixes the lookup to only use the container's /etc/passwd
and /etc/group instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rewrite the range-detection logic to prevent duplicate env-vars,
and to avoid looping over the same values multiple times.
Benchmark before / after:
goos: darwin
goarch: arm64
pkg: github.com/docker/docker/daemon/links
cpu: Apple M1 Pro
BenchmarkLinkMultipleEnv
BenchmarkLinkMultipleEnvOld-10 92817 12072 ns/op 8516 B/op 316 allocs/op
BenchmarkLinkMultipleEnvNew-10 149493 7792 ns/op 6435 B/op 213 allocs/op
PASS
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The code incorrectly created env-vars for consecutive port numbers with
a different protocol; we should only consider ports to be part of a range
if they have consecutive port-numbers and have the same protocol.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The intent of this sorting was twofold;
- the "default" port of the container to be the first TCP port (if present)
- consecutive port-mappings with the same protocol to be together so
that port-ranges would produce an env-var describing the range.
The current sorting would sort TCP ports before UDP (or SCTP) port, but
only if they had the same port-number. This could result in range-detection
incorrectly combining TCP and UDP (or SCTP) ports in the same range.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There's no need to loop and sort multiple times; this code picked
the first port after sorting, which we already did in this function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was unconditionally trying to fetch linked container, even
if the container was not using the default bridge (the only network that
supports legacy links).
Also removing the intermediate variable, as daemon.children, through
daemon.linkindex.children already returns a variable with a copy of these
links.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
with this patch:
dockerd --network-diagnostic-port -1 --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid network-diagnostic-port (-1): value must be between 0 and 65535
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
diagnosticServer is only written to during controller.New, and the
diagnostic server itself already has a mutex on EnableDiagnostic,
DisableDiagnostic, and IsDiagnosticEnabled, which should prevent
issues trying to concurrently change its state.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Handle situations where the server is already started or stopped internally,
instead of requiring the caller to do this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Handle situations where the server is already stopped internally, instead
of requiring the caller to do this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The response would not have a trailing newline, which made it difficult
to copy the path. While updating, also include the path of the stackdump
in the daemon log that's produced.
Before this:
root@fa87ff1bcd00:/go/src/github.com/docker/docker# curl -s http://127.0.0.1:123/stackdump
OK
goroutine stacks written to /tmp/goroutine-stacks-2025-01-19T160337Z.logroot@fa87ff1bcd00:/go/src/github.com/docker/docker#
After this:
root@fa87ff1bcd00:/go/src/github.com/docker/docker# curl -s http://127.0.0.1:123/stackdump
OK
goroutine stacks written to /tmp/goroutine-stacks-2025-01-19T160922Z.log
root@fa87ff1bcd00:/go/src/github.com/docker/docker#
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A terminated connection is not an error on the daemon-side, and expected
if the client disconnects. This patch detects if the error is because of
a broken pipe, and skips the warning in that case.
Before this patch:
WARN[2025-01-18T12:38:04.115298341Z] could not write error response: write unix /var/run/docker.sock->@: write: broken pipe
After this patch, no warning is logged. This patch also changes the log
format to use structured logs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this change, it would fail to detect context errors, resulting in
pullEndpoints clobbering the context error and changing it into a fallback
error; 029933578b/distribution/pull.go (L114-L119)
While the context cancellation would still be handled, the error returned
would be wrapped, causing calling code to no longer being able to detect
it as context cancellation;
029933578b/distribution/pull.go (L125)
Context cancellation are now logged as "info" in daemon-logs, as they
are not an error from the daemon's perspective;
Before:
DEBU[2025-01-18T14:59:10.079259676Z] pulling blob "sha256:8bb55f0677778c3027fcc4253dc452bc9c22de989a696391e739fb1cdbbdb4c2"
ERRO[2025-01-18T14:59:10.564076135Z] Not continuing with pull after error: context canceled
After:
DEBU[2025-01-18T15:09:56.743045420Z] pulling blob "sha256:8bb55f0677778c3027fcc4253dc452bc9c22de989a696391e739fb1cdbbdb4c2"
INFO[2025-01-18T15:09:57.390835628Z] Not continuing with pull after error error="context canceled"
This package needs a big cleanup for context- and error-handling, as it's
very messy, so these changes are only a short-term workaround.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The isBrokenPipe utility was added in 3d86b0c79b
to unwrap the error returned to detect if it was a broken pipe error.
`net.OpError` now implements Unwrap(), so we can simplify this check
using `errors.Is`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug log was slightly confusing, as it seemed the cleaning up failed,
but the log was intended to inform that cleaning up was started because
of an error.
Before this patch:
DEBU[2025-01-18T12:29:52.717452919Z] Cleaning up layer 472e3e532da3945040a6224ba36d2befcfe250aeb53536868d23e5f617e34226: exit status 1: unpigz: skipping: <stdin>: corrupted -- incomplete deflate data
With this patch:
DEBU[2025-01-18T13:01:45.307983471Z] Cleaning up cache layer after error cache-id=c4d9f38bdf6d8fcb69bd130b89d34d1511b42f3644fc49c91e253eda648bd37a error="exit status 1: unpigz: skipping: <stdin>: corrupted -- incomplete deflate data\n"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since commit 0f8fc31, the bridge driver will try to load kernel
module br_netfilter if the userland proxy is disabled. If it fails,
we're in unknown territory, so it's treated as an error. At the
very least, containers will not be able to access host ports
mapped to other containers in the same network.
Before that, and before commit 5c499fc delayed the module load
until it was needed - it was loaded unconditionally, but errors
were only logged.
So, on systems where the module is not available, or could not be
loaded/configured, no error was reported and things "worked" (as
long as you didn't try to use something that didn't work).
That behaviour has been useful to some. So, make it possible to
ignore the problem by setting env var:
DOCKER_IGNORE_BR_NETFILTER_ERROR=1
Signed-off-by: Rob Murray <rob.murray@docker.com>
Now that we moved to use containerd 2.0, the changes from
63b4688175
can now be used, removing some of gRPC's deprecated
options.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The default message size is set unconditionally in containerd's client,
so unlike Dial-options, there's no risk of implicitly dropping these
options.
TThis patch removes the options, as they were the same as the default
already set in containerd itself.
https://github.com/containerd/containerd/blob/v1.7.22/client.go#L133-L138
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The default message size is set unconditionally in containerd's client,
so unlike Dial-options, there's no risk of implicitly dropping these
options.
TThis patch removes the options, as they were the same as the default
already set in containerd itself.
https://github.com/containerd/containerd/blob/v1.7.22/client.go#L133-L138
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Dial-options passed to containerd _override_ all defaults that are set
in containerd, and containerd does not provide an option to provide
the defaults in other ways, which makes it slightly more complicated
to use the defaults combined with some custom options.
https://github.com/containerd/containerd/blob/v1.7.22/client.go#L122-L132
This patch aligns the options we set with the defaults in containerd.
grpc.FailOnNonTempDialError was added together with WithBlock in [containerd@64bc516],
but it looks like this was not copied to our options when the equivalent was
added in this repository through 9f73396dab.
grpc.WithReturnConnectionError was added in [containerd@73d28dd] to improve
handling of connection errors;
Permission errors:
% ./bin/ctr t ls
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
%
Non-existent sockets:
% ./bin/ctr -a notfound t ls
ctr: failed to dial "notfound": context deadline exceeded: connection error: desc = "transport: error while dialing: dial unix://notfound: timeout"
%
That commit failed to notice that WithReturnConnectionError implies WithBlock,
so removing that option from the list.
Note that both WithBlock and WithReturnConnectionError are deprecated in
newer versions of grpc, so we should remove these once [containerd@63b4688]
makes it into the containerd 1.7 branch (and vendored).
[containerd@64bc516]: 64bc516bbe
[containerd@73d28dd]: 73d28ddeb2
[containerd@63b4688]: 63b4688175
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 1701bce9e0 updated the debug logs
to use structured logs, and to include more information in the logs, such
as the form-data used and the status-code, and combining them into a
single log.
However, for the status-code, we need to wait for the handler to do its
thing and (ahum) this was staring me right in the face, and crossed my
mind, but then I didn't act on it; the handler may take some time to
run, and produce logs, which now means that our nice "handling request"
log will be logged _after_ the request, which is obviously confusing.
This patch splits the log into two;
- a log entry when starting to handle the request
- a log entry if a non-200 status is returned (assuming 200 status
codes are less interesting to log).
Before this patch:
DEBU[2024-10-23T15:23:31.677184128Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=8180b03fa17f9783 status=200 traceID=a5dfa9b86445467889274145ad31bb9a vars="map[]"
DEBU[2024-10-23T15:23:31.712833045Z] resolving host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
DEBU[2024-10-23T15:23:31.712883670Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.051728378Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:23:32 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.051830920Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
DEBU[2024-10-23T15:23:32.051909045Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.544987920Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:23:32 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:23:32.545112212Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
INFO[2024-10-23T15:23:32.545270087Z] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=97e231483c8d4d9a traceID=3c01e6c2cf19cf82237fdd01c0294fb4
DEBU[2024-10-23T15:23:32.550666962Z] handling POST request error-response="pull access denied for nosuchimage, repository does not exist or may require 'docker login'" method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=38286e48a07445ef status=404 traceID=3c01e6c2cf19cf82237fdd01c0294fb4 vars="map[version:1.47]"
With this patch applied:
DEBU[2024-10-23T15:18:18.876346178Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=7fd5eb011140f546 traceID=80ffd75a39de78a1f51ffda89fc4f227 vars="map[]"
DEBU[2024-10-23T15:18:18.878006428Z] handling POST request method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=40dea95727e38394 traceID=2f901d99f1cf2105e2614d6929d53d3b vars="map[version:1.47]"
DEBU[2024-10-23T15:18:18.919686136Z] resolving host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
DEBU[2024-10-23T15:18:18.919748094Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.258132303Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:18:19 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.258219803Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
DEBU[2024-10-23T15:18:19.258406886Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.755911762Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 15:18:19 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest"
DEBU[2024-10-23T15:18:19.757501928Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
INFO[2024-10-23T15:18:19.757689387Z] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=74f65a3accb19ad3 traceID=2f901d99f1cf2105e2614d6929d53d3b
DEBU[2024-10-23T15:18:19.763826095Z] error response for POST request error-response="pull access denied for nosuchimage, repository does not exist or may require 'docker login'" method=POST module=api request-url="/v1.47/images/create?fromImage=nosuchimage&tag=latest" spanID=40dea95727e38394 status=404 traceID=2f901d99f1cf2105e2614d6929d53d3b vars="map[version:1.47]"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/container/run_linux_test.go:459:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
(thanks Go)
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
This requires changes in the CLI to support fully, but matches our other boolean option handling (`no-new-privileges`).
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
This makes `WritableCgroups` a pointer so we can error when it's specified in invalid configurations (both rootless and user namespaces).
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Fixes#42040Closes#42043
Rather than making cgroups read-write by default, instead have a flag
for making it possible.
Since these security options are passed through the cli to daemon API,
no changes are needed to docker-cli.
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
Commit ebcb7d6b40 removed code that string-
matched errors to return the appropriate error-type. As part of that, it
defaulted to using a "System" (status 500) error.
We need to verify codepaths used by the archive functions, but let's
start with detecting invalid parameter errors, and preserve those.
This patch updates the code backing the following endpoints to return
a 400 (Invalid Parameter) error when present;
- `HEAD /containers/{name:.*}/archive`
- `GET /containers/{name:.*}/archive`
- `PUT /containers/{name:.*}/archive`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Windows containers do not currently support a read-only rootFS, and a
[validateReadonlyRootfs] (added in 6f7dc65847)
will reject containers created with this option. Similarly, Windows does
not support read-only volume;
Windows originally did not support volumes (47c56e4353).
Support for volumes was added later, but without support for read-only volumes
(see e5261d6e4a). Support for read-only values
was originally planned to be supported after TP4, but later moved to "post TP5"
when support for TP4 was removed in 331c8a86d4.
This patch removes code related to read-only paths on Windows, but keeps
a TODO / comment as a reminder.
[validateReadonlyRootfs]: a052449271/runconfig/hostconfig_windows.go (L56-L62)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Combine the check for read-only volume paths and the container's rootFS
to be read-only into a single check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was a no-op on Windows, which currently doesn't support it, so we
can remove this code in the meantime until there's support for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The StatsResponse type was a compatibility-wrapper introduced in
d3379946ec to differentiate responses
for API < 1.21 and API >= 1.21. API versions lower than 1.24 are
deprecated, and we can merge these types again.
The Stats type was not used directly, but deprecating it, and
making it an alias for StatsResponse, which provides a superset
of its fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The StatsResponse type was a compatibility-wrapper introduced in
d3379946ec to differentiate responses
for API < 1.21 and API >= 1.21. API versions lower than 1.24 are
deprecated, and we should merge StatsResponse and Stats, but let's
start with using the StatsResponse in our tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Networking tests have been creeping towards the limit for a
while, but they're reliably failing in a upcoming PR that adds
a relatively slow test.
Signed-off-by: Rob Murray <rob.murray@docker.com>
commit 572498be56 moved the ioutils.HashData
utility to the libnetwork resolvconf package.
After removing, we saw some tests in the reference pacakge failing;
=== Failed
=== FAIL: reference TestLoad (0.00s)
store_test.go:53: failed to parse reference: unsupported digest algorithm
=== FAIL: reference TestSave (0.00s)
store_test.go:82: failed to parse reference: unsupported digest algorithm
=== FAIL: reference TestAddDeleteGet (0.00s)
store_test.go:174: could not parse reference: unsupported digest algorithm
=== FAIL: reference TestInvalidTags (0.00s)
store_test.go:355: assertion failed: error is not nil: unsupported digest algorithm
Those failures were because those tests depended on a side-effect of the
ioutils package being imported, which (before removal of HashData) imported
crypto/sha256, registering that algorithms, which on its turn was used by
github.com/opencontainers/go-digest to determnin if a given algorithm must
be accepted (see [go-digest#64]).
As a workaround, these imports were added. pkg/ioutils is now imported
in less places, and should not be depended on for this purpose.
Let's remove this workaround; if needed, these imports can be added in
a more relevant location.
This reverts commit 98caf09f0f.
[go-digest#64]: https://github.com/opencontainers/go-digest/pull/64
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Reverts otel workaround, added in cca7085464,
as it's no longer needed:
=== Failed
=== FAIL: cmd/dockerd TestOtelMeterLeak (0.64s)
daemon_test.go:303: Allocations: 3
daemon_test.go:307: Allocations count decreased. OTEL leak workaround is no longer needed!
We're keeping the test for now, so that we can check for possible
regressions in the OTel dependencies.
Co-authored-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When a NAT-based port mapping is created with a HostIP specified, we
insert a DNAT rule in nat-DOCKER to replace the dest addr with the
container IP. Then, in filter chains, we allow access to the container
port for any packet not coming from the container's network itself (if
hairpinning is disabled), nor from another host bridge.
However we don't set any rule that prevents a rogue neighbor that shares
a L2 segment with the host, but not the one where the port binding is
expected to be published, from sending packets destined to that HostIP.
For instance, if a port binding is created with HostIP == '127.0.0.1',
this port should not be accessible from anything but the lo interface.
That's currently not the case and this provides a false sense of
security.
Since nat-DOCKER mangles the dest addr, and the nat table rejects DROP
rules, this change adds rules into raw-PREROUTING to filter ingress
packets destined to mapped ports based on the input interface, the dest
addr and the dest port.
Interfaces are dynamically resolved when packets hit the host, thanks
to iptables' addrtype extension. This extension does a fib lookup of the
dest addr and checks that it's associated with the interface reached.
Also, when a proxy-based port mapping is created, as is the case when an
IPv6 HostIP is specified but the container is only IPv4-capable, we
don't set any sort of filtering. So the same issue might happen. The
reason is a bit different - in that case, that's just how the kernel
works. But, in order to stay consistent with NAT-based mappings, these
rules are also applied.
The env var `DOCKER_DISABLE_INPUT_IFACE_FILTERING` can be set to any
true-ish value to globally disable this behavior.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This option was marked as deprecated in cc1f3c750 (released in v20.10).
The option `fluentd-async`, introduced in the same commit, should be
used instead.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This field was initialized by `driver.createNetwork` instead of being
initialized by the `newDriver` constructor. That's currently the single
place where it's initialized -- no tests override it, so it seems the
'current' netns is always used.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Close archive when context is done - this makes the cancellation
actually stop the export instead of continuing it regardless if the
client still expects the data.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
With `RWLayer` it's no longer necessary to define it for each image
service as it became a wrapper for the RWLayer's Mount and Unmount.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
An "--internal" bridge network will never set up a default route
and, with "-o com.docker.network.bridge.inhibit_ipv4", no Gateway
address will be assigned to the bridge.
So, implement the SkipGwAlloc interface in the bridge driver, and
use it to to indicate that no Gateway address is required in this
specific case.
Signed-off-by: Rob Murray <rob.murray@docker.com>
A gateway address is always reserved before the network driver is
asked to create the network. But, the driver doesn't always need a
gateway address, so the address reservation can be unnecessary.
This means, for example, an "--internal" IPv4 "/31" network cannot
be used as a point-to-point link, because one of its two addresses
is always reserved for a gateway.
So, before allocating a gateway address, ask the network driver if
it will need one (based on options that only the network driver can
interpret). Implement that as an optional interface for network
drivers.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When setting up a gateway in IpamInfo, and reserving that address in
IPAM ... the IPAM driver may return its own default gateway. That
gateway address is currently always parsed, but it's only used if the
user did not supply their own gateway address. If the user supplied
an address, it's always reserved and used. Otherwise, the IPAM driver
is asked to select and reserve an address.
The logic to deal with that was a bit confusing - and it's probably
better just to ignore the IPAM driver's gateway if it's not going to
be used.
So, simplify it little.
Signed-off-by: Rob Murray <rob.murray@docker.com>
It was added in Docker [v1.3.0] through bd130e72a0,
but never used, and its behavior never documented. There are no publicly
visible external consumers of this function, so let's remove it.
[v1.3.0]: https://github.com/moby/moby/releases/tag/v1.3.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit e9bbc41dd1 removed our fork of
pkg/archive that was in place to mitigate CVE-2019-14271. As part of that
change, a nosysFileInfo type was added to prevent tar.FileInfoHeader from
looking up user- and group-names.
A proposal was pending in go https://go.dev/issue/50102 to define an
interface for implementing custom lookup functions to be implemented,
and disable go's builtin lookup. That proposal was accepted, and is now
implemented in go1.23.
Thia patch makes the nosysFileInfo implement the tar.FileInfoNames interface
to prevent tar.FileInfoHeader from performing its own lookups. While the
mitigation implemented in e9bbc41dd1 should
already prevent this from happening, implementing the interface does not
cost us much and is complementary to the existing mitigation.
This patch keeps the mitigation added in a316b10dab
in place for any unforeseen other code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only used internally, and has no external consumers; deprecate
it to be removed in the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types are only used internally in container/streams and have no
external consumers. move them to a subpackage of container/streams.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On a freshly rebooted Linux host that's configured to use nftables
with the iptables front-end, "make test-unit" fails "TestUserChains"
on the first run - it's ok in subsequent runs.
The unit tests run in moby's dev container.
The first test in TestUserChain runs with ip6tables disabled, so the
bridge driver doesn't try to load the ip6_tables module. Then, because
the module isn't loaded (it wasn't needed by the daemon running on the
nftables host when it started), the test fails because it can't check
what's in the ip6tables filter chain.
The next test in TestUserChain does the same thing, but with ip6tables
enabled. So the module gets loaded by the bridge driver, and everything
works normally after that.
The dev container used to try to load the module on startup, but that
was removed in commit 2af19b6b ("Don't try to modprobe ip6_tables in
the moby dev container"), as part of a change to give the daemon a way
to load modules itself.
Rather that put back the dev container's code to load ip6_tables on
startup (which would mean the daemon's module-loading code not getting
to run on nftables/firewalld hosts) ...
Run the tests in TestUserChains in a different order, with iptables
enabled in the first test will make it happen to work. At least for
now.
It's not ideal, but we'll be switching to nftables soon, so the issue
will go away.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Apart from being used internally for NewWriteFlusher, it's only used
in a single location outside of this package. Copy the implementation
where it's used, and mark it deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
After implementing `RWLayer` for containerd image store, implementation
of these methods is identical for both stores.
Move the logic out of the image service into the daemon.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Implement containerd image store backed `RWLayer` and remove the
containerd-specific `PrepareSnapshot` method from the ImageService
interface.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Introduce a separate `RWLayer` interface for the `container.RWLayer`
to remove coupling with the graphdriver implementation.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
it was moved to pkg/ioutils in c30a55f14d,
and only had a single use at the time in [engine/Env.WriteTo]. That use
was removed in 531f4122bd, which removed
the engine package.
[engine/Env.WriteTo]: c30a55f14d/engine/env.go (L260-L264)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types are only used internally in container/streams and have no
external consumers. Deprecate them in preparation of moving them to
a subpackage of container/streams.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was testing errors produced by runc; both the "maximum" and
"minimum" values originate from the OCI runtime;
d48d9cfefc/libcontainer/cgroups/fs/cpu.go (L66-L83)
docker run --cpu-shares=1 alpine
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: the minimum allowed cpu-shares is 2: unknown.
Happy path for this setting is covered by TestRunWithCPUShares, and
various other tests, so we validate that the options take effect;
f5af46d4d5/integration-cli/docker_cli_run_unix_test.go (L494-L503)
This patch:
- removes the test and migrates it to an integration test
- removes the checks for errors that might be produced by runc
- updates our validation for invalid (negative) values to happen
when creating the contaienr; the existing check that happened when
creating the OCI spec is preserved, so that configs of existing containers
are still validated.
- updates validateResources to return the correct error-type
- updated unit-test to validate
With this patch:
make TEST_FILTER='TestCreateInvalidHostConfig' TEST_SKIP_INTEGRATION_CLI=1 test-integration
--- PASS: TestCreateInvalidHostConfig (0.00s)
--- PASS: TestCreateInvalidHostConfig/invalid_IpcMode (0.00s)
--- PASS: TestCreateInvalidHostConfig/invalid_CPUShares (0.00s)
--- PASS: TestCreateInvalidHostConfig/invalid_PidMode (0.00s)
--- PASS: TestCreateInvalidHostConfig/invalid_PidMode_without_container_ID (0.00s)
--- PASS: TestCreateInvalidHostConfig/invalid_Annotations (0.00s)
--- PASS: TestCreateInvalidHostConfig/invalid_UTSMode (0.00s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the fourth patch release of the 1.2.z release branch of runc. It
includes a fix for a regression introduced in 1.2.0 related to the
default device list.
- Re-add tun/tap devices to built-in allowed devices lists.
In runc 1.2.0 we removed these devices from the default allow-list
(which were added seemingly by accident early in Docker's history) as
a precaution in order to try to reduce the attack surface of device
inodes available to most containers. At the time we thought
that the vast majority of users using tun/tap would already be
specifying what devices they need (such as by using --device with
Docker/Podman) as opposed to doing the mknod manually, and thus
there would've been no user-visible change.
Unfortunately, it seems that this regressed a noticeable number of
users (and not all higher-level tools provide easy ways to specify
devices to allow) and so this change needed to be reverted. Users
that do not need these devices are recommended to explicitly disable
them by adding deny rules in their container configuration.
full diff: https://github.com/opencontainers/runc/compare/v1.2.3...v1.2.4
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type was exported in fd35494a25 to be
used in tests in pkg/archive. It's no longer used in pkg/archive since
ba90fd8da0, so we can un-export it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When saving a docker image with `docker save`, output may have the
current timestamp, resulting in slightly changed content each time the
`save` command gets run. This patch attemtps to stabilize that effort to
clean up some spots where we've missed setting the timestamps.
It's not totally clear that setting these timestamps to 0 is the correct
behavior but it will fix the hash stability problem on output.
Signed-off-by: Stephen Day <stephen.day@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure we return a proper errdefs.InvalidParameter for these, and
update some error-messages to fix linting issues.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was conditionally testing windows or non-windows fixtures,
but it looks like we can test these cross-platform (at least currently).
Also moved the custom "f" type into the test, and rewrote it to use a
regular test-table with subtests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Rewrite the test to validate decoding the container config
as a whole instead of the individial check, which didn't
provide much coverage.
- Rename to TestDecodeContainerConfigPrivileged
- Move the test to a platform-agnostic file to verify an
error is produced on Windows, but not on Linux
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test had some broken test-case where the error would only be asserted
if an error happened, but would be ignored if no error happened, e.g.;
if err := callDecodeContainerConfigIsolation(t, "invalid"); err != nil {
if !strings.Contains(err.Error(), `Invalid isolation: "invalid"`) {
t.Fatal(err)
}
}
The test also used a helper function (callDecodeContainerConfigIsolation,
added in 15e35c4470), which had multiple
output variables, but none of those, except for the error, were used.
This patch:
- removes the callDecodeContainerConfigIsolation utility
- rewrites TestDecodeContainerConfigIsolation to use sub-tests
- uses gotest.tools to assert errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Simplify the test by testing the result, instead of manually checking
specific values. This makes sure we check the actual results, and don't
miss values, or ignore unexpected values.
TestLinkPortRangeEnv was added in 611a23aa7f
to test for port-ranges to produce the expected env-vars, but used the
same input as TestLinkMultipleEnv. Now that we assert all env-vars produced,
it became a duplicate of TestLinkMultipleEnv, so we can remove that test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously the RWLayer reference was cleared without holding the
container lock. This could lead to goroutine panics in various places
that use the container.RWLayer because nil checks introduced in #36242
where not sufficient as the reference could change right before the use.
Fixes#49227
Signed-off-by: Tadeusz Dudkiewicz <tadeusz.dudkiewicz@rtbhouse.com>
Simplify the test by testing the result, instead of manually checking
specific values. This makes sure we check the actual results, and don't
miss values, or ignore unexpected values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Simplify the test by testing the result, instead of manually checking
specific values. This makes sure we check the actual results, and don't
miss values, or ignore unexpected values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was added in 12b6083c8f as a
replacement for nat.NewPort(), which before that patch would panic on
invalid values, but was changed to return an error.
Given that the utility ignores any error, and these values are fixed values
for the test, let's remove it to simplify constructing the tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Separare idtools functionality that is used internally from the
functionlality used by importers. The `pkg/idtools` package is now
much smaller and more generic.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Next commits will introduce a new internal `metrics` package, so alias
the "external" import to avoid confusion.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This was the only location in this file where pkg/errors was used,
and this function already mixed stdlib errors with pkg/errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On live-restore, the Sandbox tries to restore state in the osSbox
by telling it about interface, routes, and gateways that would
have been set up by the previous incarnation of the daemon.
Restoring gateways has been broken since commit 18327745c0 (Allow
separate IPv4/IPv6 gateway endpoints.) ... which didn't properly
deal with searching for the "dstName" of an interface based on its
IPv6 address.
This commit fixes that, and splits the osSbox restore into three
parts:
- Restore the interfaces, including finding the "dstName".
- Restore routes, unchanged, they're just a copy of the sandbox's
StaticRoutes
- Restore gateway info - if the Sandbox's gateway endpoint has an
IP address (v4 or v6, depending on which addr family/families it's
acting as the gateway for), store that. If not, the default route
is bound to the interface, so store that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Check that when a container has endpoints in an l3-ipvlan and
another network type (bridge), there's no longer any clash
betwen the ipvlan's connected default route and the bridge's
default gateway.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In L3 modes, the ipvlan driver can't set up a default gateway with
a next hop address, because there's no L2 for it to resolve the
gateway IP into a MAC address.
Instead, it sets up a route to 0.0.0.0 or [::] that's connected to
the network's interface. The end result is the same - the container
has a default route.
So, don't set up routes to 0.0.0.0/:: when applying routes when an
endpoint joins a sandbox, set them up when the endpoint is selected
as the container's gateway. And, drop those routes when another
endpoint becomes the gateway.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In L3 modes, the ipvlan driver can't set up a default gateway with
a next hop address, because there's no L2 for it to resolve the
gateway IP into a MAC address.
Instead, it sets up a route to 0.0.0.0 or [::] that's connected to
the network's interface. The end result is the same - the container
has a default route.
So, include those routes when searching for endpoints that can act
as a container's default gateway.
Signed-off-by: Rob Murray <rob.murray@docker.com>
It doesn't look like this type was intended for external use; constructing
a RingLogger can be done through the `NewRingLogger()` constructor, which
returns a `Logger` interface (implemented by `RingLogger`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch:
remove_test.go:62: timeout hit after 10s: waiting for container to be one of (exited), currently running
After this patch:
remove_test.go:62: waiting for container State.Status to be 'exited', currently 'running'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was using a confusing syntax because `Isolation.IsDefault()`
and `Isolation.IsHyperV()` don't accept an argument. It's valid (see below),
but just confusing, so let's use a more common approach.
```go
package main
import "fmt"
type NameSayer string
func (f NameSayer) SayMyName() {
fmt.Println(f)
}
func main() {
var foo NameSayer = "my name is"
foo.SayMyName()
NameSayer("my name is..").SayMyName()
// Thought SayMyName() would take no arguments? Think again!
NameSayer.SayMyName("slim shady!")
}
```
While at it, also renamed the `container` argument as it was shadowing
the `container` import.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- update github.com/Azure/go-ansiterm to v0.0.0-20250102033503-faa5f7b0171c
to fix OSC string terminator parsing.
- add security policy
- update github actions and test against go1.22, go1.23
full diff: https://github.com/moby/term/compare/v0.5.0...v0.5.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 1932091e21 removed support
for the --allow-nondistributable-artifacts, but forgot to remove
this section.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function is only used internally and has no external consumers.
Mark it deprecated to be removed in the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Validate if options are passed in the right format and if the given option
is supported on the current platform.
Before this patch, no validation would happen until the daemon was started,
and unknown options as well as incorrectly formatted options would be silently
ignored on Linux;
dockerd --exec-opt =value-only --validate
configuration OK
dockerd --exec-opt unknown-opt=unknown-value --validate
configuration OK
dockerd --exec-opt unknown-opt=unknown-value --validate
...
INFO[2024-11-28T12:07:44.255942174Z] Daemon has completed initialization
INFO[2024-11-28T12:07:44.361412049Z] API listen on /var/run/docker.sock
With this patch, exec-opts are included in the validation before the daemon
is started/created, and errors are produced when trying to use an option
that's either unknown or not supported by the platform;
dockerd --exec-opt =value-only --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid exec-opt (=value-only): must be formatted 'opt=value'
dockerd --exec-opt isolation=default --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid exec-opt (isolation=default): 'isolation' option is only supported on windows
dockerd --exec-opt unknown-opt=unknown-value --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid exec-opt (unknown-opt=unknown-value): unknown option: 'unknown-opt'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test originally added in 4352da7803,
and was a bit involved as it involved building an image, and had some
dubious test-cases, such as using `wrongimage:<ID of other image>` as
reference, and expecting that to produce a "not found" error. Possibly
this format was supported in the past, but currently it fails equally with
`correctimage:<ID of image>`.
This patch rewrites the test to an integration test, and removes the test
from integration-cli. It also removes TestCreate64ByteHexID, as it was
duplicated by this test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This code has some gnarly string-matching to detect "not found" errors
returned by hcsshim.
Hcsshim at some point changed this error to lowercase;
6d67a30859
It looks like we ran into that problem in integration tests, which was
fixed in c530c9cbb0 when updating hcsshim,
however, it was only fixed in tests, and hiding the actual issue in our
code.
It looks like hcsshim has some utilities to detect error-types, such as the
IsElementNotFoundError function in hcn, which is the newer API that also wraps
the "HNS" service;
d9a4231b9d/hcn/hcnerrors.go (L75-L77)
But unfortunately, the hns API used by us, does not return typed errors, and
returns HNS errors as a untyped formatted string.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also updated some existing ones to use `%v` instead of `%s` for consistency.
caught by go1.24
# github.com/docker/docker/libnetwork/drivers/windows/overlay
# github.com/docker/docker/libnetwork/drivers/windows/overlay/ov_network_windows.go:206:32: non-constant format string in call to github.com/docker/docker/libnetwork/types.ForbiddenErrorf
FAIL github.com/docker/docker/libnetwork/drivers/windows/overlay [build failed]
# github.com/docker/docker/libnetwork/drivers/windows
# github.com/docker/docker/libnetwork/drivers/windows/windows.go:449:33: non-constant format string in call to github.com/docker/docker/libnetwork/types.ForbiddenErrorf
FAIL github.com/docker/docker/libnetwork/drivers/windows [build failed]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
caught by go1.24
# github.com/docker/docker/distribution
# github.com/docker/docker/distribution/pull_v2_windows.go:145:35: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Debugf
FAIL github.com/docker/docker/distribution [build failed]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Based on the "lazyregexp" package in golang.org/x/mod;
https://cs.opensource.google/go/x/mod/+/refs/tags/v0.19.0:internal/lazyregexp/lazyre.go;l=66-78
This package allows defining regular expressions that should not be
compiled until used, but still providing validation to prevent
invalid regular expressions from producing a panic at runtime.
This is largely a copy of the package from golang.org/x/mod,
with FindAllStringSubmatch and ReplaceAllStringFunc added
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go has a nice tool to inspect the currently set (or default) `GOPATH`
with `go env GOPATH`.
We should do that rather than absolutely requiring people to manually
set `GOPATH` or use the `AUTO_GOPATH`.
This doesn't actually remove `AUTO_GOPATH`, people can still use it.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- use t.TempDir()
- combine various tests to check if New() sets expected values instead
of skipping tests when not.
- remove gotest.tools, as it was only used minimally
- replace uses of "path" for filepath operations.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
sysinfo.NumCPU returns the number of CPUs which are currently online,
which involves some syscalls and parsing on Windows.
Change the code to only retrieve this information when needed, and
memoize the result to prevent calling this function multiple times.
Ideally, we'd obtain this information from daemon.RawSysInfo(), but
that uses a sync.Once, which could return outdated information.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- rename variables that shadowed functions
- use strconv.FormatBool instead of fmt.Sprintf to reduce some allocations
- use structured logs for some warnings
- remove some intermediate vars in favor of early returns
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These tests failed because the error message changed in go1.24 through
https://go.dev/cl/606956.
=== Failed
=== FAIL: daemon/config TestDaemonConfigurationFeatures/invalid_config_value (0.00s)
config_linux_test.go:182: assertion failed: expected error "json: cannot unmarshal string into Go struct field Config.features of type bool", got "json: cannot unmarshal string into Go struct field Config.CommonConfig.features of type bool"
=== FAIL: daemon/config TestDaemonConfigurationFeatures (0.00s)
=== FAIL: daemon/config TestDaemonConfigurationHostGatewayIP/config_not_array (0.00s)
config_linux_test.go:354: assertion failed: expected error "json: cannot unmarshal string into Go struct field Config.host-gateway-ips of type []netip.Addr", got "json: cannot unmarshal string into Go struct field Config.CommonConfig.DNSConfig.host-gateway-ips of type []netip.Addr"
Relax the tests a bit to accept errors produced by either go1.24 or older.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 6d2c4f87af.
go.dev/issue/64759 should be fixed in go1.21.9 through https://go.dev/cl/574736,
so we can revert the workaround.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This check was in place for CentOS/RHEL 7, which use kernel 3.10. Now
that both reached EOL, and we stopped packaging for those distros, we
can remove this condition.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The roundtripper is responsible for giving back the build context when
it comes from a tar directly. So we add it to the source manager of the
containerd worker.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
This type was originally in pkg/transport, but got moved to pkg/ioutils
in 276c640be4.
This type is only used in a single location, and has no external consumers,
so we can move it where it's used and un-export it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The use of bufio for writing without flushing can lead to an incomplete
writing of the tar and subsequent unexpected EOF when importing.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Cleanup decompress logic and add a pool. The close logic should be
custom defined for each compression type since they have different
close interfaces.
Signed-off-by: Derek McGowan <derek@mcg.dev>
After the untar errors, the reader must complete in order to fill
the buffer used by the subsequent check.
Signed-off-by: Derek McGowan <derek@mcg.dev>
commit 86d1223a29 introduced a custom version
of `os.MkdirAll` for Windows to account for situations where the path to
create would start with a Windows volume name (GUID path), for example,
`"\\?\Volume{4c1b02c1-d990-11dc-99ae-806e6f6e6963}\`. At the time that patch
was added we were using [go1.4.2], which did not have special handling
for Windows in [MkdirAll], therefore would recognize such paths as regular
paths, trying to create them, which would fail.
This code was later updated in 46ec4c1ae2
to provide ACL (DACL) support on Windows.
Further updates were made in cfef1b11e5 and
55ceb5047c to allow for an early return when
detecting a volume GUID path, and the code was re-aligned with the latest
(go1.19.2) implementation in f058afc861, which
brought in the platform-specific [fixRootDirectory] handling introduced in
go1.11. While that enhancement detected UNC volume-paths (`\\?c\`, `//?/c:`),
it did not yet support volume GUID paths.
go1.22, through [golang.org/cl/86295] added support for this, and `os.MkdirAll`
now natively detects volume GUID paths, making our own implementation for
this redundant.
This patch:
- Deprecates pkg/system.MkdirAll in favor of os.MkdirAll, which now provides
the same functionality on go1.22 and up.
- Renames the (non-exported) `mkdirall` function to `mkdirAllWithACL`, and
synchronises `it` with the [implementation in go1.23.4], bringing in the
changes from [golang.org/cl/86295] and [golang.org/cl/582499].
- Adds a fast path to `MkdirAllWithACL` if no ACL / SDDL is provided.
It's worth noting that we currently still support go1.22, and that the
implementation changed in go1.23; those changes ([golang.org/cl/581517]
and [golang.org/cl/566556]) were lateral moves, therefore should be
identical to the implementation in go1.22, and we can safely use the
implementation provided by [filepath.VolumeName] on either go1.22 or go1.23.
[go1.4.2]: 86d1223a29/Dockerfile (L77)
[MkdirAll]: https://github.com/golang/go/blob/go1.4.2/src/os/path.go#L19-L60
[fixRootDirectory]: b86e766813
[golang.org/cl/86295]: cd589c8a73
[golang.org/cl/582499]: 5616ab6025
[golang.org/cl/581517]: ad22356ec6
[golang.org/cl/566556]: ceef0633b3
[1]: https://github.com/golang/go/blob/go1.23.4/src/os/path.go#L12-L66
[filepath.VolumeName]: https://pkg.go.dev/path/filepath#VolumeName
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before that change, we were passing the datastore to network drivers
through a `map[string]interface{}`. Then, each driver that needed the
store would cast the datastore to the correct type.
This was not a good design, as it was not clear which drivers were using
the store and which were not. Not all unit tests were passing the store,
leading to logs about uninitialized store being written.
This change makes the store a parameter of the `RegisterX` functions.
All unit tests are now passing a valid datastore to the drivers. A new
testutil func is added for that purpose.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Wrap `http.RoundTripper` used by distribution code (push/pull) with the
`otelhttp.Transport`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
These comments were added to prevent users from accidentally
importing using the wrong module name, but they don't have
an effect when working in go modules mode.
Remove the comments in preparation of moving this package
to a separate module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove use of github.com/pkg/errors for this package, in preparation
of it being moved to a separate module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/containerd/cgroups/compare/v3.0.3...v3.0.5
notable changes:
- chore: don't log ENOTSUP during parsing PSI files
- Add EOPNOTSUPP to err filter for PSI data
- cg2: Don't read cgroup.procs when deleting threaded cg
- Added cgroup type "domain threaded" and "domain invalid"
- switch to github.com/containerd/log
- go.mod: update to go1.22 as minimum
- deprecate RunningInUserNS()
- dont ignore failure to create cgroup after timeout
- cgroup2: Manager.Delete: handle both "threaded" and "domain threaded"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit b08ff81204 updated this code to
only produce an error if an invalid value was set by the user, and to
avoid errors on empty values.
However, the intent of this code was to allow `0` as a valid value for
cases where gc is to be handled through other properties / filters.
This patch only sets the default if no value was set by the user, but
doesn't set the default if a value is set by the user, but zero.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`imageDeleteConflict` is always returned via a reference, so adjust the
method receiver of `Conflict` to make it consistent with `Error`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
contains a fix for CVE-2024-45338 / https://go.dev/issue/70906,
but it doesn't affect our codebase:
govulncheck -show=verbose ./...
Scanning your code and 1260 packages across 211 dependent modules for known vulnerabilities...
...
Vulnerability #1: GO-2024-3333
Non-linear parsing of case-insensitive content in golang.org/x/net/html
More info: https://pkg.go.dev/vuln/GO-2024-3333
Module: golang.org/x/net
Found in: golang.org/x/net@v0.32.0
Fixed in: golang.org/x/net@v0.33.0
Your code is affected by 0 vulnerabilities.
This scan also found 0 vulnerabilities in packages you import and 1
vulnerability in modules you require, but your code doesn't appear to call these
vulnerabilities.
full diff: https://github.com/golang/net/compare/v0.32.0...v0.33.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 8fb71ce208 moved access to these to
the image service directly, so they are no longer used in the router.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Its only usage was in container creation, which also called `GetImage`
first.
This method is specific to the containerd image service and is largely
the same as `GetImage`, except it returns a manifest descriptor.
Instead, introduce add the descriptor as a new field in `Image.Details`
and set it in the containerd image service implementation of `GetImage`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Shaun is managing the runtime and build teams, and will be helping out
on triage, as well as organizing backlogs on GitHub.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When containers make DNS resolution, and the domain name they're trying
to resolve doesn't match any running container, the DNS query is
forwarded to upstream servers. In that case, when we receive a response,
we put it in an OTel spans.
This was useful to debug DNS resolution on GHA, but it leads to
excessive memory usage when DNS resolution happen in a tight loop. So,
keep the OTel event signaling that a response was received, but drop the
answer from the OTel span.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 173a9f2091)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
- Updated the example section in `swagger.yaml` for the `DataPathAddr` parameter in `SwarmJoinRequest`.
- Updated corresponding example sections in `docs/api/v1.*`
Signed-off-by: Adam Lamers <adam.lamers@wmsdev.pl>
This was deprecated in d688389f4a, and has
no known external users. The deprecation was included in the 27.4.1
release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was deprecated in 27deff4da1, and has
no known external users. The deprecation was included in the 27.4.1
release, so we can remove it from master.
This patch removes the deprecated `IPV`, and `Iptables` and `IP6Tables`
consts.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ProgramChain` method was called exclusively by the bridge driver
to insert totally unrelated ipt rules in two different table-chains.
Break down this method into two functions, and move them into the bridge
pkg.
The new function `addNATJumpRules` inserts rules that aren't related to
any specific network, and depends solely on the driver config. Call it
during driver configuration instead of during network setup.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
setupIPTablesInternal is/was called from setupIPTables to set
up some rules for a network ... "internal" behaviour of the
outer function.
Then commit 7b64b1c2 added support for "--internal" bridge
networks, calling setupInternalNetworkRules from setupIPTables
instead of setupIPTablesInternal.
So, setupIPTablesInternal is the function that deals with
everything except "--internal" networks ... rename it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
- Move variable declarations closer to where they're used.
- Put comments in the blocks they apply to.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Move the exported `Command` to a platform-agnostic file, and un-export
the platform-specific implementations. This allows us to maintain the
GoDoc in a single place, describing platform-specific differences where
needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The reexec package originally was platform-agnostic, but gained some
Linux-specific handling in 1cb17f03d0.
When Windows support was implemented in Docker, the pkg/reexec package
was adjusted accordingly in 64715c4f33,
which now made the package with with either Linux or Windows, with various
other platforms (freebsd, solaris, darwin) being added back in separate
changes.
Based on the history above, this package should be platform-agnostic, except
for Linux-specific changes introduced in 1cb17f03d0
and 5aee8807a6.
This patch:
- removes the stub-implementation to make it functional on other platforms.
- renames the files for consistency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also use a slightly different name, because "reexec" is used so
widely as term in this package, making it somewhat confusing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It no longer needs to be moved to the top of the filter-FORWARD
chain after creating a new bridge network. But, it does need to
be rearranged after setting up Swarm ingress.
Similarly, the jump to DOCKER-INGRESS no longer needs to be
shuffled back to the top of the FORWARD chain after adding a
new network.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Rules no longer need to be rearranged when creating a network.
Per-network rules are always appended to the FORWARD chain so,
after adding them, there's no need to delete the per-driver
rules to re-insert them at the top of the chain.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The netfilter module is now loaded on-demand, and no longer during daemon
startup, making these fields obsolete. These fields are now always `false`
and will be removed in the next relase.
This patch deprecates:
- the `BridgeNfIptables` field in `api/types/system.Info`
- the `BridgeNfIp6tables` field in `api/types/system.Info`
- the `BridgeNFCallIPTablesDisabled` field in `pkg/sysinfo.SysInfo`
- the `BridgeNFCallIP6TablesDisabled` field in `pkg/sysinfo.SysInfo`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The output variable was renamed in 0503cf2510,
but that commit failed to change this defer, which was now checking the
wrong error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This header is sent in its canonical format; update the docs to
reflect this.
Follow-up to 76a5ca1d4d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types and functions are only used internally (through pkg/archive).
Deprecate them, and mark them for removal.
This deprecates the `Lstat()`, `Mkdev()`, `Mknod()`, `FromStatT()`
and `Stat()` functions, and related `StatT` type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 3ec19ff62b introduced a GC goroutine to delete files where netns
were mounted. It was primarly added to work around a race in kernel
3.18-4.0.1. Since no distros we support are using such old kernels,
there's no need to keep this code around.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The iptables package has two different types to specify the IP version:
IPVersion, used by iptables code, and IPV, used by firewalld code.
Both are representing the ip version as a string.
For iptables, the case doesn't matter because the string is never used
as-is. However, for firewalld the case matters.
Make the IPV type an alias of IPVersion, and deprecate it. Also change
the case used in IPVersion strings to make IPV consts aliases of
IPVersion consts.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Since 89470a7 we ignore errors returned by `LinkSetMTU` when the MTU is
greater than 1500 but lower than 65535 to let CentOS/RHEL 7 users set
an MTU in that range (despite their kernel rejecting that value).
We dropped support for those distros, so we can now remove this code
and unconditionally error out if `LinkSetMTU` returns an error.
Commit 89470a7 introduced two unit tests - these are kept, and both now
have a proper GoDoc describing what they're testing.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Historically, the `bridge` network-driver would detect whether netfiltering
was enabled in the kernel or, if disabled, try to do a `modprobe` when
initializing the driver. This approach became problematic, as loading the
module was not always performed at startup depending on daemon configuration,
or the daemon may have failed to load the module. The `/info` response
would include a warning to inform the user that some functionality may not
be available;
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Starting with db25b0dcd0, detecting whether
netfiltering is enabled now [happens when needed][1], which was further improved
on to not depend on `modprobe` in 264c15bfc4 and
4740820716.
Because of the above, the `/info` output would now return warnings in any
situation where netfiltering was not enabled on the host before the daemon
started, which may be either _incorrect_ (i.e., the module may have been
loaded afterwards), or irrelevant, because netfiltering is not needed in
all situations.
This patch removes the warnings from the `/info` response,
[1]: 944e403502/libnetwork/drivers/bridge/setup_bridgenetfiltering.go (L16-L77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch, pull progress wouldn't show the `Extracting` layer
status which made the pull look like it got stuck when extracting a big
layer.
Use the `containerd.io/snapshot/cri.layer-digest` snapshot labels to
find a corresponding snapshot and check whether it's `active` or
`committed` to set the layer status accordingly.
Despite the `cri.` component in the label name, it's not CRI specific -
it only depends on the `snapshotters.AppendInfoHandlerWrapper`.
We _could_ also use the `Usage` snapshot method to query the exact
progress of the unpack, but it would be too expensive as the
implementation time complexity will be proportional to the snapshot size.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use noop tracer provider if the OTEL exporter is not configured.
This makes the OTEL tracing avoid doing unneeded memory allocations for
spans which aren't going to be exported anywhere anyway.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- `MkdirAll` will now no longer return an `EEXIST` error if two racing
processes are creating the same directory. We will still verify that the path
is a directory, but this will avoid spurious errors when multiple threads or
programs are trying to `MkdirAll` the same path.
full diff: https://github.com/cyphar/filepath-securejoin/compare/v0.3.4...v0.3.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the third patch release of the 1.2.z release branch of runc. It
primarily fixes some minor regressions introduced in 1.2.0.
- Fixed a regression in use of securejoin.MkdirAll, where multiple
runc processes racing to create the same mountpoint in a shared rootfs
would result in spurious EEXIST errors. In particular, this regression
caused issues with BuildKit.
- Fixed a regression in eBPF support for pre-5.6 kernels after upgrading
Cilium's eBPF library version to 0.16 in runc.
full diff: https://github.com/opencontainers/runc/compare/v1.2.2...v1.2.3
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
update to the latest version of this dependency, which has a fix for a
authorization bypass in the ssh package. We don't use this functionality,
so there's no need to backport this change (other than de-noising false positives).
This is CVE-2024-45337 and Go issue https://go.dev/issue/70779.
full diff: https://github.com/golang/crypto/compare/v0.29.0...v0.31.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Mark the AllowNondistributableArtifactsCIDRs and AllowNondistributableArtifactsHostnames
fields as deprecated in all API versions, as these fields will
no longer be propagated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Non-distributable artifacts (foreign layers) were introduced in commit
05bd04350b to accommodate Windows images,
for which the EULA did not allow layers to be distributed through registries
other than those hosted by Microsoft. The concept of foreign / non-distributable
layers was adopted by the OCI distribution spec in [oci#233].
These restrictions were relaxed later to allow distributing these images
through non-public registries, for which a configuration was added in the
daemon in 67fdf574d5. In 2022, Microsoft updated
the EULA and [removed these restrictions altogether][1], and the OCI distribution
spec deprecated the functionality in [oci#965].
In 2023, Microsoft [removed the use of foreign data layers][2] for their images,
making this functionality obsolete.
This patch:
- Deprecates the `--allow-nondistributable-artifacts` daemon flag and corresponding
`allow-nondistributable-artifacts` field in `daemon.json`. Setting either
option will no longer take an effect, but a deprecation warning log is added
to raise awareness about the deprecation. This warning is planned to become
an error in the next release.
- Deprecates the `RegistryConfig.AllowNondistributableArtifactsCIDRs` and
`RegistryConfig.AllowNondistributableArtifactsHostnames` fields in the
`GET /info` API response. For API version v1.48 and lower, the fields are
still included in the response, but always `null`. In API version v1.49 and
higher, the field will be omitted entirely.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsCIDRs`
field.
- Deprecates the `api/types/registry/ServiceConfig.AllowNondistributableArtifactsHostnames`
field.
- Deprecates the `registry.ServiceOptions.AllowNondistributableArtifacts` field.
[oci#233]: https://github.com/opencontainers/image-spec/pull/233
[oci#965]: https://github.com/opencontainers/image-spec/pull/965
[1]: https://techcommunity.microsoft.com/blog/containers/announcing-windows-container-base-image-redistribution-rights-change/3645201
[2]: https://techcommunity.microsoft.com/blog/containers/announcing-removal-of-foreign-layers-from-windows-container-images/3846833
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Starting with [containerd@52f82ac] (containerd 1.7), this dependency is no
longer needed for building containerd.
[containerd@52f82ac]: 52f82acb7b
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This dependency was added in 81d704d15d, but
I could not find a reference to it, and we may not need it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was introduced in e89a5e5e91, and probably
used for devicemapper, which we no longer support, so likely unused.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We don't build .deb packages as part of the Dockerfiles in this
repository, so we can remove this dependency.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This partially reverts e440831802 ("fix and skip some tests based on
API version"), which caused the integration-cli tests to skip all
AppArmor-related tests on SUSE.
It's not really clear why this was done originally, but I have verified
that on modern SLE 12 and SLE 15 systems the AppArmor tests pass without
any adjustments needed.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This function unconditionally constructed endpoints for mirrors when
requesting endpoints for the default (Docker Hub) registry. Doing so
involves validating the config, which involves;
- parsing the hostname
- constructing TLS config
- performing a DNS lookup to resolve the host's IP address and matching
it against CIDR masks for insecure registries.
When looking up push endpoints or endpoints to consider for authentication,
mirror endpoints were discarded to prevent sending credentials of the upstream
registry to a mirror.
This patch adds a "includeMirrors" argument to skip constructing endpoints
for mirrors when not needed. While at it, also removing named output variables,
as they didn't add much.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Reverse the order in which we call v2AuthHTTPClient and http.NewRequest.
This is mostly theoretical, but v2AuthHTTPClient makes a network connection
to ping the registry, but loginV2 may fail after this if http.NewRequest
fails. Put the (lightweight) http.NewRequest first, so that we can return
early before trying to contact the registry.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fixes compatibility with alpine 3.21
- Fix additional possible `xx-cc`/`xx-cargo` compatibility issue with Alpine 3.21
- Support for Alpine 3.21
- Fix `xx-verify` with `file` 5.46+
- Fix possible error taking lock in `xx-apk` in latest Alpine without `coreutils`
full diff: https://github.com/tonistiigi/xx/compare/v1.5.0...v1.6.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Pass the context that's used for logging, and add minimal handling of
context-cancellation. Also update logs to use structured fields.
Before this patch:
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
With this patch:
DEBU[2024-12-08T15:33:50.408445543Z] Registering route method=POST path="/networks/{id:.*}/disconnect"
DEBU[2024-12-08T15:33:50.408484335Z] Registering route method=POST path=/networks/prune
DEBU[2024-12-08T15:33:50.408505251Z] Registering route method=DELETE path="/networks/{id:.*}"
Or in JSON format:
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/connect","time":"2024-12-08T15:37:19.235209667Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/{id:.*}/disconnect","time":"2024-12-08T15:37:19.235243001Z"}
{"level":"debug","method":"POST","msg":"Registering route","path":"/networks/prune","time":"2024-12-08T15:37:19.235290876Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that debug-routes are identical to regular routers, we can pass them
the same as those routers. With this, the daemon also logs those routes
as part of its startup (when in debug mode).
Before this patch, only non-debug endpoints would be logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
With this patch, debug endpoints are also logged:
DEBU[2024-12-08T15:24:47.320933959Z] Registering routers
...
DEBU[2024-12-08T15:24:47.324420709Z] Registering POST, /networks/{id:.*}/disconnect
DEBU[2024-12-08T15:24:47.324447251Z] Registering POST, /networks/prune
DEBU[2024-12-08T15:24:47.324460626Z] Registering DELETE, /networks/{id:.*}
DEBU[2024-12-08T15:24:47.324486834Z] Registering GET, /debug/vars
DEBU[2024-12-08T15:24:47.324506751Z] Registering GET, /debug/pprof/
DEBU[2024-12-08T15:24:47.324532126Z] Registering GET, /debug/pprof/cmdline
DEBU[2024-12-08T15:24:47.324549293Z] Registering GET, /debug/pprof/profile
DEBU[2024-12-08T15:24:47.324564501Z] Registering GET, /debug/pprof/symbol
DEBU[2024-12-08T15:24:47.324582043Z] Registering GET, /debug/pprof/trace
DEBU[2024-12-08T15:24:47.324604751Z] Registering GET, /debug/pprof/{name}
INFO[2024-12-08T15:24:47.324828334Z] API listen on /var/run/docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug endpoints are currently only provided non-versioned (e.g. `/debug/vars`).
While this is convenient, we "officially" deprecated non-versioned endpoints
in the API.
This patch also registers the debug-endpoints under the API-versioned paths,
so that they can be used either without version ("latest") and versioned
paths.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The debug handlers were created for GET methods, but were registered for
any method;
curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
After this patch, they're only registered with the intended method, and a
404 is returned for incorrect ones;
curl -s -XGET --unix-socket /var/run/docker.sock http://localhost/debug/vars | jq -c .cmdline
["dockerd","--debug"]
curl -s -XPOST --unix-socket /var/run/docker.sock http://localhost/debug/vars
{"message":"page not found"}
curl -s -XDELETE --unix-socket /var/run/docker.sock http://localhost/debug/vars
{"message":"page not found"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go automatically canonicalises HTTP headers, meaning the string `API-Version` passed as a header has always been returned as `Api-Version`. Similarly, `OSType` is returned as `Ostype`.
This commit updates the documentation to reflect this behaviour and modifies the codebase to ensure that input strings are aligned with their canonical output values.
Signed-off-by: maggie44 <64841595+maggie44@users.noreply.github.com>
This function was added in eb982e7c00, at
which time networking was not yet implemented for Windows, resulting
in a panic when trying to call network-related endpoints.
That's no longer the case, so we should be able to add network-endpoints
unconditionally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was added in 19515a7ad8, but looks
to be always set for endpoints used, so we can trim remote names unconditionally.
This option was added for possible future expansion, allowing registry-
mirrors to get the full reference of the image (including domain-name),
for them to host a mirror for multiple upstreams on the same registry.
That approach will unlikely be implemented, and containerd has a different
approach for this, where the reference to the original registry is passed
through a query parameter instead.
The field is unlikely used outside of our codebased, but deprecating it
before removal just in case.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was introduced in 5b752fab32,
which added documentation in the API documentation (swagger), but did
not document the field in the API go types.
This patch adds documentation, using a variant of the description used
in swagger.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These modprobes were added as a workaround in commit cce5dfe1, but
dockerd should now be able to load the modules it needs.
Signed-off-by: Rob Murray <rob.murray@docker.com>
An ioctl() call to get the "interface index" for a kernel module triggers
the kernel to try to load the module, if the process is running with
CAP_SYS_MODULE. This tends to be more reliable than "modprobe" for
docker-in-docker.
If the ioctl() method fails, fall back to trying "modprobe".
Signed-off-by: Rob Murray <rob.murray@docker.com>
Commit facb2323 aligned the way the default bridge's IPv6 subnet
and gateway addresses are selected with IPv4.
Part of that involved looking at addresses already on the bridge,
along with daemon config options. But, for IPv6, the kernel will
assign a link-local address to the bridge.
Make sure that address is ignored when selecting "bip6" when it's
not explicitly specified.
This is made slightly complicated because we allow fixed-cidr-v6
to be a link-local subnet (either the standard "fe80::/64", or
any other non-overlapping LL subnet in "fe80::/10").
Following this change, if fixed-cidr-v6 is (or is included by)
"fe80::/64", the bridge's kernel-assigned LL address may be used
as the network's gateway address - even though it may also get an
IPAM-assigned LL address.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Env var DOCKER_TEST_CREATE_DEFAULT_BRIDGE could be set to
override the name of the default bridge - without the bridge
being user-managed (unlike the '--bridge' daemon option).
It was needed by tests looking at docker0 behaviour, using
their own instance of the daemon, without breaking the
docker0 instance belonging to CI's daemon.
Now, those tests run in their own netns using the name docker0.
So, remove the unused env var.
Signed-off-by: Rob Murray <rob.murray@docker.com>
These tests create iptables rules for different addresses on
docker0 but, unlike tests that do that for user-defined bridges,
those rules aren't removed when the test deletes the network,
because the default bridge network can't be deleted.
So, use (abuse) the L3Segment code to run the tests in their
own network namespace.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When dockerd is executed with the `dockerd-rootless.sh` script, make
/etc/cdi and /var/run/cdi available to the daemon if they exist.
This makes it possible to enable the CDI integration in rootless mode.
Fixes: #47676
Signed-off-by: Rafael Fernández López <ereslibre@ereslibre.es>
Before this patch:
go test -run TestValidateManifest
ERRO[0000] image verification failed for digest sha256:02fee8c3220ba806531f606525eceb83f4feb654f62b207191b1c9209188dedd
PASS
ok github.com/docker/docker/distribution 0.008s
With this patch:
go test -run TestValidateManifest
PASS
ok github.com/docker/docker/distribution 0.010s
Note that in verbose mode, the logs are still printed, but through t.Log;
go test -run TestValidateManifest -v
=== RUN TestValidateManifest
log_hook.go:47: time="2024-12-03T13:41:19.308383552Z" level=error msg="image verification failed for digest sha256:02fee8c3220ba806531f606525eceb83f4feb654f62b207191b1c9209188dedd" func=distribution.verifySchema1Manifest file="/go/src/github.com/docker/docker/distribution/pull_v2.go:1010" testcase=TestValidateManifest
--- PASS: TestValidateManifest (0.00s)
PASS
ok github.com/docker/docker/distribution 0.011s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Test 'TestDockerCLIRunSuite/TestRunAttachFailedNoLeak' does this ...
- start a container that exits immediately, its comment says:
- "Run a dummy container to ensure all goroutines are up and running
before we get a count"
- wait for the number of goroutines to be stable for 400ms, and remember
that number
- start a container
- start another container, expecting it to fail with a port-mapping clash
- stop the running container
- wait for up to 30s for the number of goroutines to fall back to the
remembered number.
In a CI run - hacking in some debug to count goroutines once a second,
before waiting for the number to stablilise for 400ms, showed that the
initial (dummy) container run had no immediate effect. But, three more
goroutines appeared within a few seconds. For example:
=== RUN TestDockerCLIRunSuite/TestRunAttachFailedNoLeak
docker_cli_run_test.go:3822: goroutines before container run: 47 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 47 i 0 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 48 i 1 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 48 i 2 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 48 i 3 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 48 i 4 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 50 i 5 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 50 i 6 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 50 i 7 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 50 i 8 err <nil>
docker_cli_run_test.go:3830: goroutines after container run: 50 i 9 err <nil>
That means a delay while running the rest of the test risks finding the
extra goroutines that are going to start anyway and not go away (regardless
of whether more containers are started).
So - wait for the goroutine count to be stable for 7s, rather than 400ms.
Signed-off-by: Rob Murray <rob.murray@docker.com>
If the commit, import or build targets an image tag that already exists,
persist the old image under a dangling reference instead of just
deleting it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
After the image is tagged, the engine attempts to delete a dangling
image of the source image, so the image is no longer dangling.
When the source image is not dangling, the removal errors out (as
expected), but a warning is logged to the daemon log:
```
time="2024-12-02T10:44:25.386957553Z" level=warning msg="unexpected error when deleting dangling image" error="NotFound: image \"moby-dangling@sha256:54d8c2251c811295690b53af7767ecaf246f1186c36e4f2b2a63e0bfa42df045\": not found" imageID="sha256:54d8c2251c811295690b53af7767ecaf246f1186c36e4f2b2a63e0bfa42df045" spanID=bd10a21a07830d72 tag="docker.io/library/test:latest" traceID=4cf61671c2dc6da3dc7a09c0c6ac4e16
```
Remove that log as it causes unnecessary confusion, as the failure is
expected.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
isCIDRMatch defaulted to trying to resolve a hostname to get its
IP-address(es) before trying if the given host was an IP address
already.
Let's reverse the order so that we can avoid performing a DNS lookup
when it's not needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Class field was added in a12b466183 because
Docker Hub registry required a special scope to be set for pulling plugins;
HTTP/1.1 401 Unauthorized
...
Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository(plugin):vieux/sshfs:pull",error="insufficient_scope"
This is no longer a requirement, and the field is no longer set.
This patch deprecates the field and removes its use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was unused, and support for v1 registries was removed a long
time ago. It was deprecated in d43e61758a
(part of v25.0), and marked for deletion after that release.
This removes the deprecated field and types;
- `registry.APIEndpoint.Version` field
- `registry.APIVersion` type
- `registry.APIVersion1` const
- `registry.APIVersion2` const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure the module is loaded, as we're not able to load it from within
the dev-container;
time="2024-11-29T20:40:42Z" level=error msg="Running modprobe br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/5.15.0-1072-aws\n" error="exit status 1"
Also moving these steps _before_ the "print info" step, so that docker info
doesn't show warnings that bridge-nf-call-iptables and bridge-nf-call-ip6tables
are not loaded.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Same as "nat" mode, there's masquerading and port mapping from the
host - but no port/protocol filtering for direct access to the
container's address from remote hosts.
This is the old default behaviour for IPv4 when the filter-FORWARD
chain's default policy was "ACCEPT" (the daemon would only set it
to "DROP" when it set sysctl "ip_forward" itself, but it didn't set
up DROP rules for unpublished ports).
Now, port filtering doesn't depend on the filter-FORWARD policy. So,
this mode is added as a way to restore the old/surprising/insecure
behaviour for anyone who's depending on it. Networks will need to
be re-created with this new gateway mode.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This new field is used by libnetwork to determine which endpoint
provides the default gateway for a container.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This method was only used internally as part of config.Validate; deprecate
it in favor of config.Validate and make it a non-exported function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Most validation errors have the "invalid xxxxx" prefix; format this error
to be consitent with other errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Daemon.Config() option was added to expose the Config outside of the
daemon package. Limit the use of that function to that purpose and use
the internal, non-exported functions within the daemon itself; this makes
it easier to see if there's still external consumers of the Config()
method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We were discarding the error returned by it, and its functionality
is very minimal, so inline the equivalent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ignore "dataset does not exist" error in Remove function
Signed-off-by: François Scala <github@arcenik.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Checking for `/proc/sys/net/bridge` directory alone is not enough to
decide if bridge, br_netfilter module to be loaded.
Check for specific file for br_netfilter module and then
do modprobe if the file is not found under `/proc/sys/net/bridge`
Loading br_netfilter implicitly loads bridge module.
fixes: #48948
Signed-off-by: Shreenidhi Shedi <shreenidhi.shedi@broadcom.com>
Running a container with "--add-host blah:host-gateway" adds an /etc/hosts
entry for host "blah" and an address on the docker host - to give the
container a convenient way of reaching the host.
If no --host-gateway-ip option is supplied, the IPv4 address of the
default bridge is used - and that's been fine until now, it's a host
address we know will exist. But, in a container that's only connected
to IPv6-only networks, that doesn't work.
So:
- if the default bridge has an IPv6 address, create an additional
/etc/hosts entry with that adddress
- allow two --host-gateway-ip options
- at most one IPv4 and one IPv6 address
- in daemon.json, allow a JSON array value in --host-gateway-ips (plural)
- for a single address, a JSON string is also allowed
For example:
--host-gateway-ip 192.0.2.1 --host-gateway-ip 2001:db8::1111
And the daemon.json version would be:
"host-gateway-ips": ["192.0.2.1", "2001:db8::1111"]
But, this is also still valid:
"host-gateway-ip": "192.0.2.1"
Note that the /etc/hosts entries follow the usual rules. If IPv6 is
disabled in a container (by sysctl, or lack of kernel support), IPv6
addresses are not included in the file. In other cases, IPv4 and IPv6
addresses will both be included, whether or not the container currently
has network endpoints that support IPv4 or IPv6.
buildx has its own code to interpret the host-gateway-ip option. When
it's updated to understand two addresses, moby will need to pass it
both. For now, it passes an IPv4 address if there is one, else IPv6.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Buildkit *can* take a long time to start, we don't want the daemon to fail
to startup because buildkit took too long.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Use the same logic to generate IPAMConf for IPv6 as for IPv4.
- When no fixed-cidr-v6 is specified, rather than error out, use
the default address pools (as for an IPv4 default bridge with no
fixed-cidr, and as for user-defined networks).
- Add daemon option --bip6, similar to --bip.
- Necessary because it's the only way to override an old address
on docker0 (daemon-managed default bridge), as illustrated by
test cases.
- For a user-managed default bridge (--bridge), use IPv6 addresses
on the user's bridge to determine the pool, sub-pool and gateway.
Following the same rules as IPv4.
- Don't set up IPv6 IPAMConf if IPv6 is not enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a user-managed bridge is used for the default network (--bridge),
an address from the bridge determines the subnet for the network.
If a fixed-cidr is supplied, it should fall within that subnet. If it
doesn't, it's a misconfiguration - fixed-cidr is the range of
allocatable addresses, and they need to be in the network. (Either
the user's bridge is missing an address that matches their fixed-cidr
or the fixed-cidr is wrong.)
When this happens, because it's been allowed in the past (and, because
the address-pool implementation treats fixed-cidr/SubPool as an offset
into the network rather than an actual address range, so working IP
addresses would normally still be assigned to containers) ... don't
reject the config and cause daemon startup to fail. Just log a warning
and ignore fixed-cidr.
Signed-off-by: Rob Murray <rob.murray@docker.com>
For a docker-managed default bridge (docker0), when no --bip is
supplied, the gateway address and subnet size can be inferred
from existing bridge addresses.
But, if fixed-cidr's subnet size is increased so that it's biggger
than the subnet of the bridge's existing address - the bridge's
subnet needs to be incresed to match. (fixed-cidr determines the
range of addresses that can be automatically allocated, and these
should not fall outside the default bridge's subnet.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a docker-managed default bridge (docker0) already has an
address, and the fixed-cidr subnet fits within the subnet defined
by that address, the existing address should be used as the
gateway and to define the subnet.
But, when fixed-cidr is changed, no --bip is supplied, and no
existing bridge network includes fixed-cidr ... the existing
bridge address needs to be updated.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Factor out selection of addresses from an existing bridge from
the code that uses the selected address to set up IPAM for the
default bridge.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a user-managed bridge is used instead of docker0 (--bridge), with
a fixed-cidr - the bridge should have an IP address/subnet that
encompasses fixed-cidr ... the bridge address's subnet then defines
the network's subnet, and fixed-cidr defines the allocatable range
within that.
But, selection of the correct subnet/address from the bridge depended
on the address being within fixed-cidr (within the allocatable range).
This change removes that assumption. So, a bridge address with a
subnet that includes fixed-cidr is selected.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The intention may have been to only accept a globally routable
address as the gateway address on a user-supplied default bridge.
But, the test didn't do anything, so it's been allowing LL
subnets. It's too late to change that now so, remove the broken
check.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Becuase I'm about to add tests that use netlink, and the netlink
package breaks compilation under Windows.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Currently when preparing a snapshot for a container, a lease is used to
hold that snapshot for the lifespan of a container. That is workaround
to preserve the snapshot when a container is recreated, however, the
containerd object should be able to hold this reference itself.
Signed-off-by: Derek McGowan <derek@mcg.dev>
There were a handful of direct checks against errors.Is that can be
translated to assert.ErrorIs without too much thought. Unfortunately
there are a load of other examples where ErrorIs probably makes sense
but would require testing whether this subtly breaks the test.
These transformations were done by hand.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
If we have an error type that we're checking a substring against, we
should really be checking using ErrorContains to indicate the right
semantics to assert.
Mostly done using these transforms:
find . -type f -name "*_test.go" | \
xargs gofmt -w -r 'assert.Assert(t, is.ErrorContains(e, s)) -> assert.ErrorContains(t, e, s)'
find . -type f -name "*_test.go" | \
xargs gofmt -w -r 'assert.Assert(t, is.Contains(err.Error(), s)) -> assert.ErrorContains(t, err, s)'
find . -type f -name "*_test.go" | \
xargs gofmt -w -r 'assert.Check(t, is.Contains(err.Error(), s)) -> assert.Check(t, is.ErrorContains(err, s))'
As well as some small fixups to helpers that were doing
strings.Contains explicitly.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
If a values is non-nil when we don't expect it, it would be quite
helpful to get an error message explaining what happened.
find . -type f -name "*_test.go" | \
xargs gofmt -w -r "assert.Assert(t, a == nil) -> assert.Assert(t, is.Nil(a))"
find . -type f -name "*_test.go" | \
xargs gofmt -w -r "assert.Check(t, a == nil) -> assert.Check(t, is.Nil(a))"
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Migrated using
find . -type f -name "*_test.go" |
xargs gofmt -w \
-r "assert.Check(t, strings.Contains(a, b)) -> assert.Check(t, is.Contains(a, b))"
find . -type f -name "*_test.go" |
xargs gofmt -w \
-r "assert.Assert(t, strings.Contains(a, b)) -> assert.Assert(t, is.Contains(a, b))"
Using a boolean in assert.Assert or assert.Check results in error
messages that don't contain the actual problematic string, and when
running the integration suite on an actual machine (where the source
code parsing doesn't work) this makes it almost impossible to figure out
what the actual error is.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Set the daemon.json config as a string-literal in the tests, instead of
using a map[string]interface{} as intermediary format.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Field 'generic' in 'libnetwork.Network' is used to store driver options,
it has type 'options.Generic', which is 'map[string]any'.
In that map, there may be a key 'netlabel.GenericData' holding options
known as "network generic options", used for options like:
-o com.docker.network.bridge.name=br-foo
The value type for key 'netlabel.GenericData' is always 'map[string]string'
when created via an API request. But, some unit tests use type
'options.Generic'.
That works because the bridge, ipvlan and macvlan drivers look for type
'options.Generic' as well as 'map[string]string'. If they find
'options.Generic', Go reflection is used to map keys to fields of the
config struct with the expectation that the value has the same type as
that field. But, that's only used in unit tests (so the tests aren't
testing the same code path as the API would use). The 'options.Generic'
form of the bridge name option is:
"BridgeName": "br-foo"
(Because "BridgeName" is the name of the field in the bridge driver's
network config struct.)
The libnetwork code expects "network generic options" to have type
'map[string]string', and makes no provision for 'options.Generic'. So,
for example, function Network.DriverOptions will panic if called when
'Network.generic[netlabel.GenericData]' has type 'options.Generic'.
The type of 'Network.generic[netlabel.GenericData]' can't be statically
checked, because it's just a field in a 'map[string]any'.
So - remove the driver code that converts "network generic options"
from type 'options.Generic', as it's only used in tests and just makes
things more confusing.
This should reduce the chances of things appearing to work when the
type is wrong, and converting unit tests to use 'map[string]string'
means they're testing the right thing.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Add an integration test to check that a container on a network
with gateway-mode=nat can access a container on a network with
gateway-mode=routed, but not vice-versa.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The default for a user-defined chain is RETURN anyway.
This opens up the possibilty of sorting rules into two groups
by using insert or append, without having to deal with appending
after the unconditional RETURN.
Signed-off-by: Rob Murray <rob.murray@docker.com>
After an error, there's no need for it to roll back rules
it's created, the caller already does that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
We don't yet support this at the API level, so for now it returns
an error when trying to set multiple, but this makes sure that the
client types are already ready for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- 1.2.2 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.2
- 1.2.1 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.1
- 1.2.0 release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.0
Breaking changes and deprecations are included below;
Breaking changes:
Several aspects of how mount options work has been adjusted in a way that
could theoretically break users that have very strange mount option strings.
This was necessary to fix glaring issues in how mount options were being
treated. The key changes are:
- Mount options on bind-mounts that clear a mount flag are now always
applied. Previously, if a user requested a bind-mount with only clearing
options (such as rw,exec,dev) the options would be ignored and the
original bind-mount options would be set. Unfortunately this also means
that container configurations which specified only clearing mount options
will now actually get what they asked for, which could break existing
containers (though it seems unlikely that a user who requested a specific
mount option would consider it "broken" to get the mount options they
asked foruser who requested a specific mount option would consider it
"broken" to get the mount options they asked for). This also allows us to
silently add locked mount flags the user did not explicitly request to be
cleared in rootless mode, allowing for easier use of bind-mounts for
rootless containers.
- Container configurations using bind-mounts with superblock mount flags
(i.e. filesystem-specific mount flags, referred to as "data" in
mount(2), as opposed to VFS generic mount flags like MS_NODEV) will
now return an error. This is because superblock mount flags will also
affect the host mount (as the superblock is shared when bind-mounting),
which is obviously not acceptable. Previously, these flags were silently
ignored so this change simply tells users that runc cannot fulfil their
request rather than just ignoring it.
Deprecated
- runc option --criu is now ignored (with a warning), and the option will
be removed entirely in a future release. Users who need a non-standard
criu binary should rely on the standard way of looking up binaries in
$PATH.
- runc kill option -a is now deprecated. Previously, it had to be specified
to kill a container (with SIGKILL) which does not have its own private PID
namespace (so that runc would send SIGKILL to all processes). Now, this is
done automatically.
- github.com/opencontainers/runc/libcontainer/user is now deprecated, please
use github.com/moby/sys/user instead. It will be removed in a future
release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`ImageManifestDescriptor` will contain an OCI descriptor of
platform-specific manifest of the image that was picked when creating
the container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Change the persistent container metadata to store the whole platform
(as defined by OCI) instead of only the operating system.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Some other tests on this file where skipped with this same line. Let's
skip this one, that seems to be flaky too.
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
This was originally added in 54aa3a3c21,
when there was still a wide variety of storage-drivers used, and some
hosts would be running aufs of devicemapper. Let's return to make this
an explicit override if needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This code was added in 3b1d9f1a26 when the
logentries logging-driver was removed in v25.0.0. The logentries service
was already defunct, so unlikely to have any consumers, so let's remove
this code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in 18327745c00d4d2e98e5ea7241c1a1ef43b0401b;
make BIND_DIR=. shell
make -C ./internal/gocompat/
GO111MODULE=on go test -v
# github.com/docker/docker/libnetwork
../../libnetwork/sandbox.go:588:6: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL gocompat [build failed]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in 3f2e9da0100af2ceb3ef0d6431cb2b27dc3e1e49;
make BIND_DIR=. shell
make -C ./internal/gocompat/
GO111MODULE=on go test -v
# github.com/docker/docker/api/server/router/container
../../api/server/router/container/inspect.go:29:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL gocompat [build failed]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in ba454f573b02922340761c0d44687c0b5daa60a6;
make BIND_DIR=. shell
make -C ./internal/gocompat/
GO111MODULE=on go test -v
# github.com/docker/docker/daemon/containerd
../../daemon/containerd/image_inspect.go:107:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL gocompat [build failed]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Delete the entries that were added, rather than looking at the service
map (DNS config) and trying to delete entries without accounting for
the container's --hostname.
Signed-off-by: Rob Murray <rob.murray@docker.com>
I nominated Austin to become a maintainer last week, and we reached a
majority with no blocking votes, so opening a PR to move Austin to the
maintainers section.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When a container is connected to a network, it gets /etc/hosts
entries for its addresses on that network. So, when it's connected
to two networks, it has a hosts entry for each. For example, if
the hostname is the default short-id and it's connected to two
networks (172.19.0.0/16 and 172.20.0.0/17, plus IPv6 address for
each), the hosts file might include:
172.19.0.2 4b92a573912d
fd8c:c894:d68::2 4b92a573912d
172.20.0.2 4b92a573912d
fd8c:c894:d68:1::2 4b92a573912d
If the container is disconnected from 172.19.0.2, only remove
the hosts entries with addresses on that network.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Functions in this package are only used internally in the daemon for
the `/info` endpoint (Architecture), and as part of `stats` (NumProcs).
I was not able to find external consumers, but deprecating the package
first, so that we can remove / dismantle the package in a follow-up.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While the endpoint returns a detailed information about its children, it
doesn't actually expose the descriptor of the root OCI index/manifest
list.
This commits adds the target description to the returned JSON.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
While it's still turned off by default, it's easier to just flip the
`false` to `true` inside the devcontainer.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
go1.22 now creates a copy of variables when assigned in a loop; enable the
copyloopvar linter to find locations where we capture loop vars that are
now redundant; https://go.dev/ref/spec#Go_1.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/idtools/idtools_unix_test.go:188:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/service/update_test.go:290:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
runconfig/config_test.go:43:3: The copy of the 'for' variable "f" can be deleted (Go 1.22+) (copyloopvar)
f := f
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/archive/archive_test.go:820:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
distribution/pull_v2_test.go:290:3: The copy of the 'for' variable "tt" can be deleted (Go 1.22+) (copyloopvar)
tt := tt
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/network/bridge/bridge_linux_test.go:177:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/syslog/syslog_test.go:113:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/server/middleware/version_test.go:59:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/system/disk_usage_test.go:261:5: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/docker_api_containers_test.go:1748:3: The copy of the 'for' variable "x" can be deleted (Go 1.22+) (copyloopvar)
x := x
^
integration-cli/docker_api_containers_test.go:1916:3: The copy of the 'for' variable "x" can be deleted (Go 1.22+) (copyloopvar)
x := x
^
integration-cli/docker_cli_build_test.go:6203:3: The copy of the 'for' variable "builder" can be deleted (Go 1.22+) (copyloopvar)
builder := builder
^
integration-cli/docker_cli_build_test.go:6227:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/volume/volume_test.go:202:3: The copy of the 'for' variable "ep" can be deleted (Go 1.22+) (copyloopvar)
ep := ep
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/cluster/executor/container/adapter.go:449:3: The copy of the 'for' variable "mount" can be deleted (Go 1.22+) (copyloopvar)
mount := mount
^
daemon/cluster/executor/container/container_test.go:124:3: The copy of the 'for' variable "c" can be deleted (Go 1.22+) (copyloopvar)
c := c
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/container/hostconfig_test.go:94:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/build/build_test.go:95:3: The copy of the 'for' variable "c" can be deleted (Go 1.22+) (copyloopvar)
c := c
^
integration/build/build_test.go:615:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/build/build_test.go:743:3: The copy of the 'for' variable "builderVersion" can be deleted (Go 1.22+) (copyloopvar)
builderVersion := builderVersion
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/daemon/daemon_test.go:115:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/daemon/daemon_test.go:161:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/daemon/daemon_test.go:234:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/capabilities/capabilities_linux_test.go:73:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/loggerutils/logfile.go:809:3: The copy of the 'for' variable "ra" can be deleted (Go 1.22+) (copyloopvar)
ra := ra
^
daemon/logger/loggerutils/sharedtemp_test.go:117:4: The copy of the 'for' variable "i" can be deleted (Go 1.22+) (copyloopvar)
i := i
^
daemon/logger/loggerutils/sharedtemp_test.go:179:4: The copy of the 'for' variable "i" can be deleted (Go 1.22+) (copyloopvar)
i := i
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
volume/local/local_linux_test.go:232:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
volume/local/local_linux_test.go:316:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/network/ipvlan/ipvlan_test.go:499:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/daemon_unix_test.go:277:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/delete_test.go:71:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/exec_linux_test.go:65:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/info_unix_test.go:54:3: The copy of the 'for' variable "test" can be deleted (Go 1.22+) (copyloopvar)
test := test
^
daemon/runtime_unix_test.go:173:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/runtime_unix_test.go:333:3: The copy of the 'for' variable "tt" can be deleted (Go 1.22+) (copyloopvar)
tt := tt
^
daemon/seccomp_linux_test.go:194:3: The copy of the 'for' variable "x" can be deleted (Go 1.22+) (copyloopvar)
x := x
^
daemon/top_unix_test.go:88:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/ipams/defaultipam/parallel_test.go:231:3: The copy of the 'for' variable "index" can be deleted (Go 1.22+) (copyloopvar)
index := index
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
internal/mod/mod_test.go:62:3: The copy of the 'for' variable "tt" can be deleted (Go 1.22+) (copyloopvar)
tt := tt
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
distribution/xfer/download_test.go:394:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/filters/parse_test.go:543:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
volume/mounts/lcow_parser_test.go:260:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
volume/mounts/linux_parser_test.go:253:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
volume/mounts/parser_test.go:82:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
volume/mounts/validate_test.go:59:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
volume/mounts/validate_test.go:117:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
volume/mounts/windows_parser_test.go:283:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/plugin/common/plugin_test.go:46:3: The copy of the 'for' variable "ep" can be deleted (Go 1.22+) (copyloopvar)
ep := ep
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/bitmap/sequence_test.go:1210:3: The copy of the 'for' variable "tt" can be deleted (Go 1.22+) (copyloopvar)
tt := tt
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
oci/oci_test.go:157:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/networking/bridge_linux_test.go:648:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/container/attach_test.go:39:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/container_test.go:28:3: The copy of the 'for' variable "ep" can be deleted (Go 1.22+) (copyloopvar)
ep := ep
^
integration/container/create_test.go:57:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/create_test.go:120:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/create_test.go:406:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/create_test.go:583:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/exec_test.go:218:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/kill_test.go:70:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/kill_test.go:110:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/logs_test.go:130:3: The copy of the 'for' variable "tC" can be deleted (Go 1.22+) (copyloopvar)
tC := tC
^
integration/container/overlayfs_linux_test.go:59:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/resize_test.go:107:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/restart_test.go:78:5: The copy of the 'for' variable "stopDaemon" can be deleted (Go 1.22+) (copyloopvar)
stopDaemon := stopDaemon
^
integration/container/restart_test.go:188:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/run_linux_test.go:341:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/stop_linux_test.go:58:3: The copy of the 'for' variable "d" can be deleted (Go 1.22+) (copyloopvar)
d := d
^
integration/container/wait_test.go:40:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/wait_test.go:83:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/wait_test.go:133:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/container/wait_test.go:205:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/containerd/image_list.go:162:3: The copy of the 'for' variable "img" can be deleted (Go 1.22+) (copyloopvar)
img := img
^
daemon/containerd/image_delete_test.go:222:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/image_list_test.go:89:3: The copy of the 'for' variable "count" can be deleted (Go 1.22+) (copyloopvar)
count := count
^
daemon/containerd/image_list_test.go:306:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/image_test.go:145:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/platform_matchers_test.go:131:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/containerd/platform_matchers_test.go:134:5: The copy of the 'for' variable "strict" can be deleted (Go 1.22+) (copyloopvar)
strict := strict
^
daemon/containerd/platform_matchers_test.go:159:7: The copy of the 'for' variable "p" can be deleted (Go 1.22+) (copyloopvar)
p := p
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/network/endpoint_test.go:84:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
api/types/network/endpoint_test.go:166:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
api/types/network/ipam_test.go:127:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/cluster/convert/service_test.go:312:3: The copy of the 'for' variable "c" can be deleted (Go 1.22+) (copyloopvar)
c := c
^
daemon/cluster/convert/service_test.go:375:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/cluster/convert/volume_test.go:64:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/cluster/convert/volume_test.go:116:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
internal/testutils/specialimage/partial.go:41:3: The copy of the 'for' variable "platform" can be deleted (Go 1.22+) (copyloopvar)
platform := platform
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/network/macvlan/macvlan_test.go:97:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/network/macvlan/macvlan_test.go:496:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
image/cache/compare_test.go:195:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/drivers/overlay/encryption.go:682:3: The copy of the 'for' variable "sp" can be deleted (Go 1.22+) (copyloopvar)
sp := sp
^
libnetwork/drivers/overlay/encryption.go:692:3: The copy of the 'for' variable "sa" can be deleted (Go 1.22+) (copyloopvar)
sa := sa
^
libnetwork/drivers/overlay/peerdb.go:134:3: The copy of the 'for' variable "pEntry" can be deleted (Go 1.22+) (copyloopvar)
pEntry := pEntry
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
client/client_test.go:91:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
client/client_test.go:326:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
client/client_test.go:481:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
client/image_list_test.go:183:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
client/image_push_test.go:163:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
client/image_tag_test.go:50:3: The copy of the 'for' variable "repo" can be deleted (Go 1.22+) (copyloopvar)
repo := repo
^
client/image_tag_test.go:61:3: The copy of the 'for' variable "repotag" can be deleted (Go 1.22+) (copyloopvar)
repotag := repotag
^
client/ping_test.go:114:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
client/request_test.go:53:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/image/import_test.go:107:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/image/import_test.go:174:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/image/list_test.go:189:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/image/prune_test.go:193:4: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/image/pull_test.go:177:3: The copy of the 'for' variable "ref" can be deleted (Go 1.22+) (copyloopvar)
ref := ref
^
integration/image/save_test.go:136:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/image/tag_test.go:44:3: The copy of the 'for' variable "repo" can be deleted (Go 1.22+) (copyloopvar)
repo := repo
^
integration/image/tag_test.go:77:3: The copy of the 'for' variable "name" can be deleted (Go 1.22+) (copyloopvar)
name := name
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/fluentd/fluentd_test.go:172:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/server/httputils/form_test.go:152:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/network/inspect_test.go:77:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/network/network_test.go:81:3: The copy of the 'for' variable "ep" can be deleted (Go 1.22+) (copyloopvar)
ep := ep
^
integration/network/network_test.go:143:3: The copy of the 'for' variable "ep" can be deleted (Go 1.22+) (copyloopvar)
ep := ep
^
integration/network/network_test.go:221:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
builder/remotecontext/git/gitutils_test.go:116:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/idtools/idtools_unix_test.go:188:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
profiles/seccomp/kernel_linux_test.go:52:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
profiles/seccomp/kernel_linux_test.go:111:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
profiles/seccomp/seccomp_test.go:135:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
profiles/seccomp/seccomp_test.go:223:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
profiles/seccomp/seccomp_test.go:265:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/logger/loggertest/logreader.go:183:3: The copy of the 'for' variable "tt" can be deleted (Go 1.22+) (copyloopvar)
tt := tt
^
daemon/logger/loggertest/logreader.go:201:3: The copy of the 'for' variable "i" can be deleted (Go 1.22+) (copyloopvar)
i, tail := i, tail
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/plugins/client_test.go:108:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
pkg/plugins/client_test.go:132:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
opts/env_test.go:107:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
opts/opts_test.go:72:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
opts/opts_test.go:303:3: The copy of the 'for' variable "testCase" can be deleted (Go 1.22+) (copyloopvar)
testCase := testCase
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/config/config_test.go:111:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/config/config_linux_test.go:167:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/config/config_test.go:550:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
daemon/config/config_test.go:581:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/drivers/bridge/bridge_linux_test.go:1079:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
libnetwork/drivers/bridge/port_mapping_linux_test.go:763:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/endpoint_info.go:467:3: The copy of the 'for' variable "r" can be deleted (Go 1.22+) (copyloopvar)
r := r
^
libnetwork/firewall_linux_test.go:52:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
libnetwork/libnetwork_internal_test.go:403:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
libnetwork/libnetwork_linux_test.go:2045:3: The copy of the 'for' variable "i" can be deleted (Go 1.22+) (copyloopvar)
i := i
^
libnetwork/libnetwork_linux_test.go:2165:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
registry/search_endpoint_v1_test.go:140:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
registry/search_test.go:134:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
registry/search_test.go:397:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Using the same descriptions as used in docker/cli to make it
easier to compare configurations between both repositories.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.
go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When a container sends a packet to one of its own published ports on the
host, it's normally picked up by the userland proxy and sent back.
When the userland proxy is disabled, a masquerade rule is needed in
order for responses to the container to have the host's source address.
The masquerade rule matches the container's address as source and dest,
and the published port as the dest. It's only used for the no-proxy
case.
So, when the userland proxy is enabled, don't create the masquerade
rule.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The daemon no longer depends on the iptables/ip6tables filter-FORWARD
chain's policy being DROP in order to implement its port filtering
rules.
However, if the daemon enables IP forwarding in the host's system
config, by default it will set the policy to DROP to avoid potential
security issues for other applications/networks.
If docker does need to enable IP forwarding, but other applications
on the host require filter-FORWARD's policies to be ACCEPT, this
option can be used to tell the daemon to leave the policy unchanged.
(Equivalent to enabling IP forwarding before starting the daemon,
but without needing to do that.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Before this change, for IPv4:
- sysctl net.ipv4.ip_forward was enabled during bridge driver
initialisation, if:
- not already enabled
- ip-forward=true, and
- iptables=true.
- the filter-FORWARD chain's policy was set to DROP, if the daemon
updated the sysctl.
- if setting the policy failed, the sysctl change was reverted.
But, for IPv6:
- sysctls net.ipv6.conf.[default|all].forwarding were both enabled
when creating the first IPv6-enabled network, if:
- they weren't already enabled,
- ip-forward=true, and
- ip6tables=true.
- the filter-FORWARD chain's policy was set to DROP when creating
an IPv4 enabled bridge network (inc. the default bridge), if:
- ip6tables=true.
- (this happened whether or not the daemon would ever enable
IPv6 forwarding, or even create an IPv6 network.)
The bridge driver no longer needs the default policy to be DROP to
implement its own port-filtering rules. But, enabling IP forwarding
without setting the filter-FORWARD policy to DROP would potentially
be a security risk.
This change aligns IPv4 and IPv6 behaviours:
- only try to set the sysctls when creating a bridge network that
needs them (for IPv4, that's still during daemon init because
the default bridge is IPv4 enabled).
- only check/set the filter-FORWARD policy after updating sysctls.
- if the filter-FORWARD policy can't be set, treat it as an error
and revert sysctl changes.
We enabled ip6tables by default in 27.0. Setting the filter-FORWARD
policy to DROP even when no IPv6 enabled network was created
caused issues for some users. In particular, those running with
iptables=false suddenly got the IPv6 DROP policy enabled (which
broke unrelated services on the host). This change solves that by
only setting the policy when necessary.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Also make it Linux-only, as the bridge driver is Linux only and
all of the tests had skips for Windows.
Signed-off-by: Rob Murray <rob.murray@docker.com>
A dual-stack endpoint still has priority when selecting a gateway
Endpoint for a Sandbox. But, now there are IPv6-only networks, it
is possible to have a Sandbox with only IPv4-only and IPv6-only
endpoints. This change means they are both gateway endpoints.
Tell the network driver it mustn't proxy host-IPv6 to endpoint-IPv4
when there's an IPv6 gateway endpoint (which may belong to a different
net driver). Update that when networks are connected/disconnected.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When the host has IPv6 support but the container doesn't, the
default bind address is 0.0.0.0, and a port mapping does not
include a host address - the default behaviour (of the bridge
driver) is to run a docker-proxy to map from any IPv6 host
address to the IPv4 address of a container's gateway endpoint.
Driver option com.docker.network.driver.private.no_proxy_6to4
can now be used to disable that behaviour when configuring
a Sandbox's external connectivity.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The alias is not needed as the package is already named `units`.
It was also not aliases consistently across the project.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Call the respective (`etchosts.BuildNoIPv6` or `etchosts.Build`) functions
directly instead of using the intermediate `buildf` variable.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `mergeRecords` function wasn't actually _merging_ anything, but only
appended records to the existing `/etc/hosts` content. However, doing so
was split across two functions; `Add` and `mergeRecords()`;
- `Add()` obtains a lock for the given path
- then calls `mergeRecords` which reads the file-content and appends the
new records to the content.
- Closes the file and returns the new content
- Then `Add` does a `os.WriteFile` to ... the same file
Given that we're appending, we won't have to read the file's content, and
we can append to the file itself.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This regex is constructed using user-input, which could technically
produce an invalid regex.
Given that we have an error-return to our availability, let's return
any error we get, instead of panicking.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also, libnetwork: Sandbox.buildHostsFile: rename var that shadowed type
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Rob Murray <rob.murray@docker.com>
- add a `no_gogo` build-tag to allow compiling without support for gogoproto (github.com/gogo/protobuf)
- fix panic when type is not found and no handler is hit.
- fix unmarshalling of registered types
full diff: https://github.com/containerd/typeurl/compare/v2.2.0...v2.2.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
govet produces this linting warning because the Plugin types that are
compared contain a activateErr field. This should be fine to ignore here.
pkg/plugins/discovery_unix_test.go:48:7: deepequalerrors: avoid using reflect.DeepEqual with errors (govet)
if !reflect.DeepEqual(p, pp) {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
distribution/push_v2_test.go:417:7: deepequalerrors: avoid using reflect.DeepEqual with errors (govet)
if !reflect.DeepEqual(err, tc.expectedError) {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in 9ca3bb632e, but looks like
the manifest-type was never used.
distribution/manifest.go:236:7: unusedwrite: unused write to field MediaType (govet)
desc.MediaType = mt
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/images/image.go:269:9: unusedwrite: unused write to field Variant (govet)
otherN.Variant = "" // normalization adds a default variant... which is the whole problem with `platforms.Only`
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/docker_utils_test.go:492:3: shadow: declaration of "line" shadows declaration at line 491 (govet)
line := strings.TrimSpace(line)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
layer/ro_layer.go:167:6: shadow: declaration of "n" shadows declaration at line 164 (govet)
if n, err := vrc.verifier.Write(p[:n]); err != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/plugins/plugins.go:231:6: shadow: declaration of "pl" shadows declaration at line 214 (govet)
if pl, exists := storage.plugins[name]; exists {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/graphdriver/btrfs/btrfs.go:562:3: shadow: declaration of "key" shadows declaration at line 561 (govet)
key := strings.ToLower(key)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
distribution/push_v2.go:558:9: shadow: declaration of "exists" shadows declaration at line 538 (govet)
if _, exists := digestToMetadata[meta.Digest]; exists {
^
distribution/push_v2.go:562:9: shadow: declaration of "exists" shadows declaration at line 538 (govet)
if _, exists := pd.checkedDigests[meta.Digest]; exists {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/service_linux.go:226:7: shadow: declaration of "ep" shadows declaration at line 175 (govet)
if ep := sb.getGatewayEndpoint(); ep != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/containerd/image_test.go:173:3: shadow: declaration of "service" shadows declaration at line 30 (govet)
service := &ImageService{
^
daemon/containerd/image_test.go:207:3: shadow: declaration of "service" shadows declaration at line 30 (govet)
service := &ImageService{
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/containerd/image_delete.go:378:4: shadow: declaration of "img" shadows declaration at line 355 (govet)
img := images.Image{
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/bitmap/sequence_test.go:746:5: shadow: declaration of "o" shadows declaration at line 738 (govet)
if o, err := hnd.SetAnyInRange(0, uint64(blockLen), false); err == nil {
^
libnetwork/bitmap/sequence_test.go:750:5: shadow: declaration of "o" shadows declaration at line 738 (govet)
if o, err := hnd.SetAnyInRange(0, firstAv-1, false); err == nil {
^
libnetwork/bitmap/sequence_test.go:754:5: shadow: declaration of "o" shadows declaration at line 738 (govet)
if o, err := hnd.SetAnyInRange(111*uint64(blockLen), 161*uint64(blockLen), false); err == nil {
^
libnetwork/bitmap/sequence_test.go:793:6: shadow: declaration of "o" shadows declaration at line 738 (govet)
if o, err := hnd.SetAnyInRange(0, 7, false); err != nil {
^
libnetwork/bitmap/sequence_test.go:808:6: shadow: declaration of "o" shadows declaration at line 738 (govet)
if o, err := hnd.SetAnyInRange(8, 15, false); err != nil {
^
libnetwork/bitmap/sequence_test.go:824:6: shadow: declaration of "o" shadows declaration at line 738 (govet)
if o, err := hnd.SetAnyInRange(28, 29, false); err != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/graphdriver/overlay2/overlay.go:430:3: shadow: declaration of "key" shadows declaration at line 429 (govet)
key := strings.ToLower(key)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
builder/builder-next/builder.go:435:3: shadow: declaration of "id" shadows declaration at line 294 (govet)
id, ok := resp.ExporterResponse["containerimage.digest"]
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
distribution/xfer/download_test.go:72:53: G602: slice index out of range (gosec)
return createChainIDFromParent(layer.ChainID(dgsts[0]), dgsts[1:]...)
^
distribution/xfer/download_test.go:75:69: G602: slice index out of range (gosec)
dgst := digest.FromBytes([]byte(string(parent) + " " + string(dgsts[0])))
^
distribution/xfer/download_test.go:76:59: G602: slice bounds out of range (gosec)
return createChainIDFromParent(layer.ChainID(dgst), dgsts[1:]...)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not a real issue for tests, but easy to fix;
daemon/logger/splunk/splunkhecmock_test.go:79:9: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
return http.Serve(hec.tcpListener, hec)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not a real issue for tests, but easy to fix;
pkg/authorization/authz_unix_test.go:387:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not a real issue for tests, but easy to fix;
client/hijack_test.go:23:34: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These already had a comment, so let's make it a ignore-comment
integration-cli/docker_cli_exec_test.go:409:13: G302: Expect file permissions to be 0600 or less (gosec)
f, err := os.OpenFile(netFilePath, os.O_WRONLY|os.O_SYNC|os.O_APPEND, 0o644)
^
integration-cli/docker_cli_run_test.go:3050:12: G302: Expect file permissions to be 0600 or less (gosec)
if err := os.Chmod(filename, 0o646); err != nil {
^
integration-cli/docker_cli_run_test.go:3072:12: G302: Expect file permissions to be 0600 or less (gosec)
if err := os.Chmod(filename, 0o646); err != nil {
^
integration-cli/docker_cli_run_test.go:3094:12: G302: Expect file permissions to be 0600 or less (gosec)
if err := os.Chmod(filename, 0o646); err != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/docker_cli_run_test.go:401:12: G302: Expect file permissions to be 0600 or less (gosec)
f, err := os.OpenFile(filepath.Join(dir, "test"), os.O_CREATE, 0o700)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/docker_utils_test.go:187:12: G302: Expect file permissions to be 0600 or less (gosec)
f, err := os.OpenFile(dst, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0o700)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/docker_utils_test.go:149:8: G601: Implicit memory aliasing in for loop. (gosec)
m = &c
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration/plugin/logging/cmd/discard/driver.go:40:13: G302: Expect file permissions to be 0600 or less (gosec)
f, err := os.OpenFile(req.File, syscall.O_RDONLY, 0o700)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The github.com/containerd/containerd/remotes/docker/config package
was vendored incorrectly due to 5f39567e56
(update to containerd v1.7.23) being merged from an outdated branch that
missed the changes from 8b4cb6f58c that
brought in the dependency on the config package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The govet linter now uses a new schema for its config; let's remove
for now, and work on enabling more of its functionality in a follow-up.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a consistent order, which helps comparing settings with other
repositories, such as docker/cli.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
looks like this was added in [libnetwork@be153a1], but not spotted.
libnetwork/endpoint_info.go:467:20: nilness: range of nil slice (govet)
for _, r := range tStaticRoute {
^
[libnetwork@be153a1]: be153a13e4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a fixture instead of encoding with the current definition of the type,
to make sure we don't regress if any changes are made in the type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I forgot to include this patch in 96039276b6,
which introduced the encodePlatform, which already decorates the error to
have a `invalid platform:` prefix.
While updating, also be more explicit on no result being returned on error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Interface DNSBackend.ResolveName, implemented by Network,
Sandbox (and noopDNSBackend) had a bool return value that
meant 'ipv6Miss'.
But, it was always set to true on a hit, and callers had
to deal with that.
So, changed the meaning of the return value to indicate
whether the name was found - which will also work for
'ipv4Miss' when we have IPv6-only containers/networks.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a container doesn't support IPv6 and it's joined to an IPv6
network, don't allocate an IPv6 address for it.
Update the DNS resolver to understand that it can have an 'ipv6miss'
(meaning an IPv4 address exists, but no IPv6) when a network is
IPv6 enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
For Linux, delay construction and configuration of network endpoints
until the container has been created (but not started).
Signed-off-by: Rob Murray <rob.murray@docker.com>
If config for legacy links needs to be added to a libnetwork.Sandbox,
add it when constructing the Endpoint that needs it - removing the
constraint on ordering of Endpoint construction, and the dependency
between Endpoint and Sandbox construction.
So, now a Sandbox can be constructed in one place, before the first
Endpoint.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When connecting a container to a new network, its NetworkSettings were
unconditionally updated.
But, when creating a new container, they were only updated if there
were no NetworkSettings before a network was connected. But, that's
always the case - so, make the update unconditionally.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Second attempt to stop using the OCI prestart hook to call SetKey
to set up the OS Sandbox's key and perform network config in the
new network namespace.
The first attempt was reverted because it made it impossible to
use --sysctl to set per-interface sysctls on an interface that had
not yet been moved into the new network namespace.
Now, per-interface sysctls can be used to do that (with less
ambiguity because the setting is not tied to the interface using
an unpredictably assigned name).
Signed-off-by: Rob Murray <rob.murray@docker.com>
On systems using systemd to autostart Docker on boot, containers might encounter a problem where they will not have any DNS access until the container is restarted manually. This PR fixes this issue by requiring that the Docker engine service starts after nss-lookup.target. This target is reached when DNS resolution is available. See https://wiki.archlinux.org/title/Systemd#Running_services_after_the_network_is_up (paragraph "If a service needs to perform DNS queries...")
Signed-off-by: Octol1ttle <l1ttleofficial@outlook.com>
Commit 4e567e16 added firewalld to the test matrix for various CI jobs
(namely unit, integration and integration-cli).
Commit 2807c0c2 reverted that commit as it was putting too much load on
GHA cache, and thus it was returning 429 more frequently, so builds had
a greater chance of spending time building everything from scratch. This
was slowing down our CI even more than what it was before.
This new commit re-adds firewalld to the test matrix of unit,
integration and integration-cli jobs. Unlike 4e567e16, not all
combinations of OS, storage and 'mode' will be tested. Instead,
firewalld jobs will run only on ubuntu-22.04, and with the containerd
snapshotter.
Also, the revert commit mistakenly reverted a fix that was originally
intended for commit 8883db20, but was actually 'fixed up' in the wrong
commit. Let's re-revert that too.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
It only removed the test host's libnet Bolt db file, and didn't
need to do that because the tests use a TempDir for it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In 27.x and earlier releases libnetwork's database file was in a
sub-directory "network/files" under the daemon's root data dir.
That part of the path got lost in commit ed08486ec7
So, libnet data ended up in the daemon's main Bolt db. Then, on
upgrade, config in the original file was no longer accessible.
libnet doesn't need access to any data outside its sub-dir, so
change the meaning of its OptionDataDir - it now points at libnet's
sub-dir, so the db will be created in the right place. Also, update
other uses of that data dir to match.
Signed-off-by: Rob Murray <rob.murray@docker.com>
commit 77fe35b3b9 added a warning if the
"volumeDriver" option was used in combination with --mount.
That change put the warning in the CLI, which should not have to handle
this logic. Let's move it to the daemon as we have a warnings response
in the container create response.
The warning is gated by API version, so that older clients can continue
to produce the warning client-side.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Temporarily remove firewallds jobs. These are actually not running
firewalld due to a typo in GHA definitions. Moreover, these new jobs
are putting too much load on the CI, causing GHA cache backend to return
429 for many jobs. This slows down build jobs (as they can't use
existing cache), and at least one buildkit test is now failing.
This reverts commit 4e567e1622.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Multiple images with the same repository name but different tag caused
the `RepoDigests` to contain duplicated entries for each of the image.
Deduplicate the slice before setting the `RepoDigests` field.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This package was deprecated in 3a3bb1cb50,
and moved internally. The deprecation was backported to v27.1.0 through
d1ea2b1fec, so this package can be removed
for v28.0.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Anonymous volumes get a unique, 64-character name, and intended to be a new
volume (not an existing one). While it's theoretically possible for this name
to exist in other volume drivers, this would be very unlikely, so we should
not need to check other drivers to have this volume.
This patch uses the default ("local") volume-driver for anonymous volumes,
unless the user explicitly asked for a specific driver to use. Setting the
driver skips looking up existing volumes in other drivers.
Before this patch:
DEBU[2024-10-26T15:51:12.681547126Z] container mounted via snapshotter: /var/lib/docker/rootfs/overlayfs/7e947822249514b6239657a0c54d091d90e0fed4b09da472f3f6258f2b4920bc container=7e947822249514b6239657a0c54d091d90e0fed4b09da472f3f6258f2b4920bc
DEBU[2024-10-26T15:51:12.681616084Z] Creating anonymous volume volume-name=fd46d688247c3e7d39d9bae4532d6b2dc69e82e354c4a3bf305c50bbfb9ebc6c
DEBU[2024-10-26T15:51:12.681638959Z] Probing all drivers for volume volume-name=fd46d688247c3e7d39d9bae4532d6b2dc69e82e354c4a3bf305c50bbfb9ebc6c
DEBU[2024-10-26T15:51:12.681688917Z] Registering new volume reference driver=local volume-name=fd46d688247c3e7d39d9bae4532d6b2dc69e82e354c4a3bf305c50bbfb9ebc6c
With this patch:
DEBU[2024-10-27T17:28:28.574956716Z] container mounted via snapshotter: /var/lib/docker/rootfs/overlayfs/7085cb3991b61cbb79edffcb6980ad926f99f6b6b3be617cc3e3b92673cc2eb8 container=7085cb3991b61cbb79edffcb6980ad926f99f6b6b3be617cc3e3b92673cc2eb8
DEBU[2024-10-27T17:28:28.575002549Z] Creating anonymous volume driver=local volume-name=db11c053566362499103213542402af2770a6622fe7a90b9a938a5bed84ca937
DEBU[2024-10-27T17:28:28.575016299Z] Registering new volume reference driver=local volume-name=db11c053566362499103213542402af2770a6622fe7a90b9a938a5bed84ca937
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure we're asserting the right fields to be propagated, as there
are some fields that shadow top-level fields (by design).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
ParseMountRaw returns a nil value on error, so there's nothing to
check other than it not returning a value.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These errors used a backtick, which is not commonly used in our errors.
Change them to use single quotes instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These parsers can be compiled and used cross-platform, but follow platform-
specific semantics. Remove the use of runtime.GOOS, as it would result in
confusing errors ("linux does not support tmpfs").
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These log-entries were added in 10d57fde44,
but it looks like I accidentally left them as Error-logs following some
debugging (whoops!).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Note that while it is not safe to use http fallback on non-localhost
registries, this can be avoided using the new host directories. The
previous legacy insecure configuration is ambiguous and less secure.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Use the daemon's configuration to check whether the legacy registry
configuration is used. Only attempt to merge with the legacy
configuration if it has been provided. This avoids merging in based on
a defaulted legacy config.
Signed-off-by: Derek McGowan <derek@mcg.dev>
This comment was added in f49c3f287b, following
517ba44e37, which embedded the State, which
caused the JSON presentation to change.
Referring to a very old (and now removed) API version made this confusing;
while it was added to preserve the pre-v1.11 API format, it still applies
to current API versions (i.e., we cannot change this unless an explicit
API change).
This patch;
- removes the confusing comment
- touches up the comment describing the reason for embedding the State
- also mentions the State's sync.Mutex, which acts as a lock not only
for the state itself, but for the container as a whole (which was the
motivation for 517ba44e37).
- Update GoDoc for the State struct to clarify the purpose of the Mutex.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If matched escape character like \" before enter " in begin statement, text editor return invalid Highlighting.
So included escape character outside of begin statement and did some refactoring.
Signed-off-by: Myeongjooon Kim kimmj8409@gmail.com
Signed-off-by: Myeongjoon Kim <kimmj8409@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`Parser.ParseMountRaw()` labels anonymous volumes with a `AnonymousLabel` label
(`com.docker.volume.anonymous`) label based on whether a volume has a name
(named volume) or no name (anonymous) (see [1]).
However both `VolumesService.Create()` (see [1]) and `Parser.ParseMountRaw()`
(see [2], [3]) were generating a random name for anonymous volumes. The latter
is called before `VolumesService.Create()` is called, resulting in such volumes
not being labeled as anonymous.
Generating the name was originally done in Create (fc7b904dce),
but duplicated in b3b7eb2723 with the introduction
of the new Mounts field in HostConfig. Duplicating this effort didn't have a
real effect until (`Create` would just skip generating the name), until
618f26ccbc introduced the `AnonymousLabel` in
(v24.0.0, backported to v23.0.0).
Parsing generally should not fill in defaults / generate names, so this patch;
- Removes generating volume names from `Parser.ParseMountRaw()`
- Adds a debug-log entry to `VolumesService.Create()`
- Touches up some logs to use structured logs for easier correlating logs
With this patch applied:
docker run --rm --mount=type=volume,target=/toto hello-world
DEBU[2024-10-24T22:50:36.359990376Z] creating anonymous volume volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
DEBU[2024-10-24T22:50:36.360069209Z] probing all drivers for volume volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
DEBU[2024-10-24T22:50:36.360341209Z] Registering new volume reference driver=local volume-name=0cfd63d4df363571e7b3e9c04e37c74054cc16ff1d00d9a005232d83e92eda02
[1]: 032721ff75/volume/service/service.go (L72-L83)
[2]: 032721ff75/volume/mounts/linux_parser.go (L330-L336)
[3]: 032721ff75/volume/mounts/windows_parser.go (L394-L400)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We already check if
- the key is set (otherwise default)
- a value is set (otherwise default and error)
This check can be simplified to check if they're equal (boolean cannot be both
true and false), or both false (boolean must be either true or false), although
the latter could be considered for a tri-state boolean (but we already do this
through the "not set" case).
We may need some additional checks, for example, currently it ignores invalid
values if the filter contains at least one valid one (e.g. ["true", "bananas"]).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Removes dependency on various libcontainer packages;
- github.com/opencontainers/runc/libcontainer/system
- github.com/opencontainers/runc/libcontainer/user
- github.com/opencontainers/runc/libcontainer/userns
full diff: https://github.com/opencontainers/runc/compare/v1.1.14..v1.2.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit f143f4ec51 introduced platform support
when loading images. However when loading a specific platform variant from
a tar that contains multiple, we should not ignore cases if that platform is
missing.
Before this patch, the missing platform was silently ignored, potentially
loading an empty image:
$ docker image load -i image.tar --platform=linux/riscv64
Loaded image: alpine:latest
$ docker image ls --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 0B 0B
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Added in 26.1.0, commit 6c68be24a2
Default changed to true in 27.0.0, commit 33f9a5329a
No sign of problems so, remove.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Patch from af6ada910f
Without this change, if a long-lived process uses the client and creates
connections, these connections are not released and grow over time.
We can also look into addressing this issue from the server side, but it
doesn't hurt for the `client` package to have good defaults and not
cause this.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
actions/setup-go complains:
> Restore cache failed: Dependencies file is not found in
> /home/runner/work/moby/moby. Supported file pattern: go.sum
Let's give it one to chew.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
These errors were deprecated in 3cf90ca73f
in favor of using errdefs types. They're no longer used, so we can remove
them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Daemon.containerRoot was only used in a single place, but defined far
from where it was used, and Daemon.newBaseContainer was a _very_ thin
abstraction on top of container.NewBaseContainer.
Let's remove these.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This logic is going to be updated to use the new containerd resolver and
needs all the logic handling resolution in the package where it is used.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Add an implementation of label store that's needed to create a local
content store for testing purposes.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This dependency comes from SwarmKit, but didn't update dependencies
for quite some time. Update the dependencies to their latest patch
release.
For some reason, updating this dependency brings back various legacy
versions of dependencies through `github.com/coreos/etcd/clientv3/concurrency`;
go: finding module for package github.com/coreos/etcd/clientv3/concurrency
go: finding module for package github.com/coreos/etcd/clientv3
go: found github.com/coreos/etcd/clientv3 in github.com/coreos/etcd v3.3.27+incompatible
go: found github.com/coreos/etcd/clientv3/concurrency in github.com/coreos/etcd v3.3.27+incompatible
go: finding module for package github.com/coreos/go-systemd/journal
go: finding module for package github.com/coreos/pkg/capnslog
go: found github.com/coreos/pkg/capnslog in github.com/coreos/pkg v0.0.0-20240122114842-bbd7aa9bf6fb
go: found github.com/coreos/go-systemd/journal in github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf
The above resolution results in these legacy versions to be added;
github.com/coreos/etcd v3.3.27+incompatible // indirect
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf // indirect
github.com/coreos/pkg v0.0.0-20240122114842-bbd7aa9bf6fb // indirect
However, none of those are _used_, and they don't appear as vendored file.
That change happens with v3.5.7 and up, but nothing immediately stands
out what change causes this. Possibly it's due to the switch to go1.17
in go.mod, which is the first version that flattened indirect dependencies,
potentially causing intermediate dependency versions to be used during
resolution; https://github.com/etcd-io/etcd/compare/server/v3.5.6...server/v3.5.7
changelog: f0187c38d1/CHANGELOG/CHANGELOG-3.5.md
full diff: https://github.com/etcd-io/etcd/compare/server/v3.5.6...server/v3.5.16
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was poorly documented as it uses fuzzy matching under the hood,
and it's no longer used. Mark it as deprecated, and to be removed in the
next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was poorly documented as it uses fuzzy matching under the hood,
and it's no longer used. Mark it as deprecated, and to be removed in the
next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
generateHostname was only used here, and defined as a method on Daemon
without depending on the daemon type. Inline the function and simplify
the logic (as both the "network=host" and non-host code only had to be
executed if no hostname was set).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Mostly a "nit", but it makes it clearer that we're returning an empty result,
and not a (partially) propagated struct.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
GenerateRandomID has a check to verify if the generated ID was numeric. This
check was added because a container's short-ID is used as default hostname for
containers, which isn't allowed to be consisting of only numbers (see [moby#3869]
and https://bugzilla.redhat.com/show_bug.cgi?id=1059122.
Producing an random ID with only numbers is a rare corner-case, but the check
would always be executed and wasn't optimized.
This patch applies some optimizations:
- The code was using `strconv.ParseUInt`, which has additional checks for
signs ("+" or "-"); `hex.EncodeToString` would never produce these, so
we can use `strconv.ParseInt` instead (which doesn't have these checks).
- The code was using `TruncateID(id)` to get the short-ID. The `TruncateID`
function is designed to also handle digests, and for that checks for
the given ID to contain colons (`:`), which it would split to remove
the algorithm (`sha256:`) before truncating to the short-ID length.
That check wasn't needed either, because those would not be produced
by `hex.EncodeToString`, so instead, we can just truncate the ID.
- Finally, all we _really_ need to check for is if the ID consists of only
numeric characters (`0-9`) so, let's do just that; if any non-numeric
value is found, the ID is valid, and we can terminate the loop.
I did some basic benchmark to compare all of the above in isolation;
- BenchmarkParseInt: `strconv.ParseInt(TruncateID(id), 10, 64)`
- BenchmarkParseUInt: `strconv.ParseUint(TruncateID(id), 10, 64)`
- BenchmarkParseUIntNoTrunc: `strconv.ParseUint(id[:shortLen], 10, 64)`
- BenchmarkAllNum: `allNum(id[:shortLen])`
Results of the above:
BenchmarkParseInt-10 1713937 691.0 ns/op 480 B/op 18 allocs/op
BenchmarkParseIntNoTrunc-10 3385483 356.1 ns/op 480 B/op 18 allocs/op
BenchmarkParseUInt-10 2112538 567.7 ns/op 384 B/op 12 allocs/op
BenchmarkParseUIntNoTrunc-10 4325847 266.7 ns/op 384 B/op 12 allocs/op
BenchmarkAllNum-10 77277264 15.29 ns/op 0 B/op 0 allocs/op
Difference for `GenerateRandomID` as a whole is less dramatic, as in most
cases `ParseInt` would bail out early, but still saves some allocations, and
performance is ~14% better:
BenchmarkGenerateRandomID-10 2807764 424.5 ns/op 240 B/op 6 allocs/op
BenchmarkGenerateRandomIDNew-10 3288866 366.6 ns/op 160 B/op 3 allocs/op
[moby#3869]: https://github.com/moby/moby/issues/3869
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were deprecated in 7ce1edd7c6, which
is part of v27.0.0. Move them to a test-file as they were only used for
tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These tests were named confusingly as they're testing `TruncateID`.
While renaming, let's also combine them into a single test using
a test-table, so that the test-cases can carry some description
what they're testing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- `IsShortID` was deprecated in 2100a70741
- `ValidateID` was deprecated in e19e6cf7f4
Both are part of 27.0, so we can remove these.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was originally added in 3716ec25b4
at which time it was not yet implemented for Windows, so this utility was
used to print a warning when trying to use it on that platform.
Windows support was added in bd4e8aa64e, which
kept the utility, but adjusted it to support for both Windows and Linux, and
excluding any other platform.
Let's remove this utility, given that we currently only support Windows and
Linux (there's been some partial support for other platforms, but they are
very likely broken in many ways).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was originally added in 9e9fc7b57c
at which time it was not yet implemented for Windows, so this utility was
used to print a warning when trying to use it on that platform.
Windows support was added in e0d533b1e8, which
kept the utility, but adjusted it to support for both Windows and Linux, and
excluding any other platform.
Let's remove this utility, given that we currently only support Windows and
Linux (there's been some partial support for other platforms, but they are
very likely broken in many ways).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was only used internally in the daemon. This patch splits
the implementation to a non-exported version and deprecates the exported
one.
While at it, also pass through the context (which is used for tracing),
and added a note about the function potentially not being atomic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `manifests` option, as used for the `--tree` option on `docker image ls`
currently sorts manifests to put those that are present first. The intent was
to present "available" images at the top of each tree, followed by images that
were not pulled.
However, there's some limitations to this. First of all, the current approach
makes the output non-deterministic as the order in which variants are pulled
determines the order in which they're presented, i.e., the last pulled variant
is returned first (I omitted some variants in the example for brevity);
Here's the result of pulling `linux/riscv64`, then pulling `linux/arm64`;
docker pull --platform=linux/riscv64 alpine:latest
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 10.6MB 3.37MB
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
├─ linux/amd64 33735bd63cf8 0B 0B
└─ linux/arm64/v8 9cee2b382fe2 0B 0B
docker pull --platform=linux/arm64 alpine:latest
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 24.2MB 7.46MB
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
└─ linux/amd64 33735bd63cf8 0B 0B
Repeating the steps but in reverse order results in the output to be reversed;
docker image rm alpine:latest
docker pull --platform=linux/arm64 alpine:latest
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 13.6MB 4.09MB
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
├─ linux/amd64 33735bd63cf8 0B 0B
└─ linux/riscv64 80cde017a105 0B 0B
docker image ls -a --tree
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 24.2MB 7.46MB
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
└─ linux/amd64 33735bd63cf8 0B 0B
The second limitation is that order sometimes matters; when matching a
platform from a manifest-index, implementations may find multiple suitable
candidates. In most cases the _most_ suitable candidate can be selected
(e.g., prefer `linux/arm/v7` over `linux/arm/v6`), but manifest-indices do
allow multiple entries for the same platform, in which case implementations
match the first entry found.
While these situations will be less common (and usually due to incorect use
of tooling such as `docker manifest`), being able to observe the order in
which manifests appeared in the index can help debugging or help the user
understand why a specific variant was selected.
We should therefore not re-order these manifests, and return them in the
order in which they appeared. If we decide to present "present" variants
before "non-present" variants, we can do this ordering on the client side.
With this patch applied;
docker pull --quiet --platform=linux/riscv64 alpine:latest
docker pull --quiet --platform=linux/arm64 alpine:latest
docker image ls --tree alpine
IMAGE ID DISK USAGE CONTENT SIZE USED
alpine:latest beefdbd8a1da 24.2MB 7.46MB
├─ linux/amd64 33735bd63cf8 0B 0B
├─ linux/arm/v6 50f635c8b04d 0B 0B
├─ linux/arm/v7 f2f82d424957 0B 0B
├─ linux/arm64/v8 9cee2b382fe2 13.6MB 4.09MB
├─ linux/386 b3e87f642f5c 0B 0B
├─ linux/ppc64le c7a6800e3dc5 0B 0B
├─ linux/riscv64 80cde017a105 10.6MB 3.37MB
└─ linux/s390x 2b5b26e09ca2 0B 0B
Which matches the order of the manifests in the index:
docker buildx imagetools inspect --raw alpine:latest | jq -c .manifests[].platform
{"architecture":"amd64","os":"linux"}
{"architecture":"arm","os":"linux","variant":"v6"}
{"architecture":"arm","os":"linux","variant":"v7"}
{"architecture":"arm64","os":"linux","variant":"v8"}
{"architecture":"386","os":"linux"}
{"architecture":"ppc64le","os":"linux"}
{"architecture":"riscv64","os":"linux"}
{"architecture":"s390x","os":"linux"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When running docker export command the container layer
is only released in case there is an error.
This makes the daemon not being able to remove
them when using docker rmi or docker system prune
leaving the container layer and the image used
in an orphaned state on the docker file system.
After applying this patch, the layer is always
released allowing dockerd to remove/prune it.
Signed-off-by: Joan Grau <grautxo.dev@proton.me>
I noticed that the daemon logs, even in debug, did not give any
indication that the build was cancelled. This patch adds some logs
to help debugging.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
OTEL meter implementation has a memory leak issue which causes each
meter counter invocation to create a new instrument when the meter
provider is not set.
Also add a test, which will fail once a fixed OTEL is vendored.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Now `dockerd-rootless-setuptool.sh install --force` ignores errors
from `rootlesskit`.
This might be useful when installing Rootless Docker into a container
image with `RUN` instructions.
Fix issue 48678
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
It was used in a single place, and the containerd/platforms module now
provides a format for a platform including os.version (through FormatAll),
which means we no longer have to create our own formatting for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Mostly theoretical, but let's be correct here. It's worth noting that the API
(backend) accepts uint32, but container.ResizeOptions uses uint (uint64). We
could decide to add checks for this on the client side, or to change the
type (but that would be a breaking change).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Containerd accepts uints for these, so make the backend signature align
with that, so that we don't have to cast values. Also pass the context
along.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In release 27.0, ip6tables was enabled by default. That caused a
problem on some hosts where iptables was explicitly disabled and
loading the br_netfilter module (which loads with its nf-call-iptables
settings enabled) caused user-defined iptables rules to block traffic
on bridges, breaking inter-container communication.
In 27.3.0, commit 5c499fc4b2 delayed
loading of the br_netfilter module until it was needed. The load
now happens in the function that sets bridge-nf-call-ip[6]tables when
needed. It was only called for icc=false networks.
However, br_netfilter is also needed when userland-proxy=false.
Without it, packets addressed to a host-mapped port for a container
on the same network are not DNAT'd properly (responses have the server
container's address instead of the host's).
That means, in all releases including 26.x, if br_netfilter was loaded
before the daemon started - and the OS/user/other-application had
disabled bridge-nf-call-ip[6]tables, it would not be enabled by the
daemon. So, ICC would fail for host-mapped ports with the userland-proxy
disabled.
The change in 27.3.0 made this worse - previously, loading br_netfilter
whenever iptables/ip6tables was enabled meant that bridge-netfiltering
got enabled, even though the daemon didn't check it was enabled.
So... check that br_netfilter is loaded, with bridge-nf-call-ip[6]tables
enabled, if userland-proxy=false.
Signed-off-by: Rob Murray <rob.murray@docker.com>
- Use fixtures for the JSON strings
- Add test-cases for invalid / malformed JSON
- Check error-message produced
- Add test for "happy path"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, the DOCKER-USER chains are set up on firewall reload or network
creation. If there are running containers at startup, configureNetworking won't
be called (daemon/daemon_unix.go), so the user chains won't be setup.
This commit puts the setup logic on a separate function, and calls it on the
original place and on initNetworkController.
Signed-off-by: Andrés Maldonado <maldonado@codelutin.com>
Rewrite the debug-logs produced
- Use structured logs
- Combine into a single log per request, instead of separate log-entry
for the "form-data".
- Include error-messages returned by the handler ("error-response" field)
- Include HTTP status-code returned ("status" field)
- Include the "vars" as a field; these are fields extracted from the URL
and passed to the handler
Examples below are logs for:
docker ps
docker container inspect nosuchcontainer
docker volume create --name foo
Before this change:
DEBU[2024-10-16T10:59:40.484254465Z] Calling HEAD /_ping spanID=43d76043f8e30dbb traceID=04f980a33901f35ba33c3927d3bb4bbb
DEBU[2024-10-16T10:59:40.485551840Z] Calling GET /v1.47/containers/json spanID=b9979f2b36572a43 traceID=5c2167537df2dede6bdbab030f8350bc
DEBU[2024-10-16T11:00:00.374864502Z] Calling HEAD /_ping spanID=d637e39684d56a16 traceID=efaed7838901dd6a597c5446ce3f83e2
DEBU[2024-10-16T11:00:00.384198127Z] Calling GET /v1.47/containers/nosuchcontainer/json spanID=f9cc4520b95d814b traceID=c15ae04ca248929d6e52474e711d48b0
DEBU[2024-10-16T11:00:11.576426632Z] Calling HEAD /_ping spanID=2bc30d2be873a8e5 traceID=53ccc3d2af87aa5425421306906660a6
DEBU[2024-10-16T11:00:11.588877966Z] Calling POST /v1.47/volumes/create spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
DEBU[2024-10-16T11:00:11.589198966Z] form data: {"Driver":"local","Name":"foo"} spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
DEBU[2024-10-16T11:00:11.594828216Z] using regular volume spanID=30816d2b51dd75b2 traceID=020b0e612195466468b46eb0d35a8f23
After this:
When using plain-text, we continue encoding the form-data to JSON, but
as it's now in a field, it'll be shown escaped;
DEBU[2024-10-16T11:17:35.465777379Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=9b7ea0288b2b70c3 status=200 traceID=94ef9345624e92ac0263931fbe9e15db vars="map[]"
DEBU[2024-10-16T11:17:35.468050171Z] handling GET request method=GET module=api request-url=/v1.47/containers/json spanID=04675edee7b5ec9d status=200 traceID=a9d81dcdbf2650fa6d794a7a856fb66b vars="map[version:1.47]"
DEBU[2024-10-16T11:17:38.502289297Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=7c43a8dfd8fb5043 status=200 traceID=7a2a7c71cd421570e811474749a04ccd vars="map[]"
DEBU[2024-10-16T11:17:38.504847506Z] handling GET request error-response="No such container: nosuchcontainer" method=GET module=api request-url=/v1.47/containers/nosuchcontainer/json spanID=ab721bbbe5cf8035 status=404 traceID=4a08dcd5054fc8090e3af8846beea10d vars="map[name:nosuchcontainer version:1.47]"
DEBU[2024-10-16T11:17:40.788838340Z] handling HEAD request method=HEAD module=api request-url=/_ping spanID=2dbc18ba1334635b status=200 traceID=ea9af681d096dc4a2c2f2ed7338ea417 vars="map[]"
DEBU[2024-10-16T11:17:40.790496465Z] handling POST request form-data="{\"Driver\":\"local\",\"Name\":\"foo\"}" method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"
The alternative to the above would be to unconditionally set it as-is,
but in that case it would use Go's formatting for `map[string]any`;
DEBU[2024-10-16T11:27:54.937232805Z] handling POST request form-data="map[Driver:local Name:foo]" method=POST module=api request-url=/v1.47/volumes/create spanID=2d7985a900791bf6 status=200 traceID=33feab9fd5feba3b0f4b6ec5a6971a67 vars="map[version:1.47]"
Or to use some trickery to not quote this specific field, but that may limit the
output from being parsable;
DEBU[2024-10-16T11:17:40.790496465Z] handling POST request form-data={"Driver":"local","Name":"foo"} method=POST module=api request-url=/v1.47/volumes/create spanID=03690760b6f6dec4 status=200 traceID=79a985fff0dd5fac7c90d36b04941e0a vars="map[version:1.47]"
When using `--log-format=json`, the form-data is kept as structured, becoming
part of the main JSON struct:
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"166dc12eeeadf82b","status":200,"time":"2024-10-16T11:16:09.427380423Z","traceID":"7f4f2501eee3b15ae608481ba214bd56","vars":{}}
{"level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/json","spanID":"bf95e2ce9eca41c2","status":200,"time":"2024-10-16T11:16:09.429077631Z","traceID":"041b26b30dacc240e8e3afc9c567195d","vars":{"version":"1.47"}}
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"454953906c36ea6b","status":200,"time":"2024-10-16T11:16:13.455633008Z","traceID":"3ffc0a256d6ec1a56cd7f6bf1008e55d","vars":{}}
{"error-response":"No such container: nosuchcontainer","level":"debug","method":"GET","module":"api","msg":"handling GET request","request-url":"/v1.47/containers/nosuchcontainer/json","spanID":"dcf0d42921928b29","status":404,"time":"2024-10-16T11:16:13.460309925Z","traceID":"fdfd2c89941c9c7a459bec7a05e46ef8","vars":{"name":"nosuchcontainer","version":"1.47"}}
{"level":"debug","method":"HEAD","module":"api","msg":"handling HEAD request","request-url":"/_ping","spanID":"701dc623cf1b0253","status":200,"time":"2024-10-16T11:16:16.155730884Z","traceID":"786885a9f79cbfba99097eeb4145ca1e","vars":{}}
{"form-data":{"Driver":"local","Name":"foo"},"level":"debug","method":"POST","module":"api","msg":"handling POST request","request-url":"/v1.47/volumes/create","spanID":"dc1429c1c636b30a","status":200,"time":"2024-10-16T11:16:16.162002426Z","traceID":"fc49ee4a7acafbbb8eb50ed34c434765","vars":{"version":"1.47"}}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use more structured logs, and provide a human-readable presentation of the signal
that's sent. For the human-readable presentation, we should probably look at
converting back to the signal _names_ (e.g. `SIGWINCH` or `SIGKILL`), which may
be easier to interpret, but we currently don't have a utility for that.
Before:
DEBU[2024-10-14T10:24:51.538705343Z] Sending kill signal 28 to container 7e9e99e52fc69ef1038b2fd212c8dc18948a56e6f024fbe46192f43006a229aa
DEBU[2024-10-14T10:24:51.740502218Z] Calling POST /v1.47/containers/7e9e99e52fc69ef1038b2fd212c8dc18948a56e6f024fbe46192f43006a229aa/kill?signal=WINCH spanID=9b993a93d28479f3 traceID=a37022e0429abaf9fb8b66a6cd4e4a19
DEBU[2024-10-14T10:24:51.740874218Z] Sending kill signal 28 to container 7e9e99e52fc69ef1038b2fd212c8dc18948a56e6f024fbe46192f43006a229aa
DEBU[2024-10-14T10:24:51.740501843Z] Calling POST /v1.47/containers/7e9e99e52fc69ef1038b2fd212c8dc18948a56e6f024fbe46192f43006a229aa/resize?h=39&w=127 spanID=f1563bdd86230804 traceID=9c25ff5910b30a4a04b774c8f5d0160e
After:
DEBU[2024-10-15T17:17:18.988605173Z] Calling POST /v1.47/containers/cafc94ca93a8e10eb79ce86235c4510d6bba1dab9cf4827abe490328148418c8/kill?signal=WINCH spanID=491d75545f89902a traceID=de72bdd1130bfc010ff1172ac23695b3
DEBU[2024-10-15T17:17:18.988763173Z] sending signal 28 (window changed) to container container=cafc94ca93a8e10eb79ce86235c4510d6bba1dab9cf4827abe490328148418c8 signal=28
DEBU[2024-10-15T17:17:18.988605214Z] Calling POST /v1.47/containers/cafc94ca93a8e10eb79ce86235c4510d6bba1dab9cf4827abe490328148418c8/resize?h=46&w=152 spanID=8b18f64b12931da2 traceID=0a38e4a16dbbfda72172209382faec91
...
...
DEBU[2024-10-15T20:26:16.863097005Z] sending signal 1 (hangup) to container container=824197a9af794c4bcda13914021f13d702954114d3410c9db629a51bf685bdc7 signal=1
DEBU[2024-10-15T20:26:31.431432554Z] sending signal 10 (user defined signal 1) to container container=824197a9af794c4bcda13914021f13d702954114d3410c9db629a51bf685bdc7 signal=10
DEBU[2024-10-15T19:52:41.717507211Z] shutting down container container=824197a9af794c4bcda13914021f13d702954114d3410c9db629a51bf685bdc7
DEBU[2024-10-15T19:52:41.717681920Z] sending signal 15 (terminated) to container container=824197a9af794c4bcda13914021f13d702954114d3410c9db629a51bf685bdc7 signal=15
Or in JSON format:
{"level":"debug","msg":"Calling POST /v1.47/containers/6a62783bc6a591381dd625b8dca20bebf6b0f6e927956b92a4c8ea0438f2ff76/kill?signal=WINCH","spanID":"d7622e49d248a2e5","time":"2024-10-15T19:54:36.258464042Z","traceID":"8dcc62a38b0289c9eeb7d9fa7f9a485d"}
{"container":"6a62783bc6a591381dd625b8dca20bebf6b0f6e927956b92a4c8ea0438f2ff76","level":"debug","msg":"sending signal 28 (window changed) to container","signal":28,"time":"2024-10-15T19:54:36.258546167Z"}
{"level":"debug","msg":"Calling POST /v1.47/containers/6a62783bc6a591381dd625b8dca20bebf6b0f6e927956b92a4c8ea0438f2ff76/kill?signal=WINCH","spanID":"0c908cb6fe55a921","time":"2024-10-15T19:54:36.458532084Z","traceID":"a0225edfaa0b3c3b0ce93e3d2c98f326"}
{"container":"6a62783bc6a591381dd625b8dca20bebf6b0f6e927956b92a4c8ea0438f2ff76","level":"debug","msg":"sending signal 28 (window changed) to container","signal":28,"time":"2024-10-15T19:54:36.458614126Z"}
{"level":"debug","msg":"Calling POST /v1.47/containers/6a62783bc6a591381dd625b8dca20bebf6b0f6e927956b92a4c8ea0438f2ff76/resize?h=50\u0026w=167","spanID":"1679a419b3f8b5e4","time":"2024-10-15T19:54:36.458560459Z","traceID":"cad46e855dc5975799a7c82bdbed1b81"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon used to have various implementation to adjust the container-inspect
output for different API versions, which could return different go structs,
and because of that required a function with a `interface{}` output type.
Most of those adjustments have been removed, and we no longer need separate
types for backward compatibility with old API versions.
This patch;
- Removes the Daemon.ContainerInspectCurrent method
- Introduces a backend.ContainerInspectOptions struct
- Updates the Daemon.ContainerInspect method's signature to accept the above
- Moves API-version specific adjustments to api/server/router/container,
similar to how such adjustments are made for other endpoints.
Note that we should probably change the backend's signature further,
and define separate types for the backend's inspect and the API's
inspect response. Considering that the Backend signatures should be
considered "internal", we can do that in a future change.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's good practice to use a consistent name; we couldn't use `r` as name,
as it's used for the request argument, and `s` honestly didn't make much
sense as name, so changing it to `c`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In an integration test - run a daemon, capture iptables, and feed them
to a markdown text/template describing them.
Prep for repeating that, for different network configurations.
Fail the test if the generated markdown differs from a "golden" version.
(So, at-least the golden markdown will need to be updated if the
iptables rules are deliberately changed - hopefully the corresponding
description in the template will also be updated.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
This patch fixes a bug in the daemon's restore step on startup, where
a container with a name matching another container's ID would not be
restored.
`Daemon.registerName` is used during startup as part of the daemon's
container restore code
97b1233a15/daemon/daemon.go (L331-L344)
In that process, it first registers the containers names through
[`Daemon.registerName()`][1], then registers the container's ID through
[`Daemon.Register()`][1], which calls `Daemon.containers.Add()` under the
hood.
Restoring containers is done in a goroutine, and at this stage of the daemon's
lifecycle, not all containers may be restored yet. However, `Daemon.registerName()`
has some safeguard to prevent the same container from being restored _twice_
through [`Daemon.Exists()`][3]. If a duplicate is found, an error is logged, and
the container is not restored (but kept on disk).
While it's disputable if this logic is needed at all, perhaps a panic would be
more appropriate (duplicate containers were stored on disk), there's also a
flaw in the current implementation of this check.
The [`Daemon.Exists()`][3] function uses [`Daemon.GetContainer()`][4] to look
up the container. This function performs fuzzy matching on the given reference,
first trying to match containers on their full ID, which _should_ not give a
match at this stage, before falling back to matching containers by name and
partial prefix.
This last part can be problematic in situations where a container exists that
uses the container to restore's ID as name. In such cases, the container will
be considered "already present", and not restored.
Create a container, then create a number of containers, each of which using
the ID of the previous container as name.
docker create --name one hello-world
d54301b7560f3c3544acc2d9c9dd55a194d6db37c2af64fe83fa34238c7ce6ab
docker create --name d54301b7560f3c3544acc2d9c9dd55a194d6db37c2af64fe83fa34238c7ce6ab hello-world
217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d
docker create --name 217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d hello-world
b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4
docker create --name b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4 hello-world
The daemon should now have a number of containers where the ID and name
conflict:
docker ps -a --no-trunc --format 'table {{.ID}}\t{{.Names}}'
CONTAINER ID NAMES
f59e8e4044471c45d4c9841d11a2c586cbfa4703b1344035fd51a15e15899ea7 b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4
b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4 217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d
217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d d54301b7560f3c3544acc2d9c9dd55a194d6db37c2af64fe83fa34238c7ce6ab
d54301b7560f3c3544acc2d9c9dd55a194d6db37c2af64fe83fa34238c7ce6ab one
Restart the daemon. Depending on the order in which containers are restored,
a conflict may happen, and the conflicting container will not be restored.
Logs below are from the daemon with debug enabled;
INFO[2024-10-15T11:13:38.770744797Z] Loading containers: start.
DEBU[2024-10-15T11:13:38.771152214Z] processing event stream module=libcontainerd namespace=moby
DEBU[2024-10-15T11:13:38.771599797Z] loaded container container=d54301b7560f3c3544acc2d9c9dd55a194d6db37c2af64fe83fa34238c7ce6ab paused=false running=false
DEBU[2024-10-15T11:13:38.771637464Z] loaded container container=217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d paused=false running=false
DEBU[2024-10-15T11:13:38.771672714Z] loaded container container=bbe03a6554867810c2d7464ed3cb853865c755bae797b8d1f4caf60fb3f9fa04 paused=false running=false
DEBU[2024-10-15T11:13:38.771765297Z] loaded container container=f59e8e4044471c45d4c9841d11a2c586cbfa4703b1344035fd51a15e15899ea7 paused=false running=false
DEBU[2024-10-15T11:13:38.771780839Z] loaded container container=b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4 paused=false running=false
ERRO[2024-10-15T11:13:38.772114505Z] failed to register container name: /217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d container=b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4 error="container is already loaded"
And the conflicting container (`217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d`)
is not present:
docker ps -a --no-trunc --format 'table {{.ID}}\t{{.Names}}'
CONTAINER ID NAMES
f59e8e4044471c45d4c9841d11a2c586cbfa4703b1344035fd51a15e15899ea7 b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4
b125af485d6d1875b586b314f93af1b49d5baaa94cec4199ae4ef4c6da05e7e4 217c53b9826eb7875ca2620596864d039848470befeb5f963b3ebffe509e7a6d
d54301b7560f3c3544acc2d9c9dd55a194d6db37c2af64fe83fa34238c7ce6ab one
[1]: 97b1233a15/daemon/names.go (L22-L38)
[2]: 97b1233a15/daemon/container.go (L106-L121)
[3]: 97b1233a15/daemon/container.go (L71-L76)
[4]: 97b1233a15/daemon/container.go (L30-L69)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function only checked for the ID to be non-empty, and was only
used in a single location. Also move this check as first check in
registerName, to allow for an early return.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When a UDP server is running on a multihomed server, as is the case with
pretty much _all_ Docker hosts (eg. eth0 + docker0), the kernel has to
choose which source address is used when replying to a UDP client. But
that process is based on heuristics and is fallible.
If the address picked doesn't match the original destination address
used by the client, it'll drop the datagram and return an ICMP Port
Unreachable.
To prevent that, we need to:
- `setsockopt(IP_PKTINFO)` on proxy's sockets.
- Extract the original destination address from an ancillary message
every time a new 'UDP connection' is 'established' (ie. every time we
insert a new entry into the UDP conntrack table).
- And finally, pass a control message containing the desired source
address to the kernel, every time we send a response back to the
client.
Also, update the inline comment on read errors in `(*UDPProxy).Run()`.
This comment was misleadingly referencing ECONNREFUSED - Linux's UDP
implementation never returns this error (see [1]). Instead, state why
`net.ErrClosed` is perfectly fine and doesn't need to be logged
(although, docker-proxy currently logs to nowhere).
[1]: https://github.com/search?q=repo%3Atorvalds%2Flinux+ECONNREFUSED+path%3A%2F%5Enet%5C%2F%28ipv4%7Cipv6%29%5C%2F%28udp%7Ctcp%29%2F&type=code
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Returning possibly inconsistent data avoids retrying indefinitely, and
matches the behaviour of vishvananda/netlink prior to version 1.2.1, in
which the NLM_F_DUMP_INTR flag was ignored.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Follow-up to 94dea2018e. Change these to return
errdefs types, which could allow us to move away from the sentinel errors
defined in the package, and instead use errdefs definitions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The withTxn function takes a custom function to execute; we should not
wrap those errors as the only responsibility of this function is to
execute the given function in a transaction.
This was introduced in 6549a270e9, and
an oversight of me.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- validate-prepare and smoke-prepare took 10 seconds; limiting to 10 minutes
- smoke tests took less than 3 minutes; limiting to 10 minutes
- validate: most took under a minute, but "deprecate-integration-cli" took
14 minutes; limiting to 30 minutes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- add `--quiet` to suppress pull progress output
- use `./` instead of `$(pwd)` now that relative paths are supported
- set the working directory on the container, so that we don't have to `cd`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Both the classic builder and buildkit treat empty value and "default"
as equivalent;
classic builder: c9619248d0/builder/dockerfile/internals.go (L379-L386)
buildkit: c9619248d0/builder/builder-next/builder.go (L350-L356)
Before:
DEBU[2024-10-12T11:30:47.716478170Z] Calling POST /v1.47/build?dockerfile=Dockerfile&networkmode=default&version=1 spanID=893f850bd8bda19d traceID=e7ab3b76fd7a9c27a7aa7b25f4528fb8
After:
DEBU[2024-10-12T13:42:56.799531715Z] Calling POST /v1.47/build?dockerfile=Dockerfile&version=1 spanID=d37f36ca6325422e traceID=a5eb4637fc3d0acf56cbb6a0a1d4a5ca
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "rm" option was made the default in API version 1.12 in commit
b60d647172, so the query-parameter can be
omitted unless the user opted to disable removing intermediate containers.
Before:
DEBU[2024-10-12T11:30:47.716478170Z] Calling POST /v1.47/build?dockerfile=Dockerfile&networkmode=default&rm=1&version=1 spanID=893f850bd8bda19d traceID=e7ab3b76fd7a9c27a7aa7b25f4528fb8
After:
DEBU[2024-10-12T11:30:47.716478170Z] Calling POST /v1.47/build?dockerfile=Dockerfile&networkmode=default&version=1 spanID=893f850bd8bda19d traceID=e7ab3b76fd7a9c27a7aa7b25f4528fb8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We had a couple of runs where these jobs got stuck and github
actions didn't allow terminating them, so that they were only
terminated after 120 minutes.
These jobs usually complete in 5 minutes, so let's give them
a shorter timeout. 20 minutes should be enough (don't @ me).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The platform was printed in its raw format, which didn't produce a very
readable output;
Before this change:
$ docker image save --platform=linux/amd64 -o alpine_amd64.tar alpine:latest
Error response from daemon: no suitable export target found for platform linux/amd64: no suitable image manifest found for platform {amd64 linux [] }
After this change:
$ docker image save --platform=linux/amd64 -o alpine_amd64.tar alpine:latest
Error response from daemon: no suitable export target found: image with reference alpine:latest was found but does not provide the specified platform (linux/amd64)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Changed "match" to "provide", in an attempt to indicate that the image is
a multi-platform image that doesn't contain the given platform.
- Remove the "wanted" and instead make the requested platforms to be a part
of the sentence, but within braces.
Before this patch:
docker pull --quiet --platform=linux/riscv64 alpine:latest
docker image history --platform=linux/amd64 alpine
Error response from daemon: image with reference alpine:latest was found but does not match the specified platform: wanted linux/nosuch
With this patch:
docker pull --quiet --platform=linux/riscv64 alpine:latest
docker image history --platform=linux/amd64 alpine
Error response from daemon: image with reference alpine:latest was found but does not provide the specified platform (linux/amd64)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
TestAttachDisconnectLeak starts its own daemon with iptables disabled,
but disabling ip6tables was missed when we enabled ip6tables by default.
TestNetworkStateCleanupOnDaemonStart also starts its own daemon, with
iptables and ip6tables both enabled. It isn't trying to test anything
iptables related.
These tests run in parallel, so they both modify ip6tables in the host
namespace - and could break each other by adding/removing chains at
awkward moments.
Disable iptables and ip6tables in both tests.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Slightly touching up the error message, because the "wanted/actual" output
felt too much like a assertion failure in a test-case.
- Format the image-reference using "familiar" format, which shows the
image's short name for official images.
- Move the actual and requested platforms to be a part of the sentence,
but within braces.
Before this patch:
docker image push --platform=linux/amd64 alpine:latest
Error response from daemon: image with reference docker.io/library/alpine:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux/riscv64
With this patch:
docker image push --platform=linux/amd64 alpine:latest
Error response from daemon: image with reference alpine:latest was found but its platform (linux/riscv64) does not match the specified platform (linux/amd64)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Without containerd store enabled, we were discarding underlying errors,
and instead informing the user that `--platform` is not suported;
docker pull --quiet --platform=linux/riscv64 alpine:latest
docker image push --platform=linux/amd64 alpine:latest
Error response from daemon: graphdriver backed image store doesn't support multiplatform images
However, that's not the case; platform filtering works, but if the image
is not the right platform, the push fails (which is the same as would
happen with the containerd image store enabled).
docker image push --platform=linux/amd64 alpine:latest
Error response from daemon: image with reference docker.io/library/alpine:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux/riscv64
When specifying the platform and that platform matches, it finds the image,
and the push continue;
docker image push --platform=linux/riscv64 alpine:latest
The push refers to repository [docker.io/library/alpine]
3fd4750fd687: Layer already exists
...
(The above example obviously fails because I don't have permissions to
push official images).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We had a few "runaway jobs" recently, where the job got stuck, and kept
running for 6 hours (in one case even 24 hours, probably due some github
outage). Some of those jobs could not be terminated.
While running these actions on public repositories doesn't cost us, it's
still not desirable to have jobs running for that long (as they can still
hold up the queue).
This patch adds a blanket "2 hours" time-limit to all jobs that didn't
have a limit set. We should look at tweaking those limits to actually
expected duration, but having a default at least is a start.
Also changed the position of some existing timeouts so that we have a
consistent order in which it's set; making it easier to spot locations
where no limit is defined.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was originally used to test the experimental `--stream` option with
the classic builder. This feature was removed in Docker 20.10 through
6ca3ec88ae, which also disabled the test
(adding a note to rewrite the test for BuildKit).
The above is 5 years ago, and we now run the BuildKit integration tests
as part of our CI, which (I assume) should cover this functionality.
Removing this test, as it depends on github.com/tonistiigi/fsutil, which
does not have a stable API, and frequently cannot be updated independently
from BuildKit.
Also adding a linter-rule to prevent it from accidentally being imported;
errdefs/defs.go:3:8: import 'github.com/tonistiigi/fsutil' is not allowed from list 'main': The fsutil module does not have a stable API, so we should not have a direct dependency unless necessary. (depguard)
import _ "github.com/tonistiigi/fsutil"
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
BoltDB
- Fix the freelist.allocs isn't rollbacked when a tx is rollbacked.
CMD
- Add -gobench-output option for bench command to adapt to benchstat.
Other
- Bump go version to 1.22.x.
- This patch also added dmflakey package, which can be reused by other projects. See #812.
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.10...v1.3.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The buildkit workflow uses Go to determine the version of Buildkit to run
integration-tests for. It currently uses on the default version that's
installed on the GitHub actions runners (1.21.13 currently), but this fails
if the go.mod/vendor.mod specify a higher version of Go as required version.
If this fails, the BUILDKIT_REF and REPO env-vars are not set / empty,
resulting in the workflow checking out the current (moby) repository instead
of buildkit, which fails.
This patch adds a step to explicitly install the expected version of Go.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since commit b3fabede, the Engine creates the listening sockets used by
docker-proxy by making raw syscalls (ie. socket, setsockopt, bind).
Before that commit, those sockets were created by docker-proxy through
Go's `net.ListenX` functions.
Unlike `net.ListenX` functions, the raw syscall code doesn't set the
`SO_REUSEADDR` option. This option is typically used by TCP servers to
make sure that they can be restarted even if there are client sockets
referencing the server port as their sport (eg. in TIME_WAIT state, or
any other state).
Citing UNIX Network Programming, Section 7.5 (p210):
> By default, when the listening server is restarted by calling socket,
> bind, and listen, the call to bind fails because the listening server
> is trying to bind a port that is part of an existing connection [...]
> _All_ TCP servers should specify this socket option to allow the
> server to be restarted in this situation.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Link-local connections were DNATed like other non-loopback connections,
but the kernel would drop them even before their reach the container.
This commit changes the DNAT rule inserted in ip6tables to exclude
link-local addresses. Instead, these connections will be proxied by
docker-proxy, at least if --userland-proxy=true.
If dockerd is started with the userland-proxy disabled, link-local
port-bindings won't be supported (ie. silently discarded).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This option was added in f143f4ec51,
which changed the minimum API version for "save" but forgot to update
the version for "load".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Along with this new test, a new struct `L3Segment` is introduced to
simulate hosts connected on a same switched network. This struct will
let us test various scenarios where published ports and containers
should or should not be accessible from remote hosts.
The new test introduced, `TestAccessPublishedPortFromRemoteHost`, skips
link-local address as port publishing doesn't work on those addresses
currently. This will be fixed in a future commit.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Previously, if a port mapping specified a host IP or port that
could not be used because the endpoint's network was in routed
mode (so, there's no host binding), it'd be treated as an error.
However:
- the selected gateway endpoint may change over time, as networks
are connected and disconnected - so the binding may make sense
for some other endpoint.
- the validation was complicated, duplicated logic in order to
fail early, and wasn't complete.
So, just log when fields are ignored, at the point where they're
ignored.
Signed-off-by: Rob Murray <rob.murray@docker.com>
If we fail to start an exec, the deferred error-handling block in [L181-L193](c7e42d855e/daemon/exec.go (L181-L193))
would set the exit code to `126` (`EACCES`). However, if we get far enough along
attempting to start the exec, we set the exit code according to the error returned
from starting the task [L288-L291](c7e42d855e/daemon/exec.go (L288-L291)).
For some situations (such as `docker exec [some-container]
missing-binary`), the 2nd block returns the correct exit code (`127`)
but that then gets overwritten by the 1st block.
This commit changes that logic to only set the default exit code `126`
if the exit code has not been set yet.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
- use an early return if we're not using container-mode, instead
of checking multiple times
- use ConnectedContainer() method to check if a container is specified
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this change, capabilities would be sent un-normalized, un-sorted,
and could contain duplicates;
docker create --name foo --cap-add SYS_ADMIN --cap-add sys_admin --cap-add cap_sys_admin --cap-add ALL busybox
docker container inspect --format '{{json .HostConfig.CapAdd }}' foo
["SYS_ADMIN","sys_admin","cap_sys_admin","ALL"]
After this change, capabilities are sent in their normalized form, sorted,
and with duplicates removed;
docker create --name foo --cap-add SYS_ADMIN --cap-add sys_admin --cap-add cap_sys_admin --cap-add ALL busybox
docker container inspect --format '{{json .HostConfig.CapAdd }}' foo
["ALL", "CAP_SYS_ADMIN"]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function used a locally defined `configWrapper`, which was identical
to the `container.CreateRequest`, with the exception of `CreateRequest`
defining `omitempty` for HostConfig and NetworkingConfig, but this should
not impact our use as the same type is used to handle the request on the
daemon side.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Shortened to minimum code because OCI v1 is essentially equivalent to
Docker v2.2 format. Also fixed formatting issue with jq where it wouldn't
accept newline character in chained jq commands.
Co-authored-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Jimbo Jones <jjjimbo137@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 11380a109e updated the daemon to
always treat 127.0.0.1 as insecure for all cases anytime anywhere. This
was initially a hard-coded list, but later made configurable to allow
the user to mark additional CIDRs or registries as insecure in
6aba75db4e.
This patch expands the default list of insecure registries to also
include the IPv6 loopback-address (::1); IPv6, unlike IPv4 only has
a single loopback address (::1/128).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure an error is returned straight away if there's contention on
the underlying db file. This makes sure we don't reintroduce the issue
fixed in d21d088, and it will help detect contention in parallelized
tests if they're badly written. It effectively adds a new error mode to
the daemon, but if anyone faces this error, they should fix their
process manager.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
That function was needlessly complex. Instead of relying on a struct and
a sub-struct, it now just takes two string params: a path and a bucket
name.
Libnetwork config is now initialized with default values. A new struct
is introduced in libnetwork/config to let tests customize the path and
bucket name.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This option fn was defining a custom directory, file name and bucket
name for boltdb. Users can only change data-dir through `OptionDataDir`.
Better reuse that function instead, that'll make refactorings easier.
It won't set a custom bucket name or file name as `OptionBoltdbWithRandomDBFile`
was doing, but that's not needed since every test will use a different
temp dir anyway.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
cmd/dockerd: notifyShutdown: don't use Fatal log to prevent early exit
------------------------------------------------------------------------
Commit 40868e263d added this log for situations
where the daemon failed to start, and where no event-log was created
to discover the failure.
However, it used a `Fatal` log which, when using logrus, not only logs
a message, but also calls the [logger's Exit function][1], which defaults
[to `os.Exit`][2].
The error passed to the `notifyShutdown` function can be the result of
different steps in the service's lifecycle; `cli.start()` starts a long-
lived process, and the error returned can be either because it failed
to start the service, or an error produced while shutting down the service.
We should reduce ambiguity here (which could be for `cli.Start()` to return
whether the service was started in the first place). Regardless, we want
code following the log-entry to be executed, to make sure that codepaths
following the `notifyShutdown` (which could be (defer) statements or cleanup
steps) are executed.
This patch changes the log to an `Error`, which is non-fatal, and should
make sure that at least the `service.stopped` is executed to signal the
service no longer running. The logging is also moved out of the
`notifyShutdown`.
Disclaimer: this is with the assumption that calling `service.stopped` is
always acceptable, even if the service is not running, or in other states.
cmd/dockerd: preNotifyReady: return error instead of logging and exit
------------------------------------------------------------------------
Commit 57aef3b490 added support for running
the daemon as a Windows service. As part of this, it introduced a log
for situations where the service failed to start. However, it used a
`Fatal` log for this purpose which, when using logrus, not only logs
a message, but also calls the [logger's Exit function][1], which defaults
[to `os.Exit`][2].
A result of this is that the daemon does an unclean shutdown, causing
other code (which could be (defer) statements or cleanup steps) not to
be executed.
This patch updates the `preNotifyReady` to return the error instead of
logging and exiting. `preNotifyReady` is executed as part of `cli.start()`,
which has an error-return, and the error that is returned, is already
logged by `runDaemon` (so logging it as part of `preNotifyReady` would
result in the error being logged twice).
[1]: 5098132d84/vendor/github.com/sirupsen/logrus/entry.go (L336-L339)
[2]: 5098132d84/vendor/github.com/sirupsen/logrus/logger.go (L342-L348)
- relates to https://github.com/moby/moby/pull/22340
cmd/dockerd: preNotifyReady: return error instead of logging and exit
Commit 57aef3b490 added support for running
the daemon as a Windows service. As part of this, it introduced a log
for situations where the service failed to start. However, it used a
`Fatal` log for this purpose which, when using logrus, not only logs
a message, but also calls the [logger's Exit function][1], which defaults
[to `os.Exit`][2].
A result of this is that the daemon does an unclean shutdown, causing
other code (which could be (defer) statements or cleanup steps) not to
be executed.
This patch changes the `preNotifyReady` to
[1]: 5098132d84/vendor/github.com/sirupsen/logrus/entry.go (L336-L339)
[2]: 5098132d84/vendor/github.com/sirupsen/logrus/logger.go (L342-L348)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
They're only used within this package, and are not expected to be used
externally. Some exported functions also take non-exported types as
argument, so would not be usable outside of this package either way.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Unlike Linux, which uses fixed locations as default, the Windows daemon uses
paths relative to the data-root as defaults for storing both the PIDFile, and
the daemon configuration file (daemon.json).
The data-root is configurable both through command-line options (`--data-root`),
and through the daemon configuration file (daemon.json). This patch moves Windows-
specific config handling to config-related code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Validate and apply options when creating the CLI, so that starting the
CLI does not have to mutate the config, and to have a clearer separation
between "creating", "validating", and starting the daemon.
This also allows skipping the service-registration code in situations
where we only want to validate the config.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
getDefaultDaemonConfigDir would never return an error and because of that,
neither would getDefaultDaemonConfigFile, so we can remove these error returns.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch moves Windows-specific config for the config-file location to config-
related code to help discoverability.
Unlike Linux, which uses fixed locations as default, the Windows daemon uses
paths relative to the data-root as defaults for storing both the PIDFile, and
the daemon configuration file (daemon.json).
For the PIDfile, additional changes will be needed, as using a PIDfile depends
on whether the daemon is run as a service or not.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit f9c0103 (WSL2 mirrored-mode loopback) uses netlink funcs
that were removed/wrapped by commit 00bf437.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Kernel module br_netfilter is loaded when the daemon starts with
either iptables or ip6tables enabled. That automatically sets:
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
So, when:
- docker was running happily with iptables=false, and
- no explicit ip6tables=false, and
- br_netfilter was not loaded
... the change in moby 27.0 to enable ip6tables by default, resulted
in net.bridge.bridge-nf-call-iptables being enabled.
If the host also had a firewall with default-drop on its forward
chain - that resulted in packets getting dropped between containers
on a bridge network.
So, only try to load br_netfilter when it's needed - it's only needed
to implement "--icc=false", which can only be used when iptables or
ip6tables is enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This option was added in a08abec9f8,
as part of Docker v25.0, but did not update the docs and manpage.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The value is optional for SetOpts (and NamedSetOpts), and implied
"true" when omitted.
This patch adds a test-case for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The --feature flag allows the boolean value to be omitted.
If only a name is provided, the default is "true".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When running WSL2 with mirrored mode networking, add an iptables
rule to skip DNAT for packets arriving on interface loopback0 that
are addressed to a localhost address - they're from the Windows
host.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Spot netlink functions that may return EINTR because
network configuration changed during a state dump, and
point at the nlutil wrappers.
Signed-off-by: Rob Murray <rob.murray@docker.com>
A recent change to the vishvananda/netlink package exposes
NLM_F_DUMP_INTR in some netlink responses as an EINTR (with
no data).
Retry the requests when that happens, up to five times, before
returning the error. The limit of five is arbitrary, on most
systems a single retry will be rare but, there's no guarantee
that a retry will succeed. So, on a very busy or misbehaving
system the error may still be returned. In most cases, this
will lead to failure of the operation being attempted (which
may lead to daemon startup failure, network initialisation
failure etc).
Signed-off-by: Rob Murray <rob.murray@docker.com>
The new Network.enableIPv4 flag needs to be set for IPv4
networks created before it was introduced.
Commit 903daa4 attempted to do that in the unmarshalling code
by checking Network.ipamV4Info - but, that field hadn't been
unmarshalled yet, so it was never present.
Instead, check for its presence in the saved map.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The `Commit` type was introduced in 2790ac68b3,
to assist triaging issues that were reported with an incorrect version of
runc or containerd. At the time, both `runc` and `containerd` were not yet
stable, and had to be built from a specific commit to guarantee compatibility.
We encountered various situations where unexpected (and incompatible) versions
of those binaries were packaged, resulting in hard to trace bug-reports.
For those situations, a "expected" version was set at compile time, to
indicate if the version installed was different from the expected version;
docker info
...
runc version: a592beb5bc4c4092b1b1bac971afed27687340c5 (expected: 69663f0bd4b60df09991c08812a60108003fa340)
Both `runc` and `containerd` are stable now, and docker 19.03 and up set the
expected version to the actual version since c65f0bd13c
and 23.0 did the same for the `init` binary b585c64e2b,
to prevent the CLI from reporting "unexpected version".
In short; the `Expected` fields no longer serves a real purpose.
In future, we can even consider deprecating the `ContainerdCommit`, `RuncCommit`
and `InitCommit` fields on the `/info` response (as we also include this
information as part of the components returned in `/version`), but those
can still be useful currently for situations where a user only provides
`docker info` output.
This patch starts with deprecating the `Expected` field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When untagging multiple images targetting the same digest, delete the
images in lexographic order to be consistent with graphdrivers.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Allows load to filter image manifests to load based on their platform.
For save, verify that the image platform matches the requested platform,
otherwise error out.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Platform` parameter that allows to select a specific platform to
save/load.
This is a breaking change to the Go client as it changes the signatures
of `ImageLoad` and `ImageSave`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Platform` parameter that allows to select a specific platform to
show the history for.
This is a breaking change to the Go client as it changes the signature
of `ImageHistory`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This change ensures that when --gpus=0 is selected, the injection of
NVIDIA device nodes and libraries is disabled by setting the
NVIDIA_VISIBLE_DEVICES environment variable to void instead of
leaving this unspecfied.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
When looking for failures in CI, I always search for `FAIL:` (with a
trailing colon) to find tests that fail. This test has some test-cases
that are currently expected to fail, but due to the colon would also
be included when searching;
=== RUN TestIPRangeAt64BitLimit/ipRange_at_end_of_64-bit_subnet
bridge_test.go:196: XFAIL: Container startup failed with error: Error response from daemon: no available IPv6 addresses on this network's address pools: test64bl (b014e28c35c14cc34514430a8cfe1c97632c7988c56d89cea46abb10fa32229d)
=== RUN TestIPRangeAt64BitLimit/ipRange_at_64-bit_boundary_inside_56-bit_subnet
bridge_test.go:196: XFAIL: Container startup failed with error: Error response from daemon: no available IPv6 addresses on this network's address pools: test64bl (fb70301550d7a2d1d3425f5c1010a9ef487a9a251221a2d68ac49d257b249013)
Remove the trailing `:` so that searching for unexpected failures does not
include these tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't turn images into dangling when they are used by containers created
with an image specified by an ID only (e.g. `docker run 82d1e9d`).
Keep the last image reference with the same target when all other
references would be pruned.
If the container was created with a digested and tagged reference (e.g.
`docker run alpine:latest@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1`),
the `alpine:latest` image won't get untagged.
This change makes the behavior consistent with the graphdriver
implementation.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The 27.x branch was created and is on API 1.47, so changes in master/main
should now be targeting the next version of the API (1.48).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 9160b9fda6 removed the platform from
the descriptor, but this field was still used further in the code, and now
always taking an empty platform.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit e1ea911aba removed the error return from
.Size() and .DiffSize(), but forgot to remove this error-check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The specification moved to a separate module (github.com/moby/docker-image-spec),
and this package was deprecated in in 03a17a2887,
which is part of docker v26.0.0 and up.
This patch remove the deprecated package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We moved to the major release branches with a `.x` suffix and forgot to
adjust this workflow.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
commit af0cdc36c7 officially marked these
fields as deprecated and to be removed in API v1.47 (which was targeted
for v28.0). We shipped v1.47 with the v27.2 release, but did not yet
remove the erroneous fields.
This patch updates the version to v1.48.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit af0cdc36c7 officially marked these
fields as deprecated and to be removed in API v1.47 (which was targeted
for v28.0). We shipped v1.47 with the v27.2 release, but did not yet
remove the erroneous fields.
This patch updates the version to v1.48.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The documentation was incorrect and didn't properly document the use of
tags;
- Image push currently only accepts an image-name, not an ID / digest.
- When giving a name, it's expected to be without tag included; when
including a tag, it is ignored.
- The tag parameter is required when pushing a single image (i.e., it
does not default to "latest"); omitting the tag parameter will push
all tags of the given image.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The documentation was incorrect and didn't properly document the use of
tags;
- Image push currently only accepts an image-name, not an ID / digest.
- When giving a name, it's expected to be without tag included; when
including a tag, it is ignored.
- The tag parameter is required when pushing a single image (i.e., it
does not default to "latest"); omitting the tag parameter will push
all tags of the given image.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 77f2d90e27 introduced the slices import,
which uses generics, but failed to add the go:build directives.
# github.com/docker/docker/daemon/logger/loggerutils
../../daemon/logger/loggerutils/logfile.go:770:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 2847c4b7fe switched networkdb to use
go-immutable-radix v2, which uses generics, but failed to add the go:build
directives.
# github.com/docker/docker/libnetwork/networkdb
../../libnetwork/networkdb/networkdb.go:47:19: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/networkdb/networkdb.go:259:33: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/networkdb/networkdb.go:269:25: function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/networkdb/networkdb.go:270:27: function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- https://github.com/golang/go/issues?q=milestone%3AGo1.22.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.22.6...go1.22.7
These minor releases include 3 security fixes following the security policy:
- go/parser: stack exhaustion in all Parse* functions
Calling any of the Parse functions on Go source code which contains deeply nested literals can cause a panic due to stack exhaustion.
This is CVE-2024-34155 and Go issue https://go.dev/issue/69138.
- encoding/gob: stack exhaustion in Decoder.Decode
Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion.
This is a follow-up to CVE-2022-30635.
Thanks to Md Sakib Anwar of The Ohio State University (anwar.40@osu.edu) for reporting this issue.
This is CVE-2024-34156 and Go issue https://go.dev/issue/69139.
- go/build/constraint: stack exhaustion in Parse
Calling Parse on a "// +build" build tag line with deeply nested expressions can cause a panic due to stack exhaustion.
This is CVE-2024-34158 and Go issue https://go.dev/issue/69141.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.23.1
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
diff: https://github.com/opencontainers/runc/compare/v1.1.13...v1.1.14
Release Notes:
- Fix CVE-2024-45310 / GHSA-jfvp-7x6p-h2pv, a low-severity attack that allowed maliciously configured containers to create empty files and directories on the host.
- Add support for Go 1.23.
- Revert "allow overriding VERSION value in Makefile" and add EXTRA_VERSION.
- rootfs: consolidate mountpoint creation logic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update the runc binary that's used in CI and for the static packages.
diff: https://github.com/opencontainers/runc/compare/v1.1.13...v1.1.14
Release Notes:
- Fix CVE-2024-45310, a low-severity attack that allowed maliciously configured containers to create empty files and directories on the host.
- Add support for Go 1.23.
- Revert "allow overriding VERSION value in Makefile" and add EXTRA_VERSION.
- rootfs: consolidate mountpoint creation logic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pie-mode is supported for windows/arm64 since https://go.dev/cl/452415,
which is part of go1.20. Also update link to Go source for pie-mode support
to match the location for current versions of Go because the package was
moved in https://go.dev/cl/438475.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- increase mock handler processing time to 50ms to try to prevent it from finishing before the 10ms client timeout occurs
- replace deprecated error type assertion
Signed-off-by: Adam Simon <adamsimon85100@gmail.com>
Use the same error message as the graphdrivers image store backend.
It's more informative as it also includes the requested platform and
won't break clients checking doing error check with string-matching.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
it produces many hits, some of which may be false positives, but we need to
look into these, e.g.;
container/container.go:517:72: G115: integer overflow conversion int -> uint32 (gosec)
shouldRestart, _, _ := container.RestartManager().ShouldRestart(uint32(container.ExitCode()), container.HasBeenManuallyStopped, container.FinishedAt.Sub(container.StartedAt))
^
container/view.go:401:25: G115: integer overflow conversion int -> uint16 (gosec)
PrivatePort: uint16(p),
^
container/view.go:413:25: G115: integer overflow conversion int -> uint16 (gosec)
PrivatePort: uint16(p),
^
container/view.go:414:25: G115: integer overflow conversion int -> uint16 (gosec)
PublicPort: uint16(h),
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cross-compiling for arm/v5 was failing;
#56 84.12 /usr/bin/arm-linux-gnueabi-clang -marm -o $WORK/b001/exe/a.out -Wl,--export-dynamic-symbol=_cgo_panic -Wl,--export-dynamic-symbol=_cgo_topofstack -Wl,--export-dynamic-symbol=crosscall2 -Qunused-arguments -Wl,--compress-debug-sections=zlib /tmp/go-link-759578347/go.o /tmp/go-link-759578347/000000.o /tmp/go-link-759578347/000001.o /tmp/go-link-759578347/000002.o /tmp/go-link-759578347/000003.o /tmp/go-link-759578347/000004.o /tmp/go-link-759578347/000005.o /tmp/go-link-759578347/000006.o /tmp/go-link-759578347/000007.o /tmp/go-link-759578347/000008.o /tmp/go-link-759578347/000009.o /tmp/go-link-759578347/000010.o /tmp/go-link-759578347/000011.o /tmp/go-link-759578347/000012.o /tmp/go-link-759578347/000013.o /tmp/go-link-759578347/000014.o /tmp/go-link-759578347/000015.o /tmp/go-link-759578347/000016.o /tmp/go-link-759578347/000017.o /tmp/go-link-759578347/000018.o -O2 -g -O2 -g -O2 -g -lpthread -O2 -g -no-pie -static
#56 84.12 ld.lld: error: undefined symbol: __atomic_load_4
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced 2 more times
#56 84.12
#56 84.12 ld.lld: error: undefined symbol: __atomic_store_4
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(_cgo_wait_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(x_cgo_notify_runtime_init_done)
#56 84.12 >>> referenced by gcc_libinit.c
#56 84.12 >>> /tmp/go-link-759578347/000009.o:(x_cgo_set_context_function)
#56 84.12 clang: error: linker command failed with exit code 1 (use -v to see invocation)
From discussion on GitHub;
https://github.com/moby/moby/pull/46982#issuecomment-2206992611
The arm/v5 build failure looks to be due to libatomic not being included
in the link. For reasons probably buried in mailing list archives,
[gcc](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358) and clang don't
bother to implicitly auto-link libatomic. This is not a big deal on many
modern platforms with atomic intrinsics as the compiler generates inline
instruction sequences, avoiding any libcalls into libatomic. ARMv5 is not
one of those platforms: all atomic operations require a libcall.
In theory, adding `CGO_LDFLAGS=-latomic` should fix arm/v5 builds.
While it could be argued that cgo should automatically link against
libatomic in the same way that it automatically links against libpthread,
the Go maintainers would have a valid counter-argument that it should be
the C toolchain's responsibility to link against libatomic automatically,
just like it does with libgcc or compiler-rt.
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
cross-compiling for arm/v5 fails on go1.22; a fix is included for this
in go1.23 (https://github.com/golang/go/issues/65290), but for go1.22
we can set the correct option manually.
1.189 + go build -mod=vendor -modfile=vendor.mod -o /tmp/bundles/binary-daemon/dockerd -tags 'netgo osusergo static_build journald' -ldflags '-w -X "github.com/docker/docker/dockerversion.Version=dev" -X "github.com/docker/docker/dockerversion.GitCommit=HEAD" -X "github.com/docker/docker/dockerversion.BuildTime=2024-08-29T16:59:57.000000000+00:00" -X "github.com/docker/docker/dockerversion.PlatformName=" -X "github.com/docker/docker/dockerversion.ProductName=" -X "github.com/docker/docker/dockerversion.DefaultProductLicense=" -extldflags -static ' -gcflags= github.com/docker/docker/cmd/dockerd
67.78 # runtime/cgo
67.78 gcc_libinit.c:44:8: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:47:6: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:49:10: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:69:9: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
67.78 gcc_libinit.c:71:3: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment]
78.20 + rm -f /go/src/github.com/docker/docker/go.mod
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Move the definition, but mostly keep it for documentation purposes,
to prevent having to import the registry package in all places.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Support the use of `make install` in packaging scripts, where the
$mandir tree might not exist under $DESTDIR.
For portability, create the parent directories using a separate install
command instead of relying on the non-portable `-D` flag.
Set errexit so the install target fails if any install step fails.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Before this, the image list would not show images that are not a valid
container image, but could be a valid artifact.
While they're not directly usable by docker, we should still show them
so the user can still discover them and at least be able to delete them.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
These values are configured when instantiating the allocator, and not
intended to be mutated externally. They're only used internally with
the exception of a test in the bridge driver that uses it to pick a
port that can be used for testing.
This patch:
- un-exports the Begin and End fields
- introduces a GetPortRange() utility to allow the bridge driver
to get the port, but marking it as a function for internal use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
RequestPortsInRange calls portMap.getPortRange() in a loop, but the given
port-range is always the same. Perform validation once instead of for each
iteration.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use the variable that's provided by the net package, and make the defaultIP
a property of the allocator instead of a package variable.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Test the functionality in isolation instead of using the singleton that's
returned by the `GET` function; this makes sure tests don't affect each other,
and doesn't require resetting the singleton in between tests, potentially
allowing these tests to eb run in parallel.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The doc-link was not formatted correctly and didn't work. While updating
also slightly touch-up the description to explain "defaultIP".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon/daemon.go:942:21: printf: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Errorf (govet)
log.G(ctx).Errorf(err.Error())
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
api/types/container/hostconfig.go:328:43: printf: non-constant format string in call to fmt.Errorf (govet)
return &errInvalidParameter{fmt.Errorf(msg)}
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
container/stream/streams.go:111:21: printf: non-constant format string in call to fmt.Errorf (govet)
return fmt.Errorf(strings.Join(errors, "\n"))
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/drivers/bridge/setup_ip_tables_linux.go:385:23: printf: non-constant format string in call to fmt.Errorf (govet)
return fmt.Errorf(msg)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
volume/testutils/testutils.go:98:26: printf: non-constant format string in call to fmt.Errorf (govet)
return nil, fmt.Errorf(opts["error"])
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
builder/dockerfile/internals_linux.go:38:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
return idtools.Identity{}, errors.Wrapf(err, "can't find uid for user "+userStr)
^
builder/dockerfile/internals_linux.go:42:48: printf: non-constant format string in call to github.com/docker/docker/vendor/github.com/pkg/errors.Wrapf (govet)
return idtools.Identity{}, errors.Wrapf(err, "can't find gid for group "+grpStr)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This looks to be a false positive;
layer/layer.go:202:47: G602: slice index out of range (gosec)
return createChainIDFromParent(ChainID(dgsts[0]), dgsts[1:]...)
^
layer/layer.go:205:69: G602: slice index out of range (gosec)
dgst := digest.FromBytes([]byte(string(parent) + " " + string(dgsts[0])))
^
layer/layer.go:206:53: G602: slice bounds out of range (gosec)
return createChainIDFromParent(ChainID(dgst), dgsts[1:]...)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
internal/cleanups/composite_test.go:46:9: printf: non-constant format string in call to (*testing.common).Logf (govet)
t.Logf(err.Error())
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/cnmallocator/drivers_ipam.go:43:31: printf: non-constant format string in call to (*github.com/docker/docker/vendor/github.com/sirupsen/logrus.Entry).Infof (govet)
log.G(context.TODO()).Infof("Swarm initialized global default address pool to: " + str.String())
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/controller.go:1054:32: printf: non-constant format string in call to github.com/docker/docker/libnetwork/types.NotFoundErrorf (govet)
return types.NotFoundErrorf(err.Error())
^
libnetwork/controller.go:1073:32: printf: non-constant format string in call to github.com/docker/docker/libnetwork/types.NotFoundErrorf (govet)
return types.NotFoundErrorf(err.Error())
^
libnetwork/sandbox_externalkey_unix.go:113:21: printf: non-constant format string in call to fmt.Errorf (govet)
return fmt.Errorf(string(buf[0:n]))
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also fix some unhandled errors.
integration-cli/docker_cli_swarm_test.go:697:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"Error":"failed to add veth pair: `+err.Error()+`"}`)
^
integration-cli/docker_cli_swarm_test.go:731:18: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"LocalDefaultAddressSpace":"`+lAS+`", "GlobalDefaultAddressSpace": "`+gAS+`"}`)
^
integration-cli/docker_cli_swarm_test.go:742:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"Error":"Unknown address space in pool request: `+poolRequest.AddressSpace+`"}`)
^
integration-cli/docker_cli_swarm_test.go:746:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"PoolID":"`+poolID+`", "Pool":"`+pool+`"}`)
^
integration-cli/docker_cli_swarm_test.go:763:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"Address":"`+gw+`"}`)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also rename some variables that shadowed imports, and fix some
unhandled errors.
integration-cli/docker_cli_network_unix_test.go:102:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"Error":"failed to add veth pair: `+err.Error()+`"}`)
^
integration-cli/docker_cli_network_unix_test.go:136:18: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"LocalDefaultAddressSpace":"`+lAS+`", "GlobalDefaultAddressSpace": "`+gAS+`"}`)
^
integration-cli/docker_cli_network_unix_test.go:147:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"Error":"Unknown address space in pool request: `+poolRequest.AddressSpace+`"}`)
^
integration-cli/docker_cli_network_unix_test.go:151:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"PoolID":"`+poolID+`", "Pool":"`+pool+`"}`)
^
integration-cli/docker_cli_network_unix_test.go:168:19: printf: non-constant format string in call to fmt.Fprintf (govet)
fmt.Fprintf(w, `{"Address":"`+gw+`"}`)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
integration-cli/benchmark_test.go:49:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
integration-cli/benchmark_test.go:62:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
integration-cli/benchmark_test.go:68:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
integration-cli/benchmark_test.go:73:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
integration-cli/benchmark_test.go:78:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
integration-cli/benchmark_test.go:84:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
integration-cli/benchmark_test.go:94:27: printf: non-constant format string in call to fmt.Errorf (govet)
chErr <- fmt.Errorf(out)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/sandbox_dns_unix_test.go:17:13: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
skip.If(t, runtime.GOOS == "windows", "test only works on linux")
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This condition was added in 0215a62d5b, which
removed pkg/homedir as abstraction, but didn't consider that this test
is currently only ran on Unix.
integration-cli/docker_cli_run_unix_test.go:254:5: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
if runtime.GOOS == "windows" {
^
integration-cli/docker_cli_run_unix_test.go:338:5: SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)
if runtime.GOOS == "windows" {
^
Added a TODO, because this functionality should also be tested on Windows,
probably as part of tests in docker/cli instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With graphdrivers, the old image was still kept as a dangling image.
Keep the same behavior with containerd.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When loading a multi-platform image, it's not necessary to unpack all
platforms, especially those which have a completely different OS.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Log the error to the progress output instead.
The image is still loaded into the content store and image service even
if the unpacking failed, so don't error out the whole operation to avoid
missing the load events for other image names loaded from the same
archive.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use a regular for loop instead of ApplyAll which spawns a separate
goroutine for each separate container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Breaking change: The .ToOCI() functions in the specs-go package have been
removed. This removes the dependency on the OCI runtime specification from
the CDI specification definition itself.
What's Changed
- Add workflow to mark prs and issues as stale
- Remove the ToOCI functions from the specs-go package
- docs: add a pointer to community meetings in our docs.
- Bump spec version to v0.8.0
- Update spec version in README
Full diff: https://github.com/cncf-tags/container-device-interface/compare/v0.7.2...v0.8.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
the resolvconf package is imported in BuildKit, and this is the only
location that used the errdefs package outside of the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows for an individual decode operation to be cancelled while the
log reader is reading data from a log file by closing the underlying file.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The config.logLevel field, when set, is used to set the `--log-level`
flag when starting the managed `containerd` binary. This flag is the
equivalent to setting the `Config.Debug.Level` field, as can be seen
in the [`md/containerd/command.setLogLevel()`][1] function.
As we're already producing a generated containerd configuration file,
and this file already includes `Debug` options, we might as well include
the option in that file, instead of using the `--log-level` flag.
For entertainment of whoever reads this commit-message, it's worth noting
that previously we were writing this option to the config-file, and
yours truly removed that part in b6b0b0a05f,
but to my defence, we were _also_ setting the `--log-level` flag at the
time :)
[1]: https://github.com/containerd/containerd/blob/v1.7.20/cmd/containerd/command/main.go#L348-L357
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Set the GO_MD2MAN make variable to elide building go-md2man from
vendored sources and use the specified command instead.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Vendor the go-md2man tool used to generate the man pages so that the
only dependency is a Go toolchain.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Commit a0009345f5 updated the default
MaxRecvMsgSize and MaxSendMsgSize for Linux, but did not modify the
defaults for Windows. Those options should not be platform-specific,
which means that the only difference between the Linux and Windows
config are the addresses for GRPC and Debug (Windows defaulting
to a named pipe, whereas Linux sockets within exec-root).
This patch
- implements functions to return the default addresses for each platform
- moves the defaults into `supervisor.Start()`
- removes the now redundant `remote.setDefaults()` method
It's worth noting that prior to this path, `remove.setDefaults()` would
be applied _after_ any (custom) `DaemonOpt` was applied. However, none of
the existing `DaemonOpt` options currently mutates these options. `remote`
is also a non-exported type, so no external implementations can currently
be created. It is therefore safe to set these defaults before options are
applied.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `GET /images/json` requires an optional `container-count` parameter
which set the `Containers` property of in the ImageSummary to a number
of containers using that image.
This was also propagated to the new manifest list property which
includes a list of all the container IDs that are using this specific
image manifest.
Disconnect the `ImageData.Containers` property from this option and
always include it by default without an explicit opt-in.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
An address can be assigned in the container (maybe by DHCP).
DNS lookups work once there's an address and a route, before
that, there's no source address for lookups made by the
internal resolver from the container's network namespace.
In this case, don't want to hook it up to docker_gwbridge, so
disable it in joinInfo - this currently means no DNS entry for
the container's name is set up in the internal DNS. (Even once
the interface has a user-assigned address, there won't be an
internal-DNS entry for the container until it's connected to
a with-IPAM network as well.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
An address can be assigned in the container (maybe by DHCP).
DNS lookups work once there's an address and a route, before
that, there's no source address for lookups made by the
internal resolver from the container's network namespace.
In this case, don't want to hook it up to docker_gwbridge, so
disable it in joinInfo - this currently means no DNS entry for
the container's name is set up in the internal DNS. (Even once
the interface has a user-assigned address, there won't be an
internal-DNS entry for the container until it's connected to
a with-IPAM network as well.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
When defaultipam.newPoolData is asked for a pool of 64-bits
or more, it ends up with an overflowed u64 - so, it just
subtracts one to get a nearly-big-enough range (for a 64-bit
subnet).
When defaultipam.getAddress is called with an ipr (sub-pool
range), the range it calls bitmask.SetAnyInRange with is
exclusive of end. So, its end param can't be MaxUint64,
because that's the max value for the top end of the range
and, when checking the range, SetAnyInRange fails.
When fixed-cidr-v6 behaves more like fixed-cidr, it will ask
for a 64-bit range if that's what fixed-cidr-v6 needs. So,
it hits the bug when allocating an address for, for example:
docker network create --ipv6 --subnet fddd::/64 --ip-range fddd::/64 b46
The additional check for "ipr == base" avoids the issue in
this case, by ignoring the ipr/sub-pool range if ipr is the
same as the pool itself (not really a sub-pool).
But, it still fails when ipr!=base. For example:
docker network create --ipv6 --subnet fddd::/56 --ip-range fddd::/64 b46
So, also subtract one from 'end' if it's going to hit the max
value allowed by the Bitmap.
Signed-off-by: Rob Murray <rob.murray@docker.com>
commit e0c7a839a8 stopped building the
docker-proxy on Windows as it's not used on that platform.
This removes some leftovers for building it on Windows.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Austin has been helping maintain the project for some time, and we discussed
adding Austin as a curator to grant permissions to do triage tasks.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 2ce811e632 migrated the use of the
userns package to the github.com/moby/sys/user module.
After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).
This patch migrates our code to use the new module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Defining BIND_GIT will bind mount the common .git directory into the dev
container.
This makes it possible to run some hack/* scripts which rely on `git`
when running the container in a git worktree.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add `Manifests` field to `ImageSummary` which exposes all image
manifests (which includes other blobs using the image media type, like
buildkit attestations).
There's also a new `manifests` query field that needs to be set in order
for the response to contain the new information.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The internal resolver now uses any namesever found in the host's
/etc/resolv.conf as an external nameserver, and it's accessed
from the host's network namespace.
Before this change, when no external nameservers were found (so
the host had no entries in /etc/resolv.conf) Google's DNS servers
were used as fallbacks, always accessed from the container's
network namespace. If a container's initial set of endpoints had
IPv6 enabled, the IPv6 nameservers were included.
Now we have IPv6-only networks, a similar exception would be
needed for Google's IPv4 nameservers... don't include them if
there are no IPv4 endpoints.
However, only the initial set of endpoints was considered. As
networks are connected/disconnected, IPv4 or IPv6 connectivity
may be lost.
Unlike nameservers read from the host's /etc/resolv.conf, there
is no way to tell which fallback nameservers (v4/v6) might work
from the host's namespace. So, using the host's namespace isn't
a good solution.
Since we want to get away from using fallback nameservers anyway,
this change removes them.
If a host has no /etc/resolv.conf entries, but a container does
need to use DNS, it'll need to be configured with servers via
'--dns'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The internal resolver needs to know whether to make requets
to external DNS servers from the container's network namespace
or the host's.
The original rule was that requests were always made from the
container's namespace, unless the nameserver was on a localhost
address on the host. IPv6 nameservers were left in the container's
/etc/resolv.conf.
Commit 4e8d9a4 modified that so that IPv6 nameservers were also
used as external nameservers. The internal resolver accessed
them from the host namespace if the container's initial set of
endpoints were IPv4-only, or the nameserver address contained
a zone-id, (or the nameserver was on the IPv6 loopback address).
That would break if initial IPv6 endpoints were disconnected from
the container, leaving it with no IPv6 address.
Once IPv6-only networks are allowed, another exception would need
to be made for IPv4 nameservers (they'd need to be accessed from
the host's namespace).
Instead of doing that ... this change simplifies things, if a
nameserver address is read from the host's /etc/resolv.conf, it'll
work in the host's namespace. So, the rule is now simply that
nameservers read from the host's resolv.conf are accessed from the
host's namespace. DNS servers added as overrides ('--dns') are
accessed from the container's namespace (as before).
Signed-off-by: Rob Murray <rob.murray@docker.com>
Also, return IPv6 records from Network.getSvcRecords()
so that /etc/hosts entries are deleted when an IPv6-only
endpoint is removed.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Mostly, this makes sure that literals (such as true, false, host,
private, examples of options usage etc.) are typeset in bold, except for
filenames, which are typeset in italic.
While at it,
- remove some default values from synopsis as it should not
be there;
- fix man pages references (page name in bold, volume number in
regular).
This is not a complete fix, but a step in the right direction.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Escape asterisks and underscores, that have special meaning in
Markdown. While most markdown processors are smart enough to
distinguish whether it's a literal * or _ or a formatting directive,
escaping makes things more explicit.
2. Fix using wrong level of headings in some dm options (most are ####,
but some were #####).
3. Do not use sub-heading for examples in some dm options (this is how
it's done in the rest of the man page).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This fix tries to address issues raised in moby/moby#44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.
Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
- the `--disable-legacy-registry` daemon flag was removed
- duplicate keys with conflicting values for engine labels
now produce an error instead of a warning.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Fix 19 typos, grammatical errors and duplicated words.
These fixes have minimal impact on the code as these are either in the
doc files or in comments inside the code files.
Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
This builds (and depends) on https://github.com/moby/moby/pull/34087
Version 2:
- remove --ipc argument validation (it is now done by daemon)
- add/document 'none' value
- docs/reference/run.md: add a table with better modes description
- dockerd(8) typesetting fixes
Version 3:
- remove ipc mode tests from cli/command/container/opts_test.go
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is a new option added specifically to allow for debugging of bugs
in Docker's storage drivers or libdm itself.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Prepare to move the dockerd man page back to this repository from
docker/cli, retaining history.
This partially reverts commit b5579a4ce3.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The daemon was modified to tell RootlessKit about host port
mappings directly, rather than by running rootlesskit-docker-proxy
to make those updates.
DNAT rules created in rootless mode referred to the host IP address,
rather than the address seen as host address in the rootless network
namespace.
With these changes, port mappings work in rootless mode when
--userland-proxy=false - so, don't gate the RootlessKit API calls
on starting docker-proxy.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Before commit 4f09af6, when allocating host ports for a new
port mapping, iptables rules were set up then docker-proxy was
started. If the host port was already in-use, docker-proxy exited
with an error, and the iptables rules were removed. That could
potentially interfere with a non-docker service that was already
using the host port for something unrelated.
Commit 4f09af6 swapped that problem for a different one... in
order to check that a port was available before creating iptables
rules, it attempted to start docker-proxy first. If it failed, it
could then try a different host port, without interfering with
any other service. The problem with that is docker-proxy would
start listening before the iptables rules were in place, so it
could accept connections then become unusable because new NAT
rules diverted packets directly to the container. This would leave
the client with a broken connection, causing at-least a delay
while it figured that out and reconnected.
This change creates and binds the socket in the daemon, before
creating iptables rules. If the bind fails, it may try a different
port. When or if the bind succeeds, iptables rules are created,
then the daemon calls listen on the socket. If docker-proxy is
needed, the socket is handed over to it at that point.
In rootless mode, the ports have to be bound to an address in the
rootless network namespace (where dockerd is running). DNAT rules
now use the same address.
If docker-proxy is not needed ("--userland-proxy=false"), the daemon
still listens on TCP sockets as the old dummyProxy would have done.
This makes the socket show up in "netstat" output.
The dummyProxy is no longer needed on Linux. Its job was to bind the
host ports if docker-proxy was disabled, but that's now already
handled by binding the sockets early.
This change doesn't affect SCTP, because it's not currently possible
for docker-proxy to convert the file descriptor into an SCTPListener.
So, docker-proxy is still started early, and the window for lost
connections remains.
If the user has an old docker-proxy in their path and it's given a
listener docker with '-use-listen-fd', it'll fail because of the
unknown option. In this case, the daemon's error message suggests
checking $PATH.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Before this change, when running rootless, instead of running
docker-proxy the daemon would run rootlesskit-docker-proxy.
The job of rootlesskit-docker-proxy was to tell RootlessKit
about mapped host ports before starting docker-proxy, and then
to remove the mapping when it was stopped.
So, rootlesskit-docker-proxy would need to be kept in-step
with changes to docker-proxy (particuarly the upcoming change
to bind TCP/UDP ports in the daemon and pass them to the proxy,
but also possible-future changes like running proxy per-container
rather than per-port-mapping).
This change runs the docker-proxy in rootless mode, instead of
rootlesskit-docker-proxy, and the daemon itself tells RootlessKit
about changes in host port mappings.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Get rid of "FIXME: Got an API for which error does not match
any expected type!!! error="driver failed programming external
connectivity on endpoint..." from debug logs.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In preparation for the daemon passing a listen fd, add command line
option -use-listen-fd to indicate that the fd is present (as fd 4).
If the new flag isn't given, open the listener as normal.
Refactor the TCP and UDP proxies to be constructed with an existing
TCPListener or UDPConn, respectively. Lift the responsibilty of opening
the listener to the entrypoint. Per the Single Responsibility Principle,
this structure affords changing how the listener is created without
having to touch the proxy implementations.
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
Refactor a pattern where a passed `*ocispec.Platform` was used to
create a platform matcher that matches the passed platform if not nil
and uses a default host platform otherwise into a separate function.
Also add some basic unit tests for its behavior.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When `getBestPresentImageManifest` fails with `errPlatformNotFound` -
fill the requested platform.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
All two usages only care about the "first" result from the slice sorted
according to the platform preference.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Return a similar error as the graphdrivers implementation when an image
was found, but the requested platform is not present locally or in the
image.
The message doesn't include the "actual" platform, as it doesn't make
sense with the multi-platform images. With graphdrivers all images were
single platform.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `refOrId` parameter is only needed to construct an error in case
where the doesn't have the matching platform.
Move the responsibility of creating the error message to the caller.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- removes use of deprecated github.com/containerd/containerd/errdefs package
- removes use of deprecated github.com/containerd/containerd/platforms package
- removes use of deprecated github.com/containerd/containerd/reference/docker package
- switch to dario.cat/mergo v1.0.0 dependency
- remove use of deprecated CRI Alpha API
full diff: https://github.com/containerd/nydus-snapshotter/compare/v0.13.14...v0.14.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Slightly touch-up the security policy in this repository to describe
the process in more details.
- Describe process around reporting, triage, and review.
- Describe timelines for acknowledging reports.
- Refer to supported versions / branches.
Some of this wording was adopted from containerd's policy, adjusting
where needed (e.g. the project currently does not have an embargoed
security announce list, and no formal definition of security advisors).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 0071832226 introduced
per-endpoint sysctls, and migration to them from the top-level
'--sysctl' option.
The migration was intended to be short-term, disabled in the
next major release and code was added to check for the next
API version. But now, the API version will be bumped in a
minor release - this breaking change needs to wait until the
next major release, and we don't yet know the API version
number for that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Similar to EnableIPv6:
- Set it if EnableIPv4 is specified in a create request.
- Otherwise, set it if included in `default-network-opts`.
- Apart from in a config-from network, so that it doesn't look
like the API request set the field.
- Include the new field in Network marshalling/unmarshalling test.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.
Rendering the docs (`make swagger-docs`) showed a warning:
> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was added in 812f319a57, but it
looks like redoc doesn't like the field in this location, producing a
warning.
Rendering the docs (`make swagger-docs`) showed a warning:
> Warning: Other properties are defined at the same level as $ref at
> "#/definitions/SystemInfo/properties/Containerd". They are IGNORED
> according to the JsonSchema spec
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "not mounted" error from .integration-daemon-stop is not an error,
so let's suppress that one (this is the only effect of -q option, at
least according to its man page).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In docker versions before v24.0, the dockerd daemon allowed adjusting
its own oom-score; in that case we would forward the configuration to
containerd (when running as child process) for it to do the same.
This feature was deprecated in 5a922dc162,
and removed in fb96b94ed0, but some fields
were left behind in the supervisor package.
This patch removes the oomScore field and adjustOOMScore method from
the remove type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `shim.BinaryName()` function used from containerd is part of the
"github.com/containerd/containerd/runtime/v2/shim" packaqge, which comes
with a large number of dependencies.
This patch implements a local variant of the check so that we can remove
the dependency.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Most tests in integration/networking/bridge_test.go are
skipped on Windows - and I want to add a test that uses
helpers that aren't implemented on Windows.
So, move it to bridge_linux_test.go, remove the skips
and put the couple of Windows/nat tests in their own file.
(TestPortMappedHairpin was the only dual-platform test,
it's now got two versions.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Commit 4b9dc647 (from 2016) added this in response to a
review comment, but I don't think it can ever have worked
because n.enableIPv6 = netMap["enableIPv6"].(bool), added
in 2015, would already have panic'd if the field wasn't
stored.
In any case, it's no longer needed.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The cleanups var collided with the cleanups import; rename it to use
the same name as is used in the Windows implementation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a more distinct name, so that local variables can use it. While
at it, also added GoDoc to describe its functionality.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The same code was used both on Linux and Windows; move it to a platform-
agnostic file so that both can use this function, which contains GoDoc
describing the functionality.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
[buildkit@29b4b1a537][1] applied changes to `buildkitd` to set the correct
defaults, which should be 16MB, but used the library defaults. Without that
change, builds using large Dockerfiles would fail with a `ResourceExhausted`
error;
=> [internal] load build definition from Dockerfile
=> => transferring dockerfile: 896.44kB
ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (44865299 vs. 16777216)
However those changes were applied to the `buildkitd` code, which is the
daemon when running BuildKit standalone (or in a container through the
`container` driver). When running a build with the BuildKit builder compiled
into the Docker Engine, that code is not used, so the BuildKit changes did
not fix the issue.
This patch applies the same changes as were made in [buildkit@29b4b1a537][1]
to the gRPC endpoint provided by the dockerd daemon.
[1]: 29b4b1a537
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove a special `Details` parameter from the `GetImage` options and
extract its behavior to a `ImageInspect` method as it was only used by
the `/images/{name}/json` endpoint (`docker image inspect`).
This makes it easier for the containerd image service to output an image
inspect output without having to use the same data structures as the
graphdrivers.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].
The userns package is used in many places, and currently either depends
on runc/libcontainer, or on containerd, both of which have a complex
dependency tree. This patch is part of a series of patches to unify the
implementations, and to migrate toward that implementation to simplify
the dependency tree.
[1]: 3778ae603c
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Makes TestDaemonDefaultNetworkPools re-runnable, and stops the
accumulation of bridges from the others.
Signed-off-by: Rob Murray <rob.murray@docker.com>
`dockerd-rootless-setuptool.sh check` now skips the smoke test for
running RootlessKit.
Fix docker/docker-install issue 417
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This check was added in 2355151556 to handle
non-standard behavior in CentOS/RHEL 7;
> Add check for RHEL7/CentOS7 experimental userns disabled
>
> Add a check in `check-config.sh` to see if we are running on a RHEL7 or
> CentOS7 system, which may report that CONFIG_USERNS is OK/enabled, but
> user namespaces still won't work because of the experimental feature
> flag added by Redhat.
>
> This will add a warning if it is actually disabled and notes what has to
> be added to the grub/boot command line to enable it.
Now that CentOS/RHEL 7 reached EOL, we can remove this special case.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was added in 83c2152de5 to
automatically set `/proc/sys/fs/may_detach_mounts=1` on startup.
This is a kernel config available in RHEL7.4 based kernels that enables
mountpoint removal where the mountpoint exists in other namespaces. This
setting is the default, and non-configurable, on upstream kernels since
3.15.
As this option was only supported in RHEL 7.x systems, which reached EOL,
we can remove this code, as it's not doing anything on current kernels.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Configuring CORS headers was deprecated in docker 27.0 through
7ea9acc97f, which disabled them by default
with a temporary `DOCKERD_DEPRECATED_CORS_HEADER` env-var to allow using
the option.
This patch removes the feature altogether; the flag is kept for one more
release to allow printing a more informative error, but can be removed in
the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This plumbs a context down the stack and handles cancellation as needed
so that we can have correlated traces from the API.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When there is an error in parsing an individual log file just close the
log and move on to the next one instead of erroring our the entire
request.
I investigated trying to error correct and scan ahead for corrupted log
files but found this is too much of a risk of parsing things we
shouldn't be and hence why this is just dropping the rest of the file.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This simplifies how we manage log files, especially rotated ones.
It also fixes a long-standing issue to lazily open rotated files so we
don't needlessly start decompressing files that we don't need.
Much of this is just setting things up for commits following this one.
It uses ReaderAtSize for managing all files to be tailed and manages
cleanups by passing closures.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This check was updated in f460110ef5, but
introduced some bugs;
- the regular expressions were meant to define a capturing group, but
the braces (`(`, `)`) were escaped (they previously were used by
`sed`, which requires different escaping), so no value was captured.
- the check itself was not updated to use the resulting `$target_branch`
env-var, so was comparing against the `$GITHUB_BASE_REF` (which is
the branch name before stripping minor versions).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We'll be using release branches for minor version updates, so instead
of (e.g.) a 27.0 branch, we'll be using 27.x and continue using the
branch for minor version updates.
This patch changes the validation step to only compare against the
major version.
Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Docker EE is no more. Point users looking for commercial support at the
currently-maintained commercial products based on the Moby project:
Docker Desktop and Mirantis Container Runtime.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Replace the regexp checking ID validity with a for loop.
The benefits are:
- faster (up to 10x faster with less allocations);
- no init overhead to compile the regexp.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Replace the regexp checking ID validity with a function. The benefits
are:
- function is faster (up to 10x faster with less allocations);
- no init overhead to compile the regexp;
Add a test case.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Without this, the dependency on systemd is said to be "automagic", which
can lead to breakage, for example, if a binary package of docker is
built on a system that has systemd installed then installed on a system
that does not have systemd installed.
for example: https://bugs.gentoo.org/914076
Signed-off-by: William Hubbs <w.d.hubbs@gmail.com>
The test checks that it's possible to set a per-interface sysctl
using '--sysctl' - but, after API v1.46, it's not (and driver option
'com.docker.network.endpoint.sysctls' must be used instead).
Test added in commit fde80fe2
Per-interface sysctls added, with API changes, in commit 00718322
Signed-off-by: Rob Murray <rob.murray@docker.com>
For SCTP port mappings, an iptables mangle rule was create to fix
the checksum. The code comment says it was done to fix a problem
caused by the veth driver reporting NETIF_F_SCTP_CRC since kernel
4.9, when used with NICs that do not support NETIF_F_SCTP_CRC.
But the checksum rule fills in the checksum in the IP header,
not an SCTP checksum, so it doesn't seem related. And, quoting
from the github issue, "since kernel v4.19, the xt_CHECKSUM.c
explicitly states it should only be used for UDP and only in the
OUTPUT chain - while docker is using it for SCTP in the PREROUTING
chain".
Furthermore, the rule is reported to be causing errors for SCTP
between containers.
Because we don't entirely understand why the rule exists, it
is now only added if the daemon's env has
DOCKER_IPTABLES_SCTP_CHECKSUM=1. If no problems are reported, that
escape-hatch and the code to add the rule will be removed in a
future release.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Capabilities were implemented in aa96c3176b,
as part of work on an external graphdriver-plugin. Given that none of
the builtin graphdrivers use this option, and support for graphdriver-
plugins has been removed in 555dac5e14,
we can remove this functionality.
This patch:
- removes the CapabilityDriver interface, which has no implementations
- removes the Capabilities type
- layer: remove layerStore.useTarSplit. This field was previously set
through the driver's Capabilities, but always enabled for the builtin
graphdrivers,
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was unnecessary; access to the variable was synchronized through a
mutex in all cases but one, where synchonizing through the mutex would
work just fine without any chance of deadlocks.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The netlink.RouteAdd in setupBridgeIPv6 always failed, because
the bridge is always 'down' when it's called.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This is a follow-up to 1abc8f6158, which
moved the ContainerJSONBase to api/types/container, but also renamed it
to container.InspectBase. This field is embedded into the InspectResponse
type, which meant that renaming the type also implicitly renamed the
field when creating this type from a struct-literal.
While we're planning to merge these types (which would be a breaking
change for users constructing it through struct-literals), let's keep
it backward-compatible for now (other than deprecating the old names).
We can continue the other changes separately.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.12+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.11...go1.21.12
These minor releases include 1 security fixes following the security policy:
net/http: denial of service due to improper 100-continue handling
The net/http HTTP/1.1 client mishandled the case where a server responds to a request with an "Expect: 100-continue" header with a non-informational (200 or higher) status. This mishandling could leave a client connection in an invalid state, where the next request sent on the connection will fail.
An attacker sending a request to a net/http/httputil.ReverseProxy proxy can exploit this mishandling to cause a denial of service by sending "Expect: 100-continue" requests which elicit a non-informational response from the backend. Each such request leaves the proxy with an invalid connection, and causes one subsequent request using that connection to fail.
Thanks to Geoff Franks for reporting this issue.
This is CVE-2024-24791 and Go issue https://go.dev/issue/67555.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.21.12
**- Description for the changelog**
```markdown changelog
Update Go runtime to 1.21.12
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Update the containerd binary that's used in CI and for the static packages.
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.19
- full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19
Welcome to the v1.7.19 release of containerd!
The nineteenth patch release for containerd 1.7 contains various updates and
splits the main module from the api module in preparation for the same change
in containerd 2.0. Splitting the modules will allow 1.7 and 2.x to both exist
as transitive dependencies without running into API registration errors.
Projects should use this version as the minimum 1.7 version in preparing to
use containerd 2.0 or to be imported alongside it.
Highlights
- Fix support for OTLP config
- Add API go module
- Remove overlayfs volatile option on temp mounts
- Update runc binary to v1.1.13
- Migrate platforms package to github.com/containerd/platforms
- Migrate reference/docker package to github.com/distribution/reference
Container Runtime Interface (CRI)
- Fix panic in NRI from nil CRI reference
- Fix Windows HPC working directory
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch to use github.com/containerd/platforms module, because containerd's
platforms package has moved to a separate module. This allows updating the
platforms parsing independent of the containerd module itself.
The package in containerd is deprecated, but kept as an alias to provide
compatibility between codebases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Highlights
- Fix support for OTLP config
- Add API go module
- Remove overlayfs volatile option on temp mounts
- Update runc binary to v1.1.13
- Migrate platforms package to github.com/containerd/platforms
- Migrate reference/docker package to github.com/distribution/reference
Container Runtime Interface (CRI)
- Fix panic in NRI from nil CRI reference
- Fix Windows HPC working directory
full diff: https://github.com/containerd/containerd/compare/v1.7.18...v1.7.19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Same applies to this field as for journalReadDir above it.
daemon/logger/journald/journald.go:66:2: field `readSyncTimeout` is unused (unused)
readSyncTimeout time.Duration
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was moved and deprecated in 1fc9236119
(4587688258 for v27.0). This patch removes the
temporary alias and removes the relocated api/types/container.ContainerNode
as well as the Node field on the api/types/container.Base struct.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/registry.SearchOptions in f6cc76ceb9.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types were moved to api/types/container in 0a4277abf4.
This removes the temporary aliases for:
- ThrottlingData
- CPUUsage
- CPUStats
- MemoryStats
- BlkioStatEntry
- BlkioStats
- StorageStats
- NetworkStats
- PidsStats
- Stats
- StatsJSON (moved/renamed to api/types/container.StatsResponse)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/container.StatsResponseReader in 17c3269a37.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/container.CopyToContainerOptions in fd1d8f323b.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/container.PruneReport in db2f1acd5d.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/container.ExecInspect in 5b27e71521.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's replaced by api/types/container.ExecStartOptions and ExecAttachOptions
in cd76e3e7f8. This patch removes the temporary
alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/container.ExecOptions in cd76e3e7f8.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's replaced by api/types/network.Inspect and api/types/network.Summary in
68bf0e7625. This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.EndpointResource in 68bf0e7625.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.DisconnectOptions in 245d12175f.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.ConnectOptions in 245d12175f.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.InspectOptions in 5bea0c38bc.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.CreateResponse in 89624e09e6.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.CreateOptions in 162ef4f8d1.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It moved to api/types/network.CreateRequest in 162ef4f8d1.
This patch removes the temporary alias.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `ContainerJSONBase`, `ContainerJSON` and `ContainerNode`
types to the api/types/container package and deprecates the old location.
- `ContainerJSONBase` was renamed to `InspectBase`
- `ContainerJSON` was rnamed to `InspectResponse`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `Container` type to the containere package, rename
it to `Summary`, and deprecates the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `ContainerState` type to the container package,
renames it to `State`, and deprecates the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `ImageInspect` and `RootFS` types to the image package,
and deprecates the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `GraphDriverData` type is shared between images and containers, and
putting it in either package would result in a circular import, so adding
a new package for this type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the `Health` and `HealthcheckResult` types to the container package,
as well as the related `NoHealthcheck`, `Starting`, `Healthy`, and `Unhealthy`
consts.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the NetworkSettings, NetworkSettingsBase, DefaultNetworkSettings,
and SummaryNetworkSettings types to the api/types/container package, and
deprecates the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When image is built with buildkit with containerd integration the image
service has no way of knowing that the image was tagged because buildkit
creates the image directly in containerd image store.
Add a callback that is called by the exporter wrapper.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This is only a callback that notifies about event so there is no way to
react to the error.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The execution-driver was replaced with containerd since docker 1.11 (API
v1.23) in 9c4570a958, after which the value
was no longer set. The field was left in the type definition.
Commit 1fb1136fec removed its use from the
CLI and [docker/engine-api@39c7d7e] removed it from the API type, followed
by an update to the API docs in 3c6ef4c29d.
Changes to the API types were not pulled into the engine until v1.13, and
probably because of that gated it on API version < 1.25 instead of < 1.24
(see 6d98e344c7); setting a "not supported"
value for older versions.
Based on the above; this field was deprecated in API v1.23, and empty
since then. Given that the minimum API version supported by the engine
is not v1.24, we can safely remove it.
[docker/engine-api@39c7d7e]: 39c7d7ec19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The RefCounter is used in both graphdrivers and snapshotters. Move it
to a separate package to help decoupling snapshotters and graphdrivers,
and make it internal, as it's not intended to be used as a generic utility
package (we can still make it public if there would be a need).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package is only used by the daemon, so move it to the internal
rootless package instead.
Note that technically this could be in daemon/internal, but as there's
already an existing internal/rootless package (which needs to be in the
top-level internal package because it's also used by /plugin), I'm moving
it there.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Checker interface was introduced in 1ba05cdb6a
as an optimization to allow passing a simplified check for situations that
don't require mountinfo.Mounted to be executed (as that may result in parsing
`/proc/self/mountinfo`).
The Checker was defined as an interface with a single `IsMounted` method,
possibly with the intent to allow for additional kind of checks to be added.
No new additions were made since its inception 9 Years ago, and if a need would
arrive, could probably be implemented as part of the check.
This patch simplifies the definition to a function, removing the need to
implement a wrapper struct just to satisfy the interface. The `Checker`
type definition is somewhat redundant, but is kept to have a place to
provide GoDoc.
The `NewFsChecker` and `NewDefaultChecker` utilities are removed as part
of this change, favoring a local definition for storage-drivers that
used them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While detection of filesystem-types may not be supported on all platforms,
it should be ok to define the types; this would allow for these types to
be used to print names (e.g.) in cross-platform situations.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These utilities were used in both graphdrivers and snapshotters. Move them
to a separate package, to help decoupling snapshotters and graphdrivers,
and make it internal, as it's not intended to be used as a generic utility
package (we can still make it public if there would be a need).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was deprecated in 437e1ae15e, which is
part of v27.0, and is no longer in use. This was the remaining code
in the runconfig/opts package, which is now removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The only external consumer are the `graphdriver` and `graphdriver/shim`
packages in github.com/docker/go-plugins-helpers, which depended on
[ContainerFS][1], which was removed in 9ce2b30b81.
graphdriver-plugins were deprecated in 6da604aa6a,
and support for them removed in 555dac5e14,
so removing this should not be an issue.
Ideally this package would've been moved inside `daemon/internal`, but it's used
by the `daemon` (cleanupContainer), `plugin` package, and by `graphdrivers`,
so needs to be in the top-level `internal/` package.
[1]: 6eecb7beb6/graphdriver/api.go (L218)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Improve some GoDoc to use docs links
- Change the Windows stub to an actual wrapper function, as we don't
want it to be updateable, and it currently shows as "variable" on
pkg.go.dev, which is confusing.
- Remove "import" comments in preparation of moving this package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Fix process handle leak when launching a job container
- Add EndpointState attribute to the HNSEndpoint struct to support
hyperv containers for k8s
- Add support for loadbalancer policy update in hns
- Changes for checking the global version for modify policy version support
- OutBoundNATPolicy Schema changes (add MaxPortPoolUsage to OutboundNatPolicySetting)
full diff: https://github.com/microsoft/hcsshim/compare/v0.11.5...v0.11.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Regression introduced in 01eecb6.
A port mapping from a specific IPv6 host address can be used
by a container on an IPv4-only network, docker-proxy makes the
connection.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Keep allocated subnets in-order, so that they're not mistakenly
reallocated due to a gap in the list where misplaced subnets should
have been.
Introduced in 9d288b5.
The iterator over allocated subnets was incremented too early, this
change moves it past three clauses in addrSpace.allocatePredefinedPool().
The three new unit tests correspond to a separate failure caused by
incrementing before each of them.
Signed-off-by: Rob Murray <rob.murray@docker.com>
createTarFile accepts a opts (TarOptions) argument to specify whether
userns is enabled; whe should consider always detecting locally, but
at least make sure we're consistently working with the same value within
this function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was discarding EPERM errors if it detected that userns was
enabled; move such checks to the caller-site, so that they can decide
how to handle the error (which, in case of userns may be to log and ignore).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since 838047a1f5, the overlayWhiteoutConverter
is supported with userns enabled, so we no longer need this check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The exported function was only used in tests, and identical in use when
using New with a name provided. Deprecate it, and remove the uses of it
in our (integration-)tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Better describe what this function does (currently), and add a TODO for
consideration to extract some of its magic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that support for external graphdriver-plugins is removed, these functions
are now identical in functionality; combine them, but use a non-exported
variant for internal use to get visibility into where it's used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a follow-up to 6da604aa6a, which
deprecated external graphdriver plugins.
This patch removes the functionality; some warnings / errors are kept in
place, but can be removed in a follow-up release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- the check for `ok` was redundant as the line above it would return early
- the `t` and `ok` variable declaration was redundant, as all variables to
the left of the `strings.Cut` were locally scoped variables.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like the way it picks up #nosec comments changed, causing the
linter error to re-appear;
pkg/archive/archive_linux.go:57:17: G305: File traversal when extracting zip/tar archive (gosec)
Name: filepath.Join(hdr.Name, WhiteoutOpaqueDir),
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like the way it picks up #nosec comments changed, causing the
linter error to re-appear;
builder/remotecontext/remote.go:48:17: G107: Potential HTTP request made with variable url (gosec)
if resp, err = http.Get(address); err != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `Node` field and related `ContainerNode` type were used by the classic
(standalone) Swarm API. API documentation for this field was already removed
in 234d5a78fe (API 1.41 / docker 20.10), and
as the Docker Engine didn't implement these fields for the Swarm API, it
would always have been unset / nil.
Let's do a quick deprecation, and remove it on the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Graphdriver plugins] are an experimental feature that allow extending the
Docker Engine with custom storage drivers for storing images and containers.
This feature was not maintained since its inception, and will no longer be
supported in upcoming releases.
Users of this feature are recommended to instead configure the Docker Engine
to use the [containerd image store], and a custom [snapshotter].
This patch:
- Disables loading graphdriver plugins by default, producing an error instead.
- Introduces a temporary `DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS` environment
variable to re-enable the deprecated features; this allows users to still
use the feature on a v27.0 daemon, but disabling it by default will give a
strong message that it will no longer be supported.
[Graphdriver plugins]: https://github.com/docker/cli/blob/v26.1.4/docs/extend/plugins_graphdriver.md
[containerd image store]: https://docs.docker.com/storage/containerd/
[snapshotter]: https://github.com/containerd/containerd/tree/v1.7.18/docs/snapshotters
Before this patch (ignore the "Unable to load plugin" errors, as there's no plugin);
dockerd --experimental -s my-driver
...
INFO[2024-06-21T10:42:49.574901255Z] containerd successfully booted in 0.011384s
INFO[2024-06-21T10:42:50.575891922Z] [graphdriver] trying configured driver: my-driver
WARN[2024-06-21T10:42:50.576121547Z] Unable to locate plugin: my-driver, retrying in 1s
WARN[2024-06-21T10:42:51.577131506Z] Unable to locate plugin: my-driver, retrying in 2s
WARN[2024-06-21T10:42:53.582637715Z] Unable to locate plugin: my-driver, retrying in 4s
With this patch:
dockerd --experimental -s my-driver
...
INFO[2024-06-21T10:32:35.123078845Z] [graphdriver] trying configured driver: my-driver
ERRO[2024-06-21T10:32:35.123127012Z] Failed to GetDriver graph driver=my-driver error="DEPRECATED: Experimental graphdriver plugins are deprecated, and disabled by default. This feature will be removed in the next release. See https://docs.docker.com/go/deprecated/" home-dir=/var/lib/docker
INFO[2024-06-21T10:32:35.124735595Z] stopping healthcheck following graceful shutdown module=libcontainerd
INFO[2024-06-21T10:32:35.124743137Z] stopping event stream following graceful shutdown error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: error initializing graphdriver: driver not supported: my-driver
With the `DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS` env-var set:
DOCKERD_DEPRECATED_GRAPHDRIVER_PLUGINS=1 dockerd --experimental -s my-driver
...
INFO[2024-06-21T10:35:04.149901970Z] containerd successfully booted in 0.013614s
INFO[2024-06-21T10:35:05.148195845Z] [graphdriver] trying configured driver: my-driver
WARN[2024-06-21T10:35:05.150647679Z] Unable to locate plugin: my-driver, retrying in 1s
WARN[2024-06-21T10:35:06.152531221Z] Unable to locate plugin: my-driver, retrying in 2s
WARN[2024-06-21T10:35:08.158452389Z] Unable to locate plugin: my-driver, retrying in 4s
WARN[2024-06-21T10:35:12.163699293Z] Unable to locate plugin: my-driver, retrying in 8s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since it's a [][]string, there should only be two levels of array
in the OpenAPI spec. Also, the outermost level array shouldn't have
properties: (it should have items: instead).
Co-authored-by: Mark Yen <mark.yen@suse.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows distros to put docker-proxy under libexec paths as is done
for docker-init.
Also expands the lookup to to not require a `docker/` subdir in libexec
subdir.
Since it is a generic helper that may be used for something else in the
future, this is only done for binaries with a `docker-`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Move the function internal to the daemon, where it's used. Deliberately
not mentioning the new location, as this function should not be used
externally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 17c3269a37 moved the ContainerStats
type to the container package, and renamed it to StatsResponse. However,
this name is chosen poorly, as it documents it to be the response of
the API endpoint, but is more accurately a wrapper around a reader,
used to read a (stream of) StatsJSON. We want to change StatsJSON
to StatsResponse, as it's more consistent with other response types.
As 17c3269a37 did not make it into a
non-pre-release, we can still change this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- inline the auth.TokenHandlerOptions in the auth.NewTokenHandlerWithOptions call
- construct a authHandlers slice to make it more clear that this is a variadic
list of authentication-handlers.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Found back the PR related to this contributors, and they addressed
their name in an intermediate rebase, but it got lost in a later one.
While at it, also fixed an entry next to it :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the system (or Docker) crashes while saivng the volume options, on
restart the daemon will error out when trying to read the options file
because it doesn't contain valid JSON.
In such a crash scenario, the new volume will be treated as though it
has the default options configuration. This is not ideal, but volumes
created on very old Docker versions (pre-1.11[1], circa 2016) do not
have opts.json and so doing some kind of cleanup when loading the volume
store (even if we take care to only delete empty volumes) could delete
existing volumes carried over from very old Docker versions that users
would not expect to disappear.
Ultimately, if a user creates a volume and the system crashes, a volume
that has the wrong config is better than Docker not being able to start.
[1]: commit b05b237075 ("Support mount opts for `local` volume driver")
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Update daemon versions, and minimum supported API version to be more
representative to what the API would return.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update daemon versions, and minimum supported API version to be more
representative to what the API would return.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I noticed some duplicates made their way in, in
084219a5f9 and some authors
didn't have git configured properly to include the name
they used for the sign-off
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The stringid.TruncateID utility is used to provide a consistent length
for "short IDs" (containers, networks). While the dummy interfaces need
a short identifier, they use their own format and don't have to follow
the same length as is used for "short IDs" elsewhere.
In addition, stringid.TruncateID has an additional check for the given
ID to contain colons (":"), which won't be the case for network-IDs that
are passed to it, so this check is redundant.
This patch moves the truncating local to the getDummyName function, so
that it can define its own semantics, independent of changes elsewhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function description mentions that the returned value will contain
a truncated ID, but the function was only prepending the prefix, which
meant that callers had to be aware that truncating is necessary.
This patch moves truncating the ID into the utility to make its use
less error-prone, and to make the code a bite more DRY.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The stringid.TruncateID utility is used to provide a consistent length
for "short IDs" (containers, networks). While the dummy interfaces need
a short identifier, they use their own format and don't have to follow
the same length as is used for "short IDs" elsewhere.
In addition, stringid.TruncateID has an additional check for the given
ID to contain colons (":"), which won't be the case for network-IDs that
are passed to it, so this check is redundant.
This patch moves the truncating local to the getDummyName function, so
that it can define its own semantics, independent of changes elsewhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function description mentions that the returned value will contain
a truncated ID, but the function was only prepending the prefix, which
meant that callers had to be aware that truncating is necessary.
This patch moves truncating the ID into the utility to make its use
less error-prone, and to make the code a bite more DRY.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Following changes to the port mapping code, the DNAT iptables rule was
inserted into the nat table rather than appended.
This meant DNAT was applied before the rule that should have skipped
it when a packet was from a bridge network.
So, packets sent from a container on one network to a mapped port on
the host's address were DNAT'd before docker-proxy could pick them up,
then they were dropped by a rule intended to isolate the networks.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This is the API version to be released with v27.0, and the API
is now frozen for this release, so we can create the documentation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type is included in various types used in the API, but comes from
a separate module. The go-units module may be moving to the moby org,
and it is yet to be decided if the Ulimit type is a good fit for that
module (which deals with more generic units, such as "size" and "duration"
otherwise).
This patch introduces an alias to help during the transition of this type
to it's new location. The alias makes sure that existing code continues
to work (at least for now), but we need to start updating such code after
this PR is merged.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- rename variables that shadowed imports
- remove some intermediate vars
- slight reformating for readability
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 18f4f775ed.
Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).
Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.
By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This reverts commit d365702dbd.
Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).
Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.
By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.
Signed-off-by: Rob Murray <rob.murray@docker.com>
While working on this file, I noticed the `isContainerCreateNameConflict`,
`isUnknownContainer`, and `isStoppedContainer` utilities, which are used
to perform error-type detection through string-matching.
These utilities were added in 534a90a993,
as part of the initial implementation of the Swarm executor in Docker.
At that time, the Docker API client did not return typed errors, and
various part of the code depended on string matching, which is brittle,
and it looks like `isContainerCreateNameConflict` at least is already
broken since c9d0a77657, which changed
the error-message.
Starting with ebcb7d6b40, we use typed
errors through the errdefs package, so we can replace these utilities:
The `isUnknownContainer` utility is replace by `errdefs.IsNotFound`,
which is returned if the object is not found. Interestingly, this utility
was checking for containers only (`No such container`), but was also
used for an `removeNetworks` call. Tracking back history of that use to
verify if it was _intentionally_ checking for a "container not found"
error;
- This check added in the initial implementation 534a90a993
- Moved from `controller.Remove` to `container.Shutdown` to make sure the
sandbox was removed in 680d0ba4ab
- And finally touched again in 70fa7b6a3f,
which was a follow-up to the previous one, and fixed the conditions
to prevent returning early before the network was removed.
None of those patches mention that these errors are related to containers,
and checking the codepath that's executed, we can only expect a
`libmetwork.ErrNoSuchNetwork` to be returned, so this looks to have been
a bug.
The `isStoppedContainer` utility is replaced by `errdefs.IsNotModified`,
which is the error (status) returned in situations where the container
is already stopped; caf502a0bc/daemon/stop.go (L30-L35)
This is the only
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function returns the default network to use for the daemon platform;
moving this to a location separate from runconfig, which is planned to
be dismantled and moved to the API.
While it might be convenient to move this utility inside api/types/container,
we don't want to advertise this function too widely, as the default returned
can ONLY be considered correct when ran on the daemon-side. An alternative
would be to introduce an argument (daemonPlatform), which isn't very convenient
to use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility is only used in two places, and simple enough to duplicate.
There's no external consumers, and a copy of this utility exists in docker/cli
for use on the client side, so we could consider skipping deprecation,
but just to be on the safe side ':)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These used aliases that weren't used elsewhere, so renaming / removing
to keep some consistency. Some local variables were renamed to prevent
shadowing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This populates the "Image" field on containerd containers, but only when
using the containerd image store.
This allows containerd clients to look up the image information.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This will be used in the next commit to test that changes are propagated
to the containerd store.
It is also just generally useful for debugging purposes.
- docs/api: update version history
- daemon: add fillContainerdInfo utility
- api: update swagger file with new types
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Buildkit doesn't call the engine API when it builds an image without
tag. Wrap the exporter returned by the worker that calls a callback when
a new image is exported from buildkit.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This const was exported because it was in use by pkg/symlink. This
dependency was removed in a48c6e3005,
after which this const was only used internally.
This patch deprecates the const and introduces a non-exported const
to use.
There are no known external consumers of this const, so we may skip
deprecating it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use a switch instead of if/else for readability and to reduce
the risk of duplicates in the checks.
- Align order between Windows and Linux implementation for easier
comparing of differences in the implementation.
- Add a check for `IsHost()` in the Windows implementation which
would never occur currently, but is implemented.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not a full list yet, but renaming to prevent shadowing, and to use a more
consistent short form (ctr for container).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The commit endpoint accepts a container.Config, but uses the decoder to
unmarshal the request. The decoder uses a CreateRequest, which is a superset,
and also contains HostConfig and network.NetworkConfig. Those structs are
discarded in the router, but decoder.DecodeConfig also performs validation,
so a request containing those additional fields would result in a validation
error.
We should rewrite this code to only unmarshal what's expected.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the type to api/types/container.CreateRequest, together with other
types used by the container API endpoints.
The Decoder, and related validation code is kept in the runconfig package
for now, but should likely be moved elsewhere (inside the API).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove uses of this function and mark it deprecated. There's no known
consumers of this function, but let's stay on the safe side, and mark
it deprected for 1 release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility used to be responsible for backward compatibility with old
API versions, but was reduced to a single line. Inline the code, and
inline the SetDefaultNetModeIfBlank code, which in itself also was
just 3 lines of code.
A platform check was added to only set the default network conditionally,
but other paths in the codebase don't perform this conditionally, so a
TODO was added, to verify if this behavior is needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that the backward-compatibility code has been removed, the Linux and
Windows implementations of this struct are identical, so the platform-
specific code can be removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The runconfig package contained compatibility code to unmarshal API
requests on API < v1.18, and to convert them to current API versions.
These fields were marked as deprecated, but didn't mention relevant API
versions, so some digging was needed to track back history;
API versions before 1.18 accepted top-level `Memory`, `MemorySwap`,
`CpuShares`, and `Cpuset` fields as part of the container create requests.
These fields were not considered "portable", and therefore moved to the
`HostConfig` struct in 837eec064d. The
API version at that time was [v1.18]. For backward-compatibility, the
existing top-level fields were kept, and conversion code was added in
[ContainerHostConfigFromJob] to copy their values to `HostConfig` if
present.
A refactor in 767df67e31 introduced a new
`ContainerConfigWrapper` struct, which embedded the container-config and
a (non-exported) `hostConfigWrapper`. This resulted in an incompatibility
when compiling with gccgo, sn eb97de7dee
removed the non-exported `hostConfigWrapper`, instead embedding the
`HostConfig` and adding a `CpuSet` field. The API version at that time
was [v1.19].
With the introduction of Windows containers, which did not need conversion
code as it never supported previous API versions, the `ContainerConfigWrapper`
was split to Linux and Windows implementation in f6ed590596.
This change introduced a `SetDefaultNetModeIfBlank` function to set the
default network-mode on Linux. Windows did not have a default network,
but did require a separate `ValidateNetMode` implemenation.
The `ContainerConfigWrapper` was expanded to include `NetworkingConfig`
in 2bb3fc1bc5 for API [v1.22], but did
not involve backward-compatiblity / conversion code.
Based on the above, all conversion code present in runconfig is related
to API versions [v1.18] or before. 19a04efa2f,
and other commits in [moby PR 47155] removed support for API < v1.24, so
this conversion code is no longer needed.
This patch removes the legacy fields from the `ContainerConfigWrapper`,
and removes the corresponding conversion code. The `InnerHostConfig` field
is also renamed, as it is no longer shadowed by the `container.HostConfig`
that was embedded for backward-compatibility.
[v1.18]: 837eec064d/api/common.go (L18)
[v1.19]: 767df67e31/api/common.go (L20)
[v1.22]: 2bb3fc1bc5/api/common.go (L21)
[moby PR 47155]: https://github.com/moby/moby/pull/47155
[ContainerHostConfigFromJob]: 837eec064d/runconfig/hostconfig.go (L149-L162)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was depending on top-level fields that were deprecated since
API v1.18. These fields are no longer sent by current clients.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This wrapper is now a plain alias for hcsshim.HNSEndpointRequest, so let's
remove the extra abstraction.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This synchronisation was added in [libnetwork@0a61693]:
> Adding synchronization around peerAdd and peerDelete to prevent network
> connectivity issue
>
> When multiple networks are present in a Swarm Cluster, multiple peerAdd
> or peerDelete calls are an issue for different remote endpoints. These
> threads are updating the remote endpoint to HNS parallelly. In 2016 HNS
> code base, we don't have synchronization around remoteEndpoint addition
> and deletion. So serializing the peerAdd and peerDelete calls from docker
> network driver.
We no longer support and test Windows 2016, as it reached EOL / end of
[standard support][1], so we can remove this special condition.
[libnetwork@0a61693]: c90114ce7c
[1]: https://en.wikipedia.org/wiki/Windows_10,_version_1607
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
TestChangesDirsEmpty and TestChangesDirsMutated fail on Windows V19H1 (1903)
and up, possibly due to changes in the kernel:
=== FAIL: github.com/docker/docker/pkg/archive TestChangesDirsEmpty (0.21s)
changes_test.go:261: Reported changes for identical dirs: [{\dirSymlink C}]
=== FAIL: github.com/docker/docker/pkg/archive TestChangesDirsMutated (0.14s)
changes_test.go:391: unexpected change "C \\dirSymlink" "\\dirnew"
commit 8f4b3b0ad4 added a version-dependent
skip for those tests, but as we no longer run CI on versions before V19H1,
we can remove the kernel-version check, and skip it on Windows unconditionally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While using iota can be convenient, it can also make it harder to grasp
what value is assigned. Use of iota also makes changing values implicit;
changing the order of these consts implicitly means their value changes.
This can be problematic, as some of these consts are a plain `int` and
while golang is strong-typed, it does allow plain `int` values to be
used for such values.
For example, `archive.Tar` accepts a `Compression` as second argument,
but allows a plain int to be passed, so both of these are equivalent;
archive.Tar(contextDir, archive.Uncompressed)
archive.Tar(contextDir, 0)
This patch removes the use of `iota`, and instead explicitly setting a
value for each to prevent accidental changes in their value, which can
be hard to discover.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
[pkg/system.Lgetxattr] is only implemented on Linux, and always produces
an ErrNotSupportedPlatform on other platforms.
This patch removes the call to this function, but intentionally leaves
it commented-out as a reminder to include this code if this would ever
be refactored and implemented on other platforms.
[pkg/system.Lgetxattr]: d1273b2b4a/pkg/system/xattrs_unsupported.go (L1-L8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were added in baacae8345, but are
currently only used in tests inside pkg/archive. There are no external
users of this function, so we should deprecated them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit d59758450b changed this function to
be a wrapper for `filepath.ToSlash`. It was used in the CLI for the classic
builder, but is no longer used in our codebase.
However, there may still be some consumers that copied the CLI code for the
classic builder that didn't synchronise their implementation yet, so let's
deprecate this function to give them a warning that they should no longer
use this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gosec uses a non-standard format for "automated" comments to suppress
false positives (such comments should not have a leading space, but
are not allowed to start with a non-alphabetical character). However,
current versions of gosec do allow a leading space.
This patch reformats the comments to prevent them from being changed
by IDEs when reformating code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gosec uses a non-standard format for "automated" comments to suppress
false positives (such comments should not have a leading space, but
are not allowed to start with a non-alphabetical character). However,
current versions of gosec do allow a leading space.
This patch reformats the comments to prevent them from being changed
by IDEs when reformating code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package was originally added in 46833ee1c3
for use in the devicemapper graphdriver. The devicemapper graphdriver was
deprecated and has been removed. The only remaining consumer is an integration
test.
Deprecate the package and mark it for removal in the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This value was originally added in 46833ee1c3,
at which time golang.org/x/sys/unix didn't have utilities for this syscall.
A later patch switched the implementation to use the golang/x/sys/unix
implementation in 2841b05b71, but kept the
local variable.
golang.org/x/sys now has a const for this, so let's use it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The only case where macvlan interfaces are unable to share a parent is
when the macvlan mode is passthru. This change tightens the check to
that situation.
It also makes the error message more specific to avoid suggesting that
sharing parents is never correct.
Signed-off-by: Andrew Baxter <423qpsxzhh8k3h@s.rendaw.me>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function is no longer used, and has no external users. Deprecated
the function and mark if for removal for the next release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function is only used for the legacy v1 image format.
Deprecate the function, and make image/v1 self-contained.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in 7a7357dae1 as part of the
LCOW implementation. LCOW has been removed, and this option was no longer
in use because of that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- explicitly suppress some errors
- use fmt.Fprintln instead of manually appending a newline
- remove an outdated TODO; looking at the suggestion, it's not a
realistic option
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- imageMount was shadowing the imageMount type
- copy was shadowing the copy builtin
- container was shadowing the container import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only used in a single location, and other locations were shadowing
the function through local variables. As it's a one-liner, inlining the
code may be just as transparent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The health-check start interval added in API v1.44, and the start
interval option is ignored when creating a Swarm service using an older
API version. However, due to an oversight, the option is not ignored
when older API clients _update_ a Swarm service. Fix this oversight by
moving the adjustment code into the adjustForAPIVersion function used by
both the createService and updateService handler functions.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Includes two commits from Arash Deshmeh:
add exec option to API TmpfsOptions and the related volume functions
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
feature: daemon handles tmpfs mounts exec option
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
Updated by Drew Erny
Signed-off-by: Drew Erny <derny@mirantis.com>
`Daemon.Kill()` was sending a SIGKILL to the daemon process but wasn't
waiting until the process was really killed. While the race window is
really small, better safe than sorry.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When the container stops or during `restore`, `daemon.releaseNetwork` is
used to clear all net-related state carried by a container. However, the
fields `SandboxID` and `SandboxKey` are never cleared. On the next start,
these fields will be replaced with new values. There's no point in
preserving these data since they became invalid as soon as the container
stopped.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When the daemon crashes, the host unexpectedly reboot, or the daemon
restarts with live-restore enabled, running containers might stop and the
on-disk state for containers might diverge from reality. All these
situations are currently handled by the daemon's `restore` method.
That method calls `daemon.Cleanup()` for all the dead containers. In
turn, `Cleanup` calls `daemon.releaseNetwork()`. However, this last
method won't do anything because it expects the `netController` to be
initialized when it's called. That's not the case in the `restore` code
path -- the `netController` is initialized _after_ cleaning up dead
containers.
There's a chicken-egg problem here, and fixing that would require some
important architectural changes (eg. change the way libnet's controller
is initialized).
Since `releaseNetwork()` early exits, dead containers won't ever have
their networking state cleaned. This led to bugs in Docker Desktop,
among other things.
Fix that by calling `releaseNetwork` after initializing the
`netController`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
setupBridgeIPv6 attempts to add a route to a new network while
the bridge device is 'down', so it always fails (and the route
is added anyway when the bridge is set 'up').
I'm almost sure the RouteAdd can be removed but, this close to
the moby 27.0 release, only sure-enough to demote the log message
from error to debug.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The `Sandbox.SetKey()` method is called through an OCI prestart hook
which then calls back the daemon through a UNIX socket. This method is
responsible for provisioning interfaces, etc... into the sandbox.
A new EnvironCarrier is used to propagate the trace context to the
prestart hook, which then marhsals an OTel MapCarrier into the JSON
payload sent back to the daemon. That way, every spans created from
`SetKey()` are correctly parented to the original `ContainerStart` API
call.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Add a nil check to handle a case where the image config JSON would
deserialize into a nil map.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Update the runc binary that's used in CI and for the static packages.
full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13
Release notes:
* If building with Go 1.22.x, make sure to use 1.22.4 or a later version.
* Support go 1.22.4+.
* runc list: fix race with runc delete.
* Fix set nofile rlimit error.
* libct/cg/fs: fix setting rt_period vs rt_runtime.
* Fix a debug msg for user ns in nsexec.
* script/*: fix gpg usage wrt keyboxd.
* CI fixes and misc backports.
* Fix codespell warnings.
* Silence security false positives from golang/net.
* libcontainer: allow containers to make apps think fips is enabled/disabled for testing.
* allow overriding VERSION value in Makefile.
* Vagrantfile.fedora: bump Fedora to 39.
* ci/cirrus: rm centos stream 8.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13
Release notes:
* If building with Go 1.22.x, make sure to use 1.22.4 or a later version.
* Support go 1.22.4+.
* runc list: fix race with runc delete.
* Fix set nofile rlimit error.
* libct/cg/fs: fix setting rt_period vs rt_runtime.
* Fix a debug msg for user ns in nsexec.
* script/*: fix gpg usage wrt keyboxd.
* CI fixes and misc backports.
* Fix codespell warnings.
* Silence security false positives from golang/net.
* libcontainer: allow containers to make apps think fips is enabled/disabled for testing.
* allow overriding VERSION value in Makefile.
* Vagrantfile.fedora: bump Fedora to 39.
* ci/cirrus: rm centos stream 8.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like some packages fail in go module mode, because they require
recent Go versions:
GO111MODULE=on go test -v
# github.com/docker/docker/libnetwork/ipamutils
../../libnetwork/ipamutils/utils.go:46:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipamutils/utils.go:51:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/portallocator
../../libnetwork/portallocator/portallocator.go:179:7: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/netutils
../../libnetwork/netutils/utils_linux.go:66:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/netutils/utils_linux.go:75:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/api/server/router/grpc
../../api/server/router/grpc/grpc.go:56:48: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/container
../../container/view.go:335:47: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/ipams/defaultipam
../../libnetwork/ipams/defaultipam/address_space.go:33:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:53:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:124:10: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:125:21: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:146:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:310:14: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/ipams/defaultipam/address_space.go:311:22: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/drivers/bridge
../../libnetwork/drivers/bridge/port_mapping_linux.go:76:15: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/drivers/bridge/port_mapping_linux.go:201:2: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This internal package was added in f6e44bc0e8
to preserve compatibility with go1.20 and older. At the time, our vendor.mod
still had go1.18 as minimum version requirement (see [1]), which got updated to go1.20
in 16063c7456, and go1.21 in f90b03ee5d
The version of BuildKit we use already started using context.WithoutCancel,
without a fallback, so we no longer can provide compatibility with older
versions of Go, which makes our compatiblity package redundant.
This patch removes the package, and updates our code to use stdlib's context
instead.
[1]: f6e44bc0e8/vendor.mod (L7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added to use a specific format for the vendor.mod/go.mod
file, but we should no longer need this, as go1.21 is now the
minimum.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
26.1.1 added env var DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE to make it
possible to create an IPv4-only network, even with a read-only
"/proc/sys/net" that meant IPv6 could not be disabled on an
interface.
In 27.0 it's easier to enable IPv6, just '--ipv6' when creating the
network - in particular, there's no need to allocate a subnet, because
a unique-local prefix will be assigned by default).
So, this change removes the env-var workaround. Now, the workarounds
are to enable IPv6, mount "/proc/sys/net" read-write, disable IPv6
by default in OS configuration, or remove support for IPv6 from the
kernel.
Signed-off-by: Rob Murray <rob.murray@docker.com>
On an nftables host, the ip6_tables kernel module may not be loaded,
but it needs to be for dockerd to run (with ip6tables now enabled by
default).
If ip6tables doesn't work, try the dind official image's trick for
loading the module using "ip link show".
Signed-off-by: Rob Murray <rob.murray@docker.com>
Small cleanup of this function;
- change to a regular function, as it does not depend on the daemon
- use an early return
- explicitly refer to EndpointSettings.EndpointSettings, not the top-
level EndpointSettings.
- use a struct-literal.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When bridge driver opt com.docker.network.bridge.gatway_mode_ipv[46]
is set to "routed", there is no NAT.
When there's no NAT, there's no meaning to the HostPort field in a
port mapping (all the port mapping does is open the container's port),
and the HostIP field is only used to determine the address family.
So, check port bindings, and raise errors if fields are unexpectedly
set when the mapping only applies to a gateway_mode=routed network.
Zero-addresses are allowed, to say the mapping/open-port should be
IPv4-only or IPv6-only, and host ports are not allowed.
A mapping with no host address, so it applies to IPv4 and IPv6 when
the default binding is 0.0.0.0, may include a host port if either
uses NAT. The port number is ignored for the directly-routed family.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Display a PortBinding in a format that's more like the one
used in the CLI, but includes the container IP if known.
Signed-off-by: Rob Murray <rob.murray@docker.com>
It was added so that tests could replace it before it was picked
up and used by a new network's PortMapper, so that tests were isolated
from each other. Now the PortMapper is not used by the bridge driver,
neither is driver's portAllocator.
Instead of replacing the driver.portAllocator in tests, reset the
singleton instance using its ReleaseAll().
Un-export portallocator.NewInstance, now the tests aren't using it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Without this change, if a port mapping did not specify a host address
and the network was IPv6-enabled, the same port would be allocated for
mappings from '0.0.0.0' and '::'. But, if the port mapping was specified
with explicit addresses even, for example:
-p 0.0.0.0:8080-8083:80 -p '[::]:8083-8080:80'
This change looks for port mappings that only differ in the host IP
address, and makes sure it allocates the same port for all of them. If
it can't, it fails with an error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Add bridge driver options...
com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>
com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>
If set to "routed", no NAT or masquerade rules are set up for port
mappings.
When NAT is disabled, the mapping is shown in 'inspect' output with
no host port number. For example, for "-p 80" with NAT disabled for
IPv6 but not IPv4:
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "32768"
},
{
"HostIp": "::",
"HostPort": ""
}
Signed-off-by: Rob Murray <rob.murray@docker.com>
Use the bridge driver's iptables types to set up portmapping related
iptables rules - instead of using iptables.Forward, which is bridge
specific code in the iptables package.
Remove iptables.Forward() and its unit test, the bridge driver's
version is covered by TestAddPortMappings.
Remove hairpinMode from iptables.ChainInfo hairpinMode relates to bridge
driver specific behaviour, that is now implemented in the bridge driver.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The bridge driver now does its own port-mapping, rather than using the
portmapper module (which ran as two completely separate instances, for
IPv4 and IPv6).
When asked for a mapping from any host address (0.0.0.0/0) with a range
of host ports, the same port will be allocated for IPv4 and IPv6, or the
mapping will fail with an error if that's not possible.
The bridge driver now manages its own port mappings. So, remove
linux-specific PortMapper code and make what's left Windows-only.
Also, replace the portmapper.userlandProxy interface with StartProxy().
Signed-off-by: Rob Murray <rob.murray@docker.com>
For a port mapping like '-p 8080-8083:80', when some non-docker process
is using a port in the range, try other ports in the range. And, don't
do that on live-restore.
Because the port mapping may fail on live-restore, leaving no ports
mapped for the endpoint - update the view of mapped ports shown in
'inspect' output. (The wrong mappings will still be shown in 'docker ps',
the container will be left running and connected to the network, it just
won't work. There's plenty of scope for better error handling here.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Similar to portallocator.RequestPortInRange(), but it attempts to
allocate the same port for multiple IP addresses.
Signed-off-by: Rob Murray <rob.murray@docker.com>
It's an alias for the OCI-spec type, which was only there for
convenience, but will be deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
They might still change to "Mounted from" or "Already exists" when
containerd updates the status in tracker.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The target variable was already overwritten with the new value. Use the
original value instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `Config` field returned by this endpoint (used for "image inspect") returns
additional fields that are not part of the image's configuration and not part of
the [Docker Image Spec] and the [OCI Image Spec].
These additional fields are included in the response, due to an
implementation detail, where the [api/types.ImageInspec] type used
for the response is using the [container.Config] type.
The [container.Config] type is a superset of the image config, and while the
image's Config is used as a _template_ for containers created from the image,
the additional fields are set at runtime (from options passed when creating
the container) and not taken from the image Config.
These fields are never set (and always return the default value for the type),
but are not omitted in the response when left empty. As these fields were not
intended to be part of the image configuration response, they are deprecated,
and will be removed from the API.
The following fields are currently included in the API response, but
are not part of the underlying image's Config, and deprecated:
- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)
[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
[OCI Image Spec]: https://github.com/opencontainers/image-spec/blob/v1.1.0/specs-go/v1/config.go#L24-L62
[api/types.ImageInspec]: https://github.com/moby/moby/blob/v26.1.4/api/types/types.go#L87-L104
[container.Config]: https://github.com/moby/moby/blob/v26.1.4/api/types/container/config.go#L47-L82
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Image.Config field currently reuses the ContainerConfig definition,
matching the Go implementation, which also uses that type.
However, the ContainerConfig type contains various fields that are not
part of the image config, and would never be set. The Image.Config is
used as template / default values for containers started from the image,
but will only use the fields that are part of the [Docker image spec].
This patch updates the swagger files used in the documentation to use a
separate `ImageConfig` definition for the Image.Config field. The new
definition is a copy of the existing `ContainerConfig` type, but with
updated descriptions for fields, and with an example response that omits
the fields that should not be used.
The following fields are currently included in the `Config` field of the API
response, but are not part of the underlying image's config:
- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)
[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Image.Config field currently reuses the ContainerConfig definition,
matching the Go implementation, which also uses that type.
However, the ContainerConfig type contains various fields that are not
part of the image config, and would never be set. The Image.Config is
used as template / default values for containers started from the image,
but will only use the fields that are part of the [Docker image spec].
This patch updates the swagger files used in the documentation to use a
separate `ImageConfig` definition for the Image.Config field. The new
definition is a copy of the existing `ContainerConfig` type, but with
updated descriptions for fields, and with an example response that omits
the fields that should not be used.
The following fields are currently included in the `Config` field of the API
response, but are not part of the underlying image's config:
- `Hostname`
- `Domainname`
- `AttachStdin`
- `AttachStdout`
- `AttachStderr`
- `Tty`
- `OpenStdin`
- `StdinOnce`
- `Image`
- `NetworkDisabled` (already omitted unless set)
- `MacAddress` (already omitted unless set)
- `StopTimeout` (already omitted unless set)
[Docker image spec]: https://github.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add a OCI platform fields as parameters to the `POST /images/{id}/push`
that allow to specify a specific-platform manifest to be pushed instead
of the whole image index.
When no platform was requested and pushing whole index failed, fallback
to pushing a platform-specific manifest with a best candidate (if it's
possible to choose one).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This moves the type, but we should consider removing this type, and just
returning an io.ReadCloser
This type was added in 9fd2c0feb0c131d01d727d50baa7183b976c7bdc;
> Make docker load to output json when the response content type is json
> Swarm hijacks the response from docker load and returns JSON rather
> than plain text like the Engine does. This makes the API library to return
> information to figure that out.
However the "load" endpoint unconditionally returns JSON;
7b9d2ef6e5/api/server/router/image/image_routes.go (L248-L255)
Commit 96d7db665b made the response-type depend
on whether "quiet" was set, but this logic got changed in a follow-up
2f27632cde, which made the JSON response-type
unconditionally, but the output produced depend on whether"quiet" was set.
We should deprecated the "quiet" option, as it's really a client
responsibility.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Note that RequestPrivilegeFunc could not be referenced, as it would
introduce a circular import, so copying the definition instead.
Also combining the other search-related types in the package to be in
the same file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the response type; other types related to stats are left
for now, but should be moved (as well as utilities ported from
the CLI repository).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the type to api/types/container and creates an alias for
exec attach; ContainerExecAttach currently uses the same type as
ContainerExecStart, but does not all the same options (and some
options cannot be used).
We need to split the actual types, but lets start with aliasing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Container and ContainerConfig fields have been deprecated, and removed
since API v1.45 in commit 03cddc62f4.
This patch fixes the swagger and documentation to no longer mention them
as they are no longer returned by API v1.45 and higher.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was added in 428328908dc529b1678fb3d8b033fb0591a294e3;
> Deprecate /containers/(id or name)/copy endpoint
> This endpoint has been deprecated since 1.8. Return an error starting
> from this API version (1.24) in order to make sure it's not used for the
> next API version and so that we can remove it sometimes later.
We deprecated and removed those older API versions, and the test was
effectively only verifying that a non-existing endpoint returns a 404,
so let's remove it.
This also removes api/types.CopyConfig, which was only used in this
test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove intermediate variable
- format a "todo" comment as an actual todo ':)
- explicitly suppress some unhandled errors to keep linters happy
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove deprecated UnsafeSlice and use unsafe.Slice
- Stabilize the behaviour of Prev when the cursor already points to
the first element
- Fix Cursor.Prev() out of range issues in v1.3.9
- Relates to boltdb/bolt/issues/357 (Cursor inconsistent when mixing
cursor.Delete() with Put() in same transaction)
- Bump go version to 1.21.9
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.9...v1.3.10
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Touch-up some GoDoc in the package, and remove "import" comments.
This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. The "import" comments are ignored when
used in go module mode so have little benefit. Let's remove them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. While it's not too problematic to have
this dependency, the tests only used basic assertions from gotest.tools,
which could be easily re-implemented without the dependency.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Windows, Darwin, and FreeBSD implementations were identical, other
than their GoDoc to be different. Unify them so that we don't have to
maintain separate GoDoc for each.
It's worth noting that FreeBSD also supports Pdeathsig, so could be
using the same implementation as Linux. However, we don't test/maintain
the FreeBSD implementation, and it would require updating to GoDoc to
be more specific about the use of `/proc/self/exe`, so keeping the
status quo for now.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This combines the implementations of the Self function, to allow having
a single GoDoc to document the behavior. The naiveSelf function is kept,
because it's used in unit-tests.
There is a minor change in behavior, as this patch removes the stub for
unsupported platforms (non-linux, windows, freebsd or darwin), which will
now use `os.Args[0]`. The stub was added in 21537b818d
to fix compilation of https://github.com/ethereum/go-ethereum on OpenBSD,
which had docker/docker as dependency. It looks like that repository no
longer has this dependency, and as this was only to make the code
compilable, is unlikely to be a problem.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 069fdc8a08 changed most uses of
the syscall package to switch utsname from unsigned to signed (see
069fdc8a08). Those don't seem to be
impacting the code used here, so either stdlib or golang.org/x/sys/unix
should work for this case.
I chose stdlib's syscall package for this case, in case we'd decide to
move this package to a separate module (and want to limit its dependencies).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function largely identical to GetFSMagic, except for suppressing
ENOENT errors. The only consumer of this function was fsChecker.IsMounted,
which would ignore errors either way, and only use the "success" case to
check if the detected filesystem-type was the expected one.
This patch;
- rewrites fsChecker.IsMounted to use GetFSMagic instead
- removes the now unused Mounted function
As we consider daemon/graphdriver to be "internal", and as there are no
public consumers of this, we can remove this function without deprecating
first.
The freebsd implementation also seemed to be broken, as it mixed syscall
with golang.org/x/sys/unix, which used incompatible types. I left the file
in place for now, but we can consider removing it altogether as there's no
active development on making freebsd functional.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before "ip6tables" was enabled by default, dockerd would start normally
when:
- the kernel had no IPv6 support, or
- docker is running as docker-in-docker, and the host doesn't have kernel
module 'ip6_tables' loaded.
Now, the bridge driver will try to set up its ip6tables chains and it'll
fail. By not treating that as an error, the daemon will start and IPv4
will work normally.
A subsequent attempt to create an IPv6 network will fail with an error
about ip6tables. At that point, the user's options are:
- set "ip6tables":false in daemon config
- in the DinD case, "modprobe ip6_tables" on the host, or start dockerd
on the host with ip6tables enabled (causing the kernel module load).
Signed-off-by: Rob Murray <rob.murray@docker.com>
Rob is currently a curator, and has been actively contributing to this
repo for 7 months now.
Beside day-to-day triaging and bug fixing, Rob is an instrumental
contributor to libnetwork, and amongst other things, to the ongoing work
on IPv6 improvements.
I nominated Rob as maintainer, and votes passed, so opening a PR to make
it official.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The CheckDuplicate option is no longer part of the current API; it's
only used by the client when connecting to old API versions, which need
to have this field set.
This patch:
- Removes the CheckDuplicate from the API documentation, as the API
describes the current version of the API (which does not have this
field).
- Moves the CheckDuplicate field to the CreateRequest type; this is
the type used for the network create request. The CheckDuplicate
is not an option that's set by the user, and set internally by
the client, so removing it from the CreateOptions struct moves
it entirely internal.
- Change the CheckDuplicate field to be a pointer; this makes the
"omitempty" become active, and the client will no longer include
the field in the request JSON unless it's set (API < 1.44).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
b5c50afa882e2b34aba880fd5028615e2ef94e07 changed the signature of
NewGatewayFrontend to include a slice of allowed repositories.
Docker does not allow to specify this option, so don't place any
restrictions by passing an empty slice.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
1b1c5bc08ad81add007eb647e66ed0929693f3a0 extended the function signature
with one additional return value.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
eea0b41bf4fb1d69e109ff5ff8045c63f0c0d510 added a new argument to
`instructions.Parse` to support issuing linter warnings.
Classic builder uses it to parse the Dockerfile instructions and its
usage needs adjustment.
The classic builder is deprecated and we won't be adding any new
features to it, so we just pass a nil linter callback.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't fall-back to Google's DNS servers in a network that has an
internal resolver.
Now the default bridge uses the internal resolver, the only reason a
network started by the daemon should end up without any upstream
servers is if the host's resolv.conf doesn't list any. In this case,
the '--dns' option can be used to explicitly configure nameservers
for a container if necessary.
(Note that buildkit's containers do not have an internal resolver, so
they will still set up Google's nameservers if the host has no
resolvers that can be used in the container's namespace.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Until now, containers on the default bridge network have been configured
to talk directly to external DNS servers - their resolv.conf files have
either been populated with nameservers from the host's resolv.conf, or
with servers from '--dns' (or with Google's nameservers as a fallback).
This change makes the internal bridge more like other networks by using
the internal resolver. But, the internal resolver is not populated with
container names or aliases - it's only for external DNS lookups.
Containers on the default network, on a host that has a loopback
resolver (like systemd's on 127.0.0.53) will now use that resolver
via the internal resolver. So, the logic used to find systemd's current
set of resolvers is no longer needed by the daemon.
Legacy links work just as they did before, using '/etc/hosts' and magic.
(Buildkit does not use libnetwork, so it can't use the internal resolver.
But it does use libnetwork/resolvconf's logic to configure resolv.conf.
So, code to set up resolv.conf for a legacy networking without an internal
resolver can't be removed yet.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Update the containerd binary that's used in CI and for the static packages.
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.18
- full diff: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
Welcome to the v1.7.18 release of containerd!
The eighteenth patch release for containerd 1.7 contains various updates along
with an updated version of Go. Go 1.22.4 and 1.21.11 include a fix for a symlink
time of check to time of use race condition during directory removal.
Highlights
- Update Go version to 1.21.11
- Remove uses of platforms.Platform alias
- Migrate log imports to github.com/containerd/log
- Migrate errdefs package to github.com/containerd/errdefs
- Fix usage of "unknown" platform
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update to containerd 1.7.18, which now migrated to the errdefs module. The
existing errdefs package is now an alias for the module, and should no longer
be used directly.
This patch:
- updates the containerd dependency: https://github.com/containerd/containerd/compare/v1.7.17...v1.7.18
- replaces uses of the old package in favor of the new module
- adds a linter check to prevent accidental re-introduction of the old package
- adds a linter check to prevent using the "log" package, which was also
migrated to a separate module.
There are still some uses of the old package in (indirect) dependencies,
which should go away over time.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.21.11 (released 2024-06-04) includes security fixes to the archive/zip
and net/netip packages, as well as bug fixes to the compiler, the go command,
the runtime, and the os package. See the Go 1.21.11 milestone on our issue
tracker for details;
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.11+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.10...go1.21.11
From the security announcement;
We have just released Go versions 1.22.4 and 1.21.11, minor point releases.
These minor releases include 2 security fixes following the security policy:
- archive/zip: mishandling of corrupt central directory record
The archive/zip package's handling of certain types of invalid zip files
differed from the behavior of most zip implementations. This misalignment
could be exploited to create an zip file with contents that vary depending
on the implementation reading the file. The archive/zip package now rejects
files containing these errors.
Thanks to Yufan You for reporting this issue.
This is CVE-2024-24789 and Go issue https://go.dev/issue/66869.
- net/netip: unexpected behavior from Is methods for IPv4-mapped IPv6 addresses
The various Is methods (IsPrivate, IsLoopback, etc) did not work as expected
for IPv4-mapped IPv6 addresses, returning false for addresses which would
return true in their traditional IPv4 forms.
Thanks to Enze Wang of Alioth and Jianjun Chen of Zhongguancun Lab
for reporting this issue.
This is CVE-2024-24790 and Go issue https://go.dev/issue/67680.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, starting dockerd with
`--default-network-opt=bridge=com.docker.network.enable_ipv6=true` has
no effect as `NetworkCreateRequest.EnableIPv6` is a basic bool.
This change makes it a `*bool` to make it optional. If clients don't
specify it, the default-network-opt will be applied.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This method was added in 534a90a993 as
part of adding the Swarm cluster backend, and later updated in commit
85b1fdf15c to use a swarmRequestTimeout
const for the timeout.
Nothing in this utility depends on the Cluster struct, and the abstraction
makes it appear as more than it is, which is just a wrapper for
context.WithTimeout().
Let's remove the abstraction to make it less magical.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves the type to the api/types/network package, but also introduces
a "Summary" alias; the intent here is to allow diverging the types used
for "list" and "inspect" operations, as list operations may only be
producing a subset of the fields available.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These aliases were added in ac2a028dcc,
which was part of the v26.0 and v26.1 releases. We can remove the
aliases, assuming users that depended on this have migrated to the
new location of these types.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The types.NetworkInspectOptions type was moved to the networks package
in 5bea0c38bc and deprecated, but use of it
was re-introduced in cd3804655a, which was
merged out-of-order.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 1a5dafb31e improved the error messages
produced by adding a check if the client is using as an elevated user. For
this, it attempts to open `\\.\PHYSICALDRIVE0`.
However, it looks like closing the file landed in the wrong branch of the
condition, so the file-handle would not be closed when the os.Open succeeded.
Looking further into this check, it appears the conditions were reversed;
if the check _fails_, it means the user is not running with elevated
permissions, but the check would use elevatedErr == nil.
Fix both by changing the condition to `elevatedErr != nil`.
While at it, also changing the string to use a string-literal, to reduce
the amount of escaping needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make this code slightly more idiomatic, and make it clear in what cases
we don't return an actual response, but an empty / default struct.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove redundant `if statusCode == 0 {` check, which would always be true
- use early returns in the switch
- move all conditions into the switch, and scope the `statusCode` variable
to conditions where it's used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
So far, Moby only had IPv4 prefixes in its 'default-address-pools'. To
get dynamic IPv6 subnet allocations, users had to redefine this
parameter to include IPv6 base network(s). This is needlessly complex
and against Moby's 'batteries-included' principle.
This change generates a ULA base network by deriving a ULA Global ID
from the Engine's Host ID and put that base network into
'default-address-pools'. This Host ID is stable over time (except if
users remove their '/var/lib/docker/engine-id') and thus the GID is
stable too.
This ULA base network won't be put into 'default-address-pools' if users
have manually configured it.
This is loosely based on https://datatracker.ietf.org/doc/html/rfc4193#section-3.2.2.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Until this commit, the default local address pool was initialized by the
defaultipam driver if none was provided by libnet / the daemon.
Now, defaultipam errors out if none is passed and instead the daemon is
made responsible for initializing it with the default values if the user
don'te set the related config parameter.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Until now it's been possible to set per-interface sysctls using, for
example, '--sysctl net.ipv6.conf.eth0.accept_ra=2'. But, the index in
the interface name is allocated serially, and the numbering in a container
with more than one interface may change when a container is restarted.
The change to make it possible to connect a container to more than one
network when it's created increased the ambiguity.
This change adds label "com.docker.network.endpoint.sysctls" to the
DriverOpts in EndpointSettings. This option is explicitly associated
with the interface.
Settings in "--sysctl" for "eth0" are migrated to DriverOpts.
Because using "--sysctl" with any interface apart from "eth0" would have
unpredictable results, it is now an error to use any other interface name
in the top level "--sysctl" option. The error message includes a hint at
how to use the new per-interface setting.
The per-endpoint sysctl name has the interface name replaced by
"IFNAME". For example:
net.ipv6.conf.eth0.accept_ra=2
becomes:
net.ipv6.conf.IFNAME.accept_ra=2
The value of DriverOpts["com.docker.network.endpoint.sysctls"] is a
comma separated list.
Settings from '--sysctl' are applied by the runtime lib during task
creation. So, task creation fails if the endpoint does not exist.
Applying per-endpoint settings during interface configuration means the
endpoint can be created later, which paves the way for removal of the
SetKey OCI prestart hook.
Unlike other DriverOpts, the sysctl label itself is not driver-specific,
but each driver has a chance to check settings/values and raise an error
if a setting would cause it a problem - no such checks have been added
in this initial version. As a future extension, if required, it would be
possible for the driver to echo back valid/extended/modified settings to
libnetwork for it to apply to the interface. (At that point, the syntax
for the options could become driver specific to allow, for example, a
driver to create more than one interface).
Signed-off-by: Rob Murray <rob.murray@docker.com>
- api: swagger: Network: inline examples, and add ConfigOnly, ConfigFrom
These fields were added in 9ee7b4dda9, but
not documented in the API docs / swagger.
Also move the example values per-field to reduce the risk of the example
given from diverging with the actual struct that's used for the request.
- api: swagger: POST /networks/create: document Scope, ConfigOnly, ConfigFrom
Adds missing documentation for Scope, ConfigOnly, and ConfigFrom. The ConfigOnly
and ConfigFrom fields were added in 9ee7b4dda9,
but not documented in the API docs / swagger.
- api: swagger: Network: add Peers
Add documentation for the Peers field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adds missing documentation for Scope, ConfigOnly, and ConfigFrom. The ConfigOnly
and ConfigFrom fields were added in 9ee7b4dda9,
but not documented in the API docs / swagger.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These fields were added in 9ee7b4dda9, but
not documented in the API docs / swagger.
Also move the example values per-field to reduce the risk of the example
given from diverging with the actual struct that's used for the request.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the example values per-field to reduce the risk of the example given
from diverging with the actual struct that's used for the request.
This patch updates older API versions (went back to v1.41).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the example values per-field to reduce the risk of the example given
from diverging with the actual struct that's used for the request.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
GoDoc is mostly copied from NetworkResource, which is the equivalent for
retrieving the information.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We currently depend on the containerd platform-parsing to return typed
errdefs errors; the new containerd platforms module does not return such
errors, and documents that errors returned should not be used as sentinel
errors; c1438e911a/errors.go (L21-L30)
Let's type these errors ourselves, so that we don't depend on the error-types
returned by containerd, and consider that eny platform string that results in
an error is an invalid parameter.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove redundant conversion to strslice.StrSlice
- Use assert.Check where possible to not fail early
- Remove instances of types.ExecStartCheck that used default values
- Minor code-formatting cleanup
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove redundant conversion to strslice.StrSlice
- Use assert.Assert instead of assert.Check to fail early if value is nil
- Minor code-formatting cleanup
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- fix typo in comment
- rename variable that collided with an import
- add log for an unhandled error
- slightly improve error-logs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in ee6959addc to account
for arm (32) requiring a longer timeout at the time, but it was never
used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- rename the client var to not collide with the imported client package
- remove an intermediate startCheck variable
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This var for the incoming attach request.
Just within this one function we also have `cfg`, and `ctr` already, so
`c` just makes things more confusing.
Not to mention `c` is usually referencing a container object in other
parts of the code.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In cases where the client disconnects and there is nothing to read from
a stdio stream after that disconnect, the copy goroutines and file
descriptors are leaked because `io.Copy` is just blocked waiting for
data from the container's I/O stream.
This fix only applies to Linux.
Windows will need a separate fix.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Before this change a call to `Close` could be blocked if the the channel
used to buffer logs is full.
When this happens the container state will end up wedged causing a
deadlock on anything that needs to lock the container state.
This removes the use of a channel which has semantics which are
difficult to manage to something more suitable for the situation.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Nothing was validating whether address pools' `base` prefix
were larger than the target subnet `size` they're associated to. As
such invalid address pools would yield no subnet, the error could go
unnoticed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The previous allocator was subnetting address pools eagerly
when the daemon started, and would then just iterate over that
list whenever RequestPool was called. This was leading to high
memory usage whenever IPv6 pools were configured with a target
subnet size too different from the pools prefix size.
For instance: pool = fd00::/8, target size = /64 -- 2 ^ (64-8)
subnets would be generated upfront. This would take approx.
9 * 10^18 bits -- way too much for any human computer in 2024.
Another noteworthy issue, the previous implementation was allocating
a subnet, and then in another layer was checking whether the
allocation was conflicting with some 'reserved networks'. If so,
the allocation would be retried, etc... To make it worse, 'reserved
networks' would be recomputed on every iteration. This is totally
ineffective as there could be 'reserved networks' that fully overlap
a given address pool (or many!).
To fix this issue, a new field `Exclude` is added to `RequestPool`.
It's up to each driver to take it into account. Since we don't know
whether this retry loop is useful for some remote IPAM driver, it's
reimplemented bug-for-bug directly in the remote driver.
The new allocator uses a linear-search algorithm. It takes advantage
of all lists (predefined pools, allocated subnets and reserved
networks) being sorted and logically combines 'allocated' and
'reserved' through a 'double cursor' to iterate on both lists at the
same time while preserving the total order. At the same time, it
iterates over 'predefined' pools and looks for the first empty space
that would be a good fit.
Currently, the size of the allocated subnet is still dictated by
each 'predefined' pools. We should consider hardcoding that size
instead, and let users specify what subnet size they want. This
wasn't possible before as the subnets were generated upfront. This
new allocator should be able to deal with this easily.
The method used for static allocation has been updated to make sure
the ascending order of 'allocated' is preserved. It's bug-for-bug
compatible with the previous implementation.
One consequence of this new algorithm is that we don't keep track
of where the last allocation happened, we just allocate the first
free subnet we find.
Before:
- Allocate: 10.0.1.0/24, 10.0.2.0/24 ; Deallocate: 10.0.1.0/24 ;
Allocate 10.0.3.0/24.
Now, the 3rd allocation would yield 10.0.1.0/24 once again.
As it doesn't change the semantics of the allocator, there's no
reason to worry about that.
Finally, about 'reserved networks'. The heuristics we use are
now properly documented. It was discovered that we don't check
routes for IPv6 allocations -- this can't be changed because
there's no such thing as on-link routes for IPv6.
(Kudos to Rob Murray for coming up with the linear-search idea.)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
If a node is promoted right after another node is demoted, there exists
the possibility of a race, by which the newly promoted manager attempts
to connect to the newly demoted manager for its initial Raft membership.
This connection fails, and the whole swarm Node object exits.
At this point, the daemon nodeRunner sees the exit and restarts the
Node.
However, if the address of the no-longer-manager is recorded in the
nodeRunner's config.joinAddr, the Node again attempts to connect to the
no-longer-manager, and crashes again. This repeats. The solution is to
remove the node entirely and rejoin the Swarm as a new node.
This change erases config.joinAddr from the restart of the nodeRunner,
if the node has previously become Ready. The node becoming Ready
indicates that at some point, it did successfully join the cluster, in
some fashion. If it has successfully joined the cluster, then Swarm has
its own persistent record of known manager addresses. If no joinAddr is
provided, then Swarm will choose from its persisted list of managers to
join, and will join a functioning manager.
Signed-off-by: Drew Erny <derny@mirantis.com>
InvalidParameterErrorf was used whenever an invalid value was found
during PoolID unmarshaling. This error is converted to a 400 HTTP code
by the HTTP server.
However, users never provide PoolIDs directly -- these are constructed
from user-supplied values which are already validated when the PoolID is
marshaled. Hence, if such erroneous value is found, it's an internal
error and should be converted to a 500.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Prior to this change PoolID microformat was using slashes to separate
fields. Those fields include subnet prefixes in CIDR notation, which
also include a slash. This makes future evolution harder than it should
be.
This change introduces a 'v2' microformat based on JSON. This has two
advantages:
1. Fields are clearly named to ensure each value is associated to the
right field.
2. Field values and separators are clearly distinguished to remove any
ambiguity.
The 'v1' encoding will be kept until the next major MCR LTS is released.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This method does nothing more than `return c.store`. It has no value and
adds an unecessary level of indirection. Let's ditch it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Previous commit made it clear that c.store can't be nil. Hence,
`c.store.Close()` can be called without checking if c.store is nil.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This was done in a separate method, called by the ctrler constructor.
This method was returning a nil datastore when c.cfg was nil -- but that
can't happen in practice!
This was giving the impression that the controller could be run without
a datastore properly configured. It's not the case, so make it explicit
by instantiating the datastore before `Controller`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This works around issues with the otel http handler wrapper causing
multiple calls to `WriteHeader` when a `Flush` is called before `Write`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When only stdin is attached the goroutine can only ever exit if:
1. The container pipe is closed while trying to write to it
2. The client closes the stdin read pipe
This is because `io.Copy` does a read on the read side then a write to
the write side.
If reading from the client's stdin pipe blocks, the goroutine will never
get notified that the container pipe is closed.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Pass `context.Context` through `tarexport.Load` and `tarexport.Save`.
Create OTEL spans for the most time consuming operations.
Also, handle context cancellations to actually end saving/loading when
the operation is cancelled - before this PR the daemon would still be
performing the operation even though the user already cancelled it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
In 26.1, we added daemon feature flag "windows-dns-proxy" which could
be set to "true" to make "nslookup" work in Windows containers, by
forwarding requests from the internal resolver to the container's
external DNS servers.
This changes the default to forwarding-enabled - it can be disabled by
via daemon.json using ...
"features": { "windows-dns-proxy": false }
Signed-off-by: Rob Murray <rob.murray@docker.com>
A recent change to prevent containers only connected to --internal
networks from communicating with external DNS servers inadvertently
prevented the daemon's internal DNS server from forwarding requests
within an internal network to a containerised DNS server.
Relax the check, so that only requests that need to be forwarded
from the host's network namespace are dropped.
External DNS servers remain unreachable from the internal network.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Libnet's method `(*Network).createEndpoint()` is already parsing this
netlabel to set the field `ep.iface.mac`. Later on, this same method
invoke the driver's method `CreateEndpoint` with an `InterfaceInfo` arg
and an `options` arg (an opaque map of driver otps).
The `InterfaceInfo` interface contains a `MacAddress()` method that
returns `ep.iface.mac`. And the opaque map may contain the key
`netlabel.MacAddress`.
Prior to this change, the bridge driver was calling `MacAddress()`. If
no value was returned, it'd fall back to the option set in the `options`
map, or generate a MAC address based on the IP address.
However, the expected type of the `options` value is a `net.HardwareAddr`.
This is what's set by the daemon when handing over the endpoint config
to libnet controller. If the value is a string, as is the case if the
MAC address is provided through `EndpointsSettings.DriverOpts`, it
produces an error.
As such, the opaque option and the `MacAddress()` are necessarily the
same -- either nothing or a `net.HardwareAddr`. No need to keep both.
Moreover, the struct `endpointConfiguration` was only used to store that
netlabel value. Drop it too.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
In setupIPv6BridgeNetFiltering(), the bridge should always be named.
Don't fall back to checking the "default" setting for a new bridge.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Check forwarding, then set bridge-nf-call-ip6tables, on a bridge
if IPv6 is enabled - even if no IPv6 address has been assigned.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The code to enable "bridge-nf-call-iptables" or "bridge-nf-call-ip6tables"
was gated on "--iptables=true", it didn't check "--ip6tables=true".
So, split the top level call into IPv4/IPv6 so that the iptables-enable
settings can be checked independently, and simplfied the implementation.
Signed-off-by: Rob Murray <rob.murray@docker.com>
bridgeNetwork.isolateNetwork() checks "--iptables=true" and
"--ip6tables=true" before doing anything with IPv4 and IPv6
respectively. But, it was only called if "--iptables=true".
Now, it's called if "--ip6tables=true", even if "--iptables=false".
Signed-off-by: Rob Murray <rob.murray@docker.com>
The bridge driver's setupIPChains() had an initial sanity check that
"--iptables=true".
But, it's called with "version=IPv6" when "--iptables=false" and
"--ip6tables=true" - the sanity test needed to allow for that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Tests that start a daemon disable iptables, to avoid conflicts with
other tests running in parallel and also creating iptables chains.
Do the same for ip6tables, in prep for them being enabled by-default.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Use defer funcs instead.
For no apparant reasons, a few error cases in the Join method were not
triggering a rollback. This is now fixed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Since commit befff0e1, `(*Controller).getStore()` never returns nil
except if `c.store` isn't initialized yet. This can't happen unless
`New()` returned an error and it wasn't proper caught.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
CORS headers were originally added by 6d5bdff.
These headers could be set without any Authz plugin enabled
beforehand, making this feature quite dangerous.
This commit marks the daemon flag `api-cors-header` as deprecated
and requires the env var `DOCKERD_DEPRECATED_CORS_HEADER` to be
set. When enabled, the daemon will write a deprecation warning to
the logs and the endpoint `GET /info` will return the same
deprecation warning.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.9...go1.21.10
These minor releases include 2 security fixes following the security policy:
- cmd/go: arbitrary code execution during build on darwin
On Darwin, building a Go module which contains CGO can trigger arbitrary code execution when using the Apple version of ld, due to usage of the -lto_library flag in a "#cgo LDFLAGS" directive.
Thanks to Juho Forsén of Mattermost for reporting this issue.
This is CVE-2024-24787 and Go issue https://go.dev/issue/67119.
- net: malformed DNS message can cause infinite loop
A malformed DNS message in response to a query can cause the Lookup functions to get stuck in an infinite loop.
Thanks to long-name-let-people-remember-you on GitHub for reporting this issue, and to Mateusz Poliwczak for bringing the issue to our attention.
This is CVE-2024-24788 and Go issue https://go.dev/issue/66754.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.3
**- Description for the changelog**
```markdown changelog
Update Go runtime to 1.21.10
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This was changed recently so that the bundles target is always run, but
`mkdir bundles` fails when bundles exists...
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
It'd only return true on a host with no IPv6 in its kernel.
So, removed, having fixed the two tests that used it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.
TestDaemonIPv6Enabled tried to run with IPv6 on the default bridge,
but didn't set up a "fixed-cidr-v6" - so the daemon wouldn't start.
It then tried to check the bridge had address "fe80::1", which it
expected to work because it had just used setupV6() to add that
address.
Then it checked that "LinkLocalIPv6Address" was set in container
inspect output, but it wouldn't be (the field is deprecated).
There are working IPv6 tests in the suite (TestDaemonIPv6FixedCIDR,
TestDaemonIPv6FixedCIDRAndMac, TestDaemonIPv6HostMode) - and there's
more coverage in the network integration tests.
So, deleted the test as it didn't seem worth salvaging.
Also deleted now-unused helper functions setupV6(), teardownV6().
Signed-off-by: Rob Murray <rob.murray@docker.com>
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.
If the test had run, it'd have failed because:
- it used "--listen-add", but the option is "--listen-addr"
- so, the daemon wouldn't have started
- it tried to use "--join ::1"
- address "::1" was interpreted as host:port so the Dial() failed,
it needed to be "[::1]".
- it didn't supply a join token
Signed-off-by: Rob Murray <rob.murray@docker.com>
Multicast addresses aren't added by the daemon so, if they're present,
it's because they were explicitly added - possibly to a user-managed
bridge. So, don't remove.
Signed-off-by: Rob Murray <rob.murray@docker.com>
- Remove package variable bridge.bridgeIPv6
- Use netip in more places
- Improve error messages from fixed-cidr-v6 checks
Signed-off-by: Rob Murray <rob.murray@docker.com>
Make the behaviour enabled by env var DOCKER_BRIDGE_PRESERVE_KERNEL_LL
the default...
- don't remove kernel assigned link-local addresses
- or any address in fe80::/64
- don't assign fe80::1 to a bridge
Signed-off-by: Rob Murray <rob.murray@docker.com>
If dockerd runs on a host with a read-only /proc/sys/net filesystem,
it isn't able to enable or disable IPv6 on network interfaces when
attaching a container to a network (including initial networks during
container creation).
In release 26.0.2, a read-only /proc/sys/net meant container creation
failed in all cases.
So, don't attempt to enable/disable IPv6 on an interface if it's already
set appropriately.
If it's not possible to enable IPv6 when it's needed, just log (because
that's what libnetwork has always done if IPv6 is disabled in the
kernel).
If it's not possible to disable IPv6 when it needs to be disabled,
refuse to create the container and raise an error that suggests setting
environment variable "DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE=1", to tell
the daemon it's ok to ignore the problem.
Signed-off-by: Rob Murray <rob.murray@docker.com>
If env var DOCKER_BRIDGE_PRESERVE_KERNEL_LL=1, don't assign fe80::1/64
to a bridge, and don't delete any link local address with prefix fe80::/64.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Prior to this change, cnmallocator would call
`ConfigGlobalScopeDefaultNetworks` right before initializing its
IPAM drivers. This function was mutating some global state used
during drivers init.
This change just remove the global state, and adds an arg to
ipams.Register and defaultipam.Register to pass the global pools
by arguments instead.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
All drivers except the default have a Register function. Before this
change, default's registration was handled by another package. Move
this logic into the driver pkg.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Packages in libnet/ipams are drivers, except builtin -- it's used
to register drivers. Move files one level up and delete this pkg.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
All drivers except the default ipam driver are stored in ipams/.
Since `default` isn't a valid Go pkg name, this package is
renamed to `defaultipam`, following `windowsipam` example.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Prior to this change, daemon's `default-address-pools` param would
be passed to `SetDefaultIPAddressPool()` to set a global var named
`defaultAddressPool`. This var would then be retrieved during the
`default` IPAM driver registration. Both steps were executed in
close succession during libnet's controller initialization.
This change removes the global var and just pass the user-defined
`default-address-pools` to the `default` driver's `Register` fn.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The `RequestPool` method has many args and named returns. This
makes the code hard to follow at times. This commit adds one struct,
`PoolRequest`, to replace these args, and one struct, `AllocatedPool`,
to replace these named returns.
Both structs' fields are properly documented to better define their
semantics, and their relationship with address allocation.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Address spaces are a continuum of addresses that can be used for a
specific purpose (ie. 'local' for unmanaged containers, 'global for
Swarm). v4 and v6 addresses aren't of the same size -- hence
combining them into a single address space doesn't form a continuum.
Better set them apart into two different address spaces.
Also, the upcoming rewrite of `addrSpace` will benefit from that
split.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
`addrSpace` methods are currently scattered in two different files.
As upcoming work will rewrite some of these methods, better put them
into a separate file.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Embedding `sync.Mutex` into a struct is considered a bad practice
as it makes the mutex methods part of the embedding struct's API.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Allow forwarding from any firewalld zone to the 'docker' zone.
This makes it possible to use routable IPv6 addresses on a bridge
network, with masquerading disabled, and have the host forward packets
to it.
Signed-off-by: Rob Murray <rob.murray@docker.com>
/usr/sbin/runc is confined with "runc" profile[1] introduced in AppArmor
v4.0.0. This change breaks stopping of containers, because the profile
assigned to containers doesn't accept signals from the "runc" peer.
AppArmor >= v4.0.0 is currently part of Ubuntu Mantic (23.10) and later.
In the case of Docker, this regression is hidden by the fact that
dockerd itself sends SIGKILL to the running container after runc fails
to stop it. It is still a regression, because graceful shutdowns of
containers via "docker stop" are no longer possible, as SIGTERM from
runc is not delivered to them. This can be seen in logs from dockerd
when run with debug logging enabled and also from tracing signals with
killsnoop utility from bcc[2] (in bpfcc-tools package in Debian/Ubuntu):
Test commands:
root@cloudimg:~# docker run -d --name test redis
ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
root@cloudimg:~# docker stop test
Relevant syslog messages (with wrapped long lines):
Apr 23 20:45:26 cloudimg kernel: audit:
type=1400 audit(1713905126.444:253): apparmor="DENIED"
operation="signal" class="signal" profile="docker-default" pid=9289
comm="runc" requested_mask="receive" denied_mask="receive"
signal=kill peer="runc"
Apr 23 20:45:36 cloudimg dockerd[9030]:
time="2024-04-23T20:45:36.447016467Z"
level=warning msg="Container failed to exit within 10s of kill - trying direct SIGKILL"
container=ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
error="context deadline exceeded"
Killsnoop output after "docker stop ...":
root@cloudimg:~# killsnoop-bpfcc
TIME PID COMM SIG TPID RESULT
20:51:00 9631 runc 3 9581 -13
20:51:02 9637 runc 9 9581 -13
20:51:12 9030 dockerd 9 9581 0
This change extends the docker-default profile with rules that allow
receiving signals from processes that run confined with either runc or
crun profile (crun[4] is an alternative OCI runtime that's also confined
in AppArmor >= v4.0.0, see [1]). It is backward compatible because the
peer value is a regular expression (AARE) so the referenced profile
doesn't have to exist for this profile to successfully compile and load.
Note that the runc profile has an attachment to /usr/sbin/runc. This is
the path where the runc package in Debian/Ubuntu puts the binary. When
the docker-ce package is installed from the upstream repository[3], runc
is installed as part of the containerd.io package at /usr/bin/runc.
Therefore it's still running unconfined and has no issues sending
signals to containers.
[1] https://gitlab.com/apparmor/apparmor/-/commit/2594d936
[2] https://github.com/iovisor/bcc/blob/master/tools/killsnoop.py
[3] https://download.docker.com/linux/ubuntu
[4] https://github.com/containers/crun
Signed-off-by: Tomáš Virtus <nechtom@gmail.com>
If a container is configured with the internal DNS resolver's own
address as an external server, try the next ext server rather than
recursing (return SERVFAIL if there are no other servers).
Signed-off-by: Rob Murray <rob.murray@docker.com>
no changes in vendored codee
full diff: https://github.com/golang/mod/compare/v0.13.0...v0.17.0
- modfile: do not collapse if there are unattached comments within blocks
- modfile: fix crash on AddGoStmt in empty File
- modfile: improve directory path detection and error text consistency
- modfile: use new go version string format in WorkFile.add error
- sumdb: replace globsMatchPath with module.MatchPrefixPatterns
- sumdb/tlog: make NewTiles only generate strictly necessary tiles
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't fail-fast when encountering an image that targets an unexpected
descriptor (neither a manifest nor index). Log a warning instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This error is returned when attempting to walk a descriptor that
*should* be an index or a manifest.
Without this the error is not very helpful sicne there's no way to tell
what triggered it.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Don't mutate the container's `Config.WorkingDir` permanently with a
cleaned path when creating a working directory.
Move the `filepath.Clean` to the `translateWorkingDir` instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `normalizeWorkdir` function has two branches, one that returns a
result of `filepath.Join` which always returns a cleaned path, and
another one where the input string is returned unmodified.
To make these two outputs consistent, also clean the path in the second
branch.
This also makes the cleaning of the container workdir explicit in the
`normalizeWorkdir` function instead of relying on the
`SetupWorkingDirectory` to mutate it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Make the internal DNS resolver for Windows containers forward requests
to upsteam DNS servers when it cannot respond itself, rather than
returning SERVFAIL.
Windows containers are normally configured with the internal resolver
first for service discovery (container name lookup), then external
resolvers from '--dns' or the host's networking configuration.
When a tool like ping gets a SERVFAIL from the internal resolver, it
tries the other nameservers. But, nslookup does not, and with this
change it does not need to.
The internal resolver learns external server addresses from the
container's HNSEndpoint configuration, so it will use the same DNS
servers as processes in the container.
The internal resolver for Windows containers listens on the network's
gateway address, and each container may have a different set of external
DNS servers. So, the resolver uses the source address of the DNS request
to select external resolvers.
On Windows, daemon.json feature option 'windows-no-dns-proxy' can be used
to prevent the internal resolver from forwarding requests (restoring the
old behaviour).
Signed-off-by: Rob Murray <rob.murray@docker.com>
- deprecate Prestart hook
- deprecate kernel memory limits
Additions
- config: add idmap and ridmap mount options
- config.md: allow empty mappings for [r]idmap
- features-linux: Expose idmap information
- mount: Allow relative mount destinations on Linux
- features: add potentiallyUnsafeConfigAnnotations
- config: add support for org.opencontainers.image annotations
Minor fixes:
- config: improve bind mount and propagation doc
full diff: https://github.com/opencontainers/runtime-spec/compare/v1.1.0...v1.2.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds some nolint-comments for the deprecated kernel-memory options; we
deprecated these, but they could technically still be accepted by alternative
runtimes.
daemon/daemon_unix.go:108:3: SA1019: memory.Kernel is deprecated: kernel-memory limits are not supported in cgroups v2, and were obsoleted in [kernel v5.4]. This field should no longer be used, as it may be ignored by runtimes. (staticcheck)
memory.Kernel = &config.KernelMemory
^
daemon/update_linux.go:63:3: SA1019: memory.Kernel is deprecated: kernel-memory limits are not supported in cgroups v2, and were obsoleted in [kernel v5.4]. This field should no longer be used, as it may be ignored by runtimes. (staticcheck)
memory.Kernel = &resources.KernelMemory
^
Prestart hooks are deprecated, and more granular hooks should be used instead.
CreateRuntime are the closest equivalent, and executed in the same locations
as Prestart-hooks, but depending on what these hooks do, possibly one of the
other hooks could be used instead (such as CreateContainer or StartContainer).
As these hooks are still supported, this patch adds nolint comments, but adds
some TODOs to consider migrating to something else;
daemon/nvidia_linux.go:86:2: SA1019: s.Hooks.Prestart is deprecated: use [Hooks.CreateRuntime], [Hooks.CreateContainer], and [Hooks.StartContainer] instead, which allow more granular hook control during the create and start phase. (staticcheck)
s.Hooks.Prestart = append(s.Hooks.Prestart, specs.Hook{
^
daemon/oci_linux.go:76:5: SA1019: s.Hooks.Prestart is deprecated: use [Hooks.CreateRuntime], [Hooks.CreateContainer], and [Hooks.StartContainer] instead, which allow more granular hook control during the create and start phase. (staticcheck)
s.Hooks.Prestart = append(s.Hooks.Prestart, specs.Hook{
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
No IPAM IPv6 address is given to an interface in a network with
'--ipv6=false', but the kernel would assign a link-local address and,
in a macvlan/ipvlan network, the interface may get a SLAAC-assigned
address.
So, disable IPv6 on the interface to avoid that.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This reverts commit a77e147d32.
The ipvlan integration tests have been skipped in CI because of a check
intended to ensure the kernel has ipvlan support - which failed, but
seems to be unnecessary (probably because kernels have moved on).
Signed-off-by: Rob Murray <rob.murray@docker.com>
We document that an macvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an macvlan
network was still configured with a gateway. So, DNS proxying would
be enabled in the internal resolver (and, if the host's resolver
was on a localhost address, requests to external resolvers from the
host's network namespace would succeed).
This change disables configuration of a gateway for a macvlan Endpoint
if no parent interface is specified.
(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway will still be set up. Documentation
will need to be updated to note that '--internal' should be used to
prevent DNS request forwarding in this case.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
The internal DNS resolver should only forward requests to external
resolvers if the libnetwork.Sandbox served by the resolver has external
network access (so, no forwarding for '--internal' networks).
The test for external network access was whether the Sandbox had an
Endpoint with a gateway configured.
However, an ipvlan-l3 networks with external network access does not
have a gateway, it has a default route bound to an interface.
Also, we document that an ipvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an ipvlan-l2
network was configured with a gateway. So, DNS proxying would be enabled
in the internal resolver (and, if the host's resolver was on a localhost
address, requests to external resolvers from the host's network
namespace would succeed).
So, this change adjusts the test for enabling DNS proxying to include
a check for '--internal' (as a shortcut) and, for non-internal networks,
checks for a default route as well as a gateway. It also disables
configuration of a gateway or a default route for an ipvlan Endpoint if
no parent interface is specified.
(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway/default route will still be set up
and external DNS proxying will be enabled. The network must be
configured as '--internal' to prevent that from happening.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.
These minor releases include 1 security fixes following the security policy:
- http2: close connections when receiving too many headers
Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.
Set a limit on the amount of excess header frames we will process before
closing a connection.
Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.
This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
full diff: https://github.com/golang/net/compare/v0.22.0...v0.23.0
Includes a fix for CVE-2023-45288, which is also addressed in go1.22.2
and go1.21.9;
> http2: close connections when receiving too many headers
>
> Maintaining HPACK state requires that we parse and process
> all HEADERS and CONTINUATION frames on a connection.
> When a request's headers exceed MaxHeaderBytes, we don't
> allocate memory to store the excess headers but we do
> parse them. This permits an attacker to cause an HTTP/2
> endpoint to read arbitrary amounts of data, all associated
> with a request which is going to be rejected.
>
> Set a limit on the amount of excess header frames we
> will process before closing a connection.
>
> Thanks to Bartek Nowotarski for reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diffs changes relevant to vendored code:
- https://github.com/golang/net/compare/v0.18.0...v0.22.0
- websocket: add support for dialing with context
- http2: remove suspicious uint32->v conversion in frame code
- http2: send an error of FLOW_CONTROL_ERROR when exceed the maximum octets
- https://github.com/golang/crypto/compare/v0.17.0...v0.21.0
- internal/poly1305: drop Go 1.12 compatibility
- internal/poly1305: improve sum_ppc64le.s
- ocsp: don't use iota for externally defined constants
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Unlike its stdlib counterparts, AtomicFileWriter does not take into
consideration umask due to its use of chmod. Failure to recognize this
may cause subtle problems like the one described in #47498.
Therefore the documentation has been updated to let users know that
umask is not taken into consideration when using AtomicFileWriter.
Closes#47516.
Signed-off-by: Antonio Aguilar <antonio@zoftko.com>
This was brought up by bmitch that its not expected to have a platform
object in the config descriptor.
Also checked with tianon who agreed, its not _wrong_ but is unexpected
and doesn't neccessarily make sense to have it there.
Also, while technically incorrect, ECR is throwing an error when it sees
this.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This was using `errors.Wrap` when there was no error to wrap, meanwhile
we are supposed to be creating a new error.
Found this while investigating some log corruption issues and
unexpectedly getting a nil reader and a nil error from `getTailReader`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The NetworkMode "default" is now normalized into the value it
aliases ("bridge" on Linux and "nat" on Windows) by the
ContainerCreate endpoint, the legacy image builder, Swarm's
cluster executor and by the container restore codepath.
builder-next is left untouched as it already uses the normalized
value (ie. bridge).
Going forward, this will make maintenance easier as there's one
less NetworkMode to care about.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"
Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.
But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.
This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"
Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.
But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.
This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The `identity.ChainIDs` call was accidentally removed in
b37ced2551.
This broke the shared size calculation for images with more than one
layer that were sharing the same compressed layer.
This was could be reproduced with:
```
$ docker pull docker.io/docker/desktop-kubernetes-coredns:v1.11.1
$ docker pull docker.io/docker/desktop-kubernetes-etcd:3.5.10-0
$ docker system df
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
After a535a65c4b the size reported by the
image list was changed to include all platforms of that image.
This made the "shared size" calculation consider all diff ids of all the
platforms available in the image which caused "snapshot not found"
errors when multiple images were sharing the same layer which wasn't
unpacked.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This is better because every possible platform combination
does not need to be defined in the Dockerfile. If built
for platform where Delve is not supported then it is just
skipped.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v26.0.0 release.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Mark the following tests as flaky:
- TestNetworkDBCRUDTableEntry
- TestNetworkDBCRUDTableEntries
- TestNetworkDBIslands
- TestNetworkDBNodeLeave
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
libnetwork tests tend to be flaky (namely `TestNetworkDBIslands` and
`TestNetworkDBCRUDTableEntries`).
Move execution of tests which name has `TestFlaky` prefix to a separate
gotestsum pass which allows them to be reran 4 times.
On Windows, the libnetwork test execution is not split into a separate
pass.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Benchmark the `Images` implementation (image list) against an image
store with 10, 100 and 1000 random images. Currently the images are
single-platform only.
The images are generated randomly, but a fixed seed is used so the
actual testing data will be the same across different executions.
Because the content store is not a real containerd image store but a
local implementation, a small delay (500us) is added to each content
store method call. This is to simulate a real-world usage where each
containerd client call requires a gRPC call.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit 8921897e3b introduced the uses of `clear()`,
which requires go1.21, but Go is downgrading this file to go1.16 when used in
other projects (due to us not yet being a go module);
0.175 + xx-go build '-gcflags=' -ldflags '-X github.com/moby/buildkit/version.Version=b53a13e -X github.com/moby/buildkit/version.Revision=b53a13e4f5c8d7e82716615e0f23656893df89af -X github.com/moby/buildkit/version.Package=github.com/moby/buildkit -extldflags '"'"'-static'"'" -tags 'osusergo netgo static_build seccomp ' -o /usr/bin/buildkitd ./cmd/buildkitd
181.8 # github.com/docker/docker/libnetwork/internal/resolvconf
181.8 vendor/github.com/docker/docker/libnetwork/internal/resolvconf/resolvconf.go:509:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
52a80b40e2 extracted the `imageSummary`
function but introduced a bug causing the whole caller function to
return if the image should be skipped.
`imageSummary` returns a nil error and nil image when the image doesn't
have any platform or all its platforms are not available locally.
In this case that particular image should be skipped, instead of failing
the whole image list operation.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't run filter function which would only run through the images
reading theirs config without checking any label anyway.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
commit c655b7dc78 added a check to make sure
the TMP_OUT variable was not set to an empty value, as such a situation would
perform an `rm -rf /**` during cleanup.
However, it was a bit too eager, because Makefile conditionals (`ifeq`) are
evaluated when parsing the Makefile, which happens _before_ the make target
is executed.
As a result `$@_TMP_OUT` was always empty when the `ifeq` was evaluated,
making it not possible to execute the `generate-files` target.
This patch changes the check to use a shell command to evaluate if the var
is set to an empty value.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Fix `error mounting "/etc/hosts" to rootfs at "/etc/hosts": mount
/etc/hosts:/etc/hosts (via /proc/self/fd/6), flags: 0x5021: operation
not permitted`.
This error was introduced in 7d08d84b03
(`dockerd-rootless.sh: set rootlesskit --state-dir=DIR`) that changed
the filesystem of the state dir from /tmp to /run (in a typical setup).
Fix issue 47248
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This code is currently only used in the daemon, but is also needed in other
places. We should consider moving this code to github.com/moby/sys, so that
BuildKit can also use the same implementation instead of maintaining a fork;
moving it to internal allows us to reuse this code inside the repository, but
does not allow external consumers to depend on it (which we don't want as
it's not a permanent location).
As our code only uses this in linux files, I did not add a stub for other
platforms (but we may decide to do that in the moby/sys repository).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.
This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.
Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
Adds an experimental `DOCKER_BUILDKIT_RUNC_COMMAND` variable that allows
to specify different runc-compatible binary to be used by the buildkit's
runc executor.
This allows runtimes like sysbox be used for the containers spawned by
buildkit.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
full diffs:
- https://github.com/protocolbuffers/protobuf-go/compare/v1.31.0...v1.33.0
- https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4
From the Go security announcement list;
> Version v1.33.0 of the google.golang.org/protobuf module fixes a bug in
> the google.golang.org/protobuf/encoding/protojson package which could cause
> the Unmarshal function to enter an infinite loop when handling some invalid
> inputs.
>
> This condition could only occur when unmarshaling into a message which contains
> a google.protobuf.Any value, or when the UnmarshalOptions.UnmarshalUnknown
> option is set. Unmarshal now correctly returns an error when handling these
> inputs.
>
> This is CVE-2024-24786.
In a follow-up post;
> A small correction: This vulnerability applies when the UnmarshalOptions.DiscardUnknown
> option is set (as well as when unmarshaling into any message which contains a
> google.protobuf.Any). There is no UnmarshalUnknown option.
>
> In addition, version 1.33.0 of google.golang.org/protobuf inadvertently
> introduced an incompatibility with the older github.com/golang/protobuf
> module. (https://github.com/golang/protobuf/issues/1596) Users of the older
> module should update to github.com/golang/protobuf@v1.5.4.
govulncheck results in our code:
govulncheck ./...
Scanning your code and 1221 packages across 204 dependent modules for known vulnerabilities...
=== Symbol Results ===
Vulnerability #1: GO-2024-2611
Infinite loop in JSON unmarshaling in google.golang.org/protobuf
More info: https://pkg.go.dev/vuln/GO-2024-2611
Module: google.golang.org/protobuf
Found in: google.golang.org/protobuf@v1.31.0
Fixed in: google.golang.org/protobuf@v1.33.0
Example traces found:
#1: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Peek
#2: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Read
#3: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls protojson.Unmarshal
Your code is affected by 1 vulnerability from 1 module.
This scan found no other vulnerabilities in packages you import or modules you
require.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Turn warnings into a deprecation notice and highlight that it will
prevent daemon startup in future releases.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.14
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.14
Welcome to the v1.7.14 release of containerd!
The fourteenth patch release for containerd 1.7 contains various fixes and updates.
Highlights
- Update builds to use go 1.21.8
- Fix various timing issues with docker pusher
- Register imagePullThroughput and count with MiB
- Move high volume event logs to Trace level
Container Runtime Interface (CRI)
- Handle pod transition states gracefully while listing pod stats
Runtime
- Update runc-shim to process exec exits before init
Dependency Changes
- github.com/containerd/nri v0.4.0 -> v0.6.0
- github.com/containerd/ttrpc v1.2.2 -> v1.2.3
- google.golang.org/genproto/googleapis/rpc 782d3b101e98 -> cbb8c96f2d6d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.
This makes sure we don't underflow the refcount (uint64) when
decrementing it.
The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use a separate `devcontainer` Dockerfile target, this allows to include
the `gopls` in the devcontainer so it doesn't have to be installed by
the Go vscode extension.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Make sure the `ping` command used by `TestBridgeICC` actually has
the `-6` flag when it runs IPv6 test cases. Without this flag,
IPv6 connectivity isn't tested properly.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Currently this won't have any real effect because the platform matcher
matches all platform and is only used for sorting.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Move containers counting out of `singlePlatformImage` and count them
based on the `ImageManifest` property.
(also remove ChainIDs calculation as they're no longer used)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Avoid fetching `SnapshotService` from client every time. Fetch it once
and then store when creating the image service.
This also allows to pass custom snapshotter implementation for unit
testing.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use `image.Store` and `content.Store` stored in the ImageService struct
instead of fetching it every time from containerd client.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Both containerd and graphdriver image service use the same code to
create the cache - they only supply their own `cacheAdaptor` struct.
Extract the shared code to `cache.New`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Move image store backend specific code out of the cache code and move it
to a separate interface to allow using the same cache code with
containerd image store.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Rather than error out if the host's resolv.conf has a bad ndots option,
just ignore it. Still validate ndots supplied via '--dns-option' and
treat failure as an error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.
In that case the reference count got mixed up and on
release only one of these layers was actually released.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
When IPv6 is disabled in a container by, for example, using the --sysctl
option - an IPv6 address/gateway is still allocated. Don't attempt to
apply that config because doing so enables IPv6 on the interface.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When configuring the internal DNS resolver - rather than keep IPv6
nameservers read from the host's resolv.conf in the container's
resolv.conf, treat them like IPv4 addresses and use them as upstream
resolvers.
For IPv6 nameservers, if there's a zone identifier in the address or
the container itself doesn't have IPv6 support, mark the upstream
addresses for use in the host's network namespace.
Signed-off-by: Rob Murray <rob.murray@docker.com>
In de2447c, the creation of the 'lower' file was changed from using
os.Create to using ioutils.AtomicWriteFile, which ignores the system's
umask. This means that even though the requested permission in the
source code was always 0666, it was 0644 on systems with default
umask of 0022 prior to de2447c, so the move to AtomicFile potentially
increased the file's permissions.
This is not a security issue because the parent directory does not
allow writes into the file, but it can confuse security scanners on
Linux-based systems into giving false positives.
Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
The field will still be present in the response, but will always be
`false`.
Searching for `is-automated=true` will yield no results, while
`is-automated=false` will effectively be a no-op.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When using devcontainers in VSCode, install the Go extension
automatically in the container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
While github.com/stretchr/testify is not used directly by any of the
repository code, it is a transitive dependency via Swarmkit and
therefore still easy to use without having to revendor. Add lint rules
to ban importing testify packages to make sure nobody does.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Apply command gotest.tools/v3/assert/cmd/gty-migrate-from-testify to the
cnmallocator package to be consistent with the assertion library used
elsewhere in moby.
Signed-off-by: Cory Snider <csnider@mirantis.com>
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.
The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.
So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).
This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This message accidentally changed in ac2a028dcc
because my IDE's "refactor tool" was a bit over-enthusiastic. It also went and
updated the tests accordingly, so CI didn't catch this :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Moby imports Swarmkit; Swarmkit no longer imports Moby. In order to
accomplish this feat, Swarmkit has introduced a new plugin.Getter
interface so it could stop importing our pkg/plugingetter package. This
new interface is not entirely compatible with our
plugingetter.PluginGetter interface, necessitating a thin adapter.
Swarmkit had to jettison the CNM network allocator to stop having to
import libnetwork as the cnmallocator package is deeply tied to
libnetwork. Move the CNM network allocator into libnetwork, where it
belongs. The package had a short an uninteresting Git history in the
Swarmkit repository so no effort was made to retain history.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This patch disables pulling legacy (schema1 and schema 2, version 1) images by
default.
A `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE` environment-variable is
introduced to allow re-enabling this feature, aligning with the environment
variable used in containerd 2.0 (`CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`).
With this patch, attempts to pull a legacy image produces an error:
With graphdrivers:
docker pull docker:1.0
1.0: Pulling from library/docker
[DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
With the containerd image store enabled, output is slightly different
as it returns the error before printing the `1.0: pulling ...`:
docker pull docker:1.0
Error response from daemon: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
Using the "distribution" endpoint to resolve the digest for an image also
produces an error:
curl -v --unix-socket /var/run/docker.sock http://foo/distribution/docker.io/library/docker:1.0/json
* Trying /var/run/docker.sock:0...
* Connected to foo (/var/run/docker.sock) port 80 (#0)
> GET /distribution/docker.io/library/docker:1.0/json HTTP/1.1
> Host: foo
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Api-Version: 1.45
< Content-Type: application/json
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Tue, 27 Feb 2024 16:09:42 GMT
< Content-Length: 354
<
{"message":"[DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"}
* Connection #0 to host foo left intact
Starting the daemon with the `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`
env-var set to a non-empty value allows pulling the image;
docker pull docker:1.0
[DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
b0a0e6710d13: Already exists
d193ad713811: Already exists
ba7268c3149b: Already exists
c862d82a67a2: Already exists
Digest: sha256:5e7081837926c7a40e58881bbebc52044a95a62a2ea52fb240db3fc539212fe5
Status: Image is up to date for docker:1.0
docker.io/library/docker:1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This should allow to enable host loopback by setting
DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK to false,
defaults true.
Signed-off-by: serhii.n <serhii.n@thescimus.com>
Don't use all `*.json` files blindly, take only these that are likely to
be reports from go test.
Also, use `find ... -exec` instead of piping results to `xargs`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
For current implementation of Checkpoint Restore (C/R) in docker, it
will write the checkpoint to content store. However, when restoring
libcontainerd uses .Digest().Encoded(), which will remove the info
of alg, leading to error.
Signed-off-by: huang-jl <1046678590@qq.com>
Buildkit added support for exporting metrics in:
7de2e4fb32
Explicitly set the protocol for exporting metrics like we do for the
traces. We need that because Buildkit defaults to grpc.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
30c069cb03
removed the `ResolveImageConfig` method in favor of more generic
`ResolveSourceMetadata` that can also support other things than images.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
e358792815
changed that field to a function and added an `OverrideResource`
function that allows to override it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
StaticDirSource definition changed and can no longer be initialized from
the composite literal.
a80b48544c
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
All other progress updates are emitted with truncated id.
```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit e6907243af applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.
However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.
However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.
This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.
Before this patch:
docker -H unix:///no/such/socket.sock secret ls
"secret list" requires API version 1.25, but the Docker daemon API version is 1.24
With this patch applied:
docker -H unix:///no/such/socket.sock secret ls
Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).
The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.
Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.
Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was added in 27ef09a46f, which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.
The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.
Running the test also shows that returning a response is unexpected, and
ignored:
=== RUN TestPingWithError
2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
--- PASS: TestPingWithError (0.00s)
PASS
This patch updates the test to remove the response.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't error out when mount source doesn't exist and mounts has
`CreateMountpoint` option enabled.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Any PR that is labeled with any `impact/*` label should have a
description for the changelog and an `area/*` label.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
A common pattern in libnetwork is to delete an object using
`DeleteAtomic`, ie. to check the optimistic lock, but put in a retry
loop to refresh the data and the version index used by the optimistic
lock.
This commit introduces a new `Delete` method to delete without
checking the optimistic lock. It focuses only on the few places where
it's obvious the calling code doesn't rely on the side-effects of the
retry loop (ie. refreshing the object to be deleted).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
I noticed that this log didn't use structured logs;
[resolver] failed to query DNS server: 10.115.11.146:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:46361->10.115.11.146:53: i/o timeout
[resolver] failed to query DNS server: 10.44.139.225:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:53991->10.44.139.225:53: i/o timeout
But other logs did;
DEBU[2024-02-20T15:48:51.026704088Z] [resolver] forwarding query client-addr="udp:172.19.0.2:39661" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t A"
DEBU[2024-02-20T15:48:51.028331088Z] [resolver] forwarding query client-addr="udp:172.19.0.2:35163" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t AAAA"
DEBU[2024-02-20T15:48:51.057329755Z] [resolver] received AAAA record "2a00:1450:400e:801::200e" for "google.com." from udp:192.168.65.7
DEBU[2024-02-20T15:48:51.057666880Z] [resolver] received A record "142.251.36.14" for "google.com." from udp:192.168.65.7
As we're already constructing a logger with these fields, we may as well use it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Allow to override the PAGER/GIT_PAGER variables inside the container.
Use `cat` as pager when running in Github Actions (to avoid things like
`git diff` stalling the CI).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't use OTEL tracing in this test because we're testing the HTTP proxy
behavior here and we don't want OTEL to interfere.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This will return a single entry for each name/value pair, and for now
all the "image specific" metadata (labels, config, size) should be
either "default platform" or "first platform we have locally" (which
then matches the logic for commands like `docker image inspect`, etc)
with everything else (just ID, maybe?) coming from the manifest
list/index.
That leaves room for the longer-term implementation to add new fields to
describe the _other_ images that are part of the manifest list/index.
Co-authored-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
v1.33.0 is also available, but it would also cause
`github.com/aws/aws-sdk-go-v2` change from v1.24.1 to v1.25.0
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
DNS names were only set up for user-defined networks. On Linux, none
of the built-in networks (bridge/host/none) have built-in DNS, so they
don't need DNS names.
But, on Windows, the default network is "nat" and it does need the DNS
names.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This matches the prior behavior before 2a6ff3c24f.
This also updates the Swagger documentation for the current version to note that the field might be the empty string and what that means.
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.
The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Non-swarm networks created before network-creation-time validation
was added in 25.0.0 continued working, because the checks are not
re-run.
But, swarm creates networks when needed (with 'agent=true'), to
ensure they exist on each agent - ignoring the NetworkNameError
that says the network already existed.
By ignoring validation errors on creation of a network with
agent=true, pre-existing swarm networks with IPAM config that would
fail the new checks will continue to work too.
New swarm (overlay) networks are still validated, because they are
initially created with 'agent=false'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This spec is not directly relevant for the image spec, and the Docker
documentation no longer includes the actual specification.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).
This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Replace regex matching/replacement and re-reading of generated files
with a simple parser, and struct to remember and manipulate the file
content.
Annotate the generated file with a header comment saying the file is
generated, but can be modified, and a trailing comment describing how
the file was generated and listing external nameservers.
Always start with the host's resolv.conf file, whether generating config
for host networking, or with/without an internal resolver - rather than
editing a file previously generated for a different use-case.
Resolves an issue where rewrites of the generated file resulted in
default IPv6 nameservers being unnecessarily added to the config.
Signed-off-by: Rob Murray <rob.murray@docker.com>
This const contains the minimum API version that can be supported by the
API server. The daemon is currently configured to use the same version,
but we may increment the _configured_ minimum version when deprecating
old API versions in future.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 08e4e88482 (Docker Engine v25.0.0)
deprecated API version v1.23 and lower, but older API versions could be
enabled through the DOCKER_MIN_API_VERSION environment variable.
This patch removes all support for API versions < v1.24.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.
API v1.23 and older are deprecated, and this patch removes the feature.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 322e2a7d05 changed the format of errors
returned by the API to be in JSON format for API v1.24. Older versions of
the API returned errors in plain-text format.
API v1.23 and older are deprecated, so we can remove support for plain-text
error responses.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This endpoint was deprecated in API v1.20 (Docker Engine v1.8.0) in
commit db9cc91a9e, in favor of the
`PUT /containers/{id}/archive` and `HEAD /containers/{id}/archive`
endpoints, and disabled in API v1.24 (Docker Engine v1.12.0) through
commit 428328908d.
This patch removes the endpoint, and the associated `daemon.ContainerCopy`
method in the backend.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
API v1.21 (Docker Engine v1.9.0) enforces the request to have a JSON
content-type on exec start (see 45dc57f229).
An exception was added in 0b5e628e14 to
make this check conditional (supporting API < 1.21).
API v1.23 and older are deprecated, and this patch removes the feature.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.20 and lower.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The TestInspectAPIContainerResponse mentioned that Windows does not
support API versions before v1.25.
While technically, no stable release existed for Windows with API versions
before that (see f811d5b128), API version
v1.24 was enabled in e4af39aeb3, to have
a consistend fallback version for API version negotiation.
This patch updates the test to reflect that change.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.19 and lower.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
API v1.20 and up produces an error when signalling / killing a non-running
container (see c92377e300). Older API versions
allowed this, and an exception was added in 621e3d8587.
API v1.23 and older are deprecated, so we can remove this handling.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
API versions before 1.19 allowed CpuShares that were greater than the maximum
or less than the minimum supported by the kernel, and relied on the kernel to
do the right thing.
Commit ed39fbeb2a introduced code to adjust the
CPU shares to be within the accepted range when using API version 1.18 or
lower.
API v1.23 and older are deprecated, so we can remove support for this
functionality.
Currently, there's no validation for CPU shares to be within an acceptable
range; a TODO was added to add validation for this option, and to use the
`linuxMinCPUShares` and `linuxMaxCPUShares` consts for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "pull" option was added in API v1.16 (Docker Engine v1.4.0) in commit
054e57a622, which gated the option by API
version.
API v1.23 and older are deprecated, so we can remove the gate.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "rm" option was made the default in API v1.12 (Docker Engine v1.0.0)
in commit b60d647172, and "force-rm" was
added in 667e2bd4ea.
API v1.23 and older are deprecated, so we can remove these gates.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "pause" flag was added in API v1.13 (Docker Engine v1.1.0), and is
enabled by default (see 17d870bed5).
API v1.23 and older are deprecated, so we can remove the version-gate.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Inspect and history used two different ways to find the present images.
This made history fail in some cases where image inspect would work (if
a configuration of a manifest wasn't found in the content store).
With this change we now use the same logic for both inspect and history.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Add this syscall to match the profile in containerd
containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 9f6c532f59
futex: Add sys_futex_wake()
To complement sys_futex_waitv() add sys_futex_wake(). This syscall
implements what was previously known as FUTEX_WAKE_BITSET except it
uses 'unsigned long' for the bitmask and takes FUTEX2 flags.
The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add this syscall to match the profile in containerd
containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cb8c4312af
futex: Add sys_futex_wait()
To complement sys_futex_waitv()/wake(), add sys_futex_wait(). This
syscall implements what was previously known as FUTEX_WAIT_BITSET
except it uses 'unsigned long' for the value and bitmask arguments,
takes timespec and clockid_t arguments for the absolute timeout and
uses FUTEX2 flags.
The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add this syscall to match the profile in containerd
containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 0f4b5f9722
futex: Add sys_futex_requeue()
Finish off the 'simple' futex2 syscall group by adding
sys_futex_requeue(). Unlike sys_futex_{wait,wake}() its arguments are
too numerous to fit into a regular syscall. As such, use struct
futex_waitv to pass the 'source' and 'destination' futexes to the
syscall.
This syscall implements what was previously known as FUTEX_CMP_REQUEUE
and uses {val, uaddr, flags} for source and {uaddr, flags} for
destination.
This design explicitly allows requeueing between different types of
futex by having a different flags word per uaddr.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add this syscall to match the profile in containerd
containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: c35559f94e
x86/shstk: Introduce map_shadow_stack syscall
When operating with shadow stacks enabled, the kernel will automatically
allocate shadow stacks for new threads, however in some cases userspace
will need additional shadow stacks. The main example of this is the
ucontext family of functions, which require userspace allocating and
pivoting to userspace managed stacks.
Unlike most other user memory permissions, shadow stacks need to be
provisioned with special data in order to be useful. They need to be setup
with a restore token so that userspace can pivot to them via the RSTORSSP
instruction. But, the security design of shadow stacks is that they
should not be written to except in limited circumstances. This presents a
problem for userspace, as to how userspace can provision this special
data, without allowing for the shadow stack to be generally writable.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add this syscall to match the profile in containerd
containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 09da082b07
fs: Add fchmodat2()
On the userspace side fchmodat(3) is implemented as a wrapper
function which implements the POSIX-specified interface. This
interface differs from the underlying kernel system call, which does not
have a flags argument. Most implementations require procfs [1][2].
There doesn't appear to be a good userspace workaround for this issue
but the implementation in the kernel is pretty straight-forward.
The new fchmodat2() syscall allows to pass the AT_SYMLINK_NOFOLLOW flag,
unlike existing fchmodat.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add this syscall to match the profile in containerd
containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cf264e1329
NAME
cachestat - query the page cache statistics of a file.
SYNOPSIS
#include <sys/mman.h>
struct cachestat_range {
__u64 off;
__u64 len;
};
struct cachestat {
__u64 nr_cache;
__u64 nr_dirty;
__u64 nr_writeback;
__u64 nr_evicted;
__u64 nr_recently_evicted;
};
int cachestat(unsigned int fd, struct cachestat_range *cstat_range,
struct cachestat *cstat, unsigned int flags);
DESCRIPTION
cachestat() queries the number of cached pages, number of dirty
pages, number of pages marked for writeback, number of evicted
pages, number of recently evicted pages, in the bytes range given by
`off` and `len`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This syscall is gated by CAP_SYS_NICE, matching the profile in containerd.
containerd: a6e52c74fa
libseccomp: d83cb7ac25
kernel: c6018b4b25
mm/mempolicy: add set_mempolicy_home_node syscall
This syscall can be used to set a home node for the MPOL_BIND and
MPOL_PREFERRED_MANY memory policy. Users should use this syscall after
setting up a memory policy for the specified range as shown below.
mbind(p, nr_pages * page_size, MPOL_BIND, new_nodes->maskp,
new_nodes->size + 1, 0);
sys_set_mempolicy_home_node((unsigned long)p, nr_pages * page_size,
home_node, 0);
The syscall allows specifying a home node/preferred node from which
kernel will fulfill memory allocation requests first.
...
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The compatibility depends on whether `hyperv` or `process` container
isolation is used.
This fixes cache not being used when building images based on older
Windows versions on a newer Windows host.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Only print the tag when the received reference has a tag, if
we can't cast the received tag to a `reference.Tagged` then
skip printing the tag as it's likely a digest.
Fixes panic when trying to install a plugin from a reference
with a digest such as
`vieux/sshfs@sha256:1d3c3e42c12138da5ef7873b97f7f32cf99fb6edde75fa4f0bcf9ed277855811`
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Since 964ab7158c, we explicitly set the bridge MTU if it was specified.
Unfortunately, kernel <v4.17 have a check preventing us to manually set
the MTU to anything greater than 1500 if no links is attached to the
bridge, which is how we do things -- create the bridge, set its MTU and
later on, attach veths to it.
Relevant kernel commit: 804b854d37
As we still have to support CentOS/RHEL 7 (and their old v3.10 kernels)
for a few more months, we need to ignore EINVAL if the MTU is > 1500
(but <= 65535).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Commit 4f47013feb introduced a new validation step to make sure no
IPv6 subnet is configured on a network which has EnableIPv6=false.
Commit 5d5eeac310 then removed that validation step and automatically
enabled IPv6 for networks with a v6 subnet. But this specific commit
was reverted in c59e93a67b and now the error introduced by 4f47013feb
is re-introduced.
But it turns out some users expect a network created with an IPv6
subnet and EnableIPv6=false to actually have no IPv6 connectivity.
This restores that behavior.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Previous commit made getDBhandle a one-liner returning a struct
member -- making it useless. Inline it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This parameter was used to tell the boltdb kvstore not to open/close
the underlying boltdb db file before/after each get/put operation.
Since d21d0884ae, we've a single datastore instance shared by all
components that need it. That commit set `PersistConnection=true`.
We can now safely remove this param altogether, and remove all the
code that was opening and closing the db file before and after each
operation -- it's dead code!
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This test is non-representative of what we now do in libnetwork.
Since the ability of opening the same boltdb database multiple
times in parallel will be dropped in the next commit, just remove
this test.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:
```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
The monitorDaemon() goroutine calls startContainerd() then blocks on
<-daemonWaitCh to wait for it to exit. The startContainerd() function
would (re)initialize the daemonWaitCh so a restarted containerd could be
waited on. This implementation was race-free because startContainerd()
would synchronously initialize the daemonWaitCh before returning. When
the call to start the managed containerd process was moved into the
waiter goroutine, the code to initialize the daemonWaitCh struct field
was also moved into the goroutine. This introduced a race condition.
Move the daemonWaitCh initialization to guarantee that it happens before
the startContainerd() call returns.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Containers attached to an 'internal' bridge network are unable to
communicate when the host is running firewalld.
Non-internal bridges are added to a trusted 'docker' firewalld zone, but
internal bridges were not.
DOCKER-ISOLATION iptables rules are still configured for an internal
network, they block traffic to/from addresses outside the network's subnet.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Do not set 'Config.MacAddress' in inspect output unless the MAC address
is configured.
Also, make sure it is filled in for a configured address on the default
network before the container is started (by translating the network name
from 'default' to 'config' so that the address lookup works).
Signed-off-by: Rob Murray <rob.murray@docker.com>
The API's EndpointConfig struct has a MacAddress field that's used for
both the configured address, and the current address (which may be generated).
A configured address must be restored when a container is restarted, but a
generated address must not.
The previous attempt to differentiate between the two, without adding a field
to the API's EndpointConfig that would show up in 'inspect' output, was a
field in the daemon's version of EndpointSettings, MACOperational. It did
not work, MACOperational was set to true when a configured address was
used. So, while it ensured addresses were regenerated, it failed to preserve
a configured address.
So, this change removes that code, and adds DesiredMacAddress to the wrapped
version of EndpointSettings, where it is persisted but does not appear in
'inspect' results. Its value is copied from MacAddress (the API field) when
a container is created.
Signed-off-by: Rob Murray <rob.murray@docker.com>
File paths can contain commas, particularly paths returned from
t.TempDir() in subtests which include commas in their names. There is
only one datastore provider and it only supports a single address, so
the only use of parsing the address is to break tests in mysterious
ways.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The bbolt library wants exclusive access to the boltdb file and uses
file locking to assure that is the case. The controller and each network
driver that needs persistent storage instantiates its own unique
datastore instance, backed by the same boltdb file. The boltdb kvstore
implementation works around multiple access to the same boltdb file by
aggressively closing the boltdb file between each transaction. This is
very inefficient. Have the controller pass its datastore instance into
the drivers and enable the PersistConnection option to disable closing
the boltdb between transactions.
Set data-dir in unit tests which instantiate libnetwork controllers so
they don't hang trying to lock the default boltdb database file.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The double quotes inside a single quoted string don't need to be
escaped.
Looks like different Powershell versions are treating this differently
and it started failing unexpectedly without any changes on our side.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- full diff: https://github.com/actions/setup-go/compare/v3.5.0...v5.0.0
v5
In scope of this release, we change Nodejs runtime from node16 to node20.
Moreover, we update some dependencies to the latest versions.
Besides, this release contains such changes as:
- Fix hosted tool cache usage on windows
- Improve documentation regarding dependencies caching
V4
The V4 edition of the action offers:
- Enabled caching by default
- The action will try to enable caching unless the cache input is explicitly
set to false.
Please see "Caching dependency files and build outputs" for more information:
https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If a reader has caught up to the logger and is waiting for the next
message, it should stop waiting when the logger is closed. Otherwise
the reader will unnecessarily wait the full closedDrainTimeout for no
log messages to arrive.
This case was overlooked when the journald reader was recently
overhauled to be compatible with systemd 255, and the reader tests only
failed when a logical race happened to settle in such a way to exercise
the bugged code path. It was only after implicit flushing on close was
added to the journald test harness that the Follow tests would
repeatably fail due to this bug. (No new regression tests are needed.)
Signed-off-by: Cory Snider <csnider@mirantis.com>
The journald reader test harness injects an artificial asynchronous
delay into the logging pipeline: a logged message won't be written to
the journal until at least 150ms after the Log() call returns. If a test
returns while log messages are still in flight to be written, the logs
may attempt to be written after the TempDir has been cleaned up, leading
to spurious errors.
The logger read tests which interleave writing and reading have to
include explicit synchronization points to work reliably with this delay
in place. On the other hand, tests should not be required to sync the
logger explicitly before returning. Override the Close() method in the
test harness wrapper to wait for in-flight logs to be flushed to disk.
Signed-off-by: Cory Snider <csnider@mirantis.com>
- Check the return value when logging messages
- Log the stream (stdout/stderr) and list of messages that were not read
- Wait until the logger is closed before returning early (panic/fatal)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Writing the systemd-journal-remote command output directly to os.Stdout
and os.Stderr makes it nearly impossible to tell which test case the
output is related to when the tests are not run in verbose mode. Extend
the journald sender fake to redirect output to the test log so they
interleave with the rest of the test output.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The Go race detector was detecting a data race when running the
TestLogRead/Follow/Concurrent test against the journald logging driver.
The race was in the test harness, specifically syncLogger. The waitOn
field would be reassigned each time a log entry is sent to the journal,
which is not concurrency-safe. Make it concurrency-safe using the same
patterns that are used in the log follower implementation to synchronize
with the logger.
Signed-off-by: Cory Snider <csnider@mirantis.com>
When saving an image treat `image@sha256:abcdef...` the same as
`abcdef...`, this makes it:
- Not export the digested tag as the image name
- Not try to export all tags from the image repository
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Saving an image via digested reference, ID or truncated ID doesn't store
the image reference in the archive. This also causes the save code to
not add the image's manifest to the index.json.
This commit explicitly adds the untagged manifests to the index.json if
no tagged manifests were added.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
errDrainDone is a sentinel error which is never supposed to escape the
package. Consequently, it needs to be filtered out of returns all over
the place, adding boilerplate. Forgetting to filter out these errors
would be a logic bug which the compiler would not help us catch. Replace
it with boolean multi-valued returns as they can't be accidentally
ignored or propagated.
Signed-off-by: Cory Snider <csnider@mirantis.com>
While it doesn't really matter if the reader waits for an extra
arbitrary period beyond an arbitrary hardcoded timeout, it's also
trivial and cheap to implement, and nice to have.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The journald reader uses a timer to set an upper bound on how long to
wait for the final log message of a stopped container. However, the
timer channel is only received from in non-blocking select statements!
There isn't enough benefit of using a timer to offset the cost of having
to manage the timer resource. Setting a deadline and comparing the
current time is just as effective, without having to manage the
lifecycle of any runtime resources.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Synthesize a boot ID for journal entries fed into
systemd-journal-remote, as required by systemd 255.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Following logs with a non-negative tail when the container log is empty
is broken on the journald driver when used with systemd 255. Add tests
which cover this edge case to our loggertest suite.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Previously this was done indirectly - the `compare` function didn't
check the `ArgsEscaped`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Restrict cache candidates only to images that were built locally.
This doesn't affect builds using `--cache-from`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Store additional image property which makes it possible to distinguish
if image was built locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This is a follow-up to 2cf230951f, adding
more directives to adjust for some new code added since:
Before this patch:
make -C ./internal/gocompat/
GO111MODULE=off go generate .
GO111MODULE=on go mod tidy
GO111MODULE=on go test -v
# github.com/docker/docker/internal/sliceutil
internal/sliceutil/sliceutil.go:3:12: type parameter requires go1.18 or later (-lang was set to go1.16; check go.mod)
internal/sliceutil/sliceutil.go:3:14: predeclared comparable requires go1.18 or later (-lang was set to go1.16; check go.mod)
internal/sliceutil/sliceutil.go:4:19: invalid map key type T (missing comparable constraint)
# github.com/docker/docker/libnetwork
libnetwork/endpoint.go:252:17: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/daemon
daemon/container_operations.go:682:9: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
daemon/inspect.go:42:18: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
With this patch:
make -C ./internal/gocompat/
GO111MODULE=off go generate .
GO111MODULE=on go mod tidy
GO111MODULE=on go test -v
=== RUN TestModuleCompatibllity
main_test.go:321: all packages have the correct go version specified through //go:build
--- PASS: TestModuleCompatibllity (0.00s)
PASS
ok gocompat 0.031s
make: Leaving directory '/go/src/github.com/docker/docker/internal/gocompat'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Functional programming for the win! Add a utility function to map the
values of a slice, along with a curried variant, to tide us over until
equivalent functionality gets added to the standard library
(https://go.dev/issue/61898)
Signed-off-by: Cory Snider <csnider@mirantis.com>
We need to isolate the images that we are remapping to a userns, we
can't mix them with "normal" images. In the graph driver case this means
we create a new root directory where we store the images and everything
else, in the containerd case we can use a new namespace.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
These types were deprecated in v25.0, and moved to api/types/container;
This patch removes the aliases for;
- api/types.ResizeOptions (deprecated in 95b92b1f97)
- api/types.ContainerAttachOptions (deprecated in 30f09b4a1a)
- api/types.ContainerCommitOptions (deprecated in 9498d897ab)
- api/types.ContainerRemoveOptions (deprecated in 0f77875220)
- api/types.ContainerStartOptions (deprecated in 7bce33eb0f)
- api/types.ContainerListOptions (deprecated in 9670d9364d)
- api/types.ContainerLogsOptions (deprecated in ebef4efb88)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types were deprecated in v25.0, and moved to api/types/swarm;
This patch removes the aliases for;
- api/types.ServiceUpdateResponse (deprecated in 5b3e6555a3)
- api/types.ServiceCreateResponse (deprecated in ec69501e94)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types were deprecated in 48cacbca24
(v25.0), and moved to api/types/image.
This patch removes the aliases for;
- api/types.ImageDeleteResponseItem
- api/types.ImageSummary
- api/types.ImageMetadata
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types were deprecated in b688af2226
(v25.0), and moved to api/types/checkpoint.
This patch removes the aliases for;
- api/types.CheckpointCreateOptions
- api/types.CheckpointListOptions
- api/types.CheckpointDeleteOptions
- api/types.Checkpoint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These types were deprecated in c90229ed9a
(v25.0), and moved to api/types/system.
This patch removes the aliases for;
- api/types.Info
- api/types.Commit
- api/types.PluginsInfo
- api/types.NetworkAddressPool
- api/types.Runtime
- api/types.SecurityOpt
- api/types.KeyValue
- api/types.DecodeSecurityOptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.
This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.
As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).
In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
I made a mistake in the last commit - after resolving the IP from the
passed `addr` for CIFS it would still resolve the `device` part.
Apply only one name resolution
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.
This commit reintroduces the old logic.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
- pass the cluster as an argument instead of manually setting it after
creating the router-options
- remove the "opts" variable, to prevent it accidentally being used (with
the assumption that's the value returned)
- use a struct-literal for the returned options.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 21e50b89c9 added a label on the buildkit
worker to advertise the host-gateway-ip. This option can be either set by the
user in the daemon config, or otherwise defaults to the gateway-ip.
If no value is set by the user, discovery of the gateway-ip happens when
initializing the network-controller (`NewDaemon`, `daemon.restore()`).
However d222bf097c changed how we handle the
daemon config. As a result, the `cli.Config` used when initializing the
builder only holds configuration information form the daemon config
(user-specified or defaults), but is not updated with information set
by `NewDaemon`.
This patch adds an accessor on the daemon to get the current daemon config.
An alternative could be to return the config by `NewDaemon` (which should
likely be a _copy_ of the config).
Before this patch:
docker buildx inspect default
Name: default
Driver: docker
Nodes:
Name: default
Endpoint: default
Status: running
Buildkit: v0.12.4+3b6880d2a00f
Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Labels:
org.mobyproject.buildkit.worker.moby.host-gateway-ip: <nil>
After this patch:
docker buildx inspect default
Name: default
Driver: docker
Nodes:
Name: default
Endpoint: default
Status: running
Buildkit: v0.12.4+3b6880d2a00f
Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Labels:
org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.18.0.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 8ae94cafa5 added a DNS resolution
of the `device` part of the volume option.
The previous way to resolve the passed hostname was to use `addr`
option, which was handled by the same code path as the `nfs` mount type.
The issue is that `addr` is also an SMB module option handled by kernel
and passing a hostname as `addr` produces an invalid argument error.
To fix that, restore the old behavior to handle `addr` the same way as
before, and only perform the new DNS resolution of `device` if there is
no `addr` passed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also fixing a "defer in loop" warning, instead changing to use
sub-tests, and simplifying some code, using os.WriteFile() instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The `GetImageOpts` struct is used for options to be passed to the backend,
and are not used in client code. This struct currently is intended for internal
use only.
This patch moves the `GetImageOpts` struct to the backend package to prevent
it being imported in the client, and to make it more clear that this is part
of internal APIs, and not public-facing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The MAC address of a running container was stored in the same place as
the configured address for a container.
When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.
So, remember whether the MAC address was explicitly configured, and
clear it if not.
Signed-off-by: Rob Murray <rob.murray@docker.com>
With containerd snapshotters enabled `docker run` currently fails when
creating a container from an image that doesn't have the default host
platform without an explicit `--platform` selection:
```
$ docker run image:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```
This is confusing and the graphdriver behavior is much better here,
because it runs whatever platform the image has, but prints a warning:
```
$ docker run image:amd64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
```
This commits changes the containerd snapshotter behavior to be the same
as the graphdriver. This doesn't affect container creation when platform
is specified explicitly.
```
$ docker run --rm --platform linux/arm64 asdf:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Since v25.0 (commit ff50388), we validate endpoint settings when
containers are created, instead of doing so when containers are started.
However, a container created prior to that release would still trigger
validation error at start-time. In such case, the API returns a 500
status code because the Go error isn't wrapped into an InvalidParameter
error. This is now fixed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.
Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.
Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was added in 75f6929b44, but pinned
to the API version that was current at the time (v1.20), which is now
deprecated.
Update the test to use the current API version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- add some asserts for unhandled errors
- use consts for fixed values, and slightly re-format Dockerfile contentt
- inline one-line Dockerfiles
- fix some vars to be properly camel-cased
- improve assert for error-types;
Before:
=== RUN TestBuildPlatformInvalid
build_test.go:685: assertion failed: expression is false: errdefs.IsInvalidParameter(err)
--- FAIL: TestBuildPlatformInvalid (0.01s)
FAIL
After:
=== RUN TestBuildPlatformInvalid
build_test.go:689: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter
--- FAIL: TestBuildPlatformInvalid (0.01s)
FAIL
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This matcher was only used internally in the containerd implementation of
the image store. Un-export it, and make it a local utility in that package
to prevent external use.
This package was introduced in 1616a09b61
(v24.0), and there are no known external consumers of this package, so there
should be no need to deprecate / alias the old location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When resolving names in swarm mode, services with exposed ports are
connected to user overlay network, ingress network, and local (docker_gwbridge)
networks. Name resolution should prioritize returning the VIP/IPs on user
overlay network over ingress and local networks.
Sandbox.ResolveName implemented this by taking the list of endpoints,
splitting the list into 3 separate lists based on the type of network
that the endpoint was attached to (dynamic, ingress, local), and then
creating a new list, applying the networks in that order.
This patch refactors that logic to use a custom sorter (sort.Interface),
which makes the code more transparent, and prevents iterating over the
list of endpoints multiple times.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Permit container network attachments to set any static IP address within
the network's IPAM master pool, including when a subpool is configured.
Users have come to depend on being able to statically assign container
IP addresses which are guaranteed not to collide with automatically-
assigned container addresses.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This package was introduced in af59752712
as a utility package for devicemapper, which was removed in commit
dc11d2a2d8 (v25.0.0), and the package
was deprecated in bf692d47fb.
This patch removes the package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This flag was marked deprecated in commit 5a922dc16 (released in v24.0)
and to be removed in the next release.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Some configuration in a container depends on whether it has support for
IPv6 (including default entries for '::1' etc in '/etc/hosts').
Before this change, the container's support for IPv6 was determined by
whether it was connected to any IPv6-enabled networks. But, that can
change over time, it isn't a property of the container itself.
So, instead, detect IPv6 support by looking for '::1' on the container's
loopback interface. It will not be present if the kernel does not have
IPv6 support, or the user has disabled it in new namespaces by other
means.
Once IPv6 support has been determined for the container, its '/etc/hosts'
is re-generated accordingly.
The daemon no longer disables IPv6 on all interfaces during initialisation.
It now disables IPv6 only for interfaces that have not been assigned an
IPv6 address. (But, even if IPv6 is disabled for the container using the
sysctl 'net.ipv6.conf.all.disable_ipv6=1', interfaces connected to IPv6
networks still get IPv6 addresses that appear in the internal DNS. There's
more to-do!)
Signed-off-by: Rob Murray <rob.murray@docker.com>
All components of the path are locked before the check, and
released once the path is already mounted.
This makes it impossible to replace the mounted directory until it's
actually mounted in the container.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
All subpath components are opened with openat, relative to the base
volume directory and checked against the volume escape.
The final file descriptor is mounted from the /proc/self/fd/<fd> to a
temporary mount point owned by the daemon and then passed to the
underlying container runtime.
Temporary mountpoint is removed after the container is started.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.
Symlinks are supported, but they cannot escape the base volume
directory.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
We constructed a "function level" logger, which was used once "as-is", but
also added additional Fields in a loop (for each resource), effectively
overwriting the previous one for each iteration. Adding additional
fields can result in some overhead, so let's construct a "logger" only for
inside the loop.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We have many "image" packages, so these vars easily conflict/shadow
imports. Let's rename them (and in some cases use a const) to
prevent that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For some time, when adding an interface with no IPv6 address (an
interface to a network that does not have IPv6 enabled), we've been
disabling IPv6 on that interface.
As part of a separate change, I'm removing that logic - there's nothing
wrong with having IPv6 enabled on an interface with no routable address.
The difference is that the kernel will assign a link-local address.
TestAddRemoveInterface does this...
- Assign an IPv6 link-local address to one end of a veth interface, and
add it to a namespace.
- Add a bridge with no assigned IPv6 address to the namespace.
- Remove the veth interface from the namespace.
- Put the veth interface back into the namespace, still with an
explicitly assigned IPv6 link local address.
When IPv6 is disabled on the bridge interface, the test passes.
But, when IPv6 is enabled, the bridge gets a kernel assigned link-local
address.
Then, when re-adding the veth interface, the test generates an error in
'osl/interface_linux.go:checkRouteConflict()'. The conflict is between
the explicitly assigned fe80::2 on the veth, and a route for fe80::/64
belonging to the bridge.
So, in preparation for not-disabling IPv6 on these interfaces, use a
unique-local address in the test instead of link-local.
I don't think that changes the intent of the test.
With the change to not-always disable IPv6, it is possible to repro the
problem with a real container, disconnect and re-connect a user-defined
network with '--subnet fe80::/64' while the container's connected to an
IPv4 network. So, strictly speaking, that will be a regression.
But, it's also possible to repro the problem in master, by disconnecting
and re-connecting the fe80::/64 network while another IPv6 network is
connected. So, I don't think it's a problem we need to address, perhaps
other than by prohibiting '--subnet fe80::/64'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Message is different with containerd backend. The Linux test
`TestPullLinuxImageFailsOnLinux` was adjusted before, but we missed this
one.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
All commonly used filesystems should use ref-counted mounter, so make it
the default instead of having to whitelist them.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Prior to this commit, a container running with `--net=host` had
`{"type":"network","path":"/var/run/docker/netns/default"}` in
the ``.linux.namespaces` field of the OCI Runtime Config,
but this wasn't needed.
Close issue 47100
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
The actual divergence is due to differences in the snapshotter and
graphfilter mount behaviour on Windows, but the snapshotter behaviour is
better, so we deal with it here rather than changing the snapshotter
behaviour.
We're relying on the internals of containerd's Windows mount
implementation here. Unless this code flow is replaced, future work is
to move getBackingDeviceForContainerdMount into containerd's mount
implementation.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
That means 'null', not that we can call builder-next on Windows. If and
when we do get builder-next going, this will need to be solved properly
in some way.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The existing API ImageService.GetLayerFolders didn't have access to the
ID of the container, and once we have that, the snapshotter Mounts API
provides all the information we need here.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Needed for Diff on Windows. Don't remount it afterwards as the layer is
going to be released anyway.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This is consistent with layerStore's CreateRWLayer behaviour.
Potentially this can be refactored to avoid creating the -init layer,
but as noted in layerStore's initMount, this name may be special, and
should be cleared-out all-at-once.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This change adds a TempDir function that ensures the correct permissions for
the fake-root user in rootless mode.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
XDG_RUNTIME_DIR will contain sockets so its path mustn't be too long.
Prior to this commit, it was set to very long path like
`/go/src/github.com/docker/docker/bundles/test-integration/TestDiskUsage/de4fb36576d7d/xdgrun`
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Consider only images that were built `FROM scratch` as valid candidates
for the `FROM scratch` + INSTRUCTION build step.
The images are marked as `FROM scratch` based by the classic builder
with a special label. It must be a new label instead of empty parent
label, because empty label values are not persisted.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
In order for the cache in the classic builder to work we need to:
- use the came comparison function as the graph drivers implementation
- save the container config when commiting the image
- use all images to search a 'FROM "scratch"' image
- load all images if `cacheFrom` is empty
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Protecting the environment relies on the shared state (containers,
images, etc) which might already be mutated by other tests if the test
opted in into the Parallel execution before Protect was called.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The health status and probe log of containers are not mission-criticial
data which must survive a crash. It is not worth prematrely wearing out
consumer-grade flash storage by overwriting and fsync()ing the container
config on after every probe. Update only the live Container object and
the ViewDB replica on every container health probe instead. It will
eventually get checkpointed along with some other state (or config)
change. Running containers will not be checkpointed on daemon shutdown
when live-restore is enabled, but it does not matter: the health status
and probe log will be zeroed out when the daemon starts back up.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The "builtin" port driver was marked as "Slow" in the row for the lxc-user-nic
network driver, while it was marked as "Fast" in other rows.
It had to be consistently marked as "Fast" regardless to the network driver.
It is still not as fast as rootful.
Follow-up to PR 47076
Fixes: b5a5ecf4a3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
setupTest should be called before Parallel as it modifies the test
environment which might produce:
```
fatal error: concurrent map writes
goroutine 143 [running]:
github.com/docker/docker/testutil/environment.(*Execution).ProtectContainer(...)
/go/src/github.com/docker/docker/testutil/environment/protect.go:59
github.com/docker/docker/testutil/environment.ProtectContainers({0x12e8d98, 0xc00040e420}, {0x12f2878?, 0xc0004fc340}, 0xc0001fac00)
/go/src/github.com/docker/docker/testutil/environment/protect.go:68 +0xb1
github.com/docker/docker/testutil/environment.ProtectAll({0x12e8d98, 0xc00040e210}, {0x12f2878, 0xc0004fc340}, 0xc0001fac00)
/go/src/github.com/docker/docker/testutil/environment/protect.go:45 +0xf3
github.com/docker/docker/integration/image.setupTest(0xc0004fc340)
/go/src/github.com/docker/docker/integration/image/main_test.go:46 +0x59
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Save the unmodified manifest list to keep the image ID of the
multi-platform images when not all platforms are present.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
`diffIDPaths` is not used and can be removed.
`savedConfig` stores if the config was already saved (ID of the image is
the ID of the config).
`savedLayers` stores if the layer (diff ID) was already saved.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0
=== Pasta ===
RootlessKit v2 adds the support for pasta (https://passt.top/passt/).
Pasta is similar to slirp4netns but its port forwarder achieves better
throughput than slirp4netns port driver.
It is still not faster than RootlessKit's `builtin` port driver, but unlike the
`builtin` port driver, pasta can retain source IP address information.
Network driver | Port driver | Net throughput | Port throughput | Src IP | No SUID | Note
---------------|----------------|----------------|-----------------|--------|---------|--------------------------------------------
slirp4netns | builtin | Slow | Fast ✅ | ❌ | ✅ | Default in typical setup
vpnkit | builtin | Slow | Fast ✅ | ❌ | ✅ | Default when slirp4netns is not installed
slirp4netns | slirp4netns | Slow | Slow | ✅ | ✅ |
**pasta** | **implicit** | Slow | Fast ✅ | ✅ | ✅ | Experimental
lxc-user-nic | builtin | Fast ✅ | Slow | ❌ | ❌ | Experimental
(bypass4netns) | (bypass4netns) | Fast ✅ | Fast ✅ | ✅ | ✅ | (Not integrated to RootlessKit)
=== Detach-netns ===
Aside from pasta, RootlessKit v2 also brings the support for
"detach-netns" mode, which leaves the runtime in the host network namespace to
eliminate the slirp overhead for pull/push and to allow accessing the "real"
127.0.0.1.
See containerd/nerdctl PR 2723 for how detach-netns is being adopted in
nerdctl v2.
Integrating detach-netns into Docker/Moby will need an extra work and will be
deferred to Docker v26 (or later).
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Update the TestDaemonRestartKilContainers integration test to assert
that a container's healthcheck status is always reset to the Starting
state after a daemon restart, even when the container is live-restored.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.
Also set the digest to the actual digest of the tar archive.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Switch github.com/imdario/mergo to dario.cat/mergo v1.0.0, because
the module was renamed, and reached v1.0.0
full diff: https://github.com/imdario/mergo/compare/v0.3.13...v1.0.0
vendor: github.com/containerd/containerd v1.7.12
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12
Welcome to the v1.7.12 release of containerd!
The twelfth patch release for containerd 1.7 contains various fixes and updates.
Notable Updates
- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13
Deprecation Warnings
- Emit deprecation warning for `containerd.io/restart.logpath` label usage
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12
Welcome to the v1.7.12 release of containerd!
The twelfth patch release for containerd 1.7 contains various fixes and updates.
Notable Updates
- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13
Deprecation Warnings
- Emit deprecation warning for `containerd.io/restart.logpath` label usage
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).
The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.
To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Split task creation and start into two separate method calls in the
libcontainerd API. Clients now have the opportunity to inspect the
freshly-created task and customize its runtime environment before
starting execution of the user-specified binary.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The container may have been running without health probes for an
indeterminate amount of time. The container may have become unhealthy in
the interim. We should probe it sooner than in steady-state, while also
giving it some leeway to recover from e.g. timed-out connections. This
is easy to achieve by probing the container like a freshly-started one.
The original author of health-checks came to the same conclusion; the
health monitor was reinitialized on live-restored containers before
v17.11.0, when health monitoring of live-restored containers was
accidentally broken. Revert to the original behavior.
Signed-off-by: Cory Snider <csnider@mirantis.com>
commit 4f9db655ed moved looking up the
userland-proxy binary to early in the startup process, and introduced
a log-message if the binary was missing.
However, a side-effect of this was this message would also be printed
when running "--version";
dockerd --version
time="2024-01-09T09:18:53.705271292Z" level=warning msg="failed to lookup default userland-proxy binary" error="exec: \"docker-proxy\": executable file not found in $PATH"
Docker version v25.0.0-rc.1, build 9cebefa717
We should look if we can avoid this, but let's change the message to be
a debug message as a short-term workaround.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 0b1c1877c5 updated the version in
hack/dockerfile/install/rootlesskit.installer, but forgot to update the
version in Dockerfile.
Also updating both to use a tag, instead of commit. While it's good to pin by
an immutable reference, I think it's reasonably safe to use the tag, which is
easier to use, and what we do for other binaries, such as runc as well.
Full diff: https://github.com/rootless-containers/rootlesskit/compare/v1.1.0...v1.1.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The only use is in `builder/builder-next/adapters/snapshot.EnsureLayer()`,
which always calls the function with an _empty_ `oldTarDataPath`;
7082aecd54/builder/builder-next/adapters/snapshot/layer.go (L81)
When called with an empty `oldTarDataPath`, this function was an alias for
`checksumForGraphIDNoTarsplit`, so let's make it that.
Note that this code was added in 500e77bad0, as
part of the migration from "v1" images to "v2" (content-addressable) images.
Given that the remaining code lives in a "migration" file, possibly more code
can be removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ExecBackend.ContainerKill()` function was called before removing a build-
container.
This function is backed by `daemon.ContainerKill()` which, if no signal is passed,
performed a `daemon.Kill()`, using `SIGKILL` as signal. However, the
`ExecBackend.ContainerRm()` (backed by `daemonContainerRm()`), which is called
after this, is executed with the `ForceRemove` option set, which calls
`daemon.cleanupContainer()` with `ForceRemove` set, which also results in
`daemon.Kill()` being called:
1a0c15abbb/daemon/delete.go (L84-L95)
This makes the `ExecBackend.ContainerKill()` redundant, so removing this from
the interface.
While looking at this code, one (possible) race-condition was found in
`daemon.cleanupContainer()`, where `daemon.Kill()` could return a `errdefs.Conflict`
if the container was already stopped. An extra check was added for this case to
prevent `daemon.cleanupContainer()` from terminating early.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Prevent cleanup from terminating early when failing to remove a container;
- continue trying to remove remaining containers
- ignore errors due to containers that were not found
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was just a very thin wrapper for backend.ContainerRm(), and the
error it returned was not handled, so moving this code inline.
Moving it inline also allows differentiating the error message to
distinguish the "removing all intermediate containers" from "removing container"
(when cancelling a build).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The output var was used in a `defer`, but named `err` and shadowed in various
places. Rename the var to a more explicit name to make clear where it's used
and to prevent it being accidentally shadowed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
release notes:
- NewSystemd handles UnitExists when starting units
- makefile fixes
- cgroups2: export memory max usage and swap max usage
- build(deps): bump github.com/cilium/ebpf from v0.9.1 to v0.11.0
- support psi
- feat: add Threads for cgroupv2
- Linux.Swap is defined as memory+swap combined, while in cgroup v2 swap is a separate value
- fix(): support re-enabling oom killer refs #307 by @kestrelcjx
full diff: https://github.com/containerd/cgroups/compare/v3.0.2...v3.0.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
To make the version format in the `moby-bin` consistent with the
version we use in the release pipeline.
```diff
Server: Docker Engine - Community
Engine:
- Version: v25.0.0
+ Version: 25.0.0
...
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.
For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.
This patch brings the daemon-side lookup of image-manifests on-par with the
client-side lookup (the GET /distribution endpoint) as used in API 1.30 and
higher.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, did not fall back to querying the upstream if the mirror did not
contain the given reference.
For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.
This problem was caused by the logic used in GetRepository, which had an
optimization to only return the first registry it was successfully able to
configure (and connect to), with the assumption that the mirror either contained
all images used, or to be configured as a pull-through mirror.
This patch:
- Introduces a GetRepositories method, which returns all candidates (both
mirror(s) and upstream).
- Updates the endpoint to try all
Before this patch:
# the daemon is configured to use a mirror for Docker Hub
cat /etc/docker/daemon.json
{ "registry-mirrors": ["http://localhost:5000"]}
# start the mirror (empty registry, not configured as pull-through mirror)
docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2
# querying the endpoint fails, because the image-manifest is not found in the mirror:
curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json
{
"message": "manifest unknown: manifest unknown"
}
With this patch applied:
# the daemon is configured to use a mirror for Docker Hub
cat /etc/docker/daemon.json
{ "registry-mirrors": ["http://localhost:5000"]}
# start the mirror (empty registry, not configured as pull-through mirror)
docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2
# querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry):
curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq .
{
"Descriptor": {
"mediaType": "application/vnd.oci.image.index.v1+json",
"digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e",
"size": 3849
},
"Platforms": [
{
"architecture": "amd64",
"os": "linux"
}
]
}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A check was added to the bridge driver to detect when it was called to
create the default bridge nw whereas a stale default bridge already
existed. In such case, the bridge driver was deleting the stale network
before re-creating it. This check was introduced in docker/libnetwork@6b158eac6a
to fix an issue related to newly introduced live-restore.
However, since commit docker/docker@ecffb6d58c,
the daemon doesn't even try to create default networks if there're
active sandboxes (ie. due to live-restore).
Thus, now it's impossible for the default bridge network to be stale and
to exists when the driver's CreateNetwork() method is called. As such,
the check introduced in the first commit mentioned above is dead code
and can be safely removed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Turn subsequent `Close` calls into a no-op and produce a warning with an
optional stack trace (if debug mode is enabled).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This hopefully makes the test less flakey (or removes any flake that
would be caused by the test itself).
1. Adds tail of cluster daemon logs when there is a test failure so we
can more easily see what may be happening
2. Scans the daemon logs to check if the key is rotated before
restarting the daemon. This is a little hacky but a little better
than assuming it is done after a hard-coded 3 seconds.
3. Cleans up the `node ls` check such that it uses a poll function
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The Go 1.21.5 compiler has a bug: per-file language version override
directives do not take effect when instantiating generic functions which
have certain nontrivial type constraints. Consequently, a module-mode
project with Moby as a dependency may fail to compile when the compiler
incorrectly applies go1.16 semantics to the generic function call.
As the offending function is trivial and is only used in one place, work
around the issue by converting it to a concretely-typed function.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v25.0.0 release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The following fields are never written and are now marked as deprecated:
- `HairpinMode`
- `LinkLocalIPv6Address`
- `LinkLocalIPv6PrefixLen`
- `SecondaryIPAddress`
- `SecondaryIPv6Addresses`
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This is the eleventh patch release in the 1.1.z release branch of runc.
It primarily fixes a few issues with runc's handling of containers that
are configured to join existing user namespaces, as well as improvements
to cgroupv2 support.
- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2.
Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
For cgroupv1, Usage and Failcnt are set by subtracting memory usage
from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.11
- full diff: https://github.com/opencontainers/runc/compare/v1.1.10...v1.1.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the eleventh patch release in the 1.1.z release branch of runc.
It primarily fixes a few issues with runc's handling of containers that
are configured to join existing user namespaces, as well as improvements
to cgroupv2 support.
- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2.
Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
For cgroupv1, Usage and Failcnt are set by subtracting memory usage
from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.11
- full diff: https://github.com/opencontainers/runc/compare/v1.1.10...v1.1.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use stdlib's filepath.VolumeName to get the volume-name (if present) instead
of a self-crafted implementation, and unify the implementations for Windows
and Unix.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package provided utilities to obtain the apparmor_parser version, as well
as loading a profile.
Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used, which made LoadProfile the only utility remaining in this package.
LoadProfile appears to have no external consumers, and the only use in our code
is "profiles/apparmor".
This patch moves the remaining code (LoadProfile) to profiles/apparmor as a
non-exported function, and deletes the package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used.
This patch removes the deprecated utility, and inlines the remaining code into
the LoadProfile function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function never returned an error, and was not matching an interface, so
remove the error-return.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).
This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;
docker run -d -P nginx:alpine
4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.
However, the container would still be created (but invalid);
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
869f41d7e94f nginx:alpine "/docker-entrypoint.…" 10 seconds ago Created romantic_wiles
This patch changes how the userland-proxy is configured;
- The path of the userland-proxy is now looked up / configured at daemon
startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
is now required to be set. It no longer looks up the executable, and
produces an error if no path was provided. While this change was not
required, it makes the daemon config the canonical source of truth, instead
of logic spread accross multiplee locations.
Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;
- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
preference in PATH)
With this patch:
Validating the config produces an error if the binary is not found:
dockerd --validate
WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary error="exec: \"docker-proxy\": executable file not found in $PATH"
userland-proxy is enabled, but userland-proxy-path is not set
Disabling userland-proxy prints a warning, but validates as "OK":
dockerd --userland-proxy=false --validate
WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary error="exec: \"docker-proxy\": executable file not found in $PATH"
configuration OK
Speficying a non-absolute path produces an error:
dockerd --userland-proxy-path=docker-proxy --validate
invalid userland-proxy-path: must be an absolute path: docker-proxy
Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;
docker run -d -P nginx:alpine
4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.
Specifying an invalid userland-proxy-path produces an error as well:
dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory
mkdir -p /usr/local/bin/not-a-file
dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory
touch /usr/local/bin/not-an-executable
dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied
Same when using the daemon.json config-file;
echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
dockerd --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary
dockerd --userland-proxy-path=hello --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The cleanup function never returns an error, so didn't add much value. This
patch removes the closure, and calls it inline to remove the extra
indirection, and removes the error which would never be returned.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The defer was set after the switch, but various code-paths inside the switch
could return with an error after the port was allocated / reserved, which
could result in those ports not being released.
This patch moves the defer into each individual branch of the switch to set
it immediately after succesfully reserving the port.
We can also remove a redundant ReleasePort from the cleanup function, as
it's only called if an error occurs, and the defers already take care of
that.
Note that the cleanup function was handling errors returned by ReleasePort,
but this function never returns an error, so it was fully redundant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Prevent accidentally shadowing the error, which is used in a defer.
Also re-format the code to make it more clear we're not acting on
a locally-scoped "allocatedHostPort" variable.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/klauspost/compress/compare/v1.17.2...v1.17.4
v1.17.4:
- huff0: Speed up symbol counting
- huff0: Remove byteReader
- gzhttp: Allow overriding decompression on transport
- gzhttp: Clamp compression level
- gzip: Error out if reserved bits are set
v1.17.3:
- fse: Fix max header size
- zstd: Improve better/best compression
- gzhttp: Fix missing content type on Close
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
All underlying jobs inherit from the status of all parent jobs
in the tree, not just the very parent. We need to apply the same
kind of special condition.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
If IPv6 is enabled for a bridge network, by the time configuration
is applied, the bridge will always have an address. Assert that, by
raising an error when the configuration is validated.
Use that to simplify the logic used to calculate which addresses
should be assigned to a bridge. Also remove a redundant check in
setupGatewayIPv6() and the error associated with it.
Fix unit tests that enabled IPv6, but didn't supply an IPv6 IPAM
address/pool. Before this change, these tests passed but silently
left the bridge without an IPv6 address.
(The daemon already ensured there was an IPv6 address, this change
does not add a new restriction on config at that level.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
Some checks in 'networkConfiguration.Validate()' were not running as
expected, they'd always pass - because 'parseNetworkOptions()' called
it before 'config.processIPAM()' had added IP addresses and gateways.
Signed-off-by: Rob Murray <rob.murray@docker.com>
No more concept of "anonymous endpoints". The equivalent is now an
endpoint with no DNSNames set.
Some of the code removed by this commit was mutating user-supplied
endpoint's Aliases to add container's short ID to that list. In order to
preserve backward compatibility for the ContainerInspect endpoint, this
commit also takes care of adding that short ID (and the container
hostname) to `EndpointSettings.Aliases` before returning the response.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The remapping in the commit code was in the wrong place, we would create
a diff and then remap the snapshot, but the descriptor created in
"CreateDiff" was still pointing to the old snapshot, we now remap the
snapshot before creating a diff. Also make sure we don't lose any
capabilities, they used to be lost after the chown.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
If the resolver's DNSBackend returns a name that cannot be marshaled
into a well-formed DNS message, the resolver will only discover this
when it attempts to write the reply message and it fails with an error.
No reply message is sent, leaving the client to wait out its timeout and
the user in the dark about what went wrong.
When writing the intended reply message fails, retry once with a
ServFail response to inform the client and user that the DNS query was
not resolved due to a problem with to the resolver, not the network.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The well-formedness of a DNS message is only checked when it is
serialized, through the (*dns.Msg).Pack() method. Add a call to Pack()
to our tstwriter mock to mirror the behaviour of the real
dns.ResponseWriter implementation. And fix tests which generated
ill-formed DNS query messages.
Signed-off-by: Cory Snider <csnider@mirantis.com>
They fail because exporting an image which targets a manifest list when
only one platform is available exports only the platform-specific
manifest so the ID of the loaded image is different (ID of the platform
manifest, not manifest list).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The semantics of an "anonymous" endpoint has always been weird: it was
set on endpoints which name shouldn't be taken into account when
inserting DNS records into libnetwork's `Controller.svcRecords` (and
into the NetworkDB). However, in that case the endpoint's aliases would
still be used to create DNS records; thus, making those "anonymous
endpoints" not so anonymous.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The `(*Endpoint).rename()` method is changed to only mutate `ep.name`
and let a new method `(*Endpoint).UpdateDNSNames()` handle DNS updates.
As a consequence, the rollback code that was part of
`(*Endpoint).rename()` is now removed, and DNS updates are now
rolled back by `ContainerRename`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Instead of special-casing anonymous endpoints, use the list of DNS names
associated to the endpoint.
`(*Endpoint).isAnonymous()` has no more uses, so let's delete it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This new property will be empty if the daemon was upgraded with
live-restore enabled. To not break DNS resolutions for restored
containers, we need to populate dnsNames based on endpoint's myAliases &
anonymous properties.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Instead of special-casing anonymous endpoints in libnetwork, let the
daemon specify what (non fully qualified) DNS names should be associated
to container's endpoints.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
update the package, which contains a fix in the ssh package.
full diff: https://github.com/golang/crypto/compare/v0.16.0...v0.17.0
from the security mailing:
> Hello gophers,
>
> Version v0.17.0 of golang.org/x/crypto fixes a protocol weakness in the
> golang.org/x/crypto/ssh package that allowed a MITM attacker to compromise
> the integrity of the secure channel before it was established, allowing
> them to prevent transmission of a number of messages immediately after
> the secure channel was established without either side being aware.
>
> The impact of this attack is relatively limited, as it does not compromise
> confidentiality of the channel. Notably this attack would allow an attacker
> to prevent the transmission of the SSH2_MSG_EXT_INFO message, disabling a
> handful of newer security features.
>
> This protocol weakness was also fixed in OpenSSH 9.6.
>
> Thanks to Fabian Bäumer, Marcus Brinkmann, and Jörg Schwenk from Ruhr
> University Bochum for reporting this issue.
>
> This is CVE-2023-48795 and Go issue https://go.dev/issue/64784.
>
> Cheers,
> Roland on behalf of the Go team
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ensure that when removing an image, an image is checked consistently
against the images with the same target digest. Add unit testing around
delete.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Simplify the hijack process by just performing the http request/response
on the connection and returning the raw conn after success. The client
conn from httputil is deprecated and easily replaced.
Signed-off-by: Derek McGowan <derek@mcg.dev>
This new property is meant to replace myAliases and anonymous
properties.
The end goal is to get rid of both properties by letting the daemon
determine what (non fully qualified) DNS names should be associated to
them.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
They just happen to exist on a network that doesn't support DNS-based
service discovery (ie. no embedded DNS servers are started for them).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.
(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)
IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.
Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.
Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Before this change `ParentId` was filled for images when calling the
`/images/json` (image list) endpoint but was not for the
`/images/<image>/json` (image inspect).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This command was originally added by ea7f555446
to test the code snippet put into libnet's README.md. Nothing compiles
this file and it doesn't add any value to the project. So better remove
it than maintaining it.
This commit also removes the code snippet from libnet's README.md for
the same reasons.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This repository is not yet a module (i.e., does not have a `go.mod`). This
is not problematic when building the code in GOPATH or "vendor" mode, but
when using the code as a module-dependency (in module-mode), different semantics
are applied since Go1.21, which switches Go _language versions_ on a per-module,
per-package, or even per-file base.
A condensed summary of that logic [is as follows][1]:
- For modules that have a go.mod containing a go version directive; that
version is considered a minimum _required_ version (starting with the
go1.19.13 and go1.20.8 patch releases: before those, it was only a
recommendation).
- For dependencies that don't have a go.mod (not a module), go language
version go1.16 is assumed.
- Likewise, for modules that have a go.mod, but the file does not have a
go version directive, go language version go1.16 is assumed.
- If a go.work file is present, but does not have a go version directive,
language version go1.17 is assumed.
When switching language versions, Go _downgrades_ the language version,
which means that language features (such as generics, and `any`) are not
available, and compilation fails. For example:
# github.com/docker/cli/cli/context/store
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
Note that these fallbacks are per-module, per-package, and can even be
per-file, so _(indirect) dependencies_ can still use modern language
features, as long as their respective go.mod has a version specified.
Unfortunately, these failures do not occur when building locally (using
vendor / GOPATH mode), but will affect consumers of the module.
Obviously, this situation is not ideal, and the ultimate solution is to
move to go modules (add a go.mod), but this comes with a non-insignificant
risk in other areas (due to our complex dependency tree).
We can revert to using go1.16 language features only, but this may be
limiting, and may still be problematic when (e.g.) matching signatures
of dependencies.
There is an escape hatch: adding a `//go:build` directive to files that
make use of go language features. From the [go toolchain docs][2]:
> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a `go.mod` file with a go line such as `go 1.21` or `go 1.21.3`.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding `//go:build go1.22` to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.
This patch adds `//go:build` directives to those files using recent additions
to the language. It's currently using go1.19 as version to match the version
in our "vendor.mod", but we can consider being more permissive ("any" requires
go1.18 or up), or more "optimistic" (force go1.21, which is the version we
currently use to build).
For completeness sake, note that any file _without_ a `//go:build` directive
will continue to use go1.16 language version when used as a module.
[1]: 58c28ba286/src/cmd/go/internal/gover/version.go (L9-L56)
[2]: https://go.dev/doc/toolchain
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These fields were an implementation detail of the classic image builder
and are empty when using buildkit.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When the doc job is skipped, the dependent ones will be skipped
as well. To fix this issue we need to apply special conditions
to always run dependent jobs but not if canceled or failed.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
build --squash is an experimental feature that is not implemented in the
containerd image store.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
We already have everything needed to work inside a container, with this
configuration file developing in moby is even easier: the IDE will ask
you if you want to run everything inside a container and set it up for
you. No need to know that you have to run "BIN_DIR=. make shell" any
more.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
No need to copy the parent label from the source dangling image, because
it will already be copied from the source image.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
A validation step was added to prevent the daemon from considering "logentries"
as a dynamically loaded plugin, causing it to continue trying to load the plugin;
WARN[2023-12-12T21:53:16.866857127Z] Unable to locate plugin: logentries, retrying in 1s
WARN[2023-12-12T21:53:17.868296836Z] Unable to locate plugin: logentries, retrying in 2s
WARN[2023-12-12T21:53:19.874259254Z] Unable to locate plugin: logentries, retrying in 4s
WARN[2023-12-12T21:53:23.879869881Z] Unable to locate plugin: logentries, retrying in 8s
But would ultimately be returned as an error to the user:
docker container create --name foo --log-driver=logentries nginx:alpine
Error response from daemon: error looking up logging plugin logentries: plugin "logentries" not found
With the additional validation step, an error is returned immediately:
docker container create --log-driver=logentries busybox
Error response from daemon: the logentries logging driver has been deprecated and removed
A migration step was added on container restore. Containers using the
"logentries" logging driver are migrated to use the "local" logging driver:
WARN[2023-12-12T22:38:53.108349297Z] migrated deprecated logentries logging driver container=4c9309fedce75d807340ea1820cc78dc5c774d7bfcae09f3744a91b84ce6e4f7 error="<nil>"
As an alternative to the validation step, I also considered using a "stub"
deprecation driver, however this would not result in an error when creating
the container, and only produce an error when starting:
docker container create --name foo --log-driver=logentries nginx:alpine
4c9309fedce75d807340ea1820cc78dc5c774d7bfcae09f3744a91b84ce6e4f7
docker start foo
Error response from daemon: failed to create task for container: failed to initialize logging driver: the logentries logging driver has been deprecated and removed
Error: failed to start containers: foo
For containers, this validation is added in the backend (daemon). For services,
this was not sufficient, as SwarmKit would try to schedule the task, which
caused a close loop;
docker service create --log-driver=logentries --name foo nginx:alpine
zo0lputagpzaua7cwga4lfmhp
overall progress: 0 out of 1 tasks
1/1: no suitable node (missing plugin on 1 node)
Operation continuing in background.
DEBU[2023-12-12T22:50:28.132732757Z] Calling GET /v1.43/tasks?filters=%7B%22_up-to-date%22%3A%7B%22true%22%3Atrue%7D%2C%22service%22%3A%7B%22zo0lputagpzaua7cwga4lfmhp%22%3Atrue%7D%7D
DEBU[2023-12-12T22:50:28.137961549Z] Calling GET /v1.43/nodes
DEBU[2023-12-12T22:50:28.340665007Z] Calling GET /v1.43/services/zo0lputagpzaua7cwga4lfmhp?insertDefaults=false
DEBU[2023-12-12T22:50:28.343437632Z] Calling GET /v1.43/tasks?filters=%7B%22_up-to-date%22%3A%7B%22true%22%3Atrue%7D%2C%22service%22%3A%7B%22zo0lputagpzaua7cwga4lfmhp%22%3Atrue%7D%7D
DEBU[2023-12-12T22:50:28.345201257Z] Calling GET /v1.43/nodes
So a validation was added in the service create and update endpoints;
docker service create --log-driver=logentries --name foo nginx:alpine
Error response from daemon: the logentries logging driver has been deprecated and removed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The service was discontinued on November 15, 2022, so
remove mentions of this driver in the API docs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The service was discontinued on November 15, 2022, so
remove mentions of this driver in the API docs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Logentries service will be discontinued next week:
> Dear Logentries user,
>
> We have identified you as the owner of, or collaborator of, a Logentries account.
>
> The Logentries service will be discontinued on November 15th, 2022. This means that your Logentries account access will be removed and all your log data will be permanently deleted on this date.
>
> Next Steps
> If you are interested in an alternative Rapid7 log management solution, InsightOps will be available for purchase through December 16th, 2022. Please note, there is no support to migrate your existing Logentries account to InsightOps.
>
> Thank you for being a valued user of Logentries.
>
> Thank you,
> Rapid7 Customer Success
There is no reason to preserve this code in Moby as a result.
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit switches our code to use semconv 1.21, which is the version matching
the OTEL modules, as well as the containerd code.
The BuildKit 0.12.x module currently uses an older version of the OTEL modules,
and uses the semconv 0.17 schema. Mixing schema-versions is problematic, but
we still want to consume BuildKit's "detect" package to wire-up other parts
of OTEL.
To align the versions in our code, this patch sets the BuildKit detect.Resource
with the correct semconv version.
It's worth noting that the BuildKit package has a custom "serviceNameDetector";
https://github.com/moby/buildkit/blob/v0.12.4/util/tracing/detect/detect.go#L153-L169
Whith is merged with OTEL's default resource:
https://github.com/moby/buildkit/blob/v0.12.4/util/tracing/detect/detect.go#L100-L107
There's no need to duplicate that code, as OTEL's `resource.Default()` already
provides this functionality:
- It uses fromEnv{} detector internally: https://github.com/open-telemetry/opentelemetry-go/blob/v1.19.0/sdk/resource/resource.go#L208
- fromEnv{} detector reads OTEL_SERVICE_NAME: https://github.com/open-telemetry/opentelemetry-go/blob/v1.19.0/sdk/resource/env.go#L53
This patch also removes uses of the httpconv package, which is no longer included
in semconv 1.21 and now an internal package. Removing the use of this package
means that hijacked connections will not have the HTTP attributes on the Moby
client span, which isn't ideal, but a limited loss that'd impact exec/attach.
The span itself will still exist, it just won't the additional attributes that
are added by that package.
Alternatively, the httpconv call COULD remain - it will not error and will send
syntactically valid spans but we would be mixing & matching semconv versions,
so won't be compliant.
Some parts of the httpconv package were preserved through a very minimal local
implementation; a variant of `httpconv.ClientStatus(resp.StatusCode))` is added
to set the span status (`span.SetStatus()`). The `httpconv` package has complex
logic for this, but mostly drills down to HTTP status range (1xx/2xx/3xx/4xx/5xx)
to determine if the status was successfull or non-successful (4xx/5xx).
The additional logic it provided was to validate actual status-codes, and to
convert "bogus" status codes in "success" ranges (1xx, 2xx) into an error. That
code seemed over-reaching (and not accounting for potential future _valid_
status codes). Let's assume we only get valid status codes.
- https://github.com/open-telemetry/opentelemetry-go/blob/v1.21.0/semconv/v1.17.0/httpconv/http.go#L85-L89
- https://github.com/open-telemetry/opentelemetry-go/blob/v1.21.0/semconv/internal/v2/http.go#L322-L330
- https://github.com/open-telemetry/opentelemetry-go/blob/v1.21.0/semconv/internal/v2/http.go#L356-L404
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upgrade to the latest OpenTelemetry libraries; this will unblock a lot of
downstream projects in the ecosystem to upgrade, as some of the parts here
were pre-1.0/unstable.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
update the dependency to v0.2.4 to prevent scanners from flagging the
vulnerability (GHSA-6xv5-86q9-7xr8 / GO-2023-2048). Note that that vulnerability
only affects Windows, and is currently only used in runc/libcontainer, so should
not impact our use (as that code is Linux-only).
full diff: https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.10...v1.7.11
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.11
Welcome to the v1.7.11 release of containerd!
The eleventh patch release for containerd 1.7 contains various fixes and
updates including one security issue.
Notable Updates
- Fix Windows default path overwrite issue
- Update push to always inherit distribution sources from parent
- Update shim to use net dial for gRPC shim sockets
- Fix otel version incompatibility
- Fix Windows snapshotter blocking snapshot GC on remove failure
- Mask /sys/devices/virtual/powercap path in runtime spec and deny in
default apparmor profile [GHSA-7ww5-4wqc-m92c]
Deprecation Warnings
- Emit deprecation warning for AUFS snapshotter
- Emit deprecation warning for v1 runtime
- Emit deprecation warning for deprecated CRI configs
- Emit deprecation warning for CRI v1alpha1 usage
- Emit deprecation warning for CRIU config in CRI
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When reading logs, timestamps should always be presented in UTC. Unlike
the "json-file" and other logging drivers, the "local" logging driver
was using local time.
Thanks to Roman Valov for reporting this issue, and locating the bug.
Before this change:
echo $TZ
Europe/Amsterdam
docker run -d --log-driver=local nginx:alpine
fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3
docker logs --timestamps fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3
2023-12-08T18:16:56.291023422+01:00 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2023-12-08T18:16:56.291056463+01:00 /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2023-12-08T18:16:56.291890130+01:00 /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
...
With this patch:
echo $TZ
Europe/Amsterdam
docker run -d --log-driver=local nginx:alpine
14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5
docker logs --timestamps 14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5
2023-12-08T17:18:46.635967625Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2023-12-08T17:18:46.635989792Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2023-12-08T17:18:46.636897417Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
...
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If no `dangling` filter is specified, prune should only delete dangling
images.
This wasn't visible by doing `docker image prune` because the CLI
explicitly sets this filter to true.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Isolate the prune effects by running the test in a separate daemon.
This minimizes the impact of/on other integration tests.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Config serialization performed by the graphdriver implementation
maintained the distinction between an empty array and having no Cmd set.
With containerd integration we serialize the OCI types directly that use
the `omitempty` option which doesn't persist that distinction.
Considering that both values should have exactly the same semantics (no
cmd being passed) it should be fine if in this case the Cmd would be
null instead of an empty array.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Acquire the mutex in the help handler to synchronize access to the
handlers map. While a trivial issue---a panic in the request handler if
the node joins a swarm at just the right time, which would only result
in an HTTP 500 response---it is also a trivial race condition to fix.
Signed-off-by: Cory Snider <csnider@mirantis.com>
We don't need C-style callback functions which accept a void* context
parameter: Go has closures. Drop the unnecessary httpHandlerCustom type
and refactor the diagnostic server handler functions into closures which
capture whatever context they need implicitly.
If the node leaves and rejoins a swarm, the cluster agent and its
associated NetworkDB are discarded and replaced with new instances. Upon
rejoin, the agent registers its NetworkDB instance with the diagnostic
server. These handlers would all conflict with the handlers registered
by the previous NetworkDB instance. Attempting to register a second
handler on a http.ServeMux with the same pattern will panic, which the
diagnostic server would historically deal with by ignoring the duplicate
handler registration. Consequently, the first NetworkDB instance to be
registered would "stick" to the diagnostic server for the lifetime of
the process, even after it is replaced with another instance. Improve
duplicate-handler registration such that the most recently-registered
handler for a pattern is used for all subsequent requests.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Rewrite `.build-empty-images` shell script that produced special images
(emptyfs with no layers, and empty danglign image) to a Go functions
that construct the same archives in a temporary directory.
Use them to load these images on demand only in the tests that need
them.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This struct is intended for internal use only for the backend, and is
not intended to be used externally.
This moves the plugin-related `NetworkListConfig` types to the backend
package to prevent it being imported in the client, and to make it more
clear that this is part of internal APIs, and not public-facing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These structs are intended for internal use only for the backend, and are
not intended to be used externally.
This moves the plugin-related `PluginRmConfig`, `PluginEnableConfig`, and
`PluginDisableConfig` types to the backend package to prevent them being
imported in the client, and to make it more clear that this is part of
internal APIs, and not public-facing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.21.5 (released 2023-12-05) includes security fixes to the go command,
and the net/http and path/filepath packages, as well as bug fixes to the
compiler, the go command, the runtime, and the crypto/rand, net, os, and
syscall packages. See the Go 1.21.5 milestone on our issue tracker for
details:
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.4...go1.21.5
from the security mailing:
[security] Go 1.21.5 and Go 1.20.12 are released
Hello gophers,
We have just released Go versions 1.21.5 and 1.20.12, minor point releases.
These minor releases include 3 security fixes following the security policy:
- net/http: limit chunked data overhead
A malicious HTTP sender can use chunk extensions to cause a receiver
reading from a request or response body to read many more bytes from
the network than are in the body.
A malicious HTTP client can further exploit this to cause a server to
automatically read a large amount of data (up to about 1GiB) when a
handler fails to read the entire body of a request.
Chunk extensions are a little-used HTTP feature which permit including
additional metadata in a request or response body sent using the chunked
encoding. The net/http chunked encoding reader discards this metadata.
A sender can exploit this by inserting a large metadata segment with
each byte transferred. The chunk reader now produces an error if the
ratio of real body to encoded bytes grows too small.
Thanks to Bartek Nowotarski for reporting this issue.
This is CVE-2023-39326 and Go issue https://go.dev/issue/64433.
- cmd/go: go get may unexpectedly fallback to insecure git
Using go get to fetch a module with the ".git" suffix may unexpectedly
fallback to the insecure "git://" protocol if the module is unavailable
via the secure "https://" and "git+ssh://" protocols, even if GOINSECURE
is not set for said module. This only affects users who are not using
the module proxy and are fetching modules directly (i.e. GOPROXY=off).
Thanks to David Leadbeater for reporting this issue.
This is CVE-2023-45285 and Go issue https://go.dev/issue/63845.
- path/filepath: retain trailing \ when cleaning paths like \\?\c:\
Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the
volume name in Windows paths starting with \\?\, resulting in
filepath.Clean(\\?\c:\) returning \\?\c: rather than \\?\c:\ (among
other effects). The previous behavior has been restored.
This is an update to CVE-2023-45283 and Go issue https://go.dev/issue/64028.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.21.4 (released 2023-11-07) includes security fixes to the path/filepath
package, as well as bug fixes to the linker, the runtime, the compiler, and
the go/types, net/http, and runtime/cgo packages. See the Go 1.21.4 milestone
on our issue tracker for details:
- https://github.com/golang/go/issues?q=milestone%3AGo1.21.4+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.3...go1.21.4
from the security mailing:
[security] Go 1.21.4 and Go 1.20.11 are released
Hello gophers,
We have just released Go versions 1.21.4 and 1.20.11, minor point releases.
These minor releases include 2 security fixes following the security policy:
- path/filepath: recognize `\??\` as a Root Local Device path prefix.
On Windows, a path beginning with `\??\` is a Root Local Device path equivalent
to a path beginning with `\\?\`. Paths with a `\??\` prefix may be used to
access arbitrary locations on the system. For example, the path `\??\c:\x`
is equivalent to the more common path c:\x.
The filepath package did not recognize paths with a `\??\` prefix as special.
Clean could convert a rooted path such as `\a\..\??\b` into
the root local device path `\??\b`. It will now convert this
path into `.\??\b`.
`IsAbs` did not report paths beginning with `\??\` as absolute.
It now does so.
VolumeName now reports the `\??\` prefix as a volume name.
`Join(`\`, `??`, `b`)` could convert a seemingly innocent
sequence of path elements into the root local device path
`\??\b`. It will now convert this to `\.\??\b`.
This is CVE-2023-45283 and https://go.dev/issue/63713.
- path/filepath: recognize device names with trailing spaces and superscripts
The `IsLocal` function did not correctly detect reserved names in some cases:
- reserved names followed by spaces, such as "COM1 ".
- "COM" or "LPT" followed by a superscript 1, 2, or 3.
`IsLocal` now correctly reports these names as non-local.
This is CVE-2023-45284 and https://go.dev/issue/63713.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon currently provides support for API versions all the way back
to v1.12, which is the version of the API that shipped with docker 1.0. On
Windows, the minimum supported version is v1.24.
Such old versions of the client are rare, and supporting older API versions
has accumulated significant amounts of code to remain backward-compatible
(which is largely untested, and a "best-effort" at most).
This patch updates the minimum API version to v1.24, which is the fallback
API version used when API-version negotiation fails. The intent is to start
deprecating older API versions, but no code is removed yet as part of this
patch, and a DOCKER_MIN_API_VERSION environment variable is added, which
allows overriding the minimum version (to allow restoring the behavior from
before this patch).
With this patch the daemon defaults to API v1.24 as minimum:
docker version
Client:
Version: 24.0.2
API version: 1.43
Go version: go1.20.4
Git commit: cb74dfc
Built: Thu May 25 21:50:49 2023
OS/Arch: linux/arm64
Context: default
Server:
Engine:
Version: dev
API version: 1.44 (minimum version 1.24)
Go version: go1.21.3
Git commit: 0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
Built: Mon Dec 4 15:22:17 2023
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: v1.7.9
GitCommit: 4f03e100cb967922bec7459a78d16ccbac9bb81d
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Trying to use an older version of the API produces an error:
DOCKER_API_VERSION=1.23 docker version
Client:
Version: 24.0.2
API version: 1.23 (downgraded from 1.43)
Go version: go1.20.4
Git commit: cb74dfc
Built: Thu May 25 21:50:49 2023
OS/Arch: linux/arm64
Context: default
Error response from daemon: client version 1.23 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version
To restore the previous minimum, users can start the daemon with the
DOCKER_MIN_API_VERSION environment variable set:
DOCKER_MIN_API_VERSION=1.12 dockerd
API 1.12 is the oldest supported API version on Linux;
docker version
Client:
Version: 24.0.2
API version: 1.43
Go version: go1.20.4
Git commit: cb74dfc
Built: Thu May 25 21:50:49 2023
OS/Arch: linux/arm64
Context: default
Server:
Engine:
Version: dev
API version: 1.44 (minimum version 1.12)
Go version: go1.21.3
Git commit: 0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
Built: Mon Dec 4 15:22:17 2023
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: v1.7.9
GitCommit: 4f03e100cb967922bec7459a78d16ccbac9bb81d
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0
When using the `DOCKER_MIN_API_VERSION` with a version of the API that
is not supported, an error is produced when starting the daemon;
DOCKER_MIN_API_VERSION=1.11 dockerd --validate
invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: 1.11
DOCKER_MIN_API_VERSION=1.45 dockerd --validate
invalid DOCKER_MIN_API_VERSION: maximum supported API version is 1.44: 1.45
Specifying a malformed API version also produces the same error;
DOCKER_MIN_API_VERSION=hello dockerd --validate
invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: hello
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ContainerCreateConfig` and `ContainerRmConfig` structs are used for
options to be passed to the backend, and are not used in client code.
Thess struct currently is intended for internal use only (for example, the
`AdjustCPUShares` is an internal implementation details to adjust the container's
config when older API versions are used).
Somewhat ironically, the signature of the Backend has a nicer UX than that
of the client's `ContainerCreate` signature (which expects all options to
be passed as separate arguments), so we may want to update that signature
to be closer to what the backend is using, but that can be left as a future
exercise.
This patch moves the `ContainerCreateConfig` and `ContainerRmConfig` structs
to the backend package to prevent it being imported in the client, and to make
it more clear that this is part of internal APIs, and not public-facing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Follow-up to e72c4818c4, which updated the
Dockerfile to use Debian 12 "bookworm", but forgot to update the package
repository to use for the CRIU packages. Note that the criu stage is currently
not built by default (see d3d2823edf), so to
verify the stage, it needs to be built manually;
docker build --target=criu .
This patch adds an extra `criu --version` to the build, so that it's verified
to be "functional".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These tests build new images, setupTest sets up the test cleanup
function that clears the test environment from created images,
containers, etc.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This removes various skips that accounted for running the integration tests
against older versions of the daemon before 20.10 (API version v1.41). Those
versions are EOL, and we don't run tests against them.
This reverts most of e440831802, and similar
PRs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use the minimum API version as advertised by the test-daemon, instead of the
hard-coded API version from code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was using API version 1.20 to test old behavior, but the actual change
in behavior was API v1.25; see commit 6d98e344c7
and 63b5a37203.
This updates the test to use API v1.24 to test the old behavior.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
TestKillDifferentUserContainer was migrated from integration-cli in
commit 0855922cd3. Before migration, it
was not using a specific API version, so we can assume "current"
API version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 59b83d8aae containerized these steps,
as they didn't work well on Debian Jessie:
> Because the `mount` here will sometimes fail when run in `debian:jessie`,
> which is what the environrment hosting the test suite is running if run
> from the `Makefile`.
> Also, why the heck not containerize it, all the things.
Follow-up commits, such as 228d74842f, and
1c5806cf57 updated the Debian distro, but
also updated this comment, losing the original context (the issue was
(originally) related to Debian Jessie).
This patch changes the test back to not use containers, which seems to
work fine (at least "it worked on my machine").
make TEST_IGNORE_CGROUP_CHECK=1 TEST_FILTER=TestDaemonNoSpaceLeftOnDeviceError DOCKER_GRAPHDRIVER=overlay2 test-integration
=== RUN TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError
check_test.go:589: [df36ad96a412b] daemon is not started
--- PASS: TestDockerDaemonSuite (5.12s)
--- PASS: TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError (5.12s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was originally added in 8ec8564691,
at which time the upstream debian package repositories were not always
reliable, so using a mirror helped with CI stability and performance.
Debian's package repositories are a lot more reliable now, so there's no
longer a need to use a mirror.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When creating a CIFS volume, generate an error if the device URL
includes a port number, for example:
--opt device="//some.server.com:2345/thepath"
The port must be specified in the port option instead, for example:
--opt o=username=USERNAME,password=PASSWORD,vers=3,sec=ntlmsspi,port=1234
Signed-off-by: Rob Murray <rob.murray@docker.com>
Add the TaskStatus, PortStatus and ContainerStatus to api docs. TaskStatus was moved to the swagger definitions root from anonymous type definition, and PortStatus and Container Status are its dependencies.
Signed-off-by: Martin Jirku <martin@jirku.sk>
Move the initialization logic to the attachContext itself, so that
the container doesn't have to be aware about mutexes and other logic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Skip TestListDanglingImagesWithDigests which tests graphdriver
implementation specific behavior of `docker images --filter
dangling=true`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This removes various templating functions that were added for the
docker CLI. These are not needed for the dockerd binary, which does
not have subcommands or management commands.
Revert "Only hide commands if the env variable is set."
This reverts commit a7c8bcac2b.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We used DEBIAN_FRONTEND in some places to prevent installation of packages
from being blocked. However, debian bookworm now [includes a fix][1] for
situations like this (it was specifically reported for Docker situations <3),
so we can get rid of these.
Thanks to Tianon for noticing this, and for linking to the Debian ticket!
[1]: https://bugs.debian.org/929417
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Replace `time.Sleep` with a poll that checks if process no longer exists
to avoid possible race condition.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
I noticed this log being logged as an error, but the kill logic actually
proceeds after this (doing a "direct" kill instead). While usually containers
are expected to be exiting within the given timeout, I don't think this
needs to be logged as an error (an error is returned after we fail to
kill the container).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The auth service error response is not a part of the spec and containerd
doesn't parse it like the Docker's distribution does.
Check for containerd specific errors instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Debian Woodworm ships with a newer version of iptables, which caused two
tests to fail:
=== FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)
docker_cli_daemon_test.go:841: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge6.*ext-bridge6", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge6 !ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge6 ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n"
--- FAIL: TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)
=== FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCPing (1.19s)
docker_cli_daemon_test.go:803: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge5.*ext-bridge5", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge5 !ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge5 ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n"
--- FAIL: TestDockerDaemonSuite/TestDaemonICCPing (1.19s)
Both the `TestDaemonICCPing`, and `TestDaemonICCLinkExpose` test were introduced
in dd0666e64f. These tests called `iptables` with
the `-n` (`--numeric`) option, which prevents it from doing a reverse-DNS lookup
as an optimization.
However, the `-n` option did not have an effect to the `prot` column before
commit [da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa] (iptables < v1.8.9 or v1.8.8).
Newer versions, such as the iptables version shipping with Debian Woodworm do,
so we need to update the expected output for this version.
This patch removes the `-n` option, to keep the test more portable, also when
run non-containerized, and removes the use of regular expressions to check the
result, as these regular expressions were quite permissive (using `.*` wild-
card matching). Instead, we're getting the
With this change;
make DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestDaemonICC TEST_IGNORE_CGROUP_CHECK=1 test-integration
...
--- PASS: TestDockerDaemonSuite (139.11s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCLinkExpose (54.62s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCPing (84.48s)
[da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa]: https://git.netfilter.org/iptables/commit/?id=da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When live-restore is enabled, containers with autoremove enabled
shouldn't be forcibly killed when engine restarts.
They still should be removed if they exited while the engine was down
though.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Change the repo name used as for an intermediate image so it doesn't
try to mount from the image pushed by `TestBuildMultiStageImplicitPull`.
Before this patch, this test failed because the distribution.source
labels are not cleared between tests and the busybox content still has
the distribution.source label pointing to the `dockercli/testf`
repository which is no longer present in the test registry.
So both `dockercli/busybox` and `dockercli/testf` are equally valid
mount candidates for `dockercli/crossrepopush` and containerd algorithm
just happens to select the last one.
This changes the repo name to not have the common repository component
(`dockercli`) with the `dockercli/testf` repository.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Starting with [6e0ed3d19c54603f0f7d628ea04b550151d8a262], the minimum
allowed size is now 300MB. Given that this is a sparse image, and
the size of the image is irrelevant to the test (we check for
limits defined through project-quotas, not the size of the
device itself), we can raise the size of this image.
[6e0ed3d19c54603f0f7d628ea04b550151d8a262]: https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=6e0ed3d19c54603f0f7d628ea04b550151d8a262
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On bookworm, AppArmor failed to start inside the container, which can be
seen at startup of the dev-container:
Created symlink /etc/systemd/system/systemd-firstboot.service → /dev/null.
Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
Created symlink /etc/systemd/system/multi-user.target.wants/docker-entrypoint.service → /etc/systemd/system/docker-entrypoint.service.
hack/dind-systemd: starting /lib/systemd/systemd --show-status=false --unit=docker-entrypoint.target
systemd 252.17-1~deb12u1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Detected virtualization docker.
Detected architecture x86-64.
modprobe@configfs.service: Deactivated successfully.
modprobe@dm_mod.service: Deactivated successfully.
modprobe@drm.service: Deactivated successfully.
modprobe@efi_pstore.service: Deactivated successfully.
modprobe@fuse.service: Deactivated successfully.
modprobe@loop.service: Deactivated successfully.
apparmor.service: Starting requested but asserts failed.
proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 49 (systemd-binfmt)
+ source /etc/docker-entrypoint-cmd
++ hack/make.sh dynbinary test-integration
When checking "aa-status", an error was printed that the filesystem was
not mounted:
aa-status
apparmor filesystem is not mounted.
apparmor module is loaded.
Checking if "local-fs.target" was loaded, that seemed to be the case;
systemctl status local-fs.target
● local-fs.target - Local File Systems
Loaded: loaded (/lib/systemd/system/local-fs.target; static)
Active: active since Mon 2023-11-27 10:48:38 UTC; 18s ago
Docs: man:systemd.special(7)
However, **on the host**, "/sys/kernel/security" has a mount, which was not
present inside the container:
mount | grep securityfs
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
Interestingly, on `debian:bullseye`, this was not the case either; no
`securityfs` mount was present inside the container, and apparmor actually
failed to start, but succeeded silently:
mount | grep securityfs
systemctl start apparmor
systemctl status apparmor
● apparmor.service - Load AppArmor profiles
Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2023-11-27 11:59:09 UTC; 44s ago
Docs: man:apparmor(7)
https://gitlab.com/apparmor/apparmor/wikis/home/
Process: 43 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=0/SUCCESS)
Main PID: 43 (code=exited, status=0/SUCCESS)
CPU: 10ms
Nov 27 11:59:09 9519f89cade1 apparmor.systemd[43]: Not starting AppArmor in container
Same, using the `/etc/init.d/apparmor` script:
/etc/init.d/apparmor start
Starting apparmor (via systemctl): apparmor.service.
echo $?
0
And apparmor was not actually active:
aa-status
apparmor module is loaded.
apparmor filesystem is not mounted.
aa-enabled
Maybe - policy interface not available.
After further investigating, I found that the non-systemd dind script
had a mount for AppArmor, which was added in 31638ab2ad
The systemd variant was missing this mount, which may have gone unnoticed
because `debian:bullseye` was silently ignoring this when starting the
apparmor service.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
BaseFS is not serialized and is lost after an unclean shutdown. Unmount
method in the containerd image service implementation will not work
correctly in that case.
This patch will allow Unmount to restore the BaseFS if the target is
still mounted.
The reason it works with graphdrivers is that it doesn't directly
operate on BaseFS. It uses RWLayer, which is explicitly restored
immediately as soon as container is loaded.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This is purely cosmetic - if a non-default MTU is configured, the bridge
will have the default MTU=1500 until a container's 'veth' is connected
and an MTU is set on the veth. That's a disconcerting, it looks like the
config has been ignored - so, set the bridge's MTU explicitly.
Fixes#37937
Signed-off-by: Rob Murray <rob.murray@docker.com>
The graphdriver implementation sets the ModTime of all image content to
match the `Created` time from the image config, whereas the containerd's
archive export code just leaves it empty (zero).
Adjust the test in the case where containerd integration is enabled to
check if config file ModTime is equal to zero (UNIX epoch) instead.
This behaviour is not a part of the Docker Image Specification and the
intention behind introducing it was to make the `docker save` produce
the same archive regardless of the time it was performed.
It would also be a bit problematic with the OCI archive layout which can
contain multiple images referencing the same content.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Also, err `e` is renamed into the more standard `err` as the defer
already uses `retErr` to avoid clashes (changed in f5a611a74).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
DNS config is a property of each adapter on Windows, thus we've a
dedicated `EndpointOption` for that.
The list of `EndpointOption` that should be applied to a given endpoint
is built by `buildCreateEndpointOptions`. This function contains a
seemingly flawed condition that adds the DNS config _iff_:
1. the network isn't internal ;
2. no ports are published / exposed through another sandbox endpoint ;
While 1. does make sense, there's actually no justification for 2.,
hence this commit remove this part of the condition.
This logic flaw has been made obvious by 0fd0e82, but it was originally
introduced by d1e0a78. Commit and PR comments don't mention why this is
done like so. Most probably, this was overlooked both by the original
author and the PR reviewers.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The `buildCreateEndpointOptions` does a lot of things to build the list
of `libnetwork.EndpointOption` from the `EndpointSettings` spec. To skip
ports-related options, an early return was put in the middle of that
function body.
Early returns are generally great, but put in the middle of a 150-loc
long function that does a lot, they're just a potential footgun. And I'm
the one who pulled the trigger in 052562f. Since this commit, generic
options won't be applied to endpoints if there's already one with
exposed/published ports. As a consequence, only the first endpoint can
have a user-defined MAC address right now.
Instead of moving up the code line that adds generic options, a better
change IMO is to move ports-related options, and the early-return gating
those options, to a dedicated func to make `buildCreateEndpointOptions`
slightly easier to read and reason about.
There was actually one oddity in the original
`buildCreateEndpointOptions`: the early-return also gates the addition
of `CreateOptionDNS`. These options are Windows-specific; a comment is
added to explain that. But the oddity is really: why are we checking if
an endpoint with exposed / published ports joined this sandbox to decide
whether we want to configure DNS server on the endpoint's adapter? Well,
this early-return was most probably overlooked by the original author
and by reviewers at the time these options were added (in commit d1e0a78)
Let's fix that in a follow-up commit.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The example still used the deprecated types.ContainerListOptions;
also slightly updated the example to show both stopped and running
containers, so that the example works even if no container is running.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The DirCopy() function in "graphdriver/copy/copy.go" has a special case for
skip file-attribute copying when making a hard link to an already-copied
file, if "copyMode == Hardlink". Do the same for copies of hard-links in
the source filesystem.
Significantly speeds up vfs's copy of a BusyBox filesystem (which
consists mainly of hard links to a single binary), making moby's
integration tests run more quickly and more reliably in a dev container.
Fixes#46810
Signed-off-by: Rob Murray <rob.murray@docker.com>
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10
This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.
- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
(fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
configuration. This issue is not a security issue because it requires a
malicious config.json, which is outside of our threat model.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10
This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.
- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
(fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
configuration. This issue is not a security issue because it requires a
malicious config.json, which is outside of our threat model.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a strong type for the DNS IP-addresses so that we can use flags.IPSliceVar,
instead of implementing our own option-type and validation.
Behavior should be the same, although error-messages have slightly changed:
Before this patch:
dockerd --dns 1.1.1.1oooo --validate
Status: invalid argument "1.1.1.1oooo" for "--dns" flag: 1.1.1.1oooo is not an ip address
See 'dockerd --help'., Code: 125
cat /etc/docker/daemon.json
{"dns": ["1.1.1.1"]}
dockerd --dns 2.2.2.2 --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: dns: (from flag: [2.2.2.2], from file: [1.1.1.1])
cat /etc/docker/daemon.json
{"dns": ["1.1.1.1oooo"]}
dockerd --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: 1.1.1.1ooooo is not an ip address
With this patch:
dockerd --dns 1.1.1.1oooo --validate
Status: invalid argument "1.1.1.1oooo" for "--dns" flag: invalid string being converted to IP address: 1.1.1.1oooo
See 'dockerd --help'., Code: 125
cat /etc/docker/daemon.json
{"dns": ["1.1.1.1"]}
dockerd --dns 2.2.2.2 --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: dns: (from flag: [2.2.2.2], from file: [1.1.1.1])
cat /etc/docker/daemon.json
{"dns": ["1.1.1.1oooo"]}
dockerd --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid IP address: 1.1.1.1oooo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- document accepted values
- add test-coverage for the function's behavior (including whitespace handling),
and use sub-tests.
- improve error-message to use uppercase for "IP", and to use a common prefix.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I was trying to find out why `docker info` was sometimes slow so
plumbing a context through to propagate trace data through.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The forwarding database (fdb) of Linux VXLAN links are restricted to
entries with destination VXLAN tunnel endpoint (VTEP) address of a
single address family. Which address family is permitted is set when the
link is created and cannot be modified. The overlay network driver
creates VXLAN links such that the kernel only allows fdb entries to be
created with IPv4 destination VTEP addresses. If the Swarm is configured
with IPv6 advertise addresses, creating fdb entries for remote peers
fails with EAFNOSUPPORT (address family not supported by protocol).
Make overlay networks functional over IPv6 transport by configuring the
VXLAN links for IPv6 VTEPs if the local node's advertise address is an
IPv6 address. Make encrypted overlay networks secure over IPv6 transport
by applying the iptables rules to the ip6tables when appropriate.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Early return if the iface or its address is nil to make the whole
function slightly easier to read.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
We remap the snapshot when we create a container, we have to to the
inverse when we commit the container into an image
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
This test broke in 98323ac114.
This commit renamed WithMacAddress into WithContainerWideMacAddress.
This helper sets the MacAddress field in container.Config. However, API
v1.44 now ignores this field if the NetworkMode has no matching entry in
EndpointsConfig.
This fix uses the helper WithMacAddress and specify for which
EndpointConfig the MacAddress is specified.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When server address is not provided with the auth configuration,
use the domain from the image provided with the auth.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The workaround is no longer required. The bug has been fixed in stable
versions of all supported containerd branches.
This reverts commit fb7ec1555c.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Change the non-refcounted implementation to perform the mount using the
same identity and access right. They should be the same regardless if
we're refcounting or not.
This also allows to refactor refCountMounter into a mounter decorator.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Note that this entire document needs a modernization pass, both for
current best practices as well as something approximating
s/Docker/Moby/.
In the interest of making some forward progress, we'll make minimal
changes for now, however.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This test was rewritten from an integration-cli test in commit
68d9beedbe, and originally implemented in
f4942ed864, which rewrote it from a unit-
test to an integration test.
Originally, it would check for the raw JSON response from the daemon, and
check for individual fields to be present in the output, but after commit
0fd5a65428, `client.Info()` was used, and
now the response is unmarshalled into a `system.Info`.
The remainder of the test remained the same in that rewrite, and as a
result were were now effectively testing if a `system.Info` struct,
when marshalled as JSON would show all the fields (surprise: it does).
TL;DR; the test would even pass with an empty `system.Info{}` struct,
which didn't provide much coverage, as it passed without a daemon:
func TestInfoAPI(t *testing.T) {
// always shown fields
stringsToCheck := []string{
"ID",
"Containers",
"ContainersRunning",
"ContainersPaused",
"ContainersStopped",
"Images",
"LoggingDriver",
"OperatingSystem",
"NCPU",
"OSType",
"Architecture",
"MemTotal",
"KernelVersion",
"Driver",
"ServerVersion",
"SecurityOptions",
}
out := fmt.Sprintf("%+v", system.Info{})
for _, linePrefix := range stringsToCheck {
assert.Check(t, is.Contains(out, linePrefix))
}
}
This patch makes the test _slightly_ better by checking if the fields
are non-empty. More work is needed on this test though; currently it
uses the (already running) daemon, so it's hard to check for specific
fields to be correct (withouth knowing state of the daemon), but it's
not unlikely that other tests (partially) cover some of that. A TODO
comment was added to look into that (we should probably combine some
tests to prevent overlap, and make it easier to spot "gaps" as well).
While working on this, also moving the `SystemTime` into this test,
because that field is (no longer) dependent on "debug" state
(It is was actually this change that led me down this rabbit-hole)
()_()
(-.-)
'(")(")'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Following tests are implemented in this specific commit:
- Inter-container communications for internal and non-internal
bridge networks, over IPv4 and IPv6.
- Inter-container communications using IPv6 link-local addresses for
internal and non-internal bridge networks.
- Inter-network communications for internal and non-internal bridge
networks, over IPv4 and IPv6, are disallowed.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This commit introduces a new integration test suite aimed at testing
networking features like inter-container communication, network
isolation, port mapping, etc... and how they interact with daemon-level
and network-level parameters.
So far, there's pretty much no tests making sure our networks are well
configured: 1. there're a few tests for port mapping, but they don't
cover all use cases ; 2. there're a few tests that check if a specific
iptables rule exist, but that doesn't prevent that specific iptables
rule to be wrong in the first place.
As we're planning to refactor how iptables rules are written, and change
some of them to fix known security issues, we need a way to test all
combinations of parameters. So far, this was done by hand, which is
particularly painful and time consuming. As such, this new test suite is
foundational to upcoming work.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When the start interval is 0 we should treat that as unset.
This is especially important for older API versions where we reset the
value to 0.
Instead of using the default probe value we should be using the
configured `interval` value (which may be a default as well) which gives
us back the old behavior before support for start interval was added.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This syncs the seccomp profile with changes made to containerd's default
profile in [1].
The original containerd issue and PR mention:
> Security experts generally believe io_uring to be unsafe. In fact
> Google ChromeOS and Android have turned it off, plus all Google
> production servers turn it off. Based on the blog published by Google
> below it seems like a bunch of vulnerabilities related to io_uring can
> be exploited to breakout of the container.
>
> [2]
>
> Other security reaserchers also hold this opinion: see [3] for a
> blackhat presentation on io_uring exploits.
For the record, these syscalls were added to the allowlist in [4].
[1]: a48ddf4a20
[2]: https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html
[3]: https://i.blackhat.com/BH-US-23/Presentations/US-23-Lin-bad_io_uring.pdf
[4]: https://github.com/moby/moby/pull/39415
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This test is very weird, the Size in the manifests that it creates is
wrong, graph drivers only print a warning in that case but containerd
fails because it verifies more things. The media types are also wrong in
the containerd case, the manifest list forces the media type to be
"schema2.MediaTypeManifest" but in the containerd case the media type is
an OCI one.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
The build target is not quoted and it makes it difficult for some
persons to see what the problem is.
By quoting it we emphasize that the target name is variable.
Signed-off-by: Frank Villaro-Dixon <frank.villarodixon@merkle.com>
I am finally convinced that, given two netip.Prefix values a and b, the
expression
a.Contains(b.Addr()) || b.Contains(a.Addr())
is functionally equivalent to
a.Overlaps(b)
The (netip.Prefix).Contains method works by masking the address with the
prefix's mask and testing whether the remaining most-significant bits
are equal to the same bits in the prefix. The (netip.Prefix).Overlaps
method works by masking the longer prefix to the length of the shorter
prefix and testing whether the remaining most-significant bits are
equal. This is equivalent to
shorterPrefix.Contains(longerPrefix.Addr()), therefore applying Contains
symmetrically to two prefixes will always yield the same result as
applying Overlaps to the two prefixes in either order.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This delete was originally added in b37fdc5dd1
and migrated from `deleteImages(repoName)` in commit 1e55ace875,
however, deleting `foobar-save-multi-images-test` (`foobar-save-multi-images-test:latest`)
always resulted in an error;
Error response from daemon: No such image: foobar-save-multi-images-test:latest
This patch removes the redundant image delete.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Shutting down containers on Windows can take a long time (with hyper-v),
causing this test to be flaky; seen failing on windows 2022;
=== FAIL: github.com/docker/docker/integration/image TestSaveRepoWithMultipleImages (23.16s)
save_test.go:104: timeout waiting for container to exit
Looking at the test, we run a container only to commit it, and the test
does not make changes to the container's filesystem; it only runs a container
with a custom command (`true`).
Instead of running the container, we can _create_ a container and commit it;
this simplifies the tests, and prevents having to wait for the container to
exit (before committing).
To verify:
make BIND_DIR=. DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestSaveRepoWithMultipleImages test-integration
INFO: Testing against a local daemon
=== RUN TestSaveRepoWithMultipleImages
--- PASS: TestSaveRepoWithMultipleImages (1.20s)
PASS
DONE 1 tests in 2.668s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When starting a daemon in debug mode (such as used in CI), many log-messages
are printed during startup. As a result, the log message indicating whether
graph-drivers or snapshotters are used may appear far separate from the
informational log about the daemon (and selected storage-driver).
The existing log-driver also unconditionally uses the legacy "graph-driver"
terminology, instead of the more generic "storage-driver".
This patch changes the log message shown during startup to use the generic
"graph-driver" as field, and adds a new field that indicates wheter we're
using snapshotters or graph-drivers.
Given that snapshotters will be the default at some point, an alternative
could be to include the _type_ of driver used, for example;
`io.containerd.snapshotter.v1`, which may continue to be relevant after
snapshotters become the default, and at which point (potentially) the
type of snapshotter becomes more relevant.
Before this change:
TEST_INTEGRATION_USE_SNAPSHOTTER=1 DOCKER_GRAPHDRIVER=overlayfs dockerd
...
INFO[2023-10-31T09:12:33.586269801Z] Starting daemon with containerd snapshotter integration enabled
INFO[2023-10-31T09:12:33.586322176Z] Loading containers: start.
INFO[2023-10-31T09:12:33.640514759Z] Loading containers: done.
INFO[2023-10-31T09:12:33.646498134Z] Docker daemon commit=dcf7287d647bcb515015e389df46ccf1e09855b7 graphdriver=overlayfs version=dev
INFO[2023-10-31T09:12:33.646706551Z] Daemon has completed initialization
INFO[2023-10-31T09:12:33.658840592Z] API listen on /var/run/docker.sock
With this change;
TEST_INTEGRATION_USE_SNAPSHOTTER=1 DOCKER_GRAPHDRIVER=overlayfs dockerd
...
INFO[2023-10-31T08:41:38.841155928Z] Starting daemon with containerd snapshotter integration enabled
INFO[2023-10-31T08:41:38.841207512Z] Loading containers: start.
INFO[2023-10-31T08:41:38.902461053Z] Loading containers: done.
INFO[2023-10-31T08:41:38.910535137Z] Docker daemon commit=dcf7287d647bcb515015e389df46ccf1e09855b7 containerd-snapshotter=true storage-driver=overlayfs version=dev
INFO[2023-10-31T08:41:38.910936803Z] Daemon has completed initialization
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These platforms are filled by default from containerd
introspection API and may not be normalized. Initializing
wrong platform in here results in incorrect platform
for BUILDPLATFORM and TARGETPLATFORM build-args for
Dockerfile frontend (and probably other side effects).
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Redirecting check-config.sh output to a file puts control character
output into that file, which isn't helpful for reading.
Disable colorized output if either
1. NO_COLOR environment is set to "1"
2. stdout is not a terminal.
Signed-off-by: Scott Moser <smoser@brickies.net>
In case of `docker push -a`, we need to return an error if there is no
image for the given repository.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't wrap the `no basic auth credentials` error from containerd and
return it as-is.
The error will look like:
```
failed to resolve reference "docker.io/library/aodkoakds:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The 403 error might not only be raised in swarm operations. It is
also returned when the given container is already connected to the
network and is currently running. I noticed this when during the
following PR: https://github.com/containers/podman/pull/20365
Signed-off-by: Philipp Fruck <dev@p-fruck.de>
When writing a tar file with archive/tar, extended attributes in the
deprecated (tar.Header).Xattrs map take precedence over conflicting
'SCHILY.xattr' records in the (tar.Header).PAXRecords map. Update
package tarsum to follow the same precedence rules as archive/tar.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Add a new `com.docker.network.host_ipv6` bridge option to compliment
the existing `com.docker.network.host_ipv4` option. When set to an
IPv6 address, this causes the bridge to insert `SNAT` rules instead of
`MASQUERADE` rules (assuming `ip6tables` is enabled). `SNAT` makes it
possible for users to control the source IP address used for outgoing
connections.
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
Kept `coci` import alias since we use it elsewhere,
maybe to prevent confusion with our own `oci` package.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Having a sandbox/container-wide MacAddress field makes little sense
since a container can be connected to multiple networks at the same
time. This field is an artefact of old times where a container could be
connected to a single network only.
As we now have a way to specify per-endpoint mac address, this field is
now deprecated.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Prior to this commit, only container.Config had a MacAddress field and
it's used only for the first network the container connects to. It's a
relic of old times where custom networks were not supported.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
- Merge BC conds for API < v1.42 together
- Merge BC conds for API < v1.44 together
- Re-order BC conds by API version
- Move pids-limit normalization after BC conds
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The same error is already returned by `(*Daemon).containerCreate()` but
since this function is also called by the cluster executor, the error
has to be duplicated.
Doing that allows to remove a nil check on container config in
`postContainersCreate`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
containerd's `WithUser` function now resets this property, starting with
[3eda46af12b1deedab3d0802adb2e81cb3521950][1] (v1.7.0-beta.4), so we no
longer need this function.
[1]: 3eda46af12
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The github.com/opencontainers/runc/libcontainer/user package was moved
to a separate module. While there's still uses of the old module in
our code-base, runc itself is migrating to the new module, and deprecated
the old package (for runc 1.2).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit def549c8f6 passed through the context
to the daemon.ContainerStart function. As a result, restarting containers
no longer is an atomic operation, because a context cancellation could
interrupt the restart (between "stopping" and "(re)starting"), resulting
in the container being stopped, but not restarted.
Restarting a container, or more factually; making a successful request on
the `/containers/{id]/restart` endpoint, should be an atomic operation.
This patch uses a context.WithoutCancel for restart requests.
It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).
Before this patch, starting a container that bind-mounts the docker socket,
then restarting itself from within the container would cancel the restart
operation. The container would be stopped, but not started after that:
docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
docker exec myself sh -c 'docker restart myself'
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3a2a741c65ff docker:cli "docker-entrypoint.s…" 26 seconds ago Exited (128) 7 seconds ago myself
With this patch: the stop still cancels the exec, but does not cancel the
restart operation, and the container is started again:
docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
docker exec myself sh -c 'docker restart myself'
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4393a01f7c75 docker:cli "docker-entrypoint.s…" About a minute ago Up 4 seconds myself
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Fix a silly bug in the implementation which had the effect of
len(h.Xattrs) blank entries being inserted in the middle of
orderedHeaders. Luckily this is not a load-bearing bug: empty headers
are ignored as the tarsum digest is computed by concatenating header
keys and values without any intervening delimiter.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The existing pkg/archive unit tests are primarily round-trip tests which
assert that pkg/archive produces tarballs which pkg/archive can unpack.
While these tests are effective at catching regressions in archiving or
unarchiving, they have a blind spot for regressions in compatibility
with the rest of the ecosystem. For example, a typo in the capabilities
extended attribute constant would result in subtly broken image layer
tarballs, but the existing tests would not catch the bug if both the
archiving and unarchiving implementations have the same typo.
Extend the test for archiving an overlay filesystem layer to assert that
the overlayfs style whiteouts (extended attributes and device files) are
transformed into AUFS-style whiteouts (magic file names).
Extend the test for archiving files with extended attributes to assert
that the extended attribute is encoded into the file's tar header in the
standard, interoperable format compatible with the rest of the
ecosystem.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Use context.WithoutCancel so that both the containerStop and
container.Wait can share the same parent context. This context is still
a "TODO", but can be wired up in future.
It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Follow-up to fc94ed0a86. Now that
f6e44bc0e8 added the compatcontext
package, we can start using context.WithoutCancel.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In the tagged case the error message when the image/tag is not found
should be "tag does not exist: ref"
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
While there is nothing inherently wrong with goto statements, their use
here is not helping with readability.
Signed-off-by: Cory Snider <csnider@mirantis.com>
ds.cache is never nil so the uncached code paths are unreachable in
practice. And given how many KVObject deep-copy implementations shallow
copy pointers and other reference-typed values, there is the distinct
possibility that disabling the datastore cache could break things.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The datastore cache only uses the reference to its datastore to get a
reference to the backing store. Modify the cache to take the backing
store reference directly so that methods on the datastore can't get
called, as that might result in infinite recursion between datastore and
cache methods.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Also removing some waitRun call, as they were not actually checked for
results, and the tests depended on that behavior (to get events about
the container starting etc).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When choosing the next image, don't reject images without the classic
builder parent label. The intention was to *prefer* images them instead
of making that a condition.
This fixes the ID not being filled for parent images that weren't built
with the classic builder.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The `Tags` slice of each history entry was filled with tags of parent
image. Change it to correctly assign the current image tags.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Check for accurate values that may contain content sizes unknown to the
usage test in the calculation. Avoid asserting using deep equals when
only the expected value range is known to the test.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Inline the tortured logic for deciding when to skip updating the svc
records to give us a fighting chance at deciphering the logic behind the
logic and spotting logic bugs.
Update the service records synchronously. The only potential for issues
is if this change introduces deadlocks, which should be fixed by
restrucuting the mutexes rather than papering over the issue with
sketchy hacks like deferring the operation to a goroutine.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Its only remaining purpose is to elide removing the endpoint from the
service records if it was not previously added. Deleting the service
records is an idempotent operation so it is harmless to delete service
records which do not exist.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The service db entry for each network is deleted by
(*Controller).cleanupServiceDiscovery() when the network is deleted.
There is no need to also eagerly delete it whenever the network's
endpoint count drops to zero.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The logic to rename an endpoint includes code which would synchronize
the renamed service records to peers through the distributed datastore.
It would trigger the remote peers to pick up the rename by touching a
datastore object which remote peers would have subscribed to events on.
The code also asserts that the local peer is subscribed to updates on
the network associated with the endpoint, presumably as a proxy for
asserting that the remote peers would also be subscribed.
https://github.com/moby/libnetwork/pull/712
Libnetwork no longer has support for distributed datastores or
subscribing to datastore object updates, so this logic can be deleted.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The meaning of the (*Controller).isDistributedControl() method is not
immediately clear from the name, and it does not have any doc comment.
It returns true if and only if the controller is neither a manager node
nor an agent node -- that is, if the daemon is _not_ participating in a
Swarm cluster. The method name likely comes from the old abandoned
datastore-as-IPC control plane architecture for libnetwork. Refactor
c.isDistributedControl() -> !c.isSwarmNode()
to make it easier to understand code which consumes the method.
Signed-off-by: Cory Snider <csnider@mirantis.com>
server: prohibit more than MaxConcurrentStreams handlers from running at once
(CVE-2023-44487).
In addition to this change, applications should ensure they do not leave running
tasks behind related to the RPC before returning from method handlers, or should
enforce appropriate limits on any such work.
- https://github.com/grpc/grpc-go/compare/v1.56.2...v1.56.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We support importing images for other platforms when
using the containerd image store, so we shouldn't validate
the image OS on import.
This commit also splits the test into two, so that we can
keep running the "success" import with a custom platform tests
running w/ c8d while skipping the "error/rejection" test cases.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
After a successful push, all pushed blobs should have a
distribution.source label pointing to the new registry.
Before this commit, the label was only appended to the top-level blob
(manifest or manifest list). Adjust this to also do that recursively to
its children.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use the distribution code to query the remote repository for tags and
pull them sequentially just like the non-c8d pull.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add a function to return tags for the given repository reference. This
is needed to implement the `pull -a` (pull all tags) for containerd
which doesn't directly use distribution, but we need to somehow make an
API call to the registry to obtain the available tags.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When the default bridge is disabled by setting dockerd's `--bridge=none`
option, the daemon still creates a sandbox for containers with no
network attachment specified. In that case `NetworkDisabled` will be set
to true.
However, currently the `releaseNetwork` call will early return if
NetworkDisabled is true. Thus, these sandboxes won't be deleted until
the daemon is restarted. If a high number of such containers are
created, the daemon would then take few minutes to start.
See https://github.com/moby/moby/issues/42461.
Signed-off-by: payall4u <payall4u@qq.com>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This is a follow-up to 2216d3ca8d, which
implemented the StartInterval for health-checks, but did not add validation
for the minimum accepted interval;
> The time to wait between checks in nanoseconds during the start period.
> It should be 0 or at least 1000000 (1 ms). 0 means inherit.
This patch adds validation for the minimum accepted interval (1ms).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While updating the docker/docker dependency in BuildKit, I noticed that the
dependency tree showed _two_ separate versions of the semconv package;
BuildKit and containerd were using the v1.17.0 version and docker/docker was
using v1.7.0.
This patch updates the version we use to align with BuildKit and containerd.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rename all variables/fields/map keys associated with the
`com.docker.network.host_ipv4` option from `HostIP` to `HostIPv4`.
Rationale:
* This makes the variable/field name consistent with the option
name.
* This makes the code more readable because it is clear that the
variable/field does not hold an IPv6 address. This will hopefully
avoid bugs like <https://github.com/moby/moby/issues/46445> in the
future.
* If IPv6 SNAT support is ever added, the names will be symmetric.
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
Rather than pass an `iptables.IPVersion` value alongside every
`iptRule` parameter, embed the IP version in the `iptRule` struct.
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
That field was only used to pass `-t nat` for NAT rules. Now `-t
<tableName>` (where `<tableName>` is one of the `iptables.Table`
values) is always passed, eliminating the need for `preArgs`.
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
Pass the entire `*networkConfiguration` struct to
`setupIPTablesInternal` to simplify the function signature and improve
code readability.
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
Use a unique parent view snapshot key for each diff request.
I considered using singleflight at first, but I realized it wouldn't
really be correct.
The diff can take some time, so there's a window of time between the
diff start and finish, where the file system can change.
These changes not always will be reflected in the running diff.
With singleflight, the second diff request which happened before the
previous diff was finished, would not include changes made to the
container filesystem after the first diff request has started.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Implement a behavior from the graphdriver's export where `docker save
something` (untagged reference) would export all images matching the
specified repository.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
On modern kernels this is an alias; however newer code has preferred
ctstate while older code has preferred the deprecated 'state' name.
Prefer the newer name for uniformity in the rules libnetwork creates,
and because some implementations/distributions of the xtables userland
tools may not support the legacy alias.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The image that this test pulls contains an error in the linux/amd64
manifest description, the reported size is 424 but the actual size is
524, making this test fail with containerd.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
go1.21.3 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.21.3 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.21.3+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.21.2...go1.21.3
From the security mailing:
[security] Go 1.21.3 and Go 1.20.10 are released
Hello gophers,
We have just released Go versions 1.21.3 and 1.20.10, minor point releases.
These minor releases include 1 security fixes following the security policy:
- net/http: rapid stream resets can cause excessive work
A malicious HTTP/2 client which rapidly creates requests and
immediately resets them can cause excessive server resource consumption.
While the total number of requests is bounded to the
http2.Server.MaxConcurrentStreams setting, resetting an in-progress
request allows the attacker to create a new request while the existing
one is still executing.
HTTP/2 servers now bound the number of simultaneously executing
handler goroutines to the stream concurrency limit. New requests
arriving when at the limit (which can only happen after the client
has reset an existing, in-flight request) will be queued until a
handler exits. If the request queue grows too large, the server
will terminate the connection.
This issue is also fixed in golang.org/x/net/http2 v0.17.0,
for users manually configuring HTTP/2.
The default stream concurrency limit is 250 streams (requests)
per HTTP/2 connection. This value may be adjusted using the
golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
setting and the ConfigureServer function.
This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
This is also tracked by CVE-2023-44487.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.21.2 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the compiler, the go command, the linker, the runtime,
and the runtime/metrics package. See the Go 1.21.2 milestone on our issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.21.2+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.21.1...go1.21.2
From the security mailing:
[security] Go 1.21.2 and Go 1.20.9 are released
Hello gophers,
We have just released Go versions 1.21.2 and 1.20.9, minor point releases.
These minor releases include 1 security fixes following the security policy:
- cmd/go: line directives allows arbitrary execution during build
"//line" directives can be used to bypass the restrictions on "//go:cgo_"
directives, allowing blocked linker and compiler flags to be passed during
compliation. This can result in unexpected execution of arbitrary code when
running "go build". The line directive requires the absolute path of the file in
which the directive lives, which makes exploting this issue significantly more
complex.
This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/golang/net/compare/v0.13.0...v0.17.0
This fixes the same CVE as go1.21.3 and go1.20.10;
- net/http: rapid stream resets can cause excessive work
A malicious HTTP/2 client which rapidly creates requests and
immediately resets them can cause excessive server resource consumption.
While the total number of requests is bounded to the
http2.Server.MaxConcurrentStreams setting, resetting an in-progress
request allows the attacker to create a new request while the existing
one is still executing.
HTTP/2 servers now bound the number of simultaneously executing
handler goroutines to the stream concurrency limit. New requests
arriving when at the limit (which can only happen after the client
has reset an existing, in-flight request) will be queued until a
handler exits. If the request queue grows too large, the server
will terminate the connection.
This issue is also fixed in golang.org/x/net/http2 v0.17.0,
for users manually configuring HTTP/2.
The default stream concurrency limit is 250 streams (requests)
per HTTP/2 connection. This value may be adjusted using the
golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
setting and the ConfigureServer function.
This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
This is also tracked by CVE-2023-44487.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.
This patch moves our own uses of the package to use the new module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Fix storageDriver gcs not registered in binaries
- reference: replace uses of deprecated function SplitHostname
- Dont parse errors as JSON unless Content-Type is set to JSON
- update to go1.20.8
- Set Content-Type header in registry client ReadFrom
- deprecate reference package, migrate to github.com/distribution/reference
- digestset: deprecate package in favor of go-digest/digestset
- Do not close HTTP request body in HTTP handler
full diff: https://github.com/distribution/distribution/compare/v2.8.2...v2.8.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`docker build --squash` is an experimental feature which is not
implemented for containerd image store.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
`docker.io` is present in the `IndexConfigs` so the `Mirrors` property
would get lost because a fresh `RegistryConfig` object was created.
Instead of creating a new object, reuse the existing one and just
mutate its fields.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Extract the distribution source label append into its own function and
make it not fail on any error, we do still log the error.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
These are not yet implemented with containerd snapshotters. We skip them
now because implementing this is not trivial with containerd.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
So far, internal networks were only isolated from the host by iptables
DROP rules. As a consequence, outbound connections from containers would
timeout instead of being "rejected" through an immediate ICMP dest/port
unreachable, a TCP RST or a failing `connect` syscall.
This was visible when internal containers were trying to resolve a
domain that don't match any container on the same network (be it a truly
"external" domain, or a container that don't exist/is dead). In that
case, the embedded resolver would try to forward DNS queries for the
different values of resolv.conf `search` option, making DNS resolution
slow to return an error, and the slowness being exacerbated by some libc
implementations.
This change makes `connect` syscall to return ENETUNREACH, and thus
solves the broader issue of failing fast when external connections are
attempted.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Path prefixes were originally disallowed in the `--registry-mirrors`
option because the /v1 endpoint was assumed to be at the root of the
URI. This is no longer the case in v2.
Close#36598
Signed-off-by: Régis Behmo <regis@behmo.com>
This change creates a few OTEL spans and plumb context through the DNS
resolver and DNS backends (ie. Sandbox and Network). This should help
better understand how much lock contention impacts performance, and
help debug issues related to DNS queries (we basically have no
visibility into what's happening here right now).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This isn't something that user should do, but technically the dangling
images exist in the image store and user can pass its name (`moby-dangling@digest`).
Change it so rmi now recognizes that it's actually a dangling image and
doesn't handle it like a regular tagged image.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- Pass empty containerd socket which forces the daemon to create a new
supervised containerd. Otherwise a global containerd daemon will be
used and the pulled image data will be stored in its data directory,
instead of the the newly specified `data-root` that has a limited
storage capacity.
- Don't try to use `vfs` snapshotter, instead use `native` which is
containerd's equivalent for `vfs`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Instead of passing a completely fresh context without any values, just
discard the cancellation.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This reverts commit a9fa147a92.
The commit is unfortunately broken as it is still using `providerHandle`
to write events but that handle is never actually set, so it is always
invalid. All logging fails.
Note: This is note a straight revert due to the change to
containerd/log.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
To match the graphdriver's push behavior which only shows the progress
for layers.
Exclude indexes, manifests and image configs from the push progress.
Don't explicitly check for `IsLayerType` to also handle other
potentially big blobs (like buildkit attestations).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This was mistakenly added to bklog.
Since this is getting attached to the standard logger, and bklog is
using the standard logger, we only need this added once.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Fix issue 46563 "Rootful-in-Rootless dind doesn't work since systemd v250 (due to oom score adj)"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Before this commit, `doPack`, `doUnpack` and `doUnpackLayer` were not implemented for Darwin, causing build failure.
This change allows all non-Linux Unixes to use FreeBSD reexec-based pack/unpack implementation
See also: moby/buildkit#4059
See also: 8b843732b3
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
The test was depending on the client constructing an error based on the
http-status code, and the client not reading the response body if the
response was not a JSON response.
This fix;
- adds the correct content-type headers in the response
- includes error-messages in the response
- adds additional tests to cover both the plain (non-JSON) and JSON
error responses, as well as an empty response.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was setting the content-type header after WriteHeader was
called, and the header was not sent because of that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use named return variables to make the function more self-describing
- rename variable for readability
- slightly optimize slice initialization, and keep linters happy
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Copy the implementation of `context.WithoutCancel` introduced in Go 1.21
to be able to use it when building with older versions.
This will use the stdlib directly when building with Go 1.21+.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This commit moves one-shot stats processing out of the publishing
channels, i.e. collect stats directly.
Also changes the method of getSystemCPUUsage() on Linux to return
number of online CPUs also.
Signed-off-by: Xinfeng Liu <XinfengLiu@icloud.com>
It was only used in a single place, and it was defined far away from
where it was used.
Move the code inline, so that it's clear at a glance what it's doing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The store field is only mutated by Controller.initStores(), which is
only called inside the cosntructor (libnetwork.New), so there should be
no need to protect the field with a mutex in non-exported functions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Controller.getNetworkFromStore() already returns a ErrNoSuchNetwork if
no network was found, so we don't need to convert the existing error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, all traces coming from the API have an empty operation
string, which make them indistinguishable from each other without looking
at the logs of the root span, and prevent proper filtering on Jaeger UI.
With this change, traces get the route pattern as the operation string.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The reason it doesn't change with the graphdrivers is caused by an
implementation detail and the fact that the image is loaded into the
same daemon it was saved from.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Rewrite TestSaveMultipleNames and TestSaveSingleTag so that they don't
use legacy `repositories` file (which isn't present in the OCI
archives).
`docker save` output is now OCI compatible, so we don't need
to use the legacy file.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The input archive is in the old Docker format that's not OCI compatible
and is not supported by the containerd archive import:
```
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/VERSION
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/json
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/layer.tar
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/VERSION
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/json
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/layer.tar
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/VERSION
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/json
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/layer.tar
repositories
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Now that this is a generic, we can define a struct type at the package
level, and remove the casting logic necessary when we had to use
interface{}.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
SourcePolicy was accounted for in 330cf7ae7d
TODO: replace applySourcePolicies with BuildKit's implementation, which
is currently unexported.
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:
* containerd's LeaseManager and ContentStore have to be a
(namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The following changes were required:
* integration/build: progressui's signature changed in 6b8fbed01e
* builder-next: flightcontrol.Group has become a generic type in 8ffc03b8f0
* builder-next/executor: add github.com/moby/buildkit/executor/resources types, necessitated by 6e87e4b455
* builder-next: stub util/network/Namespace.Sample(), necessitated by 963f16179f
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The DeepEqual ignore required in the daemon tests is a bit ugly, but it
works given the new protoc output.
We also have to ignore lints related to schema1 deprecations; these do
not apply as we must continue to support this schema version.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The current executor is only tested on Linux, so let's be honest about
that. Stubbing this correctly helps avoid incorrectly trying to call
into Linux-only code in e.g. libnetwork.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Simplify the lock/unlock cycle, and make the "lookupAlias" branch
more similar to the non-lookupAlias variant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Skip faster when we're looking for aliases. Also check for the list
of aliases to be empty, not just `nil` (although in practice it should
be equivalent).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use `nameOrAlias` for the name (or alias) to resolve
- use `lookupAlias` to indicate what the intent is; this function
is either looking up aliases or "regular" names. Ideally we would
split the function, but let's keep that for a future exercise.
- name the `ipv6Miss` output variable. The "ipv6 miss" logic is rather
confusing, and should probably be revisited, but let's start with
giving the variable a name to make it more apparent what it is.
- use `nw` for networks, which is the more common local name
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove some intermediate vars, or move them closer to where they're used.
- ResolveService: use strings.SplitN to limit number of elements. This
code is only used to validate the input, results are not used.
- ResolveService: return early instead of breaking the loop. This makes
it clearer from the code that were not returning anything (nil, nil).
- Controller.sandboxCleanup(): rename a var, and slight refactor of
error-handling.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type was introduced in
0a79e67e4f
Make use of it throughout our log-format handling code, and convert back
to a string before we pass it to the containerd client.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This function was used to check if the network is a multi-host, swarm-scoped
network. Part of this check involved a check whether the cluster-agent was
present.
In all places where this function was used, the next step after checking if
the network was "cluster eligible", was to get the agent, and (again) check
if it was not nil.
This patch rewrites the isClusterEligible utility into a clusterAgent utility,
which both checks if the network is cluster-eligible, and returns the agent
(if set). For convenience, an "ok" bool is added, which callers can use to
return early (although just checking for nilness would likely have been
sufficient).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes redundant nil-checks in Endpoint.deleteServiceInfoFromCluster
and Endpoint.addServiceInfoToCluster.
These functions return early if the network is not ["cluster eligible"][1],
and the function used for that (`Network.isClusterEligible`) requires the
[agent to not be `nil`][2].
This check moved around a few times ([3][3], [4][4]), but was originally
added in [libnetwork 1570][5] which, among others, tried to avoid a nil-pointer
exception reported in [moby 28712][6], which accessed the `Controller.agent`
[without locking][7]. That issue was addressed by adding locks, adding a
`Controller.getAgent` accessor, and updating deleteServiceInfoFromCluster
to use a local var. It also sprinkled this `nil` check to be on the safe
side, but as `Network.isClusterEligible` already checks for the agent
to not be `nil`, this should not be redundant.
[1]: 5b53ddfcdd/libnetwork/agent.go (L529-L534)
[2]: 5b53ddfcdd/libnetwork/agent.go (L688-L696)
[3]: f2307265c7
[4]: 6426d1e66f
[5]: 8dcf9960aa
[6]: https://github.com/moby/moby/issues/28712
[7]: 75fd88ba89/vendor/github.com/docker/libnetwork/agent.go (L452)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When graphdriver is not provided the graphdriver is looked up
from docker info, but without quotes it may fail and set the
graphdriver to an incorrect value.
Signed-off-by: Derek McGowan <derek@mcg.dev>
It's not set when containerd is used as an image store and buildkit
never sets it either, so let's skip this test if snapshotters are used
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
We try to perform API-version negotiation as lazy as possible (and only execute
when we are about to make an API request). However, some code requires API-version
dependent handling (to set options, or remove options based on the version of the
API we're using).
Currently this code depended on the caller code to perform API negotiation (or
to configure the API version) first, which may not happen, and because of that
we may be missing options (or set options that are not supported on older API
versions).
This patch:
- splits the code that triggered API-version negotiation to a separate
Client.checkVersion() function.
- updates NewVersionError to accept a context
- updates NewVersionError to perform API-version negotiation (if enabled)
- updates various Client functions to manually trigger API-version negotiation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
moby and containerd have slightly different error messages when someone
tries to pull an image that doesn't contain the current platform,
instead of looking inside the error returned by containerd we match the
errors in the test related to what image backend we are using
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Diffing a container yielded some extra changes that come from the
files/directories that we mount inside the container (/etc/resolv.conf
for example). To avoid that we create an intermediate snapshot that has
these files, with this we can now diff the container fs with its parent
and only get the differences that were made inside the container.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Final progress messages were sent after the progress updater finished
which made the "Downloading" progress not being updated into "Download
complete".
Fix by sending the final messages after the progress has finished.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
It's still not "great", but implement a `newInterface()` constructor
to create a new Interface instance, instead of creating a partial
instance and applying "options" after the fact.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We're only using the results if the interface doesn't have an address
yet, so skip this step if we don't use it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Flatten some nested "if"-statements, and improve error.
Errors returned by this function are not handled, and only logged, so
make them more informative if debugging is needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
They were not consistently used, and the locations where they were
used were already "setters", so we may as well inline the code.
Also updating Namespace.Restore to keep the lock slightly longer,
instead of locking/unlocking for each property individually, although
we should consider to keep the long for the duration of the whole
function to make it more atomic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make the mutex internal to the Namespace; locking/unlocking should not
be done externally, and this makes it easier to see where it's used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Interface.Remove() was directly accessing Namespace "internals", such
as locking/unlocking. Move the code from Interface.Remove() into the
Namespace instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We weren't checking for the asked platform in the case the image was a
manifest, only if it was a manifest list.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Makes it possible to pull `application/vnd.docker.distribution.manifest.v1+prettyjws`
legacy manifests.
They are not stored in their original form but are converted to the OCI
manifests.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
While this is not strictly necessary as the default OCI config masks this
path, it is possible that the user disabled path masking, passed their
own list, or is using a forked (or future) daemon version that has a
modified default config/allows changing the default config.
Add some defense-in-depth by also masking out this problematic hardware
device with the AppArmor LSM.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The ability to read these files may offer a power-based sidechannel
attack against any workloads running on the same kernel.
This was originally [CVE-2020-8694][1], which was fixed in
[949dd0104c496fa7c14991a23c03c62e44637e71][2] by restricting read access
to root. However, since many containers run as root, this is not
sufficient for our use case.
While untrusted code should ideally never be run, we can add some
defense in depth here by masking out the device class by default.
[Other mechanisms][3] to access this hardware exist, but they should not
be accessible to a container due to other safeguards in the
kernel/container stack (e.g. capabilities, perf paranoia).
[1]: https://nvd.nist.gov/vuln/detail/CVE-2020-8694
[2]: 949dd0104c
[3]: https://web.eece.maine.edu/~vweaver/projects/rapl/
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Return the number of containers that use an image if it was asked,
during a `docker system df` call for example.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
This issue wasn't caught on ContainerCreate or NetworkConnect (when
container wasn't started yet).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Thus far, validation code would stop as soon as a bad value was found.
Now, we try to validate as much as we can, to return all errors to the
API client.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
So far, only a subset of NetworkingConfig was validated when calling
ContainerCreate. Other parameters would be validated when the container
was started. And the same goes for EndpointSettings on NetworkConnect.
This commit adds two validation steps:
1. Check if the IP addresses set in endpoint's IPAMConfig are valid,
when ContainerCreate and ConnectToNetwork is called ;
2. Check if the network allows static IP addresses, only on
ConnectToNetwork as we need the libnetwork's Network for that and it
might not exist until NetworkAttachment requests are sent to the
Swarm leader (which happens only when starting the container) ;
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Just return a regular error, because the API converts the error to
the expected ErrorResponse. Before/After produce the same API response:
curl -v --unix-socket /var/run/docker.sock 'http://localhost/v1.43/images/search?term=hello'
* Trying /var/run/docker.sock:0...
* Connected to localhost (/var/run/docker.sock) port 80 (#0)
> GET /v1.43/images/search?term=hello HTTP/1.1
> Host: localhost
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.44
< Content-Type: application/json
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/dev (linux)
< Traceparent: 00-c38c2da5cf30305fcb66836a28e227bf-d16f4f7d2c7002a1-01
< Date: Mon, 18 Sep 2023 14:30:18 GMT
< Content-Length: 41
<
{"message":"Unexpected status code 409"}
* Connection #0 to host localhost left intact
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make `PullImage` accept `reference.Named` directly instead of
duplicating the parsing code for both graphdriver and containerd image
service implementations.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This test checks how the layer store works, so we don't need it when we
use containerd as image store
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
This package was introduced in af59752712
as a utility package for devicemapper, which was removed in commit
dc11d2a2d8 (v25.0.0).
It looks like there's no external consumers of this package, so we should
consider removing it, but deprecating it first, just in case.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use local variables and remove some intermediate variables
- handle the events inside the switch itself; this makes all the
switch branches use the same logic, instead of "some" using
a `continue`, and others falling through to have the event handled
outside of the switch.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We were sending the "Pulling from ..." message too early, if the pull
progress wasn't able to resolve the image we wouldn't sent the error
back. Sending that first message would have flushed the output stream
and image_routes.go would return a nil error.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
full diff: https://github.com/containerd/containerd/compare/v1.6.22...v1.6.24
v1.6.24 release notes:
full diff: https://github.com/containerd/containerd/compare/v1.6.23...v1.6.24
The twenty-fourth patch release for containerd 1.6 contains various fixes
and updates.
Notable Updates
- CRI: fix leaked shim caused by high IO pressure
- Update to go1.20.8
- Update runc to v1.1.9
- Backport: add configurable mount options to overlay snapshotter
- log: cleanups and improvements to decouple more from logrus
v1.6.23 release notes:
full diff: https://github.com/containerd/containerd/compare/v1.6.22...v1.6.23
The twenty-third patch release for containerd 1.6 contains various fixes
and updates.
Notable Updates
- Add stable ABI support in windows platform matcher + update hcsshim tag
- cri: Don't use rel path for image volumes
- Upgrade GitHub actions packages in release workflow
- update to go1.19.12
- backport: ro option for userxattr mount check + cherry-pick: Fix ro mount option being passed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add "The push referers to repository X" message which is present in the
push output when using the graphdrivers.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Update the version used in testing;
full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.6
v1.7.6 release notes:
full diff: https://github.com/containerd/containerd/compare/v1.7.5...v1.7.6
The sixth patch release for containerd 1.7 contains various fixes and updates.
- Fix log package for clients overwriting the global logger
- Fix blockfile snapshotter copy on Darwin
- Add support for Linux usernames on non-Linux platforms
- Update Windows platform matcher to invoke stable ABI compability function
- Update Golang to 1.20.8
- Update push to inherit distribution sources from parent
v1.7.5 release notes:
full diff: https://github.com/containerd/containerd/compare/v1.7.4...v1.7.5
The fifth patch release for containerd 1.7 fixes a versioning issue from
the previous release and includes some internal logging API changes.
v1.7.4 release notes:
full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.4
The fourth patch release for containerd 1.7 contains remote differ plugin support,
a new block file based snapshotter, and various fixes and updates.
Notable Updates
- Add blockfile snapshotter
- Add remote/proxy differ
- Update runc binary to v1.1.9
- Cri: Don't use rel path for image volumes
- Allow attaching to any combination of stdin/out/err
- Fix ro mount option being passed
- Fix leaked shim caused by high IO pressure
- Add configurable mount options to overlay snapshotter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The API endpoint `/containers/create` accepts several EndpointsConfig
since v1.22 but the daemon would error out in such case. This check is
moved from the daemon to the api and is now applied only for API < 1.44,
effectively allowing the daemon to create containers connected to
several networks.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
When registry token is provided, the authorization header can be
directly applied to the registry request. No other type of
authorization will be attempted when the registry token is provided.
Signed-off-by: Derek McGowan <derek@mcg.dev>
We occassionally receive contributions to this script that are outside
its intended scope. Let's add a comment to the script that outlines
what it's meant for, and a link to a GitHub ticket with alternatives.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Synchronize the code to do the same thing as Exec.
reap doesn't need to be called before the start event was sent.
There's already a defer block which cleans up the process in case where
an error occurs.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Error check in defer block used wrong error variable which is always nil
if the flow reaches the defer. This caused the `newProcess.Kill` to be
never called if the subsequent attemp to attach to the stdio failed.
Although this only happens in Exec (as Start does overwrite the error),
this also adjusts the Start to also use the returned error to avoid this
kind of mistake in future changes.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
None of the code using this function was setting the value, so let's
simplify and remove the argument.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When the daemon process or the host running it is abruptly terminated,
the layer metadata file can become inconsistent on the file system.
Specifically, `link` and `lower` files may exist but be empty, leading
to overlay mounting errors during layer extraction, such as:
"failed to register layer: error creating overlay mount to <path>:
too many levels of symbolic links."
This commit introduces the use of `AtomicWriteFile` to ensure that the
layer metadata files contain correct data when they exist on the file system.
Signed-off-by: Mike <mike.sul@foundries.io>
Fixes#18864, #20648, #33561, #40901.
[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:
> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.
And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:
> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).
This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.
To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.
Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):
> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.
To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.
[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Windows doesn't support "FROM scratch", and the platform was only used
for validation on other platforms if a platform was provided, so no need
to set defaults.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
strong-type the fields with the expected type, to make it more explicit
what we're expecting here.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
PR 4f47013feb added a validation step to `NetworkCreate` to ensure
no IPv6 subnet could be set on a network if its `EnableIPv6` parameter
is false.
Before that, the daemon was accepting such request but was doing nothing
with the IPv6 subnet.
This validation step is now deleted, and we automatically set
`EnableIPv6` if an IPv6 subnet was specified.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Use `ImageService.unpackImage` when we want to unpack an image and we
know the exact platform-manifest to be unpacked beforehand.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
DiffID is only a digest of the one tar layer and matches the snapshot ID
only for the first layer (DiffID = ChainID).
Instead of generating random ID as a key for rolayer, just use the
snapshot ID of the unpacked image content and use it later as a parent
for creating a new RWLayer.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
diffID is the digest of a tar archive containing changes to the parent
layer - rolayer doesn't have any changes to the parent.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8
From the security mailing:
[security] Go 1.21.1 and Go 1.20.8 are released
Hello gophers,
We have just released Go versions 1.21.1 and 1.20.8, minor point releases.
These minor releases include 4 security fixes following the security policy:
- cmd/go: go.mod toolchain directive allows arbitrary execution
The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
execute scripts and binaries relative to the root of the module when the "go"
command was executed within the module. This applies to modules downloaded using
the "go" command from the module proxy, as well as modules downloaded directly
using VCS software.
Thanks to Juho Nurminen of Mattermost for reporting this issue.
This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.
- html/template: improper handling of HTML-like comments within script contexts
The html/template package did not properly handle HMTL-like "<!--" and "-->"
comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
cause the template parser to improperly interpret the contents of <script>
contexts, causing actions to be improperly escaped. This could be leveraged to
perform an XSS attack.
Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
issue.
This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.
- html/template: improper handling of special tags within script contexts
The html/template package did not apply the proper rules for handling occurrences
of "<script", "<!--", and "</script" within JS literals in <script> contexts.
This may cause the template parser to improperly consider script contexts to be
terminated early, causing actions to be improperly escaped. This could be
leveraged to perform an XSS attack.
Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
issue.
This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.
- crypto/tls: panic when processing post-handshake message on QUIC connections
Processing an incomplete post-handshake message for a QUIC connection caused a panic.
Thanks to Marten Seemann for reporting this issue.
This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
schema1 was deprecated a while ago, containerd fails to push to a
schema1 registry, let's just skip these tests for the containerd
integration
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Graph drivers create the parent directory with
rootPair().GID:CurrentIdentity().UID owner. This change brings these in
line
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Constants for both platform-specific and platform-independent networks
are added to the api/network package.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Before this commit, setting the `com.docker.network.host_ipv4` bridge
option when `enable_ipv6` is true and the experimental `ip6tables`
option is enabled would cause Docker to fail to create the network:
> failed to create network `test-network`: Error response from daemon:
> Failed to Setup IP tables: Unable to enable NAT rule: (iptables
> failed: `ip6tables --wait -t nat -I POSTROUTING -s fd01::/64 ! -o
> br-test -j SNAT --to-source 192.168.0.2`: ip6tables
> v1.8.7 (nf_tables): Bad IP address "192.168.0.2"
>
> Try `ip6tables -h` or `ip6tables --help` for more information.
> (exit status 2))
Fix this error by passing nil -- not the `host_ipv4` address -- when
creating the IPv6 rules.
Signed-off-by: Richard Hansen <rhansen@rhansen.org>
- store linkIndex in a local variable so that it can be reused
- remove / rename some intermediate vars that shadowed existing declaration
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This argument was originally added in libnetwork:
03f440667f
At the time, this argument was conditional, but currently it's always set
to "true", so let's remove it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The code ignores these errors, but will unconditionally print a warning;
> If the kernel deletion fails for the neighbor entry still remote it
> from the namespace cache. Otherwise if the neighbor moves back to the
> same host again, kernel update can fail.
Let's reduce noise if the neighbor wasn't found, to prevent logs like:
Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.186662370+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.366585939+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
Aug 16 13:26:42 master1.local dockerd[4019880]: time="2023-08-16T13:26:42.366658513+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
While changing this code, also slightly rephrase the code-comment, and
fix a typo ("remote -> remove").
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork/osl: Namespace.DeleteNeighbor: rephrase code-comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
container.Run() should be a synchronous operation in normal circumstances;
the container is created and started, so polling after that for the
container to be in the "running" state should not be needed.
This should also prevent issues when a container (for whatever reason)
exited immediately after starting; in that case we would continue
polling for it to be running (which likely would never happen).
Let's skip the polling; if the container is not in the expected state
(i.e. exited), tests should fail as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
validateEndpoint was doing more than just validating; it was also implicitly
mutating the endpoint that was passed to it (by reference).
Given that validation only happend when constructing a new v1Endpoint, let's
merge these functions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon would pass an EndpointCreateOption to set the interface MAC
address if the network name and the provided network mode were matching.
Obviously, if the network mode is a network ID, it won't work. To make
things worse, the network mode is never normalized if it's a partial ID.
To fix that: 1. the condition under what the container's mac-address is
applied is updated to also match the full ID; 2. the network mode is
normalized to a full ID when it's only a partial one.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Without these compile flags, Delve is unable to report the value of some
variables and it's not possible to jump into inlined code.
As the contributing docs already mention that `DOCKER_DEBUG` should
disable "build optimizations", the env var is reused here instead of
introducing a new one.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
validateEndpoint uses `v1Endpoint.ping` to verify if the search API can
use a secure connection, and to fall back to basic auth. For Docker Hub,
we don't allow insecure connections, and `v1Endpoint.ping` will not connect
to Docker Hub (Docker Hub also does not implement the `_ping` endpoint,
so doing so would always fail).
Let's make it more clear that we don't do any validation, and return
early.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We have the response available, which is an io.Reader, so we don't have
to read the entire response into memory before decoding.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was making a request to the `_ping` endpoint, which (if
implemented) would return a JSON response, which we unmarshal (the only
field we use from the response is the `Standalone` field).
However, if the response had a `X-Docker-Registry-Standalone`, that header
took precedence, and would overwrite the earlier `Standalone` value we
obtained from the JSON response.
This patch adds a fast-path for situations where the header is present,
in which case we can skip handling the JSON response altogether.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We don't really want the daemon to panic for this so let's log a warning
about max downloads and uploads
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
With containerd image store the images don't depend on each other even
if they share the same content and it's totally fine to delete the
"parent" image.
The skip is necessary because deleting the "parent" image does not
produce an error with the c8d image store and deleting the `busybox`
image breaks other tests.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- The `Version` field was not used for any purpose, other than a debug log
- The `X-Docker-Registry-Version` header was part of the registry v1 spec,
however, as we're not using the `Version` field, we don't need the
header for anything.
- The `X-Docker-Registry-Config` header was only set by the mock registry;
there's no code consuming it, so we don't need to mock it (even if an
actual v1 registry / search API would return it).
It's also worth noting that we never call the `_ping` endpoint when using
Docker Hub's search API, and Docker Hub does not even implement the `_ping`
endpoint;
curl -fsSL https://index.docker.io/_ping | head -n 4
<!DOCTYPE html>
<html lang="en">
<head>
<title>Docker</title>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.
Using process substitution makes the integration tests quit.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Images built by classic builder will have an additional label (in the
containerd image object, not image config) pointing to a parent of that
image.
This allows to differentiate intermediate images (dangling
images created as a result of a each Dockerfile instruction) from the
final images.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Reduce some of the boiler-plating, and by combining the tests, we skip
the testenv.Clean() in between each of the tests. Performance gain isn't
really measurable, but every bit should help :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
container.Run should be an synchronous operation; the container should
be running after the request was made (or produce an error). Simplify
these tests, and remove the redundant polling.
These were added as part of 8f800c9415,
but no such polls were in place before the refactor, and there's no
mention of these during review of the PR, so I assume these were just
added either as a "precaution", or a result of "copy/paste" from another
test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was failing frequently on Windows, where the test was waiting
for the container to exit before continuing;
=== FAIL: github.com/docker/docker/integration/container TestResizeWhenContainerNotStarted (18.69s)
resize_test.go:58: timeout hit after 10s: waiting for container to be one of (exited), currently running
It looks like this test is merely validating that a container in any non-
running state should produce an error, so there's no need to run a container
(waiting for it to stop), and just "creating" a container (which would be
in `created` state) should work for this purpose.
Looking at 8f800c9415, I see `createSimpleContainer`
and `runSimpleContainer` utilities were added, so I'm even wondering if the
original intent was to use `createSimpleContainer` for this test.
While updating, also check if we get the expected error-type, instead of
only checking for the error-message.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.
The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This wires up the integration tests to export spans to a jager instance.
After tests are finished it exports the data out of jaeger and uploads
as an artifact to the action run.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.
Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- check if we have to download layers and print the approriate message
- show the digest of the pulled manifest(list)
- skip pulling if we already have the right manifest
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
The compressor is already closed a few lines below and there's no error
returns between so the defer is not needed.
Calling Close twice on a writerCloserWrapper is unsafe as it causes it
to put the same buffer to the pool multiple times.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When running a `docker cp` to copy files to/from a container, the
lookup of the `getent` executable happens within the container's
filesystem, so we cannot re-use the results.
Unfortunately, that also means we can't preserve the results for
any other uses of these functions, but probably the lookup should not
be "too" costly.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This causes the test to have a saner error message when the `images
-q` returns multiple images separated by newline.
Before this the test would fail with `invalid reference format` when
parsing the multiline string as an image reference.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The test-file had a duplicate definition for ErrNotImplemented, which
caused an error in this package, and was not used otherwise, so we can
remove this file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Updated the description to clarify that this is the endpoint to use if
you want to pull an image.
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
This makes the c8d code which creates/reads OCI types not lose
Docker-specific features like ONBUILD or Healthcheck.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Parent is a graph-driver only field which is stored in the ImageStore.
It's not available when using containerd snapshotters.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
These were dependent on the DOCKER_ENGINE_GOARCH environment variable
but this var was no longer set. There was also some weird check to see
if the architecture is "windows" which doesn't make sense. Seeing how
nothing failed ever since the TIMEOUT was no longer platform-dependent
we can safely remove this check.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
The test considered `Foo/bar` to be an invalid name, with the assumption
that it was `[docker.io]/Foo/bar`. However, this was incorrect, and the
test passed because the reference parsing had a bug; if the first element
(`Foo`) is not lowercase (so not a valid namespace / "path element"), then
it *should* be considered a domain (as uppercase domain names are valid).
The reference parser did not account for this, and running the test with
a version of the parser with a fix caused the test to fail:
=== Failed
=== FAIL: client TestImageTagInvalidSourceImageName/invalidRepo/FOO/bar (0.00s)
image_tag_test.go:54: assertion failed: expected error to contain "not a valid repository/tag", got "Error response from daemon: client should not have made an API call"
Error response from daemon: client should not have made an API call
=== FAIL: client TestImageTagInvalidSourceImageName (0.00s)
This patch removes the faulty test-case.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add back files at the old locations, as there may be external links
referencing the specification.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This merges the v1.2 specs to provide a single history of the
specification.
To view the combined history:
git log --follow image/spec/spec.md
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was added in b18ae8c9cc, which
was part of v1.12.0-rc1 and later, which used image spec v1.2.0.
This patch amends the v1.2 spec to include the missing field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was added in 9db5db1b94, which
was part of v1.10.0-rc1 and later, which used image spec v1.1.0.
It's worth noting that documentation for the v1.1.0 image spec was not
yet available until commit 4fa0eccd10,
which was included in v1.12.0-rc1 and up. The `ArgsEscaped` field was
also adopted by the OCI image spec since [v1.1.0-rc3][1], but considered
deprecated, and not recommended to be used.
This patch amends the v1.1 and v1.2 specifications to describe the field.
[1]: 59780aa569
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was added in commit 9f994c9646,
which was merged before the image-spec v1.0.0 was released (which happened
in commit 79910625f0).
This patch backfills the specifications to describe the property.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove some trailing commas, which made the JSON invalid (some of these
were fixed in the 1.2 spec, but not in older versions).
- synchronise some formatting / phrasing between versions, to make them
easier to compare.
- remove non-breaking spaces (`NBSP`) in example outputs, and replace
them with regular spaces.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While there's not much we can do if we failed to store a snapshot of the
container's state, let's log the error in case it happens in stad of discarding.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Daemon.handleContainerExit() returns an error if snapshotting the container's
state to disk fails. There's not much we can do with the error if it occurs,
but let's log the error if that happens, instead of discarding it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function either had to create a new StaticRoute, or add the destination
to the list of routes. Skip creating a StaticRoute struct if we're not
gonna use it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Tests are failing with this error:
E ValueError: scheme http+docker is invalid
Which is reported in docker-py in https://github.com/docker/docker-py/issues/1478.
Not sure what changed in the tests, but could be due to updated Python
version or dependencies, but let's skip it for now.
Test failure:
___________ AttachContainerTest.test_run_container_reading_socket_ws ___________
tests/integration/api_container_test.py:1245: in test_run_container_reading_socket_ws
pty_stdout = self.client.attach_socket(container, opts, ws=True)
docker/utils/decorators.py:19: in wrapped
return f(self, resource_id, *args, **kwargs)
docker/api/container.py:98: in attach_socket
return self._attach_websocket(container, params)
docker/utils/decorators.py:19: in wrapped
return f(self, resource_id, *args, **kwargs)
docker/api/client.py:312: in _attach_websocket
return self._create_websocket_connection(full_url)
docker/api/client.py:315: in _create_websocket_connection
return websocket.create_connection(url)
/usr/local/lib/python3.7/site-packages/websocket/_core.py:601: in create_connection
websock.connect(url, **options)
/usr/local/lib/python3.7/site-packages/websocket/_core.py:245: in connect
options.pop('socket', None))
/usr/local/lib/python3.7/site-packages/websocket/_http.py:117: in connect
hostname, port, resource, is_secure = parse_url(url)
/usr/local/lib/python3.7/site-packages/websocket/_url.py:62: in parse_url
raise ValueError("scheme %s is invalid" % scheme)
E ValueError: scheme http+docker is invalid
------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in 1980deffae, which
changed the implementation, but forgot to update imports in these.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There is no meaningful distinction between driverapi.Registerer and
drvregistry.DriverNotifyFunc. They are both used to register a network
driver with an interested party. They have the same function signature.
The only difference is that the latter could be satisfied by an
anonymous closure. However, in practice the only implementation of
drvregistry.DriverNotifyFunc is the
(*libnetwork.Controller).RegisterDriver method. This same method also
makes the libnetwork.Controller type satisfy the Registerer interface,
therefore the DriverNotifyFunc type is redundant. Change
drvregistry.Networks to notify a Registerer and drop the
DriverNotifyFunc type.
Signed-off-by: Cory Snider <csnider@mirantis.com>
On startup all local volumes were unmounted as a cleanup mechanism for
the non-clean exit of the last engine process.
This caused live-restored volumes that used special volume opt mount
flags to be broken. While the refcount was restored, the _data directory
was just unmounted, so all new containers mounting this volume would
just have the access to the empty _data directory instead of the real
volume.
With this patch, the mountpoint isn't unmounted. Instead, if the volume
is already mounted, just mark it as mounted, so the next time Mount is
called only the ref count is incremented, but no second attempt to mount
it is performed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
A package only needs one "import" comment to enforce, so keeping
one in the go.doc.
It should be noted that even with that; in most cases, go will ignore
these comments (if go modules are used, even in "vendor" mode).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was only used in a single test, and was not using any of
the gotest.tools features, so let's remove it as dependency.
With this, the package has no external dependencies (only stdlib).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Log a warning if we encounter an error when releasing leases. While it
may not have direct consequences, failing to release the lease should be
unexpected, so let's make them visible.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was used when the code supported both "v1" and "v2" registries.
We no longer support v1 registries, and the only v1 endpoint that's still
used is for the legacy "search" endpoint, which does not use the APIEndpoint
type.
As no code is using this field, and the value will always be set to "v2",
we can deprecated the Version field.
I'm keeping this field for 1 release, to give notice to any potential
external consumer, after which we can delete it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure that the content in the live-restored volume mounted in a new
container is the same as the content in the old container.
This checks if volume's _data directory doesn't get unmounted on
startup.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Explain that search is not supported on v2 endpoints, and include the
offending endpoint in the error-message.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
First, remove the loop over `apiVersions`. The `apiVersions` map has two
entries (`APIVersion1 => "v1"`, and `APIVersion2 => "v2"`), and `APIVersion1`
is skipped, which means that the loop effectively translates to;
if apiVersionStr == "v2" {
return "", invalidParamf("unsupported V1 version path %s", apiVersionStr)
}
Which leaves us with "anything else" being returned as-is.
This patch removes the loop, and replaces the remaining handling to check
for the "v2" suffix to produce an error, or to strip the "v1" suffix.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Combine the two tests into a TestV1EndpointParse function, and rewrite
them to use gotest.tools for asserting.
Also changing the test-cases to use "https://", as the scheme doesn't
matter for this test, but using "http://" may trip-up some linters,
so let's avoid that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Define consts for the Actions we use for events, instead of "ad-hoc" strings.
Having these consts makes it easier to find where specific events are triggered,
makes the events less error-prone, and allows documenting each Action (if needed).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 70ad5b818f changed event.Type
to be a strong type, no longer an alias for string. for some reason,
this test passed on the PR, but failed later on;
=== Failed
=== FAIL: daemon/events TestLoadBufferedEventsOnlyFromPast (0.00s)
events_test.go:203: assertion failed: network (messages[0].Type events.Type) != network (string)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make the error message slightly clearer on "what" part is not valid,
and provide suggestions on what are acceptable values.
Before this change:
docker create --restart=always:3 busybox
Error response from daemon: invalid restart policy: maximum retry count cannot be used with restart policy 'always'
docker create --restart=always:-1 busybox
Error response from daemon: invalid restart policy: maximum retry count cannot be used with restart policy 'always'
docker create --restart=unknown busybox
Error response from daemon: invalid restart policy 'unknown'
After this change:
docker create --restart=always:3 busybox
Error response from daemon: invalid restart policy: maximum retry count can only be used with 'on-failure'
docker create --restart=always:-1 busybox
Error response from daemon: invalid restart policy: maximum retry count can only be used with 'on-failure' and cannot be negative
docker create --restart=unknown busybox
Error response from daemon: invalid restart policy: unknown policy 'unknown'; use one of 'no', 'always', 'on-failure', or 'unless-stopped'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some tests were testing the deprecated fields, instead of their non-deprecated
alternatives.
This patch adds a utility to verify that they match, and rewrites the tests
to check the non-deprecated fields instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- clean up "//import" comment, as test-files cannot be imported, and only
one "//import" comment is needed per package.
- remove some intermediate variables
- rewrite assertions to use gotest.tools
- use assert.Check()) (non-fatal) where possible
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type was added in 247f4796d2, and
at the time was added as an alias for string;
> api/types/events: add "Type" type for event-type enum
>
> Currently just an alias for string, but we can change it to be an
> actual type.
Now that all code uses the defined types, we should be able to make
this an actual type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Updating the version of crun that we use in our tests to a version that
supports the "features" command (crun v1.8.6 and up). This should prevent
some warnings in our logs:
WARN[2023-08-26T17:05:35.042978552Z] Failed to run [/usr/local/bin/crun features]: "unknown command features\n" error="exit status 1"
full diff: https://github.com/containers/crun/compare/1.4.5...1.8.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add a fast-patch to some functions, to prevent locking/unlocking,
or other operations that would not be needed;
- Network.addDriverInfoToCluster
- Network.deleteDriverInfoFromCluster
- Network.addServiceInfoToCluster
- Network.deleteServiceInfoFromCluster
- Network.addDriverWatches
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- return early when failing to fetch the driver
- store network-ID and controller in a variable to prevent repeatedly
locking/unlocking. We don't expect the network's ID to change
during this operation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method is not part of any interface, and identical to Endpoint.Iface,
but one returns an Interface-type (driverapi.InterfaceInfo) and the other
returns a concrete type (EndpointInterface).
Interface-matching should generally happen on the receiver side, and this
function was only used in a single location, and passed as argument to
Driver.CreateEndpoint, which already matches the interface by accepting
a driverapi.InterfaceInfo.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Interface-matching should generally happen on the receiver side, and this
function was only used in a single location, and passed as argument to
Driver.CreateEndpoint, which already matches the interface by accepting
a driverapi.InterfaceInfo.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Interface-matching should generally happen on the receiver side, and this
function was only used in a single location, and passed as argument to
Driver.CreateEndpoint, which already matches the interface by accepting
a driverapi.InterfaceInfo.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also swapping the order of arguments; putting the "attributes" arguments
last, so that variables can be more cleanly inlined.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The content of this file was removed in c0bc14e8dd,
and all it container since was the package name.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The devicemapper graphdriver has been removed in commit
dc11d2a2d8, and we should
no longer need this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This check was added in 98fe4bd8f1, to check
whether dm_task_deferred_remove could be removed, and to conditionally set
the corresponding build-tags. Now that the devicemapper graphdriver has been
removed in dc11d2a2d8, we no longer need this.
This patch:
- removes uses of the (no longer used) `libdm`, `dlsym_deferred_remove`,
and `libdm_no_deferred_remove` build-tags.
- removes the `add_buildtag` utility, which is now unused.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This documentation moved to a different page, and the Go documentation
moved to the https://go.dev/ domain.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit ab35df454d removed most of the pre-go1.17
build-tags, but for some reason, "go fix" doesn't remove these, so removing
the remaining ones manually
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Copying relevant documentation from the EndpointInfo interface. We should
remove this interface, and the related Info() function, but it's currently
acting as a "gate" to prevent accessing the Endpoint's accessors without
making sure it's fully hydrated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While working on this code, I noticed that there's currently an issue
with userns enabled. When userns is enabled, joining another container's
namespace must also join its user-namespace.
However, a container can only be in a single user namespace, so if a
container joins namespaces from multiple containers, latter user-namespaces
overwrite former ones.
We must add validation for this, but in the meantime, add notes / todo's.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Most error-message returned would already include "container" and the
container ID in the error-message (e.g. "container %s is not running"),
so there's no need to add a custom prefix for that.
- os.Stat returns a PathError, which already includes the operation ("stat"),
the path, and the underlying error that occurred.
And while updating, let's also fix the name to be proper camelCase :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function didn't need the whole container, only its ID, so let's
use that as argument. This also makes it consistent with getIpcContainer.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`Daemon.getPidContainer()` was wrapping the error-message with a message
("cannot join PID of a non running container") that did not reflect the
actual reason for the error; `Daemon.GetContainer()` could either return
an invalid parameter (invalid / empty identifier), or a "not found" error
if the specified container-ID could not be found.
In the latter case, we don't want to return a "not found" error through
the API, as this would indicate that the container we're _starting_ was
not found (which is not the case), so we need to convert the error into
an `errdefs.ErrInvalidParameter` (the container-ID specified for the PID
namespace is invalid if the container doesn't exist).
This logic is similar to what we do for IPC namespaces. which received
a similar fix in c3d7a0c603.
This patch updates the error-types, and moves them into the getIpcContainer
and getPidContainer container functions, both of which should return
an "invalid parameter" if the container was not found.
It's worth noting that, while `WithNamespaces()` may return an "invalid
parameter" error, the `start` endpoint itself may _not_ be. as outlined
in commit bf1fb97575, starting a container
that has an invalid configuration should be considered an internal server
error, and is not an invalid _request_. However, for uses other than
container "start", `WithNamespaces()` should return the correct error
to allow code to handle it accordingly.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We were using a mixture of approaches for these; aligning them a bit
to all use switch statements.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in 12485d62ee to save some
duplication, but was really over-engineered to save a few lines of code,
at the cost of hiding away what it does and also potentially returning
inconsistent errors (not addressed in this patch). Let's start with
inlining these.
This removes;
- Daemon.checkContainer
- daemon.containerIsRunning
- daemon.containerIsNotRestarting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We used to have to clone and build the registry v2 but now that we have
updated the version we can directtly copy the binary from the official
distribution/distribution image.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Previous image created a new partially filled image.
This caused child images to lose their parent's layers.
Instead of creating a new object and trying to replace its fields, just
clone the original passed image and change its ID to the manifest
digest.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
There's only one implementation; let's use that.
Also fixing a linting issue;
libnetwork/osl/interface_linux.go:91:2: S1001: should use copy(to, from) instead of a loop (gosimple)
for i, iface := range n.iFaces {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
InterfaceOptions() returned an IfaceOptionSetter interface, which contained
"methods" that returned functional options. Such a construct could have made
sense if the functional options returned would (e.g.) be pre-propagated with
information from the Sandbox (network namespace), but none of that was the case.
There was only one implementation of IfaceOptionSetter (networkNamespace),
which happened to be the same as the only implementation of Sandbox, so remove
the interface as well, to help networkNamespace with its multi-personality
disorder.
This patch:
- removes Sandbox.Bridge() and makes it a regular function (WithIsBridge)
- removes Sandbox.Master() and makes it a regular function (WithMaster)
- removes Sandbox.MacAddress() and makes it a regular function (WithMACAddress)
- removes Sandbox.Address() and makes it a regular function (WithIPv4Address)
- removes Sandbox.AddressIPv6() and makes it a regular function (WithIPv6Address)
- removes Sandbox.LinkLocalAddresses() and makes it a regular function (WithLinkLocalAddresses)
- removes Sandbox.Routes() and makes it a regular function (WithRoutes)
- removes Sandbox.InterfaceOptions().
- removes the IfaceOptionSetter interface.
Note that the IfaceOption signature was changes as well to allow returning
an error. This is not currently used, but will be used for some options
in the near future, so adding that in preparation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
NeighborOptions() returned an NeighborOptionSetter interface, which
contained "methods" that returned functional options. Such a construct
could have made sense if the functional options returned would (e.g.)
be pre-propagated with information from the Sandbox (network namespace),
but none of that was the case.
There was only one implementation of NeighborOptionSetter (networkNamespace),
which happened to be the same as the only implementation of Sandbox, so
remove the interface as well, to help networkNamespace with its multi-personality
disorder.
This patch:
- removes Sandbox.LinkName() and makes it a regular function (WithLinkName)
- removes Sandbox.Family() and makes it a regular function (WithFamily)
- removes Sandbox.NeighborOptions().
- removes the NeighborOptionSetter interface
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
osl.NewSandbox() always returns a nil interface on Windows (and other non-Linux
platforms). This means that any code that these fields are always nil, and
any code using these fields must be considered Linux-only.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
osl.NewSandbox() always returns a nil interface on Windows (and other non-Linux
platforms). This means that any code that these fields are always nil, and
any code using these fields must be considered Linux-only;
- libnetwork/Controller.defOsSbox
- libnetwork/Sandbox.osSbox
Ideally, these fields would live in Linux-only files, but they're referenced
in various platform-neutral parts of the code, so let's start with moving
the initialization code to Linux-only files.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Copying the descriptions from the Sandbox, Info, NeighborOptionSetter,
and IfaceOptionSetter interfaces that it implements.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Check if firewalld is running before running the function, so that consumers
of the function don't have to check for the status.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test is currently failing with containerd-integration, which should
be looked into, but let's start with preventing it from panicking, to make
the test-failures less noisy;
--- FAIL: TestDiskUsage/after_container.Run (0.26s)
panic: runtime error: index out of range [0] with length 0 [recovered]
panic: runtime error: index out of range [0] with length 0
goroutine 280 [running]:
testing.tRunner.func1.2({0xb07a00, 0x40002006a8})
/usr/local/go/src/testing/testing.go:1526 +0x1c8
testing.tRunner.func1()
/usr/local/go/src/testing/testing.go:1529 +0x364
panic({0xb07a00, 0x40002006a8})
/usr/local/go/src/runtime/panic.go:884 +0x1f4
github.com/docker/docker/integration/system.TestDiskUsage.func3(0x0?, {0x0, {0x14ea4a8, 0x0, 0x0}, {0x14ea4a8, 0x0, 0x0}, {0x14ea4a8, 0x0, ...}, ...})
/go/src/github.com/docker/docker/integration/system/disk_usage_test.go:82 +0x7e4
github.com/docker/docker/integration/system.TestDiskUsage.func4(0x4000235c80?)
/go/src/github.com/docker/docker/integration/system/disk_usage_test.go:118 +0x8c
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also remove integration-cli: `DockerAPISuite.TestContainerAPIDeleteConflict`,
which was testing the same conditions as `TestRemoveContainerRunning` in
integration/container.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Saw this failure in a flaky test, and I wondered why we consider this an
error condition;
=== RUN TestKillWithStopSignalAndRestartPolicies
main_test.go:32: assertion failed: error is not nil: Error response from daemon: Could not kill running container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7, cannot remove - Container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7 is not running: failed to remove 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7
--- FAIL: TestKillWithStopSignalAndRestartPolicies (0.84s)
=== RUN TestKillWithStopSignalAndRestartPolicies/same-signal-disables-restart-policy
--- PASS: TestKillWithStopSignalAndRestartPolicies/same-signal-disables-restart-policy (0.42s)
=== RUN TestKillWithStopSignalAndRestartPolicies/different-signal-keep-restart-policy
--- PASS: TestKillWithStopSignalAndRestartPolicies/different-signal-keep-restart-policy (0.23s)
In the above;
1. `Error response from daemon: Could not kill running container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7`
2. `cannot remove - Container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7 is not running`
3. `failed to remove 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7`
So it looks like the removal fails because we couldn't kill the container
because it was already stopped, which may be a race condition where the first
check shows the container to be running (but may already be in process to be
removed or killed. In either case, we probably shouldn't fail the removal if
the container is already stopped.
This patch adds a `isNotRunning()` utility, so that we can ignore this case,
and proceed with the removal.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function never returns an error, so let's remove the error-return,
and give it a slightly more to-the-point name.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Windows uses the container-iD as ID for sandboxes, so it's not needed to
generate an ID when running on Windows.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The BuildKit dockerignore package was integrated in the patternmatcher
repository / module. This patch updates our uses of the BuildKit package
with its new location.
A small local change was made to keep the format of the existing error message,
because the "ignorefile" package is slightly more agnostic in that respect
and doesn't include ".dockerignore" in the error message.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the lease doesn't exit (for example when creating the container
failed), just ignore the not found error.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Prior to moby/moby#44968, libnetwork would happily accept a ChildSubnet
with a bigger mask than its parent subnet. In such case, it was
producing IP addresses based on the parent subnet, and the child subnet
was not allocated from the address pool.
This commit automatically fixes invalid ChildSubnet for networks stored
in libnetwork's datastore.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Currently, IPAM config is never validated by the API. Some checks
are done by the CLI, but they're not exhaustive. And some of these
misconfigurations might be caught early by libnetwork (ie. when the
network is created), and others only surface when connecting a container
to a misconfigured network. In both cases, the API would return a 500.
Although the `NetworkCreate` endpoint might already return warnings,
these are never displayed by the CLI. As such, it was decided during a
maintainer's call to return validation errors _for all API versions_.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also move the validation function to live with the type definition,
which allows it to be used outside of the daemon as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the image for the wanted platform doesn't exist then the lease
doesn't exist either. Returning this error hides the real error, so
let's not return it.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
The Controller.Sandboxes method was used by some SandboxWalkers. Now
that those have been removed, there are no longer any consumers of this
method, so let's remove it for now.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I had a CI run fail to "Upload reports":
Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
Finished backoff for retry #1, continuing with upload
Total file count: 211 ---- Processed file #160 (75.8%)
...
Total file count: 211 ---- Processed file #164 (77.7%)
Total file count: 211 ---- Processed file #164 (77.7%)
Total file count: 211 ---- Processed file #164 (77.7%)
A 503 status code has been received, will attempt to retry the upload
##### Begin Diagnostic HTTP information #####
Status Code: 503
Status Message: Service Unavailable
Header Information: {
"content-length": "592",
"content-type": "application/json; charset=utf-8",
"date": "Mon, 21 Aug 2023 14:08:10 GMT",
"server": "Kestrel",
"cache-control": "no-store,no-cache",
"pragma": "no-cache",
"strict-transport-security": "max-age=2592000",
"x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
"activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
"x-frame-options": "SAMEORIGIN"
}
###### End Diagnostic HTTP information ######
Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
Error: aborting artifact upload
Total file count: 211 ---- Processed file #165 (78.1%)
A 503 status code has been received, will attempt to retry the upload
Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0
As a result, the "Download reports" continued retrying:
...
Total file count: 1004 ---- Processed file #436 (43.4%)
Total file count: 1004 ---- Processed file #436 (43.4%)
Total file count: 1004 ---- Processed file #436 (43.4%)
An error occurred while attempting to download a file
Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
at Object.onceWrapper (node:events:627:28)
at ClientRequest.emit (node:events:513:28)
at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
at Object.onceWrapper (node:events:627:28)
at TLSSocket.emit (node:events:525:35)
at TLSSocket.Socket._onTimeout (node:net:550:8)
at listOnTimeout (node:internal/timers:559:17)
at processTimers (node:internal/timers:502:7)
Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
Total file count: 1004 ---- Processed file #436 (43.4%)
And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This functionality has been replaced with Controller.GetSandbox, and is
no longer used anywhere.
This patch removes:
- the Controller.WalkSandboxes method
- the SandboxContainerWalker SandboxWalker
- the SandboxWalker type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Various parts of the code were using "walkers" to iterate over the
controller's sandboxes, and the only condition for all of them was
to find the sandbox for a given container-ID. Iterating over all
sandboxes was also sub-optimal, because on Windows, the ContainerID
is used as Sandbox-ID, which can be used to lookup the sandbox from
the "sandboxes" map on the controller.
This patch implements a GetSandbox method on the controller that
looks up the sandbox for a given container-ID, using the most optimal
approach (depending on the platform).
The new method can return errors for invalid (empty) container-IDs, and
a "not found" error to allow consumers to detect non-existing sandboxes,
or potentially invalid IDs.
This new method replaces the (non-exported) Daemon.getNetworkSandbox(),
which was only used internally, in favor of directly accessing the
controller's method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was not exported so let's remove the abstraction to not make it look
like something more than it is.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
"Pay no attention to the implementation behind the curtain!"
There's only one implementation of the Sandbox interface, and only one implementation
of the Info interface, and they both happens to be implemented by the same type:
networkNamespace. Let's merge these interfaces.
And now that we know that there's one, and only one Info, we can drop the charade,
and relieve the Sandbox from its dual personality.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This better aligns to GHA/CI settings, and is in general a better
practice in the year 2023.
We also drop the 'unsupported' fallback for `git rev-parse` in the
Makefile; we have a better fallback behavior for an empty
DOCKER_GITCOMMIT in `hack/make.sh`.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
There are still messy special cases (e.g. DOCKER_GITCOMMIT vs VERSION),
but this makes things a little easier to follow, as we keep
GHA-specifics in the GHA files.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This is something that stood out to me: removing the intermediate
container is part of a build step, but unlike the other output from
the build, wasn't indented (and prefixed with `--->`) to be shown
as part of the build.
This patch adds the `--->` prefix, to make it clearer what step the
removal was part of.
While at it, I also updated the message itself: this output is printed
_after_ the intermediate container has been removed, so we may as well
make it match reality, so I changed "removing" to "removed".
Before:
echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM busybox
---> a416a98b71e2
Step 2/3 : RUN echo hello > /dev/null
---> Running in a1a65b9365ac
Removing intermediate container a1a65b9365ac
---> 8c6b57ebebdd
Step 3/3 : RUN echo world > /dev/null
---> Running in 9fa977b763a5
Removing intermediate container 9fa977b763a5
---> 795c1f2fc7b9
Successfully built 795c1f2fc7b9
After:
echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM busybox
---> fc9db2894f4e
Step 2/3 : RUN echo hello > /dev/null
---> Running in 38d7c34c2178
---> Removed intermediate container 38d7c34c2178
---> 7c0dbc45111d
Step 3/3 : RUN echo world > /dev/null
---> Running in 629620285d4c
---> Removed intermediate container 629620285d4c
---> b92f70f2e57d
Successfully built b92f70f2e57d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Slightly refactor Resolver.dialExtDNS:
- use net.JoinHostPort to properly format IPv6 addresses
- define a const for the default port, and avoid int -> string
conversion if no custom port is defined
- slightly simplify logic if the HostLoopback is used (at the cost of
duplicating one line); in that case we don't need to define the closure
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was added in 36fd9d02be
(libnetwork: ce6c6e8c35),
because there were multiple places where a DNS response was created,
which had to use the same options. However, new "common" options were
added since, and having it in a function separate from the other (also
common) options was just hiding logic, so let's remove it.
What the above probably _should_ have done was to create a common utility
to create a DNS response (as all other options are shared as well). This
was actually done in 0c22e1bd07 (libnetwork:
be3531759b),
which added a `createRespMsg` utility, but missed that it could be used
for both cases.
This patch:
- removes the setCommonFlags function
- uses createRespMsg instead to share common options
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Removes the deprecated consts, which moved to a separate "scope" package
in commit 6ec03d6745, and are no longer used;
- datastore.LocalScope
- datastore.GlobalScope
- datastore.SwarmScope
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
UnavailableError is now compatible with errdefs.UnavailableError. These
errors will now return a 503 instead of a 500.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
InvalidParameter is now compatible with errdefs.InvalidParameter. Thus,
these errors will now return a 400 status code instead of a 500.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Fix a failure to inspect image if any of its present manifest references
an image config which isn't present locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- un-export ZoneSettings, because it's only used internally
- make conversion to a "interface" slice a method on the struct
- remove the getDockerZoneSettings() function, and move the type-definition
close to where it's used, as it was only used in a single location
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test didn't make a lot of sense, because `checkRunning()` depends on
the `connection` package-var being set, which is done by `firewalldInit()`,
so would never be true on its own.
Add a small utility that opens its own D-Bus connection to verify if
firewalld is running, and otherwise skips the tests (preserving any
error in the process).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
DelInterfaceFirewalld returns an error if the interface to delete was
not found. Let's ignore cases where we were successfully able to get
the list of interfaces in the zone, but the interface was not part of
the zone.
This patch changes the error for these cases to an errdefs.ErrNotFound,
and updates IPTable.ProgramChain to ignore those errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
As we have a hard time figuring out what moby/moby#46099 should look
like, this drop-in replacement will solve the initial formatting problem
we have. It's made internal such that we can remove it whenever we want
and unlike moby/moby#46099 doesn't require thoughtful API changes.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.
This patch:
- introduces a `retErr` output variable, to be used in defer statements.
- explicitly changes some `err` uses to locally-scoped variables.
- moves some variable definitions closer to where they're used (where possible).
While working on this change, there was one point in the code where
error handling was ambiguous. I added a note for that, in case this
was not a bug:
> This code was previously assigning the error to the global "err"
> variable (before it was renamed to "retErr"), but in case of a
> "MaskableError" did not *return* the error:
> b325dcbff6/libnetwork/controller.go (L566-L573)
>
> Depending on code paths further down, that meant that this error
> was either overwritten by other errors (and thus not handled in
> defer statements) or handled (if no other code was overwriting it.
>
> I suspect this was a bug (but possible without effect), but it could
> have been intentional. This logic is confusing at least, and even
> more so combined with the handling in defer statements that check for
> both the "err" return AND "skipCfgEpCount":
> b325dcbff6/libnetwork/controller.go (L586-L602)
>
> To save future visitors some time to dig up history:
>
> - config-only networks were added in 25082206df
> - the special error-handling and "skipCfgEpcoung" was added in ddd22a8198
> - and updated in 87b082f365 to don't use string-matching
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There were quite some places where the type collided with variables
named `agent`. Let's rename the type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function has _four_ output variables of the same type, and several
defer statements that checked the error returned (but using the `err`
variable).
This patch names the return variables to make it clearer what's being
returned, and renames the error-return to `retErr` to make it clearer
where we're dealing with the returned error (and not any local err), to
prevent accidentally shadowing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There's nothing handling these results, and they're logged as debug-logs,
so we may as well remove the returned variables.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Both functions were generating debug logs if there was nothing to log.
The function already produces logs if things failed while deleting entries,
so these logs would only be printed if there was nothing to delete, so can
safely be discarded.
Before this change:
DEBU[2023-08-14T12:33:23.082052638Z] Revoking external connectivity on endpoint sweet_swirles (1519f9376a3abe7a1c981600c25e8df6bbd0a3bc3a074f1c2b3bcbad0438443b)
DEBU[2023-08-14T12:33:23.085782847Z] DeleteConntrackEntries purged ipv4:0, ipv6:0
DEBU[2023-08-14T12:33:23.085793847Z] DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This goroutine was added in c458bca6dc, and
looks for errors from the wait channel. If no error is returned, it attempts
to start the container, and *updates* the error if a failure happened while
doing so, so that the code below it can update the container's status, and
perform auto-remove (if set for the container).
However, due to the formatting of the code, it was easy to overlook that
the "err" variable was not local to the "if" statement.
This patch breaks up the if-statement in an attempt to make it clearer that
this is not a local "err" variable, and adds a code-comment explaining the
logic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function was checking in a loop if networking for the container was
disabled. Change the function to return early, and to only set hooks
if one needs to be set.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's only called as part of the "libnetwork-setkey" re-exec, so un-exporting
it to make clear it's not for external use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The basepath is only used on Linux, so no need to call it on other
platforms. SetBasePath was already stubbed out on other platforms,
but "osl" was still imported in various places where it was not actually
used, so trying to reduce imports to get a better picture of what parts
are used (and not used).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some tests were implicitly skipped through the `getTestEnv()` utility,
which made it hard to discover they were not ran on Windows.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes it easier to spot if code is only used on Linux. Note that "all of"
the bridge driver is Linux-only.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Removing this type, because:
- containerNotModifiedError is not an actual error, and abstracting it away
was hiding some of these details. It also wasn't used as a sentinel error
anywhere, so doesn't have to be its own type.
- Defining a type just to toggle the error-message between "not running"
and "not stopped" felt a bit over-the-top, as each variant was only used once.
- So "it only had one job", and it didn't even do that right; it produced
capitalized error messages, which makes linters unhappy.
So, let's just inline what it does in the two places it was used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There's no need for this to be a closure; let's just make it a regular
function. While moving it out, also make some minor code-changes and
add some code-comments to describe the flow / intent, which may not
be trivial for people that are not familiar with these details.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The container rw layer may already be mounted, so it's not safe to use
it in another overlay mount. Use the ref counted mounter (which will
reuse the existing mount if it exists) to avoid that.
Also, mount the parent mounts (layers of the base image) in a read-only
mode.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
To prevent mounting the container rootfs in a rw mode if it's already
mounted. This can't use `mount.WithReadonlyTempMount` because the
archive code does a chroot with a pivot_root, which creates a new
directory in the rootfs.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Check that operations that could potentially perform overlayfs mounts
that could cause undefined behaviors.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Any error that occurs while creating the spec, even if it's the
result of an invalid container config, must be considered a System
error (internal server error), as it's not an error with the request
to start the container.
Invalid configuration in the config itself must be validated when
creating the container (creating its config), but some errors are
dependent on the current state, for example when starting a container
that shares a namespace with another container, and that container
is not running (or missing).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was only used for a single test, and it was very limited
in functionality as it only allowed for a certain error-string to be
matched.
Let's change it into a more generic function; a helper that allows a
container to be created from a `TestContainerConfig` (which can be
constructed using `NewTestConfig`) and that returns the response from
client.ContainerCreate(), so that any result from that can be tested,
leaving it up to the test to check the results.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Introduce a NewTestConfig utility, to allow using the available utilities
for constructing a config, and use them with the regular API client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `client` variable was colliding with the `client` import. In some cases
the confusing `cli` name (it's not the "cli") was used. Given that such names
can easily start spreading (through copy/paste, or "code by example"), let's
make a one-time pass through all of them in this package to use the same name.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `client` variable was colliding with the `client` import in various
files. While it didn't conflict in all files, there was inconsistency
in the naming, sometimes using the confusing `cli` name (it's not the
"cli"), and such names can easily start spreading (through copy/paste,
or "code by example").
Let's make a one-time pass through all of them in this package to use
the same name.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In these cases, continuing after a non nil error will result in a nil
dereference in panic.
Change the `assert.Check` to `assert.NilError` to avoid that.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This test was testing the client-side validation, so might as well
move it there, and validate that the client invalidates before
trying to make an API call.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Attach the context to the request while we're creating it, instead of
creating the context first, and adding the context later.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Re-use the request, and change the method to GET instead of building
a new request "from scratch".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't exit immediately (due to `set -e` bash behavior) when grep returns
with a non-zero exit code. Use empty dirs instead and let it print
messages about all tests being filtered out.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
To avoid passing the `/` prefix in the -test.run to the integration test
suite, which for some reason executes all tests, but works fine with
integration-cli.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Previous check checked if ANY of the test directories isn't
integration-cli. This means it was true if TEST_FILTER matched multiple
tests from both integration and integration-cli suite.
Remove the grep `-v` inversion and replace it with a bash negation, so
it actually checks if there is no `integration-cli` in test dirs.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The mutex is only used on reads, but there's nothing protecting writes,
and it looks like nothing is mutating fields after creation, so let's
remove this altogether.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
No context in the commit that added it, but PR discussion shows that
the API was mostly exploratory, and it was 8 Years go, so let's not
head in that direction :) b646784859
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that we removed the interface, there's no need to cast the Network
to a NetworkInfo interface, so we can remove uses of the `Info()` method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These errors aren't used in our repo and seem unused by the OSS
community (this was checked with Sourcegraph).
- ErrIpamInternalError has never been used
- ErrInvalidRequest is unused since moby/libnetwork@c85356efa
- ErrPoolNotFound has never been used
- ErrOverlapPool has never been used
- ErrNoAvailablePool has never been used
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
PR moby/moby#45759 is going to use the new `errors.Join` function to
return a list of validation errors.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Check the preferredPool first, as other checks could be doing more
(such as locking, or validating / parsing). Also adding a note, as
it's unclear why we're ignoring invalid pools here.
The "invalid" conditions was added in [libnetwork#1095][1], which
moved code to reduce os-specific dependencies in the ipam package,
but also introduced a types.IsIPNetValid() function, which considers
"0.0.0.0/0" invalid, and added it to the condition to return early.
Unfortunately review does not mention this change, so there's no
context why. Possibly this was done to prevent errors further down
the line (when checking for overlaps), but returning an error here
instead would likely have avoided that as well, so we can only guess.
To make this code slightly more transparent, this patch also inlines
the "types.IsIPNetValid" function, as it's not used anywhere else,
and inlining it makes it more visible.
[1]: 5ca79d6b87 (diff-bdcd879439d041827d334846f9aba01de6e3683ed8fdd01e63917dae6df23846)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This code was only run if no preferred pool was specified, however,
since [libnetwork#1162][2], the function would already return early
if a preferred pools was set (and the overlap check to be skipped),
so this was now just dead code.
[2]: 9cc3385f44
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function intentionally holds a lock / lease on address-pools to
prevent trying the same pool repeatedly.
Let's try to make this logic slightly more transparent, and prevent
defining defers in a loop. Releasing all the pools in a singe defer
also allows us to get the network-name once, which prevents locking
and unlocking the network for each iteration.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Both functions have multiple output vars with generic types, which made
it hard to grasp what's what.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes it easier to consume, without first having to create an empty
PoolID.
Performance is the same:
BenchmarkPoolIDFromString-10 6100345 196.5 ns/op 112 B/op 3 allocs/op
BenchmarkPoolIDFromString-10 6252750 192.0 ns/op 112 B/op 3 allocs/op
Note that I opted not to change the return-type to a pointer, as that seems
to perform less;
BenchmarkPoolIDFromString-10 6252750 192.0 ns/op 112 B/op 3 allocs/op
BenchmarkPoolIDFromString-10 5288682 226.6 ns/op 192 B/op 4 allocs/op
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
As this function may be called repeatedly to convert to/from a string,
it may be worth optimizing it a bit. Adding a minimal Benchmark for
it as well.
Before/after:
BenchmarkPoolIDToString-10 2842830 424.3 ns/op 232 B/op 12 allocs/op
BenchmarkPoolIDToString-10 7176738 166.8 ns/op 112 B/op 7 allocs/op
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
network.requestPoolHelper and Allocator.RequestPool have many args and
output vars with generic types. Add names for them to make it easier to
grasp what's what.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The options are unused, other than for debug-logging, which made it look
as if they were actually consumed anywhere, but they aren't.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes it slightly more readable to see what's returned in each of
the code-paths. Also move validation of pool/subpool earlier in the
function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some tests were testing non-existing plugins, but therefore triggered
the retry-loop, which times out after 15-30 seconds. Add some options
to allow overriding this timeout during tests.
Before:
go test -v -run '^(TestGet|TestNewClientWithTimeout)$'
=== RUN TestGet
=== RUN TestGet/success
=== RUN TestGet/not_implemented
=== RUN TestGet/not_exists
WARN[0000] Unable to locate plugin: vegetable, retrying in 1s
WARN[0001] Unable to locate plugin: vegetable, retrying in 2s
WARN[0003] Unable to locate plugin: vegetable, retrying in 4s
WARN[0007] Unable to locate plugin: vegetable, retrying in 8s
--- PASS: TestGet (15.02s)
--- PASS: TestGet/success (0.00s)
--- PASS: TestGet/not_implemented (0.00s)
--- PASS: TestGet/not_exists (15.02s)
=== RUN TestNewClientWithTimeout
client_test.go:166: started remote plugin server listening on: http://127.0.0.1:36275
WARN[0015] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": context deadline exceeded (Client.Timeout exceeded while awaiting headers), retrying in 1s
WARN[0017] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": context deadline exceeded (Client.Timeout exceeded while awaiting headers), retrying in 2s
WARN[0019] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": net/http: request canceled (Client.Timeout exceeded while awaiting headers), retrying in 4s
WARN[0024] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": net/http: request canceled (Client.Timeout exceeded while awaiting headers), retrying in 8s
--- PASS: TestNewClientWithTimeout (17.64s)
PASS
ok github.com/docker/docker/pkg/plugins 32.664s
After:
go test -v -run '^(TestGet|TestNewClientWithTimeout)$'
=== RUN TestGet
=== RUN TestGet/success
=== RUN TestGet/not_implemented
=== RUN TestGet/not_exists
WARN[0000] Unable to locate plugin: this-plugin-does-not-exist, retrying in 1s
--- PASS: TestGet (1.00s)
--- PASS: TestGet/success (0.00s)
--- PASS: TestGet/not_implemented (0.00s)
--- PASS: TestGet/not_exists (1.00s)
=== RUN TestNewClientWithTimeout
client_test.go:167: started remote plugin server listening on: http://127.0.0.1:45973
--- PASS: TestNewClientWithTimeout (0.04s)
PASS
ok github.com/docker/docker/pkg/plugins 1.050s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If TEST_INTEGRATION_FAIL_FAST is not set, run the integration-cli tests
even if integration tests failed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Collect a list of all the links we successfully enabled (if any), and
use a single defer to disable them.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The iptables package has types defined for these actions; use them directly
instead of creating a string only to convert it to a known value.
As the linkContainers() function is only used internally, and with fixed
values, we can also remove the validation, and InvalidIPTablesCfgError
error, which is now unused.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Partially revert commit 94b880f.
The CheckDuplicate field has been introduced in commit 2ab94e1. At that
time, this check was done in the network router. It was then moved to
the daemon package in commit 3ca2982. However, commit 94b880f duplicated
the logic into the network router for no apparent reason. Finally,
commit ab18718 made sure a 409 would be returned instead of a 500.
As this logic is first done by the daemon, the error -> warning
conversion can't happen because CheckDuplicate has to be true for the
daemon package to return an error. If it's false, the daemon proceed
with the network creation, set the Warning field of its return value and
return no error.
Thus, the CheckDuplicate logic in the api is removed and
libnetwork.NetworkNameError now implements the ErrConflict interface.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The MediaType was changed twice in;
- b3b7eb2723 ("application/vnd.docker.plugins.v1+json" -> "application/vnd.docker.plugins.v1.1+json")
- 54587d861d ("application/vnd.docker.plugins.v1.1+json" -> "application/vnd.docker.plugins.v1.2+json")
But the (integration) tests were still using the old version, so let's
use the VersionMimeType const that's defined, and use the updated version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The MediaType was changed twice in;
- b3b7eb2723 ("application/vnd.docker.plugins.v1+json" -> "application/vnd.docker.plugins.v1.1+json")
- 54587d861d ("application/vnd.docker.plugins.v1.1+json" -> "application/vnd.docker.plugins.v1.2+json")
But the (integration) tests were still using the old version, so let's
use the VersionMimeType const that's defined, and use the updated version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The MediaType was changed twice in;
- b3b7eb2723 ("application/vnd.docker.plugins.v1+json" -> "application/vnd.docker.plugins.v1.1+json")
- 54587d861d ("application/vnd.docker.plugins.v1.1+json" -> "application/vnd.docker.plugins.v1.2+json")
But the (integration) tests were still using the old version, so let's
use the VersionMimeType const that's defined, and use the updated version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
During review, it was decided to remove `LimitNOFILE` from `docker.service` to rely on the systemd v240 implicit default of `1024:524288`. On supported platforms with systemd prior to v240, packagers will patch the service with an explicit `LimitNOFILE=1024:524288`.
- `1024` soft limit is an implicit default, avoiding unexpected breakage. Software that needs a higher limit should request to raise the soft limit for its process.
- `524288` hard limit is an implicit default since systemd v240 and is adequate for most processes (_half of the historical limit from `fs.nr_open` of `1048576`_), while 4096 is the implicit default from the kernel (often too low). Individual containers can be started with `--ulimit` when a larger hard limit is required.
- The hard limit may not exceed `fs.nr_open` (_which a value of `infinity` will resolve to_). On most systems with systemd v240 or newer, this will resolve to an excessive size of 2^30 (over 1 billion).
- When set to `infinity` (usually as the soft limit) software may experience significantly increased resource usage, resulting in a performance regression or runtime failures that are difficult to troubleshoot.
- OpenRC current config approach lacks support for different soft/hard limits being set as it adjusts additional limits and `ulimit` does not support mixed usage of `-H` + `-S`. A soft limit of `524288` is not ideal, but 2^19 is much less overhead than 2^30, whilst a hard limit of 4096 would be problematic for Docker.
Signed-off-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Remove some intermediate vars, move vars closer to where they're used,
and introduce local var for `nw.Name()` to reduce some locking/unlocking in:
- `Daemon.allocateNetwork()`
- `Daemon.releaseNetwork()`
- `Daemon.connectToNetwork()`
- `Daemon.disconnectFromNetwork()`
- `Daemon.findAndAttachNetwork()`
Also un-wrapping some lines to make it slightly easier to read the conditions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Remove intermediate variable
- Optimize the order of checks in the condition; check for unmanaged containers
first, before getting information about cluster state and network information.
- Simplify the log messages, as the error would already contain the same
information about the network (name or ID) and container (ID), so would
print the network ID twice:
error detaching from network <ID>: could not find network attachment for container <ID> to network <name or ID>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The function was declaring an err variable which was shadowed. It was
intended for directly assigning to a struct field, but as this function
is directly mutating an existing object, and the err variable was declared
far away from its use, let's use an intermediate var for that to make it
slightly more atomic.
While at it, also combined two "if" branches.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
store network.Name() in a variable to reduce repeatedly locking/unlocking
of the network (although this is very, very minimal in the grand scheme
of things).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These tests were made parallel to speed up the execution, but this
turned out to be flaky, because they mutate some shared state.
The tests use shared `storage` variable without any synchronization.
However, adding synchronization is not enough in all cases, some tests
register the same plugin, so they can't be run in parallel to each
other.
This commit adds the synchronization around `storage` variable
modification and removes parallel from the tests where it's not enough.
Before:
```
$ go test -race -v . -count 1
...
--- FAIL: TestGet (15.02s)
--- FAIL: TestGet/not_implemented (0.00s)
testing.go:1446: race detected during execution of test
testing.go:1446: race detected during execution of test
FAIL
FAIL github.com/docker/docker/pkg/plugins 17.655s
FAIL
```
After:
```
$ go test -race -v . -count 1
ok github.com/docker/docker/pkg/plugins 32.702s
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This function is called by `daemon.containerCreate()` which is already
wrapping errors coming from `verifyNetworkingConfig()` with
`errdefs.InvalidParameter()`. So `verifyNetworkingConfig()` should only
return standard errors.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
"HEAD" will still be used as a version if no DOCKER_COMMIT is provided
(for example when not running via `make`), but it won't prevent it being
set to the GITHUB_SHA variable when it's present.
This should fix `Git commit` reported by `docker version` for the
binaries generated by `moby-bin`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This code was initializing a new PortBinding, and creating a deep copy
for each binding. It's unclear what the intent was here, but at least
PortBinding.GetCopy() wasn't adding much value, as it created a new
PortBinding, [copying all values from the original][1], which includes
a [copy of IPAddresses in it][2]. Our original "template" did not have any
of that, so let's forego that, and just create new PortBindings as we go.
[1]: 454b6a7cf5/libnetwork/types/types.go (L110-L120)
[2]: 454b6a7cf5/libnetwork/types/types.go (L236-L244)
Benchmarking before/after;
BenchmarkPortBindingCopy-10 166752 6230 ns/op 1600 B/op 100 allocs/op
BenchmarkPortBindingNoCopy-10 226989 5056 ns/op 1600 B/op 100 allocs/op
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were not adding much, so just getting rid of them. Also added a
TODO to move this code to the type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move variables closer to where they're used instead of defining them all
at the start of the function.
Also removing some intermediate variables, unwrapped some lines, and combined
some checks to a single check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Outside of some tests, these options are the only code setting these fields,
so we can update them to set the value, instead of appending.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was created as a "method", but didn't use the Daemon in any
way, and all other options were checked inline, so let's not pretend this
function is more "special" than the other checks, and inline the code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- store network.Name() in a variable to reduce repeatedly locking/unlocking
of the network (although this is very, very minimal in the grand scheme
of things).
- un-wrap long conditions
- ever so slightly optimise some conditions by changeing the order of checks.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This code was initializing a new PortBinding, and creating a deep copy
for each binding. It's unclear what the intent was here, but at least
PortBinding.GetCopy() wasn't adding much value, as it created a new
PortBinding, [copying all values from the original][1], which includes
a [copy of IPAddresses in it][2]. Our original "template" did not have any
of that, so let's forego that, and just create new PortBindings as we go.
[1]: 454b6a7cf5/libnetwork/types/types.go (L110-L120)
[2]: 454b6a7cf5/libnetwork/types/types.go (L236-L244)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were not adding much, so just getting rid of them. Also added a
TODO to move this code to the type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move variables closer to where they're used instead of defining them all
at the start of the function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`getPortMapInfo` does many things; it creates a copy of all the sandbox
endpoints, gets the driver, endpoints, and network from store, and creates
port-bindings for all exposed and mapped ports.
We should look if we can create a more minimal implementation for this
purpose, but in the meantime, let's prevent it being called if we don't
need it by making it the second condition in the check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- don't initialize slices; it's not needed to append to them
- store network-ID in a var to prevent repeated lock/unlocking in nw.ID()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When we added this deprecation warning, some registries had not yet
moved away from the deprecated specification, so we made the warning
conditional for pulling from Docker Hub.
That condition was added in 647dfe99a5,
which is over 4 Years ago, which should be time enough for images
and registries to have moved to current specifications.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use the same warning for both "v1 in manifest-index" and bare "v1" images.
- Update URL to use a "/go/" redirect, which allows the docs team to more
easily redirect the URL to relevant docs (if things move).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The original code in container.Exec was potentially leaking the copy
goroutine when the context was cancelled or timed out. The new
`demultiplexStreams()` function won't return until the goroutine has
finished its work, and to ensure that it takes care of closing the
hijacked connection.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Includes a fix for CVE-2023-29409
go1.20.7 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.20.7 milestone on our issue tracker for details:
- https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.6...go1.20.7
From the mailing list announcement:
[security] Go 1.20.7 and Go 1.19.12 are released
Hello gophers,
We have just released Go versions 1.20.7 and 1.19.12, minor point releases.
These minor releases include 1 security fixes following the security policy:
- crypto/tls: restrict RSA keys in certificates to <= 8192 bits
Extremely large RSA keys in certificate chains can cause a client/server
to expend significant CPU time verifying signatures. Limit this by
restricting the size of RSA keys transmitted during handshakes to <=
8192 bits.
Based on a survey of publicly trusted RSA keys, there are currently only
three certificates in circulation with keys larger than this, and all
three appear to be test certificates that are not actively deployed. It
is possible there are larger keys in use in private PKIs, but we target
the web PKI, so causing breakage here in the interests of increasing the
default safety of users of crypto/tls seems reasonable.
Thanks to Mateusz Poliwczak for reporting this issue.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes it slightly clearer what it does, as "resolve" may give the
impression it's doing more than just returning the TLS config configured
for the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
fallbackDial was only used in a single place, and it was defined far away
from where it's used, so let's inline it, so that it's clear at a glance
what we're doing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The is-automated field is being deprecated by Docker Hub's search API,
and will always be set to "false" in future.
This patch deprecates the field and related filter for the Engine's API.
In future, the `is-automated` filter will no longer yield any results
when searching for `is-automated=true`, and will be ignored when
searching for `is-automated=false`.
Given that this field is deprecated by an external API, the deprecation
will not be versioned, and will apply to any API version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I noticed this was always being skipped because of race conditions
checking the logs.
This change adds a log scanner which will look through the logs line by
line rather than allocating a big buffer.
Additionally it adds a `poll.Check` which we can use to actually wait
for the desired log entry.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
IPv4AddrNoMatchError and IPv6AddrNoMatchError are currently implementing
BadRequestError. They are returned in two cases, and none are due to a
bad user request:
- When calling daemon's CreateNetwork route, if the bridge's IPv4
address or none of the bridge's IPv6 addresses match what's requested.
If that happens, there's a big issue somewhere in libnetwork or the
kernel.
- When restoring a network, for the same reason. In that case, the
on-disk state drifted from the interface state.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This error can only be reached because of an error in our code, so it's
not a "bad user request". As it's never type asserted, no need to keep
it around.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This error is only used in defensive checks whereas the precondition is
already checked by caller. If we reach it, we messed something else. So
it's definitely not a BadRequest. Also, it's not type asserted anywhere,
so just inline it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
It was not used as a sentinel error, and didn't carry a specific type,
which made it a rather complex way to create an error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type was added moved to the types package as part of a refactor
in 778e2a72b3
but the introduction of the sandbox API changed the existing API to
weak types (not using a plain string);
9a47be244a
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- InvalidIPTablesCfgError: implement InternalError instead of
BadRequestError. This error is returned when an invalid iptables
action is passed as argument (ie. none of -A, -I, or -D).
- ErrInvalidDriverConfig: don't implement BadRequestError. This is
returned when libnetwork controller initialization pass bad driver
config -- there's no call from an HTTP route.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
- full diff: https://github.com/containerd/containerd/compare/v1.6.21...v1.6.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.22
---
Notable Updates
- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
when `securityContext.RunAsUser`, `RunAsUsername` are empty
- CRI: Write generated CNI config atomically
- Fix concurrent writes for `UpdateContainerStats`
- Make `checkContainerTimestamps` less strict on Windows
- Port-Forward: Correctly handle known errors
- Resolve `docker.NewResolver` race condition
- SecComp: Always allow `name_to_handle_at`
- Adding support to run hcsshim from local clone
- Pinned image support
- Runtime/V2/RunC: Handle early exits w/o big locks
- CRITool: Move up to CRI-TOOLS v1.27.0
- Fix cpu architecture detection issue on emulated ARM platform
- Task: Don't `close()` io before `cancel()`
- Fix panic when remote differ returns empty result
- Plugins: Notify readiness when registered plugins are ready
- Unwrap io errors in server connection receive error handling
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- go.mod: update dependencies and go version by
- Use Go1.20
- Fix couple of typos
- Added `WithStdout` and `WithStderr` helpers
- Moved `cmdOperators` handling from `RunCmd` to `StartCmd`
- Deprecate `assert.ErrorType`
- Remove outdated Dockerfile
- add godoc links
full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.4.0...v3.5.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Follow-up to fca38bcd0a, which made the
Discover API optional for drivers to implement, but forgot to remove the
stubs from the Windows drivers, which didn't implement this API.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "Capability" type defines DataScope and ConnectivityScope fields,
but their value was set from consts in the datastore package, which
required importing that package and its dependencies for the consts
only.
This patch:
- Moves the consts to a separate "scope" package
- Adds aliases for the consts in the datastore package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
changes:
- specs-go: remove artifact prefixed annotations
- Switch from scratch to empty
- Remove artifact media type reference
- image-index: add artifactType to specs and schema
- Add artifactType to image index
- Apply version change from #1050
- Specify the content of the scratch blob
- Add language from artifacttype field to forbid allowlists of media types
- spec: clarify descriptor, align with de facto artifact usage
- Remove special guidance around wasm
- Update descriptor.go
- releases: use +dev as in-development suffix
- version: bump HEAD back to -dev
- image-index: add the subject field
full diff: https://github.com/opencontainers/image-spec/compare/v1.1.0-rc3...v1.1.0-rc4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Most drivers do not implement this, so detect if a driver implements
the discoverAPI, and remove the implementation from drivers that do
not support it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.2...v1.7.3
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.3
----
Welcome to the v1.7.3 release of containerd!
The third patch release for containerd 1.7 contains various fixes and updates.
Notable Updates
- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
when `securityContext.RunAsUser`,`RunAsUsername` are empty
- CRI: write generated CNI config atomically
- Port-Forward: Correctly handle known errors
- Resolve docker.NewResolver race condition
- Fix `net.ipv4.ping_group_range` with userns
- Runtime/V2/RunC: handle early exits w/o big locks
- SecComp: always allow `name_to_handle_at`
- CRI: Windows Pod Stats: Add a check to skip stats for containers that
are not running
- Task: don't `close()` io before cancel()
- Remove CNI conf_template deprecation
- Fix issue for HPC pod metrics
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.1...v1.7.2
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.2
----
Welcome to the v1.7.2 release of containerd!
The second patch release for containerd 1.7 includes enhancements to CRI
sandbox mode, Windows snapshot mounting support, and CRI and container IO
bug fixes.
CRI/Sandbox Updates
- Publish sandbox events
- Make stats respect sandbox's platform
Other Notable Updates
- Mount snapshots on Windows
- Notify readiness when registered plugins are ready
- Fix `cio.Cancel()` should close pipes
- CDI: Use CRI `Config.CDIDevices` field for CDI injection
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows.
This issue was not limited to the go command itself, and could also affect binaries
that use `os.Command`, `os.LookPath`, etc.
From the related blogpost (https://blog.golang.org/path-security):
> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing
At time of the go1.15 release, the Go team considered changing the behavior of
`os.LookPath()` and `exec.LookPath()` to be a breaking change, and made the
behavior "opt-in" by providing the `golang.org/x/sys/execabs` package as a
replacement.
However, for the go1.19 release, this changed, and the default behavior of
`os.LookPath()` and `exec.LookPath()` was changed. From the release notes:
https://go.dev/doc/go1.19#os-exec-path
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.
A result of this change was that registering the daemon as a Windows service
no longer worked when done from within the directory of the binary itself:
C:\> cd "Program Files\Docker\Docker\resources"
C:\Program Files\Docker\Docker\resources> dockerd --register-service
exec: "dockerd": cannot run executable found relative to current directory
Note that using an absolute path would work around the issue:
C:\Program Files\Docker\Docker>resources\dockerd.exe --register-service
This patch changes `registerService()` to use `os.Executable()`, instead of
depending on `os.Args[0]` and `exec.LookPath()` for resolving the absolute
path of the binary.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
IPv6 ipt rules are exactly the same as IPv4 rules, although both
protocol don't use the same networking model. This has bad consequences,
for instance: 1. the current v6 rules disallow Neighbor
Solication/Advertisement ; 2. multicast addresses can't be used ; 3.
link-local addresses are blocked too.
To solve this, this commit changes the following rules:
```
-A DOCKER-ISOLATION-STAGE-1 ! -s fdf1:a844:380c:b247::/64 -o br-21502e5b2c6c -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fdf1:a844:380c:b247::/64 -i br-21502e5b2c6c -j DROP
```
into:
```
-A DOCKER-ISOLATION-STAGE-1 ! -s fdf1:a844:380c:b247::/64 ! -i br-21502e5b2c6c -o br-21502e5b2c6c -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fdf1:a844:380c:b247::/64 -i br-21502e5b2c6c ! -o br-21502e5b2c6c -j DROP
```
These rules only limit the traffic ingressing/egressing the bridge, but
not traffic between veth on the same bridge.
Note that, the Kernel takes care of dropping invalid IPv6 packets, eg.
loopback spoofing, thus these rules don't need to be more specific.
Solve #45460.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
refreshImage is the only function used as a reducer and it doesn't use
the `filter *listContext`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Aggregate same images into one object and add the list of tags pointing
to it to the RepoTags array
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
- use assert.Check to continue the test even if a check fails
- assert the total number of images returned, not only their RepoTags
- use subtests
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Refactor GetContainerLayerSize to calculate unpacked image size only by
following the snapshot parent tree directly instead of following it by
using diff ids from image config.
This works even if the original manifest/config used to create that
container is no longer present in the content store.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Check for generic `errdefs.NotFound` rather than specific error helper
struct when checking if the error is caused by the image not being
present.
It still works for `ErrImageDoesNotExist` because it
implements the NotFound errdefs interface too.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Now that all helper functions are updated, we can use a struct-literal
for this function, which makes it slightly easier to read.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Split the buildDetailedNetworkResources function into separate functions for
collecting container attachments (`buildContainerAttachments`) and service
attachments (`buildServiceAttachments`). This allows us to get rid of the
"verbose" bool, and makes the logic slightly more transparent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Pass the endpoint and endpoint-info, instead of individual fields from the
endpoint.
- Remove redundant nil-check, as it's already checked on the call-side
in `buildDetailedNetworkResources`, which skips endpoints without info.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make the function return the constructed network.IPAM instead of applying
it to a network struct, and rename it to "buildIPAMResources".
Rewrite the function itself:
- Use struct-literals where possible to make it slightly more readable.
- Use a boolean (hasIPv4Config, hasIPv6Config) for both IPv4 and IPv6 to
check whether the IPAM-info needs to be added. This makes the logic the
same for both, and makes the processing order-independent. This also
allows for the `network.IpamInfo()` call to be skipped if it's not needed.
- Change order of "ipv4 config / ipv4 info" and "ipv6 config / ipv4 info"
blocks to make it slightly clearer (and to allow skipping the forementioned
call to `network.IpamInfo()`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the length-check into the function, and change the code to
be a basic type-case, as networkdb.PeerInfo and network.PeerInfo
are identical types.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This variable was only accessed from within LocalRegistry methods, but
due to being a package-level variable, tests had to deal with setting
and resetting it.
Move it to be a field scoped to the LocalRegistry. This simplifies the
tests, and to make this more transparent, also removing the "Setup()"
helper (which, wasn't marked as a t.Helper()).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The client's transport can only be set by newClientWithTransport, which
is not exported, and always uses a transport.HTTPTransport.
However, requestFactory is mocked in one of the tests, so keep the interface,
but make it a local, non-exported one.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The interface is not consumed anywhere, and only non-exported functions
produced one, so we can remove it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was exported, but never mutated outside of the package, and
effectively a rather "creative" way to define a method on LocalRegistry.
While un-exporting also store these paths in a field, instead of constructing
them on every call, as the results won't change during the lifecycle of the
LocalRegistry.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Split the exported SpecsPaths from the platform-specific implementations,
so that documentation can be maintained in a single location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since 0e5eaf8ee3, these implementations
were fully identical, so removing the duplicate, and move it to a
platform-agnostic file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use gotest.tools assertions
- use consts and struct-literals where possible
- use assert.Check instead of t.Fatal() where possible
- fix some unhandled errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was used by the postNetworkConnect() handler, but is handled
by the backend itself, starting with d63a5a1ff5.
Since that commit, this function was no longer used, so we can remove it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The rootChain variable that the Key function references is a
package-global slice. As the append() built-in may append to the slice's
backing array in place, it is theoretically possible for the temporary
slices in concurrent Key() calls to share the same backing array, which
would be a data race. Thankfully in my tests (on Go 1.20.6)
cap(rootChain) == len(rootChain)
held true, so in practice a new slice is always allocated and there is
no race. But that is a very brittle assumption to depend upon, which
could blow up in our faces at any time without warning. Rewrite the
implementation in a way which cannot lead to data races.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Before this change, integration test would fail fast and not execute all
test suites when one suite fails.
Change this behavior into opt-in enabled by TEST_INTEGRATION_FAIL_FAST
variable.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
It only had a single implementation, so let's remove the interface.
While changing, also renaming;
- datastore.DataStore -> datastore.Store
- datastore.NewDataStore -> datastore.New
- datastore.NewDataStoreFromConfig -> datastore.FromConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were only used internally, and ErrConntrackNotConfigurable was not used
as a sentinel error anywhere. Remove ErrConntrackNotConfigurable, and change
IsConntrackProgrammable to return an error instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
arrangeUserFilterRule uses the package-level [`ctrl` variable][1], which
holds a reference to a controller instance. This variable is set by
[`setupArrangeUserFilterRule()`][2], which is called when initialization
a controller ([`libnetwork.New`][3]).
In normal circumstances, there would only be one controller, created during
daemon startup, and the instance of the controller would be the same as
the controller that `NewNetwork` is called from, but there's no protection
for the `ctrl` variable, and various integration tests create their own
controller instance.
The global `ctrl` var was introduced in [54e7900fb89b1aeeb188d935f29cf05514fd419b][4],
with the assumption that [only one controller could ever exist][5].
This patch tries to reduce uses of the `ctrl` variable, and as we're calling
this code from inside a method on a specific controller, we inline the code
and use that specific controller instead.
[1]: 37b908aa62/libnetwork/firewall_linux.go (L12)
[2]: 37b908aa62/libnetwork/firewall_linux.go (L14-L17)
[3]: 37b908aa62/libnetwork/controller.go (L163)
[4]: 54e7900fb8
[5]: https://github.com/moby/libnetwork/pull/2471#discussion_r343457183
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was added in libnetwork through 50964c9948
and, based on the name of the function and its signature, I think it
was meant to be a test. This patch refactors it to be one.
Changing it into a test made it slightly broken:
go test -v -run TestErrorInterfaces
=== RUN TestErrorInterfaces
errors_test.go:15: Failed to detect err network not found is of type BadRequestError. Got type: libnetwork.ErrNoSuchNetwork
errors_test.go:15: Failed to detect err endpoint not found is of type BadRequestError. Got type: libnetwork.ErrNoSuchEndpoint
errors_test.go:42: Failed to detect err unknown driver "" is of type ForbiddenError. Got type: libnetwork.NetworkTypeError
errors_test.go:42: Failed to detect err unknown network id is of type ForbiddenError. Got type: *libnetwork.UnknownNetworkError
errors_test.go:42: Failed to detect err unknown endpoint id is of type ForbiddenError. Got type: *libnetwork.UnknownEndpointError
--- FAIL: TestErrorInterfaces (0.00s)
FAIL
This was because some errors were tested twice, but for the wrong type
(`NetworkTypeError`, `UnknownNetworkError`, `UnknownEndpointError`).
Moving them to the right test left no test-cases for `types.ForbiddenError`,
so I added `ActiveContainerError` to not make that part of the code feel lonely.
Other failures were because some errors were changed from `types.BadRequestError`
to a `types.NotFoundError` error in commit ba012a703a,
so I moved those to the right part.
Before this patch:
go test -v -run TestErrorInterfaces
=== RUN TestErrorInterfaces
--- PASS: TestErrorInterfaces (0.00s)
PASS
ok github.com/docker/docker/libnetwork 0.013s
After this patch:
go test -v -run TestErrorInterfaces
=== RUN TestErrorInterfaces
--- PASS: TestErrorInterfaces (0.00s)
PASS
ok github.com/docker/docker/libnetwork 0.013s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit ffd75c2e0c updated this function to
set up the DOCKER-USER chain for both iptables and ip6tables, however the
function would return early if a failure happened (instead of continuing
with the next iptables version).
This patch extracts setting up the chain to a separate function, and updates
arrangeUserFilterRule to log the failure as a warning, but continue with
the next iptables version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These functions were mostly identical, except for iptables being enabled
by default (unless explicitly disabled by config).
Rewrite the function to a enabledIptablesVersions, which returns the list
of iptables-versions that are enabled for the controller. This prevents
having to acquire a lock twice, and simplifies arrangeUserFilterRule, which
can now just iterate over the enabled versions.
Also moving this function to a linux-only file, as other platforms don't have
the iptables types defined.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
"ro-non-recursive", "ro-force-recursive", and "rro" are
now removed from the legacy mount API.
CLI may still support them via the new mount API (if we want).
Follow-up to PR 45278
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Unfortunately also brings in golang.org/x/tools as a dependency, due to
go-winio using a "tools.go" file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove gotest.tools dependency as it was only used in one test,
and only for a trivial check
- use t.TempDir()
- rename vars that collided with package types
- don't use un-keyed structs
- explicitly ignore some errors to please linters
- use iotest.ErrReader
- TestReadCloserWrapperClose: verify reading works before closing :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.8
full diff: https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.9
This is the eighth patch release of the 1.1.z release branch of runc.
The most notable change is the addition of RISC-V support, along with a
few bug fixes.
- Support riscv64.
- init: do not print environment variable value.
- libct: fix a race with systemd removal.
- tests/int: increase num retries for oom tests.
- man/runc: fixes.
- Fix tmpfs mode opts when dir already exists.
- docs/systemd: fix a broken link.
- ci/cirrus: enable some rootless tests on cs9.
- runc delete: call systemd's reset-failed.
- libct/cg/sd/v1: do not update non-frozen cgroup after frozen failed.
- CI: bump Fedora, Vagrant, bats.
- .codespellrc: update for 2.2.5.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This change add leases for all the content that will be exported, once
the image(s) are exported the lease is removed, thus letting
containerd's GC to do its job if needed. This fixes the case where
someone would remove an image that is still being exported.
This fixes the TestAPIImagesSaveAndLoad cli integration test.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
When resolving a reference that is both a Named and Digested, it could
be resolved to an image that has the same digest, but completely
different repository name.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
If image name is already an untagged digested reference, don't produce
additional digested ref.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.
We also better document the script, and follow some best practices for
both POSIX sh and Bash.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The imgSvcConfig is defined locally, and discarded if an error occurs,
so no need to use the intermediate vars here.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The interface is defined on the receiver-side, and returning concrete
types makes it more transparent what we're creating.
As these namespaced wrappers were not exported, let's inline them, so
that it's clear at a glance what it's doing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The containerdCli was somewhat confusing (is it the CLI?); let's rename
to make it match what it is :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)
The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.
This patch removes the gotest.tools code from this file.
Before this patch:
$ (exec -a libnetwork-setkey "$(which dockerd)" -help)
Usage of libnetwork-setkey:
-exec-root string
docker exec root (default "/run/docker")
-update
update golden values
With this patch applied:
$ (exec -a libnetwork-setkey "$(which dockerd)" -help)
Usage of libnetwork-setkey:
-exec-root string
docker exec root (default "/run/docker")
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `-f` to output nothing to tar if the curl fails, and `-S` to report
errors if they happen.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Use bind-mounts instead of a `COPY` for cli.sh, and use `COPY --link`
for rootlesskit's build stage.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Use a non-slash escape sequence to support mirrors with a path
component, and do not unconditionally replace the mirror in
Dockerfile.simple.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This aligns `docker build` as invoked by the Makefile with both `docker
buildx bake` as invoked by the Makefile and directly by the user.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
- Fix copy on windows plus tests
- Fix follow symlinkResolver on Windows
- Implement proper renameFile on Windows
- Fix potential nil pointer dereference
- Use RunWithPrivileges
- Fix leaking file handle
- handle mkdir race for diskwriter
- walk: avoid stat()'ing files unnecessarily
- ci: fix freebsd workflow
- update to Go 1.20
full diff: fb433841cb...36ef4d8c0d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
insecure-registries supports using CIDR notation, however, buildkit in
moby was not respecting these. We can update the RegistryHosts function
to support this by inserting the correct host into the lookup map if
it's explicitly marked as insecure.
Signed-off-by: Justin Chadwell <me@jedevc.com>
The RegistryHosts lookup function is used by both BuildKit and by the
containerd snapshotter. However, this function differs in behaviour from
the config parser for the RegistryConfig:
- The protocol for insecure registries is treated as significant by
RegistryHosts, while the RegistryConfig strips this information.
- RegistryConfig validates and deduplicates mirrors.
- RegistryConfig does not parse the insecure-registries as URLs, which
can lead to parsing opaque URLs as was possible by the RegistryHosts
function.
This patch updates the lookup function to ensure consistency.
Signed-off-by: Justin Chadwell <me@jedevc.com>
- remove some intermediate variables
- explicitly return "nil" if there's no error
- remove redundant check for response-headers being nil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The driver-configurations are only set when creating a new controller,
using the `config.OptionDriverConfig()` option that can be passed to
`New()`, and used as "read-only" after that.
Taking away any other paths that set these options, the only type used
for per-driver options are a `map[string]interface{}`, so we can change
the type from `map[string]interface{}` to a `map[string]map[string]interface{}`,
(or its "modern" variant: `map[string]map[string]any`), so that it's
no longer needed to cast the type before use.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't fail early if we can still test more, and be slightly more strict
in what error we're looking for.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The test already creates instances for each ip-version, so let's
re-use them. While changing, also use assert.Check to not fail early.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
New() allows for driver-options to be passed using the config.OptionDriverConfig.
Update the test to not manually mutate the controller's configuration after
creating.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not critical, but when used from ChainInfo, we had to construct an IPTable
based on the version of the ChainInfo, which then only used the version
we passed to get the right loopback.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make some variables local to the if-branches to be slightly more iodiomatic,
and to make clear it's only used in that branch.
Move the bestEffortLock locking later in IPtable.raw(), because that function'
could return before the lock was even needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that all consumers of these functions are passing non-empty values,
let's validate that no empty strings for either chain or table are passed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's only used internally, and it was last used in commit:
0220b06cd6
But moved into the iptables package in this commit:
998f3ce22c
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was not used for "Config", but for Networks and Endpoints.
Having this utility made it look like more than it was, and the related
test was effectively testing stdlib.
Abstracting the validation also was hiding that, while validation does
not allow "empty" names, it happily allows leading/trailing whitespace,
and does not remove that before creating networks or endpoints;
docker network create "bridge "
docker network create "bridge "
docker network create "bridge "
docker network create " bridge "
docker network create " bridge "
docker network create " bridge"
docker network ls --filter driver=bridge
NETWORK ID NAME DRIVER SCOPE
d4d53210f185 bridge bridge local
e9afba0d99de bridge bridge local
69fb7a7ba67c bridge bridge local
a452bf065403 bridge bridge local
49d96c59061d bridge bridge local
8eae1c4be12c bridge bridge local
86dd65b881b9 bridge bridge local
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove the NewGeneric utility as it was not used anywhere, except for
in tests.
Also "modernize" the type, and use `any` instead of `interface{}`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use a more modern approach to check error-types
- Touch-up grammar of the error-message
- Remove redundant "nil" check for errors, as it's never nil at that point.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
tlsconfig.Client() does various things, including reading certs and
checking them. So we may as well return early if we're not gonna be
able to use the config.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- don't use un-keyed structs
- use http consts where possible
- use errors.As instead of manually checking the error-type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use Client.buildRequest instead of a local copy of the same logic so
that we're using the same logic, and there's less chance of diverging.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.20.6 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to the compiler, cgo, the cover tool, the go command,
the runtime, and the crypto/ecdsa, go/build, go/printer, net/mail, and text/template
packages. See the Go 1.20.6 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved
Full diff: https://github.com/golang/go/compare/go1.20.5...go1.20.6
These minor releases include 1 security fixes following the security policy:
net/http: insufficient sanitization of Host header
The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.
Thanks to Bartek Nowotarski for reporting this issue.
Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374
[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.
The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in https://go.dev/issue/60374.
Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.
Before this patch, tests would fail on go1.20.6:
=== FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header
[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.
The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in https://go.dev/issue/60374.
Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.
This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.
Before this patch (using go1.20.6):
make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
=== RUN TestAttachWithTTY
attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
--- FAIL: TestAttachWithTTY (0.11s)
=== RUN TestAttachWithoutTTy
attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
--- FAIL: TestAttachWithoutTTy (0.02s)
FAIL
With this patch applied:
make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
INFO: Testing against a local daemon
=== RUN TestAttachWithTTY
--- PASS: TestAttachWithTTY (0.12s)
=== RUN TestAttachWithoutTTy
--- PASS: TestAttachWithoutTTy (0.02s)
PASS
[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Combine TestAttachWithTTY and TestAttachWithoutTTy to a single test using sub-tests
- Set up and tear-down the test-environment once
- Remove redundant client.ContainerRemove, as it's taken care of by testEnv.Clean()
- Run both tests in parallel
make TEST_FILTER=TestAttach DOCKER_GRAPHDRIVER=overlay2 TESTDEBUG=1 test-integration
Loaded image: busybox:latest
Loaded image: busybox:glibc
Loaded image: debian:bullseye-slim
Loaded image: hello-world:latest
Loaded image: arm32v7/hello-world:latest
INFO: Testing against a local daemon
=== RUN TestAttach
=== RUN TestAttach/without_TTY
=== PAUSE TestAttach/without_TTY
=== RUN TestAttach/with_TTY
=== PAUSE TestAttach/with_TTY
=== CONT TestAttach/without_TTY
=== CONT TestAttach/with_TTY
--- PASS: TestAttach (0.00s)
--- PASS: TestAttach/without_TTY (0.03s)
--- PASS: TestAttach/with_TTY (0.03s)
PASS
DONE 3 tests in 1.347s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Calling function returned from setupTest (which calls testEnv.Clean) in
a defer block inside a test that spawns parallel subtests caused the
cleanup function to be called before any of the subtest did anything.
Change the defer expressions to use `t.Cleanup` instead to call it only
after all subtests have also finished.
This only changes tests which have parallel subtests.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- use assert.Check() where possible to not fail early
- improve checks for error-types
- rename "testURL" var to be more descriptive, and use a const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some tests are testing timeouts and take a long time to run. Run the tests
in parallel, so that the test-suite takes shorter to run.
Before:
ok github.com/docker/docker/pkg/plugins 34.013s
After:
ok github.com/docker/docker/pkg/plugins 17.945s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Refactor setupRemotePluginServer() to be a helper, and to spin up a test-
server for each test instead of sharing the same instance between tests.
This allows the tests to be run in parallel without stepping on each-other's
toes (tearing down the server).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's convenient to have in the dev container when debugging issues which
reproduce consistently when deploying containers through compose.
Signed-off-by: Cory Snider <csnider@mirantis.com>
We can't upload the same file in a matrix so generate
metadata in prepare job instead. Also fixes wrong bake meta
file in merge job.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Use http.Header, which is more descriptive on intent, and we're already
importing the package in the client. Removing the "header" type also fixes
various locations where the type was shadowed by local variables named
"headers".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "version" header was added in c0afd9c873,
but used the wrong information to get the API version.
This issue was fixed in a9d20916c3, which switched
the API handler code to get the API version from the context. That change is part
of Docker Engine 20.10 (API v1.30 and up)
This patch updates the code to only set the header on APi v1.29 and older, as it's
not used by newer API versions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With this change, the API will now return a 403 instead of a 500 when
trying to create an overlay network on a non-manager node.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The commit befff0e13f inadvertendly
disabled the error returned when trying to create an overlay network on
a node which is not part of a Swarm cluster.
Since commit e3708a89cc the overlay
netdriver returns the error: `no VNI provided`.
This commit reinstate the original error message by checking if the node
is a manager before calling libnetwork's `controller.NewNetwork()`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This patch contains some optimizations I still had stashed when working
on eaa9494b71.
- Use the bytes package for handling the output of "lsof", instead of
converting to a string.
- Count the number of newlines in the output, instead of splitting the
output into a slice of strings. We're only interested in the number
of lines in the output.
- Use lsof's -F option to only print the file-descriptor for each line,
as we don't need other information.
- Use the -l, -n, and -P options to omit converting usernames, host names,
and port numbers.
From the [LSOF(8)][1] man-page:
-l This option inhibits the conversion of user ID numbers to
login names. It is also useful when login name lookup is
working improperly or slowly.
-n This option inhibits the conversion of network numbers to host
names for network files. Inhibiting conversion can make lsof run faster.
It is also useful when host name lookup is not working properly.
-P This option inhibits the conversion of port numbers to port names for network files.
Inhibiting the conversion can make lsof run a little faster.
It is also useful when host name lookup is not working properly.
Output looks something like;
lsof -lnP -Ff -p 39849
p39849
fcwd
ftxt
ftxt
f0
f1
f2
f3
f4
f5
f6
f7
f8
f9
f10
f11
Before/After:
BenchmarkGetTotalUsedFds-10 122 9479384 ns/op 10816 B/op 63 allocs/op
BenchmarkGetTotalUsedFds-10 154 7814697 ns/op 7257 B/op 60 allocs/op
[1]: https://opensource.apple.com/source/lsof/lsof-49/lsof/lsof.man.auto.html
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- return a errdefs.System if we fail to decode the registry's response
- use strconv.Itoa instead of fmt.Sprintf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon sleeps for 15 seconds at start up when the API binds to a TCP
socket with no TLS certificate set. That's what the hack/make/run script
does, but it doesn't explicitly disable tls, thus we're experiencing
this annoying delay every time we use this script.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Golang map iteration order is not guaranteed, so in some cases the built slice has it's output of order as well. This means that testing for exact warning messages in docker build output would result in random test failures, making it more annoying for end-users to test against this functionality.
Signed-off-by: Jose Diaz-Gonzalez <email@josediazgonzalez.com>
...that Swarmkit no longer needs now that it has been migrated to use
the new-style driver registration APIs.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The only remaining user is Swarmkit, which now has its own private copy
of the package tailored to its needs.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)
However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)
This would panic if the MountPoint was not a volume;
github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
created by github.com/docker/docker/daemon.(*Daemon).restore
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]
This issue was introduced in 647c2a6cdd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently moby drops ep sets before the entrypoint is executed.
This does mean that with combination of no-new-privileges the
file capabilities stops working with non-root containers.
This is undesired as the usability of such containers is harmed
comparing to running root containers.
This commit therefore sets the effective/permitted set in order
to allow use of file capabilities or libcap(3)/prctl(2) respectively
with combination of no-new-privileges and without respectively.
For no-new-privileges the container will be able to obtain capabilities
that are requested.
Signed-off-by: Luboslav Pivarc <lpivarc@redhat.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
...which ignore the config argument. Notably, none of the network
drivers referenced by Swarmkit use config, which is good as Swarmkit
unconditionally passes nil for the config when registering drivers.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Albin is currently a curator, has been contributing for various years prior
to that, and has taken on the daunting task to work on Moby's networking stack.
Albin would be a great addition to our list of maintainers and to allow him
to perform his work in these areas in a more official capacity.
I nominated Albin as maintainer, and votes passed, so opening a PR to
make it official.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Kevin is a maintainer for BuildKit, Buildx, and Docker's official GitHub
actions (among others), has been our "in-house GitHub actions expert"
for a long time, and has made significant contributions to the integration
with BuildKit, and to improve our build pipeline(s).
Kevin would be a great addition to our list of maintainers and to allow him
to perform his work in these areas in a more official capacity.
I nominated Kevin as maintainer, and votes passed, so opening a PR to
make it official.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Laura has done significant work on the containerd integration, helping
triage and fixing bugs, both in this repository, containerd, and the
docker CLI, and would make a great addition to our list of maintainers.
I nominated Laura as maintainer, and votes passed, so opening a PR to
make it official.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds an additional interval to be used by healthchecks during the
start period.
Typically when a container is just starting you want to check if it is
ready more quickly than a typical healthcheck might run. Without this
users have to balance between running healthchecks to frequently vs
taking a very long time to mark a container as healthy for the first
time.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fixes a case where on Docker For Mac if you need to bind mount the
bundles dir (e.g. to get test results back).
The unix socket does not work over oxsfs, so instead we put it in a
tmpfs.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
1. On failed start tail the daemon logs
2. Exposes generic tailing functions to make test debugging simpler
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
setupBridgeNetFiltering:
- Indicate that the bridgeInterface argument is unused (but it's needed
to satisfy the signature).
- Return instead of nullifying the err. Still not great, but I thought it
was very slightly more logical thing to do.
checkBridgeNetFiltering:
- Remove unused argument, and scope ipVerName to the branch where it's
used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
initConnection was effectively just part of the constructor; ot was not
used elsewhere. Merge the two functions to simplify things a bit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This const was added in 8301dcc6d7, before
being moved to libnetwork, and moved back, but it was never used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove local bridgeName variable that shadowed the const, but
used the same value
- remove some redundant `var` declarations, and changed fixed
values to a const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
None of these errors were string-matched anywhere, so let's change them
to be non-capitalized, as they should.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
looks like this error was added in 1cbdaebaa1,
and later moved to libnetwork in 44c96449c2
which also updated the description to something that doesn't match what
it means.
In either case, this error was never used as a special / sentinel error,
so we can just use a regular error return.
While at it, I also lower-cased the error-message; it's not string-matched
anywhere, so we can update it to make linters more happy.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- validate input variables before constructing the ChainInfo
- only construct the ChainInfo if things were successful
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Clarify that the argument to New is an exclusive upper bound.
Correct the documentation for SetAnyInRange: the end argument is
inclusive rather than exclusive.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The idm package wraps bitseq.Handle to provide an offset and
synchronization. bitseq.Handle wraps bitmap.Bitmap to provide
persistence in a datastore. As no datastore is passed and the offset is
zero, the idm.Idm instance is nothing more than a concurrency-safe
wrapper around a bitmap.Bitmap with differently-named methods. Switch
over to using bitmap.Bitmap directly, using the ovmanager driver's mutex
for concurrency control.
Hold the driver mutex for the entire duration that VXLANs are being
assigned to the new network. This makes allocating VXLANs for a network
an atomic operation.
Signed-off-by: Cory Snider <csnider@mirantis.com>
In the network.obtainVxlanID() method, the mutex only guards a local
variable and a function argument. Locking is therefore unnecessary.
The network.releaseVxlanID() method is only called in two contexts:
driver.NetworkAllocate(), where the network struct is a local variable
and network.releaseVxlanID() is only called in failure code-paths in
which the network does not escape; and driver.NetworkFree(), while the
driver mutex is held. Locking is therefore unnecessary.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.
Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.
Signed-off-by: Cory Snider <csnider@mirantis.com>
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Now that the MTU field was moved, this function only needs the BridgeConfig,
which contains all options for the default "bridge" network.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option is only used for the default bridge network; let's move the
field to that struct to make it clearer what it's used for.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The --mtu option is only used for the default "bridge" network on Linux.
On Windows, the flag is available, but ignored. As this option has been
available for a long time, and was always silently ignored, deprecating
or removing it would be a breaking change (and perhaps it's possible to
support it in future).
This patch:
- hides the option on Windows binaries
- logs a warning if the option is set to any non-zero value other than
the default on a Windows binary
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The OptionLocalKVProvider, OptionLocalKVProviderURL, and OptionLocalKVProviderConfig
options were only used in tests, so un-export them, and move them to the
test-files.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was implemented in dd4950f36d
which added a "key" field, but that field was never used anywhere, and
still appears unused.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `store.Watch()` was only used in `Controller.processEndpointCreate()`,
and skipped if the store was not "watchable" (`store.Watchable()`).
Whether a store is watchable depends on the store's datastore.scope;
local stores are not watchable;
func (ds *datastore) Watchable() bool {
return ds.scope != LocalScope
}
datastore is only initialized in two locations, and both locations set the
scope field to LocalScope:
datastore.newClient() (also called by datastore.NewDataStore()):
3e4c9d90cf/libnetwork/datastore/datastore.go (L213)
datastore.NewTestDataStore() (used in tests);
3e4c9d90cf/libnetwork/datastore/datastore_test.go (L14-L17)
Furthermore, the backing BoltDB kvstore does not implement the Watch()
method;
3e4c9d90cf/libnetwork/internal/kvstore/boltdb/boltdb.go (L464-L467)
Based on the above;
- our datastore is never Watchable()
- so datastore.Watch() is never used
This patch removes the Watchable(), Watch(), and RestartWatch() functions,
as well as the code handling watching.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The sequential field determined whether a lock was needed when storing
and retrieving data. This field was always set to true, with the exception
of NewTestDataStore() in the tests.
This field was added in a18e2f9965
to make locking optional for non-local scoped stores. Such stores are no
longer used, so we can remove this field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make the code slightly more idiomatic; remove some "var" declarations,
remove some intermediate variables and redundant error-checks, and remove
the filePerm const.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This boolean was not used anywhere, so we can remove it. Also cleaning up
the implementation a bit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The WriteOptions struct was only used to set the "IsDir" option. This option
was added in d635a8e32b
and was only supported by the etcd libkv store.
The BoltDB store does not support this option, making the WriteOptions
struct fully redundant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The only remaining kvstore is BoltDB, which doesn't use TLS connections
or authentication, so we can remove these options.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Prevent potential suggestion when many concurrent requests happen on
the /info endpoint. It's worth noting that with this change,
requests to the endpoint while another request is still in flight
will share the results, hence might be slightly incorrect (for example,
the output includes SystemTime, which may now be incorrect).
Assuming that under normal circumstances, requests will still
happen fast enough to not be shared, this may not be a problem,
but we could decide to update specific fields to not be shared.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Got a linter warning on this one, and I don't think eventFilter() was
intentionally using a value (not pointer).
> Struct containerConfig has methods on both value and pointer receivers.
> Such usage is not recommended by the Go Documentation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These aliases were not needed, and only used in a couple of places,
which made it inconsistent, so let's use the import without aliasing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
They're not used anywhere, so let's remove them; better to have
a compile error than a panic at runtime.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Add the field as a "deprecated" field in the API type.
- Don't error when failing to parse the options, but produce a warning
instead, because the client won't be able to fix issues in the daemon
configuration. This was unlikely to happen, as the daemon probably
would fail to start with an invalid config, but just in case.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece
This patch removes the fields from the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece
This patch removes the fields from the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece
This patch removes the fields from the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is an indirect dependency for github.com/fluent/fluent-logger-golang,
which does not yet use a go.mod. Update the dependency to the latest patch
release, which contains some fixes, and updates for newer go versions;
full diff: https://github.com/tinylib/msgp/compare/v1.1.6...v1.1.8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/containerd/cgroups/compare/v3.0.1...v3.0.2
relevant changes:
- cgroup2: only enable the cpuset controller if cpus or mems is specified
- cgroup1 delete: proceed to the next subsystem when a cgroup is not found
- Cgroup2: Reduce allocations for manager.Stat
- Improve performance by for pid stats (cgroups1) re-using readuint
- Reduce allocs in ReadUint64 by pre-allocating byte buffer
- cgroup2: rm/simplify some code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If an image is only by id instead of its name, don't prune it
completely. but only untag it and create a dangling image for it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This field was added in f0e6e135a8, and
from that change I suspect it was intended to store the default SELinux
mount-labels to be set on containers.
However, it was never used, so let's remove it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove the intermediate variable, and move the option closer
to where it's used, as in some cases we created the variable,
but could return with an error before it was used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was not using the DriverCallback interface, and only
required the Registerer interface.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
CI failed sometimes if no daemon.json was present:
Run sudo rm /etc/docker/daemon.json
sudo rm /etc/docker/daemon.json
sudo service docker restart
docker version
docker info
shell: /usr/bin/bash -e {0}
env:
DESTDIR: ./build
BUILDKIT_REPO: moby/buildkit
BUILDKIT_TEST_DISABLE_FEATURES: cache_backend_azblob,cache_backend_s3,merge_diff
BUILDKIT_REF: 798ad6b0ce9f2fe86dfb2b0277e6770d0b545871
rm: cannot remove '/etc/docker/daemon.json': No such file or directory
Error: Process completed with exit code 1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Linux 6.2 and up (commit [f1f1f2569901ec5b9d425f2e91c09a0e320768f3][1])
provides a fast path for the number of open files for the process.
From the [Linux docs][2]:
> The number of open files for the process is stored in 'size' member of
> `stat()` output for /proc/<pid>/fd for fast access.
[1]: f1f1f25699
[2]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files
This patch adds a fast-path for Kernels that support this, and falls back
to the slow path if the Size fields is zero.
Comparing on a Fedora 38 (kernel 6.2.9-300.fc38.x86_64):
Before/After:
go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
BenchmarkGetTotalUsedFds 57264 18595 ns/op 408 B/op 10 allocs/op
BenchmarkGetTotalUsedFds 370392 3271 ns/op 40 B/op 3 allocs/op
Note that the slow path has 1 more file-descriptor, due to the open
file-handle for /proc/<pid>/fd during the calculation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use File.Readdirnames instead of os.ReadDir, as we're only interested in
the number of files, and results don't have to be sorted.
Before:
BenchmarkGetTotalUsedFds-5 149272 7896 ns/op 945 B/op 20 allocs/op
After:
BenchmarkGetTotalUsedFds-5 153517 7644 ns/op 408 B/op 10 allocs/op
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 8d56108ffb moved this function from
the generic (no build-tags) fileutils.go to a unix file, adding "freebsd"
to the build-tags.
This likely was a wrong assumption (as other files had freebsd build-tags).
FreeBSD's procfs does not mention `/proc/<pid>/fd` in the manpage, and
we don't test FreeBSD in CI, so let's drop it, and make this a Linux-only
file.
While updating also dropping the import-tag, as we're planning to move
this file internal to the daemon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Update docker to support a '--log-format' option, which accepts either
'text' (default) or 'json'. Propagate the log format to containerd as
well, to ensure that everything will be logged consistently.
Signed-off-by: Philip K. Warren <pkwarren@gmail.com>
When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
I think this may be missing a sudo (as all other operations do use
sudo to access daemon.json);
Run if [ ! -e /etc/docker/daemon.json ]; then
if [ ! -e /etc/docker/daemon.json ]; then
echo '{}' | tee /etc/docker/daemon.json >/dev/null
fi
DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null
sudo service docker restart
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
GO_VERSION: 1.20.5
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
ITG_CLI_MATRIX_SIZE: 6
DOCKER_EXPERIMENTAL: 1
DOCKER_GRAPHDRIVER: overlay2
tee: /etc/docker/daemon.json: Permission denied
Error: Process completed with exit code 1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Dockerfile in this repository performs many stages in parallel. If any of
those stages fails to build (which could be due to networking congestion),
other stages are also (forcibly?) terminated, which can cause an unclean
shutdown.
In some case, this can cause `git` to be terminated, leaving a `.lock` file
behind in the cache mount. Retrying the build now will fail, and the only
workaround is to clean the build-cache (which causes many stages to be
built again, potentially triggering the problem again).
> [dockercli-integration 3/3] RUN --mount=type=cache,id=dockercli-integration-git-linux/arm64/v8,target=./.git --mount=type=cache,target=/root/.cache/go-build,id=dockercli-integration-build-linux/arm64/v8 /download-or-build-cli.sh v17.06.2-ce https://github.com/docker/cli.git /build:
#0 1.575 fatal: Unable to create '/go/src/github.com/docker/cli/.git/shallow.lock': File exists.
#0 1.575
#0 1.575 Another git process seems to be running in this repository, e.g.
#0 1.575 an editor opened by 'git commit'. Please make sure all processes
#0 1.575 are terminated then try again. If it still fails, a git process
#0 1.575 may have crashed in this repository earlier:
#0 1.575 remove the file manually to continue.
This patch:
- Updates the Dockerfile to remove `.lock` files (`shallow.lock`, `index.lock`)
that may have been left behind from previous builds. I put this code in the
Dockerfile itself (not the script), as the script may be used in other
situations outside of the Dockerfile (for which we cannot guarantee no other
git session is active).
- Adds a `docker --version` step to the stage; this is mostly to verify the
build was successful (and to be consistent with other stages).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
re-enable the DCO check, which was temporarily disabled to migrate
old commits from github.com/docker/libkv
This reverts commit 7d7225fae6.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ideally, this should actually do a lookup across images that have no parent, but I wasn't 100% sure how to accomplish that so I opted for the smaller change of having `FROM scratch` builds not be cached for now.
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
The migrated history has some commits that missed a DCO:
These commits do not have a proper 'Signed-off-by:' marker:
- 3fa22634a617e2c52d2c5f061826e5107e27985f
- 9b11053e9147884c43c9a9d8ebfcd7bb9470e8b5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A reduced set of the dependency, only taking the parts that are used. Taken from
upstream commit: dfacc563de
# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo
cd ~/projects
# create a temporary clone of docker
git clone https://github.com/docker/libkv.git temp_libkv
cd temp_libkv
# create branch to work with
git checkout -b migrate_libkv
# remove all code, except for the files we need; rename the remaining ones to their new target location
git filter-repo --force \
--path libkv.go \
--path store/store.go \
--path store/boltdb/boltdb.go \
--path-rename libkv.go:libnetwork/internal/kvstore/kvstore_manage.go \
--path-rename store/store.go:libnetwork/internal/kvstore/kvstore.go \
--path-rename store/boltdb/:libnetwork/internal/kvstore/boltdb/
# go to the target github.com/moby/moby repository
cd ~/projects/docker
# create a branch to work with
git checkout -b integrate_libkv
# add the temporary repository as an upstream and make sure it's up-to-date
git remote add temp_libkv ~/projects/temp_libkv
git fetch temp_libkv
# merge the upstream code, rewriting "pkg/symlink" to "symlink"
git merge --allow-unrelated-histories --signoff -S temp_libkv/migrate_libkv
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was only used in tests, and internally, and no longer
used since we switch to using os.UserHomeDir() from stdlib.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was last used in the pkg/mflag package, which was removed
in 14712f9ff0, and is no longer used in
libnetwork code since e6de8aec2f
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info, and had to be manually
set using the TEST_OSTYPE env-var.
This patch removes the field in favor of the equivalent in DaemonInfo. It's
more verbose, but also less ambiguous what information we're using (i.e.,
the platform the daemon is running on, not the local platform).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This env-var was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 4bafaa00aa, if the daemon was
killed while a container was running and the container shim is killed
before the daemon is restarted, such as if the host system is
hard-rebooted, the daemon would restore the container to the stopped
state and set the exit code to 255. The aforementioned commit introduced
a regression where the container's exit code would instead be set to 0.
Fix the regression so that the exit code is once against set to 255 on
restore.
Signed-off-by: Cory Snider <csnider@mirantis.com>
If an exec fails to start in such a way that containerd publishes an
exit event for it, daemon.ProcessEvent will race
daemon.ContainerExecStart in handling the failure. This race has been a
long-standing bug, which was mostly harmless until
4bafaa00aa. After that change, the daemon
would dereference a nil pointer and crash if ProcessEvent won the race.
Restore the status quo buggy behaviour by adding a check to skip the
dereference if execConfig.Process is nil.
Signed-off-by: Cory Snider <csnider@mirantis.com>
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.
To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
The stargz snapshotter cannot be re-mounted, so the reference-counted
path must be used.
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Commit 90de570cfa passed through the request
context to daemon.ContainerStop(). As a result, cancelling the context would
cancel the "graceful" stop of the container, and would proceed with forcefully
killing the container.
This patch partially reverts the changes from 90de570cfa
and breaks the context to prevent cancelling the context from cancelling the stop.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.
However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;
Loaded image: emptyfs:latest
Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
INFO: Building docker-sdk-python3:5.0.3...
tests/Dockerfile:6
--------------------
5 | ARG APT_MIRROR
6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
7 | >>> && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
8 |
--------------------
ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2
This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This enables picking up OTLP tracing context for the gRPC
requests.
Also sets up the in-memory recorder that BuildKit History API
can use to store the traces associated with specific build
in a database after build completes.
This doesn't enable Jaeger tracing endpoints from env
but this can be easily enabled by adding another import if
maintainers want it.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
go1.20.5 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, the
runtime, and the crypto/rsa, net, and os packages. See the Go 1.20.5 milestone
on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.20.4...go1.20.5
These minor releases include 3 security fixes following the security policy:
- cmd/go: cgo code injection
The go command may generate unexpected code at build time when using cgo. This
may result in unexpected behavior when running a go program which uses cgo.
This may occur when running an untrusted module which contains directories with
newline characters in their names. Modules which are retrieved using the go command,
i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
GO111MODULE=off, may be affected).
Thanks to Juho Nurminen of Mattermost for reporting this issue.
This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.
- runtime: unexpected behavior of setuid/setgid binaries
The Go runtime didn't act any differently when a binary had the setuid/setgid
bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
I/O file descriptors closed, opening any files could result in unexpected
content being read/written with elevated prilieges. Similarly if a setuid/setgid
program was terminated, either via panic or signal, it could leak the contents
of its registers.
Thanks to Vincent Dehors from Synacktiv for reporting this issue.
This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.
- cmd/go: improper sanitization of LDFLAGS
The go command may execute arbitrary code at build time when using cgo. This may
occur when running "go get" on a malicious module, or when running any other
command which builds untrusted code. This is can by triggered by linker flags,
specified via a "#cgo LDFLAGS" directive.
Thanks to Juho Nurminen of Mattermost for reporting this issue.
This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon.generateNewName() already reserves the generated name, but its name
did not indicate it did. The daemon.registerName() assumed that the generated
name still had to be reserved, which could mean it would try to reserve the
same name again.
This patch renames daemon.generateNewName to daemon.generateAndReserveName
to make it clearer what it does, and updates registerName() to return early
if it successfully generated (and registered) the container name.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The most notable change here is that the OCI's type uses a pointer for `Created`, which we probably should've been too, so most of these changes are accounting for that (and embedding our `Equal` implementation in the one single place it was used).
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
This utility was only used in a single location (as part of `docker info`),
but the `pkg/rootless` package is imported in various locations, causing
rootlesskit to be a dependency for consumers of that package.
Move GetRootlessKitClient to the daemon code, which is the only location
it was used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For configured runtimes with a runtimeType other than
io.containerd.runc.v1, io.containerd.runc.v2 and
io.containerd.runhcs.v1, the only supported way to pass configuration is
through the generic containerd "runtimeoptions/v1".Options type. Add a
unit test case which verifies that the options set in the daemon config
are correctly unmarshaled into the daemon's in-memory runtime config,
and that the map keys for the daemon config align with the ones used
when configuring cri-containerd (PascalCase, not camelCase or
snake_case).
Signed-off-by: Cory Snider <csnider@mirantis.com>
When constructing the client, and setting the User-Agent, care must be
taken to apply the header in the right location, as custom headers can
be set in the CLI configuration, and merging these custom headers should
not override the User-Agent header.
This patch adds a dedicated `WithUserAgent()` option, which stores the
user-agent separate from other headers, centralizing the merging of
other headers, so that other parts of the (CLI) code don't have to be
concerned with merging them in the right order.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Convert CreateMountpoint, ReadOnlyNonRecursive, and ReadOnlyForceRecursive.
See moby/swarmkit PR 3134
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Some snapshotters (like overlayfs or zfs) can't mount the same
directories twice. For example if the same directroy is used as an upper
directory in two mounts the kernel will output this warning:
overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
And indeed accessing the files from both mounts will result in an "No
such file or directory" error.
This change introduces reference counts for the mounts, if a directory
is already mounted the mount interface will only increment the mount
counter and return the mount target effectively making sure that the
filesystem doesn't end up in an undefined behavior.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Audit the OCI spec options used for Linux containers to ensure they are
less order-dependent. Ensure they don't assume that any pointer fields
are non-nil and that they don't unintentionally clobber mutations to the
spec applied by other options.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Many of the fields in LinuxResources struct are pointers to scalars for
some reason, presumably to differentiate between set-to-zero and unset
when unmarshaling from JSON, despite zero being outside the acceptable
range for the corresponding kernel tunables. When creating the OCI spec
for a container, the daemon sets the container's OCI spec CPUShares and
BlkioWeight parameters to zero when the corresponding Docker container
configuration values are zero, signifying unset, despite the minimum
acceptable value for CPUShares being two, and BlkioWeight ten. This has
gone unnoticed as runC does not distingiush set-to-zero from unset as it
also uses zero internally to represent unset for those fields. However,
kata-containers v3.2.0-alpha.3 tries to apply the explicit-zero resource
parameters to the container, exactly as instructed, and fails loudly.
The OCI runtime-spec is silent on how the runtime should handle the case
when those parameters are explicitly set to out-of-range values and
kata's behaviour is not unreasonable, so the daemon must therefore be in
the wrong.
Translate unset values in the Docker container's resources HostConfig to
omit the corresponding fields in the container's OCI spec when starting
and updating a container in order to maximize compatibility with
runtimes.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Switch to using t.TempDir() instead of rolling our own.
Clean up mounts leaked by the tests as otherwise the tests fail due to
the leaked mounts because unlike the old cleanup code, t.TempDir()
cleanup does not ignore errors from os.RemoveAll.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Avoids invalidation of dev-systemd-true and dev-base when changing the
CLI version/repository.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't show `error: No such remote: 'origin'` error when building for the
first time and the cached git repository doesn't a remote yet.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Installs the buildx cli plugin in the container shell by default.
Previously user had to manually download the buildx binary to use
buildkit.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use separate cli for integration-cli to allow use newer CLI for
interactive dev shell usage.
Both versions can be overriden with DOCKERCLI_VERSION or
DOCKERCLI_INTEGRATION_VERSION. Binary is downloaded from
download.docker.com if it's available, otherwise it's built from the
source.
For backwards compatibility DOCKER_CLI_PATH overrides BOTH clis.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
I missed the most important COPY in 637ca59375
Copying the source code into the dev-container does not depend on the parent
layers, so can use the --link option as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon has made a habit of mutating the DefaultRuntime and Runtimes
values in the Config struct to merge defaults. This would be fine if it
was a part of the regular configuration loading and merging process,
as is done with other config options. The trouble is it does so in
surprising places, such as in functions with 'verify' or 'validate' in
their name. It has been necessary in order to validate that the user has
not defined a custom runtime named "runc" which would shadow the
built-in runtime of the same name. Other daemon code depends on the
runtime named "runc" always being defined in the config, but merging it
with the user config at the same time as the other defaults are merged
would trip the validation. The root of the issue is that the daemon has
used the same config values for both validating the daemon runtime
configuration as supplied by the user and for keeping track of which
runtimes have been set up by the daemon. Now that a completely separate
value is used for the latter purpose, surprising contortions are no
longer required to make the validation work as intended.
Consolidate the validation of the runtimes config and merging of the
built-in runtimes into the daemon.setupRuntimes() function. Set the
result of merging the built-in runtimes config and default default
runtime on the returned runtimes struct, without back-propagating it
onto the config.Config argument.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The existing runtimes reload logic went to great lengths to replace the
directory containing runtime wrapper scripts as atomically as possible
within the limitations of the Linux filesystem ABI. Trouble is,
atomically swapping the wrapper scripts directory solves the wrong
problem! The runtime configuration is "locked in" when a container is
started, including the path to the runC binary. If a container is
started with a runtime which requires a daemon-managed wrapper script
and then the daemon is reloaded with a config which no longer requires
the wrapper script (i.e. some args -> no args, or the runtime is dropped
from the config), that container would become unmanageable. Any attempts
to stop, exec or otherwise perform lifecycle management operations on
the container are likely to fail due to the wrapper script no longer
existing at its original path.
Atomically swapping the wrapper scripts is also incompatible with the
read-copy-update paradigm for reloading configuration. A handler in the
daemon could retain a reference to the pre-reload configuration for an
indeterminate amount of time after the daemon configuration has been
reloaded and updated. It is possible for the daemon to attempt to start
a container using a deleted wrapper script if a request to run a
container races a reload.
Solve the problem of deleting referenced wrapper scripts by ensuring
that all wrapper scripts are *immutable* for the lifetime of the daemon
process. Any given runtime wrapper script must always exist with the
same contents, no matter how many times the daemon config is reloaded,
or what changes are made to the config. This is accomplished by using
everyone's favourite design pattern: content-addressable storage. Each
wrapper script file name is suffixed with the SHA-256 digest of its
contents to (probabilistically) guarantee immutability without needing
any concurrency control. Stale runtime wrapper scripts are only cleaned
up on the next daemon restart.
Split the derived runtimes configuration from the user-supplied
configuration to have a place to store derived state without mutating
the user-supplied configuration or exposing daemon internals in API
struct types. Hold the derived state and the user-supplied configuration
in a single struct value so that they can be updated as an atomic unit.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Ensure data-race-free access to the daemon configuration without
locking by mutating a deep copy of the config and atomically storing
a pointer to the copy into the daemon-wide configStore value. Any
operations which need to read from the daemon config must capture the
configStore value only once and pass it around to guarantee a consistent
view of the config.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Config reloading has interleaved validations and other fallible
operations with mutating the live daemon configuration. The daemon
configuration could be left in a partially-reloaded state if any of the
operations returns an error. Mutating a copy of the configuration and
atomically swapping the config struct on success is not currently an
option as config values are not copyable due to the presence of
sync.Mutex fields. Introduce a two-phase commit protocol to defer any
mutations of the daemon state until after all fallible operations have
succeeded.
Reload transactions are not yet entirely hermetic. The platform
reloading logic for custom runtimes on *nix could still leave the
directory of generated runtime wrapper scripts in an indeterminate state
if an error is encountered.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Historically, daemon.RegistryHosts() has returned a docker.RegistryHosts
callback function which closes over a point-in-time snapshot of the
daemon configuration. When constructing the BuildKit builder at daemon
startup, the return value of daemon.RegistryHosts() has been used.
Therefore the BuildKit builder would use the registry configuration as
it was at daemon startup for the life of the process, even if the
registry configuration is changed and the configuration reloaded.
Provide BuildKit with a RegistryHosts callback which reflects the
live daemon configuration after reloads so that registry operations
performed by BuildKit always use the same configuration as the rest of
the daemon.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Passing around a bare pointer to the map of configured features in order
to propagate to consumers changes to the configuration across reloads is
dangerous. Map operations are not atomic, so concurrently reading from
the map while it is being updated is a data race as there is no
synchronization. Use a getter function to retrieve the current features
map so the features can be retrieved race-free.
Remove the unused features argument from the build router.
Signed-off-by: Cory Snider <csnider@mirantis.com>
With this patch, the user-agent has information about the containerd-client
version and the storage-driver that's used when using the containerd-integration;
time="2023-06-01T11:27:07.959822887Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=53590f34-096a-4fd1-9c58-d3b8eb7e5092 service=registry version=2.8.2
...
172.18.0.1 - - [01/Jun/2023:11:30:12 +0000] "HEAD /v2/multifoo/blobs/sha256:c7ec7661263e5e597156f2281d97b160b91af56fa1fd2cc045061c7adac4babd HTTP/1.1" 404 157 "" "docker/dev go/go1.20.4 git-commit/8d67d0c1a8 kernel/5.15.49-linuxkit-pr os/linux arch/arm64 containerd-client/1.6.21+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was not really "inserting" anything, just formatting and appending.
Simplify this by changing this in to a `getUpstreamUserAgent()` function
which returns the upstream User-Agent (if any) into a `UpstreamClient()`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a const for the characters to escape, instead of implementing
this as a generic escaping function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this, the client would report itself as containerd, and the containerd
version from the containerd go module:
time="2023-06-01T09:43:21.907359755Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=67b89d83-eac0-4f85-b36b-b1b18e80bde1 service=registry version=2.8.2
...
172.18.0.1 - - [01/Jun/2023:09:43:33 +0000] "HEAD /v2/multifoo/blobs/sha256:cb269d7c0c1ca22fb5a70342c3ed2196c57a825f94b3f0e5ce3aa8c55baee829 HTTP/1.1" 404 157 "" "containerd/1.6.21+unknown"
With this patch, the user-agent has the docker daemon information;
time="2023-06-01T11:27:07.959822887Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=53590f34-096a-4fd1-9c58-d3b8eb7e5092 service=registry version=2.8.2
...
172.18.0.1 - - [01/Jun/2023:11:27:20 +0000] "HEAD /v2/multifoo/blobs/sha256:c7ec7661263e5e597156f2281d97b160b91af56fa1fd2cc045061c7adac4babd HTTP/1.1" 404 157 "" "docker/dev go/go1.20.4 git-commit/8d67d0c1a8 kernel/5.15.49-linuxkit-pr os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Fix timeouts from very long raft messages
- fix: code optimization
- update dependencies
full diff: 75e92ce14f...01bb7a4139
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Build-cache for the build-stages themselves are already invalidated if the
base-images they're using is updated, and the COPY operations don't depend
on previous steps (as there's no overlap between artifacts copied).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The default implementation of the containerd.Image interface provided by
the containerd operates on the parent index/manifest list of the image
and the platform matcher.
This isn't convenient when a specific manifest is already known and it's
redundant to search the whole index for a manifest that matches the
given platform matcher. It can also result in a different manifest
picked up than expected when multiple manifests with the same platform
are present.
This introduces a walkImageManifests which walks the provided image and
calls a handler with a ImageManifest, which is a simple wrapper that
implements containerd.Image interfaces and performs all containerd.Image
operations against a platform specific manifest instead of the root
manifest list/index.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Resolver.setupIPTable() checks whether it needs to flush or create the
user chains used for NATing container DNS requests by testing for the
existence of the rules which jump to said user chains. Unfortunately it
does so using the IPTable.RawCombinedOutputNative() method, which
returns a non-nil error if the iptables command returns any output even
if the command exits with a zero status code. While that is fine with
iptables-legacy as it prints no output if the rule exists, iptables-nft
v1.8.7 prints some information about the rule. Consequently,
Resolver.setupIPTable() would incorrectly think that the rule does not
exist during container restore and attempt to create it. This happened
work work by coincidence before 8f5a9a741b
because the failure to create the already-existing table would be
ignored and the new NAT rules would be inserted before the stale rules
left in the table from when the container was last started/restored. Now
that failing to create the table is treated as a fatal error, the
incompatibility with iptables-nft is no longer hidden.
Switch to using IPTable.ExistsNative() to test for the existence of the
jump rules as it correctly only checks the iptables command's exit
status without regard for whether it outputs anything.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The method to restore a network namespace takes a collection of
interfaces to restore with the options to apply. The interface names are
structured data, tuples of (SrcName, DstPrefix) but for whatever reason
are being passed into Restore() serialized to strings. A refactor,
f0be4d126d, accidentally broke the
serialization by dropping the delimiter. Rather than fix the
serialization and leave the time-bomb for someone else to trip over,
pass the interface names as structured data.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Switching snapshotter implementations would result in an error when
preparing a snapshot, check that the image is indeed unpacked for the
current snapshot before trying to prepare a snapshot.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Dockerfile.e2e is not used anymore. Integration tests run
through the main Dockerfile.
Also removes the daemon OS/Arch detection script that is not
necessary anymore. It was used to select the Dockerfile based
on the arch like Dockerfile.arm64 but we don't have those
anymore. Was also used to check referenced frozen images
in the Dockerfile.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Adds a Dockerfile and make targets to update and validate
generated files (proto, seccomp default profile)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
This makes the output of `docker save` fully OCI compliant.
When using the containerd image store, this code is not used. That
exporter will just use containerd's export method and should give us the
output we want for multi-arch images.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In cases where an exec start failed the exec process will be nil even
though the channel to signal that the exec started was closed.
Ideally ExecConfig would get a nice refactor to handle this case better
(ie. it's not started so don't close that channel).
This is a minimal fix to prevent NPE. Luckilly this would only get
called by a client and only the http request goroutine gets the panic
(http lib recovers the panic).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
While the VXLAN interface and the iptables rules to mark outgoing VXLAN
packets for encryption are configured to use the Swarm data path port,
the XFRM policies for actually applying the encryption are hardcoded to
match packets with destination port 4789/udp. Consequently, encrypted
overlay networks do not pass traffic when the Swarm is configured with
any other data path port: encryption is not applied to the outgoing
VXLAN packets and the destination host drops the received cleartext
packets. Use the configured data path port instead of hardcoding port
4789 in the XFRM policies.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This struct was never modified; let's just use consts for these.
Also remove the args return from detectContentType(), as it was
not used anywhere.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.
As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It turns out that the unnecessary serialization removed in
b75246202a happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.
Signed-off-by: Cory Snider <csnider@mirantis.com>
`docker run -v /foo:/foo:ro` is now recursively read-only on kernel >= 5.12.
Automatically falls back to the legacy non-recursively read-only mount mode on kernel < 5.12.
Use `ro-non-recursive` to disable RRO.
Use `ro-force-recursive` or `rro` to explicitly enable RRO. (Fails on kernel < 5.12)
Fix issue 44978
Fix docker/for-linux issue 788
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Feature flags are one of the configuration items which can be reloaded
without restarting the daemon. Whether the daemon uses the containerd
snapshotter service or the legacy graph drivers is controlled by a
feature flag. However, much of the code which checks the snapshotter
feature flag assumes that the flag cannot change at runtime. Make it so
that the snapshotter setting can only be changed by restarting the
daemon, even if the flag state changes after a live configuration
reload.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.
From the go1.19 release notes: https://go.dev/doc/go1.19#net
> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.
Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Docker with containerd integration emits "Exists" progress action when a
layer of the currently pulled image already exists. This is different
from the non-c8d Docker which emits "Already exists".
This makes both implementations consistent by emitting backwards
compatible "Already exists" action.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
To allow skipping integration tests that don't apply to the
containerd snapshotter.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This moves the blobs around so they follow the OCI spec.
Note that because docker reads paths from the manifest.json inside the
tar this is not a breaking change.
This does, however, remove the old layer "VERSION" file which had a big
"why is this even here" in the code comments. I suspect it does not
matter at all even for really old versions of Docker. In any case it is
a useless file for any even relatively modern version of Docker.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
TestProxyNXDOMAIN has proven to be susceptible to failing as a
consequence of unlocked threads being set to the wrong network
namespace. As the failure mode looks a lot like a bug in the test
itself, it seems prudent to add a check for mismatched namespaces to the
test so we will know for next time that the root cause lies elsewhere.
Signed-off-by: Cory Snider <csnider@mirantis.com>
osl.setIPv6 mistakenly captured the calling goroutine's thread's network
namespace instead of the network namespace of the thread getting its
namespace temporarily changed. As this function appears to only be
called from contexts in the process's initial network namespace, this
mistake would be of little consequence at runtime. The libnetwork unit
tests, on the other hand, unshare network namespaces so as not to
interfere with each other or the host's network namespace. But due to
this bug, the isolation backfires and the network namespace of
goroutines used by a test which are expected to be in the initial
network namespace can randomly become the isolated network namespace of
some other test. Symptoms include a loopback network server running in
one goroutine being inexplicably and randomly being unreachable by a
client in another goroutine.
Capture the original network namespace of the thread from the thread to
be tampered with, after locking the goroutine to the thread.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Swapping out the global logger on the fly is causing tests to flake out
by logging to a test's log output after the test function has returned.
Refactor Resolver to use a dependency-injected logger and the resolver
unit tests to inject a private logger instance into the Resolver under
test.
Signed-off-by: Cory Snider <csnider@mirantis.com>
tstwriter mocks the server-side connection between the resolver and the
container, not the resolver and the external DNS server, so returning
the external DNS server's address as w.LocalAddr() is technically
incorrect and misleading. Only the protocols need to match as the
resolver uses the client's choice of protocol to determine which
protocol to use when forwarding the query to the external DNS server.
While this change has no material impact on the tests, it makes the
tests slightly more comprehensible for the next person.
Signed-off-by: Cory Snider <csnider@mirantis.com>
commit dc11d2a2d8 removed the devicemapper
storage-driver, so these warnings are no longer relevant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 0abb8dec3f removed support for
running overlay/overlay2 on top of a backing filesystem without d_type
support, and turned it into a fatal error when starting the daemon,
so there's no need to generate warnings for this situation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
release notes: https://github.com/spf13/cobra/releases/tag/v1.7.0
Features
- Allow to preserve ordering of completions in bash, zsh, pwsh, & fish
- Add support for PowerShell 7.2+ in completions
- Allow sourcing zsh completion script
Bug fixes
- Don't remove flag values that match sub-command name
- Fix powershell completions not returning single word
- Remove masked template import variable name
- Correctly detect completions with dash in argument
Testing & CI/CD
- Deprecate Go 1.15 in CI
- Deprecate Go 1.16 in CI
- Add testing for Go 1.20 in CI
- Add tests to illustrate unknown flag bug
Maintenance
- Update main image to better handle dark backgrounds
- Fix stale.yaml mispellings
- Remove stale bot from GitHub actions
- Add makefile target for installing dependencies
- Add Sia to projects using Cobra
- Add Vitess and Arewefastyet to projects using cobra
- Fixup for Kubescape github org
- Fix route for GitHub workflows badge
- Fixup for GoDoc style documentation
- Various bash scripting improvements for completion
- Add Constellation to projects using Cobra
Documentation
- Add documentation about disabling completion descriptions
- Improve MarkFlagsMutuallyExclusive example in user guide
- Update shell_completions.md
- Update copywrite year
- Document suggested layout of subcommands
- Replace deprecated ExactValidArgs with MatchAll in doc
full diff: https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Extended attributes are set on files in container images for a reason.
Fail to unpack if extended attributes are present in a layer and setting
the attributes on the unpacked files fails for any reason.
Add an option to the vfs graph driver to opt into the old behaviour
where ENOTSUPP and EPERM errors encountered when setting extended
attributes are ignored. Make it abundantly clear to users and anyone
triaging their bug reports that they are shooting themselves in the
foot by enabling this option.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Our resolver is just a forwarder for external DNS so it should act like
it. Unless it's a server failure or refusal, take the response at face
value and forward it along to the client. RFC 8020 is only applicable to
caching recursive name servers and our resolver is neither caching nor
recursive.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This fixes a bug where, if a user pulls an image with a tag != `latest` and
a specific platform, we return an NotFound error for the wrong (`latest`) tag.
see: https://github.com/moby/moby/issues/45558
This bug was introduced in 779a5b3029
in the changes to `daemon/images/image_pull.go`, when we started returning the error from the call to
`GetImage` after the pull. We do this call, if pulling with a specified platform, to check if the platform
of the pulled image matches the requested platform (for cases with single-arch images).
However, when we call `GetImage` we're not passing the image tag, only name, so `GetImage` assumes `latest`
which breaks when the user has requested a different tag, since there might not be such an image in the store.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The internal Client request methods which accept an object as a body use
nil to signal that the request should not have a body. But it is easy to
accidentally pass a typed-nil value as the object, e.g. if the object
comes from a function argument or struct field of a concrete type. The
result is that these requests will, surprisingly, have a JSON body of
`null`. Treat typed-nil pointers the same as untyped nils for the
purposes of determining whether or not the request should include a
body.
Stop assuming that POST requests should always have a body. POST /commit
does not require a body, for example.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.
Additionally, this removes the "cgroups v1" setup code because it's more reasonable now for us to expect something _else_ to have set up cgroups appropriately (especially cgroups v2).
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
These changes add basic CDI integration to the docker daemon.
A cdi driver is added to handle cdi device requests. This
is gated by an experimental feature flag and is only supported on linux
This change also adds a CDISpecDirs (cdi-spec-dirs) option to the config.
This allows the default values of `/etc/cdi`, /var/run/cdi` to be overridden
which is useful for testing.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
When the `ServerAddress` in the `AuthConfig` provided by the client is
empty, default to the default registry (registry-1.docker.io).
This makes the behaviour the same as with the containerd image store
integration disabled.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
- use is.ErrorType
- replace uses of client.IsErrNotFound for errdefs.IsNotFound, as
the client no longer returns the old error-type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use is.ErrorType
- replace uses of client.IsErrNotFound for errdefs.IsNotFound, as
the client no longer returns the old error-type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use is.ErrorType
- replace uses of client.IsErrNotFound for errdefs.IsNotFound, as
the client no longer returns the old error-type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
None of the client will return the old error-types, so there's no need
to keep the compatibility code. We can consider deprecating this function
in favor of the errdefs equivalent this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)
Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Our templates no longer contain version-specific rules, so this function
is no longer used. This patch deprecates it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 7008a51449 removed version-conditional
rules from the template, so we no longer need the apparmor_parser Version.
This patch removes the call to `aaparser.GetVersion()`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 2e19a4d56b removed all other version-
conditional statements from the AppArmor template, but left this one in place.
These conditions were added in 8cf89245f5
to account for old versions of debian/ubuntu (apparmor_parser < 2.9)
that lacked some options;
> This allows us to use the apparmor profile we have in contrib/apparmor/
> and solves the problems where certain functions are not apparent on older
> versions of apparmor_parser on debian/ubuntu.
Those patches were from 2015/2016, and all currently supported distro
versions should now have more current versions than that. Looking at the
oldest supported versions;
Ubuntu 18.04 "Bionic":
apparmor_parser --version
AppArmor parser version 2.12
Copyright (C) 1999-2008 Novell Inc.
Copyright 2009-2012 Canonical Ltd.
Debian 10 "Buster"
apparmor_parser --version
AppArmor parser version 2.13.2
Copyright (C) 1999-2008 Novell Inc.
Copyright 2009-2018 Canonical Ltd.
This patch removes the remaining conditionals.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `execDuration` field to the event attributes map. This is useful for tracking how long the container ran.
Signed-off-by: Dorin Geman <dorin.geman@docker.com>
Ports over all the previous image delete logic, such as:
- Introduce `prune` and `force` flags
- Introduce the concept of hard and soft image delete conflics, which represent:
- image referenced in multiple tags (soft conflict)
- image being used by a stopped container (soft conflict)
- image being used by a running container (hard conflict)
- Implement delete logic such as:
- if deleting by reference, and there are other references to the same image, just
delete the passed reference
- if deleting by reference, and there is only 1 reference and the image is being used
by a running container, throw an error if !force, or delete the reference and create
a dangling reference otherwise
- if deleting by imageID, and force is true, remove all tags (otherwise soft conflict)
- if imageID, check if stopped container is using the image (soft conflict), and
delete anyway if force
- if imageID was passed in, check if running container is using the image (hard conflict)
- if `prune` is true, and the image being deleted has dangling parents, remove them
This commit also implements logic to get image parents in c8d by comparing shared layers.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This option was deprecated in 5a922dc162, which
is part of the v24.0.0 release, so we can remove it from master.
This patch;
- adds a check to ValidatePlatformConfig, and produces a fatal error
if oom-score-adjust is set
- removes the deprecated libcontainerd/supervisor.WithOOMScore
- removes the warning from docker info
With this patch:
dockerd --oom-score-adjust=-500 --validate
Flag --oom-score-adjust has been deprecated, and will be removed in the next release.
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.
And when using `daemon.json`:
dockerd --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was deprecated in dbb48e4b29, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was deprecated in 818ee96219, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were deprecated in 9d5e754caa, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was deprecated in 9f3e5eead5, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were deprecated in 2d49080056, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was deprecated in c63ea32a17, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This const was deprecated in 5c78cbd3be, which
is part of the v24.0.0 release, so we can remove it from master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field is deprecated since 1261fe69a3,
and will now be omitted on API v1.44 and up for the `GET /images/json`,
`GET /images/{id}/json`, and `GET /system/df` endpoints.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 1261fe69a3 deprecated the VirtualSize
field, but forgot to mention that it's also included in the /system/df
endpoint.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The 24.0 branch was created, so changes in master/main should now be
targeting the next version of the API (1.44).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a workaround to have buildinfo with deps embedded in the
binary. We need to create a go.mod file before building with
-modfile=vendor.mod, otherwise it fails with:
"-modfile cannot be used to set the module root directory."
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
This sets BuildKit version from the build information embedded
in running binary so we are aligned with the expected vendoring.
We iterate over all dependencies and find the BuildKit one
and set the right version. We also check if the module is
replaced and use it this case.
There is also additional checks if a pseudo version is
detected. See comments in code for more info.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Encrypted overlay networks are unique in that they are the only kind of
network for which libnetwork programs an iptables rule to explicitly
accept incoming packets. No other network driver does this. The overlay
driver doesn't even do this for unencrypted networks!
Because the ACCEPT rule is appended to the end of INPUT table rather
than inserted at the front, the rule can be entirely inert on many
common configurations. For example, FirewallD programs an unconditional
REJECT rule at the end of the INPUT table, so any ACCEPT rules appended
after it have no effect. And on systems where the rule is effective, its
presence may subvert the administrator's intentions. In particular,
automatically appending the ACCEPT rule could allow incoming traffic
which the administrator was expecting to be dropped implicitly with a
default-DROP policy.
Let the administrator always have the final say in how incoming
encrypted overlay packets are filtered by no longer automatically
programming INPUT ACCEPT iptables rules for them.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This has been around for a long time - since v17.04 (API v1.28)
but was never documented.
It allows removing a plugin even if it's still in use.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
While doing a boltdb operation and if the bucket is not found
we should not return a boltdb specific bucket not found error
because this causes leaky abstraction where in the user of libkv
needs to know about boltdb and import boltdb dependencies
neither of which is desirable. Replaced all the bucket not found
errors with the more generic `store.ErrKeyNotFound` error which
is more appropriate.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
When using AtomicPut with 'previous' set at nil, it interprets
that the Key should be created with the AtomicPut. Instead of
returning a generic error, we return store.ErrKeyExists if the
key exists in the store during the operation.
Signed-off-by: Alexandre Beslic <abronan@docker.com>
Currently boltdb uses a handle which can be accessed
concurrently from multiple go routines and all of them
trying to open and close the boldb db handle which can
cause havoc. Use a mutex to serialize db access and
handle access.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
This commit migrates the old 'go-etcd' client, which is deprecated
to the new 'coreos/etcd/client'.
One notable change is the ability to specify an 'IsDir' parameter
to the 'Put' call. This allows to circumvent the limitations of etcd
regarding the key/directory distinction while setting up Watches on
a directory. A conservative measure to set up a watch that should be
used the same way for etcd/consul/zookeeper is to enforce the 'IsDir'
parameter with 'WriteOptions' on 'Put' to avoid the 'NotANode' error
thrown by etcd on Watch call. Consul and zookeeper are not using the
'IsDir' parameter.
Signed-off-by: Alexandre Beslic <abronan@docker.com>
AtomicPut can now be used to Compare-and-Swap against the state
where the key doesn't yet exist. E.g. a race where two clients
create the same key: one succeeds, the other fails.
Pass nil for the previous argument of AtomicPut for this
behavior. Before, this would cause an error.
Implements this change for all three backends.
- G204 # G204:Subprocess launched with variable; too many false positives.
- G301 # G301:Expect directory permissions to be 0750 or less (also EXC0009); too restrictive
- G302 # G302:Expect file permissions to be 0600 or less (also EXC0009); too restrictive
- G304 # G304:Potential file inclusion via variable.
- G306 # G306:Expect WriteFile permissions to be 0600 or less (too restrictive; also flags "0o644" permissions)
- G307 # G307:Deferring unsafe method "*os.File" on type "Close" (also EXC0008); (TODO: evaluate these and fix where needed: G307: Deferring unsafe method "*os.File" on type "Close")
- G504 # G504: Blocklisted import net/http/cgi: Go versions < 1.6.3 are vulnerable to Httpoxy attack:(CVE-2016-5386); (only affects go < 1.6.3)
govet:
enable-all:true
disable:
- fieldalignment # TODO:evaluate which ones should be updated.
importas:
# Do not allow unaliased imports of aliased packages.
no-unaliased:true
alias:
# Enforce alias to prevent it accidentally being used instead of our
DOCKER_GITCOMMIT:=$(shell git rev-parse --short HEAD ||echo unsupported)
DOCKER_GITCOMMIT:=$(shell git rev-parse HEAD)
exportDOCKER_GITCOMMIT
# allow overriding the repository and branch that validation scripts are running
@@ -20,6 +10,9 @@ export VALIDATE_REPO
exportVALIDATE_BRANCH
exportVALIDATE_ORIGIN_BRANCH
exportPAGER
exportGIT_PAGER
# env vars passed through directly to Docker's build scripts
# to allow things like `make KEEPBUNDLE=1 binary` easily
# `project/PACKAGERS.md` have some limited documentation of some of these
@@ -28,10 +21,9 @@ export VALIDATE_ORIGIN_BRANCH
# option of "go build". For example, a built-in graphdriver priority list
# can be changed during build time like this:
#
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper" dynbinary
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,zfs" dynbinary
#
DOCKER_ENVS:=\
-e BUILD_APT_MIRROR \
-e BUILDFLAGS \
-e KEEPBUNDLE \
-e DOCKER_BUILD_ARGS \
@@ -39,10 +31,14 @@ DOCKER_ENVS := \
-e DOCKER_BUILD_OPTS \
-e DOCKER_BUILD_PKGS \
-e DOCKER_BUILDKIT \
-e DOCKER_BASH_COMPLETION_PATH \
-e DOCKER_CLI_PATH \
-e DOCKERCLI_VERSION \
-e DOCKERCLI_REPOSITORY \
-e DOCKERCLI_INTEGRATION_VERSION \
-e DOCKERCLI_INTEGRATION_REPOSITORY \
-e DOCKER_DEBUG \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_FIREWALLD \
-e DOCKER_GITCOMMIT \
-e DOCKER_GRAPHDRIVER \
-e DOCKER_LDFLAGS \
@@ -58,8 +54,10 @@ DOCKER_ENVS := \
-e TEST_FORCE_VALIDATE \
-e TEST_INTEGRATION_DIR \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_INTEGRATION_FAIL_FAST \
-e TEST_SKIP_INTEGRATION \
-e TEST_SKIP_INTEGRATION_CLI \
-e TEST_IGNORE_CGROUP_CHECK \
-e TESTCOVERAGE \
-e TESTDEBUG \
-e TESTDIRS \
@@ -75,7 +73,12 @@ DOCKER_ENVS := \
-e PLATFORM \
-e DEFAULT_PRODUCT_LICENSE \
-e PRODUCT \
-e PACKAGER_NAME
-e PACKAGER_NAME\
-e PAGER \
-e GIT_PAGER \
-e OTEL_EXPORTER_OTLP_ENDPOINT \
-e OTEL_EXPORTER_OTLP_PROTOCOL \
-e OTEL_SERVICE_NAME
# note: we _cannot_ add "-e DOCKER_BUILDTAGS" here because even if it's unset in the shell, that would shadow the "ENV DOCKER_BUILDTAGS" set in our Dockerfile, which is very important for our official builds
# to allow `make BIND_DIR=. shell` or `make BIND_DIR= test`
@@ -83,7 +86,7 @@ DOCKER_ENVS := \
# note: BINDDIR is supported for backwards-compatibility here
Moby is an open-source project created by Docker to enable and accelerate software containerization.
@@ -32,7 +37,7 @@ New projects can be added if they fit with the community goals. Docker is commit
However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.
The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful.
The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.
The releases are supported by the maintainers, community and users, on a best efforts basis only. For customers who want enterprise or commercial support, [Docker Desktop](https://www.docker.com/products/docker-desktop/) and [Mirantis Container Runtime](https://www.mirantis.com/software/mirantis-container-runtime/) are the appropriate products for these use cases.
The Moby maintainers take security seriously. If you discover a security issue, please bring it to their attention right away!
The maintainers of the Moby project take security seriously. If you discover
a security issue, please bring it to their attention right away!
### Reporting a Vulnerability
## Reporting a Vulnerability
Please **DO NOT** file a public issue, instead send your report privately to security@docker.com.
Please **DO NOT** file a public issue, instead send your report privately
to [security@docker.com](mailto:security@docker.com).
Security reports are greatly appreciated and we will publicly thank you for it, although we keep your name confidential if you request it. We also like to send gifts—if you're into schwag, make sure to let us know. We currently do not offer a paid security bounty program, but are not ruling it out in the future.
Reporter(s) can expect a response within 72 hours, acknowledging the issue was
received.
## Review Process
After receiving the report, an initial triage and technical analysis is
performed to confirm the report and determine its scope. We may request
additional information in this stage of the process.
Once a reviewer has confirmed the relevance of the report, a draft security
advisory will be created on GitHub. The draft advisory will be used to discuss
the issue with maintainers, the reporter(s), and where applicable, other
affected parties under embargo.
If the vulnerability is accepted, a timeline for developing a patch, public
disclosure, and patch release will be determined. If there is an embargo period
on public disclosure before the patch release, the reporter(s) are expected to
participate in the discussion of the timeline and abide by agreed upon dates
for public disclosure.
## Accreditation
Security reports are greatly appreciated and we will publicly thank you,
although we will keep your name confidential if you request it. We also like to
send gifts - if you're into swag, make sure to let us know. We do not currently
offer a paid security bounty program at this time.
## Supported Versions
This project uses long-lived branches to maintain releases. Refer to
[BRANCHES-AND-TAGS.md](project/BRANCHES-AND-TAGS.md) in the default branch to
learn about the current maintenance status of each branch.
@@ -37,6 +37,6 @@ There is hopefully enough example material in the file for you to copy a similar
When you make edits to `swagger.yaml`, you may want to check the generated API documentation to ensure it renders correctly.
Run `make swagger-docs` and a preview will be running at `http://localhost`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation.
Run `make swagger-docs` and a preview will be running at `http://localhost:9000`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation.
The production documentation is generated by vendoring `swagger.yaml` into [docker/docker.github.io](https://github.com/docker/docker.github.io).
// TODO(thaJeztah): unify this with Server.makeHTTPHandler, which also logs internal server errors as error-log. See https://github.com/moby/moby/pull/48740#discussion_r1816675574
fields["error-response"]=retErr
fields["status"]=httpstatus.FromError(retErr)
logger.WithFields(fields).Debugf("error response for %s request",r.Method)
}
}()
ifr.Method!=http.MethodPost{
returnhandler(ctx,w,r,vars)
@@ -42,11 +62,15 @@ func DebugRequestMiddleware(handler func(ctx context.Context, w http.ResponseWri
varpostFormmap[string]interface{}
iferr:=json.Unmarshal(b,&postForm);err==nil{
maskSecretKeys(postForm)
formStr,errMarshal:=json.Marshal(postForm)
iferrMarshal==nil{
logrus.Debugf("formdata: %s",string(formStr))
// TODO(thaJeztah): is there a better way to detect if we're using JSON-formatted logs?
expectedErr:fmt.Sprintf("invalid API version: the minimum API version (%s) is higher than the default version (%s)",api.DefaultVersion,api.MinSupportedAPIVersion),
},
{
doc:"invalid default too low",
defaultVersion:"0.1",
minVersion:api.MinSupportedAPIVersion,
expectedErr:fmt.Sprintf("invalid default API version (0.1): must be between %s and %s",api.MinSupportedAPIVersion,api.DefaultVersion),
},
{
doc:"invalid default too high",
defaultVersion:"9999.9999",
minVersion:api.DefaultVersion,
expectedErr:fmt.Sprintf("invalid default API version (9999.9999): must be between %s and %s",api.MinSupportedAPIVersion,api.DefaultVersion),
},
{
doc:"invalid minimum too low",
defaultVersion:api.MinSupportedAPIVersion,
minVersion:"0.1",
expectedErr:fmt.Sprintf("invalid minimum API version (0.1): must be between %s and %s",api.MinSupportedAPIVersion,api.DefaultVersion),
},
{
doc:"invalid minimum too high",
defaultVersion:api.DefaultVersion,
minVersion:"9999.9999",
expectedErr:fmt.Sprintf("invalid minimum API version (9999.9999): must be between %s and %s",api.MinSupportedAPIVersion,api.DefaultVersion),
errString:"client version 0.1 is too old. Minimum supported API version is 1.2.0, please upgrade your client to a newer version",
errString:fmt.Sprintf("client version 0.1 is too old. Minimum supported API version is %s, please upgrade your client to a newer version",api.MinSupportedAPIVersion),
},
{
reqVersion:"9999.9999",
errString:"client version 9999.9999 is too new. Maximum supported API version is 1.10.0",
errString:fmt.Sprintf("client version 9999.9999 is too new. Maximum supported API version is %s",api.DefaultVersion),
// Driver was configured for this mount, so no ambiguity.
continue
}
return"WARNING: the container-wide volume-driver configuration is ignored for volumes specified via 'mount'. Use '--mount type=volume,volume-driver=...' instead"
}
return""
}
// handleMACAddressBC takes care of backward-compatibility for the container-wide MAC address by mutating the
// networkingConfig to set the endpoint-specific MACAddress field introduced in API v1.44. It returns a warning message
// or an error if the container-wide field was specified for API >= v1.44.
return"",errors.Wrap(err,"unable to migrate container-wide MAC address to a specific network")
}
// ep is the endpoint that needs the container-wide MAC address; migrate the address
// to it, or bail out if there's a mismatch.
ifep.MacAddress==""{
ep.MacAddress=deprecatedMacAddress
}elseifep.MacAddress!=deprecatedMacAddress{
return"",errdefs.InvalidParameter(errors.New("the container-wide MAC address must match the endpoint-specific MAC address for the main network, or be left empty"))
}
}
warning="The container-wide MacAddress field is now deprecated. It should be specified in EndpointsConfig instead."
config.MacAddress=""//nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
returnwarning,nil
}
// handleSysctlBC migrates top level network endpoint-specific '--sysctl'
// settings to an DriverOpts for an endpoint. This is necessary because sysctls
// are applied during container task creation, but sysctls that name an interface
// (for example 'net.ipv6.conf.eth0.forwarding') cannot be applied until the
// interface has been created. So, these settings are removed from hostConfig.Sysctls
// and added to DriverOpts[netlabel.EndpointSysctls].
//
// Because interface names ('ethN') are allocated sequentially, and the order of
// network connections is not deterministic on container restart, only 'eth0'
// would work reliably in a top-level '--sysctl' option, and then only when
// there's a single initial network connection. So, settings for 'eth0' are
// migrated to the primary interface, identified by 'hostConfig.NetworkMode'.
// Settings for other interfaces are treated as errors.
//
// In the DriverOpts, because the interface name cannot be determined in advance, the
// interface name is replaced by "IFNAME". For example, 'net.ipv6.conf.eth0.forwarding'
// becomes 'net.ipv6.conf.IFNAME.forwarding'. The value in DriverOpts is a
// comma-separated list.
//
// A warning is generated when settings are migrated.
funchandleSysctlBC(
hostConfig*container.HostConfig,
netConfig*network.NetworkingConfig,
versionstring,
)(string,error){
if!hostConfig.NetworkMode.IsPrivate(){
return"",nil
}
varep*network.EndpointSettings
vartoDelete[]string
varnetIfSysctls[]string
fork,v:=rangehostConfig.Sysctls{
// If the sysctl name matches "net.*.*.eth0.*" ...
expError:"unable to migrate container-wide MAC address to a specific network: HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks",
returntypes.NetworkResource{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on name)",term,len(listByFullName)))
returnnetwork.Inspect{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on name)",term,len(listByFullName)))
}
// Find based on partial ID, returns true only if no duplicates
returntypes.NetworkResource{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on ID prefix)",term,len(listByPartialID)))
returnnetwork.Inspect{},errdefs.InvalidParameter(errors.Errorf("network %s is ambiguous (%d matches found based on ID prefix)",term,len(listByPartialID)))
// it's part of ContainerJSONBase and returned by "inspect" command
typeStatestruct{
Statusstring// String representation of the container state. Can be one of "created", "running", "paused", "restarting", "removing", "exited", or "dead"
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
