Commit Graph

403 Commits

Author SHA1 Message Date
Derek McGowan
afd6487b2e Create github.com/moby/moby/api module
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-21 09:30:05 -07:00
Derek McGowan
f05652867d Move opts to daemon/pkg/opts
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-07-14 09:25:05 -07:00
Matthieu MOREL
9b5d8cd186 fix thelper linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-05 21:42:05 +00:00
Rob Murray
37259540e9 Remove/replace integration-cli tests that use iptables directly
In preparation for testing with an nftables backend.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-05-12 17:51:59 +01:00
Rob Murray
3fc5bfd565 Merge pull request #49778 from robmry/fix_delete_legacy_link
Fix removal of legacy links
2025-04-10 15:16:00 +01:00
Rob Murray
4d35864c3d Fix removal of legacy links
It's possible to remove a legacy link from running containers.
When that happens, the Sandbox's Endpoints are removed and
re-added.

Since commit 65120d5 ("Create bridge veth in container netns")
the veth device has been created in the container's netns. When
that happens, a flag is set on the Endpoint to note that it
does not need to be moved into the netns.

But, during the Leave/Join (Sandbox.Refresh) the veth is moved
out of the netns. So, clear the flag during the Leave, to note
that it needs to be moved back in during the Join.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-04-10 14:22:40 +01:00
Derek McGowan
cd89a35ea0 Run CLI tests with cgroups v2
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-04-09 11:05:14 -07:00
Rob Murray
984a610a21 Add chain DOCKER-FORWARD
In 28.0.0, Docker appended to the FORWARD chain - breaking other
applications that had appended their own rules that needed to execute
after Docker's rules.

Move most of Docker's rules out of the filter-FORWARD chain into a
new DOCKER-FORWARD chain, so that Docker can append to DOCKER-FORWARD
without affecting the order of rules in the FORWARD chain.

After daemon startup inserts jumps to DOCKER-USER and DOCKER-FORWARD,
the bridge driver will not touch the FORWARD chain again. DOCKER-INGRESS
is still added to the FORWARD chain, if used, as it was in 27.x and
earlier.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-02-24 17:09:09 +00:00
Rob Murray
bc130f367d bridge/macvlan endpoints always use a random MAC address
Now a gratuitous/unsolicted ARP is sent, there's no need to
use an IPv4-based MAC address to preserve arp-cache mappings
between an endpoint's IP addresses and its MAC addresses.

Because a random MAC address is used for the default bridge,
it no longer makes sense to derive container IPv6 addresses
from the MAC address. This "postIPv6" behaviour was needed
before IPv6 addresses could be configured, but not now. So,
IPv6 addresses will now be IPAM-allocated on the default
bridge network, just as they are for user-defined bridges.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2025-01-22 16:59:27 +00:00
Albin Kerouanton
f053beb34c libnet/osl: drop netns path GC
Commit 3ec19ff62b introduced a GC goroutine to delete files where netns
were mounted. It was primarly added to work around a race in kernel
3.18-4.0.1. Since no distros we support are using such old kernels,
there's no need to keep this code around.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-12-16 11:40:18 +01:00
Aleksa Sarai
0553d3d994 tests: migrate away from assert.Assert(err == nil)
Unfortunately, gofmt doesn't know about types so it was necessary to
find all of the err == nil statements through trial and error. Note that
there is no is.NilError, so for assert.Check(t, err == nil) we need to
switch to just doing assert.Check(t, err). If err is an error type, this
is equivalent (and there isn't another trivial way of doing it). Here
are the full set of rules used:

Generic "err == nil":

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, err == nil) -> assert.NilError(t, err)"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, err == nil) -> assert.Check(t, err)"

Generic, but with a different variable name:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, sr.err == nil) -> assert.NilError(t, sr.err)"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, sr.err == nil) -> assert.Check(t, sr.err)"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, err2 == nil) -> assert.NilError(t, err2)"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, err2 == nil) -> assert.Check(t, err2)"

JSON-related error assertions:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, json.Unmarshal(a, b) == nil) -> assert.NilError(t, json.Unmarshal(a, b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, json.Unmarshal(a, b) == nil) -> assert.Check(t, json.Unmarshal(a, b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, json.NewDecoder(a).Decode(b) == nil) -> assert.NilError(t, json.NewDecoder(a).Decode(b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, json.NewDecoder(a).Decode(b) == nil) -> assert.Check(t, json.NewDecoder(a).Decode(b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, json.NewEncoder(a).Encode(b) == nil) -> assert.NilError(t, json.NewEncoder(a).Encode(b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, json.NewEncoder(a).Encode(b) == nil) -> assert.Check(t, json.NewEncoder(a).Encode(b))"

Process-related error assertions:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, a.Start() == nil) -> assert.NilError(t, a.Start())"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, a.Start() == nil) -> assert.Check(t, a.Start())"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, a.Kill() == nil) -> assert.NilError(t, a.Kill())"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, a.Kill() == nil) -> assert.Check(t, a.Kill())"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, a.Signal(b) == nil) -> assert.NilError(t, a.Signal(b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, a.Signal(b) == nil) -> assert.Check(t, a.Signal(b))"

waitInspect:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, waitInspect(a, b, c, d) == nil) -> assert.NilError(t, waitInspect(a, b, c, d))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, waitInspect(a, b, c, d) == nil) -> assert.Check(t, waitInspect(a, b, c, d))"

File closing error assertions:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, a.Close() == nil) -> assert.NilError(t, a.Close())"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, a.Close() == nil) -> assert.Check(t, a.Close())"

mount.MakeRShared:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, mount.MakeRShared(a) == nil) -> assert.NilError(t, mount.MakeRShared(a))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, mount.MakeRShared(a) == nil) -> assert.Check(t, mount.MakeRShared(a))"

daemon.SwarmLeave:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, d.SwarmLeave(a, b, c) == nil) -> assert.NilError(t, d.SwarmLeave(a, b, c))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, d.SwarmLeave(a, b, c) == nil) -> assert.Check(t, d.SwarmLeave(a, b, c))"

os.MkdirAll:

  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Assert(t, os.MkdirAll(a, b) == nil) -> assert.NilError(t, os.MkdirAll(a, b))"
  find . -type f -name "*_test.go" | \
    xargs gofmt -w -r "assert.Check(t, os.MkdirAll(a, b) == nil) -> assert.Check(t, os.MkdirAll(a, b))"

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2024-11-22 23:59:20 +11:00
Aleksa Sarai
5e4e34a966 tests: migrate strings.Contains -> is.Contains in assertions
Migrated using

  find . -type f -name "*_test.go" |
    xargs gofmt -w \
      -r "assert.Check(t, strings.Contains(a, b)) -> assert.Check(t, is.Contains(a, b))"

  find . -type f -name "*_test.go" |
    xargs gofmt -w \
      -r "assert.Assert(t, strings.Contains(a, b)) -> assert.Assert(t, is.Contains(a, b))"

Using a boolean in assert.Assert or assert.Check results in error
messages that don't contain the actual problematic string, and when
running the integration suite on an actual machine (where the source
code parsing doesn't work) this makes it almost impossible to figure out
what the actual error is.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2024-11-22 23:59:20 +11:00
Sebastiaan van Stijn
41b5645ba2 integration-cli: TestDaemonStartWithDefaultTLSHost: fix G402: TLS MinVersion too low
integration-cli/docker_cli_daemon_test.go:1528:101: G402: TLS MinVersion too low. (gosec)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-11-06 12:50:22 +01:00
Rob Murray
b3fabedecc Create docker-proxy TCP/UDP listener sockets in the daemon
Before commit 4f09af6, when allocating host ports for a new
port mapping, iptables rules were set up then docker-proxy was
started. If the host port was already in-use, docker-proxy exited
with an error, and the iptables rules were removed. That could
potentially interfere with a non-docker service that was already
using the host port for something unrelated.

Commit 4f09af6 swapped that problem for a different one... in
order to check that a port was available before creating iptables
rules, it attempted to start docker-proxy first. If it failed, it
could then try a different host port, without interfering with
any other service. The problem with that is docker-proxy would
start listening before the iptables rules were in place, so it
could accept connections then become unusable because new NAT
rules diverted packets directly to the container. This would leave
the client with a broken connection, causing at-least a delay
while it figured that out and reconnected.

This change creates and binds the socket in the daemon, before
creating iptables rules. If the bind fails, it may try a different
port. When or if the bind succeeds, iptables rules are created,
then the daemon calls listen on the socket. If docker-proxy is
needed, the socket is handed over to it at that point.

In rootless mode, the ports have to be bound to an address in the
rootless network namespace (where dockerd is running). DNAT rules
now use the same address.

If docker-proxy is not needed ("--userland-proxy=false"), the daemon
still listens on TCP sockets as the old dummyProxy would have done.
This makes the socket show up in "netstat" output.

The dummyProxy is no longer needed on Linux. Its job was to bind the
host ports if docker-proxy was disabled, but that's now already
handled by binding the sockets early.

This change doesn't affect SCTP, because it's not currently possible
for docker-proxy to convert the file descriptor into an SCTPListener.
So, docker-proxy is still started early, and the window for lost
connections remains.

If the user has an old docker-proxy in their path and it's given a
listener docker with '-use-listen-fd', it'll fail because of the
unknown option. In this case, the daemon's error message suggests
checking $PATH.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-08-05 14:04:05 +01:00
Sebastiaan van Stijn
c3ac7fee26 integration-cli: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 23:18:48 +02:00
Rob Murray
4aff2fbc98 Remove integration-cli TestDaemonSuite/TestDaemonIPv6Enabled
The test hadn't been running, because it used testRequires(c, IPv6)
and predicate "IPv6" returns the opposite of the expected result.

TestDaemonIPv6Enabled tried to run with IPv6 on the default bridge,
but didn't set up a "fixed-cidr-v6" - so the daemon wouldn't start.

It then tried to check the bridge had address "fe80::1", which it
expected to work because it had just used setupV6() to add that
address.

Then it  checked that "LinkLocalIPv6Address" was set in container
inspect output, but it wouldn't be (the field is deprecated).

There are working IPv6 tests in the suite (TestDaemonIPv6FixedCIDR,
TestDaemonIPv6FixedCIDRAndMac, TestDaemonIPv6HostMode) - and there's
more coverage in the network integration tests.

So, deleted the test as it didn't seem worth salvaging.

Also deleted now-unused helper functions setupV6(), teardownV6().

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-05-01 19:26:06 +01:00
Sebastiaan van Stijn
4adc40ac40 fix duplicate words (dupwords)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 10:57:03 +01:00
Sebastiaan van Stijn
20bd690844 integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
4f9db655ed portmapper: move userland-proxy lookup to daemon config
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).

This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

However, the container would still be created (but invalid);

    docker ps -a
    CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS    PORTS     NAMES
    869f41d7e94f   nginx:alpine   "/docker-entrypoint.…"   10 seconds ago   Created             romantic_wiles

This patch changes how the userland-proxy is configured;

- The path of the userland-proxy is now looked up / configured at daemon
  startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
  produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
  is now required to be set. It no longer looks up the executable, and
  produces an error if no path was provided. While this change was not
  required, it makes the daemon config the canonical source of truth, instead
  of logic spread accross multiplee locations.

Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;

- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
  preference in PATH)

With this patch:

Validating the config produces an error if the binary is not found:

    dockerd --validate
    WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    userland-proxy is enabled, but userland-proxy-path is not set

Disabling userland-proxy prints a warning, but validates as "OK":

    dockerd --userland-proxy=false --validate
    WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    configuration OK

Speficying a non-absolute path produces an error:

    dockerd --userland-proxy-path=docker-proxy --validate
    invalid userland-proxy-path: must be an absolute path: docker-proxy

Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

Specifying an invalid userland-proxy-path produces an error as well:

    dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
    userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory

    mkdir -p /usr/local/bin/not-a-file
    dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory

    touch /usr/local/bin/not-an-executable
    dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied

Same when using the daemon.json config-file;

    echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary

    dockerd --userland-proxy-path=hello --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 16:23:18 +01:00
Sebastiaan van Stijn
aa2fa2caa3 un-containerize TestDaemonNoSpaceLeftOnDeviceError
Commit 59b83d8aae containerized these steps,
as they didn't work well on Debian Jessie:

> Because the `mount` here will sometimes fail when run in `debian:jessie`,
> which is what the environrment hosting the test suite is running if run
> from the `Makefile`.
> Also, why the heck not containerize it, all the things.

Follow-up commits, such as 228d74842f, and
1c5806cf57 updated the Debian distro, but
also updated this comment, losing the original context (the issue was
(originally) related to Debian Jessie).

This patch changes the test back to not use containers, which seems to
work fine (at least "it worked on my machine").

    make TEST_IGNORE_CGROUP_CHECK=1 TEST_FILTER=TestDaemonNoSpaceLeftOnDeviceError DOCKER_GRAPHDRIVER=overlay2 test-integration

    === RUN   TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError
        check_test.go:589: [df36ad96a412b] daemon is not started
    --- PASS: TestDockerDaemonSuite (5.12s)
        --- PASS: TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError (5.12s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 18:51:15 +01:00
Sebastiaan van Stijn
3bfb6a9420 frozen images: update to debian:bookworm-slim
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 12:01:06 +01:00
Sebastiaan van Stijn
e72c4818c4 Dockerfile: update to Debian "bookworm" (current stable)
Also switch yamllint to be installed from debian's packages, which are
currently at v1.29.0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:50:34 +01:00
Sebastiaan van Stijn
c3eed9fa3e integration-cli: fix TestDaemonICC tests for newer iptables versions
Debian Woodworm ships with a newer version of iptables, which caused two
tests to fail:

    === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)
    docker_cli_daemon_test.go:841: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge6.*ext-bridge6", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge6 !ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge6 ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n"
    --- FAIL: TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)

    === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCPing (1.19s)
    docker_cli_daemon_test.go:803: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge5.*ext-bridge5", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge5 !ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge5 ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n"
    --- FAIL: TestDockerDaemonSuite/TestDaemonICCPing (1.19s)

Both the `TestDaemonICCPing`, and `TestDaemonICCLinkExpose` test were introduced
in dd0666e64f. These tests called `iptables` with
the `-n` (`--numeric`) option, which prevents it from doing a reverse-DNS lookup
as an optimization.

However, the `-n` option did not have an effect to the `prot` column before
commit [da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa] (iptables < v1.8.9 or v1.8.8).
Newer versions, such as the iptables version shipping with Debian Woodworm do,
so we need to update the expected output for this version.

This patch removes the `-n` option, to keep the test more portable, also when
run non-containerized, and removes the use of regular expressions to check the
result, as these regular expressions were quite permissive (using `.*` wild-
card matching). Instead, we're getting the

With this change;

make DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestDaemonICC TEST_IGNORE_CGROUP_CHECK=1 test-integration
...
--- PASS: TestDockerDaemonSuite (139.11s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCLinkExpose (54.62s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCPing (84.48s)

[da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa]: https://git.netfilter.org/iptables/commit/?id=da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-28 18:58:03 +01:00
Sebastiaan van Stijn
981a878cf5 vendor: github.com/docker/go-connections fa09c952e3ea (v0.5.0-dev)
0b8c1f4e07...fa09c952e3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-10 23:23:37 +01:00
Sebastiaan van Stijn
d1108b8a06 integration-cli: DockerDaemonSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:59:10 +02:00
Djordje Lukic
23f2e0fea7 Skip tests that check max concurrency
These are not yet implemented with containerd snapshotters. We skip them
now because implementing this is not trivial with containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-10 11:36:36 +02:00
Paweł Gronowski
1f6bacc486 TestDaemonNoSpaceLeftOnDeviceError: Adjust to snapshotters
- Pass empty containerd socket which forces the daemon to create a new
  supervised containerd. Otherwise a global containerd daemon will be
  used and the pulled image data will be stored in its data directory,
  instead of the the newly specified `data-root` that has a limited
  storage capacity.

- Don't try to use `vfs` snapshotter, instead use `native` which is
  containerd's equivalent for `vfs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 15:59:30 +02:00
Djordje Lukic
6c563cda82 c8d: fix TestRemoveContainerAfterLiveRestore
There is no need to pass the storage driver to the daemon the test
starts

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-18 15:34:20 +02:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Sebastiaan van Stijn
225e2562c9 integration-cli: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:08:27 +02:00
Cory Snider
0f6eeecac0 daemon: consolidate runtimes config validation
The daemon has made a habit of mutating the DefaultRuntime and Runtimes
values in the Config struct to merge defaults. This would be fine if it
was a part of the regular configuration loading and merging process,
as is done with other config options. The trouble is it does so in
surprising places, such as in functions with 'verify' or 'validate' in
their name. It has been necessary in order to validate that the user has
not defined a custom runtime named "runc" which would shadow the
built-in runtime of the same name. Other daemon code depends on the
runtime named "runc" always being defined in the config, but merging it
with the user config at the same time as the other defaults are merged
would trip the validation. The root of the issue is that the daemon has
used the same config values for both validating the daemon runtime
configuration as supplied by the user and for keeping track of which
runtimes have been set up by the daemon. Now that a completely separate
value is used for the latter purpose, surprising contortions are no
longer required to make the validation work as intended.

Consolidate the validation of the runtimes config and merging of the
built-in runtimes into the daemon.setupRuntimes() function. Set the
result of merging the built-in runtimes config and default default
runtime on the returned runtimes struct, without back-propagating it
onto the config.Config argument.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
Sebastiaan van Stijn
ab35df454d remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Sebastiaan van Stijn
dc11d2a2d8 remove deprecated devicemapper storage-driver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-20 23:51:54 +02:00
Sebastiaan van Stijn
018347f802 integration-cli: remove waitInspectWithArgs()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 18:26:12 +01:00
Sebastiaan van Stijn
61c434c030 integration-cli: remove inspectMountPointJSON()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 18:26:00 +01:00
Sebastiaan van Stijn
1981706196 daemon: remove migrateTrustKeyID()
The migration code is in the 22.06 branch, and if we don't migrate
the only side-effect is the daemon's ID being regenerated (as a
UUID).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 20:28:55 +01:00
Sebastiaan van Stijn
dc0c2340b8 integration-cli: fix empty-lines (revive)
integration-cli/docker_cli_pull_test.go:55:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_exec_test.go:46:64: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_service_health_test.go:86:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_images_test.go:128:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_node_test.go:79:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_health_test.go:51:57: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_health_test.go:159:73: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_swarm_unix_test.go:60:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_inspect_test.go:30:33: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_build_test.go:429:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_attach_unix_test.go:19:78: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_build_test.go:470:70: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_history_test.go:29:64: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_links_test.go:93:86: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:33:61: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_links_test.go:145:78: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:114:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_attach_test.go:226:153: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_by_digest_test.go:239:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_create_test.go:135:49: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:143:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:181:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_inspect_test.go:72:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:98:77: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:144:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_rmi_test.go:63:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:199:79: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_rmi_test.go:69:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:300:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_prune_unix_test.go:35:25: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_events_unix_test.go:393:60: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_events_unix_test.go:441:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_ps_test.go:33:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_ps_test.go:559:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_events_test.go:117:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:547:74: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_api_containers_test.go:1054:84: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:1076:87: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:1232:72: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_api_containers_test.go:1801:21: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:58:95: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:750:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:765:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_swarm_test.go:617:100: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_swarm_test.go:892:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_daemon_test.go:119:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_daemon_test.go:981:68: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_daemon_test.go:1951:87: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_run_test.go:83:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_run_test.go:357:72: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_build_test.go:89:83: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:114:83: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:183:80: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:290:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:314:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:331:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:366:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:403:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:648:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:708:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:938:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1018:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1097:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1182:62: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1244:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1524:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1546:80: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1716:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1730:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2162:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2270:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2288:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3206:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3392:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3433:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3678:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3732:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3759:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3802:61: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3898:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4107:9: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4791:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4821:73: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4854:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:5341:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:5593:81: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:2145:11: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:49 +02:00
Sebastiaan van Stijn
306b8c89e8 linting: host:port in url should be constructed with net.JoinHostPort
integration-cli/docker_cli_daemon_test.go:545:54: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            cmdArgs = append(cmdArgs, "--tls=false", "--host", fmt.Sprintf("tcp://%s:%s", l.daemon, l.port))
                                                               ^
    opts/hosts_test.go:35:31: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            "tcp://:5555":              fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                        ^
    opts/hosts_test.go:91:30: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            ":5555":                   fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                       ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-04 15:36:53 +02:00
Cory Snider
547da0d575 daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-27 14:22:49 -04:00
Sebastiaan van Stijn
968ff5ab44 fix some minor linting issues
libnetwork/firewall_linux.go:11:21: var-declaration: should drop = nil from declaration of var ctrl; it is the zero value (revive)
        ctrl *controller = nil
                           ^
    distribution/pull_v2_test.go:213:4: S1038: should use t.Fatalf(...) instead of t.Fatal(fmt.Sprintf(...)) (gosimple)
                t.Fatal(fmt.Sprintf("expected formatPlatform to show windows platform with a version, but got '%s'", result))
                ^
    integration-cli/docker_cli_build_test.go:5951:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("Bug fixed in 18.06 or higher.Skipping it for %s", testEnv.DaemonInfo.ServerVersion))
            ^
    integration-cli/docker_cli_daemon_test.go:240:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("New base device size (%v) must be greater than (%s)", units.HumanSize(float64(newBasesizeBytes)), units.HumanSize(float64(oldBasesizeBytes))))
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-04 10:15:28 +02:00
Sebastiaan van Stijn
18529568d3 integration-cli: TestRemoveContainerAfterLiveRestore use overlay2
the overlay storage driver is deprecated, so we might as well use overlay2
for this test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-24 17:28:14 +02:00
Sebastiaan van Stijn
070da63310 daemon: only create trust-key if DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE is set
The libtrust trust-key is only used for pushing legacy image manifests;
pushing these images has been deprecated, and we only need to be able
to push them in our CI.

This patch disables generating the trust-key (and related paths) unless
the DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE env-var is set (which we do in
our CI).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-04 20:18:08 +02:00
Eng Zer Jun
36049a04d2 test: use T.Setenv to set env vars in tests
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-23 17:44:16 +08:00
Sebastiaan van Stijn
40182954fa daemon/logger/fluentd: add coverage for ValidateLogOpt(), parseAddress()
This exposed a bug where host is ignored on some valid cases (to be fixed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:55:31 +02:00
Sebastiaan van Stijn
c2ca3e1118 daemon/logger/syslog: remove uses of pkg/urlutil.IsTransportURL()
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

This patch:

- removes a redundant use of urlutil.IsTransportURL(); instead adding some code
  to check if the given scheme (protocol) is supported.
- define a `defaultPort` const for the default port.
- use `net.JoinHostPort()` instead of string concatenating, to account for possible
  issues with IPv6 addresses.
- renames a variable that collided with an imported package.
- improves test coverage, and moves an integration test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:48:40 +02:00
Sebastiaan van Stijn
d2c6602cac integration-cli: use unique names for custom bridges
Use unique names to prevent tests from interfering, using a shorter
name, as there's a maximum length for these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:19 +01:00
Sebastiaan van Stijn
65b92a730a integration(-cli): remove discovery related tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:13 +01:00
Sebastiaan van Stijn
f415028976 integration-cli: delete existing docker0 bridge to prevent it interfering
A number of tests in the TestDockerDaemonSuite create a custom bridge as part
of the test. In some cases, an existing `docker0` bridge could interfere with
those tests. For example, the `TestDaemonICCLinkExpose` and `TestDaemonICCPing`
verify that no "icc" communication is possible, and for this create a new
bridge with a custom IP-range.

However, depending on which tests ran before the test, a default `docker0` bridge
may exist (e.g., if the`TestDefaultGatewayIPv4Implicit`) with the same IP-range,
in which iptables rules may have been set up that allow communication, and thus
make the "icc" tests fail.

This patch removes the `docker0` interface at the start of tests that create
their own bridge to prevent it from interfering.

Note that alternatively, we could update those tests to use an IP-range that's
less likely to overlap, but this may be more brittle (but could still be done
in addition to this change as a follow-up).

To verify these changes;

    make DOCKER_GRAPHDRIVER=vfs TEST_SKIP_INTEGRATION=1 TESTFLAGS='-test.run TestDockerDaemonSuite/TestDaemon(DefaultGatewayIPv4|ICC)' test-integration-cli

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-21 16:12:24 +01:00
Eng Zer Jun
c55a4ac779 refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00