This package was originally internal, but was moved out when BuildKit
used it for its integration tests. That's no longer the case, so we
can make it internal again.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the option-types to the client and in some cases create a
copy for the backend. These types are used to construct query-
args, and not marshaled to JSON, and can be replaced with functional
options in the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The stdcopy package is used to produce and read multiplexed streams for
"attach" and "logs". It is used both by the API server (to produce), and
the client (to read / de-multiplex).
Move it to the api package, so that it can be included in the api module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These comments were added to enforce using the correct import path for
our packages ("github.com/docker/docker", not "github.com/moby/moby").
However, when working in go module mode (not GOPATH / vendor), they have
no effect, so their impact is limited.
Remove these imports in preparation of migrating our code to become an
actual go module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Move logging out of config.Reload and daemon.Reload itself, as it was not
the right place to know whether it was a "signal" that triggered the reload.
- Use Daemon.Config() to get the new config after reloading. This returns an
immutable copy of the daemon's config, so we can redact fields without having
to use an ad-hoc struct to shadow the underlying fields.
- Use structured logs for logging config reload events.
Before this (plain text):
INFO[2025-02-08T12:13:53.389649297Z] Got signal to reload configuration, reloading from: /etc/docker/daemon.json
INFO[2025-02-08T12:30:34.857691260Z] Reloaded configuration: {"pidfile":"/var/run/docker.pid","data-root":"/var/lib/docker","exec-root":"/var/run/docker","group":"docker","max-concurrent-downloads":3,"max-concurrent-uploads":5,"max-download-attempts":5,"shutdown-timeout":15,"hosts":["unix:///var/run/docker.sock"],"log-level":"info","log-format":"text","swarm-default-advertise-addr":"","swarm-raft-heartbeat-tick":0,"swarm-raft-election-tick":0,"metrics-addr":"","host-gateway-ips":[""],"log-driver":"json-file","mtu":1500,"ip":"0.0.0.0","icc":true,"iptables":true,"ip6tables":true,"ip-forward":true,"ip-masq":true,"userland-proxy":true,"userland-proxy-path":"/usr/local/bin/docker-proxy","default-address-pools":{"Values":null},"network-control-plane-mtu":1500,"experimental":false,"containerd":"/var/run/docker/containerd/containerd.sock","features":{"containerd-snapshotter":false},"builder":{"GC":{},"Entitlements":{}},"containerd-namespace":"moby","containerd-plugin-namespace":"plugins.moby","default-runtime":"runc","runtimes":{"crun":{"path":"/usr/local/bin/crun"}},"seccomp-profile":"builtin","default-shm-size":67108864,"default-ipc-mode":"private","default-cgroupns-mode":"private","resolv-conf":"/etc/resolv.conf","proxies":{}}
Before this (JSON logs):
{"level":"info","msg":"Reloaded configuration: {\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"json\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}","time":"2025-02-08T12:24:38.600761054Z"}
After this (plain text):
INFO[2025-02-08T12:30:34.835953594Z] Got signal to reload configuration config-file=/etc/docker/daemon.json
INFO[2025-02-08T12:30:34.857614135Z] Reloaded configuration config="{\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"text\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}"
After this (JSON logs):
{"config-file":"/etc/docker/daemon.json","level":"info","msg":"Got signal to reload configuration","time":"2025-02-08T12:24:38.589955637Z"}
{"config":"{\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"json\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}","level":"info","msg":"Reloaded configuration","time":"2025-02-08T12:24:38.600736179Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Becuase I'm about to add tests that use netlink, and the netlink
package breaks compilation under Windows.
Signed-off-by: Rob Murray <rob.murray@docker.com>
integration/daemon/daemon_test.go:115:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/daemon/daemon_test.go:161:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/daemon/daemon_test.go:234:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, the DOCKER-USER chains are set up on firewall reload or network
creation. If there are running containers at startup, configureNetworking won't
be called (daemon/daemon_unix.go), so the user chains won't be setup.
This commit puts the setup logic on a separate function, and calls it on the
original place and on initNetworkController.
Signed-off-by: Andrés Maldonado <maldonado@codelutin.com>
Tests that start a daemon disable iptables, to avoid conflicts with
other tests running in parallel and also creating iptables chains.
Do the same for ip6tables, in prep for them being enabled by-default.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't use OTEL tracing in this test because we're testing the HTTP proxy
behavior here and we don't want OTEL to interfere.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Replace `time.Sleep` with a poll that checks if process no longer exists
to avoid possible race condition.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When live-restore is enabled, containers with autoremove enabled
shouldn't be forcibly killed when engine restarts.
They still should be removed if they exited while the engine was down
though.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.
Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Make sure that the content in the live-restored volume mounted in a new
container is the same as the content in the old container.
This checks if volume's _data directory doesn't get unmounted on
startup.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Also move the validation function to live with the type definition,
which allows it to be used outside of the daemon as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I noticed this was always being skipped because of race conditions
checking the logs.
This change adds a log scanner which will look through the logs line by
line rather than allocating a big buffer.
Additionally it adds a `poll.Check` which we can use to actually wait
for the desired log entry.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)
However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)
This would panic if the MountPoint was not a volume;
github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
created by github.com/docker/docker/daemon.(*Daemon).restore
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]
This issue was introduced in 647c2a6cdd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Since cc19eba (backported to v23.0.4), the PreferredPool for docker0 is
set only when the user provides the bip config parameter or when the
default bridge already exist. That means, if a user provides the
fixed-cidr parameter on a fresh install or reboot their computer/server
without bip set, dockerd throw the following error when it starts:
> failed to start daemon: Error initializing network controller: Error
> creating default "bridge" network: failed to parse pool request for
> address space "LocalDefault" pool "" subpool "100.64.0.0/26": Invalid
> Address SubPool
See #45356.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
The migration code is in the 22.06 branch, and if we don't migrate
the only side-effect is the daemon's ID being regenerated (as a
UUID).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this change restarting the daemon in live-restore with running
containers + a restart policy meant that volume refs were not restored.
This specifically happens when the container is still running *and*
there is a restart policy that would make sure the container was running
again on restart.
The bug allows volumes to be removed even though containers are
referencing them. 😱
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This change is in preparation of deprecating support for old manifests.
Currently the daemon's ID is based on the trust-key ID, which will be
removed once we fully deprecate support for old manifests (the trust
key is currently only used in tests).
This patch:
- looks if a trust-key is present; if so, it migrates the trust-key
ID to the new "engine-id" file within the daemon's root.
- if no trust-key is present (so in case it's a "fresh" install), we
generate a UUID instead and use that as ID.
The migration is to prevent engines from getting a new ID on upgrades;
while we don't provide any guarantees on the engine's ID, users may
expect the ID to be "stable" (not change) between upgrades.
A test has been added, which can be ran with;
make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestConfigDaemonID' test-integration
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
