This change defines the generic `Storage` type for use in container inspect responses when using containerd snapshotter backend.
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
This package was originally internal, but was moved out when BuildKit
used it for its integration tests. That's no longer the case, so we
can make it internal again.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the option-types to the client and in some cases create a
copy for the backend. These types are used to construct query-
args, and not marshaled to JSON, and can be replaced with functional
options in the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This change makes the `GraphDriver` field in `image.InspectResponse` optional. This field will only be returned when using moby engine graph drivers as a backend storage implementation. It will be omitted when using the containerd image backend.
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
Fix a bug causing containers not being loaded when storage driver wasn't
chosen explicitly.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When no explicit driver was specified, the containerd store by default
was also applied to existing graphdriver setups.
Fix this and add a test.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This change restores the environment variable configuration of daemon storage driver through the DOCKER_DRIVER environment variable.
Signed-off-by: Austin Vazquez <austin.vazquez@docker.com>
Add layer migration on startup
Use image size threshold rather than image count
Add daemon integration test
Add test for migrating to containerd snapshotters
Add vfs migration
Add tar export for containerd migration
Add containerd migration test with save and load
Signed-off-by: Derek McGowan <derek@mcg.dev>
The stdcopy package is used to produce and read multiplexed streams for
"attach" and "logs". It is used both by the API server (to produce), and
the client (to read / de-multiplex).
Move it to the api package, so that it can be included in the api module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These comments were added to enforce using the correct import path for
our packages ("github.com/docker/docker", not "github.com/moby/moby").
However, when working in go module mode (not GOPATH / vendor), they have
no effect, so their impact is limited.
Remove these imports in preparation of migrating our code to become an
actual go module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When an IPv6 network is first created with no specific IPAM config,
network inspect adds a CIDR range to the gateway address. After the
daemon has been restarted, it's just a plain address.
Once the daaemon's been restated, "info" becomes "config", and the
address is reported correctly from "config".
Make the IPv6 code to report the gateway from "info" use net.IPNet.IP
instead of the whole net.IPNet - like the IPv4 code.
Signed-off-by: Rob Murray <rob.murray@docker.com>
- Move logging out of config.Reload and daemon.Reload itself, as it was not
the right place to know whether it was a "signal" that triggered the reload.
- Use Daemon.Config() to get the new config after reloading. This returns an
immutable copy of the daemon's config, so we can redact fields without having
to use an ad-hoc struct to shadow the underlying fields.
- Use structured logs for logging config reload events.
Before this (plain text):
INFO[2025-02-08T12:13:53.389649297Z] Got signal to reload configuration, reloading from: /etc/docker/daemon.json
INFO[2025-02-08T12:30:34.857691260Z] Reloaded configuration: {"pidfile":"/var/run/docker.pid","data-root":"/var/lib/docker","exec-root":"/var/run/docker","group":"docker","max-concurrent-downloads":3,"max-concurrent-uploads":5,"max-download-attempts":5,"shutdown-timeout":15,"hosts":["unix:///var/run/docker.sock"],"log-level":"info","log-format":"text","swarm-default-advertise-addr":"","swarm-raft-heartbeat-tick":0,"swarm-raft-election-tick":0,"metrics-addr":"","host-gateway-ips":[""],"log-driver":"json-file","mtu":1500,"ip":"0.0.0.0","icc":true,"iptables":true,"ip6tables":true,"ip-forward":true,"ip-masq":true,"userland-proxy":true,"userland-proxy-path":"/usr/local/bin/docker-proxy","default-address-pools":{"Values":null},"network-control-plane-mtu":1500,"experimental":false,"containerd":"/var/run/docker/containerd/containerd.sock","features":{"containerd-snapshotter":false},"builder":{"GC":{},"Entitlements":{}},"containerd-namespace":"moby","containerd-plugin-namespace":"plugins.moby","default-runtime":"runc","runtimes":{"crun":{"path":"/usr/local/bin/crun"}},"seccomp-profile":"builtin","default-shm-size":67108864,"default-ipc-mode":"private","default-cgroupns-mode":"private","resolv-conf":"/etc/resolv.conf","proxies":{}}
Before this (JSON logs):
{"level":"info","msg":"Reloaded configuration: {\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"json\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}","time":"2025-02-08T12:24:38.600761054Z"}
After this (plain text):
INFO[2025-02-08T12:30:34.835953594Z] Got signal to reload configuration config-file=/etc/docker/daemon.json
INFO[2025-02-08T12:30:34.857614135Z] Reloaded configuration config="{\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"text\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}"
After this (JSON logs):
{"config-file":"/etc/docker/daemon.json","level":"info","msg":"Got signal to reload configuration","time":"2025-02-08T12:24:38.589955637Z"}
{"config":"{\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"docker\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"max-download-attempts\":5,\"shutdown-timeout\":15,\"hosts\":[\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"log-format\":\"json\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"host-gateway-ips\":[\"\"],\"log-driver\":\"json-file\",\"mtu\":1500,\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip6tables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"userland-proxy-path\":\"/usr/local/bin/docker-proxy\",\"default-address-pools\":{\"Values\":null},\"network-control-plane-mtu\":1500,\"experimental\":false,\"containerd\":\"/var/run/docker/containerd/containerd.sock\",\"features\":{\"containerd-snapshotter\":false},\"builder\":{\"GC\":{},\"Entitlements\":{}},\"containerd-namespace\":\"moby\",\"containerd-plugin-namespace\":\"plugins.moby\",\"default-runtime\":\"runc\",\"runtimes\":{\"crun\":{\"path\":\"/usr/local/bin/crun\"}},\"seccomp-profile\":\"builtin\",\"default-shm-size\":67108864,\"default-ipc-mode\":\"private\",\"default-cgroupns-mode\":\"private\",\"resolv-conf\":\"/etc/resolv.conf\",\"proxies\":{}}","level":"info","msg":"Reloaded configuration","time":"2025-02-08T12:24:38.600736179Z"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit facb2323 aligned the way the default bridge's IPv6 subnet
and gateway addresses are selected with IPv4.
Part of that involved looking at addresses already on the bridge,
along with daemon config options. But, for IPv6, the kernel will
assign a link-local address to the bridge.
Make sure that address is ignored when selecting "bip6" when it's
not explicitly specified.
This is made slightly complicated because we allow fixed-cidr-v6
to be a link-local subnet (either the standard "fe80::/64", or
any other non-overlapping LL subnet in "fe80::/10").
Following this change, if fixed-cidr-v6 is (or is included by)
"fe80::/64", the bridge's kernel-assigned LL address may be used
as the network's gateway address - even though it may also get an
IPAM-assigned LL address.
Signed-off-by: Rob Murray <rob.murray@docker.com>
These tests create iptables rules for different addresses on
docker0 but, unlike tests that do that for user-defined bridges,
those rules aren't removed when the test deletes the network,
because the default bridge network can't be deleted.
So, use (abuse) the L3Segment code to run the tests in their
own network namespace.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Use the same logic to generate IPAMConf for IPv6 as for IPv4.
- When no fixed-cidr-v6 is specified, rather than error out, use
the default address pools (as for an IPv4 default bridge with no
fixed-cidr, and as for user-defined networks).
- Add daemon option --bip6, similar to --bip.
- Necessary because it's the only way to override an old address
on docker0 (daemon-managed default bridge), as illustrated by
test cases.
- For a user-managed default bridge (--bridge), use IPv6 addresses
on the user's bridge to determine the pool, sub-pool and gateway.
Following the same rules as IPv4.
- Don't set up IPv6 IPAMConf if IPv6 is not enabled.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a user-managed bridge is used for the default network (--bridge),
an address from the bridge determines the subnet for the network.
If a fixed-cidr is supplied, it should fall within that subnet. If it
doesn't, it's a misconfiguration - fixed-cidr is the range of
allocatable addresses, and they need to be in the network. (Either
the user's bridge is missing an address that matches their fixed-cidr
or the fixed-cidr is wrong.)
When this happens, because it's been allowed in the past (and, because
the address-pool implementation treats fixed-cidr/SubPool as an offset
into the network rather than an actual address range, so working IP
addresses would normally still be assigned to containers) ... don't
reject the config and cause daemon startup to fail. Just log a warning
and ignore fixed-cidr.
Signed-off-by: Rob Murray <rob.murray@docker.com>
For a docker-managed default bridge (docker0), when no --bip is
supplied, the gateway address and subnet size can be inferred
from existing bridge addresses.
But, if fixed-cidr's subnet size is increased so that it's biggger
than the subnet of the bridge's existing address - the bridge's
subnet needs to be incresed to match. (fixed-cidr determines the
range of addresses that can be automatically allocated, and these
should not fall outside the default bridge's subnet.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a docker-managed default bridge (docker0) already has an
address, and the fixed-cidr subnet fits within the subnet defined
by that address, the existing address should be used as the
gateway and to define the subnet.
But, when fixed-cidr is changed, no --bip is supplied, and no
existing bridge network includes fixed-cidr ... the existing
bridge address needs to be updated.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When a user-managed bridge is used instead of docker0 (--bridge), with
a fixed-cidr - the bridge should have an IP address/subnet that
encompasses fixed-cidr ... the bridge address's subnet then defines
the network's subnet, and fixed-cidr defines the allocatable range
within that.
But, selection of the correct subnet/address from the bridge depended
on the address being within fixed-cidr (within the allocatable range).
This change removes that assumption. So, a bridge address with a
subnet that includes fixed-cidr is selected.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Becuase I'm about to add tests that use netlink, and the netlink
package breaks compilation under Windows.
Signed-off-by: Rob Murray <rob.murray@docker.com>
integration/daemon/daemon_test.go:115:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/daemon/daemon_test.go:161:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
integration/daemon/daemon_test.go:234:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, the DOCKER-USER chains are set up on firewall reload or network
creation. If there are running containers at startup, configureNetworking won't
be called (daemon/daemon_unix.go), so the user chains won't be setup.
This commit puts the setup logic on a separate function, and calls it on the
original place and on initNetworkController.
Signed-off-by: Andrés Maldonado <maldonado@codelutin.com>
Tests that start a daemon disable iptables, to avoid conflicts with
other tests running in parallel and also creating iptables chains.
Do the same for ip6tables, in prep for them being enabled by-default.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
