Go maintainers started to unconditionally update the minimum go version
for golang.org/x/ dependencies to go1.23, which means that we'll no longer
be able to support any version below that when updating those dependencies;
> all: upgrade go directive to at least 1.23.0 [generated]
>
> By now Go 1.24.0 has been released, and Go 1.22 is no longer supported
> per the Go Release Policy (https://go.dev/doc/devel/release#policy).
>
> For golang/go#69095.
This updates our minimum version to go1.23, as we won't be able to maintain
compatibility with older versions because of the above.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since commit 925b484 ("No fallback nameservers for internal
resolver"), if the host's resolv.conf has no nameservers and
no servers are supplied via config, the internal resolver will
not use Google's DNS - so the container will not be able to
resolve external DNS requests.
That can happen when container's are "restart-always" and the
docker daemon starts before the host's DNS is configured.
So, to highlight the issue (which may not be an error, but
probably is), include a warning in the container's resolv.conf
file.
Also, log a warning - logs currently say "No non-localhost DNS
nameservers are left in resolv.conf. Using default external
servers". But, that's misleading because it's from an initial
resolv.conf setup, before the internal resolver configured without
those fallbacks - we'll drop the fallbacks completely once the
default bridge has an internal resolver).
Signed-off-by: Rob Murray <rob.murray@docker.com>
Changing to use binary.LittleEndian.AppendUint64, which does not require
the slice to have an initial size, and makes the code slightly more
straightforward.
libnetwork/internal/kvstore/boltdb/boltdb.go:79:11: append to slice `dbval` with non-zero initialized length (makezero)
dbval = append(dbval, value...)
^
libnetwork/internal/kvstore/boltdb/boltdb.go:228:11: append to slice `dbval` with non-zero initialized length (makezero)
dbval = append(dbval, value...)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit a0807e7cfe configured golangci-lint
to use go1.23 semantics, which alowed linters like `copyloopvar` to lint
using thee correct semantics.
go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure an error is returned straight away if there's contention on
the underlying db file. This makes sure we don't reintroduce the issue
fixed in d21d088, and it will help detect contention in parallelized
tests if they're badly written. It effectively adds a new error mode to
the daemon, but if anyone faces this error, they should fix their
process manager.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
That function was needlessly complex. Instead of relying on a struct and
a sub-struct, it now just takes two string params: a path and a bucket
name.
Libnetwork config is now initialized with default values. A new struct
is introduced in libnetwork/config to let tests customize the path and
bucket name.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
the resolvconf package is imported in BuildKit, and this is the only
location that used the errdefs package outside of the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The internal resolver now uses any namesever found in the host's
/etc/resolv.conf as an external nameserver, and it's accessed
from the host's network namespace.
Before this change, when no external nameservers were found (so
the host had no entries in /etc/resolv.conf) Google's DNS servers
were used as fallbacks, always accessed from the container's
network namespace. If a container's initial set of endpoints had
IPv6 enabled, the IPv6 nameservers were included.
Now we have IPv6-only networks, a similar exception would be
needed for Google's IPv4 nameservers... don't include them if
there are no IPv4 endpoints.
However, only the initial set of endpoints was considered. As
networks are connected/disconnected, IPv4 or IPv6 connectivity
may be lost.
Unlike nameservers read from the host's /etc/resolv.conf, there
is no way to tell which fallback nameservers (v4/v6) might work
from the host's namespace. So, using the host's namespace isn't
a good solution.
Since we want to get away from using fallback nameservers anyway,
this change removes them.
If a host has no /etc/resolv.conf entries, but a container does
need to use DNS, it'll need to be configured with servers via
'--dns'.
Signed-off-by: Rob Murray <rob.murray@docker.com>
The internal resolver needs to know whether to make requets
to external DNS servers from the container's network namespace
or the host's.
The original rule was that requests were always made from the
container's namespace, unless the nameserver was on a localhost
address on the host. IPv6 nameservers were left in the container's
/etc/resolv.conf.
Commit 4e8d9a4 modified that so that IPv6 nameservers were also
used as external nameservers. The internal resolver accessed
them from the host namespace if the container's initial set of
endpoints were IPv4-only, or the nameserver address contained
a zone-id, (or the nameserver was on the IPv6 loopback address).
That would break if initial IPv6 endpoints were disconnected from
the container, leaving it with no IPv6 address.
Once IPv6-only networks are allowed, another exception would need
to be made for IPv4 nameservers (they'd need to be accessed from
the host's namespace).
Instead of doing that ... this change simplifies things, if a
nameserver address is read from the host's /etc/resolv.conf, it'll
work in the host's namespace. So, the rule is now simply that
nameservers read from the host's resolv.conf are accessed from the
host's namespace. DNS servers added as overrides ('--dns') are
accessed from the container's namespace (as before).
Signed-off-by: Rob Murray <rob.murray@docker.com>
This reverts commit d365702dbd.
Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).
Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.
By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Don't fall-back to Google's DNS servers in a network that has an
internal resolver.
Now the default bridge uses the internal resolver, the only reason a
network started by the daemon should end up without any upstream
servers is if the host's resolv.conf doesn't list any. In this case,
the '--dns' option can be used to explicitly configure nameservers
for a container if necessary.
(Note that buildkit's containers do not have an internal resolver, so
they will still set up Google's nameservers if the host has no
resolvers that can be used in the container's namespace.)
Signed-off-by: Rob Murray <rob.murray@docker.com>
The previous allocator was subnetting address pools eagerly
when the daemon started, and would then just iterate over that
list whenever RequestPool was called. This was leading to high
memory usage whenever IPv6 pools were configured with a target
subnet size too different from the pools prefix size.
For instance: pool = fd00::/8, target size = /64 -- 2 ^ (64-8)
subnets would be generated upfront. This would take approx.
9 * 10^18 bits -- way too much for any human computer in 2024.
Another noteworthy issue, the previous implementation was allocating
a subnet, and then in another layer was checking whether the
allocation was conflicting with some 'reserved networks'. If so,
the allocation would be retried, etc... To make it worse, 'reserved
networks' would be recomputed on every iteration. This is totally
ineffective as there could be 'reserved networks' that fully overlap
a given address pool (or many!).
To fix this issue, a new field `Exclude` is added to `RequestPool`.
It's up to each driver to take it into account. Since we don't know
whether this retry loop is useful for some remote IPAM driver, it's
reimplemented bug-for-bug directly in the remote driver.
The new allocator uses a linear-search algorithm. It takes advantage
of all lists (predefined pools, allocated subnets and reserved
networks) being sorted and logically combines 'allocated' and
'reserved' through a 'double cursor' to iterate on both lists at the
same time while preserving the total order. At the same time, it
iterates over 'predefined' pools and looks for the first empty space
that would be a good fit.
Currently, the size of the allocated subnet is still dictated by
each 'predefined' pools. We should consider hardcoding that size
instead, and let users specify what subnet size they want. This
wasn't possible before as the subnets were generated upfront. This
new allocator should be able to deal with this easily.
The method used for static allocation has been updated to make sure
the ascending order of 'allocated' is preserved. It's bug-for-bug
compatible with the previous implementation.
One consequence of this new algorithm is that we don't keep track
of where the last allocation happened, we just allocate the first
free subnet we find.
Before:
- Allocate: 10.0.1.0/24, 10.0.2.0/24 ; Deallocate: 10.0.1.0/24 ;
Allocate 10.0.3.0/24.
Now, the 3rd allocation would yield 10.0.1.0/24 once again.
As it doesn't change the semantics of the allocator, there's no
reason to worry about that.
Finally, about 'reserved networks'. The heuristics we use are
now properly documented. It was discovered that we don't check
routes for IPv6 allocations -- this can't be changed because
there's no such thing as on-link routes for IPv6.
(Kudos to Rob Murray for coming up with the linear-search idea.)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Commit 8921897e3b introduced the uses of `clear()`,
which requires go1.21, but Go is downgrading this file to go1.16 when used in
other projects (due to us not yet being a go module);
0.175 + xx-go build '-gcflags=' -ldflags '-X github.com/moby/buildkit/version.Version=b53a13e -X github.com/moby/buildkit/version.Revision=b53a13e4f5c8d7e82716615e0f23656893df89af -X github.com/moby/buildkit/version.Package=github.com/moby/buildkit -extldflags '"'"'-static'"'" -tags 'osusergo netgo static_build seccomp ' -o /usr/bin/buildkitd ./cmd/buildkitd
181.8 # github.com/docker/docker/libnetwork/internal/resolvconf
181.8 vendor/github.com/docker/docker/libnetwork/internal/resolvconf/resolvconf.go:509:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Rather than error out if the host's resolv.conf has a bad ndots option,
just ignore it. Still validate ndots supplied via '--dns-option' and
treat failure as an error.
Signed-off-by: Rob Murray <rob.murray@docker.com>
When configuring the internal DNS resolver - rather than keep IPv6
nameservers read from the host's resolv.conf in the container's
resolv.conf, treat them like IPv4 addresses and use them as upstream
resolvers.
For IPv6 nameservers, if there's a zone identifier in the address or
the container itself doesn't have IPv6 support, mark the upstream
addresses for use in the host's network namespace.
Signed-off-by: Rob Murray <rob.murray@docker.com>
A common pattern in libnetwork is to delete an object using
`DeleteAtomic`, ie. to check the optimistic lock, but put in a retry
loop to refresh the data and the version index used by the optimistic
lock.
This commit introduces a new `Delete` method to delete without
checking the optimistic lock. It focuses only on the few places where
it's obvious the calling code doesn't rely on the side-effects of the
retry loop (ie. refreshing the object to be deleted).
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Replace regex matching/replacement and re-reading of generated files
with a simple parser, and struct to remember and manipulate the file
content.
Annotate the generated file with a header comment saying the file is
generated, but can be modified, and a trailing comment describing how
the file was generated and listing external nameservers.
Always start with the host's resolv.conf file, whether generating config
for host networking, or with/without an internal resolver - rather than
editing a file previously generated for a different use-case.
Resolves an issue where rewrites of the generated file resulted in
default IPv6 nameservers being unnecessarily added to the config.
Signed-off-by: Rob Murray <rob.murray@docker.com>
Previous commit made getDBhandle a one-liner returning a struct
member -- making it useless. Inline it.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
This parameter was used to tell the boltdb kvstore not to open/close
the underlying boltdb db file before/after each get/put operation.
Since d21d0884ae, we've a single datastore instance shared by all
components that need it. That commit set `PersistConnection=true`.
We can now safely remove this param altogether, and remove all the
code that was opening and closing the db file before and after each
operation -- it's dead code!
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
File paths can contain commas, particularly paths returned from
t.TempDir() in subtests which include commas in their names. There is
only one datastore provider and it only supports a single address, so
the only use of parsing the address is to break tests in mysterious
ways.
Signed-off-by: Cory Snider <csnider@mirantis.com>
