Compare commits

...

50 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
94d3ad69cc Merge pull request #45410 from rumpl/23.0_backport-fix-buildx-tag-events
[23.0 backport] Use the image service instead of the reference store for tagging
2023-04-26 17:29:05 +02:00
Sebastiaan van Stijn
933a9f83b7 Merge pull request #45407 from thaJeztah/23.0_backport_vendor_runc_1.1.5
[23.0 backport] vendor: github.com/opencontainers/runc v1.1.5
2023-04-26 16:43:32 +02:00
Djordje Lukic
fe0a414613 Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 16:32:25 +02:00
Sebastiaan van Stijn
1b1230eaaf Merge pull request #45375 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83
2023-04-26 15:22:10 +02:00
Sebastiaan van Stijn
1b263035af Merge pull request #45374 from thaJeztah/23.0_backport_stfu_grpc
[23.0 backport] Silence GRPC logs unless our log level is debug
2023-04-26 15:21:47 +02:00
Sebastiaan van Stijn
682542fd08 Merge pull request #45373 from thaJeztah/23.0_backport_assorted_test_and_packaging
[23.0 backport] assorted test- and build/packaging fixes
2023-04-26 15:21:31 +02:00
Sebastiaan van Stijn
40ec0eb9fe vendor: github.com/opencontainers/runc v1.1.5
no changes in vendored code, just keeping scanners happy :)

release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.5
diff: https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5

This is the fifth patch release in the 1.1.z series of runc, which fixes
three CVEs found in runc.

* CVE-2023-25809 is a vulnerability involving rootless containers where
  (under specific configurations), the container would have write access
  to the /sys/fs/cgroup/user.slice/... cgroup hierarchy. No other
  hierarchies on the host were affected. This vulnerability was
  discovered by Akihiro Suda.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-m8cg-xc2p-r3fc>

* CVE-2023-27561 was a regression which effectively re-introduced
  CVE-2019-19921. This bug was present from v1.0.0-rc95 to v1.1.4. This
  regression was discovered by @Beuc.
  <https://github.com/advisories/GHSA-vpvm-3wq2-2wvm>

* CVE-2023-28642 is a variant of CVE-2023-27561 and was fixed by the same
  patch. This variant of the above vulnerability was reported by Lei
  Wang.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-g2j6-57v7-gm8c>

In addition, the following other fixes are included in this release:

* Fix the inability to use `/dev/null` when inside a container.
* Fix changing the ownership of host's `/dev/null` caused by fd redirection
  (a regression in 1.1.1).
* Fix rare runc exec/enter unshare error on older kernels, including
  CentOS < 7.7.
* nsexec: Check for errors in `write_log()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a17029ba49)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-26 14:50:34 +02:00
Sebastiaan van Stijn
9af6762d34 Merge pull request #45403 from akerouanton/23.0-cherrypick-2d31697
[23.0 backport] daemon: set docker0 subpool as the IPAM pool
2023-04-26 14:34:41 +02:00
Sebastiaan van Stijn
0b6449a477 Merge pull request #45376 from thaJeztah/23.0_backport_delete_network_more_atomically
[23.0 backport] libnetwork: clean up inDelete network atomically
2023-04-26 13:11:30 +02:00
Albin Kerouanton
227d3f39a3 daemon: set docker0 subpool as the IPAM pool
Since cc19eba (backported to v23.0.4), the PreferredPool for docker0 is
set only when the user provides the bip config parameter or when the
default bridge already exist. That means, if a user provides the
fixed-cidr parameter on a fresh install or reboot their computer/server
without bip set, dockerd throw the following error when it starts:

> failed to start daemon: Error initializing network controller: Error
> creating default "bridge" network: failed to parse pool request for
> address space "LocalDefault" pool "" subpool "100.64.0.0/26": Invalid
> Address SubPool

See #45356.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 2d31697)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-04-26 11:38:55 +02:00
Sebastiaan van Stijn
6f83b8c3f4 Merge pull request #45372 from thaJeztah/23.0_backport_really_deprecate_newclient
[23.0 backport] client: deprecate NewClient properly
2023-04-25 18:31:47 +02:00
Cory Snider
c82be35412 libnetwork: clean up inDelete network atomically
The (*network).ipamRelease function nils out the network's IPAM info
fields, putting the network struct into an inconsistent state. The
network-restore startup code panics if it tries to restore a network
from a struct which has fewer IPAM config entries than IPAM info
entries. Therefore (*network).delete contains a critical section: by
persisting the network to the store after ipamRelease(), the datastore
will contain an inconsistent network until the deletion operation
completes and finishes deleting the network from the datastore. If for
any reason the deletion operation is interrupted between ipamRelease()
and deleteFromStore(), the daemon will crash on startup when it tries to
restore the network.

Updating the datastore after releasing the network's IPAM pools may have
served a purpose in the past, when a global datastore was used for
intra-cluster communication and the IPAM allocator had persistent global
state, but nowadays there is no global datastore and the IPAM allocator
has no persistent state whatsoever. Remove the vestigial datastore
update as it is no longer necessary and only serves to cause problems.
If the network deletion is interrupted before the network is deleted
from the datastore, the deletion will resume during the next daemon
startup, including releasing the IPAM pools.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit c957ad0067)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:47:01 +02:00
Sebastiaan van Stijn
ccccb7a170 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83
- CSI: Manager PublishVolume verify CSI node ID is not empty

full diff: a745a8755c...e28e8ba9bc

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 72dc7a0f7b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:42:24 +02:00
Brian Goff
8a6bec6e16 Silence GRPC logs unless our log level is debug
GRPC is logging a *lot* of garbage at info level.
This configures the GRPC logger such that it is only giving us logs when
at debug level and also adds a log field indicating where the logs are
coming from.

containerd is still currently spewing these same log messages and needs
a separate update.

Without this change `docker build` is extremely noisy in the daemon
logs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit c7ccc68b15)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:33:33 +02:00
Aneesh Kulkarni
844fd50c67 Added timeout to validate job in github actions
Signed-off-by: Aneesh Kulkarni <askthefactorcamera@gmail.com>
(cherry picked from commit e5dbd28afd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:29:21 +02:00
CrazyMax
8cec683574 Dockerfile: update xx to 1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 106c694993)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:28:53 +02:00
Albin Kerouanton
7ff53c3c78 Update delve version
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 4f48a4ea2a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:28:27 +02:00
Paweł Gronowski
4d57ee692d TestLogs: Increase stop check poll timeout on Windows
Stopping container on Windows can sometimes take longer than 10s which
caused this test to be flaky.
Increase the timeout to 75s when running this test on Windows.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 74dbb721aa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:26:59 +02:00
xin.li
838d5ea058 chore: use http constants to replace numbers
Signed-off-by: xin.li <xin.li@daocloud.io>
(cherry picked from commit a770153af0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:13:31 +02:00
Cory Snider
61d084593c client: deprecate NewClient properly
The 'Deprecated:' line in NewClient's doc comment was not in a new
paragraph, so GoDoc, linters, and IDEs were unaware that it was
deprecated. The package documentation also continued to reference
NewClient. Update the doc comments to finish documenting that NewClient
is deprecated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 6b9968e8b1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:05:15 +02:00
Sebastiaan van Stijn
8fdca288c5 Merge pull request #45350 from PettitWesley/awslogs-non-blocking-bug-23.0
[23.0 backport] awslogs: fix non-blocking log drop bug
2023-04-19 12:59:45 +02:00
Wesley Pettit
28b694d32d awslogs: fix non-blocking log drop bug
Previously, the AWSLogs driver attempted to implement
non-blocking itself. Non-blocking is supposed to
implemented solely by the Docker RingBuffer that
wraps the log driver.

Please see issue and explanation here:
https://github.com/moby/moby/issues/45217

Signed-off-by: Wesley Pettit <wppttt@amazon.com>
(cherry picked from commit c8f8d11ac4)
2023-04-17 11:59:53 -07:00
Sebastiaan van Stijn
cbce331930 Merge pull request #45330 from kevingentile/buildkit-3770
[23.0] vendor: github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242
2023-04-14 02:59:18 +02:00
Bjorn Neergaard
5f684cb072 Merge pull request #45331 from thaJeztah/23.0_backport_rootless_script_bugs
[23.0 backport] Fix argument quoting bugs in dockerd-rootless.sh
2023-04-13 18:48:55 -06:00
kpcyrd
3731ce10d4 Fix argument quoting bugs in dockerd-rootless.sh
Signed-off-by: kpcyrd <git@rxv.cc>
(cherry picked from commit 7fe0f73838)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-14 01:26:56 +02:00
Kevin Gentile
23774ada04 vendor: github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242
Signed-off-by: Kevin Gentile <kevin.gentile@rockwellautomation.com>
2023-04-13 23:29:30 +02:00
Sebastiaan van Stijn
90e8a0bbf5 Merge pull request #45323 from thaJeztah/23.0_backport_vendor_sctp
[23.0 backport] vendor: github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2
2023-04-13 21:59:27 +02:00
Sebastiaan van Stijn
9277e64444 vendor: github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2
- Prevent from descriptor leak
- Fixes optlen in getsockopt() for s390x

full diff: 9a39160e90...7ff4192f6f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 893d28469f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 15:02:16 +02:00
Sebastiaan van Stijn
cdb6200887 Merge pull request #45293 from AkihiroSuda/backport-45283-23
[23.0 backport] docker-rootless-setuptools.sh: improve readability of messages
2023-04-12 20:30:21 +02:00
Akihiro Suda
09fbbd5677 docker-rootless-setuptools.sh: improve readability of messages
Before:
```console
$ docker-rootless-setuptool.sh install
...
[INFO] Use CLI context "rootless"
Current context is now "rootless"

[INFO] Make sure the following environment variables are set (or add them to ~/.bashrc):

export PATH=/usr/local/bin:$PATH
Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1001/docker.sock
```

After:
```console
$ docker-rootless-setuptool.sh install
...
[INFO] Using CLI context "rootless"
Current context is now "rootless"

[INFO] Make sure the following environment variable(s) are set (or add them to ~/.bashrc):
export PATH=/usr/local/bin:$PATH

[INFO] Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1001/docker.sock
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 4aa2876c75)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-04-07 10:08:16 +09:00
Sebastiaan van Stijn
4ca4705bf7 Merge pull request #45277 from thaJeztah/23.0_bump_go1.19.8
[23.0] update go to go1.19.8
2023-04-06 01:48:49 +02:00
Sebastiaan van Stijn
d3e52936c3 [23.0] update go to go1.19.8
go1.19.8 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well as
bug fixes to the linker, the runtime, and the time package. See the Go 1.19.8
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.7...go1.19.8

Further details from the announcement on the mailing list:

We have just released Go versions 1.20.3 and 1.19.8, minor point releases.
These minor releases include 4 security fixes following the security policy:

- go/parser: infinite loop in parsing

  Calling any of the Parse functions on Go source code which contains `//line`
  directives with very large line numbers can cause an infinite loop due to
  integer overflow.
  Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
  This is CVE-2023-24537 and Go issue https://go.dev/issue/59180.

- html/template: backticks not treated as string delimiters

  Templates did not properly consider backticks (`) as Javascript string
  delimiters, and as such did not escape them as expected. Backticks are
  used, since ES6, for JS template literals. If a template contained a Go
  template action within a Javascript template literal, the contents of the
  action could be used to terminate the literal, injecting arbitrary Javascript
  code into the Go template.

  As ES6 template literals are rather complex, and themselves can do string
  interpolation, we've decided to simply disallow Go template actions from being
  used inside of them (e.g. "var a = {{.}}"), since there is no obviously safe
  way to allow this behavior. This takes the same approach as
  github.com/google/safehtml. Template.Parse will now return an Error when it
  encounters templates like this, with a currently unexported ErrorCode with a
  value of 12. This ErrorCode will be exported in the next major release.

  Users who rely on this behavior can re-enable it using the GODEBUG flag
  jstmpllitinterp=1, with the caveat that backticks will now be escaped. This
  should be used with caution.

  Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue.

  This is CVE-2023-24538 and Go issue https://go.dev/issue/59234.

- net/http, net/textproto: denial of service from excessive memory allocation

  HTTP and MIME header parsing could allocate large amounts of memory, even when
  parsing small inputs.

  Certain unusual patterns of input data could cause the common function used to
  parse HTTP and MIME headers to allocate substantially more memory than
  required to hold the parsed headers. An attacker can exploit this behavior to
  cause an HTTP server to allocate large amounts of memory from a small request,
  potentially leading to memory exhaustion and a denial of service.
  Header parsing now correctly allocates only the memory required to hold parsed
  headers.

  Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

  This is CVE-2023-24534 and Go issue https://go.dev/issue/58975.

- net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption

  Multipart form parsing can consume large amounts of CPU and memory when
  processing form inputs containing very large numbers of parts. This stems from
  several causes:

  mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form
  can consume. ReadForm could undercount the amount of memory consumed, leading
  it to accept larger inputs than intended. Limiting total memory does not
  account for increased pressure on the garbage collector from large numbers of
  small allocations in forms with many parts. ReadForm could allocate a large
  number of short-lived buffers, further increasing pressure on the garbage
  collector. The combination of these factors can permit an attacker to cause an
  program that parses multipart forms to consume large amounts of CPU and
  memory, potentially resulting in a denial of service. This affects programs
  that use mime/multipart.Reader.ReadForm, as well as form parsing in the
  net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm now does a better job of estimating the memory consumption of parsed
  forms, and performs many fewer short-lived allocations.

  In addition, mime/multipart.Reader now imposes the following limits on the
  size of parsed forms:

  Forms parsed with ReadForm may contain no more than 1000 parts. This limit may
  be adjusted with the environment variable GODEBUG=multipartmaxparts=. Form
  parts parsed with NextPart and NextRawPart may contain no more than 10,000
  header fields. In addition, forms parsed with ReadForm may contain no more
  than 10,000 header fields across all parts. This limit may be adjusted with
  the environment variable GODEBUG=multipartmaxheaders=.

  Thanks to Jakob Ackermann for discovering this issue.

  This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 22:02:01 +02:00
Sebastiaan van Stijn
347bce43e9 Merge pull request #45279 from thaJeztah/23.0_fix_vendor
[23.0] fix vendor.mod: add hashicorp/go-multierror as direct dependency
2023-04-05 18:33:33 +02:00
Sebastiaan van Stijn
a0a597cbbe [23.0] fix vendor.mod: add hashicorp/go-multierror as direct dependency
commit 59118bff50 made this a direct
dependency (previously it was indirect). That commit was part of an
advisory, and didn't run the vendor validation check because of that.

This patch fixes the vendor.mod to unblock CI in this branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 17:31:29 +02:00
Sebastiaan van Stijn
c9f6343c24 Merge pull request #45220 from vvoland/test-windows-execstartfails-23
[23.0 backport] integration-cli: Enable TestExecStartFails on Windows
2023-04-05 02:44:27 +02:00
Sebastiaan van Stijn
0ce1e22243 Merge pull request #45237 from thaJeztah/23.0_backport_update_runc_binary_1.1.5
[23.0 backport] update runc binary to v1.1.5
2023-04-05 02:44:15 +02:00
Sebastiaan van Stijn
3648a4915c Merge pull request #45240 from akerouanton/cherrypick-45211
[23.0 backport] libnet/d/windows: log EnableInternalDNS val after setting it
2023-04-05 02:44:02 +02:00
Sebastiaan van Stijn
aaa102b30f Merge pull request #45245 from thaJeztah/23.0_containerd_binary_1.6.20
[23.0] update containerd binary to v1.6.20
2023-04-05 02:43:49 +02:00
Sebastiaan van Stijn
15d6037c1e Merge pull request #45246 from akerouanton/cherrypick-44827
[23.0 backport] daemon: let libnetwork assign default bridge IPAM
2023-04-05 02:42:50 +02:00
Sebastiaan van Stijn
59118bff50 Merge pull request from GHSA-232p-vwff-86mp
[23.0 backport] libnetwork: ensure encryption is mandatory on encrypted overlay networks
2023-04-04 20:03:51 +02:00
Cory Snider
063d3a6f1a daemon: let libnetwork assign default bridge IPAM
The netutils.ElectInterfaceAddresses function is only used in one place
outside of tests: in the daemon, to configure the default bridge
network. The function is also messy to reason about as it references the
shared mutable state of ipamutils.PredefinedLocalScopeDefaultNetworks.
It uses the list of predefined default networks to always return an IPv4
address even if the named interface does not exist or does not have any
IPv4 addresses. This list happens to be the same as the one used to
initialize the address pool of the 'builtin' IPAM driver, though that is
far from obvious. (Start with "./libnetwork".initIPAMDrivers and trace
the dataflow of the addressPool value. Surprise! Global state is being
mutated using the value of other global mutable state.)

The daemon does not need the fallback behaviour of
ElectInterfaceAddresses. In fact, the daemon does not have to configure
an address pool for the network at all! libnetwork will acquire one of
the available address ranges from the network's IPAM driver when the
preferred-pool configuration is unset. It will do so using the same list
of address ranges and the exact same logic
(netutils.FindAvailableNetworks) as ElectInterfaceAddresses. So unless
the daemon needs to force the network to use a specific address range
because the bridge interface already exists, it can leave the details
up to libnetwork.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit cc19eba)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-03-31 09:38:21 +02:00
Sebastiaan van Stijn
389e18081d update containerd binary to v1.6.20
Notable Updates

- Disable looking up usernames and groupnames on host
- Add support for Windows ArgsEscaped images
- Update hcsshim to v0.9.8
- Fix debug flag in shim
- Add WithReadonlyTempMount to support readonly temporary mounts
- Update ttrpc to fix file descriptor leak
- Update runc binary to v1.1.5
= Update image config to support ArgsEscaped

full diff: https://github.com/containerd/containerd/compare/v1.6.19...v1.6.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-31 00:48:19 +02:00
Albin Kerouanton
348544e6e8 libnet/d/windows: log EnableInternalDNS val after setting it
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit bae49ff)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-03-30 14:47:55 +02:00
Sebastiaan van Stijn
3923e302a4 update runc binary to v1.1.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.5
diff: https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5

This is the fifth patch release in the 1.1.z series of runc, which fixes
three CVEs found in runc.

* CVE-2023-25809 is a vulnerability involving rootless containers where
  (under specific configurations), the container would have write access
  to the /sys/fs/cgroup/user.slice/... cgroup hierarchy. No other
  hierarchies on the host were affected. This vulnerability was
  discovered by Akihiro Suda.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-m8cg-xc2p-r3fc>

* CVE-2023-27561 was a regression which effectively re-introduced
  CVE-2019-19921. This bug was present from v1.0.0-rc95 to v1.1.4. This
  regression was discovered by @Beuc.
  <https://github.com/advisories/GHSA-vpvm-3wq2-2wvm>

* CVE-2023-28642 is a variant of CVE-2023-27561 and was fixed by the same
  patch. This variant of the above vulnerability was reported by Lei
  Wang.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-g2j6-57v7-gm8c>

In addition, the following other fixes are included in this release:

* Fix the inability to use `/dev/null` when inside a container.
* Fix changing the ownership of host's `/dev/null` caused by fd redirection
  (a regression in 1.1.1).
* Fix rare runc exec/enter unshare error on older kernels, including
  CentOS < 7.7.
* nsexec: Check for errors in `write_log()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 77be7b777c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-30 14:45:16 +02:00
Paweł Gronowski
f3946c14fe integration-cli: Enable TestExecStartFails on Windows
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit be34e93f20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-28 11:10:24 +02:00
Cory Snider
b87f7f18b8 libnet/d/overlay: insert the input-drop rule
FirewallD creates the root INPUT chain with a default-accept policy and
a terminal rule which rejects all packets not accepted by any prior
rule. Any subsequent rules appended to the chain are therefore inert.
The administrator would have to open the VXLAN UDP port to make overlay
networks work at all, which would result in all VXLAN traffic being
accepted and defeating our attempts to enforce encryption on encrypted
overlay networks.

Insert the rule to drop unencrypted VXLAN packets tagged for encrypted
overlay networks at the top of the INPUT chain so that enforcement of
mandatory encryption takes precedence over any accept rules configured
by the administrator. Continue to append the accept rule to the bottom
of the chain so as not to override any administrator-configured drop
rules.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 965eda3b9a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-22 20:54:32 -04:00
Cory Snider
98cbcb8003 libnet/d/overlay: add BPF-powered VNI matcher
Some newer distros such as RHEL 9 have stopped making the xt_u32 kernel
module available with the kernels they ship. They do ship the xt_bpf
kernel module, which can do everything xt_u32 can and more. Add an
alternative implementation of the iptables match rule which uses xt_bpf
to implement exactly the same logic as the u32 filter using a BPF
program. Try programming the BPF-powered rules as a fallback when
programming the u32-powered rules fails.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 105b9834fb)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:22:16 -04:00
Cory Snider
5c5fac2374 libnet/d/overlay: extract VNI match rule builder
The iptables rule clause used to match on the VNI of VXLAN datagrams
looks like line noise to the uninitiated. It doesn't help that the
expression is repeated twice and neither copy has any commentary.
DRY out the rule builder to a common function, and document what the
rule does and how it works.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 44cf27b5fc)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:22:16 -04:00
Cory Snider
c492a22287 libn/d/overlay: enforce encryption on sandbox init
The iptables rules which make encryption mandatory on an encrypted
overlay network are only programmed once there is a second node
participating in the network. This leaves single-node encrypted overlay
networks vulnerable to packet injection. Furthermore, failure to program
the rules is not treated as a fatal error.

Program the iptables rules to make encryption mandatory before creating
the VXLAN link to guarantee that there is no window of time where
incoming cleartext VXLAN packets for the network would be accepted, or
outgoing cleartext packets be transmitted. Only create the VXLAN link if
programming the rules succeeds to ensure that it fails closed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 142f46cac1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:22:13 -04:00
Cory Snider
018edb0284 libnet/d/overlay: document some encryption code
The overlay-network encryption code is woefully under-documented, which
is especially problematic as it operates on under-documented kernel
interfaces. Document what I have puzzled out of the implementation for
the benefit of the next poor soul to touch this code.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d4fd582fb2)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:21:05 -04:00
49 changed files with 590 additions and 188 deletions

View File

@@ -15,7 +15,7 @@ on:
default: false
env:
GO_VERSION: 1.19.7
GO_VERSION: 1.19.8
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
WINDOWS_BASE_IMAGE: mcr.microsoft.com/windows/servercore

View File

@@ -15,7 +15,7 @@ on:
pull_request:
env:
GO_VERSION: 1.19.7
GO_VERSION: 1.19.8
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
ITG_CLI_MATRIX_SIZE: 6
@@ -82,6 +82,7 @@ jobs:
validate:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- validate-prepare
- build-dev

View File

@@ -1,9 +1,9 @@
# syntax=docker/dockerfile:1
ARG GO_VERSION=1.19.7
ARG GO_VERSION=1.19.8
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
ARG XX_VERSION=1.1.2
ARG XX_VERSION=1.2.1
ARG VPNKIT_VERSION=0.5.0
ARG DOCKERCLI_VERSION=v17.06.2-ce
@@ -135,7 +135,7 @@ RUN git init . && git remote add origin "https://github.com/go-delve/delve.git"
# from the https://github.com/go-delve/delve repository.
# It can be used to run Docker with a possibility of
# attaching debugger to it.
ARG DELVE_VERSION=v1.9.1
ARG DELVE_VERSION=v1.20.1
RUN git fetch -q --depth 1 origin "${DELVE_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS delve-build
@@ -192,7 +192,7 @@ RUN git init . && git remote add origin "https://github.com/containerd/container
# When updating the binary version you may also need to update the vendor
# version to pick up bug fixes or new APIs, however, usually the Go packages
# are built from a commit from the master branch.
ARG CONTAINERD_VERSION=v1.6.19
ARG CONTAINERD_VERSION=v1.6.20
RUN git fetch -q --depth 1 origin "${CONTAINERD_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerd-build
@@ -280,7 +280,7 @@ RUN git init . && git remote add origin "https://github.com/opencontainers/runc.
# that is used. If you need to update runc, open a pull request in the containerd
# project first, and update both after that is merged. When updating RUNC_VERSION,
# consider updating runc in vendor.mod accordingly.
ARG RUNC_VERSION=v1.1.4
ARG RUNC_VERSION=v1.1.5
RUN git fetch -q --depth 1 origin "${RUNC_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS runc-build

View File

@@ -1,4 +1,4 @@
ARG GO_VERSION=1.19.7
ARG GO_VERSION=1.19.8
FROM golang:${GO_VERSION}-alpine AS base
ENV GO111MODULE=off

View File

@@ -5,7 +5,7 @@
# This represents the bare minimum required to build and test Docker.
ARG GO_VERSION=1.19.7
ARG GO_VERSION=1.19.8
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"

View File

@@ -165,10 +165,10 @@ FROM microsoft/windowsservercore
# Use PowerShell as the default shell
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG GO_VERSION=1.19.7
ARG GO_VERSION=1.19.8
ARG GOTESTSUM_VERSION=v1.8.2
ARG GOWINRES_VERSION=v0.3.0
ARG CONTAINERD_VERSION=v1.6.19
ARG CONTAINERD_VERSION=v1.6.20
# Environment variable notes:
# - GO_VERSION must be consistent with 'Dockerfile' used by Linux.

View File

@@ -92,7 +92,7 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
stdout := config.ProgressWriter.StdoutFormatter
fmt.Fprintf(stdout, "Successfully built %s\n", stringid.TruncateID(imageID))
}
if imageID != "" {
if imageID != "" && !useBuildKit {
err = tagger.TagImages(image.ID(imageID))
}
return imageID, err

View File

@@ -33,7 +33,7 @@ func TestJsonContentType(t *testing.T) {
func TestReadJSON(t *testing.T) {
t.Run("nil body", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", nil)
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", nil)
if err != nil {
t.Error(err)
}
@@ -45,7 +45,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("empty body", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(""))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(""))
if err != nil {
t.Error(err)
}
@@ -60,7 +60,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("with valid request", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"}`))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"}`))
if err != nil {
t.Error(err)
}
@@ -75,7 +75,7 @@ func TestReadJSON(t *testing.T) {
}
})
t.Run("with whitespace", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`
{"SomeField":"some value"}
@@ -95,7 +95,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("with extra content", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"} and more content`))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"} and more content`))
if err != nil {
t.Error(err)
}
@@ -112,7 +112,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("invalid JSON", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`{invalid json`))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{invalid json`))
if err != nil {
t.Error(err)
}

View File

@@ -15,6 +15,7 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/builder"
containerimageexp "github.com/docker/docker/builder/builder-next/exporter"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/daemon/images"
"github.com/docker/docker/libnetwork"
@@ -70,6 +71,7 @@ type Opt struct {
Root string
Dist images.DistributionServices
NetworkController libnetwork.NetworkController
ImageTagger containerimageexp.ImageTagger
DefaultCgroupParent string
RegistryHosts docker.RegistryHosts
BuilderConfig config.BuilderConfig

View File

@@ -159,9 +159,9 @@ func newController(rt http.RoundTripper, opt Opt) (*control.Controller, error) {
}
exp, err := containerimageexp.New(containerimageexp.Opt{
ImageStore: dist.ImageStore,
ReferenceStore: dist.ReferenceStore,
Differ: differ,
ImageStore: dist.ImageStore,
Differ: differ,
ImageTagger: opt.ImageTagger,
})
if err != nil {
return nil, err

View File

@@ -7,10 +7,10 @@ import (
"strconv"
"strings"
"github.com/docker/distribution/reference"
distref "github.com/docker/distribution/reference"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/reference"
"github.com/moby/buildkit/exporter"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
"github.com/moby/buildkit/util/compression"
@@ -29,11 +29,15 @@ type Differ interface {
EnsureLayer(ctx context.Context, key string) ([]layer.DiffID, error)
}
type ImageTagger interface {
TagImageWithReference(imageID image.ID, newTag reference.Named) error
}
// Opt defines a struct for creating new exporter
type Opt struct {
ImageStore image.Store
ReferenceStore reference.Store
Differ Differ
ImageStore image.Store
Differ Differ
ImageTagger ImageTagger
}
type imageExporter struct {
@@ -206,10 +210,10 @@ func (e *imageExporterInstance) Export(ctx context.Context, inp exporter.Source,
}
_ = configDone(nil)
if e.opt.ReferenceStore != nil {
if e.opt.ImageTagger != nil {
for _, targetName := range e.targetNames {
tagDone := oneOffProgress(ctx, "naming to "+targetName.String())
if err := e.opt.ReferenceStore.AddTag(targetName, digest.Digest(id), true); err != nil {
if err := e.opt.ImageTagger.TagImageWithReference(image.ID(digest.Digest(id)), targetName); err != nil {
return nil, tagDone(err)
}
_ = tagDone(nil)

View File

@@ -6,9 +6,10 @@ https://docs.docker.com/engine/api/
# Usage
You use the library by creating a client object and calling methods on it. The
client can be created either from environment variables with NewClientWithOpts(client.FromEnv),
or configured manually with NewClient().
You use the library by constructing a client object using [NewClientWithOpts]
and calling methods on it. The client can be configured from environment
variables by passing the [FromEnv] option, or configured manually by passing any
of the other available [Opts].
For example, to list running containers (the equivalent of "docker ps"):

View File

@@ -9,7 +9,11 @@ import "net/http"
// It won't send any version information if the version number is empty. It is
// highly recommended that you set a version or your client may break if the
// server is upgraded.
// Deprecated: use NewClientWithOpts
//
// Deprecated: use [NewClientWithOpts] passing the [WithHost], [WithVersion],
// [WithHTTPClient] and [WithHTTPHeaders] options. We recommend enabling API
// version negotiation by passing the [WithAPIVersionNegotiation] option instead
// of WithVersion.
func NewClient(host string, version string, client *http.Client, httpHeaders map[string]string) (*Client, error) {
return NewClientWithOpts(WithHost(host), WithVersion(version), WithHTTPClient(client), WithHTTPHeaders(httpHeaders))
}
@@ -17,7 +21,7 @@ func NewClient(host string, version string, client *http.Client, httpHeaders map
// NewEnvClient initializes a new API client based on environment variables.
// See FromEnv for a list of support environment variables.
//
// Deprecated: use NewClientWithOpts(FromEnv)
// Deprecated: use [NewClientWithOpts] passing the [FromEnv] option.
func NewEnvClient() (*Client, error) {
return NewClientWithOpts(FromEnv)
}

View File

@@ -295,6 +295,7 @@ func newRouterOptions(config *config.Config, d *daemon.Daemon) (routerOptions, e
SessionManager: sm,
Root: filepath.Join(config.Root, "buildkit"),
Dist: d.DistributionServices(),
ImageTagger: d.ImageService(),
NetworkController: d.NetworkController(),
DefaultCgroupParent: cgroupParent,
RegistryHosts: d.RegistryHosts(),

View File

@@ -88,6 +88,7 @@ func main() {
_, stdout, stderr := term.StdStreams()
initLogging(stdout, stderr)
configureGRPCLog()
onError := func(err error) {
fmt.Fprintf(stderr, "%s\n", err)

17
cmd/dockerd/grpclog.go Normal file
View File

@@ -0,0 +1,17 @@
package main
import (
"github.com/sirupsen/logrus"
"google.golang.org/grpc/grpclog"
)
// grpc's default logger is *very* noisy and uses "info" and even "warn" level logging for mostly useless messages.
// This function configures the grpc logger to step down the severity of all messages.
//
// info => trace
// warn => debug
// error => warn
func configureGRPCLog() {
l := logrus.WithField("library", "grpc")
grpclog.SetLoggerV2(grpclog.NewLoggerV2(l.WriterLevel(logrus.TraceLevel), l.WriterLevel(logrus.DebugLevel), l.WriterLevel(logrus.WarnLevel)))
}

View File

@@ -390,18 +390,18 @@ cmd_entrypoint_install() {
cli_ctx_create "${CLI_CONTEXT}" "unix://${XDG_RUNTIME_DIR}/docker.sock" "Rootless mode"
fi
INFO "Use CLI context \"${CLI_CONTEXT}\""
INFO "Using CLI context \"${CLI_CONTEXT}\""
cli_ctx_use "${CLI_CONTEXT}"
echo
INFO "Make sure the following environment variables are set (or add them to ~/.bashrc):"
echo
INFO "Make sure the following environment variable(s) are set (or add them to ~/.bashrc):"
if [ -n "$XDG_RUNTIME_DIR_CREATED" ]; then
echo "# WARNING: systemd not found. You have to remove XDG_RUNTIME_DIR manually on every logout."
echo "export XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR}"
fi
echo "export PATH=${BIN}:\$PATH"
echo "Some applications may require the following environment variable too:"
echo
INFO "Some applications may require the following environment variable too:"
echo "export DOCKER_HOST=unix://${XDG_RUNTIME_DIR}/docker.sock"
echo
@@ -433,7 +433,7 @@ cmd_entrypoint_uninstall() {
unset DOCKER_HOST
unset DOCKER_CONTEXT
cli_ctx_use "default"
INFO 'Configured CLI use the "default" context.'
INFO 'Configured CLI to use the "default" context.'
INFO
INFO 'Make sure to unset or update the environment PATH, DOCKER_HOST, and DOCKER_CONTEXT environment variables if you have added them to `~/.bashrc`.'
INFO "This uninstallation tool does NOT remove Docker binaries and data."

View File

@@ -107,7 +107,7 @@ if [ -z "$_DOCKERD_ROOTLESS_CHILD" ]; then
--copy-up=/etc --copy-up=/run \
--propagation=rslave \
$DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS \
$0 $@
"$0" "$@"
else
[ "$_DOCKERD_ROOTLESS_CHILD" = 1 ]
# remove the symlinks for the existing files in the parent namespace if any,
@@ -130,6 +130,5 @@ else
mount --rbind ${realpath_etc_ssl} /etc/ssl
fi
# shellcheck disable=SC2086
exec $dockerd "$@"
exec "$dockerd" "$@"
fi

View File

@@ -4,16 +4,19 @@ import (
"bufio"
"fmt"
"io"
"net"
"os"
"regexp"
"strings"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/libnetwork/ns"
"github.com/docker/docker/libnetwork/resolvconf"
"github.com/moby/sys/mount"
"github.com/moby/sys/mountinfo"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/vishvananda/netlink"
)
// On Linux, plugins use a static path for storing execution state,
@@ -141,3 +144,41 @@ func setupResolvConf(config *config.Config) {
}
config.ResolvConf = resolvconf.Path()
}
// ifaceAddrs returns the IPv4 and IPv6 addresses assigned to the network
// interface with name linkName.
//
// No error is returned if the named interface does not exist.
func ifaceAddrs(linkName string) (v4, v6 []*net.IPNet, err error) {
nl := ns.NlHandle()
link, err := nl.LinkByName(linkName)
if err != nil {
if !errors.As(err, new(netlink.LinkNotFoundError)) {
return nil, nil, err
}
return nil, nil, nil
}
get := func(family int) ([]*net.IPNet, error) {
addrs, err := nl.AddrList(link, family)
if err != nil {
return nil, err
}
ipnets := make([]*net.IPNet, len(addrs))
for i := range addrs {
ipnets[i] = addrs[i].IPNet
}
return ipnets, nil
}
v4, err = get(netlink.FAMILY_V4)
if err != nil {
return nil, nil, err
}
v6, err = get(netlink.FAMILY_V6)
if err != nil {
return nil, nil, err
}
return v4, v6, nil
}

View File

@@ -4,6 +4,7 @@
package daemon // import "github.com/docker/docker/daemon"
import (
"net"
"os"
"path/filepath"
"strings"
@@ -11,8 +12,12 @@ import (
containertypes "github.com/docker/docker/api/types/container"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/libnetwork/testutils"
"github.com/docker/docker/libnetwork/types"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/moby/sys/mount"
"github.com/moby/sys/mountinfo"
"github.com/vishvananda/netlink"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
@@ -343,3 +348,66 @@ func TestRootMountCleanup(t *testing.T) {
assert.Assert(t, d.cleanupMounts())
})
}
func TestIfaceAddrs(t *testing.T) {
CIDR := func(cidr string) *net.IPNet {
t.Helper()
nw, err := types.ParseCIDR(cidr)
assert.NilError(t, err)
return nw
}
for _, tt := range []struct {
name string
nws []*net.IPNet
}{
{
name: "Single",
nws: []*net.IPNet{CIDR("172.101.202.254/16")},
},
{
name: "Multiple",
nws: []*net.IPNet{
CIDR("172.101.202.254/16"),
CIDR("172.102.202.254/16"),
},
},
} {
t.Run(tt.name, func(t *testing.T) {
defer testutils.SetupTestOSContext(t)()
createBridge(t, "test", tt.nws...)
ipv4Nw, ipv6Nw, err := ifaceAddrs("test")
if err != nil {
t.Fatal(err)
}
assert.Check(t, is.DeepEqual(tt.nws, ipv4Nw,
cmpopts.SortSlices(func(a, b *net.IPNet) bool { return a.String() < b.String() })))
// IPv6 link-local address
assert.Check(t, is.Len(ipv6Nw, 1))
})
}
}
func createBridge(t *testing.T, name string, bips ...*net.IPNet) {
t.Helper()
link := &netlink.Bridge{
LinkAttrs: netlink.LinkAttrs{
Name: name,
},
}
if err := netlink.LinkAdd(link); err != nil {
t.Fatalf("Failed to create interface via netlink: %v", err)
}
for _, bip := range bips {
if err := netlink.AddrAdd(link, &netlink.Addr{IPNet: bip}); err != nil {
t.Fatal(err)
}
}
if err := netlink.LinkSetUp(link); err != nil {
t.Fatal(err)
}
}

View File

@@ -34,7 +34,6 @@ import (
nwconfig "github.com/docker/docker/libnetwork/config"
"github.com/docker/docker/libnetwork/drivers/bridge"
"github.com/docker/docker/libnetwork/netlabel"
"github.com/docker/docker/libnetwork/netutils"
"github.com/docker/docker/libnetwork/options"
lntypes "github.com/docker/docker/libnetwork/types"
"github.com/docker/docker/opts"
@@ -950,30 +949,37 @@ func initBridgeDriver(controller libnetwork.NetworkController, config *config.Co
ipamV4Conf := &libnetwork.IpamConf{AuxAddresses: make(map[string]string)}
nwList, nw6List, err := netutils.ElectInterfaceAddresses(bridgeName)
// By default, libnetwork will request an arbitrary available address
// pool for the network from the configured IPAM allocator.
// Configure it to use the IPv4 network ranges of the existing bridge
// interface if one exists with IPv4 addresses assigned to it.
nwList, nw6List, err := ifaceAddrs(bridgeName)
if err != nil {
return errors.Wrap(err, "list bridge addresses failed")
}
nw := nwList[0]
if len(nwList) > 1 && config.BridgeConfig.FixedCIDR != "" {
_, fCIDR, err := net.ParseCIDR(config.BridgeConfig.FixedCIDR)
if err != nil {
return errors.Wrap(err, "parse CIDR failed")
}
// Iterate through in case there are multiple addresses for the bridge
for _, entry := range nwList {
if fCIDR.Contains(entry.IP) {
nw = entry
break
if len(nwList) > 0 {
nw := nwList[0]
if len(nwList) > 1 && config.BridgeConfig.FixedCIDR != "" {
_, fCIDR, err := net.ParseCIDR(config.BridgeConfig.FixedCIDR)
if err != nil {
return errors.Wrap(err, "parse CIDR failed")
}
// Iterate through in case there are multiple addresses for the bridge
for _, entry := range nwList {
if fCIDR.Contains(entry.IP) {
nw = entry
break
}
}
}
}
ipamV4Conf.PreferredPool = lntypes.GetIPNetCanonical(nw).String()
hip, _ := lntypes.GetHostPartIP(nw.IP, nw.Mask)
if hip.IsGlobalUnicast() {
ipamV4Conf.Gateway = nw.IP.String()
ipamV4Conf.PreferredPool = lntypes.GetIPNetCanonical(nw).String()
hip, _ := lntypes.GetHostPartIP(nw.IP, nw.Mask)
if hip.IsGlobalUnicast() {
ipamV4Conf.Gateway = nw.IP.String()
}
}
if config.BridgeConfig.IP != "" {
@@ -994,6 +1000,9 @@ func initBridgeDriver(controller libnetwork.NetworkController, config *config.Co
}
ipamV4Conf.SubPool = fCIDR.String()
if ipamV4Conf.PreferredPool == "" {
ipamV4Conf.PreferredPool = fCIDR.String()
}
}
if config.BridgeConfig.DefaultGatewayIPv4 != nil {

View File

@@ -78,7 +78,6 @@ type logStream struct {
logGroupName string
logCreateGroup bool
logCreateStream bool
logNonBlocking bool
forceFlushInterval time.Duration
multilinePattern *regexp.Regexp
client api
@@ -93,7 +92,6 @@ type logStreamConfig struct {
logGroupName string
logCreateGroup bool
logCreateStream bool
logNonBlocking bool
forceFlushInterval time.Duration
maxBufferedEvents int
multilinePattern *regexp.Regexp
@@ -155,12 +153,13 @@ func New(info logger.Info) (logger.Logger, error) {
return nil, err
}
logNonBlocking := info.Config["mode"] == "non-blocking"
containerStream := &logStream{
logStreamName: containerStreamConfig.logStreamName,
logGroupName: containerStreamConfig.logGroupName,
logCreateGroup: containerStreamConfig.logCreateGroup,
logCreateStream: containerStreamConfig.logCreateStream,
logNonBlocking: containerStreamConfig.logNonBlocking,
forceFlushInterval: containerStreamConfig.forceFlushInterval,
multilinePattern: containerStreamConfig.multilinePattern,
client: client,
@@ -168,7 +167,7 @@ func New(info logger.Info) (logger.Logger, error) {
}
creationDone := make(chan bool)
if containerStream.logNonBlocking {
if logNonBlocking {
go func() {
backoff := 1
maxBackoff := 32
@@ -224,8 +223,6 @@ func newStreamConfig(info logger.Info) (*logStreamConfig, error) {
}
}
logNonBlocking := info.Config["mode"] == "non-blocking"
forceFlushInterval := defaultForceFlushInterval
if info.Config[forceFlushIntervalKey] != "" {
forceFlushIntervalAsInt, err := strconv.Atoi(info.Config[forceFlushIntervalKey])
@@ -264,7 +261,6 @@ func newStreamConfig(info logger.Info) (*logStreamConfig, error) {
logGroupName: logGroupName,
logCreateGroup: logCreateGroup,
logCreateStream: logCreateStream,
logNonBlocking: logNonBlocking,
forceFlushInterval: forceFlushInterval,
maxBufferedEvents: maxBufferedEvents,
multilinePattern: multilinePattern,
@@ -439,14 +435,6 @@ func (l *logStream) Log(msg *logger.Message) error {
if l.closed {
return errors.New("awslogs is closed")
}
if l.logNonBlocking {
select {
case l.messages <- msg:
return nil
default:
return errors.New("awslogs buffer is full")
}
}
l.messages <- msg
return nil
}

View File

@@ -390,40 +390,16 @@ func TestLogBlocking(t *testing.T) {
}
}
func TestLogNonBlockingBufferEmpty(t *testing.T) {
func TestLogBufferEmpty(t *testing.T) {
mockClient := &mockClient{}
stream := &logStream{
client: mockClient,
messages: make(chan *logger.Message, 1),
logNonBlocking: true,
client: mockClient,
messages: make(chan *logger.Message, 1),
}
err := stream.Log(&logger.Message{})
assert.NilError(t, err)
}
func TestLogNonBlockingBufferFull(t *testing.T) {
mockClient := &mockClient{}
stream := &logStream{
client: mockClient,
messages: make(chan *logger.Message, 1),
logNonBlocking: true,
}
stream.messages <- &logger.Message{}
errorCh := make(chan error, 1)
started := make(chan bool)
go func() {
started <- true
err := stream.Log(&logger.Message{})
errorCh <- err
}()
<-started
select {
case err := <-errorCh:
assert.Check(t, err != nil)
case <-time.After(30 * time.Second):
t.Fatal("Expected Log call to not block")
}
}
func TestPublishBatchSuccess(t *testing.T) {
mockClient := &mockClient{}
stream := &logStream{

View File

@@ -15,7 +15,7 @@ set -e
# the binary version you may also need to update the vendor version to pick up
# bug fixes or new APIs, however, usually the Go packages are built from a
# commit from the master branch.
: "${CONTAINERD_VERSION:=v1.6.19}"
: "${CONTAINERD_VERSION:=v1.6.20}"
install_containerd() (
echo "Install containerd version $CONTAINERD_VERSION"

View File

@@ -9,7 +9,7 @@ set -e
# the containerd project first, and update both after that is merged.
#
# When updating RUNC_VERSION, consider updating runc in vendor.mod accordingly
: "${RUNC_VERSION:=v1.1.4}"
: "${RUNC_VERSION:=v1.1.5}"
install_runc() {
RUNC_BUILDTAGS="${RUNC_BUILDTAGS:-"seccomp"}"

View File

@@ -509,16 +509,18 @@ func (s *DockerCLIExecSuite) TestExecUlimits(c *testing.T) {
// #15750
func (s *DockerCLIExecSuite) TestExecStartFails(c *testing.T) {
// TODO Windows CI. This test should be portable. Figure out why it fails
// currently.
testRequires(c, DaemonIsLinux)
name := "exec-15750"
runSleepingContainer(c, "-d", "--name", name)
assert.NilError(c, waitRun(name))
out, _, err := dockerCmdWithError("exec", name, "no-such-cmd")
assert.ErrorContains(c, err, "", out)
assert.Assert(c, strings.Contains(out, "executable file not found"))
expectedMsg := "executable file not found"
if DaemonIsWindows() {
expectedMsg = "The system cannot find the file specified"
}
assert.Assert(c, is.Contains(out, expectedMsg))
}
// Fix regression in https://github.com/docker/docker/pull/26461#issuecomment-250287297

View File

@@ -124,6 +124,12 @@ func testLogs(t *testing.T, logDriver string) {
},
}
pollTimeout := time.Second * 10
if testEnv.OSType == "windows" {
// hcs can take longer than 10s to stop a container.
pollTimeout = time.Second * 75
}
for _, tC := range testCases {
tC := tC
t.Run(tC.desc, func(t *testing.T) {
@@ -136,7 +142,9 @@ func testLogs(t *testing.T, logDriver string) {
)
defer client.ContainerRemove(ctx, id, types.ContainerRemoveOptions{Force: true})
poll.WaitOn(t, container.IsStopped(ctx, client, id), poll.WithDelay(time.Millisecond*100))
poll.WaitOn(t, container.IsStopped(ctx, client, id),
poll.WithDelay(time.Millisecond*100),
poll.WithTimeout(pollTimeout))
logs, err := client.ContainerLogs(ctx, id, tC.logOps)
assert.NilError(t, err)

View File

@@ -21,6 +21,7 @@ import (
"github.com/docker/docker/testutil/daemon"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
"gotest.tools/v3/icmd"
"gotest.tools/v3/skip"
)
@@ -416,3 +417,30 @@ func testLiveRestoreVolumeReferences(t *testing.T) {
runTest(t, "no")
})
}
func TestDaemonDefaultBridgeWithFixedCidrButNoBip(t *testing.T) {
skip.If(t, runtime.GOOS == "windows")
bridgeName := "ext-bridge1"
d := daemon.New(t, daemon.WithEnvVars("DOCKER_TEST_CREATE_DEFAULT_BRIDGE="+bridgeName))
defer func() {
d.Stop(t)
d.Cleanup(t)
}()
defer func() {
// No need to clean up when running this test in rootless mode, as the
// interface is deleted when the daemon is stopped and the netns
// reclaimed by the kernel.
if !testEnv.IsRootless() {
deleteInterface(t, bridgeName)
}
}()
d.StartWithBusybox(t, "--bridge", bridgeName, "--fixed-cidr", "192.168.130.0/24")
}
func deleteInterface(t *testing.T, ifName string) {
icmd.RunCommand("ip", "link", "delete", ifName).Assert(t, icmd.Success)
icmd.RunCommand("iptables", "-t", "nat", "--flush").Assert(t, icmd.Success)
icmd.RunCommand("iptables", "--flush").Assert(t, icmd.Success)
}

View File

@@ -16,8 +16,14 @@ import (
// SetupDevice create a new bridge interface/
func setupDevice(config *networkConfiguration, i *bridgeInterface) error {
// We only attempt to create the bridge when the requested device name is
// the default one.
if config.BridgeName != DefaultBridgeName && config.DefaultBridge {
// the default one. The default bridge name can be overridden with the
// DOCKER_TEST_CREATE_DEFAULT_BRIDGE env var. It should be used only for
// test purpose.
var defaultBridgeName string
if defaultBridgeName = os.Getenv("DOCKER_TEST_CREATE_DEFAULT_BRIDGE"); defaultBridgeName == "" {
defaultBridgeName = DefaultBridgeName
}
if config.BridgeName != defaultBridgeName && config.DefaultBridge {
return NonDefaultBridgeExistError(config.BridgeName)
}

View File

@@ -0,0 +1,47 @@
package overlay
import (
"fmt"
"strings"
"golang.org/x/net/bpf"
)
// vniMatchBPF returns a BPF program suitable for passing to the iptables bpf
// match which matches on the VXAN Network ID of encapsulated packets. The
// program assumes that it will be used in a rule which only matches UDP
// datagrams.
func vniMatchBPF(vni uint32) []bpf.RawInstruction {
asm, err := bpf.Assemble([]bpf.Instruction{
bpf.LoadMemShift{Off: 0}, // ldx 4*([0] & 0xf) ; Load length of IPv4 header into X
bpf.LoadIndirect{Off: 12, Size: 4}, // ld [x + 12] ; Load VXLAN ID (UDP header + 4 bytes) into A
bpf.ALUOpConstant{Op: bpf.ALUOpAnd, Val: 0xffffff00}, // and #0xffffff00 ; VXLAN ID is in top 24 bits
bpf.JumpIf{Cond: bpf.JumpEqual, Val: vni << 8, SkipTrue: 1}, // jeq ($vni << 8), match
bpf.RetConstant{Val: 0}, // ret #0
bpf.RetConstant{Val: ^uint32(0)}, // match: ret #-1
})
// bpf.Assemble() only errors if an instruction is invalid. As the only variable
// part of the program is an instruction value for which the entire range is
// valid, whether the program can be successfully assembled is independent of
// the input. Given that the only recourse is to fix this function and
// recompile, there's little value in bubbling the error up to the caller.
if err != nil {
panic(err)
}
return asm
}
// marshalXTBPF marshals a BPF program into the "decimal" byte code format
// which is suitable for passing to the [iptables bpf match].
//
// iptables -m bpf --bytecode
//
// [iptables bpf match]: https://ipset.netfilter.org/iptables-extensions.man.html#lbAH
func marshalXTBPF(prog []bpf.RawInstruction) string { //nolint:unused
var b strings.Builder
fmt.Fprintf(&b, "%d", len(prog))
for _, ins := range prog {
fmt.Fprintf(&b, ",%d %d %d %d", ins.Op, ins.Jt, ins.Jf, ins.K)
}
return b.String()
}

View File

@@ -0,0 +1,14 @@
package overlay
import (
"testing"
)
func FuzzVNIMatchBPFDoesNotPanic(f *testing.F) {
for _, seed := range []uint32{0, 1, 42, 0xfffffe, 0xffffff, 0xfffffffe, 0xffffffff} {
f.Add(seed)
}
f.Fuzz(func(t *testing.T, vni uint32) {
_ = vniMatchBPF(vni)
})
}

View File

@@ -18,12 +18,41 @@ import (
"github.com/docker/docker/libnetwork/iptables"
"github.com/docker/docker/libnetwork/ns"
"github.com/docker/docker/libnetwork/types"
"github.com/hashicorp/go-multierror"
"github.com/sirupsen/logrus"
"github.com/vishvananda/netlink"
)
/*
Encrypted overlay networks use IPsec in transport mode to encrypt and
authenticate the VXLAN UDP datagrams. This driver implements a bespoke control
plane which negotiates the security parameters for each peer-to-peer tunnel.
IPsec Terminology
- ESP: IPSec Encapsulating Security Payload
- SPI: Security Parameter Index
- ICV: Integrity Check Value
- SA: Security Association https://en.wikipedia.org/wiki/IPsec#Security_association
Developer documentation for Linux IPsec is rather sparse online. The following
slide deck provides a decent overview.
https://libreswan.org/wiki/images/e/e0/Netdev-0x12-ipsec-flow.pdf
The Linux IPsec stack is part of XFRM, the netlink packet transformation
interface.
https://man7.org/linux/man-pages/man8/ip-xfrm.8.html
*/
const (
r = 0xD0C4E3
// Value used to mark outgoing packets which should have our IPsec
// processing applied. It is also used as a label to identify XFRM
// states (Security Associations) and policies (Security Policies)
// programmed by us so we know which ones we can clean up without
// disrupting other VPN connections on the system.
mark = 0xD0C4E3
pktExpansion = 26 // SPI(4) + SeqN(4) + IV(8) + PadLength(1) + NextHeader(1) + ICV(8)
)
@@ -33,7 +62,9 @@ const (
bidir
)
var spMark = netlink.XfrmMark{Value: uint32(r), Mask: 0xffffffff}
// Mark value for matching packets which should have our IPsec security policy
// applied.
var spMark = netlink.XfrmMark{Value: mark, Mask: 0xffffffff}
type key struct {
value []byte
@@ -47,6 +78,9 @@ func (k *key) String() string {
return ""
}
// Security Parameter Indices for the IPsec flows between local node and a
// remote peer, which identify the Security Associations (XFRM states) to be
// applied when encrypting and decrypting packets.
type spi struct {
forward int
reverse int
@@ -79,8 +113,8 @@ func (e *encrMap) String() string {
return b.String()
}
func (d *driver) checkEncryption(nid string, rIP net.IP, vxlanID uint32, isLocal, add bool) error {
logrus.Debugf("checkEncryption(%.7s, %v, %d, %t)", nid, rIP, vxlanID, isLocal)
func (d *driver) checkEncryption(nid string, rIP net.IP, isLocal, add bool) error {
logrus.Debugf("checkEncryption(%.7s, %v, %t)", nid, rIP, isLocal)
n := d.network(nid)
if n == nil || !n.secure {
@@ -115,7 +149,7 @@ func (d *driver) checkEncryption(nid string, rIP net.IP, vxlanID uint32, isLocal
if add {
for _, rIP := range nodes {
if err := setupEncryption(lIP, aIP, rIP, vxlanID, d.secMap, d.keys); err != nil {
if err := setupEncryption(lIP, aIP, rIP, d.secMap, d.keys); err != nil {
logrus.Warnf("Failed to program network encryption between %s and %s: %v", lIP, rIP, err)
}
}
@@ -130,22 +164,14 @@ func (d *driver) checkEncryption(nid string, rIP net.IP, vxlanID uint32, isLocal
return nil
}
func setupEncryption(localIP, advIP, remoteIP net.IP, vni uint32, em *encrMap, keys []*key) error {
logrus.Debugf("Programming encryption for vxlan %d between %s and %s", vni, localIP, remoteIP)
// setupEncryption programs the encryption parameters for secure communication
// between the local node and a remote node.
func setupEncryption(localIP, advIP, remoteIP net.IP, em *encrMap, keys []*key) error {
logrus.Debugf("Programming encryption between %s and %s", localIP, remoteIP)
rIPs := remoteIP.String()
indices := make([]*spi, 0, len(keys))
err := programMangle(vni, true)
if err != nil {
logrus.Warn(err)
}
err = programInput(vni, true)
if err != nil {
logrus.Warn(err)
}
for i, k := range keys {
spis := &spi{buildSPI(advIP, remoteIP, k.tag), buildSPI(remoteIP, advIP, k.tag)}
dir := reverse
@@ -200,67 +226,96 @@ func removeEncryption(localIP, remoteIP net.IP, em *encrMap) error {
return nil
}
func programMangle(vni uint32, add bool) (err error) {
type matchVXLANFunc func(port, vni uint32) []string
// programVXLANRuleFunc returns a function which tries calling programWithMatch
// with the u32 match, falling back to the BPF match if installing u32 variant
// of the rules fails.
func programVXLANRuleFunc(programWithMatch func(matchVXLAN matchVXLANFunc, vni uint32, add bool) error) func(vni uint32, add bool) error {
return func(vni uint32, add bool) error {
if add {
if err := programWithMatch(matchVXLANWithU32, vni, add); err != nil {
// That didn't work. Maybe the xt_u32 module isn't available? Try again with xt_bpf.
err2 := programWithMatch(matchVXLANWithBPF, vni, add)
if err2 != nil {
return multierror.Append(err, err2)
}
}
return nil
} else {
// Delete both flavours.
err := programWithMatch(matchVXLANWithU32, vni, add)
return multierror.Append(err, programWithMatch(matchVXLANWithBPF, vni, add)).ErrorOrNil()
}
}
}
var programMangle = programVXLANRuleFunc(func(matchVXLAN matchVXLANFunc, vni uint32, add bool) error {
var (
p = strconv.FormatUint(uint64(overlayutils.VXLANUDPPort()), 10)
c = fmt.Sprintf("0>>22&0x3C@12&0xFFFFFF00=%d", int(vni)<<8)
m = strconv.FormatUint(uint64(r), 10)
m = strconv.FormatUint(mark, 10)
chain = "OUTPUT"
rule = []string{"-p", "udp", "--dport", p, "-m", "u32", "--u32", c, "-j", "MARK", "--set-mark", m}
a = "-A"
rule = append(matchVXLAN(overlayutils.VXLANUDPPort(), vni), "-j", "MARK", "--set-mark", m)
a = iptables.Append
action = "install"
)
// TODO IPv6 support
iptable := iptables.GetIptable(iptables.IPv4)
if add == iptable.Exists(iptables.Mangle, chain, rule...) {
return
}
if !add {
a = "-D"
a = iptables.Delete
action = "remove"
}
if err = iptable.RawCombinedOutput(append([]string{"-t", string(iptables.Mangle), a, chain}, rule...)...); err != nil {
logrus.Warnf("could not %s mangle rule: %v", action, err)
if err := iptable.ProgramRule(iptables.Mangle, chain, a, rule); err != nil {
return fmt.Errorf("could not %s mangle rule: %w", action, err)
}
return
}
return nil
})
func programInput(vni uint32, add bool) (err error) {
var programInput = programVXLANRuleFunc(func(matchVXLAN matchVXLANFunc, vni uint32, add bool) error {
var (
port = strconv.FormatUint(uint64(overlayutils.VXLANUDPPort()), 10)
vniMatch = fmt.Sprintf("0>>22&0x3C@12&0xFFFFFF00=%d", int(vni)<<8)
plainVxlan = []string{"-p", "udp", "--dport", port, "-m", "u32", "--u32", vniMatch, "-j"}
ipsecVxlan = append([]string{"-m", "policy", "--dir", "in", "--pol", "ipsec"}, plainVxlan...)
block = append(plainVxlan, "DROP")
accept = append(ipsecVxlan, "ACCEPT")
plainVxlan = matchVXLAN(overlayutils.VXLANUDPPort(), vni)
chain = "INPUT"
action = iptables.Append
msg = "add"
)
rule := func(policy, jump string) []string {
args := append([]string{"-m", "policy", "--dir", "in", "--pol", policy}, plainVxlan...)
return append(args, "-j", jump)
}
// TODO IPv6 support
iptable := iptables.GetIptable(iptables.IPv4)
if !add {
action = iptables.Delete
msg = "remove"
}
if err := iptable.ProgramRule(iptables.Filter, chain, action, accept); err != nil {
logrus.Errorf("could not %s input rule: %v. Please do it manually.", msg, err)
action := func(a iptables.Action) iptables.Action {
if !add {
return iptables.Delete
}
return a
}
if err := iptable.ProgramRule(iptables.Filter, chain, action, block); err != nil {
logrus.Errorf("could not %s input rule: %v. Please do it manually.", msg, err)
// Accept incoming VXLAN datagrams for the VNI which were subjected to IPSec processing.
// Append to the bottom of the chain to give administrator-configured rules precedence.
if err := iptable.ProgramRule(iptables.Filter, chain, action(iptables.Append), rule("ipsec", "ACCEPT")); err != nil {
return fmt.Errorf("could not %s input accept rule: %w", msg, err)
}
return
}
// Drop incoming VXLAN datagrams for the VNI which were received in cleartext.
// Insert at the top of the chain so the packets are dropped even if an
// administrator-configured rule exists which would otherwise unconditionally
// accept incoming VXLAN traffic.
if err := iptable.ProgramRule(iptables.Filter, chain, action(iptables.Insert), rule("none", "DROP")); err != nil {
return fmt.Errorf("could not %s input drop rule: %w", msg, err)
}
return nil
})
func programSA(localIP, remoteIP net.IP, spi *spi, k *key, dir int, add bool) (fSA *netlink.XfrmState, rSA *netlink.XfrmState, err error) {
var (
@@ -280,7 +335,7 @@ func programSA(localIP, remoteIP net.IP, spi *spi, k *key, dir int, add bool) (f
Proto: netlink.XFRM_PROTO_ESP,
Spi: spi.reverse,
Mode: netlink.XFRM_MODE_TRANSPORT,
Reqid: r,
Reqid: mark,
}
if add {
rSA.Aead = buildAeadAlgo(k, spi.reverse)
@@ -306,7 +361,7 @@ func programSA(localIP, remoteIP net.IP, spi *spi, k *key, dir int, add bool) (f
Proto: netlink.XFRM_PROTO_ESP,
Spi: spi.forward,
Mode: netlink.XFRM_MODE_TRANSPORT,
Reqid: r,
Reqid: mark,
}
if add {
fSA.Aead = buildAeadAlgo(k, spi.forward)
@@ -355,7 +410,7 @@ func programSP(fSA *netlink.XfrmState, rSA *netlink.XfrmState, add bool) error {
Proto: netlink.XFRM_PROTO_ESP,
Mode: netlink.XFRM_MODE_TRANSPORT,
Spi: fSA.Spi,
Reqid: r,
Reqid: mark,
},
},
}
@@ -569,7 +624,7 @@ func updateNodeKey(lIP, aIP, rIP net.IP, idxs []*spi, curKeys []*key, newIdx, pr
Proto: netlink.XFRM_PROTO_ESP,
Mode: netlink.XFRM_MODE_TRANSPORT,
Spi: fSA2.Spi,
Reqid: r,
Reqid: mark,
},
},
}
@@ -638,7 +693,7 @@ func clearEncryptionStates() {
}
for _, sa := range saList {
sa := sa
if sa.Reqid == r {
if sa.Reqid == mark {
if err := nlh.XfrmStateDel(&sa); err != nil {
logrus.Warnf("Failed to delete stale SA %s: %v", sa, err)
continue

View File

@@ -0,0 +1,17 @@
package overlay
import (
"strconv"
)
// matchVXLANWithBPF returns an iptables rule fragment which matches VXLAN
// datagrams with the given destination port and VXLAN Network ID utilizing the
// xt_bpf netfilter kernel module. The returned slice's backing array is
// guaranteed not to alias any other slice's.
func matchVXLANWithBPF(port, vni uint32) []string {
dport := strconv.FormatUint(uint64(port), 10)
vniMatch := marshalXTBPF(vniMatchBPF(vni))
// https://ipset.netfilter.org/iptables-extensions.man.html#lbAH
return []string{"-p", "udp", "--dport", dport, "-m", "bpf", "--bytecode", vniMatch}
}

View File

@@ -0,0 +1,30 @@
package overlay
import (
"fmt"
"strconv"
)
// matchVXLANWithU32 returns an iptables rule fragment which matches VXLAN
// datagrams with the given destination port and VXLAN Network ID utilizing the
// xt_u32 netfilter kernel module. The returned slice's backing array is
// guaranteed not to alias any other slice's.
func matchVXLANWithU32(port, vni uint32) []string {
dport := strconv.FormatUint(uint64(port), 10)
// The u32 expression language is documented in iptables-extensions(8).
// https://ipset.netfilter.org/iptables-extensions.man.html#lbCK
//
// 0>>22&0x3C ; Compute number of octets in IPv4 header
// @ ; Make this the new offset into the packet
// ; (jump to start of UDP header)
// 12&0xFFFFFF00 ; Read 32-bit value at offset 12 and mask off the bottom octet
// = ; Test whether the value is equal to a constant
//
// A UDP header is eight octets long so offset 12 from the start of the
// UDP header is four octets into the payload: the VNI field of the
// VXLAN header.
vniMatch := fmt.Sprintf("0>>22&0x3C@12&0xFFFFFF00=%d", int(vni)<<8)
return []string{"-p", "udp", "--dport", dport, "-m", "u32", "--u32", vniMatch}
}

View File

@@ -117,7 +117,7 @@ func (d *driver) Join(nid, eid string, sboxKey string, jinfo driverapi.JoinInfo,
d.peerAdd(nid, eid, ep.addr.IP, ep.addr.Mask, ep.mac, net.ParseIP(d.advertiseAddress), false, false, true)
if err = d.checkEncryption(nid, nil, n.vxlanID(s), true, true); err != nil {
if err = d.checkEncryption(nid, nil, true, true); err != nil {
logrus.Warn(err)
}

View File

@@ -24,6 +24,7 @@ import (
"github.com/docker/docker/libnetwork/resolvconf"
"github.com/docker/docker/libnetwork/types"
"github.com/docker/docker/pkg/reexec"
"github.com/hashicorp/go-multierror"
"github.com/sirupsen/logrus"
"github.com/vishvananda/netlink"
"github.com/vishvananda/netlink/nl"
@@ -666,6 +667,18 @@ func (n *network) initSubnetSandbox(s *subnet, restore bool) error {
brName := n.generateBridgeName(s)
vxlanName := n.generateVxlanName(s)
// Program iptables rules for mandatory encryption of the secure
// network, or clean up leftover rules for a stale secure network which
// was previously assigned the same VNI.
if err := programMangle(s.vni, n.secure); err != nil {
return err
}
if err := programInput(s.vni, n.secure); err != nil {
if n.secure {
return multierror.Append(err, programMangle(s.vni, false))
}
}
if restore {
if err := n.restoreSubnetSandbox(s, brName, vxlanName); err != nil {
return err

View File

@@ -387,7 +387,7 @@ func (d *driver) peerAddOp(nid, eid string, peerIP net.IP, peerIPMask net.IPMask
return fmt.Errorf("subnet sandbox join failed for %q: %v", s.subnetIP.String(), err)
}
if err := d.checkEncryption(nid, vtep, n.vxlanID(s), false, true); err != nil {
if err := d.checkEncryption(nid, vtep, false, true); err != nil {
logrus.Warn(err)
}
@@ -447,7 +447,7 @@ func (d *driver) peerDeleteOp(nid, eid string, peerIP net.IP, peerIPMask net.IPM
return nil
}
if err := d.checkEncryption(nid, vtep, 0, localPeer, false); err != nil {
if err := d.checkEncryption(nid, vtep, localPeer, false); err != nil {
logrus.Warn(err)
}

View File

@@ -681,8 +681,8 @@ func (d *driver) CreateEndpoint(nid, eid string, ifInfo driverapi.InterfaceInfo,
}
if n.driver.name == "nat" && !epOption.DisableDNS {
logrus.Debugf("endpointStruct.EnableInternalDNS =[%v]", endpointStruct.EnableInternalDNS)
endpointStruct.EnableInternalDNS = true
logrus.Debugf("endpointStruct.EnableInternalDNS =[%v]", endpointStruct.EnableInternalDNS)
}
endpointStruct.DisableICC = epOption.DisableICC

View File

@@ -1061,9 +1061,6 @@ func (n *network) delete(force bool, rmLBEndpoint bool) error {
}
n.ipamRelease()
if err = c.updateToStore(n); err != nil {
logrus.Warnf("Failed to update store after ipam release for network %s (%s): %v", n.Name(), n.ID(), err)
}
// We are about to delete the network. Leave the gossip
// cluster for the network to stop all incoming network

View File

@@ -42,19 +42,20 @@ require (
github.com/gorilla/mux v1.8.0
github.com/hashicorp/go-immutable-radix v1.3.1
github.com/hashicorp/go-memdb v1.3.2
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/memberlist v0.4.0
github.com/hashicorp/serf v0.8.5
github.com/imdario/mergo v0.3.12
github.com/ishidawataru/sctp v0.0.0-20210707070123-9a39160e9062
github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2
github.com/klauspost/compress v1.15.12
github.com/miekg/dns v1.1.43
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible
github.com/moby/buildkit v0.10.7-0.20230306143919-70f2ad56d3e5
github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242
github.com/moby/ipvs v1.1.0
github.com/moby/locker v1.0.1
github.com/moby/patternmatcher v0.5.0
github.com/moby/pubsub v1.0.0
github.com/moby/swarmkit/v2 v2.0.0-20230309194213-a745a8755ce3
github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83
github.com/moby/sys/mount v0.3.3
github.com/moby/sys/mountinfo v0.6.2
github.com/moby/sys/sequential v0.5.0
@@ -64,7 +65,7 @@ require (
github.com/morikuni/aec v1.0.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.0.3-0.20220303224323-02efb9a75ee1
github.com/opencontainers/runc v1.1.3
github.com/opencontainers/runc v1.1.5
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417
github.com/opencontainers/selinux v1.10.2
github.com/pelletier/go-toml v1.9.5
@@ -125,7 +126,6 @@ require (
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-msgpack v0.5.5 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-sockaddr v1.0.2 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect

View File

@@ -644,8 +644,8 @@ github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/ishidawataru/sctp v0.0.0-20210707070123-9a39160e9062 h1:G1+wBT0dwjIrBdLy0MIG0i+E4CQxEnedHXdauJEIH6g=
github.com/ishidawataru/sctp v0.0.0-20210707070123-9a39160e9062/go.mod h1:co9pwDoBCm1kGxawmb4sPq0cSIOOWNPT4KnHotMP1Zg=
github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2 h1:i2fYnDurfLlJH8AyyMOnkLHnHeP8Ff/DDpuZA/D3bPo=
github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2/go.mod h1:co9pwDoBCm1kGxawmb4sPq0cSIOOWNPT4KnHotMP1Zg=
github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56/go.mod h1:ymszkNOg6tORTn+6F6j+Jc8TOr5osrynvN6ivFWZ2GA=
github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jmespath/go-jmespath v0.0.0-20160803190731-bd40a432e4c7/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
@@ -728,8 +728,8 @@ github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0Qu
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A=
github.com/moby/buildkit v0.10.7-0.20230306143919-70f2ad56d3e5 h1:1CEKLCfx4WEWbP4A+cI61IR6pC5h6cAN+20CXsU7NRU=
github.com/moby/buildkit v0.10.7-0.20230306143919-70f2ad56d3e5/go.mod h1:tQuuyTWtOb9D+RE425cwOCUkX0/oZ+5iBZ+uWpWQ9bU=
github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242 h1:YEWromfSEDvSx13xK7jZwoNTNnYEgpguSpedsXpZ6PA=
github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242/go.mod h1:tQuuyTWtOb9D+RE425cwOCUkX0/oZ+5iBZ+uWpWQ9bU=
github.com/moby/ipvs v1.1.0 h1:ONN4pGaZQgAx+1Scz5RvWV4Q7Gb+mvfRh3NsPS+1XQQ=
github.com/moby/ipvs v1.1.0/go.mod h1:4VJMWuf098bsUMmZEiD4Tjk/O7mOn3l1PTD3s4OoYAs=
github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
@@ -738,8 +738,8 @@ github.com/moby/patternmatcher v0.5.0 h1:YCZgJOeULcxLw1Q+sVR636pmS7sPEn1Qo2iAN6M
github.com/moby/patternmatcher v0.5.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/pubsub v1.0.0 h1:jkp/imWsmJz2f6LyFsk7EkVeN2HxR/HTTOY8kHrsxfA=
github.com/moby/pubsub v1.0.0/go.mod h1:bXSO+3h5MNXXCaEG+6/NlAIk7MMZbySZlnB+cUQhKKc=
github.com/moby/swarmkit/v2 v2.0.0-20230309194213-a745a8755ce3 h1:05e6sB9az9OINsgqSy1PiSC9i0ffkpfQd0oJGAigo6k=
github.com/moby/swarmkit/v2 v2.0.0-20230309194213-a745a8755ce3/go.mod h1:GvjR7mC8YuUd9Mq44lrrIZPaXyKPAGEUMBpAQzaj3dI=
github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83 h1:jUbNDiRMDXd2rYoa4bcI+g3nIb4A1R8HNCe9wdCdh8I=
github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83/go.mod h1:GvjR7mC8YuUd9Mq44lrrIZPaXyKPAGEUMBpAQzaj3dI=
github.com/moby/sys/mount v0.3.3 h1:fX1SVkXFJ47XWDoeFW4Sq7PdQJnV2QIDZAqjNqgEjUs=
github.com/moby/sys/mount v0.3.3/go.mod h1:PBaEorSNTLG5t/+4EgukEQVlAvVEc6ZjTySwKdqp5K0=
github.com/moby/sys/mountinfo v0.4.0/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2JI+6q0qou+A=
@@ -810,8 +810,8 @@ github.com/opencontainers/runc v1.0.0-rc8.0.20190926000215-3e425f80a8c9/go.mod h
github.com/opencontainers/runc v1.0.0-rc9/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opencontainers/runc v1.0.0-rc93/go.mod h1:3NOsor4w32B2tC0Zbl8Knk4Wg84SM2ImC1fxBuqJ/H0=
github.com/opencontainers/runc v1.0.2/go.mod h1:aTaHFFwQXuA71CiyxOdFFIorAoemI04suvGRQFzWTD0=
github.com/opencontainers/runc v1.1.3 h1:vIXrkId+0/J2Ymu2m7VjGvbSlAId9XNRPhn2p4b+d8w=
github.com/opencontainers/runc v1.1.3/go.mod h1:1J5XiS+vdZ3wCyZybsuxXZWGrgSr8fFJHLXuG2PsnNg=
github.com/opencontainers/runc v1.1.5 h1:L44KXEpKmfWDcS02aeGm8QNTFXTo2D+8MYGDIJ/GDEs=
github.com/opencontainers/runc v1.1.5/go.mod h1:1J5XiS+vdZ3wCyZybsuxXZWGrgSr8fFJHLXuG2PsnNg=
github.com/opencontainers/runtime-spec v0.1.2-0.20190507144316-5b71a03e2700/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.2-0.20190207185410-29686dbc5559/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=

View File

@@ -22,6 +22,7 @@ import (
"sync/atomic"
"syscall"
"unsafe"
"runtime"
)
func setsockopt(fd int, optname, optval, optlen uintptr) (uintptr, uintptr, error) {
@@ -40,6 +41,9 @@ func setsockopt(fd int, optname, optval, optlen uintptr) (uintptr, uintptr, erro
}
func getsockopt(fd int, optname, optval, optlen uintptr) (uintptr, uintptr, error) {
if runtime.GOARCH == "s390x" {
optlen = uintptr(unsafe.Pointer(&optlen))
}
// FIXME: syscall.SYS_GETSOCKOPT is undefined on 386
r0, r1, errno := syscall.Syscall6(syscall.SYS_GETSOCKOPT,
uintptr(fd),
@@ -292,7 +296,7 @@ func dialSCTPExtConfig(network string, laddr, raddr *SCTPAddr, options InitMsg,
laddr.IPAddrs = append(laddr.IPAddrs, net.IPAddr{IP: net.IPv6zero})
}
}
err := SCTPBind(sock, laddr, SCTP_BINDX_ADD_ADDR)
err = SCTPBind(sock, laddr, SCTP_BINDX_ADD_ADDR)
if err != nil {
return nil, err
}

View File

@@ -301,7 +301,14 @@ func (cm *cacheManager) GetByBlob(ctx context.Context, desc ocispecs.Descriptor,
cm.records[id] = rec
return rec.ref(true, descHandlers, nil), nil
ref := rec.ref(true, descHandlers, nil)
if s := unlazySessionOf(opts...); s != nil {
if err := ref.unlazy(ctx, ref.descHandlers, ref.progress, s, true); err != nil {
return nil, err
}
}
return ref, nil
}
// init loads all snapshots from metadata state and tries to load the records

View File

@@ -37,3 +37,14 @@ func (m NeedsRemoteProviderError) Error() string {
}
type ProgressKey struct{}
type Unlazy session.Group
func unlazySessionOf(opts ...RefOption) session.Group {
for _, opt := range opts {
if opt, ok := opt.(session.Group); ok {
return opt
}
}
return nil
}

View File

@@ -98,7 +98,16 @@ func getContentStore(ctx context.Context, sm *session.Manager, g session.Group,
if err != nil {
return nil, err
}
return sessioncontent.NewCallerStore(caller, storeID), nil
return &unlazyProvider{sessioncontent.NewCallerStore(caller, storeID), g}, nil
}
type unlazyProvider struct {
content.Store
s session.Group
}
func (p *unlazyProvider) UnlazySession(desc ocispecs.Descriptor) session.Group {
return p.s
}
func attrsToCompression(attrs map[string]string) (*compression.Config, error) {

View File

@@ -91,6 +91,7 @@ func parseUID(str string) (uint32, error) {
// once the PR in containerd is merged we should remove this function.
func WithUIDGID(uid, gid uint32, sgids []uint32) containerdoci.SpecOpts {
return func(_ context.Context, _ containerdoci.Client, _ *containers.Container, s *containerdoci.Spec) error {
defer ensureAdditionalGids(s)
setProcess(s)
s.Process.User.UID = uid
s.Process.User.GID = gid
@@ -106,3 +107,15 @@ func setProcess(s *containerdoci.Spec) {
s.Process = &specs.Process{}
}
}
// ensureAdditionalGids ensures that the primary GID is also included in the additional GID list.
// From https://github.com/containerd/containerd/blob/v1.7.0-beta.4/oci/spec_opts.go#L124-L133
func ensureAdditionalGids(s *containerdoci.Spec) {
setProcess(s)
for _, f := range s.Process.User.AdditionalGids {
if f == s.Process.User.GID {
return
}
}
s.Process.User.AdditionalGids = append([]uint32{s.Process.User.GID}, s.Process.User.AdditionalGids...)
}

View File

@@ -6,6 +6,7 @@ import (
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/errdefs"
"github.com/moby/buildkit/session"
digest "github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -90,3 +91,23 @@ func (mp *MultiProvider) Add(dgst digest.Digest, p content.Provider) {
defer mp.mu.Unlock()
mp.sub[dgst] = p
}
func (mp *MultiProvider) UnlazySession(desc ocispecs.Descriptor) session.Group {
type unlazySession interface {
UnlazySession(ocispecs.Descriptor) session.Group
}
mp.mu.RLock()
if p, ok := mp.sub[desc.Digest]; ok {
mp.mu.RUnlock()
if cd, ok := p.(unlazySession); ok {
return cd.UnlazySession(desc)
}
} else {
mp.mu.RUnlock()
}
if cd, ok := mp.base.(unlazySession); ok {
return cd.UnlazySession(desc)
}
return nil
}

View File

@@ -6,11 +6,14 @@ import (
"fmt"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"github.com/container-storage-interface/spec/lib/go/csi"
"github.com/docker/docker/pkg/plugingetter"
"github.com/moby/swarmkit/v2/api"
"github.com/moby/swarmkit/v2/internal/csi/capability"
"github.com/moby/swarmkit/v2/log"
)
// Plugin is the interface for a CSI controller plugin.
@@ -204,6 +207,11 @@ func (p *plugin) PublishVolume(ctx context.Context, v *api.Volume, nodeID string
if !p.publisher {
return nil, nil
}
csiNodeID := p.swarmToCSI[nodeID]
if csiNodeID == "" {
log.L.Errorf("CSI node ID not found for given Swarm node ID. Plugin: %s , Swarm node ID: %s", p.name, nodeID)
return nil, status.Error(codes.FailedPrecondition, "CSI node ID not found for given Swarm node ID")
}
req := p.makeControllerPublishVolumeRequest(v, nodeID)
c, err := p.Client(ctx)

8
vendor/modules.txt vendored
View File

@@ -459,7 +459,7 @@ github.com/imdario/mergo
# github.com/inconshreveable/mousetrap v1.0.1
## explicit; go 1.18
github.com/inconshreveable/mousetrap
# github.com/ishidawataru/sctp v0.0.0-20210707070123-9a39160e9062
# github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2
## explicit; go 1.12
github.com/ishidawataru/sctp
# github.com/jmespath/go-jmespath v0.4.0
@@ -486,7 +486,7 @@ github.com/mistifyio/go-zfs
# github.com/mitchellh/hashstructure/v2 v2.0.2
## explicit; go 1.14
github.com/mitchellh/hashstructure/v2
# github.com/moby/buildkit v0.10.7-0.20230306143919-70f2ad56d3e5
# github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242
## explicit; go 1.17
github.com/moby/buildkit/api/services/control
github.com/moby/buildkit/api/types
@@ -611,7 +611,7 @@ github.com/moby/patternmatcher
# github.com/moby/pubsub v1.0.0
## explicit; go 1.19
github.com/moby/pubsub
# github.com/moby/swarmkit/v2 v2.0.0-20230309194213-a745a8755ce3
# github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83
## explicit; go 1.18
github.com/moby/swarmkit/v2/agent
github.com/moby/swarmkit/v2/agent/configs
@@ -711,7 +711,7 @@ github.com/opencontainers/go-digest/digestset
github.com/opencontainers/image-spec/identity
github.com/opencontainers/image-spec/specs-go
github.com/opencontainers/image-spec/specs-go/v1
# github.com/opencontainers/runc v1.1.3
# github.com/opencontainers/runc v1.1.5
## explicit; go 1.16
github.com/opencontainers/runc/libcontainer/cgroups
github.com/opencontainers/runc/libcontainer/configs