On startup all local volumes were unmounted as a cleanup mechanism for
the non-clean exit of the last engine process.
This caused live-restored volumes that used special volume opt mount
flags to be broken. While the refcount was restored, the _data directory
was just unmounted, so all new containers mounting this volume would
just have the access to the empty _data directory instead of the real
volume.
With this patch, the mountpoint isn't unmounted. Instead, if the volume
is already mounted, just mark it as mounted, so the next time Mount is
called only the ref count is incremented, but no second attempt to mount
it is performed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2689484402)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make sure that the content in the live-restored volume mounted in a new
container is the same as the content in the old container.
This checks if volume's _data directory doesn't get unmounted on
startup.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit aef703fa1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This documentation moved to a different page, and the Go documentation
moved to the https://go.dev/ domain.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2aabd64477)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previous image created a new partially filled image.
This caused child images to lose their parent's layers.
Instead of creating a new object and trying to replace its fields, just
clone the original passed image and change its ID to the manifest
digest.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 01214bafd2)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
If the lease doesn't exit (for example when creating the container
failed), just ignore the not found error.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit bedcc94de4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows the legacy builder to apply changes to the `FROM scratch`
layer.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit dfaff9598c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the diff is empty and don't produce an empty layer.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit eb56493f4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If the image for the wanted platform doesn't exist then the lease
doesn't exist either. Returning this error hides the real error, so
let's not return it.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit b8ff8ea58e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This better aligns to GHA/CI settings, and is in general a better
practice in the year 2023.
We also drop the 'unsupported' fallback for `git rev-parse` in the
Makefile; we have a better fallback behavior for an empty
DOCKER_GITCOMMIT in `hack/make.sh`.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit d125823d3f)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
There are still messy special cases (e.g. DOCKER_GITCOMMIT vs VERSION),
but this makes things a little easier to follow, as we keep
GHA-specifics in the GHA files.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit ad91fc1b00)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Dockerfile.e2e is not used anymore. Integration tests run
through the main Dockerfile.
Also removes the daemon OS/Arch detection script that is not
necessary anymore. It was used to select the Dockerfile based
on the arch like Dockerfile.arm64 but we don't have those
anymore. Was also used to check referenced frozen images
in the Dockerfile.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 5efe72415d)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
When we added this deprecation warning, some registries had not yet
moved away from the deprecated specification, so we made the warning
conditional for pulling from Docker Hub.
That condition was added in 647dfe99a5,
which is over 4 Years ago, which should be time enough for images
and registries to have moved to current specifications.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c4af5dacb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Use the same warning for both "v1 in manifest-index" and bare "v1" images.
- Update URL to use a "/go/" redirect, which allows the docs team to more
easily redirect the URL to relevant docs (if things move).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 982bc0e228)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I had a CI run fail to "Upload reports":
Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
Finished backoff for retry #1, continuing with upload
Total file count: 211 ---- Processed file #160 (75.8%)
...
Total file count: 211 ---- Processed file #164 (77.7%)
Total file count: 211 ---- Processed file #164 (77.7%)
Total file count: 211 ---- Processed file #164 (77.7%)
A 503 status code has been received, will attempt to retry the upload
##### Begin Diagnostic HTTP information #####
Status Code: 503
Status Message: Service Unavailable
Header Information: {
"content-length": "592",
"content-type": "application/json; charset=utf-8",
"date": "Mon, 21 Aug 2023 14:08:10 GMT",
"server": "Kestrel",
"cache-control": "no-store,no-cache",
"pragma": "no-cache",
"strict-transport-security": "max-age=2592000",
"x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
"activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
"x-frame-options": "SAMEORIGIN"
}
###### End Diagnostic HTTP information ######
Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
Error: aborting artifact upload
Total file count: 211 ---- Processed file #165 (78.1%)
A 503 status code has been received, will attempt to retry the upload
Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0
As a result, the "Download reports" continued retrying:
...
Total file count: 1004 ---- Processed file #436 (43.4%)
Total file count: 1004 ---- Processed file #436 (43.4%)
Total file count: 1004 ---- Processed file #436 (43.4%)
An error occurred while attempting to download a file
Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
at Object.onceWrapper (node:events:627:28)
at ClientRequest.emit (node:events:513:28)
at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
at Object.onceWrapper (node:events:627:28)
at TLSSocket.emit (node:events:525:35)
at TLSSocket.Socket._onTimeout (node:net:550:8)
at listOnTimeout (node:internal/timers:559:17)
at processTimers (node:internal/timers:502:7)
Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
Total file count: 1004 ---- Processed file #436 (43.4%)
And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d6f340e784)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The container rw layer may already be mounted, so it's not safe to use
it in another overlay mount. Use the ref counted mounter (which will
reuse the existing mount if it exists) to avoid that.
Also, mount the parent mounts (layers of the base image) in a read-only
mode.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6da42ca830)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
To prevent mounting the container rootfs in a rw mode if it's already
mounted. This can't use `mount.WithReadonlyTempMount` because the
archive code does a chroot with a pivot_root, which creates a new
directory in the rootfs.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 051d51b222)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Check that operations that could potentially perform overlayfs mounts
that could cause undefined behaviors.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 303e2b124e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
"HEAD" will still be used as a version if no DOCKER_COMMIT is provided
(for example when not running via `make`), but it won't prevent it being
set to the GITHUB_SHA variable when it's present.
This should fix `Git commit` reported by `docker version` for the
binaries generated by `moby-bin`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d7a9f15775)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Fix a failure to inspect image if any of its present manifest references
an image config which isn't present locally.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a64adda4e7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Currently moby drops ep sets before the entrypoint is executed.
This does mean that with combination of no-new-privileges the
file capabilities stops working with non-root containers.
This is undesired as the usability of such containers is harmed
comparing to running root containers.
This commit therefore sets the effective/permitted set in order
to allow use of file capabilities or libcap(3)/prctl(2) respectively
with combination of no-new-privileges and without respectively.
For no-new-privileges the container will be able to obtain capabilities
that are requested.
Signed-off-by: Luboslav Pivarc <lpivarc@redhat.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 3aef732e61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a workaround to have buildinfo with deps embedded in the
binary. We need to create a go.mod file before building with
-modfile=vendor.mod, otherwise it fails with:
"-modfile cannot be used to set the module root directory."
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7665feeb52)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.6.21...v1.6.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.22
---
Notable Updates
- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
when `securityContext.RunAsUser`, `RunAsUsername` are empty
- CRI: Write generated CNI config atomically
- Fix concurrent writes for `UpdateContainerStats`
- Make `checkContainerTimestamps` less strict on Windows
- Port-Forward: Correctly handle known errors
- Resolve `docker.NewResolver` race condition
- SecComp: Always allow `name_to_handle_at`
- Adding support to run hcsshim from local clone
- Pinned image support
- Runtime/V2/RunC: Handle early exits w/o big locks
- CRITool: Move up to CRI-TOOLS v1.27.0
- Fix cpu architecture detection issue on emulated ARM platform
- Task: Don't `close()` io before `cancel()`
- Fix panic when remote differ returns empty result
- Plugins: Notify readiness when registered plugins are ready
- Unwrap io errors in server connection receive error handling
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4d674897f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows.
This issue was not limited to the go command itself, and could also affect binaries
that use `os.Command`, `os.LookPath`, etc.
From the related blogpost (https://blog.golang.org/path-security):
> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing
At time of the go1.15 release, the Go team considered changing the behavior of
`os.LookPath()` and `exec.LookPath()` to be a breaking change, and made the
behavior "opt-in" by providing the `golang.org/x/sys/execabs` package as a
replacement.
However, for the go1.19 release, this changed, and the default behavior of
`os.LookPath()` and `exec.LookPath()` was changed. From the release notes:
https://go.dev/doc/go1.19#os-exec-path
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.
A result of this change was that registering the daemon as a Windows service
no longer worked when done from within the directory of the binary itself:
C:\> cd "Program Files\Docker\Docker\resources"
C:\Program Files\Docker\Docker\resources> dockerd --register-service
exec: "dockerd": cannot run executable found relative to current directory
Note that using an absolute path would work around the issue:
C:\Program Files\Docker\Docker>resources\dockerd.exe --register-service
This patch changes `registerService()` to use `os.Executable()`, instead of
depending on `os.Args[0]` and `exec.LookPath()` for resolving the absolute
path of the binary.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e8fda0a70)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
IPv6 ipt rules are exactly the same as IPv4 rules, although both
protocol don't use the same networking model. This has bad consequences,
for instance: 1. the current v6 rules disallow Neighbor
Solication/Advertisement ; 2. multicast addresses can't be used ; 3.
link-local addresses are blocked too.
To solve this, this commit changes the following rules:
```
-A DOCKER-ISOLATION-STAGE-1 ! -s fdf1:a844:380c:b247::/64 -o br-21502e5b2c6c -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fdf1:a844:380c:b247::/64 -i br-21502e5b2c6c -j DROP
```
into:
```
-A DOCKER-ISOLATION-STAGE-1 ! -s fdf1:a844:380c:b247::/64 ! -i br-21502e5b2c6c -o br-21502e5b2c6c -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fdf1:a844:380c:b247::/64 -i br-21502e5b2c6c ! -o br-21502e5b2c6c -j DROP
```
These rules only limit the traffic ingressing/egressing the bridge, but
not traffic between veth on the same bridge.
Note that, the Kernel takes care of dropping invalid IPv6 packets, eg.
loopback spoofing, thus these rules don't need to be more specific.
Solve #45460.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit da9e44a620)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If TEST_INTEGRATION_FAIL_FAST is not set, run the integration-cli tests
even if integration tests failed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6841a53d17)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Includes a fix for CVE-2023-29409
go1.20.7 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.20.7 milestone on our issue tracker for details:
- https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.6...go1.20.7
From the mailing list announcement:
[security] Go 1.20.7 and Go 1.19.12 are released
Hello gophers,
We have just released Go versions 1.20.7 and 1.19.12, minor point releases.
These minor releases include 1 security fixes following the security policy:
- crypto/tls: restrict RSA keys in certificates to <= 8192 bits
Extremely large RSA keys in certificate chains can cause a client/server
to expend significant CPU time verifying signatures. Limit this by
restricting the size of RSA keys transmitted during handshakes to <=
8192 bits.
Based on a survey of publicly trusted RSA keys, there are currently only
three certificates in circulation with keys larger than this, and all
three appear to be test certificates that are not actively deployed. It
is possible there are larger keys in use in private PKIs, but we target
the web PKI, so causing breakage here in the interests of increasing the
default safety of users of crypto/tls seems reasonable.
Thanks to Mateusz Poliwczak for reporting this issue.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d5cb7cdeae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also fixes up some cleanup issues.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1a51898d2e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I noticed this was always being skipped because of race conditions
checking the logs.
This change adds a log scanner which will look through the logs line by
line rather than allocating a big buffer.
Additionally it adds a `poll.Check` which we can use to actually wait
for the desired log entry.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 476e788090)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Allows tests to report their proxy settings for easier troubleshooting
on failures.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 8197752d68)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. On failed start tail the daemon logs
2. Exposes generic tailing functions to make test debugging simpler
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 914888cf8b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- go.mod: update dependencies and go version by
- Use Go1.20
- Fix couple of typos
- Added `WithStdout` and `WithStderr` helpers
- Moved `cmdOperators` handling from `RunCmd` to `StartCmd`
- Deprecate `assert.ErrorType`
- Remove outdated Dockerfile
- add godoc links
full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.4.0...v3.5.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ce053a14aa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I found that the errors returned weren't matched with `errors.Is()` when
wrapped.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 490fee7d45)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.2...v1.7.3
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.3
----
Welcome to the v1.7.3 release of containerd!
The third patch release for containerd 1.7 contains various fixes and updates.
Notable Updates
- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
when `securityContext.RunAsUser`,`RunAsUsername` are empty
- CRI: write generated CNI config atomically
- Port-Forward: Correctly handle known errors
- Resolve docker.NewResolver race condition
- Fix `net.ipv4.ping_group_range` with userns
- Runtime/V2/RunC: handle early exits w/o big locks
- SecComp: always allow `name_to_handle_at`
- CRI: Windows Pod Stats: Add a check to skip stats for containers that
are not running
- Task: don't `close()` io before cancel()
- Remove CNI conf_template deprecation
- Fix issue for HPC pod metrics
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bf48d3ec29)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.1...v1.7.2
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.2
----
Welcome to the v1.7.2 release of containerd!
The second patch release for containerd 1.7 includes enhancements to CRI
sandbox mode, Windows snapshot mounting support, and CRI and container IO
bug fixes.
CRI/Sandbox Updates
- Publish sandbox events
- Make stats respect sandbox's platform
Other Notable Updates
- Mount snapshots on Windows
- Notify readiness when registered plugins are ready
- Fix `cio.Cancel()` should close pipes
- CDI: Use CRI `Config.CDIDevices` field for CDI injection
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a78381c399)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Refactor GetContainerLayerSize to calculate unpacked image size only by
following the snapshot parent tree directly instead of following it by
using diff ids from image config.
This works even if the original manifest/config used to create that
container is no longer present in the content store.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4d8e3f54cc)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
refreshImage is the only function used as a reducer and it doesn't use
the `filter *listContext`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 13180c1c49)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Check for generic `errdefs.NotFound` rather than specific error helper
struct when checking if the error is caused by the image not being
present.
It still works for `ErrImageDoesNotExist` because it
implements the NotFound errdefs interface too.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5a39bee635)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Before this change, integration test would fail fast and not execute all
test suites when one suite fails.
Change this behavior into opt-in enabled by TEST_INTEGRATION_FAIL_FAST
variable.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 48cc28e4ef)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit 0d8087fbbc)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.
Additionally, this removes the "cgroups v1" setup code because it's more reasonable now for us to expect something _else_ to have set up cgroups appropriately (especially cgroups v2).
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit ae737656f9)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.8
full diff: https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.9
This is the eighth patch release of the 1.1.z release branch of runc.
The most notable change is the addition of RISC-V support, along with a
few bug fixes.
- Support riscv64.
- init: do not print environment variable value.
- libct: fix a race with systemd removal.
- tests/int: increase num retries for oom tests.
- man/runc: fixes.
- Fix tmpfs mode opts when dir already exists.
- docs/systemd: fix a broken link.
- ci/cirrus: enable some rootless tests on cs9.
- runc delete: call systemd's reset-failed.
- libct/cg/sd/v1: do not update non-frozen cgroup after frozen failed.
- CI: bump Fedora, Vagrant, bats.
- .codespellrc: update for 2.2.5.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit df86d855f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This change add leases for all the content that will be exported, once
the image(s) are exported the lease is removed, thus letting
containerd's GC to do its job if needed. This fixes the case where
someone would remove an image that is still being exported.
This fixes the TestAPIImagesSaveAndLoad cli integration test.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit f3a6b0fd08)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.
We also better document the script, and follow some best practices for
both POSIX sh and Bash.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 4ecc01f3ad)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
When resolving a reference that is both a Named and Digested, it could
be resolved to an image that has the same digest, but completely
different repository name.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 48fc306764fc5c39d4284021520a0337ef7e0cb0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
If image name is already an untagged digested reference, don't produce
additional digested ref.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 028eab9ebb)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)
The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.
This patch removes the gotest.tools code from this file.
Before this patch:
$ (exec -a libnetwork-setkey "$(which dockerd)" -help)
Usage of libnetwork-setkey:
-exec-root string
docker exec root (default "/run/docker")
-update
update golden values
With this patch applied:
$ (exec -a libnetwork-setkey "$(which dockerd)" -help)
Usage of libnetwork-setkey:
-exec-root string
docker exec root (default "/run/docker")
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1aa17222e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `-f` to output nothing to tar if the curl fails, and `-S` to report
errors if they happen.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 780e8b2332)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Use bind-mounts instead of a `COPY` for cli.sh, and use `COPY --link`
for rootlesskit's build stage.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 12a19dcd84)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Use a non-slash escape sequence to support mirrors with a path
component, and do not unconditionally replace the mirror in
Dockerfile.simple.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 235cd6c6b2)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This aligns `docker build` as invoked by the Makefile with both `docker
buildx bake` as invoked by the Makefile and directly by the user.
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit bcea83ab9b)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Use Client.buildRequest instead of a local copy of the same logic so
that we're using the same logic, and there's less chance of diverging.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c219b09d4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Just making my IDE and some linters slightly happier.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e11555218b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We can't upload the same file in a matrix so generate
metadata in prepare job instead. Also fixes wrong bake meta
file in merge job.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 0a126a85a4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon sleeps for 15 seconds at start up when the API binds to a TCP
socket with no TLS certificate set. That's what the hack/make/run script
does, but it doesn't explicitly disable tls, thus we're experiencing
this annoying delay every time we use this script.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 6b1b71ced4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With this change, the API will now return a 403 instead of a 500 when
trying to create an overlay network on a non-manager node.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit d29240d9eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The commit befff0e13f inadvertendly
disabled the error returned when trying to create an overlay network on
a node which is not part of a Swarm cluster.
Since commit e3708a89cc the overlay
netdriver returns the error: `no VNI provided`.
This commit reinstate the original error message by checking if the node
is a manager before calling libnetwork's `controller.NewNetwork()`.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 21dcbada2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also removed some newlines from t.Fatal() as they shouldn't be needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c3d533f37f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202907b14c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.20.6 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to the compiler, cgo, the cover tool, the go command,
the runtime, and the crypto/ecdsa, go/build, go/printer, net/mail, and text/template
packages. See the Go 1.20.6 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved
Full diff: https://github.com/golang/go/compare/go1.20.5...go1.20.6
These minor releases include 1 security fixes following the security policy:
net/http: insufficient sanitization of Host header
The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.
Thanks to Bartek Nowotarski for reporting this issue.
Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374
[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1ead2dd35d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.
The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in https://go.dev/issue/60374.
Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.
Before this patch, tests would fail on go1.20.6:
=== FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header
[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7705d5b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.
The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in https://go.dev/issue/60374.
Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.
This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.
Before this patch (using go1.20.6):
make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
=== RUN TestAttachWithTTY
attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
--- FAIL: TestAttachWithTTY (0.11s)
=== RUN TestAttachWithoutTTy
attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
--- FAIL: TestAttachWithoutTTy (0.02s)
FAIL
With this patch applied:
make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
INFO: Testing against a local daemon
=== RUN TestAttachWithTTY
--- PASS: TestAttachWithTTY (0.12s)
=== RUN TestAttachWithoutTTy
--- PASS: TestAttachWithoutTTy (0.02s)
PASS
[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92975f0c11)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Calling function returned from setupTest (which calls testEnv.Clean) in
a defer block inside a test that spawns parallel subtests caused the
cleanup function to be called before any of the subtest did anything.
Change the defer expressions to use `t.Cleanup` instead to call it only
after all subtests have also finished.
This only changes tests which have parallel subtests.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit f9e2eed55d)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.
Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit cdcb7c28c5)
Signed-off-by: Cory Snider <csnider@mirantis.com>
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9cee34bc94)
Signed-off-by: Cory Snider <csnider@mirantis.com>
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)
However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)
This would panic if the MountPoint was not a volume;
github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
created by github.com/docker/docker/daemon.(*Daemon).restore
/root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]
This issue was introduced in 647c2a6cdd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a490248f4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f6258f70cb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 892e9f2c23)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a4bdfb963f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 85ccb25eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This option was added in 8cb2229cd1 for
API version 1.28, but forgot to update the documentation and version
history.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aba8e04ab1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This has been around for a long time - since v17.04 (API v1.28)
but was never documented.
It allows removing a plugin even if it's still in use.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
(cherry picked from commit eb0edeafdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ed0dbb8518)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b770a50dee)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 18d77ff455)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92f1ddaf0a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece
This patch removes the fields from the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e8f206972a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece
This patch removes the fields from the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e58a60902c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `ClusterStore` and `ClusterAdvertise` fields were deprecated in commit
616e64b42f (and would no longer be included in
the `/info` API response), and were fully removed in 24.0.0 through commit
68bf777ece
This patch removes the fields from the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c905d0db9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/containerd/cgroups/compare/v3.0.1...v3.0.2
relevant changes:
- cgroup2: only enable the cpuset controller if cpus or mems is specified
- cgroup1 delete: proceed to the next subsystem when a cgroup is not found
- Cgroup2: Reduce allocations for manager.Stat
- Improve performance by for pid stats (cgroups1) re-using readuint
- Reduce allocs in ReadUint64 by pre-allocating byte buffer
- cgroup2: rm/simplify some code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f379af6d17)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If an image is only by id instead of its name, don't prune it
completely. but only untag it and create a dangling image for it.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e638351ef9)
Resolved conflicts:
daemon/containerd/image_prune.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Linux 6.2 and up (commit [f1f1f2569901ec5b9d425f2e91c09a0e320768f3][1])
provides a fast path for the number of open files for the process.
From the [Linux docs][2]:
> The number of open files for the process is stored in 'size' member of
> `stat()` output for /proc/<pid>/fd for fast access.
[1]: f1f1f25699
[2]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files
This patch adds a fast-path for Kernels that support this, and falls back
to the slow path if the Size fields is zero.
Comparing on a Fedora 38 (kernel 6.2.9-300.fc38.x86_64):
Before/After:
go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
BenchmarkGetTotalUsedFds 57264 18595 ns/op 408 B/op 10 allocs/op
BenchmarkGetTotalUsedFds 370392 3271 ns/op 40 B/op 3 allocs/op
Note that the slow path has 1 more file-descriptor, due to the open
file-handle for /proc/<pid>/fd during the calculation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec79d0fc05)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use File.Readdirnames instead of os.ReadDir, as we're only interested in
the number of files, and results don't have to be sorted.
Before:
BenchmarkGetTotalUsedFds-5 149272 7896 ns/op 945 B/op 20 allocs/op
After:
BenchmarkGetTotalUsedFds-5 153517 7644 ns/op 408 B/op 10 allocs/op
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit eaa9494b71)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 8d56108ffb moved this function from
the generic (no build-tags) fileutils.go to a unix file, adding "freebsd"
to the build-tags.
This likely was a wrong assumption (as other files had freebsd build-tags).
FreeBSD's procfs does not mention `/proc/<pid>/fd` in the manpage, and
we don't test FreeBSD in CI, so let's drop it, and make this a Linux-only
file.
While updating also dropping the import-tag, as we're planning to move
this file internal to the daemon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 252e94f499)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
CI failed sometimes if no daemon.json was present:
Run sudo rm /etc/docker/daemon.json
sudo rm /etc/docker/daemon.json
sudo service docker restart
docker version
docker info
shell: /usr/bin/bash -e {0}
env:
DESTDIR: ./build
BUILDKIT_REPO: moby/buildkit
BUILDKIT_TEST_DISABLE_FEATURES: cache_backend_azblob,cache_backend_s3,merge_diff
BUILDKIT_REF: 798ad6b0ce9f2fe86dfb2b0277e6770d0b545871
rm: cannot remove '/etc/docker/daemon.json': No such file or directory
Error: Process completed with exit code 1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 264dbad43a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 647c2a6cdd)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I think this may be missing a sudo (as all other operations do use
sudo to access daemon.json);
Run if [ ! -e /etc/docker/daemon.json ]; then
if [ ! -e /etc/docker/daemon.json ]; then
echo '{}' | tee /etc/docker/daemon.json >/dev/null
fi
DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null
sudo service docker restart
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
GO_VERSION: 1.20.5
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
ITG_CLI_MATRIX_SIZE: 6
DOCKER_EXPERIMENTAL: 1
DOCKER_GRAPHDRIVER: overlay2
tee: /etc/docker/daemon.json: Permission denied
Error: Process completed with exit code 1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d8bc5828cd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Dockerfile in this repository performs many stages in parallel. If any of
those stages fails to build (which could be due to networking congestion),
other stages are also (forcibly?) terminated, which can cause an unclean
shutdown.
In some case, this can cause `git` to be terminated, leaving a `.lock` file
behind in the cache mount. Retrying the build now will fail, and the only
workaround is to clean the build-cache (which causes many stages to be
built again, potentially triggering the problem again).
> [dockercli-integration 3/3] RUN --mount=type=cache,id=dockercli-integration-git-linux/arm64/v8,target=./.git --mount=type=cache,target=/root/.cache/go-build,id=dockercli-integration-build-linux/arm64/v8 /download-or-build-cli.sh v17.06.2-ce https://github.com/docker/cli.git /build:
#0 1.575 fatal: Unable to create '/go/src/github.com/docker/cli/.git/shallow.lock': File exists.
#0 1.575
#0 1.575 Another git process seems to be running in this repository, e.g.
#0 1.575 an editor opened by 'git commit'. Please make sure all processes
#0 1.575 are terminated then try again. If it still fails, a git process
#0 1.575 may have crashed in this repository earlier:
#0 1.575 remove the file manually to continue.
This patch:
- Updates the Dockerfile to remove `.lock` files (`shallow.lock`, `index.lock`)
that may have been left behind from previous builds. I put this code in the
Dockerfile itself (not the script), as the script may be used in other
situations outside of the Dockerfile (for which we cannot guarantee no other
git session is active).
- Adds a `docker --version` step to the stage; this is mostly to verify the
build was successful (and to be consistent with other stages).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9f6dbbc7ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ideally, this should actually do a lookup across images that have no parent, but I wasn't 100% sure how to accomplish that so I opted for the smaller change of having `FROM scratch` builds not be cached for now.
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit 1741771b67)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I'm considering deprecating the "Key()" utility, as it was only
used in tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0215a62d5b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Added code to correctly retrieve and convert the Topology from the gRPC
Swarm Node.
Signed-off-by: Drew Erny <derny@mirantis.com>
(cherry picked from commit cdb1293eea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 4bafaa00aa, if the daemon was
killed while a container was running and the container shim is killed
before the daemon is restarted, such as if the host system is
hard-rebooted, the daemon would restore the container to the stopped
state and set the exit code to 255. The aforementioned commit introduced
a regression where the container's exit code would instead be set to 0.
Fix the regression so that the exit code is once against set to 255 on
restore.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 165dfd6c3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If an exec fails to start in such a way that containerd publishes an
exit event for it, daemon.ProcessEvent will race
daemon.ContainerExecStart in handling the failure. This race has been a
long-standing bug, which was mostly harmless until
4bafaa00aa. After that change, the daemon
would dereference a nil pointer and crash if ProcessEvent won the race.
Restore the status quo buggy behaviour by adding a check to skip the
dereference if execConfig.Process is nil.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 3b28a24e97)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use `WithFields()` instead of chaining multiple `WithField()` calls.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit de363f1404)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.
To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 21e50b89c9)
Audit the OCI spec options used for Linux containers to ensure they are
less order-dependent. Ensure they don't assume that any pointer fields
are non-nil and that they don't unintentionally clobber mutations to the
spec applied by other options.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 8a094fe609)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Many of the fields in LinuxResources struct are pointers to scalars for
some reason, presumably to differentiate between set-to-zero and unset
when unmarshaling from JSON, despite zero being outside the acceptable
range for the corresponding kernel tunables. When creating the OCI spec
for a container, the daemon sets the container's OCI spec CPUShares and
BlkioWeight parameters to zero when the corresponding Docker container
configuration values are zero, signifying unset, despite the minimum
acceptable value for CPUShares being two, and BlkioWeight ten. This has
gone unnoticed as runC does not distingiush set-to-zero from unset as it
also uses zero internally to represent unset for those fields. However,
kata-containers v3.2.0-alpha.3 tries to apply the explicit-zero resource
parameters to the container, exactly as instructed, and fails loudly.
The OCI runtime-spec is silent on how the runtime should handle the case
when those parameters are explicitly set to out-of-range values and
kata's behaviour is not unreasonable, so the daemon must therefore be in
the wrong.
Translate unset values in the Docker container's resources HostConfig to
omit the corresponding fields in the container's OCI spec when starting
and updating a container in order to maximize compatibility with
runtimes.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit dea870f4ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch to using t.TempDir() instead of rolling our own.
Clean up mounts leaked by the tests as otherwise the tests fail due to
the leaked mounts because unlike the old cleanup code, t.TempDir()
cleanup does not ignore errors from os.RemoveAll.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9ff169ccf4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
daemon.generateNewName() already reserves the generated name, but its name
did not indicate it did. The daemon.registerName() assumed that the generated
name still had to be reserved, which could mean it would try to reserve the
same name again.
This patch renames daemon.generateNewName to daemon.generateAndReserveName
to make it clearer what it does, and updates registerName() to return early
if it successfully generated (and registered) the container name.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3ba67ee214)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 90de570cfa passed through the request
context to daemon.ContainerStop(). As a result, cancelling the context would
cancel the "graceful" stop of the container, and would proceed with forcefully
killing the container.
This patch partially reverts the changes from 90de570cfa
and breaks the context to prevent cancelling the context from cancelling the stop.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fc94ed0a86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some snapshotters (like overlayfs or zfs) can't mount the same
directories twice. For example if the same directroy is used as an upper
directory in two mounts the kernel will output this warning:
overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
And indeed accessing the files from both mounts will result in an "No
such file or directory" error.
This change introduces reference counts for the mounts, if a directory
is already mounted the mount interface will only increment the mount
counter and return the mount target effectively making sure that the
filesystem doesn't end up in an undefined behavior.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 32d58144fd)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.
However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;
Loaded image: emptyfs:latest
Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
INFO: Building docker-sdk-python3:5.0.3...
tests/Dockerfile:6
--------------------
5 | ARG APT_MIRROR
6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
7 | >>> && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
8 |
--------------------
ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2
This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 19d860fa9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.20.5 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, the
runtime, and the crypto/rsa, net, and os packages. See the Go 1.20.5 milestone
on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.20.4...go1.20.5
These minor releases include 3 security fixes following the security policy:
- cmd/go: cgo code injection
The go command may generate unexpected code at build time when using cgo. This
may result in unexpected behavior when running a go program which uses cgo.
This may occur when running an untrusted module which contains directories with
newline characters in their names. Modules which are retrieved using the go command,
i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
GO111MODULE=off, may be affected).
Thanks to Juho Nurminen of Mattermost for reporting this issue.
This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.
- runtime: unexpected behavior of setuid/setgid binaries
The Go runtime didn't act any differently when a binary had the setuid/setgid
bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
I/O file descriptors closed, opening any files could result in unexpected
content being read/written with elevated prilieges. Similarly if a setuid/setgid
program was terminated, either via panic or signal, it could leak the contents
of its registers.
Thanks to Vincent Dehors from Synacktiv for reporting this issue.
This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.
- cmd/go: improper sanitization of LDFLAGS
The go command may execute arbitrary code at build time when using cgo. This may
occur when running "go get" on a malicious module, or when running any other
command which builds untrusted code. This is can by triggered by linker flags,
specified via a "#cgo LDFLAGS" directive.
Thanks to Juho Nurminen of Mattermost for reporting this issue.
This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 98a44bb18e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Fix timeouts from very long raft messages
- fix: code optimization
- update dependencies
full diff: 75e92ce14f...01bb7a4139
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06aaf87aab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Avoids invalidation of dev-systemd-true and dev-base when changing the
CLI version/repository.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 0f9c8e684a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't show `error: No such remote: 'origin'` error when building for the
first time and the cached git repository doesn't a remote yet.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5433b88e2d)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Installs the buildx cli plugin in the container shell by default.
Previously user had to manually download the buildx binary to use
buildkit.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 49f76a34b5)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Use separate cli for integration-cli to allow use newer CLI for
interactive dev shell usage.
Both versions can be overriden with DOCKERCLI_VERSION or
DOCKERCLI_INTEGRATION_VERSION. Binary is downloaded from
download.docker.com if it's available, otherwise it's built from the
source.
For backwards compatibility DOCKER_CLI_PATH overrides BOTH clis.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 17c99f7164)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The default implementation of the containerd.Image interface provided by
the containerd operates on the parent index/manifest list of the image
and the platform matcher.
This isn't convenient when a specific manifest is already known and it's
redundant to search the whole index for a manifest that matches the
given platform matcher. It can also result in a different manifest
picked up than expected when multiple manifests with the same platform
are present.
This introduces a walkImageManifests which walks the provided image and
calls a handler with a ImageManifest, which is a simple wrapper that
implements containerd.Image interfaces and performs all containerd.Image
operations against a platform specific manifest instead of the root
manifest list/index.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fabc1d5bef)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
I missed the most important COPY in 637ca59375
Copying the source code into the dev-container does not depend on the parent
layers, so can use the --link option as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ff2342154b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With this patch, the user-agent has information about the containerd-client
version and the storage-driver that's used when using the containerd-integration;
time="2023-06-01T11:27:07.959822887Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=53590f34-096a-4fd1-9c58-d3b8eb7e5092 service=registry version=2.8.2
...
172.18.0.1 - - [01/Jun/2023:11:30:12 +0000] "HEAD /v2/multifoo/blobs/sha256:c7ec7661263e5e597156f2281d97b160b91af56fa1fd2cc045061c7adac4babd HTTP/1.1" 404 157 "" "docker/dev go/go1.20.4 git-commit/8d67d0c1a8 kernel/5.15.49-linuxkit-pr os/linux arch/arm64 containerd-client/1.6.21+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d099e47e00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Allow additional metadata to be passed as part of the generated User-Agent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a6da1480b5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was not really "inserting" anything, just formatting and appending.
Simplify this by changing this in to a `getUpstreamUserAgent()` function
which returns the upstream User-Agent (if any) into a `UpstreamClient()`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9a1f2e6d7c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a const for the characters to escape, instead of implementing
this as a generic escaping function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ff40d2d787)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switching snapshotter implementations would result in an error when
preparing a snapshot, check that the image is indeed unpacked for the
current snapshot before trying to prepare a snapshot.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit ed32f5e241)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Build-cache for the build-stages themselves are already invalidated if the
base-images they're using is updated, and the COPY operations don't depend
on previous steps (as there's no overlap between artifacts copied).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 637ca59375)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this, the client would report itself as containerd, and the containerd
version from the containerd go module:
time="2023-06-01T09:43:21.907359755Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=67b89d83-eac0-4f85-b36b-b1b18e80bde1 service=registry version=2.8.2
...
172.18.0.1 - - [01/Jun/2023:09:43:33 +0000] "HEAD /v2/multifoo/blobs/sha256:cb269d7c0c1ca22fb5a70342c3ed2196c57a825f94b3f0e5ce3aa8c55baee829 HTTP/1.1" 404 157 "" "containerd/1.6.21+unknown"
With this patch, the user-agent has the docker daemon information;
time="2023-06-01T11:27:07.959822887Z" level=info msg="listening on [::]:5000" go.version=go1.19.9 instance.id=53590f34-096a-4fd1-9c58-d3b8eb7e5092 service=registry version=2.8.2
...
172.18.0.1 - - [01/Jun/2023:11:27:20 +0000] "HEAD /v2/multifoo/blobs/sha256:c7ec7661263e5e597156f2281d97b160b91af56fa1fd2cc045061c7adac4babd HTTP/1.1" 404 157 "" "docker/dev go/go1.20.4 git-commit/8d67d0c1a8 kernel/5.15.49-linuxkit-pr os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 66137ae429)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Resolver.setupIPTable() checks whether it needs to flush or create the
user chains used for NATing container DNS requests by testing for the
existence of the rules which jump to said user chains. Unfortunately it
does so using the IPTable.RawCombinedOutputNative() method, which
returns a non-nil error if the iptables command returns any output even
if the command exits with a zero status code. While that is fine with
iptables-legacy as it prints no output if the rule exists, iptables-nft
v1.8.7 prints some information about the rule. Consequently,
Resolver.setupIPTable() would incorrectly think that the rule does not
exist during container restore and attempt to create it. This happened
work work by coincidence before 8f5a9a741b
because the failure to create the already-existing table would be
ignored and the new NAT rules would be inserted before the stale rules
left in the table from when the container was last started/restored. Now
that failing to create the table is treated as a fatal error, the
incompatibility with iptables-nft is no longer hidden.
Switch to using IPTable.ExistsNative() to test for the existence of the
jump rules as it correctly only checks the iptables command's exit
status without regard for whether it outputs anything.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 1178319313)
Signed-off-by: Cory Snider <csnider@mirantis.com>
The method to restore a network namespace takes a collection of
interfaces to restore with the options to apply. The interface names are
structured data, tuples of (SrcName, DstPrefix) but for whatever reason
are being passed into Restore() serialized to strings. A refactor,
f0be4d126d, accidentally broke the
serialization by dropping the delimiter. Rather than fix the
serialization and leave the time-bomb for someone else to trip over,
pass the interface names as structured data.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 50eb2d2782)
Signed-off-by: Cory Snider <csnider@mirantis.com>
In cases where an exec start failed the exec process will be nil even
though the channel to signal that the exec started was closed.
Ideally ExecConfig would get a nice refactor to handle this case better
(ie. it's not started so don't close that channel).
This is a minimal fix to prevent NPE. Luckilly this would only get
called by a client and only the http request goroutine gets the panic
(http lib recovers the panic).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 487ea81316)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While the VXLAN interface and the iptables rules to mark outgoing VXLAN
packets for encryption are configured to use the Swarm data path port,
the XFRM policies for actually applying the encryption are hardcoded to
match packets with destination port 4789/udp. Consequently, encrypted
overlay networks do not pass traffic when the Swarm is configured with
any other data path port: encryption is not applied to the outgoing
VXLAN packets and the destination host drops the received cleartext
packets. Use the configured data path port instead of hardcoding port
4789 in the XFRM policies.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9a692a3802)
Signed-off-by: Cory Snider <csnider@mirantis.com>
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.
As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 37d4b0bee9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It turns out that the unnecessary serialization removed in
b75246202a happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit fb7ec1555c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Feature flags are one of the configuration items which can be reloaded
without restarting the daemon. Whether the daemon uses the containerd
snapshotter service or the legacy graph drivers is controlled by a
feature flag. However, much of the code which checks the snapshotter
feature flag assumes that the flag cannot change at runtime. Make it so
that the snapshotter setting can only be changed by restarting the
daemon, even if the flag state changes after a live configuration
reload.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9b9c5242eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.
From the go1.19 release notes: https://go.dev/doc/go1.19#net
> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.
Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 53d1b12bc0)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Docker with containerd integration emits "Exists" progress action when a
layer of the currently pulled image already exists. This is different
from the non-c8d Docker which emits "Already exists".
This makes both implementations consistent by emitting backwards
compatible "Already exists" action.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a7bc65fbd8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
TestProxyNXDOMAIN has proven to be susceptible to failing as a
consequence of unlocked threads being set to the wrong network
namespace. As the failure mode looks a lot like a bug in the test
itself, it seems prudent to add a check for mismatched namespaces to the
test so we will know for next time that the root cause lies elsewhere.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 871cf72363)
Signed-off-by: Cory Snider <csnider@mirantis.com>
osl.setIPv6 mistakenly captured the calling goroutine's thread's network
namespace instead of the network namespace of the thread getting its
namespace temporarily changed. As this function appears to only be
called from contexts in the process's initial network namespace, this
mistake would be of little consequence at runtime. The libnetwork unit
tests, on the other hand, unshare network namespaces so as not to
interfere with each other or the host's network namespace. But due to
this bug, the isolation backfires and the network namespace of
goroutines used by a test which are expected to be in the initial
network namespace can randomly become the isolated network namespace of
some other test. Symptoms include a loopback network server running in
one goroutine being inexplicably and randomly being unreachable by a
client in another goroutine.
Capture the original network namespace of the thread from the thread to
be tampered with, after locking the goroutine to the thread.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 6d79864135)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Swapping out the global logger on the fly is causing tests to flake out
by logging to a test's log output after the test function has returned.
Refactor Resolver to use a dependency-injected logger and the resolver
unit tests to inject a private logger instance into the Resolver under
test.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d4f3858a40)
Signed-off-by: Cory Snider <csnider@mirantis.com>
tstwriter mocks the server-side connection between the resolver and the
container, not the resolver and the external DNS server, so returning
the external DNS server's address as w.LocalAddr() is technically
incorrect and misleading. Only the protocols need to match as the
resolver uses the client's choice of protocol to determine which
protocol to use when forwarding the query to the external DNS server.
While this change has no material impact on the tests, it makes the
tests slightly more comprehensible for the next person.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0cc6e445d7)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Our resolver is just a forwarder for external DNS so it should act like
it. Unless it's a server failure or refusal, take the response at face
value and forward it along to the client. RFC 8020 is only applicable to
caching recursive name servers and our resolver is neither caching nor
recursive.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 41356227f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 967c7bc5d3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
This fixes a bug where, if a user pulls an image with a tag != `latest` and
a specific platform, we return an NotFound error for the wrong (`latest`) tag.
see: https://github.com/moby/moby/issues/45558
This bug was introduced in 779a5b3029
in the changes to `daemon/images/image_pull.go`, when we started returning the error from the call to
`GetImage` after the pull. We do this call, if pulling with a specified platform, to check if the platform
of the pulled image matches the requested platform (for cases with single-arch images).
However, when we call `GetImage` we're not passing the image tag, only name, so `GetImage` assumes `latest`
which breaks when the user has requested a different tag, since there might not be such an image in the store.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit f450ea64e6)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
When the `ServerAddress` in the `AuthConfig` provided by the client is
empty, default to the default registry (registry-1.docker.io).
This makes the behaviour the same as with the containerd image store
integration disabled.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2ad499f93e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)
Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 4e0319c878)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ports over all the previous image delete logic, such as:
- Introduce `prune` and `force` flags
- Introduce the concept of hard and soft image delete conflics, which represent:
- image referenced in multiple tags (soft conflict)
- image being used by a stopped container (soft conflict)
- image being used by a running container (hard conflict)
- Implement delete logic such as:
- if deleting by reference, and there are other references to the same image, just
delete the passed reference
- if deleting by reference, and there is only 1 reference and the image is being used
by a running container, throw an error if !force, or delete the reference and create
a dangling reference otherwise
- if deleting by imageID, and force is true, remove all tags (otherwise soft conflict)
- if imageID, check if stopped container is using the image (soft conflict), and
delete anyway if force
- if imageID was passed in, check if running container is using the image (hard conflict)
- if `prune` is true, and the image being deleted has dangling parents, remove them
This commit also implements logic to get image parents in c8d by comparing shared layers.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit cad97135b3)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Our templates no longer contain version-specific rules, so this function
is no longer used. This patch deprecates it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e3e715666f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 7008a51449 removed version-conditional
rules from the template, so we no longer need the apparmor_parser Version.
This patch removes the call to `aaparser.GetVersion()`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ecaab085db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 2e19a4d56b removed all other version-
conditional statements from the AppArmor template, but left this one in place.
These conditions were added in 8cf89245f5
to account for old versions of debian/ubuntu (apparmor_parser < 2.9)
that lacked some options;
> This allows us to use the apparmor profile we have in contrib/apparmor/
> and solves the problems where certain functions are not apparent on older
> versions of apparmor_parser on debian/ubuntu.
Those patches were from 2015/2016, and all currently supported distro
versions should now have more current versions than that. Looking at the
oldest supported versions;
Ubuntu 18.04 "Bionic":
apparmor_parser --version
AppArmor parser version 2.12
Copyright (C) 1999-2008 Novell Inc.
Copyright 2009-2012 Canonical Ltd.
Debian 10 "Buster"
apparmor_parser --version
AppArmor parser version 2.13.2
Copyright (C) 1999-2008 Novell Inc.
Copyright 2009-2018 Canonical Ltd.
This patch removes the remaining conditionals.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f445ee1e6c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `execDuration` field to the event attributes map. This is useful for tracking how long the container ran.
Signed-off-by: Dorin Geman <dorin.geman@docker.com>
(cherry picked from commit 2ad37e1832)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 1261fe69a3 deprecated the VirtualSize
field, but forgot to mention that it's also included in the /system/df
endpoint.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fdc7a78652)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:31:44 +02:00
485 changed files with 7908 additions and 4354 deletions
returnerrors.WithStack(notAvailableError("This node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again."))
returnerrors.WithStack(notAvailableError(`This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.`))
}
ifst.swarmNode.Manager()!=nil{
returnerrors.WithStack(notAvailableError("This node is not a swarm manager. Manager is being prepared or has trouble connecting to the cluster."))
errNoSwarmnotAvailableError="This node is not part of a swarm"
// errSwarmExists is returned on initialize or join request for a cluster that has already been activated
errSwarmExistsnotAvailableError="This node is already part of a swarm. Use \"docker swarm leave\" to leave this swarm and join another one."
errSwarmExistsnotAvailableError=`This node is already part of a swarm. Use "docker swarm leave" to leave this swarm and join another one.`
// errSwarmJoinTimeoutReached is returned when cluster join could not complete before timeout was reached.
errSwarmJoinTimeoutReachednotAvailableError="Timeout was reached before node joined. The attempt to join the swarm will continue in the background. Use the \"docker info\" command to see the current swarm status of your node."
errSwarmJoinTimeoutReachednotAvailableError=`Timeout was reached before node joined. The attempt to join the swarm will continue in the background. Use the "docker info" command to see the current swarm status of your node.`
// errSwarmLocked is returned if the swarm is encrypted and needs a key to unlock it.
errSwarmLockednotAvailableError="Swarm is encrypted and needs to be unlocked before it can be used. Please use \"docker swarm unlock\" to unlock it."
errSwarmLockednotAvailableError=`Swarm is encrypted and needs to be unlocked before it can be used. Please use "docker swarm unlock" to unlock it.`
// errSwarmCertificatesExpired is returned if docker was not started for the whole validity period and they had no chance to renew automatically.
errSwarmCertificatesExpirednotAvailableError="Swarm certificates have expired. To replace them, leave the swarm and join again."
logrus.WithError(err).WithField("image",name).Warn("Ignoring error about platform mismatch where the manifest list points to an image whose configuration does not match the platform in the manifest.")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
