The latest libkv uses a different etcd library. Unfortunately
that library uses some funky import paths, so I've added a new cleanup
routine for our vendor scripts to be able to normalize the imports
to be consistent with how imports work in this tree.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
introduced --subnet, --ip-range and --gateway options in docker network
command. Also, user can allocate driver specific ip-address if any using
the --aux-address option.
Supports multiple subnets per network and also sharing ip range
across networks if the network-driver and ipam-driver supports it.
Example, Bridge driver doesnt support sharing same ip range across
networks.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
* Made use of IPAM driver primitives for legacy IP configurations
* Replaced custom Generics with backend labels
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Tags and digests are kept in the same storage. We want to make sure that they are completely separated - tags are something users set and digests can only be set by pull-by-digest code path.
Reverts #14664
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
It is possible that network files do not exist, especially in the case
of `--net=host` where a host OS (like CoreOS) does not use certain
standard network files. This patch verifies that the source file of a
network mount point exists before adding it to the list of mount points
for bind mounting from the container's metadata directory.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
this allows jsonfile logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`.
Extra attributes are saved into `attrs` attributes for each log data.
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows journald logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows fluentd logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows gelf logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Additional log field will be prefixed with `_` as per gelf protocol
https://www.graylog.org/resources/gelf/
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Add a unit test for validateManifest which ensures extra data can't be
injected by adding data to the JSON object outside the payload area.
This also removes validation of legacy signatures at pull time. This
starts the path of deprecating legacy signatures, whose presence in the
very JSON document they attempt to sign is problematic. These
signatures were only checked for official images, and since they only
caused a weakly-worded message to be printed, removing the verification
should not cause impact.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Generate a hash chain involving the image configuration, layer digests,
and parent image hashes. Use the digests to compute IDs for each image
in a manifest, instead of using the remotely specified IDs.
To avoid breaking users' caches, check for images already in the graph
under old IDs, and avoid repulling an image if the version on disk under
the legacy ID ends up with the same digest that was computed from the
manifest for that image.
When a calculated ID already exists in the graph but can't be verified,
continue trying SHA256(digest) until a suitable ID is found.
"save" and "load" are not changed to use a similar scheme. "load" will
preserve the IDs present in the tar file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Docker 1.7 got support for specifying the CFS scheduler's CPU quota and
period. The work appears to have started around commit
dcc50e1d59. This change updates the API
documentation to reflect the existing support.
Signed-off-by: Victor Costan <costan@gmail.com>
All the go-lint work forced any existing "Uid" -> "UID", but seems to
not have the same rules for Gid, so stat package has calls UID() and
Gid().
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The first param on opts.ParseHost() wasn't being used for anything.
Once we get rid of that param we can then also clean-up some code
that calls ParseHost() because the param that was passed in wasn't
being used for anything else.
Signed-off-by: Doug Davis <dug@us.ibm.com>
`docker network` is the second command with subcommands.
This refactoring pulls out parsing and processing of subcommands
from `docker volume` completion and thus makes its logic available
for other commands.
Also enables `__docker_pos_first_nonflag` for subcommand completion.
Signed-off-by: Harald Albers <github@albersweb.de>
GitHub flavored markdown is now supported for links and images. Also, ran LinkChecker and FileResolver. Yay!
Fixes from Spider check
Output for docker/docker now goes into engine directory
Signed-off-by: Mary Anthony <mary@docker.com>
This adds the capability to turn on user namespace support when using an
experimental build Docker daemon binary using the `--userns-remap` flag.
Also documentation is added to the experimental docs.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Refactor so that the Host flag validation doesn't destroy the user's input,
and then post process the flags when we know the TLS options
Signed-off-by: Sven Dowideit <SvenDowideit@home.org.au>
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The `pkg/idtools` package supports the creation of user(s) for
retrieving /etc/sub{u,g}id ranges and creation of the UID/GID mappings
provided to clone() to add support for user namespaces in Docker.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
- add daemon methods Authenticate & SearchRegistryForImages
- use new methods instead of directly accessing RegistryService
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
add needed clarity for
1) using STDIN to pass build context
2) --cpu-shares flag use
also a few typos
Signed-off-by: Sally O'Malley <somalley@redhat.com>
When `-s` is not specified, there is no need to ask if there is a plugin
with the specified name.
This speeds up unit tests dramatically since they don't need to wait the
timeout period for each call to `graphdriver.New`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
If I have some unsupported sysinfo, it's warning on daemon
side every time I use `docker info`, it seems unnecessay and
annoying to me, let's keep it quiet.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Pick up name regexp change in distribution to allow matching of hostnames as a valid component of a repository.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
A recent change made `graphdriver.NaiveDiffDriver` from a function to a
struct. New function is `graphdriver.NewNaiveDiffDriver.
The graphdriver plugin PR was created (and tests run) before this change
was introduced causing the failure to not be seen until after merge.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- use daemon member directly rather than through a function call
- create GetImage method for use external to deamon
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Fixes a race when starting a container when there is an error, the stdio
streams are not always written.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
There is no need to call `os.Stat` on the driver filesystem path of a
container as `os.RemoveAll` already handles (properly) the case where
the path no longer exists.
Given the results of the stat() were not even being used, there is no
value in erroring out because of the stat call failure, and worse, it
prevents daemon cleanup of containers in "Dead" state unless you re-create
directories that were already removed via a manual cleanup after a
failure. This brings removal in overlay in line with aufs/devicemapper
drivers which don't error out if the filesystem path no longer exists.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The registry doesn't have containers, only images.
Also updated the Docker Hub link to the new location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This leverages recent additions to libkv enabling client
authentication via TLS so the discovery back-end can be locked
down with mutual TLS. Example usage:
docker daemon [other args] \
--cluster-advertise 192.168.122.168:2376 \
--cluster-store etcd://192.168.122.168:2379 \
--cluster-store-opt kv.cacertfile=/path/to/ca.pem \
--cluster-store-opt kv.certfile=/path/to/cert.pem \
--cluster-store-opt kv.keyfile=/path/to/key.pem
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
- refactor to make it easier to split the api in the future
- additional tests for non existent container case
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
- create pass through in daemon for access to functions on daemon
member
- import image
push image
export image and corrections
lookup image & comments
load image
list images
image history & comments
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
* fixed weird logic with "*"
* return error if fdNum is failed to parse
* check if listener at offset is nil
* close unused listeners
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Now we're start to serve early, but all Accept calls are intercepted by
listenbuffer or systemd socket.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Right now we check for the existence of device but don't make sure it is
a thin pool device. We assume it is a thin pool device and call poolStatus()
on the device which returns an error EOF. And that error does not tell
anything.
So before we reach the stage of calling poolStatus() make sure we are working
with a thin pool device otherwise error out.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Before, typing `docker volume` with no args would forward to the handler
for `docker volume ls`, except the flags for the `ls` subcommand were
not supported.
Instead just print the cmd usage.
This makes the behavior of the `docker volume` subcommand behave exactly
like the `docker network` subcommand.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
I disagree with #14546 that pushed the help text past 80 chars.
Aside from it now making the help text look ugly on 80 char displays,
which I use, one thing I like about the previous limitation is that it
forced us to keep our options down to more reasonable phrases/words.
For example, I think
` --disable-content-trust=true`
could have been:
` --disable-trust=true`
or even:
` --disable-ctrust=true`
But regardless, let's at least make the comments match what the code does.
Signed-off-by: Doug Davis <dug@us.ibm.com>
* Exiting experimental network UX
* removed experimental service UX
* integrated with the new network remote API
Signed-off-by: Madhu Venugopal <madhu@docker.com>
* Moving Network Remote APIs out of experimental
* --net can now accept user created networks using network drivers/plugins
* Removed the experimental services concept and --default-network option
* Neccessary backend changes to accomodate multiple networks per container
* Integration Tests
Signed-off-by: David Calavera <david.calavera@gmail.com>
Signed-off-by: Madhu Venugopal <madhu@docker.com>
progressreader.Broadcaster becomes broadcaster.Buffered and
broadcastwriter.Writer becomes broadcaster.Unbuffered.
The package broadcastwriter is thus renamed to broadcaster.
Signed-off-by: Tibor Vass <tibor@docker.com>
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.
It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.
builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Start a goroutine which runs every 30 seconds and if there are deferred
deleted devices, it tries to clean those up.
Also it moves the call to cleanupDeletedDevices() into goroutine and
moves the locking completely inside the function. Now function does not
assume that device lock is held at the time of entry.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Finally here is the patch to implement deferred deletion functionality.
Deferred deleted devices are marked as "Deleted" in device meta file.
First we try to delete the device and only if deletion fails and user has
enabled deferred deletion, device is marked for deferred deletion.
When docker starts up again, we go through list of deleted devices and
try to delete these again.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Provide a command line option dm.use_deferred_deletion to enable deferred
device deletion feature. By default feature will be turned off.
Not sure if there is much value in deferred deletion being turned on
without deferred removal being turned on. So for now, this feature can
be enabled only if deferred removal is on.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Currently during startup we walk through all the device files and read
their device ID and mark in a bitmap that device id is used.
We are anyway going through all device files. So we can as well load all
that data into device hash map. This will save us little time when
container is actually launched later.
Also this will help with later patches where cleanup deferred device
wants to go through all the devices and see which have been marked for
deletion and delete these.
So re-organize the code a bit.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Simplify setupBaseImage() even further. Move some more code in a separate
function. Pure code reorganization. No functionality change.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Move thin pool related checks in a separate function. Pure code reorganization.
Makes reading code easier.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
This moves base device creation function in a separate function. Pure
code reorganization. Makes reading code little easier.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
W/o this PR the logs would show:
```
ERRO[0000] API listen on [::]:2375
ERRO[0000] API listen on /var/run/docker.sock
```
which is misleading since these aren't "errors" they're just informational.
Signed-off-by: Doug Davis <dug@us.ibm.com>
It prevents occupying of those resources (ports, unix-sockets) by
containers.
Also fixed false-positive test for that case.
Fix#15912
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
This patch does three things. Following are the descriptions.
===
Create a separate function for delete transactions so that parent function
is little smaller.
Also close transaction if an error happens.
===
When docker is being shutdown, save deviceset metadata first before
trying to remove the devices. Generally caller gives only 10 seconds
for shutdown to complete and then kills it after that. So if some device
is busy, we will wait 20 seconds for it removal and never be able to save
metadata. So first save metadata and then deal with device removal.
===
Move issue discard operation in a separate function. This makes reading code
little easier.
Also don't issue discards if device is still open. That means devices is
still probably being used and issuing discards is not a good idea.
This is especially true in case of deferred deletion. We want to issue
discards when device is not open. At that time device can be deleted too.
Otherwise we will issue discards and deletion will actually fail. Later
we will try deletion again and issue discards again and deletion will
fail again as device is open and busy.
So this will ensure that discards are issued once when device is not open
and it can actually be deleted.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
- missing help option in `docs/reference/commandline/*.md` (some files
have it, the other I fixed didn't)
- missing `[OPTIONS]` in Usage description
- missing options
- formatting
- start/stop idempotence
Signed-off-by: Antonio Murdaca <amurdaca@redhat.com>
Search terms shouldn't be restricted to only full valid repository
names. It should be perfectly valid to search using a part of a name,
even if it ends with a period, dash or underscore.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Fixes an issue where a `Dead` container has no names so the API returns
`null` instead of an empty array.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This commit brings in end to end integration of Docker Discovery with
libnetwork multi-host networking.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Exec start was sending HTTP 500 for every error.
Fixed an error where pausing a container and then calling exec start
caused the daemon to freeze.
Updated API docs which incorrectly showed that a successful exec start
was an HTTP 201, in reality it is HTTP 200.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Right now we seem to have 3 locks.
- devinfo.lock
This is a per device lock
- metaData.devicesLock
This is supposedely protecting map of devices.
- Global DeviceSet lock
This is protecting map of devices as well as serializing calls to libdevmapper.
Semantics of per devices lock and global deviceset lock seem to be very clear.
Even ordering between these two locks has been defined properly.
What is not clear is the need and ordering of metaData.devicesLock. Looks like
this lock is not necessary and global DeviceSet lock should be used to
protect map of devices as it is part of DeviceSet.
This patchset gets rid of metaData.devicesLock and instead uses DeviceSet
lock to protect map of devices.
Also at couple of places during initialization takes devices.Lock(). That
is not strictly necessary as there is supposed to be one thread of execution
during initializaiton. Still it makes the code clearer.
I think this makes code more clear and easier to understand and easier to
make further changes.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
maxDeviceID is upper limit on device Id thin pool can support. Right now
we have this check only during startup. It is a good idea to move this
check in loadMetadata so that any time a device file is loaded and if it
is corrupted and device Id is more than maxDevieceID, it will be detected
right then and there.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Use deactivateDevice() instead of removeDevice() directly. This will make
sure for device deletion, deferred removal is used if user has configured
it in. Also this makes reading code litle easier as there is single function
to remove a device and that is deactivateDevice().
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Fixing user namespaces (again) with a vendor update from runc
(specifically, the remount() only if special flags change)
Other changes are very minimal.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
If a device is still mounted at the time of DeleteDevice(), that means
higher layers have not called Put() properly on the device and are trying
to delete it. This is a bug in the code where Get() and Put() have not been
properly paired up. Fail device deletion if it is still mounted.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Exists() and HasDevice() just check if device file exists or not. It does
not say anything about if device is mounted or not. Fix comments.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
device has map (device.Devices), contains valid devices and we skip all
the files which are not device files. transaction metadata file is not
device file. Skip this file when devices files are being read and loaded
into map.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Implement basic interfaces to write custom routers that can be plugged
to the server. Remove server coupling with the daemon.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This passes through the container hostname to HCS, which in Windows Server
2016 TP4 will set the container's hostname in the registry before starting
it. This will be silently ignored by TP3.
Signed-off-by: John Starks <jostarks@microsoft.com>
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.
Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).
This reverts commit de41640435, reversing
changes made to 7daeecd42d.
Signed-off-by: Tibor Vass <tibor@docker.com>
Conflicts:
api/server/container.go
builder/internals.go
daemon/container_unix.go
daemon/create.go
This reverts commit ff92f45be4, reversing
changes made to 80e31df3b6.
Reverting to make the next revert easier.
Signed-off-by: Tibor Vass <tibor@docker.com>
This fixes the case where directory is removed in
aufs and then the same layer is imported to a
different graphdriver.
Currently when you do `rm -rf /foo && mkdir /foo`
in a layer in aufs the files under `foo` would
only be be hidden on aufs.
The problems with this fix:
1) When a new diff is recreated from non-aufs driver
the `opq` files would not be there. This should not
mean layer differences for the user but still
different content in the tar (one would have one
`opq` file, the others would have `.wh.*` for every
file inside that folder). This difference also only
happens if the tar-split file isn’t stored for the
layer.
2) New files that have the filenames before `.wh..wh..opq`
when they are sorted do not get picked up by non-aufs
graphdrivers. Fixing this would require a bigger
refactoring that is planned in the future.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This test is failing once in a while on the CI, because the docker
attach command might be called after the container ends.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Fixes an issue where `VOLUME some_name:/foo` would be parsed as a named
volume, allowing access from the builder to any volume on the host.
This makes sure that named volumes must always be passed in as a bind.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Before this patch libcontainer badly errored out with `invalid
argument` or `numerical result out of range` while trying to write
to cpuset.cpus or cpuset.mems with an invalid value provided.
This patch adds validation to --cpuset-cpus and --cpuset-mems flag along with
validation based on system's available cpus/mems before starting a container.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Add a daemon flag to control this behaviour. Add a warning message when pulling
an image from a v1 registry. The default order of pull is slightly altered
with this changset.
Previously it was:
https v2, https v1, http v2, http v1
now it is:
https v2, http v2, https v1, http v1
Prevent login to v1 registries by explicitly setting the version before ping to
prevent fallback to v1.
Add unit tests for v2 only mode. Create a mock server that can register
handlers for various endpoints. Assert no v1 endpoints are hit with legacy
registries disabled for the following commands: pull, push, build, run and
login. Assert the opposite when legacy registries are not disabled.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Use `pkg/discovery` to provide nodes discovery between daemon instances.
The functionality is driven by two different command-line flags: the
experimental `--cluster-store` (previously `--kv-store`) and
`--cluster-advertise`. It can be used in two ways by interested
components:
1. Externally by calling the `/info` API and examining the cluster store
field. The `pkg/discovery` package can then be used to hit the same
endpoint and watch for appearing or disappearing nodes. That is the
method that will for example be used by Swarm.
2. Internally by using the `Daemon.discoveryWatcher` instance. That is
the method that will for example be used by libnetwork.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Absorb Swarm's discovery package in order to provide a common node
discovery mechanism to be used by both Swarm and networking code.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
* Thanks to the Default gateway service in libnetwork, we dont have to add
containers explicitly to secondary public network. This is handled
automatically regardless of the primary network driver.
* Fixed the URL convention for kv-store to be aligned with the upcoming
changes to discovery URL
* Also, in order to bring consistency between external and internal network
drivers, we moved the driver configs via controller Init.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
- Libnetwork brings in :
* Default Gateway as a service for network drivers
* Persistence for local scoped networks using libkv
* BATS based Multi-host Integration-test infra and end-to-end tests
* libnetwork fixes for zookeeper and etcd backend
- Libkv upgrade brings in :
* boltdb support for local kv persistence
* other general bug fixes
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Avoid creating a global context object that will be used while the daemon is running.
Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.
This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Add 10 seconds timeout when running `sudo service docker stop`. This is
especially needed when running `sudo service docker restart`. Otherwise,
`restart` results in exitstatus 1, because `start` has nothing to do.
Signed-off-by: Ewa Czechowska <ewa@ai-traders.com>
The mount syscall does not handle string flags like "noatime",
we must use bitmasks like MS_NOATIME instead.
pkg/mount.Mount already handles this work.
Signed-off-by: Carl Henrik Lunde <chlunde@ping.uio.no>
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
- Print the mount table as in /proc/self/mountinfo
- Do not exit prematurely when one of the ipc mounts doesn't exist.
- Do not exit prematurely when one of the ipc mounts cannot be unmounted.
- Add a unit test to see if the cleanup really works.
- Use syscall.MNT_DETACH to cleanup mounts after a crash.
- Unmount IPC mounts when the daemon unregisters an old running container.
Signed-off-by: David Calavera <david.calavera@gmail.com>
On boot, the init script is invoked from `/etc/rcN.d/S20docker` (where N
is the runlevel). Consequently, the init script tried to find the
defaults at `/etc/defaults/S20docker` and the binary at
`/usr/bin/S20docker`. This causes the docker daemon to fail at boot with
the error
/usr/bin/S20docker not present or not executable
Starting it manually works because it invokes `/etc/init.d/docker` which
has the correct basename.
Fix this by hardcoding "docker" as the base name.
Signed-off-by: Matthias Rampke <mr@soundcloud.com>
The shell builtin `compopt` is not available on the outdated bash
version 3.2.57 that ships with Mac OS.
It is used in Docker's bash completion to suppress trailing spaces
in advanced completions of hash map options, e.g. `--log-opt`.
If `compopt` is not available, the new behavior is to do nothing,
i.e. the user will have to delete the additional space.
Signed-off-by: Harald Albers <github@albersweb.de>
Once there have '\t' in the "CREATED BY" section, it will possibly
cause a messy output of docker history.
Signed-off-by: Hu Keping <hukeping@huawei.com>
The tests added cover the case when the Writer field returns and error
and, related to that, when the number of written bytes is less than 0.
Signed-off-by: Federico Gimenez <fgimenez@coit.es>
* Update format for 'Note:' to match other pages.
* Add link to Go's RFC3339Nano timestamp information.
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
It defines global middlewares for every request.
This makes the server slightly more composable.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Pull in version e5fea92a6c8a5968bdb8005bf959c6e23113b689
Fixes libnetwork regressions that stopped drivers that set routes from working.
Pulls in libnetwork PRs #546#543
Signed-off-by: Tom Denham <tom@tomdee.co.uk>
- Add unit tests to make sure the functionality is correct.
- Add FilterByDriver to allow filtering volumes by driver, for future
`volume ls` filtering and whatnot.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This comment was wrongly referring to the old job mechanism and it
wasn't clear what it was trying to document.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
If an invalid logger address is provided on daemon start it will
silently fail. As syslog driver is doing, this check should be done on
daemon start and prevent it from starting even in other drivers.
This patch also adds integration tests for this behavior.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
If you don't have cgroup swap memory support, `dockerCmd`'s output in
these tests will be polluted by a warning from the daemon and will fail
the tests.
No need to have memory swap support for these tests to run as it will
be reset to -1 and everything will continue correctly.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
`docker rename foo ''` would result in:
```
usage: docker rename OLD_NAME NEW_NAME
```
which is the old engine's way of return errors - yes that's in the
daemon code. So I fixed that error msg to just be normal.
While doing that I noticed that using an empty string for the
source container name failed but didn't print any error message at all.
This is because we would generate a URL like: ../containers//rename/..
which would cause a 301 redirect to ../containers/rename/..
however the CLI code doesn't actually deal with 301's - it just ignores
them and returns back to the CLI code/caller.
Rather than changing the CLI to deal with 3xx error codes, which would
probably be a good thing to do in a follow-on PR, for this immediate
issue I just added a cli-side check for empty strings for both old and
new names. This way we catch it even before we hit the daemon.
API callers will get a 404, assuming they follow the 301, for the
case of the src being empty, and the new error msg when the destination
is empty - so we should be good now.
Add tests for both cases too.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Fixes#11957Fixes#12319
Also removes check for Darwin when the stdin reader is closed as it
doesn't appear to block any more.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This way provide both Time and TimeNano in the event. For the display of
the JSONMessage, use either, but prefer TimeNano Proving only TimeNano
would break Subscribers that are using the `Time` field, so both are set
for backwards compatibility.
The events logging uses nano formatting, but only provides a Unix()
time, therefor ordering may get lost in the output. Example:
```
2015-09-15T14:18:51.000000000-04:00 ee46febd64ac629f7de9cd8bf58582e6f263d97ff46896adc5b508db804682da: (from busybox) resize
2015-09-15T14:18:51.000000000-04:00 a78c9149b1c0474502a117efaa814541926c2ae6ec3c76607e1c931b84c3a44b: (from busybox) resize
```
By having a field just for Nano time, when set, the marshalling back to
`time.Unix(sec int64, nsec int64)` has zeros exactly where it needs to.
This does not break any existing use of jsonmessage.JSONMessage, but now
allows for use of `UnixNano()` and get event formatting that has
distinguishable order. Example:
```
2015-09-15T15:37:23.810295632-04:00 6adcf8ed9f5f5ec059a915466cd1cde86a18b4a085fc3af405e9cc9fecbbbbaf: (from busybox) resize
2015-09-15T15:37:23.810412202-04:00 6b7c5bfdc3f902096f5a91e628f21bd4b56e32590c5b4b97044aafc005ddcb0d: (from busybox) resize
```
Including tests for TimeNano and updated event API reference doc.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
GET /containers/json route used to reply with and empty array `[]` when no
containers where available. Daemon containers list refactor introduced
this bug by declaring an empty slice istead of initializing it as well
and it was now replying with `null`.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
When use `docker stats` to minitor a running container and then stop it,
there are some fields need to be reset to zero. Otherwise it will keep
displaying the data it received last time.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
cleaning up docker run -v documentation for man and web reference guide
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
- refactor to make it easier to split the api in the future
- addition to check the existing test case and make sure it contains
some expected output
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
To fix an issue with experimental multihost networking.
git hash: 00a92f066e628e4c6d50979c070df377575aad18
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Using @mavenugo's patch for enabling the libcontainer pre-start hook to
be used for network namespace initialization (correcting the conflict
with user namespaces); updated the boolean check to the more generic
SupportsHooks() name, and fixed the hook state function signature.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.
- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.
- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.
Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
With go1.5's concurrency, the use of a goroutine in Log'ing events was
causing the resulting events to not be in order.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
For now docker stats will sum the rxbytes, txbytes, etc. of all
the interfaces.
It is OK for the output of CLI `docker stats` but not good for
the API response, especially when the container is in sereval
subnets.
It's better to leave these origianl data to user.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Volumes are accounted when a container is created.
If the creation fails we should remove the reference from the counter.
Do not log ErrVolumeInUse as an error, having other volume references is
not an error.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This is the first step in converting out static strings into well-defined
error types. This shows just a few examples of it to get a feel for how things
will look. Once we agree on the basic outline we can then work on converting
the rest of the code over.
Signed-off-by: Doug Davis <dug@us.ibm.com>
@noxiouz points out that we don't need to check for a nil result from
C.CString(), since an out-of-memory condition causes a runtime panic
instead.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Notary is capable of acting in offline mode, making use of cache TUF data.
When ping is not successful, notary should still be attempted without error.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Changes include :
* libnetwork support for userns
* driver api change to have 1 interface per endpoint
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Replace comma with column
Remove extra curly brace
Fix typo in REST API documentation
Replace comma with column in documentation.
Signed-off-by: Adrian Oprea <adrian@codesi.nz>
Remove extra curly brace
Remove extra curly brace from remote API docs.
Signed-off-by: Adrian Oprea <adrian@codesi.nz>
This is not a public interface and users are trying to use it.
We only use it to be able to generate the rpc protocol to talk with a plugin.
Signed-off-by: David Calavera <david.calavera@gmail.com>
If a logdriver doesn't register a callback function to validate log
options, it won't be usable. Fix the journald driver by adding a dummy
validator.
Teach the client and the daemon's "logs" logic that the server can also
supply "logs" data via the "journald" driver. Update documentation and
tests that depend on error messages.
Add support for reading log data from the systemd journal to the
journald log driver. The internal logic uses a goroutine to scan the
journal for matching entries after any specified cutoff time, formats
the messages from those entries as JSONLog messages, and stuffs the
results down a pipe whose reading end we hand back to the caller.
If we are missing any of the 'linux', 'cgo', or 'journald' build tags,
however, we don't implement a reader, so the 'logs' endpoint will still
return an error.
Make the necessary changes to the build setup to ensure that support for
reading container logs from the systemd journal is built.
Rename the Jmap member of the journald logdriver's struct to "vars" to
make it non-public, and to make it easier to tell that it's just there
to hold additional variable values that we want journald to record along
with log data that we're sending to it.
In the client, don't assume that we know which logdrivers the server
implements, and remove the check that looks at the server. It's
redundant because the server already knows, and the check also makes
using older clients with newer servers (which may have new logdrivers in
them) unnecessarily hard.
When we try to "logs" and have to report that the container's logdriver
doesn't support reading, send the error message through the
might-be-a-multiplexer so that clients which are expecting multiplexed
data will be able to properly display the error, instead of tripping
over the data and printing a less helpful "Unrecognized input header"
error.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Noteworthy changes:
- Add Prestart/Poststop hook support
- Fix bug finding cgroup mount directory
- Add OomScoreAdj as a container configuration option
- Ensure the cleanup jobs in the deferrer are executed on error
- Don't make modifications to /dev when it is bind mounted
Other changes in runc:
https://github.com/opencontainers/runc/compare/v0.0.3...v0.0.4
Signed-off-by: David Calavera <david.calavera@gmail.com>
This PR makes a user visible behavior change with userland
proxy disabled by default and rely on hairpin NAT to be enabled
by default. This may not work in older (unsupported) kernels
where the user will be forced to enable userlandproxy if needed.
- Updated the Docs
- Changed the integration-cli to start with userlandproxy
desiabled by default.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
This changeset creates /dev/shm and /dev/mqueue mounts for each container under
/var/lib/containers/<id>/ and bind mounts them into the container. When --ipc:container<id/name>
is used, then the /dev/shm and /dev/mqueue of the ipc container are used instead of creating
new ones for the container.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
(cherry picked from commit d88fe447df)
Also, use the channel to determine if the broadcaster is closed,
removing the redundant isClosed variable.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The original purpose of this was to cancel downloads if pullV2Tag
returns an error, preventing an associated crash (see #15353). The
broadcaster now accomplishes the same thing that the pipe does, making
the pipe redundant. When pullV2Tag returns, all broadcasters are closed,
which means all further writes to those broadcasters will return errors.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Before, this only waited for the download to complete. There was no
guarantee that the layer had been registered in the graph and was ready
use. This is especially problematic with v2 pulls, which wait for all
downloads before extracting layers.
Change Broadcaster to allow an error value to be propagated from Close
to the waiters.
Make the wait stop when the extraction is finished, rather than just the
download.
This also fixes v2 layer downloads to prefix the pool key with "layer:"
instead of "img:". "img:" is the wrong prefix, because this is what v1
uses for entire images. A v1 pull waiting for one of these operations to
finish would only wait for that particular layer, not all its
dependencies.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Ensure that layers are not excluded from manifests based on previous pushes.
Continue skipping pushes on layers which were pushed by a previous tag.
Update push multiple tag tests.
Ensure that each tag pushed exists on the registry and is pullable.
Add output comparison on multiple tag push check.
fixes#15536
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Allow to set the signal to stop a container in `docker run`:
- Use `--stop-signal` with docker-run to set the default signal the container will use to exit.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Without this change, there was a narrow race condition that would allow
writers to finish when there was still data left to write. This is
likely to be what was causing some integration tests to fail with
truncated pull output.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Allows people to create out-of-process graphdrivers that can be used
with Docker.
Extensions must be started before Docker otherwise Docker will fail to
start.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This defines a 'context' object that is passed to each API handler.
Right now the context just has a unique 'requestID' for each API call.
The next steps would be:
- use this 'requestID' in our logging.
- determine the best way to format the logging to include this info.
In particular for log events that generate multiple entries in the log
we can use the requestID to help correlate the log entries.
Adding the requestID to the logging will be a challenge since it could mean
changing every single logrus.XXX() call to pass in the 'context' object.
But first step is to agree on a format, which we can discus in a subsequent
PR, but my initial thoughts are to add it right after the timestamp:
current format:
INFO[0039] POST /v1.21/build?buildargs=%7B%22foo%22%3A%22xxx%22%7D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null
proposed format:
INFO[0039-83dea1222191] POST /v1.21/build?buildargs=%7B%22foo%22%3A%22xxx%22%7D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null
Signed-off-by: Doug Davis <dug@us.ibm.com>
- utils_test.go and docker_utils_test.go
- Moved docker related function to docker_utils.go
- add a test for integration-cli/checker
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
String.Trim(string, " ") can only get ride of the blank, but actully we need
to remove all leading and trailing white space include "\t" for example.
Prior to this patch, one can login with "[Tab][Tab]username", and it will
not be consider as the same with "username".
Signed-off-by: Hu Keping <hukeping@huawei.com>
Underlying volume data may have been removed by some other tool.
Ignore and remove the reference in this case.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Closes#16146
While in there, modified the testing infrastructure for the help text
so that we can get commands with nested commands - like "volume".
Signed-off-by: Doug Davis <dug@us.ibm.com>
For both commands, volume is _not_ optional. Several volumes may
be specified.
Both commands now use the same name (VOLUME) for the command argument.
Signed-off-by: Harald Albers <github@albersweb.de>
+ Fix a couple of bugs introduced by previous vendoring:
- in bitseq which prevents to use experimental overlay networking
- in docker service ls cli o/p
+ Add missing http subrouter for newly introduced sandboxes
+ Fix fragmentation issue on vxlan header addition for overlay network driver
+ Remove libnetwork test code utilities from vendoring
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Remove extraneous '-' in sudo note.
- Correct space formatting (minor).
- Simplify instructions for install EPEL and install with yum directly.
- Add `nodejs` to yum install list explicitly to match the comments -- more transparent for new users who might not be clear `npm` package also installs `nodejs`.
- Remove '--noprealloc` from MongoDB example as the option has been depcated since Mongo 2.6 and is now the default behaviour. See: http://docs.mongodb.org/manual/reference/program/mongod/
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
The docker volume ls -f dangling=true filter was
inverted; the filtered results actually returned all
non-dangling volumes.
This fixes the filter and adds some integration tests
to test the correct behavior.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For now CLI `docker stats` will not block even if the container was
not running is because there is a 2s timeout setting when waiting for
the response.
I think why we hang there waiting for the container to run is because we
want to get the stats of container immediately when it starts running.
But it will block when use the API directly, for example
- curl
- Google Chrome plugin, Postman
- Firefox plugin, RESTClient
This patch keeps the feature that getting info immediately when container
starts running and in the meantime, it will not block when using the API
directrly.
Signed-off-by: Hu Keping <hukeping@huawei.com>
TL;DR: stop building static binary that may fail
Linker flag --unresolved-symbols=ignore-in-shared-libs was added
in commit 06d0843 two years ago for the static build case, presumably
to avoid dealing with problem of missing libraries.
For the record, this is what ld(1) man page says:
> --unresolved-symbols=method
> Determine how to handle unresolved symbols. There are four
> possible values for method:
> .........
> ignore-in-shared-libs
> Report unresolved symbols that come from regular object files,
> but ignore them if they come from shared libraries. This can
> be useful when creating a dynamic binary and it is known that
> all the shared libraries that it should be referencing are
> included on the linker's command line.
Here, the flag is not used for its purpose ("creating a dynamic binary")
and does more harm than good. Instead of complaining about missing symbols
as it should do if some libraries are missing from LIBS/LDFLAGS, it lets
ld create a binary with unresolved symbols, ike this:
$ readelf -s bundles/1.7.1/binary/docker-1.7.1 | grep -w UND
........
21029: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND dlopen
.........
Such binary is working just fine -- until code calls one of those
functions, then it crashes (for apparently no reason, i.e. it is
impossible to tell why from the diagnistics printed).
In other words, adding this flag allows to build a static binary
with missing libraries, hiding the problem from both a developer
(who forgot to add a library to #cgo: LDFLAGS -- I was one such
developer a few days ago when I was working on ploop graphdriver)
and from a user (who expects the binary to work without crashing,
and it does that until the code calls a function in one of those
libraries).
Removing the flag immediately unveils the problem (as it should):
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o):
In function `unixDlError':
(.text+0x20971): undefined reference to `dlerror'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o):
In function `unixDlClose':
(.text+0x8814): undefined reference to `dlclose'
The problem is, gosqlite package says:
#cgo LDFLAGS: -lsqlite3
which is enough for dynamic linking, as indirect dependencies (i.e.
libraries required by libsqlite3.so) are listed in .so file and will be
resolved dynamically by ldd upon executing the binary.
For static linking though, one has to list all the required libraries,
both direct and indirect. For libraries with pkgconfig support the
list of required libraries can be obtained with pkg-config:
$ pkg-config --libs sqlite3 # dynamic linking case
-lsqlite3
$ pkg-config --libs --static sqlite3 # static case
-lsqlite3 -ldl -lpthread
It seems that all one has to do is to fix gosqlite this way:
-#cgo LDFLAGS: -lsqlite3
+#cgo pkg-config: sqlite3
Unfortunately, cmd/go doesn't know that it needs to pass --static
flag to pkg-config in case of static linking
(see https://github.com/golang/go/issues/12058).
So, for one, one has to do one of these things:
1. Patch sqlite.go like this:
-#cgo LDFLAGS: -lsqlite3
+#cgo pkg-config: --static sqlite3
(this is exactly what I do in goploop, see
https://github.com/kolyshkin/goploop/commit/e9aa072f51)
2. Patch sqlite.go like this:
-#cgo LDFLAGS: -lsqlite3
+#cgo LDFLAGS: -lsqlite3 -ldl -lpthread
(I would submit this patch to gosqlite but it seems that
https://code.google.com/p/gosqlite/ is deserted and not maintained,
and patching it here is not right as it is "vendored")
3. Explicitly add -ldl for the static link case.
This is what this patch does.
4. Fork sqlite to github and maintain it there. Personally I am not
ready for that, as I'm neither a Go expert nor gosqlite user.
Now, #3 doesn't look like a clear solution, but nevertheless it makes
the build much better than it was before.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Calling runtime.Stack requires the buffer to be big enough to fit the
goroutines dump. If it's not big enough the dump will be truncated and
the value returned will be the same size as the buffer.
The code was changed to handle this situation and try again with a
bigger buffer. Each time the dump doesn't fit in the buffer its size is
doubled.
Signed-off-by: Cezar Sa Espinola <cezarsa@gmail.com>
Fixed issues related to network subcommand tests
- "network" in exempted list of short help check
- Condition for exact test modified to meet experimental commands
- Sorting of commands done in flags_experimental
Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
Update and migrate existing tests to the `DockerHubPullSuite`. Most
tests were preserved, but refactored and made more exhaustive. One test
was deliberately removed (`TestPullVerified`) as it is unreliable and
that the feature was obsoleted by content trust.
Move all trust related tests to `docker_cli_pull_trusted_test.go`.
Move tests depending on a local registry to `docker_cli_pull_local_test.go`.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Add a `checker` package that adds some utility Checker implementation,
the first one being `checker.Contains`, as well as brining all go-check
provided Checker implementations in scope as a commodity.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Regarding the outdated error check, there's no `docker.PortMapping`
struct anymore and this is linked to something really old #1334
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Introduce the `DockerHubPullSuite` that interacts with its own dedicated
daemon, thus allowing to start from a clean environment and finely test
against the impact of isolated push and pull operations.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
When using a named volume without --volume-driver, the driver was
hardcoded to "local".
Even when the volume was already created by some other driver (and
visible in `docker volume ls`), the container would store in it's own
config that it was the `local` driver.
The external driver would work perfecly fine until the daemon is
restarted, at which point the `local` driver was assumed because that is
as it was set in the container config.
Set the bind driver to the driver returned by createVolume.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Main changes in this vendoring are to allow user name space integration in docker.
And it includes major fix for network namespace handling
Signed-off-by: Alessandro Boch <aboch@docker.com>
Signed-off-by: Alessandro Boch <aboch@docker.com>
Avoid duplicate definitions of NewSqliteConn when cgo isn't enabled, so
that we can at least build the daemon.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
The structure of this header has never been specified in the docs and has
been a pain point for those who have tried to get registry authentication
to work during builds. This Patch *finally* adds a description of JSON
object which should be base64-url-safe-encoded and used as the value of
the X-Registry-Config HTTP request header during `POST /build`.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This route was deprecated more than two years ago in the linked
commit[1]. It's not referenced anywhere in the documentation and it's
time to stop maintaning it.
[1]: 4f9443927e
Signed-off-by: David Calavera <david.calavera@gmail.com>
Separate container iteration, filtering and reducing.
This will make easier in the future to improve the implementation of
docker ps as we know it.
The end goal is to unify the objects returned by the api for docker ps
and docker inspect, leaving all docker ps transformations to the client.
Signed-off-by: David Calavera <david.calavera@gmail.com>
The Docker Daemon should send actual actions client ask for to issue tokens,
not all the permissions that client is guaranteed.
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
This file was not well documented and had very high cyclomatic complexity.
This patch completely rearranges this file and the ImageDelete method to
be easier to follow and more maintainable in the future.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This means the writing to a WriteFlusher will flush in the same places
as it would if the broadcaster wasn't sitting in front of it.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- some method names were changed to have a 'Locking' suffix, as the
downcased versions already existed, and the existing functions simply
had locks around the already downcased version.
- deleting unused functions
- package comment
- magic numbers replaced by golang constants
- comments all over
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
* Fixed exp/beginner label colour background
* Added thead element to label table
* Updated issue list screenshot (old one referenced old whitebelt label)
* Content tweaks as per style guide
Signed-off-by: Lynda O'Leary <lyndaoleary29@gmail.com>
- Rename to Broadcaster
- Document exported types
- Change Wait function to just wait. Writing a message to the writer and
adding the writer to the observers list are now handled by separate
function calls.
- Avoid importing logrus (the condition where it was used should never
happen, anyway).
- Make writes non-blocking
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Previously, its other return value was used even when it returned an
error. This is awkward and goes against the convention. It also could
have resulted in a nil pointer dereference when an error was returned
because of an unknown pool type. This changes the unknown pool type
error to a panic (since the pool types are hardcoded at call sites and
must always be "push" or "pull"), and returns a "found" boolean instead
of an error.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Based on #12874 from Sam Abed <sam.abed@gmail.com>. His original commit
was brought up to date by manually porting the changes in pull.go into
the new code in pull_v1.go and pull_v2.go.
Fixes#8385
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Dharmit Shah <shahdharmit@gmail.com>
Added note about process interaction with container in detached mode
Signed-off-by: Dharmit Shah <shahdharmit@gmail.com>
This adds a data struct in the aufs driver for including more
information about active mounts along with their reference count.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Makes it possible to filter containers by image, using
--filter=ancestor=busybox and get all the container running busybox
image and image based on busybox (to the bottom).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
The shebang for OpenRC init scripts is now #!/sbin/openrc-run, and we
can also use the default start and stop functions built into OpenRC.
Signed-off-by: William Hubbs <w.d.hubbs@gmail.com>
The practice of buffering to a tempfile during a pushing contributes massively
to slow V2 push performance perception. The protocol was actually designed to
avoid precalculation, supporting cut-through data push. This means we can
assemble the layer, calculate its digest and push to the remote endpoint, all
at the same time.
This should increase performance massively on systems with slow disks or IO
bottlenecks.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
- Refer to Joyent's blogs and docs rather than following from our end
- All of this information exists on Joyent's site
Updating with thaJeztah's comments
Signed-off-by: Mary Anthony <mary@docker.com>
Now scheduler makes order of events pretty random, so I added little
sleeps to make order intact. Also I renamed to test so name better
describes its nature.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
This fixes two problems:
1. docker run --device /dev/sda:rw ubuntu bash doesn't work
2. --device /dev/zero:/dev/noro:ro doesn't show clear error message,
but fail when writing to cgroup file.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
1. rename it from ValidateMountMode to ValidMountMode
Because it's a function simply check mount mode is valid or not.
2. remove the rw check return value
It's not supposed to be combined into this function, and we already
have a function for that check.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
In particular I want to make sure that calling getEnv() when the same
var name appears more than once in the env list that we only pick up
the first one. PR #15182 counts on this
Signed-off-by: Doug Davis <dug@us.ibm.com>
* Fix Windows 7 (not 7.1 ;p)
* Clarify virtualization technology section, add link to wiki
* Fix heading level for various ways of starting Docker container
* Clarify `default` VM
* Various minor changes
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
This allows someone running the image to use `-v` to mount a non-tmpfs `/tmp` into their image if they so require/desire.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
Maps rely on the keys being comparable.
Using an interface type as the map key is dangerous,
because some interface types are not comparable.
I talked about this in my "Stupid Gopher Tricks" talk:
https://talks.golang.org/2015/tricks.slide
In this case, if the user-provided writer is backed by a slice
(such as io.MultiWriter) then the code will panic at run time.
Signed-off-by: Andrew Gerrand <adg@golang.org>
Hardcode the release tag, so that:
- it goes straight to the changes for 1.2, not whatever the current
version is
- it still works now that we've renamed CHANGES.md to CHANGELOG.md
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
With this, you can specify a branch name in the
vendor script instead of a commit ID. This makes it easier
to quickly test changes in dep'd repos outside of the DIND
environment.
Signed-off-by: Christy Perez <christy@linux.vnet.ibm.com>
run.md states that the operator can override all defaults set in the Dockerfile, and explicitly says that `--expose` overrides the `EXPOSE` instruction. Neither of these are true. An `EXPOSE` instruction cannot be overridden, `--expose` can only add additional exposed ports.
This change fixes the instructions, and also takes the liberty of crisping up the grammar and phrasing in a place or two.
Signed-off-by: Spencer Brown <spencer@spencerbrown.org>
This changeset creates /dev/shm and /dev/mqueue mounts for each container under
/var/lib/containers/<id>/ and bind mounts them into the container. When --ipc:container<id/name>
is used, then the /dev/shm and /dev/mqueue of the ipc container are used instead of creating
new ones for the container.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This patch makes it such that plugin initialization is synchronized
based on the plugin name and not globally
Signed-off-by: Darren Shepherd <darren@rancher.com>
- rewrite intro to Dockerfile reference usage section to remove
references to 'source repository'
- Closes#14714
- Fixes: #8648
- Updating with Seb's comments
Signed-off-by: Mary Anthony <mary@docker.com>
This was making logrus warn on each container create and start.
These warnings are not needed as the code below already warns when these
various cgroup settings aren't supported but have been set.
Warnings were originally introduced by #15381, which appear to be a
side-effect of that change and not the intention.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This option was incorrectly ported to the new `daemon` subcommand
structure.
Beside the obvious effect that completion of `docker daemon --log-opt`
did not work, this also caused completion of `docker` and `docker xxx`
to fail on macs with
> bash: words: bad array subscript
Signed-off-by: Harald Albers <github@albersweb.de>
Docker ps default format should not take precedence over cli flags.
This happens effectively for other flags except `-q`.
We need to let the cli to set the format as table to print the
expected output with `-q`.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Windows: add support for images stored in alternate location.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Changing vendor/src/github.com/docker/libnetwork to match lindenlab/libnetwork custom-host-port-ranges-1.7 branch
Updating netlink package to 4b5dce31de6d42af5bb9811c6d265472199e0fec
to fix certain wierd netlink issues seen.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
The bool logic around setting up the TTY ended up getting flipped
accidentally.
Also added a test for exec with TTY.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
..also update the text that says "TO.BE.DETERMINED" to the actual URL
where the experimental nightlies are published.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Allow the user to configure how Docker's bash completion works for the
"events", "history", "inspect", "run", "rmi" and "save" commands through the
following environment variables:
DOCKER_COMPLETION_SHOW_IMAGE_IDS
"none" - Show names only (default)
"non-intermediate" - Show names and ids, but omit intermediate image IDs
"all" - Show names and ids, including intermediate image IDs
DOCKER_COMPLETION_SHOW_TAGS
"yes" - include tags in completion options (default)
"no" - don't include tags in completion options
Fixes#9474.
Signed-off-by: Rory Hunter <roryhunter2@gmail.com>
Was reading this to explain it to someone and noticed the missing "s"
which makes it not so great for nice English readability..
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Fixes rare edge case of handling GNU LongLink and LongName entries.
Perf improvements. /dev/null writes were taking CPU time during docker
push. Thanks @LK4D4
Various cleanup too.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Introduce a write denial for files at the root of /proc.
This prohibits root users from performing a chmod of those
files. The rules for denials in proc are also cleaned up,
making the rules better match their targets.
Locally tested on:
- Ubuntu precise (12.04) with AppArmor 2.7
- Ubuntu trusty (14.04) with AppArmor 2.8.95
Signed-off-by: Eric Windisch <eric@windisch.us>
* use downloadInfo pointers everywhere
* use downloads slice only for things that we really download
* cleanup tmp files in all cases
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Make command like "docker images ubuntu:14.04" work and filter out the
image with the given tag.
Closes#8048.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
it is possible to pass an UID that has not been created inside the container, clarify this in the docs.
This should fix issue #14795
Signed-off-by: Kai Blin <kai@samba.org>
Commit e27c904 added a wrong and misleading comment
to GetMetadata(). Fix it using the wording from
commit 407a626 which introduced GetMetadata().
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
These functions are not part of the graphdriver.Driver
interface and should therefore be private.
Also, remove comments added by commit e27c904 as they are
* pretty obvious
* no longer required by golint
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Rather than keeping a branch named "release" where we merge every release,
we want to keep "disposable" release branches for each release.
This will make much easier to create new major releases, since we always
want to start these releases from what we have in master. We create a
disposable release branch from master that we'll only use during the lifecycle
of the major release.
Once the release branch is created, the release captain will use a "bump" branch
in her own fork to cherry pick changes that will need to be applied to this
release. We'll track those changes in a pull request between the disposable release
branch and the bump branch.
Signed-off-by: David Calavera <david.calavera@gmail.com>
All docker subcommands support `-h` as an alias for `--help`
unless they have `-h` aliased to something else like `docker run`,
which uses `-h` for `--hostname`.
`-h` is not included in the help messages of the commands, though.
It ist visible in
* reference: only in `docker daemon` reference,
see output of `grep -Rse --help=false docs`
* man pages: only in `docker` man page
see output of `grep -RF '**-h**' man`
For consistency reasons, this commit removes `-h` as an alias for
`--help` from the reference page, man page and the bash completion.
Signed-off-by: Harald Albers <github@albersweb.de>
Removing references to regsitry 1.0, pointing to distribution
Updating links and title
Adding in comments
Signed-off-by: Mary Anthony <mary@docker.com>
- started from Diogo's work
- updated after discussions with team
- Updating with new key names
- fixing weight
- adding in sandbox
- adding in gliffy for images
- backing out to old names for now
- Copy edit pass
- Entering comments from the content trust team
- Update name of branch and image name
- Removing the last diogo reference
- Updating with Seb's comments
Signed-off-by: Mary Anthony <mary@docker.com>
Ploop graph driver provides its own ext4 filesystem to every
container. It so happens that ext4 root comes with lost+found
directory, causing failures from DriverTestCreateEmpty() and
DriverTestCreateBase() tests on ploop.
While I am not yet ready to submit ploop graph driver for review,
this change looks simple enough to push.
Note that filtering is done without any additional allocations,
as described in https://github.com/golang/go/wiki/SliceTricks.
[v2: added a comment about lost+found]
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
The documentation for Debian and Ubuntu explains how to enable memory
and swap accounting but doesn't explain why it is disabled in the first
place. The problem with those subsystems is that they incurs a
performance hit even when not used at all. Add this explanation.
The provided figure are quite vague. The memory overhead is easily
verifiable. It is for example cited in [RedHat documentation][]. For the
performance hit, maybe the performance is better now, but a few years
ago, it was said to be [between 6 and 15%][].
The goal is that people don't just enable memory accounting if they
don't have a use for it.
[RedHat documentation]: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html
[between 6 and 15%]: https://lwn.net/Articles/517562/
Signed-off-by: Vincent Bernat <vincent@bernat.im>
This update fixes Windows client console bugs and increases VT100
compatibility. With this change, nano and emacs become usable, and bash
works better.
Signed-off-by: John Starks <jostarks@microsoft.com>
Have network files mounted read-only when mounted using the -v
open and -v parameter has 'ro' passed.
Signed-off-by: Stefan Berger <stefanb@us.ibm.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
It may happen that host system settings are changed while the daemon is running.
This will cause errors at libcontainer level when starting a container with a
particular hostConfig (e.g. hostConfig with memory swappiness but the memory
cgroup was umounted).
This patch adds an hostConfig check on container start to prevent the daemon
from even calling libcontainer with the wrong configuration as we're already
doing on container's creation).
Signed-off-by: Antonio Murdaca <runcom@linux.com>
(cherry picked from commit 0d2628cdf1)
sysinfo struct was initialized at daemon startup to make sure
kernel configs such as device cgroup are present and error out if not.
The struct was embedded in daemon struct making impossible to detect
if some system config is changed at daemon runtime (i.e. someone
umount the memory cgroup). This leads to container's starts failure if
some config is changed at daemon runtime.
This patch moves sysinfo out of daemon and initilize and check it when
needed (daemon startup, containers creation, contaienrs startup for
now).
Signed-off-by: Antonio Murdaca <runcom@linux.com>
(cherry picked from commit 472b6f66e0)
This patch splits LookupEndpoints into LookupPullEndpoints and
LookupPushEndpoints so that mirrors added with --registry-mirror are
skipped in the list returned by LookupPushEndpoints.
Fixes https://github.com/docker/distribution/issues/823
Signed-off-by: Tibor Vass <tibor@docker.com>
I just tried the installation but docker didn't start automatically. So I had to execute the newly added command in order to get the hello-world verification running.
Signed-off-by: Jenny Gebske <jennifer@gebske.de>
The docker script in contrib/init/sysvinit-redhat will fail silently on
a start if Docker is not installed in the default /usr/bin/ location.
While a non-zero exit code is returned the user will receive no visible
indication (i.e. error message) as to why Docker was not started.
This commit changes the logic so that in the case that the docker
executable is not found in the expected location or the user does not
have execute permissions on the executable appropriate error messages
are now shown to the user as well as exiting with a non-zero exit code
Signed-off-by: Rob Vesse <rvesse@dotnetrdf.org>
- Removing references to Boot2Docker replacing with Docker Machine
- Removing sudo warnings in instances where appropriate (no sudo in file)
- Updating with comments
Signed-off-by: Mary Anthony <mary@docker.com>
The process of pulling an image spawns a new goroutine for each layer in the
image manifest. If any of these downloads fail we would stop everything and
return the error, even though other goroutines would still be running and
writing output through a progress reader which is attached to an http response
writer. Since the request handler had already returned from the first error,
the http server panics when one of these download goroutines makes a write to
the response writer buffer.
This patch prevents this crash in the daemon http server by waiting for all of
the download goroutines to complete, even if one of them fails. Only then does
it return, terminating the request handler.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Don't assume that any random image will have 'top' and explicitely use
the busybox image for testing.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Carried: #14015
If kernel is compiled with CONFIG_FAIR_GROUP_SCHED disabled cpu.shares
doesn't exist.
If kernel is compiled with CONFIG_CFQ_GROUP_IOSCHED disabled blkio.weight
doesn't exist.
If kernel is compiled with CONFIG_CPUSETS disabled cpuset won't be
supported.
We need to handle these conditions by checking sysinfo and verifying them.
Signed-off-by: Zefan Li <lizefan@huawei.com>
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
"Options:" listed when you run "docker --help" and "docker daemon
--help" do not match the options listed in "man/docker.1.md". This PR
makes 'docker --help', 'docker daemon --help' and 'man docker' consistent.
Also 2 typo fixes.
Signed-off-by: Sally O'Malley <somalley@redhat.com>
We noticed a regression since the 1.7.1 patch after some refactoring. This
patch corrects the behavior and adds integration tests for modified manifest
and rootfs layer blobs.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The custom configuration will also be used in docker invocations made
by the completion script itself, just like `-H`.
Signed-off-by: Harald Albers <github@albersweb.de>
- fixing headings in run.md
- creating a table for readability
- adding index for logging
- moving logging overview into logging
- Updating with Seb's comments
Signed-off-by: Mary Anthony <mary@docker.com>
Minor fixes:
* v1.19: GET /containers/(id)/logs - add missing '/'
* v1.18: Break up POST /containers/create and POST /containers/(id)/start into separate lines.
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
manually stopped
If a container is running with a restart policy of always and it's
manually stopped, then on daemon restart it will be running.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
After tailing a file, if the number of lines requested is > the number
of lines in the file, this would cause a json unmarshalling error to
occur when we later try to go follow the file.
So brute force set it to the end if any tailing occurred.
There is potential that there could be some missing log messages if logs
are being written very quickly, however I was not able to make this
happen even with `while true; do echo hello; done`, so this is probably
acceptable.
While testing this I also found a panic in LogWatcher.Close can be
called twice due to a race. Fix channel close to only close when there
has been no signal to the channel.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
It's a bit confusing: the "global options" are valid as "global options"
for all client commands (i.e. all but daemon).
Example: `docker --log-level info run`
For `docker daemon`, these "global options" are only valid as "command
options".
Example: `docker daemon --log-level info`
As command completion cannot tell which command the user is going to
type next, completion for the daemon command has to allow illegal
syntaxes like
`docker --log-level info daemon --log-level info`
Signed-off-by: Harald Albers <github@albersweb.de>
This patch fixes a bug where a user specifies a v1 mirror for
--registry-mirror and pull an image from the Hub.
It used to not fallback because of an unexpected error returned when
trying to JSON marshal nginx output.
We now ensure that any unexpected error falls back to the next endpoint
in the list.
Signed-off-by: Tibor Vass <tibor@docker.com>
Expand the godoc documentation for the graph package.
Centralize DefaultTag in the graphs/tag package instead of defining it
twice.
Remove some unnecessary "config" structs that are only used to pass
a few parameters to a function.
Simplify the GetParentsSize function - there's no reason for it to take
an accumulator argument.
Unexport some functions that aren't needed outside the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
[pkg/archive] Update archive/copy path handling
- Remove unused TarOptions.Name field.
- Add new TarOptions.RebaseNames field.
- Update some of the logic around path dir/base splitting.
- Update some of the logic behind archive entry name rebasing.
[api/types] Add LinkTarget field to PathStat
[daemon] Fix stat, archive, extract of symlinks
These operations *should* resolve symlinks that are in the path but if the
resource itself is a symlink then it *should not* be resolved. This patch
puts this logic into a common function `resolvePath` which resolves symlinks
of the path's dir in scope of the container rootfs but does not resolve the
final element of the path. Now archive, extract, and stat operations will
return symlinks if the path is indeed a symlink.
[api/client] Update cp path hanling
[docs/reference/api] Update description of stat
Add the linkTarget field to the header of the archive endpoint.
Remove path field.
[integration-cli] Fix/Add cp symlink test cases
Copying a symlink should do just that: copy the symlink NOT
copy the target of the symlink. Also, the resulting file from
the copy should have the name of the symlink NOT the name of
the target file.
Copying to a symlink should copy to the symlink target and not
modify the symlink itself.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.
Quoting MkdirAll documentation:
> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.
This means two things:
1. If a directory to be created already exists, no error is returned.
2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.
The above is a theory, based on quoted documentation and my UNIX
knowledge.
3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of #1, IsExist check after MkdirAll is not needed.
Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.
Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.
[v2: a separate aufs commit is merged into this one]
[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
- don't use relative references '..'
- registry-index-spec became hub-registry-spec
- reference new 'glossary' instead of old 'terms'
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
The 'deny ptrace' statement was supposed to only ignore
ptrace failures in the AUDIT log. However, ptrace was implicitly
allowed from unconfined processes (such as the docker daemon and
its integration tests) due to the abstractions/base include.
This rule narrows the definition such that it will only ignore
the failures originating inside of the container and will not
cause denials when the daemon or its tests ptrace inside processes.
Introduces positive and negative tests for ptrace /w apparmor.
Signed-off-by: Eric Windisch <eric@windisch.us>
Add tools to the apparmor profile that are needed when -s devicemapper is
in the docker daemon's command line.
Signed-off-by: Stefan Berger <stefanb@us.ibm.com>
* Add space between values in docker stats output for easier parsing
Old output could not be parsed easily because there were columns
that did not have any separator. Also values that are together
without any space is difficult to read even for humans.
* Update unit.HumanSize comment to match what the does actually does
Signed-off-by: Otto Kekäläinen <otto@seravo.fi>
Integration tests were failing due to proc filter behavior
changes with new apparmor policies.
Also include the missing docker-unconfined policy resolving
potential startup errors. This policy is complain-only so
it should behave identically to the standard unconfined policy,
but will not apply system path-based policies within containers.
Signed-off-by: Eric Windisch <eric@windisch.us>
- downcase and privatize exported variables that were unused
- make accurate an error message
- added package comments
- remove unused var ReadLogsNotSupported
- enable linter
- some spelling corrections
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Currently some notary tests change the system clock to check for expiration.
Skip these tests until the code can be refactored to not rely on updating the system clock.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Crypto rand is a much better seed for math/rand than
time. In the event we use math/rand where we should not,
this will make it a safer source of random numbers.
Although potentially dangerous, this will still fallback
to time should crypto/rand for any reason fail.
Signed-off-by: Eric Windisch <eric@windisch.us>
If I run two containers with the same network they share the same /etc/resolv.conf.
The current code changes the labels of the /etc/resolv.conf currently to the
private label which causes it to be unusable in the first container.
This patch changes the labels to a shared label if more then one container
will use the content.
Docker-DCO-1.1-Signed-off-by: Dan Walsh dwalsh@redhat.com (github: rhatdan)
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Currently the service type is 'simple', the default, meaning that
docker.service is considered to be started straight after
spawning. This is incorrect as there is significant amount of time
between spawning and docker ready to accept connections on the passed
sockets. Docker does implement systemd socket activate and
notification protocol, and send the ready signal to systemd, once it
is ready. However for systemd to take those notifications into
account, the service file type should be set to notify.
Signed-off-by: Dimitri John Ledkov <dimitri.j.ledkov@intel.com>
- Tested Mac/iterated with Jeff on fixes
- Waiting on formal uninstall;manual now
- 4 hour work
- Fixe #14562 converting Windows to Mac
- Found errors in installer need fixes/another iteration
- Updated as far as possible with limited installation
- 3 Hours work
- Entering comments from PR review
- New screen captures and migration material
- Entering Sven's comment
- Testing with powershell, Seb's comments
- Fix link in upgrade
- Minor tweaks to http and typos
- Adding deprecation note
Signed-off-by: Mary Anthony <mary@docker.com>
Builds where the base images have been resolved to trusted digest
references will now be tagged with the original tag reference from
the Dockerfile on a successful build.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The engine policy will now only complain
as a temporary measure to ensure we do not
cause breakages while users exercise this
policy.
This is NOT the policy for containers, but
for the newly-introduced policy for the
daemon itself.
Signed-off-by: Eric Windisch <eric@windisch.us>
Implements the policies for the remaining binaries
called by the Docker engine and eliminates the
giant whitelisted 'all files' permission in favor
of granular whitelisting and child-specific policies.
It should be possible now to remove the 'file' permission,
but for the sake of keeping Docker unbroken, we'll try
to gradually tighten the policy.
Signed-off-by: Eric Windisch <eric@windisch.us>
Will attempt to load profiles automatically. If loading fails
but the profiles are already loaded, execution will continue.
A hard failure will only occur if Docker cannot load
the profiles *and* they have not already been loaded via
some other means.
Also introduces documentation for AppArmor.
Signed-off-by: Eric Windisch <eric@windisch.us>
The `ApplyDiff` function takes a tar archive stream that is
automagically decompressed later. This was causing a double
decompression, and when the layer was empty, that causes an early EOF.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
In `ApplyLayer` and `Untar`, the stream is magically decompressed. Since
this is not able to be toggled, rather than break this ./pkg/ API, add
an `ApplyUncompressedLayer` and `UntarUncompressed` that does not
magically decompress the layer stream.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Return an error when the container is stopped only in api versions
equal or greater than 1.20 (docker 1.8).
Signed-off-by: David Calavera <david.calavera@gmail.com>
Options for zfs storage driver were incorrectly placed
under 'exec driver options' header. Move the header to
the correct place.
Now, this is the second time I am fixing this. First time
it was commit 68efb27, but the following commit 9af7afb
screwed it up again, so the header appears twice now.
Get rid of the the wrong one.
Cc: David Calavera <david.calavera@gmail.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
- comments on exported values
- constant string replaced by constant reference
- unexport implementation details of VolumeDriver 'local'
- add fixed packages to linter list
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Currently login and search do not load per registry certificates.
This is a regression caused by the last refactor since this was recently fixed.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
daemon_test.go supposted to be unit test for daemon, so
don't see reason why we need another daemon_unit_test.go.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
- boot2docker is deprecated in the 1.8.0
- docker-machine replaces it
- this fixes#14563
- Updating with thaJetzah comments
Signed-off-by: Mary Anthony <mary@docker.com>
Actually determine the relative path of the Dockerfile to the context
directory. Error out if the relative path starts with "../".
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
A bash completion file shouldn't have a executable bit set.
Just change file mode to 644 (instead of 755).
Signed-off-by: Dieter Reuter <dieter.reuter@me.com>
This version includes a fix that avoids checking against specific HTTP
status codes. The previous behavior violated the registry API spec.
Fixes#14975
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
There was a mistake in the plugin_volume.md file regarding the /VolumeDriver.Remove call.
Added fix and a better explanation.
Signed-off-by: Bardia Keyoumarsi <b.keyoumarsi@gmail.com>
Fix#14915. Add unit test for #14915.
Thanks @runcom for the test case: when the client calls 1.18 api
version w/o hostconfig it results in a nil pointer dereference.
Signed-off-by: Stephen Rust <srust@blockbridge.com>
PR #11907 added support for import using file (path), but it missed
the update of cli/import.md. This fixes that.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
In 1.6.2 we were decoding inspect API response into interface{}.
time.Time fields were JSON encoded as RFC3339Nano in the response
and when decoded into interface{} they were just strings so the inspect
template treated them as just strings.
From 1.7 we are decoding into types.ContainerJSON and when the template
gets executed it now gets a time.Time and it's formatted as
2015-07-22 05:02:38.091530369 +0000 UTC.
This patch brings back the old behavior by typing time.Time fields
as string so they gets formatted as they were encoded in JSON -- RCF3339Nano
Signed-off-by: Antonio Murdaca <runcom@linux.com>
* Clarify the list of supported instructions.
* Clarify behavior of ONBUILD, based on comments by @SvenDowideit, @theJeztah in PR #14735.
* Reorder list of instructions in alphabetical order.
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
If you have some kind of bogus `other_args` in `/etc/sysconfig/docker` the start script will report "started" but it's full of lies. This enhances the flow so that if the pidfile never shows up (failure to start) you get a proper failure message.
I also added dots for fun.
Signed-off-by: Jeff Minard <jeff.minard@creditkarma.com>
The Ansi parser and their associated actions have been decoupled. Now
parsing results in call backs to an interface which performs the
appropriate actions depending on the environment.
This improvement provides a functional Vi experience and the vttest no
longer panics.
This PR replaces docker/docker #13224 with the latest console updates.
Signed-off-by: John Howard <jhoward@microsoft.com>
When a container is removed but it had an exec, that still hasn't been
GC'd per PR #14476, and someone tries to inspect the exec we should
return a 404, not a 500+container not running. Returning "..not running" is
not only misleading because it could lead people to think the container is
actually still around, but after 5 minutes the error will change to a 404
after the GC. This means that we're externalizing our internall soft-deletion/GC
logic which shouldn't be any of the end user's concern. They should get the
same results immediate or after 5 minutes.
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-10 20:19:42 -07:00
1587 changed files with 130929 additions and 40136 deletions
If you know of another project underway that should be listed here, please help
us keep this list up-to-date by submitting a PR.
Awesome-Docker
==============
You can find more projects, tools and articles related to Docker on the [awesome-docker list](https://github.com/veggiemonk/awesome-docker). Add your project there.
cmd:=Cli.Subcmd("import",[]string{"file|URL|- [REPOSITORY[:TAG]]"},"Create an empty filesystem image and import the contents of the\ntarball (.tar, .tar.gz, .tgz, .bzip, .tar.xz, .txz) into it, then\noptionally tag it.",true)
cmd:=Cli.Subcmd("login",[]string{"[SERVER]"},"Register or login to a Docker registry server, if no server is\nspecified \""+registry.IndexServer+"\" is the default.",true)
cmd:=Cli.Subcmd("login",[]string{"[SERVER]"},Cli.DockerCommands["login"].Description+".\nIf no server isspecified \""+registry.IndexServer+"\" is the default.",true)
cmd:=Cli.Subcmd("logout",[]string{"[SERVER]"},"Logout from a Docker registry, if no server is\nspecified \""+registry.IndexServer+"\" is the default.",true)
cmd:=Cli.Subcmd("logout",[]string{"[SERVER]"},Cli.DockerCommands["logout"].Description+".\nIf no server isspecified \""+registry.IndexServer+"\" is the default.",true)
// Usage: docker port CONTAINER [PRIVATE_PORT[/PROTO]]
func(cli*DockerCli)CmdPort(args...string)error{
cmd:=Cli.Subcmd("port",[]string{"CONTAINER [PRIVATE_PORT[/PROTO]]"},"List port mappings for the CONTAINER, or lookup the public-facing port that\nis NAT-ed to the PRIVATE_PORT",true)
// A running container is stopped by first sending SIGTERM and then SIGKILL if the container fails to stop within a grace period (the default is 10 seconds).
cmd:=Cli.Subcmd("stop",[]string{"CONTAINER [CONTAINER...]"},"Stop a running container by sending SIGTERM and then SIGKILL after a\ngrace period",true)
cmd:=Cli.Subcmd("stop",[]string{"CONTAINER [CONTAINER...]"},Cli.DockerCommands["stop"].Description+".\nSending SIGTERM and then SIGKILL after agrace period",true)
nSeconds:=cmd.Int([]string{"t","-time"},10,"Seconds to wait for stop before killing it")
fmt.Fprintf(cli.err,"[DEPRECATED] The environment variable DOCKER_CONTENT_TRUST_OFFLINE_PASSPHRASE has been deprecated and will be removed in v1.10. Please use DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE\n")
fmt.Fprintf(cli.err,"[DEPRECATED] The environment variable DOCKER_CONTENT_TRUST_TAGGING_PASSPHRASE has been deprecated and will be removed in v1.10. Please use DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE\n")
This package includes types for legacy API versions. The stable version of the API types live in `api/types/*.go`.
Consider moving a type here when you need to keep backwards compatibility in the API. This legacy types are organized by the latest API version they appear in. For instance, types in the `v1p19` package are valid for API versions below or equal `1.19`. Types in the `v1p20` package are valid for the API version `1.20`, since the versions below that will use the legacy types in `v1p19`.
### Package name conventions
The package name convention is to use `v` as a prefix for the version number and `p`(patch) as a separator. We use this nomenclature due to a few restrictions in the Go package name convention:
1. We cannot use `.` because it's interpreted by the language, think of `v1.20.CallFunction`.
2. We cannot use `_` because golint complains abount it. The code is actually valid, but it looks probably more weird: `v1_20.CallFunction`.
For instance, if you want to modify a type that was available in the version `1.21` of the API but it will have different fields in the version `1.22`, you want to create a new package under `api/types/versions/v1p21`.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
