Compare commits

...

27 Commits

Author SHA1 Message Date
David Calavera
0a8c2e3717 Bump to version 1.8.2.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-10 14:43:43 -04:00
Tianon Gravi
57e9c4f7e5 Swap "ubuntu-debootstrap" for just "ubuntu"
See https://github.com/docker-library/official-images/pull/982#issuecomment-133207587.

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-09-10 14:43:43 -04:00
Jessica Frazelle
3e8da36017 use apt-ftparchive and reprepro to enable apt-pinning;
Signed-off-by: Jessica Frazelle <acidburn@docker.com>
(cherry picked from commit 12a71c8954)
2015-09-03 05:20:58 -04:00
David Calavera
1cce9a26a3 Bump libnetwork version to bc565c2d295067c1a43674a23a473ec6336d7fd4
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-02 17:24:37 -04:00
Tonis Tiigi
d7f8b4d43e Fix goroutine leak on pull
Close the pipeWriter even if there was no error.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-01 09:31:43 -04:00
Derek McGowan
6f7bbc3171 Fix sanitize URL bug on layer upload
Update the distribution version to include sanitize URL fix

Fixes #15875

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-08-31 03:37:50 -04:00
Harald Albers
947087fb24 Update bash completion for docker run
Also fixed sort order of options using `sort -d`

Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 644c158837)
2015-08-28 05:03:01 -04:00
Harald Albers
ffe7e48ed6 Add options for the json-file logging driver to bash completion
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 5c1ad6f90c)
2015-08-28 05:02:55 -04:00
Harald Albers
eeecd1cf59 Add storage options to bash completion
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit e4d8a8e1ca)

Conflicts:
	contrib/completion/bash/docker
2015-08-28 05:02:45 -04:00
Harald Albers
3f411db15b Add missing storage drivers to bash completion
Signed-off-by: Harald Albers <github@albersweb.de>
2015-08-28 04:59:30 -04:00
Harald Albers
789197f33d Remove -h flag from completion and daemon reference
All docker subcommands support `-h` as an alias for `--help`
unless they have `-h` aliased to something else like `docker run`,
which uses `-h` for `--hostname`.

`-h` is not included in the help messages of the commands, though.

It ist visible in
* reference: only in `docker daemon` reference,
  see output of `grep -Rse --help=false docs`
* man pages: only in `docker` man page
  see output of `grep -RF '**-h**' man`

For consistency reasons, this commit removes `-h` as an alias for
`--help` from the reference page, man page and the bash completion.

Signed-off-by: Harald Albers <github@albersweb.de>
2015-08-28 04:59:09 -04:00
Harald Albers
0c71d09921 Add docker ps --format to bash completion
Signed-off-by: Harald Albers <github@albersweb.de>
2015-08-28 04:57:54 -04:00
Tonis Tiigi
cc8320cb58 Fix pull on client disconnect
Fixes #15589

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-08-28 04:26:22 -04:00
Arnaud Porterie
c22b292719 Update vendoring
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-08-26 04:45:56 -04:00
Tonis Tiigi
14d2083f14 Remove nil contexts
Causes daemon panic because loggers can’t be found.

Fixes #15724

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 0c08913d52)
2015-08-25 12:25:38 -04:00
Tonis Tiigi
ea56c5e1ce Update volumes userguide
Fixes #15644

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-08-25 12:25:37 -04:00
Harald Albers
341ff018a2 Fix bash completion for log driver options
This option was incorrectly ported to the new `daemon` subcommand
structure.

Beside the obvious effect that completion of `docker daemon --log-opt`
did not work, this also caused completion of `docker` and `docker xxx`
to fail on macs with

> bash: words: bad array subscript

Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 18381faee6)

Conflicts:
	contrib/completion/bash/docker
2015-08-25 12:25:37 -04:00
Shijiang Wei
e07819293a a quick fix to #15626
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2015-08-25 12:25:37 -04:00
Marius Sturm
6ec8d40ae7 Initialize LogConfig in daemon mode
Signed-off-by: Marius Sturm <marius@graylog.com>
(cherry picked from commit e904cbec03)

Conflicts:
	docker/daemon.go
2015-08-25 12:25:37 -04:00
Jana Radhakrishnan
16d64608f3 Vendoring in vishvananda/netlink
Updating netlink package to 4b5dce31de6d42af5bb9811c6d265472199e0fec
to fix certain wierd netlink issues seen.

Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
(cherry picked from commit 7948b755c7)

Conflicts:
	hack/vendor.sh
2015-08-25 12:25:37 -04:00
David Calavera
00a27b6872 Fix ignore -q flag in docker ps when there is a default format.
Docker ps default format should not take precedence over cli flags.
This happens effectively for other flags except `-q`.
We need to let the cli to set the format as table to print the
expected output with `-q`.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-08-25 04:32:20 -04:00
Vincent Batts
fc12b9ddce vendor: update tar-split to v0.9.6
Fixes rare edge case of handling GNU LongLink and LongName entries.
Perf improvements. /dev/null writes were taking CPU time during docker
push. Thanks @LK4D4
Various cleanup too.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-08-25 04:32:07 -04:00
Jessie Frazelle
b66e5ef208 Merge pull request #15544 from icecrime/bump_1.8.1
Bump 1.8.1
2015-08-12 20:19:06 -07:00
Arnaud Porterie
d12ea79c9d Update VERSION and CHANGELOG.md
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-08-12 18:54:41 -07:00
Arnaud Porterie
a9aaa66780 Update regression test
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-08-12 18:52:01 -07:00
Arnaud Porterie
e19060dcea Revert #14884
This reverts commit 810d3b2642.

Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-08-12 18:51:57 -07:00
David Calavera
b0e0dbb33b Merge pull request #15091 from calavera/bump_v1.8.0
Bump v1.8.0
2015-08-11 13:16:04 -05:00
47 changed files with 492 additions and 235 deletions

View File

@@ -1,5 +1,38 @@
# Changelog
## 1.8.2 (2015-09-10)
### Distribution:
- Fixes rare edge case of handling GNU LongLink and LongName entries.
- Fix ^C on docker pull.
- Fix docker pull issues on client disconnection.
- Fix issue that caused the daemon to panic when loggers weren't configured properly.
- Fix goroutine leak pulling images from registry V2.
### Runtime:
- Fix a bug mounting cgroups for docker daemons running inside docker containers.
- Initialize log configuration properly.
### Client:
- Handle `-q` flag in `docker ps` properly when there is a default format.
### Networking:
- Fix several corner cases with netlink.
### Contrib:
- Fix several issues with bash completion.
## 1.8.1 (2015-08-12)
### Distribution
- Fix a bug where pushing multiple tags would result in invalid images
## 1.8.0 (2015-08-11)
### Distribution

View File

@@ -127,7 +127,7 @@ RUN git clone https://github.com/golang/lint.git /go/src/github.com/golang/lint
RUN gem install --no-rdoc --no-ri fpm --version 1.3.2
# Install registry
ENV REGISTRY_COMMIT 2317f721a3d8428215a2b65da4ae85212ed473b4
ENV REGISTRY_COMMIT ec87e9b6971d831f0eff752ddb54fb64693e51cd
RUN set -x \
&& export GOPATH="$(mktemp -d)" \
&& git clone https://github.com/docker/distribution.git "$GOPATH/src/github.com/docker/distribution" \

View File

@@ -1 +1 @@
1.8.0
1.8.2

View File

@@ -95,7 +95,7 @@ func (cli *DockerCli) CmdPs(args ...string) error {
f := *format
if len(f) == 0 {
if len(cli.PsFormat()) > 0 {
if len(cli.PsFormat()) > 0 && !*quiet {
f = cli.PsFormat()
} else {
f = "table"

View File

@@ -2,7 +2,7 @@
# THIS FILE IS AUTOGENERATED; SEE "contrib/builder/deb/generate.sh"!
#
FROM ubuntu-debootstrap:precise
FROM ubuntu:precise
RUN apt-get update && apt-get install -y bash-completion build-essential curl ca-certificates debhelper git libapparmor-dev libsqlite3-dev --no-install-recommends && rm -rf /var/lib/apt/lists/*

View File

@@ -2,7 +2,7 @@
# THIS FILE IS AUTOGENERATED; SEE "contrib/builder/deb/generate.sh"!
#
FROM ubuntu-debootstrap:wily
FROM ubuntu:trusty
RUN apt-get update && apt-get install -y bash-completion btrfs-tools build-essential curl ca-certificates debhelper dh-systemd git libapparmor-dev libdevmapper-dev libsqlite3-dev --no-install-recommends && rm -rf /var/lib/apt/lists/*

View File

@@ -2,7 +2,7 @@
# THIS FILE IS AUTOGENERATED; SEE "contrib/builder/deb/generate.sh"!
#
FROM ubuntu-debootstrap:vivid
FROM ubuntu:vivid
RUN apt-get update && apt-get install -y bash-completion btrfs-tools build-essential curl ca-certificates debhelper dh-systemd git libapparmor-dev libdevmapper-dev libsqlite3-dev --no-install-recommends && rm -rf /var/lib/apt/lists/*

View File

@@ -2,7 +2,7 @@
# THIS FILE IS AUTOGENERATED; SEE "contrib/builder/deb/generate.sh"!
#
FROM ubuntu-debootstrap:trusty
FROM ubuntu:wily
RUN apt-get update && apt-get install -y bash-completion btrfs-tools build-essential curl ca-certificates debhelper dh-systemd git libapparmor-dev libdevmapper-dev libsqlite3-dev --no-install-recommends && rm -rf /var/lib/apt/lists/*

View File

@@ -139,7 +139,7 @@ __docker_value_of_option() {
local counter=$((command_pos + 1))
while [ $counter -lt $cword ]; do
case ${words[$counter]} in
$option_glob )
@($option_glob) )
echo ${words[$counter + 1]}
break
;;
@@ -229,11 +229,12 @@ __docker_log_driver_options() {
# see docs/reference/logging/index.md
local fluentd_options="fluentd-address fluentd-tag"
local gelf_options="gelf-address gelf-tag"
local json_file_options="max-file max-size"
local syslog_options="syslog-address syslog-facility syslog-tag"
case $(__docker_value_of_option --log-driver) in
'')
COMPREPLY=( $( compgen -W "$fluentd_options $gelf_options $syslog_options" -S = -- "$cur" ) )
COMPREPLY=( $( compgen -W "$fluentd_options $gelf_options $json_file_options $syslog_options" -S = -- "$cur" ) )
;;
fluentd)
COMPREPLY=( $( compgen -W "$fluentd_options" -S = -- "$cur" ) )
@@ -241,6 +242,9 @@ __docker_log_driver_options() {
gelf)
COMPREPLY=( $( compgen -W "$gelf_options" -S = -- "$cur" ) )
;;
json-file)
COMPREPLY=( $( compgen -W "$json_file_options" -S = -- "$cur" ) )
;;
syslog)
COMPREPLY=( $( compgen -W "$syslog_options" -S = -- "$cur" ) )
;;
@@ -320,7 +324,7 @@ __docker_signals() {
_docker_docker() {
local boolean_options="
$global_boolean_options
--help -h
--help
--version -v
"
@@ -338,8 +342,6 @@ _docker_docker() {
;;
esac
__docker_complete_log_driver_options && return
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "$boolean_options $global_options_with_args" -- "$cur" ) )
@@ -460,7 +462,7 @@ _docker_create() {
_docker_daemon() {
local boolean_options="
$global_boolean_options
--help -h
--help
--icc=false
--ip-forward=false
--ip-masq=false
@@ -512,7 +514,39 @@ _docker_daemon() {
return
;;
--storage-driver|-s)
COMPREPLY=( $( compgen -W "aufs devicemapper btrfs overlay" -- "$(echo $cur | tr '[:upper:]' '[:lower:]')" ) )
COMPREPLY=( $( compgen -W "aufs btrfs devicemapper overlay vfs zfs" -- "$(echo $cur | tr '[:upper:]' '[:lower:]')" ) )
return
;;
--storage-opt)
local devicemapper_options="
dm.basesize
dm.blkdiscard
dm.blocksize
dm.fs
dm.loopdatasize
dm.loopmetadatasize
dm.mkfsarg
dm.mountopt
dm.override_udev_sync_check
dm.thinpooldev
"
local zfs_options="zfs.fsname"
case $(__docker_value_of_option '--storage-driver|-s') in
'')
COMPREPLY=( $( compgen -W "$devicemapper_options $zfs_options" -S = -- "$cur" ) )
;;
devicemapper)
COMPREPLY=( $( compgen -W "$devicemapper_options" -S = -- "$cur" ) )
;;
zfs)
COMPREPLY=( $( compgen -W "$zfs_options" -S = -- "$cur" ) )
;;
*)
return
;;
esac
compopt -o nospace
return
;;
--log-level|-l)
@@ -528,6 +562,27 @@ _docker_daemon() {
;;
esac
__docker_complete_log_driver_options && return
case "${words[$cword-2]}$prev=" in
*dm.blkdiscard=*)
COMPREPLY=( $( compgen -W "false true" -- "${cur#=}" ) )
return
;;
*dm.fs=*)
COMPREPLY=( $( compgen -W "ext4 xfs" -- "${cur#=}" ) )
return
;;
*dm.override_udev_sync_check=*)
COMPREPLY=( $( compgen -W "false true" -- "${cur#=}" ) )
return
;;
*dm.thinpooldev=*)
_filedir
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "$boolean_options $options_with_args" -- "$cur" ) )
@@ -869,7 +924,7 @@ _docker_ps() {
compopt -o nospace
return
;;
-n)
--format|-n)
return
;;
esac
@@ -893,7 +948,7 @@ _docker_ps() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--all -a --before --filter -f --help --latest -l -n --no-trunc --quiet -q --size -s --since" -- "$cur" ) )
COMPREPLY=( $( compgen -W "--all -a --before --filter -f --format --help --latest -l -n --no-trunc --quiet -q --size -s --since" -- "$cur" ) )
;;
esac
}
@@ -998,15 +1053,16 @@ _docker_rmi() {
_docker_run() {
local options_with_args="
--add-host
--blkio-weight
--attach -a
--blkio-weight
--cap-add
--cap-drop
--cgroup-parent
--cidfile
--cpuset
--cpu-period
--cpu-quota
--cpuset-cpus
--cpuset-mems
--cpu-shares -c
--device
--dns
@@ -1018,8 +1074,8 @@ _docker_run() {
--group-add
--hostname -h
--ipc
--label -l
--label-file
--label -l
--link
--log-driver
--log-opt
@@ -1027,14 +1083,15 @@ _docker_run() {
--mac-address
--memory -m
--memory-swap
--memory-swappiness
--name
--net
--pid
--publish -p
--restart
--security-opt
--user -u
--ulimit
--user -u
--uts
--volumes-from
--volume -v
@@ -1042,8 +1099,10 @@ _docker_run() {
"
local all_options="$options_with_args
--disable-content-trust=false
--help
--interactive -i
--oom-kill-disable
--privileged
--publish-all -P
--read-only
@@ -1053,7 +1112,7 @@ _docker_run() {
[ "$command" = "run" ] && all_options="$all_options
--detach -d
--rm
--sig-proxy
--sig-proxy=false
"
local options_with_args_glob=$(__docker_to_extglob "$options_with_args")

View File

@@ -979,7 +979,7 @@ func getDefaultRouteMtu() (int, error) {
return 0, err
}
for _, r := range routes {
if r.Default {
if r.Default && r.Iface != nil {
return r.Iface.MTU, nil
}
}

View File

@@ -76,6 +76,7 @@ func NewDaemonCli() *DaemonCli {
// TODO(tiborvass): remove InstallFlags?
daemonConfig := new(daemon.Config)
daemonConfig.LogConfig.Config = make(map[string]string)
daemonConfig.InstallFlags(daemonFlags, presentInHelp)
daemonConfig.InstallFlags(flag.CommandLine, absentFromHelp)
registryOptions := new(registry.Options)
@@ -208,10 +209,6 @@ func (cli *DaemonCli) CmdDaemon(args ...string) error {
}()
}
if cli.LogConfig.Config == nil {
cli.LogConfig.Config = make(map[string]string)
}
serverConfig := &apiserver.ServerConfig{
Logging: true,
EnableCors: cli.EnableCors,

View File

@@ -28,7 +28,7 @@ func main() {
flag.Merge(flag.CommandLine, clientFlags.FlagSet, commonFlags.FlagSet)
flag.Usage = func() {
fmt.Fprint(os.Stdout, "Usage: docker [OPTIONS] COMMAND [arg...]\n"+daemonUsage+" docker [ -h | --help | -v | --version ]\n\n")
fmt.Fprint(os.Stdout, "Usage: docker [OPTIONS] COMMAND [arg...]\n"+daemonUsage+" docker [ --help | -v | --version ]\n\n")
fmt.Fprint(os.Stdout, "A self-sufficient runtime for containers.\n\nOptions:\n")
flag.CommandLine.SetOutput(os.Stdout)

View File

@@ -16,7 +16,7 @@ or execute `docker help`:
$ docker
Usage: docker [OPTIONS] COMMAND [arg...]
docker daemon [ --help | ... ]
docker [ -h | --help | -v | --version ]
docker [ --help | -v | --version ]
-H, --host=[]: The socket(s) to bind to in daemon mode, specified using one or more tcp://host:port, unix:///path/to/socket, fd://* or fd://socketfd.

View File

@@ -32,7 +32,7 @@ parent = "smn_cli"
-G, --group="docker" Group for the unix socket
-g, --graph="/var/lib/docker" Root of the Docker runtime
-H, --host=[] Daemon socket(s) to connect to
-h, --help=false Print usage
--help=false Print usage
--icc=true Enable inter-container communication
--insecure-registry=[] Enable insecure registry communication
--ip=0.0.0.0 Default IP when binding container ports

View File

@@ -74,16 +74,21 @@ The output will provide details on the container configurations including the
volumes. The output should look something similar to the following:
...
"Volumes": {
"/webapp": "/var/lib/docker/volumes/fac362...80535"
},
"VolumesRW": {
"/webapp": true
}
Mounts": [
{
"Name": "fac362...80535",
"Source": "/var/lib/docker/volumes/fac362...80535/_data",
"Destination": "/webapp",
"Driver": "local",
"Mode": "",
"RW": true
}
]
...
You will notice in the above 'Volumes' is specifying the location on the host and
'VolumesRW' is specifying that the volume is read/write.
You will notice in the above 'Source' is specifying the location on the host and
'Destination' is specifying the volume location inside the container. `RW` shows
if the volume is read/write.
### Mount a host directory as a data volume

View File

@@ -77,7 +77,7 @@ func (p *v2Puller) pullV2Repository(tag string) (err error) {
if err != nil {
if c != nil {
// Another pull of the same repository is already taking place; just wait for it to finish
p.sf.FormatStatus("", "Repository %s already being pulled by another client. Waiting.", p.repoInfo.CanonicalName)
p.config.OutStream.Write(p.sf.FormatStatus("", "Repository %s already being pulled by another client. Waiting.", p.repoInfo.CanonicalName))
<-c
return nil
}
@@ -140,9 +140,9 @@ func (p *v2Puller) download(di *downloadInfo) {
return
}
blobs := p.repo.Blobs(nil)
blobs := p.repo.Blobs(context.Background())
desc, err := blobs.Stat(nil, di.digest)
desc, err := blobs.Stat(context.Background(), di.digest)
if err != nil {
logrus.Debugf("Error statting layer: %v", err)
di.err <- err
@@ -150,7 +150,7 @@ func (p *v2Puller) download(di *downloadInfo) {
}
di.size = desc.Size
layerDownload, err := blobs.Open(nil, di.digest)
layerDownload, err := blobs.Open(context.Background(), di.digest)
if err != nil {
logrus.Debugf("Error fetching layer: %v", err)
di.err <- err
@@ -223,6 +223,9 @@ func (p *v2Puller) pullV2Tag(tag, taggedName string) (verified bool, err error)
go func() {
if _, err := io.Copy(out, pipeReader); err != nil {
logrus.Errorf("error copying from layer download progress reader: %s", err)
if err := pipeReader.CloseWithError(err); err != nil {
logrus.Errorf("error closing the progress reader: %s", err)
}
}
}()
defer func() {
@@ -232,6 +235,9 @@ func (p *v2Puller) pullV2Tag(tag, taggedName string) (verified bool, err error)
// set the error. All successive reads/writes will return with this
// error.
pipeWriter.CloseWithError(errors.New("download canceled"))
} else {
// If no error then just close the pipe.
pipeWriter.Close()
}
}()

View File

@@ -29,13 +29,12 @@ func (s *TagStore) NewPusher(endpoint registry.APIEndpoint, localRepo Repository
switch endpoint.Version {
case registry.APIVersion2:
return &v2Pusher{
TagStore: s,
endpoint: endpoint,
localRepo: localRepo,
repoInfo: repoInfo,
config: imagePushConfig,
sf: sf,
layersSeen: make(map[string]bool),
TagStore: s,
endpoint: endpoint,
localRepo: localRepo,
repoInfo: repoInfo,
config: imagePushConfig,
sf: sf,
}, nil
case registry.APIVersion1:
return &v1Pusher{

View File

@@ -27,11 +27,6 @@ type v2Pusher struct {
config *ImagePushConfig
sf *streamformatter.StreamFormatter
repo distribution.Repository
// layersSeen is the set of layers known to exist on the remote side.
// This avoids redundant queries when pushing multiple tags that
// involve the same layers.
layersSeen map[string]bool
}
func (p *v2Pusher) Push() (fallback bool, err error) {
@@ -92,6 +87,8 @@ func (p *v2Pusher) pushV2Tag(tag string) error {
return fmt.Errorf("tag does not exist: %s", tag)
}
layersSeen := make(map[string]bool)
layer, err := p.graph.Get(layerId)
if err != nil {
return err
@@ -120,7 +117,7 @@ func (p *v2Pusher) pushV2Tag(tag string) error {
return err
}
if p.layersSeen[layer.ID] {
if layersSeen[layer.ID] {
break
}
@@ -141,7 +138,7 @@ func (p *v2Pusher) pushV2Tag(tag string) error {
dgst, err := p.graph.GetDigest(layer.ID)
switch err {
case nil:
_, err := p.repo.Blobs(nil).Stat(nil, dgst)
_, err := p.repo.Blobs(context.Background()).Stat(context.Background(), dgst)
switch err {
case nil:
exists = true
@@ -161,7 +158,7 @@ func (p *v2Pusher) pushV2Tag(tag string) error {
// if digest was empty or not saved, or if blob does not exist on the remote repository,
// then fetch it.
if !exists {
if pushDigest, err := p.pushV2Image(p.repo.Blobs(nil), layer); err != nil {
if pushDigest, err := p.pushV2Image(p.repo.Blobs(context.Background()), layer); err != nil {
return err
} else if pushDigest != dgst {
// Cache new checksum
@@ -175,7 +172,7 @@ func (p *v2Pusher) pushV2Tag(tag string) error {
m.FSLayers = append(m.FSLayers, manifest.FSLayer{BlobSum: dgst})
m.History = append(m.History, manifest.History{V1Compatibility: string(jsonData)})
p.layersSeen[layer.ID] = true
layersSeen[layer.ID] = true
}
logrus.Infof("Signed manifest for %s:%s using daemon's key: %s", p.repo.Name(), tag, p.trustKey.KeyID())
@@ -229,7 +226,7 @@ func (p *v2Pusher) pushV2Image(bs distribution.BlobService, img *image.Image) (d
// Send the layer
logrus.Debugf("rendered layer for %s of [%d] size", img.ID, size)
layerUpload, err := bs.Create(nil)
layerUpload, err := bs.Create(context.Background())
if err != nil {
return "", err
}
@@ -253,7 +250,7 @@ func (p *v2Pusher) pushV2Image(bs distribution.BlobService, img *image.Image) (d
}
desc := distribution.Descriptor{Digest: dgst}
if _, err := layerUpload.Commit(nil, desc); err != nil {
if _, err := layerUpload.Commit(context.Background(), desc); err != nil {
return "", err
}

View File

@@ -20,21 +20,70 @@ APTDIR=$DOCKER_RELEASE_DIR/apt/repo
# setup the apt repo (if it does not exist)
mkdir -p "$APTDIR/conf" "$APTDIR/db"
# supported arches/sections
arches=( amd64 i386 )
components=( main testing experimental )
# create/update distributions file
if [[ ! -f "$APTDIR/conf/distributions" ]]; then
if [ ! -f "$APTDIR/conf/distributions" ]; then
for suite in $(exec contrib/reprepro/suites.sh); do
cat <<-EOF
Origin: Docker
Suite: $suite
Codename: $suite
Architectures: amd64 i386
Components: main testing experimental
Architectures: ${arches[*]}
Components: ${components[*]}
Description: Docker APT Repository
EOF
done > "$APTDIR/conf/distributions"
fi
# create/update distributions file
if [ ! -f "$APTDIR/conf/apt-ftparchive.conf" ]; then
cat <<-EOF > "$APTDIR/conf/apt-ftparchive.conf"
Dir {
ArchiveDir "${APTDIR}";
CacheDir "${APTDIR}/db";
};
Default {
Packages::Compress ". gzip bzip2";
Sources::Compress ". gzip bzip2";
Contents::Compress ". gzip bzip2";
};
TreeDefault {
BinCacheDB "packages-\$(SECTION)-\$(ARCH).db";
Directory "pool/\$(SECTION)";
Packages "\$(DIST)/\$(SECTION)/binary-\$(ARCH)/Packages";
SrcDirectory "pool/\$(SECTION)";
Sources "\$(DIST)/\$(SECTION)/source/Sources";
Contents "\$(DIST)/\$(SECTION)/Contents-\$(ARCH)";
FileList "$APTDIR/\$(DIST)/\$(SECTION)/filelist";
};
EOF
for suite in $(exec contrib/reprepro/suites.sh); do
cat <<-EOF
Tree "dists/${suite}" {
Sections "main testing experimental";
Architectures "${arches[*]}";
}
EOF
done >> "$APTDIR/conf/apt-ftparchive.conf"
fi
if [ ! -f "$APTDIR/conf/docker-engine-release.conf" ]; then
cat <<-EOF > "$APTDIR/conf/docker-engine-release.conf"
APT::FTPArchive::Release::Origin "Docker";
APT::FTPArchive::Release::Components "${components[*]}";
APT::FTPArchive::Release::Label "Docker APT Repository";
APT::FTPArchive::Release::Architectures "${arches[*]}";
EOF
fi
# set the component and priority for the version being released
component="main"
priority=700
@@ -67,4 +116,35 @@ for dir in contrib/builder/deb/*/; do
reprepro -v --keepunreferencedfiles \
-S docker-engine -P "$priority" -C "$component" \
-b "$APTDIR" includedeb "$codename" "${DEBFILE[@]}"
# update the filelist for this codename/component
find "$APTDIR/pool/$component" \
-name *~${codename#*-}*.deb > "$APTDIR/dists/$codename/$component/filelist"
done
# run the apt-ftparchive commands so we can have pinning
apt-ftparchive generate "$APTDIR/conf/apt-ftparchive.conf"
for dir in contrib/builder/deb/*/; do
version="$(basename "$dir")"
codename="${version//debootstrap-}"
apt-ftparchive \
-o "APT::FTPArchive::Release::Codename=$codename" \
-o "APT::FTPArchive::Release::Suite=$codename" \
-c "$APTDIR/conf/docker-engine-release.conf" \
release \
"$APTDIR/dists/$codename" > "$APTDIR/dists/$codename/Release"
for arch in "${arches[@]}"; do
apt-ftparchive \
-o "APT::FTPArchive::Release::Codename=$codename" \
-o "APT::FTPArchive::Release::Suite=$codename" \
-o "APT::FTPArchive::Release::Component=$component" \
-o "APT::FTPArchive::Release::Architecture=$arch" \
-c "$APTDIR/conf/docker-engine-release.conf" \
release \
"$APTDIR/dists/$codename/$component/binary-$arch" > "$APTDIR/dists/$codename/$component/binary-$arch/Release"
done
done

View File

@@ -21,29 +21,29 @@ clone git golang.org/x/net 3cffabab72adf04f8e3b01c5baf775361837b5fe https://gith
clone hg code.google.com/p/gosqlite 74691fb6f837
#get libnetwork packages
clone git github.com/docker/libnetwork bd3eecc96f3c05a4acef1bedcf74397bc6850d22
clone git github.com/docker/libnetwork bc565c2d295067c1a43674a23a473ec6336d7fd4
clone git github.com/armon/go-metrics eb0af217e5e9747e41dd5303755356b62d28e3ec
clone git github.com/hashicorp/go-msgpack 71c2886f5a673a35f909803f38ece5810165097b
clone git github.com/hashicorp/memberlist 9a1e242e454d2443df330bdd51a436d5a9058fc4
clone git github.com/hashicorp/serf 7151adcef72687bf95f451a2e0ba15cb19412bf2
clone git github.com/docker/libkv 60c7c881345b3c67defc7f93a8297debf041d43c
clone git github.com/vishvananda/netns 493029407eeb434d0c2d44e02ea072ff2488d322
clone git github.com/vishvananda/netlink 20397a138846e4d6590e01783ed023ed7e1c38a6
clone git github.com/vishvananda/netlink 4b5dce31de6d42af5bb9811c6d265472199e0fec
clone git github.com/BurntSushi/toml f706d00e3de6abe700c994cdd545a1a4915af060
clone git github.com/samuel/go-zookeeper d0e0d8e11f318e000a8cc434616d69e329edc374
clone git github.com/coreos/go-etcd v2.0.0
clone git github.com/hashicorp/consul v0.5.2
# get graph and distribution packages
clone git github.com/docker/distribution 7dc8d4a26b689bd4892f2f2322dbce0b7119d686
clone git github.com/vbatts/tar-split v0.9.4
clone git github.com/docker/distribution ec87e9b6971d831f0eff752ddb54fb64693e51cd # docker/1.8 branch
clone git github.com/vbatts/tar-split v0.9.6
clone git github.com/docker/notary 8e8122eb5528f621afcd4e2854c47302f17392f7
clone git github.com/endophage/gotuf a592b03b28b02bb29bb5878308fb1abed63383b5
clone git github.com/tent/canonical-json-go 96e4ba3a7613a1216cbd1badca4efe382adea337
clone git github.com/agl/ed25519 d2b94fd789ea21d12fac1a4443dd3a3f79cda72c
clone git github.com/opencontainers/runc v0.0.2 # libcontainer
clone git github.com/opencontainers/runc v0.0.2.1 # libcontainer
# libcontainer deps (see src/github.com/docker/libcontainer/update-vendor.sh)
clone git github.com/coreos/go-systemd v2
clone git github.com/godbus/dbus v2

View File

@@ -2,7 +2,10 @@ package main
import (
"fmt"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"reflect"
"strconv"
"strings"
@@ -554,3 +557,23 @@ func (s *DockerSuite) TestPsFormatHeaders(c *check.C) {
c.Fatalf(`Expected 'NAMES\ntest\n', got %v`, out)
}
}
func (s *DockerSuite) TestPsDefaultFormatAndQuiet(c *check.C) {
config := `{
"psFormat": "{{ .ID }} default"
}`
d, err := ioutil.TempDir("", "integration-cli-")
c.Assert(err, check.IsNil)
defer os.RemoveAll(d)
err = ioutil.WriteFile(filepath.Join(d, "config.json"), []byte(config), 0644)
c.Assert(err, check.IsNil)
out, _ := dockerCmd(c, "run", "--name=test", "-d", "busybox", "top")
id := strings.TrimSpace(out)
out, _ = dockerCmd(c, "--config", d, "ps", "-q")
if !strings.HasPrefix(id, strings.TrimSpace(out)) {
c.Fatalf("Expected to print only the container id, got %v\n", out)
}
}

View File

@@ -369,3 +369,40 @@ func (s *DockerTrustSuite) TestTrustedPullWithExpiredSnapshot(c *check.C) {
}
})
}
// Test that pull continues after client has disconnected. #15589
func (s *DockerTrustSuite) TestPullClientDisconnect(c *check.C) {
testRequires(c, Network)
repoName := "hello-world:latest"
dockerCmdWithError(c, "rmi", repoName) // clean just in case
pullCmd := exec.Command(dockerBinary, "pull", repoName)
stdout, err := pullCmd.StdoutPipe()
c.Assert(err, check.IsNil)
err = pullCmd.Start()
c.Assert(err, check.IsNil)
// cancel as soon as we get some output
buf := make([]byte, 10)
_, err = stdout.Read(buf)
c.Assert(err, check.IsNil)
err = pullCmd.Process.Kill()
c.Assert(err, check.IsNil)
maxAttempts := 20
for i := 0; ; i++ {
if _, _, err := dockerCmdWithError(c, "inspect", repoName); err == nil {
break
}
if i >= maxAttempts {
c.Fatal("Timeout reached. Image was not pulled after client disconnected.")
}
time.Sleep(500 * time.Millisecond)
}
}

View File

@@ -60,31 +60,40 @@ func (s *DockerRegistrySuite) TestPushMultipleTags(c *check.C) {
dockerCmd(c, "tag", "busybox", repoTag2)
out, _ := dockerCmd(c, "push", repoName)
dockerCmd(c, "push", repoName)
// There should be no duplicate hashes in the output
imageSuccessfullyPushed := ": Image successfully pushed"
// Ensure layer list is equivalent for repoTag1 and repoTag2
out1, _ := dockerCmd(c, "pull", repoTag1)
if strings.Contains(out1, "Tag t1 not found") {
c.Fatalf("Unable to pull pushed image: %s", out1)
}
imageAlreadyExists := ": Image already exists"
imagePushHashes := make(map[string]struct{})
outputLines := strings.Split(out, "\n")
for _, outputLine := range outputLines {
if strings.Contains(outputLine, imageSuccessfullyPushed) {
hash := strings.TrimSuffix(outputLine, imageSuccessfullyPushed)
if _, present := imagePushHashes[hash]; present {
c.Fatalf("Duplicate image push: %s", outputLine)
}
imagePushHashes[hash] = struct{}{}
} else if strings.Contains(outputLine, imageAlreadyExists) {
hash := strings.TrimSuffix(outputLine, imageAlreadyExists)
if _, present := imagePushHashes[hash]; present {
c.Fatalf("Duplicate image push: %s", outputLine)
}
imagePushHashes[hash] = struct{}{}
var out1Lines []string
for _, outputLine := range strings.Split(out1, "\n") {
if strings.Contains(outputLine, imageAlreadyExists) {
out1Lines = append(out1Lines, outputLine)
}
}
if len(imagePushHashes) == 0 {
c.Fatal(`Expected at least one line containing "Image successfully pushed"`)
out2, _ := dockerCmd(c, "pull", repoTag2)
if strings.Contains(out2, "Tag t2 not found") {
c.Fatalf("Unable to pull pushed image: %s", out1)
}
var out2Lines []string
for _, outputLine := range strings.Split(out2, "\n") {
if strings.Contains(outputLine, imageAlreadyExists) {
out1Lines = append(out1Lines, outputLine)
}
}
if len(out1Lines) != len(out2Lines) {
c.Fatalf("Mismatched output length:\n%s\n%s", out1, out2)
}
for i := range out1Lines {
if out1Lines[i] != out2Lines[i] {
c.Fatalf("Mismatched output line:\n%s\n%s", out1Lines[i], out2Lines[i])
}
}
}

View File

@@ -23,7 +23,7 @@ its own man page which explain usage and arguments.
To see the man page for a command run **man docker <command>**.
# OPTIONS
**-h**, **--help**
**--help**
Print usage statement
**--api-cors-header**=""

View File

@@ -359,25 +359,18 @@ type blobs struct {
distribution.BlobDeleter
}
func sanitizeLocation(location, source string) (string, error) {
func sanitizeLocation(location, base string) (string, error) {
baseURL, err := url.Parse(base)
if err != nil {
return "", err
}
locationURL, err := url.Parse(location)
if err != nil {
return "", err
}
if locationURL.Scheme == "" {
sourceURL, err := url.Parse(source)
if err != nil {
return "", err
}
locationURL = &url.URL{
Scheme: sourceURL.Scheme,
Host: sourceURL.Host,
Path: location,
}
location = locationURL.String()
}
return location, nil
return baseURL.ResolveReference(locationURL).String(), nil
}
func (bs *blobs) Stat(ctx context.Context, dgst digest.Digest) (distribution.Descriptor, error) {

View File

@@ -19,6 +19,7 @@ import (
"github.com/docker/libnetwork/netutils"
"github.com/docker/libnetwork/options"
"github.com/docker/libnetwork/portmapper"
"github.com/docker/libnetwork/sandbox"
"github.com/docker/libnetwork/types"
"github.com/vishvananda/netlink"
)
@@ -544,6 +545,8 @@ func (d *driver) getNetworks() []*bridgeNetwork {
func (d *driver) CreateNetwork(id types.UUID, option map[string]interface{}) error {
var err error
defer sandbox.InitOSContext()()
// Sanity checks
d.Lock()
if _, ok := d.networks[id]; ok {
@@ -695,6 +698,8 @@ func (d *driver) CreateNetwork(id types.UUID, option map[string]interface{}) err
func (d *driver) DeleteNetwork(nid types.UUID) error {
var err error
defer sandbox.InitOSContext()()
// Get network handler and remove it from driver
d.Lock()
n, ok := d.networks[nid]
@@ -822,6 +827,8 @@ func (d *driver) CreateEndpoint(nid, eid types.UUID, epInfo driverapi.EndpointIn
err error
)
defer sandbox.InitOSContext()()
if epInfo == nil {
return errors.New("invalid endpoint info passed")
}
@@ -1029,6 +1036,8 @@ func (d *driver) CreateEndpoint(nid, eid types.UUID, epInfo driverapi.EndpointIn
func (d *driver) DeleteEndpoint(nid, eid types.UUID) error {
var err error
defer sandbox.InitOSContext()()
// Get the network handler and make sure it exists
d.Lock()
n, ok := d.networks[nid]
@@ -1168,6 +1177,8 @@ func (d *driver) EndpointOperInfo(nid, eid types.UUID) (map[string]interface{},
// Join method is invoked when a Sandbox is attached to an endpoint.
func (d *driver) Join(nid, eid types.UUID, sboxKey string, jinfo driverapi.JoinInfo, options map[string]interface{}) error {
defer sandbox.InitOSContext()()
network, err := d.getNetwork(nid)
if err != nil {
return err
@@ -1211,6 +1222,8 @@ func (d *driver) Join(nid, eid types.UUID, sboxKey string, jinfo driverapi.JoinI
// Leave method is invoked when a Sandbox detaches from an endpoint.
func (d *driver) Leave(nid, eid types.UUID) error {
defer sandbox.InitOSContext()()
network, err := d.getNetwork(nid)
if err != nil {
return err

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"github.com/docker/libnetwork/netutils"
"github.com/docker/libnetwork/sandbox"
"github.com/docker/libnetwork/types"
"github.com/vishvananda/netlink"
)
@@ -21,6 +22,8 @@ func validateID(nid, eid types.UUID) error {
}
func createVethPair() (string, string, error) {
defer sandbox.InitOSContext()()
// Generate a name for what will be the host side pipe interface
name1, err := netutils.GenerateIfaceName(vethPrefix, vethLen)
if err != nil {
@@ -45,6 +48,8 @@ func createVethPair() (string, string, error) {
}
func createVxlan(vni uint32) (string, error) {
defer sandbox.InitOSContext()()
name, err := netutils.GenerateIfaceName("vxlan", 7)
if err != nil {
return "", fmt.Errorf("error generating vxlan name: %v", err)
@@ -68,6 +73,8 @@ func createVxlan(vni uint32) (string, error) {
}
func deleteVxlan(name string) error {
defer sandbox.InitOSContext()()
link, err := netlink.LinkByName(name)
if err != nil {
return fmt.Errorf("failed to find vxlan interface with name %s: %v", name, err)

View File

@@ -1,41 +0,0 @@
package netutils
import (
"flag"
"runtime"
"syscall"
"testing"
)
var runningInContainer = flag.Bool("incontainer", false, "Indicates if the test is running in a container")
// IsRunningInContainer returns whether the test is running inside a container.
func IsRunningInContainer() bool {
return (*runningInContainer)
}
// SetupTestNetNS joins a new network namespace, and returns its associated
// teardown function.
//
// Example usage:
//
// defer SetupTestNetNS(t)()
//
func SetupTestNetNS(t *testing.T) func() {
runtime.LockOSThread()
if err := syscall.Unshare(syscall.CLONE_NEWNET); err != nil {
t.Fatalf("Failed to enter netns: %v", err)
}
fd, err := syscall.Open("/proc/self/ns/net", syscall.O_RDONLY, 0)
if err != nil {
t.Fatal("Failed to open netns file")
}
return func() {
if err := syscall.Close(fd); err != nil {
t.Logf("Warning: netns closing failed (%v)", err)
}
runtime.UnlockOSThread()
}
}

View File

@@ -26,6 +26,8 @@ var (
gpmWg sync.WaitGroup
gpmCleanupPeriod = 60 * time.Second
gpmChan = make(chan chan struct{})
nsOnce sync.Once
initNs netns.NsHandle
)
// The networkNamespace type is the linux implementation of the Sandbox
@@ -242,15 +244,37 @@ func (n *networkNamespace) InvokeFunc(f func()) error {
})
}
func nsInvoke(path string, prefunc func(nsFD int) error, postfunc func(callerFD int) error) error {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
func getLink() (string, error) {
return os.Readlink(fmt.Sprintf("/proc/%d/task/%d/ns/net", os.Getpid(), syscall.Gettid()))
}
origns, err := netns.Get()
if err != nil {
return err
func nsInit() {
var err error
if initNs, err = netns.Get(); err != nil {
log.Errorf("could not get initial namespace: %v", err)
}
defer origns.Close()
}
// InitOSContext initializes OS context while configuring network resources
func InitOSContext() func() {
runtime.LockOSThread()
nsOnce.Do(nsInit)
if err := netns.Set(initNs); err != nil {
linkInfo, linkErr := getLink()
if linkErr != nil {
linkInfo = linkErr.Error()
}
log.Errorf("failed to set to initial namespace, %v, initns fd %d: %v",
linkInfo, initNs, err)
}
return runtime.UnlockOSThread
}
func nsInvoke(path string, prefunc func(nsFD int) error, postfunc func(callerFD int) error) error {
defer InitOSContext()()
f, err := os.OpenFile(path, os.O_RDONLY, 0)
if err != nil {
@@ -269,10 +293,10 @@ func nsInvoke(path string, prefunc func(nsFD int) error, postfunc func(callerFD
if err = netns.Set(netns.NsHandle(nsFD)); err != nil {
return err
}
defer netns.Set(origns)
defer netns.Set(initNs)
// Invoked after the namespace switch.
return postfunc(int(origns))
return postfunc(int(initNs))
}
func (n *networkNamespace) nsPath() string {

View File

@@ -21,3 +21,8 @@ func NewSandbox(key string, osCreate bool) (Sandbox, error) {
// and waits for it.
func GC() {
}
// InitOSContext initializes OS context while configuring network resources
func InitOSContext() func() {
return func() {}
}

View File

@@ -21,3 +21,8 @@ func NewSandbox(key string, osCreate bool) (Sandbox, error) {
// and waits for it.
func GC() {
}
// InitOSContext initializes OS context while configuring network resources
func InitOSContext() func() {
return func() {}
}

View File

@@ -3,6 +3,7 @@ package fluent
import (
"errors"
"fmt"
"io"
"math"
"net"
"reflect"
@@ -33,7 +34,7 @@ type Config struct {
type Fluent struct {
Config
conn net.Conn
conn io.WriteCloser
pending []byte
reconnecting bool
mu sync.Mutex

View File

@@ -1,3 +1,3 @@
package fluent
const Version = "0.5.1"
const Version = "1.0.0"

View File

@@ -236,7 +236,7 @@ func getCgroupData(c *configs.Cgroup, pid int) (*data, error) {
}
func (raw *data) parent(subsystem, mountpoint, src string) (string, error) {
initPath, err := cgroups.GetInitCgroupDir(subsystem)
initPath, err := cgroups.GetThisCgroupDir(subsystem)
if err != nil {
return "", err
}

View File

@@ -159,17 +159,19 @@ func (tr *Reader) Next() (*Header, error) {
if err != nil {
return nil, err
}
var b []byte
var buf []byte
if tr.RawAccounting {
if _, err = tr.rawBytes.Write(realname); err != nil {
return nil, err
}
b = tr.RawBytes()
buf = make([]byte, tr.rawBytes.Len())
copy(buf[:], tr.RawBytes())
}
hdr, err := tr.Next()
// since the above call to Next() resets the buffer, we need to throw the bytes over
if tr.RawAccounting {
if _, err = tr.rawBytes.Write(b); err != nil {
buf = append(buf, tr.RawBytes()...)
if _, err = tr.rawBytes.Write(buf); err != nil {
return nil, err
}
}
@@ -181,17 +183,19 @@ func (tr *Reader) Next() (*Header, error) {
if err != nil {
return nil, err
}
var b []byte
var buf []byte
if tr.RawAccounting {
if _, err = tr.rawBytes.Write(realname); err != nil {
return nil, err
}
b = tr.RawBytes()
buf = make([]byte, tr.rawBytes.Len())
copy(buf[:], tr.RawBytes())
}
hdr, err := tr.Next()
// since the above call to Next() resets the buffer, we need to throw the bytes over
if tr.RawAccounting {
if _, err = tr.rawBytes.Write(b); err != nil {
buf = append(buf, tr.RawBytes()...)
if _, err = tr.rawBytes.Write(buf); err != nil {
return nil, err
}
}

View File

@@ -9,7 +9,7 @@ import (
"github.com/vbatts/tar-split/tar/storage"
)
// NewOutputTarStream returns an io.ReadCloser that is an assemble tar archive
// NewOutputTarStream returns an io.ReadCloser that is an assembled tar archive
// stream.
//
// It takes a storage.FileGetter, for mapping the file payloads that are to be read in,
@@ -62,7 +62,6 @@ func NewOutputTarStream(fg storage.FileGetter, up storage.Unpacker) io.ReadClose
fh.Close()
}
}
pw.Close()
}()
return pr
}

View File

@@ -22,8 +22,8 @@ func NewInputTarStream(r io.Reader, p storage.Packer, fp storage.FilePutter) (io
// What to do here... folks will want their own access to the Reader that is
// their tar archive stream, but we'll need that same stream to use our
// forked 'archive/tar'.
// Perhaps do an io.TeeReader that hand back an io.Reader for them to read
// from, and we'll mitm the stream to store metadata.
// Perhaps do an io.TeeReader that hands back an io.Reader for them to read
// from, and we'll MITM the stream to store metadata.
// We'll need a storage.FilePutter too ...
// Another concern, whether to do any storage.FilePutter operations, such that we
@@ -32,7 +32,7 @@ func NewInputTarStream(r io.Reader, p storage.Packer, fp storage.FilePutter) (io
// Perhaps we have a DiscardFilePutter that is a bit bucket.
// we'll return the pipe reader, since TeeReader does not buffer and will
// only read what the outputRdr Read's. Since Tar archive's have padding on
// only read what the outputRdr Read's. Since Tar archives have padding on
// the end, we want to be the one reading the padding, even if the user's
// `archive/tar` doesn't care.
pR, pW := io.Pipe()
@@ -55,13 +55,15 @@ func NewInputTarStream(r io.Reader, p storage.Packer, fp storage.FilePutter) (io
}
// even when an EOF is reached, there is often 1024 null bytes on
// the end of an archive. Collect them too.
_, err := p.AddEntry(storage.Entry{
Type: storage.SegmentType,
Payload: tr.RawBytes(),
})
if err != nil {
pW.CloseWithError(err)
return
if b := tr.RawBytes(); len(b) > 0 {
_, err := p.AddEntry(storage.Entry{
Type: storage.SegmentType,
Payload: b,
})
if err != nil {
pW.CloseWithError(err)
return
}
}
break // not return. We need the end of the reader.
}
@@ -69,12 +71,15 @@ func NewInputTarStream(r io.Reader, p storage.Packer, fp storage.FilePutter) (io
break // not return. We need the end of the reader.
}
if _, err := p.AddEntry(storage.Entry{
Type: storage.SegmentType,
Payload: tr.RawBytes(),
}); err != nil {
pW.CloseWithError(err)
return
if b := tr.RawBytes(); len(b) > 0 {
_, err := p.AddEntry(storage.Entry{
Type: storage.SegmentType,
Payload: b,
})
if err != nil {
pW.CloseWithError(err)
return
}
}
var csum []byte

View File

@@ -5,7 +5,7 @@ Packing and unpacking the Entries of the stream. The types of streams are
either segments of raw bytes (for the raw headers and various padding) and for
an entry marking a file payload.
The raw bytes are stored precisely in the packed (marshalled) Entry. Where as
The raw bytes are stored precisely in the packed (marshalled) Entry, whereas
the file payload marker include the name of the file, size, and crc64 checksum
(for basic file integrity).
*/

View File

@@ -19,11 +19,11 @@ const (
// SegmentType represents a raw bytes segment from the archive stream. These raw
// byte segments consist of the raw headers and various padding.
//
// It's payload is to be marshalled base64 encoded.
// Its payload is to be marshalled base64 encoded.
SegmentType
)
// Entry is a the structure for packing and unpacking the information read from
// Entry is the structure for packing and unpacking the information read from
// the Tar archive.
//
// FileType Payload checksum is using `hash/crc64` for basic file integrity,
@@ -32,8 +32,8 @@ const (
// collisions in a sample of 18.2 million, CRC64 had none.
type Entry struct {
Type Type `json:"type"`
Name string `json:"name",omitempty`
Size int64 `json:"size",omitempty`
Payload []byte `json:"payload"` // SegmentType store payload here; FileType store crc64 checksum here;
Name string `json:"name,omitempty"`
Size int64 `json:"size,omitempty"`
Payload []byte `json:"payload"` // SegmentType stores payload here; FileType stores crc64 checksum here;
Position int `json:"position"`
}

View File

@@ -5,14 +5,13 @@ import (
"errors"
"hash/crc64"
"io"
"io/ioutil"
"os"
"path/filepath"
)
// FileGetter is the interface for getting a stream of a file payload, address
// by name/filename. Presumably, the names will be scoped to relative file
// paths.
// FileGetter is the interface for getting a stream of a file payload,
// addressed by name/filename. Presumably, the names will be scoped to relative
// file paths.
type FileGetter interface {
// Get returns a stream for the provided file path
Get(filename string) (output io.ReadCloser, err error)
@@ -60,15 +59,15 @@ func (bfgp bufferFileGetPutter) Get(name string) (io.ReadCloser, error) {
}
func (bfgp *bufferFileGetPutter) Put(name string, r io.Reader) (int64, []byte, error) {
c := crc64.New(CRCTable)
tRdr := io.TeeReader(r, c)
b := bytes.NewBuffer([]byte{})
i, err := io.Copy(b, tRdr)
crc := crc64.New(CRCTable)
buf := bytes.NewBuffer(nil)
cw := io.MultiWriter(crc, buf)
i, err := io.Copy(cw, r)
if err != nil {
return 0, nil, err
}
bfgp.files[name] = b.Bytes()
return i, c.Sum(nil), nil
bfgp.files[name] = buf.Bytes()
return i, crc.Sum(nil), nil
}
type readCloserWrapper struct {
@@ -77,7 +76,7 @@ type readCloserWrapper struct {
func (w *readCloserWrapper) Close() error { return nil }
// NewBufferFileGetPutter is simple in memory FileGetPutter
// NewBufferFileGetPutter is a simple in-memory FileGetPutter
//
// Implication is this is memory intensive...
// Probably best for testing or light weight cases.
@@ -97,8 +96,7 @@ type bitBucketFilePutter struct {
func (bbfp *bitBucketFilePutter) Put(name string, r io.Reader) (int64, []byte, error) {
c := crc64.New(CRCTable)
tRdr := io.TeeReader(r, c)
i, err := io.Copy(ioutil.Discard, tRdr)
i, err := io.Copy(c, r)
return i, c.Sum(nil), err
}

View File

@@ -8,8 +8,8 @@ import (
"path/filepath"
)
// ErrDuplicatePath is occured when a tar archive has more than one entry for
// the same file path
// ErrDuplicatePath occurs when a tar archive has more than one entry for the
// same file path
var ErrDuplicatePath = errors.New("duplicates of file paths not supported")
// Packer describes the methods to pack Entries to a storage destination
@@ -65,7 +65,7 @@ func (jup *jsonUnpacker) Next() (*Entry, error) {
if _, ok := jup.seen[cName]; ok {
return nil, ErrDuplicatePath
}
jup.seen[cName] = emptyByte
jup.seen[cName] = struct{}{}
}
return &e, err
@@ -90,11 +90,7 @@ type jsonPacker struct {
seen seenNames
}
type seenNames map[string]byte
// used in the seenNames map. byte is a uint8, and we'll re-use the same one
// for minimalism.
const emptyByte byte = 0
type seenNames map[string]struct{}
func (jp *jsonPacker) AddEntry(e Entry) (int, error) {
// check early for dup name
@@ -103,7 +99,7 @@ func (jp *jsonPacker) AddEntry(e Entry) (int, error) {
if _, ok := jp.seen[cName]; ok {
return -1, ErrDuplicatePath
}
jp.seen[cName] = emptyByte
jp.seen[cName] = struct{}{}
}
e.Position = jp.pos
@@ -117,7 +113,7 @@ func (jp *jsonPacker) AddEntry(e Entry) (int, error) {
return e.Position, nil
}
// NewJSONPacker provides an Packer that writes each Entry (SegmentType and
// NewJSONPacker provides a Packer that writes each Entry (SegmentType and
// FileType) as a json document.
//
// The Entries are delimited by new line.

View File

@@ -157,6 +157,7 @@ type Vxlan struct {
L2miss bool
L3miss bool
NoAge bool
GBP bool
Age int
Limit int
Port int

View File

@@ -73,10 +73,7 @@ func LinkSetMTU(link Link, mtu int) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_UNSPEC)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = syscall.IFLA_MTU
req.AddData(msg)
b := make([]byte, 4)
@@ -97,10 +94,7 @@ func LinkSetName(link Link, name string) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_UNSPEC)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = syscall.IFLA_IFNAME
req.AddData(msg)
data := nl.NewRtAttr(syscall.IFLA_IFNAME, []byte(name))
@@ -118,10 +112,7 @@ func LinkSetHardwareAddr(link Link, hwaddr net.HardwareAddr) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_UNSPEC)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = syscall.IFLA_ADDRESS
req.AddData(msg)
data := nl.NewRtAttr(syscall.IFLA_ADDRESS, []byte(hwaddr))
@@ -151,10 +142,7 @@ func LinkSetMasterByIndex(link Link, masterIndex int) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_UNSPEC)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = syscall.IFLA_MASTER
req.AddData(msg)
b := make([]byte, 4)
@@ -176,10 +164,7 @@ func LinkSetNsPid(link Link, nspid int) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_UNSPEC)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = syscall.IFLA_NET_NS_PID
req.AddData(msg)
b := make([]byte, 4)
@@ -201,10 +186,7 @@ func LinkSetNsFd(link Link, fd int) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_UNSPEC)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = nl.IFLA_NET_NS_FD
req.AddData(msg)
b := make([]byte, 4)
@@ -266,6 +248,10 @@ func addVxlanAttrs(vxlan *Vxlan, linkInfo *nl.RtAttr) {
nl.NewRtAttrChild(data, nl.IFLA_VXLAN_L2MISS, boolAttr(vxlan.L2miss))
nl.NewRtAttrChild(data, nl.IFLA_VXLAN_L3MISS, boolAttr(vxlan.L3miss))
if vxlan.GBP {
nl.NewRtAttrChild(data, nl.IFLA_VXLAN_GBP, boolAttr(vxlan.GBP))
}
if vxlan.NoAge {
nl.NewRtAttrChild(data, nl.IFLA_VXLAN_AGEING, nl.Uint32Attr(0))
} else if vxlan.Age > 0 {
@@ -627,10 +613,7 @@ func setProtinfoAttr(link Link, mode bool, attr int) error {
req := nl.NewNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
msg := nl.NewIfInfomsg(syscall.AF_BRIDGE)
msg.Type = syscall.RTM_SETLINK
msg.Flags = syscall.NLM_F_REQUEST
msg.Index = int32(base.Index)
msg.Change = syscall.IFLA_PROTINFO | syscall.NLA_F_NESTED
req.AddData(msg)
br := nl.NewRtAttr(syscall.IFLA_PROTINFO|syscall.NLA_F_NESTED, nil)
@@ -683,6 +666,8 @@ func parseVxlanData(link Link, data []syscall.NetlinkRouteAttr) {
vxlan.L2miss = int8(datum.Value[0]) != 0
case nl.IFLA_VXLAN_L3MISS:
vxlan.L3miss = int8(datum.Value[0]) != 0
case nl.IFLA_VXLAN_GBP:
vxlan.GBP = int8(datum.Value[0]) != 0
case nl.IFLA_VXLAN_AGEING:
vxlan.Age = int(native.Uint32(datum.Value[0:4]))
vxlan.NoAge = vxlan.Age == 0

View File

@@ -47,7 +47,15 @@ const (
IFLA_VXLAN_PORT
IFLA_VXLAN_GROUP6
IFLA_VXLAN_LOCAL6
IFLA_VXLAN_MAX = IFLA_VXLAN_LOCAL6
IFLA_VXLAN_UDP_CSUM
IFLA_VXLAN_UDP_ZERO_CSUM6_TX
IFLA_VXLAN_UDP_ZERO_CSUM6_RX
IFLA_VXLAN_REMCSUM_TX
IFLA_VXLAN_REMCSUM_RX
IFLA_VXLAN_GBP
IFLA_VXLAN_REMCSUM_NOPARTIAL
IFLA_VXLAN_FLOWBASED
IFLA_VXLAN_MAX = IFLA_VXLAN_FLOWBASED
)
const (

View File

@@ -39,8 +39,9 @@ func NativeEndian() binary.ByteOrder {
var x uint32 = 0x01020304
if *(*byte)(unsafe.Pointer(&x)) == 0x01 {
nativeEndian = binary.BigEndian
} else {
nativeEndian = binary.LittleEndian
}
nativeEndian = binary.LittleEndian
}
return nativeEndian
}

View File

@@ -20,6 +20,15 @@ func NewRtMsg() *RtMsg {
}
}
func NewRtDelMsg() *RtMsg {
return &RtMsg{
RtMsg: syscall.RtMsg{
Table: syscall.RT_TABLE_MAIN,
Scope: syscall.RT_SCOPE_NOWHERE,
},
}
}
func (msg *RtMsg) Len() int {
return syscall.SizeofRtMsg
}

View File

@@ -14,22 +14,21 @@ import (
// Equivalent to: `ip route add $route`
func RouteAdd(route *Route) error {
req := nl.NewNetlinkRequest(syscall.RTM_NEWROUTE, syscall.NLM_F_CREATE|syscall.NLM_F_EXCL|syscall.NLM_F_ACK)
return routeHandle(route, req)
return routeHandle(route, req, nl.NewRtMsg())
}
// RouteAdd will delete a route from the system.
// Equivalent to: `ip route del $route`
func RouteDel(route *Route) error {
req := nl.NewNetlinkRequest(syscall.RTM_DELROUTE, syscall.NLM_F_ACK)
return routeHandle(route, req)
return routeHandle(route, req, nl.NewRtDelMsg())
}
func routeHandle(route *Route, req *nl.NetlinkRequest) error {
func routeHandle(route *Route, req *nl.NetlinkRequest, msg *nl.RtMsg) error {
if (route.Dst == nil || route.Dst.IP == nil) && route.Src == nil && route.Gw == nil {
return fmt.Errorf("one of Dst.IP, Src, or Gw must not be nil")
}
msg := nl.NewRtMsg()
msg.Scope = uint8(route.Scope)
family := -1
var rtAttrs []*nl.RtAttr