Commit Graph

2652 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
5037c5a8ce Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
2018-06-05 02:16:20 +02:00
Anusha Ragunathan
71cd53e4a1 Merge pull request #37177 from cpuguy83/minor_plugin_addr_change
Move plugin client to separate interface
2018-05-31 08:22:04 -07:00
Vincent Demeester
2ea4ece674 Merge pull request #37174 from thaJeztah/no_additional_magic_matching
Be explicit about github.com prefix being a legacy feature
2018-05-31 09:45:59 +02:00
Brian Goff
7c77df8acc Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-30 15:22:10 -04:00
Sebastiaan van Stijn
babb0c14fd Be explicit about github.com prefix being a legacy feature
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-30 11:09:14 +02:00
Brian Goff
f51a96c016 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 15:18:53 -04:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
f23c00d870 Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Kir Kolyshkin
d78e885326 pkg/mount/TestMount: fix wrt selinux
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:

--- FAIL: TestMount (0.12s)
    --- FAIL: TestMount/none-remount,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
    --- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"

Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).

As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.

While at it, fix error messages:
 - add missing commas;
 - fix a typo;
 - allow for clear distinction between mount
   and vfs (per-superblock) options.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-22 23:30:47 -07:00
Sebastiaan van Stijn
3646562a5c Merge pull request #37088 from ohbarye/fix-typos-duplicated-the
Fix typos: remove duplicated "the"
2018-05-19 20:57:47 +02:00
Sebastiaan van Stijn
4f8c870d62 Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-17 19:28:27 +02:00
Vincent Demeester
6a9bcc51df Merge pull request #37074 from skyc024/fix-typos
Fix typos
2018-05-17 16:00:26 +02:00
ohbarye
0f95b23d98 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
2018-05-17 21:49:51 +09:00
chenyuzhu
5ac0d2f0b8 Fix typos
Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>
2018-05-16 15:31:12 +08:00
Tonis Tiigi
f784907eae jsonmessage: pass message to aux callback
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 15:48:25 -07:00
Sebastiaan van Stijn
d4e48af48d Merge pull request #36918 from Sh4d1/master
add verification to check if escapeKeys have elements (pkg/term/proxy.go)
2018-04-30 11:57:21 +02:00
Patrik Cyvoct
d339130f30 Add verification and test to check if escapeKeys is not empty
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read

Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-04-30 00:32:43 +02:00
Kir Kolyshkin
7d62e40f7e Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-23 13:52:44 -07:00
Yong Tang
5c233cf431 Merge pull request #36913 from vdemeester/test-skip-non-root
Skip some tests requires root uid when run as user…
2018-04-23 11:42:42 -07:00
Sebastiaan van Stijn
20b524bf2e Merge pull request #36920 from kolyshkin/cancel-func
context.WithTimeout: do call the cancel func
2018-04-23 20:14:25 +02:00
Vincent Demeester
a7999aaa53 Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:39 +02:00
Vincent Demeester
5d8b88b114 Fix typo in idtools tests
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:07 +02:00
Vincent Demeester
53982e3fc1 Merge pull request #36091 from kolyshkin/mount
pkg/mount improvements
2018-04-21 11:03:54 +02:00
Kir Kolyshkin
05e2f7e2fa context.WithTimeout: do call the cancel func
govet complains (when using standard "context" package):

> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-20 12:27:13 -07:00
Kir Kolyshkin
a1d095199d mount.Unmount(): don't look into /proc/self/mountinfo
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).

Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.

Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:50 -07:00
Kir Kolyshkin
c611f18a7f pkg/mount/mountinfo_linux: parser speed up
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:

> BenchmarkParsingScanf-4      	     300	  22294112 ns/op
> BenchmarkParsingSplit-4      	    3000	   2780703 ns/op

I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:

> BenchmarkParsingMixed-4      	    1000	   8827058 ns/op

Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.

I have also added more comments to aid in future development.

Finally, the test data is fixed to now have white space before the first field.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:42 -07:00
Kir Kolyshkin
871c957242 getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:17 -07:00
Kir Kolyshkin
bb934c6aca pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:48:09 -07:00
Alexandre Jomin
0d82bc142c pkg/term add build tag for netbsd
Signed-off-by: Alexandre Jomin <alexandrejomin@gmail.com>
2018-04-18 15:16:06 +02:00
Tibor Vass
61138fb5fc Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
2018-04-13 13:56:38 -10:00
Anusha Ragunathan
ff6a103067 Merge pull request #36778 from odg0318/master
Using authorization plugin, I changed Content-Type check routine.
2018-04-12 17:39:38 -07:00
Derek McGowan
fb170206ba Set format in archiver
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:33 -07:00
Derek McGowan
a422774e59 Fix tarsum for go 1.10
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:14 -07:00
Brian Goff
74f8e47352 Limit authz response buffer
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-04-11 15:36:36 -04:00
Vincent Demeester
5219725890 Merge pull request #35518 from cyphar/libdm-dlsym-deferred_remove
pkg: devmapper: dynamically load dm_task_deferred_remove
2018-04-11 14:11:16 +02:00
odg0318
6ac73d34e4 If Content-Type is application/json;charset=UTF-8, RequestBody is empty.
Signed-off-by: odg0318 <odg0318@gmail.com>
2018-04-05 04:03:45 -04:00
Manuel Rüger
108bbd96cb Fix tests for pkg/archive
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2018-04-03 01:17:52 +02:00
Brian Goff
9d46c4c138 Support cancellation in directory.Size()
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-29 15:49:15 -04:00
Vincent Demeester
7270b01e5d Merge pull request #36617 from Microsoft/jjh/kernel-registry
Windows: Move kernel_windows to use golang registry functions
2018-03-19 11:47:17 +01:00
John Howard
29f93c4bc7 Windows: Move kernel_windows to use golang registry functions
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-16 09:47:45 -07:00
Daniel Nephin
7d8815ea70 Cleanup pkg/jsonmessage progress tests
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:47 -04:00
Daniel Nephin
c9e52bd0da Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:46 -04:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
ef01dea893 Cleanup some assertions
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 10:54:54 -04:00
Vincent Demeester
a21d5bf669 Merge pull request #36506 from kolyshkin/pkg-mount-slice
pkg/mount: use sort.Slice
2018-03-09 09:46:53 +01:00
Akihiro Suda
4a1d35c546 Merge pull request #36505 from kolyshkin/pkg-mount-tests
pkg/mount unit tests: skip some test under non-root
2018-03-08 17:27:31 +09:00
John Stephens
8cf8fe9cf8 Merge pull request #36451 from Microsoft/jjh/ubr
Windows: Report Version and UBR
2018-03-07 06:27:01 -08:00
Kir Kolyshkin
a00310b54c pkg/mount: use sort.Slice
Sorting by mount point length can be implemented in a more
straightforward fashion since Go 1.8 introduced sort.Slice()
with an ability to provide a less() function in place.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-06 12:46:58 -08:00
Kir Kolyshkin
4aae77602a pkg/mount unit tests: skip some test under non-root
This makes `go test .` to pass if run as non-root user, skipping
those tests that require superuser privileges (for `mount`).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-06 12:37:27 -08:00
Sebastiaan van Stijn
390b74cb63 Merge pull request #36493 from cpuguy83/add_plugin_timeout_test
Adds a unit test for plugin request timeout
2018-03-06 12:45:17 +01:00