Created method to handle the node state change with cleanup operation
associated.
Realign testing client with the new diagnostic interface
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
This commit introduces the possibility to enable a debug mode
for the networkDB, this will allow the opening of a tcp port
on localhost that will expose the networkDB api for debugging
purposes.
The API can be discovered using curl localhost:<port>/help
It support json output if passed json as URL query parameter
option and pretty printing if passing json=pretty
All the binaries values are serialized in base64 encoding, this
can be skip passing the unsafe option as url query parameter
A simple go client will follow up
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Update Dockerfile, curl is used for the healthcheck
Add /dump for creating the routine stack trace
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Separate the hostname from the node identifier. All the messages
that are exchanged on the network are containing a nodeName field
that today was hostname-uniqueid. Now being encoded as strings in
the protobuf without any length restriction they plays a role
on the effieciency of protocol itself. If the hostname is very long
the overhead will increase and will degradate the performance of
the database itself that each single cycle by default allows 1400
bytes payload
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
- Changed the loop per network. Previous implementation was taking a
ReadLock to update the reapTime but now with the residualReapTime
also the bulkSync is using the same ReadLock creating possible
issues in concurrent read and update of the value.
The new logic fetches the list of networks and proceed to the
cleanup network by network locking the database and releasing it
after each network. This should ensure a fair locking avoiding
to keep the database blocked for too much time.
Note: The ticker does not guarantee that the reap logic runs
precisely every reapTimePeriod, actually documentation says that
if the routine is too long will skip ticks. In case of slowdown
of the process itself it is possible that the lifetime of the
deleted entries increases, it still should not be a huge problem
because now the residual reaptime is propagated among all the nodes
a slower node will let the deleted entry being repropagate multiple
times but the state will still remain consistent.
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
- Added remainingReapTime field in the table event.
Wihtout it a node that did not have a state for the element
was marking the element for deletion setting the max reapTime.
This was creating the possibility to keep the entry being resync
between nodes forever avoding the purpose of the reap time
itself.
- On broadcast of the table event the node owner was rewritten
with the local node name, this was not correct because the owner
should continue to remain the original one of the message
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
- Diagnose framework that exposes REST API for db interaction
- Dockerfile to build the test image
- Periodic print of stats regarding queue size
- Client and server side for integration with testkit
- Added write-delete-leave-join
- Added test write-delete-wait-leave-join
- Added write-wait-leave-join
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
With docker 1.11 based dnet tests, the dnet_exec is sometimes exiting
with exit code 129 because the process is getting a SIGHUP. Although the
reason or source of the SIGHUP is unknown, it is making the tests flaky
because non-zero exit code. Fixed it by trapping SIGHUP inside the
container so that we can run the test code successfully.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Add a notion of service in libnetwork so that a group of endpoints
which form a service can be treated as such so that service level
features can be added on top. Initially as part of this PR the support
to assign a name to the said service is added which results in DNS
queries to the service name to return all the IPs of the backing
endpoints so that DNS RR behavior on the service name can be achieved.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
With DNS resolution happening within the container namespace
test should not try to ping a DNS name and expect a packet
loss message. It will only show up as a DNS name resolution
failure. Changed the bridge internal test to test for a
well known IP address.
Also rearranged the overlay internal tests so that it gets
to run before the dnet container is removed which was
happening in previous tests.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Restoring original behavior where on disconnect
from overlay network (only connected network), it also
disconnects from default gw network.
- On sandbox delete, the leave and delete of each
endpoint is performed, regardless of whether the endpoint
is the gw network endpoint. This endpoint is already
automatically removed in endpoint.sbLeave()
- Also do not let internal network dictate container does
not need external connectivity. Before this fix, if a container
was connected to an overlay and an internal network, it may not
get attached to the default gw network.
Signed-off-by: Alessandro Boch <aboch@docker.com>
- On sandbox delete, the leave and delete of each
endpoint is performed, regardless of whether the endpoint
is the gw network endpoint. This endpoint is already
automatically removed in endpoint.sbLeave() by
sb.clearDefaultGW() when the sandbox is marked for
deletion.
- Also restoring otiginal behavior where on disconnect
from overlay network (only connected network), it also
disconnects from default gw network.
- Also do not let internal network dictate container does
not need external connectivity. Before this fix, if a container
was connected to an overlay and an internal network, it may not
get attached to the default gw network.
- needDefaultGw() takes now into account whether the sandbox
is marked for deletion
Signed-off-by: Alessandro Boch <aboch@docker.com>
By removing the need to clear the default gateway during sbJoin and
sbLeave to account for other bridge network, the default-gw endpoint
will stay with the container, it will also help retain the container
property.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Add support for overlay networking in older kernels.
Following were done to achieve this:
+ Create the vxlan network in host namespace.
+ This may create conflicts with other private
networks so check for conflicts and fail a
join if there is any conflict.
+ Add iptable based filtering to only allow
subnet bridges in the same network to forward
traffic while different network bridges will
not be able to forward b/w each other. Also
block traffic to overlay network originating
from the host itself.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Compile the dnet tool for Linux (x86, amd64 and arm)
and Windows (x86 and amd64)
- Moved installation of dependencies into `Dockerfile.build`
- Remove `start-services` from Makefile
- That's the responsibility of Docker or build environment
- Removed utils depending on `netlink` from `netutils/utils.go`
Unable to add `make cross` to CircleCI just yet as there are some
issues to solve that are unrelated to this PR
Also fix `.gitignore` which was not updated after changing the build
image name in #667
Signed-off-by: Dave Tucker <dt@docker.com>
There are cases as seen in https://github.com/docker/docker/issues/17984
the sandbox could be stale in endpoint structure, when the actual
sandbox is removed during the cleanup phase. Hence instead of just
validating for sandboxID, make sure if it is actually present in the
sandboxes DB managed by the controller.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Added IT cases for external connectivity check for bridge
and overlay networks, both initially and after a restart.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Added an IT case for checking proper /etc/hosts
handling in the overlay network. This also to see
if there are any stale entries in the /etc/hosts
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
A local endpoint is known to the watch database only
during Join. But the same endpoint can be known to the
watch database as remote endpoint well before the Join
because a CreateEndpoint updates the endpoint to the store.
So on Join when you come to know that this is indeed a
local endpoint remove it from remote endpoint list and add it
to local endpoint list.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Added etcd integration test for overlay
- Added etcd integration test for multinode
with mock test driver suitable for circleci
- Added multinode tests for zookeeper
- Made the script smart enough to only start
data stores necessary for the requested suites
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- With the selectively running a suite support
one can do the following to select which suite
of tests to run:
SUITES="simple multi" sudo -E make integration-tests
- Refactored and cleaned up some ununsed code in helpers.bash
- Added discover string parse function to parse discovery
string into provide and address
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Currently integration test is a bit flaky because of
variability in the dnet bootup time. Fixed it to wait for
dnet to come up before performing any tests.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
