Docker and weave on two hosts can't ping each other

Question

OS: window7 virtualization tool: virtualbox virtual hypervisor: centos7 linux core as below

[root@localhost ~]# uname -a
Linux localhost.localdomain 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Two host IPs:

192.168.100.101
192.168.100.102

The steps:

curl -L git.io/weave -o /usr/local/bin/weave
chmod a+x /usr/local/bin/weave

[root@localhost ~]# weave launch 192.168.100.102
WARNING: existing iptables rule
'-A FORWARD -j REJECT --reject-with icmp-host-prohibited'
will block name resolution via weaveDNS - please reconfigure your firewall.
cannot locate running docker daemon
Warning: unable to detect proxy TLS configuration. To enable TLS, launch the proxy with ‘weave launch’ and supply TLS options. To suppress this warning, supply the ‘–no-detect-tls’ option.
3227932d5be77917c4e0b780cafe1171287c1029637f2360ece580fe6239cb4f

[root@localhost ~]# weave status
Version: 2.1.1 (failed to check latest version - see logs; next check at 2017/11/28 19:18:07)
Service: router
Protocol: weave 1..2
Name: 06:e0:c4:68:0c:ae(localhost.localdomain)
Encryption: disabled
PeerDiscovery: enabled
Targets: 1
Connections: 1 (1 failed)
Peers: 1
TrustedSubnets: none
Service: ipam
Status: ready
Range: 10.32.0.0/12
DefaultSubnet: 10.32.0.0/12
Service: dns
Domain: weave.local.
Upstream: 135.251.4.190, 135.251.38.218, 192.168.1.1
TTL: 1
Entries: 0
Service: proxy
Address: unix:///var/run/weave/weave.sock
Service: plugin (legacy)
DriverName: weave

[root@localhost ~]# weave version
weave script 2.1.1
weave 2.1.1

As above , the Connections: 1 (1 failed) , the docker run on these two hosts can’t ping each other.

And when I type weave version, it only showed weave script 2.1.1 and weave 2.1.1, not like other articles said it will show weaveexec and plugin or weavedns and so on. Is this the difference between versions or some mistakes I have made ?

Please help , thank you very much!

my docker log as below

[root@localhost ~]# docker logs weave
INFO: 2017/11/29 01:08:07.807752 Command line options: map[dns-effective-listen-address:172.17.0.1 nickname:localhost.localdomain dns-listen-address:172.17.0.1:53 ipalloc-range:10.32.0.0/12 status-addr:127.0.0.1:6782 weave-bridge:weave H:[unix:///var/run/weave/weave.sock] host-root:/host http-addr:127.0.0.1:6784 port:6783 proxy:true resolv-conf:/var/run/weave/etc/resolv.conf datapath:datapath docker-bridge:docker0 plugin:true]
INFO: 2017/11/29 01:08:07.807841 weave  2.1.1
INFO: 2017/11/29 01:08:07.859209 Docker API on unix:///var/run/docker.sock: &[ApiVersion=1.24 GoVersion=go1.8.3 Os=linux BuildTime=2017-10-24T15:40:21.112972404+00:00 PkgVersion=docker-1.12.6-61.git85d7426.el7.centos.x86_64 Version=1.12.6 Arch=amd64 KernelVersion=3.10.0-693.5.2.el7.x86_64 GitCommit=85d7426/1.12.6]
INFO: 2017/11/29 01:08:07.859520 Using docker bridge IP for DNS: 172.17.0.1
INFO: 2017/11/29 01:08:07.863781 proxy listening on unix:///var/run/weave/weave.sock
INFO: 2017/11/29 01:08:08.940871 Bridge type is bridged_fastdp
INFO: 2017/11/29 01:08:08.940885 Communication between peers is unencrypted.
INFO: 2017/11/29 01:08:08.961891 Our name is 06:e0:c4:68:0c:ae(localhost.localdomain)
INFO: 2017/11/29 01:08:08.962058 Restart/resume detected - using persisted peer list: [192.168.100.102]
INFO: 2017/11/29 01:08:08.972210 Docker API on unix:///var/run/docker.sock: &[KernelVersion=3.10.0-693.5.2.el7.x86_64 PkgVersion=docker-1.12.6-61.git85d7426.el7.centos.x86_64 GoVersion=go1.8.3 Os=linux Arch=amd64 BuildTime=2017-10-24T15:40:21.112972404+00:00 Version=1.12.6 ApiVersion=1.24 GitCommit=85d7426/1.12.6]
INFO: 2017/11/29 01:08:08.974990 Checking for pre-existing addresses on weave bridge
INFO: 2017/11/29 01:08:09.009949 [allocator 06:e0:c4:68:0c:ae] Initialising with persisted data
INFO: 2017/11/29 01:08:09.034491 Listening for DNS queries on 172.17.0.1
INFO: 2017/11/29 01:08:09.086102 Sniffing traffic on datapath (via ODP)
INFO: 2017/11/29 01:08:09.114882 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:09.116392 Listening for HTTP control messages on 127.0.0.1:6784
INFO: 2017/11/29 01:08:09.116576 Listening for metrics requests on 127.0.0.1:6782
INFO: 2017/11/29 01:08:09.125917 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:08:09.151109 Waiting for Weave API Server...
INFO: 2017/11/29 01:08:09.159548 Finished waiting for Weave API Server
INFO: 2017/11/29 01:08:09.159706 Listening on /run/docker/plugins/weave.sock for global scope
INFO: 2017/11/29 01:08:09.159811 Listening on /run/docker/plugins/weavemesh.sock for local scope
INFO: 2017/11/29 01:08:09.159822 Creating default "weave" network
INFO: 2017/11/29 01:08:09.462160 Discovered local MAC 06:e0:c4:68:0c:ae
INFO: 2017/11/29 01:08:09.547179 Discovered local MAC 46:2b:0d:08:12:be
INFO: 2017/11/29 01:08:09.554830 Discovered local MAC 0e:46:f3:dd:57:96
INFO: 2017/11/29 01:08:11.612424 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:11.614477 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:08:13.980824 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:13.982289 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:08:18.124543 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:18.125556 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:08:23.294574 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:23.322022 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:08:37.070537 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:37.073928 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:08:39.222651 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=1.12.6&flag_kernel-version=3.10.0-693.5.2.el7.x86_64&os=linux&signature=fvXv9SDD9r8gjV6d2HrXkVdBv5U72%2BeXQ6NT2u0JkKc%3D&version=2.1.1: dial tcp: lookup checkpoint-api.weave.works on 135.252.166.21:53: read udp 192.168.100.101:34840->135.252.166.21:53: i/o timeout
INFO: 2017/11/29 01:08:46.009136 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:08:46.011168 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:09:16.169210 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:09:16.171278 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:09:42.294136 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:09:42.296081 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:10:28.752091 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:10:28.756481 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:12:03.755330 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:12:03.760374 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:14:30.481453 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:14:30.486632 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:17:41.166716 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:17:41.168341 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:22:38.820826 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:22:38.829815 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host
INFO: 2017/11/29 01:23:08.968136 Expired MAC 06:e0:c4:68:0c:ae at 06:e0:c4:68:0c:ae(localhost.localdomain)
INFO: 2017/11/29 01:23:08.968199 Expired MAC 46:2b:0d:08:12:be at 06:e0:c4:68:0c:ae(localhost.localdomain)
INFO: 2017/11/29 01:23:08.968219 Expired MAC 0e:46:f3:dd:57:96 at 06:e0:c4:68:0c:ae(localhost.localdomain)
INFO: 2017/11/29 01:30:27.085406 ->[192.168.100.102:6783] attempting connection
INFO: 2017/11/29 01:30:27.089200 ->[192.168.100.102:6783] error during connection attempt: dial tcp4 :0->192.168.100.102:6783: getsockopt: no route to host

but on 192.168.100.101 , I can ping 192.168.100.102

[root@localhost ~]# ping 192.168.100.102
PING 192.168.100.102 (192.168.100.102) 56(84) bytes of data.
64 bytes from 192.168.100.102: icmp_seq=1 ttl=64 time=1.19 ms
64 bytes from 192.168.100.102: icmp_seq=2 ttl=64 time=1.05 ms
64 bytes from 192.168.100.102: icmp_seq=3 ttl=64 time=0.906 ms
^C
--- 192.168.100.102 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.906/1.051/1.195/0.120 ms

Answer 1

it only showed weave script 2.1.1 and weave 2.1.1, not like other articles said it will show weaveexec and plugin or weavedns and so on

This is expected with your version of Weave Net.

In prior versions, Weave Net started different containers, but starting Weave Net 2.0, these have been merged together.

See also:

• https://github.com/weaveworks/weave/blob/master/CHANGELOG.md#release-200

  All of Weave Net now runs in one container

  Previously we had three separate containers for routing, Docker API proxy and Docker plugin. Running everything in one simplifies start-up and removes the need to detect various error conditions. #1642,#2897,#2936,#2945,#2946,#2951,#2960

• https://www.weave.works/blog/weave-net-2-released

  In Weave Net 2.0, the various processes forming Weave Net were merged into a single process.

  and what follows.

Would you mind pointing to the docs which caused the confusion, so that we could improve these?

Answer 2

The issue has been resolved and shared my experience here.

As I described in my question

[root@localhost ~]# weave launch 192.168.100.102
WARNING: existing iptables rule
'-A FORWARD -j REJECT --reject-with icmp-host-prohibited'
will block name resolution via weaveDNS - please reconfigure your firewall.
cannot locate running docker daemon
Warning: unable to detect proxy TLS configuration. To enable TLS, launch the proxy with ‘weave launch’ and supply TLS options. To suppress this warning, supply the ‘–no-detect-tls’ option.
3227932d5be77917c4e0b780cafe1171287c1029637f2360ece580fe6239cb4f

Then on both hosts ,I run command

[root@localhost ~]# iptables -F

And then stop and rm all docker container and stop weave and then restart docker daemon and then on host 192.168.100.101 ,run command

[root@localhost ~]# weave launch
[root@localhost ~]# eval $(weave env)
[root@localhost ~]# docker run --name bbox1 -itd busybox
[root@localhost ~]# docker run --name bbox2 -itd busybox

and on host 192.168.100.102 , run command

[root@localhost ~]# weave launch 192.168.100.101
[root@localhost ~]# eval $(weave env)
[root@localhost ~]# docker run --name bbox3 -itd busybox

And then test

[root@localhost ~]# docker exec bbox3 ping -c2 bbox1
PING bbox1 (10.32.0.1): 56 data bytes
64 bytes from 10.32.0.1: seq=0 ttl=64 time=0.940 ms
64 bytes from 10.32.0.1: seq=1 ttl=64 time=2.362 ms

--- bbox1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.940/1.651/2.362 ms

Done! so it seemed that the iptables rule is the root cause. I will learn more iptables rules later so that avoid merely using "iptables -F". Thanks for @Marc Carré very much.