Skip to content
This repository has been archived by the owner on Sep 26, 2021. It is now read-only.

Docker Machine Fails to deploy Ubuntu 14.04 & 16.04 images to Openstack #4197

Closed
dmccullo opened this issue Jul 14, 2017 · 13 comments
Closed

Comments

@dmccullo
Copy link

I am running docker-machine version:
docker-machine version 0.12.2, build 9371605
I had already opened a Moby ticket number:
moby/moby#34095
They told me there was a bug fixed in this version of Docker-machine.
I have deployed many instances with Docker machine in openstack without issues.
Since upgrading to Docker CE I have had these issues.

Ubuntu 14.04 Kernel version:
3.13.0-37-generic

the Docker Command:
docker-machine create --openstack-flavor-name s1.small --openstack-image-id 97527772-e68c-415d-a498-3ca1942788b0 --openstack-nova-network -d openstack Hostname

the response I get:
Running pre-create checks... Creating machine... (Hostname) Creating machine... Waiting for machine to be running, this may take a few minutes... Detecting operating system of created instance... Waiting for SSH to be available... Detecting the provisioner... Provisioning with ubuntu(upstart)... Installing Docker... Copying certs to the local machine directory... Copying certs to the remote machine... Setting Docker configuration on the remote daemon... Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded

Docker-machine ssh hostname logs me into the instance.

here is the docker.log:
Waiting for /var/run/docker.sock time="2017-07-14T15:18:42.559578431Z" level=info msg="libcontainerd: new containerd process, pid: 9657" time="2017-07-14T15:18:42.581118102Z" level=warning msg="failed to rename /var/lib/docker/tmp for background deletion: rename /var/lib/docker/tmp /var/lib/docker/tmp-old: no such file or directory. Deleting synchronously" /var/run/docker.sock is up time="2017-07-14T15:18:42.701328292Z" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded." time="2017-07-14T15:18:42.702563545Z" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded." time="2017-07-14T15:18:42.863149814Z" level=warning msg="devmapper: Usage of loopback devices is strongly discouraged for production use. Please use --storage-opt dm.thinpooldevor useman docker to refer to dm.thinpooldev section." time="2017-07-14T15:18:42.897423971Z" level=warning msg="devmapper: XFS is not supported in your system. Either the kernel doesn't support it or mkfs.xfs is not in your PATH. Defaulting to ext4 filesystem" time="2017-07-14T15:18:42.899998053Z" level=info msg="devmapper: Creating filesystem ext4 on device docker-253:1-131817-base" time="2017-07-14T15:18:44.685079941Z" level=info msg="devmapper: Successfully created filesystem ext4 on device docker-253:1-131817-base" time="2017-07-14T15:18:44.794347870Z" level=info msg="Graph migration to content-addressability took 0.00 seconds" time="2017-07-14T15:18:44.794708590Z" level=warning msg="Your kernel does not support swap memory limit" time="2017-07-14T15:18:44.794870983Z" level=warning msg="Your kernel does not support cgroup rt period" time="2017-07-14T15:18:44.794989573Z" level=warning msg="Your kernel does not support cgroup rt runtime" time="2017-07-14T15:18:44.795219186Z" level=warning msg="mountpoint for pids not found" time="2017-07-14T15:18:44.795569953Z" level=info msg="Loading containers: start." time="2017-07-14T15:18:44.844080545Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found.\ninsmod /lib/modules/3.13.0-37-generic/kernel/net/llc/llc.ko \ninsmod /lib/modules/3.13.0-37-generic/kernel/net/802/stp.ko \ninsmod /lib/modules/3.13.0-37-generic/kernel/net/bridge/bridge.ko \n, error: exit status 1" time="2017-07-14T15:18:45.162298392Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address" time="2017-07-14T15:18:45.261610718Z" level=info msg="Loading containers: done." time="2017-07-14T15:18:45.273164262Z" level=info msg="Daemon has completed initialization" time="2017-07-14T15:18:45.273318653Z" level=info msg="Docker daemon" commit=02c1d87 graphdriver=devicemapper version=17.06.0-ce time="2017-07-14T15:18:45.286939464Z" level=info msg="API listen on /var/run/docker.sock" time="2017-07-14T15:18:48.084048663Z" level=info msg="Processing signal 'terminated'" INFO[0005] stopping containerd after receiving terminated Waiting for /var/run/docker.sock time="2017-07-14T15:18:53.676763888Z" level=info msg="libcontainerd: new containerd process, pid: 10403" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up /var/run/docker.sock is up time="2017-07-14T15:18:53.811297250Z" level=info msg="libcontainerd: new containerd process, pid: 10451" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:53.881343863Z" level=info msg="libcontainerd: new containerd process, pid: 10499" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:53.958452230Z" level=info msg="libcontainerd: new containerd process, pid: 10549" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.035050017Z" level=info msg="libcontainerd: new containerd process, pid: 10597" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.107651301Z" level=info msg="libcontainerd: new containerd process, pid: 10644" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.179747934Z" level=info msg="libcontainerd: new containerd process, pid: 10693" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.251748931Z" level=info msg="libcontainerd: new containerd process, pid: 10741" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.320027273Z" level=info msg="libcontainerd: new containerd process, pid: 10789" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.421297653Z" level=info msg="libcontainerd: new containerd process, pid: 10868" Error starting daemon: error initializing graphdriver: driver not supported /var/run/docker.sock is up time="2017-07-14T15:18:54.516045108Z" level=info msg="libcontainerd: new containerd process, pid: 10915" Error starting daemon: error initializing graphdriver: driver not supported

Ubuntu 16.04 Kernel version:
4.4.0-53-generic
Command I run:
docker-machine create --openstack-flavor-name s1.small --openstack-image-id 9d018c75-c18d-40bf-853f-6266d1267d0d --openstack-nova-network -d openstack HostnameUB16

Result I get:
Creating machine... (HostnameUB16) Creating machine... Waiting for machine to be running, this may take a few minutes... Detecting operating system of created instance... Waiting for SSH to be available... Detecting the provisioner... Provisioning with ubuntu(systemd)... Installing Docker... Copying certs to the local machine directory... Copying certs to the remote machine... Setting Docker configuration on the remote daemon... Error creating machine: Error running provisioning: ssh command error: command : sudo systemctl -f start docker err : exit status 1 output : Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.

$ systemctl status docker.service

● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: inactive (dead) (Result: exit-code) since Fri 2017-07-14 15:32:30 UTC; 7min ago
     Docs: https://docs.docker.com
  Process: 5279 ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.p
 Main PID: 5279 (code=exited, status=1/FAILURE)

Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jul 14 15:32:30 HostnameUB16 systemd[1]: Failed to start Docker Application Container Engine.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Unit entered failed state.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 14 15:32:30 HostnameUB16 systemd[1]: Stopped Docker Application Container Engine.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Start request repeated too quickly.
Jul 14 15:32:30 HostnameUB16 systemd[1]: Failed to start Docker Application Container Engine.

$ journalctl | grep docker

Jul 14 15:32:05 HostnameUB16 sudo[2896]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/sh -c add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial edge"
Jul 14 15:32:09 HostnameUB16 sudo[1972]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/sh -c apt-get install -y -q docker-ce
Jul 14 15:32:17 HostnameUB16 groupadd[4397]: group added to /etc/group: name=docker, GID=999
Jul 14 15:32:17 HostnameUB16 groupadd[4397]: group added to /etc/gshadow: name=docker
Jul 14 15:32:17 HostnameUB16 groupadd[4397]: new group: name=docker, GID=999
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.135304715Z" level=info msg="libcontainerd: new containerd process, pid: 4513"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.150306472Z" level=warning msg="failed to rename /var/lib/docker/tmp for background deletion: rename /var/lib/docker/tmp /var/lib/docker/tmp-old: no such file or directory. Deleting synchronously"
Jul 14 15:32:18 HostnameUB16 audit[4525]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="docker-default" pid=4525 comm="apparmor_parser"
Jul 14 15:32:18 HostnameUB16 kernel: audit: type=1400 audit(1500046338.196:15): apparmor="STATUS" operation="profile_load" profile="unconfined" name="docker-default" pid=4525 comm="apparmor_parser"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.316921529Z" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.317554120Z" level=warning msg="Your kernel does not support swap memory limit"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.317821572Z" level=warning msg="Your kernel does not support cgroup rt period"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.318041930Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.318748993Z" level=info msg="Loading containers: start."
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.748473718Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Jul 14 15:32:18 HostnameUB16 systemd-udevd[4594]: Could not generate persistent MAC address for docker0: No such file or directory
Jul 14 15:32:18 HostnameUB16 kernel: IPv6: ADDRCONF(NETDEV_UP): docker0: link is not ready
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.832118128Z" level=info msg="Loading containers: done."
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.845487282Z" level=info msg="Daemon has completed initialization"
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.845777622Z" level=info msg="Docker daemon" commit=02c1d87 graphdriver=overlay2 version=17.06.0-ce
Jul 14 15:32:18 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:18.873066554Z" level=info msg="API listen on /var/run/docker.sock"
Jul 14 15:32:20 HostnameUB16 sudo[4705]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/sh -c docker version
Jul 14 15:32:21 HostnameUB16 sudo[4757]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/bin/docker version
Jul 14 15:32:21 HostnameUB16 sudo[4807]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/systemctl -f stop docker
Jul 14 15:32:21 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:21.872029757Z" level=info msg="Processing signal 'terminated'"
Jul 14 15:32:21 HostnameUB16 dockerd[4506]: time="2017-07-14T15:32:21.873462966Z" level=info msg="stopping containerd after receiving terminated"
Jul 14 15:32:22 HostnameUB16 sudo[4859]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/sbin/ip link delete docker0
Jul 14 15:32:23 HostnameUB16 sudo[4914]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/bin/tee /etc/docker/ca.pem
Jul 14 15:32:24 HostnameUB16 sudo[4964]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/bin/tee /etc/docker/server.pem
Jul 14 15:32:24 HostnameUB16 sudo[5014]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/bin/tee /etc/docker/server-key.pem
Jul 14 15:32:26 HostnameUB16 sudo[5112]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/mkdir -p /etc/systemd/system/docker.service.d
Jul 14 15:32:26 HostnameUB16 sudo[5115]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/bin/tee /etc/systemd/system/docker.service.d/10-machine.conf
Jul 14 15:32:28 HostnameUB16 sudo[5229]:   ubuntu : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/systemctl -f start docker
Jul 14 15:32:28 HostnameUB16 dockerd[5232]: time="2017-07-14T15:32:28.849909640Z" level=info msg="libcontainerd: new containerd process, pid: 5239"
Jul 14 15:32:28 HostnameUB16 dockerd[5232]: Error starting daemon: error initializing graphdriver: driver not supported
Jul 14 15:32:28 HostnameUB16 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jul 14 15:32:28 HostnameUB16 systemd[1]: docker.service: Unit entered failed state.
Jul 14 15:32:28 HostnameUB16 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 14 15:32:29 HostnameUB16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 14 15:32:29 HostnameUB16 dockerd[5259]: time="2017-07-14T15:32:29.193162823Z" level=info msg="libcontainerd: new containerd process, pid: 5264"
Jul 14 15:32:29 HostnameUB16 dockerd[5259]: Error starting daemon: error initializing graphdriver: driver not supported
Jul 14 15:32:29 HostnameUB16 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jul 14 15:32:29 HostnameUB16 systemd[1]: docker.service: Unit entered failed state.
Jul 14 15:32:29 HostnameUB16 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 14 15:32:29 HostnameUB16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 14 15:32:29 HostnameUB16 dockerd[5279]: time="2017-07-14T15:32:29.442423858Z" level=info msg="libcontainerd: new containerd process, pid: 5285"
Jul 14 15:32:30 HostnameUB16 dockerd[5279]: Error starting daemon: error initializing graphdriver: driver not supported
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Unit entered failed state.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.service: Start request repeated too quickly.
Jul 14 15:32:30 HostnameUB16 systemd[1]: docker.socket: Unit entered failed state.

Any insight on what is causing this would be appreciated!
I am running docker-machine commands from OSX and have deployed Centos containers with the exact same environment variables and command other than changing the image ID without issues..
Just ubuntu images are causing these issues.

@brandontamm
Copy link

Try running with
docker-machine -D create ...

This will provide the details above in addition to the details that Docker echo's

@brandontamm
Copy link

docker-machine create is having issues in general - fails upon creation, leaving you able to ssh to server ... however, users cannot upgrade or provision server and there are issues on the actual server itself. For instance, when attempting to resize a digital ocean machine, the re-size fails and Digitalocean support is blaming the docker-machine create command that left something in a broken state.

@dmccullo
Copy link
Author

dmccullo commented Jul 17, 2017

I am using a openstack environment and only Ubuntu images are doing this I Ran with -D and I get allot of detail but no errors other than cannot connect to ssh seems to be trying 0.0.0.0:22 not sure why?
Docker machine has the IP of the server in the first ssh command
and when I do a docker-machine ssh hostname voila I am in...

`Using SSH client type: external
Using SSH private key: /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none ubuntu@208.xx.xx.38 -o IdentitiesOnly=yes -i /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa -p 22] /usr/bin/ssh }
About to run SSH command:
netstat -tln
SSH cmd err, output: : Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN

(Hostname) Calling .GetSSHHostname
(Hostname) Calling .GetSSHPort
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none ubuntu@208.xx.xx.38 -o IdentitiesOnly=yes -i /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa -p 22] /usr/bin/ssh }
About to run SSH command:
netstat -tln
SSH cmd err, output: : Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN

(Hostname) Calling .GetSSHHostname
(Hostname) Calling .GetSSHPort
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none ubuntu@208.xx.xx.38 -o IdentitiesOnly=yes -i /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa -p 22] /usr/bin/ssh }
About to run SSH command:
netstat -tln
SSH cmd err, output: : Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN

(Hostname) Calling .GetSSHHostname
(Hostname) Calling .GetSSHPort
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none ubuntu@208.xx.xx.38 -o IdentitiesOnly=yes -i /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa -p 22] /usr/bin/ssh }
About to run SSH command:
netstat -tln
SSH cmd err, output: : Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN

(Hostname) Calling .GetSSHHostname
(Hostname) Calling .GetSSHPort
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none ubuntu@208.xx.xx.38 -o IdentitiesOnly=yes -i /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa -p 22] /usr/bin/ssh }
About to run SSH command:
netstat -tln
SSH cmd err, output: : Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN

(Hostname) Calling .GetSSHHostname
(Hostname) Calling .GetSSHPort
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHKeyPath
(Hostname) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none ubuntu@208.xx.xx.38 -o IdentitiesOnly=yes -i /Users/don.mccullough/.docker/machine/machines/Hostname/id_rsa -p 22] /usr/bin/ssh }
About to run SSH command:
netstat -tln
SSH cmd err, output: : Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN

Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded
Opting out of crash reporting.`

@brandontamm
Copy link

@shin - Is this ticket the root cause to the docker-machine create issue that is being reported in multiple recent tickets:

#4156
#4166
#4189
#4197
#4198

@brandontamm
Copy link

brandontamm commented Jul 17, 2017 via email

@dmccullo
Copy link
Author

I agree with brandontamm
I see all the same issues working with Openstack.

Docker-machine upgrade, regenerate-certs .... all fail...

this is only occurring with Ubuntu images just deployed a Centos 7 image no issues at all
docker-machine create --openstack-flavor-name s1.small --openstack-sec-groups DonsRules --openstack-image-id 1b8a53f4-1986-492d-a6ad-4907b69fc76b --openstack-nova-network -d openstack HostnameCE2 Running pre-create checks... Creating machine... (HostnameCE2) Creating machine... Waiting for machine to be running, this may take a few minutes... Detecting operating system of created instance... Waiting for SSH to be available... Detecting the provisioner... Provisioning with centos... Copying certs to the local machine directory... Copying certs to the remote machine... Setting Docker configuration on the remote daemon... Checking connection to Docker... Docker is up and running! To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env HostnameCE2

@brandontamm
Copy link

brandontamm commented Jul 17, 2017

@shin- I can confirm that these issues still occur in docker-machine v0.12.2

@shin-
Copy link
Contributor

shin- commented Jul 17, 2017

For the 14.04 image, the kernel is too old to use the overlay storage driver (3.18 is the minimum required). You can provision using a custom --storage-driver flag (aufs should work for example) or use a newer kernel.

The problem with your 16.04 image is similar, the default storage driver there is aufs but your image seems to be missing aufs support. Either use an alternative storage driver or make sure your image has aufs modules enabled.

@dmccullo
Copy link
Author

Ok I will look into our images we can update them to new images to allow our environment to work for our users, but I believe we are using standard ubuntu images and the Kernels are current LTS supported. I am running docker-machine v0.12.2 so this is definitely happening on the current most recent build.

@dmccullo
Copy link
Author

BTW I appreciate all the details and help! I will post any fixes we find work.

@dmccullo
Copy link
Author

Setting up both our images for ubuntu 14.04 and 16.04 with aufs has resolved this issue. testing has confirmed all is working perfectly now. Thanks for the guidance!

@shin-
Copy link
Contributor

shin- commented Jul 24, 2017

@dmccullo Glad to hear that - thank you for reporting back!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants