OSX 10.11, Docker 1.9.1 and Cisco AnyConnect 4.1.02011

[iamKurt]

Is there a preferred method of running Docker on a Mac when connected to a VPN? Running

docker-machine create --driver virtualbox default

results in the following:

This machine has been allocated an IP address, but Docker Machine could not
reach it successfully.

SSH for the machine should still work, but connecting to exposed ports, such as
the Docker daemon port (usually <ip>:2376), may not work properly.

You may need to add the route manually, or use another related workaround.

This could be due to a VPN, proxy, or host file configuration issue.

Anyone have a solution for this?

Thanks!

[nathanleclaire]

You have to either do the creation and usage of Machine without the VPN enabled, or add the route to the VM on the host only network manually.

[nathanleclaire]

Alternatively, you could just run docker commands directly inside the VM after docker-machine sshing in. Depends on your use case.

[iamKurt]

Thanks for the response! docker-machine ssh does work. How do you add the route on the VM?

[nathanleclaire]

Something like should work: #1500 (comment)

[cantolick]

I am having similar issues trying to get it to work while on VPN. I was able to get reconnected after disconnecting from VPN though.

I first had to make sure I was on the latest VirtualBox (VirtualBox 5.0.12) - I was on 5.0.0

chantra comment is valid, my challenge was figuring out the vboxnet name.

To do this find out your current hostonlyifs

$ VBoxManage list hostonlyifs
Name:            vboxnet0
GUID:            786f6276-656e-4074-8000-0a0027000000
DHCP:            Disabled
IPAddress:       192.168.99.1
NetworkMask:     255.255.255.0
IPV6Address:
IPV6NetworkMaskPrefixLength: 0
HardwareAddress: 0a:00:27:00:00:00
MediumType:      Ethernet
Status:          Up

Then run chantra's command the the name
sudo route add -net 192.168.99.0/24 -interface vboxnet0

Then run the docker-machine environment command.
docker-machine env dev

Then run the eval call
eval $(docker-machine env aem)

[jakirkham]

This also seems related ( #2258 ).

[iamKurt]

Okay, will this still allow me to access the Docker host via IP address in a browser? I have a couple containers running on my Docker host -- one with a webapp and the other with a database. I want to ensure that after I make this change I will still be able to use the IP address of the Docker host/VM to access the database and webapp.

Thanks,
Kurt

[nathanleclaire]

Okay, will this still allow me to access the Docker host via IP address in a browser? I have a couple containers running on my Docker host -- one with a webapp and the other with a database. I want to ensure that after I make this change I will still be able to use the IP address of the Docker host/VM to access the database and webapp.

@formerlyKurt If you use the route command like that it should work for accessing from the browser too.

Alternatively, might be able to use SSH port forwarding. e.g. if your webapp and database ports are 8080 and 6379, this should start a process that forwards them to your computer's native localhost:

$ docker-machine ssh default -L 8080:localhost:8080 -L 6379:localhost:6379 -N

[iamKurt]

Somehow adding the route is not working for me. I ran the command:

sudo route add -net 192.168.99.0/24 -interface vboxnet0
route: writing to routing socket: File exists
add net 192.168.99.0: gateway vboxnet0: File exists

But when I run

docker-machine env default

I get:

Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.

Be advised that this will trigger a Docker daemon restart which will stop running containers.

I've tried removing the route and re-adding it and get the same error. What am I missing here?

[nathanleclaire]

@formerlyKurt The output of route looks a bit suspect to me.

Are you sure that's the correct vbox interface? What's the output of VBoxManage list hostonlyifs?

[iamKurt]

@nathanleclaire sadly, this seems to be the right interface.

VBoxManage list hostonlyifs
Name:            vboxnet0
GUID:            786f6276-656e-4074-8000-0a0027000000
DHCP:            Disabled
IPAddress:       192.168.99.1
NetworkMask:     255.255.255.0
IPV6Address:
IPV6NetworkMaskPrefixLength: 0
HardwareAddress: 0a:00:27:00:00:00
MediumType:      Ethernet
Status:          Up
VBoxNetworkName: HostInterfaceNetworking-vboxnet0

[nathanleclaire]

What's the output of the route command if run with -v?

[iamKurt]

With the -v command, I get:

sudo route -v add -net 192.168.99.0/24 -interface vboxnet0
u: inet 192.168.99.0; u: link vboxnet0:a.0.27.0.0.0; RTM_ADD: Add Route: len 140, pid: 0, seq 1, errno 0, flags:<UP,STATIC>
locks:  inits:
sockaddrs: <DST,GATEWAY,NETMASK>
 192.168.99.0 vboxnet0:a.0.27.0.0.0 (0) 0 ffff ff
route: writing to routing socket: File exists
add net 192.168.99.0: gateway vboxnet0: File exists

[cloudsan42-zz]

@formerlyKurt

are u able solve this issue? I have similar issue with ciscoVpn.

[iamKurt]

@cloudsan42 I was not able to resolve this issue. Hoping that @nathanleclaire has a good suggestion.

Found instructions on port forwarding in the boot2docker docs but that did not resolve this issue.

:(

[dantran]

same here :-)

[kelsin]

Same exact behavior as @formerlyKurt on my cisco VPN:

sudo route -v add -net 192.168.99.0/24 -interface vboxnet0
u: inet 192.168.99.0; u: link vboxnet0:a.0.27.0.0.0; RTM_ADD: Add Route: len 140, pid: 0, seq 1, errno 0, flags:<UP,STATIC>
locks:  inits: 
sockaddrs: <DST,GATEWAY,NETMASK>
 192.168.99.0 vboxnet0:a.0.27.0.0.0 (0) 0 ffff ff
route: writing to routing socket: File exists
add net 192.168.99.0: gateway vboxnet0: File exists

[kelsin]

As a note... I just tried out http://www.infradead.org/openconnect/ and ALL of my cisco anyconnect problems go away. Default routes aren't messed with. So it turns out it's not company VPN setup, and it's not my computer and it's not docker or virtual box. It's just AnyConnect doing stupid stuff!

[nathanleclaire]

One possible workaround (although not perfect) is to use SSH port forwarding. docker-machine ssh default -L 8080:localhost:8080 will forward the VM's SSH port 8080 to localhost:8080 on the host computer for as long as the SSH session is enabled.

[iamKurt]

@kelsin if only I could take that path. Security group would, and I quote, 😱 As it stands right now, I seem to be having difficulty connecting to VPN.

@dantran @cloudsan42 have you tried the suggestion by @nathanleclaire?

[dantran]

ssh port forwarding works for me

[johntdyer]

Hey guys, just to be clear, AnyConnect can "own" your routes if configured to do so. If this is the case, as it is for me, you can try to modify your routes all day long and it will make no difference. Unfortunately this is considered a security feature to prevent split routing.

Is there any way to get docker-machine to bind to localhost and port forward :2376 to localhost instead of the hostonly adapter?

[nathanleclaire]

Is there any way to get docker-machine to bind to localhost and port forward :2376 to localhost instead of the hostonly adapter?

At present, there's no way to do this directly in create, but you could potentially use SSH port forwarding to forward :2376 and then set DOCKER_HOST yourself manually. It's a bit of a hack, but should work for the interim.

i.e.:

$ docker-machine ssh default -L 2376:localhost:2376 -fN
$ eval $(docker-machine env)
$ export DOCKER_HOST=:2376
$ docker info
Containers: 8
 Running: 0
 Paused: 0
 Stopped: 8
Images: 17
...
Labels:
 provider=virtualbox

By the way, I would be happy to consider proposals or PRs to enable this as a more first-class feature of Machine (e.g. --virtualbox-nat-only or what have you).

[sadhasivam]

I followed some of the hacks mentioned above in vain. Is there any documented workaround for this issue. Issue really stalling our docker development largely. please help.

I am using
Mac OSX- 10.11.4 ( EL Captain),
Docker 11.1

[mattlubner]

@sadhasivam Have you tried using the open-source version of Cisco's VPN client, openconnect? You can install it via Homebrew and it won't mess with the underlying network table entries.

I can confirm that openconnect works with Docker Machine (and Dinghy) while connected to my office's Cisco SSL VPN. I'm on the latest El Cap release (btw I think you have a typo there – no Mac OS 10.11.14 release yet).

[johntdyer]

Fwiw Cisco Anyconnect administrators can easily enable policy's which prevent users from using openconnect.

Sent from mobile

On May 8, 2016, at 17:59, Matt Lubner notifications@github.com wrote:

@sadhasivam Have you tried using the open-source version of Cisco's VPN client, openconnect? You can install it via Homebrew and it won't mess with the underlying network table entries.

I can confirm that openconnect works with Docker Machine (and Dinghy) while connected to my office's Cisco SSL VPN. I'm on the latest El Cap release (btw I think you have a typo there – no Mac OS 10.11.14 release yet).

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

[sadhasivam]

@mattlubner corrected the typo. i haven't tried openconnect yet. let me check and ping back the group.

[pdxmholmes]

Very similar issues on Windows 7 64-bit using Juno Pulse. I have to create the VM off of the VPN (which is fine), but once I connect to the VPN I can no longer interact with the VM through Docker. What's odd is that I can still SSH in to the machine and ping it.

Still playing with some port forwarding things and some routing, but so far nothing has solved it with my particular setup.

[iamKurt]

@nathanleclaire those steps didn't work for me. Here's what happened:

$ docker-machine ssh default -L 2376:localhost:2376 -fN
$ eval $(docker-machine env)
Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which will stop running containers.

So, I thought maybe you meant to add default and then I tried this:

$ eval $(docker-machine env default)

I basically got the same error message (above).

Running the regenerate-certs command also failed:

$ docker-machine regenerate-certs default
Regenerate TLS machine certs?  Warning: this is irreversible. (y/n): y
Regenerating TLS certificates
Waiting for SSH to be available...
Detecting the provisioner...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...

This machine has been allocated an IP address, but Docker Machine could not reach it successfully.

SSH for the machine should still work, but connecting to exposed ports, such as the Docker daemon port (usually <ip>:2376), may not work properly.

You may need to add the route manually, or use another related workaround.

This could be due to a VPN, proxy, or host file configuration issue.

You also might want to clear any VirtualBox host only interfaces you are not using.

[stevenmcdermott8]

looked for a solution for hours, thanks @nathanleclaire for the solution with the ssh port forwarding

[sosimon]

I struggled with this issue on and off for the last few weeks and this is what worked for me: http://www.iancollington.com/docker-and-cisco-anyconnect-vpn/. Thank you Ian!

[sadhasivam]

Folks. With the latest Mac OSX Docker binaries, Ref: (beta.docker.com) everything simplified to the core. Since VirtualBox image concept is removed in the latest version. Anyconnect is not the stopper AnyMore

checkout blog.docker.com for more information.

[sarusso]

This should do all the machinery required for setting up docker-machine with local port forwarding: https://github.com/onejli/docker-vpn-helper. Plus explains very well where problems are in using docker-machine with a VPN that intercepts all the traffic.