Any tips on how to tunnel pwd through VPN?

[exactsameproblem]

Hello there,
I would like to pass all the containers created inside PWD through a VPN.
I edited the docker compose file to reflect the changes (see code below).

I tried different things like:

• removing haproxy in order to reach the vpn service first and redirect it to PWD : I can't reach the backend even while specifying port 3000 in the playground domain : INTERNET -> Cloudflare domain -> VPN -> PWD
• putting the VPN in front of haproxy - not sure on how to redirect ports together : INTERNET -> Cloudflare domain -> VPN -> haproxy-> PWD
• adding a new frontend and backend to haproxy: INTERNET -> Cloudflare -> haproxy -> VPN -> haproxy -> PWD

On my playground domain page, I either get "503", "521" or "522 timeout" errors.

I'm using cloudflare with a ssl key configured inside haproxy, so the "normal" setup (without the vpn service) works like a charm, but I cannot pass pwd inside the VPN and reach it outside of localhost (by using the playground domain through cloudflare).

Here is my docker compose file:

version: '3.2'
services:
    haproxy:
        container_name: haproxy
        image: haproxy
        ports:
           - "80:8080" # http-in
           - "443:443" # https-in with ssl cert (cloudflare)
        volumes:
           - ./haproxy:/usr/local/etc/haproxy

vpn:
        depends_on:
            - haproxy
        container_name: nordvpn
        image: azinchen/nordvpn:latest
        cap_add:
            - net_admin
        devices:
            - /dev/net/tun
        environment:
           - USER=mail
           - PASS=pass
           - COUNTRY=Germany;IE;131
           - GROUP=Standard VPN servers
           - RANDOM_TOP=10
           - RECREATE_VPN_CRON=5 */3 * * *
           - NETWORK=172.26.0.0/16;177.17.0.0/16;177.20.0.0/16 # tried differents ip with no sucess
           - OPENVPN_OPTS=--mute-replay-warnings
     
        restart: unless-stopped

 pwd:
        depends_on:
            - vpn
        network_mode: service:vpn 
        # pwd daemon container always needs to be named this way
        container_name: pwd
        # use the latest golang image
        image: golang
        # go to the right place and starts the app
        command: /bin/sh -c 'ssh-keygen -N "" -t rsa -f /etc/ssh/ssh_host_rsa_key >/dev/null; cd /go/src/; if [ -e /runbin/pwd ]; then /runbin/pwd  -save /pwd/sessions -name l2; else go run api.go -save /pwd/sessions -name l2; fi'
        volumes:
            # since this app creates networks and launches containers, we need to talk to docker daemon
            - /var/run/docker.sock:/var/run/docker.sock
            # mount the box mounted shared folder to the container
            - $PWD:/go/src
            - sessions:/pwd

    l2:
        depends_on:
            - vpn2
        network_mode: service:vpn2	
        container_name: l2
        # use the latest golang image
        image: golang
        # go to the right place and starts the app
        command: /bin/sh -c 'ssh-keygen -N "" -t rsa -f /etc/ssh/ssh_host_rsa_key >/dev/null; cd /go/src/router/l2; if [ -e /runbin/l2 ]; then /runbin/l2 -ssh_key_path /etc/ssh/ssh_host_rsa_key -name l2 -save /pwd/networks; else go run l2.go -ssh_key_path /etc/ssh/ssh_host_rsa_key -name l2 -save /pwd/networks; fi'
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - $PWD:/go/src
            - networks:/pwd
        ports:
            - "8022:22"
            - "8053:53"   

volumes:
    sessions:
    networks:

Any help on how it would be possible to achieve this?
Thank you.

[marcosnils]

If you configure your host to connect to the VPN, all the containers inside PWD should use the VPN since the traffic is routed through the host. Would that work for you?

[exactsameproblem]

Hi Marcos,
Thank you for the advice! I didn't think about that. Unfortunately I lose the connecion (SSH) to my server when I activate the VPN. I'm trying to add a route and/or whitelist the port 22 but without sucess for now. I feel like it may have something to do with the network settings added to make docker/PWD run when I installed them.
By the way, I want to thank you for your work, I'm watching some of your conferences on Youtube and I learn a lot. You are an inspiration to me.
Best,
Max

[marcosnils]

Awesome!

don't hesitate to ask here in case it's giving too much trouble.

Cheers!

[exactsameproblem]

I may need your help if you know about this networking stuff 😀

Here is my config (X.X.X.X being my server public IP):

ip route show
default via X.X.X.254 dev eno1 proto dhcp src X.X.X.X metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.20.0.0/16 dev docker_gwbridge proto kernel scope link src 172.20.0.1
172.21.0.0/16 dev br-6c8ed30a7002 proto kernel scope link src 172.21.0.1 linkdown
172.27.0.0/16 dev br-74614dc112f3 proto kernel scope link src 172.27.0.1 linkdown
X.X.X.0/24 dev eno1 proto kernel scope link src X.X.X.X
X.X.X.254 dev eno1 proto dhcp scope link src X.X.X.X metric 100


route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         X.X.X.254 0.0.0.0         UG    100    0        0 eno1
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.20.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker_gwbridge
172.21.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-6c8ed30a7002
172.27.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-74614dc112f3
X.X.X.0   0.0.0.0         255.255.255.0   U     0      0        0 eno1
X.X.X.254 0.0.0.0         255.255.255.255 UH    100    0        0 eno1

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether e0:69:95:50:24:84 brd ff:ff:ff:ff:ff:ff
    inet X.X.X.X/24 brd X.X.X.255 scope global dynamic eno1
       valid_lft 86208sec preferred_lft 86208sec
    inet6 2001:41d0:2:bae1::/56 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::e269:95ff:fe50:2584/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:56:ff:c7:c7 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
4: br-6c8ed30a7002: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:54:58:9a:46 brd ff:ff:ff:ff:ff:ff
    inet 172.21.0.1/16 brd 172.21.255.255 scope global br-6c8ed30a7002
       valid_lft forever preferred_lft forever
5: br-74614dc112f3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:dd:aa:8d:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.27.0.1/16 brd 172.27.255.255 scope global br-74614dc112f3
       valid_lft forever preferred_lft forever
6: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:66:9e:ef:8a brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.1/16 brd 172.20.255.255 scope global docker_gwbridge
       valid_lft forever preferred_lft forever
    inet6 fe80::42:66ff:fe9e:ef8a/64 scope link
       valid_lft forever preferred_lft forever
12: vethbc26922@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
    link/ether 3a:e7:bb:ec:31:39 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::38e7:bbff:feec:3139/64 scope link
       valid_lft forever preferred_lft forever

I install nordvpn .deb package according to https://www.makeuseof.com/how-to-install-nordvpn-ubuntu/

I typed:

nordvpn login --legacy
Email: **
Pass: **

nordvpn connect

and poof* no more connection. I'm even unable to connect, I have to reboot the server from my hosting account.

I tried these commands: https://unix.stackexchange.com/questions/432709/what-exactly-does-these-ip-lines-do/432784#432784
To add a route. But with no sucess... I don't know if you are familiar with nordvpn or if you are able to reproduce the issue, but I used nordvpn whitelist add 22 which was supposed to help me connect to SSH...
I took days and I'm about to give up :-/
Thank you again!

[marcosnils]

I think I can help. I'll check it out tomorrow.

[exactsameproblem]

Hi Marcos,
Almost a year passed since my first message and I still don't have a solution.
I connect to the VPN from the host but PWD instances don't load at all (same problem than trying to SSH to the server: nothing happens).
I guess the best way would be to load a nordvpn container with docker-compose and pass the other containers to it? I tried something like this in my first message but I don't know docker well enough to make this work.
If you have any tips for that I'd gladly ready them
Best