Issue with restarting VPN container

[amberflag48]

Hello,

I have autoheal set to restart my VPN container when it goes down. It is also configured to restart the containers linked to it when they go down.

The issue is when the VPN goes down, it gets restart, that is fine and expected. It does not however restart the containers depending to it. This is an issue because the web UI ports of those containers are mapped to the VPN container so I can access them like normal.

So for those web UI ports to be available back those containers need to be started AFTER the VPN container starts. With this configuration, only the VPN container is restarted since the other ones are still healthy anyway.

Is there a solution for that?

Thanks!

[mfizz1]

You need to set your other containers to have some sort of health check in place to indicate they are unhealthy for autoheal to pick up there is a problem. By default, it would go in order.

Simplest way would be to ping your VPN container or a port that can be accessed if everything is working properly.

[Snuffy2]

Here is how I did it:
https://gist.github.com/Snuffy2/1d49250df3a5c8fdb3a24d486df92015

[heisenberg2980]

@mfizz1 that solution you are propossing seems interesting, could you expand on that? do you know how to add a health check to a container running from docker-compose?

For example I am running qbittorrent which is dependent on gluetun, and I would like to mark the qbittorrent container as unhealthy when the qbittorrent port is not responding so autoheal can restart it, how can I do that?

This is my docker-compose.yaml file:

---
version: "3"
services:
  gluetun:
    image: qmcgaw/gluetun
    container_name: gluetun
    # line above must be uncommented to allow external containers to connect. See https://github.com/qdm12/gluetun/wiki/Connect-a-container-to-gluetun#external$
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun:/dev/net/tun
    ports:
      - 8888:8888/tcp # HTTP proxy
      - 8388:8388/tcp # Shadowsocks
      - 8388:8388/udp # Shadowsocks
      - 8080:8080     #qbittorrent
      - 6881:6881     #qbittorrent
      - 6881:6881/udp #qbittorrent
    volumes:
      - ./config/gluetun:/gluetun
###   environment removed

  qbittorrent:
    container_name: qbittorrent
    image: lscr.io/linuxserver/qbittorrent
    restart: unless-stopped
    network_mode: service:gluetun
    environment:
      - PUID=${PUID} # default user id, defined in .env
      - PGID=${PGID} # default group id, defined in .env
      - TZ=${TZ} # timezone, defined in .env
    volumes:
      - ./config/torrent:/config
      - /mnt/Download:/downloads
    depends_on:
       - gluetun

[hasnat]

its good to cover self + all dependencies within healthcheck, not via depends_on but via test within healthcheck.

e.g. am testing withing qbittorrent healthcheck if http of gluetun 8888 is ok, if ever not autoheal will attempt restart qbittorrent.

You'd need gluetun own healthcheck if you want autoheal to pickup when unhealthy and restart. e.g. curl -f http://127.0.0.1:8888

gluetun:
  ...

qbittorrent:
  ...
  healthcheck:
    # can possibly also add       "curl", "-f", "http://SELF_QBIT_TEST_URL", "&&", 
    test: ["CMD", "curl", "-f", "http://gluetun:8888"] 
    interval: 1m30s
    timeout: 10s
    retries: 3
  start_period: 40s
  ...

docker-autoheal:
  ...

depends_on is more linked to startup order/control, ref: https://docs.docker.com/compose/startup-order/
e.g. compose now will only start qbittorrent if gluetun has healthcheck and is healthy; (not much to do with autoheal)

qbittorrent:
  depends_on:
    gluetun:
      condition: service_healthy

[heisenberg2980]

@hasnat thanks for the quick reply, why are you using "http://gluetun:8888" instead of localhost:8888 or similar? http://gluetun:8888 is not a correct url, right? I have tried to run "curl -f http://gluetun:8888" within the container and got the error "Could not resolve host: gluetun" (as expected tbh).

I also tried using test: curl -f localhost:8080 || exit 1 in the healthcheck but the container always appears as healthy even if the qbittorrent service is not accesible at port 8080, but then I realise if I run the command curl --fail localhost:8080 || exit 1 inside the container it always returns a valid response, which make sense as the container is able to access the service internally and the issue only appears when I try to access the service from outside the container

[heisenberg2980]

Finally I found the way to do it. For anyone coming here with the same issue, the key was to add the parameter FIREWALL_OUTBOUND_SUBNETS=192.168.0.0/24 to Gluetun (not sure if this will apply to other vpn services) so the containers linked to Gluetun are able to communicate with the network (192.168.0.X). After that I just used the external IP of the container for the healthcheck so the container can check if its internal IP and port is accesible:

    healthcheck:
      test: curl -f 192.168.0.208:8080 || exit 1
      interval: 1m30s
      timeout: 10s
      retries: 2
      start_period: 30s

@amberflag48 you might want to close this defect as it is solved now

[hasnat]

@heisenberg2980 that was only a demonstration note, suggesting you can test/curl other containers in compose file/stack to validate health of the container.
say if gluetun had some 8888 http port, you could curl that from qbittorrent healthcheck to determine qbittorrent is fully healthy/needs-restart..

I didn't use IPs since containers were all in same compose file/network; accessible via container-name as hostname by eachother.

[heisenberg2980]

Thanks for the clarification, I also thought about that but as autoheal restarts gluetun automatically within seconds as soon as it becomes unhealthy, the healtcheck in qbittorrent wouldn´t even realised gluetun was down, so the only way is for each container (qbittorrent in this case) to check its own external port.

Regarding using the container name as hostname to ping/curl, I didn´t know that was possible, and I have tried different combinations inside the qbittorrent container (i.e. ping gluetun, ping gluetun:8888, ping http://gluetun:888, even ping qbittorrent and curl qbittorrent:8080) and none of them works, not sure why

[hasnat]

@heisenberg2980 those prots might not exist, even if they did might not be http for curl to understand; I only made them up for examples to show how to structure your health checks.
If you know ports or communication to your side-car/dependencies containers, do ping/pong tests to determine communication is healthy between them and no restart needed

same like for mysql+php stack, you could netcat mysql 3306 port (which isn't http) from php container.

good luck with it