Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to connect to containers after upgrade to MacOS docker desktop v4.23 #2723

Open
8 tasks done
polleke69 opened this issue Sep 18, 2023 · 38 comments
Open
8 tasks done

Comments

@polleke69
Copy link

Is there a pinned issue for this?

  • I have read the pinned issues and could not find my issue

Is there an existing or similar issue/discussion for this?

  • I have searched the existing issues
  • I have searched the existing discussions

Is there any comment in the documentation for this?

  • I have read the documentation, especially the FAQ and Troubleshooting parts

Is this related to a provider?

  • I have checked the provider repo for issues
  • My issue is NOT related to a provider

Are you using the latest release?

  • I am using the latest release

Have you tried using the dev branch latest?

  • I have tried using dev branch

Docker run config used

"version: '3.3'
services:
pollekevpn:
image: haugene/transmission-openvpn:dev
restart: unless-stopped
cap_add:
- NET_ADMIN
volumes:
- ./data:/data
- ./config:/config
environment:
- OPENVPN_PROVIDER=NORDVPN
- OPENVPN_CONFIG=romania
- NORDVPN_COUNTRY=RO
- NORDVPN_PROTOCOL=TCP
- NORDVPN_CATEGORY=legacy_p2p
- OPENVPN_USERNAME=
- OPENVPN_PASSWORD=
- LOCAL_NETWORK=192.168.178.0/24
logging:
driver: json-file
options:
max-size: 10m
ports:
- '9091:9091' # Transmission

Current Behavior

All related containers start up correctly and are in working order (according to the logs). The only issue I have is that I cannot connect to the container running on http://localhost:9091 with a webbrowser (safari or Brave)

a netcat (nc -v localhost 9091) gives a "connected"

if connecting from a webbrowser the connection times out with:
Safari can't open the page "localhost:9091/transmission/web/ because the server unexpectedly dropped the connection...

Expected Behavior

I would expect to see the transmission web interface.

How have you tried to solve the problem?

  1. From within the container I have checked network connectivity with a dns leak test script, which gives the expected output (DNS of VPN provider).

Log output

pollekevpn | Starting container with revision: 52d432d
pollekevpn | TRANSMISSION_HOME is currently set to: /config/transmission-home
pollekevpn | Creating TUN device /dev/net/tun
pollekevpn | Using OpenVPN provider: NORDVPN
pollekevpn | Running with VPN_CONFIG_SOURCE auto
pollekevpn | Provider NORDVPN has a bundled setup script. Defaulting to internal config
pollekevpn | Executing setup script for NORDVPN
pollekevpn | /etc/openvpn/nordvpn/..
pollekevpn | INFO: OVPN: Checking curl installation
pollekevpn | INFO: OVPN: DNS resolution ok
pollekevpn | INFO: OVPN: ok, configurations download site reachable
pollekevpn | INFO: OVPN: Removing existing configs in /etc/openvpn/nordvpn
pollekevpn | Checking NORDPVN API responses
pollekevpn | INFO: OVPN:Selecting the best server...
pollekevpn | INFO: OVPN: Searching for country : RO (179)
pollekevpn | INFO: OVPN: Searching for group: legacy_p2p
pollekevpn | INFO: OVPN:Searching for technology: openvpn_tcp
pollekevpn | INFO: OVPN: Best server : ro77.nordvpn.com, load: 7
pollekevpn | Best server : ro77.nordvpn.com
pollekevpn | INFO: OVPN: Downloading config: ro77.nordvpn.com.ovpn
pollekevpn | INFO: OVPN: Downloading from: https://downloads.nordcdn.com/configs/files/ovpn_tcp/servers/ro77.nordvpn.com.tcp.ovpn
pollekevpn | OVPN: NORDVPN: selected: ro77.nordvpn.com, VPN_PROVIDER_HOME: /etc/openvpn/nordvpn
pollekevpn | Starting OpenVPN using config ro77.nordvpn.com.ovpn
pollekevpn | Modifying /etc/openvpn/nordvpn/ro77.nordvpn.com.ovpn for best behaviour in this container
pollekevpn | Modification: Point auth-user-pass option to the username/password file
pollekevpn | Modification: Change ca certificate path
pollekevpn | Modification: Change ping options
pollekevpn | Modification: Update/set resolv-retry to 15 seconds
pollekevpn | Modification: Change tls-crypt keyfile path
pollekevpn | Modification: Set output verbosity to 3
pollekevpn | Modification: Remap SIGUSR1 signal to SIGTERM, avoid OpenVPN restart loop
pollekevpn | Modification: Updating status for config failure detection
pollekevpn | Setting OpenVPN credentials...
pollekevpn | adding route to local network 192.168.178.0/24 via 172.18.0.1 dev eth0
pollekevpn | 2023-09-18 16:58:39 OpenVPN 2.5.5 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 14 2022
pollekevpn | 2023-09-18 16:58:39 library versions: OpenSSL 3.0.2 15 Mar 2022, LZO 2.10
pollekevpn | 2023-09-18 16:58:39 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
pollekevpn | 2023-09-18 16:58:39 NOTE: --fast-io is disabled since we are not using UDP
pollekevpn | 2023-09-18 16:58:39 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
pollekevpn | 2023-09-18 16:58:39 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
pollekevpn | 2023-09-18 16:58:39 TCP/UDP: Preserving recently used remote address: [AF_INET]86.106.137.11:443
pollekevpn | 2023-09-18 16:58:39 Socket Buffers: R=[131072->131072] S=[16384->16384]
pollekevpn | 2023-09-18 16:58:39 Attempting to establish TCP connection with [AF_INET]86.106.137.11:443 [nonblock]
pollekevpn | 2023-09-18 16:58:39 TCP connection established with [AF_INET]86.106.137.11:443
pollekevpn | 2023-09-18 16:58:39 TCP_CLIENT link local: (not bound)
pollekevpn | 2023-09-18 16:58:39 TCP_CLIENT link remote: [AF_INET]86.106.137.11:443
pollekevpn | 2023-09-18 16:58:39 TLS: Initial packet from [AF_INET]86.106.137.11:443, sid=6484fdfd 302a6858
pollekevpn | 2023-09-18 16:58:40 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA
pollekevpn | 2023-09-18 16:58:40 VERIFY OK: depth=1, O=NordVPN, CN=NordVPN CA8
pollekevpn | 2023-09-18 16:58:40 VERIFY KU OK
pollekevpn | 2023-09-18 16:58:40 Validating certificate extended key usage
pollekevpn | 2023-09-18 16:58:40 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
pollekevpn | 2023-09-18 16:58:40 VERIFY EKU OK
pollekevpn | 2023-09-18 16:58:40 VERIFY X509NAME OK: CN=ro77.nordvpn.com
pollekevpn | 2023-09-18 16:58:40 VERIFY OK: depth=0, CN=ro77.nordvpn.com
pollekevpn | 2023-09-18 16:58:40 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512
pollekevpn | 2023-09-18 16:58:40 [ro77.nordvpn.com] Peer Connection Initiated with [AF_INET]86.106.137.11:443
pollekevpn | 2023-09-18 16:58:40 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 103.86.96.100,dhcp-option DNS 103.86.99.100,explicit-exit-notify,comp-lzo no,route-gateway 10.7.1.1,topology subnet,ping 60,ping-restart 180,ifconfig 10.7.1.2 255.255.255.0,peer-id 0,cipher AES-256-CBC'
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: timers and/or timeouts modified
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: --explicit-exit-notify can only be used with --proto udp
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: compression parms modified
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: --ifconfig/up options modified
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: route options modified
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: route-related options modified
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: peer-id set
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: adjusting link_mtu to 1659
pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: data channel crypto options modified
pollekevpn | 2023-09-18 16:58:40 Outgoing Data Channel: Cipher 'AES-256-CBC' initialized with 256 bit key
pollekevpn | 2023-09-18 16:58:40 Outgoing Data Channel: Using 512 bit message hash 'SHA512' for HMAC authentication
pollekevpn | 2023-09-18 16:58:40 Incoming Data Channel: Cipher 'AES-256-CBC' initialized with 256 bit key
pollekevpn | 2023-09-18 16:58:40 Incoming Data Channel: Using 512 bit message hash 'SHA512' for HMAC authentication
pollekevpn | 2023-09-18 16:58:40 net_route_v4_best_gw query: dst 0.0.0.0
pollekevpn | 2023-09-18 16:58:40 net_route_v4_best_gw result: via 172.18.0.1 dev eth0
pollekevpn | 2023-09-18 16:58:40 ROUTE_GATEWAY 172.18.0.1/255.255.0.0 IFACE=eth0 HWADDR=02:42:ac:12:00:02
pollekevpn | 2023-09-18 16:58:40 TUN/TAP device tun0 opened
pollekevpn | 2023-09-18 16:58:40 net_iface_mtu_set: mtu 1500 for tun0
pollekevpn | 2023-09-18 16:58:40 net_iface_up: set tun0 up
pollekevpn | 2023-09-18 16:58:40 net_addr_v4_add: 10.7.1.2/24 dev tun0
pollekevpn | 2023-09-18 16:58:40 net_route_v4_add: 86.106.137.11/32 via 172.18.0.1 dev [NULL] table 0 metric -1
pollekevpn | 2023-09-18 16:58:40 net_route_v4_add: 0.0.0.0/1 via 10.7.1.1 dev [NULL] table 0 metric -1
pollekevpn | 2023-09-18 16:58:40 net_route_v4_add: 128.0.0.0/1 via 10.7.1.1 dev [NULL] table 0 metric -1
pollekevpn | Up script executed with device=tun0 ifconfig_local=10.7.1.2
pollekevpn | Updating TRANSMISSION_BIND_ADDRESS_IPV4 to the ip of tun0 : 10.7.1.2
pollekevpn |
pollekevpn | -------------------------------------
pollekevpn | Transmission will run as
pollekevpn | -------------------------------------
pollekevpn | User name: root
pollekevpn | User uid: 0
pollekevpn | User gid: 0
pollekevpn | -------------------------------------
pollekevpn |
pollekevpn | Updating Transmission settings.json with values from env variables
pollekevpn | Attempting to use existing settings.json for Transmission
pollekevpn | Successfully used existing settings.json /config/transmission-home/settings.json
pollekevpn | Overriding bind-address-ipv4 because TRANSMISSION_BIND_ADDRESS_IPV4 is set to 10.7.1.2
pollekevpn | Overriding download-dir because TRANSMISSION_DOWNLOAD_DIR is set to /data/completed
pollekevpn | Overriding incomplete-dir because TRANSMISSION_INCOMPLETE_DIR is set to /data/incomplete
pollekevpn | Overriding rpc-password because TRANSMISSION_RPC_PASSWORD is set to [REDACTED]
pollekevpn | Overriding rpc-port because TRANSMISSION_RPC_PORT is set to 9091
pollekevpn | Overriding rpc-username because TRANSMISSION_RPC_USERNAME is set to
pollekevpn | Overriding watch-dir because TRANSMISSION_WATCH_DIR is set to /data/watch
pollekevpn | sed'ing True to true
pollekevpn | STARTING TRANSMISSION
pollekevpn | Transmission startup script complete.
pollekevpn | 2023-09-18 16:58:40 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
pollekevpn | 2023-09-18 16:58:40 Initialization Sequence Completed

HW/SW Environment

- OS: MacOS Ventura 13.5.2
- Docker: Docker Desktop 4.23

Anything else?

I am not completely sure, but it looks like after upgrading to docker desktop 4.23 this issue started...

@polleke69
Copy link
Author

Just downgraded docker desktop to 4.22.1 and its working again.

@pkishino
Copy link
Collaborator

this is a docker issue.. please check after upgrade if your file sharing changed to VirtioFS instead of gRPC Fuse..if so, change back to Fuse..this causes issues and there is a long open issue for this on the docker for-mac issue tracker..
I run this container on macOS with 4.23 and it works fine using Fuse

@pkishino pkishino closed this as not planned Won't fix, can't repro, duplicate, stale Sep 19, 2023
@polleke69
Copy link
Author

I'm sorry to say, but upgrading to 4.23 again, changing VirtioFS to gRPC Fuse doesn't work...

@pkishino
Copy link
Collaborator

After upgrading, did you try to stop/start the container again? Not just start after the upgrade of docker?
and you are trying to connect to http://localhost:9091/transmission/web/ right?

@polleke69
Copy link
Author

Yes, I even Rebooted and recreated the containers...

@Diegus83
Copy link

I experienced the same issue today with Docker 4.23

I'm running macOS 11.7.10 so the VirtioFS option is not available for me (macOS 12 and above only.

So my settings are still set to gRPC Fuse.

I'm installing 4.22 now and will update if this changes anything.

@pkishino
Copy link
Collaborator

Strange, I created a new container on my Mac rubbing 13.6 with 4.23 docker using the 5.2.0 tag and had no issues, tested to load a few torrents etc..

@Diegus83
Copy link

I installed 4.22 from Docker's website (my previous install of 4.23 was using brew) and it is working.

I didn't had to recreate my containers, simply started them and could connect to Transmission on the first try.

One thing to note is that I checked my compose file and I'm running the 4.1 image, I never got around to update past that.

So it seems whatever this issue is affects newer and older versions of the image but was introduced in Docker 4.23

Is there any other information I could provide to help?

@pkishino
Copy link
Collaborator

please check here..im also trying to find if a related issue already exists
https://github.com/docker/for-mac/issues

@pkishino
Copy link
Collaborator

curious, for those of you that it does not work are you running on apple silicone?

@Diegus83
Copy link

In my case I'm running on an Intel mac mini 2011 that I just updated to Big Sur using the OpenCore patcher, so before this week I was running an old version of Docker on an even older OS (High Sierra).

The only difference I noticed with my other containers that worked fine on 4.23 is that none of them use

cap_add: 
       - NET_ADMIN

Everything else in the .yml file is standard across containers: the time zone, user and group, etc.

@polleke69
Copy link
Author

Same here, the only container that doesn't work is the one with cap_add: net_admin.
All others run fine...

Running on Intel macOS Ventura 13.6 and docker desktop 4.23

@pkishino
Copy link
Collaborator

please try upgrade to 4.24 and see if problem still remains

@faximan
Copy link

faximan commented Sep 30, 2023

please try upgrade to 4.24 and see if problem still remains

Same problem for me on 4.24.

@polleke69
Copy link
Author

Same issue also on 4.24...

@nickpainter
Copy link

Seems to also impact any docker release after 4.22.1 (118664) on windows.

@pkishino pkishino mentioned this issue Oct 5, 2023
8 tasks
@mrchrisster
Copy link

mrchrisster commented Oct 7, 2023

Can't connect to the docker container anymore from my server. I'm using latest Haugen's on MacOS as well. No issues before I upgraded docker .. Is there a solution?
I tried 4.23 and 4.24 - both didn't work. Downgrading to 4.22 made it work again.

@pkishino
Copy link
Collaborator

pkishino commented Oct 7, 2023 via email

@pkishino
Copy link
Collaborator

pkishino commented Oct 7, 2023 via email

@pkishino
Copy link
Collaborator

pkishino commented Nov 9, 2023

this should be fixed, cannot encounter this any longer

@pkishino pkishino unpinned this issue Nov 9, 2023
@polleke69
Copy link
Author

@pkishino Which Docker Desktop version are u using?

@pkishino
Copy link
Collaborator

4.25.0

@polleke69
Copy link
Author

Any specific settings that should be mentioned?

@faximan
Copy link

faximan commented Nov 11, 2023

Still seeing this on 4.25.0, I'm staying on 4.22.1 which works well.

My compose:

transmission-openvpn:
    container_name: transmission
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
    volumes:
      - '/path:/config'
      - '/path:/data'
    environment:
      - OPENVPN_PROVIDER=PIA
      - OPENVPN_CONFIG=sweden
      - OPENVPN_USERNAME=user
      - OPENVPN_PASSWORD=pass
      - LOCAL_NETWORK=192.168.2.0/24
      - GLOBAL_APPLY_PERMISSIONS=true
      - TRANSMISSION_DOWNLOAD_DIR=/data
      - TRANSMISSION_INCOMPLETE_DIR_ENABLED=false
    logging:
      driver: json-file
      options:
        max-size: 10m
    ports:
      - '9091:9091'
    image: haugene/transmission-openvpn:5.2.0
    privileged: true

@mrchrisster
Copy link

Thanks for your report Alex. In that case I will hold off on updating

@pkishino pkishino reopened this Nov 12, 2023
@pkishino
Copy link
Collaborator

Sorry, my bad. They WireGuard beta version works on 4.25.0 but not the standard openvpn version strangely.. that one doesn’t work part 4.22.. haven’t had time recently to check the docker for Mac issue board in case anyone else might have a chance to

@polleke69
Copy link
Author

Have been searching for a couple of weeks now, tried this, tried that, but the only solution for now I found to "work" is to separate the openvpn and other containers but do place them in the same docker network. Not ideal, because the traffic from the other containers will not go through the vpn (transmission traffic will).

At the moment I'm also running on 4.22.1

`version: '3.3'

networks:
vpn:
name: vpn
driver: bridge
external: false

services:
vpn:
image: vpn:latest
container_name: vpn
cap_add:
- NET_ADMIN
restart: unless-stopped
volumes:
- ./data:/data
- ./config:/config
environment:
- OPENVPN_PROVIDER=
- OPENVPN_CONFIG=
- NORDVPN_COUNTRY=
- NORDVPN_PROTOCOL=TCP
- NORDVPN_CATEGORY=legacy_p2p
- LOCAL_NETWORK=192.168.178.0/24
logging:
driver: json-file
options:
max-size: 10m
ports:
- 9091:9091 # Transmission
- 8118:8118 # webproxy
networks:
- vpn

sonarr:
container_name: sonarr
image: hotio/sonarr
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- UMASK=002
volumes:
- ./sonarr/config:/config
- ./sonarr/data:/data
healthcheck:
test: curl --fail -s http://localhost:8989 || exit 1
interval: 1m00s
timeout: 20s
retries: 3
networks:
- vpn
ports:
- 8989:8989`

@ThePragmaticArt
Copy link

Just revisiting this and tried to update to 4.26.1 (131620) with the same ongoing issues and no resolution. Anyone know what issue is being tracked by Docker for this?

@wivaku
Copy link

wivaku commented Dec 17, 2023

Worth considering in general: split VPN and the standard (Transmission) app(s).

https://code.mendhak.com/run-docker-through-vpn-container/

Support for OpenVPN as well as WireGuard.
Simple to switch / set up:

version: "3"
services:
  vpn:
    image: qmcgaw/gluetun
    container_name: vpn
    # ...
    ports:
      # Transmission ports:
      - "0.0.0.0:9091:9091/tcp"   # <-- ports go here, not below
      - 51413:51413/tcp
      - 51413:51413/udp
  transmission:
    image: lscr.io/linuxserver/transmission:latest
    network_mode: "service:vpn"  # <-- important bit, don't forget
    # ...
    #ports:
    #  - "9091:9091"   # <-- ports don't go here

From browser you can continue to use http://localhost:9091.
In other Docker containers that refer to Transmission use http://vpn:9091 instead of http://transmission:9091.

No Docker issues.

@faximan
Copy link

faximan commented Feb 4, 2024

Just revisiting this and tried to update to 4.26.1 (131620) with the same ongoing issues and no resolution. Anyone know what issue is being tracked by Docker for this?

Bumping this. I believe I have other issues with this old version of Docker Desktop and I'd love to be able to upgrade, but this issue is blocking that.

Anybody has any new insights since last year?

@jacobonorte
Copy link

I second the above.
I am trying to transition into MAC and I am on the latest MAC OS. It is hard to find any older versions to even installer.

Do we have another way around this?

@mrchrisster
Copy link

mrchrisster commented Feb 4, 2024

I moved to another setup with wire guard for pia and QBT. There is a python script to move all torrents to QBT. Works with latest docker. Also a great speed improvement (from 2.5 MB/s to 25MB/s)

@fortinmike
Copy link

@mrchrisster Do you think this alternative setup works with the latest Docker because it's using Wireguard instead of OpenVPN? Or is there something else that this image does that's triggers the Docker bug/limitation?

I'd be interested in details about your alternative setup and how it compares to docker-transmission-openvpn in terms of user-friendliness.

@polleke69
Copy link
Author

I have switched to [https://github.com/qdm12/gluetun]

Works perfectly, also with latest docker desktop versions.

@mrchrisster
Copy link

mrchrisster commented Feb 4, 2024

@mrchrisster Do you think this alternative setup works with the latest Docker because it's using Wireguard instead of OpenVPN? Or is there something else that this image does that's triggers the Docker bug/limitation?

I'd be interested in details about your alternative setup and how it compares to docker-transmission-openvpn in terms of user-friendliness.

Haugene has said it works with wire guard and newest docker, so yes, it seems to be openVPN related

In terms of user friendliness, I prefer it because of the category path management system. I'm coming from a decade of transmission usage.

The best thing is you can set the network interface to wg0 so you don't need to worry about DNS leakage.

Here's my current setup:
thrnz/docker-wireguard-pia#98 (comment)

Glutun is great but doesn't work for pia with wire guard

@faximan
Copy link

faximan commented Feb 15, 2024

@mrchrisster Thank you for this - I have made the switch now as well and used your setup as inspiration. Works great.

With ~1300 torrents qBit is also more responsive in the UI, especially while downloading where for some reason Transmission always used to freeze up for me. Transfers also start quicker and I can reach higher max speeds (maybe that's because Wireguard instead of OpenVPN, not sure).

@alexdawn
Copy link

alexdawn commented Apr 3, 2024

Same issue on 4.28

@ssuess
Copy link

ssuess commented Apr 26, 2024

same issue on 4.29

@pkishino pkishino mentioned this issue May 7, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests