Seeding/leeching over multiple internet connections

[thanhminhmr]

The situation

I have a RPi that run 24/7 to seeding/leeching stuff. The previous setup is RPi -> Modem/Router combo -> ISP. This obviously work fine.
Now I have multiple seperate internet connections with different public IP addresses. I changed my setup to this: (simplified to 2 ISP)

    -> Modem/Router combo A -> ISP A
   /
RPi
   \
    -> Modem/Router combo B -> ISP B

This works, but the download/upload speed is still limited to single connection.

I also tried running multiple torrent clients and assign each client to its own internet connection. This approach works, except that each client download to a separate files, this takes a lot of disk space/io time (which is limited on the RPi).

Goal

My goal is to maximize the speed, that means the RPi (or to be more precise, the torrent client) needs to be aware of multiple internet connections and use it wisely. How do I achieve this goal?

More info

• The speedtest result of those ISPs were around 20Mbit/s. (Yes, it sucks. No, I can't upgrade any of it.)
• I'm quite sure that the RPi disk/cpu/ethernet speed is not a problem, at least for the case of those ISPs.
• AFAIK, there is no way to perfectly join two different internet connections, at least for the general use case.
  • I learned (and also tried) to implement a way to simply round-robin the network connections to different internet connections. This will not work since the public IP address changes between connections. Most public servers/services will see this as a huge red flag.
  • I also learned that different internet connections can also have different latency to the same server, so even joining multiple internet connections using a VPN will be kinda sucks.
  • I think that the only way for a program to use multiple internet connections properly and effectively is to make that program aware of multiple internet connections.
• I don't know if this question is off topic or not. And I'm so sorry for my terrible English.

[AllSeeingEyeTolledEweSew]

I've pursued a similar configuration, where I tried to use of several VPN connections over a single underlying connection, to maximize bandwidth while still having VPN protection. I got the best results using:

• One Alpine docker container, with several (VPN or other) WAN links, and pingu to round-robin them, such that each outgoing connection is pinned to a random WAN link
• Another docker container running my bittorrent client, using --network=container:the_vpn_container

One thing I noticed is that at any given time, your torrent client must announce itself on just one IP. My outgoing peer connections were distributed between WAN links, but all incoming connections came over the single link used for my last announce, which created a huge bias. The target of the bias shifts with each announce, but peer connections are never evenly distributed between links. I never found a way around this problem.

I was using private http trackers in this case, so DHT wasn't an option. I'm not sure how libtorrent's DHT code handles multiple WAN links.

[arvidn]

This setup is supposed to be supported natively by libtorrent. I would be very interested in your results.

set listen_interfaces to a comma-separated list of IP:port, where IP can either be the name of your network device or the IP address it's configured with. This controls which interfaces you will accept incoming connections over.

set outgoing_interfaces to a comma separated list of IP. Again, where IP can be either the network device name or its IP address. This controls what local interface outgoing connections are bound to. If there's more than one, outgoing connections will round-robin this list. Even though you can specify an IP address here, in order for this to actually work, you need to specify device names. Using device names will trigger libtorrent to use BINDTODEVICE, which is the only way (I know of) to actually force a connection to use a network other than the default route.

With those two settings, libtorrent will:

• announce to trackers once for each listen socket, over each network. This means trackers will see both of your IPs, and hand them out to peers.
• attempt to forward ports using NAT-PMP/NAT-PCP/UPnP for each network. i.e. if both of your internet connections go via a NAT router, ports will be forwarded on both, to enable receiving incoming connections.
• run a DHT node for each network, and announce your presence from both network connections.

=limitations

• there was a bug that was fixed recently (in RC_2_0, not yet released) causing tracker announces to not bind to the source network correctly.
• The peer list, by default, only allows a single connection per remote IP. Even though you have two networks on your end, you can still only have a single connection to a single peer, not one for each network connection.

[thanhminhmr]

yes, the same peer-id will be reported to the trackers. I would also expect private trackers wouldn't appreciate the same user seeding/downloading from more than one IP at the same time.

The DHT doesn't use peer IDs, just IP addresses for peers.

This is exactly the problem I'm facing with. Is there a way to assign different peer-id to different network interfaces? That way the client will appear as 2 different clients from the tracker point of view.

[arvidn]

is there a benefit to appear as two separate clients? I would expect that public torrents would prefer you show up as the same and private torrents may cause the trackers to think you shared the torrent file with someone unauthorized.

Are you sure that's what you want? If private trackers dislikes seeing the same peer ID from multiple IPs, maybe they have a reason.

[AllSeeingEyeTolledEweSew]

I expect on public http trackers, it's fine to have multiple peer-ids and would avoid the single-interface bias problem. But private trackers would probably quickly ban clients who do this.

Most private trackers are based on gazelle, which uses peer-id as the key for clients. I expect a single client announcing with multiple peer-ids would mess with reporting total upload/download, which they care a lot about.

Policy-wise, all private trackers I've seen are OK with downloading/uploading from different clients on different IPs, since seedboxes are common. They're OK with a single peer-id changing IPs, since this is common with residential internet connections and/or CGNAT. I've never seen a private tracker have specific policies about "continuously" changing IPs (as my round-robin case), but probably only because it's uncommon to do so.

In theory, private tracker admins should welcome a single client using different peer-id for each IP, since it can increase swarm bandwidth. In practice, they really don't like experimenting with new ideas or policies. If libtorrent introduces an easy option to do this, some private trackers will respond by just banning all libtorrent-based clients.

[thanhminhmr]

Wow, so the torrent/tracker scene is much more complicated than just softwares and services. I never use any private trackers, and just trying to maximize the resource usage when seeding/leeching files with multiple ISP at once. If the problem is with the policy, I guess there is nothing much a software library can do.

[Delfistyaosani]

Hey guys, I use qbittorrent (it uses libtorrent right?), I got two WAN connections with different metrics and qbittorrent specifically bound to lower priority one to not interfere with gaming traffic. However when set to any interface it never actually uses both for leeching speed. Is that a qbittorrent-specific problem or metric-overriding behavior not coded?