A libtorrent beginner seeking help.

[leoncwu]

Hi,

I am completely new to both bittorrent and the libtorrent library. I was struggling with the concept of tracker and seed yesterday. So, my questions will be quite naive to all of you. I have successfully installed the library and was able to compile and run the example posted on the official website of libtorrent (https://www.libtorrent.org/examples.html). However, I am not able to test if I can download anything from the web due to the security of network I am using. I will probably have to set up my own tracker. Here are my questions:

1. I can create a torrent, but how to create a magnet url from the torrent? In the client example posted on the libtorrent website, the code takes a magnet url as input to get a torrent.

2. Does a client require both a tracker and magnet url to download?

3. I understand that a client can choose to be a seed after downloading the whole file. Then, why do we have the option to provide a web seed when making the torrent?

4. Does libtorrent support creating a private tracker? If not, what tool do you recommend to create a private tracker?

I will also really appreciate it if anyone can recommend any book or reading material to explain all the concepts of bittorrent. Please bear with all these simple questions I posted in such a professional community.

Thanks in advance for the help.

[arvidn]

(1) you can take a look at this example: https://github.com/arvidn/libtorrent/blob/RC_2_0/examples/torrent2magnet.cpp

(2) as long as you can join the DHT, you don't need a tracker, you can look up peers from the DHT. The tracker is associated with a torrent, not with the client downloading the torrent.

(3) Having a client seed the content or a web server are orthogonal. You can do one or the other or both. Typically, a client that has all content and stays in the swarm is referred to as a seed, even though it's not the initial copy (as the term seed would suggest).

(4) libtorrent does not come with a tracker implementation. If you need to run one, you could check out https://github.com/arvidn/utrack or open tracker. libtorrent does support creating private torrents.

I gave a lecture on bittorrent in 2006, and you can find its slides here: http://libtorrent.org/bittorrent.pdf

[leoncwu]

Hi Arvidn. It is me again. How to mimic a broadcast process using libtorrent? In other words, only peer A has the file among a group of peers (A, B, C, D).

1. If all peers start to download the file, will peer B, C, D only grab file pieces from the peer A? Or does libtorrent optimize such that other peers (B, C, D) will also download pieces from each other when some have the pieces that others do not have?
2. Do I need to implement a linear or binomial tree algorithm to avoid performance issue if we had thousands of peers downloading the same file from a peer?
3. In this scenario, do we have any solution that does not require a tracker to coordinate the file sharing for a fixed group of peers that do not change?
   Thank you in advance for your help.

[arvidn]

This looks like it should be a new "question", not a comment on an existing one.

1. A central feature of bittorrent is that all downloaders also participate in sharing the pieces they have completed with other peers in the swarm. So, as long as B, C and D can connect to each other (e.g. they have a public IP address, or have UPnP/PCP port forwarding), they should be able to help each other.
2. The way bittorrent optimizes piece picking (i.e. the order in which pieces are downloaded) is by picking the "rarest first". You can find this described on more detail on the internet; the gist of it is that the peers a client is connected to is used as a proxy for the piece distribution in the whole swarm, and the pieces that the fewest peers have are prioritized. This makes the piece distribution tend towards being even, which means maximizing robustness against dropped peers and maximizing the upload capacity. For pieces that have the same rarity, a piece is picked at random, to make different peers pick different pieces.
3. the only thing the tracker does is to let peers know about each other. If you know of every peer up-front, you can just tell libtorrent about them via connect_peer(). If peers fail too many times, you may need to re-add them via this function for libtorrent to try again.

[leoncwu]

Thank you so much for your detailed explanation.

[leoncwu]

Hi Arvidn, I am trying to build the library of the latest libtorrent myself. I tried to build with cmake. So, I have cmake, ninja, an boost installed. I was able to get went throught step 1 withour any error. However, when I tried to execute "ninja", the build failed with the following error:
/usr/bin/c++ -DBOOST_ASIO_ENABLE_CANCELIO -DBOOST_ASIO_HAS_STD_CHRONO -DBOOST_ASIO_NO_DEPRECATED -DBOOST_EXCEPTION_DISABLE -DOPENSSL_NO_SSL2 -DTORRENT_BUILDING_LIBRARY -DTORRENT_BUILDING_SHARED -DTORRENT_SSL_PEERS -DTORRENT_USE_LIBCRYPTO -DTORRENT_USE_OPENSSL -D_FILE_OFFSET_BITS=64 -Dtorrent_rasterbar_EXPORTS -I../include -I../deps/try_signal -O3 -DNDEBUG -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wextra -Wpedantic -Wvla -Wno-noexcept-type -Wno-format-zero-length -ftemplate-depth=512 -fexceptions -pthread -std=gnu++14 -MD -MT CMakeFiles/torrent-rasterbar.dir/src/alert_manager.cpp.o -MF CMakeFiles/torrent-rasterbar.dir/src/alert_manager.cpp.o.d -o CMakeFiles/torrent-rasterbar.dir/src/alert_manager.cpp.o -c ../src/alert_manager.cpp
In file included from ../include/libtorrent/torrent_handle.hpp:71:0,
from ../include/libtorrent/alert_types.hpp:50,
from ../include/libtorrent/aux_/alert_manager.hpp:42,
from ../src/alert_manager.cpp:37:
../include/libtorrent/address.hpp:47:40: fatal error: boost/asio/ip/network_v4.hpp: No such file or directory
#include <boost/asio/ip/network_v4.hpp>

Do you happen to know what caused this?

[arvidn]

you're posting this question in the wrong place. Please post it as a question (instead of a comment to an answer)