Limiting disk I/O when creating new torrent (hashing)

[ngjermundshaug]

We run libtorrent on servers that also has a lot of disk IO with higher Priority.

I am trying to limit the disk IO when creating torrents (file and piece hashing).

So far I have successfully been able to do the SHA1 hash of the file myself (limiting disk IO) - and then add the hash using add_file_borrow

But I am having some trouble navigating the documentation on how to do the piece hashes.
Any pointers would be appreciated.

(Also open to other ways of solving this).

[arvidn]

So far I have successfully been able to do the SHA1 hash of the file myself (limiting disk IO) - and then add the hash using add_file_borrow

The sha-1 hash of the file is an optional extension (and it's deprecated since bittorrent v2 solves this problem properly).

Looking at the make_torrent.cpp example for reference, I take it it's this line that's the problem for you:

https://github.com/arvidn/libtorrent/blob/RC_2_0/examples/make_torrent.cpp#L285

	lt::set_piece_hashes(t, branch_path(full_path)
		, [num] (lt::piece_index_t const p) {
			std::cerr << "\r" << p << "/" << num;
		});

Because it will read the content of all files and hash them. It will even spawn multiple threads for the hashing.

There are multiple overloads of that function (docs), one of them takes both a settings_pack and a disk_io_constructor_type.
You may want to experiment passing in a settings pack where you set settings_pack::hashing_threads to 1 (docs) and possibly where you pass in posix_disk_io_constructor (docs) as the disk I/O.

[ngjermundshaug]

I "solved" it by passing a progress callback method to lt::set_piece_hashes. This method get invoked for every hashed piece, so I added some good old fashioned sleeping here (dynamically calculated based on the current throughput). Not the most elegant solution, but it seems to work fine.