New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Page-Align Writes #7573
Comments
This seems familiar to something written here about multi-file torrents: |
It doesn't. I suspect it's really the memory mapped file that's causing issues. |
I'm seeing it using read/write for the files via strace, so I don't think
mmap is an issue.
|
right, there's a special case for small files. They are read and written using regular |
Hi there,
I've been digging a lot into performance issues I experience when using qBittorrent with its disk mounted over SMB/CIFS with a high-latency network. Also relevant is that it's an asymmetric network: I can write files about 25x faster than I can read them.
What I've found is that while (when CIFS is configured correctly), the page cache can elide most writes, some writes actually produce a synchronous read. This synchronous read, when downloading a torrent, ends up being a significant bottleneck (due to the asymmetric network).
libtorrent seems to switch from using
mmap
to usingread/write
when it detects that there's a network filesystem, however this doesn't fix all of the performance issues with it.In particular, what I believe happens is as follows:
write
call when making unaligned writes), Linux must backfill the unwritten data from disk in order to store that newly dirtied page in the page cache.write
call returns, having not written anything! The page will get written to the SMB server asynchronously later.I'll use an example: if a torrent has a 1M chunk size, is exactly 3 chunks in length, and contains two files, the first file is 1M+2k (1026 KiB), and the second is 2M-2k (2046 KiB), with a 4k page size. Let's also say that libtorrent is making
write
calls by passing in 1 MiB buffers. You'll get:I think the way to fix this is by adding a small buffer to defer partial writes to pages, and if you want to get complicated/fancy with it, you could calculate "these chunks combined together allow aligned writes" and prefer downloading those together.
The text was updated successfully, but these errors were encountered: