Slow download from FTP with TLS on #1684
Comments
|
What |
|
Do you mean what did the command show? I attached it in file, but I can do that also here: Compile-time Settings: CFLAGS: -g2 -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong -fno-strict-aliasing -Wall -fno-omit-frame-pointer -fno-strict-aliasing -Werror=implicit-function-declaration Files: Info: Features: Tunable Options: Under features is + OpenSSL support (OpenSSL 1.1.1o-freebsd 3 May 2022) |
|
Thank you. And to help provide perspective, can you explain what you mean by "slow"? That is, when was it "fast" (timings, ProFTPD/OpenSSL/FreeBSD versions), before it became "slow"? I ask because usually differences in download timings occur because of a number of possibilities:
so having some idea of what might have changed, before and after, with regard to the above factors can help me narrow down what might be involved here. Thanks! |
|
This is a very hard question. |
|
I have conducted numerous tests with various different parameters. The best result was achieved by replacing version 1.3.8 with version 1.3.7f. The download of files in TLS remains slower, but at least at more acceptable speeds. In comparison, I installed pure-ftpd and to my surprise, this software does not support TLS in the data channel, i.e., there's no way to compare results from pure-ftpd with proftpd. The other program that supports TLS, vsftpd, had its latest update in 2021, so I think that proftpd is the program that is most up-to-date in terms of TLS. There would need to be a bit more effort from the developers to identify and resolve the cause of the problem, as it only happens on download and not on upload with TLS. |
I'd like to remind you that this is purely a volunteer effort, and "the developers" (just me, mostly) work on this project when we can, in our spare time (which isn't always available). |
|
I understand that this project is a volunteer effort and that the developers work on it in their spare time. I apologize for any inconvenience caused. I'm more than willing to assist with testing and provide any help within my capabilities to find a solution. Please let me know how I can contribute and support your efforts. Together, we can work towards resolving the issue at hand. |
|
What does the command |
Here is my proftpd.conf: Important: I also activated AESNI and added cryptodev_load="YES" to /boot/loader.conf. |
|
With the above configuration I could download files at ~40MB/second. |
With the above configuration I could download files at ~40MB/second. I tried this and it did not improve. I saw by multiple users that the problem occurs on FreeBSD/TrueNAS operating system. It may somehow be a culprit. I will try to downgrade proftpd and upgrade openSSL to get more similar environment to @mcoelho80. |
|
I'm not sure if it'd be possible in your environments, but another good comparison to try would be ProFTPD on Linux. That might help provide more data on how much the underlying OS (and its libraries, networking, etc) might play into this situation. |
OK, I will try this. |
|
Hmm. Since you mention a NAS as well, I wonder if the filesystem hosting the files being downloaded (i.e. mounted from the NAS or not) factors in as well. That is, if a large file not on the NAS was being downloaded (using ProFTPD on FreeBSD), does the download speed change? |
I am actually not using NAS. I just mentioned TrueNAS, because I saw it's users also complaining about slow speed and TrueNAS OS is based on FreeBSD. |
Ah, I see. Thanks for the clarification! |
In my case, the files are served from an SSD disk. In the tests conducted on a local network, the upload speed reached ~112MB/s, while the download speed was slightly over 40MB/s. However, I was only able to achieve these download speeds after making modifications. Specifically, I adjusted the SocketOptions in the proftpd.conf file and added devcrypto_load="YES" to /boot/loader.conf. |
Yes, this is the area. With these changes I was able to get download rate from around 100 kB to 3 MB per second. I am not able to set buffer size to more than 1600000, then I get "no buffer space available", I need to look at it deeper. However, it still significantly slower than upload and also download without TLS. For now though, thank you! |
|
@Castaglia |
Thanks for the reference. Before looking into that too much, though, I'm hoping we can track down the bottlenecks with your existing setup, so that we know we're tweaking/changing the areas to have the most effect. Performance tuning of file transfers involves quite a few factors, which is why it can take a frustratingly long time to figure out just what the bottlenecks are, as it covers:
It may be that tweaking socket buffer sizes and TLS ciphersuites helps -- or may not. I've seen cases where either the server network interface was saturated -- or the client network interface was saturated. Or cases where it was the client-side filesystem (e.g. writing a downloaded to an NFS mount which was slow) which caused the perception of "slow downloads". I'm not exactly sure of what the best way is to measure all of the above factors in your case; I'm just trying to point out all of the places we'll want to look, to make sure that we are making changes in the right areas, to have the most impact. Now, what helps is your earlier observation: Using the same client, same server, same file, the above does rule out a lot of the factors as being the most likely bottlenecks. You mention:
By "local network", do you mean you have the client running on the same host as the server, i.e. connecting to localhost/127.0.0.1? I ask because I'm wondering if there are any network routers or firewalls in the network path, between server and client, that might also be a factor. Even things like iptables/pf or any other kind of packet filtering on the server host might unexpectedly add latency. |
|
Per #1314 (comment), I'm re-examining code differences between 1.3.7e and 1.3.8. One that pertains to FTPS is the support for TLSv1.3. In your downloads, can you see which TLS protocol version (and ciphersuite) is being used? If you add |
|
Hi, I apologize for longer not writing as the solution with buffer was "somewhat" satisfying. However only on one client which I was regularly using. |
|
Same issue here, Downloading caps at 400KB/s and uploading caps around 3MB/s when TLS is enabled. It only happens with Proftpd, while Proftpd is crawling slow with TLS active, I can download from Nginx or vsftpd with TLS at 110MB/s 1 thread without problem. The FTP client doesn't matter, FileZilla, FlashFXP or CoreFTP are all crawling slow with any of their internal buffer settings (if available) until I manually set Proftpd's buffers with Also, Proftpd doesn't need huge SocketOptions buffers to be manually set to fix the issue, setting the buffers at the same value as the
edit: clarifications |
|
If you're able to build ProFTPD from source, I recommend trying out the latest code in the With these changes, you may no longer need (or want) the |
|
In addition, I've also filed #1729 to track support for KTLS via |
Hello, I am not sure if the topic was somehow resolved, but I am experiencing exactly the same issue as multiple users described in
#1314
Transfer times for a 10MB file:
TLSEngine off upload/download: 1 second
TLSEngine on upload: 1 second
TLSEngine on download: 2 minutes
It does not matter if I try it on local network or from outside, or client (WinSCP, Total Commander, Filezilla, AnyFTP), or client platform, it is always very slow.
I am attaching output of proftpd -V and proftpd.conf without ip address
ftpd.txt
proftpd -V output.txt
Thank you very much in advance for looking into it.
Stefan
The text was updated successfully, but these errors were encountered: