smb/cifs performance problem

[jumoog]

I mount an external storage via cifs and then use it as source for gocryptfs. When I copy a single file, the transfer speed is always ~24MB/s. If I copy multiple files with one process, the speed is also only ~24MB/s. But when I use different processes, I can reach the maximum 125MB/s of my network card. Does gocryptfs limit the speed per process?

Im using Ubuntu 22.04 and the newest gocryptfs version.

//xxxxxxxxxxxxx /mnt/crypt cifs credentials=/root/.smbcredentials,iocharset=utf8,rw,_netdev,uid=0,gid=0,file_mode=0660,dir_mode=0770  0 0
/mnt/crypt /mnt/uncrypt  fuse./root/gocryptfs/gocryptfs rw,nofail,auto,x-systemd.idle-timeout=10,x-systemd.automount,allow_other,quiet,passfile=/root/.gocryptfs 0 0

[lechner]

Hi,

I know very little about cifs but could this be a consequence of multiple processes waiting (and perhaps disk limits) on the server side?

Kind regards
Felix Lechner

[jumoog]

No. If you copy the files in 4 different processes, each process has a transfer rate of ~24MB/s. In the network monitor can also see that I really transfer with 4 x ~24MB/s.

[rfjakob]

Looks like a latency problem.

What throughput do you get when you copy to the cifs mount without gocryptfs?

[rfjakob]

I have a cifs server in the lan

rtt min/avg/max/mdev = 0.290/0.306/0.323/0.011 ms

and throughput is the same with and without gocryptfs on cifs (about 70MB/s).

Is this a Hetzner Storagebox?

[rfjakob]

(my guess is that the extra latency from gocryptfs makes cifs unable to utilize the whole bandwidth. Having multiple cp running concurrently works around the latency problem)

[jumoog]

Yes it's a Hetzner Storagebox. The perfomance is much much better now with gofuse v2.3.0

[rfjakob]

That's good to hear. You should try gocryptfs v2.3.1 or later, though, as this fixed a slowdown with NFS streaming ( https://github.com/rfjakob/gocryptfs#v231-2023-03-04 ). This could affect cifs just the same.

FYI I tested copying to a tmpfs samba share on a VM, where I added 10ms of latency to the VM like this:

tc qdisc add dev eth0 root netem delay 10ms

Testing with dd I get this:

dd if=/dev/zero of=zero bs=1M count=1000 status=progress

• Direct to cifs mount: 256 MiB/s
• gocryptfs mount on cifs mount: 233 MiB/s

In other words, I think you should be able to max out your gigabit link with gocryptfs.

[jumoog]

Thank you! Can I give you a donation?