Too much precision shown for transfer speed

[fulldecent]

The associated forum post URL from https://forum.rclone.org

n/a

What is the problem you are having with rclone?

When performing a transfer (rclone sync) too much precision is shown by default.

Currently up to 7 digits of precision are shown, which is unnecessary. The maximum should be 2 or maybe 3.

What is your rclone version (output from rclone version)

rclone v1.64.2
- os/version: darwin 14.2.1 (64 bit)
- os/kernel: 23.2.0 (arm64)
- os/type: darwin
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.21.3
- go/linking: dynamic
- go/tags: none
rclone v1.64.2
- os/version: darwin 14.2.1 (64 bit)
- os/kernel: 23.2.0 (arm64)
- os/type: darwin
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.21.3
- go/linking: dynamic
- go/tags: none

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

macOS 14

Which cloud storage system are you using? (e.g. Google Drive)

n/a

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

rclone sync --progress --multi-thread-streams=30 $from $to

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

n/a

[ncw]

I think you have chosen the worst case in your screenshot!

I agree though, that is too much precision.

Here is the reason why

rclone/fs/sizesuffix.go

Line 70 in 4e07a72

return fmt.Sprintf("%.3f", scaled), suffix

Maybe we should scale down the precision here

@albertony - I know this is an area you are interested in - thoughts?

[albertony]

Hm. I can agree that 3 decimals may be a bit much. But I'm not sure I would want to reduce it too much either, my feeling is we would quickly loose relevant information.

Quick look at others, git seem to use 2 decimals: 399.97 KiB | 1.47 MiB/s, couldn't we just go for the same?

Robocopy on Windows on the other hand is quite detailed... 😄

   Speed :           225 195 451 Bytes/sec.
   Speed :             12885,787 MegaBytes/min.

Was the suggestion to take a bit more drastically different approach to the whole scaling, could you give an example?

[ncw]

I was wondering whether we should be thinking about significant figures rather than decimals. So if we aimed for 5 Sig figs then we'd get

9999.9
999.99
99.999
9.9999

Which might be more sensible? And also constant width.

[albertony]

Which might be more sensible?

I think so - I like it.

Although 9.9999Ki/s could be argued is still unnecessary precision, I think if we were to reduce to 4 figures then 1023Mi instead of 1023.9Mi would loose relevant details.

And also constant width.

This is a big plus.

I don't know if there is a magic format string that would give exactly this result, but after some quick experimenting %#.5g is pretty close

package main

import (
	"fmt"
)

func printit(v float64) {
	fmt.Printf("%#.5g\n", v)
}
func main() {
	printit(111111.1111)
	printit(11111.11111)
	printit(1111.111111)
	printit(111.1111111)
	printit(11.11111111)
	printit(1.111111111)
	printit(.1111111111)
	printit(.0111111111)
	printit(.0011111111)
	printit(.0001111111)
}

1.1111e+05

11111.
1111.1
111.11
11.111
1.1111

0.11111
0.011111
0.0011111
0.00011111

Since we do the scaling to units we wouldn't get scientific notation like in the first result (until 999999Ei).

The last four could happen, but only for low unscaled byte values from speed calculations, like 0.011111 B/s. So to ensure constant width in all cases, we would need some more tricks to avoid this...

[albertony]

If not considering performance, a silly sprintf+parse+sprintf combo for values below zero would give the constant width aspect.

Changing the function to:

func printit(v float64) {
	if v < 1.0 {
		x, _ := strconv.ParseFloat(fmt.Sprintf("%#.5g", v), 64)
		fmt.Printf("%.4f\n", x)
	} else {
		fmt.Printf("%#.5g\n", v)
	}
}

Would change only the last four results, they would fit the width of the rest:

0.1111
0.0111
0.0011
0.0001

[fulldecent]

This number updates three times per second. There is no value in showing 12345 if the next 0.3 seconds from now it's going to be 15347. And this scenario happens frequently unless you are only transferring large files (eg 50Mb+) over a relatively slow connection (eg 50mb/s).

Here are good numbers

1.00k
6.35k
13.4k
74.3k
312k
736k There's an extra space after that for alignment, if you care about that
1.44m
8.35m
23.4m
74.5m

[albertony]

It is "binary" units, it would need to be at least 4 significant digits:

999.0Ki
1023.Ki
1.000Mi

[fulldecent]

999.0Ki
1023.Ki
1.000Mi

Those are easy to fix. Nobody cares about the .0 in 999.0Ki. And numbers do not format with an orphan . at end. And 1.000 is too much precision.

So a good formatting is

999Ki
1.0Mi
1.0Mi

If somebody really really cares about that extra third digit of precision in some of these things they're going to be misled. This probably means they're doing network performance profiling, and there's already overhead in all of the protocols that are clone uses. Therefore, they need to actually use a network performance monitoring tool to get the data they want.

[ncw]

This number updates three times per second. There is no value in showing 12345 if the next 0.3 seconds from now it's going to be 15347. And this scenario happens frequently unless you are only transferring large files (eg 50Mb+) over a relatively slow connection (eg 50mb/s).

For transfers you are right.

However this conversion is used for a lot of other things not just fluctuating bandwidth, for example

$ rclone size .
Total objects: 1.254M (1254146)
Total size: 43.023 GiB (46195194521 Byte)

We could have multiple conversions I suppose, but I'd rather pick one we can use everywhere for consistency.

The fixed width ones are attractive because they will look good in -P...