Benji Read Performance

[adambmedent]

We have been doing some more testing with Benji and seem to be running into a odd performance issue.

We are pointing Benji at a logical volume. It reads the logical volume at around 200MB/s. I know my storage has considerably more performance. For example, if I have 2 benji backups running at the same time to the same storage, I can hit 400MB/s. Is there anything I can do so that a single backups can achieve those same speeds?

I have tried messing with the simultaneousReads and simultaneousWrites options but they don't seem to have an effect. With those set to 15 I hit 200MB/s, set to 30 and I still hit 200MB/s.

The storage has almost no latency and is under very little load.

[elemental-lf]

It's hard to say without some profiling to find the bottleneck. Benji has only seen limited benchmarking and performance tuning and my use case has relatively small volumes and the change rate is small too, so it isn't a priority for me at the moment. If someone with experience in Python profiling would look into this, I'd be willing to help interpret the results and work on a solution.
You could also try to look at the debugging output to see if you can deduce something from it. The reading and writing of blocks should output some time measurements.

[adambmedent]

Ok, ill see what I can figure out, but this is definitely a bit over my head.