lock contention (?) while downloading from the gateway/linksharing with high throughput

[amwolff]

We found out that a single AMD EPYC 7702P can only push out as much as 7.5-10 Gbps. We think that it might be possible to push out at least as much as twice more, but there's some kind of contention happening deep within the runtime (maybe caused by how we use the language?).

Flamegraph: https://pprof.host/qw0g/flamegraph

[azdagron]

Do we know what version of the Linux kernel was running on that box?

[azdagron]

And the version of Go used to compile the gateway-mt binary?

[halkyon]

@azdagron

• Linux 6.2 (we also saw it with 5.19 before it got upgraded on Ubuntu 22.04 HWE)
• Go 1.20.6

[amwolff]

@pwilloughby I bumped the priority of this. Feel free to grab it.