Profile-Guided Optimization (PGO) benchmark results

[zamazan4ik]

Hi!

I evaluate Profile-Guided Optimization (PGO) performance improvements for applications (and libraries) in different software domains - all current results can be found here. According to the tests, PGO helps to improve performance in various software domains. Since Rustls cares about performance I decided to perform PGO benchmarks on it. Here are the results.

Test environment

• Fedora 39
• Linux kernel 6.7.4
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.76
• rustls version: the latest for now from the main branch on commit eb0791bc94cfdc4c42f743902a35074ce58ced11

Benchmark

For benchmark purpose, I use ci-bench benchmark. Release measurements are done with taskset -c 0 cargo run --release -- walltime --iterations-per-scenario 100, PGO optimized with taskset -c 0 cargo pgo optimize run -- --release -- walltime --iterations-per-scenario 100, PGO training phase - with cargo pgo run -- --release -- walltime --iterations-per-scenario 1. Benchmarks are done in the single-threaded mode for better consistency across runs.

All PGO optimizations are done with cargo-pgo.

All measurements are done multiple times - the results are consistent enough across runs.

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/a52f89a4507d90c349bc23821afa11fc
• PGO-optimized: https://gist.github.com/zamazan4ik/e5f1a56b32266c5762ddc24741ed56a1

According to the benchmarks, I see improvements in many scenarios.

I think these results can be interesting to anyone who wants to improve Rustls performance for their crypto routines. Maybe mentioning these results somewhere in the README (or any other user-facing documentation) will be a good idea.

Please do not treat the issue as a bug or something like that - it's just a benchmark report.