Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: iroh-perf #2186

Merged
merged 7 commits into from
May 22, 2024
Merged

feat: iroh-perf #2186

merged 7 commits into from
May 22, 2024

Conversation

Arqu
Copy link
Collaborator

@Arqu Arqu commented Apr 15, 2024

Description

In progress thing for what should later be iroh-perf. For now you can run with cargo run --release -- iroh and cargo run --release -- quinn to do a simple comparison benchmark.

More knobs are available for tuning with --help.
There's a much more stark difference on macOS compared to running on Linux.

Next steps bumping to test against quinn 0.11

Sample output:

iroh (macOS)

Client 0 stats:
Connect time: 13.788583ms
Overall download stats:

Transferred 1073741824 bytes on 1 streams in 5.47s (187.27 MiB/s)

Time to first byte (TTFB): 1ms

Total chunks: 130319

Average chunk time: 41.936ms

Average chunk size: 8.04KiB

Stream download metrics:

      │  Throughput   │ Duration 
──────┼───────────────┼──────────
 AVG  │  187.31 MiB/s │     5.47s
 P0   │  187.25 MiB/s │     5.46s
 P10  │  187.37 MiB/s │     5.47s
 P50  │  187.37 MiB/s │     5.47s
 P90  │  187.37 MiB/s │     5.47s
 P100 │  187.37 MiB/s │     5.47s
quinn (macOS)

Client 0 stats:
Connect time: 1.81875ms
Overall download stats:

Transferred 1073741824 bytes on 1 streams in 2.85s (358.92 MiB/s)

Time to first byte (TTFB): 61.168ms

Total chunks: 47412

Average chunk time: 60.176ms

Average chunk size: 22.12KiB

Stream download metrics:

      │  Throughput   │ Duration 
──────┼───────────────┼──────────
 AVG  │  358.88 MiB/s │     2.85s
 P0   │  358.75 MiB/s │     2.85s
 P10  │  359.00 MiB/s │     2.85s
 P50  │  359.00 MiB/s │     2.85s
 P90  │  359.00 MiB/s │     2.85s
 P100 │  359.00 MiB/s │     2.85s
iroh (linux)

Client 0 stats:
Connect time: 3.83554ms
Overall download stats:

Transferred 1073741824 bytes on 1 streams in 1.16s (884.52 MiB/s)

Time to first byte (TTFB): 2.014ms

Total chunks: 38046

Average chunk time: 30.408ms

Average chunk size: 27.55KiB

Stream download metrics:

      │  Throughput   │ Duration 
──────┼───────────────┼──────────
 AVG  │  884.75 MiB/s │     1.16s
 P0   │  884.50 MiB/s │     1.16s
 P10  │  885.00 MiB/s │     1.16s
 P50  │  885.00 MiB/s │     1.16s
 P90  │  885.00 MiB/s │     1.16s
 P100 │  885.00 MiB/s │     1.16s
quinn (linux)

Client 0 stats:
Connect time: 1.22085ms
Overall download stats:

Transferred 1073741824 bytes on 1 streams in 994.57ms (1029.59 MiB/s)

Time to first byte (TTFB): 1.483ms

Total chunks: 32809

Average chunk time: 30.296ms

Average chunk size: 31.96KiB

Stream download metrics:

      │  Throughput   │ Duration 
──────┼───────────────┼──────────
 AVG  │ 1030.50 MiB/s │  993.00ms
 P0   │ 1030.00 MiB/s │  993.00ms
 P10  │ 1031.00 MiB/s │  993.00ms
 P50  │ 1031.00 MiB/s │  993.00ms
 P90  │ 1031.00 MiB/s │  993.00ms
 P100 │ 1031.00 MiB/s │  993.00ms

Notes & open questions

Change checklist

  • Self-review.
  • Documentation updates if relevant.
  • Tests if relevant.

@Arqu Arqu self-assigned this Apr 15, 2024
@dignifiedquire dignifiedquire added this to the v0.17.0 milestone May 15, 2024
@Arqu Arqu marked this pull request as ready for review May 22, 2024 11:43
@Arqu Arqu changed the title draft: iroh-perf feat iroh-perf May 22, 2024
@Arqu Arqu added perf performance related issues benchmarks labels May 22, 2024
@Arqu Arqu changed the title feat iroh-perf feat: iroh-perf May 22, 2024
@@ -1,6 +1,6 @@
[package]
name = "iroh-net-bench"
version = "0.16.0"
version = "0.13.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bad merge

Copy link
Contributor

@dignifiedquire dignifiedquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good, can you recheck this is all checked in CI to keep working please?

Comment on lines 89 to 90
.context("connecting")
.unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't do anything, you probably want .expect("connecting") instead.

Comment on lines 225 to 226
stats.upload_stats.total_duration = start.elapsed();
stats.download_stats.total_duration = start.elapsed();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe move this up to right after acquire_many()? Probably not a big difference though

@Arqu Arqu added this pull request to the merge queue May 22, 2024
Merged via the queue into main with commit 98d45f3 May 22, 2024
22 of 23 checks passed
@Arqu Arqu deleted the arqu/perf branch May 22, 2024 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmarks perf performance related issues
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants