Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: irmin-pack.unix: Measure IO activity. #2250

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

adatario
Copy link
Contributor

@adatario adatario commented May 9, 2023

This adds counters that meassure i/o activity at file level.

@adatario adatario changed the title irmin-pack.unix: Measure IO activity. WIP: irmin-pack.unix: Measure IO activity. May 9, 2023
@adatario adatario force-pushed the io-stats branch 2 times, most recently from 06bab22 to 530ed0e Compare May 9, 2023 16:05
@@ -168,7 +169,6 @@ module Unix = struct
usage is safe. *)
let buf = Bytes.unsafe_of_string s in
let () = Util.really_write t.fd off buf 0 len in
Index.Stats.add_write len;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just remembered that the stats here are used as a quick and dirty way to aggregate IO activity for the benchmark summary.

We need to double check the "official" summary printer but you can see how it is used for the bench/irmin-pack tree replay.

pb ~f:(`RM, `RM) "Disk bytes read" (fun s -> s.index.bytes_read);
pb ~f:(`RM, `RM) "Disk bytes written" (fun s -> s.index.bytes_written);
pb ~f:(`RM, `RM) "Disk bytes both" (fun s -> s.index.bytes_both);
`Spacer;
pb "Disk reads" (fun s -> s.index.nb_reads);
pb "Disk writes" (fun s -> s.index.nb_writes);
pb "Disk both" (fun s -> s.index.nb_both);

I am definitely in favor of cleaning this up and making it more clear that these are not "index" stats. Perhaps your new Io stats can help the summary aggregate the index activity as well.

@codecov-commenter
Copy link

Codecov Report

Merging #2250 (945791d) into main (1f046dd) will decrease coverage by 0.07%.
The diff coverage is 71.42%.

❗ Current head 945791d differs from pull request most recent head 0557349. Consider uploading reports for the commit 0557349 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main    #2250      +/-   ##
==========================================
- Coverage   68.15%   68.09%   -0.07%     
==========================================
  Files         137      137              
  Lines       16669    16692      +23     
==========================================
+ Hits        11361    11366       +5     
- Misses       5308     5326      +18     
Impacted Files Coverage Δ
src/irmin-pack/unix/stats_intf.ml 85.00% <ø> (ø)
src/irmin-pack/unix/io.ml 66.10% <71.42%> (-0.28%) ⬇️
src/irmin-pack/unix/stats.ml 56.60% <71.42%> (+3.66%) ⬆️

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants