Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

irmin-pack: extract GC logic from ext #2063

Merged
merged 3 commits into from Aug 31, 2022
Merged

Conversation

metanivek
Copy link
Member

This PR is a part of #2039. It moves the core GC logic out of ext.ml and into its own set of files, gc.ml and gc_worker.ml. It also does a few cleanups in this area of the code. I recommend reviewing commit-by-commit to see the changes more easily.

let flush_and_raise () = Ao.flush new_suffix |> Errs.raise_if_error in
let* () =
Errs.catch (fun () ->
Worker.transfer_append_exn ~read_exn ~append_exn ~off:copy_end_offset
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might be a code smell for how we are reusing worker code but no longer in the context of the worker. I kept it as-is for now, but we might want to revisit in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the answer to this concern and @icristescu's about the duplication of Args is to create a third file (gc_utils?) that would contain these

@codecov-commenter
Copy link

Codecov Report

Merging #2063 (af53b9b) into main (5374885) will decrease coverage by 0.02%.
The diff coverage is 41.63%.

@@            Coverage Diff             @@
##             main    #2063      +/-   ##
==========================================
- Coverage   64.34%   64.32%   -0.03%     
==========================================
  Files         131      133       +2     
  Lines       15584    15580       -4     
==========================================
- Hits        10028    10022       -6     
- Misses       5556     5558       +2     
Impacted Files Coverage Δ
src/irmin-pack/unix/gc_worker.ml 8.59% <8.59%> (ø)
src/irmin-pack/unix/ext.ml 61.15% <70.00%> (-5.51%) ⬇️
src/irmin-pack/unix/gc.ml 75.72% <75.25%> (+67.13%) ⬆️
src/irmin-pack/unix/gc_intf.ml 100.00% <100.00%> (ø)
src/irmin/watch.ml 78.87% <0.00%> (-2.12%) ⬇️
src/irmin-fs/unix/irmin_fs_unix.ml 68.38% <0.00%> (+0.64%) ⬆️
src/irmin-fs/irmin_fs.ml 82.42% <0.00%> (+1.21%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more


exception Pack_error = Errors.Pack_error

module type Args = sig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you include gc_intf here, you can avoid redefining this module type, I think. maybe even move module type S (renamed) to gc_intf. For me it make sense to have gc_intf shared between gc.ml and gc_worker, but if you disagree, you can leave it as is too :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll lump my response to @Ngoguey42's comment in with this.

Coming back to this with fresh eyes today, I think we ought to bring the worker functionality into gc.ml (as a private Worker module). The worker has no use outside of the gc.ml context, and all these little design tensions tell me that it would fit best as one module (and avoid adding another third module for sharing purposes), at least for now.

@icristescu icristescu added the no-changelog-needed No changelog is needed here label Aug 31, 2022
@metanivek
Copy link
Member Author

Okay, I pushed the change that does not create a new gc_worker.ml but rather has the worker as a private module for gc.ml. It feels much better to me to have this all in one module. Args is now shared and the call into Worker from the main gc module doesn't feel as bad since it is internal (still probably needs more consideration for if there is a better design).

@icristescu @Ngoguey42 can I get a final review?

@icristescu
Copy link
Contributor

thanks LGTM!

@metanivek metanivek merged commit 97bcdab into mirage:main Aug 31, 2022
@metanivek metanivek deleted the gc-worker branch August 31, 2022 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-changelog-needed No changelog is needed here
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants