Automatically update lib.rs, mod.rs and DIRECTORY.md

[RenzoGerritzen]

Closes #693

Pull Request Template

Description

Adds a build script that will automatically ad anything public to the correct lib.rs and mod.rs when it becomes available. Additionally updates the DIRECTORY.md file accordingly.

When /* auto-exports start */ and /* auto-exports end */ are added to a lib.rs or mod.rs file, the build script will loop through files in the directory and make a list of anything on the file root with with pub security. Once it has found all exports, it will place mod <file> and pub use <file>{<fn>, <trait>, etc} in the matching mod.rs file. It will then additionaly create a tree of exports and write it in the DIRECTORY.md in a familiar manner.

Type of change

• Breaking change (fix or feature that would cause existing functionality to not work as expected)

While I tried to keep everything as close to original as possible, there are some changes in sorting and the way pub things are exported.

Checklist:

• I ran bellow commands using the latest version of rust nightly.
• I ran cargo clippy --all -- -D warnings just before my last commit and fixed any issue that was found.
  I found some issues, but none of them seemed to have anything to do with my code. I went through all of them and they all happen in the latest stable build too :(.
• I ran cargo fmt just before my last commit.
• I ran cargo test just before my last commit and all tests passed.
• I added my algorithm to the corresponding mod.rs file within its own folder, and in any parent folder(s).
  This is not relevant for my change.
• I added my algorithm to DIRECTORY.md with the correct link.
  This is not relevant for my change.
• I checked COUNTRIBUTING.md and my code follows its guidelines.

[Miraksi]

Would it be possible to allow manually changing of mod.rs and DIRECTORY.md, or would the automation of mod.rs and DIRECTORY.md collide with the manual changes?

I'm asking this especially because I don't like the way collisions in namespaces are resolved here (e.g. segment tree implementations) and I would rather have the programmer themselves fix these collisions, than having it automatically be named <mod_name><struct name>.

What are youre ideas on that topic?

[Miraksi]

it seems like public functions and structs, that are created by declerative macros are overlooked by the build script, e.g. ciphers/sha3.rs

[RenzoGerritzen]

Good feedback, i understand your concerns. Manually changing mod.rs can be done by adding an exclusion to auto-export and adding it manually below, like done with sort-utils. I will make sure to add something similar for the DIRECTORY.md, but preferable somewhere out of sight of DIRECTORY.md.

As for the collisions, i will make sure to panic when a collision takes place. It will then tell the dev to either rename it, or custom export it outside of the auto-exports using 'use as'.

I will also fix the mod exports.

[RenzoGerritzen]

My apologies, i seem to have misunderstood what you meant with the declerative macros. I must admit i am not quite sure how to automate exporting something like that. For the time being i will add a manual export rule.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.51%. Comparing base (d5157e6) to head (c86e72f).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #699   +/-   ##
=======================================
  Coverage   94.51%   94.51%           
=======================================
  Files         293      293           
  Lines       21854    21854           
=======================================
  Hits        20656    20656           
  Misses       1198     1198           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Miraksi]

My apologies, i seem to have misunderstood what you meant with the declerative macros. I must admit i am not quite sure how to automate exporting something like that. For the time being i will add a manual export rule.

I am also not sure how to achieve this, but I'll have a look into it (sounds interesting ^^)

[RenzoGerritzen]

Colissions now panic instead of the build script trying to fix it. I have also changed the directory ordering so it matches the previous structure. I have added some manual exports for 'edge cases'. Additionally, it is now possible to add small commands in mod.rs files that change the behaviour of the automatic DIRECTORY.md file. For example, you can put // DIRECTORY.md override under sorting-utils insert Useful for debugging to add an extra comment or // DIRECTORY.md overrde at sorting place sorting-utils to move sorting-utils above sorting. It's not ideal, but in general i don't think this will be needed much.

[siriak]

The code looks good, could you add the empty line between mod and pub use back? I think it visually separates different sections of the file nicely.

And could you add a GitHub workflow that would run this script and commit changes to the branch? It would be awesome to have it automated to make sure the code is always up to date.

[RenzoGerritzen]

The code looks good, could you add the empty line between mod and pub use back? I think it visually separates different sections of the file nicely.

And could you add a GitHub workflow that would run this script and commit changes to the branch? It would be awesome to have it automated to make sure the code is always up to date.

Sure, no problem! I am not very familiar with GitHub workflows, but I'm sure i can figure it out!

[RenzoGerritzen]

I added an empty line between 'pub mod', 'mod', and 'pub use lines' as requested.

As i created the auto-exporting functionality in the build script, the existing Github workflows for building the project uses 'cargo test', which automatically updates the exports there. However, when the script creates 'pub use' statements for files that have multiple exports, it does all exports on a separate line, which does not match the way 'cargo fmt' expects it. I do not think running 'cargo fmt' at the end is a great idea. I could try to find out how exactly cargo fmt handles linebreaks in that case and try to replicate it, but this does not seem like a reliable solution to me. Does anyone have a better idea?

[siriak]

Why do you think that running cargo fmt at the end is not a great idea? This seems like the most straightforward and robust solution to me

[siriak]

@vil02 what do you think?

[vil02]

@vil02 what do you think?

• I am fine with running cargo fmt,
• are the lines in the generated files always sorted?
• I would suggest to add it to the CI to check if the generated files are as they should be. In case they are not, I would fail the CI (somehow dependabot has some problems when rebasing automated commits - I do not have much date about that, I just observed some strange behavior here Chore(deps): bump com.puppycrawl.tools:checkstyle from 9.3 to 10.16.0 Java#5134).