Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change project recovery to save one recovery script rather than one for each workspace #36859

Merged
merged 7 commits into from
Feb 16, 2024

Conversation

jhaigh0
Copy link
Contributor

@jhaigh0 jhaigh0 commented Feb 16, 2024

Description of work

See #36847 for why this was an a problem.

Previously, project recovery would save the history of each workspace in separate, numbered python files. Then, on recovery, these were put through the OrderWorkspaceHistory algorithm to create the ordered_recovery.py script. To stop as many files being written I've used the project_recovery.utils function get_all_workspace_history_from_ads() (which was being used to generate a recovery script via File -> Generate Recovery Script) to generate the recovery script at the point of saving a checkpoint. Then when loading the checkpoint back in, this script just needs to be copied over to where workbench looks for ordered_recovery.py.

Possibly this means project saving is slower (depends on if running OrderWorkspaceHistory is slower than writing all the separate files) but this all runs on a separate thread, so I think it will be fine.

I have also fixed a bug where, when loading opening and running ordered_recovery.py, if workbench had the file open on startup, an older out of date version would be run.

Fixes #36847
Fixes #36737

To test:

Since this is so close to release this probably needs the full project recovery manual tests.
https://developer.mantidproject.org/Testing/ErrorReporter-ProjectRecovery/ProjectRecoveryTesting.html

Test instructions from #36737


Reviewer

Please comment on the points listed below (full description).
Your comments will be used as part of the gatekeeper process, so please comment clearly on what you have checked during your review. If changes are made to the PR during the review process then your final comment will be the most important for gatekeepers. In this comment you should make it clear why any earlier review is still valid, or confirm that all requested changes have been addressed.

Code Review

  • Is the code of an acceptable quality?
  • Does the code conform to the coding standards?
  • Are the unit tests small and test the class in isolation?
  • If there is GUI work does it follow the GUI standards?
  • If there are changes in the release notes then do they describe the changes appropriately?
  • Do the release notes conform to the release notes guide?

Functional Tests

  • Do changes function as described? Add comments below that describe the tests performed?
  • Do the changes handle unexpected situations, e.g. bad input?
  • Has the relevant (user and developer) documentation been added/updated?

Does everything look good? Mark the review as Approve. A member of @mantidproject/gatekeepers will take care of it.

Gatekeeper

If you need to request changes to a PR then please add a comment and set the review status to "Request changes". This will stop the PR from showing up in the list for other gatekeepers.

@jhaigh0 jhaigh0 added ISIS Team: Core Issue and pull requests managed by the Core subteam at ISIS Reported By User Issues that were found or highlighted by a user/scientist labels Feb 16, 2024
@jhaigh0 jhaigh0 added this to the Release 6.9 milestone Feb 16, 2024
@jhaigh0 jhaigh0 marked this pull request as ready for review February 16, 2024 11:42
@jhaigh0 jhaigh0 changed the base branch from main to release-next February 16, 2024 11:44
@jclarkeSTFC jclarkeSTFC self-assigned this Feb 16, 2024
@sf1919
Copy link
Contributor

sf1919 commented Feb 16, 2024

Does this also fix #36737 ?

@jhaigh0 jhaigh0 force-pushed the 36847_project_recovery_making_lots_of_files branch from dd626ba to b82cc18 Compare February 16, 2024 14:01
jclarkeSTFC
jclarkeSTFC previously approved these changes Feb 16, 2024
Copy link
Contributor

@jclarkeSTFC jclarkeSTFC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through the manual testing and didn't have any problems. I think that means that #36737 is also fixed, since reproducing that simply requires following the manual testing instructions to get the error.

On Linux (but weirdly not on Windows), if the script string is empty then
this algorithm will explode.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ISIS Team: Core Issue and pull requests managed by the Core subteam at ISIS Reported By User Issues that were found or highlighted by a user/scientist
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Project recovery causing lots of file deletions on IDAaaS
3 participants