Improve repository migration script #296
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 17 commits
January 25, 2022 12:20
We will need these to unequivocally identify the correct parent to create new repository folders in.
This approach has several advantages: - it does not depend on operation order - it will allow to correctly handle some cases of position creation where the bundle import would fail because it finds several positions with the same reference number. The mapping will allow to use the GUIDs in the bundle instead of the reference number.
For example position 11, if the parent (position 1) is moved to a different position and a new repository folder is created with reference number 1, we would need to move position 11 in order for it to remain 11.
njohner
changed the title
njohner
force-pushed
the
nj/CA-3243/improve_repo_migrator
branch
from
4841c86
to
58c49da
Compare
This notably allows to correctly reindex objects that are in the trash.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To finally get a few benefits, I had to modify the migration script quite a bit. Before we were generating a mapping of old and new positions in the order of the rows in the excel file and at the same time analysing the corresponding operation. This made the whole script very sensitive to operation ordering in the excel file, and defining the correct order was really not always easy (as the operations themselves actually happen in the order
create,move,merge). Moreover we were sometimes usingpositions (i.e. reference numbers) to identify the parents (for example for creation operations). All the creation operations happening first, can lead to temporarily having 2 repository folders with the same reference number (e.g. are creating a new repofolder having the same number as an already existing one which will be moved later in the migration), which in turn would lead to issues identifying the correct parent in other creation operations. To solve these issues (and also better identify some move operations), we now:guidto all existing repository foldersWith this it is easier to correctly identify different operation types and the identification is less dependent on the order in the excel file.
Also note that I've updated the corresponding tests in opengever.core, to make sure the migration script was still working as before and added a new test for the newly supported edge cases (see 4teamwork/opengever.core#6792, last 5 commits)
For https://4teamwork.atlassian.net/browse/CA-3245 and https://4teamwork.atlassian.net/browse/CA-3243