Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(upgrade): Update repository on startup #1643

Merged
merged 25 commits into from May 18, 2020

Conversation

benjamingeer
Copy link

@benjamingeer benjamingeer commented Apr 30, 2020

User story:

https://dasch.myjetbrains.com/youtrack/issue/DSP-165

  • Move the code from the upgrade package into webapi, translating the Bash scripts into Scala, and incorporating the code into ApplicationActor and the new classes RepositoryUpdater and RepositoryUpdatePlan.
  • Implement the necessary HTTP operations (download and upload the whole repository) in HttpTriplestoreConnector.
  • Add an upgrade plugin for PR feat(api-v2): Remove ForbiddenResource #1615, to remove the instance of forbiddenResource from the Knora system data. (That PR only removed the ForbiddenResource class, and GraphDB's consistency checker rejects the instance if the class doesn't exist anymore.)
  • In ApplicationActor and its messages, rename the parameter skipLoadingOfOntologies (i.e. don't load ontologies from the triplestore into the cache) to ignoreRepository. When Knora is started for a test, before the test data is loaded into the triplestore, this is true. We can also use it to skip checking whether we should update the repository.
  • Rename some things in ApplicationMessages and TriplestoreMessages to distinguish clearly between the triplestore and the repository.
  • Use a sealed trait instead of an Enumeration for AppState, so the compiler can check that all cases are matched (e.g. in HealthRoute).
  • Update the GitHub action for testing repository update.
    • Add a script webapi/scripts/wait-for-knora.sh that polls Knora's /health route until Knora has finished starting up.
    • Add a Makefile target test-repository-update that loads old repository test data into Knora, restarts Knora, and waits for the update to complete.
  • Update docs.

@benjamingeer
Copy link
Author

benjamingeer commented May 7, 2020

@subotic I've commented out the GitHub CI upgrade-integration-tests workflow, because I removed the scripts that it was using. How do you want to do this now?

@subotic
Copy link
Collaborator

subotic commented May 7, 2020

We could add a new one based on the new feature, e.g., start the stack without api, load the v7 data, start the api and then check if it starts up.

For most of the steps there is a makefile target. Loading V7 data and checking if knora-api is running could be done with curl.

Would you like to try to do this?

@benjamingeer
Copy link
Author

benjamingeer commented May 7, 2020

@subotic

Would you like to try to do this?

I don't really understand how the Docker containers fit together, but I can try. How do I check whether the API started without errors?

Copy link
Collaborator

@subotic subotic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks! Was a lot of work at the end. LGTM :-)

@benjamingeer benjamingeer self-assigned this May 13, 2020
Copy link
Contributor

@nemmK nemmK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe improve the documentation a bit as mentioned before otherwise all good :-)

@benjamingeer
Copy link
Author

@subotic @nemmK Thank you both for the reviews!

@benjamingeer benjamingeer merged commit 0127dca into develop May 18, 2020
@benjamingeer benjamingeer deleted the wip/DSP-165-upgrade-at-startup branch May 18, 2020 13:01
@benjamingeer benjamingeer mentioned this pull request May 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants