Skip to content

Latest commit

 

History

History
81 lines (59 loc) · 3.33 KB

updating-repositories.md

File metadata and controls

81 lines (59 loc) · 3.33 KB
Raw

Updating Repositories

As explained in @ref:Knora Ontology Versions, the knora-base ontology contains a version string to ensure compatibility between a repository and a given version of Knora. The same version string is therefore hard-coded in the Knora source code, in the string constant org.knora.webapi.KnoraBaseVersion. For new pull requests, the format of this string is knora-base vN, where N is an integer that is incremented for each version. Each time a pull request introduces changes that are not compatible with existing data, the following must happen:

  • The knora-base version number must be incremented in knora-base.ttl and in the string constant org.knora.webapi.KnoraBaseVersion.

  • A plugin must be added in the upgrade subproject under org.knora.upgrade.plugins, and registered in org.knora.upgrade.Main, to transform existing repositories so that they are compatible with the code changes introduced in the pull request.

The order of version numbers must correspond to the order in which the pull requests are merged.

Adding an Upgrade Plugin

An upgrade plugin is a Scala class that extends UpgradePlugin. The name of the plugin class should refer to the pull request that made the transformation necessary, using the format UpgradePluginPRNNNN, where NNNN is the number of the pull request.

A plugin's transform method takes an RDF4J Model (a mutable object representing the repository) and modifies it as needed. For details on how to do this, see The RDF Model API in the RDF4J documentation.

The plugin must then be added to the collection pluginsForVersions in org.knora.upgrade.Main.

Testing Update Plugins

Each plugin should have a unit test that extends UpgradePluginSpec. A typical test loads a TriG file containing test data into a Model, runs the plugin, makes an RDF4J SailRepository containing the transformed Model, and uses SPARQL to check the result.

Design Rationale

We tried and rejected other designs:

  • Running SPARQL updates in the triplestore: too slow, and no way to report progress during the update.

  • Downloading the repository and transforming it in Python using rdflib: too slow.

  • Downloading the repository and transforming it in C++ using Redland: also too slow.

The Scala implementation is the fastest by far.

The whole repository is uploaded in a single transaction because GraphDB's consistency checker can enforce dependencies between named graphs.