Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
refactor(triplestore): remove graphDB support (#2037)
* remove graphDB related scripts + clean docs

* remove consistency checking

* remove graphDB config

* refactor twirl templates related to graphDB

* remove embedded-jena-graphdb config

* refactor twirl templates related to graphDB in admin routes

* refactor twirl templates related to graphDB in v1 routes

* remove GraphDBAPI

* remove more graphDB inference related stuff

* update docs

* remove more graphDB stuff

* PR changes

* more PR changes

* chore: release 20.1.0 (#2036)

* commit to trigger bad gh actions

Co-authored-by: DaSCH Bot <50987250+daschbot@users.noreply.github.com>
  • Loading branch information
mpro7 and daschbot committed Apr 13, 2022
1 parent 57d40f7 commit bf17bca
Show file tree
Hide file tree
Showing 251 changed files with 2,232 additions and 10,910 deletions.
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -137,7 +137,7 @@ We write automated tests using [ScalaTest](https://www.scalatest.org). You can r

There are three sets of automated tests:

* Unit tests, route-to-route tests, and end-to-end tests are under `webapi/src/test`. To run these, type `graphdb:test` or `graphdb-free:test` (depending on which triplestore you're using) at the SBT console in the `webapi` project. To run a single test, use `graphdb:test-only *NameOfTestSpec`.
* Unit tests, route-to-route tests, and end-to-end tests are under `webapi/src/test`. To run these, type `test` at the SBT console in the `webapi` project. To run a single test, use `testOnly *NameOfTestSpec`.
* Integration tests, which can involve [Sipi](https://github.com/daschswiss/sipi), are under `src/it`. To run these, first start Sipi, then type `it:test` at the SBT console in the `webapi` project.

Whenever you add a new feature or fix a bug, you should add one or more tests
Expand Down
3 changes: 0 additions & 3 deletions docs/02-knora-ontologies/knora-base.md
Expand Up @@ -953,9 +953,6 @@ in user-created ontologies. Three types of consistency rules are enforced:
- Constraints on the types of the subjects and objects of OWL object properties must be satisfied.
- A datatype property may not have an empty string as an object.

The implementation of consistency checking is partly triplestore-dependent; Knora may be able to provide stricter checks
with some triplestores than with others.

### OWL Cardinalities

As noted in [Resources](#resources), each subclass of
Expand Down
4 changes: 0 additions & 4 deletions docs/04-publishing-deployment/configuration.md
Expand Up @@ -53,10 +53,6 @@ A number of core settings is additionally configurable through system environmen
| app.triplestore.use-https | KNORA_WEBAPI_TRIPLESTORE_USE_HTTPS | false |
| app.triplestore.host | KNORA_WEBAPI_TRIPLESTORE_HOST | localhost |
| app.triplestore.auto-init | KNORA_WEBAPI_TRIPLESTORE_AUTOINIT | false |
| app.triplestore.graphdb.port | KNORA_WEBAPI_TRIPLESTORE_GRAPHDB_PORT | 7200 |
| app.triplestore.graphdb.repository-name | KNORA_WEBAPI_TRIPLESTORE_GRAPHDB_REPOSITORY_NAME | knora-test |
| app.triplestore.graphdb.username | KNORA_WEBAPI_TRIPLESTORE_GRAPHDB_USERNAME | admin |
| app.triplestore.graphdb.password | KNORA_WEBAPI_TRIPLESTORE_GRAPHDB_PASSWORD | root |
| app.triplestore.fuseki.port | KNORA_WEBAPI_TRIPLESTORE_FUSEKI_PORT | 3030 |
| app.triplestore.fuseki.repository-name | KNORA_WEBAPI_TRIPLESTORE_FUSEKI_REPOSITORY_NAME | knora-test |
| app.triplestore.fuseki.username | KNORA_WEBAPI_TRIPLESTORE_FUSEKI_USERNAME | admin |
Expand Down
3 changes: 0 additions & 3 deletions docs/04-publishing-deployment/getting-started.md
Expand Up @@ -50,9 +50,6 @@ DSP-API is designed to work with any standards-compliant
triplestore. It is primarily tested with
[Apache Jena Fuseki](https://jena.apache.org), an open source triplestore.

Built-in support and configuration for a high-performance, proprietary
triplestore [Ontotext GraphDB](http://ontotext.com/products/graphdb/) is
provided but unmaintained (GraphDB must be licensed separately by the user).
Other triplestores are planned.

## Running the DSP-Stack
Expand Down
51 changes: 2 additions & 49 deletions docs/04-publishing-deployment/updates.md
Expand Up @@ -15,54 +15,7 @@ manual changes are needed, these will be described in the release notes, and mus
done first.

Before starting a new version of Knora, back up your repository, so you can restore it
in case the automatic repository update fails. You can use one of these scripts
in `webapi/scripts`:

- `fuseki-dump-repository.sh` for Fuseki
- `graphdb-dump-repository.sh` for GraphDB

in case the automatic repository update fails. For Fuseki use `fuseki-dump-repository.sh`
script located in `webapi/scripts`.

For information on command-line options, run the script with no arguments.

## Upgrading from a Knora Version Before 7.0.0

**WARNING**: If you do not follow this procedure, your data may be
corrupted, and Knora may not work.

You must first upgrade to Knora 7.0.0, then upgrade again to the current
version.

The overall procedure is:

1. Back up your repository as described above.

2. Install Knora release 7.0.0, and read the general instructions in
`upgrade/graphdb-se/old/README.md` in that release.

3. Follow the instructions in one of the subsections below for the version you are
upgrading from.

4. Back up your repository again.

5. Install the current release of Knore, and follow any manual update instructions
in its release notes.

6. Start Knora to continue the automatic upgrade.

### Upgrading from Knora 6.0.0 or 6.0.1

1. Follow the instructions in `upgrade/graphdb-se/old/1263-knora-admin/README.md`.

2. Follow the instructions in
[Upgrading from Knora 7.0.0 or Later](#upgrading-from-knora-7-0-0-or-later).

### Upgrading from Knora 5.0.0

1. Follow the instructions in `upgrade/graphdb-se/old/1211-datetime/README.md`.

2. Follow the instructions in `upgrade/graphdb-se/old/1230-delete-previews/README.md`.

3. Follow the instructions in `upgrade/graphdb-se/old/1263-knora-admin/README.md`.

4. Follow the instructions in
[Upgrading from Knora 7.0.0 or Later](#upgrading-from-knora-7-0-0-or-later).
4 changes: 1 addition & 3 deletions docs/05-internals/design/api-v2/gravsearch.md
Expand Up @@ -309,9 +309,7 @@ When the triplestore-specific version of the query is generated:

Gravsearch also provides some virtual properties, which take advantage of forward-chaining inference
as an optimisation if the triplestore provides it. For example, the virtual property
`knora-api:standoffTagHasStartAncestor` is equivalent to `knora-base:standoffTagHasStartParent*`, but
with GraphDB it is implemented using a custom inference rule (in `KnoraRules.pie`) and is therefore more
efficient. If Knora is not using the triplestore's inference, `SparqlTransformer.transformStatementInWhereForNoInference`
`knora-api:standoffTagHasStartAncestor` is equivalent to `knora-base:standoffTagHasStartParent*`. If Knora is not using the triplestore's inference, `SparqlTransformer.transformStatementInWhereForNoInference`
replaces `knora-api:standoffTagHasStartAncestor` with `knora-base:standoffTagHasStartParent*`.

# Optimisation of generated SPARQL
Expand Down
3 changes: 3 additions & 0 deletions docs/05-internals/design/principles/consistency-checking.md
Expand Up @@ -5,6 +5,9 @@

# Consistency Checking

**Attention!** GraphDB is not supported anymore, therefore parts related
to it in this document are redundant.

## Requirements

Knora is designed to prevent inconsistencies in RDF data,
Expand Down
2 changes: 1 addition & 1 deletion docs/05-internals/design/principles/design-overview.md
Expand Up @@ -26,7 +26,7 @@ Knora is written in [Scala](http://www.scala-lang.org/), using the
[Akka](http://akka.io/) framework for message-based concurrency. It is
designed to work with any standards-compliant triplestore via
the [SPARQL 1.1 Protocol](http://www.w3.org/TR/sparql11-protocol/), but is currently
tested only with [Ontotext GraphDB](http://graphdb.ontotext.com/) (with support
tested only with [Apache Jena Fuseki](https://jena.apache.org) (with support
for other triplestores coming soon).

## Knora APIs
Expand Down
10 changes: 3 additions & 7 deletions docs/05-internals/design/principles/store-module.md
Expand Up @@ -26,14 +26,10 @@ HTTP-based triplestore support is implemented in the
`org.knora.webapi.triplestore.http` package.

An HTTP-based triplestore is one that is accessed remotely over the HTTP
protocol. `HttpTriplestoreConnector` supports the following triplestores:
protocol. `HttpTriplestoreConnector` supports the open source triplestore
- [Apache Jena Fuseki](https://jena.apache.org).

- Ontotext GraphDB
- Fuseki 2

### GraphDB

### Fuseki 2
### Apache Jena Fuseki

## Embedded Triplestores

Expand Down
15 changes: 5 additions & 10 deletions docs/05-internals/design/principles/triplestore-updates.md
Expand Up @@ -36,16 +36,11 @@ the same data, and may eventually provide functionality to facilitate
this coordination.)

We can assume that each SPARQL update operation will run in its own
database transaction with an isolation level of 'read committed'. This
is what GraphDB does when it receives a SPARQL update over HTTP (see
[GraphDB SE
Transactions](http://graphdb.ontotext.com/documentation/standard/storage.html#transaction-control)).
database transaction with an isolation level of 'read committed'.
We cannot assume that it is possible to run more than one SPARQL update
in a single database transaction. (The [SPARQL 1.1
Protocol](http://www.w3.org/TR/sparql11-protocol/) does not provide a
way to do this, and currently it can be done only by embedding the
triplestore in the application and using a vendor-specific API, but we
cannot require this in Knora.)
in a single database transaction. The [SPARQL 1.1 Protocol](http://www.w3.org/TR/sparql11-protocol/) does not provide a way to do this, and currently it
can be done only by embedding the triplestore in the application and
using a vendor-specific API, but we cannot require this in Knora.)

### Permissions

Expand Down Expand Up @@ -253,7 +248,7 @@ Knora enforces consistency constraints using three redundant mechanisms:
1. By doing pre-update checks using SPARQL SELECT queries and cached
ontology data.
2. By doing checks in the `WHERE` clauses of SPARQL updates.
3. By using GraphDB's built-in consistency checker (see
3. **Deprecated**: By using GraphDB's built-in consistency checker (see
[Consistency Checking](consistency-checking.md)).

We take the view that redundant consistency checks are a good thing.
Expand Down
18 changes: 2 additions & 16 deletions docs/05-internals/development/docker-compose.md
Expand Up @@ -5,27 +5,13 @@

# Starting the Knora Stack inside Docker Container

To run Knora locally, we provide `docker-compose.yml` which can be used to start GraphDB, Sipi,
To run Knora locally, we provide `docker-compose.yml` which can be used to start Fuseki, Sipi,
Webapi running each in its own Docker container.

For GraphDB it is additionally necessary to define two environment variables:

```
$ export KNORA_GDB_LICENSE # full path to the GraphDB-SE license file, e.g., /Users/name/GDB/GDB.license
$ export KNORA_GDB_HOME # full path to a local folder where GraphDB should store it's data, e.g., /users/name/GDB/home
```

Per default, GraphDB-SE is started. If GraphDB-Free is needed, because there is no awailable license,
then a third environment variable can be set to something like:

```
$ export KNORA_GDB_IMAGE=daschswiss/graphdb-free:8.3.1
```

To run the whole stack:

```
$ docker-compose up
$ make stack-up
```

For additional information please see the [Docker Compose documentation](https://docs.docker.com/compose/)
3 changes: 1 addition & 2 deletions docs/05-internals/development/overview.md
Expand Up @@ -10,8 +10,7 @@ installation of Knora. The different parts are:

1. The cloned [DSP-API](https://github.com/dasch-swiss/dsp-api) Github
repository
1. One of the supplied triplestores in the DSP-API Github repository
(GraphDB-SE 8 or Fuseki 3).
1. Fuseki - supplied triplestore in the DSP-API Github repository.
1. Sipi by building from
[source](https://github.com/dasch-swiss/sipi) or using the docker
[image](https://hub.docker.com/r/daschswiss/sipi/)
Expand Down
6 changes: 0 additions & 6 deletions docs/05-internals/development/testing.md
Expand Up @@ -5,12 +5,6 @@

# Testing

Prerequisite: Before running any tests, a supported triplestore needs to
be started and initialized through a script inside the "scripts" folder.
For example, when using "GraphDB Free", the nedded script is
"graphdb-free-init-knora-test-unit.sh". Please note the occurrence of
"test-unit" in the name of the script.

## How to Write Unit Tests

1) Inside a test, at the beginning, add the following (change the paths
Expand Down
2 changes: 1 addition & 1 deletion docs/07-lucene/index.md
Expand Up @@ -6,6 +6,6 @@
# Lucene

The Lucene full-text index provided by the triplestore is used to perform full-text searches in Knora.
The exact behavior can be different depending on the triplestore, e.g., GraphDB or Fuseki.
The exact behavior can be different depending on the triplestore.

* [Lucene Query Parser Syntax](lucene-query-parser-syntax.md)
5 changes: 1 addition & 4 deletions docs/faq/index.md
Expand Up @@ -29,10 +29,7 @@ we recommend converting it to Knora resources, which are stored as RDF.

### Which triplestores can be used with Knora?

Knora is tested with [Ontotext GraphDB SE](http://graphdb.ontotext.com/).
Our goal is to support several triplestores, including open-source options.
Integration with [Apache Jena Fuseki](https://jena.apache.org/documentation/fuseki2/)
has been partly implemented, but is not currently supported.
Knora is tested with [Apache Jena Fuseki](https://jena.apache.org/).

## Knora Ontologies

Expand Down

0 comments on commit bf17bca

Please sign in to comment.