diff --git a/docs/05-internals/design/principles/rdf-api.md b/docs/05-internals/design/principles/rdf-api.md index d551fb5f3d..ad97252852 100644 --- a/docs/05-internals/design/principles/rdf-api.md +++ b/docs/05-internals/design/principles/rdf-api.md @@ -52,9 +52,37 @@ The API is in the package `org.knora.webapi.messages.util.rdf`. It includes: To work with RDF models, start with `RdfFeatureFactory`, which returns instances of `RdfNodeFactory`, `RdfModelFactory`, and `RdfFormatUtil`, using feature toggle -configuration. +configuration. `JsonLDUtil` does not need a feature factory. -`JsonLDUtil` does not need a feature factory. +To iterate efficiently over the statements in an `RdfModel`, use its `iterator` method. +An `RdfModel` cannot be modified while you are iterating over it. +If you are iterating to look for statements to modify, you can +collect a `Set` of statements to remove and a `Set` of statements +to add, and perform these update operations after you have finished +the iteration. + +## RDF stream processing + +To read or write a large amount of RDF data without generating a large string +object, you can use the stream processing methods in `RdfFormatUtil`. + +To parse an `InputStream` to an `RdfModel`, use `inputStreamToRdfModel`. +To format an `RdfModel` to an `OutputStream`, use `rdfModelToOutputStream`. + +To parse RDF data from an `InputStream` and process it one statement at a time, +you can write a class that implements the `RdfStreamProcessor` trait, and +use it with the `RdfFormatUtil.parseWithStreamProcessor` method. +Your `RdfStreamProcessor` can also send one statement at a time to a +formatting stream processor, which knows how to write RDF to an `OutputStream` +in a particular format. Use `RdfFormatUtil.makeFormattingStreamProcessor` to +construct one of these. + + +## SPARQL queries + +In tests, it can be useful to run SPARQL queries to check the content of +an `RdfModel`. To do this, use the `RdfModel.asRepository` method, which +returns an `RdfRepository` that can run `SELECT` queries. ## Implementations @@ -74,38 +102,10 @@ The RDF API uses the feature toggle `jena-rdf-library`: - `off` (the default): use the RDF4J implementation. - The default setting is used on startup, e.g. to read ontologies from the repository. After startup, the per-request setting is used. -## What still uses RDF4J directly - -Before this API was added, Knora mainly used the RDF4J API directly, and still does -in some places: - -- Code that uses RDF4J's streaming API to process large amounts of data, especially to - avoid constructing a large string in TriG format: - - - `ProjectsResponderADM.projectDataGetRequestADM` - - - `HttpTriplestoreConnector.turtleToTrig` - - - `RepositoryUpdater` - -- The repository update plugin tests, which use SPARQL. - -- `TEIHeader`: uses XSLT that depends on the exact format of the RDF/XML generated by RDF4J. - The XSLT would need to be improved to handle `rdf:Description`. - -- `GravsearchParser`: uses RDF4J's SPARQL parser. This is probably - not worth changing. - - ## TODO - SHACL validation. - -- SPARQL querying. - -- A streaming parsing/formatting API for processing large graphs. diff --git a/docs/05-internals/development/updating-repositories.md b/docs/05-internals/development/updating-repositories.md index d4aad4e38f..8ce655d439 100644 --- a/docs/05-internals/development/updating-repositories.md +++ b/docs/05-internals/development/updating-repositories.md @@ -51,18 +51,18 @@ it to `org.knora.webapi.store.triplestore.upgrade.RepositoryUpdater`. 3. Download the entire repository from the triplestore into a TriG file. -4. Read the TriG file into an RDF4J `Model`. +4. Read the TriG file into an `RdfModel`. -5. Update the `Model` by running the necessary transformations, and replacing the +5. Update the `RdfModel` by running the necessary transformations, and replacing the built-in Knora ontologies with the current ones. -6. Save the `Model` to a new TriG file. +6. Save the `RdfModel` to a new TriG file. 7. Empty the repository in the triplestore. 8. Upload the transformed repository file to the triplestore. -To update the `Model`, `RepositoryUpdater` runs a sequence of upgrade plugins, each of which +To update the `RdfModel`, `RepositoryUpdater` runs a sequence of upgrade plugins, each of which is a class in `org.knora.webapi.store.triplestore.upgrade.plugins` and is registered in `RepositoryUpdatePlan`. @@ -94,32 +94,27 @@ with existing data, the following must happen: in the string constant `org.knora.webapi.KnoraBaseVersion`. - A plugin must be added in the package `org.knora.webapi.store.triplestore.upgrade.plugins`, - and registered in `RepositoryUpdatePlan`, to transform - existing repositories so that they are compatible with the code changes - introduced in the pull request. + to transform existing repositories so that they are compatible with the code changes + introduced in the pull request. Each new plugin must be registered + by adding it to the sequence returned by `RepositoryUpdatePlan.makePluginsForVersions`. -The order of version numbers must correspond to the order in which the pull requests -are merged. +The order of version numbers (and the plugins) must correspond to the order in which the +pull requests are merged. An upgrade plugin is a Scala class that extends `UpgradePlugin`. The name of the plugin class should refer to the pull request that made the transformation necessary, using the format `UpgradePluginPRNNNN`, where `NNNN` is the number of the pull request. -A plugin's `transform` method takes an RDF4J `Model` (a mutable object representing -the repository) and modifies it as needed. For details on how to do this, see -[The RDF Model API](https://rdf4j.eclipse.org/documentation/programming/model/) -in the RDF4J documentation. +A plugin's `transform` method takes an `RdfModel` (a mutable object representing +the repository) and modifies it as needed. Before transforming the data, a plugin can check whether a required manual transformation has been carried out. If the requirement is not met, the plugin can throw -`InconsistentTriplestoreDataException` to abort the upgrade process. - -The plugin must then be appended to the sequence `pluginsForVersions` in -`RepositoryUpdatePlan`. +`InconsistentRepositoryDataException` to abort the upgrade process. ## Testing Update Plugins Each plugin should have a unit test that extends `UpgradePluginSpec`. A typical -test loads a TriG file containing test data into a `Model`, runs the plugin, -makes an RDF4J `SailRepository` containing the transformed `Model`, and uses +test loads a TriG file containing test data into a `RdfModel`, runs the plugin, +makes an `RdfRepository` containing the transformed `RdfModel`, and uses SPARQL to check the result. diff --git a/test_data/test_route/texts/beol/header.xsl b/test_data/test_route/texts/beol/header.xsl index 2329743f54..f733c09347 100644 --- a/test_data/test_route/texts/beol/header.xsl +++ b/test_data/test_route/texts/beol/header.xsl @@ -1,4 +1,11 @@ + - - - - - - - - - - - - - - - - - @@ -137,10 +125,9 @@ - - - - + + + @@ -164,31 +151,31 @@ - + + - + - + + select="//rdf:Description[@rdf:about=$authorIri]//beol:hasIAFIdentifier/@rdf:resource"/> + select="//rdf:Description[@rdf:about=$authorIri]//beol:hasFamilyName/@rdf:resource"/> + select="//rdf:Description[@rdf:about=$authorIri]//beol:hasGivenName/@rdf:resource"/> - + select="//rdf:Description[@rdf:about=$authorIAFValue]/knora-api:valueAsString/text()"/> + select="//rdf:Description[@rdf:about=$authorFamilyNameValue]/knora-api:valueAsString/text()"/> + select="//rdf:Description[@rdf:about=$authorGivenNameValue]/knora-api:valueAsString/text()"/> @@ -203,32 +190,32 @@ + select="//rdf:Description[@rdf:about=$dateValue]"/> - + - + + select="//rdf:Description[@rdf:about=$recipientIri]//beol:hasIAFIdentifier/@rdf:resource"/> + select="//rdf:Description[@rdf:about=$recipientIri]//beol:hasFamilyName/@rdf:resource"/> + select="//rdf:Description[@rdf:about=$recipientIri]//beol:hasGivenName/@rdf:resource"/> + select="//rdf:Description[@rdf:about=$recipientIAFValue]/knora-api:valueAsString/text()"/> + select="//rdf:Description[@rdf:about=$recipientFamilyNameValue]/knora-api:valueAsString/text()"/> + select="//rdf:Description[@rdf:about=$recipientGivenNameValue]/knora-api:valueAsString/text()"/> @@ -244,7 +231,7 @@ - + diff --git a/webapi/src/main/scala/org/knora/webapi/app/ApplicationActor.scala b/webapi/src/main/scala/org/knora/webapi/app/ApplicationActor.scala index 0c113b1801..f100614081 100644 --- a/webapi/src/main/scala/org/knora/webapi/app/ApplicationActor.scala +++ b/webapi/src/main/scala/org/knora/webapi/app/ApplicationActor.scala @@ -31,7 +31,7 @@ import ch.megard.akka.http.cors.scaladsl.settings.CorsSettings import com.typesafe.scalalogging.LazyLogging import kamon.Kamon import org.knora.webapi.core.LiveActorMaker -import org.knora.webapi.exceptions.{InconsistentTriplestoreDataException, SipiException, UnexpectedMessageException, UnsupportedValueException} +import org.knora.webapi.exceptions.{InconsistentRepositoryDataException, SipiException, UnexpectedMessageException, UnsupportedValueException} import org.knora.webapi.feature.{FeatureFactoryConfig, KnoraSettingsFeatureFactoryConfig} import org.knora.webapi.http.handler import org.knora.webapi.http.version.ServerVersion @@ -143,7 +143,7 @@ class ApplicationActor extends Actor with Stash with LazyLogging with AroundDire case _: ArithmeticException => Resume case _: NullPointerException => Restart case _: IllegalArgumentException => Stop - case e: InconsistentTriplestoreDataException => + case e: InconsistentRepositoryDataException => logger.info(s"Received a 'InconsistentTriplestoreDataException', will shutdown now. Cause: {}", e.message) Stop case e: SipiException => @@ -552,6 +552,11 @@ class ApplicationActor extends Actor with Stash with LazyLogging with AroundDire msg += s"DSP-API Server started: http://${knoraSettings.internalKnoraApiHost}:${knoraSettings.internalKnoraApiPort}\n" msg += "------------------------------------------------\n" + defaultFeatureFactoryConfig.makeToggleSettingsString match { + case Some(toggleSettingsString) => msg += s"Default feature toggle settings: $toggleSettingsString\n" + case None => () + } + if (allowReloadOverHTTPState | knoraSettings.allowReloadOverHTTP) { msg += "WARNING: Resetting DB over HTTP is turned ON.\n" msg += "------------------------------------------------\n" diff --git a/webapi/src/main/scala/org/knora/webapi/exceptions/Exceptions.scala b/webapi/src/main/scala/org/knora/webapi/exceptions/Exceptions.scala index 956a983b4e..29f60b5f86 100644 --- a/webapi/src/main/scala/org/knora/webapi/exceptions/Exceptions.scala +++ b/webapi/src/main/scala/org/knora/webapi/exceptions/Exceptions.scala @@ -324,15 +324,15 @@ object TriplestoreResponseException { } /** - * Indicates that the triplestore returned inconsistent data. + * Indicates an inconsistency in repository data. * * @param message a description of the error. */ -case class InconsistentTriplestoreDataException(message: String, cause: Option[Throwable] = None) extends TriplestoreException(message, cause) +case class InconsistentRepositoryDataException(message: String, cause: Option[Throwable] = None) extends InternalServerException(message, cause) -object InconsistentTriplestoreDataException { - def apply(message: String, e: Throwable, log: LoggingAdapter): InconsistentTriplestoreDataException = - InconsistentTriplestoreDataException(message, Some(ExceptionUtil.logAndWrapIfNotSerializable(e, log))) +object InconsistentRepositoryDataException { + def apply(message: String, e: Throwable, log: LoggingAdapter): InconsistentRepositoryDataException = + InconsistentRepositoryDataException(message, Some(ExceptionUtil.logAndWrapIfNotSerializable(e, log))) } /** diff --git a/webapi/src/main/scala/org/knora/webapi/feature/FeatureFactory.scala b/webapi/src/main/scala/org/knora/webapi/feature/FeatureFactory.scala index 36b649c80e..f78033b70a 100644 --- a/webapi/src/main/scala/org/knora/webapi/feature/FeatureFactory.scala +++ b/webapi/src/main/scala/org/knora/webapi/feature/FeatureFactory.scala @@ -227,9 +227,9 @@ abstract class FeatureFactoryConfig(protected val maybeParent: Option[FeatureFac protected[feature] def getLocalConfig(featureName: String): Option[FeatureToggle] /** - * Returns an [[HttpHeader]] giving the state of all feature toggles. + * Returns a string giving the state of all feature toggles. */ - def makeHttpResponseHeader: Option[HttpHeader] = { + def makeToggleSettingsString: Option[String] = { // Convert each toggle to its string representation. val enabledToggles: Set[String] = getAllBaseConfigs.map { baseConfig: FeatureToggleBaseConfig => @@ -246,13 +246,22 @@ abstract class FeatureFactoryConfig(protected val maybeParent: Option[FeatureFac // Are any toggles enabled? if (enabledToggles.nonEmpty) { // Yes. Return a header. - Some(RawHeader(FeatureToggle.RESPONSE_HEADER, enabledToggles.mkString(","))) + Some(enabledToggles.mkString(",")) } else { // No. Don't return a header. None } } + /** + * Returns an [[HttpHeader]] giving the state of all feature toggles. + */ + def makeHttpResponseHeader: Option[HttpHeader] = { + makeToggleSettingsString.map { + settingsStr: String => RawHeader(FeatureToggle.RESPONSE_HEADER, settingsStr) + } + } + /** * Adds an [[HttpHeader]] to an [[HttpResponse]] indicating which feature toggles are enabled. */ diff --git a/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel b/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel index 3a7ba37c53..a1cd974630 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel +++ b/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel @@ -34,6 +34,9 @@ scala_library( "@maven//:org_apache_commons_commons_text", "@maven//:org_apache_jena_apache_jena_libs", "@maven//:org_eclipse_rdf4j_rdf4j_client", + "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", + "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", + "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory", "@maven//:org_jodd_jodd", "@maven//:org_scala_lang_modules_scala_xml_2_12", "@maven//:org_scala_lang_scala_library", diff --git a/webapi/src/main/scala/org/knora/webapi/messages/OntologyConstants.scala b/webapi/src/main/scala/org/knora/webapi/messages/OntologyConstants.scala index fdafcc18ea..9fe9fd1393 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/OntologyConstants.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/OntologyConstants.scala @@ -19,7 +19,7 @@ package org.knora.webapi.messages -import org.knora.webapi.exceptions.InconsistentTriplestoreDataException +import org.knora.webapi.exceptions.InconsistentRepositoryDataException import org.knora.webapi._ /** @@ -618,7 +618,7 @@ object OntologyConstants { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"salsah-gui attribute type not found: $name") + case None => throw InconsistentRepositoryDataException(s"salsah-gui attribute type not found: $name") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/StringFormatter.scala b/webapi/src/main/scala/org/knora/webapi/messages/StringFormatter.scala index dec01bf27c..6dd73c2fed 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/StringFormatter.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/StringFormatter.scala @@ -1453,7 +1453,7 @@ class StringFormatter private(val maybeSettings: Option[KnoraSettingsImpl] = Non } ) } else { - throw InconsistentTriplestoreDataException(s"Link value predicate IRI $iri does not end with 'Value'") + throw InconsistentRepositoryDataException(s"Link value predicate IRI $iri does not end with 'Value'") } } @@ -2985,10 +2985,10 @@ class StringFormatter private(val maybeSettings: Option[KnoraSettingsImpl] = Non } /** - * Calls `decodeUuidWithErr`, throwing [[InconsistentTriplestoreDataException]] if the string cannot be parsed. + * Calls `decodeUuidWithErr`, throwing [[InconsistentRepositoryDataException]] if the string cannot be parsed. */ def decodeUuid(uuidStr: String): UUID = { - decodeUuidWithErr(uuidStr, throw InconsistentTriplestoreDataException(s"Invalid UUID: $uuidStr")) + decodeUuidWithErr(uuidStr, throw InconsistentRepositoryDataException(s"Invalid UUID: $uuidStr")) } /** diff --git a/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/permissionsmessages/PermissionsMessagesADM.scala b/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/permissionsmessages/PermissionsMessagesADM.scala index c6c9c4f234..f9940f0c35 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/permissionsmessages/PermissionsMessagesADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/permissionsmessages/PermissionsMessagesADM.scala @@ -23,7 +23,7 @@ import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, ForbiddenException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{BadRequestException, ForbiddenException, InconsistentRepositoryDataException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants import org.knora.webapi.messages.admin.responder.permissionsmessages.PermissionDataType.PermissionProfileType @@ -1019,7 +1019,7 @@ object PermissionDataType extends Enumeration { /** * Given the name of a value in this enumeration, returns the value. If the value is not found, throws an - * [[InconsistentTriplestoreDataException]]. + * [[InconsistentRepositoryDataException]]. * * @param name the name of the value. * @return the requested value. @@ -1027,7 +1027,7 @@ object PermissionDataType extends Enumeration { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"Permission profile type not supported: $name") + case None => throw InconsistentRepositoryDataException(s"Permission profile type not supported: $name") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/usersmessages/UsersMessagesADM.scala b/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/usersmessages/UsersMessagesADM.scala index 228d967ab1..e62fdbd3c3 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/usersmessages/UsersMessagesADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/admin/responder/usersmessages/UsersMessagesADM.scala @@ -23,7 +23,7 @@ import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, DataConversionException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{BadRequestException, DataConversionException, InconsistentRepositoryDataException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.admin.responder.groupsmessages.{GroupADM, GroupsADMJsonProtocol} import org.knora.webapi.messages.admin.responder.permissionsmessages.{PermissionsADMJsonProtocol, PermissionsDataADM} @@ -745,7 +745,7 @@ object UserInformationTypeADM extends Enumeration { /** * Given the name of a value in this enumeration, returns the value. If the value is not found, throws an - * [[InconsistentTriplestoreDataException]]. + * [[InconsistentRepositoryDataException]]. * * @param name the name of the value. * @return the requested value. @@ -753,7 +753,7 @@ object UserInformationTypeADM extends Enumeration { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"User profile type not supported: $name") + case None => throw InconsistentRepositoryDataException(s"User profile type not supported: $name") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/store/triplestoremessages/TriplestoreMessages.scala b/webapi/src/main/scala/org/knora/webapi/messages/store/triplestoremessages/TriplestoreMessages.scala index b175a05e62..7b59d40343 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/store/triplestoremessages/TriplestoreMessages.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/store/triplestoremessages/TriplestoreMessages.scala @@ -60,66 +60,12 @@ case class CheckConnection() extends TriplestoreRequest case class CheckConnectionACK() /** - * Represents a SPARQL SELECT query to be sent to the triplestore. A successful response will be a [[SparqlSelectResponse]]. + * Represents a SPARQL SELECT query to be sent to the triplestore. A successful response will be a [[SparqlSelectResult]]. * * @param sparql the SPARQL string. */ case class SparqlSelectRequest(sparql: String) extends TriplestoreRequest -/** - * Represents a response to a SPARQL SELECT query, containing a parsed representation of the response (JSON, etc.) - * returned by the triplestore - * - * @param head the header of the response, containing the variable names. - * @param results the body of the response, containing rows of query results. - */ -case class SparqlSelectResponse(head: SparqlSelectResponseHeader, results: SparqlSelectResponseBody) { - - /** - * Returns the contents of the first row of results. - * - * @return a [[Map]] representing the contents of the first row of results. - */ - @throws[InconsistentTriplestoreDataException]("if the query returned no results.") - def getFirstRow: VariableResultsRow = { - if (results.bindings.isEmpty) { - throw TriplestoreResponseException(s"A SPARQL query unexpectedly returned an empty result") - } - - results.bindings.head - } -} - -/** - * Represents the header of a JSON response to a SPARQL SELECT query. - * - * @param vars the names of the variables that were used in the SPARQL SELECT statement. - */ -case class SparqlSelectResponseHeader(vars: Seq[String]) - -/** - * Represents the body of a JSON response to a SPARQL SELECT query. - * - * @param bindings the bindings of values to the variables used in the SPARQL SELECT statement. - * Empty rows are not allowed. - */ -case class SparqlSelectResponseBody(bindings: Seq[VariableResultsRow]) { - require(bindings.forall(_.rowMap.nonEmpty), "Empty rows are not allowed in a SparqlSelectResponseBody") -} - - -/** - * Represents a row of results in a JSON response to a SPARQL SELECT query. - * - * @param rowMap a map of variable names to values in the row. An empty string is not allowed as a variable - * name or value. - */ -case class VariableResultsRow(rowMap: ErrorHandlingMap[String, String]) { - require(rowMap.forall { - case (key, value) => key.nonEmpty && value.nonEmpty - }, "An empty string is not allowed as a variable name or value in a VariableResultsRow") -} - /** * Represents a SPARQL CONSTRUCT query to be sent to the triplestore. A successful response will be a * [[SparqlConstructResponse]]. @@ -134,13 +80,15 @@ case class SparqlConstructRequest(sparql: String, * Represents a SPARQL CONSTRUCT query to be sent to the triplestore. The triplestore's will be * written to the specified file in Trig format. A successful response message will be a [[FileWrittenResponse]]. * - * @param sparql the SPARQL string. - * @param graphIri the named graph IRI to be used in the TriG file. - * @param outputFile the file to be written. + * @param sparql the SPARQL string. + * @param graphIri the named graph IRI to be used in the TriG file. + * @param outputFile the file to be written. + * @param featureFactoryConfig the feature factory configuration. */ case class SparqlConstructFileRequest(sparql: String, graphIri: IRI, - outputFile: File) extends TriplestoreRequest + outputFile: File, + featureFactoryConfig: FeatureFactoryConfig) extends TriplestoreRequest /** * A response to a [[SparqlConstructRequest]]. @@ -185,7 +133,7 @@ object SparqlExtendedConstructResponse { val statementMap: mutable.Map[SubjectV2, ConstructPredicateObjects] = mutable.Map.empty val rdfModel: RdfModel = rdfFormatUtil.parseToRdfModel(rdfStr = turtleStr, rdfFormat = Turtle) - for (st: Statement <- rdfModel.getStatements) { + for (st: Statement <- rdfModel) { val subject: SubjectV2 = st.subj match { case iriNode: IriNode => IriSubjectV2(iriNode.iri) case blankNode: BlankNode => BlankNodeSubjectV2(blankNode.id) @@ -203,26 +151,26 @@ object SparqlExtendedConstructResponse { datatypeLiteral.datatype match { case datatypeIri if OntologyConstants.Xsd.integerTypes.contains(datatypeIri) => IntLiteralV2( - datatypeLiteral.integerValue(throw InconsistentTriplestoreDataException(s"Invalid integer: ${datatypeLiteral.value}")).toInt + datatypeLiteral.integerValue(throw InconsistentRepositoryDataException(s"Invalid integer: ${datatypeLiteral.value}")).toInt ) case OntologyConstants.Xsd.DateTime => DateTimeLiteralV2( stringFormatter.xsdDateTimeStampToInstant( datatypeLiteral.value, - throw InconsistentTriplestoreDataException(s"Invalid xsd:dateTime: ${datatypeLiteral.value}") + throw InconsistentRepositoryDataException(s"Invalid xsd:dateTime: ${datatypeLiteral.value}") ) ) case OntologyConstants.Xsd.Boolean => BooleanLiteralV2( - datatypeLiteral.booleanValue(throw InconsistentTriplestoreDataException(s"Invalid xsd:boolean: ${datatypeLiteral.value}")) + datatypeLiteral.booleanValue(throw InconsistentRepositoryDataException(s"Invalid xsd:boolean: ${datatypeLiteral.value}")) ) case OntologyConstants.Xsd.String => StringLiteralV2(value = datatypeLiteral.value, language = None) case OntologyConstants.Xsd.Decimal => DecimalLiteralV2( - datatypeLiteral.decimalValue(throw InconsistentTriplestoreDataException(s"Invalid xsd:decimal: ${datatypeLiteral.value}")) + datatypeLiteral.decimalValue(throw InconsistentRepositoryDataException(s"Invalid xsd:decimal: ${datatypeLiteral.value}")) ) case OntologyConstants.Xsd.Uri => IriLiteralV2(datatypeLiteral.value) @@ -267,11 +215,13 @@ case class SparqlExtendedConstructResponse(statements: Map[SubjectV2, SparqlExte * Requests a named graph, which will be written to the specified file in Trig format. A successful response * will be a [[FileWrittenResponse]]. * - * @param graphIri the IRI of the named graph. - * @param outputFile the destination file. + * @param graphIri the IRI of the named graph. + * @param outputFile the destination file. + * @param featureFactoryConfig the feature factory configuration. */ case class NamedGraphFileRequest(graphIri: IRI, - outputFile: File) extends TriplestoreRequest + outputFile: File, + featureFactoryConfig: FeatureFactoryConfig) extends TriplestoreRequest /** * Requests a named graph, which will be returned as Turtle. A successful response @@ -394,8 +344,11 @@ case class UpdateRepositoryRequest() extends TriplestoreRequest /** * Requests that the repository is downloaded to a TriG file. A successful response will be a [[FileWrittenResponse]]. + * + * @param outputFile the output file. + * @param featureFactoryConfig the feature factory configuration. */ -case class DownloadRepositoryRequest(outputFile: File) extends TriplestoreRequest +case class DownloadRepositoryRequest(outputFile: File, featureFactoryConfig: FeatureFactoryConfig) extends TriplestoreRequest /** * Indicates that a file was written successfully. @@ -717,6 +670,56 @@ case class DateTimeLiteralV2(value: Instant) extends LiteralV2 { ////////////////////////////////////////////////////////////////////////////////////////////////////////////////// // JSON formatting +/** + * A spray-json protocol that parses JSON returned by a SPARQL endpoint. Empty values and empty rows are + * ignored. + */ +object SparqlResultProtocol extends DefaultJsonProtocol { + + /** + * Converts a [[JsValue]] to a [[VariableResultsRow]]. + */ + implicit object VariableResultsJsonFormat extends JsonFormat[VariableResultsRow] { + def read(jsonVal: JsValue): VariableResultsRow = { + + // Collapse the JSON structure into a simpler Map of SPARQL variable names to values. + val mapToWrap: Map[String, String] = jsonVal.asJsObject.fields.foldLeft(Map.empty[String, String]) { + case (acc, (key, value)) => value.asJsObject.getFields("value") match { + case Seq(JsString(valueStr)) if valueStr.nonEmpty => // Ignore empty strings. + acc + (key -> valueStr) + case _ => acc + } + } + + // Wrap that Map in an ErrorHandlingMap that will gracefully report errors about missing values when they + // are accessed later. + VariableResultsRow(new ErrorHandlingMap(mapToWrap, { key: String => s"No value found for SPARQL query variable '$key' in query result row" })) + } + + def write(variableResultsRow: VariableResultsRow): JsValue = ??? + } + + /** + * Converts a [[JsValue]] to a [[SparqlSelectResultBody]]. + */ + implicit object SparqlSelectResponseBodyFormat extends JsonFormat[SparqlSelectResultBody] { + def read(jsonVal: JsValue): SparqlSelectResultBody = { + jsonVal.asJsObject.fields.get("bindings") match { + case Some(bindingsJson: JsArray) => + // Filter out empty rows. + SparqlSelectResultBody(bindingsJson.convertTo[Seq[VariableResultsRow]].filter(_.rowMap.keySet.nonEmpty)) + + case _ => SparqlSelectResultBody(Nil) + } + } + + def write(sparqlSelectResponseBody: SparqlSelectResultBody): JsValue = ??? + } + + implicit val headerFormat: JsonFormat[SparqlSelectResultHeader] = jsonFormat1(SparqlSelectResultHeader) + implicit val responseFormat: JsonFormat[SparqlSelectResult] = jsonFormat2(SparqlSelectResult) +} + /** * A spray-json protocol for generating Knora API v1 JSON providing data about resources and their properties. */ diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/ConstructResponseUtilV2.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/ConstructResponseUtilV2.scala index aaecdc1766..2720c47f3b 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/ConstructResponseUtilV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/ConstructResponseUtilV2.scala @@ -27,7 +27,7 @@ import akka.http.scaladsl.util.FastFuture import akka.util.Timeout import akka.pattern.ask import org.knora.webapi._ -import org.knora.webapi.exceptions.{AssertionException, InconsistentTriplestoreDataException, NotImplementedException} +import org.knora.webapi.exceptions.{AssertionException, InconsistentRepositoryDataException, NotImplementedException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectGetRequestADM, ProjectGetResponseADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages.UserADM @@ -117,7 +117,7 @@ object ConstructResponseUtilV2 { */ def maybeStringObject(predicateIri: SmartIri): Option[String] = { assertions.get(predicateIri).map { - literal => literal.asStringLiteral(throw InconsistentTriplestoreDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value + literal => literal.asStringLiteral(throw InconsistentRepositoryDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value } } @@ -129,7 +129,7 @@ object ConstructResponseUtilV2 { * @return the string object of the predicate. */ def requireStringObject(predicateIri: SmartIri): String = { - maybeStringObject(predicateIri).getOrElse(throw InconsistentTriplestoreDataException(s"Subject $subjectIri does not have predicate $predicateIri")) + maybeStringObject(predicateIri).getOrElse(throw InconsistentRepositoryDataException(s"Subject $subjectIri does not have predicate $predicateIri")) } /** @@ -140,7 +140,7 @@ object ConstructResponseUtilV2 { */ def maybeIriObject(predicateIri: SmartIri): Option[IRI] = { assertions.get(predicateIri).map { - literal => literal.asIriLiteral(throw InconsistentTriplestoreDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value + literal => literal.asIriLiteral(throw InconsistentRepositoryDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value } } @@ -152,7 +152,7 @@ object ConstructResponseUtilV2 { * @return the IRI object of the predicate. */ def requireIriObject(predicateIri: SmartIri): IRI = { - maybeIriObject(predicateIri).getOrElse(throw InconsistentTriplestoreDataException(s"Subject $subjectIri does not have predicate $predicateIri")) + maybeIriObject(predicateIri).getOrElse(throw InconsistentRepositoryDataException(s"Subject $subjectIri does not have predicate $predicateIri")) } /** @@ -163,7 +163,7 @@ object ConstructResponseUtilV2 { */ def maybeIntObject(predicateIri: SmartIri): Option[Int] = { assertions.get(predicateIri).map { - literal => literal.asIntLiteral(throw InconsistentTriplestoreDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value + literal => literal.asIntLiteral(throw InconsistentRepositoryDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value } } @@ -175,7 +175,7 @@ object ConstructResponseUtilV2 { * @return the integer object of the predicate. */ def requireIntObject(predicateIri: SmartIri): Int = { - maybeIntObject(predicateIri).getOrElse(throw InconsistentTriplestoreDataException(s"Subject $subjectIri does not have predicate $predicateIri")) + maybeIntObject(predicateIri).getOrElse(throw InconsistentRepositoryDataException(s"Subject $subjectIri does not have predicate $predicateIri")) } /** @@ -186,7 +186,7 @@ object ConstructResponseUtilV2 { */ def maybeBooleanObject(predicateIri: SmartIri): Option[Boolean] = { assertions.get(predicateIri).map { - literal => literal.asBooleanLiteral(throw InconsistentTriplestoreDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value + literal => literal.asBooleanLiteral(throw InconsistentRepositoryDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value } } @@ -198,7 +198,7 @@ object ConstructResponseUtilV2 { * @return the boolean object of the predicate. */ def requireBooleanObject(predicateIri: SmartIri): Boolean = { - maybeBooleanObject(predicateIri).getOrElse(throw InconsistentTriplestoreDataException(s"Subject $subjectIri does not have predicate $predicateIri")) + maybeBooleanObject(predicateIri).getOrElse(throw InconsistentRepositoryDataException(s"Subject $subjectIri does not have predicate $predicateIri")) } /** @@ -209,7 +209,7 @@ object ConstructResponseUtilV2 { */ def maybeDecimalObject(predicateIri: SmartIri): Option[BigDecimal] = { assertions.get(predicateIri).map { - literal => literal.asDecimalLiteral(throw InconsistentTriplestoreDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value + literal => literal.asDecimalLiteral(throw InconsistentRepositoryDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value } } @@ -221,7 +221,7 @@ object ConstructResponseUtilV2 { * @return the decimal object of the predicate. */ def requireDecimalObject(predicateIri: SmartIri): BigDecimal = { - maybeDecimalObject(predicateIri).getOrElse(throw InconsistentTriplestoreDataException(s"Subject $subjectIri does not have predicate $predicateIri")) + maybeDecimalObject(predicateIri).getOrElse(throw InconsistentRepositoryDataException(s"Subject $subjectIri does not have predicate $predicateIri")) } @@ -233,7 +233,7 @@ object ConstructResponseUtilV2 { */ def maybeDateTimeObject(predicateIri: SmartIri): Option[Instant] = { assertions.get(predicateIri).map { - literal => literal.asDateTimeLiteral(throw InconsistentTriplestoreDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value + literal => literal.asDateTimeLiteral(throw InconsistentRepositoryDataException(s"Unexpected object of $subjectIri $predicateIri: $literal")).value } } @@ -245,7 +245,7 @@ object ConstructResponseUtilV2 { * @return the timestamp object of the predicate. */ def requireDateTimeObject(predicateIri: SmartIri): Instant = { - maybeDateTimeObject(predicateIri).getOrElse(throw InconsistentTriplestoreDataException(s"Subject $subjectIri does not have predicate $predicateIri")) + maybeDateTimeObject(predicateIri).getOrElse(throw InconsistentRepositoryDataException(s"Subject $subjectIri does not have predicate $predicateIri")) } } @@ -330,7 +330,7 @@ object ConstructResponseUtilV2 { // Make sure all the subjects are IRIs, because blank nodes are not used in resources. val resultsWithIriSubjects: Statements = constructQueryResults.statements.map { case (iriSubject: IriSubjectV2, statements: ConstructPredicateObjects) => iriSubject.value -> statements - case (otherSubject: SubjectV2, _: ConstructPredicateObjects) => throw InconsistentTriplestoreDataException(s"Unexpected subject: $otherSubject") + case (otherSubject: SubjectV2, _: ConstructPredicateObjects) => throw InconsistentRepositoryDataException(s"Unexpected subject: $otherSubject") } // split statements about resources and other statements (value objects and standoff) @@ -374,7 +374,7 @@ object ConstructResponseUtilV2 { case (pred: SmartIri, objs: Seq[LiteralV2]) if pred.toString == OntologyConstants.KnoraBase.HasValue => objs.map { case IriLiteralV2(iri) => iri - case other => throw InconsistentTriplestoreDataException(s"Unexpected object for $resourceIri knora-base:hasValue: $other") + case other => throw InconsistentRepositoryDataException(s"Unexpected object for $resourceIri knora-base:hasValue: $other") } }.flatten.toSet @@ -511,7 +511,7 @@ object ConstructResponseUtilV2 { val resourceProjectLiteral: LiteralV2 = assertionsExplicit.getOrElse( OntologyConstants.KnoraBase.AttachedToProject.toSmartIri, - throw InconsistentTriplestoreDataException(s"Resource $resourceIri has no knora-base:attachedToProject") + throw InconsistentRepositoryDataException(s"Resource $resourceIri has no knora-base:attachedToProject") ).head // add the resource's project to the value's assertions, and get the user's permission on the value @@ -557,10 +557,10 @@ object ConstructResponseUtilV2 { } // Get the rdf:type of the value. - val rdfTypeLiteral: LiteralV2 = valueStatements.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"Value $valObjIri has no rdf:type")) + val rdfTypeLiteral: LiteralV2 = valueStatements.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"Value $valObjIri has no rdf:type")) val valueObjectClass: SmartIri = rdfTypeLiteral.asIriLiteral( - throw InconsistentTriplestoreDataException(s"Unexpected object of $valObjIri rdf:type: $rdfTypeLiteral") + throw InconsistentRepositoryDataException(s"Unexpected object of $valObjIri rdf:type: $rdfTypeLiteral") ).value.toSmartIri // check if it is a link value @@ -1128,9 +1128,9 @@ object ConstructResponseUtilV2 { ontologySchema = InternalSchema, valueHasStartJDN = valueObject.requireIntObject(OntologyConstants.KnoraBase.ValueHasStartJDN.toSmartIri), valueHasEndJDN = valueObject.requireIntObject(OntologyConstants.KnoraBase.ValueHasEndJDN.toSmartIri), - valueHasStartPrecision = DatePrecisionV2.parse(startPrecisionStr, throw InconsistentTriplestoreDataException(s"Invalid date precision: $startPrecisionStr")), - valueHasEndPrecision = DatePrecisionV2.parse(endPrecisionStr, throw InconsistentTriplestoreDataException(s"Invalid date precision: $endPrecisionStr")), - valueHasCalendar = CalendarNameV2.parse(calendarNameStr, throw InconsistentTriplestoreDataException(s"Invalid calendar name: $calendarNameStr")), + valueHasStartPrecision = DatePrecisionV2.parse(startPrecisionStr, throw InconsistentRepositoryDataException(s"Invalid date precision: $startPrecisionStr")), + valueHasEndPrecision = DatePrecisionV2.parse(endPrecisionStr, throw InconsistentRepositoryDataException(s"Invalid date precision: $endPrecisionStr")), + valueHasCalendar = CalendarNameV2.parse(calendarNameStr, throw InconsistentRepositoryDataException(s"Invalid calendar name: $calendarNameStr")), comment = valueCommentOption )) @@ -1302,7 +1302,7 @@ object ConstructResponseUtilV2 { val resourceLabel: String = resourceWithValueRdfData.requireStringObject(OntologyConstants.Rdfs.Label.toSmartIri) val resourceClassStr: IRI = resourceWithValueRdfData.requireIriObject(OntologyConstants.Rdf.Type.toSmartIri) - val resourceClass = resourceClassStr.toSmartIriWithErr(throw InconsistentTriplestoreDataException(s"Couldn't parse rdf:type of resource <$resourceIri>: <$resourceClassStr>")) + val resourceClass = resourceClassStr.toSmartIriWithErr(throw InconsistentRepositoryDataException(s"Couldn't parse rdf:type of resource <$resourceIri>: <$resourceClassStr>")) val resourceAttachedToUser: IRI = resourceWithValueRdfData.requireIriObject(OntologyConstants.KnoraBase.AttachedToUser.toSmartIri) val resourceAttachedToProject: IRI = resourceWithValueRdfData.requireIriObject(OntologyConstants.KnoraBase.AttachedToProject.toSmartIri) val resourcePermissions: String = resourceWithValueRdfData.requireStringObject(OntologyConstants.KnoraBase.HasPermissions.toSmartIri) diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/ErrorHandlingMap.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/ErrorHandlingMap.scala index f8f803a8c6..e7119c2bec 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/ErrorHandlingMap.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/ErrorHandlingMap.scala @@ -19,13 +19,13 @@ package org.knora.webapi.messages.util -import org.knora.webapi.exceptions.InconsistentTriplestoreDataException +import org.knora.webapi.exceptions.InconsistentRepositoryDataException import scala.collection.{GenTraversableOnce, Iterator, MapLike} /** * A [[Map]] that facilitates error-handling, by wrapping an ordinary [[Map]] and overriding the `default` - * method to provide custom behaviour (by default, throwing an [[InconsistentTriplestoreDataException]]) if a required + * method to provide custom behaviour (by default, throwing an [[InconsistentRepositoryDataException]]) if a required * value is missing. * * @param toWrap the [[Map]] to wrap. @@ -38,7 +38,7 @@ import scala.collection.{GenTraversableOnce, Iterator, MapLike} */ class ErrorHandlingMap[A, B](toWrap: Map[A, B], private val errorTemplateFun: A => String, - private val errorFun: String => B = { errorMessage: String => throw InconsistentTriplestoreDataException(errorMessage) }) + private val errorFun: String => B = { errorMessage: String => throw InconsistentRepositoryDataException(errorMessage) }) extends Map[A, B] with MapLike[A, B, ErrorHandlingMap[A, B]] { // As an optimization, if the Map we're supposed to wrap is another ErrorHandlingMap, wrap its underlying wrapped Map instead. diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/OntologyUtil.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/OntologyUtil.scala index b46257c649..0dded609b5 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/OntologyUtil.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/OntologyUtil.scala @@ -19,7 +19,7 @@ package org.knora.webapi.messages.util -import org.knora.webapi.exceptions.InconsistentTriplestoreDataException +import org.knora.webapi.exceptions.InconsistentRepositoryDataException import org.knora.webapi.messages.SmartIri /** @@ -42,7 +42,7 @@ object OntologyUtil { baseDefsSequence ++ baseDefsSequence.flatMap { baseDef => if (baseDef == initialIri) { - throw InconsistentTriplestoreDataException(s"Entity $initialIri has an inheritance cycle with entity $baseDef") + throw InconsistentRepositoryDataException(s"Entity $initialIri has an inheritance cycle with entity $baseDef") } else { getAllBaseDefsRec(initialIri, baseDef) } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/PermissionUtilADM.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/PermissionUtilADM.scala index eca1b935ea..4fbfca3b58 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/PermissionUtilADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/PermissionUtilADM.scala @@ -24,7 +24,7 @@ import akka.util.Timeout import akka.pattern.ask import com.typesafe.scalalogging.LazyLogging import org.knora.webapi.IRI -import org.knora.webapi.exceptions.{BadRequestException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{BadRequestException, InconsistentRepositoryDataException} import org.knora.webapi.messages.admin.responder.groupsmessages.{GroupGetResponseADM, MultipleGroupsGetRequestADM} import org.knora.webapi.messages.admin.responder.permissionsmessages.PermissionType.PermissionType import org.knora.webapi.messages.admin.responder.permissionsmessages.{PermissionADM, PermissionType} @@ -413,9 +413,9 @@ object PermissionUtilADM extends LazyLogging { val assertionMap: Map[IRI, String] = assertions.toMap // Anything with permissions must have an creator and a project. - val entityCreator: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToUser, throw InconsistentTriplestoreDataException(s"Entity $entityIri has no creator")) - val entityProject: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentTriplestoreDataException(s"Entity $entityIri has no project")) - val entityPermissionLiteral: String = assertionMap.getOrElse(OntologyConstants.KnoraBase.HasPermissions, throw InconsistentTriplestoreDataException(s"Entity $entityIri has no knora-base:hasPermissions predicate")) + val entityCreator: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToUser, throw InconsistentRepositoryDataException(s"Entity $entityIri has no creator")) + val entityProject: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentRepositoryDataException(s"Entity $entityIri has no project")) + val entityPermissionLiteral: String = assertionMap.getOrElse(OntologyConstants.KnoraBase.HasPermissions, throw InconsistentRepositoryDataException(s"Entity $entityIri has no knora-base:hasPermissions predicate")) getUserPermissionADM( entityCreator = entityCreator, @@ -433,7 +433,7 @@ object PermissionUtilADM extends LazyLogging { * [[OntologyConstants.KnoraBase.EntityPermissionAbbreviations]], and the values are sets of * user group IRIs. */ - def parsePermissions(permissionLiteral: String, errorFun: String => Nothing = { permissionLiteral: String => throw InconsistentTriplestoreDataException(s"invalid permission literal: $permissionLiteral") }): Map[EntityPermission, Set[IRI]] = { + def parsePermissions(permissionLiteral: String, errorFun: String => Nothing = { permissionLiteral: String => throw InconsistentRepositoryDataException(s"invalid permission literal: $permissionLiteral") }): Map[EntityPermission, Set[IRI]] = { val permissions: Seq[String] = permissionLiteral.split(OntologyConstants.KnoraBase.PermissionListDelimiter) permissions.map { @@ -478,7 +478,7 @@ object PermissionUtilADM extends LazyLogging { permissionType match { case PermissionType.AP => if (!OntologyConstants.KnoraAdmin.AdministrativePermissionAbbreviations.contains(abbreviation)) { - throw InconsistentTriplestoreDataException(s"Unrecognized permission abbreviation '$abbreviation'") + throw InconsistentRepositoryDataException(s"Unrecognized permission abbreviation '$abbreviation'") } if (splitPermission.length > 1) { @@ -491,7 +491,7 @@ object PermissionUtilADM extends LazyLogging { case PermissionType.OAP => if (!OntologyConstants.KnoraBase.EntityPermissionAbbreviations.contains(abbreviation)) { - throw InconsistentTriplestoreDataException(s"Unrecognized permission abbreviation '$abbreviation'") + throw InconsistentRepositoryDataException(s"Unrecognized permission abbreviation '$abbreviation'") } val shortGroups: Array[String] = splitPermission(1).split(OntologyConstants.KnoraBase.GroupListDelimiter) val groups: Set[IRI] = shortGroups.map(_.replace(OntologyConstants.KnoraAdmin.KnoraAdminPrefix, OntologyConstants.KnoraAdmin.KnoraAdminPrefixExpansion)).toSet @@ -519,7 +519,7 @@ object PermissionUtilADM extends LazyLogging { logger.debug(s"buildPermissionObject - ProjectResourceCreateRestrictedPermission - iris: $iris") iris.map(iri => PermissionADM.projectResourceCreateRestrictedPermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } case OntologyConstants.KnoraAdmin.ProjectAdminAllPermission => Set(PermissionADM.ProjectAdminAllPermission) @@ -530,7 +530,7 @@ object PermissionUtilADM extends LazyLogging { if (iris.nonEmpty) { iris.map(iri => PermissionADM.projectAdminGroupRestrictedPermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } case OntologyConstants.KnoraAdmin.ProjectAdminRightsAllPermission => Set(PermissionADM.ProjectAdminRightsAllPermission) @@ -539,35 +539,35 @@ object PermissionUtilADM extends LazyLogging { if (iris.nonEmpty) { iris.map(iri => PermissionADM.changeRightsPermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } case OntologyConstants.KnoraBase.DeletePermission => if (iris.nonEmpty) { iris.map(iri => PermissionADM.deletePermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } case OntologyConstants.KnoraBase.ModifyPermission => if (iris.nonEmpty) { iris.map(iri => PermissionADM.modifyPermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } case OntologyConstants.KnoraBase.ViewPermission => if (iris.nonEmpty) { iris.map(iri => PermissionADM.viewPermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } case OntologyConstants.KnoraBase.RestrictedViewPermission => if (iris.nonEmpty) { iris.map(iri => PermissionADM.restrictedViewPermission(iri)) } else { - throw InconsistentTriplestoreDataException(s"Missing additional permission information.") + throw InconsistentRepositoryDataException(s"Missing additional permission information.") } } @@ -651,7 +651,7 @@ object PermissionUtilADM extends LazyLogging { } } } else { - throw InconsistentTriplestoreDataException("Permissions cannot be empty") + throw InconsistentRepositoryDataException("Permissions cannot be empty") } case PermissionType.AP => @@ -669,7 +669,7 @@ object PermissionUtilADM extends LazyLogging { } } else { - throw InconsistentTriplestoreDataException("Permissions cannot be empty") + throw InconsistentRepositoryDataException("Permissions cannot be empty") } } } @@ -740,9 +740,9 @@ object PermissionUtilADM extends LazyLogging { val assertionMap: Map[IRI, String] = assertions.toMap // Anything with permissions must have an creator and a project. - val entityCreator: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToUser, throw InconsistentTriplestoreDataException(s"entity $entityIri has no creator")) - val entityProject: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentTriplestoreDataException(s"entity $entityIri has no project")) - val entityPermissionLiteral: String = assertionMap.getOrElse(OntologyConstants.KnoraBase.HasPermissions, throw InconsistentTriplestoreDataException(s"entity $entityIri has no knora-base:hasPermissions predicate")) + val entityCreator: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToUser, throw InconsistentRepositoryDataException(s"entity $entityIri has no creator")) + val entityProject: IRI = assertionMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentRepositoryDataException(s"entity $entityIri has no project")) + val entityPermissionLiteral: String = assertionMap.getOrElse(OntologyConstants.KnoraBase.HasPermissions, throw InconsistentRepositoryDataException(s"entity $entityIri has no knora-base:hasPermissions predicate")) getUserPermissionV1(entityIri = entityIri, entityCreator = entityCreator, entityProject = entityProject, entityPermissionLiteral = entityPermissionLiteral, userProfile = userProfile) } @@ -790,11 +790,11 @@ object PermissionUtilADM extends LazyLogging { val providedProjects = Vector(valuePropsProject, entityProject).flatten.distinct if (providedProjects.isEmpty) { - throw InconsistentTriplestoreDataException(s"No knora-base:attachedToProject was provided for entity $valueIri") + throw InconsistentRepositoryDataException(s"No knora-base:attachedToProject was provided for entity $valueIri") } if (providedProjects.size > 1) { - throw InconsistentTriplestoreDataException(s"Two different values of knora-base:attachedToProject were provided for entity $valueIri: ${valuePropsProject.get} and ${entityProject.get}") + throw InconsistentRepositoryDataException(s"Two different values of knora-base:attachedToProject were provided for entity $valueIri: ${valuePropsProject.get} and ${entityProject.get}") } val valuePropsAssertionsWithoutProject: Vector[(IRI, IRI)] = valuePropsAssertions.filter(_._1 != OntologyConstants.KnoraBase.AttachedToProject) diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/SparqlResultProtocol.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/SparqlResultProtocol.scala deleted file mode 100644 index d2443200fd..0000000000 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/SparqlResultProtocol.scala +++ /dev/null @@ -1,73 +0,0 @@ -/* - * Copyright © 2015-2018 the contributors (see Contributors.md). - * - * This file is part of Knora. - * - * Knora is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published - * by the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Knora is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Affero General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public - * License along with Knora. If not, see . - */ - -package org.knora.webapi.messages.util - -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectResponse, SparqlSelectResponseBody, SparqlSelectResponseHeader, VariableResultsRow} -import spray.json.{DefaultJsonProtocol, JsArray, JsString, JsValue, JsonFormat} - -/** - * A spray-json protocol that parses JSON returned by a SPARQL endpoint. Empty values and empty rows are - * ignored. - */ -object SparqlResultProtocol extends DefaultJsonProtocol { - - /** - * Converts a [[JsValue]] to a [[VariableResultsRow]]. - */ - implicit object VariableResultsJsonFormat extends JsonFormat[VariableResultsRow] { - def read(jsonVal: JsValue): VariableResultsRow = { - - // Collapse the JSON structure into a simpler Map of SPARQL variable names to values. - val mapToWrap: Map[String, String] = jsonVal.asJsObject.fields.foldLeft(Map.empty[String, String]) { - case (acc, (key, value)) => value.asJsObject.getFields("value") match { - case Seq(JsString(valueStr)) if valueStr.nonEmpty => // Ignore empty strings. - acc + (key -> valueStr) - case _ => acc - } - } - - // Wrap that Map in an ErrorHandlingMap that will gracefully report errors about missing values when they - // are accessed later. - VariableResultsRow(new ErrorHandlingMap(mapToWrap, { key: String => s"No value found for SPARQL query variable '$key' in query result row" })) - } - - def write(variableResultsRow: VariableResultsRow): JsValue = ??? - } - - /** - * Converts a [[JsValue]] to a [[SparqlSelectResponseBody]]. - */ - implicit object SparqlSelectResponseBodyFormat extends JsonFormat[SparqlSelectResponseBody] { - def read(jsonVal: JsValue): SparqlSelectResponseBody = { - jsonVal.asJsObject.fields.get("bindings") match { - case Some(bindingsJson: JsArray) => - // Filter out empty rows. - SparqlSelectResponseBody(bindingsJson.convertTo[Seq[VariableResultsRow]].filter(_.rowMap.keySet.nonEmpty)) - - case _ => SparqlSelectResponseBody(Nil) - } - } - - def write(sparqlSelectResponseBody: SparqlSelectResponseBody): JsValue = ??? - } - - implicit val headerFormat: JsonFormat[SparqlSelectResponseHeader] = jsonFormat1(SparqlSelectResponseHeader) - implicit val responseFormat: JsonFormat[SparqlSelectResponse] = jsonFormat2(SparqlSelectResponse) -} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/ValueUtilV1.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/ValueUtilV1.scala index d219380ccd..715606e72d 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/ValueUtilV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/ValueUtilV1.scala @@ -23,11 +23,11 @@ import akka.actor.ActorRef import akka.pattern._ import akka.util.Timeout import org.knora.webapi._ -import org.knora.webapi.exceptions.{InconsistentTriplestoreDataException, NotImplementedException, OntologyConstraintException} +import org.knora.webapi.exceptions.{InconsistentRepositoryDataException, NotImplementedException, OntologyConstraintException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.triplestoremessages.VariableResultsRow import org.knora.webapi.messages.util.GroupedProps._ +import org.knora.webapi.messages.util.rdf.VariableResultsRow import org.knora.webapi.messages.util.standoff.StandoffTagUtilV2 import org.knora.webapi.messages.v1.responder.ontologymessages._ import org.knora.webapi.messages.v1.responder.resourcemessages.{LiteralValueType, LocationV1, ResourceCreateValueObjectResponseV1, ResourceCreateValueResponseV1} @@ -555,7 +555,7 @@ class ValueUtilV1(private val settings: KnoraSettingsImpl) { val timeStampStr = predicates(OntologyConstants.KnoraBase.ValueHasTimeStamp).literals.head Future(TimeValueV1( - timeStamp = stringFormatter.xsdDateTimeStampToInstant(timeStampStr, throw InconsistentTriplestoreDataException(s"Can't parse timestamp: $timeStampStr")) + timeStamp = stringFormatter.xsdDateTimeStampToInstant(timeStampStr, throw InconsistentRepositoryDataException(s"Can't parse timestamp: $timeStampStr")) )) } @@ -577,7 +577,7 @@ class ValueUtilV1(private val settings: KnoraSettingsImpl) { userProfile: UserADM)(implicit timeout: Timeout, executionContext: ExecutionContext): Future[TextValueWithStandoffV1] = { // get the IRI of the mapping - val mappingIri = valueProps.literalData.getOrElse(OntologyConstants.KnoraBase.ValueHasMapping, throw InconsistentTriplestoreDataException(s"no mapping IRI associated with standoff belonging to textValue ${valueProps.valueIri}")).literals.head + val mappingIri = valueProps.literalData.getOrElse(OntologyConstants.KnoraBase.ValueHasMapping, throw InconsistentRepositoryDataException(s"no mapping IRI associated with standoff belonging to textValue ${valueProps.valueIri}")).literals.head for { @@ -631,7 +631,7 @@ class ValueUtilV1(private val settings: KnoraSettingsImpl) { userProfile: UserADM)(implicit timeout: Timeout, executionContext: ExecutionContext): Future[ApiValueV1] = { - val valueHasString: String = valueProps.literalData.get(OntologyConstants.KnoraBase.ValueHasString).map(_.literals.head).getOrElse(throw InconsistentTriplestoreDataException(s"Value ${valueProps.valueIri} has no knora-base:valueHasString")) + val valueHasString: String = valueProps.literalData.get(OntologyConstants.KnoraBase.ValueHasString).map(_.literals.head).getOrElse(throw InconsistentRepositoryDataException(s"Value ${valueProps.valueIri} has no knora-base:valueHasString")) val valueHasLanguage: Option[String] = valueProps.literalData.get(OntologyConstants.KnoraBase.ValueHasLanguage).map(_.literals.head) if (valueProps.standoff.nonEmpty) { diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/JsonLDUtil.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/JsonLDUtil.scala index cc11d26dfb..eaab17405e 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/JsonLDUtil.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/JsonLDUtil.scala @@ -1253,8 +1253,8 @@ object JsonLDUtil { // Are there still conflicts? if (hasPrefixConflicts(longKnoraPrefixes)) { - // Yes. This shouldn't happen, so throw InconsistentTriplestoreDataException. - throw InconsistentTriplestoreDataException(s"Can't make distinct prefixes for ontologies: ${(fixedPrefixes.values ++ knoraOntologiesNeedingPrefixes.map(_.toString)).mkString(", ")}") + // Yes. This shouldn't happen, so throw InconsistentRepositoryDataException. + throw InconsistentRepositoryDataException(s"Can't make distinct prefixes for ontologies: ${(fixedPrefixes.values ++ knoraOntologiesNeedingPrefixes.map(_.toString)).mkString(", ")}") } else { // No. Use the long prefixes. longKnoraPrefixes.toMap @@ -1470,7 +1470,7 @@ object JsonLDUtil { // Have we already processed this subject? if (!processedSubjects.contains(subj)) { // No. Get the statements about it. - val statements: Set[Statement] = model.find(Some(subj), None, None) + val statements: Set[Statement] = model.find(Some(subj), None, None).toSet // Make a JsonLDObject representing the entity and any nested entities. val jsonLDObject: JsonLDObject = entityToJsonLDObject( @@ -1660,7 +1660,7 @@ object JsonLDUtil { case None => // No. See if it's in the model. - val resourceStatements: Set[Statement] = model.find(Some(resource), None, None) + val resourceStatements: Set[Statement] = model.find(Some(resource), None, None).toSet // Is it in the model and not yet marked as processed? if (resourceStatements.nonEmpty && !processedSubjects.contains(resource)) { diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtil.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtil.scala index 157b4a5643..dbcf216d89 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtil.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtil.scala @@ -20,8 +20,9 @@ package org.knora.webapi.messages.util.rdf import akka.http.scaladsl.model.MediaType -import org.knora.webapi.{RdfMediaTypes, SchemaOption, SchemaOptions} +import org.knora.webapi.{IRI, RdfMediaTypes, SchemaOption, SchemaOptions} import org.knora.webapi.exceptions.{BadRequestException, InvalidRdfException} +import java.io.{InputStream, OutputStream} /** * A trait for supported RDF formats. @@ -107,14 +108,58 @@ case object RdfXml extends NonJsonLD { } /** - * Formats and parses RDF. + * A trait for classes that process streams of RDF data. */ -trait RdfFormatUtil { +trait RdfStreamProcessor { /** - * Returns an [[RdfModelFactory]] with the same underlying implementation as this [[RdfFormatUtil]]. + * Signals the start of the RDF data. */ - def getRdfModelFactory: RdfModelFactory + def start(): Unit + + /** + * Processes a namespace declaration. + * + * @param prefix the prefix. + * @param namespace the namespace. + */ + def processNamespace(prefix: String, namespace: IRI): Unit + /** + * Processes a statement. + * + * @param statement the statement. + */ + def processStatement(statement: Statement): Unit + + /** + * Signals the end of the RDF data. + */ + def finish(): Unit +} + +/** + * Represents a source of RDF data to be processed using an [[RdfStreamProcessor]]. + */ +sealed trait RdfSource + +/** + * An [[RdfSource]] that reads RDF data from a string. + * + * @param rdfStr a string containing RDF data. + */ +case class RdfStringSource(rdfStr: String) extends RdfSource + +/** + * An [[RdfSource]] that reads data from an [[InputStream]]. + * + * @param inputStream the input stream. + */ +case class RdfInputStreamSource(inputStream: InputStream) extends RdfSource + +/** + * Formats and parses RDF. + */ +trait RdfFormatUtil { /** * Parses an RDF string to an [[RdfModel]]. * @@ -193,7 +238,7 @@ trait RdfFormatUtil { throw BadRequestException(s"Named graphs are not supported in $rdfFormat") } - // Use an implementation-specific function to convert to other formats. + // Use an implementation-specific function to convert to formats other than JSON-LD. formatNonJsonLD( rdfModel = rdfModel, rdfFormat = nonJsonLD, @@ -202,6 +247,49 @@ trait RdfFormatUtil { } } + /** + * Parses RDF input, processing it with an [[RdfStreamProcessor]]. + * + * @param rdfSource the input source from which the RDF data should be read. + * @param rdfFormat the input format. + * @param rdfStreamProcessor the [[RdfStreamProcessor]] that will be used to process the input. + */ + def parseWithStreamProcessor(rdfSource: RdfSource, + rdfFormat: NonJsonLD, + rdfStreamProcessor: RdfStreamProcessor): Unit + + /** + * Reads RDF data from an [[InputStream]] and returns it as an [[RdfModel]]. + * + * @param inputStream the input stream. + * @param rdfFormat the data format. + * @return the corresponding [[RdfModel]]. + */ + def inputStreamToRdfModel(inputStream: InputStream, rdfFormat: NonJsonLD): RdfModel + + /** + * Formats an [[RdfModel]], writing the output to an [[OutputStream]]. + * + * @param rdfModel the model to be written. + * @param outputStream the output stream. + * @param rdfFormat the output format. + */ + def rdfModelToOutputStream(rdfModel: RdfModel, outputStream: OutputStream, rdfFormat: NonJsonLD): Unit + + /** + * Creates an [[RdfStreamProcessor]] that writes formatted output. + * + * @param outputStream the output stream to which the formatted RDF data should be written. + * @param rdfFormat the output format. + * @return an an [[RdfStreamProcessor]]. + */ + def makeFormattingStreamProcessor(outputStream: OutputStream, rdfFormat: NonJsonLD): RdfStreamProcessor + + /** + * Returns an [[RdfModelFactory]] with the same underlying implementation as this [[RdfFormatUtil]]. + */ + def getRdfModelFactory: RdfModelFactory + /** * Parses RDF in a format other than JSON-LD to an [[RdfModel]]. * diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfModel.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfModel.scala index abd80deced..58dad31f4f 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfModel.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/RdfModel.scala @@ -139,8 +139,10 @@ trait Statement { /** * Represents an RDF model consisting of a default graph and/or one or more named graphs. + * An [[RdfModel]] is mutable, so don't try to modify it while iterating over its statements + * using an iterator. */ -trait RdfModel { +trait RdfModel extends Iterable[Statement] { /** * Returns an [[RdfNodeFactory]] that can be used create nodes for use with this model. */ @@ -164,6 +166,17 @@ trait RdfModel { } } + /** + * Adds all the statements from another model to this model. + * + * @param otherModel another [[RdfModel]]. + */ + def addStatementsFromModel(otherModel: RdfModel): Unit = { + for (statement <- otherModel) { + addStatement(statement) + } + } + /** * Constructs a statement and adds it to the model. * @@ -190,9 +203,15 @@ trait RdfModel { def removeStatement(statement: Statement): Unit /** - * Returns the set of all statements in the model. + * Removes a set of statements from the model. + * + * @param statements the statements to remove. */ - def getStatements: Set[Statement] + def removeStatements(statements: Set[Statement]): Unit = { + for (statement <- statements) { + removeStatement(statement) + } + } /** * Returns statements that match a pattern. @@ -201,9 +220,20 @@ trait RdfModel { * @param pred the predicate, or `None` to match any predicate. * @param obj the object, or `None` to match any object. * @param context the IRI of a named graph, or `None` to match any graph. - * @return the statements matching the pattern. + * @return an iterator over the statements that match the pattern. + */ + def find(subj: Option[RdfResource], + pred: Option[IriNode], + obj: Option[RdfNode], + context: Option[IRI] = None): Iterator[Statement] + + /** + * Checks whether the model contains the specified statement. + * + * @param statement the statement. + * @return `true` if the model contains the statement. */ - def find(subj: Option[RdfResource], pred: Option[IriNode], obj: Option[RdfNode], context: Option[IRI] = None): Set[Statement] + def contains(statement: Statement): Boolean /** * Returns a set of all the subjects in the model. @@ -242,6 +272,21 @@ trait RdfModel { */ def getContexts: Set[IRI] + /** + * @return the number of statements in the model. + */ + def size: Int + + /** + * Empties this model. + */ + def clear(): Unit + + /** + * Returns an [[RdfRepository]] that can be used to query this model. + */ + def asRepository: RdfRepository + override def hashCode(): Int = super.hashCode() override def equals(obj: Any): Boolean = { @@ -333,3 +378,22 @@ trait RdfNodeFactory { trait RdfModelFactory { def makeEmptyModel: RdfModel } + +/** + * Represents a simple in-memory repository based on an [[RdfModel]]. + */ +trait RdfRepository { + /** + * Does a SPARQL SELECT query. + * + * @param selectQuery the query. + * @return the query result. + */ + def doSelect(selectQuery: String): SparqlSelectResult + + /** + * Shuts down this repository. The underlying [[RdfModel]] may not be usable after its + * [[RdfRepository]] has been shut down. + */ + def shutDown(): Unit +} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/SparqlSelectResult.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/SparqlSelectResult.scala new file mode 100644 index 0000000000..4cabf24f21 --- /dev/null +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/SparqlSelectResult.scala @@ -0,0 +1,72 @@ +/* + * Copyright © 2015-2018 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ + +package org.knora.webapi.messages.util.rdf + +import org.knora.webapi.exceptions.InconsistentRepositoryDataException + +/** + * Represents the result of a SPARQL SELECT query. + * + * @param head the header of the response, containing the variable names. + * @param results the body of the response, containing rows of query results. + */ +case class SparqlSelectResult(head: SparqlSelectResultHeader, results: SparqlSelectResultBody) { + + /** + * Returns the contents of the first row of results. + * + * @return a [[Map]] representing the contents of the first row of results. + */ + def getFirstRow: VariableResultsRow = { + results.bindings.headOption match { + case Some(row: VariableResultsRow) => row + case None => throw InconsistentRepositoryDataException(s"A SPARQL query unexpectedly returned an empty result") + } + } +} + +/** + * Represents the header of the result of a SPARQL SELECT query. + * + * @param vars the names of the variables that were used in the SPARQL SELECT statement. + */ +case class SparqlSelectResultHeader(vars: Seq[String]) + +/** + * Represents the body of the result of a SPARQL SELECT query. + * + * @param bindings the bindings of values to the variables used in the SPARQL SELECT statement. + * Empty rows are not allowed. + */ +case class SparqlSelectResultBody(bindings: Seq[VariableResultsRow]) { + require(bindings.forall(_.rowMap.nonEmpty), "Empty rows are not allowed in a SparqlSelectResponseBody") +} + +/** + * Represents a row of results in the result of a SPARQL SELECT query. + * + * @param rowMap a map of variable names to values in the row. An empty string is not allowed as a variable + * name or value. + */ +case class VariableResultsRow(rowMap: Map[String, String]) { + require(rowMap.forall { + case (key, value) => key.nonEmpty && value.nonEmpty + }, "An empty string is not allowed as a variable name or value in a VariableResultsRow") +} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaFormatUtil.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaFormatUtil.scala index 5a99fd34c0..fc4e244ca0 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaFormatUtil.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaFormatUtil.scala @@ -19,30 +19,78 @@ package org.knora.webapi.messages.util.rdf.jenaimpl -import java.io.{StringReader, StringWriter} +import java.io.{InputStream, OutputStream, StringReader, StringWriter} import org.apache.jena +import org.knora.webapi.IRI import org.knora.webapi.feature.Feature import org.knora.webapi.messages.util.rdf._ +/** + * Wraps an [[RdfStreamProcessor]] in a [[jena.riot.system.StreamRDF]]. + */ +class StreamProcessorAsStreamRDF(streamProcessor: RdfStreamProcessor) extends jena.riot.system.StreamRDF { + override def start(): Unit = streamProcessor.start() + + override def triple(triple: jena.graph.Triple): Unit = { + streamProcessor.processStatement( + JenaStatement(jena.sparql.core.Quad.create(jena.sparql.core.Quad.defaultGraphIRI, triple)) + ) + } + + override def quad(quad: jena.sparql.core.Quad): Unit = { + streamProcessor.processStatement(JenaStatement(quad)) + } + + override def base(s: String): Unit = {} + + override def prefix(prefixStr: String, namespace: String): Unit = { + streamProcessor.processNamespace(prefixStr, namespace) + } + + override def finish(): Unit = streamProcessor.finish() +} + +/** + * Wraps a [[jena.riot.system.StreamRDF]] in a [[RdfStreamProcessor]]. + */ +class StreamRDFAsStreamProcessor(streamRDF: jena.riot.system.StreamRDF) extends RdfStreamProcessor { + + import JenaConversions._ + + override def start(): Unit = streamRDF.start() + + override def processNamespace(prefix: String, namespace: IRI): Unit = { + streamRDF.prefix(prefix, namespace) + } + + override def processStatement(statement: Statement): Unit = { + streamRDF.quad(statement.asJenaQuad) + } + + override def finish(): Unit = streamRDF.finish() +} + /** * An implementation of [[RdfFormatUtil]] that uses the Jena API. */ class JenaFormatUtil(private val modelFactory: JenaModelFactory) extends RdfFormatUtil with Feature { override def getRdfModelFactory: RdfModelFactory = modelFactory - override def parseNonJsonLDToRdfModel(rdfStr: String, rdfFormat: NonJsonLD): RdfModel = { - val jenaModel: JenaModel = modelFactory.makeEmptyModel - - val parsingLang: jena.riot.Lang = rdfFormat match { + private def rdfFormatToJenaParsingLang(rdfFormat: NonJsonLD): jena.riot.Lang = { + rdfFormat match { case Turtle => jena.riot.RDFLanguages.TURTLE case TriG => jena.riot.RDFLanguages.TRIG case RdfXml => jena.riot.RDFLanguages.RDFXML } + } + + override def parseNonJsonLDToRdfModel(rdfStr: String, rdfFormat: NonJsonLD): RdfModel = { + val jenaModel: JenaModel = modelFactory.makeEmptyModel jena.riot.RDFParser.create() .source(new StringReader(rdfStr)) - .lang(parsingLang) + .lang(rdfFormatToJenaParsingLang(rdfFormat)) .errorHandler(jena.riot.system.ErrorHandlerFactory.errorHandlerStrictNoLogging) .parse(jenaModel.getDataset) @@ -57,33 +105,95 @@ class JenaFormatUtil(private val modelFactory: JenaModelFactory) extends RdfForm rdfFormat match { case Turtle => - val rdfFormat: jena.riot.RDFFormat = if (prettyPrint) { + val jenaRdfFormat: jena.riot.RDFFormat = if (prettyPrint) { jena.riot.RDFFormat.TURTLE_PRETTY } else { jena.riot.RDFFormat.TURTLE_FLAT } - jena.riot.RDFDataMgr.write(stringWriter, datasetGraph.getDefaultGraph, rdfFormat) + jena.riot.RDFDataMgr.write(stringWriter, datasetGraph.getDefaultGraph, jenaRdfFormat) case RdfXml => - val rdfFormat: jena.riot.RDFFormat = if (prettyPrint) { + val jenaRdfFormat: jena.riot.RDFFormat = if (prettyPrint) { jena.riot.RDFFormat.RDFXML_PRETTY } else { jena.riot.RDFFormat.RDFXML_PLAIN } - jena.riot.RDFDataMgr.write(stringWriter, datasetGraph.getDefaultGraph, rdfFormat) + jena.riot.RDFDataMgr.write(stringWriter, datasetGraph.getDefaultGraph, jenaRdfFormat) case TriG => - val rdfFormat: jena.riot.RDFFormat = if (prettyPrint) { + val jenaRdfFormat: jena.riot.RDFFormat = if (prettyPrint) { jena.riot.RDFFormat.TRIG_PRETTY } else { jena.riot.RDFFormat.TRIG_FLAT } - jena.riot.RDFDataMgr.write(stringWriter, datasetGraph, rdfFormat) + jena.riot.RDFDataMgr.write(stringWriter, datasetGraph, jenaRdfFormat) } stringWriter.toString } + + override def parseWithStreamProcessor(rdfSource: RdfSource, + rdfFormat: NonJsonLD, + rdfStreamProcessor: RdfStreamProcessor): Unit = { + // Wrap the RdfStreamProcessor in a StreamProcessorAsStreamRDF. + val streamRDF = new StreamProcessorAsStreamRDF(rdfStreamProcessor) + + // Build a parser. + val parser = jena.riot.RDFParser.create() + + // Configure it to read from the input source. + rdfSource match { + case RdfStringSource(rdfStr) => parser.source(new StringReader(rdfStr)) + case RdfInputStreamSource(inputStream) => parser.source(inputStream) + } + + // Add the other configuration and run the parser. + parser.lang(rdfFormatToJenaParsingLang(rdfFormat)) + .errorHandler(jena.riot.system.ErrorHandlerFactory.errorHandlerStrictNoLogging) + .parse(streamRDF) + } + + override def inputStreamToRdfModel(inputStream: InputStream, rdfFormat: NonJsonLD): RdfModel = { + val model: JenaModel = modelFactory.makeEmptyModel + + jena.riot.RDFDataMgr.read( + model.getDataset.asDatasetGraph, + inputStream, + rdfFormatToJenaParsingLang(rdfFormat) + ) + + model + } + + override def makeFormattingStreamProcessor(outputStream: OutputStream, + rdfFormat: NonJsonLD): RdfStreamProcessor = { + // Construct a Jena StreamRDF for the requested format. + val streamRDF: jena.riot.system.StreamRDF = jena.riot.system.StreamRDFWriter.getWriterStream( + outputStream, + rdfFormatToJenaParsingLang(rdfFormat) + ) + + // Wrap it in a StreamRDFAsStreamProcessor. + new StreamRDFAsStreamProcessor(streamRDF) + } + + override def rdfModelToOutputStream(rdfModel: RdfModel, outputStream: OutputStream, rdfFormat: NonJsonLD): Unit = { + import JenaConversions._ + + val datasetGraph: jena.sparql.core.DatasetGraph = rdfModel.asJenaDataset.asDatasetGraph + + rdfFormat match { + case Turtle => + jena.riot.RDFDataMgr.write(outputStream, datasetGraph.getDefaultGraph, jena.riot.RDFFormat.TURTLE_FLAT) + + case RdfXml => + jena.riot.RDFDataMgr.write(outputStream, datasetGraph.getDefaultGraph, jena.riot.RDFFormat.RDFXML_PLAIN) + + case TriG => + jena.riot.RDFDataMgr.write(outputStream, datasetGraph, jena.riot.RDFFormat.TRIG_FLAT) + } + } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaModel.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaModel.scala index 77c560966a..a95f39cde5 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaModel.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/jenaimpl/JenaModel.scala @@ -24,9 +24,11 @@ import org.knora.webapi.IRI import org.knora.webapi.exceptions.RdfProcessingException import org.knora.webapi.feature.Feature import org.knora.webapi.messages.OntologyConstants +import org.knora.webapi.messages.util.ErrorHandlingMap import org.knora.webapi.messages.util.rdf._ import scala.collection.JavaConverters._ +import scala.collection.mutable.ArrayBuffer sealed trait JenaNode extends RdfNode { @@ -182,6 +184,12 @@ class JenaModel(private val dataset: jena.query.Dataset, private val datasetGraph: jena.sparql.core.DatasetGraph = dataset.asDatasetGraph + private class StatementIterator(jenaIterator: java.util.Iterator[jena.sparql.core.Quad]) extends Iterator[Statement] { + override def hasNext: Boolean = jenaIterator.hasNext + + override def next(): Statement = JenaStatement(jenaIterator.next()) + } + /** * Returns the underlying [[jena.query.Dataset]]. */ @@ -193,8 +201,6 @@ class JenaModel(private val dataset: jena.query.Dataset, datasetGraph.add(statement.asJenaQuad) } - override def getStatements: Set[Statement] = datasetGraph.find.asScala.map(JenaStatement).toSet - /** * Converts an optional [[RdfNode]] to a [[jena.graph.Node]], converting * `None` to a wildcard that will match any node. @@ -225,13 +231,22 @@ class JenaModel(private val dataset: jena.query.Dataset, datasetGraph.delete(statement.asJenaQuad) } - override def find(subj: Option[RdfResource], pred: Option[IriNode], obj: Option[RdfNode], context: Option[IRI] = None): Set[Statement] = { - datasetGraph.find( - contextNodeOrWildcard(context), - asJenaNodeOrWildcard(subj), - asJenaNodeOrWildcard(pred), - asJenaNodeOrWildcard(obj) - ).asScala.map(JenaStatement).toSet + override def find(subj: Option[RdfResource], + pred: Option[IriNode], + obj: Option[RdfNode], + context: Option[IRI] = None): Iterator[Statement] = { + new StatementIterator( + datasetGraph.find( + contextNodeOrWildcard(context), + asJenaNodeOrWildcard(subj), + asJenaNodeOrWildcard(pred), + asJenaNodeOrWildcard(obj) + ) + ) + } + + override def contains(statement: Statement): Boolean = { + datasetGraph.contains(statement.asJenaQuad) } override def setNamespace(prefix: String, namespace: IRI): Unit = { @@ -306,6 +321,33 @@ class JenaModel(private val dataset: jena.query.Dataset, node: jena.graph.Node => node.getURI } } + + override def asRepository: RdfRepository = { + new JenaRepository(dataset) + } + + override def size: Int = { + // Jena's DatasetGraph doesn't have a method for this, so we have to do it ourselves. + + // Get the size of the default graph. + val defaultGraphSize: Int = datasetGraph.getDefaultGraph.size + + // Get the sum of the sizes of the named graphs. + val sumOfNamedGraphSizes: Int = datasetGraph.listGraphNodes.asScala.map { + namedGraphIri => datasetGraph.getGraph(namedGraphIri) + }.map(_.size).sum + + // Return the sum of those sizes. + defaultGraphSize + sumOfNamedGraphSizes + } + + override def iterator: Iterator[Statement] = { + new StatementIterator(datasetGraph.find) + } + + override def clear(): Unit = { + datasetGraph.clear() + } } /** @@ -385,3 +427,46 @@ class JenaModelFactory(private val nodeFactory: JenaNodeFactory) extends RdfMode nodeFactory = nodeFactory ) } + +/** + * An [[RdfRepository]] that wraps a [[jena.query.Dataset]]. + * + * @param dataset the dataset to be queried. + */ +class JenaRepository(private val dataset: jena.query.Dataset) extends RdfRepository { + override def doSelect(selectQuery: String): SparqlSelectResult = { + // Run the query. + + val queryExecution: jena.query.QueryExecution = + jena.query.QueryExecutionFactory.create(selectQuery, dataset) + + val resultSet: jena.query.ResultSet = queryExecution.execSelect + + // Convert the query result to a SparqlSelectResponse. + + val header = SparqlSelectResultHeader(resultSet.getResultVars.asScala) + val rowBuffer = ArrayBuffer.empty[VariableResultsRow] + + while (resultSet.hasNext) { + val querySolution: jena.query.QuerySolution = resultSet.next + val varNames: Iterator[String] = querySolution.varNames.asScala + + val rowMap: Map[String, String] = varNames.map { + varName => varName -> querySolution.get(varName).asNode.toString + }.toMap + + rowBuffer.append(VariableResultsRow(new ErrorHandlingMap[String, String](rowMap, { key: String => s"No value found for SPARQL query variable '$key' in query result row" }))) + } + + queryExecution.close() + + SparqlSelectResult( + head = header, + results = SparqlSelectResultBody(bindings = rowBuffer) + ) + } + + override def shutDown(): Unit = { + dataset.close() + } +} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JFormatUtil.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JFormatUtil.scala index 2116173143..96d3ea0f60 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JFormatUtil.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JFormatUtil.scala @@ -19,12 +19,52 @@ package org.knora.webapi.messages.util.rdf.rdf4jimpl -import java.io.{StringReader, StringWriter} +import java.io.{InputStream, OutputStream, StringReader, StringWriter} import org.eclipse.rdf4j +import org.knora.webapi.IRI import org.knora.webapi.feature.Feature import org.knora.webapi.messages.util.rdf._ +/** + * Wraps an [[RdfStreamProcessor]] in an [[rdf4j.rio.RDFHandler]]. + */ +class StreamProcessorAsRDFHandler(streamProcessor: RdfStreamProcessor) extends rdf4j.rio.RDFHandler { + override def startRDF(): Unit = streamProcessor.start() + + override def endRDF(): Unit = streamProcessor.finish() + + override def handleNamespace(prefix: String, namespace: String): Unit = { + streamProcessor.processNamespace(prefix, namespace) + } + + override def handleStatement(statement: rdf4j.model.Statement): Unit = { + streamProcessor.processStatement(RDF4JStatement(statement)) + } + + override def handleComment(comment: String): Unit = {} +} + +/** + * Wraps an [[rdf4j.rio.RDFHandler]] in an [[RdfStreamProcessor]]. + */ +class RDFHandlerAsStreamProcessor(rdfWriter: rdf4j.rio.RDFHandler) extends RdfStreamProcessor { + + import RDF4JConversions._ + + override def start(): Unit = rdfWriter.startRDF() + + override def processNamespace(prefix: String, namespace: IRI): Unit = { + rdfWriter.handleNamespace(prefix, namespace) + } + + override def processStatement(statement: Statement): Unit = { + rdfWriter.handleStatement(statement.asRDF4JStatement) + } + + override def finish(): Unit = rdfWriter.endRDF() +} + /** * An implementation of [[RdfFormatUtil]] that uses the RDF4J API. */ @@ -40,7 +80,7 @@ class RDF4JFormatUtil(private val modelFactory: RDF4JModelFactory, } } - protected def parseNonJsonLDToRdfModel(rdfStr: String, rdfFormat: NonJsonLD): RdfModel = { + override def parseNonJsonLDToRdfModel(rdfStr: String, rdfFormat: NonJsonLD): RdfModel = { new RDF4JModel( model = rdf4j.rio.Rio.parse( new StringReader(rdfStr), @@ -59,16 +99,66 @@ class RDF4JFormatUtil(private val modelFactory: RDF4JModelFactory, val rdfWriter: rdf4j.rio.RDFWriter = rdfFormat match { case Turtle => rdf4j.rio.Rio.createWriter(rdf4j.rio.RDFFormat.TURTLE, stringWriter) case TriG => rdf4j.rio.Rio.createWriter(rdf4j.rio.RDFFormat.TRIG, stringWriter) - case RdfXml => new rdf4j.rio.rdfxml.util.RDFXMLPrettyWriter(stringWriter) + case RdfXml => rdf4j.rio.Rio.createWriter(rdf4j.rio.RDFFormat.RDFXML, stringWriter) } // Configure the RDFWriter. - rdfWriter.getWriterConfig. - set[java.lang.Boolean](rdf4j.rio.helpers.BasicWriterSettings.INLINE_BLANK_NODES, true). - set[java.lang.Boolean](rdf4j.rio.helpers.BasicWriterSettings.PRETTY_PRINT, true) + if (prettyPrint) { + rdfWriter.getWriterConfig. + set[java.lang.Boolean](rdf4j.rio.helpers.BasicWriterSettings.INLINE_BLANK_NODES, true). + set[java.lang.Boolean](rdf4j.rio.helpers.BasicWriterSettings.PRETTY_PRINT, prettyPrint) + } // Format the RDF. rdf4j.rio.Rio.write(rdfModel.asRDF4JModel, rdfWriter) stringWriter.toString } + + override def parseWithStreamProcessor(rdfSource: RdfSource, + rdfFormat: NonJsonLD, + rdfStreamProcessor: RdfStreamProcessor): Unit = { + // Construct an RDF4J parser for the requested format. + val parser: rdf4j.rio.RDFParser = rdf4j.rio.Rio.createParser(rdfFormatToRDF4JFormat(rdfFormat)) + + // Wrap the RdfStreamProcessor in a StreamProcessorAsRDFHandler and set it as the parser's RDFHandler. + parser.setRDFHandler(new StreamProcessorAsRDFHandler(rdfStreamProcessor)) + + // Parse from the input source. + rdfSource match { + case RdfStringSource(rdfStr) => parser.parse(new StringReader(rdfStr), "") + case RdfInputStreamSource(inputStream) => parser.parse(inputStream, "") + } + } + + override def inputStreamToRdfModel(inputStream: InputStream, rdfFormat: NonJsonLD): RdfModel = { + val model: rdf4j.model.Model = rdf4j.rio.Rio.parse( + inputStream, + "", + rdfFormatToRDF4JFormat(rdfFormat) + ) + + new RDF4JModel( + model = model, + nodeFactory = nodeFactory + ) + } + + override def makeFormattingStreamProcessor(outputStream: OutputStream, + rdfFormat: NonJsonLD): RdfStreamProcessor = { + // Construct an RDF4J writer for the requested format. + val rdfWriter: rdf4j.rio.RDFWriter = rdf4j.rio.Rio.createWriter(rdfFormatToRDF4JFormat(rdfFormat), outputStream) + + // Wrap it in an RDFHandlerAsStreamProcessor. + new RDFHandlerAsStreamProcessor(rdfWriter) + } + + override def rdfModelToOutputStream(rdfModel: RdfModel, outputStream: OutputStream, rdfFormat: NonJsonLD): Unit = { + import RDF4JConversions._ + + // Construct an RDF4J writer for the requested format. + val rdfWriter: rdf4j.rio.RDFWriter = rdf4j.rio.Rio.createWriter(rdfFormatToRDF4JFormat(rdfFormat), outputStream) + + // Format the RDF. + rdf4j.rio.Rio.write(rdfModel.asRDF4JModel, rdfWriter) + } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JModel.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JModel.scala index a3a71eb8e0..62ecdbd911 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JModel.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/rdf/rdf4jimpl/RDF4JModel.scala @@ -23,10 +23,12 @@ import org.eclipse.rdf4j import org.knora.webapi.IRI import org.knora.webapi.exceptions.RdfProcessingException import org.knora.webapi.feature.Feature +import org.knora.webapi.messages.util.ErrorHandlingMap import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.util.JavaUtil._ import scala.collection.JavaConverters._ +import scala.collection.mutable.ArrayBuffer sealed trait RDF4JNode extends RdfNode { def rdf4jValue: rdf4j.model.Value @@ -169,6 +171,12 @@ class RDF4JModel(private val model: rdf4j.model.Model, private val valueFactory: rdf4j.model.ValueFactory = rdf4j.model.impl.SimpleValueFactory.getInstance + private class StatementIterator(rdf4jIterator: java.util.Iterator[rdf4j.model.Statement]) extends Iterator[Statement] { + override def hasNext: Boolean = rdf4jIterator.hasNext + + override def next(): Statement = RDF4JStatement(rdf4jIterator.next()) + } + /** * Returns the underlying [[rdf4j.model.Model]]. */ @@ -176,8 +184,6 @@ class RDF4JModel(private val model: rdf4j.model.Model, override def getNodeFactory: RdfNodeFactory = nodeFactory - override def getStatements: Set[Statement] = model.asScala.toSet.map(RDF4JStatement) - override def addStatement(statement: Statement): Unit = { model.add(statement.asRDF4JStatement) } @@ -221,15 +227,18 @@ class RDF4JModel(private val model: rdf4j.model.Model, } override def removeStatement(statement: Statement): Unit = { - remove( - Some(statement.subj), - Some(statement.pred), - Some(statement.obj), - statement.context + model.remove( + statement.subj.asRDF4JResource, + statement.pred.asRDF4JIri, + statement.obj.asRDF4JValue, + statement.context.map(definedContext => valueFactory.createIRI(definedContext)).orNull ) } - override def find(subj: Option[RdfResource], pred: Option[IriNode], obj: Option[RdfNode], context: Option[IRI] = None): Set[Statement] = { + override def find(subj: Option[RdfResource], + pred: Option[IriNode], + obj: Option[RdfNode], + context: Option[IRI] = None): Iterator[Statement] = { val filteredModel: rdf4j.model.Model = context match { case Some(definedContext) => model.filter( @@ -247,7 +256,16 @@ class RDF4JModel(private val model: rdf4j.model.Model, ) } - filteredModel.asScala.map(RDF4JStatement).toSet + new StatementIterator(filteredModel.iterator) + } + + override def contains(statement: Statement): Boolean = { + model.contains( + statement.subj.asRDF4JResource, + statement.pred.asRDF4JIri, + statement.obj.asRDF4JValue, + statement.context.map(definedContext => valueFactory.createIRI(definedContext)).orNull + ) } override def setNamespace(prefix: String, namespace: IRI): Unit = { @@ -275,6 +293,20 @@ class RDF4JModel(private val model: rdf4j.model.Model, context: rdf4j.model.Resource => context.stringValue } } + + override def asRepository: RdfRepository = { + new RDF4JRepository(model) + } + + override def size: Int = model.size + + override def iterator: Iterator[Statement] = { + new StatementIterator(model.iterator) + } + + override def clear(): Unit = { + model.remove(null, null, null) + } } /** @@ -337,3 +369,52 @@ class RDF4JModelFactory(private val nodeFactory: RDF4JNodeFactory) extends RdfMo nodeFactory = nodeFactory ) } + +/** + * An [[RdfRepository]] that wraps an [[rdf4j.model.Model]] in an [[rdf4j.repository.sail.SailRepository]]. + * + * @param model the model to be queried. + */ +class RDF4JRepository(model: rdf4j.model.Model) extends RdfRepository { + // Construct an in-memory SailRepository containing the model. + val repository = new rdf4j.repository.sail.SailRepository(new rdf4j.sail.memory.MemoryStore()) + repository.init() + val connection: rdf4j.repository.sail.SailRepositoryConnection = repository.getConnection + connection.add(model) + connection.close() + + override def doSelect(selectQuery: String): SparqlSelectResult = { + // Run the query. + + val connection = repository.getConnection + val tupleQuery: rdf4j.query.TupleQuery = connection.prepareTupleQuery(selectQuery) + val tupleQueryResult: rdf4j.query.TupleQueryResult = tupleQuery.evaluate + + // Convert the query result to a SparqlSelectResponse. + + val header = SparqlSelectResultHeader(tupleQueryResult.getBindingNames.asScala) + val rowBuffer = ArrayBuffer.empty[VariableResultsRow] + + while (tupleQueryResult.hasNext) { + val bindings: Iterable[rdf4j.query.Binding] = tupleQueryResult.next.asScala + + val rowMap: Map[String, String] = bindings.map { + binding => binding.getName -> binding.getValue.stringValue + }.toMap + + rowBuffer.append(VariableResultsRow(new ErrorHandlingMap[String, String](rowMap, { key: String => s"No value found for SPARQL query variable '$key' in query result row" }))) + } + + tupleQueryResult.close() + connection.close() + + SparqlSelectResult( + head = header, + results = SparqlSelectResultBody(bindings = rowBuffer) + ) + } + + override def shutDown(): Unit = { + repository.shutDown() + } +} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/mainquery/GravsearchMainQueryGenerator.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/mainquery/GravsearchMainQueryGenerator.scala index fa47eab456..bf884f6317 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/mainquery/GravsearchMainQueryGenerator.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/mainquery/GravsearchMainQueryGenerator.scala @@ -22,8 +22,8 @@ package org.knora.webapi.messages.util.search.gravsearch.mainquery import org.knora.webapi._ import org.knora.webapi.exceptions.GravsearchException import org.knora.webapi.messages.IriConversions._ -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectResponse, VariableResultsRow} import org.knora.webapi.messages.util.ErrorHandlingMap +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.search.{AndExpression, CompareExpression, CompareExpressionOperator, ConstructClause, ConstructQuery, FilterPattern, IriRef, OptionalPattern, QueryPattern, QueryVariable, StatementPattern, UnionPattern, ValuesPattern, WhereClause, XsdLiteral} import org.knora.webapi.messages.util.search.gravsearch.prequery.{AbstractPrequeryGenerator, NonTriplestoreSpecificGravsearchToPrequeryTransformer} import org.knora.webapi.messages.{OntologyConstants, StringFormatter} @@ -111,7 +111,7 @@ object GravsearchMainQueryGenerator { * @param mainResourceVar the variable representing the main resource. * @return a [[DependentResourcesPerMainResource]]. */ - def getDependentResourceIrisPerMainResource(prequeryResponse: SparqlSelectResponse, + def getDependentResourceIrisPerMainResource(prequeryResponse: SparqlSelectResult, transformer: NonTriplestoreSpecificGravsearchToPrequeryTransformer, mainResourceVar: QueryVariable): DependentResourcesPerMainResource = { @@ -165,7 +165,7 @@ object GravsearchMainQueryGenerator { * @param mainResourceVar the variable representing the main resource. * @return [[ValueObjectVariablesAndValueObjectIris]]. */ - def getValueObjectVarsAndIrisPerMainResource(prequeryResponse: SparqlSelectResponse, + def getValueObjectVarsAndIrisPerMainResource(prequeryResponse: SparqlSelectResult, transformer: NonTriplestoreSpecificGravsearchToPrequeryTransformer, mainResourceVar: QueryVariable): ValueObjectVariablesAndValueObjectIris = { diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/standoff/StandoffTagUtilV2.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/standoff/StandoffTagUtilV2.scala index d1a820032c..89834c063f 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/standoff/StandoffTagUtilV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/standoff/StandoffTagUtilV2.scala @@ -139,14 +139,14 @@ object StandoffTagUtilV2 { case Some(SmartIriLiteralV2(SmartIri(OntologyConstants.Xsd.DateTime))) => StandoffTagTimeAttributeV2(standoffPropertyIri = standoffTagPropIri, value = stringFormatter.xsdDateTimeStampToInstant(attr.value, throw BadRequestException(s"Invalid timestamp attribute: '${attr.value}'"))) - case None => throw InconsistentTriplestoreDataException(s"did not find ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} for $standoffTagPropIri") + case None => throw InconsistentRepositoryDataException(s"did not find ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} for $standoffTagPropIri") - case other => throw InconsistentTriplestoreDataException(s"triplestore returned unknown ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} '$other' for $standoffTagPropIri") + case other => throw InconsistentRepositoryDataException(s"triplestore returned unknown ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} '$other' for $standoffTagPropIri") } } else { // only properties with a `ObjectDatatypeConstraint` are allowed here (linking properties have to be created via data type standoff classes) - throw InconsistentTriplestoreDataException(s"no ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} given for property '$standoffTagPropIri'") + throw InconsistentRepositoryDataException(s"no ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} given for property '$standoffTagPropIri'") } }.toList @@ -655,7 +655,7 @@ object StandoffTagUtilV2 { ) - case unknownDataType => throw InconsistentTriplestoreDataException(s"the triplestore returned the data type $unknownDataType for $standoffClassIri that could be handled") + case unknownDataType => throw InconsistentRepositoryDataException(s"the triplestore returned the data type $unknownDataType for $standoffClassIri that could be handled") } } @@ -782,7 +782,7 @@ object StandoffTagUtilV2 { val standoffTagSmartIri: SmartIri = standoffTagIri.toSmartIri if (!standoffTagSmartIri.isKnoraStandoffIri) { - throw InconsistentTriplestoreDataException(s"Invalid standoff tag IRI: $standoffTagIri") + throw InconsistentRepositoryDataException(s"Invalid standoff tag IRI: $standoffTagIri") } // The start index in the tag's IRI should match the one in its assertions. @@ -791,7 +791,7 @@ object StandoffTagUtilV2 { val startIndexFromAssertions: Int = standoffTagAssertions(OntologyConstants.KnoraBase.StandoffTagHasStartIndex).toInt if (startIndexFromAssertions != startIndexFromIri) { - throw InconsistentTriplestoreDataException(s"Standoff tag $standoffTagIri has start index $startIndexFromAssertions (expected $startIndexFromIri)") + throw InconsistentRepositoryDataException(s"Standoff tag $standoffTagIri has start index $startIndexFromAssertions (expected $startIndexFromIri)") } // create a sequence of `StandoffTagAttributeV2` from the given attributes @@ -818,7 +818,7 @@ object StandoffTagUtilV2 { case None => // If a v1 SPARQL template was used, we have to get the target node and to get its XML ID. - standoffAssertions(value).getOrElse(OntologyConstants.KnoraBase.StandoffTagHasOriginalXMLID, throw InconsistentTriplestoreDataException(s"referred standoff $value node has no original XML id")) + standoffAssertions(value).getOrElse(OntologyConstants.KnoraBase.StandoffTagHasOriginalXMLID, throw InconsistentRepositoryDataException(s"referred standoff $value node has no original XML id")) } // recreate the original id reference @@ -852,13 +852,13 @@ object StandoffTagUtilV2 { case Some(SmartIriLiteralV2(SmartIri(OntologyConstants.Xsd.Uri))) => StandoffTagUriAttributeV2(standoffPropertyIri = propSmartIri, value = value) - case None => throw InconsistentTriplestoreDataException(s"did not find ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} for $propIri") + case None => throw InconsistentRepositoryDataException(s"did not find ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} for $propIri") - case other => throw InconsistentTriplestoreDataException(s"triplestore returned unknown ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} '$other' for $propIri") + case other => throw InconsistentRepositoryDataException(s"triplestore returned unknown ${OntologyConstants.KnoraBase.ObjectDatatypeConstraint} '$other' for $propIri") } } else { - throw InconsistentTriplestoreDataException(s"no object class or data type constraint found for property '$propIri'") + throw InconsistentRepositoryDataException(s"no object class or data type constraint found for property '$propIri'") } }.toVector @@ -1029,7 +1029,7 @@ object StandoffTagUtilV2 { case None => convertStandoffAttributeTags(xmlItemForStandoffClass.attributes, standoffTagV2.attributes) - case unknownDataType => throw InconsistentTriplestoreDataException(s"the triplestore returned an unknown data type for ${standoffTagV2.standoffTagClassIri} that could not be handled") + case unknownDataType => throw InconsistentRepositoryDataException(s"the triplestore returned an unknown data type for ${standoffTagV2.standoffTagClassIri} that could not be handled") } @@ -1060,7 +1060,7 @@ object StandoffTagUtilV2 { startPosition = standoffTagV2.startPosition, endPosition = standoffTagV2.endPosition, startIndex = standoffTagV2.startIndex, - endIndex = standoffTagV2.endIndex.getOrElse(throw InconsistentTriplestoreDataException(s"end index is missing for a free standoff tag")), + endIndex = standoffTagV2.endIndex.getOrElse(throw InconsistentRepositoryDataException(s"end index is missing for a free standoff tag")), startParentIndex = standoffTagV2.startParentIndex, endParentIndex = standoffTagV2.endParentIndex, attributes = attributesWithClass.toSet diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala index b5c0fac37e..bf81abb491 100755 --- a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala @@ -24,7 +24,7 @@ import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, DataConversionException, InconsistentTriplestoreDataException, InvalidApiJsonException} +import org.knora.webapi.exceptions.{BadRequestException, DataConversionException, InconsistentRepositoryDataException, InvalidApiJsonException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants import org.knora.webapi.messages.admin.responder.projectsmessages.ProjectADM @@ -826,7 +826,7 @@ object SalsahGuiConversions { def iri2SalsahGuiElement(iri: IRI): String = { iris2SalsahGuiElements.get(iri) match { case Some(salsahGuiElement) => salsahGuiElement - case None => throw new InconsistentTriplestoreDataException(s"No SALSAH GUI element found for IRI: $iri") + case None => throw new InconsistentRepositoryDataException(s"No SALSAH GUI element found for IRI: $iri") } } @@ -839,7 +839,7 @@ object SalsahGuiConversions { def salsahGuiElement2Iri(salsahGuiElement: String): IRI = { salsahGuiElements2Iris.get(salsahGuiElement) match { case Some(iri) => iri - case None => throw new InconsistentTriplestoreDataException(s"No IRI found for SALSAH GUI element: $salsahGuiElement") + case None => throw new InconsistentRepositoryDataException(s"No IRI found for SALSAH GUI element: $salsahGuiElement") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/usermessages/UserMessagesV1.scala b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/usermessages/UserMessagesV1.scala index 65ec56d329..41687e1ba4 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/usermessages/UserMessagesV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/usermessages/UserMessagesV1.scala @@ -23,7 +23,7 @@ import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{BadRequestException, InconsistentRepositoryDataException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.admin.responder.permissionsmessages.{PermissionsADMJsonProtocol, PermissionsDataADM} import org.knora.webapi.messages.v1.responder.projectmessages.{ProjectInfoV1, ProjectV1JsonProtocol} @@ -392,7 +392,7 @@ object UserProfileTypeV1 extends Enumeration { /** * Given the name of a value in this enumeration, returns the value. If the value is not found, throws an - * [[InconsistentTriplestoreDataException]]. + * [[InconsistentRepositoryDataException]]. * * @param name the name of the value. * @return the requested value. @@ -400,7 +400,7 @@ object UserProfileTypeV1 extends Enumeration { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"User profile type not supported: $name") + case None => throw InconsistentRepositoryDataException(s"User profile type not supported: $name") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala index 9a5ae139aa..b6fc84a802 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala @@ -24,7 +24,7 @@ import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, InconsistentTriplestoreDataException, NotImplementedException} +import org.knora.webapi.exceptions.{BadRequestException, InconsistentRepositoryDataException, NotImplementedException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.admin.responder.projectsmessages.ProjectADM import org.knora.webapi.messages.admin.responder.usersmessages.UserADM @@ -591,7 +591,7 @@ object KnoraCalendarV1 extends Enumeration { /** * Given the name of a value in this enumeration, returns the value. If the value is not found, throws an - * [[InconsistentTriplestoreDataException]]. + * [[InconsistentRepositoryDataException]]. * * @param name the name of the value. * @return the requested value. @@ -599,7 +599,7 @@ object KnoraCalendarV1 extends Enumeration { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"Calendar type not supported: $name") + case None => throw InconsistentRepositoryDataException(s"Calendar type not supported: $name") } } } @@ -617,7 +617,7 @@ object KnoraPrecisionV1 extends Enumeration { /** * Given the name of a value in this enumeration, returns the value. If the value is not found, throws an - * [[InconsistentTriplestoreDataException]]. + * [[InconsistentRepositoryDataException]]. * * @param name the name of the value. * @return the requested value. @@ -625,7 +625,7 @@ object KnoraPrecisionV1 extends Enumeration { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"Calendar precision not supported: $name") + case None => throw InconsistentRepositoryDataException(s"Calendar precision not supported: $name") } } } @@ -779,7 +779,7 @@ case class TextValueWithStandoffV1(utf8str: String, // unescape utf8str since it contains escaped sequences while the string returned by the triplestore does not stringFormatter.fromSparqlEncodedString(utf8str) == otherText.utf8str - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -808,7 +808,7 @@ case class TextValueWithStandoffV1(utf8str: String, utf8strIdentical && standoffIdentical && textValueWithStandoffV1.mappingIri == this.mappingIri - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -847,7 +847,7 @@ case class TextValueSimpleV1(utf8str: String, language: Option[String] = None) e override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case otherText: TextValueV1 => otherText.utf8str == utf8str - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -864,7 +864,7 @@ case class TextValueSimpleV1(utf8str: String, language: Option[String] = None) e currentVersion match { case textValueSimpleV1: TextValueSimpleV1 => textValueSimpleV1 == this case _: TextValueWithStandoffV1 => false - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } @@ -932,7 +932,7 @@ case class LinkUpdateV1(targetResourceIri: IRI, targetExists: Boolean = true) ex other match { case linkV1: LinkV1 => targetResourceIri == linkV1.targetResourceIri case linkValueV1: LinkValueV1 => targetResourceIri == linkValueV1.objectIri - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -992,7 +992,7 @@ case class HierarchicalListValueV1(hierarchicalListIri: IRI) extends UpdateValue override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case listValueV1: HierarchicalListValueV1 => listValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1005,7 +1005,7 @@ case class HierarchicalListValueV1(hierarchicalListIri: IRI) extends UpdateValue override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case listValueV1: HierarchicalListValueV1 => listValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1032,7 +1032,7 @@ case class IntegerValueV1(ival: Int) extends UpdateValueV1 with ApiValueV1 { override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case integerValueV1: IntegerValueV1 => integerValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1045,7 +1045,7 @@ case class IntegerValueV1(ival: Int) extends UpdateValueV1 with ApiValueV1 { override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case integerValueV1: IntegerValueV1 => integerValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1081,7 +1081,7 @@ case class BooleanValueV1(bval: Boolean) extends UpdateValueV1 with ApiValueV1 { override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case booleanValueV1: BooleanValueV1 => booleanValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1108,7 +1108,7 @@ case class UriValueV1(uri: String) extends UpdateValueV1 with ApiValueV1 { override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case uriValueV1: UriValueV1 => uriValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1121,7 +1121,7 @@ case class UriValueV1(uri: String) extends UpdateValueV1 with ApiValueV1 { override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case uriValueV1: UriValueV1 => uriValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1147,7 +1147,7 @@ case class DecimalValueV1(dval: BigDecimal) extends UpdateValueV1 with ApiValueV override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case decimalValueV1: DecimalValueV1 => decimalValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1160,7 +1160,7 @@ case class DecimalValueV1(dval: BigDecimal) extends UpdateValueV1 with ApiValueV override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case decimalValueV1: DecimalValueV1 => decimalValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } @@ -1192,7 +1192,7 @@ case class IntervalValueV1(timeval1: BigDecimal, timeval2: BigDecimal) extends U override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case intervalValueV1: IntervalValueV1 => intervalValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1205,7 +1205,7 @@ case class IntervalValueV1(timeval1: BigDecimal, timeval2: BigDecimal) extends U override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case intervalValueV1: IntervalValueV1 => intervalValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1232,7 +1232,7 @@ case class TimeValueV1(timeStamp: Instant) extends UpdateValueV1 with ApiValueV1 override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case timeValueV1: TimeValueV1 => timeValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1245,7 +1245,7 @@ case class TimeValueV1(timeStamp: Instant) extends UpdateValueV1 with ApiValueV1 override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case timeValueV1: TimeValueV1 => timeValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1270,7 +1270,7 @@ case class JulianDayNumberValueV1(dateval1: Int, override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case dateValueV1: DateValueV1 => DateUtilV1.julianDayNumberValueV1ToDateValueV1(this) == other - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1358,7 +1358,7 @@ case class ColorValueV1(color: String) extends UpdateValueV1 with ApiValueV1 { override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case colorValueV1: ColorValueV1 => colorValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1371,7 +1371,7 @@ case class ColorValueV1(color: String) extends UpdateValueV1 with ApiValueV1 { override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case colorValueV1: ColorValueV1 => colorValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1398,7 +1398,7 @@ case class GeomValueV1(geom: String) extends UpdateValueV1 with ApiValueV1 { override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case geomValueV1: GeomValueV1 => geomValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1411,7 +1411,7 @@ case class GeomValueV1(geom: String) extends UpdateValueV1 with ApiValueV1 { override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case geomValueV1: GeomValueV1 => geomValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1438,7 +1438,7 @@ case class GeonameValueV1(geonameCode: String) extends UpdateValueV1 with ApiVal override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case geonameValueV1: GeonameValueV1 => geonameValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1451,7 +1451,7 @@ case class GeonameValueV1(geonameCode: String) extends UpdateValueV1 with ApiVal override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case geonameValueV1: GeonameValueV1 => geonameValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } } @@ -1501,7 +1501,7 @@ case class StillImageFileValueV1(internalMimeType: String, override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case stillImageFileValueV1: StillImageFileValueV1 => stillImageFileValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1514,7 +1514,7 @@ case class StillImageFileValueV1(internalMimeType: String, override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case stillImageFileValueV1: StillImageFileValueV1 => stillImageFileValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } @@ -1554,7 +1554,7 @@ case class MovingImageFileValueV1(internalMimeType: String, override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case movingImageFileValueV1: MovingImageFileValueV1 => movingImageFileValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1567,7 +1567,7 @@ case class MovingImageFileValueV1(internalMimeType: String, override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case movingImageFileValueV1: MovingImageFileValueV1 => movingImageFileValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } @@ -1597,7 +1597,7 @@ case class TextFileValueV1(internalMimeType: String, override def isDuplicateOfOtherValue(other: ApiValueV1): Boolean = { other match { case textFileValueV1: TextFileValueV1 => textFileValueV1 == this - case otherValue => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") + case otherValue => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${otherValue.valueTypeIri}") } } @@ -1610,7 +1610,7 @@ case class TextFileValueV1(internalMimeType: String, override def isRedundant(currentVersion: ApiValueV1): Boolean = { currentVersion match { case textFileValueV1: TextFileValueV1 => textFileValueV1 == this - case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/OntologyMessagesV2.scala b/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/OntologyMessagesV2.scala index 6f4634ffcf..2b1354d3ae 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/OntologyMessagesV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/OntologyMessagesV2.scala @@ -27,7 +27,7 @@ import akka.event.LoggingAdapter import akka.util.Timeout import org.apache.commons.lang3.builder.HashCodeBuilder import org.knora.webapi._ -import org.knora.webapi.exceptions.{AssertionException, BadRequestException, DataConversionException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{AssertionException, BadRequestException, DataConversionException, InconsistentRepositoryDataException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.admin.responder.usersmessages.UserADM @@ -1657,11 +1657,11 @@ object Cardinality extends Enumeration { */ case class OwlCardinalityInfo(owlCardinalityIri: IRI, owlCardinalityValue: Int, guiOrder: Option[Int] = None) { if (!OntologyConstants.Owl.cardinalityOWLRestrictions.contains(owlCardinalityIri)) { - throw InconsistentTriplestoreDataException(s"Invalid OWL cardinality property: $owlCardinalityIri") + throw InconsistentRepositoryDataException(s"Invalid OWL cardinality property: $owlCardinalityIri") } if (!(owlCardinalityValue == 0 || owlCardinalityValue == 1)) { - throw InconsistentTriplestoreDataException(s"Invalid OWL cardinality value: $owlCardinalityValue") + throw InconsistentRepositoryDataException(s"Invalid OWL cardinality value: $owlCardinalityValue") } override def toString: String = s"<$owlCardinalityIri> $owlCardinalityValue" @@ -1706,7 +1706,7 @@ object Cardinality extends Enumeration { /** * Given the name of a value in this enumeration, returns the value. If the value is not found, throws an - * [[InconsistentTriplestoreDataException]]. + * [[InconsistentRepositoryDataException]]. * * @param name the name of the value. * @return the requested value. @@ -1714,7 +1714,7 @@ object Cardinality extends Enumeration { def lookup(name: String): Value = { valueMap.get(name) match { case Some(value) => value - case None => throw InconsistentTriplestoreDataException(s"Cardinality not found: $name") + case None => throw InconsistentRepositoryDataException(s"Cardinality not found: $name") } } @@ -1726,7 +1726,7 @@ object Cardinality extends Enumeration { * @return a [[Value]]. */ def owlCardinality2KnoraCardinality(propertyIri: IRI, owlCardinality: OwlCardinalityInfo): KnoraCardinalityInfo = { - val cardinality = owlCardinality2KnoraCardinalityMap.getOrElse(owlCardinality.copy(guiOrder = None), throw InconsistentTriplestoreDataException(s"Invalid OWL cardinality $owlCardinality for $propertyIri")) + val cardinality = owlCardinality2KnoraCardinalityMap.getOrElse(owlCardinality.copy(guiOrder = None), throw InconsistentRepositoryDataException(s"Invalid OWL cardinality $owlCardinality for $propertyIri")) KnoraCardinalityInfo( cardinality = cardinality, @@ -1804,7 +1804,7 @@ sealed trait EntityInfoContentV2 { } /** - * A convenience method that returns the canonical `rdf:type` of this entity. Throws [[InconsistentTriplestoreDataException]] + * A convenience method that returns the canonical `rdf:type` of this entity. Throws [[InconsistentRepositoryDataException]] * if the entity's predicates do not include `rdf:type`. * * @return the entity's `rdf:type`. @@ -1812,7 +1812,7 @@ sealed trait EntityInfoContentV2 { def getRdfType: SmartIri /** - * A convenience method that returns all the objects of this entity's `rdf:type` predicate. Throws [[InconsistentTriplestoreDataException]] + * A convenience method that returns all the objects of this entity's `rdf:type` predicate. Throws [[InconsistentRepositoryDataException]] * * if the entity's predicates do not include `rdf:type`. * * @return all the values of `rdf:type` for this entity, sorted for determinism. @@ -2619,11 +2619,11 @@ case class ClassInfoContentV2(classIri: SmartIri, if (classTypeSet.size == 1) { classTypeSet.head } else { - throw InconsistentTriplestoreDataException(s"The rdf:type of $classIri is invalid") + throw InconsistentRepositoryDataException(s"The rdf:type of $classIri is invalid") } } - override def getRdfTypes: Seq[SmartIri] = requireIriObjects(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"The rdf:type of $classIri is missing or invalid")).toVector.sorted + override def getRdfTypes: Seq[SmartIri] = requireIriObjects(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"The rdf:type of $classIri is missing or invalid")).toVector.sorted /** * Undoes the SPARQL-escaping of predicate objects. This method is meant to be used after an update, when the @@ -2835,7 +2835,7 @@ case class PropertyInfoContentV2(propertyIri: SmartIri, val predicatesWithAdjustedRdfType: Map[SmartIri, PredicateInfoV2] = if (ontologySchema == InternalSchema && targetSchema == ApiV2Simple) { // Yes. Is this an object property? val rdfTypeIri = OntologyConstants.Rdf.Type.toSmartIri - val sourcePropertyType: SmartIri = getPredicateIriObject(rdfTypeIri).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no rdf:type")) + val sourcePropertyType: SmartIri = getPredicateIriObject(rdfTypeIri).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no rdf:type")) if (sourcePropertyType.toString == OntologyConstants.Owl.ObjectProperty) { // Yes. See if we need to change it to a datatype property. Does it have a knora-base:objectClassConstraint? @@ -2903,11 +2903,11 @@ case class PropertyInfoContentV2(propertyIri: SmartIri, if (propertyTypeSet.size == 1) { propertyTypeSet.head } else { - throw InconsistentTriplestoreDataException(s"The rdf:type of $propertyIri is invalid") + throw InconsistentRepositoryDataException(s"The rdf:type of $propertyIri is invalid") } } - override def getRdfTypes: Seq[SmartIri] = requireIriObjects(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"The rdf:type of $propertyIri is missing or invalid")).toVector.sorted + override def getRdfTypes: Seq[SmartIri] = requireIriObjects(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"The rdf:type of $propertyIri is missing or invalid")).toVector.sorted /** * Undoes the SPARQL-escaping of predicate objects. This method is meant to be used after an update, when the @@ -3005,18 +3005,18 @@ case class IndividualInfoContentV2(individualIri: SmartIri, predicates: Map[SmartIri, PredicateInfoV2], ontologySchema: OntologySchema) extends EntityInfoContentV2 with KnoraContentV2[IndividualInfoContentV2] { override def getRdfType: SmartIri = { - val rdfTypePred = predicates.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"OWL named individual $individualIri has no rdf:type")) + val rdfTypePred = predicates.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"OWL named individual $individualIri has no rdf:type")) val nonIndividualTypes: Seq[SmartIri] = getRdfTypes.filter(iri => iri.toString != OntologyConstants.Owl.NamedIndividual) if (nonIndividualTypes.size != 1) { - throw InconsistentTriplestoreDataException(s"OWL named individual $individualIri has too many objects for rdf:type: ${rdfTypePred.objects.mkString(", ")}") + throw InconsistentRepositoryDataException(s"OWL named individual $individualIri has too many objects for rdf:type: ${rdfTypePred.objects.mkString(", ")}") } nonIndividualTypes.head } - override def getRdfTypes: Seq[SmartIri] = requireIriObjects(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"The rdf:type of $individualIri is missing or invalid")).toVector.sorted + override def getRdfTypes: Seq[SmartIri] = requireIriObjects(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"The rdf:type of $individualIri is missing or invalid")).toVector.sorted override def toOntologySchema(targetSchema: OntologySchema): IndividualInfoContentV2 = { copy( diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/resourcemessages/ResourceMessagesV2.scala b/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/resourcemessages/ResourceMessagesV2.scala index 5fb04e604e..b0a37722f5 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/resourcemessages/ResourceMessagesV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/resourcemessages/ResourceMessagesV2.scala @@ -19,7 +19,6 @@ package org.knora.webapi.messages.v2.responder.resourcemessages -import java.io.{StringReader, StringWriter} import java.time.Instant import java.util.UUID @@ -27,8 +26,7 @@ import akka.actor.ActorRef import akka.event.LoggingAdapter import akka.pattern._ import akka.util.Timeout -import org.eclipse.rdf4j.rio.rdfxml.util.RDFXMLPrettyWriter -import org.eclipse.rdf4j.rio.{RDFFormat, RDFParser, RDFWriter, Rio} +import org.knora.webapi._ import org.knora.webapi.exceptions._ import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ @@ -44,7 +42,6 @@ import org.knora.webapi.messages.v2.responder.valuemessages._ import org.knora.webapi.messages.{OntologyConstants, SmartIri, StringFormatter} import org.knora.webapi.settings.KnoraSettingsImpl import org.knora.webapi.util._ -import org.knora.webapi.{messages, _} import scala.concurrent.{ExecutionContext, Future} @@ -193,14 +190,13 @@ case class ResourceTEIGetRequestV2(resourceIri: IRI, */ case class ResourceTEIGetResponseV2(header: TEIHeader, body: TEIBody) { - def toXML: String = + def toXML: String = { s""" | |${header.toXML} |${body.toXML} - | - """.stripMargin - + |""".stripMargin + } } /** @@ -210,29 +206,31 @@ case class ResourceTEIGetResponseV2(header: TEIHeader, body: TEIBody) { * @param headerXSLT XSLT to be applied to the resource's metadata in RDF/XML. * */ -case class TEIHeader(headerInfo: ReadResourceV2, headerXSLT: Option[String], settings: KnoraSettingsImpl) { +case class TEIHeader(headerInfo: ReadResourceV2, + headerXSLT: Option[String], + featureFactoryConfig: FeatureFactoryConfig, + settings: KnoraSettingsImpl) { def toXML: String = { - if (headerXSLT.nonEmpty) { + val rdfFormatUtil: RdfFormatUtil = RdfFeatureFactory.getRdfFormatUtil(featureFactoryConfig) - val headerJSONLD: JsonLDDocument = ReadResourcesSequenceV2(Vector(headerInfo)).toJsonLDDocument(ApiV2Complex, settings) + // Convert the resource to a JsonLDDocument. + val headerJsonLD: JsonLDDocument = ReadResourcesSequenceV2(Seq(headerInfo)).toJsonLDDocument(ApiV2Complex, settings) - // TODO: Change the XSLT transformation used here to handle rdf:Description, so we can use RdfFormatUtil - // here instead of RDF4J. + // Convert the JsonLDDocument to an RdfModel. + val rdfModel: RdfModel = headerJsonLD.toRdfModel(rdfFormatUtil.getRdfModelFactory) - val rdfParser: RDFParser = Rio.createParser(RDFFormat.JSONLD) - val stringReader = new StringReader(headerJSONLD.toCompactString()) - val stringWriter = new StringWriter() - - val rdfWriter: RDFWriter = new RDFXMLPrettyWriter(stringWriter) - - rdfParser.setRDFHandler(rdfWriter) - rdfParser.parse(stringReader, "") - - val teiHeaderInfos = stringWriter.toString - XMLUtil.applyXSLTransformation(teiHeaderInfos, headerXSLT.get) + // Format the RdfModel as RDF/XML. To ensure that it contains only rdf:Description elements, + // set prettyPrint to false. + val teiXmlHeader: String = rdfFormatUtil.format( + rdfModel = rdfModel, + rdfFormat = RdfXml, + prettyPrint = false + ) + // Run an XSL transformation to convert the RDF/XML to a TEI/XML header. + XMLUtil.applyXSLTransformation(xml = teiXmlHeader, xslt = headerXSLT.get) } else { s""" | diff --git a/webapi/src/main/scala/org/knora/webapi/responders/BUILD.bazel b/webapi/src/main/scala/org/knora/webapi/responders/BUILD.bazel index 806ebf214d..58527bd5af 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/BUILD.bazel +++ b/webapi/src/main/scala/org/knora/webapi/responders/BUILD.bazel @@ -11,10 +11,10 @@ scala_library( "//webapi/src/main/scala/org/knora/webapi/annotation", "//webapi/src/main/scala/org/knora/webapi/core", "//webapi/src/main/scala/org/knora/webapi/exceptions", + "//webapi/src/main/scala/org/knora/webapi/feature", "//webapi/src/main/scala/org/knora/webapi/instrumentation", "//webapi/src/main/scala/org/knora/webapi/messages", "//webapi/src/main/scala/org/knora/webapi/settings", - "//webapi/src/main/scala/org/knora/webapi/feature", "//webapi/src/main/scala/org/knora/webapi/util", "//webapi/src/main/scala/org/knora/webapi/util/cache", "@maven//:com_typesafe_akka_akka_actor_2_12", @@ -25,10 +25,8 @@ scala_library( "@maven//:com_typesafe_play_twirl_api_2_12", "@maven//:com_typesafe_scala_logging_scala_logging_2_12", "@maven//:io_spray_spray_json_2_12", - "@maven//:org_eclipse_rdf4j_rdf4j_client", "@maven//:org_scala_lang_modules_scala_xml_2_12", "@maven//:org_slf4j_slf4j_api", "@maven//:org_springframework_security_spring_security_core", - "@maven//:org_apache_jena_apache_jena_libs", ], ) diff --git a/webapi/src/main/scala/org/knora/webapi/responders/Responder.scala b/webapi/src/main/scala/org/knora/webapi/responders/Responder.scala index 357efa9446..4c0651a8fb 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/Responder.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/Responder.scala @@ -27,8 +27,9 @@ import akka.util.Timeout import com.typesafe.scalalogging.{LazyLogging, Logger} import org.knora.webapi._ import org.knora.webapi.exceptions.{DuplicateValueException, UnexpectedMessageException} -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectRequest, SparqlSelectResponse} +import org.knora.webapi.messages.store.triplestoremessages.SparqlSelectRequest import org.knora.webapi.messages.util.ResponderData +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.{SmartIri, StringFormatter} import org.knora.webapi.settings.{KnoraDispatchers, KnoraSettings, KnoraSettingsImpl} @@ -126,7 +127,7 @@ abstract class Responder(responderData: ResponderData) extends LazyLogging { ignoreRdfSubjectAndObject = ignoreRdfSubjectAndObject ).toString()) - isEntityUsedResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(isEntityUsedSparql)).mapTo[SparqlSelectResponse] + isEntityUsedResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(isEntityUsedSparql)).mapTo[SparqlSelectResult] _ = if (isEntityUsedResponse.results.bindings.nonEmpty) { errorFun diff --git a/webapi/src/main/scala/org/knora/webapi/responders/admin/GroupsResponderADM.scala b/webapi/src/main/scala/org/knora/webapi/responders/admin/GroupsResponderADM.scala index e382b7d665..dd292cc0c4 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/admin/GroupsResponderADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/admin/GroupsResponderADM.scala @@ -31,6 +31,7 @@ import org.knora.webapi.messages.admin.responder.groupsmessages._ import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectADM, ProjectGetADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages._ import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.v1.responder.projectmessages._ import org.knora.webapi.messages.{OntologyConstants, SmartIri} @@ -91,7 +92,7 @@ class GroupsResponderADM(responderData: ResponderData) extends Responder(respond groups: Seq[Future[GroupADM]] = statements.map { case (groupIri: SubjectV2, propsMap: Map[SmartIri, Seq[LiteralV2]]) => - val projectIri: IRI = propsMap.getOrElse(OntologyConstants.KnoraAdmin.BelongsToProject.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no project attached")).head.asInstanceOf[IriLiteralV2].value + val projectIri: IRI = propsMap.getOrElse(OntologyConstants.KnoraAdmin.BelongsToProject.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no project attached")).head.asInstanceOf[IriLiteralV2].value for { maybeProjectADM: Option[ProjectADM] <- (responderManager ? ProjectGetADM( @@ -102,16 +103,16 @@ class GroupsResponderADM(responderData: ResponderData) extends Responder(respond projectADM: ProjectADM = maybeProjectADM match { case Some(project) => project - case None => throw InconsistentTriplestoreDataException(s"Project $projectIri was referenced by $groupIri but was not found in the triplestore.") + case None => throw InconsistentRepositoryDataException(s"Project $projectIri was referenced by $groupIri but was not found in the triplestore.") } group = GroupADM( id = groupIri.toString, - name = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupName.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no name attached")).head.asInstanceOf[StringLiteralV2].value, - description = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupDescription.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no description attached")).head.asInstanceOf[StringLiteralV2].value, + name = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupName.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no name attached")).head.asInstanceOf[StringLiteralV2].value, + description = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupDescription.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no description attached")).head.asInstanceOf[StringLiteralV2].value, project = projectADM, - status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no status attached")).head.asInstanceOf[BooleanLiteralV2].value, - selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no status attached")).head.asInstanceOf[BooleanLiteralV2].value + status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no status attached")).head.asInstanceOf[BooleanLiteralV2].value, + selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no status attached")).head.asInstanceOf[BooleanLiteralV2].value ) } yield group @@ -261,7 +262,7 @@ class GroupsResponderADM(responderData: ResponderData) extends Responder(respond ).toString()) //_ = log.debug(s"groupMembersByIRIGetRequestV1 - query: $sparqlQueryString") - groupMembersResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + groupMembersResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"groupMembersByIRIGetRequestV1 - result: {}", MessageUtil.toSource(groupMembersResponse)) // get project member IRI from results rows @@ -674,7 +675,7 @@ class GroupsResponderADM(responderData: ResponderData) extends Responder(respond val maybeProjectIri = propsMap.get(OntologyConstants.KnoraAdmin.BelongsToProject.toSmartIri) val projectIriFuture: Future[IRI] = maybeProjectIri match { case Some(iri) => FastFuture.successful(iri.head.asInstanceOf[IriLiteralV2].value) - case None => FastFuture.failed(throw InconsistentTriplestoreDataException(s"Group $groupIri has no project attached")) + case None => FastFuture.failed(throw InconsistentRepositoryDataException(s"Group $groupIri has no project attached")) } if (propsMap.nonEmpty) { @@ -686,15 +687,15 @@ class GroupsResponderADM(responderData: ResponderData) extends Responder(respond requestingUser = KnoraSystemInstances.Users.SystemUser )).mapTo[Option[ProjectADM]] - project: ProjectADM = maybeProject.getOrElse(throw InconsistentTriplestoreDataException(s"Group $groupIri has no project attached.")) + project: ProjectADM = maybeProject.getOrElse(throw InconsistentRepositoryDataException(s"Group $groupIri has no project attached.")) groupADM: GroupADM = GroupADM( id = groupIri, - name = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupName.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no groupName attached")).head.asInstanceOf[StringLiteralV2].value, - description = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupDescription.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no description attached")).head.asInstanceOf[StringLiteralV2].value, + name = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupName.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no groupName attached")).head.asInstanceOf[StringLiteralV2].value, + description = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GroupDescription.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no description attached")).head.asInstanceOf[StringLiteralV2].value, project = project, - status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no status attached")).head.asInstanceOf[BooleanLiteralV2].value, - selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled.toSmartIri, throw InconsistentTriplestoreDataException(s"Group $groupIri has no selfJoin attached")).head.asInstanceOf[BooleanLiteralV2].value + status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no status attached")).head.asInstanceOf[BooleanLiteralV2].value, + selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled.toSmartIri, throw InconsistentRepositoryDataException(s"Group $groupIri has no selfJoin attached")).head.asInstanceOf[BooleanLiteralV2].value ) } yield Some(groupADM) } else { diff --git a/webapi/src/main/scala/org/knora/webapi/responders/admin/ListsResponderADM.scala b/webapi/src/main/scala/org/knora/webapi/responders/admin/ListsResponderADM.scala index 211d9d527b..366c685ef2 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/admin/ListsResponderADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/admin/ListsResponderADM.scala @@ -32,6 +32,7 @@ import org.knora.webapi.messages.admin.responder.listsmessages._ import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectADM, ProjectGetADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages._ import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.{OntologyConstants, SmartIri} import org.knora.webapi.responders.Responder.handleUnexpectedMessage @@ -109,7 +110,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde ListRootNodeInfoADM( id = listIri.toString, - projectIri = propsMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject.toSmartIri, throw InconsistentTriplestoreDataException("The required property 'attachedToProject' not found.")).head.asInstanceOf[IriLiteralV2].value, + projectIri = propsMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject.toSmartIri, throw InconsistentRepositoryDataException("The required property 'attachedToProject' not found.")).head.asInstanceOf[IriLiteralV2].value, name = name, labels = StringLiteralSequenceV2(labels.toVector.sortBy(_.language)), comments = StringLiteralSequenceV2(comments.toVector.sortBy(_.language)) @@ -158,8 +159,8 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde rootNodeInfo = maybeRootNodeInfo match { case Some(info: ListRootNodeInfoADM) => info.asInstanceOf[ListRootNodeInfoADM] - case Some(info: ListChildNodeInfoADM) => throw InconsistentTriplestoreDataException("A child node info was found, although we are expecting a root node info. Please report this as a possible bug.") - case Some(_) | None => throw InconsistentTriplestoreDataException("No info about list node found, although list node should exist. Please report this as a possible bug.") + case Some(info: ListChildNodeInfoADM) => throw InconsistentRepositoryDataException("A child node info was found, although we are expecting a root node info. Please report this as a possible bug.") + case Some(_) | None => throw InconsistentRepositoryDataException("No info about list node found, although list node should exist. Please report this as a possible bug.") } list = ListADM(listinfo = rootNodeInfo, children = children) @@ -289,7 +290,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde case Some(iris: Seq[LiteralV2]) => iris.headOption match { case Some(iri: IriLiteralV2) => Some(iri.value) - case other => throw InconsistentTriplestoreDataException(s"Expected attached to project Iri as an IriLiteralV2 for list node $nodeIri, but got $other") + case other => throw InconsistentRepositoryDataException(s"Expected attached to project Iri as an IriLiteralV2 for list node $nodeIri, but got $other") } case None => None @@ -299,7 +300,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde case Some(iris: Seq[LiteralV2]) => iris.headOption match { case Some(iri: IriLiteralV2) => Some(iri.value) - case other => throw InconsistentTriplestoreDataException(s"Expected root node Iri as an IriLiteralV2 for list node $nodeIri, but got $other") + case other => throw InconsistentRepositoryDataException(s"Expected root node Iri as an IriLiteralV2 for list node $nodeIri, but got $other") } case None => None @@ -309,7 +310,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde case Some(values: Seq[LiteralV2]) => values.headOption match { case Some(value: BooleanLiteralV2) => value.value - case Some(other) => throw InconsistentTriplestoreDataException(s"Expected isRootNode as an BooleanLiteralV2 for list node $nodeIri, but got $other") + case Some(other) => throw InconsistentRepositoryDataException(s"Expected isRootNode as an BooleanLiteralV2 for list node $nodeIri, but got $other") case None => false } @@ -321,7 +322,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde if (isRootNode) { ListRootNodeInfoADM( id = nodeIri.toString, - projectIri = attachedToProjectOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required attachedToProject property missing for list node $nodeIri.")), + projectIri = attachedToProjectOption.getOrElse(throw InconsistentRepositoryDataException(s"Required attachedToProject property missing for list node $nodeIri.")), name = propsMap.get(OntologyConstants.KnoraBase.ListNodeName.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), labels = StringLiteralSequenceV2(labels.toVector.sortBy(_.language)), comments = StringLiteralSequenceV2(comments.toVector.sortBy(_.language)) @@ -332,8 +333,8 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde name = propsMap.get(OntologyConstants.KnoraBase.ListNodeName.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), labels = StringLiteralSequenceV2(labels.toVector.sortBy(_.language)), comments = StringLiteralSequenceV2(comments.toVector.sortBy(_.language)), - position = positionOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required position property missing for list node $nodeIri.")), - hasRootNode = hasRootNodeOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required hasRootNode property missing for list node $nodeIri.")) + position = positionOption.getOrElse(throw InconsistentRepositoryDataException(s"Required position property missing for list node $nodeIri.")), + hasRootNode = hasRootNodeOption.getOrElse(throw InconsistentRepositoryDataException(s"Required hasRootNode property missing for list node $nodeIri.")) ) } } @@ -426,7 +427,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde case Some(iris: Seq[LiteralV2]) => iris.headOption match { case Some(iri: IriLiteralV2) => Some(iri.value) - case other => throw InconsistentTriplestoreDataException(s"Expected attached to project Iri as an IriLiteralV2 for list node $nodeIri, but got $other") + case other => throw InconsistentRepositoryDataException(s"Expected attached to project Iri as an IriLiteralV2 for list node $nodeIri, but got $other") } case None => None @@ -436,7 +437,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde case Some(iris: Seq[LiteralV2]) => iris.headOption match { case Some(iri: IriLiteralV2) => Some(iri.value) - case other => throw InconsistentTriplestoreDataException(s"Expected root node Iri as an IriLiteralV2 for list node $nodeIri, but got $other") + case other => throw InconsistentRepositoryDataException(s"Expected root node Iri as an IriLiteralV2 for list node $nodeIri, but got $other") } case None => None @@ -446,7 +447,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde case Some(values: Seq[LiteralV2]) => values.headOption match { case Some(value: BooleanLiteralV2) => value.value - case Some(other) => throw InconsistentTriplestoreDataException(s"Expected isRootNode as an BooleanLiteralV2 for list node $nodeIri, but got $other") + case Some(other) => throw InconsistentRepositoryDataException(s"Expected isRootNode as an BooleanLiteralV2 for list node $nodeIri, but got $other") case None => false } @@ -458,7 +459,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde if (isRootNode) { ListRootNodeADM( id = nodeIri.toString, - projectIri = attachedToProjectOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required attachedToProject property missing for list node $nodeIri.")), + projectIri = attachedToProjectOption.getOrElse(throw InconsistentRepositoryDataException(s"Required attachedToProject property missing for list node $nodeIri.")), name = propsMap.get(OntologyConstants.KnoraBase.ListNodeName.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), labels = StringLiteralSequenceV2(labels.toVector.sortBy(_.language)), comments = StringLiteralSequenceV2(comments.toVector.sortBy(_.language)), @@ -470,8 +471,8 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde name = propsMap.get(OntologyConstants.KnoraBase.ListNodeName.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), labels = StringLiteralSequenceV2(labels.toVector.sortBy(_.language)), comments = StringLiteralSequenceV2(comments.toVector.sortBy(_.language)), - position = positionOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required position property missing for list node $nodeIri.")), - hasRootNode = hasRootNodeOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required hasRootNode property missing for list node $nodeIri.")), + position = positionOption.getOrElse(throw InconsistentRepositoryDataException(s"Required position property missing for list node $nodeIri.")), + hasRootNode = hasRootNodeOption.getOrElse(throw InconsistentRepositoryDataException(s"Required hasRootNode property missing for list node $nodeIri.")), children = children ) } @@ -514,7 +515,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde val propsMap: Map[SmartIri, Seq[LiteralV2]] = statements.filter(_._1 == IriSubjectV2(nodeIri)).head._2 - val hasRootNode: IRI = propsMap.getOrElse(OntologyConstants.KnoraBase.HasRootNode.toSmartIri, throw InconsistentTriplestoreDataException(s"Required hasRootNode property missing for list node $nodeIri.")).head.toString + val hasRootNode: IRI = propsMap.getOrElse(OntologyConstants.KnoraBase.HasRootNode.toSmartIri, throw InconsistentRepositoryDataException(s"Required hasRootNode property missing for list node $nodeIri.")).head.toString val nameOption = propsMap.get(OntologyConstants.KnoraBase.ListNodeName.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value) @@ -522,7 +523,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde val comments: Seq[StringLiteralV2] = propsMap.getOrElse(OntologyConstants.Rdfs.Comment.toSmartIri, Seq.empty[StringLiteralV2]).map(_.asInstanceOf[StringLiteralV2]) val positionOption: Option[Int] = propsMap.get(OntologyConstants.KnoraBase.ListNodePosition.toSmartIri).map(_.head.asInstanceOf[IntLiteralV2].value) - val position = positionOption.getOrElse(throw InconsistentTriplestoreDataException(s"Required position property missing for list node $nodeIri.")) + val position = positionOption.getOrElse(throw InconsistentRepositoryDataException(s"Required position property missing for list node $nodeIri.")) val children: Seq[ListChildNodeADM] = propsMap.get(OntologyConstants.KnoraBase.HasSubListNode.toSmartIri) match { case Some(iris: Seq[LiteralV2]) => @@ -641,7 +642,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde ).toString() } - nodePathResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(nodePathQuery)).mapTo[SparqlSelectResponse] + nodePathResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(nodePathQuery)).mapTo[SparqlSelectResult] /* @@ -1220,10 +1221,10 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde * Delete a list (root node) or a child node after verifying that neither the node itself nor any of its children * are used. If not used, delete the children of the node first, then delete the node itself. * - * @param nodeIri the node's IRI. - * @param projectIri the feature factory configuration. - * @param children the children of the node. - * @param isRootNode the flag to determine the type of the node, root or child. + * @param nodeIri the node's IRI. + * @param projectIri the feature factory configuration. + * @param children the children of the node. + * @param isRootNode the flag to determine the type of the node, root or child. * @return a [[IRI]] * @throws UpdateNotPerformedException in case a node is in use. */ @@ -1245,11 +1246,11 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde * Update the parent node of the deleted node by updating its remaining children. * Shift the remaining children of the parent node with respect to the position of the deleted node. * - * @param deletedNodeIri the IRI of the deleted node. - * @param positionOfDeletedNode the position of the deleted node. - * @param parentNodeIri the IRI of the deleted node's parent. - * @param dataNamedGraph the data named graph. - * @param featureFactoryConfig the feature factory configuration. + * @param deletedNodeIri the IRI of the deleted node. + * @param positionOfDeletedNode the position of the deleted node. + * @param parentNodeIri the IRI of the deleted node's parent. + * @param dataNamedGraph the data named graph. + * @param featureFactoryConfig the feature factory configuration. * @return a [[ListNodeADM]] * @throws UpdateNotPerformedException if the node that had to be deleted is still in the list of parent's children. */ @@ -1516,10 +1517,12 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde * @return a [[IRI]]. */ private def getProjectIriFromNode(nodeIri: IRI, featureFactoryConfig: FeatureFactoryConfig): Future[IRI] = for { - maybeNode <- listNodeGetADM(nodeIri = nodeIri, + maybeNode <- listNodeGetADM( + nodeIri = nodeIri, shallow = true, featureFactoryConfig = featureFactoryConfig, - requestingUser = KnoraSystemInstances.Users.SystemUser) + requestingUser = KnoraSystemInstances.Users.SystemUser + ) projectIri <- maybeNode match { case Some(rootNode: ListRootNodeADM) => Future(rootNode.projectIri) @@ -1551,9 +1554,10 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde errorFun: => Nothing): Future[Unit] = for { isNodeUsedSparql <- Future(org.knora.webapi.messages.twirl.queries.sparql.admin.txt.isNodeUsed( triplestore = settings.triplestoreType, - nodeIri = nodeIri).toString()) + nodeIri = nodeIri + ).toString()) - isNodeUsedResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(isNodeUsedSparql)).mapTo[SparqlSelectResponse] + isNodeUsedResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(isNodeUsedSparql)).mapTo[SparqlSelectResult] _ = if (isNodeUsedResponse.results.bindings.nonEmpty) { errorFun @@ -1565,7 +1569,7 @@ class ListsResponderADM(responderData: ResponderData) extends Responder(responde * * @param projectIri the IRI of the project. * @param featureFactoryConfig the feature factory configuration. - * @return a [[IRI]]. + * @return an [[IRI]]. */ protected def getDataNamedGraph(projectIri: IRI, featureFactoryConfig: FeatureFactoryConfig): Future[IRI] = for { /* Get the project information */ diff --git a/webapi/src/main/scala/org/knora/webapi/responders/admin/PermissionsResponderADM.scala b/webapi/src/main/scala/org/knora/webapi/responders/admin/PermissionsResponderADM.scala index bb802a1bdc..1adbdd3de1 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/admin/PermissionsResponderADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/admin/PermissionsResponderADM.scala @@ -38,6 +38,7 @@ import org.knora.webapi.responders.{IriLocker, Responder} import org.knora.webapi.responders.Responder.handleUnexpectedMessage import org.knora.webapi.util.cache.CacheUtil import org.knora.webapi.messages.IriConversions._ +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import scala.collection.immutable.Iterable import scala.collection.mutable.ListBuffer @@ -110,7 +111,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re requestingUser = KnoraSystemInstances.Users.SystemUser )).mapTo[Option[GroupADM]] - group = maybeGroup.getOrElse(throw InconsistentTriplestoreDataException(s"Cannot find information for group: '$groupIri'. Please report as possible bug.")) + group = maybeGroup.getOrElse(throw InconsistentRepositoryDataException(s"Cannot find information for group: '$groupIri'. Please report as possible bug.")) res = (group.project.id, groupIri) } yield res } @@ -348,7 +349,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"administrativePermissionsForProjectGetRequestADM - query: $sparqlQueryString") - permissionsQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionsQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"getProjectAdministrativePermissionsV1 - result: ${MessageUtil.toSource(permissionsQueryResponse)}") /* extract response rows */ @@ -366,7 +367,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re val hasPermissions: Set[PermissionADM] = PermissionUtilADM.parsePermissionsWithType(propsMap.get(OntologyConstants.KnoraBase.HasPermissions), PermissionType.AP) /* construct permission object */ - AdministrativePermissionADM(iri = permissionIri, forProject = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentTriplestoreDataException(s"Administrative Permission $permissionIri has no project attached.")), forGroup = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ForGroup, throw InconsistentTriplestoreDataException(s"Administrative Permission $permissionIri has no group attached.")), hasPermissions = hasPermissions) + AdministrativePermissionADM(iri = permissionIri, forProject = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentRepositoryDataException(s"Administrative Permission $permissionIri has no project attached.")), forGroup = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ForGroup, throw InconsistentRepositoryDataException(s"Administrative Permission $permissionIri has no group attached.")), hasPermissions = hasPermissions) }.toSeq /* construct response object */ @@ -394,7 +395,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"administrativePermissionForIriGetRequestV1 - query: $sparqlQueryString") - permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"getAdministrativePermissionForIriV1 - result: ${MessageUtil.toSource(permissionQueryResponse)}") /* extract response rows */ @@ -414,7 +415,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re //_ = log.debug(s"administrativePermissionForIriGetRequestV1 - hasPermissions: ${MessageUtil.toSource(hasPermissions)}") /* construct the permission object */ - permission = permissionsmessages.AdministrativePermissionADM(iri = administrativePermissionIri, forProject = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentTriplestoreDataException(s"Permission $administrativePermissionIri has no project attached")).head, forGroup = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForGroup, throw InconsistentTriplestoreDataException(s"Permission $administrativePermissionIri has no group attached")).head, hasPermissions = hasPermissions) + permission = permissionsmessages.AdministrativePermissionADM(iri = administrativePermissionIri, forProject = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentRepositoryDataException(s"Permission $administrativePermissionIri has no project attached")).head, forGroup = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForGroup, throw InconsistentRepositoryDataException(s"Permission $administrativePermissionIri has no group attached")).head, hasPermissions = hasPermissions) /* construct the response object */ response = AdministrativePermissionForIriGetResponseADM(permission) @@ -442,7 +443,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"administrativePermissionForProjectGroupGetADM - query: $sparqlQueryString") - permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"administrativePermissionForProjectGroupGetADM - result: ${MessageUtil.toSource(permissionQueryResponse)}") permissionQueryResponseRows: Seq[VariableResultsRow] = permissionQueryResponse.results.bindings @@ -451,7 +452,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re /* check if we only got one administrative permission back */ val apCount: Int = permissionQueryResponseRows.groupBy(_.rowMap("s")).size - if (apCount > 1) throw InconsistentTriplestoreDataException(s"Only one administrative permission instance allowed for project: $projectIri and group: $groupIri combination, but found $apCount.") + if (apCount > 1) throw InconsistentRepositoryDataException(s"Only one administrative permission instance allowed for project: $projectIri and group: $groupIri combination, but found $apCount.") /* get the iri of the retrieved permission */ val returnedPermissionIri = permissionQueryResponse.getFirstRow.rowMap("s") @@ -604,7 +605,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"objectAccessPermissionsForResourceGetV1 - query: $sparqlQueryString") - permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"objectAccessPermissionsForResourceGetV1 - result: ${MessageUtil.toSource(permissionQueryResponse)}") permissionQueryResponseRows: Seq[VariableResultsRow] = permissionQueryResponse.results.bindings @@ -644,7 +645,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"objectAccessPermissionsForValueGetV1 - query: $sparqlQueryString") - permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"objectAccessPermissionsForValueGetV1 - result: ${MessageUtil.toSource(permissionQueryResponse)}") permissionQueryResponseRows: Seq[VariableResultsRow] = permissionQueryResponse.results.bindings @@ -689,7 +690,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"defaultObjectAccessPermissionsForProjectGetRequestADM - query: $sparqlQueryString") - permissionsQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionsQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"defaultObjectAccessPermissionsForProjectGetRequestADM - result: ${MessageUtil.toSource(permissionsQueryResponse)}") /* extract response rows */ @@ -707,7 +708,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re val hasPermissions: Set[PermissionADM] = PermissionUtilADM.parsePermissionsWithType(propsMap.get(OntologyConstants.KnoraBase.HasPermissions), PermissionType.OAP) /* construct permission object */ - DefaultObjectAccessPermissionADM(iri = permissionIri, forProject = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentTriplestoreDataException(s"Permission $permissionIri has no project.")), forGroup = propsMap.get(OntologyConstants.KnoraAdmin.ForGroup), forResourceClass = propsMap.get(OntologyConstants.KnoraAdmin.ForResourceClass), forProperty = propsMap.get(OntologyConstants.KnoraAdmin.ForProperty), hasPermissions = hasPermissions) + DefaultObjectAccessPermissionADM(iri = permissionIri, forProject = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentRepositoryDataException(s"Permission $permissionIri has no project.")), forGroup = propsMap.get(OntologyConstants.KnoraAdmin.ForGroup), forResourceClass = propsMap.get(OntologyConstants.KnoraAdmin.ForResourceClass), forProperty = propsMap.get(OntologyConstants.KnoraAdmin.ForProperty), hasPermissions = hasPermissions) }.toSeq /* construct response object */ @@ -737,7 +738,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re ).toString()) //_ = log.debug(s"defaultObjectAccessPermissionForIriGetRequestADM - query: $sparqlQueryString") - permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"defaultObjectAccessPermissionForIriGetRequestADM - result: ${MessageUtil.toSource(permissionQueryResponse)}") permissionQueryResponseRows: Seq[VariableResultsRow] = permissionQueryResponse.results.bindings @@ -753,7 +754,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re hasPermissions = PermissionUtilADM.parsePermissionsWithType(groupedPermissionsQueryResponse.get(OntologyConstants.KnoraBase.HasPermissions).map(_.head), PermissionType.OAP) - defaultObjectAccessPermission = permissionsmessages.DefaultObjectAccessPermissionADM(iri = permissionIri, forProject = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentTriplestoreDataException(s"Permission $permissionIri has no project.")).head, forGroup = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForGroup).map(_.head), forResourceClass = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForResourceClass).map(_.head), forProperty = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForProperty).map(_.head), hasPermissions = hasPermissions) + defaultObjectAccessPermission = permissionsmessages.DefaultObjectAccessPermissionADM(iri = permissionIri, forProject = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentRepositoryDataException(s"Permission $permissionIri has no project.")).head, forGroup = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForGroup).map(_.head), forResourceClass = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForResourceClass).map(_.head), forProperty = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForProperty).map(_.head), hasPermissions = hasPermissions) result = DefaultObjectAccessPermissionForIriGetResponseADM(defaultObjectAccessPermission) @@ -803,7 +804,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re // _ = logger.debug(s"defaultObjectAccessPermissionGetADM - query: $sparqlQueryString") - permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + permissionQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] // _ = log.debug(s"defaultObjectAccessPermissionGetADM - result: ${MessageUtil.toSource(permissionQueryResponse)}") permissionQueryResponseRows: Seq[VariableResultsRow] = permissionQueryResponse.results.bindings @@ -812,7 +813,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re /* check if we only got one default object access permission back */ val doapCount: Int = permissionQueryResponseRows.groupBy(_.rowMap("s")).size - if (doapCount > 1) throw InconsistentTriplestoreDataException(s"Only one default object permission instance allowed for project: $projectIri and combination of group: $groupIri, resourceClass: $resourceClassIri, property: $propertyIri combination, but found: $doapCount.") + if (doapCount > 1) throw InconsistentRepositoryDataException(s"Only one default object permission instance allowed for project: $projectIri and combination of group: $groupIri, resourceClass: $resourceClassIri, property: $propertyIri combination, but found: $doapCount.") /* get the iri of the retrieved permission */ val permissionIri = permissionQueryResponse.getFirstRow.rowMap("s") @@ -821,7 +822,7 @@ class PermissionsResponderADM(responderData: ResponderData) extends Responder(re case (predicate, rows) => predicate -> rows.map(_.rowMap("o")) } val hasPermissions: Set[PermissionADM] = PermissionUtilADM.parsePermissionsWithType(groupedPermissionsQueryResponse.get(OntologyConstants.KnoraBase.HasPermissions).map(_.head), PermissionType.OAP) - val doap: DefaultObjectAccessPermissionADM = DefaultObjectAccessPermissionADM(iri = permissionIri, forProject = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentTriplestoreDataException(s"Permission has no project.")).head, forGroup = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForGroup).map(_.head), forResourceClass = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForResourceClass).map(_.head), forProperty = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForProperty).map(_.head), hasPermissions = hasPermissions) + val doap: DefaultObjectAccessPermissionADM = DefaultObjectAccessPermissionADM(iri = permissionIri, forProject = groupedPermissionsQueryResponse.getOrElse(OntologyConstants.KnoraAdmin.ForProject, throw InconsistentRepositoryDataException(s"Permission has no project.")).head, forGroup = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForGroup).map(_.head), forResourceClass = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForResourceClass).map(_.head), forProperty = groupedPermissionsQueryResponse.get(OntologyConstants.KnoraAdmin.ForProperty).map(_.head), hasPermissions = hasPermissions) // write permission to cache PermissionsMessagesUtilADM.writeDefaultObjectAccessPermissionADMToCache(doap) diff --git a/webapi/src/main/scala/org/knora/webapi/responders/admin/ProjectsResponderADM.scala b/webapi/src/main/scala/org/knora/webapi/responders/admin/ProjectsResponderADM.scala index 6685b629ca..f46ff63b40 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/admin/ProjectsResponderADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/admin/ProjectsResponderADM.scala @@ -25,8 +25,6 @@ import java.util.UUID import akka.http.scaladsl.util.FastFuture import akka.pattern._ -import org.eclipse.rdf4j.model.Statement -import org.eclipse.rdf4j.rio.{RDFFormat, RDFHandler, RDFWriter, Rio} import org.knora.webapi._ import org.knora.webapi.annotation.ApiMayChange import org.knora.webapi.exceptions._ @@ -41,10 +39,12 @@ import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.v1.responder.projectmessages._ import org.knora.webapi.messages.v2.responder.ontologymessages.{OntologyMetadataGetByProjectRequestV2, ReadOntologyMetadataV2} import org.knora.webapi.messages.{OntologyConstants, SmartIri, StringFormatter} +import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.responders.Responder.handleUnexpectedMessage import org.knora.webapi.responders.{IriLocker, Responder} import scala.concurrent.Future +import scala.util.{Failure, Success, Try} /** * Returns information about Knora projects. @@ -134,7 +134,7 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo } yield ontologyMetadataResponse.ontologies.map { ontology => val ontologyIri: IRI = ontology.ontologyIri.toString - val projectIri: IRI = ontology.projectIri.getOrElse(throw InconsistentTriplestoreDataException(s"Ontology $ontologyIri has no project")).toString + val projectIri: IRI = ontology.projectIri.getOrElse(throw InconsistentRepositoryDataException(s"Ontology $ontologyIri has no project")).toString projectIri -> ontologyIri }.groupBy(_._1).map { case (projectIri, projectIriAndOntologies: Set[(IRI, IRI)]) => @@ -457,32 +457,30 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo } /** - * An [[RDFHandler]] for combining several named graphs into one. + * An [[RdfStreamProcessor]] for combining several named graphs into one. * - * @param outputWriter an [[RDFWriter]] for writing the combined result. + * @param formattingStreamProcessor an [[RdfStreamProcessor]] for writing the combined result. */ - class CombiningRdfHandler(outputWriter: RDFWriter) extends RDFHandler { + class CombiningRdfProcessor(formattingStreamProcessor: RdfStreamProcessor) extends RdfStreamProcessor { private var startedStatements = false // Ignore this, since it will be done before the first file is written. - override def startRDF(): Unit = {} + override def start(): Unit = {} // Ignore this, since it will be done after the last file is written. - override def endRDF(): Unit = {} + override def finish(): Unit = {} - override def handleNamespace(prefix: IRI, uri: IRI): Unit = { + override def processNamespace(prefix: IRI, namespace: IRI): Unit = { // Only accept namespaces from the first graph, to prevent conflicts. if (!startedStatements) { - outputWriter.handleNamespace(prefix, uri) + formattingStreamProcessor.processNamespace(prefix, namespace) } } - override def handleStatement(st: Statement): Unit = { + override def processStatement(statement: Statement): Unit = { startedStatements = true - outputWriter.handleStatement(st) + formattingStreamProcessor.processStatement(statement) } - - override def handleComment(comment: IRI): Unit = outputWriter.handleComment(comment) } /** @@ -492,33 +490,51 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo * @param resultFile the output file. */ def combineGraphs(namedGraphTrigFiles: Seq[NamedGraphTrigFile], resultFile: File): Unit = { - // TODO: Provide a streaming API in RdfFormatUtil for this. + val rdfFormatUtil: RdfFormatUtil = RdfFeatureFactory.getRdfFormatUtil(featureFactoryConfig) + var maybeBufferedFileOutputStream: Option[BufferedOutputStream] = None + + val trigFileTry: Try[Unit] = Try { + maybeBufferedFileOutputStream = Some(new BufferedOutputStream(new FileOutputStream(resultFile))) - var maybeBufferedFileWriter: Option[BufferedWriter] = None + val formattingStreamProcessor: RdfStreamProcessor = rdfFormatUtil.makeFormattingStreamProcessor( + outputStream = maybeBufferedFileOutputStream.get, + rdfFormat = TriG + ) - try { - maybeBufferedFileWriter = Some(new BufferedWriter(new FileWriter(resultFile))) - val trigFileWriter: RDFWriter = Rio.createWriter(RDFFormat.TRIG, maybeBufferedFileWriter.get) - val combiningRdfHandler = new CombiningRdfHandler(trigFileWriter) - trigFileWriter.startRDF() + val combiningRdfProcessor = new CombiningRdfProcessor(formattingStreamProcessor) + formattingStreamProcessor.start() for (namedGraphTrigFile: NamedGraphTrigFile <- namedGraphTrigFiles) { - var maybeBufferedFileReader: Option[BufferedReader] = None + var maybeBufferedFileInputStream: Option[BufferedInputStream] = None + + val namedGraphTry: Try[Unit] = Try { + maybeBufferedFileInputStream = Some(new BufferedInputStream(new FileInputStream(namedGraphTrigFile.dataFile))) + + rdfFormatUtil.parseWithStreamProcessor( + rdfSource = RdfInputStreamSource(maybeBufferedFileInputStream.get), + rdfFormat = TriG, + rdfStreamProcessor = combiningRdfProcessor + ) - try { - maybeBufferedFileReader = Some(new BufferedReader(new FileReader(namedGraphTrigFile.dataFile))) - val trigFileParser = Rio.createParser(RDFFormat.TRIG) - trigFileParser.setRDFHandler(combiningRdfHandler) - trigFileParser.parse(maybeBufferedFileReader.get, "") namedGraphTrigFile.dataFile.delete - } finally { - maybeBufferedFileReader.foreach(_.close) + } + + maybeBufferedFileInputStream.foreach(_.close) + + namedGraphTry match { + case Success(_) => () + case Failure(ex) => throw ex } } - trigFileWriter.endRDF() - } finally { - maybeBufferedFileWriter.foreach(_.close) + formattingStreamProcessor.finish() + } + + maybeBufferedFileOutputStream.foreach(_.close) + + trigFileTry match { + case Success(_) => () + case Failure(ex) => throw ex } } @@ -554,7 +570,8 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo storeManager ? NamedGraphFileRequest( graphIri = trigFile.graphIri, - outputFile = trigFile.dataFile + outputFile = trigFile.dataFile, + featureFactoryConfig = featureFactoryConfig ) ).mapTo[FileWrittenResponse] } yield fileWrittenResponse @@ -574,7 +591,8 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo _: FileWrittenResponse <- (storeManager ? SparqlConstructFileRequest( sparql = adminDataSparql, graphIri = adminDataNamedGraphTrigFile.graphIri, - outputFile = adminDataNamedGraphTrigFile.dataFile + outputFile = adminDataNamedGraphTrigFile.dataFile, + featureFactoryConfig = featureFactoryConfig )).mapTo[FileWrittenResponse] // Download the project's permission data. @@ -589,7 +607,8 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo _: FileWrittenResponse <- (storeManager ? SparqlConstructFileRequest( sparql = permissionDataSparql, graphIri = permissionDataNamedGraphTrigFile.graphIri, - outputFile = permissionDataNamedGraphTrigFile.dataFile + outputFile = permissionDataNamedGraphTrigFile.dataFile, + featureFactoryConfig = featureFactoryConfig )).mapTo[FileWrittenResponse] // Stream the combined results into the output file. @@ -1041,15 +1060,15 @@ class ProjectsResponderADM(responderData: ResponderData) extends Responder(respo ProjectADM( id = projectIri, - shortname = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortname.toSmartIri, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no shortname defined.")).head.asInstanceOf[StringLiteralV2].value, - shortcode = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortcode.toSmartIri, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no shortcode defined.")).head.asInstanceOf[StringLiteralV2].value, + shortname = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortname.toSmartIri, throw InconsistentRepositoryDataException(s"Project: $projectIri has no shortname defined.")).head.asInstanceOf[StringLiteralV2].value, + shortcode = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortcode.toSmartIri, throw InconsistentRepositoryDataException(s"Project: $projectIri has no shortcode defined.")).head.asInstanceOf[StringLiteralV2].value, longname = propsMap.get(OntologyConstants.KnoraAdmin.ProjectLongname.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), - description = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectDescription.toSmartIri, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no description defined.")).map(_.asInstanceOf[StringLiteralV2]), + description = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectDescription.toSmartIri, throw InconsistentRepositoryDataException(s"Project: $projectIri has no description defined.")).map(_.asInstanceOf[StringLiteralV2]), keywords = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectKeyword.toSmartIri, Seq.empty[String]).map(_.asInstanceOf[StringLiteralV2].value).sorted, logo = propsMap.get(OntologyConstants.KnoraAdmin.ProjectLogo.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), ontologies = ontologies, - status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no status defined.")).head.asInstanceOf[BooleanLiteralV2].value, - selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled.toSmartIri, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no hasSelfJoinEnabled defined.")).head.asInstanceOf[BooleanLiteralV2].value + status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentRepositoryDataException(s"Project: $projectIri has no status defined.")).head.asInstanceOf[BooleanLiteralV2].value, + selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled.toSmartIri, throw InconsistentRepositoryDataException(s"Project: $projectIri has no hasSelfJoinEnabled defined.")).head.asInstanceOf[BooleanLiteralV2].value ) } diff --git a/webapi/src/main/scala/org/knora/webapi/responders/admin/SipiResponderADM.scala b/webapi/src/main/scala/org/knora/webapi/responders/admin/SipiResponderADM.scala index c5446f5b03..d29d98224b 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/admin/SipiResponderADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/admin/SipiResponderADM.scala @@ -22,7 +22,7 @@ package org.knora.webapi.responders.admin import akka.actor.Status import akka.http.scaladsl.util.FastFuture import akka.pattern._ -import org.knora.webapi.exceptions.{InconsistentTriplestoreDataException, NotFoundException} +import org.knora.webapi.exceptions.{InconsistentRepositoryDataException, NotFoundException} import org.knora.webapi.messages.SmartIri import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectIdentifierADM, ProjectRestrictedViewSettingsADM, ProjectRestrictedViewSettingsGetADM} import org.knora.webapi.messages.admin.responder.sipimessages.{SipiFileInfoGetRequestADM, SipiFileInfoGetResponseADM, SipiResponderRequestADM} @@ -75,11 +75,11 @@ class SipiResponderADM(responderData: ResponderData) extends Responder(responder )).mapTo[SparqlExtendedConstructResponse] _ = if (queryResponse.statements.isEmpty) throw NotFoundException(s"No file value was found for filename ${request.filename}") - _ = if (queryResponse.statements.size > 1) throw InconsistentTriplestoreDataException(s"Filename ${request.filename} is used in more than one file value") + _ = if (queryResponse.statements.size > 1) throw InconsistentRepositoryDataException(s"Filename ${request.filename} is used in more than one file value") fileValueIriSubject: IriSubjectV2 = queryResponse.statements.keys.head match { case iriSubject: IriSubjectV2 => iriSubject - case _ => throw InconsistentTriplestoreDataException(s"The subject of the file value with filename ${request.filename} is not an IRI") + case _ => throw InconsistentRepositoryDataException(s"The subject of the file value with filename ${request.filename} is not an IRI") } assertions: Seq[(String, String)] = queryResponse.statements(fileValueIriSubject).toSeq.flatMap { diff --git a/webapi/src/main/scala/org/knora/webapi/responders/admin/UsersResponderADM.scala b/webapi/src/main/scala/org/knora/webapi/responders/admin/UsersResponderADM.scala index 04c54a8b1a..b2c329632d 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/admin/UsersResponderADM.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/admin/UsersResponderADM.scala @@ -34,6 +34,7 @@ import org.knora.webapi.messages.admin.responder.usersmessages.UserInformationTy import org.knora.webapi.messages.admin.responder.usersmessages.{UserUpdatePayloadADM, _} import org.knora.webapi.messages.store.cacheservicemessages.{CacheServiceGetUserADM, CacheServicePutUserADM, CacheServiceRemoveValues} import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.v1.responder.usermessages._ import org.knora.webapi.messages.{OntologyConstants, SmartIri} @@ -120,12 +121,12 @@ class UsersResponderADM(responderData: ResponderData) extends Responder(responde UserADM( id = userIri.toString, - username = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Username.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'username' defined.")).head.asInstanceOf[StringLiteralV2].value, - email = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Email.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'email' defined.")).head.asInstanceOf[StringLiteralV2].value, - givenName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GivenName.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'givenName' defined.")).head.asInstanceOf[StringLiteralV2].value, - familyName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.FamilyName.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'familyName' defined.")).head.asInstanceOf[StringLiteralV2].value, - status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'status' defined.")).head.asInstanceOf[BooleanLiteralV2].value, - lang = propsMap.getOrElse(OntologyConstants.KnoraAdmin.PreferredLanguage.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'preferedLanguage' defined.")).head.asInstanceOf[StringLiteralV2].value) + username = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Username.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'username' defined.")).head.asInstanceOf[StringLiteralV2].value, + email = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Email.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'email' defined.")).head.asInstanceOf[StringLiteralV2].value, + givenName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GivenName.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'givenName' defined.")).head.asInstanceOf[StringLiteralV2].value, + familyName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.FamilyName.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'familyName' defined.")).head.asInstanceOf[StringLiteralV2].value, + status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'status' defined.")).head.asInstanceOf[BooleanLiteralV2].value, + lang = propsMap.getOrElse(OntologyConstants.KnoraAdmin.PreferredLanguage.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'preferedLanguage' defined.")).head.asInstanceOf[StringLiteralV2].value) } } yield users.sorted @@ -770,7 +771,7 @@ class UsersResponderADM(responderData: ResponderData) extends Responder(responde //_ = log.debug("userDataByIRIGetV1 - sparqlQueryString: {}", sparqlQueryString) - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] groupedUserData: Map[String, Seq[String]] = userDataQueryResponse.results.bindings.groupBy(_.rowMap("p")).map { case (predicate, rows) => predicate -> rows.map(_.rowMap("o")) @@ -1100,7 +1101,7 @@ class UsersResponderADM(responderData: ResponderData) extends Responder(responde featureFactoryConfig = featureFactoryConfig, requestingUser = KnoraSystemInstances.Users.SystemUser )).mapTo[Option[GroupADM]] - projectIri = maybeGroupADM.getOrElse(throw InconsistentTriplestoreDataException(s"Group $groupIri does not exist")).project.id + projectIri = maybeGroupADM.getOrElse(throw InconsistentRepositoryDataException(s"Group $groupIri does not exist")).project.id // check if the requesting user is allowed to perform updates _ = if (!requestingUser.permissions.isProjectAdmin(projectIri) && !requestingUser.permissions.isSystemAdmin) { @@ -1181,7 +1182,7 @@ class UsersResponderADM(responderData: ResponderData) extends Responder(responde requestingUser = KnoraSystemInstances.Users.SystemUser )).mapTo[Option[GroupADM]] - projectIri = maybeGroupADM.getOrElse(throw exceptions.InconsistentTriplestoreDataException(s"Group $groupIri does not exist")).project.id + projectIri = maybeGroupADM.getOrElse(throw exceptions.InconsistentRepositoryDataException(s"Group $groupIri does not exist")).project.id // check if the requesting user is allowed to perform updates _ = if (!requestingUser.permissions.isProjectAdmin(projectIri) && !requestingUser.permissions.isSystemAdmin && !requestingUser.isSystemUser) { @@ -1597,14 +1598,14 @@ class UsersResponderADM(responderData: ResponderData) extends Responder(responde /* construct the user profile from the different parts */ user = UserADM( id = userIri, - username = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Username.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'username' defined.")).head.asInstanceOf[StringLiteralV2].value, - email = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Email.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'email' defined.")).head.asInstanceOf[StringLiteralV2].value, + username = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Username.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'username' defined.")).head.asInstanceOf[StringLiteralV2].value, + email = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Email.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'email' defined.")).head.asInstanceOf[StringLiteralV2].value, password = propsMap.get(OntologyConstants.KnoraAdmin.Password.toSmartIri).map(_.head.asInstanceOf[StringLiteralV2].value), token = None, - givenName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GivenName.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'givenName' defined.")).head.asInstanceOf[StringLiteralV2].value, - familyName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.FamilyName.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'familyName' defined.")).head.asInstanceOf[StringLiteralV2].value, - status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'status' defined.")).head.asInstanceOf[BooleanLiteralV2].value, - lang = propsMap.getOrElse(OntologyConstants.KnoraAdmin.PreferredLanguage.toSmartIri, throw InconsistentTriplestoreDataException(s"User: $userIri has no 'preferredLanguage' defined.")).head.asInstanceOf[StringLiteralV2].value, groups = groups, projects = projects, + givenName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.GivenName.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'givenName' defined.")).head.asInstanceOf[StringLiteralV2].value, + familyName = propsMap.getOrElse(OntologyConstants.KnoraAdmin.FamilyName.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'familyName' defined.")).head.asInstanceOf[StringLiteralV2].value, + status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'status' defined.")).head.asInstanceOf[BooleanLiteralV2].value, + lang = propsMap.getOrElse(OntologyConstants.KnoraAdmin.PreferredLanguage.toSmartIri, throw InconsistentRepositoryDataException(s"User: $userIri has no 'preferredLanguage' defined.")).head.asInstanceOf[StringLiteralV2].value, groups = groups, projects = projects, sessionId = None, permissions = permissionData ) diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/CkanResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/CkanResponderV1.scala index dbabc7251b..312b492a89 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/CkanResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/CkanResponderV1.scala @@ -28,7 +28,8 @@ import org.knora.webapi.IRI import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectRequest, SparqlSelectResponse, VariableResultsRow} +import org.knora.webapi.messages.store.triplestoremessages.SparqlSelectRequest +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.v1.responder.ckanmessages._ import org.knora.webapi.messages.v1.responder.listmessages.{NodePathGetRequestV1, NodePathGetResponseV1} @@ -211,7 +212,7 @@ class CkanResponderV1(responderData: ResponderData) extends Responder(responderD for { sparqlQuery <- Future(org.knora.webapi.messages.twirl.queries.sparql.v1.txt.ckanDokubib(settings.triplestoreType, projectIri, limit).toString()) - response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] responseRows: Seq[VariableResultsRow] = response.results.bindings bilder: Seq[String] = responseRows.groupBy(_.rowMap("bild")).keys.toVector @@ -331,7 +332,7 @@ class CkanResponderV1(responderData: ResponderData) extends Responder(responderD for { sparqlQuery <- Future(org.knora.webapi.messages.twirl.queries.sparql.v1.txt.ckanIncunabula(settings.triplestoreType, projectIri, limit).toString()) - response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] responseRows: Seq[VariableResultsRow] = response.results.bindings booksWithPages: Map[String, Seq[String]] = responseRows.groupBy(_.rowMap("book")).map { @@ -397,7 +398,7 @@ class CkanResponderV1(responderData: ResponderData) extends Responder(responderD projectIri = projectIri, resType = resType ).toString()) - resourcesResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + resourcesResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] resourcesResponseRows: Seq[VariableResultsRow] = resourcesResponse.results.bindings resIri = resourcesResponseRows.groupBy(_.rowMap("s")).keys.toVector result = limit match { diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ListsResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ListsResponderV1.scala index db9e668069..244699fcb3 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ListsResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ListsResponderV1.scala @@ -22,8 +22,9 @@ package org.knora.webapi.responders.v1 import akka.pattern._ import org.knora.webapi._ import org.knora.webapi.exceptions.NotFoundException -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectRequest, SparqlSelectResponse, VariableResultsRow} +import org.knora.webapi.messages.store.triplestoremessages.SparqlSelectRequest import org.knora.webapi.messages.util.ResponderData +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.v1.responder.listmessages._ import org.knora.webapi.messages.v1.responder.usermessages.UserProfileV1 import org.knora.webapi.responders.Responder @@ -155,7 +156,7 @@ class ListsResponderV1(responderData: ResponderData) extends Responder(responder fallbackLanguage = settings.fallbackLanguage ).toString() } - listQueryResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(listQuery)).mapTo[SparqlSelectResponse] + listQueryResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(listQuery)).mapTo[SparqlSelectResult] // Group the results to map each node to the SPARQL query results representing its children. groupedByNodeIri: Map[IRI, Seq[VariableResultsRow]] = listQueryResponse.results.bindings.groupBy(row => row.rowMap("node")) @@ -227,7 +228,7 @@ class ListsResponderV1(responderData: ResponderData) extends Responder(responder fallbackLanguage = settings.fallbackLanguage ).toString() } - nodePathResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(nodePathQuery)).mapTo[SparqlSelectResponse] + nodePathResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(nodePathQuery)).mapTo[SparqlSelectResult] /* diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/OntologyResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/OntologyResponderV1.scala index 6cfb7fcbc7..bd28249751 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/OntologyResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/OntologyResponderV1.scala @@ -21,7 +21,7 @@ package org.knora.webapi.responders.v1 import akka.pattern._ import org.knora.webapi._ -import org.knora.webapi.exceptions.{InconsistentTriplestoreDataException, NotFoundException} +import org.knora.webapi.exceptions.{InconsistentRepositoryDataException, NotFoundException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.OntologyConstants @@ -183,7 +183,7 @@ class OntologyResponderV1(responderData: ResponderData) extends Responder(respon vocabulary = entityInfo.ontologyIri, occurrence = cardinalityInfo.cardinality.toString, valuetype_id = OntologyConstants.KnoraBase.LinkValue, - attributes = valueUtilV1.makeAttributeString(entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))), + attributes = valueUtilV1.makeAttributeString(entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))), gui_name = entityInfo.getPredicateObject(OntologyConstants.SalsahGui.GuiElementProp).map(iri => SalsahGuiConversions.iri2SalsahGuiElement(iri)), guiorder = cardinalityInfo.guiOrder ) @@ -197,14 +197,14 @@ class OntologyResponderV1(responderData: ResponderData) extends Responder(respon description = entityInfo.getPredicateObject(predicateIri = OntologyConstants.Rdfs.Comment, preferredLangs = Some(userProfile.lang, settings.fallbackLanguage)), vocabulary = entityInfo.ontologyIri, occurrence = cardinalityInfo.cardinality.toString, - valuetype_id = entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")), + valuetype_id = entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")), attributes = valueUtilV1.makeAttributeString(entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute)), gui_name = entityInfo.getPredicateObject(OntologyConstants.SalsahGui.GuiElementProp).map(iri => SalsahGuiConversions.iri2SalsahGuiElement(iri)), guiorder = cardinalityInfo.guiOrder ) } case None => - throw new InconsistentTriplestoreDataException(s"Resource type $resourceTypeIri is defined as having property $propertyIri, which doesn't exist") + throw new InconsistentRepositoryDataException(s"Resource type $resourceTypeIri is defined as having property $propertyIri, which doesn't exist") } }.toVector.sortBy(_.guiorder) @@ -291,8 +291,8 @@ class OntologyResponderV1(responderData: ResponderData) extends Responder(respon NamedGraphV1( id = ontologyMetadata.ontologyIri.toString, shortname = project.shortname, - longname = project.longname.getOrElse(throw InconsistentTriplestoreDataException(s"Project ${project.id} has no longname")), - description = project.description.headOption.getOrElse(throw InconsistentTriplestoreDataException(s"Project ${project.id} has no description")).toString, + longname = project.longname.getOrElse(throw InconsistentRepositoryDataException(s"Project ${project.id} has no longname")), + description = project.description.headOption.getOrElse(throw InconsistentRepositoryDataException(s"Project ${project.id} has no description")).toString, project_id = project.id, uri = ontologyMetadata.ontologyIri.toString, active = project.status @@ -362,13 +362,13 @@ class OntologyResponderV1(responderData: ResponderData) extends Responder(respon prop => PropertyTypeV1( id = prop.id, - label = prop.label.getOrElse(throw InconsistentTriplestoreDataException(s"No label given for ${prop.id}")) + label = prop.label.getOrElse(throw InconsistentRepositoryDataException(s"No label given for ${prop.id}")) ) }.toVector ResourceTypeV1( id = resClassIri, - label = resInfo.restype_info.label.getOrElse(throw InconsistentTriplestoreDataException(s"No label given for $resClassIri")), + label = resInfo.restype_info.label.getOrElse(throw InconsistentRepositoryDataException(s"No label given for $resClassIri")), properties = properties ) }.toVector @@ -434,7 +434,7 @@ class OntologyResponderV1(responderData: ResponderData) extends Responder(respon description = entityInfo.getPredicateObject(predicateIri = OntologyConstants.Rdfs.Comment, preferredLangs = Some(userProfile.lang, settings.fallbackLanguage)), vocabulary = entityInfo.ontologyIri, valuetype_id = OntologyConstants.KnoraBase.LinkValue, - attributes = valueUtilV1.makeAttributeString(entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))), + attributes = valueUtilV1.makeAttributeString(entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))), gui_name = entityInfo.getPredicateObject(OntologyConstants.SalsahGui.GuiElementProp).map(iri => SalsahGuiConversions.iri2SalsahGuiElement(iri)) ) @@ -445,7 +445,7 @@ class OntologyResponderV1(responderData: ResponderData) extends Responder(respon label = entityInfo.getPredicateObject(predicateIri = OntologyConstants.Rdfs.Label, preferredLangs = Some(userProfile.lang, settings.fallbackLanguage)), description = entityInfo.getPredicateObject(predicateIri = OntologyConstants.Rdfs.Comment, preferredLangs = Some(userProfile.lang, settings.fallbackLanguage)), vocabulary = entityInfo.ontologyIri, - valuetype_id = entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")), + valuetype_id = entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")), attributes = valueUtilV1.makeAttributeString(entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute)), gui_name = entityInfo.getPredicateObject(OntologyConstants.SalsahGui.GuiElementProp).map(iri => SalsahGuiConversions.iri2SalsahGuiElement(iri)) ) diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ProjectsResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ProjectsResponderV1.scala index cae0aedd18..ffd0445a66 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ProjectsResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ProjectsResponderV1.scala @@ -22,11 +22,12 @@ package org.knora.webapi.responders.v1 import akka.http.scaladsl.util.FastFuture import akka.pattern._ import org.knora.webapi._ -import org.knora.webapi.exceptions.{InconsistentTriplestoreDataException, NotFoundException} +import org.knora.webapi.exceptions.{InconsistentRepositoryDataException, NotFoundException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants import org.knora.webapi.messages.admin.responder.usersmessages.{UserADM, UserGetRequestADM, UserIdentifierADM, UserResponseADM} import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.v1.responder.ontologymessages.{NamedGraphV1, NamedGraphsGetRequestV1, NamedGraphsResponseV1} import org.knora.webapi.messages.v1.responder.projectmessages._ @@ -102,7 +103,7 @@ class ProjectsResponderV1(responderData: ResponderData) extends Responder(respon ).toString()) //_ = log.debug(s"getProjectsResponseV1 - query: $sparqlQueryString") - projectsResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + projectsResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"getProjectsResponseV1 - result: $projectsResponse") projectsResponseRows: Seq[VariableResultsRow] = projectsResponse.results.bindings @@ -134,16 +135,16 @@ class ProjectsResponderV1(responderData: ResponderData) extends Responder(respon ProjectInfoV1( id = projectIri, - shortname = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortname, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no shortname defined.")).head, - shortcode = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortcode, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no shortcode defined.")).head, + shortname = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortname, throw InconsistentRepositoryDataException(s"Project: $projectIri has no shortname defined.")).head, + shortcode = propsMap.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortcode, throw InconsistentRepositoryDataException(s"Project: $projectIri has no shortcode defined.")).head, longname = propsMap.get(OntologyConstants.KnoraAdmin.ProjectLongname).map(_.head), description = propsMap.get(OntologyConstants.KnoraAdmin.ProjectDescription).map(_.head), keywords = maybeKeywords, logo = propsMap.get(OntologyConstants.KnoraAdmin.ProjectLogo).map(_.head), institution = propsMap.get(OntologyConstants.KnoraAdmin.BelongsToInstitution).map(_.head), ontologies = ontologies, - status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no status defined.")).head.toBoolean, - selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no hasSelfJoinEnabled defined.")).head.toBoolean + status = propsMap.getOrElse(OntologyConstants.KnoraAdmin.Status, throw InconsistentRepositoryDataException(s"Project: $projectIri has no status defined.")).head.toBoolean, + selfjoin = propsMap.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled, throw InconsistentRepositoryDataException(s"Project: $projectIri has no hasSelfJoinEnabled defined.")).head.toBoolean ) }.toSeq @@ -249,7 +250,7 @@ class ProjectsResponderV1(responderData: ResponderData) extends Responder(respon projectIri = projectIri ).toString()) - projectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + projectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] ontologiesForProjects: Map[IRI, Seq[IRI]] <- getOntologiesForProjects( projectIris = Set(projectIri), @@ -297,7 +298,7 @@ class ProjectsResponderV1(responderData: ResponderData) extends Responder(respon ).toString()) //_ = log.debug(s"getProjectInfoByShortnameGetRequest - query: $sparqlQueryString") - projectResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + projectResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(s"getProjectInfoByShortnameGetRequest - result: $projectResponse") @@ -363,16 +364,16 @@ class ProjectsResponderV1(responderData: ResponderData) extends Responder(respon /* create and return the project info */ ProjectInfoV1( id = projectIri, - shortname = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortname, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no shortname defined.")).head, - shortcode = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortcode, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no shortcode defined.")).head, + shortname = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortname, throw InconsistentRepositoryDataException(s"Project: $projectIri has no shortname defined.")).head, + shortcode = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.ProjectShortcode, throw InconsistentRepositoryDataException(s"Project: $projectIri has no shortcode defined.")).head, longname = projectProperties.get(OntologyConstants.KnoraAdmin.ProjectLongname).map(_.head), description = projectProperties.get(OntologyConstants.KnoraAdmin.ProjectDescription).map(_.head), keywords = maybeKeywords, logo = projectProperties.get(OntologyConstants.KnoraAdmin.ProjectLogo).map(_.head), institution = projectProperties.get(OntologyConstants.KnoraAdmin.BelongsToInstitution).map(_.head), ontologies = ontologies, - status = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.Status, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no status defined.")).head.toBoolean, - selfjoin = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled, throw InconsistentTriplestoreDataException(s"Project: $projectIri has no hasSelfJoinEnabled defined.")).head.toBoolean + status = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.Status, throw InconsistentRepositoryDataException(s"Project: $projectIri has no status defined.")).head.toBoolean, + selfjoin = projectProperties.getOrElse(OntologyConstants.KnoraAdmin.HasSelfJoinEnabled, throw InconsistentRepositoryDataException(s"Project: $projectIri has no hasSelfJoinEnabled defined.")).head.toBoolean ) } else { diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala index 37f4433b3e..2080dd6fe1 100755 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala @@ -35,6 +35,7 @@ import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.twirl.SparqlTemplateResourceToCreate import org.knora.webapi.messages.util.GroupedProps._ import org.knora.webapi.messages.util._ +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.v1.responder.ontologymessages._ import org.knora.webapi.messages.v1.responder.projectmessages._ import org.knora.webapi.messages.v1.responder.resourcemessages._ @@ -172,7 +173,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // _ = println(sparql) - response: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResponse] + response: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResult] rows = response.results.bindings // Did we get any results? @@ -306,7 +307,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // _ = println(sparql) - response: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResponse] + response: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResult] rows = response.results.bindings _ = if (rows.isEmpty) { @@ -368,7 +369,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo val resourceClassLabel = entityInfoResponse.resourceClassInfoMap(node.nodeClass).getPredicateObject( predicateIri = OntologyConstants.Rdfs.Label, preferredLangs = Some(graphDataGetRequest.userADM.lang, settings.fallbackLanguage) - ).getOrElse(throw InconsistentTriplestoreDataException(s"Resource class ${node.nodeClass} has no rdfs:label")) + ).getOrElse(throw InconsistentRepositoryDataException(s"Resource class ${node.nodeClass} has no rdfs:label")) GraphNodeV1( resourceIri = node.nodeIri, @@ -385,7 +386,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo val propertyLabel = entityInfoResponse.propertyInfoMap(edge.linkProp).getPredicateObject( predicateIri = OntologyConstants.Rdfs.Label, preferredLangs = Some(graphDataGetRequest.userADM.lang, settings.fallbackLanguage) - ).getOrElse(throw InconsistentTriplestoreDataException(s"Property ${edge.linkProp} has no rdfs:label")) + ).getOrElse(throw InconsistentRepositoryDataException(s"Property ${edge.linkProp} has no rdfs:label")) GraphEdgeV1( source = edge.sourceNodeIri, @@ -457,13 +458,13 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo ) // Get information about the references pointing from other resources to this resource. - val maybeIncomingRefsFuture: Future[Option[SparqlSelectResponse]] = if (getIncoming) { + val maybeIncomingRefsFuture: Future[Option[SparqlSelectResult]] = if (getIncoming) { for { incomingRefsSparql <- Future(org.knora.webapi.messages.twirl.queries.sparql.v1.txt.getIncomingReferences( triplestore = settings.triplestoreType, resourceIri = resourceIri ).toString()) - response <- (storeManager ? SparqlSelectRequest(incomingRefsSparql)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(incomingRefsSparql)).mapTo[SparqlSelectResult] } yield Some(response) } else { Future(None) @@ -483,7 +484,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo val resType = objMap.literalData.get(OntologyConstants.Rdf.Type) match { case Some(value: ValueLiterals) => value.literals - case None => throw InconsistentTriplestoreDataException(s"$obj has no rdf:type") + case None => throw InconsistentRepositoryDataException(s"$obj has no rdf:type") } resTypeAcc ++ resType @@ -495,7 +496,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // Group incoming reference rows by the IRI of the referring resource, and construct an IncomingV1 for each one. - maybeIncomingRefsResponse: Option[SparqlSelectResponse] <- maybeIncomingRefsFuture + maybeIncomingRefsResponse: Option[SparqlSelectResult] <- maybeIncomingRefsFuture incomingRefFutures: Vector[Future[Vector[IncomingV1]]] = maybeIncomingRefsResponse match { case Some(incomingRefsResponse) => @@ -508,7 +509,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo case (incomingIri: IRI, rows: Seq[VariableResultsRow]) => // Make a resource info for each referring resource, and check the permissions on the referring resource. - val rowsForResInfo = rows.filterNot(row => stringFormatter.optionStringToBoolean(row.rowMap.get("isLinkValue"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isLinkValue: ${row.rowMap.get("isLinkValue")}"))) + val rowsForResInfo = rows.filterNot(row => stringFormatter.optionStringToBoolean(row.rowMap.get("isLinkValue"), throw InconsistentRepositoryDataException(s"Invalid boolean for isLinkValue: ${row.rowMap.get("isLinkValue")}"))) for { (incomingResPermission, incomingResInfo) <- makeResourceInfoV1( @@ -525,7 +526,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // Yes. For each link from the referring resource, check whether the user has permission to see the link. If so, make an IncomingV1 for the link. // Filter to get only the rows representing LinkValues. - val rowsWithLinkValues = rows.filter(row => stringFormatter.optionStringToBoolean(row.rowMap.get("isLinkValue"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isLinkValue: ${row.rowMap.get("isLinkValue")}"))) + val rowsWithLinkValues = rows.filter(row => stringFormatter.optionStringToBoolean(row.rowMap.get("isLinkValue"), throw InconsistentRepositoryDataException(s"Invalid boolean for isLinkValue: ${row.rowMap.get("isLinkValue")}"))) // Group them by LinkValue IRI. val groupedByLinkValue: Map[String, Seq[VariableResultsRow]] = rowsWithLinkValues.groupBy(_.rowMap("obj")) @@ -549,7 +550,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo linkValueV1: LinkValueV1 = apiValueV1 match { case linkValueV1: LinkValueV1 => linkValueV1 - case _ => throw InconsistentTriplestoreDataException(s"Expected $linkValueIri to be a knora-base:LinkValue, but its type is ${apiValueV1.valueTypeIri}") + case _ => throw InconsistentRepositoryDataException(s"Expected $linkValueIri to be a knora-base:LinkValue, but its type is ${apiValueV1.valueTypeIri}") } // Check the permissions on the LinkValue. @@ -698,7 +699,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo guielement = propertyEntityInfo.getPredicateObject(OntologyConstants.SalsahGui.GuiElementProp).map(guiElementIri => SalsahGuiConversions.iri2SalsahGuiElement(guiElementIri)), label = propertyEntityInfo.getPredicateObject(predicateIri = OntologyConstants.Rdfs.Label, preferredLangs = Some(userProfile.lang, settings.fallbackLanguage)), occurrence = Some(propsAndCardinalities(propertyIri).cardinality.toString), - attributes = (propertyEntityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(propertyEntityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))).mkString(";"), + attributes = (propertyEntityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(propertyEntityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))).mkString(";"), value_rights = Nil ) @@ -896,7 +897,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo triplestore = settings.triplestoreType, resourceIri = resourceIri ).toString() - isPartOfResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(isPartOfSparqlQuery)).mapTo[SparqlSelectResponse] + isPartOfResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(isPartOfSparqlQuery)).mapTo[SparqlSelectResult] (containingResourceIriOption: Option[IRI], containingResInfoV1Option: Option[ResourceInfoV1]) <- isPartOfResponse.results.bindings match { case rows if rows.nonEmpty => @@ -943,14 +944,14 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // _ = println(contextSparqlQuery) - contextQueryResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(contextSparqlQuery)).mapTo[SparqlSelectResponse] + contextQueryResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(contextSparqlQuery)).mapTo[SparqlSelectResult] rows: Seq[VariableResultsRow] = contextQueryResponse.results.bindings // The results consist of one row per source object. sourceObjects: Seq[SourceObject] = rows.map { row: VariableResultsRow => val sourceObject: IRI = row.rowMap("sourceObject") - val projectShortcode: String = sourceObject.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid resource IRI: $sourceObject")) + val projectShortcode: String = sourceObject.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid resource IRI: $sourceObject")) createSourceObjectFromResultRow(projectShortcode, row) } @@ -992,7 +993,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo triplestore = settings.triplestoreType, resourceIri = resourceIri ).toString()) - regionQueryResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(regionSparqlQuery)).mapTo[SparqlSelectResponse] + regionQueryResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(regionSparqlQuery)).mapTo[SparqlSelectResult] regionRows = regionQueryResponse.results.bindings regionPropertiesSequencedFutures: Seq[Future[PropsGetForRegionV1]] = regionRows.filter { @@ -1038,7 +1039,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resClassIcon: Option[String] = regionInfo.predicates.get(OntologyConstants.KnoraBase.ResourceIcon) match { case Some(predicateInfo: PredicateInfoV1) => - Some(valueUtilV1.makeResourceClassIconURL(resClass, predicateInfo.objects.headOption.getOrElse(throw InconsistentTriplestoreDataException(s"resourceClass $resClass has no value for ${OntologyConstants.KnoraBase.ResourceIcon}")))) + Some(valueUtilV1.makeResourceClassIconURL(resClass, predicateInfo.objects.headOption.getOrElse(throw InconsistentRepositoryDataException(s"resourceClass $resClass has no value for ${OntologyConstants.KnoraBase.ResourceIcon}")))) case None => None } @@ -1153,7 +1154,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // _ = println(searchResourcesSparql) - searchResponse <- (storeManager ? SparqlSelectRequest(searchResourcesSparql)).mapTo[SparqlSelectResponse] + searchResponse <- (storeManager ? SparqlSelectRequest(searchResourcesSparql)).mapTo[SparqlSelectResult] resultFutures: Seq[Future[ResourceSearchResultRowV1]] = searchResponse.results.bindings.map { row: VariableResultsRow => @@ -1323,7 +1324,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo readOntologyMetadataV2: ReadOntologyMetadataV2 <- (responderManager ? OntologyMetadataGetByIriRequestV2(resourceClassOntologyIris, requestingUser)).mapTo[ReadOntologyMetadataV2] _ = for (ontologyMetadata <- readOntologyMetadataV2.ontologies) { - val ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentTriplestoreDataException(s"Ontology ${ontologyMetadata.ontologyIri} has no project")).toString + val ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentRepositoryDataException(s"Ontology ${ontologyMetadata.ontologyIri} has no project")).toString if (resourceProjectIri != ontologyProjectIri && !(ontologyMetadata.ontologyIri.isKnoraBuiltInDefinitionIri || ontologyMetadata.ontologyIri.isKnoraSharedDefinitionIri)) { throw BadRequestException(s"Cannot create a resource in project $resourceProjectIri with a resource class from ontology ${ontologyMetadata.ontologyIri}, which belongs to another project and is not shared") @@ -1580,7 +1581,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo case (acc, (propertyIri, valuesWithComments)) => val propertyInfo = propertyInfoMap.getOrElse(propertyIri, throw NotFoundException(s"Property not found: $propertyIri")) val propertyObjectClassConstraint = propertyInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse { - throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") + throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") } acc ++ valuesWithComments.map { @@ -1771,7 +1772,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceIri = resourceIri ).toString()) - createdResourceResponse <- (storeManager ? SparqlSelectRequest(createdResourcesSparql)).mapTo[SparqlSelectResponse] + createdResourceResponse <- (storeManager ? SparqlSelectRequest(createdResourcesSparql)).mapTo[SparqlSelectResult] _ = if (createdResourceResponse.results.bindings.isEmpty) { log.error(s"Attempted a SPARQL update to create a new resource, but it inserted no rows:\n\n$createNewResourceSparql") @@ -2011,7 +2012,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceClassOntologyIri: SmartIri = resourceClassIri.toSmartIri.getOntologyFromEntity readOntologyMetadataV2: ReadOntologyMetadataV2 <- (responderManager ? OntologyMetadataGetByIriRequestV2(Set(resourceClassOntologyIri), userProfile)).mapTo[ReadOntologyMetadataV2] ontologyMetadata: OntologyMetadataV2 = readOntologyMetadataV2.ontologies.headOption.getOrElse(throw BadRequestException(s"Ontology $resourceClassOntologyIri not found")) - ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentTriplestoreDataException(s"Ontology $resourceClassOntologyIri has no project")).toString + ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentRepositoryDataException(s"Ontology $resourceClassOntologyIri has no project")).toString _ = if (resourceProjectIri != ontologyProjectIri && !(ontologyMetadata.ontologyIri.isKnoraBuiltInDefinitionIri || ontologyMetadata.ontologyIri.isKnoraSharedDefinitionIri)) { throw BadRequestException(s"Cannot create a resource in project $resourceProjectIri with resource class $resourceClassIri, which is defined in a non-shared ontology in another project") @@ -2097,10 +2098,10 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo triplestore = settings.triplestoreType, resourceIri = resourceDeleteRequest.resourceIri ).toString() - sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = sparqlSelectResponse.results.bindings - _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { + _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentRepositoryDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { throw UpdateNotPerformedException(s"Resource ${resourceDeleteRequest.resourceIri} was not marked as deleted. Please report this as a possible bug.") } } yield ResourceDeleteResponseV1(id = resourceDeleteRequest.resourceIri) @@ -2229,7 +2230,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo label = label ).toString() - sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = sparqlSelectResponse.results.bindings // we expect exactly one row to be returned if the label was updated correctly in the data. @@ -2283,7 +2284,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo triplestore = settings.triplestoreType, resourceIri = resourceIri ).toString()) - resInfoResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + resInfoResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] resInfoResponseRows = resInfoResponse.results.bindings resInfo: (Option[Int], ResourceInfoV1) <- makeResourceInfoV1( @@ -2317,8 +2318,8 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceIri = resourceIri ).toString()) - resclassQueryResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(resclassSparqlQuery)).mapTo[SparqlSelectResponse] - resclass = resclassQueryResponse.results.bindings.headOption.getOrElse(throw InconsistentTriplestoreDataException(s"No resource class given for $resourceIri")) + resclassQueryResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(resclassSparqlQuery)).mapTo[SparqlSelectResult] + resclass = resclassQueryResponse.results.bindings.headOption.getOrElse(throw InconsistentRepositoryDataException(s"No resource class given for $resourceIri")) properties: Seq[PropertyV1] <- getResourceProperties( resourceIri = resourceIri, @@ -2379,7 +2380,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo Future((Map.empty[IRI, PropertyInfoV1], Map.empty[IRI, ClassInfoV1], Map.empty[IRI, KnoraCardinalityInfo])) } - projectShortcode = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid resource IRI: $resourceIri")) + projectShortcode = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid resource IRI: $resourceIri")) queryResult <- queryResults2PropertyV1s( containingResourceIri = resourceIri, @@ -2426,11 +2427,11 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo case (subject, predicate) => subject == OntologyConstants.KnoraBase.AttachedToProject } - resourceProject = maybeResourceProjectStatement.getOrElse(throw InconsistentTriplestoreDataException(s"Resource $resourceIri has no knora-base:attachedToProject"))._2 - projectShortcode: String = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid resource IRI $resourceIri")) + resourceProject = maybeResourceProjectStatement.getOrElse(throw InconsistentRepositoryDataException(s"Resource $resourceIri has no knora-base:attachedToProject"))._2 + projectShortcode: String = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid resource IRI $resourceIri")) // Get the rows describing file values from the query results, grouped by file value IRI. - fileValueGroupedRows: Seq[(IRI, Seq[VariableResultsRow])] = resInfoResponseRows.filter(row => stringFormatter.optionStringToBoolean(row.rowMap.get("isFileValue"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isFileValue: ${row.rowMap.get("isFileValue")}"))).groupBy(row => row.rowMap("obj")).toVector + fileValueGroupedRows: Seq[(IRI, Seq[VariableResultsRow])] = resInfoResponseRows.filter(row => stringFormatter.optionStringToBoolean(row.rowMap.get("isFileValue"), throw InconsistentRepositoryDataException(s"Invalid boolean for isFileValue: ${row.rowMap.get("isFileValue")}"))).groupBy(row => row.rowMap("obj")).toVector // Convert the file value rows to ValueProps objects, and filter out the ones that the user doesn't have permission to see. valuePropsForFileValues: Seq[(IRI, ValueProps)] = fileValueGroupedRows.map { @@ -2460,7 +2461,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo } yield valueV1 match { case fileValueV1: FileValueV1 => fileValueV1 - case otherValueV1 => throw InconsistentTriplestoreDataException(s"Value $fileValueIri is not a knora-base:FileValue, it is an instance of ${otherValueV1.valueTypeIri}") + case otherValueV1 => throw InconsistentRepositoryDataException(s"Value $fileValueIri is not a knora-base:FileValue, it is an instance of ${otherValueV1.valueTypeIri}") } } @@ -2551,7 +2552,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceIri = resourceIri ).toString()) // _ = println(sparqlQuery) - resPropsResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + resPropsResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] // Partition the property result rows into rows with value properties and rows with link properties. (rowsWithLinks: Seq[VariableResultsRow], rowsWithValues: Seq[VariableResultsRow]) = resPropsResponse.results.bindings.partition(_.rowMap.get("isLinkProp").exists(_.toBoolean)) @@ -2615,7 +2616,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo attributes = propertyEntityInfo match { case Some(entityInfo) => if (entityInfo.isLinkProp) { - (entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))).mkString(";") + (entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute) + valueUtilV1.makeAttributeRestype(entityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")))).mkString(";") } else { entityInfo.getPredicateStringObjectsWithoutLang(OntologyConstants.SalsahGui.GuiAttribute).mkString(";") } @@ -2653,7 +2654,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo val valueObjectsV1WithFuture: Iterable[Future[ValueObjectV1]] = valueObject.valueObjects.map { case (valObjIri: IRI, valueProps: ValueProps) => // Make sure the value object has an rdf:type. - valueProps.literalData.getOrElse(OntologyConstants.Rdf.Type, throw InconsistentTriplestoreDataException(s"$valObjIri has no rdf:type")) + valueProps.literalData.getOrElse(OntologyConstants.Rdf.Type, throw InconsistentRepositoryDataException(s"$valObjIri has no rdf:type")) for { // Convert the SPARQL query results to a ValueV1. @@ -2764,10 +2765,10 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // Get the details of the link value that's pointed to by that link value property, and that has the target resource as its rdf:object. val (linkValueIri, linkValueProps) = groupedPropertiesByType.groupedLinkValueProperties.groupedProperties.getOrElse(linkValuePropertyIri, - throw InconsistentTriplestoreDataException(s"Resource $containingResourceIri has link property $propertyIri but does not have a corresponding link value property")).valueObjects.find { + throw InconsistentRepositoryDataException(s"Resource $containingResourceIri has link property $propertyIri but does not have a corresponding link value property")).valueObjects.find { case (someLinkValueIri, someLinkValueProps) => - someLinkValueProps.literalData.getOrElse(OntologyConstants.Rdf.Object, throw InconsistentTriplestoreDataException(s"Link value $someLinkValueIri has no rdf:object")).literals.head == targetResourceIri - }.getOrElse(throw InconsistentTriplestoreDataException(s"Link property $propertyIri of resource $containingResourceIri points to resource $targetResourceIri, but there is no corresponding link value with the target resource as its rdf:object")) + someLinkValueProps.literalData.getOrElse(OntologyConstants.Rdf.Object, throw InconsistentRepositoryDataException(s"Link value $someLinkValueIri has no rdf:object")).literals.head == targetResourceIri + }.getOrElse(throw InconsistentRepositoryDataException(s"Link property $propertyIri of resource $containingResourceIri points to resource $targetResourceIri, but there is no corresponding link value with the target resource as its rdf:object")) val linkValueOrder = linkValueProps.literalData.get(OntologyConstants.KnoraBase.ValueHasOrder) match { // this should not be necessary as an order should always be given (also if there is only one value) @@ -2786,7 +2787,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo linkValueV1: LinkValueV1 = apiValueV1ForLinkValue match { case linkValueV1: LinkValueV1 => linkValueV1 - case _ => throw InconsistentTriplestoreDataException(s"Expected $linkValueIri to be a knora-base:LinkValue, but its type is ${apiValueV1ForLinkValue.valueTypeIri}") + case _ => throw InconsistentRepositoryDataException(s"Expected $linkValueIri to be a knora-base:LinkValue, but its type is ${apiValueV1ForLinkValue.valueTypeIri}") } // Check the permissions on the LinkValue. diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/SearchResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/SearchResponderV1.scala index cbf5da6e04..f7f8710fc2 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/SearchResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/SearchResponderV1.scala @@ -21,11 +21,12 @@ package org.knora.webapi.responders.v1 import akka.pattern._ import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{BadRequestException, InconsistentRepositoryDataException} import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.OntologyConstants -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectRequest, SparqlSelectResponse, VariableResultsRow} +import org.knora.webapi.messages.store.triplestoremessages.SparqlSelectRequest import org.knora.webapi.messages.twirl.SearchCriterion +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.{DateUtilV1, PermissionUtilADM, ResponderData, ValueUtilV1} import org.knora.webapi.messages.v1.responder.ontologymessages.{EntityInfoGetRequestV1, EntityInfoGetResponseV1, _} import org.knora.webapi.messages.v1.responder.searchmessages._ @@ -168,7 +169,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde // _ = println("================" + pagingSparql) - searchResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(searchSparql)).mapTo[SparqlSelectResponse] + searchResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(searchSparql)).mapTo[SparqlSelectResult] // Get the IRIs of all the properties mentioned in the search results. propertyIris: Set[IRI] = searchResponse.results.bindings.flatMap(_.rowMap.get("resourceProperty")).toSet @@ -199,7 +200,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde val resourceCreator = firstRowMap("resourceCreator") val resourceProject = firstRowMap("resourceProject") - val resourceProjectShortcode = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid resource IRI: $resourceIri")) + val resourceProjectShortcode = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid resource IRI: $resourceIri")) val resourcePermissions = firstRowMap("resourcePermissions") val resourcePermissionCode: Option[Int] = PermissionUtilADM.getUserPermissionV1( @@ -220,7 +221,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde preferredLangs = Some(searchGetRequest.userProfile.lang, settings.fallbackLanguage) ) val resourceClassIcon = resourceEntityInfo.getPredicateObject(OntologyConstants.KnoraBase.ResourceIcon) - val resourceLabel = firstRowMap.getOrElse("resourceLabel", throw InconsistentTriplestoreDataException(s"Resource $resourceIri has no rdfs:label")) + val resourceLabel = firstRowMap.getOrElse("resourceLabel", throw InconsistentRepositoryDataException(s"Resource $resourceIri has no rdfs:label")) // Collect the matching values in the resource. val mapOfMatchingValues: Map[IRI, MatchingValue] = rows.filter(_.rowMap.get("valueObject").nonEmpty).foldLeft(Map.empty[IRI, MatchingValue]) { @@ -243,7 +244,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde val propertyIri = row.rowMap("resourceProperty") val propertyLabel = entityInfoResponse.propertyInfoMap(propertyIri).getPredicateObject(OntologyConstants.Rdfs.Label, preferredLangs = Some(searchGetRequest.userProfile.lang, settings.fallbackLanguage)) match { case Some(label) => label - case None => throw InconsistentTriplestoreDataException(s"Property $propertyIri has no rdfs:label") + case None => throw InconsistentRepositoryDataException(s"Property $propertyIri has no rdfs:label") } valueIri -> MatchingValue( @@ -351,7 +352,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde val propertyObjectClassConstraint: IRI = if (propertyEntityInfo.isLinkProp) { OntologyConstants.KnoraBase.Resource } else { - propertyEntityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $prop has no knora-base:objectClassConstraint")) + propertyEntityInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $prop has no knora-base:objectClassConstraint")) } // Ensure that the property's objectClassConstraint is valid, and that the specified operator can be @@ -505,7 +506,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde // _ = println(searchSparql) - searchResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(searchSparql)).mapTo[SparqlSelectResponse] + searchResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(searchSparql)).mapTo[SparqlSelectResult] // Collect all the resource class IRIs mentioned in the search results. resourceClassIris: Set[IRI] = searchResponse.results.bindings.map(_.rowMap("resourceClass")).toSet @@ -532,7 +533,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde val resourceCreator = firstRowMap("resourceCreator") val resourceProject = firstRowMap("resourceProject") - val resourceProjectShortcode = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid resource IRI: $resourceIri")) + val resourceProjectShortcode = resourceIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid resource IRI: $resourceIri")) val resourcePermissions = firstRowMap("resourcePermissions") val resourcePermissionCode: Option[Int] = PermissionUtilADM.getUserPermissionV1( @@ -550,7 +551,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde val resourceEntityInfo = entityInfoResponse.resourceClassInfoMap(resourceClassIri) val resourceClassLabel = resourceEntityInfo.getPredicateObject(predicateIri = OntologyConstants.Rdfs.Label, preferredLangs = Some(searchGetRequest.userProfile.lang, settings.fallbackLanguage)) val resourceClassIcon = resourceEntityInfo.getPredicateObject(OntologyConstants.KnoraBase.ResourceIcon) - val resourceLabel = firstRowMap.getOrElse("resourceLabel", throw InconsistentTriplestoreDataException(s"Resource $resourceIri has no rdfs:label")) + val resourceLabel = firstRowMap.getOrElse("resourceLabel", throw InconsistentRepositoryDataException(s"Resource $resourceIri has no rdfs:label")) // If there were search criteria referring to values, collect the matching values in the resource. val matchingValues: Vector[MatchingValue] = if (searchCriteria.nonEmpty) { @@ -606,7 +607,7 @@ class SearchResponderV1(responderData: ResponderData) extends Responder(responde val propertyIri = searchCriterion.propertyIri val propertyLabel = propertyInfo.propertyInfoMap(propertyIri).getPredicateObject(predicateIri = OntologyConstants.Rdfs.Label, preferredLangs = Some(searchGetRequest.userProfile.lang, settings.fallbackLanguage)) match { case Some(label) => label - case None => throw InconsistentTriplestoreDataException(s"Property $propertyIri has no rdfs:label") + case None => throw InconsistentRepositoryDataException(s"Property $propertyIri has no rdfs:label") } valueIri -> MatchingValue( diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/UsersResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/UsersResponderV1.scala index 20a7c1ddf0..db4e12fd31 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/UsersResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/UsersResponderV1.scala @@ -29,6 +29,7 @@ import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants import org.knora.webapi.messages.admin.responder.permissionsmessages.{PermissionDataGetADM, PermissionsDataADM} import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.{KnoraSystemInstances, ResponderData} import org.knora.webapi.messages.v1.responder.projectmessages.{ProjectInfoByIRIGetV1, ProjectInfoV1} import org.knora.webapi.messages.v1.responder.usermessages.UserProfileTypeV1.UserProfileType @@ -87,7 +88,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder triplestore = settings.triplestoreType ).toString()) - usersResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + usersResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] usersResponseRows: Seq[VariableResultsRow] = usersResponse.results.bindings @@ -147,7 +148,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder // _ = log.debug("userDataByIRIGetV1 - sparqlQueryString: {}", sparqlQueryString) - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] maybeUserDataV1 <- userDataQueryResponse2UserDataV1(userDataQueryResponse, short) @@ -186,7 +187,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder // _ = log.debug(s"userProfileByIRIGetV1 - sparqlQueryString: {}", sparqlQueryString) - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] maybeUserProfileV1 <- userDataQueryResponse2UserProfileV1( userDataQueryResponse = userDataQueryResponse, @@ -264,7 +265,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder ).toString()) //_ = log.debug(s"userProfileByEmailGetV1 - sparqlQueryString: $sparqlQueryString") - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] //_ = log.debug(MessageUtil.toSource(userDataQueryResponse)) maybeUserProfileV1 <- userDataQueryResponse2UserProfileV1( @@ -331,7 +332,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder //_ = log.debug("userDataByIRIGetV1 - sparqlQueryString: {}", sparqlQueryString) - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] groupedUserData: Map[String, Seq[String]] = userDataQueryResponse.results.bindings.groupBy(_.rowMap("p")).map { case (predicate, rows) => predicate -> rows.map(_.rowMap("o")) @@ -365,7 +366,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder //_ = log.debug("userDataByIRIGetV1 - sparqlQueryString: {}", sparqlQueryString) - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] groupedUserData: Map[String, Seq[String]] = userDataQueryResponse.results.bindings.groupBy(_.rowMap("p")).map { case (predicate, rows) => predicate -> rows.map(_.rowMap("o")) @@ -399,7 +400,7 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder //_ = log.debug("userDataByIRIGetV1 - sparqlQueryString: {}", sparqlQueryString) - userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResponse] + userDataQueryResponse <- (storeManager ? SparqlSelectRequest(sparqlQueryString)).mapTo[SparqlSelectResult] groupedUserData: Map[String, Seq[String]] = userDataQueryResponse.results.bindings.groupBy(_.rowMap("p")).map { case (predicate, rows) => predicate -> rows.map(_.rowMap("o")) @@ -421,13 +422,13 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder //////////////////// /** - * Helper method used to create a [[UserDataV1]] from the [[SparqlSelectResponse]] containing user data. + * Helper method used to create a [[UserDataV1]] from the [[SparqlSelectResult]] containing user data. * - * @param userDataQueryResponse a [[SparqlSelectResponse]] containing user data. + * @param userDataQueryResponse a [[SparqlSelectResult]] containing user data. * @param short denotes if all information should be returned. If short == true, then no token and password should be returned. * @return a [[UserDataV1]] containing the user's basic data. */ - private def userDataQueryResponse2UserDataV1(userDataQueryResponse: SparqlSelectResponse, short: Boolean): Future[Option[UserDataV1]] = { + private def userDataQueryResponse2UserDataV1(userDataQueryResponse: SparqlSelectResult, short: Boolean): Future[Option[UserDataV1]] = { // log.debug("userDataQueryResponse2UserDataV1 - " + MessageUtil.toSource(userDataQueryResponse)) @@ -462,13 +463,13 @@ class UsersResponderV1(responderData: ResponderData) extends Responder(responder } /** - * Helper method used to create a [[UserProfileV1]] from the [[SparqlSelectResponse]] containing user data. + * Helper method used to create a [[UserProfileV1]] from the [[SparqlSelectResult]] containing user data. * - * @param userDataQueryResponse a [[SparqlSelectResponse]] containing user data. + * @param userDataQueryResponse a [[SparqlSelectResult]] containing user data. * @param featureFactoryConfig the feature factory configuration. * @return a [[UserProfileV1]] containing the user's data. */ - private def userDataQueryResponse2UserProfileV1(userDataQueryResponse: SparqlSelectResponse, + private def userDataQueryResponse2UserProfileV1(userDataQueryResponse: SparqlSelectResult, featureFactoryConfig: FeatureFactoryConfig): Future[Option[UserProfileV1]] = { // log.debug("userDataQueryResponse2UserProfileV1 - userDataQueryResponse: {}", MessageUtil.toSource(userDataQueryResponse)) diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala index ebbe19ce85..d36444cda6 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala @@ -31,6 +31,7 @@ import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectADM, P import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.twirl.SparqlTemplateLinkUpdate +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.{KnoraSystemInstances, MessageUtil, PermissionUtilADM, ResponderData, ValueUtilV1} import org.knora.webapi.messages.v1.responder.ontologymessages.{EntityInfoGetRequestV1, EntityInfoGetResponseV1} import org.knora.webapi.messages.v1.responder.projectmessages.{ProjectInfoByIRIGetV1, ProjectInfoV1} @@ -148,7 +149,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde propertyInfo = entityInfoResponse.propertyInfoMap(createValueRequest.propertyIri) propertyObjectClassConstraint = propertyInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse { - throw InconsistentTriplestoreDataException(s"Property ${createValueRequest.propertyIri} has no knora-base:objectClassConstraint") + throw InconsistentRepositoryDataException(s"Property ${createValueRequest.propertyIri} has no knora-base:objectClassConstraint") } // Check that the object of the property (the value to be created, or the target of the link to be created) will have @@ -202,10 +203,10 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde } // Get the IRI of project of the containing resource. - projectIri: IRI = resourceFullResponse.resinfo.getOrElse(throw InconsistentTriplestoreDataException(s"Did not find resource info for resource ${createValueRequest.resourceIri}")).project_id + projectIri: IRI = resourceFullResponse.resinfo.getOrElse(throw InconsistentRepositoryDataException(s"Did not find resource info for resource ${createValueRequest.resourceIri}")).project_id // Get the resource class of the containing resource - resourceClassIri: IRI = resourceFullResponse.resinfo.getOrElse(throw InconsistentTriplestoreDataException(s"Did not find resource info for resource ${createValueRequest.resourceIri}")).restype_id + resourceClassIri: IRI = resourceFullResponse.resinfo.getOrElse(throw InconsistentRepositoryDataException(s"Did not find resource info for resource ${createValueRequest.resourceIri}")).restype_id defaultObjectAccessPermissions <- { responderManager ? DefaultObjectAccessPermissionsStringForPropertyGetADM( @@ -661,7 +662,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde resourceIri = resourceIri ).toString() //_ = print(getFileValuesSparql) - getFileValuesResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(getFileValuesSparql)).mapTo[SparqlSelectResponse] + getFileValuesResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(getFileValuesSparql)).mapTo[SparqlSelectResult] // _ <- Future(println(getFileValuesResponse)) // check that the resource to be updated exists and it is a subclass of knora-base:Representation @@ -808,7 +809,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde propertyInfo = entityInfoResponse.propertyInfoMap(propertyIri) propertyObjectClassConstraint = propertyInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse { - throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") + throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") } // Check that the object of the property (the value to be updated, or the target of the link to be updated) will have @@ -863,7 +864,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde } // Get the resource class of the containing resource - resourceClassIri: IRI = resourceFullResponse.resinfo.getOrElse(throw InconsistentTriplestoreDataException(s"Did not find resource info for resource ${findResourceWithValueResult.resourceIri}")).restype_id + resourceClassIri: IRI = resourceFullResponse.resinfo.getOrElse(throw InconsistentRepositoryDataException(s"Did not find resource info for resource ${findResourceWithValueResult.resourceIri}")).restype_id _ = log.debug(s"changeValueV1 - DefaultObjectAccessPermissionsStringForPropertyGetV1 - projectIri ${findResourceWithValueResult.projectIri}, propertyIri: ${findResourceWithValueResult.propertyIri}, permissions: ${changeValueRequest.userProfile.permissions} ") defaultObjectAccessPermissions <- { @@ -924,7 +925,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde val valuePermissions = currentValueQueryResult.permissionRelevantAssertions.find { case (p, o) => p == OntologyConstants.KnoraBase.HasPermissions - }.map(_._2).getOrElse(throw InconsistentTriplestoreDataException(s"Value ${changeValueRequest.valueIri} has no permissions")) + }.map(_._2).getOrElse(throw InconsistentRepositoryDataException(s"Value ${changeValueRequest.valueIri} has no permissions")) changeOrdinaryValueV1AfterChecks( projectIri = currentValueQueryResult.projectIri, @@ -1113,7 +1114,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde val valuePermissions: String = currentValueQueryResult.permissionRelevantAssertions.find { case (p, o) => p == OntologyConstants.KnoraBase.HasPermissions - }.map(_._2).getOrElse(throw InconsistentTriplestoreDataException(s"Value ${deleteValueRequest.valueIri} has no permissions")) + }.map(_._2).getOrElse(throw InconsistentRepositoryDataException(s"Value ${deleteValueRequest.valueIri} has no permissions")) val linkPropertyIri = stringFormatter.linkValuePropertyIriToLinkPropertyIri(findResourceWithValueResult.propertyIri) @@ -1218,10 +1219,10 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde triplestore = settings.triplestoreType, valueIri = deletedValueIri ).toString() - sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = sparqlSelectResponse.results.bindings - _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { + _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentRepositoryDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { throw UpdateNotPerformedException(s"The request to mark value ${deleteValueRequest.valueIri} (or a new version of that value) as deleted did not succeed. Please report this as a possible bug.") } } yield DeleteValueResponseV1(id = deletedValueIri) @@ -1288,7 +1289,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde currentValueIri = versionHistoryRequest.currentValueIri ).toString() } - selectResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + selectResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = selectResponse.results.bindings _ = if (rows.isEmpty) { @@ -1314,7 +1315,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde val valuePermissions = rowMap("valuePermissions") // Permission-checking on LinkValues is special, because they can be system-created rather than user-created. - val valuePermissionCode = if (stringFormatter.optionStringToBoolean(rowMap.get("isLinkValue"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isLinkValue: ${rowMap.get("isLinkValue")}"))) { + val valuePermissionCode = if (stringFormatter.optionStringToBoolean(rowMap.get("isLinkValue"), throw InconsistentRepositoryDataException(s"Invalid boolean for isLinkValue: ${rowMap.get("isLinkValue")}"))) { // It's a LinkValue. PermissionUtilADM.getUserPermissionV1( entityIri = valueIri, @@ -1500,7 +1501,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde valueIri = valueIri ).toString()) - response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows: Seq[VariableResultsRow] = response.results.bindings maybeValueQueryResult <- sparqlQueryResults2ValueQueryResult( @@ -1540,7 +1541,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde linkValueIri = linkValueIri ).toString()) - response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = response.results.bindings _ = if (rows.isEmpty) { @@ -1600,7 +1601,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde ).toString() } - response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows: Seq[VariableResultsRow] = response.results.bindings maybeLinkValueQueryResult <- sparqlQueryResults2LinkValueQueryResult( @@ -1643,7 +1644,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde ).toString() } - response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + response <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows: Seq[VariableResultsRow] = response.results.bindings maybeLinkValueQueryResult <- sparqlQueryResults2LinkValueQueryResult( @@ -1683,7 +1684,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde val valueProps = valueUtilV1.createValueProps(valueIri, rows) for { - projectShortcode: String <- Future(valueIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid value IRI: $valueIri"))) + projectShortcode: String <- Future(valueIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid value IRI: $valueIri"))) value <- valueUtilV1.makeValueV1( valueProps = valueProps, @@ -1694,16 +1695,16 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde ) // Get the value's class IRI. - valueClassIri = getValuePredicateObject(predicateIri = OntologyConstants.Rdf.Type, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"Value $valueIri has no rdf:type")) + valueClassIri = getValuePredicateObject(predicateIri = OntologyConstants.Rdf.Type, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"Value $valueIri has no rdf:type")) // Get the IRI of the value's creator. - creatorIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToUser, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"Value $valueIri has no knora-base:attachedToUser")) + creatorIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToUser, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"Value $valueIri has no knora-base:attachedToUser")) // Get the value's project IRI. - projectIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToProject, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"The resource containing value $valueIri has no knora-base:attachedToProject")) + projectIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToProject, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"The resource containing value $valueIri has no knora-base:attachedToProject")) // Get the value's creation date. - creationDate = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.ValueCreationDate, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"Value $valueIri has no valueCreationDate")) + creationDate = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.ValueCreationDate, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"Value $valueIri has no valueCreationDate")) // Get the optional comment on the value. comment = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.ValueHasComment, rows = rows) @@ -1720,7 +1721,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde maybePermissionCode = valueClassIri match { case OntologyConstants.KnoraBase.LinkValue => - val linkPredicateIri = getValuePredicateObject(predicateIri = OntologyConstants.Rdf.Predicate, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"Link value $valueIri has no rdf:predicate")) + val linkPredicateIri = getValuePredicateObject(predicateIri = OntologyConstants.Rdf.Predicate, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"Link value $valueIri has no rdf:predicate")) PermissionUtilADM.getUserPermissionWithValuePropsV1( valueIri = valueIri, @@ -1777,7 +1778,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde val valueProps = valueUtilV1.createValueProps(linkValueIri, rows) for { - projectShortcode: String <- Future(linkValueIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentTriplestoreDataException(s"Invalid value IRI: $linkValueIri"))) + projectShortcode: String <- Future(linkValueIri.toSmartIri.getProjectCode.getOrElse(throw InconsistentRepositoryDataException(s"Invalid value IRI: $linkValueIri"))) linkValueMaybe <- valueUtilV1.makeValueV1( valueProps = valueProps, @@ -1789,17 +1790,17 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde linkValueV1: LinkValueV1 = linkValueMaybe match { case linkValue: LinkValueV1 => linkValue - case other => throw InconsistentTriplestoreDataException(s"Expected value $linkValueIri to be of type ${OntologyConstants.KnoraBase.LinkValue}, but it was read with type ${other.valueTypeIri}") + case other => throw InconsistentRepositoryDataException(s"Expected value $linkValueIri to be of type ${OntologyConstants.KnoraBase.LinkValue}, but it was read with type ${other.valueTypeIri}") } // Get the IRI of the value's owner. - creatorIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToUser, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"Value $linkValueIri has no knora-base:attachedToUser")) + creatorIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToUser, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"Value $linkValueIri has no knora-base:attachedToUser")) // Get the value's project IRI. - projectIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToProject, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"The resource containing value $linkValueIri has no knora-base:attachedToProject")) + projectIri = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.AttachedToProject, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"The resource containing value $linkValueIri has no knora-base:attachedToProject")) // Get the value's creation date. - creationDate = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.ValueCreationDate, rows = rows).getOrElse(throw InconsistentTriplestoreDataException(s"Value $linkValueIri has no valueCreationDate")) + creationDate = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.ValueCreationDate, rows = rows).getOrElse(throw InconsistentRepositoryDataException(s"Value $linkValueIri has no valueCreationDate")) // Get the optional comment on the value. comment = getValuePredicateObject(predicateIri = OntologyConstants.KnoraBase.ValueHasComment, rows = rows) @@ -1927,7 +1928,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde ).toString() } - updateVerificationResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + updateVerificationResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = updateVerificationResponse.results.bindings resultOption <- sparqlQueryResults2ValueQueryResult( @@ -2016,7 +2017,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde triplestore = settings.triplestoreType, searchValueIri = valueIri ).toString()) - findResourceResponse <- (storeManager ? SparqlSelectRequest(findResourceSparqlQuery)).mapTo[SparqlSelectResponse] + findResourceResponse <- (storeManager ? SparqlSelectRequest(findResourceSparqlQuery)).mapTo[SparqlSelectResult] _ = if (findResourceResponse.results.bindings.isEmpty) { throw NotFoundException(s"No resource found containing value $valueIri") @@ -2659,7 +2660,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde case None => // We didn't find the LinkValue. This shouldn't happen. - throw InconsistentTriplestoreDataException(s"There should be a knora-base:LinkValue describing a direct link from resource $sourceResourceIri to resource $targetResourceIri using property $linkPropertyIri, but it seems to be missing") + throw InconsistentRepositoryDataException(s"There should be a knora-base:LinkValue describing a direct link from resource $sourceResourceIri to resource $targetResourceIri using property $linkPropertyIri, but it seems to be missing") } } yield linkUpdate } diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/OntologyResponderV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/OntologyResponderV2.scala index 01c38f7fe3..8ad2d7a84e 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/OntologyResponderV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/OntologyResponderV2.scala @@ -31,6 +31,7 @@ import org.knora.webapi.messages.StringFormatter.{SalsahGuiAttribute, SalsahGuiA import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectGetRequestADM, ProjectGetResponseADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.util.{ErrorHandlingMap, KnoraSystemInstances, OntologyUtil, ResponderData} import org.knora.webapi.messages.v2.responder.SuccessResponseV2 import org.knora.webapi.messages.v2.responder.ontologymessages.Cardinality.{KnoraCardinalityInfo, OwlCardinalityInfo} @@ -151,14 +152,14 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon // Get all ontology metadata. allOntologyMetadataSparql <- FastFuture.successful(org.knora.webapi.messages.twirl.queries.sparql.v2.txt.getAllOntologyMetadata(triplestore = settings.triplestoreType).toString()) - allOntologyMetadataResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(allOntologyMetadataSparql)).mapTo[SparqlSelectResponse] + allOntologyMetadataResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(allOntologyMetadataSparql)).mapTo[SparqlSelectResult] allOntologyMetadata: Map[SmartIri, OntologyMetadataV2] = buildOntologyMetadata(allOntologyMetadataResponse) - knoraBaseOntologyMetadata: OntologyMetadataV2 = allOntologyMetadata.getOrElse(OntologyConstants.KnoraBase.KnoraBaseOntologyIri.toSmartIri, throw InconsistentTriplestoreDataException(s"No knora-base ontology found")) - knoraBaseOntologyVersion: String = knoraBaseOntologyMetadata.ontologyVersion.getOrElse(throw InconsistentTriplestoreDataException("The knora-base ontology in the repository is not up to date. See the Knora documentation on repository updates.")) + knoraBaseOntologyMetadata: OntologyMetadataV2 = allOntologyMetadata.getOrElse(OntologyConstants.KnoraBase.KnoraBaseOntologyIri.toSmartIri, throw InconsistentRepositoryDataException(s"No knora-base ontology found")) + knoraBaseOntologyVersion: String = knoraBaseOntologyMetadata.ontologyVersion.getOrElse(throw InconsistentRepositoryDataException("The knora-base ontology in the repository is not up to date. See the Knora documentation on repository updates.")) _ = if (knoraBaseOntologyVersion != KnoraBaseVersion) { - throw InconsistentTriplestoreDataException(s"The knora-base ontology in the repository has version '$knoraBaseOntologyVersion', but this version of Knora requires '$KnoraBaseVersion'. See the Knora documentation on repository updates.") + throw InconsistentRepositoryDataException(s"The knora-base ontology in the repository has version '$knoraBaseOntologyVersion', but this version of Knora requires '$KnoraBaseVersion'. See the Knora documentation on repository updates.") } // Get the contents of each named graph containing an ontology. @@ -428,7 +429,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon constraintValueToBeChecked = subjectClassConstraint, allSuperPropertyIris = allSuperPropertyIris, errorSchema = InternalSchema, - errorFun = { msg: String => throw InconsistentTriplestoreDataException(msg) } + errorFun = { msg: String => throw InconsistentRepositoryDataException(msg) } ) // If the property is defined in a project-specific ontology, its subject class constraint, if provided, must be a Knora resource or standoff class. @@ -437,7 +438,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon if (!(baseClassesOfSubjectClassConstraint.contains(OntologyConstants.KnoraBase.Resource.toSmartIri) || baseClassesOfSubjectClassConstraint.contains(OntologyConstants.KnoraBase.StandoffTag.toSmartIri))) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri is defined in a project-specific ontology, but its knora-base:subjectClassConstraint, $subjectClassConstraint, is not a subclass of knora-base:Resource or knora-base:StandoffTag") + throw InconsistentRepositoryDataException(s"Property $propertyIri is defined in a project-specific ontology, but its knora-base:subjectClassConstraint, $subjectClassConstraint, is not a subclass of knora-base:Resource or knora-base:StandoffTag") } } @@ -454,13 +455,13 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon constraintValueToBeChecked = objectClassConstraint, allSuperPropertyIris = allSuperPropertyIris, errorSchema = InternalSchema, - errorFun = { msg: String => throw InconsistentTriplestoreDataException(msg) } + errorFun = { msg: String => throw InconsistentRepositoryDataException(msg) } ) case None => // A resource property must have an object class constraint, unless it's knora-base:resourceProperty. if (readPropertyInfo.isResourceProp && propertyIri != OntologyConstants.KnoraBase.ResourceProperty.toSmartIri) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") + throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") } } } @@ -581,7 +582,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon checkOntologyReferencesInPropertyDef( ontologyCacheData = ontologyCacheData, propertyDef = propertyInfo.entityInfoContent, - errorFun = { msg: String => throw InconsistentTriplestoreDataException(msg) } + errorFun = { msg: String => throw InconsistentRepositoryDataException(msg) } ) } @@ -589,7 +590,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon checkOntologyReferencesInClassDef( ontologyCacheData = ontologyCacheData, classDef = classInfo.entityInfoContent, - errorFun = { msg: String => throw InconsistentTriplestoreDataException(msg) } + errorFun = { msg: String => throw InconsistentRepositoryDataException(msg) } ) } } @@ -609,7 +610,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ontologyGraph => val entityIrisInGraph: Set[SmartIri] = ontologyGraph.constructResponse.statements.foldLeft(Set.empty[SmartIri]) { case (acc, (subjectIri: IriSubjectV2, subjectStatements: Map[SmartIri, Seq[LiteralV2]])) => - val subjectTypeLiterals: Seq[IriLiteralV2] = subjectStatements.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"Subject $subjectIri has no rdf:type")).collect { + val subjectTypeLiterals: Seq[IriLiteralV2] = subjectStatements.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"Subject $subjectIri has no rdf:type")).collect { case iriLiteral: IriLiteralV2 => iriLiteral } @@ -633,31 +634,31 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon * @param allOntologyMetadataResponse the triplestore's response to the SPARQL query `getAllOntologyMetadata.scala.txt`. * @return a map of ontology IRIs to ontology metadata. */ - private def buildOntologyMetadata(allOntologyMetadataResponse: SparqlSelectResponse): Map[SmartIri, OntologyMetadataV2] = { + private def buildOntologyMetadata(allOntologyMetadataResponse: SparqlSelectResult): Map[SmartIri, OntologyMetadataV2] = { allOntologyMetadataResponse.results.bindings.groupBy(_.rowMap("ontologyGraph")).map { case (ontologyGraph: IRI, rows: Seq[VariableResultsRow]) => val ontologyIri = rows.head.rowMap("ontologyIri") if (ontologyIri != ontologyGraph) { - throw InconsistentTriplestoreDataException(s"Ontology $ontologyIri must be stored in named graph $ontologyIri, but it is in $ontologyGraph") + throw InconsistentRepositoryDataException(s"Ontology $ontologyIri must be stored in named graph $ontologyIri, but it is in $ontologyGraph") } val ontologySmartIri = ontologyIri.toSmartIri if (!ontologySmartIri.isKnoraOntologyIri) { - throw InconsistentTriplestoreDataException(s"Ontology $ontologySmartIri is not a Knora ontology") + throw InconsistentRepositoryDataException(s"Ontology $ontologySmartIri is not a Knora ontology") } val ontologyMetadataMap: Map[IRI, String] = rows.map { row => - val pred = row.rowMap.getOrElse("ontologyPred", throw InconsistentTriplestoreDataException(s"Empty predicate in ontology $ontologyIri")) - val obj = row.rowMap.getOrElse("ontologyObj", throw InconsistentTriplestoreDataException(s"Empty object for predicate $pred in ontology $ontologyIri")) + val pred = row.rowMap.getOrElse("ontologyPred", throw InconsistentRepositoryDataException(s"Empty predicate in ontology $ontologyIri")) + val obj = row.rowMap.getOrElse("ontologyObj", throw InconsistentRepositoryDataException(s"Empty object for predicate $pred in ontology $ontologyIri")) pred -> obj }.toMap - val projectIri: SmartIri = ontologyMetadataMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentTriplestoreDataException(s"Ontology $ontologyIri has no knora-base:attachedToProject")).toSmartIri + val projectIri: SmartIri = ontologyMetadataMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentRepositoryDataException(s"Ontology $ontologyIri has no knora-base:attachedToProject")).toSmartIri val ontologyLabel: String = ontologyMetadataMap.getOrElse(OntologyConstants.Rdfs.Label, ontologySmartIri.getOntologyName) - val lastModificationDate: Option[Instant] = ontologyMetadataMap.get(OntologyConstants.KnoraBase.LastModificationDate).map(instant => stringFormatter.xsdDateTimeStampToInstant(instant, throw InconsistentTriplestoreDataException(s"Invalid UTC instant: $instant"))) + val lastModificationDate: Option[Instant] = ontologyMetadataMap.get(OntologyConstants.KnoraBase.LastModificationDate).map(instant => stringFormatter.xsdDateTimeStampToInstant(instant, throw InconsistentRepositoryDataException(s"Invalid UTC instant: $instant"))) val ontologyVersion: Option[String] = ontologyMetadataMap.get(OntologyConstants.KnoraBase.OntologyVersion) ontologySmartIri -> OntologyMetadataV2( @@ -720,7 +721,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val missingLinkValueProps = linkPropsInClass.map(_.fromLinkPropToLinkValueProp) -- linkValuePropsInClass if (missingLinkValueProps.nonEmpty) { - throw InconsistentTriplestoreDataException(s"Resource class $classIri has cardinalities for one or more link properties without corresponding link value properties. The missing (or incorrectly defined) property or properties: ${missingLinkValueProps.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Resource class $classIri has cardinalities for one or more link properties without corresponding link value properties. The missing (or incorrectly defined) property or properties: ${missingLinkValueProps.mkString(", ")}") } // Make sure there is a link property for each link value property. @@ -728,7 +729,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val missingLinkProps = linkValuePropsInClass.map(_.fromLinkValuePropToLinkProp) -- linkPropsInClass if (missingLinkProps.nonEmpty) { - throw InconsistentTriplestoreDataException(s"Resource class $classIri has cardinalities for one or more link value properties without corresponding link properties. The missing (or incorrectly defined) property or properties: ${missingLinkProps.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Resource class $classIri has cardinalities for one or more link value properties without corresponding link properties. The missing (or incorrectly defined) property or properties: ${missingLinkProps.mkString(", ")}") } // Make sure that the cardinality for each link property is the same as the cardinality for the corresponding link value property. @@ -738,7 +739,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val linkValuePropCardinality: OwlCardinalityInfo = allOwlCardinalitiesForClass(linkValueProp) if (!linkPropCardinality.equalsWithoutGuiOrder(linkValuePropCardinality)) { - throw InconsistentTriplestoreDataException(s"In class $classIri, the cardinality for $linkProp is different from the cardinality for $linkValueProp") + throw InconsistentRepositoryDataException(s"In class $classIri, the cardinality for $linkProp is different from the cardinality for $linkValueProp") } } @@ -759,14 +760,14 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon if (!ontologyIri.isKnoraBuiltInDefinitionIri) { // It must be either a resource class or a standoff class, but not both. if (!(isKnoraResourceClass ^ isStandoffClass)) { - throw InconsistentTriplestoreDataException(s"Class $classIri must be a subclass either of knora-base:Resource or of knora-base:StandoffTag (but not both)") + throw InconsistentRepositoryDataException(s"Class $classIri must be a subclass either of knora-base:Resource or of knora-base:StandoffTag (but not both)") } // All its cardinalities must be on properties that are defined. val cardinalitiesOnMissingProps = directCardinalityPropertyIris.filterNot(allPropertyDefs.keySet) if (cardinalitiesOnMissingProps.nonEmpty) { - throw InconsistentTriplestoreDataException(s"Class $classIri has one or more cardinalities on undefined properties: ${cardinalitiesOnMissingProps.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Class $classIri has one or more cardinalities on undefined properties: ${cardinalitiesOnMissingProps.mkString(", ")}") } // It cannot have cardinalities both on property P and on a subproperty of P. @@ -778,7 +779,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon maybePropertyAndSubproperty match { case Some((basePropertyIri, propertyIri)) => - throw InconsistentTriplestoreDataException(s"Class $classIri has a cardinality on property $basePropertyIri and on its subproperty $propertyIri") + throw InconsistentRepositoryDataException(s"Class $classIri has a cardinality on property $basePropertyIri and on its subproperty $propertyIri") case None => () } @@ -789,13 +790,13 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val cardinalitiesOnInvalidProps = directCardinalityPropertyIris.filterNot(allKnoraResourceProps) if (cardinalitiesOnInvalidProps.nonEmpty) { - throw InconsistentTriplestoreDataException(s"Resource class $classIri has one or more cardinalities on properties that are not Knora resource properties: ${cardinalitiesOnInvalidProps.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Resource class $classIri has one or more cardinalities on properties that are not Knora resource properties: ${cardinalitiesOnInvalidProps.mkString(", ")}") } Set(OntologyConstants.KnoraBase.ResourceProperty, OntologyConstants.KnoraBase.HasValue).foreach { invalidProp => if (directCardinalityPropertyIris.contains(invalidProp.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Class $classIri has a cardinality on property $invalidProp, which is not allowed") + throw InconsistentRepositoryDataException(s"Class $classIri has a cardinality on property $invalidProp, which is not allowed") } } @@ -803,26 +804,26 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val invalidCardinalitiesOnBooleanProps: Set[SmartIri] = directCardinalities.filter { case (propertyIri, knoraCardinalityInfo) => - val propertyObjectClassConstraint: SmartIri = allPropertyDefs(propertyIri).requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")) + val propertyObjectClassConstraint: SmartIri = allPropertyDefs(propertyIri).requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint")) propertyObjectClassConstraint == OntologyConstants.KnoraBase.BooleanValue.toSmartIri && !(knoraCardinalityInfo.cardinality == Cardinality.MustHaveOne || knoraCardinalityInfo.cardinality == Cardinality.MayHaveOne) }.keySet if (invalidCardinalitiesOnBooleanProps.nonEmpty) { - throw InconsistentTriplestoreDataException(s"Class $classIri has one or more invalid cardinalities on boolean properties: ${invalidCardinalitiesOnBooleanProps.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Class $classIri has one or more invalid cardinalities on boolean properties: ${invalidCardinalitiesOnBooleanProps.mkString(", ")}") } // All its base classes with Knora IRIs must also be resource classes. for (baseClass <- classDef.subClassOf) { if (baseClass.isKnoraDefinitionIri && !allSubClassOfRelations(baseClass).contains(OntologyConstants.KnoraBase.Resource.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Class $classIri is a subclass of knora-base:Resource, but its base class $baseClass is not") + throw InconsistentRepositoryDataException(s"Class $classIri is a subclass of knora-base:Resource, but its base class $baseClass is not") } } // It must have an rdfs:label. if (!classDef.predicates.contains(OntologyConstants.Rdfs.Label.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Class $classIri has no rdfs:label") + throw InconsistentRepositoryDataException(s"Class $classIri has no rdfs:label") } } else { // If it's a standoff class, none of its cardinalities must be on Knora resource properties. @@ -830,14 +831,14 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val cardinalitiesOnInvalidProps = directCardinalityPropertyIris.filter(allKnoraResourceProps) if (cardinalitiesOnInvalidProps.nonEmpty) { - throw InconsistentTriplestoreDataException(s"Standoff class $classIri has one or more cardinalities on properties that are Knora resource properties: ${cardinalitiesOnInvalidProps.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Standoff class $classIri has one or more cardinalities on properties that are Knora resource properties: ${cardinalitiesOnInvalidProps.mkString(", ")}") } // All its base classes with Knora IRIs must also be standoff classes. for (baseClass <- classDef.subClassOf) { if (baseClass.isKnoraDefinitionIri) { if (isStandoffClass && !allSubClassOfRelations(baseClass).contains(OntologyConstants.KnoraBase.StandoffTag.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Class $classIri is a subclass of knora-base:StandoffTag, but its base class $baseClass is not") + throw InconsistentRepositoryDataException(s"Class $classIri is a subclass of knora-base:StandoffTag, but its base class $baseClass is not") } } } @@ -850,7 +851,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon allBaseClassIris = allBaseClasses.toSet, allClassCardinalityKnoraPropertyDefs = allPropertyDefs.filterKeys(allOwlCardinalitiesForClass.keySet), errorSchema = InternalSchema, - errorFun = { msg: String => throw InconsistentTriplestoreDataException(msg) } + errorFun = { msg: String => throw InconsistentRepositoryDataException(msg) } ) val inheritedCardinalities: Map[SmartIri, KnoraCardinalityInfo] = allOwlCardinalitiesForClass.filterNot { @@ -866,7 +867,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val standoffDataType: Set[SmartIri] = allSubClassOfRelations(classIri).toSet.intersect(StandoffDataTypeClasses.getStandoffClassIris.map(_.toSmartIri)) if (standoffDataType.size > 1) { - throw InconsistentTriplestoreDataException(s"Class $classIri is a subclass of more than one standoff datatype: ${standoffDataType.mkString(", ")}") + throw InconsistentRepositoryDataException(s"Class $classIri is a subclass of more than one standoff datatype: ${standoffDataType.mkString(", ")}") } // A class can be instantiated if it's in a built-in ontology and marked with knora-base:canBeInstantiated, or if it's @@ -895,7 +896,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon standoffDataType = standoffDataType.headOption match { case Some(dataType: SmartIri) => Some(StandoffDataTypeClasses.lookup(dataType.toString, - throw InconsistentTriplestoreDataException(s"$dataType is not a valid standoff datatype"))) + throw InconsistentRepositoryDataException(s"$dataType is not a valid standoff datatype"))) case None => None } @@ -936,7 +937,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon validateGuiAttributes( propertyInfoContent = propertyDef, allGuiAttributeDefinitions = allGuiAttributeDefinitions, - errorFun = { msg: String => throw InconsistentTriplestoreDataException(msg) } + errorFun = { msg: String => throw InconsistentRepositoryDataException(msg) } ) val isResourceProp = allKnoraResourceProps.contains(propertyIri) @@ -949,24 +950,24 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon if (!propertyIri.isKnoraBuiltInDefinitionIri && isResourceProp) { // The property must be a subproperty of knora-base:hasValue or knora-base:hasLinkTo, but not both. if (isValueProp && isLinkProp) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri cannot be a subproperty of both knora-base:hasValue and knora-base:hasLinkTo") + throw InconsistentRepositoryDataException(s"Property $propertyIri cannot be a subproperty of both knora-base:hasValue and knora-base:hasLinkTo") } // It can't be a subproperty of knora-base:hasFileValue. if (isFileValueProp) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri cannot be a subproperty of knora-base:hasFileValue") + throw InconsistentRepositoryDataException(s"Property $propertyIri cannot be a subproperty of knora-base:hasFileValue") } // Each of its base properties that has a Knora IRI must also be a Knora resource property. for (baseProperty <- propertyDef.subPropertyOf) { if (baseProperty.isKnoraDefinitionIri && !allKnoraResourceProps.contains(baseProperty)) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri is a subproperty of knora-base:hasValue or knora-base:hasLinkTo, but its base property $baseProperty is not") + throw InconsistentRepositoryDataException(s"Property $propertyIri is a subproperty of knora-base:hasValue or knora-base:hasLinkTo, but its base property $baseProperty is not") } } // It must have an rdfs:label. if (!propertyDef.predicates.contains(OntologyConstants.Rdfs.Label.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri has no rdfs:label") + throw InconsistentRepositoryDataException(s"Property $propertyIri has no rdfs:label") } } @@ -1028,11 +1029,11 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon case StringLiteralV2(attributeDefStr, None) => stringFormatter.toSalsahGuiAttributeDefinition( attributeDefStr, - throw InconsistentTriplestoreDataException(s"Invalid salsah-gui:guiAttributeDefinition in $guiElementIri: $attributeDefStr") + throw InconsistentRepositoryDataException(s"Invalid salsah-gui:guiAttributeDefinition in $guiElementIri: $attributeDefStr") ) case other => - throw InconsistentTriplestoreDataException(s"Invalid salsah-gui:guiAttributeDefinition in $guiElementIri: $other") + throw InconsistentRepositoryDataException(s"Invalid salsah-gui:guiAttributeDefinition in $guiElementIri: $other") }.toSet case None => Set.empty[SalsahGuiAttributeDefinition] @@ -1055,7 +1056,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon // Find out which salsah-gui:Guielement the property uses, if any. val maybeGuiElementPred: Option[PredicateInfoV2] = predicates.get(OntologyConstants.SalsahGui.GuiElementProp.toSmartIri) - val maybeGuiElementIri: Option[SmartIri] = maybeGuiElementPred.map(_.requireIriObject(throw InconsistentTriplestoreDataException(s"Property $propertyIri has an invalid object for ${OntologyConstants.SalsahGui.GuiElementProp}"))) + val maybeGuiElementIri: Option[SmartIri] = maybeGuiElementPred.map(_.requireIriObject(throw InconsistentRepositoryDataException(s"Property $propertyIri has an invalid object for ${OntologyConstants.SalsahGui.GuiElementProp}"))) // Get that Guielement's attribute definitions, if any. val guiAttributeDefs: Set[SalsahGuiAttributeDefinition] = maybeGuiElementIri match { @@ -1413,7 +1414,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon label = classInfo.entityInfoContent.getPredicateStringLiteralObject( predicateIri = OntologyConstants.Rdfs.Label.toSmartIri, preferredLangs = Some(requestingUser.lang, settings.fallbackLanguage) - ).getOrElse(throw InconsistentTriplestoreDataException(s"Resource class $subClassIri has no rdfs:label")) + ).getOrElse(throw InconsistentRepositoryDataException(s"Resource class $subClassIri has no rdfs:label")) ) } } yield SubClassesGetResponseV2( @@ -1647,29 +1648,29 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon } } - val projectIris: Seq[String] = statementMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri has no knora-base:attachedToProject")) + val projectIris: Seq[String] = statementMap.getOrElse(OntologyConstants.KnoraBase.AttachedToProject, throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri has no knora-base:attachedToProject")) val labels: Seq[String] = statementMap.getOrElse(OntologyConstants.Rdfs.Label, Seq.empty[String]) val comments: Seq[String] = statementMap.getOrElse(OntologyConstants.Rdfs.Comment, Seq.empty[String]) val lastModDates: Seq[String] = statementMap.getOrElse(OntologyConstants.KnoraBase.LastModificationDate, Seq.empty[String]) val projectIri = if (projectIris.size > 1) { - throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri has more than one knora-base:attachedToProject") + throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri has more than one knora-base:attachedToProject") } else { projectIris.head.toSmartIri } if (!internalOntologyIri.isKnoraBuiltInDefinitionIri) { if (projectIri.toString == OntologyConstants.KnoraAdmin.SystemProject) { - throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri cannot be in project ${OntologyConstants.KnoraAdmin.SystemProject}") + throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri cannot be in project ${OntologyConstants.KnoraAdmin.SystemProject}") } if (internalOntologyIri.isKnoraSharedDefinitionIri && projectIri.toString != OntologyConstants.KnoraAdmin.DefaultSharedOntologiesProject) { - throw InconsistentTriplestoreDataException(s"Shared ontology $internalOntologyIri must be in project ${OntologyConstants.KnoraAdmin.DefaultSharedOntologiesProject}") + throw InconsistentRepositoryDataException(s"Shared ontology $internalOntologyIri must be in project ${OntologyConstants.KnoraAdmin.DefaultSharedOntologiesProject}") } } val label: String = if (labels.size > 1) { - throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri has more than one rdfs:label") + throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri has more than one rdfs:label") } else if (labels.isEmpty) { internalOntologyIri.getOntologyName } else { @@ -1677,16 +1678,16 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon } val comment: Option[String] = if (comments.size > 1) { - throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri has more than one rdfs:comment") + throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri has more than one rdfs:comment") } else comments.headOption val lastModificationDate: Option[Instant] = if (lastModDates.size > 1) { - throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri has more than one ${OntologyConstants.KnoraBase.LastModificationDate}") + throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri has more than one ${OntologyConstants.KnoraBase.LastModificationDate}") } else if (lastModDates.isEmpty) { None } else { val dateStr = lastModDates.head - Some(stringFormatter.xsdDateTimeStampToInstant(dateStr, throw InconsistentTriplestoreDataException(s"Invalid ${OntologyConstants.KnoraBase.LastModificationDate}: $dateStr"))) + Some(stringFormatter.xsdDateTimeStampToInstant(dateStr, throw InconsistentRepositoryDataException(s"Invalid ${OntologyConstants.KnoraBase.LastModificationDate}: $dateStr"))) } Some(OntologyMetadataV2( @@ -2050,7 +2051,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) _ = if (loadedClassDef != unescapedClassDefWithLinkValueProps) { - throw InconsistentTriplestoreDataException(s"Attempted to save class definition $unescapedClassDefWithLinkValueProps, but $loadedClassDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save class definition $unescapedClassDefWithLinkValueProps, but $loadedClassDef was saved") } // Update the cache. @@ -2270,7 +2271,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon case (propertyIri, propertyDef) => propertyDef.predicates.get(OntologyConstants.KnoraBase.SubjectClassConstraint.toSmartIri) match { case Some(subjectClassConstraintPred) => - val subjectClassConstraint = subjectClassConstraintPred.requireIriObject(throw InconsistentTriplestoreDataException(s"Property $propertyIri has an invalid object for ${OntologyConstants.KnoraBase.SubjectClassConstraint}")) + val subjectClassConstraint = subjectClassConstraintPred.requireIriObject(throw InconsistentRepositoryDataException(s"Property $propertyIri has an invalid object for ${OntologyConstants.KnoraBase.SubjectClassConstraint}")) if (!allBaseClassIris.contains(subjectClassConstraint)) { val hasOrWouldInherit = if (internalClassDef.directCardinalities.contains(propertyIri)) { @@ -2425,7 +2426,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) _ = if (loadedClassDef != newInternalClassDefWithLinkValueProps) { - throw InconsistentTriplestoreDataException(s"Attempted to save class definition $newInternalClassDefWithLinkValueProps, but $loadedClassDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save class definition $newInternalClassDefWithLinkValueProps, but $loadedClassDef was saved") } // Update the cache. @@ -2600,7 +2601,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) _ = if (loadedClassDef != newInternalClassDefWithLinkValueProps) { - throw InconsistentTriplestoreDataException(s"Attempted to save class definition $newInternalClassDefWithLinkValueProps, but $loadedClassDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save class definition $newInternalClassDefWithLinkValueProps, but $loadedClassDef was saved") } // Update the cache. @@ -2911,7 +2912,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon propertyIris = ontology.properties.keySet ).toString() - isOntologyUsedResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(isOntologyUsedSparql)).mapTo[SparqlSelectResponse] + isOntologyUsedResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(isOntologyUsedSparql)).mapTo[SparqlSelectResult] _ = if (isOntologyUsedResponse.results.bindings.nonEmpty) { val subjects: Seq[String] = isOntologyUsedResponse.results.bindings.map { @@ -3189,7 +3190,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon unescapedInputPropertyDef = internalPropertyDef.unescape _ = if (loadedPropertyDef != unescapedInputPropertyDef) { - throw InconsistentTriplestoreDataException(s"Attempted to save property definition $unescapedInputPropertyDef, but $loadedPropertyDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save property definition $unescapedInputPropertyDef, but $loadedPropertyDef was saved") } maybeLoadedLinkValuePropertyDefFuture: Option[Future[PropertyInfoContentV2]] = maybeLinkValuePropertyDef.map { @@ -3206,7 +3207,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon _ = (maybeLoadedLinkValuePropertyDef, maybeUnescapedNewLinkValuePropertyDef) match { case (Some(loadedLinkValuePropertyDef), Some(unescapedNewLinkPropertyDef)) => if (loadedLinkValuePropertyDef != unescapedNewLinkPropertyDef) { - throw InconsistentTriplestoreDataException(s"Attempted to save link value property definition $unescapedNewLinkPropertyDef, but $loadedLinkValuePropertyDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save link value property definition $unescapedNewLinkPropertyDef, but $loadedLinkValuePropertyDef was saved") } case _ => () @@ -3318,7 +3319,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon maybeCurrentLinkValueReadPropertyInfo: Option[ReadPropertyInfoV2] = if (currentReadPropertyInfo.isLinkProp) { val linkValuePropertyIri = internalPropertyIri.fromLinkPropToLinkValueProp - Some(ontology.properties.getOrElse(linkValuePropertyIri, throw InconsistentTriplestoreDataException(s"Link value property $linkValuePropertyIri not found"))) + Some(ontology.properties.getOrElse(linkValuePropertyIri, throw InconsistentRepositoryDataException(s"Link value property $linkValuePropertyIri not found"))) } else { None } @@ -3367,7 +3368,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) _ = if (loadedPropertyDef != unescapedNewPropertyDef) { - throw InconsistentTriplestoreDataException(s"Attempted to save property definition $unescapedNewPropertyDef, but $loadedPropertyDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save property definition $unescapedNewPropertyDef, but $loadedPropertyDef was saved") } maybeLoadedLinkValuePropertyDefFuture: Option[Future[PropertyInfoContentV2]] = maybeCurrentLinkValueReadPropertyInfo.map { @@ -3387,7 +3388,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) if (loadedLinkValuePropertyDef != unescapedNewLinkPropertyDef) { - throw InconsistentTriplestoreDataException(s"Attempted to save link value property definition $unescapedNewLinkPropertyDef, but $loadedLinkValuePropertyDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save link value property definition $unescapedNewLinkPropertyDef, but $loadedLinkValuePropertyDef was saved") } unescapedNewLinkPropertyDef @@ -3533,7 +3534,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) _ = if (loadedClassDef != unescapedNewClassDef) { - throw InconsistentTriplestoreDataException(s"Attempted to save class definition $unescapedNewClassDef, but $loadedClassDef was saved") + throw InconsistentRepositoryDataException(s"Attempted to save class definition $unescapedNewClassDef, but $loadedClassDef was saved") } // Update the ontology cache, using the unescaped definition(s). @@ -3660,7 +3661,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon case ontoLiteral: OntologyLiteralV2 => ontoLiteral - case other => throw InconsistentTriplestoreDataException(s"Predicate $predicateIri has an invalid object: $other") + case other => throw InconsistentRepositoryDataException(s"Predicate $predicateIri has an invalid object: $other") } ) @@ -3698,7 +3699,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val ontologyIri = propertyIri.getOntologyFromEntity if (!ontologyIri.isKnoraOntologyIri) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri is not in a Knora ontology") + throw InconsistentRepositoryDataException(s"Property $propertyIri is not in a Knora ontology") } val statements = constructResponse.statements @@ -3710,7 +3711,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon case Some(baseProperties) => baseProperties.map { case iriLiteral: IriLiteralV2 => iriLiteral.value.toSmartIri - case other => throw InconsistentTriplestoreDataException(s"Unexpected object for rdfs:subPropertyOf: $other") + case other => throw InconsistentRepositoryDataException(s"Unexpected object for rdfs:subPropertyOf: $other") }.toSet case None => Set.empty[SmartIri] @@ -3720,7 +3721,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon // salsah-gui:guiOrder isn't allowed here. if (otherPreds.contains(OntologyConstants.SalsahGui.GuiOrder.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Property $propertyIri contains salsah-gui:guiOrder") + throw InconsistentRepositoryDataException(s"Property $propertyIri contains salsah-gui:guiOrder") } val propertyDef = PropertyInfoContentV2( @@ -3731,7 +3732,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon ) if (!propertyIri.isKnoraBuiltInDefinitionIri && propertyDef.getRdfTypes.contains(OntologyConstants.Owl.TransitiveProperty.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Project-specific property $propertyIri cannot be an owl:TransitiveProperty") + throw InconsistentRepositoryDataException(s"Project-specific property $propertyIri cannot be an owl:TransitiveProperty") } propertyDef @@ -3831,7 +3832,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val ontologyIri = classIri.getOntologyFromEntity if (!ontologyIri.isKnoraOntologyIri) { - throw InconsistentTriplestoreDataException(s"Class $classIri is not in a Knora ontology") + throw InconsistentRepositoryDataException(s"Class $classIri is not in a Knora ontology") } val statements = constructResponse.statements @@ -3855,20 +3856,20 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon val directCardinalities: Map[SmartIri, KnoraCardinalityInfo] = restrictionBlankNodeIDs.map { blankNodeID => - val blankNode: Map[SmartIri, Seq[LiteralV2]] = statements.getOrElse(BlankNodeSubjectV2(blankNodeID.value), throw InconsistentTriplestoreDataException(s"Blank node '${blankNodeID.value}' not found in construct query result")) + val blankNode: Map[SmartIri, Seq[LiteralV2]] = statements.getOrElse(BlankNodeSubjectV2(blankNodeID.value), throw InconsistentRepositoryDataException(s"Blank node '${blankNodeID.value}' not found in construct query result")) - val blankNodeTypeObjs: Seq[LiteralV2] = blankNode.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentTriplestoreDataException(s"Blank node '${blankNodeID.value}' has no rdf:type")) + val blankNodeTypeObjs: Seq[LiteralV2] = blankNode.getOrElse(OntologyConstants.Rdf.Type.toSmartIri, throw InconsistentRepositoryDataException(s"Blank node '${blankNodeID.value}' has no rdf:type")) blankNodeTypeObjs match { case Seq(IriLiteralV2(OntologyConstants.Owl.Restriction)) => () - case _ => throw InconsistentTriplestoreDataException(s"Blank node '${blankNodeID.value}' is not an owl:Restriction") + case _ => throw InconsistentRepositoryDataException(s"Blank node '${blankNodeID.value}' is not an owl:Restriction") } - val onPropertyObjs: Seq[LiteralV2] = blankNode.getOrElse(OntologyConstants.Owl.OnProperty.toSmartIri, throw InconsistentTriplestoreDataException(s"Blank node '${blankNodeID.value}' has no owl:onProperty")) + val onPropertyObjs: Seq[LiteralV2] = blankNode.getOrElse(OntologyConstants.Owl.OnProperty.toSmartIri, throw InconsistentRepositoryDataException(s"Blank node '${blankNodeID.value}' has no owl:onProperty")) val propertyIri: SmartIri = onPropertyObjs match { case Seq(propertyIri: IriLiteralV2) => propertyIri.value.toSmartIri - case other => throw InconsistentTriplestoreDataException(s"Invalid object for owl:onProperty: $other") + case other => throw InconsistentRepositoryDataException(s"Invalid object for owl:onProperty: $other") } val owlCardinalityPreds: Set[SmartIri] = blankNode.keySet.filter { @@ -3876,30 +3877,30 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon } if (owlCardinalityPreds.size != 1) { - throw InconsistentTriplestoreDataException(s"Expected one cardinality predicate in blank node '${blankNodeID.value}', got ${owlCardinalityPreds.size}") + throw InconsistentRepositoryDataException(s"Expected one cardinality predicate in blank node '${blankNodeID.value}', got ${owlCardinalityPreds.size}") } val owlCardinalityIri = owlCardinalityPreds.head val owlCardinalityValue: Int = blankNode(owlCardinalityIri) match { case Seq(IntLiteralV2(intVal)) => intVal - case other => throw InconsistentTriplestoreDataException(s"Expected one integer object for predicate $owlCardinalityIri in blank node '${blankNodeID.value}', got $other") + case other => throw InconsistentRepositoryDataException(s"Expected one integer object for predicate $owlCardinalityIri in blank node '${blankNodeID.value}', got $other") } val guiOrder: Option[Int] = blankNode.get(OntologyConstants.SalsahGui.GuiOrder.toSmartIri) match { case Some(Seq(IntLiteralV2(intVal))) => Some(intVal) case None => None - case other => throw InconsistentTriplestoreDataException(s"Expected one integer object for predicate ${OntologyConstants.SalsahGui.GuiOrder} in blank node '${blankNodeID.value}', got $other") + case other => throw InconsistentRepositoryDataException(s"Expected one integer object for predicate ${OntologyConstants.SalsahGui.GuiOrder} in blank node '${blankNodeID.value}', got $other") } // salsah-gui:guiElement and salsah-gui:guiAttribute aren't allowed here. if (blankNode.contains(OntologyConstants.SalsahGui.GuiElementProp.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Class $classIri contains salsah-gui:guiElement in an owl:Restriction") + throw InconsistentRepositoryDataException(s"Class $classIri contains salsah-gui:guiElement in an owl:Restriction") } if (blankNode.contains(OntologyConstants.SalsahGui.GuiAttribute.toSmartIri)) { - throw InconsistentTriplestoreDataException(s"Class $classIri contains salsah-gui:guiAttribute in an owl:Restriction") + throw InconsistentRepositoryDataException(s"Class $classIri contains salsah-gui:guiAttribute in an owl:Restriction") } propertyIri -> Cardinality.owlCardinality2KnoraCardinality( @@ -3968,7 +3969,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon // make a map of superproperty IRIs to superproperty constraint values. val superPropertyConstraintValues: Map[SmartIri, SmartIri] = superPropertyInfos.flatMap { superPropertyInfo => - superPropertyInfo.entityInfoContent.predicates.get(constraintPredicateIri).map(_.requireIriObject(throw InconsistentTriplestoreDataException(s"Property ${superPropertyInfo.entityInfoContent.propertyIri} has an invalid object for $constraintPredicateIri"))).map { + superPropertyInfo.entityInfoContent.predicates.get(constraintPredicateIri).map(_.requireIriObject(throw InconsistentRepositoryDataException(s"Property ${superPropertyInfo.entityInfoContent.propertyIri} has an invalid object for $constraintPredicateIri"))).map { superPropertyConstraintValue => superPropertyInfo.entityInfoContent.propertyIri -> superPropertyConstraintValue } }.toMap @@ -4052,7 +4053,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon errorFun } - case None => throw InconsistentTriplestoreDataException(s"Ontology $internalOntologyIri has no ${OntologyConstants.KnoraBase.LastModificationDate}") + case None => throw InconsistentRepositoryDataException(s"Ontology $internalOntologyIri has no ${OntologyConstants.KnoraBase.LastModificationDate}") } case None => throw NotFoundException(s"Ontology $internalOntologyIri (corresponding to ${internalOntologyIri.toOntologySchema(ApiV2Complex)}) not found") @@ -4304,7 +4305,7 @@ class OntologyResponderV2(responderData: ResponderData) extends Responder(respon inheritableCardinalities = cardinalitiesAvailableToInherit, allSubPropertyOfRelations = allSubPropertyOfRelations, errorSchema = InternalSchema, - { msg: String => throw InconsistentTriplestoreDataException(msg) } + { msg: String => throw InconsistentRepositoryDataException(msg) } ) } diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourcesResponderV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourcesResponderV2.scala index 0cd4d6f222..55a34d59a5 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourcesResponderV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourcesResponderV2.scala @@ -41,6 +41,7 @@ import org.knora.webapi.messages.util.search.ConstructQuery import org.knora.webapi.messages.util.search.gravsearch.GravsearchParser import org.knora.webapi.messages.util.standoff.StandoffTagUtilV2 import org.knora.webapi.messages.util._ +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, VariableResultsRow} import org.knora.webapi.messages.v2.responder.ontologymessages._ import org.knora.webapi.messages.v2.responder.resourcemessages._ import org.knora.webapi.messages.v2.responder.searchmessages.GravsearchRequestV2 @@ -231,7 +232,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt resourceClassOntologyIri: SmartIri = createResourceRequestV2.createResource.resourceClassIri.getOntologyFromEntity readOntologyMetadataV2: ReadOntologyMetadataV2 <- (responderManager ? OntologyMetadataGetByIriRequestV2(Set(resourceClassOntologyIri), createResourceRequestV2.requestingUser)).mapTo[ReadOntologyMetadataV2] ontologyMetadata: OntologyMetadataV2 = readOntologyMetadataV2.ontologies.headOption.getOrElse(throw BadRequestException(s"Ontology $resourceClassOntologyIri not found")) - ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentTriplestoreDataException(s"Ontology $resourceClassOntologyIri has no project")).toString + ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentRepositoryDataException(s"Ontology $resourceClassOntologyIri has no project")).toString _ = if (projectIri != ontologyProjectIri && !(ontologyMetadata.ontologyIri.isKnoraBuiltInDefinitionIri || ontologyMetadata.ontologyIri.isKnoraSharedDefinitionIri)) { throw BadRequestException(s"Cannot create a resource in project <$projectIri> with resource class <${createResourceRequestV2.createResource.resourceClassIri}>, which is defined in a non-shared ontology in another project") @@ -468,11 +469,11 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt resourceIri = deleteResourceV2.resourceIri ).toString() - sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = sparqlSelectResponse.results.bindings - _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { + _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentRepositoryDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { throw UpdateNotPerformedException(s"Resource <${deleteResourceV2.resourceIri}> was not marked as deleted. Please report this as a possible bug.") } } yield SuccessResponseV2("Resource marked as deleted") @@ -842,7 +843,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt val propertyIriForObjectClassConstraint = propertyInfoForObjectClassConstraint.entityInfoContent.propertyIri val objectClassConstraint: SmartIri = propertyInfoForObjectClassConstraint.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, - throw InconsistentTriplestoreDataException(s"Property <$propertyIriForObjectClassConstraint> has no knora-api:objectType")) + throw InconsistentRepositoryDataException(s"Property <$propertyIriForObjectClassConstraint> has no knora-api:objectType")) // Check each value. for (valueToCreate: CreateValueInNewResourceV2 <- valuesToCreate) { @@ -1410,11 +1411,11 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt case Some(values: Seq[ReadValueV2]) if values.size == 1 => values.head match { case value: ReadValueV2 => value.valueContent match { case textRepr: TextFileValueContentV2 => (value.valueIri, textRepr) - case _ => throw InconsistentTriplestoreDataException(s"Resource $gravsearchTemplateIri is supposed to have exactly one value of type ${OntologyConstants.KnoraBase.TextFileValue}") + case _ => throw InconsistentRepositoryDataException(s"Resource $gravsearchTemplateIri is supposed to have exactly one value of type ${OntologyConstants.KnoraBase.TextFileValue}") } } - case None => throw InconsistentTriplestoreDataException(s"Resource $gravsearchTemplateIri has no property ${OntologyConstants.KnoraBase.HasTextFileValue}") + case None => throw InconsistentRepositoryDataException(s"Resource $gravsearchTemplateIri has no property ${OntologyConstants.KnoraBase.HasTextFileValue}") } // check if gravsearchFileValueContent represents a text file @@ -1666,6 +1667,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt header = TEIHeader( headerInfo = headerResource, headerXSLT = headerXSLT, + featureFactoryConfig = featureFactoryConfig, settings = settings ), body = TEIBody( @@ -1757,7 +1759,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt // _ = println(sparql) - response: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResponse] + response: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResult] rows: Seq[VariableResultsRow] = response.results.bindings // Did we get any results? @@ -1895,7 +1897,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt // _ = println(sparql) - response: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResponse] + response: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(sparql)).mapTo[SparqlSelectResult] rows: Seq[VariableResultsRow] = response.results.bindings _ = if (rows.isEmpty) { @@ -2005,7 +2007,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt maybeEndDate = resourceHistoryRequest.endDate ).toString() - valueHistoryResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(historyRequestSparql)).mapTo[SparqlSelectResponse] + valueHistoryResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(historyRequestSparql)).mapTo[SparqlSelectResult] valueHistoryEntries: Seq[ResourceHistoryEntry] = valueHistoryResponse.results.bindings.map { row: VariableResultsRow => @@ -2013,7 +2015,7 @@ class ResourcesResponderV2(responderData: ResponderData) extends ResponderWithSt val author: IRI = row.rowMap("author") ResourceHistoryEntry( - versionDate = stringFormatter.xsdDateTimeStampToInstant(versionDateStr, throw InconsistentTriplestoreDataException(s"Could not parse version date: $versionDateStr")), + versionDate = stringFormatter.xsdDateTimeStampToInstant(versionDateStr, throw InconsistentRepositoryDataException(s"Could not parse version date: $versionDateStr")), author = author ) } diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala index ec5cb5bc96..04933e7314 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala @@ -22,12 +22,13 @@ package org.knora.webapi.responders.v2 import akka.http.scaladsl.util.FastFuture import akka.pattern._ import org.knora.webapi._ -import org.knora.webapi.exceptions.{AssertionException, BadRequestException, GravsearchException, InconsistentTriplestoreDataException} +import org.knora.webapi.exceptions.{AssertionException, BadRequestException, GravsearchException, InconsistentRepositoryDataException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.util.ConstructResponseUtilV2.MappingAndXSLTransformation +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, SparqlSelectResultBody, VariableResultsRow} import org.knora.webapi.messages.util.search.gravsearch.GravsearchQueryChecker import org.knora.webapi.messages.util.search.gravsearch.prequery.{AbstractPrequeryGenerator, NonTriplestoreSpecificGravsearchToCountPrequeryTransformer, NonTriplestoreSpecificGravsearchToPrequeryTransformer} import org.knora.webapi.messages.util.search.gravsearch.types._ @@ -100,7 +101,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand // _ = println(countSparql) - countResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(countSparql)).mapTo[SparqlSelectResponse] + countResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(countSparql)).mapTo[SparqlSelectResult] // query response should contain one result with one row with the name "count" _ = if (countResponse.results.bindings.length != 1) { @@ -155,7 +156,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand // _ = println(searchSparql) - prequeryResponseNotMerged: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(searchSparql)).mapTo[SparqlSelectResponse] + prequeryResponseNotMerged: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(searchSparql)).mapTo[SparqlSelectResult] // _ = println(prequeryResponseNotMerged) mainResourceVar = QueryVariable("resource") @@ -341,7 +342,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand // _ = println(triplestoreSpecificCountQuery.toSparql) - countResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(triplestoreSpecificCountQuery.toSparql)).mapTo[SparqlSelectResponse] + countResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(triplestoreSpecificCountQuery.toSparql)).mapTo[SparqlSelectResult] // query response should contain one result with one row with the name "count" _ = if (countResponse.results.bindings.length != 1) { @@ -429,7 +430,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand triplestoreSpecificPrequerySparql = triplestoreSpecificPrequery.toSparql _ = log.debug(triplestoreSpecificPrequerySparql) - prequeryResponseNotMerged: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(triplestoreSpecificPrequerySparql)).mapTo[SparqlSelectResponse] + prequeryResponseNotMerged: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(triplestoreSpecificPrequerySparql)).mapTo[SparqlSelectResult] pageSizeBeforeFiltering: Int = prequeryResponseNotMerged.results.bindings.size // Merge rows with the same main resource IRI. This could happen if there are unbound variables in a UNION. @@ -616,7 +617,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand } // Get the value class that's the object of the knora-base:objectClassConstraint of the ORDER BY property. - val orderByValueType: SmartIri = internalOrderByPropertyDef.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentTriplestoreDataException(s"Property <$internalOrderByPropertyIri> has no knora-base:objectClassConstraint")) + val orderByValueType: SmartIri = internalOrderByPropertyDef.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentRepositoryDataException(s"Property <$internalOrderByPropertyIri> has no knora-base:objectClassConstraint")) // Determine which subproperty of knora-base:valueHas corresponds to that value class. val orderByValuePredicate = orderByValueType.toString match { @@ -647,7 +648,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand offset = resourcesInProjectGetRequestV2.page * settings.v2ResultsPerPage ).toString - sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(prequery)).mapTo[SparqlSelectResponse] + sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(prequery)).mapTo[SparqlSelectResult] mainResourceIris: Seq[IRI] = sparqlSelectResponse.results.bindings.map(_.rowMap("resource")) // Find out whether to query standoff along with text values. This boolean value will be passed to @@ -751,7 +752,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand // _ = println(countSparql) - countResponse: SparqlSelectResponse <- (storeManager ? SparqlSelectRequest(countSparql)).mapTo[SparqlSelectResponse] + countResponse: SparqlSelectResult <- (storeManager ? SparqlSelectRequest(countSparql)).mapTo[SparqlSelectResult] // query response should contain one result with one row with the name "count" _ = if (countResponse.results.bindings.length != 1) { @@ -819,7 +820,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand if (subjectIsMainResource) { val subjIri: IRI = subject match { case IriSubjectV2(value) => value - case other => throw InconsistentTriplestoreDataException(s"Unexpected subject of resource: $other") + case other => throw InconsistentRepositoryDataException(s"Unexpected subject of resource: $other") } acc + subjIri @@ -860,7 +861,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand * @param mainResourceVar the name of the column representing the main resource. * @return the merged results. */ - private def mergePrequeryResults(prequeryResponseNotMerged: SparqlSelectResponse, mainResourceVar: QueryVariable): SparqlSelectResponse = { + private def mergePrequeryResults(prequeryResponseNotMerged: SparqlSelectResult, mainResourceVar: QueryVariable): SparqlSelectResult = { // Make a Map of merged results per main resource IRI. val prequeryRowsMergedMap: Map[IRI, VariableResultsRow] = prequeryResponseNotMerged.results.bindings.groupBy { row => @@ -906,7 +907,7 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand } prequeryResponseNotMerged.copy( - results = SparqlSelectResponseBody(prequeryRowsMerged) + results = SparqlSelectResultBody(prequeryRowsMerged) ) } -} \ No newline at end of file +} diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/StandoffResponderV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/StandoffResponderV2.scala index cc37ec0855..587871416c 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/StandoffResponderV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/StandoffResponderV2.scala @@ -188,11 +188,11 @@ class StandoffResponderV2(responderData: ResponderData) extends Responder(respon case Some(values: Seq[ReadValueV2]) if values.size == 1 => values.head match { case value: ReadValueV2 => value.valueContent match { case textRepr: TextFileValueContentV2 => (value.valueIri, textRepr) - case _ => throw InconsistentTriplestoreDataException(s"${OntologyConstants.KnoraBase.XSLTransformation} $xslTransformationIri is supposed to have exactly one value of type ${OntologyConstants.KnoraBase.TextFileValue}") + case _ => throw InconsistentRepositoryDataException(s"${OntologyConstants.KnoraBase.XSLTransformation} $xslTransformationIri is supposed to have exactly one value of type ${OntologyConstants.KnoraBase.TextFileValue}") } } - case None => throw InconsistentTriplestoreDataException(s"${OntologyConstants.KnoraBase.XSLTransformation} has no property ${OntologyConstants.KnoraBase.HasTextFileValue}") + case None => throw InconsistentRepositoryDataException(s"${OntologyConstants.KnoraBase.XSLTransformation} has no property ${OntologyConstants.KnoraBase.HasTextFileValue}") } // check if xsltFileValueContent represents an XSL transformation diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/ValuesResponderV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/ValuesResponderV2.scala index 7c4cebd385..40bb24b067 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/ValuesResponderV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/ValuesResponderV2.scala @@ -33,6 +33,7 @@ import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.twirl.SparqlTemplateLinkUpdate import org.knora.webapi.messages.util.PermissionUtilADM._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.search.gravsearch.GravsearchParser import org.knora.webapi.messages.util.{KnoraSystemInstances, PermissionUtilADM, ResponderData} import org.knora.webapi.messages.v2.responder.SuccessResponseV2 @@ -181,7 +182,7 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde currentValuesForProp: Seq[ReadValueV2] = resourceInfo.values.getOrElse(submittedInternalPropertyIri, Seq.empty[ReadValueV2]) _ = if ((cardinalityInfo.cardinality == Cardinality.MustHaveOne || cardinalityInfo.cardinality == Cardinality.MustHaveSome) && currentValuesForProp.isEmpty) { - throw InconsistentTriplestoreDataException(s"Resource class <${resourceInfo.resourceClassIri.toOntologySchema(ApiV2Complex)}> has a cardinality of ${cardinalityInfo.cardinality} on property <${createValueRequest.createValue.propertyIri}>, but resource <${createValueRequest.createValue.resourceIri}> has no value for that property") + throw InconsistentRepositoryDataException(s"Resource class <${resourceInfo.resourceClassIri.toOntologySchema(ApiV2Complex)}> has a cardinality of ${cardinalityInfo.cardinality} on property <${createValueRequest.createValue.propertyIri}>, but resource <${createValueRequest.createValue.resourceIri}> has no value for that property") } _ = if (cardinalityInfo.cardinality == Cardinality.MustHaveOne || (cardinalityInfo.cardinality == Cardinality.MayHaveOne && currentValuesForProp.nonEmpty)) { @@ -1513,7 +1514,7 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde classInfoResponse: ReadOntologyV2 <- (responderManager ? classInfoRequest).mapTo[ReadOntologyV2] classInfo: ReadClassInfoV2 = classInfoResponse.classes(resourceInfo.resourceClassIri) - cardinalityInfo: Cardinality.KnoraCardinalityInfo = classInfo.allCardinalities.getOrElse(submittedInternalPropertyIri, throw InconsistentTriplestoreDataException(s"Resource <${deleteValueRequest.resourceIri}> belongs to class <${resourceInfo.resourceClassIri.toOntologySchema(ApiV2Complex)}>, which has no cardinality for property <${deleteValueRequest.propertyIri}>")) + cardinalityInfo: Cardinality.KnoraCardinalityInfo = classInfo.allCardinalities.getOrElse(submittedInternalPropertyIri, throw InconsistentRepositoryDataException(s"Resource <${deleteValueRequest.resourceIri}> belongs to class <${resourceInfo.resourceClassIri.toOntologySchema(ApiV2Complex)}>, which has no cardinality for property <${deleteValueRequest.propertyIri}>")) // Check that the resource class's cardinality for the submitted property allows this value to be deleted. @@ -1549,10 +1550,10 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde valueIri = deletedValueIri ).toString() - sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResponse] + sparqlSelectResponse <- (storeManager ? SparqlSelectRequest(sparqlQuery)).mapTo[SparqlSelectResult] rows = sparqlSelectResponse.results.bindings - _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentTriplestoreDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { + _ = if (rows.isEmpty || !stringFormatter.optionStringToBoolean(rows.head.rowMap.get("isDeleted"), throw InconsistentRepositoryDataException(s"Invalid boolean for isDeleted: ${rows.head.rowMap.get("isDeleted")}"))) { throw UpdateNotPerformedException(s"The request to mark value <${deleteValueRequest.valueIri}> (or a new version of that value) as deleted did not succeed. Please report this as a possible bug.") } } yield SuccessResponseV2(s"Value <$deletedValueIri> marked as deleted") @@ -1839,7 +1840,7 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde requestingUser: UserADM): Future[ReadResourceV2] = { for { // Get the property's object class constraint. - objectClassConstraint: SmartIri <- Future(propertyInfo.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentTriplestoreDataException(s"Property ${propertyInfo.entityInfoContent.propertyIri} has no knora-base:objectClassConstraint"))) + objectClassConstraint: SmartIri <- Future(propertyInfo.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentRepositoryDataException(s"Property ${propertyInfo.entityInfoContent.propertyIri} has no knora-base:objectClassConstraint"))) // If the property points to a text value, also query the resource's standoff links. maybeStandoffLinkToPropertyIri: Option[SmartIri] = if (objectClassConstraint.toString == OntologyConstants.KnoraBase.TextValue) { @@ -2025,7 +2026,7 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde featureFactoryConfig: FeatureFactoryConfig, requestingUser: UserADM): Future[Unit] = { for { - objectClassConstraint: SmartIri <- Future(propertyInfo.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentTriplestoreDataException(s"Property ${propertyInfo.entityInfoContent.propertyIri} has no knora-base:objectClassConstraint"))) + objectClassConstraint: SmartIri <- Future(propertyInfo.entityInfoContent.requireIriObject(OntologyConstants.KnoraBase.ObjectClassConstraint.toSmartIri, throw InconsistentRepositoryDataException(s"Property ${propertyInfo.entityInfoContent.propertyIri} has no knora-base:objectClassConstraint"))) result: Unit <- valueContent match { case linkValueContent: LinkValueContentV2 => @@ -2227,7 +2228,7 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde case None => // We didn't find the LinkValue. This shouldn't happen. - throw InconsistentTriplestoreDataException(s"There should be a knora-base:LinkValue describing a direct link from resource <${sourceResourceInfo.resourceIri}> to resource <$targetResourceIri> using property <$linkPropertyIri>, but it seems to be missing") + throw InconsistentRepositoryDataException(s"There should be a knora-base:LinkValue describing a direct link from resource <${sourceResourceInfo.resourceIri}> to resource <$targetResourceIri> using property <$linkPropertyIri>, but it seems to be missing") } } @@ -2285,7 +2286,7 @@ class ValuesResponderV2(responderData: ResponderData) extends Responder(responde case None => // We didn't find the LinkValue. This shouldn't happen. - throw InconsistentTriplestoreDataException(s"There should be a knora-base:LinkValue describing a direct link from resource <${sourceResourceInfo.resourceIri}> to resource <$targetResourceIri> using property <$linkPropertyIri>, but it seems to be missing") + throw InconsistentRepositoryDataException(s"There should be a knora-base:LinkValue describing a direct link from resource <${sourceResourceInfo.resourceIri}> to resource <$targetResourceIri> using property <$linkPropertyIri>, but it seems to be missing") } } diff --git a/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala b/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala index 17d81fd10b..bbc10d0a8c 100644 --- a/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala @@ -35,7 +35,7 @@ import javax.xml.XMLConstants import javax.xml.transform.stream.StreamSource import javax.xml.validation.{Schema, SchemaFactory, Validator} import org.knora.webapi._ -import org.knora.webapi.exceptions.{AssertionException, BadRequestException, ForbiddenException, InconsistentTriplestoreDataException, SipiException} +import org.knora.webapi.exceptions.{AssertionException, BadRequestException, ForbiddenException, InconsistentRepositoryDataException, SipiException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.StringFormatter.XmlImportNamespaceInfoV1 @@ -161,7 +161,7 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) resourceReferences: Set[IRI] = stringFormatter.getResourceIrisFromStandoffTags(textWithStandoffTags.standoffTagV2) } yield CreateValueV1WithComment(TextValueWithStandoffV1( - utf8str = stringFormatter.toSparqlEncodedString(textWithStandoffTags.text, throw InconsistentTriplestoreDataException("utf8str for TextValue contains invalid characters")), + utf8str = stringFormatter.toSparqlEncodedString(textWithStandoffTags.text, throw InconsistentRepositoryDataException("utf8str for TextValue contains invalid characters")), language = richtext.language, resource_reference = resourceReferences, standoff = textWithStandoffTags.standoffTagV2, @@ -466,7 +466,7 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) ontologyIrisFromObjectClassConstraints: Set[IRI] = entityInfoResponse.propertyInfoMap.map { case (propertyIri, propertyInfo) => val propertyObjectClassConstraint = propertyInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse { - throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") + throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint") } propertyObjectClassConstraint.toSmartIri.getOntologyFromEntity.toString diff --git a/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala b/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala index 3578d63a7f..d9ec608e42 100644 --- a/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala @@ -27,7 +27,7 @@ import akka.http.scaladsl.server.Route import akka.http.scaladsl.util.FastFuture import akka.pattern._ import org.knora.webapi._ -import org.knora.webapi.exceptions.{BadRequestException, InconsistentTriplestoreDataException, NotFoundException} +import org.knora.webapi.exceptions.{BadRequestException, InconsistentRepositoryDataException, NotFoundException} import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants import org.knora.webapi.messages.admin.responder.usersmessages.UserADM @@ -123,7 +123,7 @@ class ValuesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) wit resourceReferences: Set[IRI] = stringFormatter.getResourceIrisFromStandoffTags(textWithStandoffTags.standoffTagV2) } yield (TextValueWithStandoffV1( - utf8str = stringFormatter.toSparqlEncodedString(textWithStandoffTags.text, throw InconsistentTriplestoreDataException("utf8str for for TextValue contains invalid characters")), + utf8str = stringFormatter.toSparqlEncodedString(textWithStandoffTags.text, throw InconsistentRepositoryDataException("utf8str for for TextValue contains invalid characters")), language = textWithStandoffTags.language, resource_reference = resourceReferences, standoff = textWithStandoffTags.standoffTagV2, @@ -231,7 +231,7 @@ class ValuesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) wit resourceReferences: Set[IRI] = stringFormatter.getResourceIrisFromStandoffTags(textWithStandoffTags.standoffTagV2) } yield (TextValueWithStandoffV1( - utf8str = stringFormatter.toSparqlEncodedString(textWithStandoffTags.text, throw InconsistentTriplestoreDataException("utf8str for for TextValue contains invalid characters")), + utf8str = stringFormatter.toSparqlEncodedString(textWithStandoffTags.text, throw InconsistentRepositoryDataException("utf8str for for TextValue contains invalid characters")), language = richtext.language, resource_reference = resourceReferences, standoff = textWithStandoffTags.standoffTagV2, diff --git a/webapi/src/main/scala/org/knora/webapi/store/BUILD.bazel b/webapi/src/main/scala/org/knora/webapi/store/BUILD.bazel index 932e9b9323..b2e2578f05 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/BUILD.bazel +++ b/webapi/src/main/scala/org/knora/webapi/store/BUILD.bazel @@ -10,11 +10,11 @@ scala_library( "//webapi/src/main/scala/org/knora/webapi", "//webapi/src/main/scala/org/knora/webapi/core", "//webapi/src/main/scala/org/knora/webapi/exceptions", + "//webapi/src/main/scala/org/knora/webapi/feature", "//webapi/src/main/scala/org/knora/webapi/instrumentation", "//webapi/src/main/scala/org/knora/webapi/messages", "//webapi/src/main/scala/org/knora/webapi/routing", "//webapi/src/main/scala/org/knora/webapi/settings", - "//webapi/src/main/scala/org/knora/webapi/feature", "//webapi/src/main/scala/org/knora/webapi/util", "@maven//:com_twitter_chill_2_12", "@maven//:com_typesafe_akka_akka_actor_2_12", @@ -41,7 +41,6 @@ scala_library( "@maven//:org_apache_jena_jena_tdb", "@maven//:org_apache_jena_jena_text", "@maven//:org_apache_lucene_lucene_core", - "@maven//:org_eclipse_rdf4j_rdf4j_client", "@maven//:org_slf4j_slf4j_api", "@maven//:redis_clients_jedis", ], diff --git a/webapi/src/main/scala/org/knora/webapi/store/StoreManager.scala b/webapi/src/main/scala/org/knora/webapi/store/StoreManager.scala index 938f0ca23f..ad5c8d9fe9 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/StoreManager.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/StoreManager.scala @@ -23,6 +23,7 @@ import akka.actor._ import akka.event.LoggingReceive import org.knora.webapi.core.{LiveActorMaker, _} import org.knora.webapi.exceptions.UnexpectedMessageException +import org.knora.webapi.feature.{FeatureFactoryConfig, KnoraSettingsFeatureFactoryConfig} import org.knora.webapi.messages.store.cacheservicemessages.CacheServiceRequest import org.knora.webapi.messages.store.sipimessages.IIIFRequest import org.knora.webapi.messages.store.triplestoremessages.TriplestoreRequest @@ -59,10 +60,19 @@ class StoreManager(appActor: ActorRef) extends Actor with ActorLogging { */ protected val settings: KnoraSettingsImpl = KnoraSettings(system) + /** + * The default feature factory configuration. + */ + protected val defaultFeatureFactoryConfig: FeatureFactoryConfig = new KnoraSettingsFeatureFactoryConfig(settings) + /** * Starts the Triplestore Manager Actor */ - protected lazy val triplestoreManager: ActorRef = makeActor(Props(new TriplestoreManager(appActor) with LiveActorMaker).withDispatcher(KnoraDispatchers.KnoraActorDispatcher), TriplestoreManagerActorName) + protected lazy val triplestoreManager: ActorRef = makeActor(Props(new TriplestoreManager( + appActor = appActor, + settings = settings, + defaultFeatureFactoryConfig = defaultFeatureFactoryConfig + ) with LiveActorMaker).withDispatcher(KnoraDispatchers.KnoraActorDispatcher), TriplestoreManagerActorName) /** * Starts the IIIF Manager Actor diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/TriplestoreManager.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/TriplestoreManager.scala index 78e2457323..f8085e604a 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/TriplestoreManager.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/TriplestoreManager.scala @@ -24,9 +24,10 @@ import akka.event.LoggingReceive import akka.routing.FromConfig import org.knora.webapi.core.ActorMaker import org.knora.webapi.exceptions.UnsupportedTriplestoreException +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.store.triplestoremessages.UpdateRepositoryRequest import org.knora.webapi.messages.util.FakeTriplestore -import org.knora.webapi.settings.{KnoraDispatchers, KnoraSettings, TriplestoreTypes, _} +import org.knora.webapi.settings._ import org.knora.webapi.store.triplestore.embedded.JenaTDBActor import org.knora.webapi.store.triplestore.http.HttpTriplestoreConnector import org.knora.webapi.store.triplestore.upgrade.RepositoryUpdater @@ -38,12 +39,15 @@ import scala.concurrent.ExecutionContext * This actor receives messages representing SPARQL requests, and forwards them to instances of one of the configured * triple stores. * - * @param appActor a reference to the main application actor. + * @param appActor a reference to the main application actor. + * @param settings the application settings. + * @param defaultFeatureFactoryConfig the application's default feature factory configuration. */ -class TriplestoreManager(appActor: ActorRef) extends Actor with ActorLogging { +class TriplestoreManager(appActor: ActorRef, + settings: KnoraSettingsImpl, + defaultFeatureFactoryConfig: FeatureFactoryConfig) extends Actor with ActorLogging { this: ActorMaker => - private val settings = KnoraSettings(context.system) protected implicit val executionContext: ExecutionContext = context.system.dispatchers.lookup(KnoraDispatchers.KnoraActorDispatcher) private var storeActorRef: ActorRef = _ @@ -69,7 +73,8 @@ class TriplestoreManager(appActor: ActorRef) extends Actor with ActorLogging { private val repositoryUpdater: RepositoryUpdater = new RepositoryUpdater( system = context.system, appActor = appActor, - settings = settings + settings = settings, + featureFactoryConfig = defaultFeatureFactoryConfig ) override def preStart() { diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/embedded/JenaTDBActor.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/embedded/JenaTDBActor.scala index 29307a21f1..8e793c4bc1 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/embedded/JenaTDBActor.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/embedded/JenaTDBActor.scala @@ -31,16 +31,16 @@ import org.apache.jena.sparql.core.Quad import org.apache.jena.tdb.{TDB, TDBFactory} import org.apache.jena.update.{UpdateAction, UpdateFactory, UpdateRequest} import org.apache.lucene.store._ -import org.knora.webapi._ import org.knora.webapi.exceptions.{TriplestoreInternalException, TriplestoreResponseException, UnexpectedMessageException} import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.util.ErrorHandlingMap +import org.knora.webapi.messages.util.rdf.{SparqlSelectResult, SparqlSelectResultBody, SparqlSelectResultHeader, VariableResultsRow} import org.knora.webapi.settings.KnoraSettings import org.knora.webapi.store.triplestore.RdfDataObjectFactory import org.knora.webapi.util.ActorUtil._ import scala.collection.JavaConverters._ -import scala.concurrent.Future +import scala.concurrent.{ExecutionContextExecutor, Future} /** @@ -49,7 +49,7 @@ import scala.concurrent.Future class JenaTDBActor extends Actor with ActorLogging { private val system = context.system - private implicit val executionContext = system.dispatcher + private implicit val executionContext: ExecutionContextExecutor = system.dispatcher private val settings = KnoraSettings(system) @@ -57,7 +57,6 @@ class JenaTDBActor extends Actor with ActorLogging { private val loadExistingData = settings.tripleStoreConfig.getBoolean("embedded-jena-tdb.loadExistingData") private val storagePath = new File(settings.tripleStoreConfig.getString("embedded-jena-tdb.storage-path")) private val tdbStoragePath = new File(storagePath + "/db") - private val luceneIndexPath = new File(storagePath + "/lucene") private val tsType = settings.triplestoreType @@ -70,11 +69,8 @@ class JenaTDBActor extends Actor with ActorLogging { false } - private val unionGraphModelName = "urn:x-arq:UnionGraph" - - // Jena prefers to have 1 global dataset - lazy val dataset = getDataset + lazy val dataset: Dataset = getDataset var initialized: Boolean = false @@ -129,12 +125,12 @@ class JenaTDBActor extends Actor with ActorLogging { } /** - * Submits a SPARQL query to the embedded Jena TDB store and returns the response as a [[SparqlSelectResponse]]. + * Submits a SPARQL query to the embedded Jena TDB store and returns the response as a [[SparqlSelectResult]]. * * @param queryString the SPARQL request to be submitted. - * @return [[SparqlSelectResponse]]. + * @return [[SparqlSelectResult]]. */ - private def executeSparqlSelectQuery(queryString: String): Future[SparqlSelectResponse] = { + private def executeSparqlSelectQuery(queryString: String): Future[SparqlSelectResult] = { // Start read transaction this.dataset.begin(ReadWrite.READ) @@ -192,9 +188,9 @@ class JenaTDBActor extends Actor with ActorLogging { //println("==>> SparqlSelect End") Future.successful( - SparqlSelectResponse( - SparqlSelectResponseHeader(resultVars.asScala), - SparqlSelectResponseBody(variableResultsRows) + SparqlSelectResult( + SparqlSelectResultHeader(resultVars.asScala), + SparqlSelectResultBody(variableResultsRows) ) ) } catch { diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/http/HttpTriplestoreConnector.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/http/HttpTriplestoreConnector.scala index 20c7d68f51..1260d3b46b 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/http/HttpTriplestoreConnector.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/http/HttpTriplestoreConnector.scala @@ -40,14 +40,14 @@ import org.apache.http.impl.client.{BasicAuthCache, BasicCredentialsProvider, Cl import org.apache.http.message.BasicNameValuePair import org.apache.http.util.EntityUtils import org.apache.http.{Consts, HttpEntity, HttpHost, HttpRequest, NameValuePair} -import org.eclipse.rdf4j import org.knora.webapi._ import org.knora.webapi.exceptions._ +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.instrumentation.InstrumentationSupport +import org.knora.webapi.messages.store.triplestoremessages.SparqlResultProtocol._ import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.util.FakeTriplestore -import org.knora.webapi.messages.util.SparqlResultProtocol._ -import org.knora.webapi.messages.util.rdf.{RdfFeatureFactory, RdfFormatUtil, RdfModel, Statement, Turtle} +import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.settings.{KnoraDispatchers, KnoraSettings, TriplestoreTypes} import org.knora.webapi.store.triplestore.RdfDataObjectFactory import org.knora.webapi.util.ActorUtil._ @@ -187,8 +187,8 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat case SparqlSelectRequest(sparql: String) => try2Message(sender(), sparqlHttpSelect(sparql), log) case sparqlConstructRequest: SparqlConstructRequest => try2Message(sender(), sparqlHttpConstruct(sparqlConstructRequest), log) case sparqlExtendedConstructRequest: SparqlExtendedConstructRequest => try2Message(sender(), sparqlHttpExtendedConstruct(sparqlExtendedConstructRequest), log) - case SparqlConstructFileRequest(sparql: String, graphIri: IRI, outputFile: File) => try2Message(sender(), sparqlHttpConstructFile(sparql, graphIri, outputFile), log) - case NamedGraphFileRequest(graphIri: IRI, outputFile: File) => try2Message(sender(), sparqlHttpGraphFile(graphIri, outputFile), log) + case SparqlConstructFileRequest(sparql: String, graphIri: IRI, outputFile: File, featureFactoryConfig: FeatureFactoryConfig) => try2Message(sender(), sparqlHttpConstructFile(sparql, graphIri, outputFile, featureFactoryConfig), log) + case NamedGraphFileRequest(graphIri: IRI, outputFile: File, featureFactoryConfig: FeatureFactoryConfig) => try2Message(sender(), sparqlHttpGraphFile(graphIri, outputFile, featureFactoryConfig), log) case NamedGraphDataRequest(graphIri: IRI) => try2Message(sender(), sparqlHttpGraphData(graphIri), log) case SparqlUpdateRequest(sparql: String) => try2Message(sender(), sparqlHttpUpdate(sparql), log) case SparqlAskRequest(sparql: String) => try2Message(sender(), sparqlHttpAsk(sparql), log) @@ -198,22 +198,22 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat case HelloTriplestore(msg: String) if msg == triplestoreType => sender ! HelloTriplestore(triplestoreType) case CheckTriplestoreRequest() => try2Message(sender(), checkTriplestore(), log) case SearchIndexUpdateRequest(subjectIri: Option[String]) => try2Message(sender(), updateLuceneIndex(subjectIri), log) - case DownloadRepositoryRequest(outputFile: File) => try2Message(sender(), downloadRepository(outputFile), log) + case DownloadRepositoryRequest(outputFile: File, featureFactoryConfig: FeatureFactoryConfig) => try2Message(sender(), downloadRepository(outputFile, featureFactoryConfig), log) case UploadRepositoryRequest(inputFile: File) => try2Message(sender(), uploadRepository(inputFile), log) case InsertGraphDataContentRequest(graphContent: String, graphName: String) => try2Message(sender(), insertDataGraphRequest(graphContent, graphName), log) case other => sender ! Status.Failure(UnexpectedMessageException(s"Unexpected message $other of type ${other.getClass.getCanonicalName}")) } /** - * Given a SPARQL SELECT query string, runs the query, returning the result as a [[SparqlSelectResponse]]. + * Given a SPARQL SELECT query string, runs the query, returning the result as a [[SparqlSelectResult]]. * * @param sparql the SPARQL SELECT query string. - * @return a [[SparqlSelectResponse]]. + * @return a [[SparqlSelectResult]]. */ - private def sparqlHttpSelect(sparql: String): Try[SparqlSelectResponse] = { - def parseJsonResponse(sparql: String, resultStr: String): Try[SparqlSelectResponse] = { + private def sparqlHttpSelect(sparql: String): Try[SparqlSelectResult] = { + def parseJsonResponse(sparql: String, resultStr: String): Try[SparqlSelectResult] = { val parseTry = Try { - resultStr.parseJson.convertTo[SparqlSelectResponse] + resultStr.parseJson.convertTo[SparqlSelectResult] } parseTry match { @@ -247,6 +247,7 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat responseMessage <- parseJsonResponse(sparql, resultStr) } yield responseMessage } + /** * Given a SPARQL CONSTRUCT query string, runs the query, returning the result as a [[SparqlConstructResponse]]. * @@ -263,7 +264,7 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat val rdfModel: RdfModel = rdfFormatUtil.parseToRdfModel(rdfStr = turtleStr, rdfFormat = Turtle) val statementMap: mutable.Map[IRI, Seq[(IRI, String)]] = mutable.Map.empty - for (st: Statement <- rdfModel.getStatements) { + for (st: Statement <- rdfModel) { val subjectIri = st.subj.stringValue val predicateIri = st.pred.stringValue val objectIri = st.obj.stringValue @@ -294,56 +295,79 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat } /** - * Adds a named graph to CONSTRUCT query results. + * Adds a context IRI to RDF statements. * - * @param graphIri the IRI of the named graph. - * @param rdfWriter an [[rdf4j.rio.RDFWriter]] for writing the result. + * @param graphIri the IRI of the named graph. + * @param formattingStreamProcessor an [[RdfStreamProcessor]] for writing the result. + * @param featureFactoryConfig the feature factory configuration. */ - private class ConstructToGraphHandler(graphIri: IRI, rdfWriter: rdf4j.rio.RDFWriter) extends rdf4j.rio.RDFHandler { - private val valueFactory: rdf4j.model.impl.SimpleValueFactory = rdf4j.model.impl.SimpleValueFactory.getInstance() - private val context: rdf4j.model.Resource = valueFactory.createIRI(graphIri) - - override def startRDF(): Unit = rdfWriter.startRDF() + private class ContextAddingProcessor(graphIri: IRI, + formattingStreamProcessor: RdfStreamProcessor, + featureFactoryConfig: FeatureFactoryConfig) extends RdfStreamProcessor { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) - override def endRDF(): Unit = rdfWriter.endRDF() + override def start(): Unit = formattingStreamProcessor.start() - override def handleNamespace(prefix: IRI, uri: IRI): Unit = rdfWriter.handleNamespace(prefix, uri) + override def processNamespace(prefix: String, namespace: IRI): Unit = { + formattingStreamProcessor.processNamespace(prefix = prefix, namespace = namespace) + } - override def handleStatement(st: rdf4j.model.Statement): Unit = { - val outputStatement = valueFactory.createStatement( - st.getSubject, - st.getPredicate, - st.getObject, - context + override def processStatement(statement: Statement): Unit = { + val outputStatement = nodeFactory.makeStatement( + subj = statement.subj, + pred = statement.pred, + obj = statement.obj, + context = Some(graphIri) ) - rdfWriter.handleStatement(outputStatement) + formattingStreamProcessor.processStatement(outputStatement) } - override def handleComment(comment: IRI): Unit = rdfWriter.handleComment(comment) + override def finish(): Unit = formattingStreamProcessor.finish() } /** - * Saves a graph in Turtle format to a file in TriG format. + * Reads RDF data in Turtle format from an [[RdfSource]], adds a named graph IRI to each statement, + * and writes the result to a file in TriG format. * - * @param input the Turtle data. - * @param outputFile the output file. + * @param rdfSource the RDF data source. + * @param graphIri the named graph IRI to be added. + * @param outputFile the output file. + * @param featureFactoryConfig the feature factory configuration. */ - private def turtleToTrig(input: Reader, graphIri: IRI, outputFile: File): Unit = { - // TODO: Provide a streaming API in RdfFormatUtil for this. + private def turtleToTrig(rdfSource: RdfSource, + graphIri: IRI, + outputFile: File, + featureFactoryConfig: FeatureFactoryConfig): Unit = { + val rdfFormatUtil: RdfFormatUtil = RdfFeatureFactory.getRdfFormatUtil(featureFactoryConfig) + var maybeBufferedFileOutputStream: Option[BufferedOutputStream] = None + + val parseTry: Try[Unit] = Try { + maybeBufferedFileOutputStream = Some(new BufferedOutputStream(new FileOutputStream(outputFile))) + + val formattingStreamProcessor: RdfStreamProcessor = rdfFormatUtil.makeFormattingStreamProcessor( + outputStream = maybeBufferedFileOutputStream.get, + rdfFormat = TriG + ) + + val contextAddingProcessor = new ContextAddingProcessor( + graphIri = graphIri, + formattingStreamProcessor = formattingStreamProcessor, + featureFactoryConfig = featureFactoryConfig + ) + + rdfFormatUtil.parseWithStreamProcessor( + rdfSource = rdfSource, + rdfFormat = Turtle, + rdfStreamProcessor = contextAddingProcessor + ) + } - var maybeBufferedFileWriter: Option[BufferedWriter] = None + maybeBufferedFileOutputStream.foreach(_.close) - try { - maybeBufferedFileWriter = Some(new BufferedWriter(new FileWriter(outputFile))) - val turtleParser = rdf4j.rio.Rio.createParser(rdf4j.rio.RDFFormat.TURTLE) - val trigFileWriter: rdf4j.rio.RDFWriter = rdf4j.rio.Rio.createWriter(rdf4j.rio.RDFFormat.TRIG, maybeBufferedFileWriter.get) - val constructToGraphHandler = new ConstructToGraphHandler(graphIri = graphIri, rdfWriter = trigFileWriter) - turtleParser.setRDFHandler(constructToGraphHandler) - turtleParser.parse(input, "") - } finally { - maybeBufferedFileWriter.foreach(_.close) - input.close() + parseTry match { + case Success(_) => () + case Failure(ex) => throw ex } } @@ -355,10 +379,19 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat * @param outputFile the output file. * @return a [[FileWrittenResponse]]. */ - private def sparqlHttpConstructFile(sparql: String, graphIri: IRI, outputFile: File): Try[FileWrittenResponse] = { + private def sparqlHttpConstructFile(sparql: String, + graphIri: IRI, + outputFile: File, + featureFactoryConfig: FeatureFactoryConfig): Try[FileWrittenResponse] = { for { turtleStr <- getSparqlHttpResponse(sparql, isUpdate = false, acceptMimeType = mimeTypeTextTurtle) - _ = turtleToTrig(input = new StringReader(turtleStr), graphIri = graphIri, outputFile = outputFile) + + _ = turtleToTrig( + rdfSource = RdfStringSource(turtleStr), + graphIri = graphIri, + outputFile = outputFile, + featureFactoryConfig = featureFactoryConfig + ) } yield FileWrittenResponse() } @@ -589,18 +622,16 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat val httpGet = new HttpGet(checkRepositoryPath) httpGet.addHeader("Accept", mimeTypeApplicationJson) - val responseStr = { + val responseStr: String = { var maybeResponse: Option[CloseableHttpResponse] = None - try { + val responseTry: Try[String] = Try { maybeResponse = Some(queryHttpClient.execute(targetHost, httpGet, context)) EntityUtils.toString(maybeResponse.get.getEntity) - } finally { - maybeResponse match { - case Some(response) => response.close() - case None => () - } } + + maybeResponse.foreach(_.close()) + responseTry.get } val nameShouldBe = settings.triplestoreDatabaseName @@ -687,27 +718,25 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat val httpGet = new HttpGet(checkRepositoryPath) httpGet.addHeader("Accept", mimeTypeApplicationJson) - val responseStr = { + val responseStr: String = { var maybeResponse: Option[CloseableHttpResponse] = None - try { + val responseTry: Try[String] = Try { maybeResponse = Some(queryHttpClient.execute(targetHost, httpGet, context)) EntityUtils.toString(maybeResponse.get.getEntity) - } finally { - maybeResponse match { - case Some(response) => response.close() - case None => () - } } + + maybeResponse.foreach(_.close()) + responseTry.get } - val jsonArr = JsonParser(responseStr).asInstanceOf[JsArray] + val jsonArr: JsArray = JsonParser(responseStr).asInstanceOf[JsArray] // parse json and check if the repository defined in 'application.conf' is present and correctly defined val repositories: Seq[GraphDBRepository] = jsonArr.elements.map(_.convertTo[GraphDBRepository]) - val idShouldBe = settings.triplestoreDatabaseName + val idShouldBe: String = settings.triplestoreDatabaseName val sesameTypeForSEShouldBe = "owlim:MonitorRepository" val sesameTypeForFreeShouldBe = "graphdb:FreeSailRepository" @@ -750,15 +779,24 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat /** * Requests the contents of a named graph in TriG format, saving the response in a file. * - * @param graphIri the IRI of the named graph. - * @param outputFile the file to be written. + * @param graphIri the IRI of the named graph. + * @param outputFile the file to be written. + * @param featureFactoryConfig the feature factory configuration. * @return a string containing the contents of the graph in TriG format. */ - private def sparqlHttpGraphFile(graphIri: IRI, outputFile: File): Try[FileWrittenResponse] = { + private def sparqlHttpGraphFile(graphIri: IRI, + outputFile: File, + featureFactoryConfig: FeatureFactoryConfig): Try[FileWrittenResponse] = { val httpContext: HttpClientContext = makeHttpContext val httpGet = new HttpGet(makeNamedGraphDownloadUri(graphIri)) httpGet.addHeader("Accept", mimeTypeTextTurtle) - val makeResponse: CloseableHttpResponse => FileWrittenResponse = writeResponseFile(outputFile, Some(graphIri), convertToTrig = true) + + val makeResponse: CloseableHttpResponse => FileWrittenResponse = writeResponseFile( + outputFile = outputFile, + featureFactoryConfig = featureFactoryConfig, + maybeGraphIri = Some(graphIri), + convertToTrig = true + ) doHttpRequest( client = queryHttpClient, @@ -796,7 +834,9 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat * @param acceptMimeType the MIME type to be provided in the HTTP Accept header. * @return the triplestore's response. */ - private def getSparqlHttpResponse(sparql: String, isUpdate: Boolean, acceptMimeType: String = mimeTypeApplicationSparqlResultsJson): Try[String] = { + private def getSparqlHttpResponse(sparql: String, + isUpdate: Boolean, + acceptMimeType: String = mimeTypeApplicationSparqlResultsJson): Try[String] = { val httpContext: HttpClientContext = makeHttpContext @@ -835,9 +875,11 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat /** * Dumps the whole repository in TriG format, saving the response in a file. * + * @param outputFile the output file. + * @param featureFactoryConfig the feature factory configuration. * @return a string containing the contents of the graph in TriG format. */ - private def downloadRepository(outputFile: File): Try[FileWrittenResponse] = { + private def downloadRepository(outputFile: File, featureFactoryConfig: FeatureFactoryConfig): Try[FileWrittenResponse] = { val httpContext: HttpClientContext = makeHttpContext val uriBuilder: URIBuilder = new URIBuilder(repositoryDownloadPath) @@ -864,7 +906,11 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat .setDefaultCredentialsProvider(credsProvider) .setDefaultRequestConfig(queryRequestConfig) .build - val makeResponse: CloseableHttpResponse => FileWrittenResponse = writeResponseFile(outputFile) + + val makeResponse: CloseableHttpResponse => FileWrittenResponse = writeResponseFile( + outputFile = outputFile, + featureFactoryConfig = featureFactoryConfig + ) doHttpRequest( client = queryHttpClient, @@ -1024,20 +1070,11 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat } def writeResponseFile(outputFile: File, + featureFactoryConfig: FeatureFactoryConfig, maybeGraphIri: Option[IRI] = None, convertToTrig: Boolean = false)(response: CloseableHttpResponse): FileWrittenResponse = { Option(response.getEntity) match { - case None => - if (maybeGraphIri.nonEmpty) { - log.error(s"Triplestore returned no content for graph ${maybeGraphIri.get}") - throw TriplestoreResponseException(s"Triplestore returned no content for graph ${maybeGraphIri.get}") - } else { - log.error(s"Triplestore returned no content for repository dump") - throw TriplestoreResponseException(s"Triplestore returned no content for for repository dump") - } case Some(responseEntity: HttpEntity) => - // TODO: Provide a streaming API in RdfFormatUtil for this. - // Stream the HTTP entity to the output file. if (convertToTrig) { // Stream the HTTP entity to the temporary .ttl file. @@ -1045,15 +1082,41 @@ class HttpTriplestoreConnector extends Actor with ActorLogging with Instrumentat Files.copy(responseEntity.getContent, Paths.get(turtleFile.getCanonicalPath)) // Convert the Turtle to Trig. - val bufferedReader = new BufferedReader(new FileReader(turtleFile)) - turtleToTrig(input = bufferedReader, graphIri = maybeGraphIri.get, outputFile = outputFile) + + var maybeBufferedInputStream: Option[BufferedInputStream] = None + + val processFileTry: Try[Unit] = Try { + maybeBufferedInputStream = Some(new BufferedInputStream(new FileInputStream(turtleFile))) + + turtleToTrig( + rdfSource = RdfInputStreamSource(maybeBufferedInputStream.get), + graphIri = maybeGraphIri.get, + outputFile = outputFile, + featureFactoryConfig = featureFactoryConfig + ) + } + + maybeBufferedInputStream.foreach(_.close()) turtleFile.delete() + processFileTry match { + case Success(_) => () + case Failure(ex) => throw ex + } } else { Files.copy(responseEntity.getContent, Paths.get(outputFile.getCanonicalPath)) } FileWrittenResponse() + + case None => + if (maybeGraphIri.nonEmpty) { + log.error(s"Triplestore returned no content for graph ${maybeGraphIri.get}") + throw TriplestoreResponseException(s"Triplestore returned no content for graph ${maybeGraphIri.get}") + } else { + log.error(s"Triplestore returned no content for repository dump") + throw TriplestoreResponseException(s"Triplestore returned no content for for repository dump") + } } } } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdatePlan.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdatePlan.scala index 843ad60601..7076529815 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdatePlan.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdatePlan.scala @@ -1,24 +1,28 @@ package org.knora.webapi.store.triplestore.upgrade -import org.knora.webapi.store.triplestore.upgrade.plugins.{NoopPlugin, UpgradePluginPR1307, UpgradePluginPR1322, UpgradePluginPR1367, UpgradePluginPR1372, UpgradePluginPR1615, UpgradePluginPR1746} +import com.typesafe.scalalogging.Logger +import org.knora.webapi.feature.FeatureFactoryConfig +import org.knora.webapi.store.triplestore.upgrade.plugins._ /** * The plan for updating a repository to work with the current version of Knora. */ object RepositoryUpdatePlan { /** - * A list of all repository update plugins in chronological order. + * Constructs list of all repository update plugins in chronological order. + * + * @param featureFactoryConfig the feature factor configuration. */ - val pluginsForVersions: Seq[PluginForKnoraBaseVersion] = Seq( - PluginForKnoraBaseVersion(versionNumber = 1, plugin = new UpgradePluginPR1307, prBasedVersionString = Some("PR 1307")), - PluginForKnoraBaseVersion(versionNumber = 2, plugin = new UpgradePluginPR1322, prBasedVersionString = Some("PR 1322")), - PluginForKnoraBaseVersion(versionNumber = 3, plugin = new UpgradePluginPR1367, prBasedVersionString = Some("PR 1367")), - PluginForKnoraBaseVersion(versionNumber = 4, plugin = new UpgradePluginPR1372, prBasedVersionString = Some("PR 1372")), + def makePluginsForVersions(featureFactoryConfig: FeatureFactoryConfig, log: Logger): Seq[PluginForKnoraBaseVersion] = Seq( + PluginForKnoraBaseVersion(versionNumber = 1, plugin = new UpgradePluginPR1307(featureFactoryConfig), prBasedVersionString = Some("PR 1307")), + PluginForKnoraBaseVersion(versionNumber = 2, plugin = new UpgradePluginPR1322(featureFactoryConfig), prBasedVersionString = Some("PR 1322")), + PluginForKnoraBaseVersion(versionNumber = 3, plugin = new UpgradePluginPR1367(featureFactoryConfig), prBasedVersionString = Some("PR 1367")), + PluginForKnoraBaseVersion(versionNumber = 4, plugin = new UpgradePluginPR1372(featureFactoryConfig), prBasedVersionString = Some("PR 1372")), PluginForKnoraBaseVersion(versionNumber = 5, plugin = new NoopPlugin, prBasedVersionString = Some("PR 1440")), PluginForKnoraBaseVersion(versionNumber = 6, plugin = new NoopPlugin), // PR 1206 PluginForKnoraBaseVersion(versionNumber = 7, plugin = new NoopPlugin), // PR 1403 - PluginForKnoraBaseVersion(versionNumber = 8, plugin = new UpgradePluginPR1615), - PluginForKnoraBaseVersion(versionNumber = 9, plugin = new UpgradePluginPR1746) + PluginForKnoraBaseVersion(versionNumber = 8, plugin = new UpgradePluginPR1615(featureFactoryConfig)), + PluginForKnoraBaseVersion(versionNumber = 9, plugin = new UpgradePluginPR1746(featureFactoryConfig, log)) ) /** diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdater.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdater.scala index f997a4c0f4..af7b23b877 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdater.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/RepositoryUpdater.scala @@ -8,31 +8,35 @@ import akka.http.scaladsl.util.FastFuture import akka.pattern._ import akka.util.Timeout import com.typesafe.scalalogging.{LazyLogging, Logger} -import org.eclipse.rdf4j.model.impl.{LinkedHashModel, SimpleValueFactory} -import org.eclipse.rdf4j.model.{Model, Statement} -import org.eclipse.rdf4j.rio.helpers.StatementCollector -import org.eclipse.rdf4j.rio.{RDFFormat, RDFParser, Rio} -import org.knora.webapi.exceptions.InconsistentTriplestoreDataException +import org.knora.webapi.IRI +import org.knora.webapi.exceptions.InconsistentRepositoryDataException +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.store.triplestore.upgrade.RepositoryUpdatePlan.PluginForKnoraBaseVersion import org.knora.webapi.util.FileUtil import org.knora.webapi.settings.{KnoraDispatchers, KnoraSettingsImpl} import org.knora.webapi.messages.StringFormatter +import org.knora.webapi.messages.util.rdf._ -import scala.collection.JavaConverters._ import scala.concurrent.{ExecutionContext, Future} /** * Updates a Knora repository to work with the current version of Knora. * - * @param system the Akka [[ActorSystem]]. - * @param appActor a reference to the main application actor. - * @param settings the Knora application settings. + * @param system the Akka [[ActorSystem]]. + * @param appActor a reference to the main application actor. + * @param featureFactoryConfig the feature factory configuration. + * @param settings the Knora application settings. */ class RepositoryUpdater(system: ActorSystem, appActor: ActorRef, + featureFactoryConfig: FeatureFactoryConfig, settings: KnoraSettingsImpl) extends LazyLogging { + // RDF factories. + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) + private val rdfFormatUtil: RdfFormatUtil = RdfFeatureFactory.getRdfFormatUtil(featureFactoryConfig) + // A SPARQL query to find out the knora-base version in a repository. private val knoraBaseVersionQuery = """PREFIX knora-base: | @@ -60,15 +64,13 @@ class RepositoryUpdater(system: ActorSystem, */ private val log: Logger = logger - /** - * Constructs RDF4J values. - */ - private val valueFactory: SimpleValueFactory = SimpleValueFactory.getInstance - /** * A map of version strings to plugins. */ - private val pluginsForVersionsMap: Map[String, PluginForKnoraBaseVersion] = RepositoryUpdatePlan.pluginsForVersions.map { + private val pluginsForVersionsMap: Map[String, PluginForKnoraBaseVersion] = RepositoryUpdatePlan.makePluginsForVersions( + featureFactoryConfig = featureFactoryConfig, + log = log + ).map { knoraBaseVersion => knoraBaseVersion.versionString -> knoraBaseVersion }.toMap @@ -106,7 +108,7 @@ class RepositoryUpdater(system: ActorSystem, */ private def getRepositoryVersion: Future[Option[String]] = { for { - repositoryVersionResponse: SparqlSelectResponse <- (appActor ? SparqlSelectRequest(knoraBaseVersionQuery)).mapTo[SparqlSelectResponse] + repositoryVersionResponse: SparqlSelectResult <- (appActor ? SparqlSelectRequest(knoraBaseVersionQuery)).mapTo[SparqlSelectResult] bindings = repositoryVersionResponse.results.bindings @@ -130,14 +132,14 @@ class RepositoryUpdater(system: ActorSystem, // The repository has a version string. Get the plugins for all subsequent versions. val pluginForRepositoryVersion: PluginForKnoraBaseVersion = pluginsForVersionsMap.getOrElse( repositoryVersion, - throw InconsistentTriplestoreDataException(s"No such repository version $repositoryVersion") + throw InconsistentRepositoryDataException(s"No such repository version $repositoryVersion") ) - RepositoryUpdatePlan.pluginsForVersions.filter(_.versionNumber > pluginForRepositoryVersion.versionNumber) + pluginsForVersionsMap.values.filter(_.versionNumber > pluginForRepositoryVersion.versionNumber).toSeq case None => // The repository has no version string. Include all updates. - RepositoryUpdatePlan.pluginsForVersions + pluginsForVersionsMap.values.toSeq } } @@ -171,7 +173,10 @@ class RepositoryUpdater(system: ActorSystem, for { // Ask the store actor to download the repository to the file. - _: FileWrittenResponse <- (appActor ? DownloadRepositoryRequest(downloadedRepositoryFile)).mapTo[FileWrittenResponse] + _: FileWrittenResponse <- (appActor ? DownloadRepositoryRequest( + outputFile = downloadedRepositoryFile, + featureFactoryConfig = featureFactoryConfig + )).mapTo[FileWrittenResponse] // Run the transformations to produce an output file. _ = doTransformations( @@ -206,7 +211,7 @@ class RepositoryUpdater(system: ActorSystem, pluginsForNeededUpdates: Seq[PluginForKnoraBaseVersion]): Unit = { // Parse the input file. log.info("Reading repository file...") - val model = readFileIntoModel(downloadedRepositoryFile, RDFFormat.TRIG) + val model = readFileIntoModel(file = downloadedRepositoryFile, rdfFormat = TriG) log.info(s"Read ${model.size} statements.") // Run the update plugins. @@ -221,80 +226,88 @@ class RepositoryUpdater(system: ActorSystem, // Write the output file. log.info(s"Writing output file (${model.size} statements)...") - val fileWriter = new FileWriter(transformedRepositoryFile) - val bufferedWriter = new BufferedWriter(fileWriter) - Rio.write(model, fileWriter, RDFFormat.TRIG) - bufferedWriter.close() - fileWriter.close() + writeModelToFile( + rdfModel = model, + file = transformedRepositoryFile, + rdfFormat = TriG + ) } /** - * Adds Knora's built-in named graphs to a [[Model]]. + * Adds Knora's built-in named graphs to an [[RdfModel]]. * - * @param model the [[Model]]. + * @param model the [[RdfModel]]. */ - private def addBuiltInNamedGraphsToModel(model: Model): Unit = { + private def addBuiltInNamedGraphsToModel(model: RdfModel): Unit = { + // Add each built-in named graph to the model. for (builtInNamedGraph <- RepositoryUpdatePlan.builtInNamedGraphs) { - val context = valueFactory.createIRI(builtInNamedGraph.iri) - model.remove(null, null, null, context) - - val namedGraphModel: Model = readResourceIntoModel(builtInNamedGraph.filename, RDFFormat.TURTLE) - - // Set the context on each statement. - for (statement: Statement <- namedGraphModel.asScala.toSet) { - namedGraphModel.remove( - statement.getSubject, - statement.getPredicate, - statement.getObject, - statement.getContext - ) + val context: IRI = builtInNamedGraph.iri + + // Remove the existing named graph from the model. + model.remove( + subj = None, + pred = None, + obj = None, + context = Some(context) + ) + + // Read the current named graph from a file. + val namedGraphModel: RdfModel = readResourceIntoModel(builtInNamedGraph.filename, Turtle) - namedGraphModel.add( - statement.getSubject, - statement.getPredicate, - statement.getObject, - context + // Copy it into the model, adding the named graph IRI to each statement. + for (statement: Statement <- namedGraphModel) { + model.add( + subj = statement.subj, + pred = statement.pred, + obj = statement.obj, + context = Some(context) ) } - - model.addAll(namedGraphModel) } } /** - * Reads an RDF file into a [[Model]]. + * Reads an RDF file into an [[RdfModel]]. * * @param file the file. - * @param format the file format. - * @return a [[Model]] representing the contents of the file. + * @param rdfFormat the file format. + * @return a [[RdfModel]] representing the contents of the file. */ - def readFileIntoModel(file: File, format: RDFFormat): Model = { - val fileReader = new FileReader(file) - val bufferedReader = new BufferedReader(fileReader) - val model = new LinkedHashModel() - val trigParser: RDFParser = Rio.createParser(format) - trigParser.setRDFHandler(new StatementCollector(model)) - trigParser.parse(bufferedReader, "") - fileReader.close() - bufferedReader.close() - model + def readFileIntoModel(file: File, rdfFormat: NonJsonLD): RdfModel = { + val fileInputStream = new BufferedInputStream(new FileInputStream(file)) + val rdfModel: RdfModel = rdfFormatUtil.inputStreamToRdfModel(inputStream = fileInputStream, rdfFormat = rdfFormat) + fileInputStream.close() + rdfModel } /** - * Reads a file from the CLASSPATH into a [[Model]]. + * Reads a file from the CLASSPATH into an [[RdfModel]]. * * @param filename the filename. - * @param format the file format. - * @return a [[Model]] representing the contents of the file. + * @param rdfFormat the file format. + * @return an [[RdfModel]] representing the contents of the file. */ - def readResourceIntoModel(filename: String, format: RDFFormat): Model = { + def readResourceIntoModel(filename: String, rdfFormat: NonJsonLD): RdfModel = { val fileContent: String = FileUtil.readTextResource(filename) - val stringReader = new StringReader(fileContent) - val model = new LinkedHashModel() - val trigParser: RDFParser = Rio.createParser(format) - trigParser.setRDFHandler(new StatementCollector(model)) - trigParser.parse(stringReader, "") - model + rdfFormatUtil.parseToRdfModel(fileContent, rdfFormat) } -} + /** + * Writes an [[RdfModel]] to a file. + * + * @param rdfModel the model to be written. + * @param file the file. + * @param rdfFormat the file format. + */ + def writeModelToFile(rdfModel: RdfModel, file: File, rdfFormat: NonJsonLD): Unit = { + val fileOutputStream = new BufferedOutputStream(new FileOutputStream(file)) + + rdfFormatUtil.rdfModelToOutputStream( + rdfModel = rdfModel, + outputStream = fileOutputStream, + rdfFormat = rdfFormat + ) + + fileOutputStream.close() + } +} diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/UpgradePlugin.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/UpgradePlugin.scala index bee775f7c2..21880e5854 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/UpgradePlugin.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/UpgradePlugin.scala @@ -1,6 +1,7 @@ package org.knora.webapi.store.triplestore.upgrade -import org.eclipse.rdf4j.model.Model +import org.knora.webapi.messages.util.rdf.RdfModel + /** * A trait for plugins that update a repository. @@ -9,7 +10,7 @@ trait UpgradePlugin { /** * Transforms a repository. * - * @param model a [[Model]] containing the repository data. + * @param model an [[RdfModel]] containing the repository data. */ - def transform(model: Model): Unit + def transform(model: RdfModel): Unit } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/NoopPlugin.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/NoopPlugin.scala index 51ec6e6ab9..eb2c7e793a 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/NoopPlugin.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/NoopPlugin.scala @@ -19,12 +19,12 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.Model +import org.knora.webapi.messages.util.rdf.RdfModel import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin /** * An update plugin that does nothing. Used for updates in which only the built-in named graphs have changed. */ class NoopPlugin extends UpgradePlugin { - override def transform(model: Model): Unit = {} + override def transform(model: RdfModel): Unit = {} } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307.scala index 9338dbf0e1..07d1cc2c75 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307.scala @@ -19,30 +19,27 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.impl.SimpleValueFactory -import org.eclipse.rdf4j.model.util.Models -import org.eclipse.rdf4j.model.vocabulary.RDF -import org.eclipse.rdf4j.model.{IRI, Model, Statement, Value} -import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin -import org.knora.webapi.util.JavaUtil._ -import org.knora.webapi.exceptions.InconsistentTriplestoreDataException +import org.knora.webapi.IRI +import org.knora.webapi.exceptions.InconsistentRepositoryDataException +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants - -import scala.collection.JavaConverters._ +import org.knora.webapi.messages.util.rdf._ +import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin /** * Transforms a repository for Knora PR 1307. */ -class UpgradePluginPR1307 extends UpgradePlugin { - private val valueFactory = SimpleValueFactory.getInstance - - // RDF4J IRI objects representing the IRIs used in this transformation. - private val TextValueIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.TextValue) - private val ValueHasStandoffIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.ValueHasStandoff) - private val StandoffTagHasStartIndexIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.StandoffTagHasStartIndex) - private val StandoffTagHasStartParentIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.StandoffTagHasStartParent) - private val StandoffTagHasEndParentIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.StandoffTagHasEndParent) - private val ValueHasMaxStandoffStartIndexIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.ValueHasMaxStandoffStartIndex) +class UpgradePluginPR1307(featureFactoryConfig: FeatureFactoryConfig) extends UpgradePlugin { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) + + // IRI objects representing the IRIs used in this transformation. + private val rdfTypeIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.Rdf.Type) + private val TextValueIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.TextValue) + private val ValueHasStandoffIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueHasStandoff) + private val StandoffTagHasStartIndexIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.StandoffTagHasStartIndex) + private val StandoffTagHasStartParentIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.StandoffTagHasStartParent) + private val StandoffTagHasEndParentIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.StandoffTagHasEndParent) + private val ValueHasMaxStandoffStartIndexIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueHasMaxStandoffStartIndex) /** * Represents a standoff tag to be transformed. @@ -50,49 +47,53 @@ class UpgradePluginPR1307 extends UpgradePlugin { * @param oldIri the tag's old IRI. * @param statements the statements about the tag. */ - case class StandoffRdf(oldIri: IRI, statements: Model) { + case class StandoffRdf(oldIri: IriNode, statements: Set[Statement]) { + def notFound = throw InconsistentRepositoryDataException(s"$oldIri does not have knora-base:standoffTagHasStartIndex with an integer object") + /** * The value of knora-base:standoffTagHasStartIndex. */ - val startIndex: Int = Models.getPropertyLiteral(statements, oldIri, StandoffTagHasStartIndexIri).toOption match { - case Some(index) => index.intValue - case None => throw InconsistentTriplestoreDataException(s"$oldIri has no knora-base:standoffTagHasStartIndex") + val startIndex: Int = statements.find { + statement => statement.subj == oldIri && statement.pred == StandoffTagHasStartIndexIri + } match { + case Some(statement: Statement) => + statement.obj match { + case datatypeLiteral: DatatypeLiteral => datatypeLiteral.integerValue(notFound).toInt + case _ => notFound + } + + case None => notFound } /** * The tag's new IRI. */ - lazy val newIri: IRI = { + lazy val newIri: IriNode = { val oldSubjStr: String = oldIri.stringValue val slashPos: Int = oldSubjStr.lastIndexOf('/') - valueFactory.createIRI(oldSubjStr.substring(0, slashPos + 1) + startIndex.toString) + nodeFactory.makeIriNode(oldSubjStr.substring(0, slashPos + 1) + startIndex.toString) } - def transform(model: Model, standoff: Map[IRI, StandoffRdf]): Unit = { - for (statement: Statement <- statements.asScala.toSet) { + def transform(model: RdfModel, standoff: Map[IriNode, StandoffRdf]): Unit = { + for (statement: Statement <- statements) { // Change statements with knora-base:standoffTagHasStartParent and knora-base:standoffTagHasEndParent to point // to the new IRIs of those tags. - val newStatementObj: Value = if (statement.getPredicate == StandoffTagHasStartParentIri || statement.getPredicate == StandoffTagHasEndParentIri) { - val targetTagOldIri: IRI = valueFactory.createIRI(statement.getObject.stringValue) + val newStatementObj: RdfNode = if (statement.pred == StandoffTagHasStartParentIri || statement.pred == StandoffTagHasEndParentIri) { + val targetTagOldIri: IriNode = nodeFactory.makeIriNode(statement.obj.stringValue) standoff(targetTagOldIri).newIri } else { - statement.getObject + statement.obj } // Remove each statement that uses this tag's old IRI. - model.remove( - oldIri, - statement.getPredicate, - statement.getObject, - statement.getContext - ) + model.removeStatement(statement) // Replace it with a statement that uses this tag's new IRI. model.add( - newIri, - statement.getPredicate, - newStatementObj, - statement.getContext + subj = newIri, + pred = statement.pred, + obj = newStatementObj, + context = statement.context ) } } @@ -108,48 +109,43 @@ class UpgradePluginPR1307 extends UpgradePlugin { * @param standoff the standoff tags attached to this text value, as a map of old standoff tag IRIs to * [[StandoffRdf]] objects. */ - case class TextValueRdf(iri: IRI, context: IRI, valueHasStandoffStatements: Model, standoff: Map[IRI, StandoffRdf]) { - def transform(model: Model): Unit = { + case class TextValueRdf(iri: IriNode, context: Option[IRI], valueHasStandoffStatements: Set[Statement], standoff: Map[IriNode, StandoffRdf]) { + def transform(model: RdfModel): Unit = { // Transform the text value's standoff tags. for (standoffTag <- standoff.values) { standoffTag.transform(model = model, standoff = standoff) } if (standoff.nonEmpty) { - for (statement: Statement <- valueHasStandoffStatements.asScala.toSet) { + for (statement: Statement <- valueHasStandoffStatements) { // Replace each statement in valueHasStandoffStatements with one that points to the standoff // tag's new IRI. - val targetTagOldIri: IRI = valueFactory.createIRI(statement.getObject.stringValue) - val targetTagNewIri: IRI = standoff(targetTagOldIri).newIri + val targetTagOldIri: IriNode = nodeFactory.makeIriNode(statement.obj.stringValue) + val targetTagNewIri: IriNode = standoff(targetTagOldIri).newIri - model.remove( - iri, - statement.getPredicate, - statement.getObject, - statement.getContext - ) + model.removeStatement(statement) model.add( - iri, - statement.getPredicate, - targetTagNewIri, - statement.getContext + subj = iri, + pred = statement.pred, + obj = targetTagNewIri, + context = statement.context ) } // Add a statement to the text value with the predicate knora-base:valueHasMaxStandoffStartIndex. model.add( - iri, - ValueHasMaxStandoffStartIndexIri, - valueFactory.createLiteral(new java.math.BigInteger(standoff.values.map(_.startIndex).max.toString)), - context + subj = iri, + pred = ValueHasMaxStandoffStartIndexIri, + obj = nodeFactory.makeDatatypeLiteral(standoff.values.map(_.startIndex).max.toString, OntologyConstants.Xsd.Integer), + context = context ) } } } - override def transform(model: Model): Unit = { + override def transform(model: RdfModel): Unit = { for (textValue <- collectTextValues(model)) { textValue.transform(model) } @@ -158,32 +154,51 @@ class UpgradePluginPR1307 extends UpgradePlugin { /** * Collects the text values and standoff tags in the repository. */ - private def collectTextValues(model: Model): Vector[TextValueRdf] = { - // Pairs of text value IRI and text value context. - val textValueSubjectsAndContexts: Vector[(IRI, IRI)] = model.filter(null, RDF.TYPE, TextValueIri).asScala.map { + private def collectTextValues(model: RdfModel): Vector[TextValueRdf] = { + + + // A map of text value IRIs to their contexts. + val textValueSubjectsAndContexts: Map[IriNode, Option[IRI]] = model.find( + subj = None, + pred = Some(rdfTypeIri), + obj = Some(TextValueIri) + ).map { statement => - (valueFactory.createIRI(statement.getSubject.stringValue), valueFactory.createIRI(statement.getContext.stringValue)) - }.toVector + (nodeFactory.makeIriNode(statement.subj.stringValue), statement.context) + }.toMap textValueSubjectsAndContexts.map { - case (textValueSubj: IRI, textValueContext: IRI) => + case (textValueSubj: IriNode, textValueContext: Option[IRI]) => // Get the statements about the text value. - val textValueStatements: Model = model.filter(textValueSubj, null, null) + val textValueStatements: Set[Statement] = model.find( + subj = Some(textValueSubj), + pred = None, + obj = None + ).toSet // Get the statements whose subject is the text value and whose predicate is knora-base:valueHasStandoff. - val valueHasStandoffStatements: Model = textValueStatements.filter(null, ValueHasStandoffIri, null) + val valueHasStandoffStatements: Set[Statement] = textValueStatements.filter { + statement => statement.pred == ValueHasStandoffIri + } // Get the IRIs of the text value's standoff tags. - val standoffSubjects: Set[IRI] = valueHasStandoffStatements.objects.asScala.map { - value => valueFactory.createIRI(value.stringValue) - }.toSet + val standoffSubjects: Set[IriNode] = valueHasStandoffStatements.map { + statement => statement.obj match { + case iriNode: IriNode => iriNode + case other => throw InconsistentRepositoryDataException(s"Unexpected object for $textValueSubj $ValueHasStandoffIri: $other") + } + } // Make a map of standoff IRIs to StandoffRdf objects. - val standoff: Map[IRI, StandoffRdf] = standoffSubjects.map { - standoffSubj: IRI => + val standoff: Map[IriNode, StandoffRdf] = standoffSubjects.map { + standoffSubj: IriNode => standoffSubj -> StandoffRdf( oldIri = standoffSubj, - statements = model.filter(standoffSubj, null, null) + statements = model.find( + subj = Some(standoffSubj), + pred = None, + obj = None + ).toSet ) }.toMap @@ -193,6 +208,6 @@ class UpgradePluginPR1307 extends UpgradePlugin { valueHasStandoffStatements = valueHasStandoffStatements, standoff = standoff ) - } + }.toVector } } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322.scala index 68280fa664..a6b7ba8d0b 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322.scala @@ -19,32 +19,32 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.impl.SimpleValueFactory -import org.eclipse.rdf4j.model.{IRI, Model, Resource} -import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin +import org.knora.webapi.exceptions.InconsistentRepositoryDataException +import org.knora.webapi.feature.FeatureFactoryConfig +import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.messages.{OntologyConstants, StringFormatter} +import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin -import scala.collection.JavaConverters._ /** * Transforms a repository for Knora PR 1322. */ -class UpgradePluginPR1322 extends UpgradePlugin { - private val valueFactory = SimpleValueFactory.getInstance +class UpgradePluginPR1322(featureFactoryConfig: FeatureFactoryConfig) extends UpgradePlugin { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) private implicit val stringFormatter: StringFormatter = StringFormatter.getInstanceForConstantOntologies - // RDF4J IRI objects representing the IRIs used in this transformation. - private val ValueHasUUIDIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.ValueHasUUID) - private val ValueCreationDateIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.ValueCreationDate) - private val PreviousValueIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.PreviousValue) + // IRI objects representing the IRIs used in this transformation. + private val ValueHasUUIDIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueHasUUID) + private val ValueCreationDateIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueCreationDate) + private val PreviousValueIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.PreviousValue) - override def transform(model: Model): Unit = { + override def transform(model: RdfModel): Unit = { // Add a random UUID to each current value version. - for (valueIri: IRI <- collectCurrentValueIris(model)) { + for (valueIri: IriNode <- collectCurrentValueIris(model)) { model.add( - valueIri, - ValueHasUUIDIri, - valueFactory.createLiteral(stringFormatter.makeRandomBase64EncodedUuid) + subj = valueIri, + pred = ValueHasUUIDIri, + obj = nodeFactory.makeStringLiteral(stringFormatter.makeRandomBase64EncodedUuid) ) } } @@ -52,11 +52,16 @@ class UpgradePluginPR1322 extends UpgradePlugin { /** * Collects the IRIs of all values that are current value versions. */ - private def collectCurrentValueIris(model: Model): Set[IRI] = { - model.filter(null, ValueCreationDateIri, null).subjects.asScala.toSet.filter { - resource: Resource => model.filter(null, PreviousValueIri, resource).asScala.toSet.isEmpty + private def collectCurrentValueIris(model: RdfModel): Iterator[IriNode] = { + model.find(None, Some(ValueCreationDateIri), None).map(_.subj).filter { + resource: RdfResource => model.find( + subj = None, + pred = Some(PreviousValueIri), + obj = Some(resource) + ).isEmpty }.map { - resource: Resource => valueFactory.createIRI(resource.stringValue) + case iriNode: IriNode => iriNode + case other => throw InconsistentRepositoryDataException(s"Unexpected subject for $ValueCreationDateIri: $other") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367.scala index c38f758620..c5d1452960 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367.scala @@ -19,44 +19,42 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.impl.SimpleValueFactory -import org.eclipse.rdf4j.model.{IRI, Literal, Model, Statement} +import org.knora.webapi.feature.FeatureFactoryConfig +import org.knora.webapi.messages.OntologyConstants +import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin -import scala.collection.JavaConverters._ - /** * Transforms a repository for Knora PR 1367. */ -class UpgradePluginPR1367 extends UpgradePlugin { - private val valueFactory = SimpleValueFactory.getInstance - - // RDF4J IRI objects representing the IRIs used in this transformation. - private val XsdValueHasDecimalIri: IRI = valueFactory.createIRI("http://www.w3.org/2001/XMLSchema#valueHasDecimal") +class UpgradePluginPR1367(featureFactoryConfig: FeatureFactoryConfig) extends UpgradePlugin { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) - override def transform(model: Model): Unit = { + override def transform(model: RdfModel): Unit = { // Fix the datatypes of decimal literals. - for (statement: Statement <- model.asScala.toSet) { - statement.getObject match { - case literal: Literal => - if (literal.getDatatype == XsdValueHasDecimalIri) { - model.remove( - statement.getSubject, - statement.getPredicate, - statement.getObject, - statement.getContext - ) - model.add( - statement.getSubject, - statement.getPredicate, - valueFactory.createLiteral(BigDecimal(statement.getObject.stringValue).underlying), - statement.getContext + val statementsToRemove: collection.mutable.Set[Statement] = collection.mutable.Set.empty + val statementsToAdd: collection.mutable.Set[Statement] = collection.mutable.Set.empty + + for (statement: Statement <- model) { + statement.obj match { + case literal: DatatypeLiteral => + if (literal.datatype == "http://www.w3.org/2001/XMLSchema#valueHasDecimal") { + statementsToRemove += statement + + statementsToAdd += nodeFactory.makeStatement( + subj = statement.subj, + pred = statement.pred, + obj = nodeFactory.makeDatatypeLiteral(literal.value, OntologyConstants.Xsd.Decimal), + context = statement.context ) } case _ => () } } + + model.removeStatements(statementsToRemove.toSet) + model.addStatements(statementsToAdd.toSet) } } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372.scala index c346bab3a9..d60d6e0bc8 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372.scala @@ -19,31 +19,30 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.impl.SimpleValueFactory -import org.eclipse.rdf4j.model.{IRI, Model, Resource} -import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin +import org.knora.webapi.exceptions.InconsistentRepositoryDataException +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.OntologyConstants - -import scala.collection.JavaConverters._ +import org.knora.webapi.messages.util.rdf._ +import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin /** * Transforms a repository for Knora PR 1372. */ -class UpgradePluginPR1372 extends UpgradePlugin { - private val valueFactory = SimpleValueFactory.getInstance +class UpgradePluginPR1372(featureFactoryConfig: FeatureFactoryConfig) extends UpgradePlugin { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) - // RDF4J IRI objects representing the IRIs used in this transformation. - private val ValueCreationDateIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.ValueCreationDate) - private val PreviousValueIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.PreviousValue) - private val HasPermissionsIri: IRI = valueFactory.createIRI(OntologyConstants.KnoraBase.HasPermissions) + // IRI objects representing the IRIs used in this transformation. + private val ValueCreationDateIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueCreationDate) + private val PreviousValueIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.PreviousValue) + private val HasPermissionsIri: IriNode = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.HasPermissions) - override def transform(model: Model): Unit = { + override def transform(model: RdfModel): Unit = { // Remove knora-base:hasPermissions from all past value versions. - for (valueIri: IRI <- collectPastValueIris(model)) { + for (valueIri: IriNode <- collectPastValueIris(model)) { model.remove( - valueIri, - HasPermissionsIri, - null + subj = Some(valueIri), + pred = Some(HasPermissionsIri), + obj = None ) } } @@ -51,11 +50,16 @@ class UpgradePluginPR1372 extends UpgradePlugin { /** * Collects the IRIs of all values that are past value versions. */ - private def collectPastValueIris(model: Model): Set[IRI] = { - model.filter(null, ValueCreationDateIri, null).subjects.asScala.toSet.filter { - resource: Resource => model.filter(null, PreviousValueIri, resource).asScala.toSet.nonEmpty + private def collectPastValueIris(model: RdfModel): Iterator[IriNode] = { + model.find(None, Some(ValueCreationDateIri), None).map(_.subj).filter { + resource: RdfResource => model.find( + subj = None, + pred = Some(PreviousValueIri), + obj = Some(resource) + ).nonEmpty }.map { - resource: Resource => valueFactory.createIRI(resource.stringValue) + case iriNode: IriNode => iriNode + case other => throw InconsistentRepositoryDataException(s"Unexpected subject for $ValueCreationDateIri: $other") } } } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615.scala index 52fd692a7d..7a61027995 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615.scala @@ -1,20 +1,43 @@ +/* + * Copyright © 2015-2019 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ + package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.Model -import org.eclipse.rdf4j.model.impl.SimpleValueFactory +import org.knora.webapi.feature.FeatureFactoryConfig +import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin /** * Transforms a repository for Knora PR 1615. */ -class UpgradePluginPR1615 extends UpgradePlugin { - private val valueFactory = SimpleValueFactory.getInstance +class UpgradePluginPR1615(featureFactoryConfig: FeatureFactoryConfig) extends UpgradePlugin { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) - // RDF4J IRI objects representing the IRIs used in this transformation. - private val ForbiddenResourceIri = valueFactory.createIRI("http://rdfh.ch/0000/forbiddenResource") + // IRI objects representing the IRIs used in this transformation. + private val ForbiddenResourceIri: IriNode = nodeFactory.makeIriNode("http://rdfh.ch/0000/forbiddenResource") - override def transform(model: Model): Unit = { + override def transform(model: RdfModel): Unit = { // Remove the singleton instance of knora-base:ForbiddenResource. - model.remove(ForbiddenResourceIri, null, null) + model.remove( + subj = Some(ForbiddenResourceIri), + pred = None, + obj = None + ) } } diff --git a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746.scala b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746.scala index cfeefcb2c4..31d8b4f752 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746.scala @@ -1,52 +1,88 @@ -package org.knora.webapi.store.triplestore.upgrade.plugins +/* + * Copyright © 2015-2019 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ +package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.{Literal, Model, Statement} -import org.eclipse.rdf4j.model.impl.{SimpleLiteral, SimpleValueFactory} +import com.typesafe.scalalogging.Logger +import org.knora.webapi.feature.FeatureFactoryConfig +import org.knora.webapi.messages.OntologyConstants +import org.knora.webapi.messages.util.rdf._ import org.knora.webapi.store.triplestore.upgrade.UpgradePlugin -import scala.collection.JavaConverters._ - /** * Transforms a repository for Knora PR 1746. */ -class UpgradePluginPR1746 extends UpgradePlugin { - private val valueFactory = SimpleValueFactory.getInstance - - override def transform(model: Model): Unit = { - // change the empty strings to FIXME - def replaceEmptyStringWithDummy(statement: Statement, languageTag: String) = { - val fixMeString: Literal = if(languageTag.isEmpty) { - valueFactory.createLiteral("FIXME") - } else { - valueFactory.createLiteral("FIXME", languageTag) +class UpgradePluginPR1746(featureFactoryConfig: FeatureFactoryConfig, log: Logger) extends UpgradePlugin { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) + + private val dummyString = "FIXME" + + override def transform(model: RdfModel): Unit = { + val statementsToRemove: collection.mutable.Set[Statement] = collection.mutable.Set.empty + val statementsToAdd: collection.mutable.Set[Statement] = collection.mutable.Set.empty + + def replaceEmptyStringWithDummy(statement: Statement, languageTag: Option[String]): Unit = { + val fixMeString: RdfLiteral = languageTag match { + case Some(definedLanguageTag) => + nodeFactory.makeStringWithLanguage(dummyString, definedLanguageTag) + + case None => nodeFactory.makeStringLiteral(dummyString) } - model.remove( - statement.getSubject, - statement.getPredicate, - statement.getObject, - statement.getContext - ) + statementsToRemove += statement - model.add( - statement.getSubject, - statement.getPredicate, - fixMeString, - statement.getContext + statementsToAdd += nodeFactory.makeStatement( + subj = statement.subj, + pred = statement.pred, + obj = fixMeString, + context = statement.context ) + + log.warn(s"Changed empty object of <${statement.subj}> <${statement.pred}> to FIXME") } - for (statement: Statement <- model.asScala.toSet) { - statement.getObject match { - case literal: SimpleLiteral => - if (literal.stringValue().isEmpty) { - val lang = literal.getLanguage.orElse("") - replaceEmptyStringWithDummy(statement, lang) + for (statement: Statement <- model) { + statement.obj match { + case rdfLiteral: RdfLiteral => + if (rdfLiteral.stringValue.isEmpty) { + rdfLiteral match { + case datatypeLiteral: DatatypeLiteral => + if (datatypeLiteral.datatype == OntologyConstants.Xsd.String) { + replaceEmptyStringWithDummy( + statement = statement, + languageTag = None + ) + } + + case stringWithLanguage: StringWithLanguage => + replaceEmptyStringWithDummy( + statement = statement, + languageTag = Some(stringWithLanguage.language) + ) + } } case _ => () } } + + model.removeStatements(statementsToRemove.toSet) + model.addStatements(statementsToAdd.toSet) } } diff --git a/webapi/src/main/twirl/org/knora/webapi/messages/twirl/xsd/v1/xmlImport.scala.xml b/webapi/src/main/twirl/org/knora/webapi/messages/twirl/xsd/v1/xmlImport.scala.xml index 330252d833..79e8404aae 100644 --- a/webapi/src/main/twirl/org/knora/webapi/messages/twirl/xsd/v1/xmlImport.scala.xml +++ b/webapi/src/main/twirl/org/knora/webapi/messages/twirl/xsd/v1/xmlImport.scala.xml @@ -37,7 +37,7 @@ *@ @import org.knora.webapi._ -@import org.knora.webapi.exceptions.{AssertionException, InconsistentTriplestoreDataException, SparqlGenerationException} +@import org.knora.webapi.exceptions.{AssertionException, InconsistentRepositoryDataException, SparqlGenerationException} @import org.knora.webapi.messages.v1.responder.ontologymessages._ @import org.knora.webapi.messages.v2.responder.ontologymessages.Cardinality @import org.knora.webapi.messages.OntologyConstants @@ -154,7 +154,7 @@ } } - @defining(propertyInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentTriplestoreDataException(s"Property $propertyIri has no knora-base:objectClassConstraint"))) { propertyObjectClassConstraint => + @defining(propertyInfo.getPredicateObject(OntologyConstants.KnoraBase.ObjectClassConstraint).getOrElse(throw InconsistentRepositoryDataException(s"Property $propertyIri has no knora-base:objectClassConstraint"))) { propertyObjectClassConstraint => @if(propertyInfo.isLinkProp) { diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtilSpec.scala b/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtilSpec.scala index c881a0a2fe..9380e86d0a 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtilSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfFormatUtilSpec.scala @@ -19,7 +19,7 @@ package org.knora.webapi.util.rdf -import java.io.File +import java.io.{BufferedInputStream, ByteArrayInputStream, ByteArrayOutputStream, File, FileInputStream} import org.knora.webapi.{CoreSpec, IRI} import org.knora.webapi.feature.{FeatureFactoryConfig, FeatureToggle, KnoraSettingsFeatureFactoryConfig, TestFeatureFactoryConfig} @@ -31,6 +31,8 @@ import org.knora.webapi.util.FileUtil * Tests implementations of [[RdfFormatUtil]]. */ abstract class RdfFormatUtilSpec(featureToggle: FeatureToggle) extends CoreSpec { + StringFormatter.initForTest() + private val featureFactoryConfig: FeatureFactoryConfig = new TestFeatureFactoryConfig( testToggles = Set(featureToggle), parent = new KnoraSettingsFeatureFactoryConfig(settings) @@ -40,14 +42,55 @@ abstract class RdfFormatUtilSpec(featureToggle: FeatureToggle) extends CoreSpec private val rdfNodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) private val rdfModelFactory: RdfModelFactory = RdfFeatureFactory.getRdfModelFactory(featureFactoryConfig) - StringFormatter.initForTest() + private val expectedThingLabelStatement = rdfNodeFactory.makeStatement( + rdfNodeFactory.makeIriNode("http://www.knora.org/ontology/0001/anything#Thing"), + rdfNodeFactory.makeIriNode(OntologyConstants.Rdfs.Label), + rdfNodeFactory.makeStringWithLanguage(value = "Thing", language = "en") + ) + + /** + * Processes `anything-onto.ttl` and checks whether the expected content is received. + */ + class TestStreamProcessor extends RdfStreamProcessor { + var startCalled: Boolean = false + var finishCalled: Boolean = false + var gotKnoraBaseNamespace = false + var gotThingLabelStatement: Boolean = false + + override def start(): Unit = { + startCalled = true + } + + override def processNamespace(prefix: String, namespace: IRI): Unit = { + if (prefix == "knora-base" && namespace == "http://www.knora.org/ontology/knora-base#") { + gotKnoraBaseNamespace = true + } + } + + override def processStatement(statement: Statement): Unit = { + if (statement == expectedThingLabelStatement) { + gotThingLabelStatement = true + } + } + + override def finish(): Unit = { + finishCalled = true + } + + def check(): Unit = { + assert(startCalled) + assert(gotKnoraBaseNamespace) + assert(gotThingLabelStatement) + assert(finishCalled) + } + } private def checkModelForRdfTypeBook(rdfModel: RdfModel): Unit = { val statements: Set[Statement] = rdfModel.find( subj = Some(rdfNodeFactory.makeIriNode("http://rdfh.ch/0803/2a6221216701")), pred = Some(rdfNodeFactory.makeIriNode(OntologyConstants.Rdf.Type)), obj = None - ) + ).toSet assert(statements.size == 1) assert(statements.head.obj == rdfNodeFactory.makeIriNode("http://0.0.0.0:3333/ontology/0803/incunabula/v2#book")) @@ -145,5 +188,71 @@ abstract class RdfFormatUtilSpec(featureToggle: FeatureToggle) extends CoreSpec val outputTurtle: String = rdfFormatUtil.format(rdfModel = outputModel, rdfFormat = Turtle) assert(outputTurtle.contains("\"JULIAN:1481 CE\"^^knora-api:Date")) } + + "parse RDF from a stream and process it using an RdfStreamProcessor" in { + val inputStream = new BufferedInputStream(new FileInputStream("test_data/ontologies/anything-onto.ttl")) + val testStreamProcessor = new TestStreamProcessor + + rdfFormatUtil.parseWithStreamProcessor( + rdfSource = RdfInputStreamSource(inputStream), + rdfFormat = Turtle, + rdfStreamProcessor = testStreamProcessor + ) + + inputStream.close() + testStreamProcessor.check() + } + + "process streamed RDF and write the formatted result to an output stream" in { + // Read the file, process it with an RdfStreamProcessor, and write the result + // to a ByteArrayOutputStream. + + val fileInputStream = new BufferedInputStream(new FileInputStream("test_data/ontologies/anything-onto.ttl")) + val byteArrayOutputStream = new ByteArrayOutputStream() + + val formattingStreamProcessor = rdfFormatUtil.makeFormattingStreamProcessor( + outputStream = byteArrayOutputStream, + rdfFormat = Turtle + ) + + rdfFormatUtil.parseWithStreamProcessor( + rdfSource = RdfInputStreamSource(fileInputStream), + rdfFormat = Turtle, + rdfStreamProcessor = formattingStreamProcessor + ) + + fileInputStream.close() + + // Read back the ByteArrayOutputStream and check that it's correct. + + val byteArrayInputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray) + val testStreamProcessor = new TestStreamProcessor + + rdfFormatUtil.parseWithStreamProcessor( + rdfSource = RdfInputStreamSource(byteArrayInputStream), + rdfFormat = Turtle, + rdfStreamProcessor = testStreamProcessor + ) + + byteArrayInputStream.close() + testStreamProcessor.check() + } + + "stream RDF data from an InputStream into an RdfModel, then into an OutputStream, then back into an RdfModel" in { + val fileInputStream = new BufferedInputStream(new FileInputStream("test_data/ontologies/anything-onto.ttl")) + val rdfModel: RdfModel = rdfFormatUtil.inputStreamToRdfModel(inputStream = fileInputStream, rdfFormat = Turtle) + fileInputStream.close() + assert(rdfModel.contains(expectedThingLabelStatement)) + + val byteArrayOutputStream = new ByteArrayOutputStream() + rdfFormatUtil.rdfModelToOutputStream(rdfModel = rdfModel, outputStream = byteArrayOutputStream, rdfFormat = Turtle) + byteArrayOutputStream.close() + + val byteArrayInputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray) + val copyOfRdfModel: RdfModel = rdfFormatUtil.inputStreamToRdfModel(inputStream = byteArrayInputStream, rdfFormat = Turtle) + byteArrayInputStream.close() + + assert(copyOfRdfModel == rdfModel) + } } } diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfModelSpec.scala b/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfModelSpec.scala index c667a8d8a2..94b92c75a9 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfModelSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/rdf/RdfModelSpec.scala @@ -19,11 +19,13 @@ package org.knora.webapi.util.rdf +import java.io.{BufferedInputStream, FileInputStream} + import org.knora.webapi.exceptions.AssertionException import org.knora.webapi.feature._ import org.knora.webapi.messages.OntologyConstants -import org.knora.webapi.{CoreSpec, IRI} import org.knora.webapi.messages.util.rdf._ +import org.knora.webapi.{CoreSpec, IRI} /** * Tests implementations of [[RdfModel]]. @@ -39,6 +41,7 @@ abstract class RdfModelSpec(featureToggle: FeatureToggle) extends CoreSpec { private val model: RdfModel = RdfFeatureFactory.getRdfModelFactory(featureFactoryConfig).makeEmptyModel private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(featureFactoryConfig) + private val rdfFormatUtil: RdfFormatUtil = RdfFeatureFactory.getRdfFormatUtil(featureFactoryConfig) /** * Adds a statement, then searches for it by subject and predicate. @@ -51,7 +54,7 @@ abstract class RdfModelSpec(featureToggle: FeatureToggle) extends CoreSpec { private def addAndFindBySubjAndPred(subj: RdfResource, pred: IriNode, obj: RdfNode, context: Option[IRI] = None): Unit = { val statement: Statement = nodeFactory.makeStatement(subj = subj, pred = pred, obj = obj) model.addStatement(statement) - assert(model.find(subj = Some(subj), pred = Some(pred), obj = None) == Set(statement)) + assert(model.find(subj = Some(subj), pred = Some(pred), obj = None).toSet == Set(statement)) } "An RdfModel" should { @@ -62,7 +65,7 @@ abstract class RdfModelSpec(featureToggle: FeatureToggle) extends CoreSpec { model.add(subj = subj, pred = pred, obj = obj) val expectedStatement: Statement = nodeFactory.makeStatement(subj = subj, pred = pred, obj = obj) - assert(model.find(subj = Some(subj), pred = Some(pred), obj = None) == Set(expectedStatement)) + assert(model.find(subj = Some(subj), pred = Some(pred), obj = None).toSet == Set(expectedStatement)) } "add a triple with a datatype literal object" in { @@ -131,13 +134,13 @@ abstract class RdfModelSpec(featureToggle: FeatureToggle) extends CoreSpec { ) model.addStatements(statements) - assert(model.find(subj = Some(subj), pred = None, obj = None) == statements) + assert(model.find(subj = Some(subj), pred = None, obj = None).toSet == statements) val stringWithLangFindResult: Set[Statement] = model.find( subj = Some(subj), pred = Some(stringWithLangStatement.pred), obj = Some(stringWithLangStatement.obj) - ) + ).toSet assert(stringWithLangFindResult.size == 1) @@ -160,7 +163,7 @@ abstract class RdfModelSpec(featureToggle: FeatureToggle) extends CoreSpec { } } - "add and find quads" in { + "add, find, and remove quads" in { val labelPred: IriNode = nodeFactory.makeIriNode(OntologyConstants.Rdfs.Label) val commentPred: IriNode = nodeFactory.makeIriNode(OntologyConstants.Rdfs.Comment) val context1 = "http://example.org/graph1" @@ -208,10 +211,123 @@ abstract class RdfModelSpec(featureToggle: FeatureToggle) extends CoreSpec { model.addStatements(graph1) model.addStatements(graph2) - assert(model.find(subj = None, pred = None, obj = None, context = Some(context1)) == graph1) - assert(model.find(subj = None, pred = None, obj = None, context = Some(context2)) == graph2) - assert(model.find(subj = None, pred = Some(labelPred), obj = None) == Set(graph1LabelStatement, graph2LabelStatement)) - assert(model.find(subj = None, pred = Some(commentPred), obj = None) == Set(graph1CommentStatement, graph2CommentStatement)) + assert(model.find(subj = None, pred = None, obj = None, context = Some(context1)).toSet == graph1) + assert(model.find(subj = None, pred = None, obj = None, context = Some(context2)).toSet == graph2) + assert(model.find(subj = None, pred = Some(labelPred), obj = None).toSet == Set(graph1LabelStatement, graph2LabelStatement)) + assert(model.find(subj = None, pred = Some(commentPred), obj = None).toSet == Set(graph1CommentStatement, graph2CommentStatement)) + + model.removeStatement(graph1CommentStatement) + assert(!model.contains(graph1CommentStatement)) + + assert(model.contains(graph1LabelStatement)) + assert(model.contains(graph2LabelStatement)) + assert(model.contains(graph2CommentStatement)) + } + + "Remove a statement from the default graph, rather than an otherwise identical statement in a named graph" in { + val subj: IriNode = nodeFactory.makeIriNode("http://example.org/foo") + val pred: IriNode = nodeFactory.makeIriNode(OntologyConstants.Rdfs.Label) + val obj: DatatypeLiteral = nodeFactory.makeDatatypeLiteral(value = "Foo", datatype = OntologyConstants.Xsd.String) + + val statementInDefaultGraph: Statement = nodeFactory.makeStatement( + subj = subj, + pred = pred, + obj = obj, + context = None + ) + + val context = "http://example.org/namedGraph" + + val statementInNamedGraph = nodeFactory.makeStatement( + subj = subj, + pred = pred, + obj = obj, + context = Some(context) + ) + + model.addStatement(statementInDefaultGraph) + model.addStatement(statementInNamedGraph) + + assert(model.contains(statementInDefaultGraph)) + assert(model.contains(statementInNamedGraph)) + + model.removeStatement(statementInDefaultGraph) + + assert(!model.contains(statementInDefaultGraph)) + assert(model.contains(statementInNamedGraph)) + } + + "Remove a statement from the default graph, and an otherwise identical statement in a named graph" in { + val subj: IriNode = nodeFactory.makeIriNode("http://example.org/bar") + val pred: IriNode = nodeFactory.makeIriNode(OntologyConstants.Rdfs.Label) + val obj: DatatypeLiteral = nodeFactory.makeDatatypeLiteral(value = "Bar", datatype = OntologyConstants.Xsd.String) + + val statementInDefaultGraph: Statement = nodeFactory.makeStatement( + subj = subj, + pred = pred, + obj = obj, + context = None + ) + + val context = "http://example.org/namedGraph" + + val statementInNamedGraph = nodeFactory.makeStatement( + subj = subj, + pred = pred, + obj = obj, + context = Some(context) + ) + + model.addStatement(statementInDefaultGraph) + model.addStatement(statementInNamedGraph) + + assert(model.contains(statementInDefaultGraph)) + assert(model.contains(statementInNamedGraph)) + + model.remove( + subj = Some(subj), + pred = Some(pred), + obj = Some(obj) + ) + + assert(!model.contains(statementInDefaultGraph)) + assert(!model.contains(statementInNamedGraph)) + } + + + "do a SPARQL SELECT query" in { + val fileInputStream = new BufferedInputStream(new FileInputStream("test_data/all_data/anything-data.ttl")) + val anythingModel: RdfModel = rdfFormatUtil.inputStreamToRdfModel(inputStream = fileInputStream, rdfFormat = Turtle) + fileInputStream.close() + + val rdfRepository: RdfRepository = anythingModel.asRepository + + val selectQuery = + """PREFIX knora-base: + |PREFIX anything: + | + |SELECT ?resource ?value WHERE { + | ?resource anything:hasDecimal ?value . + |} ORDER BY ?resource""".stripMargin + + val queryResult: SparqlSelectResult = rdfRepository.doSelect(selectQuery) + + assert(queryResult.head.vars == Seq("resource", "value")) + + val results: Seq[Map[String, String]] = queryResult.results.bindings.map(_.rowMap) + + val expectedResults = Seq( + Map( + "resource" -> "http://rdfh.ch/0001/H6gBWUuJSuuO-CilHV8kQw", + "value" -> "http://rdfh.ch/0001/H6gBWUuJSuuO-CilHV8kQw/values/bXMwnrHvQH2DMjOFrGmNzg" + ), + Map( + "resource" -> "http://rdfh.ch/0001/uqmMo72OQ2K2xe7mkIytlg", + "value" -> "http://rdfh.ch/0001/uqmMo72OQ2K2xe7mkIytlg/values/85et-o-STOmn2JcVqrGTCQ" + ) + ) + + assert(results == expectedResults) } } } diff --git a/webapi/src/test/scala/org/knora/webapi/messages/v2/responder/metadatamessages/BUILD.bazel b/webapi/src/test/scala/org/knora/webapi/messages/v2/responder/metadatamessages/BUILD.bazel index 9133636813..217cb21ef7 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/v2/responder/metadatamessages/BUILD.bazel +++ b/webapi/src/test/scala/org/knora/webapi/messages/v2/responder/metadatamessages/BUILD.bazel @@ -18,6 +18,5 @@ scala_test( deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", "//webapi:test_library", - "@maven//:org_apache_jena_apache_jena_libs", ] + BASE_TEST_DEPENDENCIES, ) diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala index d11f4dd05c..e659a0ab46 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala @@ -30,6 +30,7 @@ import org.knora.webapi.exceptions.{BadRequestException, NotFoundException, Onto import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.admin.responder.permissionsmessages.{ObjectAccessPermissionADM, ObjectAccessPermissionsForResourceGetADM, PermissionADM} import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.{DateUtilV1, KnoraSystemInstances, MessageUtil, ValueUtilV1} import org.knora.webapi.messages.v1.responder.resourcemessages._ import org.knora.webapi.messages.v1.responder.valuemessages._ @@ -754,7 +755,7 @@ class ResourcesResponderV1Spec extends CoreSpec(ResourcesResponderV1Spec.config) storeManager ! SparqlSelectRequest(lastModSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings assert(rows.size <= 1, s"Resource $resourceIri has more than one instance of knora-base:lastModificationDate") diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala index 2747eb82a3..a804fde60e 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala @@ -30,6 +30,7 @@ import org.knora.webapi.app.ApplicationActor import org.knora.webapi.exceptions._ import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.v1.responder.resourcemessages.{LocationV1, ResourceFullGetRequestV1, ResourceFullResponseV1} import org.knora.webapi.messages.v1.responder.valuemessages._ import org.knora.webapi.messages.v2.responder.standoffmessages._ @@ -219,7 +220,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(lastModSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings assert(rows.size <= 1, s"Resource $resourceIri has more than one instance of knora-base:lastModificationDate") @@ -390,7 +391,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case sparqlSelectResponse: SparqlSelectResponse => + case sparqlSelectResponse: SparqlSelectResult => assert(sparqlSelectResponse.results.bindings.head.rowMap.keySet == Set("value")) } } @@ -863,7 +864,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with // The new LinkValue should have no previous version, and there should be a direct link between the resources. expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(false) @@ -952,7 +953,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with // There should be no new version of the LinkValue, and the direct link should still be there. expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(false) @@ -1031,7 +1032,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with // still be there. expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(true) @@ -1090,7 +1091,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with // The LinkValue should point to its previous version, and the direct link should still be there. expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => standoffLinkValueIri.set(response.results.bindings.head.rowMap("linkValue")) val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) @@ -1156,7 +1157,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with // The LinkValue should point to its previous version. There should be no direct link. expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => standoffLinkValueIri.unset() val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) @@ -1233,7 +1234,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(false) @@ -1403,7 +1404,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(false) @@ -1487,7 +1488,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(oldLinkValueSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(row => row.rowMap("objPred") == OntologyConstants.KnoraBase.IsDeleted && row.rowMap("objObj").toBoolean) should ===(true) @@ -1507,7 +1508,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(newLinkValueSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(false) @@ -1550,7 +1551,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(deletedLinkValueSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(row => row.rowMap("objPred") == OntologyConstants.KnoraBase.IsDeleted && row.rowMap("objObj").toBoolean) should ===(true) @@ -1900,7 +1901,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with // It should have no previousValue, and the direct link should exist. expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(false) @@ -1968,7 +1969,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(decrementedLinkValueSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) rows.exists(_.rowMap("objPred") == OntologyConstants.KnoraBase.PreviousValue) should ===(true) @@ -2029,7 +2030,7 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with storeManager ! SparqlSelectRequest(deletedLinkValueSparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => standoffLinkValueIri.unset() val rows = response.results.bindings rows.groupBy(_.rowMap("linkValue")).size should ===(1) diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v2/OntologyResponderV2Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v2/OntologyResponderV2Spec.scala index 6a6d7a702e..e4a7e02348 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v2/OntologyResponderV2Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v2/OntologyResponderV2Spec.scala @@ -4402,7 +4402,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a class that's missing a cardinality for a link value property" in { @@ -4411,7 +4411,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } @@ -4421,7 +4421,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a class with a cardinality whose subject class constraint is incompatible with the class" in { @@ -4430,7 +4430,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class without an rdfs:label" in { @@ -4439,7 +4439,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource property without an rdfs:label" in { @@ -4448,7 +4448,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class that is also a standoff class" in { @@ -4457,7 +4457,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class with a cardinality on an undefined property" in { @@ -4466,7 +4466,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class with a directly defined cardinality on a non-resource property" in { @@ -4475,7 +4475,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class with a cardinality on knora-base:resourceProperty" in { @@ -4484,7 +4484,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class with a cardinality on knora-base:hasValue" in { @@ -4493,7 +4493,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource class with a base class that has a Knora IRI but isn't a resource class" in { @@ -4502,7 +4502,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a standoff class with a cardinality on a resource property" in { @@ -4511,7 +4511,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a standoff class with a base class that's not a standoff class" in { @@ -4520,7 +4520,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property with a subject class constraint of foaf:Person" in { @@ -4529,7 +4529,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a Knora value property with a subject class constraint of knora-base:TextValue" in { @@ -4538,7 +4538,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property with a subject class constraint of salsah-gui:Guielement" in { @@ -4547,7 +4547,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property with an object class constraint of foaf:Person" in { @@ -4556,7 +4556,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property whose object class constraint is incompatible with its base property" in { @@ -4565,7 +4565,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a class with cardinalities for a link property and a matching link value property, except that the link property isn't really a link property" in { @@ -4574,7 +4574,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a class with cardinalities for a link property and a matching link value property, except that the link value property isn't really a link value property" in { @@ -4583,7 +4583,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource property with no object class constraint" ignore { // Consistency checks don't allow this in GraphDB. @@ -4592,7 +4592,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource property with no rdfs:label" in { @@ -4601,7 +4601,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property that's a subproperty of both knora-base:hasValue and knora-base:hasLinkTo" in { @@ -4610,7 +4610,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property that's a subproperty of knora-base:hasFileValue" in { @@ -4619,7 +4619,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a resource property with a base property that has a Knora IRI but isn't a resource property" in { @@ -4628,7 +4628,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a property that contains salsah-gui:guiOrder" ignore { // Consistency checks don't allow this in GraphDB. @@ -4637,7 +4637,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a cardinality that contains salsah-gui:guiElement" in { @@ -4646,7 +4646,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load an ontology containing a cardinality that contains salsah-gui:guiAttribute" in { @@ -4655,7 +4655,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing an owl:TransitiveProperty" in { @@ -4664,7 +4664,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology with a class that has cardinalities both on property P and on a subproperty of P" in { @@ -4673,7 +4673,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing mismatched cardinalities for a link property and a link value property" in { @@ -4682,7 +4682,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing an invalid cardinality on a boolean property" in { @@ -4691,7 +4691,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing a class with a cardinality on a property from a non-shared ontology in another project" in { @@ -4700,7 +4700,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing a class with a base class defined in a non-shared ontology in another project" in { @@ -4709,7 +4709,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing a property with a base property defined in a non-shared ontology in another project" in { @@ -4718,7 +4718,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing a property whose subject class constraint is defined in a non-shared ontology in another project" in { @@ -4727,7 +4727,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing a property whose object class constraint is defined in a non-shared ontology in another project" in { @@ -4736,7 +4736,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } "not load a project-specific ontology containing a class with two cardinalities that override the same base class cardinality of 1 or 0-1" in { @@ -4745,7 +4745,7 @@ class OntologyResponderV2Spec extends CoreSpec() with ImplicitSender { )) customLoadTestData(invalidOnto) - expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true) + expectMsgType[akka.actor.Status.Failure](timeout).cause.isInstanceOf[InconsistentRepositoryDataException] should ===(true) } } } diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2Spec.scala index 7b9b67e98b..dbe83133ec 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2Spec.scala @@ -31,6 +31,7 @@ import org.knora.webapi.exceptions._ import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.{CalendarNameGregorian, DatePrecisionYear, KnoraSystemInstances, PermissionUtilADM} import org.knora.webapi.messages.v2.responder.SuccessResponseV2 import org.knora.webapi.messages.v2.responder.resourcemessages._ @@ -524,7 +525,7 @@ class ResourcesResponderV2Spec extends CoreSpec() with ImplicitSender { storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case sparqlSelectResponse: SparqlSelectResponse => + case sparqlSelectResponse: SparqlSelectResult => sparqlSelectResponse.results.bindings.map { row => row.rowMap("standoffTag") }.toSet @@ -540,7 +541,7 @@ class ResourcesResponderV2Spec extends CoreSpec() with ImplicitSender { storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case sparqlSelectResponse: SparqlSelectResponse => + case sparqlSelectResponse: SparqlSelectResult => val savedDeleteDateStr = sparqlSelectResponse.getFirstRow.rowMap("deleteDate") stringFormatter.xsdDateTimeStampToInstant( @@ -2329,7 +2330,7 @@ class ResourcesResponderV2Spec extends CoreSpec() with ImplicitSender { storeManager ! SparqlSelectRequest(isEntityUsedSparql) expectMsgPF(timeout) { - case entityUsedResponse: SparqlSelectResponse => + case entityUsedResponse: SparqlSelectResult => assert(entityUsedResponse.results.bindings.isEmpty, s"Link value was not erased") } } diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v2/ValuesResponderV2Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v2/ValuesResponderV2Spec.scala index fcb03ac659..092b51fb1e 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v2/ValuesResponderV2Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v2/ValuesResponderV2Spec.scala @@ -30,6 +30,7 @@ import org.knora.webapi.exceptions._ import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.triplestoremessages._ +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.messages.util.search.gravsearch.GravsearchParser import org.knora.webapi.messages.util.{CalendarNameGregorian, DatePrecisionYear, KnoraSystemInstances, PermissionUtilADM} import org.knora.webapi.messages.v2.responder._ @@ -237,7 +238,7 @@ class ValuesResponderV2Spec extends CoreSpec() with ImplicitSender { storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case sparqlSelectResponse: SparqlSelectResponse => + case sparqlSelectResponse: SparqlSelectResult => val savedDeleteDateStr = sparqlSelectResponse.getFirstRow.rowMap("deleteDate") val savedDeleteDate: Instant = stringFormatter.xsdDateTimeStampToInstant( @@ -320,7 +321,7 @@ class ValuesResponderV2Spec extends CoreSpec() with ImplicitSender { storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings if (rows.isEmpty) { @@ -346,7 +347,7 @@ class ValuesResponderV2Spec extends CoreSpec() with ImplicitSender { storeManager ! SparqlSelectRequest(sparqlQuery) expectMsgPF(timeout) { - case response: SparqlSelectResponse => + case response: SparqlSelectResult => val rows = response.results.bindings if (rows.isEmpty) { diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/AllTriplestoreSpec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/AllTriplestoreSpec.scala index ffb47e4d52..3301463eed 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/AllTriplestoreSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/AllTriplestoreSpec.scala @@ -23,8 +23,9 @@ import akka.testkit.ImplicitSender import com.typesafe.config.ConfigFactory import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.CoreSpec +import org.knora.webapi.messages.util.rdf.SparqlSelectResult import org.knora.webapi.settings.TriplestoreTypes -import org.knora.webapi.sharedtestdata.{SharedOntologyTestDataADM} +import org.knora.webapi.sharedtestdata.SharedOntologyTestDataADM import scala.concurrent.duration._ import scala.language.postfixOps @@ -237,7 +238,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(countTriplesQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println(msg) afterLoadCount = msg.results.bindings.head.rowMap("no").toInt (afterLoadCount > 0) should ===(true) @@ -250,7 +251,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(namedGraphQuery) //println(result) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println(msg) msg.results.bindings.nonEmpty should ===(true) } @@ -264,7 +265,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(countTriplesQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println("vor insert: " + msg) msg.results.bindings.head.rowMap("no").toInt should ===(afterLoadCount) } @@ -275,7 +276,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(checkInsertQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println(msg) msg.results.bindings.size should ===(3) } @@ -283,7 +284,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(countTriplesQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println("nach instert" + msg) afterChangeCount = msg.results.bindings.head.rowMap("no").toInt (afterChangeCount - afterLoadCount) should ===(3) @@ -299,7 +300,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(countTriplesQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println("vor revert: " + msg) msg.results.bindings.head.rowMap("no").toInt should ===(afterChangeCount) } @@ -309,7 +310,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(countTriplesQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println("nach revert: " + msg) msg.results.bindings.head.rowMap("no").toInt should ===(afterLoadCount) } @@ -317,7 +318,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic storeManager ! SparqlSelectRequest(checkInsertQuery) expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println("check: " + msg) msg.results.bindings.size should ===(0) } @@ -333,7 +334,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic case _ => storeManager ! SparqlSelectRequest(textSearchQueryFusekiValueHasString) } expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println(msg) msg.results.bindings.size should ===(3) } @@ -347,7 +348,7 @@ class AllTriplestoreSpec extends CoreSpec(AllTriplestoreSpec.config) with Implic case _ => storeManager ! SparqlSelectRequest(textSearchQueryFusekiDRFLabel) } expectMsgPF(timeout) { - case msg: SparqlSelectResponse => + case msg: SparqlSelectResult => //println(msg) msg.results.bindings.size should ===(1) } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/BUILD.bazel b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/BUILD.bazel index f784170774..b4df3bde19 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/BUILD.bazel +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/BUILD.bazel @@ -1,8 +1,7 @@ package(default_visibility = ["//visibility:public"]) load("@io_bazel_rules_scala//scala:scala.bzl", "scala_test") -load("//third_party:dependencies.bzl", "ALL_WEBAPI_MAIN_DEPENDENCIES") -load("//third_party:dependencies.bzl", "BASE_TEST_DEPENDENCIES") +load("//third_party:dependencies.bzl", "ALL_WEBAPI_MAIN_DEPENDENCIES", "BASE_TEST_DEPENDENCIES") scala_test( name = "UpgradePluginPR1307Spec", @@ -12,15 +11,15 @@ scala_test( "UpgradePluginSpec.scala" ], data = [ + "//knora-ontologies", + "//test_data", "//test_data/upgrade", ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], # unused_dependency_checker_mode = "warn", deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", - "@maven//:org_eclipse_rdf4j_rdf4j_client", - "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory" + "//webapi:test_library", ] + BASE_TEST_DEPENDENCIES, ) @@ -32,15 +31,15 @@ scala_test( "UpgradePluginSpec.scala" ], data = [ + "//knora-ontologies", + "//test_data", "//test_data/upgrade", ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], # unused_dependency_checker_mode = "warn", deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", - "@maven//:org_eclipse_rdf4j_rdf4j_client", - "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory" + "//webapi:test_library", ] + BASE_TEST_DEPENDENCIES, ) @@ -52,15 +51,15 @@ scala_test( "UpgradePluginSpec.scala" ], data = [ + "//knora-ontologies", + "//test_data", "//test_data/upgrade", ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], # unused_dependency_checker_mode = "warn", deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", - "@maven//:org_eclipse_rdf4j_rdf4j_client", - "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory" + "//webapi:test_library", ] + BASE_TEST_DEPENDENCIES, ) @@ -72,15 +71,15 @@ scala_test( "UpgradePluginSpec.scala" ], data = [ + "//knora-ontologies", + "//test_data", "//test_data/upgrade", ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], # unused_dependency_checker_mode = "warn", deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", - "@maven//:org_eclipse_rdf4j_rdf4j_client", - "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory" + "//webapi:test_library", ] + BASE_TEST_DEPENDENCIES, ) @@ -92,15 +91,15 @@ scala_test( "UpgradePluginSpec.scala" ], data = [ + "//knora-ontologies", + "//test_data", "//test_data/upgrade", ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], # unused_dependency_checker_mode = "warn", deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", - "@maven//:org_eclipse_rdf4j_rdf4j_client", - "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory" + "//webapi:test_library", ] + BASE_TEST_DEPENDENCIES, ) @@ -112,14 +111,14 @@ scala_test( "UpgradePluginSpec.scala" ], data = [ + "//knora-ontologies", + "//test_data", "//test_data/upgrade", ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], # unused_dependency_checker_mode = "warn", deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ "//webapi:main_library", - "@maven//:org_eclipse_rdf4j_rdf4j_client", - "@maven//:org_eclipse_rdf4j_rdf4j_repository_sail", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_api", - "@maven//:org_eclipse_rdf4j_rdf4j_sail_memory" + "//webapi:test_library", ] + BASE_TEST_DEPENDENCIES, ) diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307Spec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307Spec.scala index 51200ad265..032f0bc01b 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1307Spec.scala @@ -19,23 +19,20 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.Model -import org.eclipse.rdf4j.repository.sail.SailRepository -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectResponse, SparqlSelectResponseBody} +import org.knora.webapi.messages.util.rdf._ class UpgradePluginPR1307Spec extends UpgradePluginSpec { "Upgrade plugin PR1307" should { "update text values with standoff" in { // Parse the input file. - val model: Model = trigFileToModel("test_data/upgrade/pr1307.trig") + val model: RdfModel = trigFileToModel("test_data/upgrade/pr1307.trig") // Use the plugin to transform the input. - val plugin = new UpgradePluginPR1307 + val plugin = new UpgradePluginPR1307(defaultFeatureFactoryConfig) plugin.transform(model) // Make an in-memory repository containing the transformed model. - val repository: SailRepository = makeRepository(model) - val connection = repository.getConnection + val repository: RdfRepository = model.asRepository // Check that knora-base:valueHasMaxStandoffStartIndex was added. @@ -48,9 +45,9 @@ class UpgradePluginPR1307Spec extends UpgradePluginSpec { |} |""".stripMargin - val queryResult1: SparqlSelectResponse = doSelect(selectQuery = query1, connection = connection) + val queryResult1: SparqlSelectResult = repository.doSelect(selectQuery = query1) - val expectedResult1: SparqlSelectResponseBody = expectedResult( + val expectedResult1: SparqlSelectResultBody = expectedResult( Seq( Map( "s" -> "http://rdfh.ch/0001/qN1igiDRSAemBBktbRHn6g/values/xyUIf8QHS5aFrlt7Q4F1FQ", @@ -72,9 +69,9 @@ class UpgradePluginPR1307Spec extends UpgradePluginSpec { |} ORDER BY ?tag |""".stripMargin - val queryResult2: SparqlSelectResponse = doSelect(selectQuery = query2, connection = connection) + val queryResult2: SparqlSelectResult = repository.doSelect(selectQuery = query2) - val expectedResult2: SparqlSelectResponseBody = expectedResult( + val expectedResult2: SparqlSelectResultBody = expectedResult( Seq( Map("tag" -> "http://rdfh.ch/0001/qN1igiDRSAemBBktbRHn6g/values/xyUIf8QHS5aFrlt7Q4F1FQ/standoff/0"), Map("tag" -> "http://rdfh.ch/0001/qN1igiDRSAemBBktbRHn6g/values/xyUIf8QHS5aFrlt7Q4F1FQ/standoff/1"), @@ -104,9 +101,9 @@ class UpgradePluginPR1307Spec extends UpgradePluginSpec { |} ORDER BY ?tag |""".stripMargin - val queryResult3: SparqlSelectResponse = doSelect(selectQuery = query3, connection = connection) + val queryResult3: SparqlSelectResult = repository.doSelect(selectQuery = query3) - val expectedResult3: SparqlSelectResponseBody = expectedResult( + val expectedResult3: SparqlSelectResultBody = expectedResult( Seq( Map( "startIndex" -> "0", @@ -151,7 +148,7 @@ class UpgradePluginPR1307Spec extends UpgradePluginSpec { ) assert(queryResult3.results == expectedResult3) - connection.close() + repository.shutDown() } } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322Spec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322Spec.scala index ef69e0c3b4..d4cf040958 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1322Spec.scala @@ -19,23 +19,20 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.Model -import org.eclipse.rdf4j.repository.sail.SailRepository -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectResponse, SparqlSelectResponseBody} +import org.knora.webapi.messages.util.rdf._ class UpgradePluginPR1322Spec extends UpgradePluginSpec { "Upgrade plugin PR1322" should { "add UUIDs to values" in { // Parse the input file. - val model: Model = trigFileToModel("test_data/upgrade/pr1322.trig") + val model: RdfModel = trigFileToModel("test_data/upgrade/pr1322.trig") // Use the plugin to transform the input. - val plugin = new UpgradePluginPR1322 + val plugin = new UpgradePluginPR1322(defaultFeatureFactoryConfig) plugin.transform(model) // Make an in-memory repository containing the transformed model. - val repository: SailRepository = makeRepository(model) - val connection = repository.getConnection + val repository: RdfRepository = model.asRepository // Check that UUIDs were added. @@ -48,9 +45,9 @@ class UpgradePluginPR1322Spec extends UpgradePluginSpec { |} ORDER BY ?value |""".stripMargin - val queryResult1: SparqlSelectResponse = doSelect(selectQuery = query, connection = connection) + val queryResult1: SparqlSelectResult = repository.doSelect(selectQuery = query) - val expectedResultBody: SparqlSelectResponseBody = expectedResult( + val expectedResultBody: SparqlSelectResultBody = expectedResult( Seq( Map( "value" -> "http://rdfh.ch/0001/thing-with-history/values/1c" @@ -63,7 +60,6 @@ class UpgradePluginPR1322Spec extends UpgradePluginSpec { assert(queryResult1.results == expectedResultBody) - connection.close() repository.shutDown() } } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367Spec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367Spec.scala index 1c89271f84..5c6cb26f04 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1367Spec.scala @@ -19,32 +19,43 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.impl.SimpleValueFactory -import org.eclipse.rdf4j.model.util.Models -import org.eclipse.rdf4j.model.{Literal, Model} -import org.knora.webapi.util.JavaUtil._ +import org.knora.webapi.exceptions.AssertionException import org.knora.webapi.messages.OntologyConstants +import org.knora.webapi.messages.util.rdf._ class UpgradePluginPR1367Spec extends UpgradePluginSpec { - private val valueFactory = SimpleValueFactory.getInstance + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(defaultFeatureFactoryConfig) "Upgrade plugin PR1367" should { "fix the datatypes of decimal literals" in { // Parse the input file. - val model: Model = trigFileToModel("test_data/upgrade/pr1367.trig") + val model: RdfModel = trigFileToModel("test_data/upgrade/pr1367.trig") // Use the plugin to transform the input. - val plugin = new UpgradePluginPR1367 + val plugin = new UpgradePluginPR1367(defaultFeatureFactoryConfig) plugin.transform(model) // Check that the decimal datatype was fixed. - val literal: Literal = Models.getPropertyLiteral( - model, - valueFactory.createIRI("http://rdfh.ch/0001/thing-with-history/values/1"), - valueFactory.createIRI(OntologyConstants.KnoraBase.ValueHasDecimal) - ).toOption.get - assert(literal.getDatatype == valueFactory.createIRI(OntologyConstants.Xsd.Decimal)) + val subj = nodeFactory.makeIriNode("http://rdfh.ch/0001/thing-with-history/values/1") + val pred = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueHasDecimal) + + model.find( + subj = Some(subj), + pred = Some(pred), + obj = None + ).toSet.headOption match { + case Some(statement: Statement) => + statement.obj match { + case datatypeLiteral: DatatypeLiteral => + assert(datatypeLiteral.datatype == OntologyConstants.Xsd.Decimal) + + case other => + throw AssertionException(s"Unexpected object for $pred: $other") + } + + case None => throw AssertionException(s"No statement found with subject $subj and predicate $pred") + } } } } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372Spec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372Spec.scala index 3f9b4ac7df..1697941812 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1372Spec.scala @@ -19,23 +19,20 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.Model -import org.eclipse.rdf4j.repository.sail.SailRepository -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectResponse, SparqlSelectResponseBody} +import org.knora.webapi.messages.util.rdf._ class UpgradePluginPR1372Spec extends UpgradePluginSpec { "Upgrade plugin PR1372" should { "remove permissions from past versions of values" in { // Parse the input file. - val model: Model = trigFileToModel("test_data/upgrade/pr1372.trig") + val model: RdfModel = trigFileToModel("test_data/upgrade/pr1372.trig") // Use the plugin to transform the input. - val plugin = new UpgradePluginPR1372 + val plugin = new UpgradePluginPR1372(defaultFeatureFactoryConfig) plugin.transform(model) // Make an in-memory repository containing the transformed model. - val repository: SailRepository = makeRepository(model) - val connection = repository.getConnection + val repository: RdfRepository = model.asRepository // Check that permissions were removed. @@ -49,9 +46,9 @@ class UpgradePluginPR1372Spec extends UpgradePluginSpec { |} ORDER BY ?value |""".stripMargin - val queryResult1: SparqlSelectResponse = doSelect(selectQuery = query, connection = connection) + val queryResult1: SparqlSelectResult = repository.doSelect(selectQuery = query) - val expectedResultBody: SparqlSelectResponseBody = expectedResult( + val expectedResultBody: SparqlSelectResultBody = expectedResult( Seq( Map( "value" -> "http://rdfh.ch/0001/thing-with-history/values/1c" @@ -67,7 +64,6 @@ class UpgradePluginPR1372Spec extends UpgradePluginSpec { assert(queryResult1.results == expectedResultBody) - connection.close() repository.shutDown() } } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615Spec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615Spec.scala index 7ad5c04afc..a1000a6ffc 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1615Spec.scala @@ -19,23 +19,20 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.Model -import org.eclipse.rdf4j.repository.sail.SailRepository -import org.knora.webapi.messages.store.triplestoremessages.SparqlSelectResponse +import org.knora.webapi.messages.util.rdf._ class UpgradePluginPR1615Spec extends UpgradePluginSpec { "Upgrade plugin PR1615" should { "remove the instance of ForbiddenResource" in { // Parse the input file. - val model: Model = trigFileToModel("test_data/upgrade/pr1615.trig") + val model: RdfModel = trigFileToModel("test_data/upgrade/pr1615.trig") // Use the plugin to transform the input. - val plugin = new UpgradePluginPR1615 + val plugin = new UpgradePluginPR1615(defaultFeatureFactoryConfig) plugin.transform(model) // Make an in-memory repository containing the transformed model. - val repository: SailRepository = makeRepository(model) - val connection = repository.getConnection + val repository: RdfRepository = model.asRepository // Check that was removed. @@ -45,7 +42,7 @@ class UpgradePluginPR1615Spec extends UpgradePluginSpec { |} |""".stripMargin - val queryResult1: SparqlSelectResponse = doSelect(selectQuery = query1, connection = connection) + val queryResult1: SparqlSelectResult = repository.doSelect(selectQuery = query1) assert(queryResult1.results.bindings.isEmpty) // Check that other data is still there. @@ -56,10 +53,9 @@ class UpgradePluginPR1615Spec extends UpgradePluginSpec { |} |""".stripMargin - val queryResult2: SparqlSelectResponse = doSelect(selectQuery = query2, connection = connection) + val queryResult2: SparqlSelectResult = repository.doSelect(selectQuery = query2) assert(queryResult2.results.bindings.nonEmpty) - connection.close() repository.shutDown() } } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746Spec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746Spec.scala index 9273a4a12f..a0eafe89ac 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginPR1746Spec.scala @@ -19,42 +19,55 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import org.eclipse.rdf4j.model.impl.SimpleValueFactory -import org.eclipse.rdf4j.model.util.Models -import org.eclipse.rdf4j.model.{Literal, Model} +import com.typesafe.scalalogging.LazyLogging +import org.knora.webapi.exceptions.AssertionException import org.knora.webapi.messages.OntologyConstants -import org.knora.webapi.util.JavaUtil._ +import org.knora.webapi.messages.util.rdf._ -class UpgradePluginPR1746Spec extends UpgradePluginSpec { - private val valueFactory = SimpleValueFactory.getInstance +class UpgradePluginPR1746Spec extends UpgradePluginSpec with LazyLogging { + private val nodeFactory: RdfNodeFactory = RdfFeatureFactory.getRdfNodeFactory(defaultFeatureFactoryConfig) + + private def checkLiteral(model: RdfModel, subj: IriNode, pred: IriNode, expectedObj: RdfLiteral): Unit = { + model.find( + subj = Some(subj), + pred = Some(pred), + obj = None + ).toSet.headOption match { + case Some(statement: Statement) => + statement.obj match { + case rdfLiteral: RdfLiteral => assert(rdfLiteral == expectedObj) + case other => throw AssertionException(s"Unexpected object for $pred: $other") + } + + case None => throw AssertionException(s"No statement found with subject $subj and predicate $pred") + } + } "Upgrade plugin PR1746" should { "replace empty string with FIXME" in { // Parse the input file. - val model: Model = trigFileToModel("test_data/upgrade/pr1746.trig") + val model: RdfModel = trigFileToModel("test_data/upgrade/pr1746.trig") // Use the plugin to transform the input. - val plugin = new UpgradePluginPR1746 + val plugin = new UpgradePluginPR1746(defaultFeatureFactoryConfig, logger) plugin.transform(model) // Check that the empty valueHasString is replaced with FIXME. - val literal: Literal = Models.getPropertyLiteral( - model, - valueFactory.createIRI("http://rdfh.ch/0001/thing-with-empty-string/values/1"), - valueFactory.createIRI(OntologyConstants.KnoraBase.ValueHasString) - ).toOption.get + checkLiteral( + model = model, + subj = nodeFactory.makeIriNode("http://rdfh.ch/0001/thing-with-empty-string/values/1"), + pred = nodeFactory.makeIriNode(OntologyConstants.KnoraBase.ValueHasString), + expectedObj = nodeFactory.makeStringLiteral("FIXME") + ) - assert(literal.stringValue() == "FIXME") // Check that the empty string literal value with lang tag is replaced with FIXME. - val stringLiteral: Literal = Models.getPropertyLiteral( - model, - valueFactory.createIRI("http://rdfh.ch/projects/XXXX"), - valueFactory.createIRI("http://www.knora.org/ontology/knora-admin#projectDescription") - ).toOption.get - - assert(stringLiteral.getLabel == "FIXME") - assert(stringLiteral.getLanguage.get == "en") + checkLiteral( + model = model, + subj = nodeFactory.makeIriNode("http://rdfh.ch/projects/XXXX"), + pred = nodeFactory.makeIriNode("http://www.knora.org/ontology/knora-admin#projectDescription"), + expectedObj = nodeFactory.makeStringWithLanguage(value = "FIXME", language = "en") + ) } } } diff --git a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginSpec.scala b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginSpec.scala index 70a2a10296..5b59e8f74c 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/triplestore/upgrade/plugins/UpgradePluginSpec.scala @@ -19,101 +19,42 @@ package org.knora.webapi.store.triplestore.upgrade.plugins -import java.io.{BufferedReader, File, FileReader} +import java.io.{BufferedInputStream, FileInputStream} -import org.eclipse.rdf4j.model.Model -import org.eclipse.rdf4j.model.impl.LinkedHashModel -import org.eclipse.rdf4j.query.{Binding, TupleQuery, TupleQueryResult} -import org.eclipse.rdf4j.repository.sail.{SailRepository, SailRepositoryConnection} -import org.eclipse.rdf4j.rio.helpers.StatementCollector -import org.eclipse.rdf4j.rio.{RDFFormat, RDFParser, Rio} -import org.eclipse.rdf4j.sail.memory.MemoryStore -import org.knora.webapi.messages.store.triplestoremessages.{SparqlSelectResponse, SparqlSelectResponseBody, SparqlSelectResponseHeader, VariableResultsRow} +import org.knora.webapi.CoreSpec import org.knora.webapi.messages.util.ErrorHandlingMap -import org.scalatest.matchers.should.Matchers -import org.scalatest.wordspec.AnyWordSpecLike - -import scala.collection.JavaConverters._ -import scala.collection.mutable.ArrayBuffer +import org.knora.webapi.messages.util.rdf._ /** * Provides helper methods for specs that test upgrade plugins. */ -abstract class UpgradePluginSpec extends AnyWordSpecLike with Matchers { - /** - * Parses a TriG file and returns it as an RDF4J [[Model]]. - * - * @param path the file path of the TriG file. - * @return a [[Model]]. - */ - def trigFileToModel(path: String): Model = { - val trigParser: RDFParser = Rio.createParser(RDFFormat.TRIG) - println("trigFileToModel - file path: {}", path) - val fileReader = new BufferedReader(new FileReader(new File(path))) - val model = new LinkedHashModel() - trigParser.setRDFHandler(new StatementCollector(model)) - trigParser.parse(fileReader, "") - model - } +abstract class UpgradePluginSpec extends CoreSpec() { + private val rdfFormatUtil: RdfFormatUtil = RdfFeatureFactory.getRdfFormatUtil(defaultFeatureFactoryConfig) /** - * Makes an in-memory RDF4J [[SailRepository]] containing a [[Model]]. + * Parses a TriG file and returns it as an [[RdfModel]]. * - * @param model the model to be added to the repository. - * @return the repository. + * @param path the file path of the TriG file. + * @return an [[RdfModel]]. */ - def makeRepository(model: Model): SailRepository = { - val repository = new SailRepository(new MemoryStore()) - repository.init() - val connection: SailRepositoryConnection = repository.getConnection - connection.add(model) - connection.close() - repository + def trigFileToModel(path: String): RdfModel = { + val fileInputStream = new BufferedInputStream(new FileInputStream(path)) + val rdfModel: RdfModel = rdfFormatUtil.inputStreamToRdfModel(inputStream = fileInputStream, rdfFormat = TriG) + fileInputStream.close() + rdfModel } /** - * Wraps expected SPARQL SELECT results in a [[SparqlSelectResponseBody]]. + * Wraps expected SPARQL SELECT results in a [[SparqlSelectResultBody]]. * * @param rows the expected results. - * @return a [[SparqlSelectResponseBody]] containing the expected results. + * @return a [[SparqlSelectResultBody]] containing the expected results. */ - def expectedResult(rows: Seq[Map[String, String]]): SparqlSelectResponseBody = { + def expectedResult(rows: Seq[Map[String, String]]): SparqlSelectResultBody = { val rowMaps = rows.map { mapToWrap => VariableResultsRow(new ErrorHandlingMap[String, String](mapToWrap, { key: String => s"No value found for SPARQL query variable '$key' in query result row" })) } - SparqlSelectResponseBody(bindings = rowMaps) - } - - /** - * Does a SPARQL SELECT query using a connection to an RDF4J [[SailRepository]]. - * - * @param selectQuery the query. - * @param connection a connection to the repository. - * @return a [[SparqlSelectResponse]] containing the query results. - */ - def doSelect(selectQuery: String, connection: SailRepositoryConnection): SparqlSelectResponse = { - val tupleQuery: TupleQuery = connection.prepareTupleQuery(selectQuery) - val tupleQueryResult: TupleQueryResult = tupleQuery.evaluate - - val header = SparqlSelectResponseHeader(tupleQueryResult.getBindingNames.asScala) - val rowBuffer = ArrayBuffer.empty[VariableResultsRow] - - while (tupleQueryResult.hasNext) { - val bindings: Iterable[Binding] = tupleQueryResult.next.asScala - - val rowMap: Map[String, String] = bindings.map { - binding => binding.getName -> binding.getValue.stringValue - }.toMap - - rowBuffer.append(VariableResultsRow(new ErrorHandlingMap[String, String](rowMap, { key: String => s"No value found for SPARQL query variable '$key' in query result row" }))) - } - - tupleQueryResult.close() - - SparqlSelectResponse( - head = header, - results = SparqlSelectResponseBody(bindings = rowBuffer) - ) + SparqlSelectResultBody(bindings = rowMaps) } }