diff --git a/docs/03-apis/api-v2/query-language.md b/docs/03-apis/api-v2/query-language.md index 83338d4487..5ed2b651f0 100644 --- a/docs/03-apis/api-v2/query-language.md +++ b/docs/03-apis/api-v2/query-language.md @@ -1211,3 +1211,61 @@ CONSTRUCT { } ORDER BY (?int) ``` + +## Query Optimization by Dependency + +The query performance of triplestores, such as Fuseki, is highly dependent on the order of query +patterns. To improve performance, Gravsearch automatically reorders the +statement patterns in the WHERE clause according to their dependencies on each other, to minimise +the number of possible matches for each pattern. +This optimization can be controlled using `gravsearch-dependency-optimisation` +[feature toggle](../feature-toggles.md), which is turned on by default. + +Consider the following Gravsearch query: + +```sparql +PREFIX beol: +PREFIX knora-api: + +CONSTRUCT { + ?letter knora-api:isMainResource true . + ?letter ?linkingProp1 ?person1 . + ?letter ?linkingProp2 ?person2 . + ?letter beol:creationDate ?date . +} WHERE { + ?letter beol:creationDate ?date . + + ?letter ?linkingProp1 ?person1 . + FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) + + ?letter ?linkingProp2 ?person2 . + FILTER(?linkingProp2 = beol:hasAuthor || ?linkingProp2 = beol:hasRecipient ) + + ?person1 beol:hasIAFIdentifier ?gnd1 . + ?gnd1 knora-api:valueAsString "(DE-588)118531379" . + + ?person2 beol:hasIAFIdentifier ?gnd2 . + ?gnd2 knora-api:valueAsString "(DE-588)118696149" . +} ORDER BY ?date +``` + +Gravsearch optimises the performance of this query by moving these statements +to the top of the WHERE clause: + +``` + ?gnd1 knora-api:valueAsString "(DE-588)118531379" . + ?gnd2 knora-api:valueAsString "(DE-588)118696149" . +``` + +The rest of the WHERE clause then reads: + +``` + ?person1 beol:hasIAFIdentifier ?gnd1 . + ?person2 beol:hasIAFIdentifier ?gnd2 . + ?letter ?linkingProp1 ?person1 . + FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) + + ?letter ?linkingProp2 ?person2 . + FILTER(?linkingProp2 = beol:hasAuthor || ?linkingProp2 = beol:hasRecipient ) + ?letter beol:creationDate ?date . +``` diff --git a/docs/05-internals/design/api-v2/figures/query_graph.png b/docs/05-internals/design/api-v2/figures/query_graph.png new file mode 100644 index 0000000000..a8db48a7e9 Binary files /dev/null and b/docs/05-internals/design/api-v2/figures/query_graph.png differ diff --git a/docs/05-internals/design/api-v2/gravsearch.md b/docs/05-internals/design/api-v2/gravsearch.md index 52c2e17c3a..dc6728302d 100644 --- a/docs/05-internals/design/api-v2/gravsearch.md +++ b/docs/05-internals/design/api-v2/gravsearch.md @@ -332,4 +332,166 @@ replaces `knora-api:standoffTagHasStartAncestor` with `knora-base:standoffTagHas The triplestore-specific transformers in `SparqlTransformer.scala` can run optimisations on the generated SPARQL, in the method `optimiseQueryPatterns` inherited from `WhereTransformer`. For example, `moveLuceneToBeginning` moves -Lucene queries to the beginning of the block in which they occur. \ No newline at end of file +Lucene queries to the beginning of the block in which they occur. + +## Query Optimization by Topological Sorting of Statements + +GraphDB seems to have inherent algorithms to optimize the query time, however query performance of Fuseki highly depends +on the order of the query statements. For example, a query such as the one below: + +```sparql +PREFIX beol: +PREFIX knora-api: + +CONSTRUCT { + ?letter knora-api:isMainResource true . + ?letter ?linkingProp1 ?person1 . + ?letter ?linkingProp2 ?person2 . + ?letter beol:creationDate ?date . +} WHERE { + ?letter beol:creationDate ?date . + + ?letter ?linkingProp1 ?person1 . + FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) + + ?letter ?linkingProp2 ?person2 . + FILTER(?linkingProp2 = beol:hasAuthor || ?linkingProp2 = beol:hasRecipient ) + + ?person1 beol:hasIAFIdentifier ?gnd1 . + ?gnd1 knora-api:valueAsString "(DE-588)118531379" . + + ?person2 beol:hasIAFIdentifier ?gnd2 . + ?gnd2 knora-api:valueAsString "(DE-588)118696149" . +} ORDER BY ?date +``` + +takes a very long time with Fuseki. The performance of this query can be improved +by moving up the statements with literal objects that are not dependent on any other statement: + +``` + ?gnd1 knora-api:valueAsString "(DE-588)118531379" . + ?gnd2 knora-api:valueAsString "(DE-588)118696149" . +``` + +The rest of the query then reads: + +``` + ?person1 beol:hasIAFIdentifier ?gnd1 . + ?person2 beol:hasIAFIdentifier ?gnd2 . + ?letter ?linkingProp1 ?person1 . + FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) + + ?letter ?linkingProp2 ?person2 . + FILTER(?linkingProp2 = beol:hasAuthor || ?linkingProp2 = beol:hasRecipient ) + ?letter beol:creationDate ?date . +``` + +Since we cannot expect clients to know about performance of triplestores in order to write efficient queries, we have +implemented an optimization method to automatically rearrange the statements of the given queries. +Upon receiving the Gravsearch query, the algorithm converts the query to a graph. For each statement pattern, +the subject of the statement is the origin node, the predicate is a directed edge, and the object +is the target node. For the query above, this conversion would result in the following graph: + +![query_graph](figures/query_graph.png) + +The [Graph for Scala](http://www.scala-graph.org/) library is used to construct the graph and sort it using [Kahn's +topological sorting algorithm](https://en.wikipedia.org/wiki/Topological_sorting#Kahn's_algorithm). + +The algorithm returns the nodes of the graph ordered in several layers, where the +root element `?letter` is in layer 0, `[?date, ?person1, ?person2]` are in layer 1, `[?gnd1, ?gnd2]` in layer 2, and the +leaf nodes `[(DE-588)118531379, (DE-588)118696149]` are given in the last layer (i.e. layer 3). +According to Kahn's algorithm, there are multiple valid permutations of the topological order. The graph in the example + above has 24 valid permutations of topological order. Here are two of them (nodes are ordered from left to right with the highest + order to the lowest): + +- `(?letter, ?date, ?person2, ?person1, ?gnd2, ?gnd1, (DE-588)118696149, (DE-588)118531379)` +- `(?letter, ?date, ?person1, ?person2, ?gnd1, ?gnd2, (DE-588)118531379, (DE-588)118696149)`. + +From all valid topological orders, one is chosen based on certain criteria; for example, the leaf should node should not +belong to a statement that has predicate `rdf:type`, since that could match all resources of the specified type. +Once the best order is chosen, it is used to re-arrange the query +statements. Starting from the last leaf node, i.e. +`(DE-588)118696149`, the method finds the statement pattern which has this node as its object, and brings this statement +to the top of the query. This rearrangement continues so that the statements with the fewest dependencies on other +statements are all brought to the top of the query. The resulting query is as follows: + +```sparql +PREFIX beol: +PREFIX knora-api: + +CONSTRUCT { + ?letter knora-api:isMainResource true . + ?letter ?linkingProp1 ?person1 . + ?letter ?linkingProp2 ?person2 . + ?letter beol:creationDate ?date . +} WHERE { + ?gnd2 knora-api:valueAsString "(DE-588)118696149" . + ?gnd1 knora-api:valueAsString "(DE-588)118531379" . + ?person2 beol:hasIAFIdentifier ?gnd2 . + ?person1 beol:hasIAFIdentifier ?gnd1 . + ?letter ?linkingProp2 ?person2 . + ?letter ?linkingProp1 ?person1 . + ?letter beol:creationDate ?date . + FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) + FILTER(?linkingProp2 = beol:hasAuthor || ?linkingProp2 = beol:hasRecipient ) +} ORDER BY ?date +``` + +Note that position of the FILTER statements does not play a significant role in the optimization. + +If a Gravsearch query contains statements in `UNION`, `OPTIONAL`, `MINUS`, or +`FILTER NOT EXISTS`, they are reordered +by defining a graph per block. For example, consider the following query with `UNION`: + +```sparql +{ + ?thing anything:hasRichtext ?richtext . + FILTER knora-api:matchText(?richtext, "test") + ?thing anything:hasInteger ?int . + ?int knora-api:intValueAsInt 1 . +} +UNION +{ + ?thing anything:hasText ?text . + FILTER knora-api:matchText(?text, "test") + ?thing anything:hasInteger ?int . + ?int knora-api:intValueAsInt 3 . +} +``` +This would result in one graph per block of the `UNION`. Each graph is then sorted, and the statements of its +block are rearranged according to the topological order of graph. This is the result: + +```sparql +{ + ?int knora-api:intValueAsInt 1 . + ?thing anything:hasRichtext ?richtext . + ?thing anything:hasInteger ?int . + FILTER(knora-api:matchText(?richtext, "test")) +} UNION { + ?int knora-api:intValueAsInt 3 . + ?thing anything:hasText ?text . + ?thing anything:hasInteger ?int . + FILTER(knora-api:matchText(?text, "test")) +} +``` + +### Cyclic Graphs + +The topological sorting algorithm can only be used for DAGs (directed acyclic graphs). However, +a Gravsearch query can contains statements that result in a cyclic graph, e.g.: + +``` +PREFIX anything: +PREFIX knora-api: + +CONSTRUCT { + ?thing knora-api:isMainResource true . +} WHERE { + ?thing anything:hasOtherThing ?thing1 . + ?thing1 anything:hasOtherThing ?thing2 . + ?thing2 anything:hasOtherThing ?thing . + +``` + +In this case, the algorithm tries to break the cycles in order to sort the graph. If this is not possible, +the query statements are not reordered. diff --git a/third_party/dependencies.bzl b/third_party/dependencies.bzl index f783e67940..8bab8aac64 100644 --- a/third_party/dependencies.bzl +++ b/third_party/dependencies.bzl @@ -137,6 +137,9 @@ def dependencies(): # Additional Selenium libraries besides the ones pulled in during init # of io_bazel_rules_webtesting "org.seleniumhq.selenium:selenium-support:3.141.59", + + # Graph for Scala + "org.scala-graph:graph-core_2.12:1.13.1", ], repositories = [ "https://repo.maven.apache.org/maven2", @@ -187,6 +190,7 @@ BASE_TEST_DEPENDENCIES = [ "@maven//:org_scalatest_scalatest_shouldmatchers_2_12", "@maven//:org_scalatest_scalatest_compatible", "@maven//:org_scalactic_scalactic_2_12", + "@maven//:org_scala_graph_graph_core_2_12", ] BASE_TEST_DEPENDENCIES_WITH_JSON = BASE_TEST_DEPENDENCIES + [ diff --git a/webapi/src/main/resources/application.conf b/webapi/src/main/resources/application.conf index 151af9735e..aca986c94a 100644 --- a/webapi/src/main/resources/application.conf +++ b/webapi/src/main/resources/application.conf @@ -292,6 +292,21 @@ app { "Benjamin Geer " ] } + + gravsearch-dependency-optimisation { + description = "Optimise Gravsearch queries by reordering query patterns according to their dependencies." + + available-versions = [ 1 ] + default-version = 1 + enabled-by-default = yes + override-allowed = yes + expiration-date = "2021-12-01T00:00:00Z" + + developer-emails = [ + "Sepideh Alassi " + "Benjamin Geer " + ] + } } shacl { diff --git a/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel b/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel index f6f697eea2..15d4298463 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel +++ b/webapi/src/main/scala/org/knora/webapi/messages/BUILD.bazel @@ -44,5 +44,6 @@ scala_library( "@maven//:org_scala_lang_scala_reflect", "@maven//:org_slf4j_slf4j_api", "@maven//:org_springframework_security_spring_security_core", + "@maven//:org_scala_graph_graph_core_2_12", ], ) diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/AbstractPrequeryGenerator.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/AbstractPrequeryGenerator.scala index 71f4c2f5df..60bd9718a0 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/AbstractPrequeryGenerator.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/AbstractPrequeryGenerator.scala @@ -55,10 +55,6 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, // suffix appended to variables that are returned by a SPARQL aggregation function. protected val groupConcatVariableSuffix = "__Concat" - // A set of types that can be treated as dates by the knora-api:toSimpleDate function. - private val dateTypes: Set[IRI] = - Set(OntologyConstants.KnoraApiV2Complex.DateValue, OntologyConstants.KnoraApiV2Complex.StandoffTag) - /** * A container for a generated variable representing a value literal. * @@ -352,14 +348,14 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, } - val (maybeSubjectTypeIri: Option[SmartIri], subjectIsResource: Boolean) = + val maybeSubjectType: Option[NonPropertyTypeInfo] = typeInspectionResult.getTypeOfEntity(statementPattern.subj) match { - case Some(NonPropertyTypeInfo(subjectTypeIri, isResourceType, _)) => (Some(subjectTypeIri), isResourceType) - case _ => (None, false) + case Some(nonPropertyTypeInfo: NonPropertyTypeInfo) => Some(nonPropertyTypeInfo) + case _ => None } // Is the subject of the statement a resource? - if (subjectIsResource) { + if (maybeSubjectType.exists(_.isResourceType)) { // Yes. Is the object of the statement also a resource? if (propertyTypeInfo.objectIsResourceType) { // Yes. This is a link property. Make sure that the object is either an IRI or a variable (cannot be a literal). @@ -490,7 +486,7 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, if (querySchema == ApiV2Complex) { // Yes. If the subject is a standoff tag and the object is a resource, that's an error, because the client // has to use the knora-api:standoffLink function instead. - if (maybeSubjectTypeIri.contains(OntologyConstants.KnoraApiV2Complex.StandoffTag.toSmartIri) && propertyTypeInfo.objectIsResourceType) { + if (maybeSubjectType.exists(_.isStandoffTagType) && propertyTypeInfo.objectIsResourceType) { throw GravsearchException( s"Invalid statement pattern (use the knora-api:standoffLink function instead): ${statementPattern.toSparql.trim}") } else { @@ -769,7 +765,7 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, case xsdLiteral: XsdLiteral if xsdLiteral.datatype.toString == OntologyConstants.KnoraApiV2Simple.ListNode => xsdLiteral.value - case other => + case _ => throw GravsearchException(s"Invalid type for literal ${OntologyConstants.KnoraApiV2Simple.ListNode}") } @@ -1259,7 +1255,7 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, val langLiteral: XsdLiteral = compareExpression.rightArg match { case strLiteral: XsdLiteral if strLiteral.datatype == OntologyConstants.Xsd.String.toSmartIri => strLiteral - case other => + case _ => throw GravsearchException( s"Right argument of comparison statement must be a string literal for use with 'lang' function call") } @@ -1821,9 +1817,8 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, val standoffTagVar = functionCallExpression.getArgAsQueryVar(pos = 1) typeInspectionResult.getTypeOfEntity(standoffTagVar) match { - case Some(nonPropertyTypeInfo: NonPropertyTypeInfo) - if nonPropertyTypeInfo.typeIri.toString == OntologyConstants.KnoraApiV2Complex.StandoffTag => - () + case Some(nonPropertyTypeInfo: NonPropertyTypeInfo) if nonPropertyTypeInfo.isStandoffTagType => () + case _ => throw GravsearchException( s"The second argument of ${functionIri.toSparql} must represent a knora-api:StandoffTag") @@ -1930,7 +1925,7 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, typeInspectionResult.getTypeOfEntity(dateBaseVar) match { case Some(nonPropInfo: NonPropertyTypeInfo) => - if (!dateTypes.contains(nonPropInfo.typeIri.toString)) { + if (!(nonPropInfo.isStandoffTagType || nonPropInfo.typeIri.toString == OntologyConstants.KnoraApiV2Complex.DateValue)) { throw GravsearchException( s"${dateBaseVar.toSparql} must represent a knora-api:DateValue or a knora-api:StandoffDateTag") } @@ -2042,53 +2037,4 @@ abstract class AbstractPrequeryGenerator(constructClause: ConstructClause, } } - - /** - * Optimises a query by removing `rdf:type` statements that are known to be redundant. A redundant - * `rdf:type` statement gives the type of a variable whose type is already restricted by its - * use with a property that can only be used with that type (unless the property - * statement is in an `OPTIONAL` block). - * - * @param patterns the query patterns. - * @return the optimised query patterns. - */ - protected def removeEntitiesInferredFromProperty(patterns: Seq[QueryPattern]): Seq[QueryPattern] = { - - // Collect all entities which are used as subject or object of an OptionalPattern. - val optionalEntities = patterns - .filter { - case OptionalPattern(_) => true - case _ => false - } - .flatMap { - case optionalPattern: OptionalPattern => - optionalPattern.patterns.flatMap { - case pattern: StatementPattern => - GravsearchTypeInspectionUtil.maybeTypeableEntity(pattern.subj) ++ GravsearchTypeInspectionUtil - .maybeTypeableEntity(pattern.obj) - case _ => None - } - case _ => None - } - - // remove statements whose predicate is rdf:type, type of subject is inferred from a property, and the subject is not in optionalEntities. - val optimisedPatterns = patterns.filter { - case statementPattern: StatementPattern => - statementPattern.pred match { - case iriRef: IriRef => - val subject = GravsearchTypeInspectionUtil.maybeTypeableEntity(statementPattern.subj) - subject match { - case Some(typeableEntity) => - !(iriRef.iri.toString == OntologyConstants.Rdf.Type && typeInspectionResult.entitiesInferredFromProperties.keySet - .contains(typeableEntity) - && !optionalEntities.contains(typeableEntity)) - case _ => true - } - - case _ => true - } - case _ => true - } - optimisedPatterns - } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/GravsearchQueryOptimisationFactory.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/GravsearchQueryOptimisationFactory.scala new file mode 100644 index 0000000000..23c38cd519 --- /dev/null +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/GravsearchQueryOptimisationFactory.scala @@ -0,0 +1,387 @@ +/* + * Copyright © 2015-2018 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ + +package org.knora.webapi.messages.util.search.gravsearch.prequery + +import org.knora.webapi.ApiV2Schema +import org.knora.webapi.feature.{Feature, FeatureFactory, FeatureFactoryConfig} +import org.knora.webapi.messages.OntologyConstants +import org.knora.webapi.messages.util.search._ +import org.knora.webapi.messages.util.search.gravsearch.types.{ + GravsearchTypeInspectionResult, + GravsearchTypeInspectionUtil, + TypeableEntity +} +import scalax.collection.Graph +import scalax.collection.GraphEdge.DiHyperEdge + +/** + * Represents optimisation algorithms that transform Gravsearch input queries. + * + * @param typeInspectionResult the type inspection result. + * @param querySchema the query schema. + */ +abstract class GravsearchQueryOptimisationFeature(protected val typeInspectionResult: GravsearchTypeInspectionResult, + protected val querySchema: ApiV2Schema) { + + /** + * Performs the optimisation. + * + * @param patterns the query patterns. + * @return the optimised query patterns. + */ + def optimiseQueryPatterns(patterns: Seq[QueryPattern]): Seq[QueryPattern] +} + +/** + * A feature factory that constructs Gravsearch query optimisation algorithms. + */ +object GravsearchQueryOptimisationFactory extends FeatureFactory { + + /** + * Returns a [[GravsearchQueryOptimisationFeature]] implementing one or more optimisations, depending + * on the feature factory configuration. + * + * @param typeInspectionResult the type inspection result. + * @param querySchema the query schema. + * @param featureFactoryConfig the feature factory configuration. + * @return a [[GravsearchQueryOptimisationFeature]] implementing one or more optimisations. + */ + def getGravsearchQueryOptimisationFeature( + typeInspectionResult: GravsearchTypeInspectionResult, + querySchema: ApiV2Schema, + featureFactoryConfig: FeatureFactoryConfig): GravsearchQueryOptimisationFeature = { + new GravsearchQueryOptimisationFeature(typeInspectionResult: GravsearchTypeInspectionResult, + querySchema: ApiV2Schema) { + override def optimiseQueryPatterns(patterns: Seq[QueryPattern]): Seq[QueryPattern] = { + if (featureFactoryConfig.getToggle("gravsearch-dependency-optimisation").isEnabled) { + new ReorderPatternsByDependencyOptimisationFeature(typeInspectionResult, querySchema).optimiseQueryPatterns( + new RemoveEntitiesInferredFromPropertyOptimisationFeature(typeInspectionResult, querySchema) + .optimiseQueryPatterns(patterns) + ) + } else { + new RemoveEntitiesInferredFromPropertyOptimisationFeature(typeInspectionResult, querySchema) + .optimiseQueryPatterns(patterns) + } + } + } + } +} + +/** + * Optimises a query by removing `rdf:type` statements that are known to be redundant. A redundant + * `rdf:type` statement gives the type of a variable whose type is already restricted by its + * use with a property that can only be used with that type (unless the property + * statement is in an `OPTIONAL` block). + */ +class RemoveEntitiesInferredFromPropertyOptimisationFeature(typeInspectionResult: GravsearchTypeInspectionResult, + querySchema: ApiV2Schema) + extends GravsearchQueryOptimisationFeature(typeInspectionResult, querySchema) + with Feature { + + /** + * Performs the optimisation. + * + * @param patterns the query patterns. + * @return the optimised query patterns. + */ + override def optimiseQueryPatterns(patterns: Seq[QueryPattern]): Seq[QueryPattern] = { + + // Collect all entities which are used as subject or object of an OptionalPattern. + val optionalEntities: Seq[TypeableEntity] = patterns + .collect { + case optionalPattern: OptionalPattern => optionalPattern + } + .flatMap { + case optionalPattern: OptionalPattern => + optionalPattern.patterns.flatMap { + case pattern: StatementPattern => + GravsearchTypeInspectionUtil.maybeTypeableEntity(pattern.subj) ++ GravsearchTypeInspectionUtil + .maybeTypeableEntity(pattern.obj) + + case _ => None + } + + case _ => None + } + + // Remove statements whose predicate is rdf:type, type of subject is inferred from a property, + // and the subject is not in optionalEntities. + patterns.filterNot { + case statementPattern: StatementPattern => + // Is the predicate an IRI? + statementPattern.pred match { + case predicateIriRef: IriRef => + // Yes. Is this an rdf:type statement? + if (predicateIriRef.iri.toString == OntologyConstants.Rdf.Type) { + // Yes. Is the subject a typeable entity? + val subjectAsTypeableEntity = GravsearchTypeInspectionUtil.maybeTypeableEntity(statementPattern.subj) + + subjectAsTypeableEntity match { + case Some(typeableEntity) => + // Yes. Was the type of the subject inferred from another predicate? + if (typeInspectionResult.entitiesInferredFromProperties.keySet.contains(typeableEntity)) { + // Yes. Is the subject in optional entities? + if (optionalEntities.contains(typeableEntity)) { + // Yes. Keep the statement. + false + } else { + // Remove the statement. + true + } + } else { + // The type of the subject was not inferred from another predicate. Keep the statement. + false + } + + case _ => + // The subject isn't a typeable entity. Keep the statement. + false + } + } else { + // This isn't an rdf:type statement. Keep it. + false + } + + case _ => + // The predicate isn't an IRI. Keep the statement. + false + } + + case _ => + // This isn't a statement pattern. Keep it. + false + } + } +} + +/** + * Optimises query patterns by reordering them on the basis of dependencies between subjects and objects. + */ +class ReorderPatternsByDependencyOptimisationFeature(typeInspectionResult: GravsearchTypeInspectionResult, + querySchema: ApiV2Schema) + extends GravsearchQueryOptimisationFeature(typeInspectionResult, querySchema) + with Feature { + + /** + * Converts a sequence of query patterns into DAG representing dependencies between + * the subjects and objects used, performs a topological sort of the graph, and reorders + * the query patterns according to the topological order. + * + * @param statementPatterns the query patterns to be reordered. + * @return the reordered query patterns. + */ + private def createAndSortGraph(statementPatterns: Seq[StatementPattern]): Seq[QueryPattern] = { + @scala.annotation.tailrec + def makeGraphWithoutCycles(graphComponents: Seq[(String, String)]): Graph[String, DiHyperEdge] = { + val graph = graphComponents.foldLeft(Graph.empty[String, DiHyperEdge]) { (graph, edgeDef) => + val edge = DiHyperEdge(edgeDef._1, edgeDef._2) + graph + edge // add nodes and edges to graph + } + + if (graph.isCyclic) { + // get the cycle + val cycle: graph.Cycle = graph.findCycle.get + + // the cyclic node is the one that cycle starts and ends with + val cyclicNode: graph.NodeT = cycle.endNode + val cyclicEdge: graph.EdgeT = cyclicNode.edges.last + val originNodeOfCyclicEdge: String = cyclicEdge._1.value + val TargetNodeOfCyclicEdge: String = cyclicEdge._2.value + val graphComponentsWithOutCycle = + graphComponents.filterNot(edgeDef => edgeDef.equals((originNodeOfCyclicEdge, TargetNodeOfCyclicEdge))) + + makeGraphWithoutCycles(graphComponentsWithOutCycle) + } else { + graph + } + } + + def createGraph: Graph[String, DiHyperEdge] = { + val graphComponents: Seq[(String, String)] = statementPatterns.map { statementPattern => + // transform every statementPattern to pair of nodes that will consist an edge. + val node1 = statementPattern.subj.toSparql + val node2 = statementPattern.obj.toSparql + (node1, node2) + } + + makeGraphWithoutCycles(graphComponents) + } + + /** + * Finds topological orders that don't end with an object of rdf:type. + * + * @param orders the orders to be filtered. + * @param statementPatterns the statement patterns that the orders are based on. + * @return the filtered topological orders. + */ + def findOrdersNotEndingWithObjectOfRdfType( + orders: Set[Vector[Graph[String, DiHyperEdge]#NodeT]], + statementPatterns: Seq[StatementPattern]): Set[Vector[Graph[String, DiHyperEdge]#NodeT]] = { + type NodeT = Graph[String, DiHyperEdge]#NodeT + + // Find the nodes that are objects of rdf:type in the statement patterns. + val nodesThatAreObjectsOfRdfType: Set[String] = statementPatterns + .filter { statementPattern => + statementPattern.pred match { + case iriRef: IriRef => iriRef.iri.toString == OntologyConstants.Rdf.Type + case _ => false + } + } + .map { statementPattern => + statementPattern.obj.toSparql + } + .toSet + + // Filter out the topological orders that end with any of those nodes. + orders.filterNot { order: Vector[NodeT] => + nodesThatAreObjectsOfRdfType.contains(order.last.value) + } + } + + /** + * Tries to find the best topological order for the graph, by finding all possible topological orders + * and eliminating those whose last node is the object of rdf:type. + * + * @param graph the graph to be ordered. + * @param statementPatterns the statement patterns that were used to create the graph. + * @return a topological order. + */ + def findBestTopologicalOrder(graph: Graph[String, DiHyperEdge], + statementPatterns: Seq[StatementPattern]): Vector[Graph[String, DiHyperEdge]#NodeT] = { + type NodeT = Graph[String, DiHyperEdge]#NodeT + + /** + * An ordering for sorting topological orders. + */ + object TopologicalOrderOrdering extends Ordering[Vector[NodeT]] { + private def orderToString(order: Vector[NodeT]) = order.map(_.value).mkString("|") + + override def compare(left: Vector[NodeT], right: Vector[NodeT]): Int = + orderToString(left).compare(orderToString(right)) + } + + // Get all the possible topological orders for the graph. + val allTopologicalOrders: Set[Vector[NodeT]] = TopologicalSortUtil.findAllTopologicalOrderPermutations(graph) + + // Did we find any topological orders? + if (allTopologicalOrders.isEmpty) { + // No, the graph is cyclical. + Vector.empty + } else { + // Yes. Is there only one possible order? + if (allTopologicalOrders.size == 1) { + // Yes. Don't bother filtering. + allTopologicalOrders.head + } else { + // There's more than one possible order. Find orders that don't end with an object of rdf:type. + val ordersNotEndingWithObjectOfRdfType: Set[Vector[NodeT]] = + findOrdersNotEndingWithObjectOfRdfType(allTopologicalOrders, statementPatterns) + + // Are there any? + val preferredOrders = if (ordersNotEndingWithObjectOfRdfType.nonEmpty) { + // Yes. Use one of those. + ordersNotEndingWithObjectOfRdfType + } else { + // No. Use any order. + allTopologicalOrders + } + + // Sort the preferred orders to produce a deterministic result, and return one of them. + preferredOrders.min(TopologicalOrderOrdering) + } + } + } + + def sortStatementPatterns(createdGraph: Graph[String, DiHyperEdge], + statementPatterns: Seq[StatementPattern]): Seq[QueryPattern] = { + type NodeT = Graph[String, DiHyperEdge]#NodeT + + // Try to find the best topological order for the graph. + val topologicalOrder: Vector[NodeT] = + findBestTopologicalOrder(graph = createdGraph, statementPatterns = statementPatterns) + + // Was a topological order found? + if (topologicalOrder.nonEmpty) { + // Start from the end of the ordered list (the nodes with lowest degree). + // For each node, find statements which have the node as object and bring them to top. + topologicalOrder.foldRight(Vector.empty[QueryPattern]) { (node, sortedStatements) => + val statementsOfNode: Set[QueryPattern] = statementPatterns + .filter(p => p.obj.toSparql.equals(node.value)) + .toSet[QueryPattern] + sortedStatements ++ statementsOfNode.toVector + } + } else { + // No topological order found. + statementPatterns + } + } + + sortStatementPatterns(createGraph, statementPatterns) + } + + /** + * Performs the optimisation. + * + * @param patterns the query patterns. + * @return the optimised query patterns. + */ + override def optimiseQueryPatterns(patterns: Seq[QueryPattern]): Seq[QueryPattern] = { + // Separate the statement patterns from the other patterns. + val (statementPatterns: Seq[StatementPattern], otherPatterns: Seq[QueryPattern]) = + patterns.foldLeft((Vector.empty[StatementPattern], Vector.empty[QueryPattern])) { + case ((statementPatternAcc, otherPatternAcc), pattern: QueryPattern) => + pattern match { + case statementPattern: StatementPattern => (statementPatternAcc :+ statementPattern, otherPatternAcc) + case _ => (statementPatternAcc, otherPatternAcc :+ pattern) + } + } + + val sortedStatementPatterns: Seq[QueryPattern] = createAndSortGraph(statementPatterns) + + val sortedOtherPatterns: Seq[QueryPattern] = otherPatterns.map { + // sort statements inside each UnionPattern block + case unionPattern: UnionPattern => + val sortedUnionBlocks: Seq[Seq[QueryPattern]] = + unionPattern.blocks.map(block => optimiseQueryPatterns(block)) + UnionPattern(blocks = sortedUnionBlocks) + + // sort statements inside OptionalPattern + case optionalPattern: OptionalPattern => + val sortedOptionalPatterns: Seq[QueryPattern] = optimiseQueryPatterns(optionalPattern.patterns) + OptionalPattern(patterns = sortedOptionalPatterns) + + // sort statements inside MinusPattern + case minusPattern: MinusPattern => + val sortedMinusPatterns: Seq[QueryPattern] = optimiseQueryPatterns(minusPattern.patterns) + MinusPattern(patterns = sortedMinusPatterns) + + // sort statements inside FilterNotExistsPattern + case filterNotExistsPattern: FilterNotExistsPattern => + val sortedFilterNotExistsPatterns: Seq[QueryPattern] = + optimiseQueryPatterns(filterNotExistsPattern.patterns) + FilterNotExistsPattern(patterns = sortedFilterNotExistsPatterns) + + // return any other query pattern as it is + case pattern: QueryPattern => pattern + } + + sortedStatementPatterns ++ sortedOtherPatterns + } +} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformer.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformer.scala index 202a3e6921..084aff64a9 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformer.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformer.scala @@ -20,6 +20,7 @@ package org.knora.webapi.messages.util.search.gravsearch.prequery import org.knora.webapi.ApiV2Schema +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.util.search._ import org.knora.webapi.messages.util.search.gravsearch.types.GravsearchTypeInspectionResult @@ -31,10 +32,12 @@ import org.knora.webapi.messages.util.search.gravsearch.types.GravsearchTypeInsp * @param constructClause the CONSTRUCT clause from the input query. * @param typeInspectionResult the result of type inspection of the input query. * @param querySchema the ontology schema used in the input query. + * @param featureFactoryConfig the feature factory configuration. */ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformer(constructClause: ConstructClause, typeInspectionResult: GravsearchTypeInspectionResult, - querySchema: ApiV2Schema) + querySchema: ApiV2Schema, + featureFactoryConfig: FeatureFactoryConfig) extends AbstractPrequeryGenerator(constructClause = constructClause, typeInspectionResult = typeInspectionResult, querySchema = querySchema) @@ -87,7 +90,11 @@ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformer(constructClause } override def optimiseQueryPatterns(patterns: Seq[QueryPattern]): Seq[QueryPattern] = { - removeEntitiesInferredFromProperty(patterns) + GravsearchQueryOptimisationFactory + .getGravsearchQueryOptimisationFeature(typeInspectionResult = typeInspectionResult, + querySchema = querySchema, + featureFactoryConfig = featureFactoryConfig) + .optimiseQueryPatterns(patterns) } override def transformLuceneQueryPattern(luceneQueryPattern: LuceneQueryPattern): Seq[QueryPattern] = diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformer.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformer.scala index e1861c65ef..1ef31a96ef 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformer.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformer.scala @@ -21,6 +21,7 @@ package org.knora.webapi.messages.util.search.gravsearch.prequery import org.knora.webapi._ import org.knora.webapi.exceptions.{AssertionException, GravsearchException} +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.util.search._ import org.knora.webapi.messages.util.search.gravsearch.types.{ GravsearchTypeInspectionResult, @@ -39,11 +40,13 @@ import org.knora.webapi.settings.KnoraSettingsImpl * @param typeInspectionResult the result of type inspection of the input query. * @param querySchema the ontology schema used in the input query. * @param settings application settings. + * @param featureFactoryConfig the feature factory configuration. */ class NonTriplestoreSpecificGravsearchToPrequeryTransformer(constructClause: ConstructClause, typeInspectionResult: GravsearchTypeInspectionResult, querySchema: ApiV2Schema, - settings: KnoraSettingsImpl) + settings: KnoraSettingsImpl, + featureFactoryConfig: FeatureFactoryConfig) extends AbstractPrequeryGenerator( constructClause = constructClause, typeInspectionResult = typeInspectionResult, @@ -408,6 +411,10 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformer(constructClause: Con * @return the optimised query patterns. */ override def optimiseQueryPatterns(patterns: Seq[QueryPattern]): Seq[QueryPattern] = { - removeEntitiesInferredFromProperty(patterns) + GravsearchQueryOptimisationFactory + .getGravsearchQueryOptimisationFeature(typeInspectionResult = typeInspectionResult, + querySchema = querySchema, + featureFactoryConfig = featureFactoryConfig) + .optimiseQueryPatterns(patterns) } } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/TopologicalSortUtil.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/TopologicalSortUtil.scala new file mode 100644 index 0000000000..cef35e4470 --- /dev/null +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/TopologicalSortUtil.scala @@ -0,0 +1,85 @@ +/* + * Copyright © 2015-2018 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ + +package org.knora.webapi.messages.util.search.gravsearch.prequery + +import scalax.collection.Graph +import scalax.collection.GraphEdge.DiHyperEdge + +import scala.collection.mutable + +/** + * A utility for finding all topological orders of a graph. + * Based on [[https://github.com/scala-graph/scala-graph/issues/129#issuecomment-485398400]]. + */ +object TopologicalSortUtil { + + /** + * Finds all possible topological order permutations of a graph. If the graph is cyclical, returns an empty set. + * + * @param graph the graph to be sorted. + * @tparam T the type of the nodes in the graph. + */ + def findAllTopologicalOrderPermutations[T](graph: Graph[T, DiHyperEdge]): Set[Vector[Graph[T, DiHyperEdge]#NodeT]] = { + type NodeT = Graph[T, DiHyperEdge]#NodeT + + def findPermutations(listOfLists: List[Vector[NodeT]]): List[Vector[NodeT]] = { + def makePermutations(next: Vector[NodeT], acc: List[Vector[NodeT]]): List[Vector[NodeT]] = { + next.permutations.toList.flatMap(i => acc.map(j => j ++ i)) + } + + @scala.annotation.tailrec + def makePermutationsRec(next: Vector[NodeT], + rest: List[Vector[NodeT]], + acc: List[Vector[NodeT]]): List[Vector[NodeT]] = { + if (rest.isEmpty) { + makePermutations(next, acc) + } else { + makePermutationsRec(rest.head, rest.tail, makePermutations(next, acc)) + } + } + + listOfLists match { + case Nil => Nil + case one :: Nil => one.permutations.toList + case one :: two :: tail => makePermutationsRec(two, tail, one.permutations.toList) + } + } + + // Accumulates topological orders. + val allOrders: Set[Vector[NodeT]] = graph.topologicalSort match { + // Is there any topological order? + case Right(topOrder) => + // Yes. Find all valid permutations. + val nodesOfLayers: List[Vector[NodeT]] = + topOrder.toLayered.iterator.foldRight(List.empty[Vector[NodeT]]) { (layer, acc) => + val layerNodes: Vector[NodeT] = layer._2.toVector + layerNodes +: acc + } + + findPermutations(nodesOfLayers).toSet + + case Left(_) => + // No, The graph has a cycle, so don't try to sort it. + Set.empty[Vector[NodeT]] + } + + allOrders.filter(_.nonEmpty) + } +} diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectionResult.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectionResult.scala index 1bbb0d126b..6a7875f306 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectionResult.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectionResult.scala @@ -33,12 +33,24 @@ sealed trait GravsearchEntityTypeInfo * @param objectTypeIri an IRI representing the type of the objects of the property. * @param objectIsResourceType `true` if the property's object type is a resource type. Property is a link. * @param objectIsValueType `true` if the property's object type is a value type. Property is not a link. + * @param objectIsStandoffTagType `true` if the property's object type is a standoff tag type. Property is not a link. */ case class PropertyTypeInfo(objectTypeIri: SmartIri, objectIsResourceType: Boolean = false, - objectIsValueType: Boolean = false) + objectIsValueType: Boolean = false, + objectIsStandoffTagType: Boolean = false) extends GravsearchEntityTypeInfo { override def toString: String = s"knora-api:objectType ${IriRef(objectTypeIri).toSparql}" + + /** + * Converts this [[PropertyTypeInfo]] to a [[NonPropertyTypeInfo]]. + */ + def toNonPropertyTypeInfo: NonPropertyTypeInfo = NonPropertyTypeInfo( + typeIri = objectTypeIri, + isResourceType = objectIsResourceType, + isValueType = objectIsValueType, + isStandoffTagType = objectIsStandoffTagType + ) } /** @@ -48,10 +60,24 @@ case class PropertyTypeInfo(objectTypeIri: SmartIri, * @param typeIri an IRI representing the entity's type. * @param isResourceType `true` if this is a resource type. * @param isValueType `true` if this is a value type. + * @param isStandoffTagType `true` if this is a standoff tag type. */ -case class NonPropertyTypeInfo(typeIri: SmartIri, isResourceType: Boolean = false, isValueType: Boolean = false) +case class NonPropertyTypeInfo(typeIri: SmartIri, + isResourceType: Boolean = false, + isValueType: Boolean = false, + isStandoffTagType: Boolean = false) extends GravsearchEntityTypeInfo { override def toString: String = s"rdf:type ${IriRef(typeIri).toSparql}" + + /** + * Converts this [[NonPropertyTypeInfo]] to a [[PropertyTypeInfo]]. + */ + def toPropertyTypeInfo: PropertyTypeInfo = PropertyTypeInfo( + objectTypeIri = typeIri, + objectIsResourceType = isResourceType, + objectIsValueType = isValueType, + objectIsStandoffTagType = isStandoffTagType + ) } /** diff --git a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/InferringGravsearchTypeInspector.scala b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/InferringGravsearchTypeInspector.scala index 44b3f4fa72..06c97d95b1 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/InferringGravsearchTypeInspector.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/util/search/gravsearch/types/InferringGravsearchTypeInspector.scala @@ -138,15 +138,13 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe case Some(classDef) => // Yes. Is it a resource class? if (classDef.isResourceClass) { - // Yes. Infer rdf:type knora-api:Resource. - val inferredType = NonPropertyTypeInfo(classDef.entityInfoContent.classIri, - isResourceType = classDef.isResourceClass, - isValueType = classDef.isValueClass) + // Yes. Use that class as the inferred type. + val inferredType = NonPropertyTypeInfo(classDef.entityInfoContent.classIri, isResourceType = true) log.debug("InferTypeOfSubjectOfRdfTypePredicate: {} {} .", entityToType, inferredType) Some(inferredType) } else if (classDef.isStandoffClass) { - // It's not a resource class, it's a standoff class. Infer rdf:type knora-api:StandoffTag. - val inferredType = NonPropertyTypeInfo(OntologyConstants.KnoraApiV2Complex.StandoffTag.toSmartIri) + val inferredType = + NonPropertyTypeInfo(classDef.entityInfoContent.classIri, isStandoffTagType = true) log.debug("InferTypeOfSubjectOfRdfTypePredicate: {} {} .", entityToType, inferredType) Some(inferredType) } else { @@ -212,19 +210,11 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe entityInfo.propertyInfoMap.get(iri) match { case Some(readPropertyInfo: ReadPropertyInfoV2) => // Yes. Try to infer its knora-api:objectType from the provided information. - InferenceRuleUtil.readPropertyInfoToObjectType(readPropertyInfo, entityInfo, usageIndex.querySchema) match { - case Some(objectTypeIri: SmartIri) => - val isValue = GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(objectTypeIri.toString) - val inferredType = PropertyTypeInfo(objectTypeIri = objectTypeIri, - objectIsResourceType = readPropertyInfo.isLinkProp, - objectIsValueType = isValue) - log.debug("InferTypeOfPropertyFromItsIri: {} {} .", entityToType, inferredType) - Set(inferredType) - - case None => - // Its knora-api:objectType couldn't be inferred. - Set.empty[GravsearchEntityTypeInfo] - } + val inferredObjectTypes: Set[GravsearchEntityTypeInfo] = InferenceRuleUtil + .readPropertyInfoToObjectType(readPropertyInfo, entityInfo, usageIndex.querySchema) + .toSet + log.debug("InferTypeOfPropertyFromItsIri: {} {} .", entityToType, inferredObjectTypes.mkString(", ")) + inferredObjectTypes case None => // The ontology responder hasn't provided a definition of this property. This should have caused @@ -280,10 +270,7 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe // Yes. Use the knora-api:objectType of each PropertyTypeInfo. entityTypes.flatMap { case propertyTypeInfo: PropertyTypeInfo => - val inferredType: GravsearchEntityTypeInfo = - NonPropertyTypeInfo(propertyTypeInfo.objectTypeIri, - isResourceType = propertyTypeInfo.objectIsResourceType, - isValueType = propertyTypeInfo.objectIsValueType) + val inferredType: GravsearchEntityTypeInfo = propertyTypeInfo.toNonPropertyTypeInfo log.debug("InferTypeOfObjectFromPredicate: {} {} .", entityToType, inferredType) Some(inferredType) case _ => @@ -384,24 +371,11 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe // Yes. Has the ontology responder provided a property definition for it? entityInfo.propertyInfoMap.get(predIri) match { case Some(readPropertyInfo: ReadPropertyInfoV2) => - // Yes. Can we infer the property's knora-api:subjectType from that definition? - InferenceRuleUtil.readPropertyInfoToSubjectType(readPropertyInfo, - entityInfo, - usageIndex.querySchema) match { - case Some(subjectTypeIri: SmartIri) => - // Yes. Use that type. - val isValue = - GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(subjectTypeIri.toString) - val inferredType = NonPropertyTypeInfo(subjectTypeIri, - isResourceType = readPropertyInfo.isResourceProp, - isValueType = isValue) - log.debug("InferTypeOfSubjectFromPredicateIri: {} {} .", entityToType, inferredType) - Some(inferredType) - - case None => - // No. This rule can't infer the entity's type. - None - } + // Yes. Try to get the property's knora-api:subjectType from that definition, + // and infer that type as the type of the entity. + InferenceRuleUtil + .readPropertyInfoToSubjectType(readPropertyInfo, entityInfo, usageIndex.querySchema) + .map(_.toNonPropertyTypeInfo) case None => // The ontology responder hasn't provided a definition of this property. This should have caused @@ -455,14 +429,36 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe updatedIntermediateResult.entities.get(typeableObj) match { case Some(entityTypes: Set[GravsearchEntityTypeInfo]) => // Yes. Use those types. + + val alreadyInferredPropertyTypes: Set[PropertyTypeInfo] = + updatedIntermediateResult.entities.getOrElse(entityToType, Set.empty).collect { + case propertyTypeInfo: PropertyTypeInfo => propertyTypeInfo + } + entityTypes.flatMap { case nonPropertyTypeInfo: NonPropertyTypeInfo => - val inferredType: GravsearchEntityTypeInfo = - PropertyTypeInfo(objectTypeIri = nonPropertyTypeInfo.typeIri, - objectIsResourceType = nonPropertyTypeInfo.isResourceType, - nonPropertyTypeInfo.isValueType) - log.debug("InferTypeOfPredicateFromObject: {} {} .", entityToType, inferredType) - Some(inferredType) + // Is this type a subclass of an object type that we already have for this property, + // which we may have got from the property's definition in an ontology? + val baseClassesOfInferredType: Set[SmartIri] = + entityInfo.classInfoMap.get(nonPropertyTypeInfo.typeIri) match { + case Some(classDef) => classDef.allBaseClasses.toSet + case None => Set.empty + } + + val isSubclassOfAlreadyInferredType: Boolean = alreadyInferredPropertyTypes.exists { + alreadyInferredType: PropertyTypeInfo => + baseClassesOfInferredType.contains(alreadyInferredType.objectTypeIri) + } + + if (!isSubclassOfAlreadyInferredType) { + // No. Use the inferred type. + val inferredType: GravsearchEntityTypeInfo = nonPropertyTypeInfo.toPropertyTypeInfo + log.debug("InferTypeOfPredicateFromObject: {} {} .", entityToType, inferredType) + Some(inferredType) + } else { + // Yes. Don't infer the more specific type for the property. + None + } case _ => None @@ -506,9 +502,14 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe val typesFromFilters: Set[GravsearchEntityTypeInfo] = usageIndex.typedEntitiesInFilters.get(entityToType) match { case Some(typesFromFilters: Set[SmartIri]) => // Yes. Return those types. - typesFromFilters.map { typeFromFilter => - val isValue = GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(typeFromFilter.toString) - val inferredType = NonPropertyTypeInfo(typeFromFilter, isResourceType = !isValue, isValueType = isValue) + typesFromFilters.map { typeFromFilter: SmartIri => + val isValueType = GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(typeFromFilter.toString) + val isStandoffTagType = typeFromFilter.toString == OntologyConstants.KnoraApiV2Complex.StandoffTag + val isResourceType = !(isValueType || isStandoffTagType) + val inferredType = NonPropertyTypeInfo(typeFromFilter, + isResourceType = isResourceType, + isValueType = isValueType, + isStandoffTagType = isStandoffTagType) log.debug("InferTypeOfEntityFromKnownTypeInFilter: {} {} .", entityToType, inferredType) inferredType } @@ -551,24 +552,10 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe // Has the ontology responder provided a definition of this property? entityInfo.propertyInfoMap.get(propertyIri) match { case Some(readPropertyInfo: ReadPropertyInfoV2) => - // Yes. Can we determine the property's knora-api:objectType from that definition? - InferenceRuleUtil.readPropertyInfoToObjectType(readPropertyInfo, - entityInfo, - usageIndex.querySchema) match { - case Some(objectTypeIri: SmartIri) => - // Yes. Use that type. - val isValue = - GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(objectTypeIri.toString) - val inferredType = PropertyTypeInfo(objectTypeIri = objectTypeIri, - objectIsResourceType = readPropertyInfo.isLinkProp, - objectIsValueType = isValue) - log.debug("InferTypeOfEntityFromKnownTypeInFilter: {} {} .", variableToType, inferredType) - Some(inferredType) - - case None => - // No knora-api:objectType could be determined for the property IRI. - None - } + // Yes. Try to determine the property's knora-api:objectType from that definition. + InferenceRuleUtil + .readPropertyInfoToObjectType(readPropertyInfo, entityInfo, usageIndex.querySchema) + .toSet case None => // The ontology responder hasn't provided a definition of this property. This should have caused @@ -675,7 +662,7 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe */ def readPropertyInfoToSubjectType(readPropertyInfo: ReadPropertyInfoV2, entityInfo: EntityInfoGetResponseV2, - querySchema: ApiV2Schema): Option[SmartIri] = { + querySchema: ApiV2Schema): Option[PropertyTypeInfo] = { // Get the knora-api:subjectType that the ontology responder provided. readPropertyInfo.entityInfoContent .getPredicateIriObject(OntologyConstants.KnoraApiV2Simple.SubjectType.toSmartIri) @@ -687,14 +674,14 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe // Is it a resource class? if (readPropertyInfo.isResourceProp) { // Yes. Use it. - Some(subjectType) + Some(PropertyTypeInfo(subjectType, objectIsResourceType = true)) } else if (subjectTypeStr == OntologyConstants.KnoraApiV2Complex.Value || OntologyConstants.KnoraApiV2Complex.ValueBaseClasses .contains(subjectTypeStr)) { // If it's knora-api:Value or one of the knora-api:ValueBase classes, don't use it. None } else if (OntologyConstants.KnoraApiV2Complex.FileValueClasses.contains(subjectTypeStr)) { // If it's a file value class, return the representation of file values in the specified schema. - Some(getFileTypeForSchema(querySchema)) + Some(PropertyTypeInfo(getFileTypeForSchema(querySchema), objectIsValueType = true)) } else { // It's not any of those types. Is it a standoff class? val isStandoffClass: Boolean = entityInfo.classInfoMap.get(subjectType) match { @@ -703,11 +690,11 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe } if (isStandoffClass) { - // Yes. Infer knora-api:subjectType knora-api:StandoffTag. - Some(OntologyConstants.KnoraApiV2Complex.StandoffTag.toSmartIri) + // Yes. Return it as a standoff tag type. + Some(PropertyTypeInfo(subjectType, objectIsStandoffTagType = true)) } else if (GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(subjectTypeStr)) { // It's not any of those. If it's a value type, return it. - Some(subjectType) + Some(PropertyTypeInfo(subjectType, objectIsValueType = true)) } else { // It's not valid in a type inspection result. This must mean it's not allowed in Gravsearch queries. throw GravsearchException( @@ -719,7 +706,7 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe // Subject type of the predicate is not known but this is a resource property? if (readPropertyInfo.isResourceProp) { // Yes. Infer knora-api:subjectType knora-api:Resource. - Some(getResourceTypeIriForSchema(querySchema)) + Some(PropertyTypeInfo(getResourceTypeIriForSchema(querySchema), objectIsResourceType = true)) } else None } } @@ -733,11 +720,11 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe */ def readPropertyInfoToObjectType(readPropertyInfo: ReadPropertyInfoV2, entityInfo: EntityInfoGetResponseV2, - querySchema: ApiV2Schema): Option[SmartIri] = { + querySchema: ApiV2Schema): Option[PropertyTypeInfo] = { // Is this a file value property? if (readPropertyInfo.isFileValueProp) { // Yes, return the representation of file values in the specified schema. - Some(getFileTypeForSchema(querySchema)) + Some(PropertyTypeInfo(getFileTypeForSchema(querySchema), objectIsValueType = true)) } else { // It's not a link property. Get the knora-api:objectType that the ontology responder provided. readPropertyInfo.entityInfoContent @@ -759,14 +746,14 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe } if (isStandoffClass) { - // Yes. Infer knora-api:objectType knora-api:StandoffTag. - Some(OntologyConstants.KnoraApiV2Complex.StandoffTag.toSmartIri) + // Yes. Return it as a standoff tag type. + Some(PropertyTypeInfo(objectType, objectIsStandoffTagType = true)) } else if (readPropertyInfo.isLinkProp) { // No. Is this a link property? - // Yes. return the object type resource class. - Some(objectType) + // Yes. Return it as a resource type. + Some(PropertyTypeInfo(objectType, objectIsResourceType = true)) } else if (GravsearchTypeInspectionUtil.GravsearchValueTypeIris.contains(objectTypeStr)) { // It's not any of those. If it's a value type, return it. - Some(objectType) + Some(PropertyTypeInfo(objectType, objectIsValueType = true)) } else { // No. This must mean it's not allowed in Gravsearch queries. throw GravsearchException( @@ -781,7 +768,9 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe } // The inference rule pipeline for the first iteration. Includes rules that cannot return additional - // information if they are run more than once. + // information if they are run more than once. It's important that InferTypeOfPropertyFromItsIri + // is run before InferTypeOfPredicateFromObject, so that the latter doesn't add a subtype of a type + // already added by the former. private val firstIterationRulePipeline = new InferTypeOfSubjectOfRdfTypePredicate( Some(new InferTypeOfPropertyFromItsIri(Some(new InferTypeOfSubjectFromPredicateIri( Some(new InferTypeOfEntityFromKnownTypeInFilter(Some(new InferTypeOfVariableFromComparisonWithPropertyIriInFilter( @@ -1009,23 +998,14 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe entityInfo: EntityInfoGetResponseV2): IntermediateTypeInspectionResult = { /** - * Returns `true` if the specified [[GravsearchEntityTypeInfo]] refers to a resource type. - */ - def getIsResourceFlags(typeInfo: GravsearchEntityTypeInfo): Boolean = { - typeInfo match { - case propertyTypeInfo: PropertyTypeInfo => propertyTypeInfo.objectIsResourceType - case nonPropertyTypeInfo: NonPropertyTypeInfo => nonPropertyTypeInfo.isResourceType - case _ => throw GravsearchException(s"There is an invalid type") - } - } - - /** - * Given a set of resource classes, this method finds a common base class. + * Given a set of classes, this method finds a common base class. * - * @param typesToBeChecked a set of resource classes. + * @param typesToBeChecked a set of classes. + * @param defaultBaseClassIri the default base class IRI if none is found. * @return the IRI of a common base class. */ - def findCommonBaseResourceClass(typesToBeChecked: Set[GravsearchEntityTypeInfo]): SmartIri = { + def findCommonBaseClass(typesToBeChecked: Set[GravsearchEntityTypeInfo], + defaultBaseClassIri: SmartIri): SmartIri = { val baseClassesOfFirstType: Seq[SmartIri] = entityInfo.classInfoMap.get(iriOfGravsearchTypeInfo(typesToBeChecked.head)) match { case Some(classDef: ReadClassInfoV2) => classDef.allBaseClasses @@ -1048,26 +1028,26 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe // returns the most specific common base class. commonBaseClasses.head } else { - InferenceRuleUtil.getResourceTypeIriForSchema(querySchema) + defaultBaseClassIri } } else { - InferenceRuleUtil.getResourceTypeIriForSchema(querySchema) + defaultBaseClassIri } } /** - * Replaces inconsistent resource types with a common base class. + * Replaces inconsistent types with a common base class. */ - def replaceInconsistentResourceTypes(acc: IntermediateTypeInspectionResult, - typedEntity: TypeableEntity, - typesToBeChecked: Set[GravsearchEntityTypeInfo], - newType: GravsearchEntityTypeInfo): IntermediateTypeInspectionResult = { + def replaceInconsistentTypes(acc: IntermediateTypeInspectionResult, + typedEntity: TypeableEntity, + typesToBeChecked: Set[GravsearchEntityTypeInfo], + newType: GravsearchEntityTypeInfo): IntermediateTypeInspectionResult = { val withoutInconsistentTypes: IntermediateTypeInspectionResult = typesToBeChecked.foldLeft(acc) { - (sanitizeResults, currType) => - sanitizeResults.removeType(typedEntity, currType) + (sanitizeResults: IntermediateTypeInspectionResult, currType: GravsearchEntityTypeInfo) => + sanitizeResults.removeType(entity = typedEntity, typeToRemove = currType) } - withoutInconsistentTypes.addTypes(typedEntity, Set(newType)) + withoutInconsistentTypes.addTypes(entity = typedEntity, entityTypes = Set(newType)) } // get inconsistent types @@ -1078,26 +1058,62 @@ class InferringGravsearchTypeInspector(nextInspector: Option[GravsearchTypeInspe inconsistentEntities.keySet.foldLeft(lastResults) { (acc, typedEntity) => // all inconsistent types val typesToBeChecked: Set[GravsearchEntityTypeInfo] = inconsistentEntities.getOrElse(typedEntity, Set.empty) - val commonBaseClassIri: SmartIri = findCommonBaseResourceClass(typesToBeChecked) - - // Are all inconsistent types NonPropertyTypeInfo and resourceType? - if (typesToBeChecked.count(elem => elem.isInstanceOf[NonPropertyTypeInfo]) == typesToBeChecked.size && - typesToBeChecked.count(elem => getIsResourceFlags(elem)) == typesToBeChecked.size) { + // Are all inconsistent types NonPropertyTypeInfo representing resource classes? + if (typesToBeChecked.forall { + case nonPropertyTypeInfo: NonPropertyTypeInfo => nonPropertyTypeInfo.isResourceType + case _ => false + }) { // Yes. Remove inconsistent types and replace with a common base class. + val commonBaseClassIri: SmartIri = + findCommonBaseClass(typesToBeChecked, InferenceRuleUtil.getResourceTypeIriForSchema(querySchema)) val newResourceType = NonPropertyTypeInfo(commonBaseClassIri, isResourceType = true) - replaceInconsistentResourceTypes(acc, typedEntity, typesToBeChecked, newResourceType) - - // No. Are they PropertyTypeInfo types with a object of a resource type? - } else if (typesToBeChecked.count(elem => elem.isInstanceOf[PropertyTypeInfo]) == typesToBeChecked.size && - typesToBeChecked.count(elem => getIsResourceFlags(elem)) == typesToBeChecked.size) { - - // Yes. Remove inconsistent types and replace with a common base class + replaceInconsistentTypes(acc = acc, + typedEntity = typedEntity, + typesToBeChecked = typesToBeChecked, + newType = newResourceType) + } else if (typesToBeChecked.forall { + case nonPropertyTypeInfo: NonPropertyTypeInfo => nonPropertyTypeInfo.isStandoffTagType + case _ => false + }) { + // No, they're NonPropertyTypeInfo representing standoff tag classes. + // Yes. Remove inconsistent types and replace with a common base class. + val commonBaseClassIri: SmartIri = + findCommonBaseClass(typesToBeChecked, OntologyConstants.KnoraApiV2Complex.StandoffTag.toSmartIri) + val newStandoffTagType = NonPropertyTypeInfo(commonBaseClassIri, isStandoffTagType = true) + replaceInconsistentTypes(acc = acc, + typedEntity = typedEntity, + typesToBeChecked = typesToBeChecked, + newType = newStandoffTagType) + } else if (typesToBeChecked.forall { + case nonPropertyTypeInfo: PropertyTypeInfo => nonPropertyTypeInfo.objectIsResourceType + case _ => false + }) { + // No, they're PropertyTypeInfo types with object types representing resource classes. + // Remove inconsistent types and replace with a common base class. + val commonBaseClassIri: SmartIri = + findCommonBaseClass(typesToBeChecked, InferenceRuleUtil.getResourceTypeIriForSchema(querySchema)) val newObjectType = PropertyTypeInfo(commonBaseClassIri, objectIsResourceType = true) - replaceInconsistentResourceTypes(acc, typedEntity, typesToBeChecked, newObjectType) - - // No. Don't touch the determined inconsistent types, later an error is returned for this. + replaceInconsistentTypes(acc = acc, + typedEntity = typedEntity, + typesToBeChecked = typesToBeChecked, + newType = newObjectType) + + } else if (typesToBeChecked.forall { + case nonPropertyTypeInfo: PropertyTypeInfo => nonPropertyTypeInfo.objectIsStandoffTagType + case _ => false + }) { + // No, they're PropertyTypeInfo types with object types representing standoff tag classes. + // Remove inconsistent types and replace with a common base class. + val commonBaseClassIri: SmartIri = + findCommonBaseClass(typesToBeChecked, OntologyConstants.KnoraApiV2Complex.StandoffTag.toSmartIri) + val newObjectType = PropertyTypeInfo(commonBaseClassIri, objectIsStandoffTagType = true) + replaceInconsistentTypes(acc = acc, + typedEntity = typedEntity, + typesToBeChecked = typesToBeChecked, + newType = newObjectType) } else { + // None of the above. Don't touch the determined inconsistent types, later an error is returned for this. acc } } diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala index 2eacb892f4..7159124955 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala @@ -407,7 +407,8 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand nonTriplestoreSpecificConstructToSelectTransformer: NonTriplestoreSpecificGravsearchToCountPrequeryTransformer = new NonTriplestoreSpecificGravsearchToCountPrequeryTransformer( constructClause = inputQuery.constructClause, typeInspectionResult = typeInspectionResult, - querySchema = inputQuery.querySchema.getOrElse(throw AssertionException(s"WhereClause has no querySchema")) + querySchema = inputQuery.querySchema.getOrElse(throw AssertionException(s"WhereClause has no querySchema")), + featureFactoryConfig = featureFactoryConfig ) nonTriplestoreSpecificPrequery: SelectQuery = QueryTraverser.transformConstructToSelect( @@ -493,7 +494,8 @@ class SearchResponderV2(responderData: ResponderData) extends ResponderWithStand constructClause = inputQuery.constructClause, typeInspectionResult = typeInspectionResult, querySchema = inputQuery.querySchema.getOrElse(throw AssertionException(s"WhereClause has no querySchema")), - settings = settings + settings = settings, + featureFactoryConfig = featureFactoryConfig ) // TODO: if the ORDER BY criterion is a property whose occurrence is not 1, then the logic does not work correctly diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/search/BUILD.bazel b/webapi/src/test/scala/org/knora/webapi/messages/util/search/BUILD.bazel index 9bedb5f558..f0d968721f 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/util/search/BUILD.bazel +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/search/BUILD.bazel @@ -32,6 +32,24 @@ scala_test( ] + BASE_TEST_DEPENDENCIES, ) +scala_test( + name = "TopologicalSortUtilSpec", + size = "small", # 60s + srcs = [ + "gravsearch/prequery/TopologicalSortUtilSpec.scala", + ], + data = [ + "//knora-ontologies", + "//test_data", + ], + jvm_flags = ["-Dconfig.resource=fuseki.conf"], + # unused_dependency_checker_mode = "warn", + deps = ALL_WEBAPI_MAIN_DEPENDENCIES + [ + "//webapi:main_library", + "//webapi:test_library", + ] + BASE_TEST_DEPENDENCIES, +) + scala_test( name = "NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec", size = "small", # 60s diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec.scala b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec.scala index 70794bb12a..1f27f2f9a0 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec.scala @@ -2,6 +2,7 @@ package org.knora.webapi.util.search.gravsearch.prequery import org.knora.webapi.CoreSpec import org.knora.webapi.exceptions.AssertionException +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.StringFormatter import org.knora.webapi.messages.admin.responder.usersmessages.UserADM @@ -13,7 +14,6 @@ import org.knora.webapi.messages.util.search.gravsearch.types.{ GravsearchTypeInspectionUtil } import org.knora.webapi.messages.util.search.gravsearch.{GravsearchParser, GravsearchQueryChecker} -import org.knora.webapi.settings.KnoraSettingsImpl import org.knora.webapi.sharedtestdata.SharedTestDataADM import scala.collection.mutable.ArrayBuffer @@ -26,7 +26,9 @@ private object CountQueryHandler { val anythingUser: UserADM = SharedTestDataADM.anythingAdminUser - def transformQuery(query: String, responderData: ResponderData, settings: KnoraSettingsImpl): SelectQuery = { + def transformQuery(query: String, + responderData: ResponderData, + featureFactoryConfig: FeatureFactoryConfig): SelectQuery = { val constructQuery = GravsearchParser.parseQuery(query) @@ -51,7 +53,8 @@ private object CountQueryHandler { new NonTriplestoreSpecificGravsearchToCountPrequeryTransformer( constructClause = constructQuery.constructClause, typeInspectionResult = typeInspectionResult, - querySchema = constructQuery.querySchema.getOrElse(throw AssertionException(s"WhereClause has no querySchema")) + querySchema = constructQuery.querySchema.getOrElse(throw AssertionException(s"WhereClause has no querySchema")), + featureFactoryConfig = featureFactoryConfig ) val nonTriplestoreSpecficPrequery: SelectQuery = QueryTraverser.transformConstructToSelect( @@ -71,30 +74,6 @@ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec extends Cor implicit val stringFormatter: StringFormatter = StringFormatter.getGeneralInstance - "The NonTriplestoreSpecificGravsearchToCountPrequeryGenerator object" should { - - "transform an input query with a decimal as an optional sort criterion and a filter" in { - - val transformedQuery = - CountQueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilter, responderData, settings) - - assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilter) - - } - - "transform an input query with a decimal as an optional sort criterion and a filter (submitted in complex schema)" in { - - val transformedQuery = - CountQueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilterComplex, - responderData, - settings) - - assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilterComplex) - - } - - } - val inputQueryWithDecimalOptionalSortCriterionAndFilter: String = """ |PREFIX anything: @@ -244,17 +223,18 @@ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec extends Cor val transformedQueryWithDecimalOptionalSortCriterionAndFilterComplex: SelectQuery = SelectQuery( + fromClause = None, variables = Vector( Count( - inputVariable = QueryVariable(variableName = "thing"), + outputVariableName = "count", distinct = true, - outputVariableName = "count" + inputVariable = QueryVariable(variableName = "thing") )), offset = 0, groupBy = Nil, orderBy = Nil, whereClause = WhereClause( - patterns = ArrayBuffer( + patterns = Vector( StatementPattern( subj = QueryVariable(variableName = "thing"), pred = IriRef( @@ -285,6 +265,15 @@ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec extends Cor ), OptionalPattern( patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "decimal"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasDecimal".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "decimalVal"), + namedGraph = None + ), StatementPattern( subj = QueryVariable(variableName = "thing"), pred = IriRef( @@ -310,15 +299,6 @@ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec extends Cor propertyPathOperator = None )) ), - StatementPattern( - subj = QueryVariable(variableName = "decimal"), - pred = IriRef( - iri = "http://www.knora.org/ontology/knora-base#valueHasDecimal".toSmartIri, - propertyPathOperator = None - ), - obj = QueryVariable(variableName = "decimalVal"), - namedGraph = None - ), FilterPattern(expression = CompareExpression( leftArg = QueryVariable(variableName = "decimalVal"), operator = CompareExpressionOperator.GREATER_THAN, @@ -336,4 +316,29 @@ class NonTriplestoreSpecificGravsearchToCountPrequeryTransformerSpec extends Cor useDistinct = true ) + "The NonTriplestoreSpecificGravsearchToCountPrequeryGenerator object" should { + + "transform an input query with a decimal as an optional sort criterion and a filter" in { + + val transformedQuery = + CountQueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilter, + responderData, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilter) + + } + + "transform an input query with a decimal as an optional sort criterion and a filter (submitted in complex schema)" in { + + val transformedQuery = + CountQueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilterComplex, + responderData, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilterComplex) + + } + + } } diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec.scala b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec.scala index 8805cbdce2..2893741de9 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec.scala @@ -2,6 +2,7 @@ package org.knora.webapi.util.search.gravsearch.prequery import org.knora.webapi.CoreSpec import org.knora.webapi.exceptions.AssertionException +import org.knora.webapi.feature.FeatureFactoryConfig import org.knora.webapi.messages.IriConversions._ import org.knora.webapi.messages.StringFormatter import org.knora.webapi.messages.admin.responder.usersmessages.UserADM @@ -27,7 +28,10 @@ private object QueryHandler { val anythingUser: UserADM = SharedTestDataADM.anythingAdminUser - def transformQuery(query: String, responderData: ResponderData, settings: KnoraSettingsImpl): SelectQuery = { + def transformQuery(query: String, + responderData: ResponderData, + settings: KnoraSettingsImpl, + featureFactoryConfig: FeatureFactoryConfig): SelectQuery = { val constructQuery = GravsearchParser.parseQuery(query) @@ -53,7 +57,8 @@ private object QueryHandler { constructClause = constructQuery.constructClause, typeInspectionResult = typeInspectionResult, querySchema = constructQuery.querySchema.getOrElse(throw AssertionException(s"WhereClause has no querySchema")), - settings = settings + settings = settings, + featureFactoryConfig = featureFactoryConfig ) val nonTriplestoreSpecificPrequery: SelectQuery = QueryTraverser.transformConstructToSelect( @@ -967,12 +972,13 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec val transformedQueryWithDecimalOptionalSortCriterionAndFilterComplex: SelectQuery = SelectQuery( + fromClause = None, variables = Vector( QueryVariable(variableName = "thing"), GroupConcat( inputVariable = QueryVariable(variableName = "decimal"), separator = StringFormatter.INFORMATION_SEPARATOR_ONE, - outputVariableName = "decimal__Concat", + outputVariableName = "decimal__Concat" ) ), offset = 0, @@ -991,7 +997,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec ) ), whereClause = WhereClause( - patterns = ArrayBuffer( + patterns = Vector( StatementPattern( subj = QueryVariable(variableName = "thing"), pred = IriRef( @@ -1022,6 +1028,15 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec ), OptionalPattern( patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "decimal"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasDecimal".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "decimalVal"), + namedGraph = None + ), StatementPattern( subj = QueryVariable(variableName = "thing"), pred = IriRef( @@ -1060,15 +1075,6 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec propertyPathOperator = None )) ), - StatementPattern( - subj = QueryVariable(variableName = "decimal"), - pred = IriRef( - iri = "http://www.knora.org/ontology/knora-base#valueHasDecimal".toSmartIri, - propertyPathOperator = None - ), - obj = QueryVariable(variableName = "decimalVal"), - namedGraph = None - ), FilterPattern(expression = CompareExpression( leftArg = QueryVariable(variableName = "decimalVal"), operator = CompareExpressionOperator.GREATER_THAN, @@ -1115,6 +1121,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec """.stripMargin val TransformedQueryWithRdfsLabelAndLiteral: SelectQuery = SelectQuery( + fromClause = None, variables = Vector(QueryVariable(variableName = "book")), offset = 0, groupBy = Vector(QueryVariable(variableName = "book")), @@ -1124,7 +1131,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec isAscending = true )), whereClause = WhereClause( - patterns = ArrayBuffer( + patterns = Vector( StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( @@ -1144,24 +1151,24 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( - iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + iri = "http://www.w3.org/2000/01/rdf-schema#label".toSmartIri, propertyPathOperator = None ), - obj = IriRef( - iri = "http://www.knora.org/ontology/0803/incunabula#book".toSmartIri, - propertyPathOperator = None + obj = XsdLiteral( + value = "Zeitgl\u00F6cklein des Lebens und Leidens Christi", + datatype = "http://www.w3.org/2001/XMLSchema#string".toSmartIri ), namedGraph = None ), StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( - iri = "http://www.w3.org/2000/01/rdf-schema#label".toSmartIri, + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, propertyPathOperator = None ), - obj = XsdLiteral( - value = "Zeitgl\u00F6cklein des Lebens und Leidens Christi", - datatype = "http://www.w3.org/2001/XMLSchema#string".toSmartIri + obj = IriRef( + iri = "http://www.knora.org/ontology/0803/incunabula#book".toSmartIri, + propertyPathOperator = None ), namedGraph = None ) @@ -1232,6 +1239,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec |}""".stripMargin val TransformedQueryWithRdfsLabelAndVariable: SelectQuery = SelectQuery( + fromClause = None, variables = Vector(QueryVariable(variableName = "book")), offset = 0, groupBy = Vector(QueryVariable(variableName = "book")), @@ -1241,7 +1249,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec isAscending = true )), whereClause = WhereClause( - patterns = ArrayBuffer( + patterns = Vector( StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( @@ -1261,22 +1269,22 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( - iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, - propertyPathOperator = None - ), - obj = IriRef( - iri = "http://www.knora.org/ontology/0803/incunabula#book".toSmartIri, + iri = "http://www.w3.org/2000/01/rdf-schema#label".toSmartIri, propertyPathOperator = None ), + obj = QueryVariable(variableName = "label"), namedGraph = None ), StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( - iri = "http://www.w3.org/2000/01/rdf-schema#label".toSmartIri, + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/0803/incunabula#book".toSmartIri, propertyPathOperator = None ), - obj = QueryVariable(variableName = "label"), namedGraph = None ), FilterPattern( @@ -1297,6 +1305,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec ) val TransformedQueryWithRdfsLabelAndRegex: SelectQuery = SelectQuery( + fromClause = None, variables = Vector(QueryVariable(variableName = "book")), offset = 0, groupBy = Vector(QueryVariable(variableName = "book")), @@ -1306,7 +1315,7 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec isAscending = true )), whereClause = WhereClause( - patterns = ArrayBuffer( + patterns = Vector( StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( @@ -1326,22 +1335,22 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( - iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, - propertyPathOperator = None - ), - obj = IriRef( - iri = "http://www.knora.org/ontology/0803/incunabula#book".toSmartIri, + iri = "http://www.w3.org/2000/01/rdf-schema#label".toSmartIri, propertyPathOperator = None ), + obj = QueryVariable(variableName = "bookLabel"), namedGraph = None ), StatementPattern( subj = QueryVariable(variableName = "book"), pred = IriRef( - iri = "http://www.w3.org/2000/01/rdf-schema#label".toSmartIri, + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/0803/incunabula#book".toSmartIri, propertyPathOperator = None ), - obj = QueryVariable(variableName = "bookLabel"), namedGraph = None ), FilterPattern( @@ -1424,6 +1433,47 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec ), OptionalPattern( patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "recipient"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "recipient"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasFamilyName".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "familyName"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "familyName"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), StatementPattern( subj = QueryVariable(variableName = "document"), pred = IriRef( @@ -1491,47 +1541,6 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec propertyPathOperator = None )) ), - StatementPattern( - subj = QueryVariable(variableName = "recipient"), - pred = IriRef( - iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, - propertyPathOperator = None - ), - obj = XsdLiteral( - value = "false", - datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri - ), - namedGraph = Some( - IriRef( - iri = "http://www.knora.org/explicit".toSmartIri, - propertyPathOperator = None - )) - ), - StatementPattern( - subj = QueryVariable(variableName = "recipient"), - pred = IriRef( - iri = "http://www.knora.org/ontology/0801/beol#hasFamilyName".toSmartIri, - propertyPathOperator = None - ), - obj = QueryVariable(variableName = "familyName"), - namedGraph = None - ), - StatementPattern( - subj = QueryVariable(variableName = "familyName"), - pred = IriRef( - iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, - propertyPathOperator = None - ), - obj = XsdLiteral( - value = "false", - datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri - ), - namedGraph = Some( - IriRef( - iri = "http://www.knora.org/explicit".toSmartIri, - propertyPathOperator = None - )) - ), LuceneQueryPattern( subj = QueryVariable(variableName = "familyName"), obj = QueryVariable(variableName = "familyName__valueHasString"), @@ -1766,166 +1775,1541 @@ class NonTriplestoreSpecificGravsearchToPrequeryTransformerSpec extends CoreSpec useDistinct = true ) - "The NonTriplestoreSpecificGravsearchToPrequeryGenerator object" should { - - "transform an input query with an optional property criterion without removing the rdf:type statement" in { - - val transformedQuery = QueryHandler.transformQuery(queryWithOptional, responderData, settings) - assert(transformedQuery === TransformedQueryWithOptional) - } - - "transform an input query with a date as a non optional sort criterion" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterion, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterion) - - } - - "transform an input query with a date as a non optional sort criterion (submitted in complex schema)" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterionComplex, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterion) - - } - - "transform an input query with a date as non optional sort criterion and a filter" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterionAndFilter, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterionAndFilter) - - } - - "transform an input query with a date as non optional sort criterion and a filter (submitted in complex schema)" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterionAndFilterComplex, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterionAndFilter) - - } - - "transform an input query with a date as an optional sort criterion" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterion, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateOptionalSortCriterion) - - } - - "transform an input query with a date as an optional sort criterion (submitted in complex schema)" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterionComplex, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateOptionalSortCriterion) - - } - - "transform an input query with a date as an optional sort criterion and a filter" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterionAndFilter, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateOptionalSortCriterionAndFilter) - - } - - "transform an input query with a date as an optional sort criterion and a filter (submitted in complex schema)" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterionAndFilterComplex, responderData, settings) - - assert(transformedQuery === transformedQueryWithDateOptionalSortCriterionAndFilter) - - } - - "transform an input query with a decimal as an optional sort criterion" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterion, responderData, settings) - - assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterion) - } - - "transform an input query with a decimal as an optional sort criterion (submitted in complex schema)" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionComplex, responderData, settings) - - assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterion) - } - - "transform an input query with a decimal as an optional sort criterion and a filter" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilter, responderData, settings) - - assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilter) - } - - "transform an input query with a decimal as an optional sort criterion and a filter (submitted in complex schema)" in { - - val transformedQuery = - QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilterComplex, responderData, settings) - - // TODO: user provided statements and statement generated for sorting should be unified (https://github.com/dhlab-basel/Knora/issues/1195) - assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilterComplex) - } - - "transform an input query using rdfs:label and a literal in the simple schema" in { - val transformedQuery = - QueryHandler.transformQuery(InputQueryWithRdfsLabelAndLiteralInSimpleSchema, responderData, settings) + val queryToReorder: String = """ + |PREFIX beol: + |PREFIX knora-api: + | + |CONSTRUCT { + | ?letter knora-api:isMainResource true . + | ?letter ?linkingProp1 ?person1 . + | ?letter ?linkingProp2 ?person2 . + | ?letter beol:creationDate ?date . + |} WHERE { + | ?letter beol:creationDate ?date . + | + | ?letter ?linkingProp1 ?person1 . + | FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) + | + | ?letter ?linkingProp2 ?person2 . + | FILTER(?linkingProp2 = beol:hasAuthor || ?linkingProp2 = beol:hasRecipient ) + | + | ?person1 beol:hasIAFIdentifier ?gnd1 . + | ?gnd1 knora-api:valueAsString "(DE-588)118531379" . + | + | ?person2 beol:hasIAFIdentifier ?gnd2 . + | ?gnd2 knora-api:valueAsString "(DE-588)118696149" . + |} ORDER BY ?date""".stripMargin + + val transformedQueryToReorder: SelectQuery = SelectQuery( + fromClause = None, + variables = Vector( + QueryVariable(variableName = "letter"), + GroupConcat( + inputVariable = QueryVariable(variableName = "person1"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "person1__Concat" + ), + GroupConcat( + inputVariable = QueryVariable(variableName = "person2"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "person2__Concat" + ), + GroupConcat( + inputVariable = QueryVariable(variableName = "date"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "date__Concat" + ), + GroupConcat( + inputVariable = QueryVariable(variableName = "letter__linkingProp1__person1__LinkValue"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "letter__linkingProp1__person1__LinkValue__Concat" + ), + GroupConcat( + inputVariable = QueryVariable(variableName = "letter__linkingProp2__person2__LinkValue"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "letter__linkingProp2__person2__LinkValue__Concat" + ) + ), + offset = 0, + groupBy = Vector( + QueryVariable(variableName = "letter"), + QueryVariable(variableName = "date__valueHasStartJDN") + ), + orderBy = Vector( + OrderCriterion( + queryVariable = QueryVariable(variableName = "date__valueHasStartJDN"), + isAscending = true + ), + OrderCriterion( + queryVariable = QueryVariable(variableName = "letter"), + isAscending = true + ) + ), + whereClause = WhereClause( + patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "gnd2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasString".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "(DE-588)118696149", + datatype = "http://www.w3.org/2001/XMLSchema#string".toSmartIri + ), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "gnd1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasString".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "(DE-588)118531379", + datatype = "http://www.w3.org/2001/XMLSchema#string".toSmartIri + ), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "person2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "person2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasIAFIdentifier".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "gnd2"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "gnd2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "person1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "person1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasIAFIdentifier".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "gnd1"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "gnd1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter"), + pred = QueryVariable(variableName = "linkingProp2"), + obj = QueryVariable(variableName = "person2"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "letter"), + pred = QueryVariable(variableName = "linkingProp2__hasLinkToValue"), + obj = QueryVariable(variableName = "letter__linkingProp2__person2__LinkValue"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "letter__linkingProp2__person2__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/knora-base#LinkValue".toSmartIri, + propertyPathOperator = None + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter__linkingProp2__person2__LinkValue"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter__linkingProp2__person2__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#object".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "person2"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter"), + pred = QueryVariable(variableName = "linkingProp1"), + obj = QueryVariable(variableName = "person1"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "letter"), + pred = QueryVariable(variableName = "linkingProp1__hasLinkToValue"), + obj = QueryVariable(variableName = "letter__linkingProp1__person1__LinkValue"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "letter__linkingProp1__person1__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/knora-base#LinkValue".toSmartIri, + propertyPathOperator = None + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter__linkingProp1__person1__LinkValue"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter__linkingProp1__person1__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#object".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "person1"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "letter"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#creationDate".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "date"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "date"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "date"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasStartJDN".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "date__valueHasStartJDN"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + FilterPattern( + expression = OrExpression( + leftArg = CompareExpression( + leftArg = QueryVariable(variableName = "linkingProp1"), + operator = CompareExpressionOperator.EQUALS, + rightArg = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasAuthor".toSmartIri, + propertyPathOperator = None + ) + ), + rightArg = CompareExpression( + leftArg = QueryVariable(variableName = "linkingProp1"), + operator = CompareExpressionOperator.EQUALS, + rightArg = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasRecipient".toSmartIri, + propertyPathOperator = None + ) + ) + )), + FilterPattern( + expression = OrExpression( + leftArg = CompareExpression( + leftArg = QueryVariable(variableName = "linkingProp2"), + operator = CompareExpressionOperator.EQUALS, + rightArg = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasAuthor".toSmartIri, + propertyPathOperator = None + ) + ), + rightArg = CompareExpression( + leftArg = QueryVariable(variableName = "linkingProp2"), + operator = CompareExpressionOperator.EQUALS, + rightArg = IriRef( + iri = "http://www.knora.org/ontology/0801/beol#hasRecipient".toSmartIri, + propertyPathOperator = None + ) + ) + )) + ), + positiveEntities = Set(), + querySchema = None + ), + limit = Some(25), + useDistinct = true + ) + + val queryToReorderWithCycle: String = """ + |PREFIX anything: + |PREFIX knora-api: + | + |CONSTRUCT { + | ?thing knora-api:isMainResource true . + |} WHERE { + | ?thing anything:hasOtherThing ?thing1 . + | ?thing1 anything:hasOtherThing ?thing2 . + | ?thing2 anything:hasOtherThing ?thing . + |} """.stripMargin + + val transformedQueryToReorderWithCycle: SelectQuery = SelectQuery( + fromClause = None, + variables = Vector(QueryVariable(variableName = "thing")), + offset = 0, + groupBy = Vector(QueryVariable(variableName = "thing")), + orderBy = Vector( + OrderCriterion( + queryVariable = QueryVariable(variableName = "thing"), + isAscending = true + )), + whereClause = WhereClause( + patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "thing2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasOtherThing".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "thing"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "thing2"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasOtherThingValue".toSmartIri, + propertyPathOperator = None + ), + obj = + QueryVariable(variableName = "thing2__httpwwwknoraorgontology0001anythinghasOtherThing__thing__LinkValue"), + namedGraph = None + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing2__httpwwwknoraorgontology0001anythinghasOtherThing__thing__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/knora-base#LinkValue".toSmartIri, + propertyPathOperator = None + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing2__httpwwwknoraorgontology0001anythinghasOtherThing__thing__LinkValue"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing2__httpwwwknoraorgontology0001anythinghasOtherThing__thing__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#object".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "thing"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasOtherThing".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "thing2"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "thing1"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasOtherThingValue".toSmartIri, + propertyPathOperator = None + ), + obj = + QueryVariable(variableName = "thing1__httpwwwknoraorgontology0001anythinghasOtherThing__thing2__LinkValue"), + namedGraph = None + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing1__httpwwwknoraorgontology0001anythinghasOtherThing__thing2__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/knora-base#LinkValue".toSmartIri, + propertyPathOperator = None + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing1__httpwwwknoraorgontology0001anythinghasOtherThing__thing2__LinkValue"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing1__httpwwwknoraorgontology0001anythinghasOtherThing__thing2__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#object".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "thing2"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasOtherThing".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "thing1"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasOtherThingValue".toSmartIri, + propertyPathOperator = None + ), + obj = + QueryVariable(variableName = "thing__httpwwwknoraorgontology0001anythinghasOtherThing__thing1__LinkValue"), + namedGraph = None + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing__httpwwwknoraorgontology0001anythinghasOtherThing__thing1__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/knora-base#LinkValue".toSmartIri, + propertyPathOperator = None + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing__httpwwwknoraorgontology0001anythinghasOtherThing__thing1__LinkValue"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = + QueryVariable(variableName = "thing__httpwwwknoraorgontology0001anythinghasOtherThing__thing1__LinkValue"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#object".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "thing1"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ) + ), + positiveEntities = Set(), + querySchema = None + ), + limit = Some(25), + useDistinct = true + ) + + val queryToReorderWithMinus: String = + """PREFIX knora-api: + |PREFIX anything: + | + |CONSTRUCT { + | ?thing knora-api:isMainResource true . + |} WHERE { + | ?thing a knora-api:Resource . + | ?thing a anything:Thing . + | MINUS { + | ?thing anything:hasInteger ?intVal . + | ?intVal a xsd:integer . + | FILTER(?intVal = 123454321 || ?intVal = 999999999) + | } + |}""".stripMargin + + val transformedQueryToReorderWithMinus: SelectQuery = SelectQuery( + fromClause = None, + variables = Vector(QueryVariable(variableName = "thing")), + offset = 0, + groupBy = Vector(QueryVariable(variableName = "thing")), + orderBy = Vector( + OrderCriterion( + queryVariable = QueryVariable(variableName = "thing"), + isAscending = true + )), + whereClause = WhereClause( + patterns = Vector( + MinusPattern(patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some(IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "intVal"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "intVal"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some(IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "intVal"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "intVal__valueHasInteger"), + namedGraph = Some(IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + FilterPattern(expression = OrExpression( + leftArg = CompareExpression( + leftArg = QueryVariable(variableName = "intVal__valueHasInteger"), + operator = CompareExpressionOperator.EQUALS, + rightArg = XsdLiteral( + value = "123454321", + datatype = "http://www.w3.org/2001/XMLSchema#integer".toSmartIri + ) + ), + rightArg = CompareExpression( + leftArg = QueryVariable(variableName = "intVal__valueHasInteger"), + operator = CompareExpressionOperator.EQUALS, + rightArg = XsdLiteral( + value = "999999999", + datatype = "http://www.w3.org/2001/XMLSchema#integer".toSmartIri + ) + ) + )) + ))), + positiveEntities = Set(), + querySchema = None + ), + limit = Some(25), + useDistinct = true + ) + + val queryToReorderWithUnion: String = + s"""PREFIX knora-api: + |PREFIX anything: + |CONSTRUCT { + | ?thing knora-api:isMainResource true . + | ?thing anything:hasInteger ?int . + | ?thing anything:hasRichtext ?richtext . + | ?thing anything:hasText ?text . + |} WHERE { + | ?thing a knora-api:Resource . + | ?thing a anything:Thing . + | ?thing anything:hasInteger ?int . + | + | { + | ?thing anything:hasRichtext ?richtext . + | FILTER knora-api:matchText(?richtext, "test") + | + | ?thing anything:hasInteger ?int . + | ?int knora-api:intValueAsInt 1 . + | } + | UNION + | { + | ?thing anything:hasText ?text . + | FILTER knora-api:matchText(?text, "test") + | + | ?thing anything:hasInteger ?int . + | ?int knora-api:intValueAsInt 3 . + | } + |} + |ORDER BY (?int)""".stripMargin + + val transformedQueryToReorderWithUnion: SelectQuery = SelectQuery( + fromClause = None, + variables = Vector( + QueryVariable(variableName = "thing"), + GroupConcat( + inputVariable = QueryVariable(variableName = "int"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "int__Concat" + ), + GroupConcat( + inputVariable = QueryVariable(variableName = "richtext"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "richtext__Concat" + ), + GroupConcat( + inputVariable = QueryVariable(variableName = "text"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "text__Concat" + ) + ), + offset = 0, + groupBy = Vector( + QueryVariable(variableName = "thing"), + QueryVariable(variableName = "int__valueHasInteger") + ), + orderBy = Vector( + OrderCriterion( + queryVariable = QueryVariable(variableName = "int__valueHasInteger"), + isAscending = true + ), + OrderCriterion( + queryVariable = QueryVariable(variableName = "thing"), + isAscending = true + ) + ), + whereClause = WhereClause( + patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "int"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "int__valueHasInteger"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + UnionPattern( + blocks = Vector( + Vector( + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "1", + datatype = "http://www.w3.org/2001/XMLSchema#integer".toSmartIri + ), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasRichtext".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "richtext"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "richtext"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "int"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "int__valueHasInteger"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + LuceneQueryPattern( + subj = QueryVariable(variableName = "richtext"), + obj = QueryVariable(variableName = "richtext__valueHasString"), + queryString = LuceneQueryString(queryString = "test"), + literalStatement = Some(StatementPattern( + subj = QueryVariable(variableName = "richtext"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasString".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "richtext__valueHasString"), + namedGraph = Some(IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + )) + ) + ), + Vector( + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "3", + datatype = "http://www.w3.org/2001/XMLSchema#integer".toSmartIri + ), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasText".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "text"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "text"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "int"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "int"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasInteger".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "int__valueHasInteger"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + LuceneQueryPattern( + subj = QueryVariable(variableName = "text"), + obj = QueryVariable(variableName = "text__valueHasString"), + queryString = LuceneQueryString(queryString = "test"), + literalStatement = Some(StatementPattern( + subj = QueryVariable(variableName = "text"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasString".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "text__valueHasString"), + namedGraph = Some(IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + )) + ) + ) + )) + ), + positiveEntities = Set(), + querySchema = None + ), + limit = Some(25), + useDistinct = true + ) + + val queryWithStandoffTagHasStartAncestor: String = + """ + |PREFIX knora-api: + |PREFIX standoff: + |PREFIX anything: + |PREFIX knora-api-simple: + | + |CONSTRUCT { + | ?thing knora-api:isMainResource true . + | ?thing anything:hasText ?text . + |} WHERE { + | ?thing a anything:Thing . + | ?thing anything:hasText ?text . + | ?text knora-api:textValueHasStandoff ?standoffDateTag . + | ?standoffDateTag a knora-api:StandoffDateTag . + | FILTER(knora-api:toSimpleDate(?standoffDateTag) = "GREGORIAN:2016-12-24 CE"^^knora-api-simple:Date) + | ?standoffDateTag knora-api:standoffTagHasStartAncestor ?standoffParagraphTag . + | ?standoffParagraphTag a standoff:StandoffParagraphTag . + |}""".stripMargin + + val transformedQueryWithStandoffTagHasStartAncestor: SelectQuery = SelectQuery( + fromClause = None, + variables = Vector( + QueryVariable(variableName = "thing"), + GroupConcat( + inputVariable = QueryVariable(variableName = "text"), + separator = StringFormatter.INFORMATION_SEPARATOR_ONE, + outputVariableName = "text__Concat" + ) + ), + offset = 0, + groupBy = Vector(QueryVariable(variableName = "thing")), + orderBy = Vector( + OrderCriterion( + queryVariable = QueryVariable(variableName = "thing"), + isAscending = true + )), + whereClause = WhereClause( + patterns = Vector( + StatementPattern( + subj = QueryVariable(variableName = "standoffParagraphTag"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/standoff#StandoffParagraphTag".toSmartIri, + propertyPathOperator = None + ), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "standoffDateTag"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#standoffTagHasStartAncestor".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "standoffParagraphTag"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "standoffDateTag"), + pred = IriRef( + iri = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type".toSmartIri, + propertyPathOperator = None + ), + obj = IriRef( + iri = "http://www.knora.org/ontology/knora-base#StandoffDateTag".toSmartIri, + propertyPathOperator = None + ), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "text"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasStandoff".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "standoffDateTag"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "thing"), + pred = IriRef( + iri = "http://www.knora.org/ontology/0001/anything#hasText".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "text"), + namedGraph = None + ), + StatementPattern( + subj = QueryVariable(variableName = "text"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#isDeleted".toSmartIri, + propertyPathOperator = None + ), + obj = XsdLiteral( + value = "false", + datatype = "http://www.w3.org/2001/XMLSchema#boolean".toSmartIri + ), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "standoffDateTag"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasStartJDN".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "standoffDateTag__valueHasStartJDN"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + StatementPattern( + subj = QueryVariable(variableName = "standoffDateTag"), + pred = IriRef( + iri = "http://www.knora.org/ontology/knora-base#valueHasEndJDN".toSmartIri, + propertyPathOperator = None + ), + obj = QueryVariable(variableName = "standoffDateTag__valueHasEndJDN"), + namedGraph = Some( + IriRef( + iri = "http://www.knora.org/explicit".toSmartIri, + propertyPathOperator = None + )) + ), + FilterPattern( + expression = AndExpression( + leftArg = CompareExpression( + leftArg = XsdLiteral( + value = "2457747", + datatype = "http://www.w3.org/2001/XMLSchema#integer".toSmartIri + ), + operator = CompareExpressionOperator.LESS_THAN_OR_EQUAL_TO, + rightArg = QueryVariable(variableName = "standoffDateTag__valueHasEndJDN") + ), + rightArg = CompareExpression( + leftArg = XsdLiteral( + value = "2457747", + datatype = "http://www.w3.org/2001/XMLSchema#integer".toSmartIri + ), + operator = CompareExpressionOperator.GREATER_THAN_OR_EQUAL_TO, + rightArg = QueryVariable(variableName = "standoffDateTag__valueHasStartJDN") + ) + )) + ), + positiveEntities = Set(), + querySchema = None + ), + limit = Some(25), + useDistinct = true + ) - assert(transformedQuery === TransformedQueryWithRdfsLabelAndLiteral) + "The NonTriplestoreSpecificGravsearchToPrequeryGenerator object" should { + + "transform an input query with an optional property criterion without removing the rdf:type statement" in { + + val transformedQuery = + QueryHandler.transformQuery(queryWithOptional, responderData, settings, defaultFeatureFactoryConfig) + assert(transformedQuery === TransformedQueryWithOptional) + } + + "transform an input query with a date as a non optional sort criterion" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterion, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterion) + + } + + "transform an input query with a date as a non optional sort criterion (submitted in complex schema)" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterionComplex, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterion) + + } + + "transform an input query with a date as non optional sort criterion and a filter" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterionAndFilter, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterionAndFilter) + + } + + "transform an input query with a date as non optional sort criterion and a filter (submitted in complex schema)" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateNonOptionalSortCriterionAndFilterComplex, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateNonOptionalSortCriterionAndFilter) + + } + + "transform an input query with a date as an optional sort criterion" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterion, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateOptionalSortCriterion) + + } + + "transform an input query with a date as an optional sort criterion (submitted in complex schema)" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterionComplex, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateOptionalSortCriterion) + + } + + "transform an input query with a date as an optional sort criterion and a filter" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterionAndFilter, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateOptionalSortCriterionAndFilter) + + } + + "transform an input query with a date as an optional sort criterion and a filter (submitted in complex schema)" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDateOptionalSortCriterionAndFilterComplex, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDateOptionalSortCriterionAndFilter) + + } + + "transform an input query with a decimal as an optional sort criterion" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterion, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterion) + } + + "transform an input query with a decimal as an optional sort criterion (submitted in complex schema)" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionComplex, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterion) + } + + "transform an input query with a decimal as an optional sort criterion and a filter" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilter, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilter) + } + + "transform an input query with a decimal as an optional sort criterion and a filter (submitted in complex schema)" in { + + val transformedQuery = + QueryHandler.transformQuery(inputQueryWithDecimalOptionalSortCriterionAndFilterComplex, + responderData, + settings, + defaultFeatureFactoryConfig) + + // TODO: user provided statements and statement generated for sorting should be unified (https://github.com/dhlab-basel/Knora/issues/1195) + assert(transformedQuery === transformedQueryWithDecimalOptionalSortCriterionAndFilterComplex) + } + + "transform an input query using rdfs:label and a literal in the simple schema" in { + val transformedQuery = + QueryHandler.transformQuery(InputQueryWithRdfsLabelAndLiteralInSimpleSchema, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery == TransformedQueryWithRdfsLabelAndLiteral) } "transform an input query using rdfs:label and a literal in the complex schema" in { val transformedQuery = - QueryHandler.transformQuery(InputQueryWithRdfsLabelAndLiteralInComplexSchema, responderData, settings) + QueryHandler.transformQuery(InputQueryWithRdfsLabelAndLiteralInComplexSchema, + responderData, + settings, + defaultFeatureFactoryConfig) assert(transformedQuery === TransformedQueryWithRdfsLabelAndLiteral) } "transform an input query using rdfs:label and a variable in the simple schema" in { val transformedQuery = - QueryHandler.transformQuery(InputQueryWithRdfsLabelAndVariableInSimpleSchema, responderData, settings) + QueryHandler.transformQuery(InputQueryWithRdfsLabelAndVariableInSimpleSchema, + responderData, + settings, + defaultFeatureFactoryConfig) assert(transformedQuery === TransformedQueryWithRdfsLabelAndVariable) } "transform an input query using rdfs:label and a variable in the complex schema" in { val transformedQuery = - QueryHandler.transformQuery(InputQueryWithRdfsLabelAndVariableInComplexSchema, responderData, settings) + QueryHandler.transformQuery(InputQueryWithRdfsLabelAndVariableInComplexSchema, + responderData, + settings, + defaultFeatureFactoryConfig) assert(transformedQuery === TransformedQueryWithRdfsLabelAndVariable) } "transform an input query using rdfs:label and a regex in the simple schema" in { val transformedQuery = - QueryHandler.transformQuery(InputQueryWithRdfsLabelAndRegexInSimpleSchema, responderData, settings) + QueryHandler.transformQuery(InputQueryWithRdfsLabelAndRegexInSimpleSchema, + responderData, + settings, + defaultFeatureFactoryConfig) assert(transformedQuery === TransformedQueryWithRdfsLabelAndRegex) } "transform an input query using rdfs:label and a regex in the complex schema" in { val transformedQuery = - QueryHandler.transformQuery(InputQueryWithRdfsLabelAndRegexInComplexSchema, responderData, settings) + QueryHandler.transformQuery(InputQueryWithRdfsLabelAndRegexInComplexSchema, + responderData, + settings, + defaultFeatureFactoryConfig) assert(transformedQuery === TransformedQueryWithRdfsLabelAndRegex) } "transform an input query with UNION scopes in the simple schema" in { val transformedQuery = - QueryHandler.transformQuery(InputQueryWithUnionScopes, responderData, settings) + QueryHandler.transformQuery(InputQueryWithUnionScopes, responderData, settings, defaultFeatureFactoryConfig) assert(transformedQuery === TransformedQueryWithUnionScopes) } + + "transform an input query with knora-api:standoffTagHasStartAncestor" in { + val transformedQuery = + QueryHandler.transformQuery(queryWithStandoffTagHasStartAncestor, + responderData, + settings, + defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryWithStandoffTagHasStartAncestor) + } + + "reorder query patterns in where clause" in { + val transformedQuery = + QueryHandler.transformQuery(queryToReorder, responderData, settings, defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryToReorder) + } + + "reorder query patterns in where clause with union" in { + val transformedQuery = + QueryHandler.transformQuery(queryToReorderWithUnion, responderData, settings, defaultFeatureFactoryConfig) + + assert(transformedQuery === transformedQueryToReorderWithUnion) + } + + "reorder query patterns in where clause with optional" in { + val transformedQuery = + QueryHandler.transformQuery(queryWithOptional, responderData, settings, defaultFeatureFactoryConfig) + + assert(transformedQuery === TransformedQueryWithOptional) + } + + "reorder query patterns with minus scope" in { + val transformedQuery = + QueryHandler.transformQuery(queryToReorderWithMinus, responderData, settings, defaultFeatureFactoryConfig) + + assert(transformedQuery == transformedQueryToReorderWithMinus) + } + + "reorder a query with a cycle" in { + val transformedQuery = + QueryHandler.transformQuery(queryToReorderWithCycle, responderData, settings, defaultFeatureFactoryConfig) + + assert(transformedQuery == transformedQueryToReorderWithCycle) + } } } diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/TopologicalSortUtilSpec.scala b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/TopologicalSortUtilSpec.scala new file mode 100644 index 0000000000..101c20f658 --- /dev/null +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/prequery/TopologicalSortUtilSpec.scala @@ -0,0 +1,78 @@ +/* + * Copyright © 2015-2018 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ + +package org.knora.webapi.util.search.gravsearch.prequery + +import org.knora.webapi.CoreSpec +import org.knora.webapi.messages.util.search.gravsearch.prequery.TopologicalSortUtil +import scalax.collection.Graph +import scalax.collection.GraphEdge._ + +/** + * Tests [[TopologicalSortUtil]]. + */ +class TopologicalSortUtilSpec extends CoreSpec() { + type NodeT = Graph[Int, DiHyperEdge]#NodeT + + private def nodesToValues(orders: Set[Vector[NodeT]]): Set[Vector[Int]] = { + orders.map { order: Vector[NodeT] => + order.map(_.value) + } + } + + "TopologicalSortUtilSpec" should { + + "return all topological orders of a graph" in { + val graph: Graph[Int, DiHyperEdge] = + Graph[Int, DiHyperEdge](DiHyperEdge[Int](2, 4), DiHyperEdge[Int](2, 7), DiHyperEdge[Int](4, 5)) + + val allOrders: Set[Vector[Int]] = nodesToValues( + TopologicalSortUtil + .findAllTopologicalOrderPermutations(graph)) + + val expectedOrders = Set( + Vector(2, 4, 7, 5), + Vector(2, 7, 4, 5) + ) + + assert(allOrders == expectedOrders) + } + + "return an empty set of orders for an empty graph" in { + val graph: Graph[Int, DiHyperEdge] = Graph[Int, DiHyperEdge]() + + val allOrders: Set[Vector[Int]] = nodesToValues( + TopologicalSortUtil + .findAllTopologicalOrderPermutations(graph)) + + assert(allOrders.isEmpty) + } + + "return an empty set of orders for a cyclic graph" in { + val graph: Graph[Int, DiHyperEdge] = + Graph[Int, DiHyperEdge](DiHyperEdge[Int](2, 4), DiHyperEdge[Int](4, 7), DiHyperEdge[Int](7, 2)) + + val allOrders: Set[Vector[Int]] = nodesToValues( + TopologicalSortUtil + .findAllTopologicalOrderPermutations(graph)) + + assert(allOrders.isEmpty) + } + } +} diff --git a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectorSpec.scala b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectorSpec.scala index 9acdec81f7..f6573b6747 100644 --- a/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectorSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/messages/util/search/gravsearch/types/GravsearchTypeInspectorSpec.scala @@ -767,86 +767,198 @@ class GravsearchTypeInspectorSpec extends CoreSpec() with ImplicitSender { entities = Map( TypeableVariable(variableName = "book4") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "titleProp1") -> PropertyTypeInfo( objectTypeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - objectIsValueType = true), + objectIsResourceType = false, + objectIsValueType = true, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "page1") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "book1") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "titleProp2") -> PropertyTypeInfo( objectTypeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - objectIsValueType = true), + objectIsResourceType = false, + objectIsValueType = true, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "page3") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableIri(iri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#partOf".toSmartIri) -> PropertyTypeInfo( objectTypeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, - objectIsResourceType = true), + objectIsResourceType = true, + objectIsValueType = false, + objectIsStandoffTagType = false + ), TypeableIri(iri = "http://rdfh.ch/1749ad09ac06".toSmartIri) -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "linkObj") -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Resource".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "title2") -> NonPropertyTypeInfo( typeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - isValueType = true), + isResourceType = false, + isValueType = true, + isStandoffTagType = false + ), TypeableIri(iri = "http://rdfh.ch/52431ecfab06".toSmartIri) -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "title3") -> NonPropertyTypeInfo( typeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - isValueType = true), + isResourceType = false, + isValueType = true, + isStandoffTagType = false + ), TypeableIri(iri = "http://rdfh.ch/dc4e3c44ac06".toSmartIri) -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableIri(iri = "http://api.knora.org/ontology/knora-api/simple/v2#isRegionOf".toSmartIri) -> PropertyTypeInfo( - objectTypeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, - objectIsResourceType = true), + objectTypeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Representation".toSmartIri, + objectIsResourceType = true, + objectIsValueType = false, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "page2") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "page4") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableIri(iri = "http://api.knora.org/ontology/knora-api/simple/v2#hasLinkTo".toSmartIri) -> PropertyTypeInfo( - objectTypeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, - objectIsResourceType = true), + objectTypeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Resource".toSmartIri, + objectIsResourceType = true, + objectIsValueType = false, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "titleProp4") -> PropertyTypeInfo( objectTypeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - objectIsValueType = true), + objectIsResourceType = false, + objectIsValueType = true, + objectIsStandoffTagType = false + ), TypeableIri(iri = "http://rdfh.ch/8d3d8f94ab06".toSmartIri) -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "title1") -> NonPropertyTypeInfo( typeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - isValueType = true), + isResourceType = false, + isValueType = true, + isStandoffTagType = false + ), TypeableVariable(variableName = "titleProp3") -> PropertyTypeInfo( objectTypeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - objectIsValueType = true), + objectIsResourceType = false, + objectIsValueType = true, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "linkProp2") -> PropertyTypeInfo( objectTypeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, - objectIsResourceType = true), + objectIsResourceType = true, + objectIsValueType = false, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "partOfProp") -> PropertyTypeInfo( objectTypeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, - objectIsResourceType = true), + objectIsResourceType = true, + objectIsValueType = false, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "title4") -> NonPropertyTypeInfo( typeIri = "http://www.w3.org/2001/XMLSchema#string".toSmartIri, - isValueType = true), + isResourceType = false, + isValueType = true, + isStandoffTagType = false + ), TypeableVariable(variableName = "book3") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableVariable(variableName = "linkProp1") -> PropertyTypeInfo( objectTypeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, - objectIsResourceType = true), + objectIsResourceType = true, + objectIsValueType = false, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "book2") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, - isResourceType = true) - )) + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ) + ), + entitiesInferredFromProperties = Map( + TypeableVariable(variableName = "page1") -> Set( + NonPropertyTypeInfo( + typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#page".toSmartIri, + isResourceType = true, + isValueType = false, + isStandoffTagType = false + )), + TypeableVariable(variableName = "linkObj") -> Set( + NonPropertyTypeInfo( + typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Resource".toSmartIri, + isResourceType = true, + isValueType = false, + isStandoffTagType = false + )), + TypeableIri(iri = "http://rdfh.ch/8d3d8f94ab06".toSmartIri) -> Set( + NonPropertyTypeInfo( + typeIri = "http://api.knora.org/ontology/knora-api/simple/v2#Region".toSmartIri, + isResourceType = true, + isValueType = false, + isStandoffTagType = false + )), + TypeableVariable(variableName = "book1") -> Set( + NonPropertyTypeInfo( + typeIri = "http://0.0.0.0:3333/ontology/0803/incunabula/simple/v2#book".toSmartIri, + isResourceType = true, + isValueType = false, + isStandoffTagType = false + )) + ) + ) val TypeInferenceResult1: GravsearchTypeInspectionResult = GravsearchTypeInspectionResult( entities = Map( @@ -984,21 +1096,50 @@ class GravsearchTypeInspectorSpec extends CoreSpec() with ImplicitSender { entities = Map( TypeableIri(iri = "http://0.0.0.0:3333/ontology/0801/beol/v2#hasText".toSmartIri) -> PropertyTypeInfo( objectTypeIri = "http://api.knora.org/ontology/knora-api/v2#TextValue".toSmartIri, - objectIsValueType = true), + objectIsResourceType = false, + objectIsValueType = true, + objectIsStandoffTagType = false + ), TypeableVariable(variableName = "text") -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/v2#TextValue".toSmartIri, - isValueType = true), + isResourceType = false, + isValueType = true, + isStandoffTagType = false + ), TypeableVariable(variableName = "letter") -> NonPropertyTypeInfo( typeIri = "http://0.0.0.0:3333/ontology/0801/beol/v2#letter".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableIri(iri = "http://rdfh.ch/biblio/up0Q0ZzPSLaULC2tlTs1sA".toSmartIri) -> NonPropertyTypeInfo( typeIri = "http://api.knora.org/ontology/knora-api/v2#Resource".toSmartIri, - isResourceType = true), + isResourceType = true, + isValueType = false, + isStandoffTagType = false + ), TypeableIri(iri = "http://api.knora.org/ontology/knora-api/v2#textValueHasStandoff".toSmartIri) -> PropertyTypeInfo( - objectTypeIri = "http://api.knora.org/ontology/knora-api/v2#StandoffTag".toSmartIri), + objectTypeIri = "http://api.knora.org/ontology/knora-api/v2#StandoffTag".toSmartIri, + objectIsResourceType = false, + objectIsValueType = false, + objectIsStandoffTagType = true + ), TypeableVariable(variableName = "standoffLinkTag") -> NonPropertyTypeInfo( - typeIri = "http://api.knora.org/ontology/knora-api/v2#StandoffTag".toSmartIri) - )) + typeIri = "http://api.knora.org/ontology/knora-api/v2#StandoffLinkTag".toSmartIri, + isResourceType = false, + isValueType = false, + isStandoffTagType = true + ) + ), + entitiesInferredFromProperties = Map( + TypeableVariable(variableName = "text") -> Set( + NonPropertyTypeInfo( + typeIri = "http://api.knora.org/ontology/knora-api/v2#TextValue".toSmartIri, + isResourceType = false, + isValueType = true, + isStandoffTagType = false + ))) + ) val QueryWithRdfsLabelAndLiteral: String = """