Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
feat!: add isSequenceOf to knora-base ontology (DEV-745) (#2061)
* add isSequenceOf to knora-base ontology

* add hasSequenceBounds to knora-base

* bump knora-base version

* better error messages

* test: write test to create isSequenceOf resource classes for video resources in ontology

* update testdata according to modified knora-base ontology

* test: add test for audio sequence in ontology

* make ontology model slightly slimmer

* add sequence bounds property to ontology tests

* fix minor oversights in knora-base

* minor refactorings

* start working on a test ontology and dataset for sequences

* add ontology constants

* add test for dynamically creating a video resource

* bump API version again

* add audio (including subclasses of isSequenceOf etc.) to sequences ontology

* add audio resource to sequences data

* add tests for audio sequences with ontology-specific sub-props of isSequenceOf etc.

* refactor: remove some code duplication

* create client test data

* add newly created test data to list of expected test data

* add noop upgrade plugin

* use Validation in OntologyHelpers

* replace more throws with validations

* fix validation

* createPropertyCommand value object in schema slice

* remove client test data

* add isSequenceOf to anything ontology

* update test data to change anything ontology

* add unit tests to ontology part of the changes

* add documentation

* first improvements according to review

* Update LangString.scala

* improve LangString value object

* add tests for LangString value object

* move language code error messages in the right place

* add documentation to SchemaCommands

* add tests for SchemaCommands

* rename V3 smartIri in OntologiesRoute

* apply feedback from review

* fix problems after merge conflicts

* add logging to unsafeMake

* tidy up language code stuff

Co-authored-by: Ivan Subotic <400790+subotic@users.noreply.github.com>
  • Loading branch information
BalduinLandolt and subotic committed Aug 3, 2022
1 parent 3eddfc4 commit 74366d4
Show file tree
Hide file tree
Showing 60 changed files with 10,565 additions and 6,973 deletions.
20 changes: 19 additions & 1 deletion build.sbt
Expand Up @@ -202,7 +202,7 @@ lazy val webapi: Project = Project(id = "webapi", base = file("webapi"))
),
buildInfoPackage := "org.knora.webapi.http.version"
)
.dependsOn(shared)
.dependsOn(shared, schemaCore)

lazy val webapiJavaRunOptions = Seq(
// "-showversion",
Expand Down Expand Up @@ -382,6 +382,24 @@ lazy val userCore = project
)
.dependsOn(shared)

// schema projects

lazy val schemaCore = project
.in(file("dsp-schema/core"))
.settings(
scalacOptions ++= Seq(
"-feature",
"-unchecked",
"-deprecation",
"-Yresolve-term-conflict:package",
"-Ymacro-annotations"
),
name := "schemaCore",
libraryDependencies ++= Dependencies.schemaCoreLibraryDependencies,
testFrameworks := Seq(new TestFramework("zio.test.sbt.ZTestFramework"))
)
.dependsOn(shared)

// Shared project

lazy val shared = project
Expand Down
25 changes: 21 additions & 4 deletions docs/02-knora-ontologies/knora-base.md
Expand Up @@ -571,22 +571,39 @@ containing metadata about the link. We can visualise the result as the following
Knora allows a user to see a link if the requesting user has permission to see the source and target resources as well
as the `kb:LinkValue`.

### Part-of (part-whole) relation between resources
### Part-Whole-Relations between Resources

#### isPartOf

A special case of linked resources are _part-of related resources_, i.e. a resource consisting of several other resources.
In order to create a part-of relation between two resources, the resource that is part of another resource needs to have
a property that is a subproperty of `kb:isPartOf`. This property needs to point to the resource class it is part of via
its predicate `knora-api:objectType`.
its predicate `kb:objectType`.
`kb:isPartOf` itself is a subproperty of `kb:hasLinkTo`. Same as described above for link properties, a corresponding
part-of value property is created automatically. This value property has the same name as the part-of property with
`Value` appended. For example, if in an ontology `data` a property `data:partOf` was defined, the corresponding value
property would be named `data:partOfValue`. This newly created property `data:partOfValue` is defined as a subproperty
of `kb:isPartOfValue`.

Part-of relations are recommended for resources of type `StillImageRepresentation`. In that case, the resource that is
part of another resource needs to have a property that is a subproperty of `knora-api:seqnum` with an integer as value.
Part-of relations are recommended for resources of type `kb:StillImageRepresentation`. In that case, the resource that is
part of another resource needs to have a property that is a subproperty of `kb:seqnum` with an integer as value.
A client can then use this information to leaf through the parts of the compound resource (p.ex. to leaf through the
pages of a book like in [this](https://docs.dasch.swiss/DSP-API/01-introduction/example-project/#resource-classes) example).

#### isSequenceOf

Similar to `kb:isPartOf` for `kb:StillImageRepresentations`, part-whole-relations can be defined for resources that have a time
dimension by using `kb:isSequenceOf`. You can use it for video or audio resources that are subtypes of `kb:MovingImageRepresentation`
and `kb:AudioRepresentation`.

`kb:isSequenceOf` is intended to be used in combination with the property `kb:hasSequenceBounds` which points to a `kb:IntervalValue`.
This defines the start and end point of the subseqence in relation to the entire audio/video resource as an [interval](#intervalvalue).
A dedicated frontend behaviour is planned, if these properties are used in combination.

There is an important difference between `kb:isSequenceOf` and `kb:isPartOf`: For `kb:isPartOf`, each part *is a* `kb:StillImageRepresentation` and
the whole consists of multiple such parts. In `kb:isSequenceOf` on the other hand, the whole is one `kb:MovingImageRepresentation` or `kb:AudioRepresentation`.
The parts only define which sub-sequence of this representation they are.

### Text with Standoff Markup

DSP-API is designed to be able to store text with markup, which can indicate formatting and structure, as well as the
Expand Down
@@ -0,0 +1,59 @@
package dsp.schema.domain

import dsp.errors.ValidationException
import dsp.valueobjects.LangString
import dsp.valueobjects.Schema
import zio.prelude.Validation

import java.time.Instant

/**
* SmartIri placeholder value object.
* WARNING: don't use this in production code. First find a solution how we deal with SmartIri in the new codebase.
*
* // TODO: this is only a placeholder for SmartIri - eventually we need a proper solution for IRI value objects.
*/
case class SmartIri(value: String)

/**
* Command/Value object representing a command to create a property on a schema/ontology.
* WARNING: This should not be used in production code before the SmartIri value object is propertly implemented.
*/
sealed abstract case class CreatePropertyCommand private (
ontologyIri: SmartIri,
lastModificationDate: Instant,
propertyIri: SmartIri,
subjectType: Option[SmartIri],
objectType: SmartIri, // must be `kb:Value`, unless it's a link property, then it should be a `kb:Resource`
label: LangString,
comment: Option[LangString],
superProperties: List[SmartIri],
guiObject: Schema.GuiObject
)

object CreatePropertyCommand {
def make(
ontologyIri: SmartIri, // TODO: should eventally be schemaId value object, etc.
lastModificationDate: Instant,
propertyIri: SmartIri,
subjectType: Option[SmartIri],
objectType: SmartIri,
label: LangString,
comment: Option[LangString],
superProperties: List[SmartIri],
guiObject: Schema.GuiObject
): Validation[ValidationException, CreatePropertyCommand] =
Validation.succeed(
new CreatePropertyCommand(
ontologyIri = ontologyIri,
lastModificationDate = lastModificationDate,
propertyIri = propertyIri,
subjectType = subjectType,
objectType = objectType,
label = label,
comment = comment,
superProperties = superProperties,
guiObject = guiObject
) {}
)
}
@@ -0,0 +1,51 @@
package dsp.schema.domain

import dsp.constants.SalsahGui
import dsp.valueobjects.LangString
import dsp.valueobjects.LanguageCode
import dsp.valueobjects.Schema
import zio._
import zio.prelude.Validation
import zio.test.Assertion._
import zio.test._

import java.time.Instant

/**
* This spec is used to test [[dsp.schema.domain.SchemaCommands]].
*/
object SchemaCommandsSpec extends ZIOSpecDefault {

def spec = (createPropertyCommandTest)

private val createPropertyCommandTest = suite("CreatePropertyCommand")(
test("create a createPropertyCommand") {
val ontologyIri = SmartIri("Ontology IRI")
val lastModificationDate = Instant.now()
val propertyIri = SmartIri("")
val subjectType = None
val objectType = SmartIri("Object Type")
val superProperties = List(SmartIri("Super Property IRI"))
(for {
label <- LangString.make(LanguageCode.en, "some label")
commentLangString <- LangString.make(LanguageCode.en, "some comment")
comment = Some(commentLangString)
guiAttribute <- Schema.GuiAttribute.make("hlist=<http://rdfh.ch/lists/082F/PbRLUy66TsK10qNP1mBwzA>")
guiElement <- Schema.GuiElement.make(SalsahGui.List)
guiObject <- Schema.GuiObject.make(guiAttributes = List(guiAttribute), guiElement = Some(guiElement))
command = CreatePropertyCommand.make(
ontologyIri = ontologyIri,
lastModificationDate = lastModificationDate,
propertyIri = propertyIri,
subjectType = subjectType,
objectType = objectType,
label = label,
comment = comment,
superProperties = superProperties,
guiObject = guiObject
)
} yield assert(command.toEither)(isRight)).toZIO
}
)

}
64 changes: 64 additions & 0 deletions dsp-shared/src/main/scala/dsp/valueobjects/LangString.scala
@@ -0,0 +1,64 @@
package dsp.valueobjects

import dsp.errors.ValidationException
import zio.prelude.Validation
import zio._

/**
* LangString value object
*/
sealed abstract case class LangString private (language: LanguageCode, value: String)

object LangString {

/**
* Creates a [[zio.prelude.Validation]] that either fails with a ValidationException or succeeds with a LangString value object.
*
* @param language a [[LanguageCode]] value object representing the language of the LangString.
* @param value the string value of the LangString.
* @return a Validation of a LangString value object.
*/
def make(language: LanguageCode, value: String): Validation[ValidationException, LangString] =
if (value.isBlank()) {
Validation.fail(ValidationException(LangStringErrorMessages.LangStringValueEmpty))
} else {
Validation.succeed(new LangString(language, value) {})
}

/**
* Creates a [[zio.prelude.Validation]] that either fails with a ValidationException or succeeds with a LangString value object.
*
* @param language a two-digit language code string representing the language of the LangString.
* @param value the string value of the LangString.
* @return a Validation of a LangString value object.
*/
def makeFromStrings(language: String, value: String): Validation[ValidationException, LangString] =
for {
languageCode <- LanguageCode.make(language)
langString <- LangString.make(languageCode, value)
} yield langString

/**
* Unsafely creates a LangString value object.
* Warning: skips all validation. Should not be used unless there is no possibility for the data to be invalid.
*
* @param languagea [[LanguageCode]] value object representing the language of the LangString.
* @param value the string value of the LangString.
* @return a LanguageCode value object
*/
def unsafeMake(language: LanguageCode, value: String): LangString =
LangString
.make(language = language, value = value)
.fold(
e => {
val unsafe = new LangString(language, value) {}
ZIO.logWarning(s"Called unsafeMake() for an invalid $unsafe: $e") // TODO-BL: get this to actually log
unsafe
},
langString => langString
)
}

object LangStringErrorMessages {
val LangStringValueEmpty = "String value cannot be empty."
}
51 changes: 51 additions & 0 deletions dsp-shared/src/main/scala/dsp/valueobjects/LanguageCode.scala
@@ -0,0 +1,51 @@
/*
* Copyright © 2021 - 2022 Swiss National Data and Service Center for the Humanities and/or DaSCH Service Platform contributors.
* SPDX-License-Identifier: Apache-2.0
*/

package dsp.valueobjects

import dsp.errors.ValidationException
import zio.prelude.Validation

/**
* LanguageCode value object.
*/
sealed abstract case class LanguageCode private (value: String)

object LanguageCode { self =>
val DE: String = "de"
val EN: String = "en"
val FR: String = "fr"
val IT: String = "it"
val RM: String = "rm"
// TODO-BL: with NewTypes we shouldn't need the strings separately anymore, as the valueobject can be used just like a string value

val SupportedLanguageCodes: Set[String] = Set(
DE,
EN,
FR,
IT,
RM
)

def make(value: String): Validation[ValidationException, LanguageCode] =
if (value.isEmpty) {
Validation.fail(ValidationException(LanguageCodeErrorMessages.LanguageCodeMissing))
} else if (!SupportedLanguageCodes.contains(value)) {
Validation.fail(ValidationException(LanguageCodeErrorMessages.LanguageCodeInvalid(value)))
} else {
Validation.succeed(new LanguageCode(value) {})
}

lazy val en: LanguageCode = new LanguageCode(EN) {}
lazy val de: LanguageCode = new LanguageCode(DE) {}
lazy val fr: LanguageCode = new LanguageCode(FR) {}
lazy val it: LanguageCode = new LanguageCode(IT) {}
lazy val rm: LanguageCode = new LanguageCode(RM) {}
}

object LanguageCodeErrorMessages {
val LanguageCodeMissing = "LanguageCode cannot be empty."
def LanguageCodeInvalid(lang: String) = s"LanguageCode '$lang' is invalid."
}
3 changes: 2 additions & 1 deletion dsp-shared/src/main/scala/dsp/valueobjects/Role.scala
Expand Up @@ -6,6 +6,7 @@
package dsp.valueobjects

import dsp.errors.BadRequestException
import dsp.valueobjects.LanguageCode
import zio.prelude.Validation

object Role {
Expand All @@ -19,7 +20,7 @@ object Role {
sealed abstract case class LangString private (value: String, isoCode: String)
object LangString {
def isIsoCodeSupported(isoCode: String): Boolean =
V2.SupportedLanguageCodes.contains(isoCode.toLowerCase) // should only lower case be supported?
LanguageCode.SupportedLanguageCodes.contains(isoCode.toLowerCase) // should only lower case be supported?

def make(value: String, isoCode: String): Validation[Throwable, LangString] =
if (value.isEmpty) {
Expand Down
20 changes: 2 additions & 18 deletions dsp-shared/src/main/scala/dsp/valueobjects/User.scala
Expand Up @@ -6,6 +6,7 @@
package dsp.valueobjects

import dsp.errors.BadRequestException
import dsp.errors.ValidationException
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder
import org.springframework.security.crypto.scrypt.SCryptPasswordEncoder
import zio._
Expand Down Expand Up @@ -199,27 +200,12 @@ object User {
Validation.succeed(new UserStatus(value) {})
}

/**
* LanguageCode value object.
*/
sealed abstract case class LanguageCode private (value: String)
object LanguageCode { self =>
def make(value: String): Validation[Throwable, LanguageCode] =
if (value.isEmpty) {
Validation.fail(BadRequestException(UserErrorMessages.LanguageCodeMissing))
} else if (!V2.SupportedLanguageCodes.contains(value)) {
Validation.fail(BadRequestException(UserErrorMessages.LanguageCodeInvalid))
} else {
Validation.succeed(new LanguageCode(value) {})
}
}

/**
* SystemAdmin value object.
*/
sealed abstract case class SystemAdmin private (value: Boolean)
object SystemAdmin {
def make(value: Boolean): Validation[Throwable, SystemAdmin] =
def make(value: Boolean): Validation[ValidationException, SystemAdmin] =
Validation.succeed(new SystemAdmin(value) {})
}
}
Expand All @@ -237,6 +223,4 @@ object UserErrorMessages {
val GivenNameInvalid = "GivenName is invalid."
val FamilyNameMissing = "FamilyName cannot be empty."
val FamilyNameInvalid = "FamilyName is invalid."
val LanguageCodeMissing = "LanguageCode cannot be empty."
val LanguageCodeInvalid = "LanguageCode is invalid."
}
13 changes: 0 additions & 13 deletions dsp-shared/src/main/scala/dsp/valueobjects/V2.scala
Expand Up @@ -19,19 +19,6 @@ import scala.util.matching.Regex
// implementations in webapi project which needed to be added temporary in order
// to avoid circular dependencies after moving value objects to separate project.
object V2 {
val DE: String = "de"
val EN: String = "en"
val FR: String = "fr"
val IT: String = "it"
val RM: String = "rm"

val SupportedLanguageCodes: Set[String] = Set(
DE,
EN,
FR,
IT,
RM
)

/**
* Represents a string with language iso. Allows sorting inside collections by value.
Expand Down

0 comments on commit 74366d4

Please sign in to comment.