Retaining non-SKOS information

[mielvds]

Quick question: we often have non-skos constructs mixed into our conceptschemes to further describe concepts.
This is more or less limited to:

• predicates from other vocabs such as skos-xl

ostr:buitengewoon-secundair-opleidingsvorm-1 a skos:Concept ;
    skosxl:altLabel ostr:buitengewoon-secundair-opleidingsvorm-1-short-label .

• additional types

<#identification>  a skos:Concept, premis:Event.

When importing this, this information is ignored, but is it also dropped? Can I get it back when exporting?

[koenedaele]

Interesting question. Not something we've worked on. Currently, anything that can't be passed through a skosprovider is dropped. It would require allowing skosproviders to retain extra info that doens't map directly on the skosprovider interface. Presumably as some form of RDF that gets passed around.

Some aspects might benefit from a more structured way, such as allowing skos-xl labels. What do you generally keep on such a label?

[mielvds]

In the example above, the skosxl labels are used to differentiate between labels via type, for example for displaying different labels in different applications. If you have multiple skos:altLabel, you cannot distinguish them.

Sidenote: because all data is stored in a relational database with ORM, these kind of changes are difficult to do. I see little reason not to use a triple store to simply store all data in RDF/SKOS. In that scenario, data would not need transformation and there would be no need to drop data. But it would be quite the architectural change.

[koenedaele]

The ORM isn't the only problem. The ecosystem depends on providers mapping input to a certain structure and outputting that again. This can be very powerful, anything that implements the provider interface can be given to the RDFProvider and turned into RDF, but it does mean that anything that's not included in the interface isn't available for the next provider. Even if we wrote a TripleStoreProvider, it wouldn't be able to pass anything not defined in the interface. So, the extra data would be dropped or couldn't be accessed in other places.

Still, it seems like a good addition to the skosprovider domain model to allow it to store any other statements that don't map to the domain model in https://skosprovider.readthedocs.io/en/latest/api.html#module-skosprovider.skos So, that would require a change there. I would make it a JSON-LD attribute, since that can be handled by pyld, which is already included in the base skosprovider package and doesn't require rdflib (which is or used to be difficullt to install due to c bindings). I've always tried to keep the basic skosprovider a very light install.

That way, extra statements can be added to a concept, collection or conceptscheme. People using the regular python code and the basic stuff (labels, notes, relations and so) can safely ignore it but to other people it doesn't get lost and it ensure that reading a rdf skos file through skosprovider_rdf and dumping it again retains all information.

[koenedaele]

Started a discussion on the skosprovider interface at OnroerendErfgoed/skosprovider#119

[mielvds]

Thanks @koenedaele , I appreciate the effort and sorry about the naive assumptions ;)

[mielvds]

I was thinking some more about this recently: if there would be a TripleStoreProvider - or more likely, a SparqlEndpointProvider (which takes a SPARQL 1.1 and SPARQL Update 1.1 endpoint as init parameters) - where SKOS info is mixed with non-SKOS info, could you not use Atramhasis "as is" to load and edit the SKOS info and not touch whatever else is in de endpoint/triple store?