Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turtle manipulation/validation #902

Open
vanhumbeecka opened this issue Mar 12, 2021 · 6 comments
Open

Turtle manipulation/validation #902

vanhumbeecka opened this issue Mar 12, 2021 · 6 comments
Labels
enhancement New feature request

Comments

@vanhumbeecka
Copy link
Contributor

Search terms you've used

Asked this question in Gitter chat. Was directed here to explain the use case.

Use Case

This is part question/feature-request.

I'm working on a prototype that extensively uses solid pods for storing/retrieving information.
Because I noticed most solid apps use schema names all over the place, I was trying to abstract away all the schema names, so potential app developers could work with datastructures without worrying about underlying schema's. In my case, I'm working on a project that involves Surveys. Based on the 'survey-ontology' (that can be found at https://cefriel.github.io/survey-ontology/docs/index.html), I'm in the process of creating a simple wrapper library to abstract away all the schema stuff (which is a public npm package for now: @consolidate/survey-ontology-ts. This package is Solid agnostic and its goals is to provide an interface for creating survey datastructures and don't worry about the underlying schemas: it can generate turtle files out of the survey you constructed with it.

However, I'm now noticing not every turtle file is solid compatible (or I'm building the wrong turtle files).
Whatever the case, I'm trying to convert my turtle files to valid SolidDatastructures. But there doesn't seem to be interfaces in the solid-client to deal with this. The only way to construct SOlidDatastructures is to use the interfaces provided by dealing with 'Quads' (which is part of the n3 library apparently).

Feature Request

It would certainly help to have some kind of 'processing' and/or 'validation' steps that can create SolidDatasets from turtle files directly. This would bypass the conversion from turtle >> quads >> load all quads in SolidDataset, and instead go directly from turtle >> SolidDataset. This could also potentially help if there is some build in validation of the provide turtlefile with helpfull messages regarding syntax and other errors.

(Another thing I noticed, is that, even though I'm able to add 'Quads' to an empty SolidDataset, that doesn't mean it's valid. When I try to save it to my solid-pod, I get a lot of errors telling me nothing was saved in the end without much further explanation. There is no build in 'validation'. You have to try and save it to know it's valid or not, but this validation step is maybe part of another feature request, so I'm leaving this here..)

Expected functionality/enhancement

Example usage:

try {
  const solidDataset = SolidDataset.fromTurtle("...")
} catch (e) {
  console.error("parsing errors: ", e)
}

So the fromTurtle could be a factory method:

class SolidDataset {
  static fromTurtle(input: string): SolidDataset
}

@vanhumbeecka vanhumbeecka added the enhancement New feature request label Mar 12, 2021
@Vinnl
Copy link
Contributor

Vinnl commented Mar 15, 2021

Hi @vanhumbeecka, thanks for the detailed description of your use case. The ability to parse plain Turtle into SolidDatasets is duly noted and something we'll hopefully be able to get to at some point.

Something we are hoping to be able to get to in the nearer-term future is an officially supported way for solid-client to accept RDF/JS data as input. That would allow you to use any parser that generates RDF/JS data to parse your Turtle (such as n3.js, but others are available) and then continue working with it using solid-client. I'll add a reminder to notify you in this issue once that is available.

Unfortunately, until then, keep in mind that we only support our publicly documented API's. Specifically, this does not include any method to insert Quads directly into SolidDatasets, which is the reason behind the errors you noticed when trying that.

One workaround I can come up with for now is to host your Turtle files somewhere (with a Content-Type of text/turtle) and to fetch those to generate SolidDatasets. For example, in a Pod:

import { overwriteFile, getSolidDataset } from "@inrupt/solid-client";

const yourTurtle = /* ... */;
await overwriteFile("https://your.pod/yourData", new Blob([yourTurtle]), { contentType: "text/turtle" });
const solidDataset = await getSolidDataset("https://your.pod/yourData");

Not great, but better than nothing...

I hope that helps for now, and apologies that I don't have a more satisfying answer at this point in time.

@pmcb55
Copy link
Contributor

pmcb55 commented Mar 16, 2021

@vanhumbeecka - by the way, if you're connected with the creators of the Survey Ontology you might notify them that the server serving up their ontology at https://w3id.org/survey-ontology# doesn't seem to be honoring the standard q quality parameter on the HTTP Accept: header. For instance, when explicitly requesting Turtle, it returns Turtle:

[~]$ curl https://w3id.org/survey-ontology# -L -H "Accept: text/turtle"
@prefix : <https://w3id.org/survey-ontology#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@base <https://w3id.org/survey-ontology#> .

<https://w3id.org/survey-ontology#> rdf:type owl:Ontology ;
                                     owl:imports <http://purl.org/linked-data/cube#> ,
                                                 <http://purl.org/wf4ever/ro#> ,
                                                 <http://purl.org/wf4ever/wfdesc#> ,
                                                 <http://purl.org/wf4ever/wfprov#> ,
                                                 <http://w3id.org/rv-ontology#> ,
                                                 <http://www.w3.org/ns/prov#> ;
                                     <http://purl.org/dc/elements/1.1/contributor> "Work on the Survey Ontology and Coney (https://survey.actionproject.eu/coney/) was partially supported by the ACTION project (grant agreement number 824603), co-funded by the European Commission under the Horizon 2020 Framework Programme."@en ;
                                     <http://purl.org/dc/elements/1.1/creator> "Damiano Scandolari"@en ,
                                                                               "Gloria Re Calegari"@en ,
                                                                               "Irene Celino"@en ,
                                                                               "Mario Scrocca"@en ;
                                     <http://purl.org/dc/elements/1.1/rights> "The Survey Ontology is available under the Creative Commons Attribution 3.0 Unported license; see http://creativecommons.org/licenses/by/3.0/. In a nutshell, you are free to copy, distribute and transmit the work; to remix/adapt the work (e.g. to import the ontology and create specializations of its elements), as long as you attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). Proper Attribution: Simply include the statement \"This work is based on the Survey Ontology, developed by the Knowledge Technologies group at Cefriel, Milan\" and link back to https://w3id.org/survey-ontology"@en ;
                                     <http://purl.org/dc/elements/1.1/subject> "Survey, Question, Answer, Variable"@en ;
                                     <http://purl.org/dc/elements/1.1/title> "The Survey Ontology"@en ;
                                     <http://purl.org/dc/terms/license> <http://creativecommons.org/licenses/by/3.0/> ;
                                     <http://purl.org/vocab/vann/preferredNamespacePrefix> "sur" ;
                                     <http://purl.org/vocab/vann/preferredNamespaceUri> "https://w3id.org/survey-ontology#" ;
                                     rdfs:comment "Ontology for surveys based on the Coney data model."@en ;
                                     rdfs:label "Survey Ontology"@en ;
                                     owl:versionInfo "V 1.0, Release 2020"@en .
:
:
:

...but when requesting Turtle with a high q value, and all other serializations with a lower q value, it still returns RDF/XML:

[~]$ curl https://w3id.org/survey-ontology# -L -H "Accept: text/turtle;q=1.0, */*;q=0.1"
<?xml version="1.0"?>
<rdf:RDF xmlns="https://w3id.org/survey-ontology#"
     xml:base="https://w3id.org/survey-ontology"
     xmlns:dc="http://purl.org/dc/elements/1.1/"
     xmlns:ns="http://www.w3.org/2003/06/sw-vocab-status/ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:wot="http://xmlns.com/wot/0.1/"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:core="http://purl.org/vocab/frbr/core#"
     xmlns:foaf="http://xmlns.com/foaf/0.1/"
     xmlns:prov="http://www.w3.org/ns/prov#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
     xmlns:skos="http://www.w3.org/2004/02/skos/core#"
     xmlns:vann="http://purl.org/vocab/vann/"
     xmlns:terms="http://purl.org/dc/terms/"
     xmlns:dcmi-terms="http://dublincore.org/documents/dcmi-terms/">
    <owl:Ontology rdf:about="https://w3id.org/survey-ontology#">
        <owl:imports rdf:resource="http://purl.org/linked-data/cube#"/>
        <owl:imports rdf:resource="http://purl.org/wf4ever/ro#"/>
        <owl:imports rdf:resource="http://purl.org/wf4ever/wfdesc#"/>
        <owl:imports rdf:resource="http://purl.org/wf4ever/wfprov#"/>
        <owl:imports rdf:resource="http://w3id.org/rv-ontology#"/>
        <owl:imports rdf:resource="http://www.w3.org/ns/prov#"/>
        <dc:contributor xml:lang="en">Work on the Survey Ontology and Coney (https://survey.actionproject.eu/coney/) was partially supported by the ACTION project (grant agreement number 824603), co-funded by the European Commission under the Horizon 2020 Framework Programme.</dc:contributor>
        <dc:creator xml:lang="en">Damiano Scandolari</dc:creator>
        <dc:creator xml:lang="en">Gloria Re Calegari</dc:creator>
        <dc:creator xml:lang="en">Irene Celino</dc:creator>
        <dc:creator xml:lang="en">Mario Scrocca</dc:creator>
        <dc:rights xml:lang="en">The Survey Ontology is available under the Creative Commons Attribution 3.0 Unported license; see http://creativecommons.org/licenses/by/3.0/. In a nutshell, you are free to copy, distribute and transmit the work; to remix/adapt the work (e.g. to import the ontology and create specializations of its elements), as long as you attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). Proper Attribution: Simply include the statement &quot;This work is based on the Survey Ontology, developed by the Knowledge Technologies group at Cefriel, Milan&quot; and link back to https://w3id.org/survey-ontology</dc:rights>
        <dc:subject xml:lang="en">Survey, Question, Answer, Variable</dc:subject>
        <dc:title xml:lang="en">The Survey Ontology</dc:title>
        <terms:license rdf:resource="http://creativecommons.org/licenses/by/3.0/"/>
        <vann:preferredNamespacePrefix>sur</vann:preferredNamespacePrefix>
        <vann:preferredNamespaceUri>https://w3id.org/survey-ontology#</vann:preferredNamespaceUri>
        <rdfs:comment xml:lang="en">Ontology for surveys based on the Coney data model.</rdfs:comment>
        <rdfs:label xml:lang="en">Survey Ontology</rdfs:label>
        <owl:versionInfo xml:lang="en">V 1.0, Release 2020</owl:versionInfo>
    </owl:Ontology>
:
:
:

As a separate issue, I'm intrigued by your desire to "to abstract away all the schema stuff", and your use of ts-rdf-mapper (which I'd never come across before). You probably know already, but attempting to map Object-Oriented beans (or POJO's) to RDF has a long, and sadly dead-project-strewn, history. For reference, see Empire, Pinto, Alibaba, PA4RDF, OLGA.
Getting into why all those projects faded away would probably get into endless, pointless, subjective, opinionated discussion - but my 2c is simply that trying to force graphs into objects is basically trying to force square pegs into round holes. There's a reason JSON-LD exists as a full-blown, and recently updated, W3C standard (i.e. JSON and objects expressed simply as nested key-value pairs just aren't expressive enough to handle the minimal complexity necessary for distributed systems between independent parties (i.e., the web today)), which I think lies at the heart of JSON-LD becoming more and more prevalent and mandated in standards. So I guess that's my way of recommending "embracing all the schema stuff", not trying to abstract it all away (except perhaps for very narrow, clearly defined, use-cases) :) !

@vanhumbeecka
Copy link
Contributor Author

vanhumbeecka commented Mar 18, 2021

Thanks for the elaborated information! I'm fairly new to this space (semantic web) so this info certainly helps.

@pmcb55 to answer your question about the use of ts-rdf-mapper for the wrapper library, it's actually for multiple reasons:

  1. for abstracting away the schemas, which reduces complexity
  2. for reuse in different parts of an application landscape (both FE and BE services) which reduces duplication

Take a look at the following sketch. There are still a lot of moving parts and unanswered question in this diagram, but the idea of the wrapper library, is to be able to use in all of these services that needs them (the 'creator' FE, the 'analytics' FE, the 'survey' FE, and possibly the aggregation service).

Untitled - Copy of Solid Survey Architecture

By using the wrapper library, I should be able to create surveys as follows (example from test folder. This is still an early version so the api could still be improved, but you get the point):

const factory = new SurveyProcedureFactory('newid')
const question1 = QuestionFactory.singleInputQuestion({text: 'Where do you live?', id: 'question_id_1'})
const question2 = QuestionFactory.singleInputQuestion({text: 'How old are you?', id: 'question_id_2'})

const survey =
  factory
  .addElement(question1)
  .addElement(question2)
  .build()
...
const turtle = survey.toTurtleString()

I also realise this is not sufficient to make the turtle output compatible with Solid datastructures (since I still need to add some subject and other URI's to the mix).

If you have any pointers in how to improve or change this approach with the use of JSON-LD, I'm eager to know.

@pmcb55
Copy link
Contributor

pmcb55 commented Mar 23, 2021

Hi @vanhumbeecka - ok, so it's certainly interesting what you're doing there. It's not how I'd personally approach the problem, but that's one of the beauties of RDF - it's really wide open to experimentation :) !
Maybe one piece of feedback is just in relation to the method .toTurtleString() (and asking if JSON-LD might improve things) - I wouldn't define methods specific to particular serializations of RDF like that. Since your const survey = ... example is just an RDF Dataset, I'd just use existing serializers to write out that RDF using any of the serializations that someone using your system might want. Maybe this article might help to explain, but in JavaScript I use this library. Unfortunately the documentation for that library isn't great, so you might find it awkward to apply it to your use-case, or perhaps in your case it's overkill. But at least the idea should be clear - basically, when it comes to RDF, I'd always recommend trying to use content negotiation to accept and produce RDF in whatever serializations your users might prefer (some might prefer Turtle, some JSON-LD, some Trig, etc. - so just plug in an existing library to allow your system support all of them!).
Best of luck with it anyways!

@calummackervoy
Copy link

Hi @vanhumbeecka I had a similar need in my own project...

I'm using axios to make requests to a server instead of the relevant functions in this library, because I like that it abstracts away the request logic and I need query parameters in my particular requests

I needed to send a SolidDataset as Turtle in a POST request, and parse a Turtle response into a SolidDataset. I did this by duplicating this library's internal functions like turtleToTriples and triplesToTurtle

Here's a gist of the code: https://gist.github.com/calummackervoy/f9cf214466044ccbefa5b49f582e00d7

e.g.

export const parseTurtleToSolidDataset = async (turtle: string) : Promise<SolidDataset> => {
    const triples = await turtleToTriples(turtle);
    const resource = createSolidDataset();
    triples.forEach((triple) => resource.add(triple));

    return resource;
}

Note that this code skips the attaching of dataset infos which would be useful to functions like fetchFrom

I wasn't sure whether to open a new issue about my use case because it's similar to this one so I thought I'd comment and ask :)

@Vinnl
Copy link
Contributor

Vinnl commented Mar 29, 2021

Thanks @calummackervoy, the 👍 on the original issue is just fine. I'm hopeful we'll have a more reliable solution at some point, and this is the issue to follow for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature request
Projects
None yet
Development

No branches or pull requests

4 participants