Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs(xmlupload): improve examples, add documentation of geometry-prop JSON format #240

Merged
merged 7 commits into from Oct 18, 2022
Merged

docs(xmlupload): improve examples, add documentation of geometry-prop JSON format #240

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b6fb6ca
edit
jnussbaum Oct 10, 2022
2ba5979
Apply suggestions from code review
jnussbaum Oct 11, 2022
81ea24a
edit
jnussbaum Oct 13, 2022
a236953
edit
jnussbaum Oct 13, 2022
94d8220
improve description of <text>
jnussbaum Oct 17, 2022
fa1230c
Merge branch 'main' into wip/dev-1404-update-docs-xmlupload
jnussbaum Oct 17, 2022
ec00596
apply reviewer's feedback
jnussbaum Oct 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
35 changes: 25 additions & 10 deletions docs/dsp-tools-usage.md
View file
Open in desktop
Expand Up @@ -89,18 +89,33 @@ dsp-tools xmlupload [options] xml_data_file.xml

The following options are available:

- `-s` | `--server` _server_: URL of the DSP server (default: 0.0.0.0:3333)
- `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
- `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
- `-V` | `--validate`: If set, only the validation of the XML file is performed.
- `-i` | `--imgdir` _dirpath_: path to the directory where the bitstream objects are stored (default: .)
- `-S` | `--sipi` _SIPIserver_: URL of the SIPI IIIF server (default: http://0.0.0.0:1024)
- `-I` | `--incremental` : If set, IRIs instead of internal IDs are expected as reference to already existing resources on DSP
- `-v` | `--verbose`: If set, more information about the uploaded resources is printed to the console.
- `-s` | `--server` (optional, default: `0.0.0.0:3333`): URL of the DSP server
- `-u` | `--user` (optional, default: `root@example.com`): username used for authentication with the DSP API
- `-p` | `--password` (optional, default: `test`): password used for authentication with the DSP API
- `-i` | `--imgdir` (optional, default: `.`): path to the directory where the bitstream objects are stored
- `-S` | `--sipi` (optional, default: `http://0.0.0.0:1024`): URL of the SIPI IIIF server
- `-I` | `--incremental` (optional) : If set, IRIs instead of internal IDs are expected as reference to already existing resources on DSP
- `-V` | `--validate` (optional): If set, the XML file will only be validated, but not uploaded.
- `-v` | `--verbose` (optional): If set, more information about the process is printed to the console.

The command is used to upload data defined in an XML file onto a DSP server. The defaults are intended for local
testing:
```bash
dsp-tools xmlupload xml_data_file.xml
```

will upload the XML file on `localhost` for local viewing. It assumes that DSP-API has been started up with the default
settings, and that potential `<bitstream>` tags contain file paths that are relative to the working directory from where
`dsp-tools` is called from.

When uploading data to a remote DSP server, you should be aware of some conventions that DaSCH follows when setting up
a DSP server:
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved

The command is used to upload data defined in an XML file onto a DSP server. The following example shows how to upload
data from the XML file `xml_data_file.xml` to the DSP server `https://admin.dasch.swiss`:
- In a server address, `admin` stands for the DSP-APP frontend that you look at in your browser
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
- `api` stands for the DSP-API (where dsp-tools sends its data to)
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
- `iiif` stands for the SIPI-server (where dsp-tools sends the multimedia files to)
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved

This means that for uploading data to the DSP server `https://admin.dasch.swiss`, you have to type the following:
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
```bash
dsp-tools xmlupload -s https://api.dasch.swiss -u root@example.com -p test -S https://iiif.dasch.swiss xml_data_file.xml
```
Expand Down
181 changes: 80 additions & 101 deletions docs/dsp-tools-xmlupload.md
View file
Open in desktop
Expand Up @@ -221,17 +221,15 @@ A complete `<resource>` element may look as follows:
</resource>
```

The `<resource>` element contains a property element (e.g. `<text-prop>`) for each property class (i.e. data field)
describing the resource. The property element itself contains one or several value elements (e.g. `<text>`) and must
have an attribute `name` with the name of the property as defined in the project specific ontology.

Example for a property element of type text (`<text-prop>`) with two value elements `<text>`:
For every property that the ontology requires, the `<resource>` element contains one property
element (e.g. `<integer-prop name="property_name>`). The property element contains one or more values.

Example of a property element of type integer with two values:
```xml
<text-prop name=":hasTranslation">
<text encoding="utf8">Dies ist eine Übersetzung.</text>
<text encoding="utf8">Dies ist eine weitere Übersetzung.</text>
</text-prop>
<integer-prop name=":hasInteger">
<integer permissions="prop-default">4711</integer>
<integer permissions="prop-default">1</integer>
</integer-prop>
```

The following property elements exist:
Expand Down Expand Up @@ -269,7 +267,7 @@ Supported file extensions:
| Representation | Supported formats |
|-----------------------------|----------------------------------------|
| `ArchiveRepresentation` | ZIP, TAR, GZ, Z, TAR.GZ, TGZ, GZIP, 7Z |
| `AudioRepresentation` | MP3, MP4, WAV |
| `AudioRepresentation` | MP3, WAV |
| `DocumentRepresentation` | PDF, DOC, DOCX, XLS, XLSX, PPT, PPTX |
| `MovingImageRepresentation` | MP4 |
| `StillImageRepresentation` | JPG, JPEG, PNG, TIF, TIFF, JP2 |
Expand All @@ -281,10 +279,11 @@ Attributes:

- `permissions` : Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)

Example:

Example of a public image inside a `StillImageRepresentation`:
```xml
<bitstream permissions="prop-restricted">postcards/images/EURUS015a.jpg</bitstream>
<resource restype=":Image" id="image_1" label="image_1" permissions="res-default">
<bitstream permissions="prop-default">postcards/images/EURUS015a.jpg</bitstream>
</resource>
```


Expand All @@ -306,16 +305,12 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a public and a hidden boolean property:
```xml
<boolean-prop name=":hasBoolean">
<boolean>true</boolean>
<boolean permissions="prop-default">true</boolean>
</boolean-prop>
```

```xml
<boolean-prop name=":hasBoolean">
<boolean-prop name=":hasHiddenBoolean">
<boolean>0</boolean>
</boolean-prop>
```
Expand All @@ -340,10 +335,10 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

A property with two color values would be defined as follows:
Example of a property with a public and a hidden color value:
```xml
<color-prop name=":hasColor">
<color>#00ff66</color>
<color permissions="prop-default">#00ff66</color>
<color>#ff00ff</color>
</color-prop>
Comment on lines 340 to 343
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why it is important here to have a public and hidden colour value. This seems out of scope (to me) at this location in the documentation where it only explains how to use the color-prop property..?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can understand your point, but it is very important for me to prevent users from creating a hidden value (<color>#ff00ff</color>). Unfortunately, the docs have only examples of hidden values until now. This is dangerous. So I wanted to find a way how I can grab the user's attention, and teach him this important difference.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the long run, I could perhaps disallow the construct <color>#ff00ff</color>, and force the user to write <color permissions="None">#ff00ff</color>. This would prevent mistakes, but it would be a breaking change.

```
Expand Down Expand Up @@ -389,16 +384,10 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

```xml
<date-prop name=":hasDate">
<date>GREGORIAN:CE:2014-01-31</date>
</date-prop>
```

Example of a property with a public and a hidden date value:
```xml
<date-prop name=":hasDate">
<date permissions="prop-default">GREGORIAN:CE:2014-01-31</date>
<date>GREGORIAN:CE:1930-09-02:CE:1930-09-03</date>
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
</date-prop>
```
Expand All @@ -422,11 +411,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a property with a public and a hidden decimal value:
```xml
<decimal-prop name=":hasDecimal">
<decimal>3.14159</decimal>
<decimal permissions="prop-default">3.14159</decimal>
<decimal>2.71828</decimal>
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
</decimal-prop>
```

Expand Down Expand Up @@ -476,11 +465,27 @@ The following example defines a polygon:
}
```

Example of a `<geometry>` element:

Example of a property with a public polygon and a hidden rectangle:
```xml
<geometry-prop name=":hasPolygon">
<geometry>{"status":"active","type"="circle","lineColor"="#ff0000","lineWidth"=2,"points":[{"x":0.5,"y":0.5}],"radius":{"x":0.1,"y":0.0}}</geometry>
<geometry-prop name=":hasRegion">
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
<geometry permissions="prop-default">
{
"status": "active", "type": "polygon", "lineColor": "#ff3333", "lineWidth": 2, "original_index": 0,
"points": [{"x": 0.1725239616613418, "y": 0.1597222222222222},
{"x": 0.8242811501597445, "y": 0.1458333333333333},
{"x": 0.8242811501597445, "y": 0.8310185185185185},
{"x": 0.1757188498402556, "y": 0.8240740740740740},
{"x": 0.1757188498402556, "y": 0.1597222222222222},
{"x": 0.1693290734824281, "y": 0.1643518518518518}]
}
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved
</geometry>
<geometry>
{
"status": "active", "type": "rectangle", "lineColor": "#ff3333", "lineWidth": 2, "original_index": 0,
"points": [{"x": 0.080985915492957750, "y": 0.16741071428571427},
{"x": 0.739436619718309900, "y": 0.72991071428571430}]
}
</geometry>
</geometry-prop>
```

Expand Down Expand Up @@ -509,11 +514,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example (city of Vienna):

Example of a property with a public link to Vienna and a hidden link to Basel:
```xml
<geoname-prop name=":hasLocation">
<geoname>2761369</geoname>
<geoname permissions="prop-default">2761369</geoname>
<geoname>2661604</geoname>
</geoname-prop>
```

Expand All @@ -536,11 +541,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a property with a public and a hidden integer value:
```xml
<integer-prop name=":hasInteger">
<integer>4711</integer>
<integer permissions="prop-default">4711</integer>
<integer>1</integer>
</integer-prop>
```

Expand All @@ -566,12 +571,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a property with a public and a hidden interval value:
```xml
<interval-prop name=":hasInterval">
<interval>60.5:120.5</interval> <!-- 0:01:00.5 - 0:02:00.5 -->
<interval>61:3600</interval> <!-- 0:01:01 - 1:00:00 -->
<interval permissions="prop-default">60.5:120.5</interval> <!-- 0:01:00.5 - 0:02:00.5 -->
<interval>61:3600</interval> <!-- 0:01:01 - 1:00:00 -->
</interval-prop>
```

Expand All @@ -597,11 +601,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a property with a public and a hidden list value:
```xml
<list-prop list="category" name=":hasCategory">
<list>physics</list>
<list permissions="prop-default">physics</list>
<list>nature</list>
</list-prop>
```

Expand All @@ -626,14 +630,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

If there is a resource defined as `<resource label="EURUS015a" restype=":Postcard" id="238807">...</resource>`, it can
be referenced as:

Example of a property with a public link to `<resource id="res_1" ...>` and a hidden link to and `<resource id="res_2" ...>`:
```xml
<resptr-prop name=":hasReferenceTo">
<resptr>238807</resptr>
<resptr permissions="prop-default">res_1</resptr>
<resptr>res_2</resptr>
</resptr-prop>
```

Expand All @@ -659,41 +660,26 @@ The `<text>` element has the following attributes:
- `comment`: a comment for this specific value (optional)

There are two variants of text: Simple (UTF8) and complex (XML). Within a text property, multiple simple and
complex text values may be mixed. Both simple and complex text values can be used inside all gui_elements
that are defined in an ontology (SimpleText, Richtext, Textarea, see [here](dsp-tools-create-ontologies.md#textvalue)).
But typically, you would use UTF8 in a SimpleText, and XML in Richtext or Textarea.


#### Simple text (UTF-8)

An example for simple text:
complex text values may be mixed. Both simple and complex text values can be used inside all `gui_element`s
that are defined in an ontology (`SimpleText`, `Richtext`, `Textarea`, see [here](dsp-tools-create-ontologies.md#textvalue)).
But typically, you would use UTF8 in a `SimpleText`, and XML in `Richtext` or `Textarea`.

Example of a public simple text and a hidden complex text:
```xml
<text-prop name=":hasComment">
<text encoding="utf8">Probe bei "Wimberger". Lokal in Wien?</text>
<text encoding="utf8" permissions="prop-default">Probe bei "Wimberger". Lokal in Wien?</text>
<text encoding="xml">
<strong>Bold text</strong> and a <a class="salsah-link" href="IRI:obj_0003:IRI">link to an ID</a>
and a <a class="salsah-link" href="http://rdfh.ch/4123/nyOODvYySV2nJ5RWRdmOdQ">link to an IRI</a>
</text>
</text-prop>
```

If your text is very long, it is not advised to add XML-"pretty-print" whitespaces after line breaks. These
whitespaces will be taken into the text field as they are.

For simple texts, it is not advised to add XML-"pretty-print" whitespaces after line breaks, because they will be taken
into the text field as they are.

#### Text with markup (XML)

dsp-tools assumes that for markup (standoff markup), the
[DSP standard mapping](https://docs.dasch.swiss/latest/DSP-API/03-apis/api-v2/xml-to-standoff-mapping/) is used (custom mapping is not yet
implemented).

Example of a text containing a link to another resource:

```xml
<text-prop name=":hasComment">
<text encoding="xml" >The <strong>third</strong> object and a <a class="salsah-link" href="IRI:obj_0003:IRI">link</a>.</text>
</text-prop>
```

Please note that the `href` option within the anchor tag (`<a>`) points to an internal resource of the DSP and has to
conform to the special format `IRI:[res-id]:IRI` where [res-id] is the resource id defined within the XML import file.
Complex texts can contain links to a resource. If the target is defined in the same XML file, it is identified by its
ID, in the format `IRI:ID:IRI`. If the target already exists on the DSP server, it is defined by its IRI.
jnussbaum marked this conversation as resolved.
Show resolved Hide resolved


### &lt;time-prop&gt;
Expand All @@ -707,8 +693,8 @@ Attributes:

#### &lt;time&gt;

The `<time>` element represents an exact datetime value in the form of `yyyy-mm-ddThh:mm:ss.sssssssssssszzzzzz`. The
following abbreviations describe this form:
The `<time>` element represents an exact datetime value in the form [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp),
which is defined as `yyyy-mm-ddThh:mm:ss.sssssssssssszzzzzz`. The following abbreviations describe this form:

- `yyyy`: a four-digit numeral that represents the year. The value cannot start with a minus (-) or a plus (+) sign.
0001 is the lexical representation of the year 1 of the Common Era (also known as 1 AD). The value cannot be 0000. The
Expand Down Expand Up @@ -743,26 +729,19 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a property with a public and a hidden time value:
```xml
<time-prop name=":hasTime">
<time>2019-10-23T13:45:12Z</time>
</time-prop>
```

The following value indicates noon on October 10, 2009, Eastern Standard Time in the United States:

```xml
<time-prop name=":hasTime">
<time>2009-10-10T12:00:00-05:00</time>
<time permissions="prop-default">2019-10-23T13:45:12Z</time>
<time>2009-10-10T12:00:00-05:00</time>
</time-prop>
```


### &lt;uri-prop&gt;

The `<uri-prop>` element is used for referencing resources with a URI. It must contain at least one `<uri>` element.
The `<uri-prop>` represents a [Uniform Resource Identifier](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier).
It must contain at least one `<uri>` element.

Attributes:

Expand All @@ -778,11 +757,11 @@ Attributes:
- `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights)
- `comment`: a comment for this specific value (optional)

Example:

Example of a property with a public and a hidden URI:
```xml
<uri-prop name=":hasURI">
<uri>http://www.groove-t-gang.ch</uri>
<uri permissions="prop-default">http://www.groove-t-gang.ch</uri>
<uri>http://dasch.swiss</uri>
</uri-prop>
```

Expand Down