diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 28db7f41e..b8d587daf 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,7 +1,7 @@ ===REMOVE=== Important! Please follow the new guidelines for naming Pull -Requests: https://docs.dasch.swiss/developers/dsp/contribution/#pull-request-guidelines +Requests: https://docs.dasch.swiss/latest/developers/dsp/contribution/#pull-request-guidelines > **Note:** If a pull request consists consists of *only one* commit when squash-merging it to main, the commit message will *not* be correct. In this case you have to manually ensure that the commit message is identical to the PR title, not the commit that the PR consists of. diff --git a/README.md b/README.md index c59bae155..a43a08112 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ For formatting Markdown files (*.md) we use the default styling configuration pr ## Publishing Publishing is automated with GitHub Actions and should _not_ be done manually. Please follow the -[Pull Request Guidelines](https://docs.dasch.swiss/developers/dsp/contribution/#pull-request-guidelines). If done +[Pull Request Guidelines](https://docs.dasch.swiss/latest/developers/dsp/contribution/#pull-request-guidelines). If done correctly, when merging a pull request into `main`, the `release-please` action will create or update a pull request for a release. This pull request will follow semantic versioning and update the change log. Once all desired features are merged, the release can be executed by merging this release pull request into `main`. This will trigger actions that diff --git a/docs/dsp-tools-create.md b/docs/dsp-tools-create.md index 05fe6a5e4..527898b84 100644 --- a/docs/dsp-tools-create.md +++ b/docs/dsp-tools-create.md @@ -9,6 +9,9 @@ determines the structure of data. Data models are typically specified by a data humanities scholar in a data modeling notation". The following sections describe the notation for ontologies in the context of DSP. +A data model as described in this document can be uploaded to a DSP server. The command to do so is described +[here](./dsp-tools-usage.md#create-a-data-model-on-a-dsp-server). + ## A short overview A complete data model definition for DSP looks like this: diff --git a/docs/dsp-tools-excel.md b/docs/dsp-tools-excel.md index 4e08492b9..0e1b08dd8 100644 --- a/docs/dsp-tools-excel.md +++ b/docs/dsp-tools-excel.md @@ -6,11 +6,14 @@ dsp-tools is able to process Excel files and output the appropriate JSON or XML used to create the ontology on the DSP server or import data to the DSP repository. dsp-tools can also be used to create a list from an Excel file. + + + ## Create the resources for a data model from an Excel file -With dsp-tools, the `resources` section used in a data model (JSON) can be created from an Excel file. Only `XLSX` -files are allowed. The `resources` section can be inserted into the ontology file and then be uploaded onto a DSP -server. +With dsp-tools, the `resources` section used in a data model (JSON) can be created from an Excel file. The command for +this is documented [here](./dsp-tools-usage.md#create-resources-from-an-excel-file). Only `XLSX` files are allowed. +The `resources` section can be inserted into the ontology file and then be uploaded onto a DSP server. **An Excel file template can be found [here](assets/templates/resources_template.xlsx). It is recommended to work from the template.** @@ -44,11 +47,15 @@ The GUI order is given by the order in which the properties are listed in the Ex For further information about resources, see [here](./dsp-tools-create-ontologies.md#resources). + + + ## Create the properties for a data model from an Excel file -With dsp-tools, the `properties` section used in a data model (JSON) can be created from an Excel file. Only the first -worksheet of the Excel file is considered and only XLSX files are allowed. The `properties` section can be inserted -into the ontology file and then be uploaded onto a DSP server. +With dsp-tools, the `properties` section used in a data model (JSON) can be created from an Excel file. The command for +this is documented [here](./dsp-tools-usage.md#create-properties-from-an-excel-file). Only the first worksheet of the +Excel file is considered and only XLSX files are allowed. The `properties` section can be inserted into the ontology +file and then be uploaded onto a DSP server. **An Excel file template can be found [here](assets/templates/properties_template.xlsx). It is recommended to work from the template.** @@ -73,6 +80,9 @@ For backwards compatibility, files containing a column `hlist` are valid, but de For further information about properties, see [here](./dsp-tools-create-ontologies.md#properties). + + + ## Create a DSP-conform XML file from an Excel file [not yet implemented] @@ -80,11 +90,11 @@ For further information about properties, see [here](./dsp-tools-create-ontologi - ## Create a list from one or several Excel files -With dsp-tools, a JSON list can be created from one or several Excel files. The list can then be inserted into a JSON -ontology and uploaded to a DSP server. It is possible to create multilingual lists. In this case, a separate +With dsp-tools, a JSON list can be created from one or several Excel files. The command for this is documented +[here](./dsp-tools-usage.md#create-a-json-list-file-from-one-or-several-excel-files). The list can then be inserted +into a JSON ontology and uploaded to a DSP server. It is possible to create multilingual lists. In this case, a separate Excel file has to be created for each language. The data must be in the first worksheet of each Excel file. It is important that all the Excel lists have the same structure. So, the translation of a label in one Excel sheet has to be in the exact same cell than the original was in the other Excel sheet (i.e. same cell index). diff --git a/docs/dsp-tools-information-for-developers.md b/docs/dsp-tools-information-for-developers.md index 04d93638c..44a8d892c 100644 --- a/docs/dsp-tools-information-for-developers.md +++ b/docs/dsp-tools-information-for-developers.md @@ -68,7 +68,7 @@ For formatting Markdown files (*.md) we use the default styling configuration pr ## Publishing Publishing is automated with GitHub Actions and should _not_ be done manually. Please follow the -[Pull Request Guidelines](https://docs.dasch.swiss/developers/dsp/contribution/#pull-request-guidelines). If done +[Pull Request Guidelines](https://docs.dasch.swiss/latest/developers/dsp/contribution/#pull-request-guidelines). If done correctly, when merging a pull request into `main`, the `release-please` action will create or update a pull request for a release. This pull request will follow semantic versioning and update the change log. Once all desired features are merged, the release can be executed by merging this release pull request into `main`. This will trigger actions that diff --git a/docs/dsp-tools-usage.md b/docs/dsp-tools-usage.md index 3a1de696a..eee904b77 100644 --- a/docs/dsp-tools-usage.md +++ b/docs/dsp-tools-usage.md @@ -4,6 +4,9 @@ The following paragraphs gives you an overview of how to install and use dsp-tools. + + + ## Installation To install the latest version run: @@ -18,6 +21,9 @@ To update to the latest version run: pip3 install --upgrade dsp-tools ``` + + + ## Create a data model on a DSP server ```bash @@ -45,6 +51,9 @@ dsp-tools create -s https://api.dsl.server.org -u root@example.com -p test data_ The description of the expected JSON format can be found [here](./dsp-tools-create.md). + + + ## Get a data model from a DSP server ```bash @@ -69,6 +78,11 @@ the password `test` are used. The data model is saved into the output file `outp dsp-tools get -s https://api.test.dasch.swiss -u root@example.com -p test -P my_project output_file.json ``` +The description of the JSON format can be found [here](./dsp-tools-create.md). + + + + ## Upload data to a DSP server ```bash @@ -98,13 +112,16 @@ dsp-tools xmlupload -s https://api.dsl.server.org -u root@example.com -p test -S The description of the expected XML format can be found [here](./dsp-tools-xmlupload.md). -An internal ID is used in the `` tag of an XML file used for `xmlupload` to reference resources inside the same -XML file. Once data is uploaded to DSP it cannot be referenced by this internal ID anymore. Instead, the resource's IRI -has to be used. The mapping of internal IDs to their respective IRIs is written to a file -called `id2iri_mapping_[timstamp].json` after a successful `xmlupload`. +An internal ID is used in the `` tag of an XML file to reference resources inside the same XML file. Once data +is uploaded to DSP, it cannot be referenced by this internal ID anymore. Instead, the resource's IRI has to be used. +After a successful `xmlupload`, the mapping of internal IDs to their respective IRIs is written to a file +called `id2iri_mapping_[timstamp].json`. See [`dsp-tools id2iri`](./dsp-tools-usage.md#replace-internal-ids-with-iris-in-xml-file) for more information about how to use this file to replace internal IDs in an existing XML file to reference existing resources. + + + ## Create a JSON list file from one or several Excel files ```bash @@ -131,6 +148,9 @@ The description of the expected Excel format can be found [here](./dsp-tools-cre information about the usage of this command can be found [here](./dsp-tools-excel.md#create-a-list-from-one-or-several-excel-files). + + + ## Create resources from an Excel file ```bash @@ -148,8 +168,10 @@ dsp-tools excel2resources Resources.xlsx resources.json ``` More information about the usage of this command can be -found [here](./dsp-tools-excel.md#create-the-resources-for-a-data-model-from-an-excel-file) -. +found [here](./dsp-tools-excel.md#create-the-resources-for-a-data-model-from-an-excel-file). + + + ## Create properties from an Excel file @@ -168,8 +190,10 @@ dsp-tools excel2properties Properties.xlsx properties.json ``` More information about the usage of this command can be found -[here](./dsp-tools-excel.md#create-the-properties-for-a-data-model-from-an-excel-file) -. +[here](./dsp-tools-excel.md#create-the-properties-for-a-data-model-from-an-excel-file). + + + ## Replace internal IDs with IRIs in XML file diff --git a/docs/dsp-tools-xmlupload.md b/docs/dsp-tools-xmlupload.md index ab3cc9243..264a22548 100644 --- a/docs/dsp-tools-xmlupload.md +++ b/docs/dsp-tools-xmlupload.md @@ -7,6 +7,8 @@ standard XML file as described on this page. After a successful upload of the da `id2iri_mapping_[timstamp].json`) with the mapping from the internal IDs used inside the XML to their corresponding IRIs which uniquely identify them inside DSP. This file should be kept if data is later added with the `--incremental` [option](#incremental-xml-upload). +The command to import an XML file on a DSP server is described [here](./dsp-tools-usage.md#upload-data-to-a-dsp-server). + The import file must start with the standard XML header: ```xml @@ -48,7 +50,7 @@ The `` element can only contain the following sub-elements: The DSP server provides access control for each resource and each field of a resource through permissions. For a thorough explanation of the permission and access system of DSP, see -[DSP permissions](https://docs.knora.org/02-knora-ontologies/knora-base/#permissions). +[DSP permissions](https://docs.dasch.swiss/latest/DSP-API/02-knora-ontologies/knora-base/#permissions). It is optional to define permissions in the XML. If not defined, default permissions are applied, so that only project and system administrators can view and edit resources. All other users have no rights at all, not even view or restricted view permissions. @@ -662,7 +664,7 @@ The `` element has the following attributes: - `encoding`: either "utf8" or "xml" (required) - `utf8`: The element describes a simple text without markup. The text is a simple UTF-8 string. - `xml`: The element describes a complex text containing markup. It must follow the XML format as defined by the - [DSP standard mapping](https://docs.knora.org/03-apis/api-v1/xml-to-standoff-mapping/). + [DSP standard mapping](https://docs.dasch.swiss/latest/DSP-API/03-apis/api-v2/xml-to-standoff-mapping/). - `permissions`: Permission ID (optional, but if omitted, users who are lower than a `ProjectAdmin` have no permissions at all, not even view rights) - `comment`: a comment for this specific value (optional) @@ -689,7 +691,7 @@ whitespaces will be taken into the text field as they are. #### Text with markup (XML) dsp-tools assumes that for markup (standoff markup), the -[DSP standard mapping](https://docs.knora.org/03-apis/api-v1/xml-to-standoff-mapping/) is used (custom mapping is not yet +[DSP standard mapping](https://docs.dasch.swiss/latest/DSP-API/03-apis/api-v2/xml-to-standoff-mapping/) is used (custom mapping is not yet implemented). Example of a text containing a link to another resource: @@ -921,7 +923,7 @@ To do an incremental XML upload, one of the following procedures is recommended. shortcode="0001" default-ontology="anything"> - + V V diff --git a/testdata/test-data.xml b/testdata/test-data.xml index d9b9e060a..9b0167c96 100644 --- a/testdata/test-data.xml +++ b/testdata/test-data.xml @@ -6,7 +6,7 @@ shortcode="4123" default-ontology="testonto"> - + RV V diff --git a/testdata/test-id2iri-data.xml b/testdata/test-id2iri-data.xml index 298a5a45f..b5d4ac673 100644 --- a/testdata/test-id2iri-data.xml +++ b/testdata/test-id2iri-data.xml @@ -6,7 +6,7 @@ shortcode="4123" default-ontology="testonto"> - + RV V