diff --git a/README.md b/README.md index f842f2342..140e5abc2 100644 --- a/README.md +++ b/README.md @@ -1,131 +1,100 @@ [![PyPI version](https://badge.fury.io/py/dsp-tools.svg)](https://badge.fury.io/py/dsp-tools) -# DSP-TOOLS - The DaSCH Service Platform Tools Repository -dsp-tools is a python package containing a command line tool for data model (ontology) creation, a library allowing -creation of single resources and mass upload using the bulk import of data into the Knora framework. +# DSP-TOOLS - DaSCH Service Platform Tools -The package consists of: -- `dsplib` Python modules for accessing Knora using the API (ontology creation, data import/export etc.) -- `dsp-tools` A command line program to perfrom several operations on a DSP server: - - create an ontology out of a simple JSON description - - dump an existing ontology from a DSP server to a JSON file - - Bulk-upload of data from a XML data file +dsp-tools is a command line tool that helps you interact with the DaSCH Service Platform server (DSP server). Go to [Full Documentation](https://dasch-swiss.github.io/dsp-tools/) -## Install +## Information for developers -_Please note that dsp-tools require Python 3.9 for working properly!_ +There is a `Makefile` for all the following tasks (and more). Type `make` to print the available targets. -To install the latest published version from PyPI, run: -``` -$ pip3 install dsp-tools -``` +## Install from source -To upgrade to the latest published version, run: -``` -$ pip3 install --upgrade dsp-tools +To install from source run: +```bash +python3 setup.py install ``` -## Local Development Environment - -- Python3 -- [Bazel](https://bazel.build) +## Install the requirements -Please consult the [https://docs.dasch.swiss/developers](https://docs.dasch.swiss/developers) -documentation on how to install these prerequisites. +To install the requirements run: -## Makefile for repository management tasks +```bash +pip3 install -r requirements.txt +``` -The project contains a Makefile defining management tasks. Please use -`make help` to see what is available. +To generate a requirements file (usually requirements.txt), that you commit with your project, run: -To install from source, i.e., this repository, run: -``` -$ make install +```bash +pip3 freeze > requirements.txt ``` ## Testing +Please note that testing requires launching the complete DSP API stack which is based on docker images. Therefore, we +recommend installing the [docker desktop client](https://www.docker.com/products). + +To run the complete test suite: ```bash -$ make test +make test ``` ## Publishing -Publishing is automated with github actions and should _not_ be done manually. - -Ensure to have only one Pull Request per feature, and follow the [conventions for commit messages and PR title](https://docs.dasch.swiss/developers/dsp/contribution/#pull-request-guidelines). +Publishing is automated with GitHub Actions and should _not_ be done manually. Please follow the +[Pull Request Guidelines](https://docs.dasch.swiss/developers/dsp/contribution/#pull-request-guidelines). If done +correctly, when merging a pull request into `main`, the `release-please` action will create or update a pull request for +a release. This pull request will follow semantic versioning and update the change log. Once all desired features are +merged, the release can be executed by merging this release pull request into `main`. This will trigger actions that +create a release on GitHub, on PyPI and the docs. -If this is done correctly, when merging a PR into `main`, the `release-please` action will create or update a release-PR. -This PR will follow semantic versioning and update the change log. -Once all desired features are merged, the release can be executed by merging the release-PR into `main`. -This will trigger actions that create a release on Github, on PyPI and the docs. +Please ensure you have only one pull request per feature. -### Publishing to PyPi +## Publishing manually -Generate distribution package. Make sure you have the latest versions of `setuptools` and `wheel` installed. -Remove the `dist`and `buil`-directories: `rm -rf bild dist` +Publishing is automated with GitHub Actions and should _not_ be done manually. If you still need to do it, follow the +steps below. +Generate the distribution package. Make sure you have the latest versions of `setuptools` and `wheel` installed: ```bash -$ make upgrade-dist-tools -$ make dist +python3 -m pip install --upgrade pip setuptools wheel +python3 setup.py sdist bdist_wheel ``` You can install the package locally from the dist: ```bash -$ python3 -m pip install ./dist/some_name.whl -``` - -Upload package with `twine`, - -first create `~/.pypirc` in your home folder: - -```bash -[distutils] -index-servers=pypi -[pypi] -repository = https://upload.pypi.org/legacy/ -username =your_username_on_pypi +python3 -m pip ./dist/some_name.whl ``` -then upload: +Upload package works also with `make`: ```bash -$ make upload +make dist +make upload ``` For local development: ```bash -$ python3 setup.py --editable . +python3 setup.py develop ``` -## Requirements - -To install the requirements: - -```bash -$ pip3 install -r requirements.txt -``` +## Contributing to the documentation -To generate a "requirements" file (usually requirements.txt), that you commit with your project, do: +The documentation is a collection of [markdown](https://en.wikipedia.org/wiki/Markdown) files in the `docs` folder. +After updates of the files, build and check the result with the following commands: ```bash -$ pip3 freeze > requirements.txt +make build-docs +make serve-docs ``` -### Running tests with Bazel - -Run all tests: -```bash -$ bazel test //... -``` +To update the changes to the official documentation pages run: -Run single test: ```bash -$ bazel test //test:test_user +make publish-docs ``` - diff --git a/docs/assets/fonts/roboto-v19-latin-300.eot b/docs/assets/fonts/roboto-v19-latin-300.eot deleted file mode 100644 index 574375d40..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-300.eot and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-300.svg b/docs/assets/fonts/roboto-v19-latin-300.svg deleted file mode 100644 index 4ded944a8..000000000 --- a/docs/assets/fonts/roboto-v19-latin-300.svg +++ /dev/null @@ -1,312 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/assets/fonts/roboto-v19-latin-300.ttf b/docs/assets/fonts/roboto-v19-latin-300.ttf deleted file mode 100644 index 57a24e82f..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-300.ttf and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-300.woff b/docs/assets/fonts/roboto-v19-latin-300.woff deleted file mode 100644 index 2f6bdb5e7..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-300.woff and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-300.woff2 b/docs/assets/fonts/roboto-v19-latin-300.woff2 deleted file mode 100644 index ef8c8836b..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-300.woff2 and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-500.eot b/docs/assets/fonts/roboto-v19-latin-500.eot deleted file mode 100644 index 2ec7bdd75..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-500.eot and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-500.svg b/docs/assets/fonts/roboto-v19-latin-500.svg deleted file mode 100644 index 67eecf442..000000000 --- a/docs/assets/fonts/roboto-v19-latin-500.svg +++ /dev/null @@ -1,305 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/assets/fonts/roboto-v19-latin-500.ttf b/docs/assets/fonts/roboto-v19-latin-500.ttf deleted file mode 100644 index dd04ff19d..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-500.ttf and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-500.woff b/docs/assets/fonts/roboto-v19-latin-500.woff deleted file mode 100644 index 869925869..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-500.woff and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-500.woff2 b/docs/assets/fonts/roboto-v19-latin-500.woff2 deleted file mode 100644 index 6362d7f64..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-500.woff2 and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-700.eot b/docs/assets/fonts/roboto-v19-latin-700.eot deleted file mode 100644 index 0168f09bc..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-700.eot and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-700.svg b/docs/assets/fonts/roboto-v19-latin-700.svg deleted file mode 100644 index 11db87dd0..000000000 --- a/docs/assets/fonts/roboto-v19-latin-700.svg +++ /dev/null @@ -1,309 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/assets/fonts/roboto-v19-latin-700.ttf b/docs/assets/fonts/roboto-v19-latin-700.ttf deleted file mode 100644 index ea06a634f..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-700.ttf and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-700.woff b/docs/assets/fonts/roboto-v19-latin-700.woff deleted file mode 100644 index 0f14effba..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-700.woff and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-700.woff2 b/docs/assets/fonts/roboto-v19-latin-700.woff2 deleted file mode 100644 index 32b25eee7..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-700.woff2 and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-italic.eot b/docs/assets/fonts/roboto-v19-latin-italic.eot deleted file mode 100644 index 0753b8839..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-italic.eot and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-italic.svg b/docs/assets/fonts/roboto-v19-latin-italic.svg deleted file mode 100644 index 4d5979710..000000000 --- a/docs/assets/fonts/roboto-v19-latin-italic.svg +++ /dev/null @@ -1,323 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/assets/fonts/roboto-v19-latin-italic.ttf b/docs/assets/fonts/roboto-v19-latin-italic.ttf deleted file mode 100644 index bdbb10850..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-italic.ttf and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-italic.woff b/docs/assets/fonts/roboto-v19-latin-italic.woff deleted file mode 100644 index b940dbcb7..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-italic.woff and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-italic.woff2 b/docs/assets/fonts/roboto-v19-latin-italic.woff2 deleted file mode 100644 index 2741d4f08..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-italic.woff2 and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-regular.eot b/docs/assets/fonts/roboto-v19-latin-regular.eot deleted file mode 100644 index 4f3480091..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-regular.eot and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-regular.svg b/docs/assets/fonts/roboto-v19-latin-regular.svg deleted file mode 100644 index 627f5a368..000000000 --- a/docs/assets/fonts/roboto-v19-latin-regular.svg +++ /dev/null @@ -1,308 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/assets/fonts/roboto-v19-latin-regular.ttf b/docs/assets/fonts/roboto-v19-latin-regular.ttf deleted file mode 100644 index a97385dfd..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-regular.ttf and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-regular.woff b/docs/assets/fonts/roboto-v19-latin-regular.woff deleted file mode 100644 index 69c882540..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-regular.woff and /dev/null differ diff --git a/docs/assets/fonts/roboto-v19-latin-regular.woff2 b/docs/assets/fonts/roboto-v19-latin-regular.woff2 deleted file mode 100644 index 1a5370151..000000000 Binary files a/docs/assets/fonts/roboto-v19-latin-regular.woff2 and /dev/null differ diff --git a/docs/img.png b/docs/assets/images/img-list-example.png similarity index 100% rename from docs/img.png rename to docs/assets/images/img-list-example.png diff --git a/docs/assets/images/knora/knora-logo.png b/docs/assets/images/knora/knora-logo.png deleted file mode 100644 index d5ea3b491..000000000 Binary files a/docs/assets/images/knora/knora-logo.png and /dev/null differ diff --git a/docs/dsp-tools-create.md b/docs/dsp-tools-create.md index 70142afff..ae975897c 100644 --- a/docs/dsp-tools-create.md +++ b/docs/dsp-tools-create.md @@ -112,7 +112,8 @@ So, a simple example definition of the "project" object could look like this: ## Simple key/value pairs At that point we will go through all of this step by step and take a more in depth view on the individual fields of the -"project" object. The first four fields of the "project" object are "key"/"value" pairs. Therefore they are quite simple. +"project" object. The first four fields of the "project" object are "key"/"value" pairs. Therefore, they are quite +simple. ### Shortcode `"shortcode": "<4-hex-characters>"` @@ -139,7 +140,7 @@ Wulfric Dumbledore". The descriptions specify the content of the project in *exactly* one or more strings. These descriptions can be supplied in several languages (currently _"en"_, _"de"_, _"fr"_ and _"it"_ are supported). The descriptions have to be -given as a JSON object with the language as "key" and the description as "value". See the example above inside the +given as a JSON object with the language as "key", and the description as "value". See the example above inside the curly brackets after "descriptions" to see what that means. ## Key/object pairs @@ -167,7 +168,7 @@ example a classification of disciplines in the Humanities might look like follow - Choirs - Orchestras - Music history - - Musictheory + - Music theory - Musicology - Jazz - Pop/Rock @@ -219,15 +220,15 @@ DSP allows to define such controlled vocabularies or thesauri. They can be arran given example about the disciplines in Humanities is). The definition of these entities are called "lists" in the DSP. Thus, the list object is used to give the resources of the ontology a taxonomic quality. A taxonomy makes it possible to categorize a resource. The big advantage of a taxonomic structure as it is implemented by the DSP -is that the user can subcategorize the objects. This allows the user to formulate his search requests more or less +is that the user can sub-categorize the objects. This allows the user to formulate his search requests more or less specifically as desired. Thus, in the example above a search for "Vocal music" would result in all works that are -characterized by a subelement of "Vocal music". However a search for "Masses" would retrun only works that +characterized by a sub-element of "Vocal music". However, a search for "Masses" would return only works that have been characterized as such. The number of hierarchy levels is not limited, but for practical reasons it should not exceed 3-4 levels. Thus, a taxonomy is a hierarchical list of categories in a tree-like structure. The taxonomy must be complete. This means that the entire set of resources must be mappable to the sub-categorization of the taxonomy. To come back to the previous -example: It must not occur that an musical work within our resource set cannot be mapped to a subcategory of our +example: It must not occur that a musical work within our resource set cannot be mapped to a subcategory of our taxonomy about classical music. The taxonomic-hierarchical structure is mapped using JSON. This is because JSON inherently implements a tree structure as well. The root of the taxonomy tree is always the name of the taxonomy. The root always stands alone at the top of the tree. It is followed by any number of levels, on which any number of @@ -256,7 +257,7 @@ a cardinality greater than 1 allows adding multiple attributes of the same time. A node of the Taxonomy may have the following elements: -- _name_: Name of the node. This should be unique for the given list. The name-element is optional but highly +- _name_: Name of the node. This should be unique within the given list. The name-element is optional but highly recommended. - _labels_: Language dependent labels in the form `{ "": "