fix(permissions): use permissions in xml upload (DEV-178) (#127)

* Create test data

* make permissions optional in schema

* use optional type for permissions in xml_upload

* update .gitignore

* precise the use of username for CLI options

* create testdata to test permissions

* update xml_upload to handle optional permissions

* Add bitstream resources to tests

* use resource permissions for bitstream objects

* remove duplicated code

* remove duplicated code

* update documentation

* Remove unused file knora.py

* use images to test permissions

* allow permissions attribute for bitstreams

* Refactor xml_upload.py

* add bitstream to models

* update documentation

* Rename Knora instances

* Update test data

* Rename output labels

* refactor bitstream

* add bitstream to bazel

* refactor resource, value and bitstream

* cleanup code

* update documentation

* Allow separate permission for bitstream

* add image related permissions to test data

* fix failing test

* refactor bitstream

* Update documentation

* update cli documentation

* update url to permissions

.gitignore

@@ -64,6 +64,7 @@ venv.bak/
 # created files
 lists.json
 out.json
+id2iri_*
 
 # bazel
 /bazel-*

docs/dsp-tools-usage.md

@@ -26,7 +26,7 @@ dsp-tools create [options] data_model_definition.json
 
 The following options are available:
 
-- `-s` | `--server` _server_: URL of the DSP server (default: localhost:3333)
+- `-s` | `--server` _server_: URL of the DSP server (default: 0.0.0.0:3333)
 - `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
 - `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
 - `-V` | `--validate`: If set, only the validation of the JSON file is performed.
@@ -53,7 +53,7 @@ dsp-tools get [options] output_file.json
 
 The following options are available:
 
-- `-s` | `--server` _server_: URL of the DSP server (default: localhost:3333)
+- `-s` | `--server` _server_: URL of the DSP server (default: 0.0.0.0:3333)
 - `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
 - `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
 - `-P` | `--project` _shortcode_ | _shortname_ | _iri_: shortcode, shortname or (mandatory)
@@ -77,7 +77,7 @@ dsp-tools xmlupload [options] xml_data_file.xml
 
 The following options are available:
 
-- `-s` | `--server` _server_: URL of the DSP server (default: localhost:3333)
+- `-s` | `--server` _server_: URL of the DSP server (default: 0.0.0.0:3333)
 - `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
 - `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
 - `-i` | `--imgdir` _dirpath_: path to the directory where the bitstream objects are stored (default: .)

docs/dsp-tools-xmlupload.md

@@ -39,7 +39,7 @@ The `<knora>` element may look as follows:
 
 The `<knora>` element can only contain the following sub-elements:
 
-- `<permissions>`
+- `<permissions>` (optional)
 - `<resource>`
 
 ## Describing permissions with &lt;permissions&gt; elements
@@ -48,6 +48,9 @@ The DSP server provides access control for each resource and each field of a res
 thorough explanation of the permission and access system of the DSP platform, see
 [DSP platform permissions](https://docs.knora.org/02-knora-ontologies/knora-base/#permissions).
 
+It is optional to define permissions in the XML. If not defined, default permissions are applied (only project and 
+system administrators can view and edit resources).
+
 The following access rights are defined by the DSP platform which apply to either a resource or a field:
 
 - `RV` _restricted view permission_: The user gets only a restricted view of the element. E.g. in case of a still image
@@ -72,7 +75,7 @@ name has to be prepended before the group name, separated by a colon, e.g. `dsp-
 
 A `<permissions>` element contains the permissions given to the selected groups and is called a _permission set_. It has
 a mandatory attribute `id` and must contain at least one `<allow>` element per group indicating the group's permission.
-It is of the following form:
+The permission is referenced inside the resource or property tag by its `id`. It is of the following form:
 
 ```xml
 <permissions id="res-default">
@@ -84,6 +87,18 @@ It is of the following form:
 </permissions>
 ```
 
+If you don't want a group to have access at all, leave it out. In the following example, only `ProjectAdmin`s will see
+the resource or property with permission `special-permission`:
+
+```xml
+<permissions id="special-permission">
+  <allow group="ProjectAdmin">CR</allow>
+</permissions>
+```
+
+Note: The permissions defined in the XML are applied to resources that are created. But only project or system administrators
+do have the permission to create resources via the XML upload.
+
 ### The &lt;allow&gt; sub-element
 
 The `<allow>` element is used to define the permission for a specific group. It is of the following form:
@@ -94,23 +109,23 @@ The `<allow>` element is used to define the permission for a specific group. It
 
 The allowed values are:
 
-- `RV` _restricted view_: The associated media is shown in reduced quality.
-- `V` _view_: The user has read access to the data.
-- `M` _modify_: The user may modify a value, but may not delete it. The original value will be preserved using the
-  history mechanism.
-- `D` _delete_: The user is able to mark a resource as deleted.
-- `CR` _change right_: The user is able to change the right of a resource or value.
+- `RV` _restricted view_: Same as `V` but if it is applied to an image, the image is shown blurred.
+- `V` _view_: The user can view a resource or a value, but can not modify it.
+- `M` _modify_: The user can modify a resource or value, but can not delete it. The original resource or value will be preserved.
+- `D` _delete_: The user can mark a resource or value as deleted. The original resource or value will be preserved.
+- `CR` _change right_: The user can change the permission of a resource or value. The user is also allowed to 
+  permanently delete (erase) a resource.
 
-The `group` attribute is mandatory. It defines the group which the permission is applied to. The DSP system groups as
+The `group` attribute is mandatory. It defines the group which the permission is applied to. DSP system groups as
 well as project specific groups are supported. A project specific group name has the form `project-shortname:groupname`.
 The available system groups are:
 
-- UnknownUser
-- KnownUser
-- ProjectMember
-- Creator
-- ProjectAdmin
-- SystemAdmin
+- UnknownUser (not logged in user)
+- KnownUser (logged in user)
+- ProjectMember (user with project membership)
+- Creator (creator of the resource or value)
+- ProjectAdmin (user with project admin membership)
+- SystemAdmin (system administrator)
 
 There are no sub-elements allowed for the `<allow>` element.
 
@@ -225,12 +240,12 @@ Note:
 
 Attributes:
 
-- none
+- `permissions` : ID or a permission set (optional, but if omitted very restricted default permissions apply)
 
 Example:
 
 ```xml
-<bitstream>postcards/images/EURUS015a.jpg</bitstream>
+<bitstream permissions="prop-restricted">postcards/images/EURUS015a.jpg</bitstream>
 ```
 
 ### `<text-prop>`

knora/BUILD.bazel

@@ -4,23 +4,6 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library")
 # make the dependencies from requirements.txt available
 load("@knora_py_deps//:requirements.bzl", "requirement")
 
-py_library(
-    name = "knora",
-    visibility = ["//visibility:public"],
-    srcs = ["knora.py"],
-    deps = [
-        requirement("rdflib"),
-        requirement("lxml"),
-        requirement("validators"),
-        requirement("requests"),
-        requirement("jsonschema"),
-        requirement("click"),
-        requirement("rfc3987"),
-        requirement("pprint"),
-    ],
-    imports = ["."],
-)
-
 py_binary(
     name = "create_ontology",
     visibility = ["//visibility:public"],

knora/dsp_tools.py

@@ -27,6 +27,16 @@ def program(user_args: list[str]) -> None:
     Returns:
         None
     """
+    # help texts
+    username_text = 'Username (e-mail) for DSP server'
+    password_text = 'The password for login'
+    url_text = 'URL of the DSP server'
+    verbose_text = 'Verbose feedback'
+
+    # default values
+    default_localhost = 'http://0.0.0.0:3333'
+    default_user = 'root@example.com'
+    default_pw = 'test'
 
     dsp_tools_version = version('dsp-tools')
     now = datetime.datetime.now()
@@ -42,41 +52,41 @@ def program(user_args: list[str]) -> None:
                                           help='Upload an ontology and/or list(s) from a JSON file to the DaSCH '
                                                'Service Platform')
     parser_create.set_defaults(action='create')
-    parser_create.add_argument('-s', '--server', type=str, default='http://0.0.0.0:3333', help='URL of the DSP server')
-    parser_create.add_argument('-u', '--user', default='root@example.com', help='Username for DSP server')
-    parser_create.add_argument('-p', '--password', default='test', help='The password for login')
+    parser_create.add_argument('-s', '--server', type=str, default=default_localhost, help=url_text)
+    parser_create.add_argument('-u', '--user', default=default_user, help=username_text)
+    parser_create.add_argument('-p', '--password', default=default_pw, help=password_text)
     parser_create.add_argument('-V', '--validate', action='store_true',
                                help='Do only validation of JSON, no upload of the '
                                     'ontology')
     parser_create.add_argument('-L', '--listfile', type=str, default='lists.json',
                                help='Name of list node informationfile')
     parser_create.add_argument('-l', '--lists', action='store_true', help='Upload only the list(s)')
-    parser_create.add_argument('-v', '--verbose', action='store_true', help='Verbose feedback')
+    parser_create.add_argument('-v', '--verbose', action='store_true', help=verbose_text)
     parser_create.add_argument('-d', '--dump', action='store_true', help='dump test files for DSP-API requests')
     parser_create.add_argument('datamodelfile', help='path to data model file')
 
     parser_get = subparsers.add_parser('get',
                                        help='Get the ontology (data model) of a project from the DaSCH Service Platform.')
     parser_get.set_defaults(action='get')
-    parser_get.add_argument('-u', '--user', default='root@example.com', help='Username for DSP server')
-    parser_get.add_argument('-p', '--password', default='test', help='The password for login')
-    parser_get.add_argument('-s', '--server', type=str, default='http://0.0.0.0:3333', help='URL of the DSP server')
+    parser_get.add_argument('-u', '--user', default=default_user, help=username_text)
+    parser_get.add_argument('-p', '--password', default=default_pw, help=password_text)
+    parser_get.add_argument('-s', '--server', type=str, default=default_localhost, help=url_text)
     parser_get.add_argument('-P', '--project', type=str, help='Shortcode, shortname or iri of project', required=True)
-    parser_get.add_argument('-v', '--verbose', action='store_true', help='Verbose feedback')
+    parser_get.add_argument('-v', '--verbose', action='store_true', help=verbose_text)
     parser_get.add_argument('datamodelfile', help='Path to the file the ontology should be written to',
                             default='onto.json')
 
     parser_upload = subparsers.add_parser('xmlupload',
                                           help='Upload data from an XML file to the DaSCH Service Platform.')
     parser_upload.set_defaults(action='xmlupload')
-    parser_upload.add_argument('-s', '--server', type=str, default='http://0.0.0.0:3333', help='URL of the DSP server')
-    parser_upload.add_argument('-u', '--user', type=str, default='root@example.com', help='Username for DSP server')
-    parser_upload.add_argument('-p', '--password', type=str, default='test', help='The password for login')
+    parser_upload.add_argument('-s', '--server', type=str, default=default_localhost, help=url_text)
+    parser_upload.add_argument('-u', '--user', type=str, default=default_user, help=username_text)
+    parser_upload.add_argument('-p', '--password', type=str, default=default_pw, help=password_text)
     parser_upload.add_argument('-V', '--validate', action='store_true',
                                help='Do only validation of XML, no upload of the data')
     parser_upload.add_argument('-i', '--imgdir', type=str, default='.', help='Path to folder containing the images')
     parser_upload.add_argument('-S', '--sipi', type=str, default='http://0.0.0.0:1024', help='URL of SIPI server')
-    parser_upload.add_argument('-v', '--verbose', action='store_true', help='Verbose feedback')
+    parser_upload.add_argument('-v', '--verbose', action='store_true', help=verbose_text)
     parser_upload.add_argument('-I', '--incremental', action='store_true', help='Incremental XML upload')
     parser_upload.add_argument('xmlfile', help='path to xml file containing the data', default='data.xml')
 
@@ -116,12 +126,14 @@ def program(user_args: list[str]) -> None:
                                          default='properties.json')
 
     parser_id2iri = subparsers.add_parser('id2iri',
-                                                    help='Replace internal IDs in an XML with their corresponding IRIs from a provided JSON file.')
+                                          help='Replace internal IDs in an XML with their corresponding IRIs from a provided JSON file.')
     parser_id2iri.set_defaults(action='id2iri')
     parser_id2iri.add_argument('xmlfile', help='Path to the XML file containing the data to be replaced')
-    parser_id2iri.add_argument('jsonfile', help='Path to the JSON file containing the mapping of internal IDs and their respective IRIs')
-    parser_id2iri.add_argument('--outfile', default=None, help='Path to the XML output file containing the replaced IDs (optional)')
-    parser_id2iri.add_argument('-v', '--verbose', action='store_true', help='Verbose feedback')
+    parser_id2iri.add_argument('jsonfile',
+                               help='Path to the JSON file containing the mapping of internal IDs and their respective IRIs')
+    parser_id2iri.add_argument('--outfile', default=None,
+                               help='Path to the XML output file containing the replaced IDs (optional)')
+    parser_id2iri.add_argument('-v', '--verbose', action='store_true', help=verbose_text)
 
     args = parser.parse_args(user_args)
 

knora/dsplib/models/BUILD.bazel

@@ -4,6 +4,19 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library")
 # make the dependencies from requirements.txt available
 load("@knora_py_deps//:requirements.bzl", "requirement")
 
+py_library(
+    name = "bitstream",
+    visibility = ["//visibility:public"],
+    srcs = ["bitstream.py"],
+    deps = [
+        ":helpers",
+        ":langstring",
+        ":permission",
+        requirement("pystrict"),
+    ],
+    imports = ["."],
+)
+
 py_library(
     name = "connection",
     visibility = ["//visibility:public"],
@@ -130,6 +143,7 @@ py_library(
     visibility = ["//visibility:public"],
     srcs = ["resource.py"],
     deps = [
+        ":bitstream",
         ":connection",
         ":helpers",
         ":langstring",

knora/dsplib/models/bitstream.py

@@ -0,0 +1,37 @@
+from typing import Dict, Optional, Any
+
+from pystrict import strict
+
+from .helpers import Actions
+from .permission import Permissions
+
+
+@strict
+class Bitstream:
+    """
+    Represents a bitstream object (file) which is attached to a resource
+    """
+    _value: str
+    _permissions: Optional[Permissions]
+
+    def __init__(self,
+                 value: str,
+                 permissions: Optional[Permissions] = None):
+        self._value = value
+        self._permissions = permissions
+
+    @property
+    def value(self) -> str:
+        return self._value
+
+    @property
+    def permissions(self) -> Optional[Permissions]:
+        return self._permissions
+
+    def toJsonLdObj(self, action: Actions) -> Dict[str, Any]:
+        tmp = {}
+        if action == Actions.Create:
+            tmp["knora-api:fileValueHasFilename"] = self._value
+            if self._permissions:
+                tmp["knora-api:hasPermissions"] = self.permissions.toJsonLdObj()
+        return tmp

knora/dsplib/models/group.py

@@ -272,7 +272,6 @@ def print(self):
                       status=True,
                       selfjoin=False).create()
     new_group.print()
-    print("iiiii")
 
     new_group.name = "GROUP TEST - modified"
     new_group = new_group.update()