diff --git a/.bazelversion b/.bazelversion deleted file mode 100644 index c5fbc5af9..000000000 --- a/.bazelversion +++ /dev/null @@ -1 +0,0 @@ -last_downstream_green diff --git a/.github/workflows/daily-test.yml b/.github/workflows/daily-test.yml index 64ec7ae90..7835d12cb 100644 --- a/.github/workflows/daily-test.yml +++ b/.github/workflows/daily-test.yml @@ -13,19 +13,6 @@ jobs: uses: actions/checkout@v2 with: fetch-depth: 1 - - name: Set up JDK - uses: joschi/setup-jdk@v2 - with: - java-version: '11' # The OpenJDK version to make available on the path - architecture: 'x64' # defaults to 'x64' - - name: Set up Node - uses: actions/setup-node@v1 - with: - node-version: '12' - - name: Install dependencies - run: | - sudo apt-get install ca-certificates-java expect - npm install --global @bazel/bazelisk - name: Set up Python 3.9 uses: actions/setup-python@v2 with: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8e8475794..d68245cf6 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -25,7 +25,7 @@ jobs: uses: actions/setup-python@v2 with: python-version: '3.9' - # install pythonn dependencies + # install python dependencies - name: Install dependencies run: | python -m pip install --upgrade pip diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 0b7be04ae..e80944e3f 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -17,19 +17,6 @@ jobs: uses: actions/checkout@v2 with: fetch-depth: 1 - - name: Set up JDK - uses: joschi/setup-jdk@v2 - with: - java-version: '11' # The OpenJDK version to make available on the path - architecture: 'x64' # defaults to 'x64' - - name: Set up Node - uses: actions/setup-node@v1 - with: - node-version: '12' - - name: Install dependencies - run: | - sudo apt-get install ca-certificates-java expect - npm install --global @bazel/bazelisk - name: Set up Python 3.9 uses: actions/setup-python@v2 with: diff --git a/.gitignore b/.gitignore index d1c3178f3..d67dad4cf 100644 --- a/.gitignore +++ b/.gitignore @@ -68,11 +68,8 @@ lists.json out.json id2iri_* **/~$*.* -testdata/tmp/lists_output.json - -# bazel -/bazel-* -/.ijwb/ +testdata/tmp/_*.json +testdata/tmp/_*.xml # for testing in development tmp/ diff --git a/BUILD.bazel b/BUILD.bazel deleted file mode 100644 index e69de29bb..000000000 diff --git a/Makefile b/Makefile index 95705f5fd..43184731c 100644 --- a/Makefile +++ b/Makefile @@ -28,7 +28,7 @@ upload: ## upload distribution package to PyPI python3 -m twine upload dist/* .PHONY: upgrade-dist-tools -upgrade-dist-tool: ## upgrade packages necessary for testing, building, packaging and uploading to PyPI +upgrade-dist-tools: ## upgrade packages necessary for testing, building, packaging and uploading to PyPI python3 -m pip install --upgrade pip setuptools wheel tqdm twine pytest mkdocs .PHONY: docs-build @@ -56,16 +56,15 @@ install: ## install from source (runs setup.py) .PHONY: test test: clean local-tmp clone-dsp-repo dsp-stack ## run all tests - # to run only one test, replace //test/... with p.ex. //test/e2e:test_tools - bazel test --test_summary=detailed --test_output=all //test/... + pytest test/ .PHONY: test-end-to-end test-end-to-end: clean local-tmp clone-dsp-repo dsp-stack ## run e2e tests - bazel test --test_summary=detailed --test_output=all //test/e2e/... + pytest test/e2e/ .PHONY: test-unittests test-unittests: ## run unit tests - bazel test --test_summary=detailed --test_output=all //test/unittests/... + pytest test/unittests/ .PHONY: local-tmp local-tmp: ## create local .tmp folder diff --git a/README.md b/README.md index 6a8a4c6aa..7b45fb5bc 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,13 @@ Go to [Full Documentation](https://dasch-swiss.github.io/dsp-tools/) ## Information for developers -There is a `Makefile` for all the following tasks (and more). Type `make` to print the available targets. +There is a `Makefile` for all the following tasks (and more). Type `make` to print the available targets. For a quick +start use: +```bash +make upgrade-dist-tools +make install-requirements +make run +``` ## Install from source @@ -49,21 +55,8 @@ When contributing to the project please make sure you use the same code style ru [autopep8](https://pypi.org/project/autopep8/) and [mypy](https://pypi.org/project/mypy/). The configuration is defined in `pyproject.toml` in the root directory of the project. -```toml -[tool.autopep8] -max_line_length = 180 -experimental = true - -[tool.mypy] -ignore_missing_imports = true -follow_imports = "silent" -show_column_numbers = true -strict = true -``` - -You can use the configuration with `autopep8 --global-config pyproject.toml [file path]` -and `mypy --config-file pyproject.toml -[file path]`. +You can use the configuration with `autopep8 --global-config pyproject.toml [file path]` and +`mypy --config-file pyproject.toml [file path]`. If you are using PyCharm we recommend installing autopep8 as external tool. You can then use it with right-click on the file > `External Tools` > `autopep8` to reformat files in-place. Due to compatibility issues with VSCode, the argument diff --git a/WORKSPACE b/WORKSPACE deleted file mode 100644 index e5d27e37c..000000000 --- a/WORKSPACE +++ /dev/null @@ -1,24 +0,0 @@ -workspace(name = "knora_py") - -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - -# register our custom toolchains for the different platforms -#register_toolchains("//:homebrew_toolchain", "//:travis_toolchain") - -# use rules_python version 0.1.0 -http_archive( - name = "rules_python", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.1.0/rules_python-0.1.0.tar.gz", - sha256 = "b6d46438523a3ec0f3cead544190ee13223a52f6a6765a29eae7b7cc24cc83a0", -) - -# allows to use requirements.txt for loading the dependencies -load("@rules_python//python:pip.bzl", "pip_install") - -# Create a central repo that knows about the dependencies needed for -# requirements.txt. -pip_install( - name = "knora_py_deps", - requirements = "//:requirements.txt", - python_interpreter = "python3", -) diff --git a/knora/BUILD.bazel b/knora/BUILD.bazel deleted file mode 100644 index e262bab94..000000000 --- a/knora/BUILD.bazel +++ /dev/null @@ -1,61 +0,0 @@ -# make the python rules available -load("@rules_python//python:defs.bzl", "py_binary", "py_library") - -# make the dependencies from requirements.txt available -load("@knora_py_deps//:requirements.bzl", "requirement") - -py_binary( - name = "create_ontology", - visibility = ["//visibility:public"], - srcs = ["create_ontology.py"], - deps = [ - ":knora", - requirement("jsonschema"), -# requirement("pprint"), - ], - data = [ - "ontology.json", - "lists.json", - ], - python_version = "PY3", -) - -py_binary( - name = "xml2knora", - visibility = ["//visibility:public"], - srcs = ["xml2knora.py"], - deps = [ - ":knora", - requirement("lxml"), - requirement("rdflib"), -# requirement("pprint"), - ], - data = [ - "knora-data-schema.xsd" - ], - python_version = "PY3", -) - -py_binary( - name = "reset_triplestore", - visibility = ["//visibility:public"], - srcs = ["reset_triplestore.py"], - deps = [":knora"], - python_version = "PY3", -) - -py_binary( - name = "knoraConsole", - visibility = ["//visibility:public"], - srcs = ["knoraConsole.py"], - deps = [":knora"], - python_version = "PY3", -) - -py_binary( - name = "knoractl", - visibility = ["//visibility:public"], - srcs = ["knoractl.py"], - deps = [":knora"], - python_version = "PY3", -) diff --git a/knora/dsplib/BUILD.bazel b/knora/dsplib/BUILD.bazel deleted file mode 100644 index e69de29bb..000000000 diff --git a/knora/dsplib/models/BUILD.bazel b/knora/dsplib/models/BUILD.bazel deleted file mode 100644 index 0ec07c13f..000000000 --- a/knora/dsplib/models/BUILD.bazel +++ /dev/null @@ -1,217 +0,0 @@ -# make the python rules available -load("@rules_python//python:defs.bzl", "py_binary", "py_library") - -# make the dependencies from requirements.txt available -load("@knora_py_deps//:requirements.bzl", "requirement") - -py_library( - name = "bitstream", - visibility = ["//visibility:public"], - srcs = ["bitstream.py"], - deps = [ - ":helpers", - ":langstring", - ":permission", - requirement("pystrict"), - ], - imports = ["."], -) - -py_library( - name = "connection", - visibility = ["//visibility:public"], - srcs = ["connection.py"], - deps = [ - ":helpers", - requirement("requests"), - ], - imports = ["."], -) - -py_library( - name = "group", - visibility = ["//visibility:public"], - srcs = ["group.py"], - deps = [ - ":helpers", - ":connection", - ":project", - requirement("requests"), - requirement("pystrict"), - ], - imports = ["."], -) - -py_library( - name = "helpers", - visibility = ["//visibility:public"], - srcs = ["helpers.py"], - deps = [ - requirement("pystrict"), - ], - imports = ["."], -) - -py_library( - name = "langstring", - visibility = ["//visibility:public"], - srcs = ["langstring.py"], - deps = [ - ":helpers", - ], - imports = ["."], -) - -py_library( - name = "listnode", - visibility = ["//visibility:public"], - srcs = ["listnode.py"], - deps = [ - "//knora/dsplib/utils:set_encoder", - ":connection", - ":helpers", - ":langstring", - ":project", - ], - imports = [".", ".."], -) - -py_library( - name = "model", - visibility = ["//visibility:public"], - srcs = ["model.py"], - deps = [ - ":connection", - ":helpers", - ], - imports = ["."], -) - -py_library( - name = "ontology", - visibility = ["//visibility:public"], - srcs = ["ontology.py"], - deps = [ - ":connection", - ":helpers", - ":project", - ":propertyclass", - ":resourceclass", - requirement("pystrict"), - ], - imports = ["."], -) - -py_library( - name = "permission", - visibility = ["//visibility:public"], - srcs = ["permission.py"], - deps = [], - imports = ["."], -) - -py_library( - name = "project", - visibility = ["//visibility:public"], - srcs = ["project.py"], - deps = [ - "//knora/dsplib/utils:set_encoder", - ":helpers", - ":langstring", - ":connection", - ":model", - ], - imports = ["."], -) - -py_library( - name = "propertyclass", - visibility = ["//visibility:public"], - srcs = ["propertyclass.py"], - deps = [ - "//knora/dsplib/utils:set_encoder", - ":connection", - ":helpers", - ":langstring", - ":listnode" - ], - imports = [".", ".."], -) - -py_library( - name = "resource", - visibility = ["//visibility:public"], - srcs = ["resource.py"], - deps = [ - ":bitstream", - ":connection", - ":helpers", - ":langstring", - ":model", - ":ontology", - ":permission", - ":project", - ":propertyclass", - ":resourceclass", - ":value", - ], - imports = ["."], -) - -py_library( - name = "resourceclass", - visibility = ["//visibility:public"], - srcs = ["resourceclass.py"], - deps = [ - "//knora/dsplib/utils:set_encoder", - ":connection", - ":helpers", - ":langstring", - ], - imports = [".", ".."], -) - -py_library( - name = "sipi", - visibility = ["//visibility:public"], - srcs = ["sipi.py"], - deps = [ - ":helpers", - ], - imports = ["."], -) - -py_library( - name = "user", - visibility = ["//visibility:public"], - srcs = ["user.py"], - deps = [ - ":connection", - ":helpers", - ":group", - ":project", - ":langstring" - ], - imports = ["."], -) - -py_library( - name = "value", - visibility = ["//visibility:public"], - srcs = ["value.py"], - deps = [ - ":connection", - ":helpers", - ":langstring", - ":listnode", - ":permission", - requirement("pystrict"), - ], - imports = ["."], -) - - - - - - diff --git a/knora/dsplib/utils/BUILD.bazel b/knora/dsplib/utils/BUILD.bazel deleted file mode 100644 index 816045d76..000000000 --- a/knora/dsplib/utils/BUILD.bazel +++ /dev/null @@ -1,135 +0,0 @@ -# make the python rules available -load("@rules_python//python:defs.bzl", "py_binary", "py_library") - -# make the dependencies from requirements.txt available -load("@knora_py_deps//:requirements.bzl", "requirement") - -py_library( - name = "excel_to_json_lists", - visibility = ["//visibility:public"], - srcs = ["excel_to_json_lists.py"], - deps = [ - requirement("jsonschema"), - requirement("openpyxl") - ] -) - -py_library( - name = "excel_to_json_resources", - visibility = ["//visibility:public"], - srcs = ["excel_to_json_resources.py"], - deps = [ - requirement("jsonschema"), - requirement("openpyxl") - ] -) - -py_library( - name = "excel_to_json_properties", - visibility = ["//visibility:public"], - srcs = ["excel_to_json_properties.py"], - deps = [ - requirement("jsonschema"), - requirement("openpyxl") - ] -) - -py_library( - name = "expand_all_lists", - visibility = ["//visibility:public"], - srcs = ["expand_all_lists.py"], - deps = [ - "//knora/dsplib/utils:excel_to_json_lists" - ] -) - -py_library( - name = "onto_create_lists", - visibility = ["//visibility:public"], - srcs = ["onto_create_lists.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:listnode", - "//knora/dsplib/models:project", - ":expand_all_lists", - ":onto_validate" - ], - imports = [".", ".."], -) - -py_library( - name = "onto_create_ontology", - visibility = ["//visibility:public"], - srcs = ["onto_create_ontology.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:group", - "//knora/dsplib/models:helpers", - "//knora/dsplib/models:langstring", - "//knora/dsplib/models:ontology", - "//knora/dsplib/models:project", - "//knora/dsplib/models:propertyclass", - "//knora/dsplib/models:resourceclass", - "//knora/dsplib/models:user", - ":expand_all_lists", - ":onto_create_lists", - ":onto_validate" - ], - imports = [".", ".."], -) - -py_library( - name = "onto_get", - visibility = ["//visibility:public"], - srcs = ["onto_get.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:listnode", - "//knora/dsplib/models:ontology", - "//knora/dsplib/models:project" - ], - imports = [".", ".."], -) - -py_library( - name = "onto_validate", - visibility = ["//visibility:public"], - srcs = ["onto_validate.py"], - deps = [ - ":expand_all_lists", - requirement("jsonschema") - ], - imports = [".", ".."], -) - -py_library( - name = "set_encoder", - visibility = ["//visibility:public"], - srcs = ["set_encoder.py"] -) - -py_library( - name = "xml_upload", - visibility = ["//visibility:public"], - srcs = ["xml_upload.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:group", - "//knora/dsplib/models:permission", - "//knora/dsplib/models:project", - "//knora/dsplib/models:resource", - "//knora/dsplib/models:sipi", - requirement("lxml") - ], - imports = [".", ".."], -) - -py_library( - name = "id_to_iri", - visibility = ["//visibility:public"], - srcs = ["id_to_iri.py"], - deps = [ - requirement("lxml") - ] -) - diff --git a/knora/dsplib/utils/onto_create_lists.py b/knora/dsplib/utils/onto_create_lists.py index 355aead51..3b0eeede8 100644 --- a/knora/dsplib/utils/onto_create_lists.py +++ b/knora/dsplib/utils/onto_create_lists.py @@ -55,7 +55,7 @@ def create_lists(input_file: str, lists_file: str, server: str, user: str, passw user: Username (e-mail) for the DSP server, has to have the permissions to create an ontology password: Password of the user verbose: Verbose output if True - dump: ??? + dump: dumps the request as JSON (used for testing) Returns: list_root_nodes: Dictionary of node names and their respective IRI diff --git a/requirements.txt b/requirements.txt index ec9010cf7..bdf146066 100644 --- a/requirements.txt +++ b/requirements.txt @@ -12,11 +12,12 @@ ghp-import==2.0.2 idna==3.3 importlib-metadata==4.8.1 importlib-resources==5.4.0 +iniconfig==1.1.1 isodate==0.6.0 Jinja2==3.0.2 joblib==1.1.0 +jsonpath-ng==1.5.3 jsonschema==4.2.1 -jsonpath_ng==1.5.3 keyring==23.2.1 livereload==2.6.3 lunr==0.5.8 @@ -31,17 +32,23 @@ mkdocs-material==7.2.3 mkdocs-material-extensions==1.0.3 mkdocstrings==0.16.2 nltk==3.6.6 +numpy==1.22.3 openpyxl==3.0.9 packaging==21.2 pandas==1.4.1 pkginfo==1.7.1 +pluggy==1.0.0 +ply==3.11 +py==1.11.0 Pygments==2.10.0 pymdown-extensions==9.0 pyparsing==2.4.7 pyrsistent==0.18.0 pystrict==1.1 +pytest==7.1.1 python-dateutil==2.8.2 pytkdocs==0.12.0 +pytz==2022.1 PyYAML==6.0 pyyaml_env_tag==0.1 rdflib==6.0.2 @@ -51,15 +58,14 @@ requests==2.26.0 requests-toolbelt==0.9.1 rfc3986==1.5.0 rfc3987==1.3.8 -setuptools six==1.16.0 +tomli==2.0.1 tornado==6.1 tqdm==4.62.3 twine==3.5.0 -typing-extensions==4.0.1 +typing_extensions==4.0.1 urllib3==1.26.7 validators==0.18.2 watchdog==2.1.6 webencodings==0.5.1 -wheel zipp==3.6.0 diff --git a/test/e2e/BUILD.bazel b/test/e2e/BUILD.bazel deleted file mode 100644 index 90fc7fb33..000000000 --- a/test/e2e/BUILD.bazel +++ /dev/null @@ -1,131 +0,0 @@ -# make the python rules available -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") - -# make the dependencies from requirements.txt available -load("@knora_py_deps//:requirements.bzl", "requirement") - - - -py_test( - name = "test_connection", - srcs = ["test_connection.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:helpers" - ] -) - -py_test( - name = "test_group", - srcs = ["test_group.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:group", - "//knora/dsplib/models:model", - ] -) - - -py_test( - name = "test_listnode", - srcs = ["test_listnode.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:langstring", - "//knora/dsplib/models:listnode", - ] -) - -py_test( - name = "test_ontology", - srcs = ["test_ontology.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:helpers", - "//knora/dsplib/models:ontology", - ] -) - -py_test( - name = "test_project", - srcs = ["test_project.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:langstring", - "//knora/dsplib/models:project", - ] -) - -py_test( - name = "test_propertyclass", - srcs = ["test_propertyclass.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:langstring", - "//knora/dsplib/models:helpers", - "//knora/dsplib/models:ontology", - "//knora/dsplib/models:propertyclass", - ] -) - -py_test( - name = "test_resourceclass", - srcs = ["test_resourceclass.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:langstring", - "//knora/dsplib/models:helpers", - "//knora/dsplib/models:ontology", - "//knora/dsplib/models:resourceclass", - ] -) - -py_test( - name = "test_user", - srcs = ["test_user.py"], - deps = [ - "//knora/dsplib/models:connection", - "//knora/dsplib/models:langstring", - "//knora/dsplib/models:user", - ] -) - -py_test( - name = "test_resource", - srcs = ["test_resource.py"], - deps = [ - "//knora/dsplib/models:bitstream", - "//knora/dsplib/models:connection", - "//knora/dsplib/models:resource", - "//knora/dsplib/models:group", - "//knora/dsplib/models:value", - "//knora/dsplib/models:permission", - "//knora/dsplib/models:sipi", - ], - data = [ - "//testdata:testdata", - "//testdata/bitstreams:bitstream" - ] -) - -py_test( - name = "test_tools", - srcs = ["test_tools.py"], - deps = [ - "//knora/dsplib/utils:onto_get", - "//knora/dsplib/utils:onto_validate", - "//knora/dsplib/utils:onto_create_ontology", - "//knora/dsplib/utils:xml_upload", - "//knora/dsplib/utils:excel_to_json_lists", - "//knora/dsplib/utils:excel_to_json_resources", - "//knora/dsplib/utils:excel_to_json_properties", - "//knora/dsplib/utils:id_to_iri" - ], - data = [ - "//testdata:testdata", - "//testdata/bitstreams:bitstream" - ] -) - - - diff --git a/test/e2e/test_tools.py b/test/e2e/test_tools.py index ad8a2e5c8..6c67a2506 100644 --- a/test/e2e/test_tools.py +++ b/test/e2e/test_tools.py @@ -17,6 +17,8 @@ class TestTools(unittest.TestCase): server = 'http://0.0.0.0:3333' user = 'root@example.com' test_onto_file = 'testdata/test-onto.json' + test_list_file = 'testdata/test-list.json' + test_data_file = 'testdata/test-data.xml' def setUp(self) -> None: """Is executed before each test""" @@ -34,13 +36,13 @@ def test_get(self) -> None: test_onto = json.loads(onto_json_str) get_ontology(project_identifier='tp', - outfile='_test-onto.json', + outfile='testdata/tmp/_test-onto.json', server=self.server, user=self.user, password='test', verbose=False) - with open('_test-onto.json') as f: + with open('testdata/tmp/_test-onto.json') as f: onto_json_str = f.read() test_onto_out = json.loads(onto_json_str) @@ -145,30 +147,30 @@ def test_get(self) -> None: def test_excel_to_json_list(self) -> None: list_excel2json(listname='my_test_list', excelfolder='testdata/lists', - outfile='_lists-out.json') + outfile='testdata/tmp/_lists-out.json') def test_excel_to_json_resources(self) -> None: resources_excel2json(excelfile='testdata/Resources.xlsx', - outfile='_resources-out.json') + outfile='testdata/tmp/_resources-out.json') def test_excel_to_json_properties(self) -> None: properties_excel2json(excelfile='testdata/Properties.xlsx', - outfile='_properties-out.json') + outfile='testdata/tmp/_properties-out.json') def test_validate_ontology(self) -> None: validate_ontology(self.test_onto_file) def test_create_ontology(self) -> None: create_ontology(input_file=self.test_onto_file, - lists_file='lists-out.json', + lists_file=self.test_list_file, server=self.server, user_mail=self.user, password='test', verbose=False, - dump=True) + dump=False) def test_xml_upload(self) -> None: - xml_upload(input_file='testdata/test-data.xml', + xml_upload(input_file=self.test_data_file, server=self.server, user=self.user, password='test', @@ -181,7 +183,7 @@ def test_xml_upload(self) -> None: def test_id_to_iri(self) -> None: id_to_iri(xml_file='testdata/test-id2iri-data.xml', json_file='testdata/test-id2iri-mapping.json', - out_file='_test-id2iri-replaced.xml', + out_file='testdata/tmp/_test-id2iri-replaced.xml', verbose=False) diff --git a/test/unittests/BUILD.bazel b/test/unittests/BUILD.bazel deleted file mode 100644 index fdd58e735..000000000 --- a/test/unittests/BUILD.bazel +++ /dev/null @@ -1,58 +0,0 @@ -# make the python rules available -load("@rules_python//python:defs.bzl", "py_test") - -# make the dependencies from requirements.txt available -load("@knora_py_deps//:requirements.bzl", "requirement") - -package(default_visibility = ["//visibility:public"]) - -py_test( - name = "test_convert_ark_v0_to_resource_iri", - srcs = ["test_convert_ark_v0_to_resource_iri.py"], - deps = [ - "//knora/dsplib/utils:xml_upload", - "//knora/dsplib/models:helpers" - ], -) - -py_test( - name = "test_excel_to_json_lists", - srcs = ["test_excel_to_json_lists.py"], - deps = [ - "//knora/dsplib/utils:excel_to_json_lists" - ], - data = [ - "//testdata", - ] -) - -py_test( - name = "test_langstring", - srcs = ["test_langstring.py"], -) - -py_test( - name = "test_value", - srcs = ["test_value.py"], - deps = [ - "//knora/dsplib/models:group", - "//knora/dsplib/models:helpers", - "//knora/dsplib/models:value", - ], -) - -py_test( - name = "test_id_to_iri", - srcs = ["test_id_to_iri.py"], - data = [ - "//testdata", - ], -) - -py_test( - name = "test_excel_to_resource", - srcs = ["test_excel_to_resource.py"], - data = [ - "//testdata", - ], -) diff --git a/test/unittests/test_create_ontology.py b/test/unittests/test_create_ontology.py index b74610508..c8d2fc2be 100644 --- a/test/unittests/test_create_ontology.py +++ b/test/unittests/test_create_ontology.py @@ -7,7 +7,9 @@ class TestOntoCreation(unittest.TestCase): - ontology: dict[str, Any] = json.loads('testdata/test-onto.json')['project']['ontologies'][0] + with open('testdata/test-onto.json', 'r') as json_file: + json_onto: dict[str, Any] = json.load(json_file) + ontology: dict[str, Any] = json_onto['project']['ontologies'][0] def test_sort_resources(self) -> None: """ diff --git a/test/unittests/test_excel_to_json_lists.py b/test/unittests/test_excel_to_json_lists.py index f7f57651c..0a8659d0c 100644 --- a/test/unittests/test_excel_to_json_lists.py +++ b/test/unittests/test_excel_to_json_lists.py @@ -14,7 +14,7 @@ class TestExcelToJSONList(unittest.TestCase): def test_excel2jsonlist(self) -> None: # check that the output file was created excelfolder = "testdata/lists" - outfile = "testdata/tmp/lists_output.json" + outfile = "testdata/tmp/_lists_output.json" e2l.list_excel2json(listname=None, excelfolder=excelfolder, outfile=outfile) self.assertTrue(os.path.exists(outfile), f'The outfile {outfile} was not created') diff --git a/test/unittests/test_excel_to_resource.py b/test/unittests/test_excel_to_resource.py index 7ada18735..175c455e2 100644 --- a/test/unittests/test_excel_to_resource.py +++ b/test/unittests/test_excel_to_resource.py @@ -11,7 +11,7 @@ class TestExcelToResource(unittest.TestCase): def test_excel2json(self) -> None: in_file = "testdata/Resources.xlsx" - out_file = "out_res.json" + out_file = "testdata/tmp/_out_res.json" e2j.resources_excel2json(in_file, out_file) self.assertTrue(os.path.exists(out_file)) diff --git a/test/unittests/test_id_to_iri.py b/test/unittests/test_id_to_iri.py index e4be8a938..cf8278cd1 100644 --- a/test/unittests/test_id_to_iri.py +++ b/test/unittests/test_id_to_iri.py @@ -8,7 +8,7 @@ class TestIdToIri(unittest.TestCase): - out_file = '_test-id2iri-replaced.xml' + out_file = 'testdata/tmp/_test-id2iri-replaced.xml' def test_invalid_xml_file_name(self): with self.assertRaises(SystemExit) as cm: diff --git a/testdata/BUILD.bazel b/testdata/BUILD.bazel deleted file mode 100644 index 1e8fa490e..000000000 --- a/testdata/BUILD.bazel +++ /dev/null @@ -1,23 +0,0 @@ -# make the python rules available -#load("@rules_python//python:defs.bzl", "py_binary", "py_library") - -# make the dependencies from requirements.txt available -#load("@knora_py_deps//:requirements.bzl", "requirement") - -filegroup( - name = "testdata", - visibility = ["//visibility:public"], - srcs = [ - "Resources.xlsx", - "Properties.xlsx", - "lists/description_en.xlsx", - "lists/Beschreibung_de.xlsx", - "test-data.xml", - "test-onto.json", - "test-id2iri-data.xml", - "test-id2iri-mapping.json", - "test-id2iri-replaced.xml", - "tmp/_test-id2iri-replaced.xml" - ], -) - diff --git a/testdata/bitstreams/BUILD.bazel b/testdata/bitstreams/BUILD.bazel deleted file mode 100644 index c2ef74bd4..000000000 --- a/testdata/bitstreams/BUILD.bazel +++ /dev/null @@ -1,21 +0,0 @@ -# make the python rules available -#load("@rules_python//python:defs.bzl", "py_binary", "py_library") - -# make the dependencies from requirements.txt available -#load("@knora_py_deps//:requirements.bzl", "requirement") - -filegroup( - name = "bitstream", - visibility = ["//visibility:public"], - srcs = [ - "clara.wav", - "TEMP11.TIF", - "TEMP12.TIF", - "TEMP13.TIF", - "TEMP14.TIF", - "TEMP15.TIF", - "test.csv", - "test.pdf", - "test.zip" - ] -) diff --git a/testdata/test-list.json b/testdata/test-list.json new file mode 100644 index 000000000..e1d28ec6f --- /dev/null +++ b/testdata/test-list.json @@ -0,0 +1,214 @@ +{ + "my-list-from-excel": { + "id": "http://rdfh.ch/lists/4123/xaUpvp27TPCSCM2VUE0beA", + "nodes": [ + { + "first-sublist": { + "id": "http://rdfh.ch/lists/4123/CmNTDARbQd60EU5aZ8GuKg", + "nodes": [ + { + "first-subnode": { + "id": "http://rdfh.ch/lists/4123/w1OAjHTfSWqsRd9KdqInpA", + "nodes": [ + { + "one": { + "id": "http://rdfh.ch/lists/4123/hOlcnyCMRDia9h3U0Yu-2w" + } + }, + { + "two": { + "id": "http://rdfh.ch/lists/4123/Da1zfGHjQfijPy1ljm0gsQ" + } + }, + { + "three": { + "id": "http://rdfh.ch/lists/4123/YuaSpYooQg2rEbha2TO4Mw", + "nodes": [ + { + "fine": { + "id": "http://rdfh.ch/lists/4123/ZZ8P5gZUREuW9KF2HQRnyQ" + } + }, + { + "medium": { + "id": "http://rdfh.ch/lists/4123/B-ImGQ2ZSRSdrd_Qe0GHmQ" + } + }, + { + "coarse": { + "id": "http://rdfh.ch/lists/4123/jdvwvtuxTiq0qH5G1rcMbA" + } + } + ] + } + }, + { + "four": { + "id": "http://rdfh.ch/lists/4123/lhjH93vYRU2Na6Qi2k5siw", + "nodes": [ + { + "fine-2": { + "id": "http://rdfh.ch/lists/4123/nTsw5YrjQouGMYKopMyVyA" + } + }, + { + "medium-2": { + "id": "http://rdfh.ch/lists/4123/p-qz6DPXS52w1kOgRPc21A" + } + }, + { + "coarse-2": { + "id": "http://rdfh.ch/lists/4123/huZ5Zlq0SL2RXP7ltjVbYg" + } + } + ] + } + }, + { + "five": { + "id": "http://rdfh.ch/lists/4123/dgWBMoNvQrGe4jXPiwP2-Q" + } + }, + { + "six": { + "id": "http://rdfh.ch/lists/4123/iXfDKQQeS065WVsWgHxN9g" + } + } + ] + } + }, + { + "second-subnode": { + "id": "http://rdfh.ch/lists/4123/kj3P7j44TCCjv35xye4wWA", + "nodes": [ + { + "one-2": { + "id": "http://rdfh.ch/lists/4123/v50lT3KFSwG-e8cooGTaAw" + } + }, + { + "two-2": { + "id": "http://rdfh.ch/lists/4123/b43EStMpTNCRzToOePmahQ" + } + }, + { + "three-2": { + "id": "http://rdfh.ch/lists/4123/TnmEO1IETOysqbO_gcMUSw" + } + }, + { + "four-2": { + "id": "http://rdfh.ch/lists/4123/hLCBzBGHTE6jn_2P7QmsgA" + } + }, + { + "five-2": { + "id": "http://rdfh.ch/lists/4123/Ortus_1KR8m0qMFLXvBgqQ" + } + }, + { + "six-2": { + "id": "http://rdfh.ch/lists/4123/1HQqlFaqQqutg8p7e3Uxcg" + } + }, + { + "seven": { + "id": "http://rdfh.ch/lists/4123/LSOI-5vFQSCWBtnc65e2fg" + } + } + ] + } + } + ] + } + }, + { + "second-sublist": { + "id": "http://rdfh.ch/lists/4123/vqlWQPHdT_auZOfO0i-Eng", + "nodes": [ + { + "node-of-second-sublist": { + "id": "http://rdfh.ch/lists/4123/8r9qdM7-QqSjUdfe_UsD9Q" + } + } + ] + } + }, + { + "third-sublist": { + "id": "http://rdfh.ch/lists/4123/9noa0WWRT-m5Rad9h6PxjA", + "nodes": [ + { + "special-characters-12-0-are-embedded": { + "id": "http://rdfh.ch/lists/4123/_7_H2nvZTCibz2Mliuza3A", + "nodes": [ + { + "very": { + "id": "http://rdfh.ch/lists/4123/RISTODH5SZea-CEQo0XYlg", + "nodes": [ + { + "deeply": { + "id": "http://rdfh.ch/lists/4123/2pq66F78TE67eCoYBGP1Jg", + "nodes": [ + { + "nested": { + "id": "http://rdfh.ch/lists/4123/gRsrGPITSI6lu7j0Xgw1eQ" + } + } + ] + } + } + ] + } + } + ] + } + } + ] + } + } + ] + }, + "notUsedList": { + "id": "http://rdfh.ch/lists/4123/voLHO1XdTuSvrBkDLEKVDQ", + "nodes": [ + { + "notUsedNode_1": { + "id": "http://rdfh.ch/lists/4123/sHBZOMcXQPecrH3jVAN3QQ" + } + } + ] + }, + "testlist": { + "id": "http://rdfh.ch/lists/4123/73s19IeyRpa7yBLGWRyehw", + "nodes": [ + { + "a": { + "id": "http://rdfh.ch/lists/4123/FAwozOvzQsCOModuUihYug" + } + }, + { + "b": { + "id": "http://rdfh.ch/lists/4123/id5Ij_dOTn-IuPAt7AL0nQ", + "nodes": [ + { + "b1": { + "id": "http://rdfh.ch/lists/4123/kjwEI3eQQxKQoRSGG-650A" + } + }, + { + "b2": { + "id": "http://rdfh.ch/lists/4123/SmgwdTXMQVWueGkPVdPr8Q" + } + } + ] + } + }, + { + "c": { + "id": "http://rdfh.ch/lists/4123/h-_T7oFdTva3TES_EVx9pQ" + } + } + ] + } +} \ No newline at end of file