test: Collect client test data from E2E tests (DSP-724) (#1724)

* test: Add ClientTestDataCollector. * fix(build): Add dependencies. * test: Fix settings. * test: Generate client test data in resources route E2E test. - Add script to download the test data as a Zip file. * test: Start Redis before running tests. - Move request templates from SharedTestDataADM into ResourcesRouteV2E2ESpec. * test: Install redis-cli for e2e tests on GitHub CI. * test: Install redis-cli without redis-server on GitHub CI. * test: Generate client test data in OntologyV2R2RSpec. * refactor (clientTestAPI) move createProject test data generation to e2e tests * refactor (clientTestAPI) remaining project test data * refactor (clientTestAPI) add generation of test data to groups e2e tests * test: Move test data generation into E2E tests for values route. * refactor (clientTestAPI) add generation of users test data to e2e tests * fix (clientTestAPi) correct directory names * refactor (clientTestAPI) move test data generation to permissions e2e tests * test: Generate client test data in SearchRouteV2R2RSpec. * test: Generate client test data in ListsRouteV2R2RSpec. * refactor (clientTestAPI) move generation test data from list route to e2e tests + add missing tests * refactor (clientTestAPI) remove AdminClientApi and ClientApiRoute * refactor(test): Remove obsolete lasses. * test: Check generated client test data. * fix: add the missing test data * test: Add missing test file paths. * style(routing): Optimise imports. * refactor: delete commented method * test: Move shared test data classes from main to test. - Clean up SharedTestDataADM. - Update docs. * test: Make response deterministic. * test: Try to pass environment variable from Makefile. * test: Try to pass environment variable from Makefile. * test: Make it_library depend on test_library and use SharedTestDataADM. * fix(build): Undo salsah1 build breakage. Co-authored-by: Sepideh Alassi <sepideh.alassi@gmail.com> Co-authored-by: Ivan Subotic <400790+subotic@users.noreply.github.com>
dasch-swiss · Oct 9, 2020 · 743cd7e · 743cd7e
1 parent a824bcc
commit 743cd7e
Show file tree

Hide file tree

Showing 133 changed files with 29,325 additions and 22,292 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -118,7 +118,7 @@ jobs:
           java-version: '11' # The OpenJDK version to make available on the path
           architecture: 'x64' # defaults to 'x64'
       - name: install requirements
-        run: sudo apt-get install ca-certificates-java expect
+        run: sudo apt-get install ca-certificates-java expect redis-tools unzip
       - name: Set up Python 3.8
         uses: actions/setup-python@v2
         with:

diff --git a/.gitignore b/.gitignore
@@ -54,3 +54,4 @@ bazel-*
 dump.rdb
 /site/
 dependencies.txt
+/client-test-data.zip
diff --git a/Makefile b/Makefile
@@ -243,6 +243,13 @@ test-unit: docker-build ## runs the dsp-api unit tests.
 test-e2e: docker-build ## runs the dsp-api e2e tests.
 	bazel test //webapi/src/test/scala/org/knora/webapi/e2e/...
 
+.PHONY: client-test-data
+client-test-data: docker-build ## runs the dsp-api e2e tests and generates client test data.
+	docker-compose -f docker-compose.yml up -d redis
+	$(CURRENT_DIR)/webapi/scripts/clear-client-test-data.sh
+	bazel test --cache_test_results=no //webapi/src/test/scala/org/knora/webapi/e2e/... --action_env=KNORA_WEBAPI_COLLECT_CLIENT_TEST_DATA=true
+	$(CURRENT_DIR)/webapi/scripts/dump-client-test-data.sh
+
 .PHONY: test-it
 test-it: docker-build ## runs the dsp-api integration tests.
 	bazel test //webapi/src/it/...

diff --git a/docs/05-internals/development/generating-client-test-data.md b/docs/05-internals/development/generating-client-test-data.md
@@ -26,19 +26,28 @@ client code without the need for a running Knora instance.
 
 ## Implementation
 
-A class for each Knora API extends the `ClientApi` trait.
-A `ClientApi` contains one or more `KnoraRoute` implementations that extend
-`ClientEndpoint`. Each endpoint provides functions that return generated
-client test data.
-
-The route `ClientApiRoute` returns a Zip file containing generated test data.
-returning source code in a Zip file.
+Client test data is generated as a side effect of running Knora's E2E tests.
+E2E tests use `ClientTestDataCollector` to collect test API requests and
+responses. The implementation of `ClientTestDataCollector` collects these
+in a Redis hash. When the E2E tests have completed, the script
+`webapi/scripts/dump-client-test-data.sh` saves the collected test data
+in a Zip file. It then checks the filenames in the Zip file by comparing them
+with the list in `webapi/scripts/expected-client-test-data.txt`.
 
 ## Usage
 
-The following route returns a Zip file containing generated client test
-data:
+To generate client test data, type:
+
+```
+make test-e2e
+```
+
+When the tests have finished running, you will find the file
+`client-test-data.zip` in the current directory.
+
+If generated client test data changes, run `make test-e2e`, then run
+this script to update the list of expected test data files:
 
 ```
-HTTP GET to http://host/clientapitest
+webapi/scripts/update-expected-client-test-data.sh
 ```
diff --git a/salsah1/src/test/scala/org/knora/salsah/ResourceCreationSpec.scala b/salsah1/src/test/scala/org/knora/salsah/ResourceCreationSpec.scala
@@ -22,8 +22,8 @@ package org.knora.salsah
 import org.openqa.selenium.WebElement
 
 /**
-  * Tests the SALSAH web interface using Selenium.
-  */
+ * Tests the SALSAH web interface using Selenium.
+ */
 class ResourceCreationSpec extends SalsahSpec {
     /*
 

diff --git a/salsah1/src/test/scala/org/knora/salsah/SearchAndEditSpec.scala b/salsah1/src/test/scala/org/knora/salsah/SearchAndEditSpec.scala
@@ -24,8 +24,8 @@ import org.scalatest.concurrent.Eventually
 import org.scalatest.concurrent.Eventually._
 
 /**
-  * Tests the SALSAH web interface using Selenium.
-  */
+ * Tests the SALSAH web interface using Selenium.
+ */
 class SearchAndEditSpec extends SalsahSpec {
     /*
 

diff --git a/test_data/ontologyR2RV2/allOntologyMetadata.jsonld b/test_data/ontologyR2RV2/allOntologyMetadata.jsonld
@@ -1,124 +1,140 @@
+
 {
-  "@graph" : [ {
-    "@id" : "http://0.0.0.0:3333/ontology/0001/anything/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/0001"
-    },
-    "knora-api:lastModificationDate" : {
-      "@type" : "xsd:dateTimeStamp",
-      "@value" : "2017-12-19T15:23:42.166Z"
-    },
-    "rdfs:label" : "The anything ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/0001/minimal/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/0001"
-    },
-    "knora-api:lastModificationDate" : {
-      "@type" : "xsd:dateTimeStamp",
-      "@value" : "2019-09-10T08:57:46.633162Z"
-    },
-    "rdfs:label" : "A minimal ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/0001/something/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/0001"
-    },
-    "rdfs:label" : "The something ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/00FF/images/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/00FF"
-    },
-    "rdfs:label" : "The images demo ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/0801/beol/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF"
-    },
-    "rdfs:label" : "The BEOL ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/0801/biblio/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF"
-    },
-    "rdfs:label" : "The Biblio ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/0803/incunabula/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/0803"
-    },
-    "rdfs:label" : "The incunabula ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/0804/dokubib/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/0804"
-    },
-    "rdfs:label" : "The dokubib ontology"
-  }, {
-    "@id" : "http://0.0.0.0:3333/ontology/08AE/webern/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://rdfh.ch/projects/08AE"
-    },
-    "rdfs:label" : "The Anton Webern project ontology"
-  }, {
-    "@id" : "http://api.knora.org/ontology/knora-admin/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://www.knora.org/ontology/knora-admin#SystemProject"
-    },
-    "knora-api:isBuiltIn" : true,
-    "rdfs:label" : "The Knora admin ontology"
-  }, {
-    "@id" : "http://api.knora.org/ontology/knora-api/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://www.knora.org/ontology/knora-admin#SystemProject"
-    },
-    "knora-api:isBuiltIn" : true,
-    "rdfs:label" : "The knora-api ontology in the complex schema"
-  }, {
-    "@id" : "http://api.knora.org/ontology/salsah-gui/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://www.knora.org/ontology/knora-admin#SystemProject"
-    },
-    "knora-api:isBuiltIn" : true,
-    "rdfs:label" : "The salsah-gui ontology"
-  }, {
-    "@id" : "http://api.knora.org/ontology/shared/example-box/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://www.knora.org/ontology/knora-admin#DefaultSharedOntologiesProject"
-    },
-    "knora-api:isShared" : true,
-    "knora-api:lastModificationDate" : {
-      "@type" : "xsd:dateTimeStamp",
-      "@value" : "2018-09-10T14:53:00Z"
-    },
-    "rdfs:label" : "An example of a shared ontology"
-  }, {
-    "@id" : "http://api.knora.org/ontology/standoff/v2",
-    "@type" : "owl:Ontology",
-    "knora-api:attachedToProject" : {
-      "@id" : "http://www.knora.org/ontology/knora-admin#SystemProject"
-    },
-    "knora-api:isBuiltIn" : true,
-    "rdfs:label" : "The standoff ontology"
-  } ],
-  "@context" : {
-    "knora-api" : "http://api.knora.org/ontology/knora-api/v2#",
-    "xsd" : "http://www.w3.org/2001/XMLSchema#",
-    "rdfs" : "http://www.w3.org/2000/01/rdf-schema#",
-    "owl" : "http://www.w3.org/2002/07/owl#"
-  }
+    "@graph": [
+        {
+            "knora-api:lastModificationDate": {
+                "@value": "2017-12-19T15:23:42.166Z",
+                "@type": "xsd:dateTimeStamp"
+            },
+            "rdfs:label": "The anything ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/0001"
+            },
+            "@type": "owl:Ontology",
+            "@id": "http://0.0.0.0:3333/ontology/0001/anything/v2"
+        },
+        {
+            "knora-api:lastModificationDate": {
+                "@value": "2019-09-10T08:57:46.633162Z",
+                "@type": "xsd:dateTimeStamp"
+            },
+            "rdfs:label": "A minimal ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/0001"
+            },
+            "@type": "owl:Ontology",
+            "@id": "http://0.0.0.0:3333/ontology/0001/minimal/v2"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/0001/something/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/0001"
+            },
+            "rdfs:label": "The something ontology"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/00FF/images/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/00FF"
+            },
+            "rdfs:label": "The images demo ontology"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/0801/beol/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF"
+            },
+            "rdfs:label": "The BEOL ontology"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/0801/biblio/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF"
+            },
+            "rdfs:label": "The Biblio ontology"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/0803/incunabula/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/0803"
+            },
+            "rdfs:label": "The incunabula ontology"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/0804/dokubib/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/0804"
+            },
+            "rdfs:label": "The dokubib ontology"
+        },
+        {
+            "@id": "http://0.0.0.0:3333/ontology/08AE/webern/v2",
+            "@type": "owl:Ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://rdfh.ch/projects/08AE"
+            },
+            "rdfs:label": "The Anton Webern project ontology"
+        },
+        {
+            "rdfs:label": "The Knora admin ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://www.knora.org/ontology/knora-admin#SystemProject"
+            },
+            "knora-api:isBuiltIn": true,
+            "@type": "owl:Ontology",
+            "@id": "http://api.knora.org/ontology/knora-admin/v2"
+        },
+        {
+            "rdfs:label": "The knora-api ontology in the complex schema",
+            "knora-api:attachedToProject": {
+                "@id": "http://www.knora.org/ontology/knora-admin#SystemProject"
+            },
+            "knora-api:isBuiltIn": true,
+            "@type": "owl:Ontology",
+            "@id": "http://api.knora.org/ontology/knora-api/v2"
+        },
+        {
+            "rdfs:label": "The salsah-gui ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://www.knora.org/ontology/knora-admin#SystemProject"
+            },
+            "knora-api:isBuiltIn": true,
+            "@type": "owl:Ontology",
+            "@id": "http://api.knora.org/ontology/salsah-gui/v2"
+        },
+        {
+            "knora-api:lastModificationDate": {
+                "@value": "2018-09-10T14:53:00Z",
+                "@type": "xsd:dateTimeStamp"
+            },
+            "rdfs:label": "An example of a shared ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://www.knora.org/ontology/knora-admin#DefaultSharedOntologiesProject"
+            },
+            "knora-api:isShared": true,
+            "@type": "owl:Ontology",
+            "@id": "http://api.knora.org/ontology/shared/example-box/v2"
+        },
+        {
+            "rdfs:label": "The standoff ontology",
+            "knora-api:attachedToProject": {
+                "@id": "http://www.knora.org/ontology/knora-admin#SystemProject"
+            },
+            "knora-api:isBuiltIn": true,
+            "@type": "owl:Ontology",
+            "@id": "http://api.knora.org/ontology/standoff/v2"
+        }
+    ],
+    "@context": {
+        "knora-api": "http://api.knora.org/ontology/knora-api/v2#",
+        "xsd": "http://www.w3.org/2001/XMLSchema#",
+        "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
+        "owl": "http://www.w3.org/2002/07/owl#"
+    }
 }
diff --git a/test_data/ontologyR2RV2/anythingHasDate.jsonld b/test_data/ontologyR2RV2/anythingHasDate.jsonld
@@ -0,0 +1,43 @@
+
+{
+    "knora-api:lastModificationDate": {
+        "@value": "2017-12-19T15:23:42.166Z",
+        "@type": "xsd:dateTimeStamp"
+    },
+    "rdfs:label": "The anything ontology",
+    "@graph": [
+        {
+            "rdfs:label": "Date",
+            "rdfs:subPropertyOf": {
+                "@id": "knora-api:hasValue"
+            },
+            "knora-api:isEditable": true,
+            "knora-api:isResourceProperty": true,
+            "knora-api:subjectType": {
+                "@id": "anything:Thing"
+            },
+            "@type": "owl:ObjectProperty",
+            "knora-api:objectType": {
+                "@id": "knora-api:DateValue"
+            },
+            "salsah-gui:guiElement": {
+                "@id": "salsah-gui:Date"
+            },
+            "@id": "anything:hasDate"
+        }
+    ],
+    "knora-api:attachedToProject": {
+        "@id": "http://rdfh.ch/projects/0001"
+    },
+    "@type": "owl:Ontology",
+    "@id": "http://0.0.0.0:3333/ontology/0001/anything/v2",
+    "@context": {
+        "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
+        "knora-api": "http://api.knora.org/ontology/knora-api/v2#",
+        "owl": "http://www.w3.org/2002/07/owl#",
+        "salsah-gui": "http://api.knora.org/ontology/salsah-gui/v2#",
+        "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
+        "xsd": "http://www.w3.org/2001/XMLSchema#",
+        "anything": "http://0.0.0.0:3333/ontology/0001/anything/v2#"
+    }
+}