Merge branch 'main' into DEV-1503-dsp-api-remove-cache-service-redis-…

…impl

.github/pull_request_template.md

@@ -1,9 +1,13 @@
 <!-- Important! Please follow the guidelines for naming Pull Requests: https://docs.dasch.swiss/latest/developers/dsp/contribution/ -->
 
-Issue Number: DEV-
-
 ## Pull Request Checklist
 
+### Task Description/Number
+
+<!-- Please add either the Jira ticket number or, in case of unscheduled work, a short description of the task at hand -->
+
+Issue Number: DEV-
+
 ### Basic Requirements
 
 Please check if your PR fulfills the following requirements:
@@ -35,5 +39,3 @@ What kind of change does this PR introduce?
 
 - [ ] Yes (don't forget to update the JS-LIB team about the change)
 - [ ] No
-
-### Other information

.github/workflows/build-and-test.yml

@@ -14,7 +14,39 @@ jobs:
       - name: Run preparatory steps
         uses: dasch-swiss/dsp-api/.github/actions/preparation@main
       - name: Run all tests
-        run: make test
+        run: sbt -v coverage test coverageAggregate
+      - name:  WebApi Unit Test Report
+        uses: dorny/test-reporter@v1
+        if: success() || failure()
+        with:
+          name: WebApi Unit Test Results
+          path: ./webapi/target/test-reports/TEST-*.xml
+          reporter: java-junit
+      - name: Upload coverage data to codacy
+        uses: codacy/codacy-coverage-reporter-action@v1
+        with:
+          project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+          coverage-reports: ./target/scala-2.13/coverage-report/cobertura.xml
+      - name: Upload coverage data to codecov
+        uses: codecov/codecov-action@v3
+        with:
+          files: ./target/scala-2.13/coverage-report/cobertura.xml
+
+  integration-test:
+    name: Build and integration-test
+    runs-on: ubuntu-latest
+    steps:
+      - name: Run preparatory steps
+        uses: dasch-swiss/dsp-api/.github/actions/preparation@main
+      - name: Run all integration tests
+        run: make integration-test
+      - name:  WebApi Integration Test Report
+        uses: dorny/test-reporter@v1
+        if: success() || failure()
+        with:
+          name: WebApi Integration Test Results
+          path: ./webapi/target/it-reports/TEST-*.xml
+          reporter: java-junit
       - name: Upload coverage data to codacy
         uses: codacy/codacy-coverage-reporter-action@v1
         with:

.github/workflows/publish-from-branch.yml

@@ -0,0 +1,25 @@
+name: Publish from branch 
+
+on:
+  workflow_dispatch:
+
+jobs:
+  publish-from-branch:
+    name: Publish from branch
+    runs-on: ubuntu-latest
+    steps:
+      - name: Build and publish all images to Dockerhub
+        run: |
+          echo ${{ secrets.DOCKER_HUB_TOKEN }} | docker login -u ${{ secrets.DOCKER_USER }} --password-stdin
+          make docker-publish
+
+  send-chat-notification:
+    name: Send google chat notification
+    needs: [publish-from-branch]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Send notification to google chat room "DSP releases"
+        uses: lakto/google-chat-action@main
+        with:
+          url: ${{ secrets.GOOGLE_CHAT_DSP_RELEASES_WEBHOOK_URL }}
+

CHANGELOG.md

@@ -1,5 +1,50 @@
 # Changelog
 
+## [25.0.0](https://www.github.com/dasch-swiss/dsp-api/compare/v24.0.8...v25.0.0) (2022-12-02)
+
+
+### ⚠ BREAKING CHANGES
+
+* partOf and sequenceOf properties are not marked as isEditable (#2268)
+* change all project IRIs to contain UUID instead of shortcode (DEV-1400) (#2255)
+
+### Bug Fixes
+
+* Allow warn logging for requests/responses which are failures ([#2273](https://www.github.com/dasch-swiss/dsp-api/issues/2273)) ([92531ce](https://www.github.com/dasch-swiss/dsp-api/commit/92531ceb0c38f0c6117bc08633bb66a89dbcabc7))
+* Ask timeouts with GetUserADM (DEV-1443) ([#2267](https://www.github.com/dasch-swiss/dsp-api/issues/2267)) ([3f77b6e](https://www.github.com/dasch-swiss/dsp-api/commit/3f77b6e5708e94f7531c5df09164cbfce20b9fc1))
+* Deprecation warnings for SCryptPasswordEncoder ([#2308](https://www.github.com/dasch-swiss/dsp-api/issues/2308)) ([86dc389](https://www.github.com/dasch-swiss/dsp-api/commit/86dc389fb2a45d99e82c601f27bcd813dcf8be12))
+* Don't log hashes (DEV-1442) ([#2265](https://www.github.com/dasch-swiss/dsp-api/issues/2265)) ([adaf4b0](https://www.github.com/dasch-swiss/dsp-api/commit/adaf4b0ca982464e38672b2cc3aaac04c802b49f))
+* Exclude characters with special meaning in Lucene Query Parser syntax from searchbylabel search (DEV-1446) ([#2269](https://www.github.com/dasch-swiss/dsp-api/issues/2269)) ([b359916](https://www.github.com/dasch-swiss/dsp-api/commit/b3599163f3e74375da657e14e47387d598f8485e))
+* fix RepositoryUpdater that is not timing out during repository upgrade (DEV-1534) ([#2313](https://www.github.com/dasch-swiss/dsp-api/issues/2313)) ([213a5f0](https://www.github.com/dasch-swiss/dsp-api/commit/213a5f0fd6cd48ffba8128e33b6a964059ed3309))
+* Increase timeout when emptying repository (DEV-1506) ([#2289](https://www.github.com/dasch-swiss/dsp-api/issues/2289)) ([39771ed](https://www.github.com/dasch-swiss/dsp-api/commit/39771ed743cd485679a05e5679a25352a8891aea))
+* key frame extraction (DEV-1513) ([#2300](https://www.github.com/dasch-swiss/dsp-api/issues/2300)) ([729f071](https://www.github.com/dasch-swiss/dsp-api/commit/729f0718ff6ac831e91279b77046e4f6b9438b65))
+* partOf and sequenceOf properties are not marked as isEditable ([#2268](https://www.github.com/dasch-swiss/dsp-api/issues/2268)) ([68f19c3](https://www.github.com/dasch-swiss/dsp-api/commit/68f19c3e570c470563d831ab7f8483e1f392de6f))
+
+
+### Enhancements
+
+* **projectsADM:** add possibility to get project and members by UUID (DEV-1408) ([#2272](https://www.github.com/dasch-swiss/dsp-api/issues/2272)) ([4b66682](https://www.github.com/dasch-swiss/dsp-api/commit/4b666829ff81a447cd1c1fae304bb5ca518fbab9))
+
+
+### Documentation
+
+* improve permissions documentation ([#2314](https://www.github.com/dasch-swiss/dsp-api/issues/2314)) ([f4004b2](https://www.github.com/dasch-swiss/dsp-api/commit/f4004b2525821285ac798b251a0a11ba42cc5daf))
+* publish architectural decision records ([#2301](https://www.github.com/dasch-swiss/dsp-api/issues/2301)) ([be6bcd0](https://www.github.com/dasch-swiss/dsp-api/commit/be6bcd0b1b6b8d321e2105f25431a43d5814d5c6))
+* Remove warning which considers v2 as not production ready ([#2282](https://www.github.com/dasch-swiss/dsp-api/issues/2282)) ([0246522](https://www.github.com/dasch-swiss/dsp-api/commit/024652259ada84f49c4615f91aa3f585da14d254))
+
+
+### Maintenance
+
+* add GH workflow to publish manually from branches ([#2316](https://www.github.com/dasch-swiss/dsp-api/issues/2316)) ([6f5020e](https://www.github.com/dasch-swiss/dsp-api/commit/6f5020ea54cd0df9a96d132043992f612ffb09cd))
+* change all project IRIs to contain UUID instead of shortcode (DEV-1400) ([#2255](https://www.github.com/dasch-swiss/dsp-api/issues/2255)) ([f2b2584](https://www.github.com/dasch-swiss/dsp-api/commit/f2b2584489748bf8a9d56044744c7821582bdce8))
+* Decrease timeout for emptying repository (DEV-1518) ([#2310](https://www.github.com/dasch-swiss/dsp-api/issues/2310)) ([a83000b](https://www.github.com/dasch-swiss/dsp-api/commit/a83000ba0b234b294c80931c5930fe15b26c9fdc))
+* Introduce ZIO HTTP (DEV-1425) ([#2256](https://www.github.com/dasch-swiss/dsp-api/issues/2256)) ([7ae6d24](https://www.github.com/dasch-swiss/dsp-api/commit/7ae6d24604cda006b85aceda689ed45089025f5a))
+* make possible to run Publish GH Action manually (DEV-1519) ([#2297](https://www.github.com/dasch-swiss/dsp-api/issues/2297)) ([bfe578a](https://www.github.com/dasch-swiss/dsp-api/commit/bfe578a37f7609cfac0b2c38e318c98641eebb49))
+* **SIPI:** add timestamp to some SIPI Lua logs ([#2311](https://www.github.com/dasch-swiss/dsp-api/issues/2311)) ([8f3f19f](https://www.github.com/dasch-swiss/dsp-api/commit/8f3f19f235a3787afa43b010afedac86bd320958))
+* slight improvements to PR template ([#2312](https://www.github.com/dasch-swiss/dsp-api/issues/2312)) ([ca3a8d0](https://www.github.com/dasch-swiss/dsp-api/commit/ca3a8d0def066e8b3b949080bd8d4e163948c73f))
+* update dependencies ([#2264](https://www.github.com/dasch-swiss/dsp-api/issues/2264)) ([41d5315](https://www.github.com/dasch-swiss/dsp-api/commit/41d53150e3fee7d91d6c67fda3511e02644e38e3))
+* update dependencies ([#2281](https://www.github.com/dasch-swiss/dsp-api/issues/2281)) ([725bc0f](https://www.github.com/dasch-swiss/dsp-api/commit/725bc0f750ec9503ea7902d54a549d848983c358))
+
 ### [24.0.8](https://www.github.com/dasch-swiss/dsp-api/compare/v24.0.7...v24.0.8) (2022-10-18)
 
 
@@ -1524,7 +1569,7 @@
 * MAJOR: Fix property names for incoming links (#1144))
 * MAJOR: Generate and resolve ARK URLs for resources (#1161). Projects
   that have resource IRIs that do not conform to the format specified in
-  <https://docs.knora.org/paradox/03-apis/api-v2/knora-iris.html#iris-for-data>
+  <https://docs.knora.org/paradox/03-endpoints/api-v2/knora-iris.html#iris-for-data>
   must update them.
 * MAJOR: Use project shortcode in IIIF URLs (#1191). If you have file value IRIs containing the substring `/reps/`, you must replace `/reps/` with `/values/`.
 

Makefile

@@ -86,7 +86,7 @@ env-file: ## write the env file used by dsp-stack.
 #################################
 
 .PHONY: stack-up
-stack-up: docker-build env-file ## starts the dsp-stack: fuseki, sipi, api.
+stack-up: docker-build env-file ## starts the dsp-stack: fuseki, sipi, api and app.
 	@docker compose -f docker-compose.yml up -d db
 	$(CURRENT_DIR)/webapi/scripts/wait-for-db.sh
 	@docker compose -f docker-compose.yml up -d
@@ -167,6 +167,10 @@ stack-config: env-file
 stack-without-api: stack-up ## starts the dsp-stack without dsp-api: fuseki and sipi only.
 	@docker compose -f docker-compose.yml stop api
 
+.PHONY: stack-without-app
+stack-without-app: stack-up ## starts the dsp-stack without dsp-app - this is the previous state of "make stack-up" command.
+	@docker compose -f docker-compose.yml stop app
+
 .PHONY: stack-without-api-and-sipi
 stack-without-api-and-sipi: stack-up ## starts the dsp-stack without dsp-api and sipi: fuseki only.
 	@docker compose -f docker-compose.yml stop api
@@ -185,7 +189,7 @@ stack-db-only: env-file  ## starts only fuseki.
 client-test-data: export KNORA_WEBAPI_COLLECT_CLIENT_TEST_DATA := true
 client-test-data: build ## runs the dsp-api e2e and r2r tests and generates client-test-data.
 	$(CURRENT_DIR)/webapi/scripts/zap-client-test-data.sh
-	sbt -v "webapi/testOnly *E2ESpec *R2RSpec"
+	sbt -v "webapi/IntegrationTest/testOnly *E2ESpec *R2RSpec"
 	$(CURRENT_DIR)/webapi/scripts/zip-client-test-data.sh
 
 .PHONY: test-repository-upgrade
@@ -201,12 +205,17 @@ test-repository-upgrade: build init-db-test-minimal ## runs DB upgrade integrati
 	# after a certain time. at startup, data should be upgraded.
 	@$(MAKE) -f $(THIS_FILE) stack-up
 
+.PHONY: test-all
+test-all: test integration-test
+
 .PHONY: test
-test: build test-shared test-user-slice test-role-slice test-project-slice ## runs all tests
-	sbt -v coverage "webapi/test"
-	sbt -v coverage "schemaCore/test"
-	sbt coverageAggregate
-
+test: ## runs all unit tests
+	sbt -v coverage test coverageAggregate
+
+.PHONY: integration-test
+integration-test: docker-build-sipi-image ## runs all integration tests
+	sbt -v coverage "IntegrationTest/test" coverageAggregate
+
 .PHONY: test-shared
 test-shared: ## tests the shared projects (build is not called from this target)
 	sbt -v coverage "shared/test"

README.md

@@ -94,10 +94,25 @@ make stack-down
 
 ### Run the automated tests
 
+Automated tests are split into different source sets into slow running integration tests (i.e. tests which do IO or are
+using [Testcontainers](https://www.testcontainers.org/)) and fast running unit tests.
+
+Run unit tests:
+
+```shell
+sbt test
+```
+
+Run integration tests:
+
+```shell
+make integration-test
+```
+
 Run all tests:
 
 ```shell
-make test
+make test-all
 ```
 
 ## Release Versioning Convention
@@ -111,4 +126,4 @@ The DSP-API release versioning follows the [Semantic Versioning](https://semver.
 > * PATCH version when you make backwards-compatible bug fixes.
 
 Additionally, we will also increment the MAJOR version in the case when any kind of changes to existing
-data would be necessary, e.g., any changes to the [knora-base ontology](https://docs.dasch.swiss/latest/DSP-API/02-knora-ontologies/knora-base/) which are not backwards compatible.
+data would be necessary, e.g., any changes to the [knora-base ontology](https://docs.dasch.swiss/latest/DSP-API/02-dsp-ontologies/knora-base/) which are not backwards compatible.