Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Replace Settings with AppConfig (DEV-1312) #2202

Merged
merged 45 commits into from Sep 20, 2022
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9c20c33
add placeholders to not invoke warnings
Sep 12, 2022
a963246
remove settings from OntologyHelpers
Sep 12, 2022
83ec9bf
fix deprecation warnings in GravsearchParser
Sep 12, 2022
a45c32e
update login form
Sep 12, 2022
497b384
remove settings from CardinalityHandler
Sep 12, 2022
8836777
remove settings from Cache
Sep 12, 2022
05a0a8a
remove config from V1 responders
Sep 12, 2022
6db9257
remove settings from V1 responders
Sep 12, 2022
fa3689c
use appConfig in V2 responders
Sep 12, 2022
fcb5f2e
replace settings in responders V2
Sep 12, 2022
43a0043
remove unnecessary brackets
Sep 12, 2022
44f7a71
remove unused code
Sep 12, 2022
e032cd9
add appConfig to Responder
Sep 12, 2022
09c8346
wip: all but one test fails
Sep 13, 2022
603ced6
Merge branch 'main' into wip/dev-1312-replace-settings-with-appconfig
Sep 13, 2022
9554024
fix failing tests
Sep 14, 2022
ad1e6c1
use appConfig for cache definitions
Sep 14, 2022
04009a2
remove settings from V1 spec
Sep 14, 2022
952a322
remove settings from V1 test data
Sep 14, 2022
85d1221
replace all but one instances of settings
Sep 14, 2022
9497616
refactor CacheUtilSpec to remove settings
Sep 14, 2022
f27a8de
add missing things and clean-up
Sep 14, 2022
ee9b42f
add missing config
Sep 14, 2022
5878836
reformat
Sep 15, 2022
36e77b7
use Try instead of try catch
Sep 15, 2022
94a0e1e
clean up
Sep 15, 2022
33dfffe
improve readability of code and clean up
Sep 15, 2022
9c66344
Create .codecov.yml
Sep 16, 2022
b2a94ab
simplify and clean-up
Sep 16, 2022
b27d8b2
fix failing test
Sep 16, 2022
60298f7
remove appConfig as param for routes
Sep 16, 2022
56f99a1
remove appConfig as param from responders
Sep 16, 2022
5f4aa8c
fix formatting
Sep 16, 2022
07c3f6b
Update ResourcesRouteV2.scala
Sep 16, 2022
85df8fd
refactor CacheUtilSpec
Sep 16, 2022
303ed62
redo refactoring on CacheUtilSpec
Sep 16, 2022
a810c2e
simplify main
Sep 19, 2022
d3ba0a2
fix stack-restart
Sep 19, 2022
9d06498
fix docstring
Sep 19, 2022
3384452
update Makefile
Sep 19, 2022
892f15f
Remove outdated documentation
Sep 19, 2022
fe68ea1
implement feedback from review
Sep 19, 2022
d892d82
Clearify requiresRepository
Sep 19, 2022
966fce8
use ZIO.unit instead of ZIO.succeed(())
Sep 19, 2022
b7f3e65
reformat
Sep 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions .codecov.yml
@@ -0,0 +1,8 @@
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
24 changes: 12 additions & 12 deletions Makefile
Expand Up @@ -86,7 +86,7 @@ env-file: ## write the env file used by knora-stack.
#################################

.PHONY: stack-up
stack-up: docker-build env-file ## starts the knora-stack: fuseki, sipi, redis, api.
stack-up: docker-build env-file ## starts the knora-stack: fuseki, sipi, api.
@docker compose -f docker-compose.yml up -d db
$(CURRENT_DIR)/webapi/scripts/wait-for-db.sh
@docker compose -f docker-compose.yml up -d
Expand All @@ -98,12 +98,16 @@ stack-up-fast: docker-build-knora-api-image env-file ## starts the knora-stack b

.PHONY: stack-up-ci
stack-up-ci: KNORA_DB_REPOSITORY_NAME := knora-test-unit
stack-up-ci: docker-build env-file print-env-file ## starts the knora-stack using 'knora-test-unit' repository: fuseki, sipi, redis, api.
stack-up-ci: docker-build env-file print-env-file ## starts the knora-stack using 'knora-test-unit' repository: fuseki, sipi, api.
docker-compose -f docker-compose.yml up -d

.PHONY: stack-restart
stack-restart: stack-up ## re-starts the knora-stack: fuseki, sipi, redis, api.
@docker compose -f docker-compose.yml restart
stack-restart: ## re-starts the knora-stack: fuseki, sipi, api.
@docker compose -f docker-compose.yml down
@docker compose -f docker-compose.yml up -d db
$(CURRENT_DIR)/webapi/scripts/wait-for-db.sh
@docker compose -f docker-compose.yml up -d
$(CURRENT_DIR)/webapi/scripts/wait-for-knora.sh

.PHONY: stack-restart-api
stack-restart-api: ## re-starts the api. Usually used after loading data into fuseki.
Expand All @@ -130,10 +134,6 @@ stack-logs-sipi: ## prints out and follows the logs of the 'sipi' container runn
stack-logs-sipi-no-follow: ## prints out the logs of the 'sipi' container running in knora-stack.
@docker compose -f docker-compose.yml logs sipi

.PHONY: stack-logs-redis
stack-logs-redis: ## prints out and follows the logs of the 'redis' container running in knora-stack.
@docker compose -f docker-compose.yml logs -f redis

.PHONY: stack-logs-api
stack-logs-api: ## prints out and follows the logs of the 'api' container running in knora-stack.
@docker compose -f docker-compose.yml logs -f api
Expand Down Expand Up @@ -164,11 +164,11 @@ stack-config: env-file

## stack without api
.PHONY: stack-without-api
stack-without-api: stack-up ## starts the knora-stack without knora-api: fuseki, sipi, redis.
stack-without-api: stack-up ## starts the knora-stack without knora-api: fuseki and sipi only.
@docker compose -f docker-compose.yml stop api

.PHONY: stack-without-api-and-sipi
stack-without-api-and-sipi: stack-up ## starts the knora-stack without knora-api and sipi: fuseki, redis.
stack-without-api-and-sipi: stack-up ## starts the knora-stack without knora-api and sipi: fuseki only.
@docker compose -f docker-compose.yml stop api
@docker compose -f docker-compose.yml stop sipi

Expand Down Expand Up @@ -348,11 +348,11 @@ clean-sipi-projects: ## deletes all files uploaded within a project
@rm -rf sipi/images/originals/[0-9A-F][0-9A-F][0-9A-F][0-9A-F]

.PHONY: check
check: # Run code formating check
check: ## Run code formating check
@sbt "check"

.PHONY: fmt
fmt: # Run code formating fix
fmt: ## Run code formating fix
@sbt "fmt"


Expand Down
Expand Up @@ -4,3 +4,5 @@
*/

package dsp.project.listener.external

object placeholder
Expand Up @@ -4,3 +4,5 @@
*/

package dsp.project.listener.internal

object placeholder
Expand Up @@ -4,3 +4,5 @@
*/

package dsp.project.route

object placeholder
Expand Up @@ -4,3 +4,5 @@
*/

package dsp.project.listener.external

object placeholder
Expand Up @@ -4,3 +4,5 @@
*/

package dsp.project.listener.internal

object placeholder
Expand Up @@ -4,3 +4,5 @@
*/

package dsp.project.route

object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.api
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.listener.external
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.listener.internal
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.route
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.listener.external
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.listener.internal
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -8,4 +8,4 @@ package dsp.role.route
/**
* To be implemented...
*/
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.api

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.listener.external

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.listener.internal

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.route

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.listener.external

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.listener.internal

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
Expand Up @@ -7,4 +7,4 @@ package dsp.user.route

// this placeholder can be removed as soon as the code are there
// without it there is a warning
object placeholder {}
object placeholder
24 changes: 10 additions & 14 deletions webapi/src/main/resources/application.conf
Expand Up @@ -267,6 +267,8 @@ app {
allow-reload-over-http = false
allow-reload-over-http = ${?KNORA_WEBAPI_ALLOW_RELOAD_OVER_HTTP}

fallback-language = "en"

knora-api {
// relevant for direct communication inside the knora stack
internal-host = "0.0.0.0"
Expand Down Expand Up @@ -304,11 +306,8 @@ app {

file-server-path = "server"

v2 {
file-metadata-route = "knora.json"
move-file-route = "store"
delete-temp-file-route = "delete_temp_file"
}
move-file-route = "store"
delete-temp-file-route = "delete_temp_file"

image-mime-types = ["image/tiff", "image/jpeg", "image/png", "image/jp2", "image/jpx"]
document-mime-types = [
Expand Down Expand Up @@ -430,8 +429,8 @@ app {
gui {
// The default size of resource type icons. TODO: put icon sizes in the triplestore instead.
default-icon-size {
dimX = 32
dimY = 32
dim-x = 32
dim-y = 32
}
}

Expand All @@ -442,7 +441,6 @@ app {
triplestore {
dbtype = "fuseki"
dbtype = ${?KNORA_WEBAPI_TRIPLESTORE_DBTYPE}
// dbtype = "fake-triplestore"

use-https = false
use-https = ${?KNORA_WEBAPI_TRIPLESTORE_USE_HTTPS}
Expand Down Expand Up @@ -471,8 +469,6 @@ app {
password = ${?KNORA_WEBAPI_TRIPLESTORE_FUSEKI_PASSWORD}
}

reload-on-start = false // ignored if "memory" as it will always reload

// If true, the time taken by each SPARQL query is logged at DEBUG level. To see these messages,
// set loglevel = "DEBUG" above, and
// <logger name="org.knora.webapi.store.triplestore.http.HttpTriplestoreConnector" level="DEBUG"/>
Expand All @@ -499,16 +495,16 @@ app {
}
}

client-test-data-service {
collect-client-test-data = false
}

monitoring {
prometheus-endpoint: false
prometheus-endpoint: ${?KNORA_WEBAPI_PROMETHEUS_ENDPOINT}
}
}

user {
default-language: "en"
}

kamon.prometheus.embedded-server {

# Hostname and port used by the embedded web server to publish the
Expand Down
13 changes: 4 additions & 9 deletions webapi/src/main/scala/org/knora/webapi/Main.scala
Expand Up @@ -2,7 +2,6 @@
* Copyright © 2021 - 2022 Swiss National Data and Service Center for the Humanities and/or DaSCH Service Platform contributors.
* SPDX-License-Identifier: Apache-2.0
*/

package org.knora.webapi

import zio._
Expand All @@ -12,6 +11,8 @@ import org.knora.webapi.core.AppServer

object Main extends ZIOApp {

override def environmentTag: EnvironmentTag[Environment] = EnvironmentTag[Environment]

/**
* The `Environment` that we require to exist at startup.
*/
Expand All @@ -21,19 +22,13 @@ object Main extends ZIOApp {
* `Bootstrap` will ensure that everything is instantiated when the Runtime is created
* and cleaned up when the Runtime is shutdown.
*/
override val bootstrap: ZLayer[
override def bootstrap: ZLayer[
ZIOAppArgs with Scope,
Any,
Environment
] = ZLayer.empty ++ Runtime.removeDefaultLoggers ++ SLF4J.slf4j ++ core.LayersLive.dspLayersLive

/* Needed for ZIO type magic */
override val environmentTag: EnvironmentTag[Environment] = EnvironmentTag[Environment]

/* Here we start our Application */
override val run =
(for {
never <- ZIO.never
} yield never).provideLayer(AppServer.live)
override def run = AppServer.live.launch

}
12 changes: 7 additions & 5 deletions webapi/src/main/scala/org/knora/webapi/auth/JWTService.scala
Expand Up @@ -3,11 +3,13 @@ package org.knora.webapi.auth
import spray.json.JsValue
import zio._

import scala.concurrent.duration

import org.knora.webapi._
import org.knora.webapi.config._
import org.knora.webapi.routing.JWTHelper

final case class JWTService(config: AppConfig) {
final case class JWTService(secret: String, longevity: duration.Duration, issuer: String) {

/**
* Creates a new JWT token for a specific user and holds some additional
Expand All @@ -20,9 +22,9 @@ final case class JWTService(config: AppConfig) {
ZIO.succeed {
JWTHelper.createToken(
userIri = id,
secret = config.jwtSecretKey,
longevity = config.jwtLongevityAsDuration,
issuer = config.knoraApi.externalKnoraApiHostPort,
secret = secret,
longevity = longevity,
issuer = issuer,
content = content
)
}
Expand All @@ -33,6 +35,6 @@ object JWTService {
ZLayer {
for {
config <- ZIO.service[AppConfig]
} yield JWTService(config)
} yield JWTService(config.jwtSecretKey, config.jwtLongevityAsDuration, config.knoraApi.externalKnoraApiHostPort)
}
}