Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: application actor (DEV-956) #2166

Merged
merged 47 commits into from Sep 9, 2022
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
2bf3c9c
refactor: zio-fying app startup
subotic Aug 10, 2022
caafc27
refactor: zio-fying app startup
subotic Aug 12, 2022
e29d247
refactor: zio-fying app startup
subotic Aug 13, 2022
0172930
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Aug 18, 2022
e3307fb
refactor: zio-fying app startup
subotic Aug 18, 2022
577497f
Merge branch 'wip/DEV-956-zio-fying-application-actor' of github.com:…
subotic Aug 18, 2022
3dfc950
refactor: zio-fying app startup
subotic Aug 20, 2022
8cbb578
refactor: zio-fying app startup
subotic Aug 20, 2022
4299729
refactor: zio-fying app startup
subotic Aug 20, 2022
267c592
refactor: zio-fying app startup
subotic Aug 21, 2022
7438d07
refactor: zio-fying app startup
subotic Aug 22, 2022
2a5cd89
refactor: zio-fying app startup
subotic Aug 23, 2022
5e9664a
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Aug 25, 2022
62f0baf
fixes after merge
subotic Aug 26, 2022
628ef82
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Aug 26, 2022
5212970
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Aug 26, 2022
0eac20e
refactoring tests
subotic Aug 26, 2022
26e8c4e
refactoring tests
subotic Aug 27, 2022
ffeae00
refactoring tests
subotic Aug 28, 2022
a128645
refactoring tests
subotic Aug 28, 2022
771d915
add structured logging to health route
subotic Aug 30, 2022
d881b60
refactoring tests
subotic Sep 1, 2022
11cdf45
refactoring tests
subotic Sep 1, 2022
d5af664
refactoring tests
subotic Sep 1, 2022
a363e63
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Sep 2, 2022
c142d89
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Sep 2, 2022
ec0a1b2
refactor: http server as layer
subotic Sep 2, 2022
f6e4369
reverting logging to SLF4J
subotic Sep 4, 2022
723f973
cleanup logging
subotic Sep 6, 2022
b8780c3
refactoring tests
subotic Sep 7, 2022
5787365
refactoring tests
subotic Sep 7, 2022
9010005
cleanup
subotic Sep 8, 2022
abf8b81
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
subotic Sep 8, 2022
bcc5dfd
Update Dependencies.scala
Sep 8, 2022
eff4737
Update Main.scala
Sep 8, 2022
e2b9289
refactor startup
subotic Sep 8, 2022
2e4a0bf
cleanup
subotic Sep 8, 2022
0401e7e
raise akka http bind timeout
subotic Sep 8, 2022
9773f46
raise akka http bind timeout
subotic Sep 8, 2022
86e5777
trying stuff out
subotic Sep 8, 2022
f77ee03
fix failing tests
Sep 9, 2022
517a8a5
fix failing tests
Sep 9, 2022
1c0de47
fix failing test
Sep 9, 2022
0ca729f
cleanup
Sep 9, 2022
46c4ba2
Merge branch 'main' into wip/DEV-956-zio-fying-application-actor
Sep 9, 2022
dff087f
cleanup
Sep 9, 2022
0e8cd7e
improve comment
Sep 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Expand Up @@ -3,6 +3,8 @@
[![Github](https://img.shields.io/github/v/tag/dasch-swiss/dsp-api?include_prereleases&label=Github%20tag)](https://github.com/dasch-swiss/dsp-api)
[![Docker](https://img.shields.io/docker/v/daschswiss/knora-api?label=Docker%20image)](https://hub.docker.com/r/daschswiss/knora-api)
[![CI](https://github.com/dasch-swiss/dsp-app/workflows/CI/badge.svg)](https://github.com/dasch-swiss/dsp-api/actions?query=workflow%3ACI)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/4c8f6736facf4e3ab6b0436c0c1ff197)](https://www.codacy.com/gh/dasch-swiss/dsp-api/dashboard?utm_source=github.com&utm_medium=referral&utm_content=dasch-swiss/dsp-api&utm_campaign=Badge_Grade)
[![Codacy Badge](https://app.codacy.com/project/badge/Coverage/4c8f6736facf4e3ab6b0436c0c1ff197)](https://www.codacy.com/gh/dasch-swiss/dsp-api/dashboard?utm_source=github.com&utm_medium=referral&utm_content=dasch-swiss/dsp-api&utm_campaign=Badge_Coverage)

[DSP](https://admin.dasch.swiss/) is a server application for storing, sharing, and working with primary sources and data in the humanities.

Expand Down
2 changes: 1 addition & 1 deletion build.sbt
Expand Up @@ -201,7 +201,7 @@ lazy val webapi: Project = Project(id = "webapi", base = file("webapi"))
// add 'config' directory to the classpath of the start script,
Universal / scriptClasspath := Seq("webapi/scripts", "knora-ontologies", "../config/") ++ scriptClasspath.value,
// need this here, so that the Manifest inside the jars has the correct main class set.
Compile / mainClass := Some("org.knora.webapi.app.Main"),
Compile / mainClass := Some("org.knora.webapi.Main"),
// add dockerCommands used to create the image
// docker:stage, docker:publishLocal, docker:publish, docker:clean
Docker / dockerRepository := Some("daschswiss"),
Expand Down
10 changes: 10 additions & 0 deletions k6/health.js
@@ -0,0 +1,10 @@
import http from "k6/http";
import { sleep } from "k6";
export const options = {
vus: 10,
duration: "30s",
};
export default function () {
http.get("http://0.0.0.0:3333/health");
sleep(1);
}
98 changes: 55 additions & 43 deletions project/Dependencies.scala
Expand Up @@ -20,28 +20,29 @@ object Dependencies {
val AkkaActorVersion = "2.6.19"
val JenaVersion = "4.6.0"

val ZioVersion = "2.0.2"
val ZioHttpVersion = "2.0.0-RC4"
val ZioJsonVersion = "0.3.0-RC11"
val ZioConfigVersion = "3.0.2"
val ZioSchemaVersion = "0.2.0"
val ZioLoggingVersion = "2.1.0"
val ZioZmxVersion = "2.0.0-RC4"
val ZioPreludeVersion = "1.0.0-RC15"
val ZioVersion = "2.0.2"
val ZioHttpVersion = "2.0.0-RC4"
val ZioJsonVersion = "0.3.0-RC11"
val ZioConfigVersion = "3.0.2"
val ZioSchemaVersion = "0.2.0"
val ZioLoggingVersion = "2.1.0"
val ZioMetricsConnectorsVersion = "2.0.0"
val ZioPreludeVersion = "1.0.0-RC15"

// ZIO - all Scala 3 compatible
val zio = "dev.zio" %% "zio" % ZioVersion
val zioMacros = "dev.zio" %% "zio-macros" % ZioVersion
val zioHttp = "io.d11" %% "zhttp" % ZioHttpVersion
val zioJson = "dev.zio" %% "zio-json" % ZioJsonVersion
val zioPrelude = "dev.zio" %% "zio-prelude" % ZioPreludeVersion
val zioLogging = "dev.zio" %% "zio-logging" % ZioLoggingVersion
val zioLoggingSlf4j = "dev.zio" %% "zio-logging-slf4j" % ZioLoggingVersion
val zioConfig = "dev.zio" %% "zio-config" % ZioConfigVersion
val zioConfigMagnolia = "dev.zio" %% "zio-config-magnolia" % ZioConfigVersion
val zioConfigTypesafe = "dev.zio" %% "zio-config-typesafe" % ZioConfigVersion
val zioTest = "dev.zio" %% "zio-test" % "2.0.2"
val zioTestSbt = "dev.zio" %% "zio-test-sbt" % "2.0.2"
val zio = "dev.zio" %% "zio" % ZioVersion
val zioMacros = "dev.zio" %% "zio-macros" % ZioVersion
val zioHttp = "io.d11" %% "zhttp" % ZioHttpVersion
val zioJson = "dev.zio" %% "zio-json" % ZioJsonVersion
val zioPrelude = "dev.zio" %% "zio-prelude" % ZioPreludeVersion
val zioLogging = "dev.zio" %% "zio-logging" % ZioLoggingVersion
val zioLoggingSlf4j = "dev.zio" %% "zio-logging-slf4j" % ZioLoggingVersion
val zioConfig = "dev.zio" %% "zio-config" % ZioConfigVersion
val zioConfigMagnolia = "dev.zio" %% "zio-config-magnolia" % ZioConfigVersion
val zioConfigTypesafe = "dev.zio" %% "zio-config-typesafe" % ZioConfigVersion
val zioTest = "dev.zio" %% "zio-test" % ZioVersion
val zioTestSbt = "dev.zio" %% "zio-test-sbt" % ZioVersion
val zioMetricsConnectors = "dev.zio" %% "zio-metrics-connectors" % ZioMetricsConnectorsVersion

// akka
val akkaActor = "com.typesafe.akka" %% "akka-actor" % AkkaActorVersion // Scala 3 compatible
Expand All @@ -55,9 +56,9 @@ object Dependencies {
val jenaText = "org.apache.jena" % "jena-text" % JenaVersion

// logging
val logbackClassic = "ch.qos.logback" % "logback-classic" % "1.2.11"
val scalaLogging = "com.typesafe.scala-logging" %% "scala-logging" % "3.9.5" // Scala 3 compatible
val slf4j = "org.slf4j" % "slf4j-simple" % "2.0.0"
val slf4jApi = "org.slf4j" % "slf4j-api" % "2.0.0" // the logging interface
val logbackClassic = "ch.qos.logback" % "logback-classic" % "1.4.0" // the logging implementation

// Metrics
val aspectjweaver = "org.aspectj" % "aspectjweaver" % "1.9.9.1"
Expand Down Expand Up @@ -115,8 +116,8 @@ object Dependencies {
akkaHttp,
akkaHttpCors,
akkaHttpSprayJson,
akkaSlf4j,
akkaHttpTestkit % Test,
akkaSlf4j % Runtime,
akkaStream,
akkaStreamTestkit % Test,
akkaTestkit % Test,
Expand All @@ -136,14 +137,15 @@ object Dependencies {
jwtSprayJson,
kamonCore,
kamonScalaFuture,
logbackClassic % Runtime,
rdf4jClient % Test,
logbackClassic,
rdf4jClient % Test,
rdf4jShacl,
saxonHE,
scalaGraph,
scalaLogging,
scalaTest % Test,
scallop,
slf4jApi,
springSecurityCore,
bouncyCastle,
swaggerAkkaHttp,
Expand All @@ -159,6 +161,7 @@ object Dependencies {
zioLogging,
zioLoggingSlf4j,
zioMacros,
zioMetricsConnectors,
zioPrelude,
zioTest % Test,
zioTestSbt % Test
Expand Down Expand Up @@ -194,70 +197,78 @@ object Dependencies {

// user projects dependencies
val userInterfaceLibraryDependencies = Seq(
slf4j % Test,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
val userHandlerLibraryDependencies = Seq(
bouncyCastle,
slf4j % Test,
springSecurityCore,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
val userCoreLibraryDependencies = Seq(
bouncyCastle,
slf4j % Test,
springSecurityCore,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
val userRepoLibraryDependencies = Seq(
slf4j % Test,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)

// role projects dependencies
val roleInterfaceLibraryDependencies = Seq(
slf4j % Test,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
val roleHandlerLibraryDependencies = Seq(
bouncyCastle,
slf4j % Test,
springSecurityCore,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
val roleCoreLibraryDependencies = Seq(
bouncyCastle,
slf4j % Test,
springSecurityCore,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
val roleRepoLibraryDependencies = Seq(
slf4j % Test,
zio,
zioMacros,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)

// shared project dependencies
Expand All @@ -267,10 +278,11 @@ object Dependencies {
commonsValidator,
gwtServlet,
scalaLogging,
slf4j % Test,
springSecurityCore,
zioPrelude,
zioTest % Test,
zioTestSbt % Test
zioTestSbt % Test,
zioLogging,
zioLoggingSlf4j
)
}
13 changes: 6 additions & 7 deletions webapi/src/main/resources/application.conf
Expand Up @@ -231,13 +231,6 @@ akka-http-cors {
}

app {

shacl {
# The directory that SHACL shapes are loaded from.
shapes-dir = "../test_data/shacl"
shapes-dir = ${?KNORA_WEBAPI_SHACLE_SHAPES_DIR}
}

print-extended-config = false // If true, an extended list of configuration parameters will be printed out at startup.
print-extended-config = ${?KNORA_WEBAPI_PRINT_EXTENDED_CONFIG}

Expand Down Expand Up @@ -484,6 +477,12 @@ app {
profile-queries = false
}

shacl {
# The directory that SHACL shapes are loaded from.
shapes-dir = "../test_data/shacl"
shapes-dir = ${?KNORA_WEBAPI_SHACLE_SHAPES_DIR}
}

cache-service {

enabled = true
Expand Down
36 changes: 14 additions & 22 deletions webapi/src/main/resources/logback.xml
@@ -1,26 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>

<!-- For assistance related to logback-translator or configuration -->
<!-- files in general, please contact the logback user mailing list -->
<!-- at http://www.qos.ch/mailman/listinfo/logback-user -->
<!-- -->
<!-- For professional support please see -->
<!-- http://www.qos.ch/shop/products/professionalSupport -->
<!-- -->
<configuration scan="false" debug="false">
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} | %-5level | %logger{0} | %msg%n</pattern>
<configuration>
<import class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"/>
<import class="ch.qos.logback.core.ConsoleAppender"/>

<appender name="STDOUT" class="ConsoleAppender">
<encoder class="PatternLayoutEncoder">
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>

<!--<conversionRule conversionWord="traceID" converterClass="kamon.instrumentation.logback.tools.LogbackTraceIDConverter" />

<appender name="STDOUTMETRICS" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} | %-5level | %traceID | %c{0} -> %m%n</pattern>
</encoder>
</appender>-->
<root level="ERROR">
<appender-ref ref="STDOUT"/>
</root>

<logger name="org.eclipse.rdf4j.rio.RDFParserRegistry" level="ERROR"/>
<logger name="org.apache.jena.riot" level="ERROR"/>
Expand All @@ -40,6 +33,7 @@

<logger name="org.knora" level="INFO"/>
<logger name="org.knora.webapi" level="INFO"/>
<logger name="org.knora.webapi.core" level="INFO"/>
<logger name="org.knora.webapi.app.ApplicationActor" level="INFO"/>
<logger name="org.knora.webapi.util.cache" level="ERROR"/>
<logger name="org.knora.webapi.util.PermissionUtilADM" level="INFO"/>
Expand Down Expand Up @@ -73,7 +67,5 @@

<!-- for logging during tests, please see/use logback-test.xml under test/resources -->

<root level="ERROR">
<appender-ref ref="stdout"/>
</root>

</configuration>
38 changes: 38 additions & 0 deletions webapi/src/main/scala/org/knora/webapi/Main.scala
@@ -0,0 +1,38 @@
/*
* Copyright © 2021 - 2022 Swiss National Data and Service Center for the Humanities and/or DaSCH Service Platform contributors.
* SPDX-License-Identifier: Apache-2.0
*/

package org.knora.webapi

import zio._
import zio.logging.backend.SLF4J

object Main extends ZIOApp {

/**
* The `Environment` that we require to exist at startup.
*/
override type Environment = core.LayersLive.DspEnvironmentLive

/**
* `Bootstrap` will ensure that everything is instantiated when the Runtime is created
* and cleaned up when the Runtime is shutdown.
*/
override val bootstrap: ZLayer[
ZIOAppArgs with Scope,
Any,
Environment
] = ZLayer.empty ++ Runtime.removeDefaultLoggers ++ SLF4J.slf4j ++ core.LayersLive.dspLayersLive

// no idea why we need that, but we do
irinaschubert marked this conversation as resolved.
Show resolved Hide resolved
override val environmentTag: EnvironmentTag[Environment] = EnvironmentTag[Environment]

/* Here we start our Application */
override val run =
for {
_ <- core.AppServer.start(true, true)
never <- ZIO.never
} yield never

}