Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Introduce ZIO HTTP (DEV-1425) #2256

Merged
merged 54 commits into from Nov 28, 2022
Merged
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
59a320e
use ZIO HTTP for user slice
Oct 18, 2022
9b6c2cf
add validation of payload
Oct 18, 2022
28056b2
wip
Oct 19, 2022
d2813c0
fmt
Oct 19, 2022
edf154e
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Oct 19, 2022
8d19b1e
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Oct 25, 2022
b25fd4d
reformat code
Oct 25, 2022
7fc8b41
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Oct 25, 2022
5e162a6
Delete MainApp.scala
Oct 25, 2022
b472d2c
add logging binding to dependencies
Oct 25, 2022
31bef4c
reduce health route logs
Oct 25, 2022
e102fb5
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Nov 14, 2022
328bf5f
fix bad request value
Nov 14, 2022
b96a6b7
update ZIO HTTP
Nov 14, 2022
f7550c7
use AppConfig
Nov 14, 2022
514a1fe
return accumulated errors
Nov 14, 2022
9669797
add new logback.xml with JSON
Nov 16, 2022
5c446a5
add appConfig as layer
Nov 16, 2022
47c361d
use PasswordStrength as type in config
Nov 16, 2022
31c3fce
improve logging
Nov 16, 2022
8dcee06
add zio http to webapi project
Nov 18, 2022
2edeccf
add accumulating route
Nov 18, 2022
8ab4ba8
improve code quality
Nov 18, 2022
22fff81
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Nov 18, 2022
0b90c95
clean up code
Nov 18, 2022
6f9e154
Merge branch 'wip/dev-1425-introduce-zio-http' of https://github.com/…
Nov 18, 2022
6adb76f
Update webapi/src/main/scala/org/knora/webapi/routing/HealthRouteWith…
Nov 18, 2022
cf12080
implement reviewer's feedback
Nov 18, 2022
fc9ef5b
Merge branch 'wip/dev-1425-introduce-zio-http' of https://github.com/…
Nov 18, 2022
d179a1b
format code
Nov 21, 2022
b45b696
unify definition of classes
Nov 21, 2022
7bbe080
introduce UuidGenerator
Nov 22, 2022
260a329
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Nov 22, 2022
083f05c
fmt
Nov 22, 2022
f737b93
Merge branch 'wip/dev-1425-introduce-zio-http' of https://github.com/…
Nov 22, 2022
8e3b26a
add CreateUserSpec
Nov 22, 2022
21b084f
add more tests for the UserRoute
Nov 23, 2022
8ce9f40
complete test impl of uuidgenerator
Nov 23, 2022
c8bd535
fmt
Nov 23, 2022
2a15118
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Nov 23, 2022
d4f533d
update docstrings in UuidGenerator
Nov 24, 2022
4252579
Merge branch 'wip/dev-1425-introduce-zio-http' of https://github.com/…
Nov 24, 2022
049de2f
make codacy happy
Nov 24, 2022
d73f695
fix file name
Nov 24, 2022
d43164b
Revert "fix file name"
Nov 24, 2022
8da7285
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Nov 24, 2022
5bdf0bb
implement reviewer's feedback
Nov 24, 2022
51279a1
implement reviewer's feedback
Nov 24, 2022
58cc651
simplify codecs
Nov 28, 2022
191e7db
move UuidGeneratorMock to test package
Nov 28, 2022
2939711
initialize the UuidGeneratorMock with 20 random UUIDs
Nov 28, 2022
4046c36
Merge branch 'main' into wip/dev-1425-introduce-zio-http
Nov 28, 2022
7256a4e
fmt
Nov 28, 2022
5cacb3f
fmt
Nov 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 13 additions & 1 deletion build.sbt
Expand Up @@ -261,6 +261,18 @@ lazy val webapiJavaTestOptions = Seq(
// DSP's new codebase
//////////////////////////////////////

// dsp-api-main project

lazy val dspApiMain = project
.in(file("dsp-api-main"))
.settings(
scalacOptions ++= customScalacOptions,
name := "dspApiMain",
libraryDependencies ++= Dependencies.dspApiMainLibraryDependencies,
testFrameworks := Seq(new TestFramework("zio.test.sbt.ZTestFramework"))
)
.dependsOn(userInterface, userHandler, userRepo)

// Role projects

lazy val roleInterface = project
Expand Down Expand Up @@ -317,7 +329,7 @@ lazy val userInterface = project
libraryDependencies ++= Dependencies.userInterfaceLibraryDependencies,
testFrameworks := Seq(new TestFramework("zio.test.sbt.ZTestFramework"))
)
.dependsOn(shared, userHandler)
.dependsOn(shared, userHandler, userRepo % "test->test")

lazy val userHandler = project
.in(file("dsp-user/handler"))
Expand Down
34 changes: 34 additions & 0 deletions dsp-api-main/src/main/resources/logback.xml
@@ -0,0 +1,34 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>

<configuration >
<import class="ch.qos.logback.core.ConsoleAppender"/>
<import class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"/>
<import class="ch.qos.logback.contrib.json.classic.JsonLayout"/>
<import class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter"/>

<appender name="STDOUT" class="ConsoleAppender">
<encoder class="PatternLayoutEncoder">
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg [%mdc]%n</pattern>
</encoder>
</appender>

<appender name="JSON" class="ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="JsonLayout">
<jsonFormatter class="JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
<timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
</layout>
</encoder>
</appender>

<root level="INFO">
<appender-ref ref="STDOUT"/>
<!-- <appender-ref ref="JSON"/> -->
</root>

<!-- for logging during tests, please see/use logback-test.xml under test/resources -->

</configuration>
41 changes: 41 additions & 0 deletions dsp-api-main/src/main/scala/dsp/api/main/DspMain.scala
@@ -0,0 +1,41 @@
package dsp.api.main

import zio._
import dsp.user.route.UserRoutes
import dsp.user.handler.UserHandler
import dsp.user.repo.impl.UserRepoLive
import zio.logging.removeDefaultLoggers
import zio.metrics.connectors.MetricsConfig
import zio.logging.backend.SLF4J
import dsp.config.AppConfig
import dsp.util.UuidGeneratorLive

object DspMain extends ZIOAppDefault {

/**
* Configures Metrics to be run at a set interval, in this case every 5 seconds
*/
val metricsConfig = ZLayer.succeed(MetricsConfig(5.seconds))
irinaschubert marked this conversation as resolved.
Show resolved Hide resolved
override val run: Task[Unit] =
ZIO
.serviceWithZIO[DspServer](_.start)
.provide(
// ZLayer.Debug.mermaid,
// server
DspServer.layer,
// configuration
AppConfig.live,
// routes
UserRoutes.layer,
// handlers
UserHandler.layer,
// repositories
UserRepoLive.layer,
// slf4j facade, we use it with logback.xml
removeDefaultLoggers,
SLF4J.slf4j,
// metricsConfig
UuidGeneratorLive.layer
)

}
24 changes: 24 additions & 0 deletions dsp-api-main/src/main/scala/dsp/api/main/DspMiddleware.scala
@@ -0,0 +1,24 @@
package dsp.api.main

import zhttp.http.middleware.HttpMiddleware
import zhttp.http._
import zio._

object DspMiddleware {
// adds a requestId to all logs that were triggered by the same request
val logging: HttpMiddleware[Any, Nothing] =
new HttpMiddleware[Any, Nothing] {
override def apply[R1 <: Any, E1 >: Nothing](
http: HttpApp[R1, E1]
): HttpApp[R1, E1] =
Http.fromOptionFunction[Request] { request =>
Random.nextUUID.flatMap { requestId =>
ZIO.logAnnotate("RequestId", requestId.toString) {
for {
result <- http(request)
} yield result
}
}
irinaschubert marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
32 changes: 32 additions & 0 deletions dsp-api-main/src/main/scala/dsp/api/main/DspServer.scala
@@ -0,0 +1,32 @@
package dsp.api.main

import dsp.user.route.UserRoutes
import zio._
import zhttp.http._
import zio.ZLayer
import zhttp.service.Server
import dsp.api.main.DspMiddleware
import dsp.config.AppConfig
import dsp.util.UuidGeneratorLive

final case class DspServer(
appConfig: AppConfig,
userRoutes: UserRoutes
) {

// adds up the routes of all slices
val dspRoutes: HttpApp[AppConfig & UuidGeneratorLive, Throwable] =
userRoutes.routes // ++ projectRoutes.routes

// starts the server with the provided settings from the appConfig
def start = {
val port = appConfig.dspApi.externalPort
Server.start(port, dspRoutes @@ DspMiddleware.logging)
}

}

object DspServer {
val layer: ZLayer[AppConfig & UserRoutes, Nothing, DspServer] =
ZLayer.fromFunction(DspServer.apply _)
}
18 changes: 0 additions & 18 deletions dsp-api-main/src/main/scala/dsp/api/main/MainApp.scala

This file was deleted.

Expand Up @@ -154,9 +154,6 @@ final case class ProjectHandler(repo: ProjectRepo) {

}

/**
* Companion object providing the layer with an initialized implementation
*/
object ProjectHandler {
val layer: ZLayer[ProjectRepo, Nothing, ProjectHandler] =
ZLayer {
Expand Down
Expand Up @@ -7,6 +7,8 @@ package dsp.role.domain

import zio.test._

import java.util.UUID

import dsp.role.sharedtestdata.RoleTestData
import dsp.valueobjects.Id
import dsp.valueobjects.Permission
Expand Down Expand Up @@ -86,7 +88,7 @@ object RoleDomainSpec extends ZIOSpecDefault {
(
for {
role <- RoleTestData.role1
newValue = List(RoleUser(Id.UserId.make().fold(e => throw e.head, v => v)))
newValue = List(RoleUser(Id.UserId.make(UUID.randomUUID()).fold(e => throw e.head, v => v)))
updatedRole <- role.updateUsers(newValue)
} yield assertTrue(updatedRole.name == role.name) &&
assertTrue(updatedRole.description == role.description) &&
Expand Down
Expand Up @@ -5,6 +5,8 @@

package dsp.role.sharedtestdata

import java.util.UUID

import dsp.role.domain.Role
import dsp.role.domain.RoleUser
import dsp.valueobjects.Id
Expand All @@ -15,10 +17,13 @@ import dsp.valueobjects.Role._
* Contains shared role test data.
*/
object RoleTestData {
val uuid1 = UUID.randomUUID()
val uuid2 = UUID.randomUUID()

val id1 = Id.RoleId.make()
val name1 = LangString.make("Name", "en")
val description1 = LangString.make("Description", "en")
val users1 = List(RoleUser(Id.UserId.make().fold(e => throw e.head, v => v)))
val users1 = List(RoleUser(Id.UserId.make(uuid1).fold(e => throw e.head, v => v)))
val permission1 = Permission.make(Permission.View)

val role1 = for {
Expand All @@ -39,7 +44,7 @@ object RoleTestData {
val id2 = Id.RoleId.make()
val name2 = LangString.make("Name 2", "en")
val description2 = LangString.make("Description 2", "en")
val users2 = List(RoleUser(Id.UserId.make().fold(e => throw e.head, v => v)))
val users2 = List(RoleUser(Id.UserId.make(uuid2).fold(e => throw e.head, v => v)))
val permission2 = Permission.make(Permission.Admin)

val role2 = for {
Expand Down
Expand Up @@ -135,9 +135,6 @@ final case class RoleHandler(repo: RoleRepo) {
} yield id).tap(_ => ZIO.logInfo(s"Deleted role with ID: $id"))
}

/**
* Companion object providing the layer with an initialized implementation
*/
object RoleHandler {
val layer: ZLayer[RoleRepo, Nothing, RoleHandler] =
ZLayer {
Expand Down
Expand Up @@ -60,9 +60,6 @@ final case class RoleRepoLive(
} yield id).commit.tap(_ => ZIO.logInfo(s"Deleted role: ${id.uuid}"))
}

/**
* Companion object providing the layer with an initialized implementation of [[RoleRepo]]
*/
object RoleRepoLive {
val layer: ZLayer[Any, Nothing, RoleRepo] =
ZLayer {
Expand Down
Expand Up @@ -60,9 +60,6 @@ final case class RoleRepoMock(
} yield id).commit.tap(_ => ZIO.logInfo(s"Deleted role: ${id.uuid}"))
}

/**
* Companion object providing the layer with an initialized implementation of [[RoleRepo]]
*/
object RoleRepoMock {
val layer: ZLayer[Any, Nothing, RoleRepo] =
ZLayer {
Expand Down
19 changes: 19 additions & 0 deletions dsp-shared/src/main/resources/application.conf
@@ -0,0 +1,19 @@
app {
dsp-api {
// relevant for direct communication inside the dsp stack
internal-host = "0.0.0.0"
internal-host = ${?DSP_API_INTERNAL_HOST}
internal-port = 4444
internal-port = ${?DSP_API_INTERNAL_PORT}

// relevant for the client, i.e. browser
external-protocol = "http" // optional ssl termination needs to be done by the proxy
external-protocol = ${?DSP_API_EXTERNAL_PROTOCOL}
external-host = "0.0.0.0"
external-host = ${?DSP_API_EXTERNAL_HOST}
external-port = 4444
external-port = ${?DSP_API_EXTERNAL_PORT}
}

bcrypt-password-strength = 12
}
46 changes: 46 additions & 0 deletions dsp-shared/src/main/scala/dsp/config/AppConfig.scala
@@ -0,0 +1,46 @@
package dsp.config

import com.typesafe.config.ConfigFactory
import zio._
import zio.config._
import zio.config.magnolia.descriptor
import zio.config.typesafe.TypesafeConfigSource

import dsp.valueobjects.User._

/**
* Configuration
*/
final case class AppConfig(
dspApi: DspApi,
bcryptPasswordStrength: PasswordStrength
)

final case class DspApi(
internalHost: String,
internalPort: Int,
externalHost: String,
externalPort: Int
)

object AppConfig {

/**
* Reads in the applicaton configuration using ZIO-Config. ZIO-Config is capable of loading
* the Typesafe-Config format. Reads the 'app' configuration from 'application.conf'.
*/
private val source: ConfigSource =
TypesafeConfigSource.fromTypesafeConfig(ZIO.attempt(ConfigFactory.load().getConfig("app").resolve))

/**
* Instantiates the config class hierarchy using the data from the 'app' configuration from 'application.conf'.
*/
private val configFromSource: IO[ReadError[String], AppConfig] = read(
descriptor[AppConfig].mapKey(toKebabCase) from source
)

/**
* Application configuration from application.conf
*/
val live: ULayer[AppConfig] = ZLayer(configFromSource.orDie)
}