Abstract Rest Service Benchmark

Requirements

• PORT: 8080

• API request: /api/test

• API response: Hello, World! (text/plain, UTF-8, 13 bytes)

Benchmark

Command for WRK:

$ wrk -t8 -c512 -d2m --timeout 10s --latency http://localhost:8080/api/test

---

Results

#	Service	Language	Framework	RPS
	Java Light 4J	Java	oracle jdk 9.0.4 + ligth4j	401511.94
	.Net Core	C#	dotnet 2.1.4	249758.46
	Haskell Warp	Haskell	ghc 7.10.3 + warp	191020.62
	Rust Iron	Rust	rust 1.24.1 + iron	186726.17
	Go Gorilla/Mux	Golang	go sdk 1.9.4 + gorilla/mux	131499.79
	.Net Core ASP.NET MVC	C#	dotnet 2.1.4	99091.85
	NodeJS	JavaScript	nodejs 8.9.4	94132.92
	Python Sanic	Python	python 3.5.2 + sanic	74854.41
	NodeJS Express	JavaScript	nodejs 8.9.4 + express	58769.29
	Java Spring Boot	Java	oracle jdk 9.0.4 + spring boot	56253.12
	.Net Core Freya	F#	dotnet 2.1.4 + freya-hopac	49736.08
	Python Aiohttp	Python	python 3.5.2 + aiohttp	4603.01
	Python Flask	Python	python 3.5.2 + flask	200.44
	.Net Core Websocket	C#	dotnet 2.1.4 + websocket	TBD
	.Net Core Giraffe	F#	dotnet 2.1.4 + giraffe	TBD
	Python API Star	Python	python 3.5.2 + apistar	TBD
	Rust Rocket	Rust	rust 1.24.1 + rocket	TBD

---

Services

.Net Core by @vanbukin

Language: C#

Framework: .Net Core 2

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./dotNetCoreRestService/bin/Release/netcoreapp2.1/publish
$ dotnet dotNetCoreRestService.dll

Result:

TBD

---

.Net Core ASP.NET MVC by @binakot

Language: C#

Framework: .Net Core 2, ASP.NET MVC

Main tutorial: https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./dotNetCoreRestService/bin/Release/netcoreapp2.0/publish
$ dotnet dotNetCoreRestService.dll

Result:

TBD

---

.Net Core Websocket by @petriashev

Language: C#

Framework: .Net Core 2, Websocket

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-websocket.dll

Result:

TBD

---

.Net Core Freya by @Szer

Language: F#

Framework: .Net Core 2, Freya

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-hopac-freya.dll

Result:

TBD

---

.Net Core Giraffe by @Szer

Language: F#

Framework: .Net Core 2, Giraffe

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-giraffe.dll

Result:

TBD

---

Go Gorilla/Mux by @binakot

Language: Go

Framework: Go SDK, Gorilla/Mux

Main tutorial: https://www.codementor.io/codehakase/building-a-restful-api-with-golang-a6yivzqdo

Run:

$ go build
$ ./go

Result:

TBD

---

Haskell Warp by @kchugalinskiy

Language: Haskell

Framework: GHC, Warp

Main tutorial: http://taylor.fausak.me/2014/10/21/building-a-json-rest-api-in-haskell

Requirements:

• Install Haskell stack: https://docs.haskellstack.org/en/stable/README
• Install Cabal package: cabal install Cabal-2.0.1.1

Run:

$ stack setup
$ stack build
$ stack exec .stack-work/dist/**/build/test-exe/test-exe

Result:

TBD

---

Java

Language: Java

Framework: Oracle JDK 8 or above

Run:

$ gradle clean run

Result:

TBD

---

Java Light 4J by @0ffer

Language: Java

Framework: Oracle JDK 9, Light4J

Main tutorial: https://github.com/networknt/light-example-4j/tree/master/demo

Run:

$ mvn clean install
$ java -jar target/service-example-0.1.0.jar

Result:

TBD

---

Java Spring Boot by @binakot

Language: Java

Framework: Oracle JDK 9, Spring Boot

Main tutorial: http://spring.io/guides/gs/rest-service

Run:

$ gradle clean build
$ java -jar build/libs/java-spring-boot-rest-service-1.0-SNAPSHOT.jar

Result:

TBD

---

NodeJS by @WilixLead

Language: JavaScript

Framework: Node.js

Main tutorial: https://nodejs.org/api/http.html#http_class_http_server

Run:

$ node ./index.js

Result:

TBD

---

NodeJS Express by @WilixLead

Language: JavaScript

Framework: Node.js, Express

Main tutorial: http://expressjs.com/en/starter/hello-world.html

Run:

$ npm i
$ node ./index.js

Result:

TBD

---

Python Aiohttp by @TyVik

Language: Python

Framework: Python 3, Aiohttp

Main tutorial: https://aiohttp.readthedocs.io/en/stable/

Run:

$ pip install -r requirements.txt
$ python app.py

Result:

TBD

---

Python API Star by @sergeibershadsky

Language: Python

Framework: Python 3, API Star

Main tutorial: https://github.com/encode/apistar

Run:

pip install -r requirements.txt
uvicorn app:app --workers=8 --bind=0.0.0.0:8080 --pid=pid

Results:

TBD

---

Python Flask by @TyVik

Language: Python

Framework: Python 3, Flask

Main tutorial: http://flask.pocoo.org/

Run:

$ pip install -r requirements.txt
$ python app.py

Result:

TBD

---

Python Sanic by @TyVik

Language: Python

Framework: Python 3, Sanic

Main tutorial: http://sanic.readthedocs.io/en/latest/

Run:

$ pip install -r requirements.txt
$ python app.py

Results:

TBD

---

Rust Iron by @kchugalinskiy

Language: Rust

Framework: Rust SDK, Iron

Main tutorial: https://github.com/iron/router/blob/master/examples/simple.rs

Run:

$ cargo run --release

Result:

TBD

---

Rust Rocket by @qezz

Language: Rust

Framework: Rust SDK, Rocket

Main tutorial: https://rocket.rs/guide/getting-started/#hello-world

Run:

$ ROCKET_ENV=prod cargo run --release

Result:

TBD

---

Hardware

• Server (rest-service) host:

TBD

• Client (wrk) host:

TBD