Meaning of numbers

[bonezinger]

What exactly do the numbers on the Datatable mean? req/s I suspect it's how many requests per second the framework can handle? Are higher numbers better? And what are the speed (concurrency?) numbers, 64, 256, 512? What's the difference between them?

[waghanza]

Hi @bonezinger,

Number displayed in datatable are computed by https://github.com/wg/wrk.

The command use is wrk -t 8 -d 15 --timeout 1 -H 'Connection: keep-alive' -c 64 http://127.0.0.1:3000/

Where :

• 64 is the concurrency level (we use 64 / 256 and 512)
• 127.0.0.1 is the hostname (framework are build and run inside a container, as of now -> will change in the future)

The results should display req/s for each framework, this only one metric (other will come)

Regards,

[waghanza]

Hi @sumonto,

Concurrency refers to simultaneous connection. The idea is this number is to test how tech stack (web framework) could scale.

Regards,

PS : Feel free to ask more questions if this is not clear for you

[sumonto]

@waghanza thank u
Is there any way you can get a comparison with Apache/Nginx pls?

Thanks

[waghanza]

This out the scope of this project. I mean apache and nginx are http servers (and not the entire stack) the idea here is to test framework (of course some are using nginx to run, since they could not run by themselves)

PS : This is private feedback, but I had not seen any big difference between this tools (depending on how they are setups) on production environments

[sharpninja]

But these tests are testing stacks, just the most minimal workload possible. So configuring a Dockerfile for each server for a few stacks that can use both is reasonable.

[waghanza]

Indeed, stacks are tested, but is their a way to benchmark in a more isolated way and reflect production usage ?