Test stats don't seem to match specified load simulation

[andreycha]

Hi,

I'm using self-hosted NBomber 5.4.1 and NBomber.Http 5.0.1 to test REST API. A test has following load simulation:

"LoadSimulationsSettings": [
  { "RampingInject": [ 800, "00:01:00", "00:01:00" ] }, // ramp up to 800 req/min within 1 minute
  { "Inject": [ 800, "00:01:00", "00:01:00" ] } // hold 800 req/min load for 1 more minute
]

So in total there should be 1600 requests over the course of 2 minutes. When I look at the reported stats there are things I don't understand.

First, why is reported total requests equal to 800?

Second, the math doesn't seem to match for RPS. 6.4 gives 384 req/min or 768 requests within 2 minutes. Which doesn't match specified load.

Third, judging by the throughput and latency charts NBomber doesn't produce any load within the first 55 seconds and then quickly ramps up to a target load within 5 seconds. Shouldn't it steadily increase the load over the whole minute?

Fourth, throughput and latency charts both start with 00:05 on the timescale (tried in Chrome, Edge, and Opera).

Could you please clarify and explain?

Thank you.

[AntyaDev]

Hi @andreycha
Let me check.

So in total there should be 1600 requests over the course of 2 minutes.

I think you can't get 1600 in total because your 1 Load Simulation(RampingInject 800) will smoothly ramp from 0 to 800, but it doesn't mean that, in total, it will be 800. It mean that at the last second 00:01:00 you will get to the rate = 800 RPS

[AntyaDev]

It's also visible from your HTML report that because of injectInterval: 1 min you had 0 request sent

[andreycha]

But I need a rate of 800 requests per minute, not 800 requests per second. Isn't it supported for ramping inject (constant load obviously works)? Or do I incorrectly understand the meaning of the second parameter?

[AntyaDev]

Yeah, that's sound like a bug :)
Let me check; it should be easy to fix.

[AntyaDev]

After reading your request carefully, I think it's not configured correctly.

But I need a rate of 800 requests per minute

This is a different use case.
{ "Inject": [ 14, "00:01:00", "00:00:01" ] } // it will send in total 840 requests

"00:01:00" - it's an injection interval, meaning how often Nbomber will wake up and send a new portion of requests.

For your use case I suggest to use
https://nbomber.com/docs/nbomber/load-simulation#iterations-for-inject

[AntyaDev]

Keep in mind that IterationsForInject - This load simulation type can't be mixed with any other simulations.
So it's more for testing on the CI/CD pipeline when you just want to send concrete requests (for example, 800 requests) for a short period and see that your system works.

[andreycha]

Thanks for the replies, Anton. After reading your answers and once again the documentation I realized I understood load simulation setup wrong. Now it's clear how it work and both RPS numbers and charts make sense.