Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow tests to run for an infinite duration #2701

Open
rts-rob opened this issue Sep 29, 2022 · 2 comments
Open

Allow tests to run for an infinite duration #2701

rts-rob opened this issue Sep 29, 2022 · 2 comments
Labels
docs evaluation needed proposal needs to be validated or tested before fully implementing it in k6 feature

Comments

@rts-rob
Copy link

rts-rob commented Sep 29, 2022

Feature Description

We would like to run tests for an infinite duration, i.e., until we manually kill them or kill them as part of a new deployment.

The docs say, under Performance and synthetic monitoring:

With k6, you could run tests with a small amount of load to continuously validate the performance and availability of your production environment.

We would like to do exactly this - use k6 to send a steady stream of traffic through several synthetics. However, I can’t find anything around how to actually configure an infinite duration in the docs, in the code, in the forums, or here on GitHub.

This is related to this forum post (I am the other of the post and this feature request).

Suggested Solution (optional)

Currently the config JavaScript object requires a duration for each stage. For example, the following configuration runs a single stage ramping up to 500 VUs over thirty seconds:

export const options = {
  stages: [
    { duration: '30s', target: 500 },
  ],
};

Proposal 1 - Allow an infinite duration value

Currently the values for each stage MUST contain a positive time value for duration. If we allow the user to specify a special string such as 'infinite', the test can run until it is manually stopped. For example, the following configuration ramps up smoothly to 500 VUs over thirty seconds then runs 500 VUs indefinitely until the process is manually stopped:

export const options = {
  stages: [
    { duration: '30s', target: 500 },
    { duration: 'infinite', target: 500 },
  ],
};

Proposal 2 - Allow an unspecified duration

Currently the values for each stage MUST contain a duration. By removing this requirement, the user could omit the duration, causing the test to run until it is manually stopped. For example, the same configuration from Proposal 1 could be specified with:

export const options = {
  stages: [
    { duration: '30s', target: 500 },
    { target: 500 },
  ],
};

All else equal I would favor the first proposal as it requires the user to perform an action to achieve an indefinitely running process. In cases where the user forgets to specify the duration, she continues to receive an error message. The error message could be updated to include something like:

"If you intended to run this test without termination, specify duration: 'infinite'."

Checks could also be added to ensure no additional stages are defined following an infinite stage.

Note: Allowing duration: '0' was considered but not proposed as this runs the given scenario exactly one time. If this behavior is relied upon in existing tests, changing it would be a breaking change.


In either case, accumulation and presentation of statistics will need to be considered. This is not a problem for users who ship results to an external datastore such as Prometheus. Other user requirements will vary.

Already existing or connected issues / PRs (optional)

I searched open and closed issues but could not identify any related feature requests.

@na-- na-- added the evaluation needed proposal needs to be validated or tested before fully implementing it in k6 label Oct 3, 2022
@na--
Copy link
Member

na-- commented Oct 3, 2022

The externally-controlled executor actually allows for infinite duration. I see that's not very well documented in the docs, but these are the relevant parts of the code:

Duration types.NullDuration `json:"duration"` // 0 is a valid value, meaning infinite duration

if maxDuration == 0 {
// Infinite duration, don't emit 0 VUs at the end since there's no planned end
return []lib.ExecutionStep{startVUs}
}

if duration > 0 { // Only keep track of duration if it's not infinite
go mex.stopWhenDurationIsReached(ctx, duration, cancel)
}

Theoretically, support for infinite duration could be added to the other executors too, but I am not sure the added complexity will be worth it 🤔 If anyone else has the same or a similar issue to @rts-rob, please 👍 the issue and share details for your specific use case 🙏

@na-- na-- added the docs label Oct 3, 2022
@mstoykov
Copy link
Collaborator

mstoykov commented Oct 4, 2022

I would argue that you can just let infinite = "1000000h" and use that as your infinite. This "random" number I wrote is >114 years.

I guess we can agree you won't be running a k6 process for over 114 years ;).

I would also recommend in those cases to add --no-summary --no-thresholds as currently this will keep accumulating memory for Trends (#763) among other problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs evaluation needed proposal needs to be validated or tested before fully implementing it in k6 feature
Projects
None yet
Development

No branches or pull requests

3 participants