Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow more flexible sub-metrics and better UX in thresholds #1313

Open
na-- opened this issue Jan 20, 2020 · 2 comments
Open

Allow more flexible sub-metrics and better UX in thresholds #1313

na-- opened this issue Jan 20, 2020 · 2 comments
Labels
enhancement evaluation needed proposal needs to be validated or tested before fully implementing it in k6 feature ux

Comments

@na--
Copy link
Member

na-- commented Jan 20, 2020

Currently, you can define thresholds on subsets of the metric data points that have specific tag values, like this:

export let options = {
    thresholds: {
        // We want the 95th percentile of request durations for HTTP request duration 
       // metrics tagged with status=200 and staticAsset=yes to be less than 500ms
        "http_req_duration{status:200,staticAsset:yes}": ["p(95)<500"],
    },
};

However, there's no way to be able to specify more complex matching queries that use something besides "equals" and "and". For example, it seems useful to be able to specify something like "the number of HTTP requests with a non-200 status" or "URLs starting with https://www.somesite.com/whatever", etc..

Needless to say, but this should probably be implemented after we refactor the thresholds to have sane parsing (#961). #1298 is something we should also consider when we decide exactly how to implement this - for example, we should be able to specify thresholds only for 403 responses that were not intended.

Additionally, we should think about refactoring how we configure thresholds. Stuffing complex matching logic in the JSON key seems to be the wrong way to go about it, even now... It may also be worth introducing some JS config-helper functions for whatever we decide, if we can provide a cleaner API that way.

@na-- na-- added enhancement ux feature evaluation needed proposal needs to be validated or tested before fully implementing it in k6 labels Jan 20, 2020
@na--
Copy link
Member Author

na-- commented Jan 20, 2020

Something else that might be worth expressing is to be able to automatically generate Rate sub-metrics on the fly from any Counter or Trend metric, based on a such a matching query. Something like this makes sense to me as a threshold: "the rate of HTTP requests to https://somesite.com/api/v1/blah/ with non-200 responses should be less than 1 percent (out of all requests to the same URL)".

Unfortunately I don't currently have even the vaguest suggestion how to configure something like this in a same manner... So, given that it (and the things above) could be expressed with some extra coding via custom metrics, it's not urgent or a must-have feature. But it would be very nice if our threshold rule engine was powerful enough that you are able to encode such rules in a declarative manner in a single place of your script...

@na--
Copy link
Member Author

na-- commented Dec 22, 2021

Another use case for this is submetrics for nested groups: #2309

@joanlopez joanlopez changed the title Allow more flexible sub-metics and better UX in thresholds Allow more flexible sub-metrics and better UX in thresholds Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement evaluation needed proposal needs to be validated or tested before fully implementing it in k6 feature ux
Projects
None yet
Development

No branches or pull requests

1 participant