Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

improve --rules.max-concurrent-evals description. #13931

Closed
machine424 opened this issue Apr 15, 2024 · 1 comment · Fixed by #14120 · May be fixed by #14022
Closed

improve --rules.max-concurrent-evals description. #13931

machine424 opened this issue Apr 15, 2024 · 1 comment · Fixed by #14120 · May be fixed by #14022
Labels
low hanging fruit

Comments

@machine424
Copy link
Collaborator

Proposal

Suggest to adjust –query.max-concurrency when changing --rules.max-concurrent-evals as the rules manager uses the same query engine than the API:

prometheus/cmd/prometheus/main.go

Lines 743 to 772 in 4a6f870

opts := promql.EngineOpts{
Logger: log.With(logger, "component", "query engine"),
Reg: prometheus.DefaultRegisterer,
MaxSamples: cfg.queryMaxSamples,
Timeout: time.Duration(cfg.queryTimeout),
ActiveQueryTracker: promql.NewActiveQueryTracker(localStoragePath, cfg.queryConcurrency, log.With(logger, "component", "activeQueryTracker")),
LookbackDelta: time.Duration(cfg.lookbackDelta),
NoStepSubqueryIntervalFn: noStepSubqueryInterval.Get,
// EnableAtModifier and EnableNegativeOffset have to be
// always on for regular PromQL as of Prometheus v2.33.
EnableAtModifier: true,
EnableNegativeOffset: true,
EnablePerStepStats: cfg.enablePerStepStats,
}
queryEngine = promql.NewEngine(opts)
ruleManager = rules.NewManager(&rules.ManagerOptions{
Appendable: fanoutStorage,
Queryable: localStorage,
QueryFunc: rules.EngineQueryFunc(queryEngine, fanoutStorage),
NotifyFunc: rules.SendAlerts(notifierManager, cfg.web.ExternalURL.String()),
Context: ctxRule,
ExternalURL: cfg.web.ExternalURL,
Registerer: prometheus.DefaultRegisterer,
Logger: log.With(logger, "component", "rule manager"),
OutageTolerance: time.Duration(cfg.outageTolerance),
ForGracePeriod: time.Duration(cfg.forGracePeriod),
ResendDelay: time.Duration(cfg.resendDelay),
MaxConcurrentEvals: cfg.maxConcurrentEvals,

Add that to --rules.max-concurrent-evals's description:

prometheus/cmd/prometheus/main.go

Line 420 in 6332248

serverOnlyFlag(a, "rules.max-concurrent-evals", "Global concurrency limit for independent rules that can run concurrently.").

Was discussed here: https://cloud-native.slack.com/archives/C167KFM6C/p1712665137550239
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
low hanging fruit
Projects
None yet
Milestone
No milestone
2 participants
@machine424 @kushalShukla-web