Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New circuit breaker on context.template.max_compilations_rate trips for ingest pipelines #13650

Open
slylock opened this issue May 13, 2024 · 3 comments
Labels
bug Something isn't working ingest-pipeline

Comments

@slylock
Copy link

slylock commented May 13, 2024

Describe the bug

This circuit breaker did not exist in version 2.7. When I upgraded to 2.11, the circuit breaker started tripping for ingest pipelines that use Mustache. Unfortunately I'm unable to increase the 75/5m limit because I'm using AWS-hosted OpenSearch. Should this setting apply to ingest pipelines? If so, the limit effectively makes it impossible to use Mustache in an ingest pipeline.

Related component

Indexing

To Reproduce

Any use of Mustache in an ingest pipeline, with a indexing job of more than 75 docs, will trip the breaker. For example:
{
"processors": [
{"set": {"field": "_source.surl" , "value": "{{_source.murl}}"}},
{"set": {"field": "_source.llevel" , "value": "{{_source.schoollevel}}"}}

Expected behavior

Perhaps ingest pipeline context needs a separate setting with a much higher default limit.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@slylock slylock added bug Something isn't working untriaged labels May 13, 2024
@github-actions github-actions bot added the Indexing Indexing, Bulk Indexing and anything related to indexing label May 13, 2024
@dhwanilpatel dhwanilpatel added ingest-pipeline and removed Indexing Indexing, Bulk Indexing and anything related to indexing labels May 15, 2024
@peternied
Copy link
Member

[Triage - attendees 1 2 3 4 5 6 7 8]
@slylock Thanks for creating this issue

@peternied
Copy link
Member

@slylock You might also want to submit a support request with the AWS service as the problem could be on their end

@slylock
Copy link
Author

slylock commented May 21, 2024

I don't think this is a real bug. After I deleted and re-PUT the offending pipeline, I'm not getting the error anymore. So it seems like an anomaly of the AWS upgrade process only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ingest-pipeline
Projects
None yet
Development

No branches or pull requests

3 participants