Alerting: Elasticsearch support

[torkelo]

Port frontend data source code to backend & Go. Includes building ES query from internal query model, and parsing ES response into time series.

Grafana internal query model example:

Elastic query json model: 
{
  "bucketAggs": [
    {
      "field": "@timestamp",
      "id": "2",
      "settings": {
        "interval": "auto",
        "min_doc_count": 0,
        "trimEdges": 0
      },
      "type": "date_histogram"
    }
  ],
  "dsType": "elasticsearch",
  "metrics": [
    {
      "field": "@value",
      "id": "1",
      "meta": {},
      "settings": {},
      "type": "avg"
    }
  ],
  "refId": "A",
  "target": "",
  "timeField": "@timestamp"
}

[bergquist]

ref #5858

[luigiberrettini]

Since #5948 is merged is this issue closed?

[bergquist]

@luigiberrettini its not merged to master just to the alerting-elasticsearch branch. It far from completed yet.

[DEvil0000]

not more far then graphite support ;)
missing graphite query features in alerting:

• template variables
• functions referencing other queries (e.g. asPercent)

[ictcorebiz]

Hi, will complete elasticsearch support come after the 4.0.0 deliverable (planned for November), because I see this issue marked with the 4.1.0 milestone?
If so: when is 4.1.0 planned for delivery?

[torkelo]

4.1 does not have a release date yet, but would guess January / early feb.

Elasticsearch alerting might not make it into Grafana v4, we tried (the work is started), but requires a lot more work to complete and some more high prio issues has pushed it out of v4

[ictcorebiz]

In that case, would it be possible to enable/disable the alert-menu items and panel-tabs based on an Organization preference (or even better: on individual user level definitions within the organization)?

[warroyo]

Has there been any progress on this? Is the elasticsearch-alerting branch still active? I am wondering if there is something that I could start testing and giving feedback on.

[xamox]

Looking forward to this getting into 4.2 so we can fully move to grafana + elasticsearch (metricbeat).

[andytsnowden]

Having this functionality would be a huge win for my company

[torkelo]

@andytsnowden enough to buy a support plan? https://grafana.net/support/plans 😜

We hope get a chance to continue work on this soon.

[gbrian]

Maybe silly (please don't answer "yes, you are") but what about kind of feature-crowdfunding ? sure many will support

[wirecutter313]

This was the whole reason I downloaded this tool, in hopes of having alerting for elastic.

[DEvil0000]

There is a branch which was at least working for most queries. But it is a bit outdated - you would need to align some things. Von meinem Samsung Gerät gesendet.

…

-------- Ursprüngliche Nachricht -------- Von: wirecutter313 <notifications@github.com> Datum: 30.01.2017 20:56 (GMT+01:00) An: grafana/grafana <grafana@noreply.github.com> Cc: "A. Binzxxxxxx" <alexander@binzberger.de>, Manual <manual@noreply.github.com> Betreff: Re: [grafana/grafana] Alerting: Elasticsearch support (#5893) This was the whole reason I downloaded this tool, in hopes of having alerting for elastic. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/grafana/grafana","title":"grafana/grafana","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/grafana/grafana"}},"updates":{"snippets":[{"icon":"PERSON","message":"@wirecutter313 in #5893: This was the whole reason I downloaded this tool, in hopes of having alerting for elastic. "}],"action":{"name":"View Issue","url":"#5893 (comment)"}}}

[yannispanousis]

Is there a strong chance this is going to make the 4.2.0 mark? Trying to come up with temporary alternatives for the alerting part while we wait for this.

[torkelo]

its not looking like that. No one is working on it that we know and we have a lot on our plate right now.

[mizalewski]

@nikskiz Grafana is open source project, and everyone can contribute to it. Also you can contribute!

Many people work on open source projects for free in their free time, and I think it is not ok to be rude and criticize their work.

[javasoze]

There is a smaller PR in progress: wph95#2 by @wph95

[4ndv]

@nikskiz right now you can use influxdb

[kvalev]

@nikskiz or you can simply use elastalert

[lucasrodcosta]

Hi everyone!

We all know that alerting for Elasticsearch is an amazing, very-welcome and long-waited feature.
But I think we should keep this thread as clean and organized as possible.

So, questions and comments like these...

Any timeline?
It would be very useful!
Can't wait for it!
Has there been any progress on this?

doesn't help in evolution of the feature, generates a lot of useless emails to people who subscribed to the thread and makes the discussion too much confusing to new participants.

Even questions about workarounds and other alerting solutions (like ElastAlert) shouldn't been posted here anymore, as it has already discussed a lot in previous comments.

So, please, stop trivializing this space! Let's use them to spread useful comments and to share the progress you've made on this feature.

[wph95]

I just finished a PR #11380 to make grafana support Alerting [Elasticsearch]

Compared to some early implementations e.g. #8943 , #10343

• base on grafana-5.0.0
• source code less than 1000 lines (and a third lib [leibowitz/moment]~=1000 line, and some test code)
• Code logic is consistent with the front

this week i will continue to improve this PR, more comprehensive test, clearer abstract model.
I'm looking forward to someone who can try to use this PR and improve it :)

[harshvladha]

@wph95 - I will be trying your PR this week for sure, will create issue in your repo, if that's ok with you, and look into contributing there.

[Guchman]

Merge that already please. Thats a hot feature.

[yossiv]

anything we can help in order to promote this ?

[i033653]

What is the estimation to merge it? Shall I wait or use X-Pack?

[jockjiang]

+1, What is the estimation to merge it? 5.2.x?

[yossiv]

Hi i know we pushing you too much on this , but this issue was open almost 2 years ago (Aug2016) .
elasticsearch + Grafana commutiy is wide and big like others i guess.
i really appreciate the great job this community doing for us, but seems like this issue stayed behind :(
is there anything we can do in order to promote this?

[demetriusnunes]

Why is it taking so long to merge the PR?

[enigy]

+1

[futurpc]

+1

[petrochen]

+1

[beriba]

Stop writing comments with only "+1" because you're spamming other people. You're not adding anything to the topic. Click thumbs up button on the main post or click Subscribe button instead.

[guohailong2006]

+1

[Trainxy]

+1

[ldktta]

🎉 🎉 🎉 💖

[mvlach]

🎉

[xamox]

Wow, it happened. I honestly thought this was going to go the way of duke nukem. 👍

[dev-e]

Thanks a lot for your efforts!