Skip to content

non-blocking mode, better monitoring

Compare
Choose a tag to compare
@Dieterbe Dieterbe released this 14 Sep 09:34
· 617 commits to master since this release
  • make blocking behavior configurable for kafkaMdm and grafanaNet routes. previously, grafanaNet was blocking, kafkaMdm was non-blocking. Now both default to non-blocking, but you can specify blocking=true.
    • nonblocking (default): when the route's buffer fills up, data will be discarded for that route, but everything else (e.g. other routes) will be unaffected. If you set your buffers large enough this won't be an issue.
      rule of thumb: rate in metrics/s times how many seconds you want to be able to buffer in case of downstream issues. memory used will be bufSize * 100B or use bufSize * 150B to be extra safe.
    • blocking: when the route's buffer fills up, ingestion into the route will slow down/block, providing backpressure to the clients, and also blocking other routes from making progress. use this only if you know what you're doing and have smart clients that can gracefully handle the backpressure
  • monitor queue drops for non-blocking queues
  • document route options better
  • monitor queue size and ram used #218
  • preliminary support for parsing out the new graphite tag format (kafkaMdm and grafanaNet route only)

the included, dashboard is updated accordingly. and also on https://grafana.com/dashboards/338