v1.0

• grafanaNet route: add errBackoffMin and errBackoffFactor controls to control backoff retry timing. #465
• grafanaNet route: automatically publish stripped (from comments) storage-aggregation.conf, similar to storage-schemas.conf
  This enables an upcoming new feature which allows customizing the configs on Grafana Cloud Graphite V5. #466
  IMPORTANT you must set the aggregationFile parameter for grafanaNet routes (pointing to your storage-aggregation.conf), similar to the existing schemasFile option (which points to your storage-schemas.conf)
• FreeBSD support. #468

build artifacts: https://app.circleci.com/pipelines/github/grafana/carbon-relay-ng/195/workflows/43170e91-7689-4480-889f-437f9f34e918/jobs/2209/artifacts

v0.14.0

Important:
Docker images are no longer being pushed to raintank/carbon-relay-ng, only to grafana/carbon-relay-ng
See https://hub.docker.com/r/grafana/carbon-relay-ng for all Docker image updates.

• Deprecate (remove) pushing to "raintank" docker hub account. #449
• Kafkamdm: make sasl mechanism configurable. #447
• Fix panels on dashboard. #387
• Fix process reporter (/proc) to behave properly in windows, freeBSD. #450
• GrafanaNet route: periodically post the schemas parsed from storage-schemas.conf (minus file comments) #458
  (this enables an upcoming new feature which allows customizing the schemas on Grafana Cloud Graphite)
• Upgrade Sarama, our kafka library from v1.19.0 to v1.23.0 #445
• add bySeriesWithTagsFnv partitioner for kafka-mdm route #460

Packages and builds can be downloaded from CircleCI

inverted filters, kafka TLS, and more

• kafkaMdm route: allow partitionBy bySeriesWithTags. #381
• Aggregation point tracker (more detailed stats for aggregator). #382
• fix: Add error check for process reporter so we don't auto-crash for osx/windows. #390
• Add support for passing grafana.net credentials via environment variables. #393
• Add a flag to enable /debug/pprof/ on admin http port. #402
• Inverted filter criteria for matchers. #422
• Document k8s deployment. #427, #428
• substring filters have been renamed from "substr" to "sub" and "notSubstr" to "notSub" everywhere, for consistency.
  change is backwards compatible, the old "substr" parameters are still accepted in all places where they've been accepted before. #419, #423
• kafkaMdm route: TLS + SASL support. #426

better pickle support, various fixes and better aggregators

• support pickle protocol versions 0, 1, 2, 3 & 4 + accept pickle arrays + send pickle tuples. #341
• fix different connections to same host:port using same spool file, by adding instance to destination key. #349
• fix aggregator flush timing + limit concurrent aggregator flushing #354
• support new tsdbgw response with detailed info about invalid metrics #338
• add 'count' aggregator. #355
• Some aggregator improvements and better stats. #361, specifically:
  • assure aggregation flushing is in timestamp asc order
  • new stat for number of metrics going into aggregator: service_is_carbon-relay-ng.instance_is_$instance.mtype_is_counter.unit_is_Metric.direction_is_in.aggregator_is_$aggregator
  • new stat for number of metrics going out of aggregator: service_is_carbon-relay-ng.instance_is_$instance.mtype_is_counter.unit_is_Metric.direction_is_out.aggregator_is_$aggregator
  • new stat for number of aggregators waiting to flush: service_is_carbon-relay-ng.instance_is_$instance.*.unit_is_aggregator.what_is_flush_waiting
  • new stats for CPU, memory and golang GC under `carbon-relay-ng.stats.$instance.*
  • give aggregators a "key" property for use in stats ($aggregator above), printed at startup and in web ui.
  • process pending aggregations more eagerly - instead of waiting - when relay is somewhat loaded.
• Fix bad regex cache clean up resulting in unbounded growth #365
• Fix deadlock in keepSafe #369
• Embed bootstrap, jquery, etc, instead of pulling in from the internet. #371

memleak fix, new logging, major input refactor and more

• BREAKING: switch to logrus for logging. #317, #326
  If you were previously using log level notice, you should now use info.
  See logging docs for more info on the log levels.
• IMPORTANT: refactor release process, docker tags and package repo. #330, #331
  see installation docs
• fix memory leak when connections should be closed. #259, #329
• Rewrite of the inputs. pickle, tcp, udp and data reading.
  They are now plugins, handle errors better, and support timeouts.
  #286, #287, #301, #303, #306, #313, #316, #318, #320
• simplify default configs, bump default stats interval, improve config docs. #276
• Make StorageResolution for CloudWatch routes configurable. #275
• re-organize docs into separate pages. #277
• track timestamps coming in to aggregators and being too old. #280
• include sysv init file in centos6/rhel6 rpm to accommodate aws images. #283
• rewriter: support "not" clause + DRY constructor. #284
• fix connection pooling for grafanaNet route. #289
• Fix sslVerify default=true for grafanaNet route using new syntax. #291
• add support for tag appendix validation. make validation inline with grafanaNet. #292
• add request logging for admin interface. #297
• make max_procs setting optional, honor GOMAXPROCS env var. #309
• switch to circleci 2.0. #294
• switch from govendor to dep. #310

aggregator optional drop raw, grafanaNet concurrent send fix, auto-windows binaries

• build windows binaries in CircleCI and save artifacts
• add support for dropping raw series consumed by an aggregator (#268, #273 )
• refactor concurrent sending for grafanaNet route. this should improve throughput and reduce buffering in some cases (#272)
• change max retry interval from 1 minute to 30 seconds, so service can restore quicker.

google pubsub, amazon cloudwatch, percentile aggregators and spool name fix (finally!)

new routes:

• google pubsub #256
• amazon cloudwatch #261, #266

aggregators:

• percentile aggregators #265

other

• fix overlapping spool names / stats / log entries when you use same endpoint tcp address multiple times. #258
  names of spool files and stats will change to include route name
• packages for debian v9/v/10/stretch #246
• grafanaNet defaults update: timeout 5s->10s, flushMaxNum 10k->5k, concurrency 10->100 #254 , #263

Release O'Time

various network setting fixes 351f807 , #243
make windows builds work #245
Add tmpfiles.d config for centos/7 #235
Use useradd to support multiple distros #233
support variable substitution in instance, and default to $HOST #236
fix blacklist #237
add support for specifying explicit prefixFilter and/or substringFilter in aggregations, which can help perf a lot. #239
show target address for GrafanaNet routes #244

massive aggregator improvements.

aggregators

• massive aggregation improvements. they now run faster (sometimes by 20x), use less memory and can handle much more load. see #227 , #230
• add derive (rate) aggregator #230
• aggregator regex cache, which lowers cpu usage and increases the max workload, in exchange for a bit more ram usage. #227 By default, the cache is enabled for aggregators set up via commands (init commands in the config) but disabled for aggregators configured via config sections (due to a limitation in our config library)

docs

• add dashboard explanation screenshot #208

packaging and versioning

• fix logging on cent6/amzn/el6 #224 43b265d
• Fix the creation of /var/run/carbon-relay-ng directory #213
• version argument to get the version

other

• disable http2.0 support for grafanaNet route, since an incompatibility with nginx was resulting in bogus 400 Bad Request responses. note this fix does not properly work, use 0.9.3-1 instead.
• track allocated memory

non-blocking mode, better monitoring

• make blocking behavior configurable for kafkaMdm and grafanaNet routes. previously, grafanaNet was blocking, kafkaMdm was non-blocking. Now both default to non-blocking, but you can specify blocking=true.
  • nonblocking (default): when the route's buffer fills up, data will be discarded for that route, but everything else (e.g. other routes) will be unaffected. If you set your buffers large enough this won't be an issue.
    rule of thumb: rate in metrics/s times how many seconds you want to be able to buffer in case of downstream issues. memory used will be bufSize * 100B or use bufSize * 150B to be extra safe.
  • blocking: when the route's buffer fills up, ingestion into the route will slow down/block, providing backpressure to the clients, and also blocking other routes from making progress. use this only if you know what you're doing and have smart clients that can gracefully handle the backpressure
• monitor queue drops for non-blocking queues
• document route options better
• monitor queue size and ram used #218
• preliminary support for parsing out the new graphite tag format (kafkaMdm and grafanaNet route only)

the included, dashboard is updated accordingly. and also on https://grafana.com/dashboards/338