Minutes from 2020 09 28

Attendees: Sunku, Barbara K, Borja, Dago, Elene M, Florian, Jack, Nikolay T, Pawel Z, Svetlana S, Z, Rafal, Hari TG, Thomas, Susan Coombs, Jack, Jabir, Maick, Octo

Distribution metrics - Presentation from Google interns

• Presented by Barbara K, Elene M and Svetlana S
• Presentation to be updated soon
• Presentation recording to be uploaded
• What is the impact on other read plugins
  • Ping used to calculate avg & std deviation, it was updated to include distribution metrics to find latency
  • Classical read plugins for 1 sec, distribution metrics are for aggregation -> req latency response latency can be found out
  • TODO: Everything that ingests metrics - network, exec, etc., plugin, all bindings - java plugin, python plugin
  • TODO: Add support to write plugins, into daemon
• Reference is available in csv plugin, its not necessarily that complex
  • Support distributions natively
  • Break the 1 distribution metric into 1 metric per bucket, ex. Write_prometheus plugin as it produces 1 line per metric
  • Calculate percentile -> treat distribution metric as a scalar metric
• Impact of distribution metrics on cache
  • Fixed bug
• Can we configure or are there any settings of buckets
  • Option available for curl for boundaries of buckets and number of buckets
  • Default is 100
• Looking to get this merged into 6.0 branch
• Thanks to interns for their contributions in a very short time

Collectd 6.0

• Merged main branch into 6.0 branch
• Created bugs for plugins to be ported, based on info gathered in spreadsheet
• With write_callback there is a line for updating the data structure
• Many plugins fall in the cracks
• CI needs to be updated too
• Guide is to be updated: https://collectd.org/wiki/index.php/V5_to_v6_migration_guide
  • Workflow, example code (CPU plugin or memory plugin)
  • Florian will update it
• How long will 5.x be supported ->
  • 4.x was supported for many years (4-5), officially declared unsupported when it dropped out off debian stable
  • Exponential back off, support this long time meaning security fixes
• New plugins for 5.x after 6.0 release, choices are:
  • Build plugins from out of tree, separate repo
  • Separate core plugins and rest of the plugins which would help core plugins
  • This would require changes to configure file to help build faster for core plugins vs. others