Configuration

The configuration format used by ingraind is TOML.

The primary goal of the configuration system is to set up the grains and the processing pipelines used by ingraind. As a rule of thumb, all settings that are sensitive (eg. credentials), or can change on every deployment, will be taken from environment variables.

The TOML file has two sections: the probes, and the pipelines. The probe array contains the configuration for the probes, and a single entry looks like this:

[[probe]]
pipelines = ["my_pipeline", "another_pipeline"]

[probe.config]
type = "Files"
monitor_dirs = ["/"]

The probe has to specify a list of pipelines that will receive its data. The probe.config section configures the probe itself; the type attribute is mandatory, anything else is probe dependent.

The different brackets in [[probe]] and [probe.config] are intentional: [[probe]] means a new entry in the probe array, while [probe.config] means "the config attribute of the probe".

The pipelines are named sequences of actions. Some steps in the pipeline will have state, so be aware that multiple probes targeting the pipeline with the same name, will use the same instance of the pipeline.

Configuration of pipelines looks like this:

[pipeline.my_pipeline.config]
backend = "StatsD"
use_tags = true

[[pipeline.statsd.steps]]
type = "Whitelist"
allow = ["k1", "k2"]

[[pipeline.statsd.steps]]
type = "AddSystemDetails"

[[pipeline.statsd.steps]]
type = "Buffer"
interval_s = 30

The distinction between the [pipeline.statsd.config] and [[pipeline.statsd.steps]] are the same as previously.

Events generated by the probe will hit the first step in the pipeline. Steps are defined in sequence, and events will trickle through every step, unless they are filtered out.

The pipeline.name.config.backend specifies which backend is used to transmit or store the events after every step in the pipeline is complete, and is a mandatory attribute. Other attributes in the pipeline.name.config section are dependent on the given backend.