This is a simple Splunk log integration using fluentd that works with any of the devices supported by Balena.
This image includes two containers: writelogs and sendlogs.
In writelogs, the src/main.sh script continually writes random events to logs/logs.txt. /logs is a shared volume accessible from both continers.
In sendlogs, fluentd tails the logs.txt file and sends to a Splunk Enterprise or Splunk Cloud HTTP endpoint collector (HEC) using the Splunk-fluentd output plugin.
To get this project up and running on the Splunk end, you'll need to have a working Splunk Cloud or Splunk Enterprise environment with the HTTP Event Collector (HEC) enabled and accessible from the internet. You'll also need to create a HEC authentication token and have a target events index enabled.
On the Balena end, signup for a balena account here, set up a device, and have a look at the Getting Started tutorial. Once you are set up with balena, you will need to clone this repo locally.
Follow these steps to push the code to your fleet to enable the data collection. Make sure to change the Dockerfile.template file to match the architecture of the target device for the build.
Once deployed, add the four OS environment variables below to enable the connection to Splunk:
SPLUNK_HOST = IP address or hostname of remote Splunk host
SPLUNK_PORT = HEC port for the remote Splunk host
SPLUNK_TOKEN = the Splunk authentication token for HEC access
SPLUNK_INDEX = the name of the Splunk target index where the data will be stored
Note: The Splunk index must be an events index (not metrics).
This is how the variables should look on the Balena console:
Once running correctly, you should see this in your logs:
In this example, the log file source and sourcetype are hard coded in the fluentd.conf. These can also be set as OS variables in Balena if preferred.
Here's an example search in Splunk. Note that the events are forwarded by fluentd as JSON:
