Skip to content

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.

License

Notifications You must be signed in to change notification settings

driskell/log-courier

Repository files navigation

Log Courier Suite

Golang Ruby Release

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.

Log Courier

Log Courier is a lightweight shipper. It reads from log files and transmits events over the Courier protocol to a remote Logstash or Log Carver instance.

  • Reads from files or the program input, following log file rotations and movements
  • Compliments log events with additional fields
  • Live configuration reload
  • Transmits securely using TLS with server and client verification
  • Codecs for client-side preprocessing of multiline events and filtering of unwanted events
  • Native JSON reader to support JSON files, even those with no line-termination that makes line-based reading problematic
  • Remote Administration Utility to inspect monitored log files and connections in real time.
  • Compatible with all supported versions of Logstash. At the time of writing this is >= 7.7.x.

Log Carver

Log Carver is a lightweight event processor and alternative to Logstash. It receives events over the Courier protocol and performs actions against them to manipulate them into the required format for storage within Elasticsearch, or further processing in Logstash. Connected clients do not receive acknowledgements until the events are acknowledged by the endpoint, whether that be Elasticsearch or another more centralised Log Carver, providing end-to-end guarantee.

Philosophy

  • Keep resource usage low and predictable at all times
  • Be efficient, reliable and scalable
  • At-least-once delivery of events, a crash should never lose events
  • Offer secure transports
  • Be easy to use

Documentation

Installation

Reference

Upgrading from 1.x to 2.x

There are many breaking changes in the configuration between 1.x and 2.x. Please check carefully the list of breaking changes here: Change Log.

Packages also now default to using a log-courier user. If you require the old behaviour of root, please be sure to modify the /etc/sysconfig/log-courier (CentOS/RedHat) or /etc/default/log-courier (Ubuntu) file.

About

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.

Resources

License

Stars

Watchers

Forks