Ethlogger Configuration

For fine-grained control over ethlogger's operation you can create a configuration file and tweak any of the settings available. Some settings can also be adjusted using CLI flags or environment variables (CLI flags and environment variables do take precedence over settings in the configuration, if specified).

The configuration file can be created either in YAML or in JSON format. You can specify the configuration file path using the --config-file (short -c) CLI flag:

$ ethlogger -c path/to/myconfig.yaml

or, if omitted, ethlogger will look for a file called ethlogger.yaml in the current working directory. If this file is not present either, then ethlogger will go with the default configuration.

The configuration file content will be layered on top of the defaults, so it is only necessary to specify settings where the default needs to be overridden.

Example

ethlogger.yaml

eth:
    url: https://rpc.gnosischain.com
    network: xdai
hec:
    default:
        url: https://localhost:8088
        token: 44422111-0000-3232-9821-26664c2e7515
        validateCertificate: false
        # Splunk 8.0 or higher support compact metrics HEC messages
        multipleMetricFormatEnabled: true
    events:
        defaultMetadata:
            index: myevents
    metrics:
        defaultMetadata:
            index: mymetrics
blockWatcher:
    startAt: latest
abi:
    directory: ./abis
internalMetrics:
    enabled: false

Debugging Ethlogger Configuration

Ethlogger has a dedicated flag --print-config to show the effective configuration after merging defaults, config file, environment variables and CLI flags.

$ ethlogger -c myconfig.yaml --print-config

JSON Schema

You can find a JSON Schema for ethlogger's configuration files here: config.schema.json.

Editor Integrations

Several editors can use JSON schemas to provide validation and auto-complete as you edit the config file.

JSON schemas in VS Code: https://code.visualstudio.com/docs/languages/json#_json-schemas-and-settings
https://json-schema.org/implementations.html#editors

Reference

Ethlogger

Root configuration schema for ethlogger

Name	Type	Description
`eth`	`Ethereum`	Ethereum node configuration
`output`	`HecOutput` \| `ConsoleOutput` \| `FileOutput` \| `DevNullOutput`	In the output configuration you can specify where ethlogger will send generated metrics and events to. By default it will send all information to Splunk HEC, but you can instead send it to console output or a file.
`hec`	`HecClients`	HTTP event collector
`checkpoint`	`Checkpoint`	Checkpoint configuration - how ethlogger keeps track of state between restarts
`abi`	`AbiRepository`	ABI repository configuration
`contractInfo`	`ContractInfo`	Contract info cache settings
`blockWatcher`	`BlockWatcher`	Block watcher settings, configure how blocks, transactions, event logs are ingested
`balanceWatchers`	`BalanceWatchers`	Balance watchers, tracking balance of ERC-20 token holders
`nftWatchers`	`NFTWatchers`	NFT watchers, tracking balance of ERC-721 token holders
`nodeMetrics`	`NodeMetrics`	Settings for the node metrics collector
`nodeInfo`	`NodeInfo`	Settings for the node info collector
`pendingTx`	`PendingTx`	Settings for collecting pending transactions from node
`peerInfo`	`PeerInfo`	Settings for collecting peer information from the node
`internalMetrics`	`InternalMetrics`	Settings for internal metrics collection

Ethereum

General Ethereum configuration including client and transport, defining how ethlogger talks to the ethereum node

Name	Type	Description
`url`	`string`	URL of JSON RPC endpoint
`network`	`string`	Network name logged as a field with every event and metric. Ethlogger will attempt to automatically determine if not specified but there are only a handful of known public networkIds associated with particular networks (ethereum mainnet, ropsten, ...). Typical values of the network name are `"mainnet"` or `"testnet"`.
`chain`	`string`	Chain name logged as a field with every event and metric. Ethlogger will attempt to automatically determine if not specified but there are only a handful of known public chainIds associated with particular ethereum-based chains. This value will allow consumers of data to distinguish between different chains in case multiple chains are being logged to one place. See https://chainid.network
`http`	`HttpTransport`	HTTP transport configuration
`client`	`EthereumClient`	Ethereum client configuration

HttpTransport

Settings for ethlogger connecting to the ethereum node via JSON RPC over HTTP

Name	Type	Description
`timeout`	`Duration`	Time before failing JSON RPC requests. Specify a number in milliseconds or a golang-style duration expression. Example: `"30s"`
`validateCertificate`	`boolean`	If set to false, the HTTP client will ignore certificate errors (eg. when using self-signed certs)
`requestKeepAlive`	`boolean`	Keep sockets to JSON RPC open
`maxSockets`	`number`	Maximum number of sockets HEC will use (per host)
`proxyUrl`	`string`	Optional proxy URL to route HTTP requests through. Note: this disables internal httpClient stats

EthereumClient

Ethereum client settings - configure batching multiple JSON RPC method calls into single HTTP requests

Name	Type	Description
`maxBatchSize`	`number`	Maximum number of JSON RPC requests to pack into a single batch. Set to `1` to disable batching.
`maxBatchTime`	`Duration`	Maximum time to wait before submitting a batch of JSON RPC requests

HecOutput

Name	Type	Description
`type`	`"hec"`
`sourcetypes`	`Sourcetypes`	Sourcetypes to use for different kinds of events we send to Splunk
`metricsPrefix`	`string`	A common prefix for all metrics emitted to Splunk

Sourcetypes

Configurable set of sourcetype field values emitted by ethlogger

Name	Type	Default
`block`	`string`	`"ethereum:block"`
`transaction`	`string`	`"ethereum:transaction"`
`event`	`string`	`"ethereum:transaction:event"`
`pendingtx`	`string`	`"ethereum:transaction:pending"`
`nodeInfo`	`string`	`"ethereum:node:info"`
`nodeMetrics`	`string`	`"ethereum:node:metrics"`
`gethPeer`	`string`	`"ethereum:geth:peer"`
`balance`	`string`	`"ethereum:balance"`
`nft`	`string`	`"ethereum:nft"`

ConsoleOutput

Console output prints all generated events and metrics to STDOUT

Name	Type
`type`	`"console"`

FileOutput

File output will append all generated messages to a file. (this output type has not been implemented)

Name	Type	Description
`type`	`"file"`
`path`	`string`	Path to output file

DevNullOutput

Null output will just drop all generated events and metrics

Name	Type
`type`	`"null"`

HecClients

Name	Type	Description
`default`	`Hec`	Base settings that apply to all HEC clients. Overrides for events, metrics and internal metrics will be layered on top of the defaults and allow for using different HEC tokens, URL or destination index.
`events`	`Hec`	HEC settings (overrides for `default`) for events sent to Splunk
`metrics`	`Hec`	HEC settings (overrides for `default`) for metrics sent to Splunk
`internal`	`Hec`	HEC settings (overrides for `default`) for internal metrics sent to Splunk

Hec

Settings for the Splunk HTTP Event Collector client

Name	Type	Description	Default
`url`	`string`	The URL of HEC. If only the base URL is specified (path is omitted) then the default path will be used
`token`	`string`	The HEC token used to authenticate HTTP requests
`defaultMetadata`	`object`	Defaults for host, source, sourcetype and index. Can be overridden for each message See Use variables in metadata
`defaultFields`	`object`	Default set of fields to apply to all events and metrics sent with this HEC client See Use variables in metadata
`maxQueueEntries`	`number`	Maximum number of entries in the HEC message queue before flushing it
`maxQueueSize`	`number`	Maximum number of bytes in the HEC message queue before flushing it
`flushTime`	`Duration`	Maximum number of milliseconds to wait before flushing the HEC message queue
`gzip`	`boolean`	Gzip compress the request body sent to HEC (Content-Encoding: gzip)
`maxRetries`	`number`	Maximum number of attempts to send a batch to HEC. By default this there is no limit
`timeout`	`Duration`	Number of milliseconds to wait before considering an HTTP request as failed
`requestKeepAlive`	`boolean`	Set to `false` to disable HTTP keep-alive for connections to Splunk
`validateCertificate`	`boolean`	If set to false, the HTTP client will ignore certificate errors (eg. when using self-signed certs)
`maxSockets`	`number`	Maximum number of sockets HEC will use (per host)
`userAgent`	`string`	User-agent header sent to HEC See Use variables in metadata	`ethlogger-hec-client/<version>`
`retryWaitTime`	`WaitTime`	Wait time before retrying to send a (batch of) HEC messages after an error
`multipleMetricFormatEnabled`	`boolean`	Enable sending multipe metrics in a single message to HEC. Supported as of Splunk 8.0.0 https://docs.splunk.com/Documentation/Splunk/8.0.0/Metrics/GetMetricsInOther#The_multiple-metric_JSON_format
`waitForAvailability`	`Duration`	If set to > 0, then ethlogger will wait for the HEC service to become available for the given amount of time by periodically attempting to request the collector/health REST endpoint. This can be useful when starting Splunk and ethlogger for example in docker-compose, where Splunk takes some time to start.

Checkpoint

The checkpoint is where ethlogger keeps track of its state, which blocks have already been processed. This allows it to resume where it left off after being shut down and restarted.

Name	Type	Description	Default
`filename`	`string`	File path (relative to the current working directory) where the checkpoint file will be stored	`checkpoints.json`
`saveInterval`	`Duration`	Maximum duration before saving updated checkpoint information to disk

AbiRepository

The ABI repository is used to decode ABI information from smart contract calls and event logs. It generates and adds some additional information in transactions and events, including smart contract method call parameter names, values and data types, as well as smart contract names associated with a particular contract address.

Name	Type	Description
`directory`	`string`	If specified, the ABI repository will recursively search this directory for ABI files
`searchRecursive`	`boolean`	`true` to search ABI directory recursively for ABI files
`abiFileExtension`	`string`	Set to `.json` by default as the file extension for ABIs
`fingerprintContracts`	`boolean`	If enabled, the ABI repository will creates hashes of all function and event signatures of an ABI (the hash is the fingerprint) and match it against the EVM bytecode obtained from live smart contracts we encounter.
`requireContractMatch`	`boolean`	If enabled, signature matches will be treated as anonymous (parameter names will be omitted from the output) if a contract cannot be tied to an ABI definition via either a fingerprint match, or a contract address match (when the ABI file includes the address of the deployed contract). Enabled by default. Setting this to `false` will output parameter names for any matching signature.
`decodeAnonymous`	`boolean`	If enabled, ethlogger will attempt to decode function calls and event logs using a set of common signatures as a fallback if no match against any supplied ABI definition was found.
`reconcileStructShapeFromTuples`	`boolean`	If enabled, ethlogger will attempt to reconcile the shape of struct definitions from decoded tuples data if the ABI definition contains names for each of the tuple components. This basically turns the decoded array into an key-value map, where the keys are the names from the ABI definition.

ContractInfo

Ethlogger checks for each address it encounters whether it is a smart contract by attempting to retrieve the contract code. To reduce the performance hit by this operation, ethlogger can cache contract information in memory.

Name	Type	Description
`maxCacheEntries`	`number`	Maximum number of contract info results to cache in memory. Set to 0 to disable the cache.

BlockWatcher

Block watcher is the component that retrieves blocks, transactions, event logs from the node and sends them to output.

Name	Type	Description
`enabled`	`boolean`	Specify `false` to disable the block watcher
`pollInterval`	`Duration`	Interval in which to look for the latest block number (if not busy processing the backlog)
`blocksMaxChunkSize`	`number`	Max. number of blocks to fetch at once
`maxParallelChunks`	`number`	Max. number of chunks to process in parallel
`startAt`	`StartBlock`	If no checkpoint exists (yet), this specifies which block should be chosen as the starting point.
`retryWaitTime`	`WaitTime`	Wait time before retrying to fetch and process blocks after failure
`decryptPrivateTransactions`	`boolean`	For chains/nodes that do support private transactions, this setting instructs block watcher to attempt to load the decrypted payload for private transactions

BalanceWatchers

Balance watchers is a component tracking transfers of tokens and reporting balances of accounts.

Name	Type	Description
`-`	map<string,`BalanceWatcher`>	Mapping of name => balancer watcher. See BalanceWatcherConfigSchema

BalanceWatcher

Balance watcher is a component tracking transfers of a token and reporting balances of its accounts.

Name	Type	Description
`contractAddress`	`string`	The address of the contract to watch.
`decimals`	`number`	The number of decimals to divide balances with.
`startAt`	`StartBlock`	If no checkpoint exists (yet), this specifies which block should be chosen as the starting point.
`endAt`	`number`	This optionally specifies at which block the watcher should stop collecting data.
`enabled`	`boolean`	Specify `false` to disable the balance watcher
`pollInterval`	`Duration`	Interval in which to look for the latest block number (if not busy processing the backlog)
`blocksMaxChunkSize`	`number`	Max. number of blocks to fetch at once
`maxParallelChunks`	`number`	Max. number of chunks to process in parallel
`retryWaitTime`	`WaitTime`	Wait time before retrying to fetch and process blocks after failure
`logEthBalance`	`boolean`	Log the account ethereum balance

NFTWatchers

NFT watchers is a component tracking transfers of NFTs and reporting their metadata.

Name	Type	Description
`-`	map<string,`NFTWatcher`>	Mapping of name => nft watcher. See NFTWatcherConfigSchema

NFTWatcher

NFT watcher is a component tracking transfers of NFTs and reporting their metadata.

Name	Type	Description
`contractAddress`	`string`	The address of the contract to watch.
`startAt`	`StartBlock`	If no checkpoint exists (yet), this specifies which block should be chosen as the starting point.
`endAt`	`number`	This optionally specifies at which block the watcher should stop collecting data.
`enabled`	`boolean`	Specify `false` to disable the balance watcher
`pollInterval`	`Duration`	Interval in which to look for the latest block number (if not busy processing the backlog)
`blocksMaxChunkSize`	`number`	Max. number of blocks to fetch at once
`maxParallelChunks`	`number`	Max. number of chunks to process in parallel
`retryWaitTime`	`WaitTime`	Wait time before retrying to fetch and process blocks after failure
`logEthBalance`	`boolean`	Log the account ethereum balance

NodeMetrics

The node metrics collector retrieves numeric measurements from nodes on a periodic basis.

Name	Type	Description
`enabled`	`boolean`	Specify `false` to disable node metrics collection
`collectInterval`	`Duration`	Interval in which to collect node metrics
`retryWaitTime`	`WaitTime`	Wait time before retrying to collect node metrics after failure

NodeInfo

Platform specific node information is collection on regular interval

Name	Type	Description
`enabled`	`boolean`	Specify `false` to disable node info collection
`collectInterval`	`Duration`	Interval in which to collect node info
`retryWaitTime`	`WaitTime`	Wait time before retrying to collect node info after failure

PendingTx

Periodic collection of pending transactions

Name	Type	Description
`enabled`	`boolean`	Enable or disable collection of pending transactions
`collectInterval`	`Duration`	Interval in which to collect pending transactions
`retryWaitTime`	`WaitTime`	Wait time before retrying to collect pending transactions after failure

PeerInfo

Periodic collection of detailed peer information. Note that this is only possible with certain types of ethereum nodes (geth atm)

Name	Type	Description
`enabled`	`boolean`	Enable or disable collection of peer information
`collectInterval`	`Duration`	Interval in which to collect peer information
`retryWaitTime`	`WaitTime`	Wait time before retrying to collect peer information after failure

InternalMetrics

Ethlogger-internal metrics allow for visibility into the operation of ethlogger itself.

Name	Type	Description
`enabled`	`boolean`	Specify `false` to disable internal metrics collection
`collectInterval`	`Duration`	Interval in which to collect internal metrics

Duration

A duration in ethlogger's config can be specified either as as a number or a string.

Type	Description	Examples
`number`	Number of milliseconds	`500`
`string`	Golang-style duration string See https://golang.org/pkg/time/#ParseDuration	`"1h"`, `"3m30s500ms"`

WaitTime

Wait time expresses how long to wait between retry attempts based on the number of attempts made so far.

Type	Description
`Duration`	Absolute and static time to wait after each attempt
`ExponentialBackoff`	Exponentially increasing wait time with randomness
`LinearBackoff`	Linear increasing wait time

ExponentialBackoff

Exponentially increasing wait time with randomness. The wait time will be a random number between min and 2^attempts (up to max) after each attempt.

Name	Type	Description
`type`	`"exponential-backoff"`
`min`	`Duration`	Minimum wait time
`max`	`Duration`	Maximum wait time

Example

{ "type": "exponential-backoff", "min": 0, "max": "5m" }

LinearBackoff

Linear increasing wait time

Name	Type	Description
`type`	`"linear-backoff"`
`min`	`Duration`	Minimum wait time (after the first failure)
`step`	`Duration`	Increase of wait time for each failure after the first until max is reached
`max`	`Duration`	Maximum wait time