v2.0.4 release

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] Improve sender log condition to record valuable debug info (#1359)
• [public] [both] [added] wait for read flush timeout instead of flushing last line immediately on file deletion (#1418)
• [public] [both] [added] refactor logic of last matched line (#1426)
• [public] [both] [added] test: add a case to test timeformat without year (#1432)
• [public] [both] [added] Update chatgroup image (#1445)
• [public] [both] [added] refactory: enable move for SourceBuffer (#1457)
• [public] [both] [added] summer-ospp-2024 (#1461)
• [public] [both] [added] default enable spl (#1475)
• [public] [both] [added] allow empty endpoint in enterprise flusher_sls to avoid unsuccessful load of config (#1486)

Fixed

• [public] [both] [fixed] fix dockershim cause containerd invalid (#1424)
• [public] [both] [fixed] fix: correct pipeline version recognition issue (#1427)
• [public] [both] [fixed] fix: refactor multiline split state (#1410)
• [public] [both] [fixed] fix: update base image to resolve 'Illegal instruction' error on old CPUs (#1429)
• [public] [both] [fixed] ensure config name in golang config and ctx matching C++ config name (#1437)
• [public] [both] [fixed] fix missing user agent info (#1444)
• [public] [both] [fixed] fix plugin alarm with default version (#1443)
• [public] [both] [fixed] fix: make LOGTAIL_LOG_LEVEL env variable effective for ilogtail.LOG (#1440)
• [public] [both] [fixed] Fix parameter errors for container metadata preview. (#1430)
• [public] [both] [fixed] fix: Ensure alarms raised during collection for profile projects are reported (#1455)
• [public] [both] [fixed] fix missing uuid and incorrect instance id (#1476)

Doc

• [public] [both] [doc] document: fix document for flusher pulsar (#1460)

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-2.0.4.linux-amd64.tar.gz	Linux	x86-64	ilogtail-2.0.4.linux-amd64.tar.gz.sha256
ilogtail-2.0.4.linux-arm64.tar.gz	Linux	arm64	ilogtail-2.0.4.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:2.0.4

v1.8.8 release

1.8.8

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] Processor rate limit #1321

Fixed

• [public] [both] [fixed] Fix apsara base field parse error #1307
• [public] [both] [fixed] Fix microsecond handling in Apsara log parser #1309
• [public] [both] [fixed] Resolve unintended Go module loading caused by high-precision timestamp settings #1336
• [public] [both] [fixed] Ensure process is able to exit when history file import is ever used #1369
• [public] [both] [fixed] Fix missing Connection_string tag in Skywalking database Span #1407
• [public] [both] [fixed] Maintain sequential info of the original log content and allow multivalue for single key for apsara processor #1416
• [public] [both] [fixed] Improve sender log condition to record valuable debug info #1419

Doc

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.8.8.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.8.8.linux-amd64.tar.gz.sha256
ilogtail-1.8.8.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.8.8.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.8.8

v2.0.0 release

2.0.0

iLogtail 2.0 is a completely new version with upgraded architecture, and its configuration files are not fully compatible with the 1.x version. For the upgrade guide, please refer to the change log #1294 and example configs.

The configuration method of version 2.0 perfectly supports cascading between C++ native plugins and cascading between C++ native plugins and Go plugins. At the same time, it introduces a more efficient data processing language, SLS Processing Language (SPL), making on-device data processing more effortless.

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] add input and flusher interface #1184
• [public] [both] [added] complete pipeline definition #1185
• [public] [both] [added] support V2 config #1188
• [public] [both] [added] Support jsonline protocol #1265
• [public] [both] [added] Support SPL processor in ilogtail for enhanced data processing #1278
• [public] [both] [added] add UsingOldContentTag. When UsingOldContentTag is set to false, the Tag is now placed in the Meta instead of Logs during cgo #1169
• [public] [both] [added] Upgrade environment variable logic using new Pipeline API for logtail environment configuration management #1282
• [public] [both] [updated] improve source file structure #1182
• [public] [both] [updated] feat: add flusher_http queue buffer and async interceptor settings #1203
• [public] [both] [updated] Enhancing the vscode development experience #1219
• [public] [both] [updated] feat: add kafka net config MaxOpenRequests #1224
• [public] [both] [updated] feat: support labels for self_metric #1240
• [public] [both] [updated] Support contents only for loki flusher #1256
• [public] [both] [updated] Fix issue where container IP is occasionally empty for Kubernetes pods in HostNetwork mode #1280
• [public] [both] [updated] Improve goprofile plugin to support real local IP inclusion #1281
• [public] [both] [updated] Added a switch to control the loading of the processor_spl, which by default remains unloaded #1312

Fixed

• [public] [both] [fixed] fix: resolve crash by setting CURLOPT_NOSIGNAL for thread safety #1283
• [public] [both] [fixed] fix: Prevent crash due to concurrent map access in Go plugin log processing #1284
• [public] [both] [fixed] Fix delimiter parsing for logs with leading spaces #1289
• [public] [both] [fixed] Fix timezone handling in native processors #1293
• [public] [both] [fixed] Fix log content duplication in ParseJsonNativeProcessor #1296
• [public] [both] [fixed] fix memory leak in ProcessorParseDelimiterNative #1300
• [public] [both] [fixed] fix: reset checkpoint dump time after dir registration completes to avoid log duplication #1291
• [public] [both] [fixed] Fix timestamp parsing compatibility issue for apsara log #1285
• [public] [both] [fixed] Fix: Delete mSourceKeyOverwritten in processorParseApsaraNative to prevent concurrency issues #1305
• [public] [both] [fixed] Fix apsara base field parse error and microsecond handling in Apsara log parser #1308

Doc

• [public] [both] [doc] update docs related to V2 config #1287

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-2.0.0.linux-amd64.tar.gz	Linux	x86-64	ilogtail-2.0.0.linux-amd64.tar.gz.sha256
ilogtail-2.0.0.linux-arm64.tar.gz	Linux	arm64	ilogtail-2.0.0.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:2.0.0

v1.8.4 release

1.8.4

Changes

All issues and pull requests are here.

Features

• [public] [both] [updated] Improve goprofile plugin to support real local IP inclusion #1281

Fixed

• [public] [both] [fixed] fix panic when k8s_event.GetEventRecorder() is nil #1264
• [public] [both] [fixed] When REGEX_LOG is set, mTimeKey will be set to "time". #1272
• [public] [both] [fixed] [backport]fix: resolve crash by setting CURLOPT_NOSIGNAL for thread safety #1286
• [public] [both] [fixed] [backport]Fix delimiter parsing for logs with leading spaces #1288
• [public] [both] [fixed] [backport]Fix timezone handling in native processors #1293
• [public] [both] [fixed] [backport]Fix log content duplication in ParseJsonNativeProcessor #1295
• [public] [both] [fixed] [backport]fix memory leak in ProcessorParseDelimiterNative #1299
• [public] [both] [fixed] [backport] fix: reset checkpoint dump time after dir registration completes to avoid log duplication #1301
• [public] [both] [fixed] [backport]Fix timestamp parsing compatibility issue for apsara log #1302
• [public] [both] [fixed] Fix: remove content if native processor parse failed #1304
• [public] [both] [fixed] Fix issue where container IP is occasionally empty for Kubernetes pods in HostNetwork mode #1280

Doc

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.8.4.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.8.4.linux-amd64.tar.gz.sha256
ilogtail-1.8.4.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.8.4.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.8.4

v1.8.3 release

1.8.3

Changes

All issues and pull requests are here.

Features

Fixed

• [public] [both] [fixed] fix log duplication problem arised from #1216
• [public] [both] [fixed] fix plugin crash due to nil in container info
• [public] [both] [fixed] resolve issue with ProcessorParseDelimiterNative capturing next line data
• [public] [both] [fixed] correct premature file read termination triggered by sender back pressure
• [public] [both] [fixed] implement panic recovery mechanism in plugin export
• [public] [both] [fixed] resolve crash issue during Apsara log parsing
• [public] [both] [fixed] correct parsing errors in ProcessorParseApsaraNative with large buffer input

Doc

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.8.3.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.8.3.linux-amd64.tar.gz.sha256
ilogtail-1.8.3.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.8.3.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.8.3

v1.8.1 release

1.8.1

Changes

All issues and pull requests are here.

Features

Fixed

• [public] [both] [fixed] fix prometheus send wrong sls logs when reuse string memory
• [public] [both] [fixed] fix dropping jvm metrics when collecting multiple skywalking instances's data
• [public] [both] [fixed] fix elasticsearch flusher authentication tls config and http config
• [public] [both] [fixed] fix profiling wrong type when the different profiling type having same stack.
• [public] [both] [fixed] fix send local buffer failed when upgrade iLogtail from version earlier than 1.3.
• [public] [both] [fixed] change strptime_ns to parse %c format from "%x %X" to "%a %b %d %H:%M:%S %Y" for consistent behavior with striptime.
• [public] [both] [fixed] fix topic key does not support underscore.
• [public] [both] [fixed] fix jmxfetch status error when exist multi jmxfetch config in the same machine.
• [public] [both] [fixed] fix increasing WSS memory issue in collected containers.
• [public] [both] [fixed] fix cannot log blacklist config error

Doc

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.8.1.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.8.1.linux-amd64.tar.gz.sha256
ilogtail-1.8.1.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.8.1.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.8.1

v1.8.0 release

1.8.0

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] refactoried C++ process pipeline
• [public] [both] [added] support use accelerate processors with go processors
• [public] [both] [added] add new logtail metric module
• [public] [both] [added] ddd JSON flatten protocol, data can be flattened and then brushed into storage such as Kafka and ES.
• [public] [both] [added] use env LOGTAIL_LOG_LEVEL to control ilogtail log level
• [public] [both] [added] add global host path blacklist
• [public] [both] [updated] support continue/end regex patterns to split multiline log
• [public] [both] [updated] support reader flush timeout
• [public] [both] [updated] flusher Kafka V2: support send the message with headers to kafka
• [public] [both] [updated] sls flusher supports nanosecond timestamp
• [public] [both] [updated] update gcc version to 9.3.1
• [public] [both] [updated] add make flag WITHOUTGDB
• [public] [both] [updated] cache incomplete line in memory to avoid repeated read system call
• [public] [both] [updated] unify metrics format
• [public] [both] [updated] verify host ip against NIC to get more accurate host ip
• [public] [both] [added] support plugin ProcessorParseTimestampNative
• [public] [both] [added] support plugin processor_otel_metric
• [public] [both] [updated] skywalking plugin support to capture db.connection_string tag
• [public] [both] [added] support plugin ProcessorParseApsaraNative
• [public] [both] [added] support plugin ProcessorParseDelimiterNative
• [public] [both] [added] support plugin ProcessorFilterNative
• [public] [both] [added] support plugin ProcessorDesensitizeNative
• [public] [both] [added] add resource tag for env created logtail config

Fixed

• [public] [both] [fixed] remove the ip restrict of config server address, allow domain and hostname
• [public] [both] [fixed] add APSARA_LOG_TRACE to solve the problem of not being able to find LOG_TRACE.
• [public] [both] [fixed] fix collection duplication for statefulset sidecar that may mount same volume on different nodes
• [public] [both] [fixed] fix multiline is splitted if not flushed to disk together
• [public] [both] [fixed] fix line is truncated if \0 is in the middle of line
• [public] [both] [fixed] container cannot exit for file reopened by checkpoint
• [public] [both] [fixed] fix filename being mismatched to the deleted file if the deleted file size is 0 and their inode is same
• [public] [both] [fixed] fix config server panic caused by concurrent read and write shared object
• [public] [both] [fixed] add expand array switch in json processor for backward compatibility
• [public] [both] [fixed] timezone adjust not working with apsara_log
• [public] [both] [fixed] fix parse json without newline may result in partial parsing
• [public] [both] [fixed] fix json does not split correctly if beginning buffer contains invalid json

Doc

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.8.0.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.8.0.linux-amd64.tar.gz.sha256
ilogtail-1.8.0.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.8.0.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.8.0

v1.7.1 release

1.7.1

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] Add new plugin: input_command
• [public] [both] [added] Add new plugin: processor_log_to_sls_metric
• [public] [both] [added] C++ Unit test CI
• [public] [both] [updated] Elasticsearch flusher new features: send batch request by bulk api and format index
• [public] [both] [updated] Service_http_server support raw type on v1
• [public] [both] [updated] Add v2 interface to processor_json
• [public] [both] [updated] Json processor support parsing array
• [public] [both] [updated] Service_prometheus support scale in kubernetes
• [public] [both] [updated] Support to export logtail listening ports
• [public] [both] [updated] Add containerd custom rootpath support
• [public] [both] [updated] Upgrade go version to 1.19
• [public] [both] [updated] Refactory LogBuffer structure
• [inner] [both] [updated] Logtail containers monitor refine code
• [inner] [both] [updated] Add too long log split alarm

Fixed

• [public] [both] [fixed] Fix service_go_profile nil panic
• [public] [both] [fixed] Fix broken container stdout log path link
• [public] [both] [fixed] Fix zstd batch send error
• [public] [both] [fixed] Fix go pb has TimeNs when not set
• [public] [both] [fixed] Fix elasticsearch flusher panic
• [public] [both] [fixed] Fix the abnormal shutdown issue of service_otlp
• [public] [both] [fixed] Fix inconsistence of ttl when param invalid
• [public] [both] [fixed] Fix proxy env key error
• [public] [both] [fixed] Update map time after mark container removed
• [public] [both] [fixed] Fix multi-bytes character cut off
• [public] [both] [fixed] Add timeout func when collect disk metrics
• [public] [both] [fixed] Fix updating agent config's check_status in heartbeat in config server
• [public] [both] [fixed] Fix otlp flusher does not set Timestamp
• [public] [both] [fixed] Fix snapshot dir not created when apsara_log_conf.json exits
• [inner] [both] [fixed] Fix file name of shennong profile data in container

Doc

• [public] [both] [doc] Elasticsearch flusher config examples
• [public] [both] [doc] Fix document of flusher kafka v2

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.7.1.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.7.1.linux-amd64.tar.gz.sha256
ilogtail-1.7.1.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.7.1.linux-arm64.tar.gz.sha256
ilogtail-1.7.1.windows-amd64.zip	Windows	x86-64	ilogtail-1.7.1.windows-amd64.zip.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.7.1

v1.7.0 release

1.7.0

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] support timestamp nanosecond
• [public] [both] [added] add RawLogTag to support original log key rename
• [public] [both] [updated] upgrade go.mod version to 1.19
• [public] [both] [updated] add too long log split alarm

Fixed

• [public] [both] [fixed] metric_system_v2 add timeout func when collect disk metrics

Doc

• [public] [both] [doc] add parse high resolution time while upload raw log config example
• [public] [both] [doc] add parse swift alliance gateway log for audit config example
• [public] [both] [doc] add guidance to setup iLogtail resource limit feature in k8s deployment.
• [public] [both] [doc] fix the description error of supported version range for Kafka

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.7.0.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.7.0.linux-amd64.tar.gz.sha256
ilogtail-1.7.0.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.7.0.linux-arm64.tar.gz.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.7.0

v1.6.0 release

1.6.0

Changes

All issues and pull requests are here.

Features

• [public] [both] [added] Decoder support Opentelemetry metric to SLS Log Protocol, service_otlp supports v1 pipeline, service_http_server v1 pipeline supports otlp metric
• [public] [both] [added] Implement influxdb DecodeV2 method
• [public] [both] [added] plugin_main support file-to-file test
• [public] [both] [added] metric_meta_kubernetes support collect kruise CRD meta
• [public] [both] [updated] Optimize flusher pulsar to improve performance in static topic scenarios
• [public] [both] [updated] Upgrade sarama version to 1.38.1, Rewrite service_kafka input plugin
• [public] [both] [updated] Optimize flusher kafka v2 to improve performance in static topic scenarios
• [public] [both] [updated] Optimize prometheus decoder 20%
• [public] [both] [updated] Change config server's GET method to POST in order to carry payload for browser
• [public] [both] [updated] Improve processor desensitize performance 20-80%
• [public] [both] [updated] service-otlp-input supports gzip decompression
• [public] [both] [updated] Add more prometheus service discovery support
• [inner] [both] [updated] Add config name in shennong log key
• [inner] [both] [updated] add tags to profile log to get container meta

Fixed

• [public] [both] [fixed] Resolved issue of double counting disk total metrics in the disk partition condition of metric_system_v2
• [public] [both] [fixed] Do not read env config from exited containers
• [public] [both] [fixed] Json converter marshal without HTML escaped
• [public] [both] [fixed] Fix various bugs in config server cause server crash
• [public] [both] [fixed] Fix container file fd not close if multiple configs monitor the same dir
• [public] [both] [fixed] Fix processor_gotime cannot config to UTC-0 time zone
• [public] [both] [fixed] Fix beginning lines lost if they do not match multiline regex in core
• [public] [both] [fixed] Fix processor_split_log_regex does not trim ending \n
• [public] [both] [fixed] Fix memory access error in evenv_handler
• [public] [both] [fixed] Fix string lost if regex not match in processor_string_replace
• [inner] [both] [fixed] Fix container label lost when update container info in static file
• [inner] [both] [fixed] Request to much to ECS meta service when using meta service api
• [inner] [both] [fixed] Fix env config cache cannot expire

Doc

• [public] [both] [doc] Add config server ui quick start
• [public] [both] [doc] Add user_contrib dir to collect configuration best practice
• [public] [both] [doc] Explain processor_gotime time format

Download

Filename	OS	Arch	SHA256 Checksum
ilogtail-1.6.0.linux-amd64.tar.gz	Linux	x86-64	ilogtail-1.6.0.linux-amd64.tar.gz.sha256
ilogtail-1.6.0.linux-arm64.tar.gz	Linux	arm64	ilogtail-1.6.0.linux-arm64.tar.gz.sha256
ilogtail-1.6.0.windows-amd64.zip	Windows	x86-64	ilogtail-1.6.0.windows-amd64.zip.sha256

Docker Image

Docker Pull Command

docker pull sls-opensource-registry.cn-shanghai.cr.aliyuncs.com/ilogtail-community-edition/ilogtail:1.6.0