debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin #1492

gpapaz · 2024-04-08T14:40:32Z

Describe the bug

All tags after the version of v1.16.3-debian-kafka2-2.0, i.e. v1.16.3-debian-kafka2-2.0, v1.16.3-debian-kafka2-2.1 and v1.16.5-debian-kafka2-1.0, suffer from this issue. They are unable to correctly start the rdkafka output plugin for fluentd.

In version fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.0 when the container is started with the configuration mentione in the 'To reproduce' section the below errors can be seen:

fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:226:in `block (2 levels) in native_config'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:216:in `each'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:216:in `block in native_config'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 <internal:kernel>:90:in `tap'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:214:in `native_config'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:178:in `producer'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:387:in `block in get_producer'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:384:in `synchronize'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:384:in `get_producer'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:421:in `write'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin/output.rb:1225:in `try_flush'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin/output.rb:1538:in `flush_thread_run'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin/output.rb:510:in `block (2 levels) in start'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
fluentd-1  | 2024-04-08 13:54:08 +0000 [warn]: #2 Send exception occurred: Configuration property "ssl.ca.location" not supported in this build: OpenSSL not available at build time at /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:226:in `block (2 levels) in native_config'

A bit of checking shows that the libssl-dev is not available any more in the container.
This is the result of from fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-1.0

# apt list | grep ssl 
libssl-dev/now 1.1.1w-0+deb11u1 amd64 [installed,local]
libssl1.1/now 1.1.1w-0+deb11u1 amd64 [installed,local]
openssl/now 1.1.1w-0+deb11u1 amd64 [installed,local]

where everything seems to work as expected.

The result from the fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.0 is

# apt list | grep ssl 
libssl3/now 3.0.11-1~deb12u2 amd64 [installed,local]
openssl/now 3.0.11-1~deb12u2 amd64 [installed,local]

You can see that libssl-dev is missing.

On fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.1 the error changes and becomes:

fluentd-1  | /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/dynamic_library.rb:65:in `load_library': Could not open library '/fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/../../ext/librdkafka.so': libsasl2.so.2: cannot open shared object file: No such file or directory. (LoadError)

fluentd-1  | Searched in <system library path>, /usr/lib, /usr/local/lib, /opt/local/lib

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:95:in `block in ffi_lib'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:94:in `map'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:94:in `ffi_lib'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:18:in `<module:Bindings>'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:7:in `<module:Rdkafka>'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:5:in `<top (required)>'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka.rb:9:in `<top (required)>'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:159:in `require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:159:in `rescue in require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:39:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:6:in `<top (required)>'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:103:in `block in search'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `each'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `search'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:44:in `lookup'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:169:in `new_impl'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:114:in `new_output'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:130:in `add_match'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:74:in `block in configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `each'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/root_agent.rb:149:in `configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:105:in `configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:80:in `run_configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/supervisor.rb:571:in `run_supervisor'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/command/fluentd.rb:352:in `<top (required)>'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/bin/fluentd:15:in `<top (required)>'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `load'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `<main>'

fluentd-1  | <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- rdkafka (LoadError)

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:6:in `<top (required)>'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:103:in `block in search'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `each'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `search'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:44:in `lookup'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:169:in `new_impl'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:114:in `new_output'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:130:in `add_match'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:74:in `block in configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `each'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/root_agent.rb:149:in `configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:105:in `configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:80:in `run_configure'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/supervisor.rb:571:in `run_supervisor'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/command/fluentd.rb:352:in `<top (required)>'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/bin/fluentd:15:in `<top (required)>'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `load'

fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `<main>'

Looking into the container we can see that libsasl2 is not present:

# ldd $(gem contents rdkafka | grep librdkafka.so)
        linux-vdso.so.1 (0x00007ffe842b7000)
        /usr/lib/libjemalloc.so.2 (0x00007ffad7d8b000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffad7caa000)
        libsasl2.so.2 => not found
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffad7ac9000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ffad8226000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ffad78af000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ffad788d000)

The same happens with the latest tag you provide as of today:
fluent/fluentd-kubernetes-daemonset:v1.16.5-debian-kafka2-1.0
Keep in mind that in both these last image the openssl-dev is not present, either.

To me it looks like this commit created this issue by purgin still necessary libraries:

fluentd-kubernetes-daemonset/templates/Dockerfile.erb

Line 52 in 29fdf03

'*-dev' \

To Reproduce

Use below configuration to start the fluentd container:

`
@type forward
port 29999
bind 0.0.0.0

@type rdkafka2 brokers "localhost:9092" default_topic "test_topic" ssl_ca_cert "/certs/test.crt" @type json `

Container using the above image tags, will create errors long before it tries to send data to the broker. The errors are described above per image tag.

Expected behavior

No errors are raised during the plugin's initialization.

Your Environment

- Tag of using fluentd-kubernetes-daemonset: 

fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.0
fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.1
fluent/fluentd-kubernetes-daemonset:v1.16.5-debian-kafka2-1.0

Your Configuration

<source>
    @type forward
    port 29999
    bind 0.0.0.0
</source>
<match>
    @type rdkafka2
    brokers "localhost:9092"
    default_topic "test_topic"
    ssl_ca_cert "/certs/test.crt"
    <format>
      @type json
    </format>
</match>

Your Error Log

fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.0

fluentd-1  | 2024-04-08 13:54:08 +0000 [warn]: #1 failed to flush the buffer. retry_times=3 next_retry_time=2024-04-08 13:54:16 +0000 chunk="615961fb1f6b351762de7dee1a9625a7" error_class=Rdkafka::Config::ConfigError error="Configuration property \"ssl.ca.location\" not supported in this build: OpenSSL not available at build time"
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:226:in `block (2 levels) in native_config'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:216:in `each'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:216:in `block in native_config'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 <internal:kernel>:90:in `tap'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:214:in `native_config'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:178:in `producer'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:387:in `block in get_producer'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:384:in `synchronize'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:384:in `get_producer'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:421:in `write'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin/output.rb:1225:in `try_flush'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin/output.rb:1538:in `flush_thread_run'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin/output.rb:510:in `block (2 levels) in start'
fluentd-1  |   2024-04-08 13:54:08 +0000 [warn]: #1 /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
fluentd-1  | 2024-04-08 13:54:08 +0000 [warn]: #2 Send exception occurred: Configuration property "ssl.ca.location" not supported in this build: OpenSSL not available at build time at /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/config.rb:226:in `block (2 levels) in native_config'

fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-kafka2-2.1
fluentd-1  | /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/dynamic_library.rb:65:in `load_library': Could not open library '/fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/../../ext/librdkafka.so': libsasl2.so.2: cannot open shared object file: No such file or directory. (LoadError)
fluentd-1  | Searched in <system library path>, /usr/lib, /usr/local/lib, /opt/local/lib
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:95:in `block in ffi_lib'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:94:in `map'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:94:in `ffi_lib'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:18:in `<module:Bindings>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:7:in `<module:Rdkafka>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:5:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka.rb:9:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:159:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:159:in `rescue in require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:39:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:6:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:103:in `block in search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:44:in `lookup'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:169:in `new_impl'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:114:in `new_output'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:130:in `add_match'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:74:in `block in configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/root_agent.rb:149:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:105:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:80:in `run_configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/supervisor.rb:571:in `run_supervisor'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/bin/fluentd:15:in `<top (required)>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `load'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `<main>'
fluentd-1  | <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- rdkafka (LoadError)
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:6:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:103:in `block in search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in `search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:44:in `lookup'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:169:in `new_impl'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:114:in `new_output'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:130:in `add_match'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:74:in `block in configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/root_agent.rb:149:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:105:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:80:in `run_configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/supervisor.rb:571:in `run_supervisor'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.3/bin/fluentd:15:in `<top (required)>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `load'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `<main>'

fluent/fluentd-kubernetes-daemonset:v1.16.5-debian-kafka2-1.0
fluentd-1  | /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/dynamic_library.rb:65:in `load_library': Could not open library '/fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/../../ext/librdkafka.so': libsasl2.so.2: cannot open shared object file: No such file or directory. (LoadError)
fluentd-1  | Searched in <system library path>, /usr/lib, /usr/local/lib, /opt/local/lib
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:95:in `block in ffi_lib'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:94:in `map'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/ffi-1.16.3/lib/ffi/library.rb:94:in `ffi_lib'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:18:in `<module:Bindings>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:7:in `<module:Rdkafka>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka/bindings.rb:5:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/rdkafka-0.11.1/lib/rdkafka.rb:9:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:40:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:6:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:103:in `block in search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:100:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:100:in `search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:44:in `lookup'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/plugin.rb:169:in `new_impl'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/plugin.rb:114:in `new_output'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:130:in `add_match'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:74:in `block in configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:64:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:64:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/root_agent.rb:149:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/engine.rb:105:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/engine.rb:80:in `run_configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/supervisor.rb:571:in `run_supervisor'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/bin/fluentd:15:in `<top (required)>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `load'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `<main>'
fluentd-1  | <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require': cannot load such file -- rdkafka (LoadError)
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb:6:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:103:in `block in search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:100:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:100:in `search'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/registry.rb:44:in `lookup'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/plugin.rb:169:in `new_impl'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/plugin.rb:114:in `new_output'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:130:in `add_match'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:74:in `block in configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:64:in `each'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/agent.rb:64:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/root_agent.rb:149:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/engine.rb:105:in `configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/engine.rb:80:in `run_configure'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/supervisor.rb:571:in `run_supervisor'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/gems/fluentd-1.16.5/bin/fluentd:15:in `<top (required)>'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `load'
fluentd-1  |    from /fluentd/vendor/bundle/ruby/3.2.0/bin/fluentd:25:in `<main>'

Additional context

No response

The text was updated successfully, but these errors were encountered:

gpapaz added the waiting-for-triage label Apr 8, 2024

gpapaz changed the title ~~debian-kafka2 after v1.16.3-debian-kafka2-2.0 fails to start kafka output plugin~~ debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin Apr 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin #1492

debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin #1492

gpapaz commented Apr 8, 2024

debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin #1492

debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin #1492

Comments

gpapaz commented Apr 8, 2024

Describe the bug

To Reproduce

Expected behavior

Your Environment

Your Configuration

Your Error Log

Additional context