Breaking change in 3.5.1

[staktrace]

Hi,

I upgraded kafkactl in my project from 3.4.0 to 3.5.1 and this command that I was running started to break:

kafkactl produce orc_test_input \
    -H "Content-Type: application/json" \
    -H "ce_specversion:1.0" \
    -H "ce_id:00001" \
    -H "ce_source:test-source" \
    -H "ce_type:type" \
    -H "ce_time:2021-08-26T00:24:10.01383Z" \
    -H "ce_partitionkey:partition-key" \
    -H "ce_datacontenttype:application/json" \
    -v '{"value": 42}'

Previously it would work fine, but after upgrading it emits this:

Failed to open Kafka producer: kafka: client has run out of available brokers to talk to: EOF

I bisected kafkactl and it appears to be coming from the dependency upgrade for github.com/IBM/sarama from v1.40.1 to v1.42.1

[staktrace]

Seems to be because of the breaking changes in sarama v1.41.0 (link). Not sure if you want to roll back to the old sarama or are ok with having breaking changes here.

[staktrace]

Also FWIW we're using the vectorized/redpanda:v22.3.11 docker image as a kafka implementation in this scenario.

[d-rk]

Hi @staktrace

thank you for the detailed analysis.
So in the end the reason for this is an incorrect implementation of the kafka api from redpanda?
or can this be reproduced with a "real" kafka?
If not I guess redpanda will have to fix this at some point and you could use kafkactl 3.4.0 for the time being.

[staktrace]

Hi @d-rk ,

I'm not sure honestly. We can't run our code against a "real" kafka and I'm not familiar with the API details to say. I did notice we were on an older version of redpanda than what's currently the latest, so I'll try upgrading that and see if that helps. But yeah for now I'm staying on kafkactl 3.4.0.

[staktrace]

So for what it's worth, even using the latest redpanda image (redpandadata/redpanda:v23.2.17) I get the same error.