You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
It's impossible to set empty for the kafka "key" field. This is annoying, since key is supposed to be a bytefield and can happily be left unset to avoid specific broker allocation.
To Reproduce
Configure a job to run on the kafka executor with the following executor config as follows: 'executor_config': {'brokerAddress': 'brokerURL:9092', 'key': '', 'message': '{}', 'topic': 'queue'},
Then try each of the following:
Leave key out of the configuration.
Set the key to null (in the json field).
Set the key to the empty string.
The empty string at least gets past the point of API ingeestion and I guess it sort of works sort of as intended, although using an empty string for this field makes absolutely no sense in any way shape or form so setting it as such could be used as a workaround to at least leave it unset.
Describe the bug
It's impossible to set empty for the kafka "key" field. This is annoying, since key is supposed to be a bytefield and can happily be left unset to avoid specific broker allocation.
To Reproduce
Configure a job to run on the kafka executor with the following executor config as follows:
'executor_config': {'brokerAddress': 'brokerURL:9092', 'key': '', 'message': '{}', 'topic': 'queue'},
Then try each of the following:
The empty string at least gets past the point of API ingeestion and I guess it sort of works sort of as intended, although using an empty string for this field makes absolutely no sense in any way shape or form so setting it as such could be used as a workaround to at least leave it unset.
Expected behavior
Using the example python kafka consumer from https://kafka-python.readthedocs.io/en/master/usage.html (modified for broker url and message) and kafka's own topic producer like so:
echo {} | /opt/kafka/bin/kafka-console-producer.sh --topic aer01_findity.job.cronEvent --bootstrap-server
produces the following message in the consumer:
queue:1:0: key=None value=b'{}'
I.e., key remains unset.
Screenshots
Using the same kafka consumer but consuming a message from the dkron kafka executor
queue:5:5: key=b'' value=b'{}'
I.e., key is set to the null string instead of being left as NULL
** Specifications:**
The text was updated successfully, but these errors were encountered: