Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Reactor Kafka channel adapter #3850

Open
rolkhas2 opened this issue Jul 16, 2022 · 4 comments
Open

[Feature Request] Reactor Kafka channel adapter #3850

rolkhas2 opened this issue Jul 16, 2022 · 4 comments

Comments

@rolkhas2
Copy link

Expected Behavior
I'd expect there to be full Apache Kafka support since this is one of the most popular technologies, but for some reason, I didn't see any reactive Kafka channel adapter. That prevents me from using the adapter. I also didn't manage to understand if there's an async Kafka channel adapter, so if there is I think that can get documented better.

Current Behavior
Only Spring-Kafka support. No relation with Reactor-Kafka.

Context
I'm trying to build a simple data pipeline in the Reactive Spring Integration way.

@rolkhas2 rolkhas2 added status: waiting-for-triage The issue need to be evaluated and its future decided type: enhancement labels Jul 16, 2022
@artembilan artembilan added in: Kafka and removed status: waiting-for-triage The issue need to be evaluated and its future decided labels Jul 16, 2022
@artembilan artembilan added this to the Backlog milestone Jul 16, 2022
@artembilan
Copy link
Member

It is possible at the moment with a combination of Reactor Kafka and MessageProducerSupport.subscribeToPublisher(Publisher<? extends Message<?>> publisher): https://docs.spring.io/spring-integration/docs/current/reference/html/reactive-streams.html#event-driven-channel-adapter.
Or there is a nice trick via ReactiveMessageSourceProducer and existing KafkaMessageSource. See the same doc.

We definitely will look into reactive channel adapters for Apache Kafka somewhere in the future.

@artembilan
Copy link
Member

BTW, the logic for Reactor Kafka channel adapters really might be borrowed from a new Reactor Kafka binder in Spring Cloud Stream: https://github.com/spring-cloud/spring-cloud-stream/blob/main/binders/kafka-binder/spring-cloud-stream-binder-kafka-reactive/src/main/java/org/springframework/cloud/stream/binder/reactorkafka/ReactorKafkaBinder.java

@rolkhas2
Copy link
Author

@artembilan yes you're right. That's pretty simple to create such binders (shouldn't take more than a couple of hours). I'll try to implement that later.

@rolkhas2
Copy link
Author

Super improtant though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants