-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DATAGO-74463: Add Dynamic Producer Destination to the binder #39839
Open
Aryelmr
wants to merge
2
commits into
Azure:main
Choose a base branch
from
Aryelmr:DATAGO-74463-Add-Dynamic-Producer-Destionation
base: main
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering why this entity type is missing from the producer properties retrieved from line 271, the getProducerPropertiesSupplier?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's because the key to return the extended properties is the destination name. When it uses the Dynamic Destination name to pull the properties to create the producer it's not going to find it. Right here:
Permalink
So it'll create a default
ProducerProperty
and if thedefaultEntityType
is set, it'll set theentityType
. As thedefaultEntityType
is not set, then it'll fail to publish to the dynamic destination. That's why in there I'm not setting theentityType
and setting thedefaultEntityType
in case the destination is dynamic is used and no ProducerProperty is register in its name.I've tested in our internal connector manually and on integration test and it works fine when
defaultEntityType
is set.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, thanks. So a better way to set the type is probably to use another header.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you help add some comments around this code? Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@saragluna
So you mean you would like to add a new producer property so the user can define in the YAML what would be the defaultEntityType?
About the code, do you mean adding some comments in the code or just in the PR so it makes easier to understand? If it's the comments in the code, wouldn't be better some Java Docs instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the update, the other way to set the entity type via the header I was talking about is exactly like this
setHeader(SolaceBinderHeaders.TARGET_DESTINATION_TYPE, "topic")
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try the NewDestinationBindingCallback to see what causes the issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Aryelmr, I've created this project https://github.com/saragluna/demo-servicebus-binder-dynamic-destinations to test the dynamic destinations. In my test, there are two ways to configure the default entity type:
spring.cloud.stream.servicebus.default.producer.entityType
, see https://github.com/saragluna/demo-servicebus-binder-dynamic-destinations/blob/main/src/main/resources/application.propertiesCould you help look to see whether these two approaches work in your case? If so, I think we don't need to make changes in this PR now. Thanks in advance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@saragluna
Thank you for coming up with those alternatives to see if we could get the same results with the code changes :)
Unfortunately, they didn't work.
On
DefaultServiceBusNamespaceProducerFactory:createProducer
it just creates a plain ProducerProperties so it doesn't get theExtendendProducerProperties
to get the default values and the NewDestinationBindingCallback didn't work as well.For the
setHeader(SolaceBinderHeaders.TARGET_DESTINATION_TYPE, "topic")
Solace approach is nice.In the same way as the destination we could check if the EntityType header is set and if so pass the value. E.g.:
What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Aryelmr, thanks. I like the header approach, and it looks better. But I am still wondering what your dynamic destination code looks like, and why it goes to the
DefaultServiceBusNamespaceProducerFactory:createProducer
? Could you help provide some code, so I could take a look.