Integration with AWS Glue Schema Registry

[albanovito]

Hello guys,

I read the documentation but I not found nothing. Is it possible to use this library with AWS Glue schema registry?

Thanks

[kevinwallimann]

Hi @albanovito No, ABRiS is only designed to work with Confluent Schema Registry. Currently, we have no plans of adding support for AWS Glue Schema Registry, but if you would like to fork or contribute, I suggest you look at AbrisRegistryClient and see how you can adapt it for AWS Glue Schema Registry.

[kevinwallimann]

Hi @codeworm2. It is not possible to use AWS Glue Schema Registry with the current version of Abris, and we are currently not planning on implementing it. Abris is tightly integrated with the Confluent Schema Registry, because it uses its SchemaMetadata data structure internally (in SchemaManager, Config, possibly other classes as well)

If you would like to contribute to Abris, and implement the support for AWS Glue Schema Registry, you would have to create an abstraction around the SchemaMetadata data structure, and then provide implementations for both Confluent and AWS Glue Schema Registries. Additionally, you'd have to provide an implementation of the AbrisRegistryClient that supports AWS Glue Schema Registry.

[codeworm2]

Hi @kevinwallimann, Yes, my teammate and I want to contribute to ABRiS for the integration to AWS Glue Schema Registry. I branched the release 6.1.1 for compatibility with our production EMR environment.

I agree that we need to abstract SchemaManager, and change some other code. Regarding the timeline, we can finish in Feb, hopefully. Would you mind if we have weekly zoom meeting to sync on the code chagnes? You can give me an email account, then I will invite you.

[kevinwallimann]

Hi @codeworm2
Thank you for your invitation. I will happily review your pull request, but I think weekly zoom meetings are not necessary at this point.

[codeworm2]

Hi @kevinwallimann, our initial integration PoC works internally at AWS, but is not on prod yet. We will provide the diff in March. Sorry for the delay.

[Roytjoo]

@codeworm2 Any update on your integration with AWS Glue Schema Registry?