Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an operator for dumping schemas #4147

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

dominiklohmann
Copy link
Member

This is pretty helpful to get started with writing schemas, as it allows one to go from inferred events over to having a manually specified schema without needing to write everything from scratch.

The operator name is still a TBD and this definitely needs documentation iff we want to keep this. This originated in a discussion in the community Discord, where a new user suggested this workflow.

This is pretty helpful to get started with writing schemas, as it allows
one to go from inferred events over to having a manually specified
schema without needing to write everything from scratch.

The operator name is still a TBD and this definitely needs documentation
_iff_ we want to keep this. This originated in a discussion in the
community Discord, where a new user suggested this workflow.
@dominiklohmann dominiklohmann added feature New functionality operator Source, transformation, and sink labels Apr 23, 2024
Copy link
Member

@lava lava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I applied this locally and verified that it works as expected.

However, the string format is a bit awkward to use, in order to do anything useful with the schema afterwards (ie. search for the presence of a field with some specific type) one has to resort to substring search.

I think it would be worth considering to turn the definition in a nested field, or as a compromise maybe in JSON format so that it can be easily postprocessed by other tools.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality operator Source, transformation, and sink
Projects
None yet
2 participants