Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data transformation node #76

Open
lalitpagaria opened this issue Apr 4, 2021 · 3 comments
Open

Data transformation node #76

lalitpagaria opened this issue Apr 4, 2021 · 3 comments

Comments

@lalitpagaria
Copy link
Collaborator

Idea to have a node which transform list of data/dict/json to one format to another format.
Ideally it can be used as data merging, and conversion purpose as well.

@lalitpagaria lalitpagaria added enhancement New feature or request medium priority labels Apr 4, 2021
@lalitpagaria lalitpagaria removed the enhancement New feature or request label May 27, 2021
@shahrukhx01
Copy link
Collaborator

@lalitpagaria could you please explain this a little more with couple of examples?

@lalitpagaria
Copy link
Collaborator Author

Let me come back to you.

@lalitpagaria
Copy link
Collaborator Author

When I originally created this ticket I had this type of conversions in my mind -
https://github.com/lalitpagaria/obsei/blob/master/obsei/sink/dailyget_sink.py#L22

But now with current state it can be made more generic and powerful.

In future now plan to have following nodes as well (these will sit between analyzer and sink) -

  • Routing/Splitter (For example if sentiment positive send to slack sink otherwise to ticketing system)
  • Transformation (To update, delete, modify, and add payload of AnalyzerResponse)
  • Merger/Joiner/Batcher/Aggregator (To merge multiple AnalyzerResponse)

Still idea if rough. So open for suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants