Support YAML input

[gf-mse]

Just discovered this project ( which is an awesome idea, btw ); I wonder if it supports | yaml -- and if not, then how hard it would be to add that ?

[rcoh]

It should be easy to add -- just follow the pattern of the JSON operator. Currently deeply nested structures aren't well supported, neither is accessing them -- you'd need to hack around that for now the same way JSON does, I think.

But when I add better support for nested structures YAML will get it too

[gf-mse]

My sincere apologies -- I meant a YAML parser. ( YAML output probably comes as a very natural next step, though )

[rcoh]

After a bit more thought, this may be quite tricky. Currently, things operate on a line-by-line basis. You'd need to somehow flatten your YAML structure onto a single line or significantly refactor that way input works.

[gf-mse]

So for json input, everything is flattened to one line first? That is, we need a plugin that converts yaml input to a sequence of json-formatted lines .. what I suppose could be an external program. Ok, I get it. Thank you.

[rcoh]

json input assumes that the entire JSON record is on one line (which is the case for most logging applications)

[rcoh]

@gf-mse are your yaml documents separated by --- or something similar? It would be easy to allow changing the document separator from \n to another string like that.

[gf-mse]

No, but it could be achieved. Worse comes to worse, I can pre-parse it and add a --- on every de-indentation, or next top-level list/dictionary Item.

What actually makes me think if I shall open another issue and ask whether there are plans for some plugin mechanism )

[rcoh]

I doubt I'd add a plugin mechanism -- the CLI is such that it's generally better to `|` from one program to another to handle more specific task. I'm considering handling indentation grouping of records, but it adds considerable complexity to the ingest pipeline which is currently very simple

…

On Sat, Jul 6, 2019 at 11:38 AM Georgy (George) Fedorov < ***@***.***> wrote: No, but it could be achieved. Worse comes to worse, I can pre-parse it and add a --- on every de-indentation, or next top-level list/dictionary Item. What actually makes me think if I shall open another issue and ask whether there are plans for some plugin mechanism ) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70?email_source=notifications&email_token=AADYKZZ2ISBXTBGHC6UI6ZDP6DRCJA5CNFSM4H24RQAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZK6M5I#issuecomment-508946037>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADYKZ65IZ2DDVWXJGLUYZLP6DRCJANCNFSM4H24RQAA> .