Notwithstanding --skip-comments, double-quoted tokens in comment are mishandled with any --c2* option

[sphakka]

I'm possibly missing something, but can't get mlr (v6.12.0-dev) to work with comments containing double quotes:

$ echo -e '# Foo "bar" quux'  | mlr --skip-comments  --c2m cat
mlr: parse error on line 1, column 7: bare " in non-quoted-field.

Shouldn't it ignore the comment line altogether?

[aborruso]

Hi @sphakka I have your same results.

A way to have an output is to change the input format in csvlite, but it may not be suitable for your CSV input.

echo -e '# Foo "bar" quux\na,b\n1,2' | mlr --skip-comments --icsvlite --omarkdown cat

a	b
1	2

[sphakka]

Thanks for the workaround, @aborruso!

[johnkerl]

Shouldn't it ignore the comment line altogether?

@sphakka this is related to #1346

In Miller <= 5 (written in C) I had my own CSV parser and I could make it do the right thing.

In Miller 6 (re-written in Go) I use the Go library's CSV parser. Unfortunately, it parses CSV liines as CSV and then lets me decide whether or not they look like comments. This isn't great.

I either need to port my own CSV parser from C to Go and stop using the Go library's CSV parser, or, maybe find some way to intervene in what it's doing ....