Fix some unit tests

[martindiphoorn]

No description provided.

[benoitx]

Thanks for this Martin. It is greatly appreciated.

What do you think should happen to 'invalid' lines?
e.g. "hello,world

At the moment it will parse as [hello,world], single column, no double quote...

and this one:
"col1",col2",col3 returns col1 col2" col3

AAARRRRGGG

[benoitx]

My thinking is: leave the parsing of the line as is BUT have the DataSet parser detect if we have an ODD number of Qualifiers (e.g. " or | or whatever). It is the number is odd -> Do not parse the line and raise an error, the line is malformed. What do you think?

[martindiphoorn]

If i use a library like flatpack i would be happy that this decision is left to the user of the library. So I would make it a setting. Something like failOnError setting and skipLineOnErrror.

failOnError=true: Stop processing and raise an exception
failOnError=false: Check the skipLineOnError value
skipLineOnError=true: Skip the line if there is an error in it and continue
skipLineOnError=false: Do our best to get the right value

Just an idea, it gives the user some flexibility. And every use case is different.

Some examples:

col1,"col2",col3       // good
col1,"co""l2",col3    // good
"col1,col2,col3"       // valid, its just one column
"col1"",""col2,col3"  // valid, its just one column
"col1,coll2,col3       // invalid, when starting with qualifier it should end also  with it
col1",col2,col3        // invalid, end but no starting
co""l1,col2,col3       // invalid, if qualifiers are escaped the field should be wrapped with it

The big question is, what should be the best effort result?

CSV lint i was talking about is here:
https://csvlint.io/

[martindiphoorn]

Just thinking, should we create an seperate unit test for checking RFC compatibility?

https://www.ietf.org/rfc/rfc4180.txt

[benoitx]

Done: issue #36 #36

…

On 18 June 2018 at 18:07, Martin Diphoorn ***@***.***> wrote: Just thinking, should we create an seperate unit test for checking RFC compatibility? https://www.ietf.org/rfc/rfc4180.txt — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#35 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAPeaV6hfzmllE548Aw0FQxGVKEJ8VjRks5t995HgaJpZM4UrT2m> .

[benoitx]

Hi Martin
Thinking of doing a release but do you know how to build the Maven site with TRAVIS CI? Getting stuck on this... Thanks
Benoit

[martindiphoorn]

Hi Benout,

i never used TRAVIS CI. I have experience with Gitlab CI.
If you can give me some pointers. I can look in to it.

Martin