Experiment with approaches to view Record-heterogeneity in csv files. (ragged csvs)

[alexhallam]

Here I am using the term "ragged csv" like miller.

In a standard csv, if a cell is missing ,NA, it is common to just omit the value, but retain the commas ,,. @lithiumfrost uploaded a "ragged csv" where the omitted data came with no commas --- or any delimiter. See line 361 in the below image

It was mentioned that miller has an option to work with ragged csvs.

#75 (comment)

This is an open issue to think about how tv should work with these types of files.

Proposal

Since tv is based on pillar I lean on the shoulders of giants and see what the creators of the fantastic GNU-R pillar library decided to do. There are two components:

1. Truncated warnings when parsing
2. A pretty print of readable records

Here is the output.

Original Test file:
en_climate_hourly_AB_3012209_05-2021_P1H.csv

[alexhallam]

Next steps:

1. see if I can wrap the reading of csv in a error type.
2. Pass the error and continue reading.
3. Print the errors (truncated if more than 5 -- add all errors to current debug option). Print the readable data.

[alexhallam]

@lithiumfrost I wanted to keep you in the loop. I am currently leaning heavily to the above proposal. This will close #75