Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When parsing a CSV that has headers with 2 or more matching names, data is lost #224

Open
MichaelFoss opened this issue Feb 13, 2023 · 1 comment

Comments

@MichaelFoss
Copy link

  • Operating System: macOS 13.2
  • Node Version: 14.15.4
  • NPM Version: 6.14.0
  • csv-parser Version: 3.0.0

Expected Behavior

The parsed data should be maintained in an array of strings instead of a single string.

Actual Behavior

Only the last read value is maintained.

How Do We Reproduce?

  1. Create a .csv file with at least columns with the same name and one row with different data for each
  2. Try to parse the data without losing data
@MichaelFoss
Copy link
Author

MichaelFoss commented Feb 13, 2023

I recommend having an option that allows changing behavior in the event of duplicate headers. Something like useArraysForDuplicateHeaders as a boolean flag that defaults to false; this way it can maintain the existing behavior, with the use of arrays for duplicated headers being a feature.

Consider the simple file:

a,a,b
1,2,3

The row object in the data event's function looks like this:

{ a: '2', b: '3' }

What I'd like to see, when this flag is enabled in the options, is this:

{ a: [ '1', '2' ], b: '3' }

Line 185 of index.js reads as follows:

o[header] = cell

This change to the writeRow function on line 185 will allow it to work as expected:

if (Array.isArray(o[header])) {
  o[header].push(cell)
} else if (o.hasOwnProperty(header)) {
  o[header] = [o[header], cell]
} else {
  o[header] = cell
}

The issue with this approach, of course, are the extra operations, which may cause a performance hit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant