Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing Field Value error when using POCO #284

Open
JimF42 opened this issue Apr 11, 2023 · 8 comments
Open

Missing Field Value error when using POCO #284

JimF42 opened this issue Apr 11, 2023 · 8 comments

Comments

@JimF42
Copy link

JimF42 commented Apr 11, 2023

I am new to ChoETL, so maybe I am doing something wrong. I have CSV that I am trying to read with a header. I have created a class and I have decorated it with ChoCSVFileHeader and numerous iterations of ChoCSVRecordObject along with decorating the individual properties with ChoCSVRecordField(FieldName = "...")

If I run this with new ChoCSVReader<VlanRecord>(@"<my file name>.csv") in my for loop, I will get a consistent error reporting that I have missing field values for one particular field. If I open the same file up in Excel, everything lines up correctly and there are values in the columns I am looking for (I don't need or want all the columns from this system generated CSV).

As a test, I tried removing unneeded columns to see if some unusually long values were throwing something off, but the only thing different is that the "missing" field changes. I even eventually removed all of the columns except the ones I needed (not practical in an on-going basis, but fine for testing), and, again, I get the same results, just a different field value is called out as "missing".

If, as an additional test, I run this into a dynamic object instead of a POCO, it all runs fine. No errors. But I wanted a POCO for better support for the rest of my application.

I'm not sure what I am doing wrong or how to diagnose this any further.

I have not uploaded the file as it would have to be heavily sanitized to be able to do that. If required, I can do that for a couple of lines (happens right away after the header).

I have updated to version 1.2.1.57 and I still see the issue.

c# 11 and .NET 7 running on Windows 10 64-bit.

@Cinchoo
Copy link
Owner

Cinchoo commented Apr 11, 2023

pls post sample csv with POCO class. Or use dotnetfiddle to share sample.

@JimF42
Copy link
Author

JimF42 commented Apr 12, 2023

@Cinchoo, try this sample. Thanks
ChoEtlSample.zip

@Cinchoo
Copy link
Owner

Cinchoo commented Apr 12, 2023

ok, got it. csv comes with quoted values including header. In order to handle it, use QuoteAllFields = true in ChoCSVRecordObjectAttribute

    [ChoCSVFileHeader,
     ChoCSVRecordObject(ErrorMode = ChoErrorMode.ReportAndContinue, ThrowAndStopOnMissingField = true, QuoteAllFields = true)]

@JimF42
Copy link
Author

JimF42 commented Apr 12, 2023

Interesting. Yup, that works.

One more question though. I have opened this file in Excel and re-saved it and Excel removed all of the quotes (header and data). Why does it still only work unless I use QuoteAllFields = true?

FYI, your documentation on Code Project says that QuoteAllFields doesn't apply to Reader. Apparently, it does apply.
Listing 6.1, the bullets below the code, "QuoteAllFields - N/A for reader."

Thanks for the fix though.

@Cinchoo
Copy link
Owner

Cinchoo commented Apr 12, 2023

I'll fix the document. thx for ur input.

Excel, I'm not sure about it.

@JimF42
Copy link
Author

JimF42 commented Apr 12, 2023

As far as Excel, it isn't specifically an Excel question. It was just the tool that I used to read in the CSV and resaved it. When it resaved it the raw .csv file no longer had even one quote in it (when opened in Notepad++). But, to get the program to work, I still need to use QuoteAllFields=true.

As long as I have this knowledge to make it work, I'm fine with it. It just seemed strange that even with zero double quotes in the file, I still need to use that property.

@Cinchoo
Copy link
Owner

Cinchoo commented Apr 19, 2023

if the csv header comes with quotes, you must use this flag to get it work. Pls share the csv when u can.

@JimF42
Copy link
Author

JimF42 commented Apr 25, 2023

ChoEtlSample.zip

Here you go. Not a single double quote in the entire file. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants