[add]: Added try/catches to primitive parsing methods #127
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Handle invalid primitives in json:
Problem we experience quite often:
Looking at the code and exception stack trace,
and relevant codes
MapOperator.java:process
my understanding is that currently there is no way to catch these exceptions via configuration options (e.g. mapreduce.job.skiprecords).
The only thing that seems viable is to catch these kind of problems using try/catch within methods of ParsePrimitiveUtils.
I understand this is probably the ugliest solution and a better pattern might be to pre-validate json, but unfortunately this doesn't seem to be a feasible situation in our case.
I will be pleased if you merge this PR, however I understand that this is not the ideal situation and you might not accept it.
In any case, I am open to your suggestions regarding this.