Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] : Streaming gets stopped and goes to "end" event if row encounters "numerics at the end of the string" like "something_10". #826

Open
1 task
rajatsinghcms opened this issue Jun 5, 2023 · 1 comment
Assignees
Labels

Comments

@rajatsinghcms
Copy link

Describe the bug
Streaming gets stopped and goes to "end" event if row encounters "numerics at the end of the string" of a row like "value","other_values","something_10".
Now at for this row, it will stop.
Parsing or Formatting?

  • Formatting
  • [*] Parsing

To Reproduce
Steps to reproduce the behavior:

  1. Example file contents if applicable
  2. Example code to reproduce the issue.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: - MacOS/Windows
  • OS Version -Mojave
  • Node Version - 12

Additional context
Add any other context about the problem here.

@rajatsinghcms
Copy link
Author

return csvStream
.on("error", error => {
logger.info("Errored at error event", {
error: JSON.stringify(error.message)
});
reject(error);
})
.on("data", async row => {
csvStream.pause();
try {
arrayData[csvIndex % CHUNK_SIZE] = processEachRecordWithThrottle(throttle, row, csvIndex, processOneRecord);
csvIndex += 1;
if (csvIndex % CHUNK_SIZE === 0) {
await Promise.all(arrayData).catch(e => {
logger.info("error_occured_in_parallelLimitChunkExecutor", { e: e.message });
});
for (let i = 0; i < CHUNK_SIZE; i++) {
arrayData[i] = 0;
}
}
} catch (e) {
logger.info("error_occured_in_parallelLimitChunkExecutor_at_data_event_at_catch", {
error: e.message,
stack: e.stack
});
reject(e);
}
csvStream.resume();
})
.on("end", async rowCount => {
try {
await Promise.all(arrayData).catch(e => {
logger.info("error_occured_in_parallelLimitChunkExecutor_at_end", { e: e.message });
});
for (let i = 0; i < CHUNK_SIZE; i++) {
arrayData[i] = 0;
}
logger.info(Executed ${rowCount} number of rows.);
csvStream.destroy();
resolve(rowCount);
} catch (e) {
logger.info(Errored at execution at end stage. Terminating the process., {
error: e.message,
stack: e.stack
});
csvStream.destroy();
reject(e);
}
});

@rajatsinghcms rajatsinghcms reopened this Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants