Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NullPointer Exception using more Threads than documents #1030

Closed
dr0i opened this issue Oct 4, 2019 · 2 comments
Closed

NullPointer Exception using more Threads than documents #1030

dr0i opened this issue Oct 4, 2019 · 2 comments
Assignees
@dr0i
Projects
lobid board

Comments

@dr0i
Copy link
Member

dr0i commented Oct 4, 2019 •
edited

Experienced this when shrinking the hbz01XmlClobs.tar.bz2 test set to one, while there are two threads to compute the data. When calling closeStream(), a NullPointer Exception appears. This stops the ETL process finishing correctly, and thus no resource (aka document) is serialized to the disk.
A workaround is to catch the Exception.
However, as this is a problem can occur everywhere, it would be nicer to somehow control this at the modules in metafacture-core, probably at the ObjectPipeDecoupler. Also, to say the least, it's rather unexpected to use more threads than documents, it may happen.
@dr0i dr0i self-assigned this Oct 4, 2019
@dr0i dr0i added this to Backlog in lobid board via automation Oct 4, 2019
dr0i added a commit that referenced this issue Oct 4, 2019
@dr0i
Catch NullPointer
c5548ee 
If there are more threads than resources to be computed "closeStream()"
interrupts the computation of all resources. Catching the Exception is
enough to make it finishing correctly.

See #1030.
()
@fsteeg fsteeg mentioned this issue Oct 7, 2019
Merged
@acka47
Copy link
Contributor

acka47 commented Sep 16, 2022

@dr0i Does this problem still exist? Shouldn't we open issue over at metafacture-core then and close this one?

@dr0i dr0i mentioned this issue Sep 16, 2022
Open
@dr0i
Copy link
Member Author

dr0i commented Sep 16, 2022

Problem still exists. Opened metafacture/metafacture-core#466. Closing this one.

@dr0i dr0i closed this as completed Sep 16, 2022
lobid board automation moved this from Backlog to Done Sep 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dr0i dr0i

Labels
None yet
Projects
lobid board
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@acka47 @dr0i