-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Warning: Empty candidate sentence detected; setting raw BERTscores to 0. #18
Comments
Hi, ideally, you shouldn't get neither the Because in the first case, it means the model generated empty sentences (i.e. empty strings) for each region, which were concatenated into an empty report. I.e. the model is not trained well. And in the second case, it means the reference report was empty, which shouldn't happen since we only used non-empty findings sections as reference reports. I.e. somehow there are empty reference reports in your test set. Looking at your 2nd screenshot, it seems like the model you evaluated was not trained well (or maybe not trained enough yet). Usually, it's normal that it will generate gibberish in the beginning of the training, but it should generate coherent sentences towards the end of its training. |
I get the same issue when trying to run the I am using the
Specifically I get this stacktrace
Has there been any work on this issue yet? Addendum: when using the exact same framework, but calling the |
Hi,
While running test_set_evaluation.py, I am getting the warning message
Warning: Empty candidate sentence detected; setting raw BERTscores to 0.
Do you recommend to continue generating reports and BERTscores?
Here is the snippet of the warning message
The text was updated successfully, but these errors were encountered: