Training Asreview: what to do with 'maybe' articles? #1451
Replies: 5 comments 8 replies
-
Hi @Hanedinges, |
Beta Was this translation helpful? Give feedback.
-
Hi @Hanedinges , in addition to the useful comments of @rohitgarud: assuming you're going through multiple stages of screening, at some point these 'maybe's' need to be addressed. To not 'confuse' the algorithm and assuming the maybe-records are very similar in text to included ones, the logical option would be to include and add a comment. |
Beta Was this translation helpful? Give feedback.
-
Your query is indeed very insightful. In the context of active learning, making a definite decision on 'maybe' articles is crucial to optimizing the machine learning model's performance in ASReview. While it can seem counterintuitive, the model actually learns the most from these 'borderline' or 'unclear' articles. The reason for this is that by making a clear decision (include/exclude) on these articles, you help refine the model's understanding of the nuances of what constitutes a relevant or irrelevant article for your specific review. This in turn helps improve the model's predictions for future articles, thereby enhancing the efficiency of your review process. Therefore, if your goal is to maximally enhance the performance of ASReview, making a final decision on these articles is highly suggested. |
Beta Was this translation helpful? Give feedback.
-
Sorry for my late reply guys! Your advice and explanation of the logic behind them are highly appreciated, thanks! @FelixWdm wrote: Ideally with the reasons for excluding per record as well. |
Beta Was this translation helpful? Give feedback.
-
A PR has been opened for adding the option to add tags to the review screen: #1527. This functionality can be used to label |
Beta Was this translation helpful? Give feedback.
-
Dear all,
Even with the best list of eligibility criteria, doubts can arise about the (ir)relevance of an article. What to do in such a situation? In Rayyan for example, it is possible to use the label 'maybe', add a reason for it and discuss it when all articles have been screened. However, AsReview doesn't have this feature.
What to with 'maybe' articles? I see three options, but there might better ones of course;)
Which option would be the best and why? Is there a guideline that I might have missed, a useful source?
What are 'maybe' articles called in offical review-terms (so I can use it as a key word in my search for help)
I'm also trying to think of the consequences of all options in terms of reliability of the review, if anyone is able to explain this it would be great!
Thanks a lot!
Simone
Beta Was this translation helpful? Give feedback.
All reactions