Does ASReview generate relevance predictions based on prior knowledge all at once?

[quazhuo]

I'm curious about whether ASReview performs relevancy ranking calculations only once, periodically, or every time a reviewer labels a record. How does ASReview deal with the computational cost?

If a new model is being trained every time a reviewer labels a citation, each labeling decision then will influence the model that ASReview uses to make further suggestions. Occasionally, researchers may mistakenly label a relevant publication as irrelevant or vice versa. I know that by using ASReview you can just undo it and correct the labeling, but I wonder if the model has already been generated and influenced by the previous decisions.

[PeterLombaers]

After every labeling decision ASReview trains a new model and uses that to calculate new relevance scores. The only caveat is that ASReview trains only one model at the same time. So if you label very quickly, or the model takes a long time to train, then the relevance scores will not have been updated yet when you get a new record. See also https://doi.org/10.31234/osf.io/g93zf for a description of the different steps in the active learning process.

So indeed, if there are mistakes in the labeling decisions this will influence the model. After the mistake you might see different records than you would have seen if you gave the correct label. After you correct the mistake the new models will use the new corrected label.

[quazhuo]

Thank you so much!