You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been attempting to reproduce the ablation study on CoQA dev set. I've tried:
perform experiment on GraphFlow model(2-His) and got 78.7% f1 score(78.3% in paper).
set "use_bert"=False in config file, the performance dropped significantly to around 70% (which is actually 76.0% in paper).
set "use_bert"=True and remove RGNN, the performance dropped slightly, which is also inconsistent with the result present in paper.
The question is that whether set "use_bert"=False means removing bert embeddings? If it is, is there anything else I should do to reproduce these results? Any advice would be appreciated.
Thanks,
Nan
The text was updated successfully, but these errors were encountered:
Hi there,
I've been attempting to reproduce the ablation study on CoQA dev set. I've tried:
The question is that whether set "use_bert"=False means removing bert embeddings? If it is, is there anything else I should do to reproduce these results? Any advice would be appreciated.
Thanks,
Nan
The text was updated successfully, but these errors were encountered: