Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to modify the code to calculate factscore between any two paragraphs/sentences without retrieval steps? #32

Open
laiviet opened this issue Nov 21, 2023 · 1 comment

Comments

@laiviet
Copy link

laiviet commented Nov 21, 2023

I have a golden paragraph that doesn't need to be retrieved from any source.
How to bypass all unnecessary retrieval steps to calculate the FactScore for a predicted sentence..

Thank you in advance.

@shmsw25
Copy link
Owner

shmsw25 commented Nov 22, 2023

Hi @laiviet, thanks for your interest with our work!!

In this case, I think you need to modify some code in factscorer.py - but it shouldn't be too complicated! What you need to do is

  • In the main function, get_score, change L170 so that _get_score will receive another argument called context.
  • Also change L209 to receive an argument context, and then, comment out L215-219 so that it won't use retrieval to obtain context and instead just use the context given as an argument.
  • Optionally, you can also comment out L111-117 as then you can bypass setting up the retrieval corpus.

Let me know if you have any further questions! =)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants