Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue about BLEU smoothing method 4 #2676

Closed
OpenSource-fan opened this issue Mar 24, 2021 · 3 comments
Closed

Issue about BLEU smoothing method 4 #2676

OpenSource-fan opened this issue Mar 24, 2021 · 3 comments
Assignees

Comments

@OpenSource-fan
Copy link
Contributor

NLTK3.5 implements smoothing method 4 proposed in the paper [https://www.aclweb.org/anthology/W14-3346/ ].
However, we suspect that the implementation is incorrect. The code at

incvnt = i + 1 * self.k / math.log(
hyp_len
) # Note that this K is different from the K from NIST.
p_n[i] = incvnt / p_i.denominator
is problematic.
This leads to the p_n[i] greater than 100% in some case. When hyp_len < 4 in 4-gram, p_n[i] can be assigned with a percentage number that is much greater than 100% (or even > 500%).
In fact, the p_n[i] should not be greater than 100%.
The correct implementation is p_n[i] = (1 / incvnt) / p_i.denominator in line585.

@stevenbird
Copy link
Member

Thanks @anonymityBoy. Would you please submit a pull request?

@stevenbird
Copy link
Member

Thanks @anonymityBoy... can I trouble you to add a regression test or two here please?

@OpenSource-fan
Copy link
Contributor Author

Sure. I have added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants