Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spelling correction mostly is not working #20

Open
stas00 opened this issue Mar 14, 2020 · 0 comments
Open

spelling correction mostly is not working #20

stas00 opened this issue Mar 14, 2020 · 0 comments

Comments

@stas00
Copy link

stas00 commented Mar 14, 2020

Came to this project for spelling in twitter text, but it doesn't quite work most of the time.

  1. spell correction seems to only work when annotate is set as in the example. Now
    take the same example and set annotate={} and spell correction is gone:
i saw the new john doe movie and it suuuuucks ! ! ! waisted <money> . . . bad movies <annoyed>

if I restore annotate={"hashtag", "...}, then it corrects suuuuucks to sucks
I'm not sure what is the connection between annotations and spell correction.

  1. spelling-correction doesn't work in general. Again, going back to your pipeline example, change the first input sentence to inject some spelling errors: CANT WAIT for the neww seaason of #TwinPeaks , run it, you get:
    cant wait for the neww seaason of twin peaks - i.e. no spell correction.
    The spell_correct_elong doesn't seem to make a difference.

Yet, if I run:

from ekphrasis.classes.spellcorrect import SpellCorrector
sp = SpellCorrector(corpus="english") 
print([sp.correct(x) for x in "neww seaason".split()])

It corrects: ['new', 'season']

@stas00 stas00 changed the title spelling correction mostly not working spelling correction mostly is not working Mar 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant