Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyfasttext predict sometimes return empty result #20

Open
nguyenhoan1988 opened this issue Oct 28, 2017 · 2 comments
Open

pyfasttext predict sometimes return empty result #20

nguyenhoan1988 opened this issue Oct 28, 2017 · 2 comments

Comments

@nguyenhoan1988
Copy link

When I use the pyfasttext with a same model, to predict this:
dwell dwelluk -> [u'company']
Joshua Katz joshuakatz4 -> [u'male']
meredith mears mered148r -> [u'female']
Shenia Pellum SheniaPellum -> [u'female']
Matt Inama mattinama -> [u'male']
maureen russell mopruss -> [u'male']
RickLeeJames RickLeeJames -> []
Lauren Townsend laurenttownsend -> [u'female']
Andy Campbell lordsnarfdog -> [u'male']
Mark Newham mhn56 -> [u'male']
Ryan DatBirdbrain -> [u'male']
MaryAnn maryannstewart1 -> [u'female']
Career & Business HustleNaija -> [u'company']
Kevin A. Thompson kevinathompson -> [u'male']
Kevin Aston AstonFarms -> [u'male']

while if I use the command line, the result will be:
__label__female
__label__male
__label__female
__label__female
__label__male
__label__female
__label__female --> missing in the above
__label__female
__label__male
__label__male
__label__male
__label__female
__label__company
__label__male
__label__male

So the results from pyfasttext and command line are different and missing (in case of pyfasttext)

Do you have any idea why it's different?

@vrasneur
Copy link
Owner

Hi @nguyenhoan1988
pyfasttext uses the same source code as the fastText binary (with a thin Python wrapper).

So, there should be no difference... But fastText adds a newline at the end of each sentence, so you may have to add '\n' manually to your lines.

If that does not work, can you upload your model?

@nguyenhoan1988
Copy link
Author

@vrasneur thank you so much
I fixed the problem by adding '\n' to every single line before calling predict
Maybe you should add this warning to the README or automatically add '\n' when predict is called

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants