Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does segment support splitting Traditional Chinese into words? #8

Open
ShawTim opened this issue Dec 26, 2020 · 1 comment
Open

Does segment support splitting Traditional Chinese into words? #8

ShawTim opened this issue Dec 26, 2020 · 1 comment

Comments

@ShawTim
Copy link

ShawTim commented Dec 26, 2020

as title.

or, do i need to convert it into simplified chinese first, and then convert it back?

@peterolson
Copy link
Owner

It probably won't work very well for traditional characters because the segmentation library used (jieba) is trained on simplified texts. For now you'll probably have to convert to simplified first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants