Issues: huggingface/tokenizers
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
BPE Trainer doesn't respect the
vocab_size
parameter when dataset size is increased
#1514
opened Apr 25, 2024 by
Abhinay1997
Extended vocab tokenizer merging text into a single string without spaces while decoding
#1501
opened Apr 17, 2024 by
savanth14
Issue in installing rudalle on google colab, !pip install rudalle
#1500
opened Apr 17, 2024 by
deepanshh786
Deepseeker model completely loses performance after using tokenizer.add_tokens(special_tokens)
#1490
opened Apr 11, 2024 by
bin123apple
Discrepancy Between GitHub Release and NPM Package Version & Missing Dependencies
#1489
opened Apr 10, 2024 by
superBertBerg
Is it possible to pass a tokenizer from Python into Rust?
Stale
#1487
opened Apr 5, 2024 by
albertsgarde
cargo build
fails for python bindings when --locked
is passed for v0.15.1
and v0.15.2
#1477
opened Mar 22, 2024 by
CobaltCause
Previous Next
ProTip!
Type g i on any issue or pull request to go back to the issue listing page.