Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This model has order 20 but KenLM was compiled to support up to 6 #5482

Open
imMonicaShaw opened this issue Apr 18, 2024 · 0 comments
Open

Comments

@imMonicaShaw
Copy link

What is your question?

Hi guys,When I execute the following command using the Lao 20gram language model in MMS,the following error occurs:
RuntimeError: /root/anaconda3/envs/meta/temp_dir/kenlm/lm/model.cc:49 in void lm::ngram::detail::{anonymous}::CheckCounts(const std::vector&) threw FormatLoadException because `counts.size() > 6'.
This model has order 20 but KenLM was compiled to support up to 6. If your build system supports changing KENLM_MAX_ORDER, change it there and recompile. With cmake:
cmake -DKENLM_MAX_ORDER=10 ..
With Moses:
bjam --max-kenlm-order=10 -a
Otherwise, edit lm/max_order.hh.

Code

python /root/anaconda3/envs/meta/fairseq/examples/mms/asr/infer/mms_infer.py --model "/root/anaconda3/envs/meta/fairseq/models_new/mms1b_all.pt" --lang "lao" --audio "/root/anaconda3/envs/ASRtest/audio/test/lao.wav" --extra-infer-args "decoding.type=kenlm decoding.beam=500 decoding.beamsizetoken=50 decoding.lmweight=2.69 decoding.lmpath=/root/anaconda3/envs/meta/lmdecode/lao_char_20gram.bin decoding.lexicon= decoding.silweight=2.8 decoding.unitlm=True common_eval.post_process=none decoding.results_path='/root/anaconda3/envs/meta/fairseq/0417'"

What have you tried?

cmake -DKENLM_MAX_ORDER=10 .., and recompiled the kenLM
the issue remains unresolved. What should I do?

Additionally, on the webpage https://huggingface.co/facebook/mms-cclms, the download links for the Mandarin and Cantonese language models have turned into errors. Where else can I download them from?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant