hypo.word file missing during MMS ASR inference #5117
Comments
|
Facing the exact same issue |
|
Hi, can you share the entire log? I just tested the code again and it works fine from my end. |
|
you need to check what the error is, change your out = subprocess.run(cmd, check=True, shell=True, stdout=subprocess.DEVNULL,)
print(out)to see the error, for me it was I needed to pass common:
cpu: true |
|
I am hitting this though and I am not sure what I am doing wrong, not sure if I am using the right lang_code, it doesn't say what the lang codes are or what standard it is referencing, I have tried 
|
|
Sure here is the full log of mine (base) hello_automate_ai@machinelearningnotebook:~/fairseqmmstest/fairseq$ python "examples/mms/asr/infer/mms_infer.py" --model "/home/hello_automate_ai/fairseqmmstest/mms1b_all.pt" --lang hin --audio "/home/hello_automate_ai/fairseqmmstest/audio.wav" |
|
This is after the fix suggested by audiolion |
|
@audiolion We expect a 3-digit language code. See 'Supported languages' section in README file for each model. |
|
@shsagnik |
you need to install the modules that are used |
|
Got these errors this time preparing tmp manifest dir ... During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): |
|
Getting pretty much the same, used the right 3 letter language code (while waiting on #5119 to be answered) and doesn't seem to have an effect, |
|
I got this error when i want to try ASR on google colab |
|
Please y'all read the error messages and try to debug yourself.
you need to install the missing modules, one of them being @altryne you need to print the error output to debug @shsagnik your hydra install has some issues, and you need to specify a checkpoint directory, it was setup to run on linux where you can make directories off the root (probably in a container) so change 
|
|
Thanks @audiolion Here's the full output I'm getting (added a print out of the cmd command as well) |
|
hi @audiolion, after installing omegaconf & hydra a new error appeared |
you need to do what I said in my first comment and output the process error message. the hyp.word file is not found because the actual ASR never ran and produced an output |
|
SIGH, I am, it prints the command and that's it. However, when I go back and recreate that temp dir, and run the command manually myself I do seem to get errors. Just for some reason not via the way you mentioned. Had to install many packages on the way, here's a partial list (in case it helps anyone) Still getting nowhere. Running the subprocess command even with |
|
Got the model to finally load and run, apparently windows doesn't allow : in directory names and the above code adds So if you pass I think the full inference ran, as the process got stuck for a few minutes, the GPU went to 8GB (impressive) and after a while, I had 2 errors again. the I'm currently staring at this error, and am pretty sure that's due to me removing the |
|
I had the same error with Google Colab and have investigated. my error As it turns out, it was crashing at the following location. fairseq/fairseq/models/wav2vec/wav2vec2.py Line 1052 in af12c9c Looking at the RAM status, I believe the crash was caused by lack of memory. So I feel that perhaps increasing the memory will solve the problem. I hope this helps you in your investigation. |
|
Getting same error. Also documentation to run sample is horrible. |
|
I would say it isn't a catch all error, but rather that error handling from the subprocess call is not done, so if the call to run the inference fails for any reason, the edit: @altryne my bad I thought by your message you were printing the command out itself, not the output of running the command. Your error does look like its failing because of the lack of |
Yeah, that's what I mean, if anything happens within the subprocess for any reason, folks are going to get the above mentioned error. Then they will likely google their way into this issue, which covers many of the possible ways it can fail.
Thanks! You helped a lot, I eventually had to rewrite that whole block like so: import os
os.environ["TMPDIR"] = str(tmpdir)
os.environ["PYTHONPATH"] = "."
os.environ["PREFIX"] = "INFER"
os.environ["HYDRA_FULL_ERROR"] = "1"
os.environ["USER"] = "micro"
cmd = f"""python examples/speech_recognition/new/infer.py -m --config-dir examples/mms/asr/config/ --config-name infer_common decoding.type=viterbi dataset.max_tokens=4000000 distributed_training.distributed_world_size=1 "common_eval.path='{args.model}'" task.data={tmpdir} dataset.gen_subset="{args.lang}" common_eval.post_process={args.format} decoding.results_path={tmpdir}
"""To even have the command execute and do something and not fail outright. |
|
glad you got it working! |
|
Hi, thanks for this discussion - I've learned a lot. This is the Dockerfile I created after a few hours trying to make it work: I built the image with: And run it with: |
|
I kept tracing error and solving them until i met this error: Does anyone know a solution ? |
i run the code based on the docker, but it fails again |
|
Same error. $ python examples/mms/asr/infer/mms_infer.py --model /idiap/temp/esarkar/cache/fairseq/mms1b_all.pt --lang shp --audio /idiap/temp/esarkar/Data/shipibo/downsampled_single_folder/short/shp-ROS-2022-03-14-2.1.wav
>>> preparing tmp manifest dir ...
>>> loading model & running inference ...
Traceback (most recent call last):
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/examples/speech_recognition/new/infer.py", line 21, in <module>
from examples.speech_recognition.new.decoders.decoder_config import (
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/examples/speech_recognition/__init__.py", line 1, in <module>
from . import criterions, models, tasks # noqa
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/examples/speech_recognition/criterions/__init__.py", line 15, in <module>
importlib.import_module(
File "/idiap/temp/esarkar/miniconda/envs/fairseq/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/examples/speech_recognition/criterions/cross_entropy_acc.py", line 13, in <module>
from fairseq import utils
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/fairseq/__init__.py", line 33, in <module>
import fairseq.criterions # noqa
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/fairseq/criterions/__init__.py", line 18, in <module>
(
TypeError: cannot unpack non-iterable NoneType object
Traceback (most recent call last):
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/examples/mms/asr/infer/mms_infer.py", line 52, in <module>
process(args)
File "/remote/idiap.svm/user.active/esarkar/speech/fairseq/examples/mms/asr/infer/mms_infer.py", line 44, in process
with open(tmpdir/"hypo.word") as fr:
FileNotFoundError: [Errno 2] No such file or directory: '/idiap/temp/esarkar/tmp/tmpnhi5rrui/hypo.word' |
|
Same issue. |
|
Ok for anyone who still have the Its really what the error says. :D you can see in the code python should already created tthe files dev.tsv, dev.uid, dev.ltr and dev.wrd in the same tmp folder. If you want to check this, simply change the
and you will see that those file will bne created. including the now the the and its again just a problem with permissions to write some files. so it trys to write in to the so now the script will have access to those folders and whrite down the inferece log (infer.log) in to that folder which includes the result of the ASR. |
I'm pretty sure I made the same changes and I still get the unpack error. I change the ENV vars before the cmd string + copied your entire cmd string. Maybe I'm missing something in infer_common.yaml or with running with args? (Windows paths do be scuffed) |
|
There is a problem with the mms1b_fl102.pt model, and the replacement model is mms1b_all.pt. |
I edited the script and it's now working for me, with an Ubuntu 22.04 image, tested with both CUDA 11.8 and 12.1.
# Also works with CUDA 12.1:
#FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04
FROM nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04
ENV DEBIAN_FRONTEND=noninteractive
WORKDIR /usr/src/app
RUN apt-get update \
&& apt-get install -y python-is-python3 git python3-pip sudo wget curl
RUN git clone https://github.com/facebookresearch/fairseq.git \
&& cd fairseq \
&& pip install pip -U \
&& pip install --no-cache-dir . \
&& pip install --no-cache-dir soundfile \
&& pip install --no-cache-dir torch \
&& pip install --no-cache-dir hydra-core \
&& pip install --no-cache-dir editdistance \
&& pip install --no-cache-dir soundfile \
&& pip install --no-cache-dir omegaconf \
&& pip install --no-cache-dir scikit-learn \
&& pip install --no-cache-dir tensorboardX \
&& python setup.py build_ext --inplace
ENV USERNAME=user
RUN echo "root:root" | chpasswd \
&& adduser --disabled-password --gecos "" "${USERNAME}" \
&& echo "${USERNAME}:${USERNAME}" | chpasswd \
&& echo "%${USERNAME} ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/${USERNAME} \
&& chmod 0440 /etc/sudoers.d/${USERNAME}
RUN mkdir -p /checkpoint/${USERNAME}/INFER \
&& chown -R ${USERNAME}:${USERNAME} /checkpoint/${USERNAME}
USER ${USERNAME}
WORKDIR /usr/src/app/fairseq
CMD [ "python", "examples/mms/asr/infer/mms_infer.py" ]Building with: docker build -t fairseq:dev -f Dockerfile.mms .Running with: docker run --rm -it --gpus all -e USER=user -v $(pwd):/mms:ro fairseq:dev python examples/mms/asr/infer/mms_infer.py --model /mms/examples/mms/mms1b_l1107.pt --lang fra --audio /mms/examples/mms/test16k.wav |
Worked for me thanks! For the ones not proficient with docker, just make sure to create on a directory where your docker file is located a directory /examples/mms and place your model and audio files in that directory. What this line |
|
Hi all, |
I get the following error after following all the steps: |
I just noticed that the MMS project is not included yet in the released version of pip uninstall fairseq && pip install git+https://github.com/facebookresearch/fairseqThe installation steps are updated accordingly. |
I have the following error now :( @abdeladim-s |
|
@bekarys0504, what model are you using ? I think you are using a wrong model! |
this one mms1b_fl102.pt downloaded through this link https://dl.fbaipublicfiles.com/mms/asr/mms1b_fl102.pt Should be the right one it is for ASR @abdeladim-s |
@bekarys0504 , yes it is a right model it seems. |
I did what you described and while it ran for 6 minutes, I get a "Killed" in the output with no other information. The RAM was basically maxed out throughout and there was no hypo.word not found error. The model is probably just too big to run on free Colab. |
|
BTW, it should now be very simple to use MMS with See: |
|
How much resources does it really take to run the l1107 model anyways? Because running it on colab maxed out 12GB of system RAM. Feels really overkill for a 10 second audio input. |
|
It takes less than 8GB with the code snippet of https://huggingface.co/facebook/mms-1b-all and can easily be run on CPU - give it a try ;-) |
That's good to know B), because even after tweaking the line where asr.py was supposed to do stuff to hypo.word it ran on Colab but after 6 minutes of maxing out the 12GB of RAM it was killed. The audio file wasn't even long, it was less than 10 seconds long. By the way I have yet to try it using 🤗 transformers, I'm referring to the colab notebook demoing ASR that's having trouble running. |
|
Here we go: https://colab.research.google.com/drive/1jqREwuNUn0SrzcVjh90JSLleSVEcx1BY?usp=sharing simple 4 cell colab |
|
I also found this error "hypo.word" on another machine (Ubuntu 20.04) while there is no problem in the other (Ubuntu 22.04). Actually there is an error before After updating Numpy (from 1.21.5 to 1.24.3) the error was gone and the the output of ASR is shown in the bottom. |
Yes you are right. Smaller model is working 😓 |
|
To fix this issue, add |
|
what files need to be changed in windows |
your project is perfect,but i want to know how to use my own voice to translate |
I have the same error as @shsagnik |
|
Sorry to bother you here. I am unable to run mms asr transcribe . I am using python3.11 and facing a range of issues from hypo.word not found, AttributeError: 'PosixPath' object has no attribute 'find' and what not. Going through the issues, there is no landing solutions except lot of comments.. There are just way too many threads relating to mms asr transcribe issues but no working solutions posted, if there is one set of installation instructions that actually work and are documented somewhere that be great. Here is my error:
/Users/spanta/Downloads/fairseq-main/examples/speech_recognition/new/infer.py:440: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=config_path, config_name="infer") Traceback (most recent call last): File "/Users/spanta/Downloads/fairseq-main/examples/speech_recognition/new/infer.py", line 499, in File "/Users/spanta/Downloads/fairseq-main/examples/speech_recognition/new/infer.py", line 495, in cli_main File "/opt/homebrew/lib/python3.11/site-packages/hydra/main.py", line 94, in decorated_main File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/utils.py", line 355, in _run_hydra File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/utils.py", line 223, in run_and_report File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/utils.py", line 220, in run_and_report File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/utils.py", line 356, in File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/hydra.py", line 61, in create_main_hydra2 File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/config_loader_impl.py", line 48, in init File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/config_repository.py", line 65, in init File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/config_repository.py", line 72, in initialize_sources File "/opt/homebrew/lib/python3.11/site-packages/hydra/_internal/config_repository.py", line 143, in _get_scheme AttributeError: 'PosixPath' object has no attribute 'find' 0 |
Hello, I output the error message according to your comment, and it printed the following error |
After I replaced the code, the error message output is as follows：
The complete log is as follows： |
and others
❓ Questions and Help
What is your question?
I'm facing the following issue while running the MMS ASR inference script
examples/mms/asr/infer/mms_infer.py:Code
What have you tried?
Tried running the ASR on different audios and languages
What's your environment?
pip, source): pipThe text was updated successfully, but these errors were encountered: