Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

我用A40推理过程中报错, RecursionError: maximum recursion depth exceeded,显示栈溢出,下面是完整报错信息,请问是什么问题呢 #99

Open
yihp opened this issue Oct 25, 2023 · 4 comments

Comments

@yihp
Copy link

yihp commented Oct 25, 2023

File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/root/miniconda3/envs/huatuo/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
RecursionError: maximum recursion depth exceeded

@galaxy2017
Copy link

galaxy2017 commented Oct 25, 2023

4090(24G)的也出现这个问题。我找了一下,是在infer.py里面这一句tokenizer = AutoTokenizer.from_pretrained(base_model)卡住了。在上述的这些报错之前还有几句,如图
wechat233
如果解决了问题,记得@一下我 : )

@galaxy2017
Copy link

问题已经解决,详情见:#86

@yihp
Copy link
Author

yihp commented Oct 30, 2023

感谢感谢!

@yihp
Copy link
Author

yihp commented Oct 30, 2023

@galaxy2017 兄弟可以加个好友吗?我这样还是报错啊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants