Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_pickle.UnpicklingError: invalid load key, 'v'. #94

Open
cuiding opened this issue May 15, 2022 · 3 comments
Open

_pickle.UnpicklingError: invalid load key, 'v'. #94

cuiding opened this issue May 15, 2022 · 3 comments

Comments

@cuiding
Copy link

cuiding commented May 15, 2022

在加载预训练模型时出现这个错误,请问这个错误是怎么回事?

@silverriver
Copy link
Collaborator

请您提供一下详情:

  1. transformer库的版本
  2. 加载的哪个模型
  3. 出错的指令

@PlaBerlin
Copy link

transformers 2.1.1

执行步骤:

  1. pip install -r requirements.txt
  2. git clone https://github.com/thu-coai/CDial-GPT.git
    3.cd CDial-GPT
  3. git clone https://huggingface.co/thu-coai/CDial-GPT_LCCC-large
  4. python train.py --pretrained --model_checkpoint ./CDial-GPT_LCCC-large --data_path data/data_toy.json --scheduler linear

报错内容:
INFO:transformers.modeling_utils:loading weights file ./CDial-GPT_LCCC-large/pytorch_model.bin
Traceback (most recent call last):
File "train.py", line 238, in
train()
File "train.py", line 104, in train
model = model_class.from_pretrained(args.model_checkpoint)
File "/home/zkti/anaconda3/envs/gpt/lib/python3.7/site-packages/transformers/modeling_utils.py", line 345, in from_pretrained
state_dict = torch.load(resolved_archive_file, map_location='cpu')
File "/home/zkti/anaconda3/envs/gpt/lib/python3.7/site-packages/torch/serialization.py", line 529, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/zkti/anaconda3/envs/gpt/lib/python3.7/site-packages/torch/serialization.py", line 692, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

@AlexKai1
Copy link

“_pickle.UnpicklingError: invalid load key, 'v'.”
这个报错是因为预训练模型没有下载完整,建议参考下面这位老师的解决方案。
#74 (comment)

ps,
git下载大文件需要安装git lfs插件,参考以下文档进行安装
https://www.jianshu.com/p/493b81544f80

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants