模型维度问题

[terminator123]

gpt2模型config里面n_positions=513，会报
size mismatch for transformer.h.0.attn.bias: copying a param with shape torch.Size([1, 1, 512, 512]) from checkpoint, the shape in current model is torch.Size([1, 1, 513, 513]).

改成512后，如果use_gpt2=True，会报
size mismatch for transformer.wpe.weight: copying a param with shape torch.Size([513, 768]) from checkpoint, the shape in current model is torch.Size([512, 768]).

[pppihf]

我也遇到这种情况了，请问怎么解决

[terminator123]

不知道呀，你要是解决了告诉我下

[zhaojunGUO]

我也是，怎么解决哇

[YuChuXi]

tokenizer = tokenizer_class.from_pretrained(checkpoint, do_lower_case=False)  
model = model_class.from_pretrained(checkpoint)  

两句中加入参数 ignore_mismatched_sizes=True 然后忽略报错就好

[YuChuXi]

不用动模型的配置文件