我在使用vit模型的时候，加载vit_base_patch16_224_in21k()模型的预训练权重，显示和某些层和预训练权重不符合，怎么解决呢

[tutoujun]

System information

• Have I written custom code:
• OS Platform(e.g., window10 or Linux Ubuntu 16.04):
• Python version:
• Deep learning framework and version(e.g., Tensorflow2.1 or Pytorch1.3):
• Use GPU or not:
• CUDA/cuDNN version(if you use GPU):
• The network you trained(e.g., Resnet34 network):

Describe the current behavior

Error info / logs

[tutoujun]

size mismatch for pos_embed: copying a param with shape torch.Size([1, 50, 768]) from checkpoint, the shape in current model is torch.Size([1, 197, 768]).
size mismatch for patch_embed.proj.weight: copying a param with shape torch.Size([768, 3, 32, 32]) from checkpoint, the shape in current model is torch.Size([768, 3, 16, 16]).
就是这两个层尺寸不匹配