merge报错

[CHIHOQ]

RuntimeError: The size of tensor a (32000) must match the size of tensor b (32002) at non-singleton dimension 0 如何解决？

[hamishivi]

Hi, looks like you're trying to merge two models with different vocabulary (and thus embedding) sizes. This can happen when you've added tokens like a padding token to the vocabulary. I would recommend merging before adding vocab items if possible. If you only have access to a llama model with added vocab items, just removed them from the model before merging (either by manually slicing the embedding matrix, or via the huggingface resize method (which I think shooouuuld work).

[notoookay]

If you are using llama2 model and add pad_token, use this base_model.resize_token_embeddings(32002) for pretrained llama2 model, then load tokenizer of peft fine-tuned locally, it should work.