How to interpret the model fine tuning on the pre-trained ViT model using the imagery with larger resolution (500 * 500) than the pre-trained dataset (224 * 224)

[lyfranki]

Hi @cdpierse ,

I am working on the downstream task of image classification. I fine tuned a pre-trained ViT model (224 * 224) using larger resoltuion imagery (500 * 500). In the process of fine tuning, I only need to set interpolate_pos_encoding as True in model(**inputs, interpolate_pos_encoding = True) to fine tune the pre-trained model. However, it seems like your code does not include the part of interpolate_pos_encoding in the class of ImageClassificationExplainer. I am wondering what do you think about the attention map of the image classified by my fine tuned model according to your understanding to ViT. Is it much different from the attention map of the image classified by a model fine tuned from the pre-trained ViT model using the same larger resolution imagery? (In this process of fine tuning, imagery is resized to 224 * 224.) Or they would be same?

Thanks,
Yin