Welcome to ChangeFormer Discussions!

[wgcban]

👋 Welcome!

Welcome to the discussion section for our project. If you have any questions, comments, or feedback related to our work, we would love to hear from you.

Please use this space to discuss any issues, ideas, or suggestions you may have regarding our project. We encourage open and constructive dialogue, and we look forward to hearing your thoughts.

As a reminder, please keep all discussions respectful and relevant to the project. Let's work together to create a productive and collaborative environment for all.

Thank you for your participation and we look forward to hearing from you.

[bskelly]

Hello

Firstly congrats on ChangeFormer! Apologies for cross posting as I also sent an email. I was unsure of the best way to reach out.

We are attempting to use your ChangeFormer model as part of a comparison in a paper we are preparing.

ChangeFormer works reasonably well on our medical imaging data but when i train the SOTA models (starting with pertaining on LEVIR) I cannot get the models to converge. After 1 epoch of training they predict a blank mask, get a validation accuracy of 0.499 and do not improve with further training. The change is small and subtle relative to the data used in your paper.

However since changeformer can learn to identify change i am curious to see if you have any tips to get the SOTA models to work.

I tried implementing a dice loss and a combined dice/ce loss but that did not help

Thanks in advance

[bskelly]

Hello,

just for competition i was able to solve my issue using a weighted dice/bce loss weighted towards the minority class

thanks so much to @wgcban for helping and for replying to the discussion

Best

BSK

[wgcban]

@bskelly Are you training the ChangeFormer model on your medical imaging data? Hmm, if it is not converging, did you try it with the multi-scale loss>

[bskelly]

Hi

Changeformer coverages quickly with ce loss on my data but none of the SOTA models will.

I have tried all the built in loss functions in the changeformer repo and also implemented a dice loss and combined dice bce loss.

It's strange that the sota models train successfully on levir but not on my data whereas changeformer converges using the same pipeline so was just wondering if you had any tips or common errors?

Much appreciated

[bskelly]

i wonder if the issue could be with the pretrained models, i think i am taking them correctly from the drop box but the SOTA pretrained models are in the oder of 12MB (60 for the basic ViT) and the changeformer in about 460MB. does that sound right?

[wgcban]

for ViTs which pre-trained weights are you using? Mae pre-trained? I prefer to use MAE pre-trained weights for MAEs: https://github.com/MCG-NJU/VideoMAE/blob/main/MODEL_ZOO.md

[bskelly]

Sorry for emailing and replying on git

Thanks so much for your advice.

When I run changeformer pretrained on levir it performs quite well on my medical imaging data

When I run the other models in the changeformer repo (unet Siamese unet basic ViT etc) also pretrained on levir they do not converge.

I will try the multiscale loss and revert

Regards

Brendan

[wgcban]

Also, ViTs needs more data and I am wondering the amount of fine-tuning data available?

[bskelly]

we have about 4000 pairs of images for fine tuning

[Yolo-huyuan]

Hello, could you please share the link about ChangeFormer again: https://www.dropbox.com/s/undtrlxiz7bkag5/pretrained_changeformer.pt? It seems to be deleted. Thank you！

[wgcban]

@bskelly @Yolo-huyuan I have removed pre-trained models from the DropBox. However, they are still available for you to download through GitHub Release