How could use?

[p00uya]

Hi, I'm very interested in using your code in my project. I was able to run demo scripts. Now I want to use your code for my own data, but unfortunately I do not know how I can do this on my data. I put them in files A, B But I think I need guidance to test
Please tell me how I can find a difference for my images
(I am a newcomer, thank you)

[wgcban]

Hi @p00uya,

Thank you for your interest in our work and welcome!

The easiest way to get predictions for your samples is to copy and paste your pre-change (A) and post-change (B) images to one of the samples_DSIFN or samples_LEVIR. Once you copied your images to a folder, update the demo.txt in the list folder which is placed inside the respective folder with your image names and run the corresponding demo file. The prediction will be saved in samples/predict_LEVIR'' or samples/predict_LEVIR''.

Please note that our model is trained on remote sensing images and it may not give correct predictions if you are testing it with images from a separate domain. In that case, I recommend you to tune our model for your dataset to get better results. To do that, you need to follow the training procedure mentioned in the latter part of the readme file. Please let me know if you need any further help and I'm happy to help you!

[p00uya]

Ok, I did this the name of img was t1.png , I got the following error
FileNotFoundError: [Errno 2] No such file or directory: './samples_LEVIR/label/t1.png'
Question: Is it like giving input and taking an output like object recognition tasks? So why ask for a label for each image?

[wgcban]

@p00uya the ground truth change mask is used to compute the performance metrics such as IoU and accuracy. Since you don’t have the ground truth labels, place a dummy black png there with same size as your pre-change and post-change imgaes. Or you can comment those lines in the code too.

[p00uya]

@wgcban
I did, but unfortunately the prediction is not correct
in second way would you please let me know where should I comment?

[wgcban]

Hi @p00uya,
Are you testing on the remote sensing data or some other images?
If your test images are different from the images that we used to train the model, we cannot expect good predictions from the model.
Also, please note that the LEVIR-CD is a building change detection dataset, so if you used a pre-trained model on the LEVIR-CD dataset, it will only detect building changes.
Can you clarify the above things because it will help me to get an idea about your project?
Also, if possible, please post a sample pre-change and post-change image here.

[p00uya]

Hi @wgcban thanks to replaying quickly
this my data, consist of A,B,label
the label size is same as A , B image

[wgcban]

Hi @p00uya
Please give me access to download the data.

[wgcban]

Hi @p00uya ,
I got the predictions for your data using the model trained on the LEVIR-CD
dataset. The predictions seem reasonable to me except val_29. This behavior can be expected because you are testing on out of distribution data. If you want better results I recommend you to train our model on your data.
You can find the prediction here. p00uya_predictions_ChangeFormer.zip
Best,

Best,
Chaminda.

[p00uya]

Yes, you are right
I got the same results
Thank a lot

[yuanjian2022]

Hi,wgcban
I try to reproduce your code ChangeFormerV6 on DSIFN, I use ''LEVIER-CD\CD_ChangeFormerV6_LEVIR_b16_lr0.0001_adamw_train_test_200_linear_ce_multi_train_True_multi_infer_False_shuffle_AB_False_embed_dim_256\best_ckpt.pt'' to initialize the model, and I have some problems.
Did you use ''from models.pixel_shuffel_up import PS_UP'', but your ''Changeformer'' did not call PS_UP. Your pre-trained model parameter size is 492671152. I found that the model I reproduced from your code is 492670048. The result I reproduced is very poor on DSIFN.
Thank you very much.

[wgcban]

Hey @yuanjian2022,

You are using the wrong model to initialize. As the name infer "LEVIER-CD\CD_ChangeFormerV6_LEVIR_b16_lr0.0001_adamw_train_test_200_linear_ce_multi_train_True_multi_infer_False_shuffle_AB_False_embed_dim_256\best_ckpt.pt" is the model trained on LEVIR-CD and not on the DSIFN CD. You should use: DSIFN-CD-Pretrained-Model.

Please use the model trained on DSIFN-CD to initialize the model if you are trying to reproduce results on DSIFN-CD. I have linked pre-trained models for each dataset in the readme file. Please see: Quick Start on DSFIN dataset.

Or you can simply run the demo file for the DSIFN dataset.

Please let me know if you have any questions!

Best,
Chaminda.

[yuanjian2022]

Hello,Chaminda
I downloaded the pre-trained model CD_ChangeFormerV6_DSIFN_b16_lr0.00006_adamw_train_test_200_linear_ce_multi_train_True_multi_infer_False_shuffle_AB_False_embed_dim_256/ from https://github.com/wgcban/ChangeFormer#quick-start-on-dsfin-dataset,I first tried demo_DSIFN.py, and got the prediction graph and yours /yj/yjcode/ChangeFormer-main/samples_DSIFN/predict_ChangeFormerV6 consistent.
Then start training from the 182nd epoch in your log. The size of the generated model parameters is 492671152 and the downloaded best_ckpt.pt and last_ckpt.pt are the same size, and F1 and iou are also similar to yours.
But I started training from the first epoch, the model will call CD_ChangeFormerV6_DSIFN_b16_lr0.00006_adamw_train_test_200_linear_ce_multi_train_True_multi_infer_False_shuffle_AB_False_embed_dim_256/best_ckpt.pt to initialize the model, the generated new model parameter is 492670448 and the model does not converge.
I also tried to train directly, initialize the model randomly, namely 'training from scratch... ', the generated model parameters are also 492670448 and also do not converge.
I think it is the loss of newly generated model parameters that is causing the non-convergence.
Best,
JianYuan

[wgcban]

@yuanjian2022,

When training on the DSIFN dataset, I initialized the ChangeFormerV6 with the model trained onLEVIR-CD for faster convergence. Since we have trained models for both LEVIR-CD and WHU-CD, I recommend you to do the same when training on DSIFN or another dataset.
Having said that, I'm not exactly sure about how starting from the 182 epoch and starting from the 0th epoch create two different models with different numbers of parameters. Please allow me some time - I will think about this and update you. At the same time, if you find the solution please post here.
Did you try what happens when you initialize the model with a model trained on LEVIR-CD?

[yuanjian2022]

Hi,Chaminda
I tried to initialize with LEVIR's pretrained model when training DSIFN. The results are basically the same as initializing with the pretrained model of DSIFN.

[wgcban]

Nice!
Best
Chaminda

[WesleyZhang1991]

DSIFN-CD-Pretrained-Model
@yuanjian2022
Did you achieve similar results on DSIFN-CD as the log (https://github.com/wgcban/ChangeFormer/releases/download/v0.1.0/CD_ChangeFormerV6_DSIFN_b16_lr0.00006_adamw_train_test_200_linear_ce_multi_train_True_multi_infer_False_shuffle_AB_False_embed_dim_256.zip) after initializing with the LEVIR-CD model? Namely about 76% iou_1 for DSIFN-CD？ I am working to reproduce results for DSIFN-CD with our framework in #5 but still struggling.
Please let me know if you have new results on DSIFN-CD.
Sincerely

[yuanjian2022]

Sorry，No results similar to the log were obtained on the DSIFN dataset. What I mean above is that neither of the two pretrained models given by the author can make the model converge on the DSIFN dataset.
yuanjian

[wgcban]

Hey @yuanjian2022 @WesleyZhang1991
Do you follow the same dataset preparation procedure that I mentioned in the paper or readme file? Please note that instead of random cropping first, I create non-overlapping patches of size 256x256 from the DSIFN dataset and then train on those images? Could you please confirm this?

[WesleyZhang1991]

@wgcban
I am quite sure I follow the non-overlapping data preparation. Today I have checked the provided DSIFN model. iou_1 is about 76% and I think the results are correct based on the evaluation code and the training logs from this repo. However, when it comes to our framework, I have followed almost all the procedures from ChangeFormer, namely non-overlapping data preparation, Transformer encoder, multi-scale training, AdamW optimizer with the provided setting, fine-tuning from LEVIR-CD pretrained model. But still archives about 50% iou_1.

I am afraid there is some unmentioned data augmentations in the current code for DSIFN, e.g., disabled random rotation, only 0.8-1.2 random scale seems not enough for DSIFN

[wgcban]

Hi @WeslyZhang1991,
Thank you for the reply.
The results reported in the paper were produced from the current version of the code.
Instead of adapting my model in your framework, can you check if it gives the same results if you run this repository? I think it should give because I also used the same code. Perhaps you are missing something very small thing when you are adopting it to your framework? Also, as a side note, when I was training the model for DSIFN, I initialized it from the model trained for LEVIR-CD to get faster convergence. have you done this?

Best
Chaminda

[Mabel0403]

Hi wgcban!
I'm really interested in your project and trying to understand it. But I have a question with your ChangeFormerNets which return a parameter called "cp". I know it's a list which consists 5 tensors, but I don't understand what it means exactly. If I want to get the image reflecting change between A image and B image, should I use the fifth tensor(cp[4]) of the cp list?
(Actually, I am an beginner in deep learning. There's a lot for me to learn and I really want to understand the changeformer totally. I will appreciate it if you can answer my question! Thanks!)
Mabel

[wgcban]

Hi @Mabel2022,

Thank you for your interest in our work!

You are correct. The output from the transformer decoder cp consists of four tensors. It basically consists of outputs from the decoder at multiple spatial scales (HxW, H/2xW/2, H/4xW/4, and H/8xW/8) that will be used to compute multi-scale loss when multi_scale_train=True in the config file.

This is the place where we configure the multi-scale loss function in the config file:

ChangeFormer/scripts/run_ChangeFormer_LEVIR.sh

Line 23 in fbe2f9e

multi_scale_train=True

This is the place where we compute the multi-scale CD loss:

ChangeFormer/models/trainer.py

Lines 295 to 306 in fbe2f9e

	if self.multi_scale_train == "True":
	i = 0
	temp_loss = 0.0
	for pred in self.G_pred:
	if pred.size(2) != gt.size(2):
	temp_loss = temp_loss + self.weights[i]*self._pxl_loss(pred, F.interpolate(gt, size=pred.size(2), mode="nearest"))
	else:
	temp_loss = temp_loss + self.weights[i]*self._pxl_loss(pred, gt)
	i+=1
	self.G_loss = temp_loss
	else:
	self.G_loss = self._pxl_loss(self.G_pred[-1], gt)

The multi-scale loss function is commonly used in CD to improve the quality of predicted change maps at multiple scales. You can read more on the multi-scale loss function for CD here.

During the inference, we use the output prediction at the HxW scale ( i.e., cp[-1]) to get the predicted change map from our ChangeFormer if multi_scale_infer == "False".

If multi_scale_infer == "True", we upsample all the intermediate predictions from the network to the HxW scale and take the average as the final prediction.

This is the place where we get the final prediction from the network during the inference.

ChangeFormer/models/trainer.py

Lines 281 to 290 in fbe2f9e

	if self.multi_scale_infer == "True":
	self.G_final_pred = torch.zeros(self.G_pred[-1].size()).to(self.device)
	for pred in self.G_pred:
	if pred.size(2) != self.G_pred[-1].size(2):
	self.G_final_pred = self.G_final_pred + F.interpolate(pred, size=self.G_pred[-1].size(2), mode="nearest")
	else:
	self.G_final_pred = self.G_final_pred + pred
	self.G_final_pred = self.G_final_pred/len(self.G_pred)
	else:
	self.G_final_pred = self.G_pred[-1]

I hope this clarifies your question and I'm happy to help you if you have any more questions.

Best,
Chaminda

[Mabel0403]

Hi Chaminda！
It's so kind of you to answer my question carefully! Now I fully understand this question. Thanks for your time and attention on it. Hope you have a good day!
Best,
Mabel