-
Notifications
You must be signed in to change notification settings - Fork 5.5k
RetinaNet:workspace blob weight shape is not matched with predtrained model weights shape on custom dataset #97
Comments
I think this is because your new dataset has a different number of categories. The "cls_pred" (and also "bbox_reg") layers are class-dependent, so are their shapes. If you load a pre-trained Detectron model and fine-tune on another dataset, you may either change the name of these related layers during fine-tuning, or not load these weight blobs when initializing. |
@KaimingHe Sorry for asking, but can you explain the fine-tuning process further? I understand that I have to change the number of classes in the /configs/pre-trained-network.yaml and have to modify the output layers of the network. My assumed steps for training on own data:
Now I'm at the third step and need some help. In my understanding of Detectron, the models are defined by a configuration file (.yaml) and a pkl file with their weights. I can edit the .yaml, but how can I change the name of the output related layers in the pkl file for the fine-tuning? It's a binary file, isn't it? Thank's for any help! |
@mattifrind We have not exposed this as a config option. Basically, what you may try is to revise the lines in https://github.com/facebookresearch/Detectron/blob/master/lib/utils/net.py#L88, for example, removing those class-dependent layers the dst list. |
@KaimingHe Sorry I didn't get it. I have to ask again...
That changed nothing. The demo still runs without problems and my dataset with just 2 categories and the corresponding .yaml file (demo file with edited NUM_CLASSES: 2 and Dataset) still throws this error: |
Hi @mattifrind I have a idea to use the pre-trained model and fine-tune it on your own dataset without removing the class-dependent layers. Suppose you have a dataset with 3 categories which all the categories not list in COCO 2014. B.T.W. In our experience, it shows that pre-trained model is very important for getting a good performance. So you might not abandon the pre-trained model and retrain a new one from the very beginning. Now I have a problem that how can I transfer the label information which is represented by the coordinate into COCO like format like this: |
@SniperZhao Thanks! It isn't really the beautiful way because I don't need the 77 other classes, but that should work. Have you tested that? I don't want to abandon the idea of a pre-trained model. I thought that for example the demo config.yaml loads a .pkl file with pre-trained weights except the weights from the output layers (Detectron prints out "not found"). So if I run train_net.py I keep the whole model (except the ones that aren't found) and fine-tune the last layers according to my new task. Am I wrong? I'm sorry but I'm not sure if I understand your last problem correctly. Do you have a problem with converting your data? I used Java with a JSON Library so I could easily bring the data into the needed format. |
@mattifrind Yes! You are right, it's not beautiful. But I am not so sure the other 77 classes would not help. Some paper said that multi-class detection task has better performance than single class detection. May be other classes can also offer some more feature information for modeling the target that you interested. But, yes, it's another story and not a perfect solution for your question. Apparently you read more code of Detectron than me did. So I can not answer your question about the code now. I'll debug the code later and wish we can figure it out. And I'm pretty sure that your theory about fine-tune is correct. For my question, I have a dataset which all the labeling samples have the bounding box coordinate in integer numbers. However, the COCO's labeling information has floating number as the coordinate. I don't know the corresponding relationship between them. So I can't transfer the data into right format. And I also don't know the meaning of segmentation which a list of floating numbers in COCO annotations. |
Ok maybe the multi class detection helps. I'm greatful for any solution. |
I was able to finetune of RetinaNet (model X-101-64x4d-FPN_2x) on my own dataset as follows:
This got my network training, with a steadily decreasing overall loss. I started out with 0's for the individual FPN focal loss values but they gradually increased after training for a few k iterations or so. |
@djr2015 Have you done the fine tuning part on the mask-rcnn? I have followed your steps (1-3), but in the mask-rcnn paper the initialisation of the layers is not clearly explained. |
Hi @GabriellaP thank you! |
When I am trying to train retinanet model(X-101-32x8d-FPN) on my own dataset, I am getting the below error.
AssertionError: Workspace blob retnet_cls_pred_fpn3_w with shape (72, 256, 3, 3) does not match weights file shape (720, 256, 3, 3)
Any solution?
The text was updated successfully, but these errors were encountered: