why Resnet-50 channel pruning is worse with reconstruction ?

[zlheos]

I try to prune resnet-50

first, Merge batchnorm and conv , the precise is 92%
second, I set channel ration = 0.9 , I only prune before res2a-brach2c , but the precise drop to 65%
I don't know what is wrong ?

then , I only prune channel without reconstruction , prune before res2a-brach2c and the precise drop less, I don't know why is worse with reconstruction ?

Thank you for your answer

[Johnson-yue]

@zlheos hi , I see you are also insterested in ResNet pruning, so am I. Do you successful? should I run add_bn() first?

[Toory465]

Any news about pruning resNet-18? I have been trying to prune it for sometimes, but I could not get better that 80% accuracy for 10 percent compression.

[bbjy]

@Toory465 @zlheos Could you please tell me how you can apply the channel pruning without 3c? I don't know how to delete the spatial decompose and the channel decompose. Thank you so much!

[Toory465]

@bbjy for resNet I only applied it on 8 filters of all net( basically 2 out of 5 conv layers of each residual block. I guess you need 3c for the filters that their input our outputs are affected by residual shortcut. However, I also tried to use 3c , prune filters from other layers and reconstruct error with linear regression which was not worked in my case.

[bbjy]

@Toory465 Actually, I donnot want to use 3c , I only want to use the channel pruning method in my case, but I don't know how to delet the other two in the code. And would you please tell me is there any modification in your cfgs.py ?Thank you so much!

[bbjy]

@zlheos Have you already pruned the resnet-50? I am very interested in it, and I am trying to prune it, but failed.

[zlheos]

@bbjy I try to prune resnet-50 with only cp, but the result will be worse
but VGG-16 with cp is good
I guess the reason is batchnorm layer, now I cannot give the conclusion

[bbjy]

@zlheos Ok,thank you.

[zlheos]

@bbjy , have you try to cp resnet-50 ?

[bbjy]

I tried, but failed.

[bbjy]

I tried, but failed.@zlheos

[JinyangGuo]

@zlheos @bbjy @Toory465 Have you succeed to prune resnet-56? I encounter same problem. The accuracy drops a lot when I just reconstruct the weights without prune the channel. I guess the problem is in the extract_feature?