Only 3x3 conv can use 2:4 sparsity?

[WeixiangXu]

I use nsys to profile the resnet34.

And I find that only 3x3 conv use 2:4 sparsity, while 1x1 conv does not. (Furthermore, I find that Linear in transformers also does not use sparsity.)

Why?

[nvpohanh]

TRT tries both sparse and dense tactics and choose a faster one. Based on our experiments, sparse conv kernels are faster than dense conv kernels if C and K are large enough (>256). Could you try increase C and K for the 1x1 Conv and see if sparse conv tactic is chosen?

3x3 Conv effectively increases the C by 9x so it favors sparse kernels.

[WeixiangXu]

Could you try increase C and K for the 1x1 Conv

Thanks for your reply!

Does C stand for C_in or C_out? And does K stands for kernel size? @nvpohanh

[nvpohanh]

Sorry, should have been clearer: C = C_in, K = C_out

[WeixiangXu]

thanks!