How to set the pruned weight blocks as a same learnable value?

[hobbitlzy]

I want to use the sparsity feature of the neural-compressor. I want to prune the model weights using block-wise granularity. Unlike traditional pruning approaches that zero out pruned weights, I aim to set the values within pruned blocks to a same learnable value. Does neural-compressor supports this functionality? If not, is there a convenient workaround to achieve this block-pruning with reparameterization functionality?

[YIYANGCAI]

Hi! Sorry for the late response. I think the method you proposed seems not supported in our current API. However, I believe your idea is interesting, and I will look for a chance to discuss it within my team. By the way, if you have some references which can support the effectiveness of your idea, please feel free to contact us! Thank you.