How to freeze params of an embedding table in EmbeddingBagCollection？

[tiankongdeguiji]

No description provided.

[tiankongdeguiji]

Hi, @henrylhtsang @IvanKobzarev @joshuadeng @PaulZhang12 can you see this problem?

[colin2328]

a few ways you could do this:
the simplest way would be to constrain (through parameter constraints) your EBC to be dense (non fused) and not to set up an optimizer for the tables you want to freeze.
You could also try setting the lr to 0 if you are interested in freezing EBCs mid training.

Keep in mind htat if you have only some tables you want to freeze (and some you want to always be trainable) that you wil likely want to create separate EBCs for each (otherwise the sharder might combine them)

[tiankongdeguiji]

a few ways you could do this: the simplest way would be to constrain (through parameter constraints) your EBC to be dense (non fused) and not to set up an optimizer for the tables you want to freeze. You could also try setting the lr to 0 if you are interested in freezing EBCs mid training.

Keep in mind htat if you have only some tables you want to freeze (and some you want to always be trainable) that you wil likely want to create separate EBCs for each (otherwise the sharder might combine them)

Thanks!