Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better random init of factorized tensors #521

Open
JeanKossaifi opened this issue Aug 19, 2023 · 1 comment
Open

Better random init of factorized tensors #521

JeanKossaifi opened this issue Aug 19, 2023 · 1 comment

Comments

@JeanKossaifi
Copy link
Member

I wrote a better init in TensorLy-Torch) we should probably bring it in at some point, it allows the reconstruction to have zero mean and given standard deviation.

@cohenjer
Copy link
Contributor

cohenjer commented Sep 14, 2023

I had a look at your code @JeanKossaifi, if I got it correctly its simply a matter of scaling the standard deviation with the rank and the order; seems like we could indeed very easily have this is pure tensorly code without any issue.
I'd be happy to review the PR quickly if you want to open the PR. Only thing I would add is a quick math explanation of the computation in the comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants