-
Notifications
You must be signed in to change notification settings - Fork 280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Non-linear tensor decomposition #477
Comments
What exactly do you mean by non-linear relationships? One could argue tensor factorizations are not linear. Re. VAE for tensor decomposition, this would be nice to have indeed. Do you have any insights/references on how this compares in practice to a simple gradient based method optimizing the rec. loss or other non-bayesian approach? Unfortunately I don't have the bandwidth at the moment but it should be straightforward to implement directly in TensorLy-Torch which already provides most of the tools including basic tensor layers and a convenient tensor factorization extending PyTorch tensors. I haven't read the paper in detail but it looks like it's mostly a matter of optimizing a lower bound with a KL divergence over reconstruction. You could directly use the FactorizedCPTensor for this in tensorly-torch and write a custom loss. |
Thanks! I'll take a look to Tensorly-PyTorch. With non-linear relationships I mean getting factors that reconstruct the tensor in a non-linear way (i.e. not using the typical sum of rank-1 tensors). That's why the VAE approach is useful because it uses the autoencoder to obtaint latent factors from the neural net training. For that reason I think using the FactorizedCPTensor and modifying the loss function is not enough because it still relies on the parafac method (please correct me if I'm wrong). If I had the experience with this I would be able to implement it, but unfortunately I haven't work on this area and don't understand all the math in the paper. Regarding a benchmarking, in Fig 2 of the paper they compared it to other methods, and presented lower RMSE than all of them. |
Is your feature request related to a problem? Please describe.
There is no method able to capture non-linear relationships using tensor decomposition in tensorly.
Describe the solution you'd like
Variational Auto-Encoder could be used for tensor decomposition to capture non-linear relationships as described here: https://arxiv.org/pdf/1611.00866.pdf
It would be great if someone with the knowledge to implement this method do so. The algorithm is described in the manuscript (https://arxiv.org/pdf/1611.00866.pdf). This could be implemented using the PyTorch backend?
The text was updated successfully, but these errors were encountered: