Implement CP Decomposition for tensorflow.sparse.SparseTensor #168

amanj120 · 2020-04-29T22:12:43Z

Aims to solve issue #167

…ly, running the command tensorly.contrib.sparse.tests.test_decomposition.test_tf_sparse_cpd() should perform a quick test of the cp decomposition function

commit 2 fixed the testing frameowrk to be faster, now, after importing tensorly, running the command tensorly.contrib.sparse.tests.test_decomposition.test_tf_sparse_cpd() should perform a quick test of the cp decomposition function All the necessary functions for CP Decomposition on tf.sparse.SparseTensor are added. I need some help on how to rebase this code to fit the tensorly guidelines

amanj120 · 2020-04-30T04:08:11Z

I am not sure how to refactor the code to work with the tensorly programming guidelines/model. Any help would be appreciated.

Running

from tensorly.contrib.sparse.tests.test_decomposition import test_tf_sparse_cpd
test_tf_sparse_cpd()

produces the correct output

JeanKossaifi · 2020-05-01T14:40:21Z

Hi @amanj120,

Thanks for the PR, I will try to go through it as soon as I have some time. For reference, you want to define a new backend for the TensorFlow Sparse tensors, as is done for the PyData Sparse backend.

The functions from the backend are dynamically dispatch (in other words, every time you call a function from the backend, an intermediate function decorator checks the current backend and calls the function from that specific backend). What happens when you use the sparse context is that the functions you define in that backend will be called instead of the ones from the dense backend.

amanj120 · 2020-05-04T19:51:22Z

tensorly/contrib/sparse/kruskal_tensor.py

+
+
+def sparse_mttkrp(tensor, factors, n, rank, dims):
+    values = tl.values(tensor)


I think all sparse backends should have a "values" and "indices" method

coveralls · 2020-05-04T19:52:36Z

Coverage decreased (-3.04%) to 87.531% when pulling 9781086 on amanj120:master into 26189bf on tensorly:master.

…sparse.backend.__init__.py

JeanKossaifi · 2020-06-19T12:43:03Z

Hi, sorry for the delay -- so in contrib.sparse, all the backends are sparse, so you only need tensorflow, not tensorflow.sparse. In other words, to use sparse tensors with tensorflow you just have to set the backend to tensorflow and use tensorly.contrib.sparse

Ideally all should then work transparently.

meyerzinn · 2020-07-07T17:13:42Z

What is the status of this PR? I'm hoping to use TensorLy for some large sparse PARAFAC decompositions and right now it's limited to CPU-only. Would TensorFlow support allow GPU acceleration?

amanj120 · 2020-07-07T18:02:20Z

Hello,

I've been really busy since school ended, so I haven't had time to look at this PR in a while. @JeanKossaifi I will try to look into the changes you mentioned this coming weekend. @20zinnm The changes I've made so far are CPU-Only.

A GPU implementation for sparse PARAFAC might use Numba CUDA or some similar alternative, but that falls outside the scope of this project.

amanj120

@JeanKossaifi can you take a look at the comments I left?

Also, the Travis builds are failing because of a unit test that I slightly altered. The documentation for sparse.random suggests that the shape of the random factor matrices NOT be a tuple, so I changed the test code, but now I get a different error saying ValueError: cannot reshape array of size 2862 into shape (2862,14036). Do you know why this might be?

amanj120 · 2020-07-14T00:19:21Z

tensorly/backend/core.py

+    def indices(tensor):
+        """ Returns all the indices of non zero values of the tensor in COO
+        """
+        raise NotImplementedError



I think in order to have static indices and values methods for all sparse backends, I had to provide an implementation for the non-sparse backends too, which is why I am changing tensorly/backend/core.py

amanj120 · 2020-07-14T00:20:45Z

tensorly/backend/tensorflow_backend.py

@@ -43,11 +43,11 @@ def to_numpy(tensor):

    @staticmethod
    def ndim(tensor):
-        return len(tensor.get_shape()._dims)
+        return len(tl.to_numpy(tensor).shape)



For some reason, the old implementation was failing for me. This might be a problem with the dynamic dispatching though. If it turns out my implementation of something else was wrong, I'll go back and fix this.

amanj120 · 2020-07-14T00:24:36Z

tensorly/contrib/sparse/kruskal_tensor.py

+
+
+def sparse_mttkrp(tensor, factors, n, rank, dims):
+    with sparse_context():


I don't know if I need to have a with sparse_context(): for every method in this file. The reason I include this line is because I call the values and indices methods inside this method, and the implementations for those only exist if the backend is sparse.

amanj120 · 2020-07-14T00:25:08Z

tensorly/contrib/sparse/tests/test_decomposition.py

+
+
+def test_tf_sparse_cpd():
+    with sparse_context():


Same thing about with sparse_context: here.

amanj120 · 2020-07-14T00:26:59Z

tensorly/contrib/sparse/backend/tensorflow_backend.py

+
+
+class TensorflowSparseBackend(Backend):
+    backend_name = 'tensorflow'


Should the backend_name be tensorflow or tensorflow.sparse? In numpy_backend the name is numpy.sparse, but if I set the name to tensorflow.sparse here, I get a ValueError: Unknown backend name 'tensorflow.sparse', known backends are ['numpy', 'tensorflow'] error.

amanj120 · 2020-07-14T01:02:19Z

tensorly/contrib/sparse/backend/tensorflow_backend.py

+    def tensor(data, dtype=np.float32, device=None, device_id=None):
+        if isinstance(data, tf.sparse.SparseTensor):
+            return data
+        elif isinstance(data, tuple):


This elif branch is necessary in order to create tensors without having to import tensorflow. Importing tensorflow is what was causing the Travis builds to fail.

JeanKossaifi · 2021-04-23T18:58:51Z

Hey @amanj120! Did you manage to make it work?

JeanKossaifi · 2022-12-28T17:56:56Z

Hi @amanj120 -- are you still working on this?

amanj120 added 5 commits April 29, 2020 18:01

sparse cpd implemented

a041ec6

commit 2

5cb4d24

fixed the testing frameowrk to be faster, now, after importing tensor…

c10374a

…ly, running the command tensorly.contrib.sparse.tests.test_decomposition.test_tf_sparse_cpd() should perform a quick test of the cp decomposition function

fix merge conflict

2bb165c

no need to change requirements.txt

b28518e

refactored code to fit the dynamic dispatching model

f3433e2

amanj120 commented May 4, 2020

View reviewed changes

amanj120 added 2 commits May 4, 2020 17:09

moved inner product and fit methods to kruskal_tensor.py

38236d9

I am not sure why there is no set_backend method in tensorly.contrib.…

92931b7

…sparse.backend.__init__.py

amanj120 commented Jul 14, 2020

View reviewed changes

amanj120 force-pushed the master branch 4 times, most recently from 6c80dbe to 99c6df2 Compare July 14, 2020 01:00

amanj120 commented Jul 14, 2020

View reviewed changes

amanj120 force-pushed the master branch from 99c6df2 to 97a66d6 Compare July 14, 2020 01:13

followed Jean's recommendation

9781086

amanj120 force-pushed the master branch from 97a66d6 to 9781086 Compare July 14, 2020 01:37

JeanKossaifi force-pushed the master branch from b651260 to f22b9bf Compare September 30, 2020 16:06

JeanKossaifi force-pushed the master branch 2 times, most recently from d45beab to 2aa9834 Compare January 3, 2021 14:51

Base automatically changed from master to main January 22, 2021 13:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement CP Decomposition for tensorflow.sparse.SparseTensor #168

Implement CP Decomposition for tensorflow.sparse.SparseTensor #168

amanj120 commented Apr 29, 2020

amanj120 commented Apr 30, 2020

JeanKossaifi commented May 1, 2020

amanj120 May 4, 2020

coveralls commented May 4, 2020 •

edited

JeanKossaifi commented Jun 19, 2020

meyerzinn commented Jul 7, 2020

amanj120 commented Jul 7, 2020

amanj120 left a comment •

edited

amanj120 Jul 14, 2020

amanj120 Jul 14, 2020

amanj120 Jul 14, 2020

amanj120 Jul 14, 2020

amanj120 Jul 14, 2020

amanj120 Jul 14, 2020

JeanKossaifi commented Apr 23, 2021

JeanKossaifi commented Dec 28, 2022



		def sparse_mttkrp(tensor, factors, n, rank, dims):
		values = tl.values(tensor)



		def sparse_mttkrp(tensor, factors, n, rank, dims):
		with sparse_context():



		class TensorflowSparseBackend(Backend):
		backend_name = 'tensorflow'

Implement CP Decomposition for tensorflow.sparse.SparseTensor #168

Are you sure you want to change the base?

Implement CP Decomposition for tensorflow.sparse.SparseTensor #168

Conversation

amanj120 commented Apr 29, 2020

amanj120 commented Apr 30, 2020

JeanKossaifi commented May 1, 2020

amanj120 May 4, 2020

Choose a reason for hiding this comment

coveralls commented May 4, 2020 • edited

JeanKossaifi commented Jun 19, 2020

meyerzinn commented Jul 7, 2020

amanj120 commented Jul 7, 2020

amanj120 left a comment • edited

Choose a reason for hiding this comment

amanj120 Jul 14, 2020

Choose a reason for hiding this comment

amanj120 Jul 14, 2020

Choose a reason for hiding this comment

amanj120 Jul 14, 2020

Choose a reason for hiding this comment

amanj120 Jul 14, 2020

Choose a reason for hiding this comment

amanj120 Jul 14, 2020

Choose a reason for hiding this comment

amanj120 Jul 14, 2020

Choose a reason for hiding this comment

JeanKossaifi commented Apr 23, 2021

JeanKossaifi commented Dec 28, 2022

coveralls commented May 4, 2020 •

edited

amanj120 left a comment •

edited