add support of scipy.linalg.solve_banded()

[AxelBreuer]

Until now, autograd only handles scipy.linalg.solve() and scipy.linalg.solve_triangular().

In contrast to scipy.linalg.solve() and scipy.linalg.solve_triangular(); scipy.linalg.solve_banded() can handle VERY large systems of equations.

In fact, I think that solve_banded() could interest a lot of autograd users: banded systems are ubiquitous in ODEs, PDEs and signal processing.

Furthermore, solve_banded() could be of interest to a wider audience, since sparse matrix solvers are not handled yet by torch.autograd (to the best of my knowledge).

In that sense, solve_banded() would be a first step towards handling a specific subset of sparse matrices, namely banded matrices.

Last but not least, I hope that, at a later stage, someone could then adapt, improve and include my code to torch.autograd and/or jax.

[j-towns]

Thanks @AxelBreuer! Could you please add a test, the right place to do it is down here:

autograd/tests/test_scipy.py

Lines 227 to 230 in e18f656

	## Linalg
	def test_sqrtm(): combo_check(spla.sqrtm, modes=['fwd'], order=2)([R(3, 3)])
	def test_sqrtm(): combo_check(symmetrize_matrix_arg(spla.sqrtm, 0), modes=['fwd', 'rev'], order=2)([R(3, 3)])
	def test_solve_sylvester(): combo_check(spla.solve_sylvester, [0, 1, 2], modes=['rev', 'fwd'], order=2)([R(3, 3)], [R(3, 3)], [R(3, 3)])

.

[AxelBreuer]

Hi @j-towns I wrote the test you asked for. This exercise actually helped me to find a bug in my original code; which I corrected. Overall my code and test could probably be more elegant, but they do the job.

[j-towns]

Hi @AxelBreuer, nice work! I reverted some unnecessary changes to test_scipy.py, but this is now ready to merge. Thanks, and sorry for the long wait.

[AxelBreuer]

Thank you so much Jamie !