Support symmetry in einsum

[fellideal]

🚀 The feature, motivation and pitch

PyTorch is an incredibly powerful tool, particularly due to its capability to parallelize and leverage GPU resources for operations like einsum. One area where I'd love to see development is in the support for tensor symmetries.

For example, let's consider a tensor contraction like A[a,b,c,d,e] * B[d,e,f,g,h] = C[a,b,c,f,g,h]. If we have symmetries such as A[a,b,c,d,e] = A[b,a,c,d,e] or B[d,e,f,g,h]=B[d,e,f,h,g], it would be great if we could use these symmetries to boost computational efficiency significantly.

Alternatives

The ctf package does offer some functionality along these lines, but it seems limited to symmetric/antisymmetric operations for adjacent indices. Here's the GitHub repository for ctf:

https://github.com/cyclops-community/ctf

For more specifics, the documentation can be found here:

https://solomon2.web.engr.illinois.edu/ctf_python/ctf.html#module-ctf.core

That said, from my observations, the performance isn't quite as efficient as one might hope. You can check out the issue I'm referring to here:

cyclops-community/ctf#136

Additional context

No response

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @lezcano

[lezcano]

We do not have anyone actively developing einsum, and it's not even clear to me how we could potentially implement this with the current tools we have. That being said, if you think that this could be done, feel free to put up a PR and tag me as a reviewer.