Rename Embedding to Scatter or something similar

[marvinpfoertner]

PR #505 introduces the Embedding linear operator, which represents a tall sparse matrix containing exactly one nonzero entry, specifically a 1, per column.
In my opinion an Embedding is much more general than that (I would essentially call any tall matrix an embedding).
The operation that is performed by this LinearOperator is called Scatter in some Deep Learning frameworks.
I propose we rename it to Scatter and add some documentation as to what it does.
Context: @JonathanWenger and I will use the more general class of embeddings for the probabilistic linear solver and will need that name then.

Is this fine with everyone? I'm also open to alternative naming suggestions.

[marvinpfoertner]

@JonathanWenger @pnkraemer @schmidtjonathan

[schmidtjonathan]

@pnkraemer and I discussed this at some point, we came to the conclusion that a per-row scaling with a non-1-entry would be not a mere embedding anymore, but also add a scaling/warping/?-aspect to the operation.

[marvinpfoertner]

I agree, it would add scaling and warping, but from my point of view this does not challenge it being an embedding, since this just means embedding into a higher-dimensional space with a different basis. I think the specific case you described could be referred to as an isometric embedding, since this excludes scaling

[JonathanWenger]

My proposal would be to stick to the mathematical definition where an embedding is any injective map which is "structure-preserving" in some way. Here structure-preserving often means continuous (\lVert Ax \rVert_Y \leq C \lVert x \rVert_X$) for example. For linear maps between finite-dimensional spaces the structure-preserving part is always given (choose $C=\lVert A \rVert_{op}$). The type of embedding described by the current implementation is oftentimes called a natural embedding (see https://en.wikipedia.org/wiki/Continuous_embedding#Examples). So my proposal would be to

• have a general class Embedding, which satisfies shape[0] >= rank >= shape[1] (thus ensuring injectivity)
• and a class NaturalEmbedding(Embedding) which has orthogonal unit vectors in its columns.

[schmidtjonathan]

I like that. Thanks, @JonathanWenger for the detailed explanation!