Thoughts on future state handling

[stefan-k]

State is needed to keep track of any kind of data from one iteration to the next. This state could be handled by the solver structs themselves, but by using a dedicated state, it is possible to define a common interface and it avoids having to re-implement certain functionality for every solver. However, I'm not really happy with the current design of IterState and PopulationState. IterState is designed to accommodate a wide range of solvers, but in doing so it is neither very space efficient, nor is it convenient to use due to its many internal type parameters.
If a solver only operates on parameters and gradients, it still carries Option<Jacobian> and others around. Some solvers are sufficiently different to justify their own kind of state, such as ParticleSwarm, which has PopulationState.

Ideally, each solver should have it's own state with a yet-to-be-designed common interface. However, I'm afraid that implementing this will be an annoying and unnecessary seeming chore. Also, the common interface must not suffer from similar problems as the current design.

My current idea is to provide "state building blocks" which can be stitched together using a declarative macro. For instance, there could be dedicated building blocks for handling parameter vectors, gradients, Hessians, Jacobians, residuals, populations, number of function calls, costs, time, termination status, ... probably each with their own Trait, which are automatically implemented for the "aggregated" state type. For each solver, one just has to define which building blocks are needed and the macro will take care of it all. I'm not sure though if all of this can be done with a declarative macro.

One potential problem I see with this is in all the places which depend on the state implementing the State trait.

This design should also allow one to reuse state from one run to another, as this could avoid unnecessary allocations. This would require some sort of resetting mechanism which each of the building blocks could provide.

Any ideas on this are highly welcome!