Write the "Incremental Computation" RFD

[alilleybrinker]

Currently, Hipcheck uses the salsa crate to provide "incremental computation," meaning the ability to write code which recognizes when some value has already been computed, and to reuse that computation rather than calculating it again. This kind of property is extremely useful for something like Hipcheck where we want to support fairly arbitrary calculation of measurements based on data, some of which may be expensive to calculate (a classic example in Hipcheck's past which motivated this was the calculation of diffs for all commits in the history of a project; diffs are slow to compute and we really only want to do it once).

Salsa was chosen more than a year ago, at a time when it was still under active development. It has always been experimental, but it was in active use in the rust-analyzer project and we believed it would continue to improve and become more stable over time. That bet has turned out to be wrong, Salsa has made some progress since we initially adopted it, but is largely defunct at this point. The rust-analyzer project maintains their own fork of it which is specialized to their own use cases and not reusable.

Given this, we ought to consider other options for incremental computation. These options could include:

• Use of an alternative system for incremental computation in Rust, if one exists.
• Vendoring of salsa to maintain and further develop it ourselves.
• Exploration of options exploiting outside database / data management solutions.

The goal of this effort would be to produce an RFD which outlines what the new system would be, and how we'd move to it.

There's a ton of good prior art in the references shared by the salsa project itself. There are also good and relevant talks related to the Have You Tried Rubbing a Database On It? (HYTRADBOI) online conference held in 2022.

[alilleybrinker]

I should add that it would also be a valid conclusion to still use upstream salsa if we investigate it and determine that it makes sense to keep using.