Recompute method in computed tables

[lfrank]

One of our main goals is to be able to share complete pipelines with results. Sharing of the various intermediate computed data outputs is one way to do this, but for data that can be computed relatively quickly, we could also enable remote users to recompute on the fly.

One possible solution would be to add a recompute method to each dj.Computed table that would regenerate the NWB file (using the same name) if it was not present locally. This would be quite a lot of work, and would also require a lot of thought as to what to do when upstream computed results are not available, but if we could get this to work we could share a much smaller subet of results when we publish papers, which would likely help.

[samuelbray32]

Thoughts on a potential structure:

• Recompute is essentially recalling the make function outside of a datajoint transaction and avoiding any insert statements therin.
  • We could add a recompute=False argument to make functions and change any insert statements in them to only run if this is False.
• Files would only need recomputed when accessed. This could be a new fallback in fetch_nwb that would call Table.make(key, recompute=True) when the analysis nwb can't be obtained another way.
  • Benefit is this would recursively handle propagating recompute up to other missing intermediate tables if they are needed in the the original recompute call since the upstream data should be accessed through fetch_nwb

I'm sure there's some edge cases in things like spikesorting I'm not thinking of, but this might handle a lot without too much change to the code