Performance and memory usage on large collections

[valentinalexeev]

With canner 2.0.10 and above and Google Firestore as a back-end I'm running into performance problems and out-of-memory errors when editing collections with 400+ entities.

A single edit in <array ui="table"/> can result in jump in memory usage by ~400MB.

The issue seems to come from React component state update sequence where canner hocs/utils.js function parseConnectionToNormal traverses the whole collection on any view, save or edit of any field in any entity.

The memory snapshot after single save operation:

Trace to the function:

The associated schema definition: https://gist.github.com/valentinalexeev/9fa39f3aabb1d83027c1b99ba097dc1b.

As a temporary workaround to avoid OOM I can use <array ui="tableRoute"/> however due to issue with filter state which is not saved on navigation it becomes unusable for editors to work with large content set.

[valentinalexeev]

It seems that #132 issue can be related to this as every update of every input launches full collection traversal as well.

[wwwy3y3]

hi @valentinalexeev

After the investigation, we knew exactly where we should fix and we're fixing it.

We also need some time to test it.

The patch will be released soon and we'll keep you updated.

thanks!

[valentinalexeev]

Dear @wwwy3y3, is there any update on this issue? So far I see the current v3 version still has the same code and memory issues in non-async mode.