Replication stability & performance

[raineorshine]

This issue captures the variety of stability and performance issues that are observed when replicating a large, real-world thoughtspace on staging2.

Observations

• Server crashes regularly
• Replication gets stuck on "Replicating..." for a long time or indefinitely.
• Replication gets stuck at 99% for a long time or indefinitely.
• IndexedDB updates collection grows as slow as one record per minute during replication
  • Perhaps at the same rate as db queries?

Questions

• Are there crash logs?
• How do crash logs align with metrics?
• Does upgrading the droplet mitigate the issue?
• How many doclog blocks are loaded into memory?
• Are db query performance issues observed in samples of 10k or 100k thoughts?
  • If all updates are stored in a single collection, then I would expect db query performance to be degraded in a new thoughtspace as well.

Solutions

• Can Hocuspocus sync differences only to conserve memory?
  • Does Hocuspocus use state vectors?
  • How would it sync different clients?
  • Each client triggers its own onDocumentLoad