Truncate redis streams

[ddimaria]

Closes #1270

• Query Redis for all active channels
• For each active channel, query the API service for the latest checkpoint (or one prior?)
• Truncate the stream (channel) up to the set sequence number of the checkpoint
• Add truncate info to stats (appears on logs)
• Force client to reload the file if the transaction isn't available

Deployment Plan

• Set new environment variables (1 week?)

TRUNCATE_FILE_CHECK_S=3600 # 1 hour
TRUNCATE_TRANSACTION_AGE_DAYS=5 # 5 days

• Create a jump box that can connect to Redis with Redis CLI
• Merge branch (wait until deploy)
• Backup Redis
• Shutdown multiplayer EC2
• Wait until File service processes everything via DataDog logs
• Clear existing redis cache (FLUSHDB)
• Restartup multiplayer EC2
• Verify files are being cleaned up

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
quadratic	✅ Ready (Inspect)	Visit Preview	May 22, 2024 10:21pm

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.17%. Comparing base (1e21d3a) to head (b4dfd94).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1278      +/-   ##
==========================================
+ Coverage   89.34%   90.17%   +0.82%     
==========================================
  Files         160      134      -26     
  Lines       26425    24393    -2032     
==========================================
- Hits        23610    21996    -1614     
+ Misses       2815     2397     -418     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.