Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache parsed results to speed up initial load #5659

Open
Rheeseyb opened this issue May 13, 2024 · 0 comments
Open

Cache parsed results to speed up initial load #5659

Rheeseyb opened this issue May 13, 2024 · 0 comments

Comments

@Rheeseyb
Copy link
Contributor

Rheeseyb commented May 13, 2024

Since parsing of large projects takes so long, and we do not store the parsed model (only the code) on the server (mainly due to the size of the requests), we would like to cache the parsed results locally. There are a few possible goals we can achieve here, so it is important when taking this ticket to assess a solution's suitability based on which of these goals it can achieve, how long it will take to implement and test, and how complex it is.

Possible goals:

  • Improve loading time of an existing (previously opened project)
  • Improve loading time of a project that is very similar to a previously opened project (e.g. creating a new project from a template that you've used before, or a fork of someone else's project that you were viewing, or a pair of projects with maybe a single file difference)
  • Reduce the time taken to clone a project from github which has previously been cloned (probably the same problem as above, but not necessarily)

Important caveats:

  • We need to make absolutely sure that the parsed model is still valid. Any editor changes which could in any affect the parsed model must cause a cache invalidation. The fallback option here is to invalidate the cache based on the editor's commit sha, but that of course means the cache will always be very short lived. We could try something which only invalidates it based on changes which actually affect the parsed model, but that would likely either be brittle (prone to human error since it would involve manually flagging these kinds of changes and updating the projectVersion each time) or time consuming (moving all parsing into a separate package so we can use the version number of that package).
  • We do not want a cache that will grow out of control, so e.g. keying based on the file contents itself would require aggressive cleaning
  • The filename is important, as without that we will definitely see UID clashes

Note this work should not be started until #5655 has been completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant