Skip to content

Data and Backups

Miguel Serrano edited this page Feb 4, 2022 · 8 revisions

Data Storage

Overleaf Community Edition and Overleaf Server Pro store their data in 3 separate places:

  • Mongo Database: user and project data lives here.
  • Redis: used as high-performant cache for in-flight data, it mainly stores data from project edition and collaboration.
  • Filesystem: stores project files (such as images) and it's also used as a temporary disk cache during project compilations.

Performing a backup

There are 3 things which need to be backed up

  • Mongodb
  • Overleaf Filesystem data
  • Redis

In order to produce a consistent backup it is mandatory to stop users from producing new data while the backup process is running.

We advise setting up a maintenance window during which users are unable to edit their projects.

The admin interface https://sharelatex.example.com/admin (Close Editor tab) provides buttons for closing the editor and disconnecting users. Closing the editor stops users from opening new editor sessions, but keeps existing sessions open. Disconnecting users is happening with a 10 seconds grace period before the page reloads.

You will need to close the editor first, then disconnect all users.

Stop the sharelatex container and starting the backup process.

Once you are done with the backup you will need to restart the sharelatex container to restore editing capability.

Backup Tips

Backups need to be on a separate server to the one Overleaf running on, ideally in a different location entirely. Running multiple instances of mongodb is also not a backup as it will not prevent against corruption.

Mongodb

mongodb comes with a tool called mongodump which will export the dataset in a safe backup format.

Filesystem data

The path where your files are stored is specified in your settings file. This might be: /var/lib/sharelatex, copying this directory recursively is needed. rsync is a good tool for the job.

Redis

Redis stores user sessions and pending document updates before they are flushed to Mongodb. To backup redis you can copy the RDB file to a secure location.

Migrating data between servers

At best you do not have any valuable data in the new instance yet. We do not have a process for merging the data of instances.

Assuming the new instance has not data yet, here are some steps you could follow. On a high level we produce a tar-ball of the mongo, redis and sharelatex volumes, copy it over to the new server and inflate it there again.

With the default docker-compose file that would be:

# Gracefully shutdown the old instance
# (Press the buttons in the admin panel as notes above.)
old-server$ docker stop sharelatex
old-server$ docker stop mongo redis

# Create the tar-ball
old-server$ tar --create --file backup.tar ~/sharelatex_data ~/mongo_data ~/redis_data

# Copy the backup.tar file from the old-server to the new-server using any method that fits

# Gracefully shutdown new instance (if started yet)
new-server$ docker stop sharelatex
new-server$ docker stop mongo redis

# Move new data, you can delete it too
new-server$ mkdir backup-new
new-server$ mv ~/sharelatex_data ~/mongo_data ~/redis_data backup-new/

# Populate data dirs again
new-server$ tar --extract --file backup.tar

# Start containers
new-server$ docker start mongo redis
new-server$ docker start sharelatex

Depending on your docker-compose config, you may need to adjust the paths of the mongo/redis/sharelatex volume.

Clone this wiki locally