Full Project History Migration
The 3.5.x
release of Overleaf Community Edition includes the Full Project History feature that is already available in our SaaS offering, overleaf.com
After upgrading your instance to Overleaf CE 3.5.5
, all new projects will be using Full Project History by default. Existing projects will continue using the legacy History system, until they’re migrated.
Note: If you upgrade to 3.5.5
and decide to downgrade to an earlier version, then you should restore from a full system backup. The history of projects created in 3.5.5
is not compatible with earlier versions of Overleaf CE.
The new Full Project History brings several improvements for users:
- It tracks changes in binary files, which is unsupported in the legacy system.
- There is support for labelled versions.
- The system is in general more robust, there is less chance of data loss.
Check Full Project History documentation for more information about full project history.
- Create a full backup of your instance with a consistent snapshot of the mongo, redis and sharelatex directories.
- Update the version of
sharelatex/sharelatex
image to3.5.5
-
toolkit users: update
./config/version
to3.5.5
.
-
toolkit users: update
- Start the instance.
- Ideally, you’d want to prevent users from accessing your instance while the migration takes place, to avoid data loss in case you need to restore your backup. In the toolkit add
SITE_OPEN=false
toconfig/variables.env
and restart the container with./bin/stop
&&./bin/up
.
- Ideally, you’d want to prevent users from accessing your instance while the migration takes place, to avoid data loss in case you need to restore your backup. In the toolkit add
- Wait until all the Overleaf CE services are up and running (see command below)
$ bin/docker-compose exec sharelatex /bin/bash -c "curl http://localhost:3000/status"
web sharelatex is alive (api)%
- Run the migration script:
# Overleaf Toolkit users:
$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web; VERBOSE_LOGGING=true node scripts/history/migrate_history.js"
# legacy docker-compose.yml users:
$ docker exec sharelatex /bin/bash -c "cd /overleaf/services/web; VERBOSE_LOGGING=true node scripts/history/migrate_history.js"
The output should look like this:
Migrated Projects : 1
Total Projects : 51
Remaining Projects : 51
Total history records to migrate: 98
Starting migration...
Migrating project: 63d29b5772dd80015a81bffe
migration result { upgraded: true, historyType: 'NoneWithoutConversion' }
Migrating project: 63d29c2e72dd80015a81c0a2
migration result { upgraded: true, historyType: 'NoneWithoutConversion' }
// …
Migration complete
==================
Projects migrated: 51
Projects failed: 0
Done.
If the migration is successful, you'll get an exit code of 0
, and the last lines indicating no failures:
Projects failed: 0
Done.
You can reopen access to your users (see next step). If there are failures, please see the troubleshooting section below. You can still reopen the site if the problems are not immediately fixed, and the unmigrated projects will remain on the legacy history system.
- Reopen the site. In the toolkit, edit
config/variables.env
to setSITE_OPEN=true
and restart the site with./bin/stop
&&./bin/up
.
We will add troubleshooting advice here. Please note that while we normally offer support only to Server Pro customers, given the nature of this migration, we will also do our best to support CE customers who experience problems specific to the full project history migration.
If the full project history migration script fails (i.e. exits with an error or prints a non-zero number of failed projects), please send the following details to our support team by email support+historymigration@overleaf.com, detailing:
Subject: Full project history migration problem
- Instance Type: CE or Server Pro (delete as appropriate)
- Installation Type: Overleaf toolkit or
docker-compose.yml
or other (delete as appropriate) - Script output (which should be located in the container under
/overleaf/services/web
) -
bin/doctor
output (if using toolkit)
The migration may fail for projects which have a malformed file tree (for example, where filenames are empty). You can find a list of these problems using the find_malformed_filetrees
script which checks all projects in the database:
$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web; node scripts/find_malformed_filetrees.js"
BAD PATH: 123456789012345678901234 rootFolder.0.1.2.3
BAD PATH: 123456789012345678901234 rootFolder.0.4.5.6
...
To fix the invalid paths, use the fix_malformed_filetree
script, running the command once for each bad path:
$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web; node scripts/fix_malformed_filetree.js 123456789012345678901234 rootFolder.0.1.2.3"
$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web; node scripts/fix_malformed_filetree.js 123456789012345678901234 rootFolder.0.4.5.6"
...
If there is a project which has been migrated to full project history but you want to go back to the legacy history, use the downgrade_project
script as follows:
$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web; PROJECT_ID=YOURPROJECTID node scripts/history/downgrade_project.js"
- Quickstart Guide (Overleaf Toolkit)
- Hardware Requirements
- Database & Dependencies
- Creating and managing users
- General configuration
- Configuring Email
- SSL & Nginx reverse proxy
- Data and Backups
- Configuring Headers, Footers & Logo
- Password Restrictions
- i18n Languages
- Logging
- Common Config Options
- F.A.Q
- Troubleshooting
- Full Project History Migration