Troubleshooting

Here are some possible causes for errors in the logs.

CLSI

Log location in the container: /var/log/sharelatex/clsi.log

(HTTP code 400) unexpected - OCI runtime create failed

...opt/synctex ... not a directory

Full message reads roughly (after removing quite a lot of noise): <path> is not a directory; Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

Check the value of the SYNCTEX_BIN_HOST_PATH environment variable --- please see this documentation.

Recovering deleted docs

Docs are logically deleted and will be in the deletedDocs array in the project document.

List the deleted docs in mongo:

db.projects.findOne({_id: ObjectId("<project_id>")}, { deletedDocs: 1} )

Then open a shell in the sharelatex container. From there you can run the restore_soft_deleted_docs script.

You will need:

• your admin user ID
• the project ID
• the list of doc names from deletedDocs

Example restoring a deleted restoreMe.tex document:

node /var/www/sharelatex/web/scripts/restore_soft_deleted_docs.js <admin_id> <project_id> restoreMe.tex
{ doc:
   { _id: 5f91b367e86f6d0001680ecc,
     name: '2020-10-22T16-29-27.066Z-restoreMe.tex' },
  folderId: null }

This will have created a bunch of docs in the root of the project that look like TIMESTAMP-original-name. They should have the content and the tracked changes and comments from before deletion.

Running Overleaf with a NFS filesystem

Mounting a NFS filesystem in an Overleaf container is technically possible, but it's not recommended and can result in different types of performance errors.

One common error that compiles see is:

EBUSY: resource busy or locked, unlink '/var/lib/sharelatex/data/compiles/62f3d57bef7cf9005c364e75-62f3d57bef7cf9005c364e7a/.nfs573663533034825247625441'

In particular we advise against using NFS backed filesystems for ephemeral data, like the directories use for compilation data. We recommend using a local scratch disk, preferably a local SSD for the following directories:

• /var/lib/sharelatex/tmp: ephemeral data such as uploads and processing zip file data, Settings.path.dumpFolder and Settings.path.uploadFolder are sub-directories of this folder
• /var/lib/sharelatex/data/cache: cache for binary files of projects that are being compiled (Settings.path.clsiCacheDir)
• /var/lib/sharelatex/data/compiles: ephemeral compile dir data (Settings.path.compilesDir)
• /var/lib/sharelatex/data/output: ephemeral compile artifacts for download by the browser (Settings.path.outputDir)

For docker-compose based setups, we suggest just overriding the bind-mount from NFS, which avoids changing paths in the application. Here's an example of a docker-compose config excerpt with the use of a scratch disk that is mounted at /scratch:

services:
  sharelatex:
    environment:
      SANDBOXED_COMPILES_HOST_DIR: /scratch/compiles/
    volumes:
      - nfs:/var/lib/sharelatex/data
      - /scratch/cache/:/var/lib/sharelatex/data/cache
      - /scratch/compiles/:/var/lib/sharelatex/data/compiles
      - /scratch/output/:/var/lib/sharelatex/data/output
      - /scratch/tmp/:/var/lib/sharelatex/tmp

There is no need to migrate any existing files from the NFS to their new home after the update. The LaTeX compiler can recreate all the files with a full compilation run again.

Upgrading to Redis 6.2 results in a restart loop

Use the docker logs redis command to output a copy of the logs.

Fatal: Can't initialize Background Jobs

The full output will look something like this:

1:M 11 Feb 2024 15:19:22.609 # Server initialized
1:M 11 Feb 2024 15:19:22.609 # Fatal: Can't initialize Background Jobs.
1:C 11 Feb 2024 15:19:26.055 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

If you see the line Fatal: Can't initialize Background Jobs, this may be related to the version of Docker currently in use. Updating to a version >=20.10.10 should resolve this issue.