This document covers the setup of S3 in Overleaf Community Edition and Server Pro. A separate document can be found on migrating existing data to S3.
Here are the most popular options for S3 compatible object storage:
- AWS S3, managed, we suggest picking AWS S3 when running Server CE/Pro on AWS
- Min.io, self-hosted
- Ceph.io, self-hosted
- Other hosting providers also have some kind of managed S3 compatible object storage, you may want to use these instead of running your own when already running Server CE/Pro at such a provider.
The latency between the Server CE/Pro instance and your S3 compatible object storage is a big contributor to the time it takes to complete the migration. The latency also impacts the file-upload performance in Server CE/Pro and slow file-downloads can have a big impact on PDF compile times as well. We suggest minimizing the geo-graphical distance between your Server CE/Pro instance and the S3 compatible object storage. In a managed environment, this would mean provisioning a bucket in the same region, and for an on-premise solution, running the two on the same campus.
We need four "buckets" and two restricted user accounts.
The buckets should not be publicly accessible.
Note: The history related items are needed from Server CE/Pro v3.5 onwards.
Bucket | Usage | Service | Previously in /var/lib/sharelatex/data
|
---|---|---|---|
overleaf-user-files |
project user files | filestore | user_files |
overleaf-template-files |
template files | filestore | template_files |
overleaf-project-blobs |
project history blobs | history | history/overleaf-project-blobs |
overleaf-chunks |
history chunks | history | history/overleaf-chunks |
You may want/need to pick a different name, be sure to use the custom buckets in all the commands.
The following will use placeholders for actual credentials:
-
SHARELATEX_FILESTORE_S3_ACCESS_KEY_ID
This is the access key/username of the restricted user of the filestore service. -
SHARELATEX_FILESTORE_S3_SECRET_ACCESS_KEY
This is the secret key/password of the restricted user of the filestore service. -
SHARELATEX_HISTORY_S3_ACCESS_KEY_ID
This is the access key/username of the restricted user of the history service. -
SHARELATEX_HISTORY_S3_SECRET_ACCESS_KEY
This is the secret key/password of the restricted user of the history service.
Server CE/Pro only needs a small set of permissions on each bucket:
- create object
- get object
- delete object
- list bucket
Here is how a policy for the filestore user could look like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::overleaf-user-files"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::overleaf-user-files/*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::overleaf-template-files"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::overleaf-template-files/*"
}
]
}
Here is how a policy for the history user could look like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::overleaf-project-blobs"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::overleaf-project-blobs/*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::overleaf-chunks"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::overleaf-chunks/*"
}
]
}
# Enable S3 backend for filestore
SHARELATEX_FILESTORE_BACKEND=s3
# Bucket name for project files
SHARELATEX_FILESTORE_USER_FILES_BUCKET_NAME=overleaf-user-files
# Bucket name for template files
SHARELATEX_FILESTORE_TEMPLATE_FILES_BUCKET_NAME=overleaf-template-files
# Key for filestore user
SHARELATEX_FILESTORE_S3_ACCESS_KEY_ID=...
# Secret for filestore user
SHARELATEX_FILESTORE_S3_SECRET_ACCESS_KEY=...
# Bucket region you picked when creating the buckets.
SHARELATEX_FILESTORE_S3_REGION=''
# Enable S3 backend for history
SHARELATEX_HISTORY_BACKEND=s3
# Bucket name for project history blobs
SHARELATEX_HISTORY_PROJECT_BLOBS_BUCKET: "overleaf-project-blobs"
# Bucket name for history chunks
SHARELATEX_HISTORY_CHUNKS_BUCKET: "overleaf-chunks"
# Key for history user
SHARELATEX_HISTORY_S3_ACCESS_KEY_ID=...
# Secret for history user
SHARELATEX_HISTORY_S3_SECRET_ACCESS_KEY=...
# Bucket region you picked when creating the buckets.
SHARELATEX_HISTORY_S3_REGION=''
# Enable S3 backend for filestore
SHARELATEX_FILESTORE_BACKEND=s3
# Bucket name for project files
SHARELATEX_FILESTORE_USER_FILES_BUCKET_NAME=overleaf-user-files
# Bucket name for template files
SHARELATEX_FILESTORE_TEMPLATE_FILES_BUCKET_NAME=overleaf-template-files
# Key for filestore user
SHARELATEX_FILESTORE_S3_ACCESS_KEY_ID=...
# Secret for filestore user
SHARELATEX_FILESTORE_S3_SECRET_ACCESS_KEY=...
# S3 provider endpoint
SHARELATEX_FILESTORE_S3_ENDPOINT=http://10.10.10.10:9000
# Path style addressing of buckets. Most likely you need to set this to 'true'.
SHARELATEX_FILESTORE_S3_PATH_STYLE='true'
# Bucket region. Most likely you do not need to configure this.
SHARELATEX_FILESTORE_S3_REGION=''
# Enable S3 backend for history
SHARELATEX_HISTORY_BACKEND=s3
# Bucket name for project history blobs
SHARELATEX_HISTORY_PROJECT_BLOBS_BUCKET: "overleaf-project-blobs"
# Bucket name for history chunks
SHARELATEX_HISTORY_CHUNKS_BUCKET: "overleaf-chunks"
# Key for history user
SHARELATEX_HISTORY_S3_ACCESS_KEY_ID=...
# Secret for history user
SHARELATEX_HISTORY_S3_SECRET_ACCESS_KEY=...
# S3 provider endpoint
SHARELATEX_HISTORY_S3_ENDPOINT=http://10.10.10.10:9000
# Path style addressing of buckets. Most likely you need to set this to 'true'.
SHARELATEX_HISTORY_S3_PATH_STYLE='true'
# Bucket region. Most likely you do not need to configure this.
SHARELATEX_HISTORY_S3_REGION=''
Note: MINIO_ROOT_USER
and MINIO_ROOT_PASSWORD
are the root credentials of
the minio instance.
Please follow the official documentation for obtaining a copy of mc
.
mc alias set s3 http://10.10.10.10:9000 MINIO_ROOT_USER MINIO_ROOT_PASSWORD
# Put the contents of the policies from the previous section in the
# respective json file policy-filestore.json and policy-history.json.
# Reminder: Replace the bucket names and credentials accordingly.
# filestore buckets, user and policy
mc mb --ignore-existing s3/overleaf-user-files
mc mb --ignore-existing s3/overleaf-template-files
mc admin user add s3 \
SHARELATEX_FILESTORE_S3_ACCESS_KEY_ID \
SHARELATEX_FILESTORE_S3_SECRET_ACCESS_KEY
mc admin policy add s3 overleaf-filestore policy-filestore.json
mc admin policy set s3 overleaf-filestore \
user=SHARELATEX_FILESTORE_S3_ACCESS_KEY_ID
# history buckets, user and policy
mc mb --ignore-existing s3/overleaf-project-blobs
mc mb --ignore-existing s3/overleaf-chunks
mc admin user add s3 \
SHARELATEX_HISTORY_S3_ACCESS_KEY_ID \
SHARELATEX_HISTORY_S3_SECRET_ACCESS_KEY
mc admin policy add s3 overleaf-history policy-history.json
mc admin policy set s3 overleaf-history \
user=SHARELATEX_HISTORY_S3_ACCESS_KEY_ID
- Quickstart Guide (Overleaf Toolkit)
- Hardware Requirements
- Database & Dependencies
- Creating and managing users
- General configuration
- Configuring Email
- SSL & Nginx reverse proxy
- Data and Backups
- Configuring Headers, Footers & Logo
- Password Restrictions
- i18n Languages
- Logging
- Common Config Options
- F.A.Q
- Troubleshooting
- Full Project History Migration