Skip to content

Latest commit

 

History

History
97 lines (71 loc) · 3.07 KB

mongodb.md

File metadata and controls

97 lines (71 loc) · 3.07 KB
Raw

MongoDB

MongoDB is the database used to search products and run aggregations (like facets), etc.

We have a single instance in production.

For staging also we have a single instance.

Refresh products scripts

The machine with MongoDB must have a product opener clone to be able to run refresh_products_tags.js in crontab.

This creates small collections that are used for aggregations (mainly for facets for anonymous users).

Staging (aka Dev) instance

This is an instance for staging (aka .net) product opener instances.

It is deployed on staging containers VM.

See https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/workflows/mongo-deploy.yml script.

Prod instance

We installed the MongoDB Community Edition Packages for Debian in version 4.4.x.

Currently this is hosted on off3, a temporary server gracefully lended by OSM France, you can access it via ssh from off1 or off2 as root (sudo ssh 10.0.0.3).

The configuration file is in /etc/mongod.conf.

  • storage.dbPath is set to a specific volume: /mongo/db
  • systemLog.LogRotate is set to reopen (needed by logrotate)
  • processManagement.pidFilePath is set to "/var/run/mongodb/mongod.pid" (strangely it's not set by default while /lib/systemd/system/mongod.service uses this path)

I also had to create the /var/run/mongodb directory and give it to mongodb user:

mkdir /var/run/mongodb/
chown mongodb:mongodb /var/run/mongodb/

Log rotate

In production, logs have to be rotated1.

Following this blog post cross checked with official docs.

See above to configure /etc/mongod.conf correctly.

Then /etc/logrotate.d/mongod:

/var/log/mongodb/mongod.log
{
   # keep 7 files (one week)
   rotate 7
   # every day
   daily
   # min file size
   size 100M
   # no problem if it does not yet exists
   missingok
   # create it if needed with owner mongodb
   create 0600 mongodb mongodb
   # compress only old file (as switch is not immediate)
   delaycompress
   # but compress old archives
   compress
   # the postrotate tells mongo to change log
   # Be sure to have processManagement.pidFilePath set to this path in mongod.conf
   # and that folder exists and is owned by mongodb
   sharedscripts
   postrotate
     /bin/kill -SIGUSR1 $(cat /var/run/mongodb/mongod.pid)
   endscript
}

We also have logs for refresh products tags /etc/logrotate.d/mongo_refresh_tags:

/var/log/mongodb/refresh_products_tags_*.log
{
  rotate 7
  weekly
  size 100M
  missingok
  # we don't know when logrotate will happen
  # script might be working, so only compress previous files
  delaycompress
  compress
}

Footnotes

  1. On staging logs are sent to docker stdout/err and we let docker handle logs. ↩