Configuration and logging

The configuration is automatically read from /etc/livereduce.conf unless specified as a command line argument. Defaults will be attempted to be determined from the environment. A minimal configuration to specify using nightly builds of mantid is

{
  "instrument": "PG3",
  "mantid_loc": "/opt/mantidnightly/bin"
}

For testing a configuration file can be supplied as a command line argument when running

$ python scripts/livereduce.py ./livereduce.conf

The logfile of what was setup for running, as well as other messages, is /var/log/SNS_applications/livereduce.log if run as the user snsdata, or livereduce.log in the current working directory (if run from the command line).

If run from inside systemd, use the standard commands for starting and stopping it.

sudo service livereduce start
sudo service livereduce stop
sudo service livereduce restart

The status of the service can be found via

sudo service livereduce status -l -n 25

to see the last 25 lines (-n 25) mantid wrote to stdout with the full information for each line (-l). People with extra permissions can run sudo journalctl -u livereduce -f and see all of the logs without them flushing on restart of the service.

The script files that are used/looked for are

• <script_dir>/reduce_<instrument>_proc.py is the processing script (for each chunk). This is required.
• <script_dir>/reduce_<instrument>_post_proc.py is the post-processing script (for the accumulated data). To disable this step rename the python script so it is not found by the daemon.

Example filenames for NOMAD with default script location is /SNS/NOM/shared/livereduce/reduce_NOM_live_proc.py and /SNS/NOM/shared/livereduce/reduce_NOM_live_post_proc.py.

Behavior

The daemon will immediately cancel StartLiveData and MonitorLiveData and restart them when one of processing scripts is changed (verified by md5sum) or removed. This is to be resilient against changes in the scripts.

The process will exit and systemd will restart it if the configuration file is changed. This is done in case the version of mantid wanted is changed.

Building and packaging

Testing is described in the test/ subdirectory.

This package uses a hand-written spec file for releasing on rpm based systems rather than the one generated by python. To run it execute

./rpmbuild.sh

And look for the results in the dist directory.

This package depends on pyinotify and (of course) mantid.

Developer notes

This repository is configured to use pre-commit. This can be done using conda via

conda env create  # looks at environment.yml
pre-commit install

Python 3 compatibility

livereduce.py is python2/3 compatible. Because the livereduce.service file is written to use system python, whether it is actually using python 2 or 3 will depend on your system's default python. The other thing to note, is that livereduce.py imports mantid directly in its namespace so you must run livereduce.py using the same python that mantid was built against.

Acknowledgements and other links

Information and ideas taken from:

• StatisticsService
• autoreduce
• post_processing_agent
• Logging in python
• systemd and systemd.service