3-UsersGuide.txt

/**

\page UsersGuide User's Guide

\section UserIntro Introduction

This chapter explains the overall philosophy behind PTS and provides overall usage guidelines.

\section UserPhil Philosophy behind PTS

\subsection UserPhilRatio Rationale for Python

The overall processes in which the SKIRT simulations take part are exceedingly complex. For example:
  - input data (e.g. from other simulations) is presented in various formats;
  - output data must be further analyzed, plotted, combined, or processed into final results;
  - a large number of simulations (thousands) must be performed on various platforms including
    high-performance clusters, and the results must be organized accordingly.

This demanding usage pattern, and the growing number of SKIRT developers and users, also begs decent testing
of the code after each change.

The core SKIRT functionality is very performance-sensitive, and thus must be written in a fast compiled language
such as C++. For the peripheral functions listed above however, ease of development is much more important than
execution speed. It seems that the Python language, appropriately extended with the relevant scientific packages,
perfectly fits the requirement list, which includes the following:
  - data must be massaged, sometimes in non-trivial ways, but performance is not critical;
  - several external libraries are needed to handle the various data formats;
  - plotting capabilities must handle quick graphs for internal analysis up to publication-quality figures;
  - the precise needs change frequently, often weekly or even daily.

Since the existing Python packages fill most of the needs for external functionality, the choice for Python also
limits the number of different tools and technologies used. Furthermore, offering all functions in the same environment
maximizes the opportunity for future reuse, perhaps gluing components together in unforeseen ways.

\subsection UserPhilFunc Intended functionality

The intention is to include the following types of functionality in PTS (the list is not exhaustive):
  - preprocessing data for input into SKIRT: data extractions and conversions on the results of hydrodynamical
    simulations, analyzing such input data (statistics, plots, ...), preparing simulation parameters (ski files)
    based on external input, ...
  - postprocessing SKIRT output data: creating plots (SED's, dust grids, ...), calculating color indices,
    combining frames into composite color images, putting results back into external data bases, creating movies, ...
  - managing (large numbers of) SKIRT simulations: preparing and queuing jobs on the various available computers,
    including local systems and remote high-performance clusters, copying and organizing the results, ...
  - testing SKIRT: performing regression tests in the form of a (hopefully large) number of simulation test cases
    (i.e. ski files), automatically verifying the test results with the provided reference data.

\section UserUse Using PTS classes and functions

\subsection UserUseDirs PTS source code directories

The PTS Python source code resides in three directories inside the \c PTS/git directory:
 - the \c pts directory contains the Python code for all toolkit classes and functions; i.e. the bulk of the PTS
   functionality. The files in this directory make up a Python \em package in the formal meaning of the word;
   each source file can be imported as a Python \em module.
 - the \c eagle directory contains Python classes and functions dedicated to the EAGLE project;
   see the \ref EagleGuide for more information.
 - the \c do directory contains a set of small Python scripts intended to be invoked from the command line.
   Strictly speaking these files are not part of a package since they cannot be imported as a module.

An overview of the files in each directory, with links to a more detailed description, can be found in the
reference material for "namespaces", a slightly unusual term automatically generated by the documentation system.

\subsection UserUsePts Using classes and functions in the pts directory

The files in the \c pts directory make up a Python \em package in the formal meaning of the word;
each source file can be imported as a \em module. For example, to obtain information on the instruments in a ski file
called "mysim.ski" (residing in the current directory), you may enter the following commands in a Terminal window:

\verbatim
$ python
Enthought Python Distribution ...
>>> from pts.skifile import SkiFile
>>> ski = SkiFile("mysim.ski")
>>> ski.instrumentnames()
['xy', 'xz', 'yz', 'o1']
>>> ski.instrumentshape()
(500, 500)
>>>
\endverbatim

Or to get a simulated preview for a SKIRT instrument with certain position angles, you may enter:

\verbatim
$ python
Enthought Python Distribution ...
>>> import pts.viewangles as va
>>> v = va.ViewAngles()
>>> v.distant(70, 35, 0)
>>>
\endverbatim

Which should bring up a seperate window showing the simulated view using a simple scene.

More likely, you will use the classes and functions in this package to build more complex Python scripts, or
to implement new functionality in the package itself.

Please refer to the documentation for each individual module for more details on its use.

\subsection UserUseDo Using scripts in the do directory

The Python scripts in the \c do directory are intended to be invoked from the command line.
Most scripts accept one or more command line arguments and call on functions in the Python Toolkit for SKIRT (PTS),
effectively exposing PTS functionality to the command line.

Before proceeding, ensure that your login script includes the extra lines as described in \ref InstallSourceConf,
and that you have restarted the Terminal application after that change.

To execute a Python script named \c example.py residing in the \c do directory, enter "pts example" on the command line.
This will work regardless of your current directory.
You can include command line arguments as well, as in "pts example arg1 arg2".
The following example invokes the trial script provided with PTS:

\verbatim
$ pts try arg1 arg2
Executing: try arg1 arg2
Starting try...
['/Users/pcamps/PTS/svn/src/do/try.py', 'arg1', 'arg2']
Finished try.
$
\endverbatim

To see a list of the available scripts, enter "pts" without any arguments:

\verbatim
$ pts
Available scripts:
  analyze
  convert
  flybymovie
  plotgrids
  ...
  upgradeskifiles
$
\endverbatim

You can shorten the script name in the pts command to the first few letters as long as there are no two scripts
with matching names. For example "pts exa arg1 arg2" would still execute \c example.py, assuming that only one
script has a name starting with the string "exa".

Use "ipts" rather than "pts" to enter interactive Python mode (with the >>> prompt) after executing the script.
This is useful for testing or experimenting with pts functionality: the script imports the relevant
pts module(s) and initializes some key objects that then can be used from the interactive Python prompt.

Please refer to the documentation for each individual script for more details on its use.

\section UserCase Some real-life PTS use cases

\subsection UserCaseGrids Plotting dust grids

When performing a simulation that includes a dust grid, SKIRT outputs some text files with information about the
dust grid (you can turn this behavior off when configuring the ski file, but it's on by default).
Each of the generated files contains data for plotting the intersection of the grid with one of
the coordinate planes.
The number of data files written depends on the dimension of the simulation's geometry: for spherical symmetry
only the intersection with the xy plane is written, for axial symmetry the intersections with the xy and xz planes
are written, and for general geometries all three intersections are written.
The data files are called prefix_ds_gridxy.dat, prefix_ds_gridxz.dat and prefix_ds_gridyz.dat.

PTS offers a simple way to create PDF files containing these plots for the results of a SKIRT simulation.
After the simulation has completed, make sure that the SKIRT output directory is your current directory, and
simply enter the command "pts plotgrids":

\verbatim
$ skirt mysim
...
$ pts plotgrids
Executing: plotgrids
Plotting mysim_ds_gridxy...
Done for mysim_ds_gridxy.
Plotting mysim_ds_gridxz...
Done for mysim_ds_gridxz.
Plotting mysim_ds_gridyz...
Done for mysim_ds_gridyz.
All done.
$
\endverbatim

The PDF files are placed next to the original data files and have the same name except for the .pdf extension.

\subsection UserCaseSEDs Plotting SEDs

When performing a panchromatic simulation, SKIRT outputs a text file with SED information for each instrument
of the appropriate type (including SED, Simple and Full instruments). These data files are called
prefix_instrument_sed.dat.

PTS offers a simple way to create a PDF file with a single plot showing these SEDs.
After the simulation has completed, make sure that the SKIRT output directory is your current directory,
and simply enter the command "pts plotseds":

\verbatim
$ pts plotseds
Executing: plotseds
Starting plotseds...
Created PDF grid plot file __/eagle_sed.pdf
Finished plotseds.
$
\endverbatim

The PDF files are placed next to the original data files and have a similar name
(after removing the instrument name) except for the .pdf extension.

\subsection UserCaseRGB Making RGB images

When performing a simulation, SKIRT outputs a FITS file with surface brightness information for each instrument
of the appropriate type (including Frame, Simple and Full instruments). These data files are called
prefix_instrument_total.fits. Each file contains a data frame per wavelength in the simulation’s wavelength grid.
The frames are sorted by increasing wave length.

PTS offers a simple way to create a regular RGB image (in PNG format) for each of these FITS files.
By default PTS uses the first FITS frame for the Blue channel, the last FITS frame for the Red channel,
and some frame in the middle for the Green channel. This works well for monochromatic simulations
(producing a grayscale image) and for oligochromatic simulations with 3 wavelengths.

After the simulation has completed, make sure that the SKIRT output directory is your current directory,
and simply enter the command "pts makergb":

\verbatim
$ pts makergb
Executing: makergbimages
Starting makergbimages...
Created RGB image file __/MonoDisk_i88_total.png
Finished makergbimages.
$
\endverbatim

The PNG files are placed next to the original data files and have the same name except for the .png extension.

\subsection UserCaseUpgr Upgrading ski files

Some changes or extensions to SKIRT's simulation capabilities cause incompatible adjustments to existing ski files.
To avoid the manual labour involved in upgrading your ski files, PTS offers a simple way to upgrade ski files
automatically. The intention is to support all changes made after January 1, 2013. In other words,
any ski file generated by a SKIRT6 version built from the repository source after that date should be
propertly adjusted to the most recent format version (at least, that's the goal).

To upgrade the ski files in your current directory, simply enter the command "pts upgradeskifiles":

\verbatim
$ pts upgradeskifiles
Executing: upgradeskifiles
-> /Users/pcamps/SKIRT/run...
   axes.ski (not changed).
** spiral.ski (UPGRADED).
   yep.ski (not changed).
All done.
$
\endverbatim

Before actually upgrading a ski file, the script makes a backup copy -- just to make sure that the original
is preserved in case anything goes wrong.
The name of the backup copy includes a time stamp and has the ".xml" filename extension.

The script also allows specifying a different target directory. It can even upgrade all ski files in all recursively
nested directories. See the documentation of the do::upgradeskifiles script for more information.

*/