cavedbmanager

This project simplifies the data management for a state and/or county-wide cave survey by eliminating the need for the manual duplication of data, which will help to save time and minimize errors. The system dynamically generates the following types of files based on the data that you upload to the database:

• GIS files: SHP, KML (Google Earth), GPX, MXF (Maptech), and images generated using Mapserver with your cave locations and cave lineplots overlayed onto them.
• A PDF that is suitable for publication.
• ZIP file whose contents include select photos, maps, and references. All of the filenames are standardized to a common naming convention. The contents of this ZIP file can be written to a CD or DVD and included with the book.
• To-do list
• CSV file (Spreadsheet)

This significantly reduces the hassle of formatting the book and allows the user to concentrate on the collection of data.

This project is currently used by the West Virginia Speleological Survey to manage over 5,000 cave and karst features within the state.

Sample Bulletin

A sample bulletin is available that shows a PDF that was generated by this application based on the data that was added through the web interface. Full instructions about how to recreate this sample document are below.

Screenshots of the web application and other generated artifacts are available in the screenshots directory.

Some notes about the generated Sample Bulletin PDF

• The title page, preamble page and contributor pages (i, ii, and iii) can all be edited in the bulletin section of the web interface. The content is styled using LaTeX.
• The Table of Contents on page iv shows the list of bulletin regions associated with the bulletin.
• The map on page v shows the position of all of the regions as they relate to each other. The rectangles labeled Coopers Rock and Test Region were dynamically drawn based on the extent of all features that are present inside each region. If you add a new feature outside of the rectangle, then the rectangle will be dynamically expanded the next time that the bulletin is regenerated.
• The maps on pages 1, 2, 7 and 8 were all dynamically generated based on the features associated with each region.
• Additional terms can be added to the index on page 13. Edit the bulletin and add your terms to the Indexed Terms section. Separate each term with a newline.

Installation

Fedora users can install docker-compose with:

sudo dnf install docker-compose podman-docker
sudo systemctl enable --now podman.socket

The project can be started by running:

sudo --preserve-env=PWD make dockerRun

This will use Docker Compose to build and start several containers.

Open https://localhost:8443 in your web browser. The default credentials are admin / password (see docker-compose.yml. Once logged in, click on the Bulletins link on the main page, then click on the generate link in the documents column. You'll see the build start in your console log. Once finished, refresh the page and you should see the generated documents.

Note: During initial startup, it will download about 2-3 GB of data, and it will transform some of the GIS data. On my laptop with an i7 processor, it takes about 15 minutes to complete. This data will be stored in the docker-volumes/ directory outside the container and will be mounted inside the containers as volumes.

Architecture

The docker-compose file sets up four containers:

• A webserver container that runs nginx, uWSGI, supervisord, and Django. Only the Django Admin Site is used at the moment to make generating the pages easy. Note that the base Docker image cavedbmanager_base is defined in Dockerfile.base.
• A database container that runs PostgreSQL and PostGIS.
• A worker container that builds GIS maps, PDFs, and other artifacts. Note that the base Docker image cavedbmanager_base is defined in Dockerfile.base. Some notable technologies used include Mapserver, GDAL, and LaTeX / TeX Live. Messages are passed to this container by writing new files as messages into a shared directory. The worker.sh script uses inotify to watch for new files that are created in a shared directory. The files are empty and the message is in the filename. See cron container for an example. If this needs to run across multiple systems, then a real messaging system like RabbitMQ can be used instead.
• A cron container schedules periodic jobs. See the crontab file for an example of how the cron container passes messages to the worker container. The docker-compose file lists /var/run/cavedb-worker as a shared directory for multiple containers.

See the note at the bottom of the docker-compose file for instructions on how to change the networking so that the webserver container is the only one that has access to the Internet to increase the security of the application. The Internet is enabled by default so that the sample data can be downloaded.

Publications

This system was used to publish the book:

• West Virginia Speleological Survey Bulletin #18: The Caves and Karst of Tucker County by Doug McCarty and Brian Masney, 2011. Includes 304 cave and karst features, 155 photos, and 96 maps; contains CD-ROM with full-sized color versions of all maps and photos, plus extras. Cover

This book is available for sale to NSS members through the West Virginia Speleological Survey.