Merge pull request #206 from CosmiQ/dev

Version bump to 0.1.2
CosmiQ · Jul 18, 2019 · eb2ce05 · eb2ce05
2 parents d0c4592 + 5c01053
commit eb2ce05
Show file tree

Hide file tree

Showing 151 changed files with 3,237 additions and 354 deletions.
diff --git a/.gitignore b/.gitignore
@@ -120,3 +120,5 @@ dmypy.json
 
 # Project-specific
 sandbox.ipynb
+solaris/nets/weights
+model_weights
diff --git a/.travis.yml b/.travis.yml
@@ -1,6 +1,7 @@
 language: python
 sudo: required
 dist: xenial
+cache: false
 python:
   - "3.6"
   - "3.7"
@@ -19,20 +20,22 @@ install:
   - export PATH="$HOME/miniconda/bin:$PATH"
   - hash -r
   - conda config --set always_yes yes --set changeps1 no
-  - conda update -q conda
   # Useful for debugging any issues with conda
+  - conda update conda -c conda-forge
   - conda info -a
   # switch python version spec in environment.yml to match TRAVIS_PYTHON_VERSION
   # annoying workaround to `conda env create python=$TRAVIS_PYTHON_VERSION` not working
   - sed -i -E 's/(python=)(.*)/\1'$TRAVIS_PYTHON_VERSION'/' ./environment.yml
   - conda env create -n solaris -f environment.yml
+  - conda list -n solaris
   - source activate solaris
   - python --version
   - pip install -q -e .[test]
-  - pip install codecov pytest pytest-cov
+  - conda install pytest=4.6.2 -c conda-forge
+  - pip install codecov pytest-cov
 # command to run tests
 script:
-  - pytest --cov=./
+  - python -m pytest --cov=./
 
 after_success:
   - codecov
diff --git a/docs/_templates/custom_sidebar.html b/docs/_templates/custom_sidebar.html
@@ -0,0 +1 @@
+<ul class="globaltoc">{{ toctree(collapse=False,includehidden=theme_globaltoc_includehidden|tobool) }}</ul>
diff --git a/docs/api/index.rst b/docs/api/index.rst
@@ -1,3 +1,5 @@
+.. _api_index:
+
 .. title:: API reference contents
 
 ###################

diff --git a/docs/conf.py b/docs/conf.py
@@ -24,8 +24,8 @@
 copyright = u'2018-{}, CosmiQ Works: an IQT Lab'.format(time.strftime("%Y"))
 
 # The full version, including alpha/beta/rc tags
-release = '0.1.1'
-version = '0.1.1'
+release = '0.1.2'
+version = '0.1.2'
 
 # -- General configuration ---------------------------------------------------
 
@@ -128,7 +128,7 @@ def setup(app):
 # default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
 # 'searchbox.html']``.
 #
-html_sidebars = {'**': ['localtoc.html', 'sourcelink.html', 'searchbox.html']}
+html_sidebars = {'**': ['custom_sidebar.html', 'sourcelink.html', 'searchbox.html']}
 
 # -- Options for HTMLHelp output ---------------------------------------------
 

diff --git a/docs/index.rst b/docs/index.rst
@@ -23,6 +23,7 @@ An open source machine learning pipeline for geospatial imagery
 
   installation
   intro
+  pretrained_models
   api/index
   tutorials/index
 
@@ -31,13 +32,14 @@ An open source machine learning pipeline for geospatial imagery
 
 User Guide
 ==========
-* `What is solaris? <intro.html>`_
-* `Installation <installation.html>`_
-* `Tutorials and recipes <tutorials/index.html>`_
+* :ref:`What is solaris? <intro>`
+* :ref:`Installation <installation>`
+* :ref:`Pretrained models available in solaris <pretrained_models>`
+* :ref:`Tutorials and recipes <tutorials_index>`
 
 Reference
 =========
-* `API reference <api/index>`_
+* :ref:`API reference <api_index>`
 
 Index
 =====

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -1,3 +1,5 @@
+.. _installation:
+
 ######################
 Installing ``solaris``
 ######################

diff --git a/docs/intro.rst b/docs/intro.rst
@@ -1,3 +1,5 @@
+.. _intro:
+
 
 ##############################
 An introduction to ``solaris``

diff --git a/docs/pretrained_models.rst b/docs/pretrained_models.rst
@@ -0,0 +1,62 @@
+.. _pretrained_models:
+
+##########################################
+Pretrained models available in ``solaris``
+##########################################
+
+``solaris`` provides access to a number of pre-trained models from
+`the SpaceNet challenges <https://spacenet.ai>`_.  See the table below for a
+summary. Note that the model name in the first column should be used as the
+``"model_name"`` argument in
+`the config file <tutorials/notebooks/creating_the_yaml_config_file.html>`_ if you wish to use that model with ``solaris``. Note that we re-trained the
+ competitors' models for compatibility with ``solaris`` and the training parameters,
+ inputs, and performance may vary slightly from their original models.
+
+Model details
+=============
+
++------------------------------------+---------------------+-----------------------+----------------+-------------+-------------+---------------------------------+---------------------------------------+
+| Model name                         | Model type          | Model details         | # Parameters   | Input shape |Output shape | Config file                     | Weights file                          |
++====================================+=====================+=======================+================+=============+=============+=================================+=======================================+
+| xdxd_spacenet4                     | Segmentation UNet   | Encoder: VGG16        | 29.3M          | 3x512x512   | 1x512x512   | `link <XDXDconfig_>`_           | `link <XDXDweights_>`_  (117 MB)      |
++------------------------------------+---------------------+-----------------------+----------------+-------------+-------------+---------------------------------+---------------------------------------+
+| selimsef_spacenet4_resnet34unet    | Segmentation UNet   | Encoder: ResNet-34    | 30.0M          | 4x416x416   | 3x416x416   | `link <ssresnet34config_>`_     | `link <ssresnet34weights_>`_ (120 MB) |
++------------------------------------+---------------------+-----------------------+----------------+-------------+-------------+---------------------------------+---------------------------------------+
+| selimsef_spacenet4_densenet121unet | Segmentation UNet   | Encoder: DenseNet-121 | 15.6M          | 3x384x384   | 3x384x384   | `link <ssdense121config_>`_     | `link <ssdense121weights_>`_ (63 MB)  |
++------------------------------------+---------------------+-----------------------+----------------+-------------+-------------+---------------------------------+---------------------------------------+
+| selimsef_spacenet4_densenet161unet | Segmentation UNet   | Encoder: DenseNet-161 | 41.1M          | 3x384x384   | 3x384x384   | `link <ssdense161config_>`_     | `link <ssdense161weights_>`_ (158 MB) |
++------------------------------------+---------------------+-----------------------+----------------+-------------+-------------+---------------------------------+---------------------------------------+
+
+Training details
+================
+
+Below is a summary of the training hyperparameters for each model. For image
+pre-processing and augmentation pipelines see the config files linked above.
+*Note that our hyperparameters may differ from the competitors' original values.*
+See `their solution descriptions <https://github.com/spacenetchallenge>`_ for
+more on their implementations.
+
++------------------------------------+-------------------------+-------------------+---------------+------------------------+-----------------+------------+-----------------+---------------------+
+| Model name                         | Loss function           | Optimizer         | Learning Rate | Training input         | Training mask   | Batch size | Training Epochs | Pre-trained weights |
++====================================+=========================+===================+===============+========================+=================+============+=================+=====================+
+| xdxd_spacenet4                     | BCE +                   | Adam              | 1e-4          | SpaceNet 4             | Footprints only | 12         | 60              | None                |
+|                                    | Jaccard (4:1)           | default params    | with decay    | Pan-sharpened RGB      |                 |            |                 |                     |
++------------------------------------+-------------------------+-------------------+---------------+------------------------+-----------------+------------+-----------------+---------------------+
+| selimsef_spacenet4_resnet34unet    | Focal + Dice            | AdamW             | 2e-4          | SpaceNet 4             | 3-channel (FP,  | 42         | 70              | ImageNet (encoder   |
+|                                    | (1:1)                   | 1e-3 weight decay | with decay    | Pan-sharpened RGB+NIR  | (edge, contact) |            |                 | only)               |
++------------------------------------+-------------------------+-------------------+---------------+------------------------+-----------------+------------+-----------------+---------------------+
+| selimsef_spacenet4_densenet121unet | Focal + Dice            | AdamW             | 2e-4          | SpaceNet 4             | 3-channel (FP,  | 32         | 70              | ImageNet (encoder   |
+|                                    | (1:1)                   | 1e-3 weight decay | with decay    | Pan-sharpened RGB      | (edge, contact) |            |                 | only)               |
++------------------------------------+-------------------------+-------------------+---------------+------------------------+-----------------+------------+-----------------+---------------------+
+| selimsef_spacenet4_densenet161unet | Focal + Dice            | AdamW             | 2e-4          | SpaceNet 4             | 3-channel (FP,  | 20         | 60              | ImageNet (encoder   |
+|                                    | (1:1)                   | 1e-3 weight decay | with decay    | Pan-sharpened RGB      | (edge, contact) |            |                 | only)               |
++------------------------------------+-------------------------+-------------------+---------------+------------------------+-----------------+------------+-----------------+---------------------+
+
+.. _XDXDconfig: https://github.com/CosmiQ/solaris/blob/master/solaris/nets/configs/xdxd_spacenet4.yml
+.. _ssresnet34config: https://github.com/CosmiQ/solaris/blob/master/solaris/nets/configs/selimsef_resnet34unet_spacenet4.yml
+.. _ssdense121config: https://github.com/CosmiQ/solaris/blob/master/solaris/nets/configs/selimsef_densenet121unet_spacenet4.yml
+.. _ssdense161config: https://github.com/CosmiQ/solaris/blob/master/solaris/nets/configs/selimsef_densenet161unet_spacenet4.yml
+.. _XDXDweights: https://s3.amazonaws.com/spacenet-dataset/spacenet-model-weights/spacenet-4/xdxd_spacenet4_solaris_weights.pth
+.. _ssresnet34weights: https://s3.amazonaws.com/spacenet-dataset/spacenet-model-weights/spacenet-4/selimsef_spacenet4_resnet34unet_solaris_weights.pth
+.. _ssdense121weights: https://s3.amazonaws.com/spacenet-dataset/spacenet-model-weights/spacenet-4/selimsef_spacenet4_densenet121unet_solaris_weights.pth
+.. _ssdense161weights: https://s3.amazonaws.com/spacenet-dataset/spacenet-model-weights/spacenet-4/selimsef_spacenet4_densenet161unet_solaris_weights.pth
diff --git a/docs/tutorials/cli_eval.rst b/docs/tutorials/cli_eval.rst
diff --git a/docs/tutorials/index.rst b/docs/tutorials/index.rst
@@ -1,3 +1,5 @@
+.. _tutorials_index:
+
 ##############################
 Solaris Tutorials and Cookbook
 ##############################
@@ -40,7 +42,7 @@ on creating configuration files and running the CLI can be found below.
 * `Creating reference files to help solaris find your imagery <notebooks/creating_im_reference_csvs.ipynb>`_
 * `Creating training masks with the solaris CLI <notebooks/cli_mask_creation.ipynb>`_
 * `Running a full deep learning pipeline using the solaris CLI <notebooks/cli_ml_pipeline.ipynb>`_
-* :doc:`Evaluating prediction quality on SpaceNet data with the solaris CLI <cli_eval>`
+* `Evaluating prediction quality on SpaceNet data with the solaris CLI <notebooks/cli_spacenet_evaluation.ipynb>`_
 
 If these relatively narrow use cases don't cover your needs, the ``solaris`` python
 API can help!
@@ -62,10 +64,10 @@ the tutorials below.
 
 * `Tiling imagery <notebooks/api_tiling_tutorial.ipynb>`_
 * `Creating training masks <notebooks/api_masks_tutorial.ipynb>`_
-* :doc:`Training a SpaceNet model <notebooks/api_training_spacenet>`
-* :doc:`Inference with a pre-trained SpaceNet model <notebooks/api_inference_spacenet>`
-* :doc:`Training a custom model <notebooks/api_training_custom>`
-* :doc:`Converting pixel masks to vector labels <notebooks/api_mask_to_vector>`
+* `Training a SpaceNet model <notebooks/api_training_spacenet.ipynb>`_
+* `Inference with a pre-trained SpaceNet model <notebooks/api_inference_spacenet.ipynb>`_
+* `Training a custom model <notebooks/api_training_custom.ipynb>`_
+* `Converting pixel masks to vector labels <notebooks/api_mask_to_vector.ipynb>`_
 * `Scoring your model's performance with the solaris Python API <notebooks/api_evaluation_tutorial.ipynb>`_
 
 

diff --git a/docs/tutorials/notebooks/api_inference_spacenet.ipynb b/docs/tutorials/notebooks/api_inference_spacenet.ipynb
@@ -0,0 +1,161 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Inferencing with included SpaceNet models and the `solaris` Python API\n",
+    "\n",
+    "We've included a number of SpaceNet models with `solaris`, including pre-trained model weights. You can find more information about your model choices [here](../pretrained_models.html) and the original competitors' code for the models [here](https://github.com/spacenetchallenge/spacenet_off_nadir_solutions).\n",
+    "\n",
+    "For this tutorial we'll walk through running inference with XD_XD's SpaceNet 4 model. We'll use the config file for that model, which you can find [here](https://github.com/CosmiQ/solaris/blob/master/solaris/nets/configs/xdxd_spacenet4.yml).\n",
+    "\n",
+    "You'll also need to [create the image reference files](creating_im_reference_csvs.ipynb) before you start.\n",
+    "\n",
+    "First, we'll load the configuration:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'model_name': 'xdxd_spacenet4',\n",
+       " 'model_path': None,\n",
+       " 'train': False,\n",
+       " 'infer': True,\n",
+       " 'pretrained': True,\n",
+       " 'nn_framework': 'torch',\n",
+       " 'batch_size': 12,\n",
+       " 'data_specs': {'width': 512,\n",
+       "  'height': 512,\n",
+       "  'image_type': 'zscore',\n",
+       "  'rescale': False,\n",
+       "  'rescale_minima': 'auto',\n",
+       "  'rescale_maxima': 'auto',\n",
+       "  'channels': 4,\n",
+       "  'label_type': 'mask',\n",
+       "  'is_categorical': False,\n",
+       "  'mask_channels': 1,\n",
+       "  'val_holdout_frac': 0.2,\n",
+       "  'data_workers': None},\n",
+       " 'training_data_csv': '/path/to/training_df.csv',\n",
+       " 'validation_data_csv': None,\n",
+       " 'inference_data_csv': '/path/to/test_df.csv',\n",
+       " 'training_augmentation': {'augmentations': {'DropChannel': {'idx': 3,\n",
+       "    'axis': 2},\n",
+       "   'HorizontalFlip': {'p': 0.5},\n",
+       "   'RandomRotate90': {'p': 0.5},\n",
+       "   'RandomCrop': {'height': 512, 'width': 512, 'p': 1.0},\n",
+       "   'Normalize': {'mean': [0.006479, 0.009328, 0.01123],\n",
+       "    'std': [0.004986, 0.004964, 0.00495],\n",
+       "    'max_pixel_value': 65535.0,\n",
+       "    'p': 1.0}},\n",
+       "  'p': 1.0,\n",
+       "  'shuffle': True},\n",
+       " 'validation_augmentation': {'augmentations': {'DropChannel': {'idx': 3,\n",
+       "    'axis': 2},\n",
+       "   'CenterCrop': {'height': 512, 'width': 512, 'p': 1.0},\n",
+       "   'Normalize': {'mean': [0.006479, 0.009328, 0.01123],\n",
+       "    'std': [0.004986, 0.004964, 0.00495],\n",
+       "    'max_pixel_value': 65535.0,\n",
+       "    'p': 1.0}},\n",
+       "  'p': 1.0},\n",
+       " 'inference_augmentation': {'augmentations': {'DropChannel': {'idx': 3,\n",
+       "    'axis': 2,\n",
+       "    'p': 1.0},\n",
+       "   'Normalize': {'mean': [0.006479, 0.009328, 0.01123],\n",
+       "    'std': [0.004986, 0.004964, 0.00495],\n",
+       "    'max_pixel_value': 65535.0,\n",
+       "    'p': 1.0}},\n",
+       "  'p': 1.0},\n",
+       " 'training': {'epochs': 60,\n",
+       "  'steps_per_epoch': None,\n",
+       "  'optimizer': 'Adam',\n",
+       "  'lr': 0.0001,\n",
+       "  'opt_args': None,\n",
+       "  'loss': {'bcewithlogits': None, 'jaccard': None},\n",
+       "  'loss_weights': {'bcewithlogits': 10, 'jaccard': 2.5},\n",
+       "  'metrics': {'training': None, 'validation': None},\n",
+       "  'checkpoint_frequency': 10,\n",
+       "  'callbacks': {'model_checkpoint': {'filepath': 'xdxd_best.pth',\n",
+       "    'monitor': 'val_loss'}},\n",
+       "  'model_dest_path': 'xdxd.pth',\n",
+       "  'verbose': True},\n",
+       " 'inference': {'window_step_size_x': None,\n",
+       "  'window_step_size_y': None,\n",
+       "  'output_dir': 'inference_out/'}}"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import solaris as sol\n",
+    "\n",
+    "config = sol.utils.config.parse('/Users/nweir/code/cosmiq_repos/solaris/solaris/nets/configs/xdxd_spacenet4.yml')\n",
+    "config"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the YAML gets parsed into a set of nested dictionaries by `solaris`. Relevant pieces of that config then get read during training.\n",
+    "\n",
+    "Inferencing is _very_ similar to [training](api_training_spacenet.ipynb), with one major difference: you'll load in and pass the reference CSV for your inference dataset as an argument to the `Inferer` object. `solaris` is set up this way so that you can quickly and easily iterate through inference on a number of inputs without having to re-instantiate your inferencer each time."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "inferer = sol.nets.infer.Inferer(config)\n",
+    "inference_df = sol.nets.infer.get_infer_df(config)\n",
+    "inferer(inference_df)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The above commands will create prediction segmentation masks for each input image in the `output_dir` specified in your `config`. You can then [use sol.vector.mask.mask_to_poly_geojson to convert these predicted masks to vector-formatted geometries](api_mask_to_vector.ipynb)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "solaris",
+   "language": "python",
+   "name": "solaris"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}