source-extraction-demo.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:7b878fd1786c77cb1fa414b2394a64a10d0915e8a5978296048945ef03aee2b6"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "<img style='float: left' width=\"150px\" src=\"http://thunder-project.github.io/images/logo-outline.svg\">\n",
      "<br>\n",
      "<br>\n",
      "<br>\n",
      "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Source extraction in <a href='http://thunder-project.github.io/'><font color='#EB8B34'>Thunder</font></a>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This notebook demos the source extraction API, including visualizations using Lightning\n",
      "\n",
      "**Environment**\n",
      "- Run on local environment (Mac OS X)\n",
      "- Spark version 1.3.0, Thunder version 0.6.0.dev, Python version 2.7.6\n",
      "\n",
      "**Data**\n",
      "- Data are generated locally using the sources example data type\n",
      "- These data are random with trivial to identify sources, and are meant **only** to illustrate the API \n"
     ]
    },
    {
     "cell_type": "heading",
     "level": 6,
     "metadata": {},
     "source": [
      "<hr>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Set up plotting and other imports"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from lightning import Lightning\n",
      "lgn = Lightning(ipython=True, host='http://lightning-sources-demo.herokuapp.com/')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 11
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from thunder import SourceExtraction"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Generate a data set"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data, ts, truth = tsc.makeExample('sources', centers=10, noise=1.0, returnParams=True)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 17
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Feature method example: localmax"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A top-level factory method enables easy construction of source extraction methods. The first type of method we'll consider is a feature method, which proceeds in two stages: create some kind of feature from the data set (usually an array), and then apply some operator to that image to identify sources. As a particularly simple example, the `localmax` method computes a mean across time as the feature, and then finds sources as regions centered on local maxima. (*Note*: keyword arguments are propgated to the component functions, which does not support built-in error checking for invalid passed keyword arguments in the factory.)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model = SourceExtraction('localmax', minDistance=5, maxSources=10, radius=5).fit(data)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A model is a collection of sources"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 19,
       "text": [
        "SourceModel\n",
        "10 sources"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It has support for easy indexing into the sources"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model[0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "Source\n",
        "center: [40.0, 102.0]\n",
        "bbox: [36, 98, 44, 106]"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Along with its primary attributes, a source as lazily computed properties that are computed once upon request and then cached (avoiding expensive recomputation). These are avaialble in aggregate on the model object. For example, each source has the ability to compute a bounding polygon. We can look at just one, rendered using lightning `imagedraw` method which draws regions on top of images and supports freehand drawing."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.imagepoly(data.mean(), model[0].polygon)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image-poly\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"122\" class=\"feed-item-container\"><div data-type=\"image-poly\" data-data=\"{&quot;coordinates&quot;:[[[100,44],[104,44],[106,42],[106,38],[104,36],[100,36],[98,38],[98,42]]]}\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/ZhGpSUkQnB85ea2yogkl2hZ7L221vAif&quot;]\" data-options=\"null\" id=\"viz-122\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image-poly\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image-poly\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "<lightning.types.images.ImagePoly at 0x10a86e850>"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Or look at all of them."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.imagepoly(data.mean(), model.polygons)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image-poly\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"123\" class=\"feed-item-container\"><div data-type=\"image-poly\" data-data=\"{&quot;coordinates&quot;:[[[100,44],[104,44],[106,42],[106,38],[104,36],[100,36],[98,38],[98,42]],[[94,35],[94,31],[92,29],[88,29],[86,31],[86,35],[88,37],[92,37]],[[164,36],[164,32],[162,30],[158,30],[156,32],[156,36],[158,38],[162,38]],[[116,68],[116,64],[114,62],[110,62],[108,64],[108,68],[110,70],[114,70]],[[178,23],[176,25],[176,29],[178,31],[182,31],[184,29],[184,25],[182,23]],[[48,45],[48,41],[46,39],[42,39],[40,41],[40,45],[42,47],[46,47]],[[67,75],[67,71],[65,69],[61,69],[59,71],[59,75],[61,77],[65,77]],[[80,54],[80,50],[78,48],[74,48],[72,50],[72,54],[74,56],[78,56]],[[44,26],[48,26],[50,24],[50,20],[48,18],[44,18],[42,20],[42,24]],[[46,67],[46,63],[44,61],[40,61],[38,63],[38,67],[40,69],[44,69]]]}\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/G1SE1BU9uSOD04O9yOBIREPQRJYm6JOC&quot;]\" data-options=\"null\" id=\"viz-123\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image-poly\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image-poly\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 22,
       "text": [
        "<lightning.types.images.ImagePoly at 0x10806c650>"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "An alternate output renders regions as masks, with or without values and outlines via the `binary` and `outline` flags"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "m = model.masks(base=data.mean(), outline=True)\n",
      "lgn.image(m)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"124\" class=\"feed-item-container\"><div data-type=\"image\" data-data=\"null\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/VLeRE1mbIrRXOnYvbXKF0rqItcB42TSz&quot;]\" data-options=\"null\" id=\"viz-124\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "<lightning.types.images.Image at 0x10a9b9150>"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "m = truth.masks(base=data.mean(), outline=True)\n",
      "lgn.image(m)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"125\" class=\"feed-item-container\"><div data-type=\"image\" data-data=\"null\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/NPwvNeoX3Kulu373lKtTcTrJOXo9J5f7&quot;]\" data-options=\"null\" id=\"viz-125\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "<lightning.types.images.Image at 0x10a9b9f90>"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If we pass a `SourceModel` as the base when generating the mask image, it'll superimpose them together, useful for comparisons"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "m = model.masks((100,200), base=truth)\n",
      "lgn.image(m)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"126\" class=\"feed-item-container\"><div data-type=\"image\" data-data=\"null\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/RaKTOrDZnz9yNtFpV30hHczbIKkeOucW&quot;]\" data-options=\"null\" id=\"viz-126\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "<lightning.types.images.Image at 0x10a9b9a10>"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Finally, we can do quantitative comparisons between two sets of regions"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Matching tries to assign each source in the first model to another source in the second model. By default, matches are not unique"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model.match(truth)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 26,
       "text": [
        "[1, 2, 6, 8, 3, 5, 0, 7, 9, 5]"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "But we can force them to be unique"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model.match(truth, unique=True)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "[1, 2, 6, 8, 3, 5, 0, 7, 9, 4]"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Computing distance estimates the distance between each source in the first model to the closest source in the second model, after uniquely matching"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model.distance(truth)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "array([  3.        ,   1.        ,   1.        ,   2.23606798,\n",
        "         1.41421356,   3.16227766,   2.        ,   3.        ,\n",
        "         2.82842712,  66.21178143])"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Similarity estimates the fraction of sources in the first model that are close to a source in the second, within some threshold. Usually we use this to estimate the fraction of truth sources that are found by the estimated model."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "truth.similarity(model)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "0.90000000000000002"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Modularity"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can do the same operation but construct from modular components rather than use the factory. The key components of a `FeatureMethod` are a `FeatureCreator` and a `FeatureAlgorithm`, as well as a more generic `Cleaner`. This obeys normal argument checking on component classes."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from thunder.extraction import FeatureMethod, LocalMaxFeatureAlgorithm, MeanFeatureCreator\n",
      "\n",
      "algorithm = LocalMaxFeatureAlgorithm(minDistance=5, maxSources=10, radius=5)\n",
      "creator = MeanFeatureCreator()\n",
      "\n",
      "model = FeatureMethod(algorithm, creator).fit(data)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 30
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.imagepoly(data.mean(), model.polygons)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image-poly\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"127\" class=\"feed-item-container\"><div data-type=\"image-poly\" data-data=\"{&quot;coordinates&quot;:[[[100,44],[104,44],[106,42],[106,38],[104,36],[100,36],[98,38],[98,42]],[[94,35],[94,31],[92,29],[88,29],[86,31],[86,35],[88,37],[92,37]],[[164,36],[164,32],[162,30],[158,30],[156,32],[156,36],[158,38],[162,38]],[[116,68],[116,64],[114,62],[110,62],[108,64],[108,68],[110,70],[114,70]],[[178,23],[176,25],[176,29],[178,31],[182,31],[184,29],[184,25],[182,23]],[[48,45],[48,41],[46,39],[42,39],[40,41],[40,45],[42,47],[46,47]],[[67,75],[67,71],[65,69],[61,69],[59,71],[59,75],[61,77],[65,77]],[[80,54],[80,50],[78,48],[74,48],[72,50],[72,54],[74,56],[78,56]],[[44,26],[48,26],[50,24],[50,20],[48,18],[44,18],[42,20],[42,24]],[[46,67],[46,63],[44,61],[40,61],[38,63],[38,67],[40,69],[44,69]]]}\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/ZuyYHobiwPErJ88emhUqul8sIr1ygzhC&quot;]\" data-options=\"null\" id=\"viz-127\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image-poly\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image-poly\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 31,
       "text": [
        "<lightning.types.images.ImagePoly at 0x10a001d10>"
       ]
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Saving and loading models"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Saving as JSON"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Models can easily be saved to JSON. We slightly modify Thunder's built-in serialization functionality to yield a simplified, more human-readable output."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model.save(\"~/sources.json\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Models contain both primary attributes (id, coordinates, values) and lazily derived attributes (center, bounding box, polygon boundary, etc.). By default, lazy attributes are not saved, as they are not neccessary for reconstituing sources. However, they can be optionally selected for inclusion, in which case they will also be persisted (and automatically reloaded, potentially saving recomputation)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model.save(\"~/sources.json\", include=\"polygon\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model.save(\"~/sources.json\", include=[\"center\", \"area\", \"polygon\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Loading from JSON"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The same model can then be loaded back in"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model = SourceExtraction.load(\"~/sources.json\")\n",
      "lgn.imagepoly(data.mean(), model.polygons)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image-poly\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"128\" class=\"feed-item-container\"><div data-type=\"image-poly\" data-data=\"{&quot;coordinates&quot;:[[[100,44],[104,44],[106,42],[106,38],[104,36],[100,36],[98,38],[98,42]],[[94,35],[94,31],[92,29],[88,29],[86,31],[86,35],[88,37],[92,37]],[[164,36],[164,32],[162,30],[158,30],[156,32],[156,36],[158,38],[162,38]],[[116,68],[116,64],[114,62],[110,62],[108,64],[108,68],[110,70],[114,70]],[[178,23],[176,25],[176,29],[178,31],[182,31],[184,29],[184,25],[182,23]],[[48,45],[48,41],[46,39],[42,39],[40,41],[40,45],[42,47],[46,47]],[[67,75],[67,71],[65,69],[61,69],[59,71],[59,75],[61,77],[65,77]],[[80,54],[80,50],[78,48],[74,48],[72,50],[72,54],[74,56],[78,56]],[[44,26],[48,26],[50,24],[50,20],[48,18],[44,18],[42,20],[42,24]],[[46,67],[46,63],[44,61],[40,61],[38,63],[38,67],[40,69],[44,69]]]}\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/9oUHXgYxxeLrq90Bp1QhNTkf4YMOfXGw&quot;]\" data-options=\"null\" id=\"viz-128\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image-poly\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image-poly\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "<lightning.types.images.ImagePoly at 0x10a86ead0>"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Getting time series"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Once a source model is constructed, we can use it to extract time series from a data set. Currently only simple averaging over regions is supported, but we will add support for weighted regions, and methods that do some form of local unmixing (via regression)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data, ts, centers = tsc.makeExample('sources', centers=3, noise=0.1, returnParams=True)\n",
      "model = SourceExtraction('localmax', minDistance=5, maxSources=3, radius=5).fit(data)\n",
      "tsrecon = model.transform(data)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 36
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.imagepoly(data.mean(), model.polygons)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image-poly\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"129\" class=\"feed-item-container\"><div data-type=\"image-poly\" data-data=\"{&quot;coordinates&quot;:[[[57,27],[57,23],[55,21],[51,21],[49,23],[49,27],[51,29],[55,29]],[[70,60],[68,62],[68,66],[70,68],[74,68],[76,66],[76,62],[74,60]],[[152,76],[156,76],[158,74],[158,70],[156,68],[152,68],[150,70],[150,74]]]}\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/sHlqlTKYIP5WmwJTtRYvixgutYH22MGn&quot;]\" data-options=\"null\" id=\"viz-129\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image-poly\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image-poly\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "<lightning.types.images.ImagePoly at 0x10a001d50>"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Compare real and estimated time series (note that colors may not correspond one-to-one because order is arbitrary)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.linestacked(tsrecon)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=line-stacked\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"130\" class=\"feed-item-container\"><div data-type=\"line-stacked\" data-data=\"{&quot;series&quot;:[[0.00738846919561331,0.008281965025323164,-0.013082899370889238,0.7203067181017226,0.006647047827574756,0.3642690227714993,0.4979834076118827,-0.007999013472731098,0.6208057674706787,0.0020605587321254797,0.004066266558995915,0.35575820979349754,-0.00416719401479157,0.36852195251675507,0.5231943439468946,0.00994691874990493,-0.011211518410564147,0.3831375118267272,0.684998567634132,-0.013826987509245405,0.002195387297132303,-0.007898217286920417,0.8409010897742193,0.002500269944862455,-0.007674479116041959,0.2590658055621959,1.1127511163766075,0.005341122146200271,-0.029186824496780513,0.9214681023991276,-0.013702877000395797,0.9741808895479903,0.2788040856021882,0.010784368181742157,1.0525596993514563,-0.0024897338670155523,0.07953677587480219,0.3616763782453909,-0.004771673937943813,0.583545213409987,0.6338044196743461,0.012919625385780778,-0.012915522605172269,-0.00467862139781401,-0.0025327118895701315,0.01601054346788824,0.009309449808632813,-0.006044305044880971,-0.005520878421870305,0.007418916634416044,-0.01615645001115472,-0.01526836267441063,1.4488382411788396,0.436354539729871,0.4670094253617956,0.15409377231843183,0.31115329140366405,0.12530292599276602,0.07670676466601123,0.09781915754716407,0.6046455064735542,0.4707254341985049,0.005242108421164706,0.24722621687244822,0.11023644092444689,0.6990905983521645,0.6045059433369989,0.0035709618455935133,0.009111145459606562,0.23168035034102324,1.3573447904298255,0.0274116761657077,0.15698127720435168,-0.020019643120768343,0.2795932796892999,0.3257953236900516,0.015876797108984122,-0.03266461437228891,0.315756014825312,-0.004133363909371336,0.003592204756035749,-0.013447900651026277,0.1699410753182611,0.00753500013373396,-0.02179845453745663,0.4459929135153513,0.008086676946579411,0.6264854871260507,0.5626042785220242,0.013963552649554926,0.590616662268075,0.19271755361982218,-0.00598422610618124,0.00645263756478906,0.45091324249684994,0.3350489281403507,0.003947058809101629,0.007437439261446298,-0.009264667890617044,0.9122357161768923],[0.6598793578503283,0.10056666373507939,0.22905321994228217,0.19842055142905418,0.09521806915906283,-0.010359785425553627,0.6084870577003456,0.49690461721315415,0.7466326921564935,0.5509428950900033,1.182622806430854,-0.009408747971100894,0.011532579595244952,0.07104724366260222,0.017863015492387897,0.0016597285206720724,0.0029953608007354587,0.14298797468672653,0.10606836443277681,0.00008676828951448681,-0.014532236406436635,0.3686286878691256,-0.012952180319901476,-0.005279885433630711,-0.01147559897188197,-0.0048331530063441645,0.006037033597627823,0.0019451551540326992,0.005599865622649883,0.002884700505621171,-0.00023076261316403232,-0.0010113286322142234,0.840110840410774,1.0581882127903441,1.0766952155399319,0.01702518711883991,-0.011603836734691215,0.4017131931036968,-0.0195040916070925,0.0012656787588109888,0.9061801395004723,0.006748984922219031,0.007647458442255225,1.6668548418322637,0.0069506523446420535,1.5406614968342383,-0.013154974019549846,-0.0006466748882028409,-0.024253481487917398,0.22720749707569468,1.5936227390202489,0.0041743761687932975,0.009714521200673875,0.02560143184423588,0.005226941882127902,0.4464120101859035,0.35499331677197726,0.6221690251379038,0.017500186996017315,0.2528453252569006,-0.0015482439811617661,-0.0019013759821757629,0.4482084037236321,1.0096376626411738,0.008558438711295297,0.7516346309092209,-0.01422324998983204,0.8648422991458142,1.0062730473197787,0.038238660726876386,0.5417084787973551,0.009429696743190587,0.6034777675589028,0.15083240232641107,-0.007570250857325366,0.008495131851954038,-0.0038297574665726217,0.1061426358114356,0.0030038401450884116,0.23576128570532992,0.004940230291401856,0.278928202120705,-0.0015212432592434425,0.00829253733256494,0.43120912846672266,0.8998707652834076,0.0021503266202676833,0.0051685093957471355,0.46436422812576306,0.014174677397329853,0.2976955827061204,0.7004012313496453,1.4279487232700114,-0.011061117348411854,-0.008003194135963392,0.00759359786715425,0.39715229133264374,0.4965404293576405,-0.004845188327127776,0.03266701890562653],[0.39684652404236676,0.6647581273724703,0.1306633928782544,0.01692694088479908,0.5733826697621645,-0.0008131867565813589,-0.00563105015596309,-0.028891820298804802,0.014352060500048494,0.01753349421192098,0.3062582497439156,-0.016531431143965575,0.8705817209467901,0.6194881185434724,0.1399579967580553,0.9466481070808721,1.7283536765990035,0.6753971361394684,0.8307879263648568,0.5807047815577016,-0.012405728680169014,-0.0077149619939173,0.997197964851307,0.011646513829893257,-0.01801638944104185,0.4591818813439798,1.0588402080497723,0.03867093934008994,0.00332583504571829,0.17624163661291253,-0.009208139853004062,0.010936171329679622,0.8710583807144057,1.5609579452748057,0.7656438966655295,-0.018366075917720805,0.23924773770229724,-0.006478523013705265,-0.008912252316566746,0.19127022776950306,0.027089893761292956,-0.012066068107954001,-0.0010980132289582117,0.02389611525273042,0.14034822679616593,-0.00549964625109814,0.6748819452105964,0.007722044497217228,-0.0027214238940185126,0.7618297078269326,-0.01455734998663601,0.009517698798219034,-0.004977194244418784,0.0037733843197389756,0.0953150597617335,-0.01629151193912042,0.009145586166017789,0.18662236922847536,-0.00859783394678012,0.0022872516642133745,0.19190275339937493,0.07109731438688088,1.338830329807276,0.03392878713701806,0.005134521404828589,0.22890487880955734,0.24353029741858445,0.052245196985417894,0.06123347057248186,0.002840240616246894,0.489270728872838,0.7151734662195555,-0.0012632847733376957,0.06829979566586739,0.03482958282019054,0.5263210880935106,1.0550830862190148,0.7293456205483618,-0.00652425682210091,0.17904144413000156,0.3096232679757598,0.00011111563854877517,1.0348187246511464,0.004962420192980402,-0.006814648013515639,0.02099113301461746,0.21011742844043504,0.1750885484630139,0.5563456010239438,0.5554973077937578,0.008132046117016685,0.5273338319527139,-0.0018926605128750742,-0.018428534189201644,0.2610543103756754,0.7205851846532568,0.021593627737761897,0.9017840796036137,0.01236861327763696,0.3311290828806827]]}\" data-images=\"null\" data-options=\"null\" id=\"viz-130\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"line-stacked\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"line-stacked\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 38,
       "text": [
        "<lightning.types.plots.LineStacked at 0x10a001c90>"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.linestacked(ts)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=line-stacked\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"131\" class=\"feed-item-container\"><div data-type=\"line-stacked\" data-data=\"{&quot;series&quot;:[[0.6776777990116172,1.1568235618978417,0.24097238325936907,0,1.0244422328391707,0,0,0,0,0,0.5123366390307289,0,1.4798639186375528,1.0859436318420883,0.23880288200848357,1.6531861833286672,2.9804350602984697,1.1473283508237184,1.434111730130378,1.0100856926415014,0,0,1.6833452941901277,0,0,0.7765674255056633,1.8383549858037471,0.05687460775589493,0,0.315322212877665,0,0,1.5024632618775913,2.687556001598072,1.3213772302207731,0,0.38518836424842046,0,0,0.29867836045599117,0,0,0,0,0.23683685733397505,0,1.1823572135271014,0,0,1.288069145741345,0,0,0,0,0.14120287305424858,0,0,0.33030803135488684,0,0,0.3212664617016314,0.144849675052796,2.2848667178358895,0.08369294421558596,0,0.37977531929045427,0.446909759441977,0.05543255042727157,0.09048091791031583,0,0.838080201889484,1.1927642031056955,0,0.11086889792761967,0,0.8976543947114808,1.7729464102332704,1.3010552503350297,0,0.3159811769419279,0.540929859413742,0,1.7725173915737302,0,0,0,0.30392905485249083,0.2923872978888191,0.9611148659017975,0.9394769962863136,0,0.9200971526748412,0,0,0.4473605910104449,1.1994481434827655,0.026175665233332288,1.5323604151858048,0,0.6044696057936731],[1.13392205313098,0.12101433616689883,0.39170084624592927,0.3740528768302972,0.15792454940981593,0,1.032825502558261,0.8892763970408439,1.2980826819756306,0.9300455194448857,2.0347962639978316,0,0,0.15063321466823304,0.052834255870007,0,0,0.2637109971017614,0.19383617011577595,0,0,0.6216653289392479,0,0,0,0,0,0,0,0,0,0,1.4567417917872514,1.8348873377685748,1.8437355265274618,0,0,0.6901004586114465,0,0,1.566461803642018,0,0,2.8117548816415177,0,2.635913363258343,0,0,0,0.3974020068173374,2.7723278750701383,0,0,0.04969249043053036,0,0.7779915654692345,0.6327990621461964,1.08352599066101,0,0.44091042180474116,0,0,0.7756072542436403,1.7361184779544778,0.019713343962586696,1.2852104467037346,0,1.4882048807937351,1.7067904003322443,0,0.9291297057060421,0,1.030837964381759,0.2806841925169632,0,0,0,0.1837903709775338,0,0.37144938061844585,0,0.47857003374985224,0,0,0.7306309600962326,1.5766741961890407,0,0,0.8160210219693965,0,0.5302640303512435,1.1959288267060966,2.5017825429282294,0,0,0,0.6613908808189964,0.8693348864030388,0,0.03588113923631083],[0,0,0,1.2521272459356152,0,0.642721739486982,0.8644054293012993,0,1.1022381999154351,0,0,0.6264787756201413,0.005233374822370221,0.5880868161857159,0.8936629701248078,0,0,0.6456333653733597,1.1579518504459803,0,0,0,1.4321955278939984,0,0,0.4377038098561162,1.9014663336131277,0,0,1.5852486998400288,0,1.684812417552854,0.47599467346047686,0,1.8402996792727435,0,0.1134340459037002,0.59613754316141,0,0.9744273967074929,1.0994426259623657,0,0,0,0,0,0,0,0,0,0,0,2.499131085530972,0.771764275547429,0.801118950783413,0.30055972034259815,0.5632588920524085,0.26864686773901825,0.14444802606556867,0.14064971629823714,1.012085191886353,0.8133584156033324,0,0.4382683245757806,0.1751476192167608,1.249271043360931,1.0427337633297116,0,0,0.39751802208037473,2.362336381706967,0,0.24007968465661353,0,0.4679611849227013,0.5429809404380737,0,0,0.5242422565990237,0,0,0,0.29004326286605886,0,0,0.7957981587451997,0,1.0804182386502912,0.9347519483059723,0,1.056301857444435,0.34816581259399687,0,0,0.7767511900883913,0.5775398449413703,0,0,0,1.5747045519208407]]}\" data-images=\"null\" data-options=\"null\" id=\"viz-131\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"line-stacked\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"line-stacked\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        "<lightning.types.plots.LineStacked at 0x10a001dd0>"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can also get both the model and the time series all at once by calling `run`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model, ts = SourceExtraction('localmax', minDistance=5, maxSources=3, radius=5).run(data)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Block method example: nmf"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Block methods use an alternative procedure in which the data set is split into small blocks of xyzt (or xyt), an algorithm is run on each block to genreate a set of candidate sources, and then those sources are merged together. The key components of a `BlockMethod` are a `BlockAlgorithm` and a `BlockMerger`, as well as a more generic `Cleaner`. Here we use the factory to call the method `nmf`, which combined an NMF decomposition as the algorithm as (currently very simple!) merger."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data, ts, truth = tsc.makeExample('sources', centers=10, noise=0.4, returnParams=True)\n",
      "model = SourceExtraction(\"nmf\", percentile=90).fit(data, size=(50,50))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Because the output is once again a model we can access and visualize geometric properties the same as above. Note that this doesn't neccessarily do a terrific job. There is much to improve even in this basic case for both the algorithm and the merging, but more importantly, the point of the architecture is to enable new, better algorithms."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.imagepoly(data.mean(), model.polygons)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image-poly\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"132\" class=\"feed-item-container\"><div data-type=\"image-poly\" data-data=\"{&quot;coordinates&quot;:[[[28,44],[36,46],[38,41],[39,35],[35,33],[30,32],[27,34],[27,41]],[[27,24],[21,22],[18,25],[15,31],[18,35],[21,38],[22,38],[28,36],[29,35],[30,31],[28,25]],[[39,41],[33,46],[36,49],[46,49],[48,46],[46,43],[40,41]],[[49,69],[44,67],[40,70],[39,74],[43,80],[48,80],[49,79]],[[81,68],[76,66],[71,69],[70,71],[70,76],[72,78],[76,80],[80,79],[83,75],[83,71]],[[102,49],[112,49],[113,48],[123,33],[124,30],[123,28],[120,25],[118,25],[115,26],[113,27],[100,45],[101,48]],[[128,27],[137,27],[138,26],[138,20],[136,17],[134,15],[128,15],[127,16],[125,20],[125,22],[126,24]],[[119,54],[111,58],[113,64],[114,65],[118,68],[120,69],[127,64],[125,57]],[[159,55],[152,55],[150,57],[150,65],[151,66],[154,68],[157,68],[162,64],[163,60]]]}\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/DN2LKpwNRyxdA0W99GEQ2lHrfw7DKkqw&quot;]\" data-options=\"null\" id=\"viz-132\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image-poly\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image-poly\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        "<lightning.types.images.ImagePoly at 0x10a001c10>"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lgn.image(model.masks([100,200], outline=True, base=truth))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, maximum-scale=1\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=10; IE=9; IE=8; IE=7; IE=EDGE\"/><meta charset=\"UTF-8\"/><link href=\"//fonts.googleapis.com/css?family=Open+Sans:400,700\" rel=\"stylesheet\" type=\"text/css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/app.css\"/><link rel=\"stylesheet\" href=\"http://lightning-sources-demo.herokuapp.com/css/dynamic/viz/?visualizations[]=image\"/><div id=\"lightning-body\" class=\"container content wrap push\"><div class=\"feed-container\"><div data-model=\"visualization\" data-model-id=\"133\" class=\"feed-item-container\"><div data-type=\"image\" data-data=\"null\" data-images=\"[&quot;https://s3.amazonaws.com/lightning.demos/sessions/20/FHWReUUBvxl35N6Y6RZGW52xZmR5dnVG&quot;]\" data-options=\"null\" id=\"viz-133\" data-initialized=\"false\" class=\"feed-item\"></div></div></div></div><script>window.lightning = window.lightning || {};\n",
        "window.lightning.host = \"http://lightning-sources-demo.herokuapp.com/\" || 'http://127.0.0.1:3000/';\n",
        "window.lightning.vizCount = (window.lightning.vizCount + 1) || 1;\n",
        "window.lightning.requiredVizTypes = window.lightning.requiredVizTypes || [];\n",
        "if(window.lightning.requiredVizTypes.indexOf(\"image\") === -1) {\n",
        "    window.lightning.requiredVizTypes.push(\"image\");\n",
        "}\n",
        "window._require = window.require;\n",
        "window.require = undefined;\n",
        "window._define = window.define;\n",
        "window.define = undefined;</script><script src=\"http://lightning-sources-demo.herokuapp.com/js/embed.js\"></script>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 43,
       "text": [
        "<lightning.types.images.Image at 0x10a8b7190>"
       ]
      }
     ],
     "prompt_number": 43
    }
   ],
   "metadata": {}
  }
 ]
}