examples/notebooks/Debugging with monkeys.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Debugging with monkeys\n",
    "\n",
    "This guide will demonstrate how `monkeys` can be used to discover and diagnose software bugs. By defining how you expect the components of your codebase to be able to interact and the invariants that should hold during those interactions, `monkeys` can find combinations for which these invariants do not hold. \n",
    "\n",
    "In this notebook, we'll be looking for bugs in a simple class which supports serialization to and deserialization from JSON. We want to make sure that under all circumstances, the class can be serialized without error, the JSON produced is valid, that instance properties are of the correct types, and that class-JSON-class round trips end up giving us the same object we had originally.\n",
    "\n",
    "First, let's define our class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from ast import literal_eval\n",
    "\n",
    "class Dog(object):\n",
    "    def __init__(self, name, age):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "        \n",
    "    def __eq__(self, other):\n",
    "        if isinstance(other, Dog):\n",
    "            return other.name == self.name and other.age == self.age\n",
    "        \n",
    "    def to_json(self):\n",
    "        return '{{\"name\": \"{0.name}\", \"age\": \"{0.age}\"}}'.format(self)\n",
    "    \n",
    "    @classmethod\n",
    "    def from_json(cls, json):\n",
    "        data = literal_eval(json)\n",
    "        return cls(\n",
    "            name=data['name'],\n",
    "            age=data['age']\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's import `monkeys` and start defining some constructs reflecting the properties we wanted to enforce (as stated above)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from monkeys import *\n",
    "\n",
    "@params(str, int)\n",
    "@rtype(Dog)\n",
    "def new_dog(name, age):\n",
    "    \"\"\"Create a new dog from a name and age.\"\"\"\n",
    "    return Dog(name, age)\n",
    "    \n",
    "@params(Dog)\n",
    "@rtype('JSON')\n",
    "def dog_to_json(dog):\n",
    "    \"\"\"JSON produced can be deserialized into a Python dict.\"\"\"\n",
    "    json = dog.to_json()\n",
    "    assert literal_eval(json), \"`to_json` did not produce valid JSON.\"\n",
    "    return json\n",
    "\n",
    "@params('JSON')\n",
    "@rtype(Dog)\n",
    "def dog_from_json(json):\n",
    "    \"\"\"Dogs can be loaded in from the JSON we produce.\"\"\"\n",
    "    return Dog.from_json(json)\n",
    "    \n",
    "@params(Dog)\n",
    "@rtype(Dog)\n",
    "def dog_type_invariants(dog):\n",
    "    \"\"\"\n",
    "    Constructs don't have to implement meaningful transformations - here, we \n",
    "    simply want to ensure at any time that Dog instances have string names and\n",
    "    integer ages.\n",
    "    \"\"\"\n",
    "    assert isinstance(dog.name, str), \"`dog.name` was not a string.\"\n",
    "    assert isinstance(dog.age, int), \"`dog.age` was not an integer.\"\n",
    "    return dog\n",
    "\n",
    "@params(Dog)\n",
    "@rtype(Dog)\n",
    "def dog_round_trip(old_dog):\n",
    "    \"\"\"Serializing and deserializing a dog gives you the same dog.\"\"\"\n",
    "    new_dog = Dog.from_json(\n",
    "        old_dog.to_json()\n",
    "    )\n",
    "    assert new_dog == old_dog, \"Round-trip serialization did not preserve dog.\"\n",
    "    return new_dog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll also need some example strings and integers, for `new_dog` to draw from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "names = [\n",
    "    constant(str, \"Fido\"), \n",
    "    constant(str, \"Rex\"), \n",
    "    constant(str, \"Pluto\"),\n",
    "]\n",
    "\n",
    "ints = [\n",
    "    constant(int, 5), \n",
    "    constant(int, 12), \n",
    "    constant(int, 1),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's visualize the type interactions we've declared, to ensure that everything looks OK:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"134pt\" height=\"332pt\"\n",
       " viewBox=\"0.00 0.00 134.00 332.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 328)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-328 130,-328 130,4 -4,4\"/>\n",
       "<!-- ε -->\n",
       "<g id=\"node1\" class=\"node\"><title>ε</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" stroke-dasharray=\"1,5\" cx=\"63\" cy=\"-306\" rx=\"18\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"63\" y=\"-301.8\" font-family=\"Times,serif\" font-size=\"14.00\">ε</text>\n",
       "</g>\n",
       "<!-- str -->\n",
       "<g id=\"node3\" class=\"node\"><title>str</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"27\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"27\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">str</text>\n",
       "</g>\n",
       "<!-- ε&#45;&gt;str -->\n",
       "<g id=\"edge8\" class=\"edge\"><title>ε&#45;&gt;str</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M55.0104,-289.465C50.615,-280.918 45.052,-270.101 40.044,-260.363\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"43.1549,-258.759 35.4688,-251.467 36.9299,-261.961 43.1549,-258.759\"/>\n",
       "</g>\n",
       "<!-- int -->\n",
       "<g id=\"node4\" class=\"node\"><title>int</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"99\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"99\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">int</text>\n",
       "</g>\n",
       "<!-- ε&#45;&gt;int -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>ε&#45;&gt;int</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M70.9896,-289.465C75.385,-280.918 80.948,-270.101 85.956,-260.363\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"89.0701,-261.961 90.5312,-251.467 82.8451,-258.759 89.0701,-261.961\"/>\n",
       "</g>\n",
       "<!-- str, int -->\n",
       "<g id=\"node2\" class=\"node\"><title>str, int</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"90,-180 36,-180 36,-144 90,-144 90,-180\"/>\n",
       "<text text-anchor=\"middle\" x=\"63\" y=\"-157.8\" font-family=\"Times,serif\" font-size=\"14.00\">str, int</text>\n",
       "</g>\n",
       "<!-- Dog -->\n",
       "<g id=\"node5\" class=\"node\"><title>Dog</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"63\" cy=\"-90\" rx=\"27.8306\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"63\" y=\"-85.8\" font-family=\"Times,serif\" font-size=\"14.00\">Dog</text>\n",
       "</g>\n",
       "<!-- str, int&#45;&gt;Dog -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>str, int&#45;&gt;Dog</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M63,-143.697C63,-135.983 63,-126.712 63,-118.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"66.5001,-118.104 63,-108.104 59.5001,-118.104 66.5001,-118.104\"/>\n",
       "</g>\n",
       "<!-- str&#45;&gt;str, int -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>str&#45;&gt;str, int</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M35.3496,-216.765C39.5428,-208.611 44.7279,-198.529 49.4709,-189.307\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"52.7324,-190.618 54.1934,-180.124 46.5074,-187.416 52.7324,-190.618\"/>\n",
       "</g>\n",
       "<!-- int&#45;&gt;str, int -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>int&#45;&gt;str, int</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M90.6504,-216.765C86.4572,-208.611 81.2721,-198.529 76.5291,-189.307\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"79.4926,-187.416 71.8066,-180.124 73.2676,-190.618 79.4926,-187.416\"/>\n",
       "</g>\n",
       "<!-- Dog&#45;&gt;Dog -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>Dog&#45;&gt;Dog</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M83.189,-102.432C96.1709,-105.675 108.665,-101.531 108.665,-90 108.665,-81.5317 101.927,-77.0474 93.2056,-76.5471\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"92.7826,-73.072 83.189,-77.5679 93.4923,-80.036 92.7826,-73.072\"/>\n",
       "</g>\n",
       "<!-- JSON -->\n",
       "<g id=\"node6\" class=\"node\"><title>JSON</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"63\" cy=\"-18\" rx=\"34.2065\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"63\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">JSON</text>\n",
       "</g>\n",
       "<!-- Dog&#45;&gt;JSON -->\n",
       "<g id=\"edge7\" class=\"edge\"><title>Dog&#45;&gt;JSON</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M57.1218,-72.055C56.2915,-64.2306 56.0543,-54.7484 56.4102,-45.9768\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"59.9164,-46.0131 57.1384,-35.789 52.9342,-45.514 59.9164,-46.0131\"/>\n",
       "</g>\n",
       "<!-- JSON&#45;&gt;Dog -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>JSON&#45;&gt;Dog</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M68.8616,-35.789C69.6998,-43.5912 69.9449,-53.067 69.5969,-61.8479\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"66.0893,-61.8338 68.8782,-72.055 73.072,-62.3255 66.0893,-61.8338\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f6af4b17290>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from monkeys.tools.display import type_graph\n",
    "type_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We might also want to add some additional interactions - e.g. allowing conversions from strings to integers (and vice-versa), or allowing use of JSON as a string - but for now, let's check that what we already have works. As we can see from the graph above, we can freely choose whether we want to generate Dog instances or JSON in order to test our system - let's choose Dog instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "41"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception_percent = 0\n",
    "for __ in range(100):\n",
    "    try:\n",
    "        build_tree(Dog).evaluate()\n",
    "    except Exception:\n",
    "        exception_percent += 1\n",
    "        \n",
    "exception_percent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oh my - about half of the time, something goes wrong! Let's have `monkeys` diagnose under what circumstances we're seeing exceptions thrown."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting exception sample...\n",
      "Discovered 2 distinct exceptions.\n",
      "Reproducing exceptions...\n",
      "Done.\n",
      "AssertionError('`dog.age` was not an integer.',):\n",
      "    3.68 | (dog_to_json) -> dog_from_json\n",
      "    1.90 | (dog_from_json) -> dog_type_invariants\n",
      "    1.66 | (dog_type_invariants) -> dog_type_invariants\n",
      "AssertionError('Round-trip serialization did not preserve dog.',):\n",
      "    2.44 | (new_dog) -> dog_round_trip\n",
      "    1.58 | (dog_round_trip) -> dog_type_invariants\n",
      "    1.36 | (_const_Rex, _const_12) -> new_dog\n"
     ]
    }
   ],
   "source": [
    "from monkeys.tools.diagnostics import diagnose\n",
    "\n",
    "diagnosis = diagnose(Dog)  # indicating the type we'd like to construct\n",
    "diagnosis.show_report()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The diagnosis tool discovers exceptions encountered when evaluating trees and tries its best to reproduce them. From the report above, we can see the two exceptions occurring when building `Dog`, as well as the most common node combinations in trees where these exceptions occur. The first exception looks to be caused when we're deserializing a `Dog` from JSON, whereas, in the second case, round-trip serialization of new `Dog` instances is failing.\n",
    "\n",
    "The diagnosis also catalogues the minimum reproducing examples it found for each exception. Let's visualize these with `node_graph`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AssertionError('`dog.age` was not an integer.',)\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"240pt\" height=\"332pt\"\n",
       " viewBox=\"0.00 0.00 240.37 332.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 328)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-328 236.374,-328 236.374,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"107.905\" cy=\"-306\" rx=\"89.773\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"107.905\" y=\"-301.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_type_invariants</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"107.905\" cy=\"-234\" rx=\"70.0906\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"107.905\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_from_json</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;1 -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M107.905,-287.697C107.905,-276.846 107.905,-262.917 107.905,-252.104\"/>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"107.905\" cy=\"-162\" rx=\"58.5752\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"107.905\" y=\"-157.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_to_json</text>\n",
       "</g>\n",
       "<!-- 1&#45;&#45;2 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>1&#45;&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M107.905,-215.697C107.905,-204.846 107.905,-190.917 107.905,-180.104\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"107.905\" cy=\"-90\" rx=\"46.374\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"107.905\" y=\"-85.8\" font-family=\"Times,serif\" font-size=\"14.00\">new_dog</text>\n",
       "</g>\n",
       "<!-- 2&#45;&#45;3 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>2&#45;&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M107.905,-143.697C107.905,-132.846 107.905,-118.917 107.905,-108.104\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\"><title>4</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"45.9048\" cy=\"-18\" rx=\"45.8099\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"45.9048\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_5</text>\n",
       "</g>\n",
       "<!-- 3&#45;&#45;4 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>3&#45;&#45;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M93.525,-72.7646C83.5087,-61.456 70.2059,-46.4367 60.2067,-35.1473\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\"><title>5</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"170.905\" cy=\"-18\" rx=\"61.4379\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"170.905\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_Pluto</text>\n",
       "</g>\n",
       "<!-- 3&#45;&#45;5 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>3&#45;&#45;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M122.517,-72.7646C132.549,-61.6175 145.826,-46.865 155.935,-35.6334\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Graph at 0x7f6ad2ffb790>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AssertionError('Round-trip serialization did not preserve dog.',)\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"232pt\" height=\"188pt\"\n",
       " viewBox=\"0.00 0.00 231.54 188.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 184)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-184 227.537,-184 227.537,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"105.905\" cy=\"-162\" rx=\"71.2365\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"105.905\" y=\"-157.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_round_trip</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"105.905\" cy=\"-90\" rx=\"46.374\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"105.905\" y=\"-85.8\" font-family=\"Times,serif\" font-size=\"14.00\">new_dog</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;1 -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M105.905,-143.697C105.905,-132.846 105.905,-118.917 105.905,-108.104\"/>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"45.9048\" cy=\"-18\" rx=\"45.8099\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"45.9048\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_1</text>\n",
       "</g>\n",
       "<!-- 1&#45;&#45;2 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>1&#45;&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M91.9888,-72.7646C82.4342,-61.6175 69.7891,-46.865 60.162,-35.6334\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"166.905\" cy=\"-18\" rx=\"56.7639\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"166.905\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_Rex</text>\n",
       "</g>\n",
       "<!-- 1&#45;&#45;3 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>1&#45;&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M120.053,-72.7646C129.767,-61.6175 142.622,-46.865 152.41,-35.6334\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Graph at 0x7f6af4bbd390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from monkeys.tools.display import node_graph\n",
    "from IPython.display import display\n",
    "\n",
    "for exception in diagnosis.exceptions:\n",
    "    print(exception)\n",
    "    repro = diagnosis.minimal_reproductions[exception]\n",
    "    display(node_graph(repro))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can clearly see that an integer is being passed in to `new_dog` as an age in both instances - but clearly not remaining one. Taking a look at `Dog.to_json`, the problem becomes obvious - we've put the dog's age in quotes! Let's quickly patch `Dog` with a new `to_json` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fixed_to_json(self):\n",
    "    return '{{\"name\": \"{0.name}\", \"age\": {0.age}}}'.format(self)\n",
    "\n",
    "Dog.to_json = fixed_to_json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's run our diagnosis again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting exception sample...\n"
     ]
    },
    {
     "ename": "UnsatisfiableConstraint",
     "evalue": "Could not find any exceptions after 250 trials.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m\u001b[0m",
      "\u001b[0;31mUnsatisfiableConstraint\u001b[0mTraceback (most recent call last)",
      "\u001b[0;32m<ipython-input-9-a11152e1f14c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdiagnosis\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdiagnose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mDog\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/root/miniconda2/lib/python2.7/site-packages/monkeys/tools/diagnostics.pyc\u001b[0m in \u001b[0;36mdiagnose\u001b[0;34m(target_type, test, sample_size)\u001b[0m\n\u001b[1;32m     84\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mencountered_exceptions\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     85\u001b[0m         \u001b[0merror_message\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Could not find any exceptions after {} trials.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 86\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mUnsatisfiableConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merror_message\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msample_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     88\u001b[0m     \u001b[0;32mprint\u001b[0m \u001b[0;34m\"Discovered {} distinct exceptions.\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mencountered_exceptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mUnsatisfiableConstraint\u001b[0m: Could not find any exceptions after 250 trials."
     ]
    }
   ],
   "source": [
    "diagnosis = diagnose(Dog)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great - `diagnose` raises an exception to let us know that it couldn't find any issues. Now that we have identified and resolved that problem, let's allow the use of JSON as a string, revisualize our interactions, and see whether this leads to any additional bugs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"142pt\" height=\"332pt\"\n",
       " viewBox=\"0.00 0.00 142.10 332.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 328)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-328 138.103,-328 138.103,4 -4,4\"/>\n",
       "<!-- ε -->\n",
       "<g id=\"node1\" class=\"node\"><title>ε</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" stroke-dasharray=\"1,5\" cx=\"71.1031\" cy=\"-306\" rx=\"18\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"71.1031\" y=\"-301.8\" font-family=\"Times,serif\" font-size=\"14.00\">ε</text>\n",
       "</g>\n",
       "<!-- str -->\n",
       "<g id=\"node3\" class=\"node\"><title>str</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"107.103\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"107.103\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">str</text>\n",
       "</g>\n",
       "<!-- ε&#45;&gt;str -->\n",
       "<g id=\"edge8\" class=\"edge\"><title>ε&#45;&gt;str</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M79.0927,-289.465C83.488,-280.918 89.0511,-270.101 94.0591,-260.363\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"97.1732,-261.961 98.6343,-251.467 90.9482,-258.759 97.1732,-261.961\"/>\n",
       "</g>\n",
       "<!-- int -->\n",
       "<g id=\"node4\" class=\"node\"><title>int</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"35.1031\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"35.1031\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">int</text>\n",
       "</g>\n",
       "<!-- ε&#45;&gt;int -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>ε&#45;&gt;int</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M63.1135,-289.465C58.7181,-280.918 53.1551,-270.101 48.1471,-260.363\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"51.258,-258.759 43.5719,-251.467 45.0329,-261.961 51.258,-258.759\"/>\n",
       "</g>\n",
       "<!-- str, int -->\n",
       "<g id=\"node2\" class=\"node\"><title>str, int</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"62.1031,-180 8.10309,-180 8.10309,-144 62.1031,-144 62.1031,-180\"/>\n",
       "<text text-anchor=\"middle\" x=\"35.1031\" y=\"-157.8\" font-family=\"Times,serif\" font-size=\"14.00\">str, int</text>\n",
       "</g>\n",
       "<!-- Dog -->\n",
       "<g id=\"node5\" class=\"node\"><title>Dog</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"34.1031\" cy=\"-90\" rx=\"27.8306\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"34.1031\" y=\"-85.8\" font-family=\"Times,serif\" font-size=\"14.00\">Dog</text>\n",
       "</g>\n",
       "<!-- str, int&#45;&gt;Dog -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>str, int&#45;&gt;Dog</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M34.8559,-143.697C34.7457,-135.983 34.6133,-126.712 34.4904,-118.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"37.99,-118.053 34.3474,-108.104 30.9907,-118.153 37.99,-118.053\"/>\n",
       "</g>\n",
       "<!-- str&#45;&gt;str, int -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>str&#45;&gt;str, int</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M92.5328,-218.834C83.1846,-209.746 70.8053,-197.71 59.9342,-187.141\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"62.3145,-184.574 52.7048,-180.113 57.4349,-189.593 62.3145,-184.574\"/>\n",
       "</g>\n",
       "<!-- int&#45;&gt;str, int -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>int&#45;&gt;str, int</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M35.1031,-215.697C35.1031,-207.983 35.1031,-198.712 35.1031,-190.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"38.6032,-190.104 35.1031,-180.104 31.6032,-190.104 38.6032,-190.104\"/>\n",
       "</g>\n",
       "<!-- Dog&#45;&gt;Dog -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>Dog&#45;&gt;Dog</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M54.292,-102.432C67.274,-105.675 79.7679,-101.531 79.7679,-90 79.7679,-81.5317 73.0298,-77.0474 64.3087,-76.5471\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"63.8857,-73.072 54.292,-77.5679 64.5954,-80.036 63.8857,-73.072\"/>\n",
       "</g>\n",
       "<!-- JSON -->\n",
       "<g id=\"node6\" class=\"node\"><title>JSON</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"34.1031\" cy=\"-18\" rx=\"34.2065\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"34.1031\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">JSON</text>\n",
       "</g>\n",
       "<!-- Dog&#45;&gt;JSON -->\n",
       "<g id=\"edge7\" class=\"edge\"><title>Dog&#45;&gt;JSON</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M28.2249,-72.055C27.3946,-64.2306 27.1574,-54.7484 27.5133,-45.9768\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"31.0195,-46.0131 28.2415,-35.789 24.0373,-45.514 31.0195,-46.0131\"/>\n",
       "</g>\n",
       "<!-- JSON&#45;&gt;str -->\n",
       "<g id=\"edge9\" class=\"edge\"><title>JSON&#45;&gt;str</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M54.3219,-32.7771C66.5511,-42.2852 81.2523,-56.0593 89.1031,-72 110.315,-115.071 111.242,-172.288 109.505,-205.635\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"105.989,-205.769 108.839,-215.973 112.974,-206.219 105.989,-205.769\"/>\n",
       "</g>\n",
       "<!-- JSON&#45;&gt;Dog -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>JSON&#45;&gt;Dog</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M39.9647,-35.789C40.8028,-43.5912 41.0479,-53.067 40.6999,-61.8479\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"37.1924,-61.8338 39.9813,-72.055 44.1751,-62.3255 37.1924,-61.8338\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f6ad2ffbc50>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json_to_str = free(str, 'JSON')  # We're already representing JSON as a string\n",
    "type_graph()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting exception sample...\n",
      "Discovered 22 distinct exceptions.\n",
      "Reproducing exceptions...\n",
      "Done.\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 5}\", \"age\": 12}')):\n",
      "    1.34 | (dog_to_json) -> dog_from_json\n",
      "    0.98 | (dog_type_invariants) -> dog_round_trip\n",
      "    0.96 | (new_dog) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 1}\", \"age\": 1}')):\n",
      "    1.80 | (dog_type_invariants) -> dog_type_invariants\n",
      "    1.47 | (dog_to_json) -> dog_from_json\n",
      "    1.05 | (new_dog) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 1}\", \"age\": 12}')):\n",
      "    2.07 | (json_to_str, _const_12) -> new_dog\n",
      "    2.07 | (dog_to_json) -> json_to_str\n",
      "    1.86 | (new_dog) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 12}\", \"age\": 12}')):\n",
      "    1.30 | (dog_to_json) -> dog_from_json\n",
      "    0.99 | (dog_type_invariants) -> dog_type_invariants\n",
      "    0.90 | (new_dog) -> dog_round_trip\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 5}\", \"age\": 5}')):\n",
      "    2.05 | (new_dog) -> dog_to_json\n",
      "    2.03 | (dog_to_json) -> dog_from_json\n",
      "    1.59 | (dog_to_json) -> json_to_str\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 5}\", \"age\": 5}')):\n",
      "    1.62 | (dog_round_trip) -> dog_round_trip\n",
      "    1.27 | (dog_to_json) -> json_to_str\n",
      "    1.27 | (json_to_str, _const_5) -> new_dog\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 1}\", \"age\": 5}')):\n",
      "    1.99 | (dog_type_invariants) -> dog_type_invariants\n",
      "    1.35 | (new_dog) -> dog_to_json\n",
      "    1.28 | (dog_to_json) -> dog_from_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 5}\", \"age\": 5}')):\n",
      "    2.88 | (dog_to_json) -> dog_from_json\n",
      "    2.66 | (dog_from_json) -> dog_to_json\n",
      "    1.27 | (new_dog) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 1}\", \"age\": 1}')):\n",
      "    2.62 | (dog_round_trip) -> dog_round_trip\n",
      "    1.76 | (dog_round_trip) -> dog_type_invariants\n",
      "    1.68 | (dog_type_invariants) -> dog_round_trip\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 1}\", \"age\": 1}')):\n",
      "    1.12 | (dog_round_trip) -> dog_to_json\n",
      "    0.98 | (dog_to_json) -> json_to_str\n",
      "    0.88 | (new_dog) -> dog_round_trip\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 5}\", \"age\": 12}')):\n",
      "    2.11 | (new_dog) -> dog_to_json\n",
      "    1.85 | (dog_to_json) -> json_to_str\n",
      "    1.85 | (json_to_str, _const_12) -> new_dog\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 1}\", \"age\": 5}')):\n",
      "    1.75 | (dog_to_json) -> dog_from_json\n",
      "    1.30 | (dog_from_json) -> dog_to_json\n",
      "    0.78 | (json_to_str, _const_5) -> new_dog\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 1}\", \"age\": 12}')):\n",
      "    1.16 | (dog_to_json) -> dog_from_json\n",
      "    0.88 | (dog_type_invariants) -> dog_type_invariants\n",
      "    0.62 | (dog_from_json) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 12}\", \"age\": 12}')):\n",
      "    0.90 | (dog_type_invariants) -> dog_to_json\n",
      "    0.72 | (dog_to_json) -> dog_from_json\n",
      "    0.47 | (dog_from_json) -> dog_type_invariants\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 1}\", \"age\": 5}')):\n",
      "    1.51 | (new_dog) -> dog_round_trip\n",
      "    1.35 | (dog_to_json) -> json_to_str\n",
      "    1.28 | (dog_round_trip) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 12}\", \"age\": 5}')):\n",
      "    1.24 | (dog_to_json) -> dog_from_json\n",
      "    0.94 | (dog_type_invariants) -> dog_round_trip\n",
      "    0.93 | (dog_round_trip) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 12}\", \"age\": 12}')):\n",
      "    2.13 | (dog_round_trip) -> dog_to_json\n",
      "    1.75 | (dog_to_json) -> dog_from_json\n",
      "    1.69 | (dog_from_json) -> dog_round_trip\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Rex\", \"age\": 5}\", \"age\": 1}')):\n",
      "    1.96 | (dog_to_json) -> dog_from_json\n",
      "    1.48 | (dog_from_json) -> dog_to_json\n",
      "    1.19 | (dog_type_invariants) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 1}\", \"age\": 12}')):\n",
      "    2.95 | (dog_to_json) -> dog_from_json\n",
      "    1.91 | (dog_type_invariants) -> dog_to_json\n",
      "    1.60 | (dog_from_json) -> dog_to_json\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Pluto\", \"age\": 5}\", \"age\": 1}')):\n",
      "    1.33 | (new_dog) -> dog_to_json\n",
      "    1.09 | (dog_to_json) -> dog_from_json\n",
      "    1.03 | (dog_to_json) -> json_to_str\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 12}\", \"age\": 1}')):\n",
      "    1.52 | (new_dog) -> dog_type_invariants\n",
      "    1.36 | (dog_type_invariants) -> dog_to_json\n",
      "    1.07 | (dog_to_json) -> json_to_str\n",
      "SyntaxError('invalid syntax', ('<unknown>', 1, 16, '{\"name\": \"{\"name\": \"Fido\", \"age\": 5}\", \"age\": 1}')):\n",
      "    1.45 | (dog_round_trip) -> dog_round_trip\n",
      "    0.83 | (dog_to_json) -> json_to_str\n",
      "    0.82 | (json_to_str, _const_1) -> new_dog\n"
     ]
    }
   ],
   "source": [
    "diagnosis = diagnose(Dog)\n",
    "diagnosis.show_report()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Something's clearly gone very wrong - let's see what a minimum reproduction looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"306pt\" height=\"476pt\"\n",
       " viewBox=\"0.00 0.00 306.20 476.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 472)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-472 302.202,-472 302.202,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"111.733\" cy=\"-450\" rx=\"89.773\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"111.733\" y=\"-445.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_type_invariants</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"111.733\" cy=\"-378\" rx=\"71.2365\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"111.733\" y=\"-373.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_round_trip</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;1 -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M111.733,-431.697C111.733,-420.846 111.733,-406.917 111.733,-396.104\"/>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"111.733\" cy=\"-306\" rx=\"46.374\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"111.733\" y=\"-301.8\" font-family=\"Times,serif\" font-size=\"14.00\">new_dog</text>\n",
       "</g>\n",
       "<!-- 1&#45;&#45;2 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>1&#45;&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M111.733,-359.697C111.733,-348.846 111.733,-334.917 111.733,-324.104\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"50.7333\" cy=\"-234\" rx=\"50.9677\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"50.7333\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_12</text>\n",
       "</g>\n",
       "<!-- 2&#45;&#45;3 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>2&#45;&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M97.5854,-288.765C87.8714,-277.618 75.0157,-262.865 65.2281,-251.633\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\"><title>4</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"173.733\" cy=\"-234\" rx=\"53.9117\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"173.733\" y=\"-229.8\" font-family=\"Times,serif\" font-size=\"14.00\">json_to_str</text>\n",
       "</g>\n",
       "<!-- 2&#45;&#45;4 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>2&#45;&#45;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M126.113,-288.765C135.986,-277.618 149.053,-262.865 159.001,-251.633\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\"><title>5</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"173.733\" cy=\"-162\" rx=\"58.5752\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"173.733\" y=\"-157.8\" font-family=\"Times,serif\" font-size=\"14.00\">dog_to_json</text>\n",
       "</g>\n",
       "<!-- 4&#45;&#45;5 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>4&#45;&#45;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M173.733,-215.697C173.733,-204.846 173.733,-190.917 173.733,-180.104\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\"><title>6</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"173.733\" cy=\"-90\" rx=\"46.374\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"173.733\" y=\"-85.8\" font-family=\"Times,serif\" font-size=\"14.00\">new_dog</text>\n",
       "</g>\n",
       "<!-- 5&#45;&#45;6 -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>5&#45;&#45;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M173.733,-143.697C173.733,-132.846 173.733,-118.917 173.733,-108.104\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\"><title>7</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"111.733\" cy=\"-18\" rx=\"45.8099\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"111.733\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_5</text>\n",
       "</g>\n",
       "<!-- 6&#45;&#45;7 -->\n",
       "<g id=\"edge7\" class=\"edge\"><title>6&#45;&#45;7</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M159.353,-72.7646C149.337,-61.456 136.034,-46.4367 126.035,-35.1473\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\"><title>8</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"236.733\" cy=\"-18\" rx=\"61.4379\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"236.733\" y=\"-13.8\" font-family=\"Times,serif\" font-size=\"14.00\">_const_Pluto</text>\n",
       "</g>\n",
       "<!-- 6&#45;&#45;8 -->\n",
       "<g id=\"edge8\" class=\"edge\"><title>6&#45;&#45;8</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.345,-72.7646C198.378,-61.6175 211.655,-46.865 221.763,-35.6334\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Graph at 0x7f6ad2fdd750>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_graph(list(diagnosis.minimal_reproductions.values())[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ah - the problem's occurring when we deserialize a `Dog` instance that, itself, had a serialized `Dog` provided as a name. It's at this point that we might do some Googling and discover that Python provides perfectly good JSON serialization and deserialization out of the box:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "def fixed_fixed_to_json(self):\n",
    "    return json.dumps({'name': self.name, 'age': self.age})\n",
    "\n",
    "@classmethod\n",
    "def fixed_from_json(cls, json_string):\n",
    "    data = json.loads(json_string)\n",
    "    return cls(\n",
    "        name=str(data['name']),\n",
    "        age=data['age']\n",
    "    )\n",
    "\n",
    "Dog.to_json = fixed_fixed_to_json\n",
    "Dog.from_json = fixed_from_json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting exception sample...\n"
     ]
    },
    {
     "ename": "UnsatisfiableConstraint",
     "evalue": "Could not find any exceptions after 250 trials.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m\u001b[0m",
      "\u001b[0;31mUnsatisfiableConstraint\u001b[0mTraceback (most recent call last)",
      "\u001b[0;32m<ipython-input-14-5f94b1bda1c4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdiagnose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mDog\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshow_report\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/root/miniconda2/lib/python2.7/site-packages/monkeys/tools/diagnostics.pyc\u001b[0m in \u001b[0;36mdiagnose\u001b[0;34m(target_type, test, sample_size)\u001b[0m\n\u001b[1;32m     84\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mencountered_exceptions\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     85\u001b[0m         \u001b[0merror_message\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Could not find any exceptions after {} trials.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 86\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mUnsatisfiableConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merror_message\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msample_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     88\u001b[0m     \u001b[0;32mprint\u001b[0m \u001b[0;34m\"Discovered {} distinct exceptions.\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mencountered_exceptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mUnsatisfiableConstraint\u001b[0m: Could not find any exceptions after 250 trials."
     ]
    }
   ],
   "source": [
    "diagnose(Dog).show_report()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, the exception lets us know that our diagnosis tool couldn't find any issues.\n",
    "\n",
    "While the use of `monkeys` for diagnosis may seem like overkill in our `Dog` example, in larger systems, bugs caused by specific combinations of components can be difficult to pin down - especially when these combinations are rare. The `monkeys` diagnostic tool allows us to hunt for these problematic combinations and gives us a means of confidently validating composability. It can also prove invaluable when trying to debug genetic programming tasks we've made in `monkeys` - since oftentimes `monkeys` combines our nodes in ways we never expected!\n",
    "\n",
    "The diagnosis tool becomes even more powerful when combined with the Python code generation functionality of `monkeys`, covered in [Monkeys in abstract syntax trees](Monkeys%20in%20abstract%20syntax%20trees.ipynb) - using these methods, we can create and diagnosis issues directly in real Python code, and even integrate with existing test suites."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}