examples/notebooks/Linting by example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linting by example\n",
    "\n",
    "In this notebook, we demonstrate the creation of custom linting rules by supplying examples and counter-examples. While there are many great linting engines for Python already which capture a wide variety of code smells, creating custom, codebase-specific linting rules is a bit more difficult. \n",
    "\n",
    "For example, supposing we've just deprecated an old method for a newer, better version -- but, for legacy reasons, need to keep supporting the old one for now. We might want to alert developers, when linting, that they're continuing to use the deprecated method. We'll first supply examples of the deprecated method and preferred method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from monkeys.asts import quoted\n",
    "import ast\n",
    "\n",
    "\n",
    "@quoted\n",
    "def bad():\n",
    "    my_obj.old_deprecated_method()\n",
    "        \n",
    "bad_ast = ast.Module(body=bad)\n",
    "\n",
    "\n",
    "@quoted\n",
    "def good():\n",
    "    my_obj.new_great_method()\n",
    "        \n",
    "good_ast = ast.Module(body=good)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create queries against code structure with the external library [`astpath`](https://github.com/hchasestevens/astpath), which uses XPath expressions to query Python abstract syntax trees. Normally, using this means having a fairly good understanding of how Python ASTs look, but we can instead just have `monkeys` figure things out for us, given our examples.\n",
    "\n",
    "We'll first off use the common XPath definitions available in `monkeys`, and register (for each of our examples) a suite of common syntax tree node names, attribute names, and attribute values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from monkeys.common.xpath import NodeName, AttributeName, AttributeValue\n",
    "\n",
    "\n",
    "registered = []\n",
    "register = registered.append\n",
    "\n",
    "\n",
    "class NodeRegistrar(ast.NodeVisitor):\n",
    "    def visit(self, node):\n",
    "        \"\"\"Register AST node names and attributes with monkeys.\"\"\"\n",
    "        register(NodeName(node.__class__.__name__))\n",
    "        \n",
    "        fields = {\n",
    "            field: getattr(node, field) \n",
    "            for field in \n",
    "            node._fields\n",
    "        }\n",
    "        for field, value in fields.items():\n",
    "            if isinstance(value, ast.AST):\n",
    "                register(NodeName(field))\n",
    "                continue\n",
    "                \n",
    "            if isinstance(value, list):\n",
    "                register(NodeName(field))\n",
    "                for item in value:\n",
    "                    if isinstance(item, str):\n",
    "                        register(AttributeValue(item))\n",
    "                continue\n",
    "                \n",
    "            register(AttributeName(field))\n",
    "            if isinstance(value, str):\n",
    "                register(AttributeValue(value))\n",
    "                \n",
    "        return super(NodeRegistrar, self).visit(node)\n",
    "\n",
    "\n",
    "NodeRegistrar().visit(good_ast.body[0])\n",
    "NodeRegistrar().visit(bad_ast.body[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now set up our scoring function, to try to find an `astpath` expression which matches on our deprecated method example, but not the new method example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating initial population of 250.\n",
      "Optimizing...\n",
      "Iteration 1:\tBest: 2.00\tAverage: 1.00\n",
      "Iteration 2:\tBest: 2.00\tAverage: 1.00\n",
      "Iteration 3:\tBest: 2.00\tAverage: 2.00\n",
      "Iteration 4:\tBest: 2.00\tAverage: 1.00\n",
      "Iteration 5:\tBest: 4.00\tAverage: 1.00\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\".//Attribute[@attr = 'old_deprecated_method']//ctx\""
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from monkeys.common.xpath import Expression\n",
    "from monkeys.typing import params\n",
    "from monkeys.search import optimize, assertions_as_score, pre_evaluate\n",
    "\n",
    "from astpath import convert_to_xml, find_in_ast\n",
    "\n",
    "\n",
    "good_xml = convert_to_xml(good_ast)\n",
    "bad_xml = convert_to_xml(bad_ast)\n",
    "\n",
    "\n",
    "@params(Expression)\n",
    "@pre_evaluate\n",
    "@assertions_as_score\n",
    "def score(expression):\n",
    "    assert expression.startswith('.//')  # as we want to capture this globally\n",
    "    matches_good = bool(find_in_ast(good_xml, expression))\n",
    "    matches_bad = bool(find_in_ast(bad_xml, expression))\n",
    "    assert not matches_good\n",
    "    assert matches_bad\n",
    "    assert matches_good ^ matches_bad\n",
    "    \n",
    "    \n",
    "best_expression = optimize(score).evaluate()\n",
    "\n",
    "best_expression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voila! We could now take this expression and run `astpath \".//Attribute[@attr = 'old_deprecated_method']//ctx\"` against our codebase to find all deprecated method usages.\n",
    "\n",
    "We can also isolate more structural elements of code, like silently ignoring unspecified exceptions, this time using a more sophisticated scoring function and more examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating initial population of 10000.\n",
      "Optimizing...\n",
      "Iteration 1:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 2:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 3:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 4:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 5:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 6:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 7:\tBest: 0.00\tAverage: 0.00\n",
      "Iteration 8:\tBest: 3.00\tAverage: 0.00\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'.//ExceptHandler[count(./Expr/ancestor-or-self::Call[self::orelse]/orelse[count(.//args) > 0]/Name/Load) = count(.//Name)]/body'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from monkeys.typing import deregister\n",
    "\n",
    "\n",
    "while registered:\n",
    "    deregister(registered.pop().constant)\n",
    "    \n",
    "    \n",
    "@quoted\n",
    "def bad():\n",
    "    try:\n",
    "        do_something()\n",
    "    except:\n",
    "        pass\n",
    "    \n",
    "bad_ast = ast.Module(body=bad)\n",
    "bad_xml = convert_to_xml(bad_ast)\n",
    "NodeRegistrar().visit(bad_ast.body[0])\n",
    "    \n",
    "    \n",
    "@quoted\n",
    "def good():\n",
    "    try:\n",
    "        do_something()\n",
    "    except Exception:\n",
    "        pass\n",
    "\n",
    "good_ast = ast.Module(body=good)\n",
    "good_xml = convert_to_xml(good_ast)\n",
    "NodeRegistrar().visit(good_ast.body[0])\n",
    "\n",
    "\n",
    "@quoted\n",
    "def neutral():\n",
    "    try:\n",
    "        do_something()\n",
    "    except Exception:\n",
    "        pass\n",
    "        raise Exception\n",
    "    \n",
    "neutral_ast = ast.Module(body=neutral)\n",
    "neutral_xml = convert_to_xml(neutral_ast)\n",
    "\n",
    "\n",
    "@params(Expression)\n",
    "@pre_evaluate\n",
    "@assertions_as_score\n",
    "def score(expression):\n",
    "    matches_good = bool(find_in_ast(good_xml, expression))\n",
    "    matches_bad = bool(find_in_ast(bad_xml, expression))\n",
    "    matches_neutral = bool(find_in_ast(neutral_xml, expression))\n",
    "    separates = (matches_good ^ matches_bad) and not matches_neutral\n",
    "    assert separates\n",
    "    if separates:\n",
    "        assert matches_bad\n",
    "        if matches_bad:\n",
    "            assert expression.startswith('.//')\n",
    "    \n",
    "    \n",
    "best_expression = optimize(score, population_size=10000).evaluate()\n",
    "\n",
    "best_expression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We end up with an expression that's a little less readable than the first, but still works -- essentially, by checking for `except` clauses with no referenced names."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}