examples/sr/Boolean_function_identification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boolean function identification problem\n",
    "In this example, we aim to identify a Boolean function $f$ from the given binary input-ouput data such that the function $f$ can produce the expected ouput given a certain input. This is a typical [symbolic regression](https://en.wikipedia.org/wiki/Symbolic_regression) problem:\n",
    "\n",
    "> Symbolic regression is a type of regression analysis that searches the space of mathematical expressions to find the model that best fits a given dataset, both in terms of accuracy and simplicity.\n",
    "\n",
    "which is most commonly solved with genetic programming and its variant, gene expression programming (GEP), presented here. \n",
    "\n",
    "Before continuing this example, you'd better first go through the two fundamental tutorials listed below to get familiar with GEP and *geppy*:\n",
    "+ [Introduction to gene expression programming](https://geppy.readthedocs.io/en/latest/intro_GEP.html)\n",
    "+ [Overview of geppy for Gene Expression Programming (GEP)](https://geppy.readthedocs.io/en/latest/overview.html)\n",
    "\n",
    "To check the detailed documentation of each function/class in *geppy*, please refer to [library reference](https://geppy.readthedocs.io/en/latest/#library-reference)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Synthetic dataset\n",
    "\n",
    "For this simple task, we first choose a ground truth Boolean function $f$ to generate a dataset $D$. Supposing there are four possible inputs $a, b, c, d$, the dataset $D$ contains 16 data samples (input-ouput pairs) in total."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(a, b, c, d):\n",
    "    \"\"\" The true model, which only involves three inputs on purpose.\"\"\"\n",
    "    return (a and d) or not c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "# generate the training set which contains all the 16 samples\n",
    "X = []\n",
    "Y = []\n",
    "for a, b, c, d in itertools.product([True, False], repeat=4):\n",
    "    X.append((a, b, c, d))\n",
    "    Y.append(f(a, b, c, d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating the primitives set\n",
    "The first step in GEP (or GP as well) is to specify the primitive set, which contains the elementary building blocks to formulate the model. For this Boolean function problem, no constant terminals are necessary. Instead, only the three Boolean logic operators and the four input terminals are needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import geppy as gep\n",
    "import operator\n",
    "\n",
    "pset = gep.PrimitiveSet('Main', input_names=['a', 'b', 'c', 'd'])\n",
    "pset.add_function(operator.and_, 2)\n",
    "pset.add_function(operator.or_, 2)\n",
    "pset.add_function(operator.not_, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create the individual and population\n",
    "Our objective is to **maximize** the number of samples that are correctly predicted by the evolved model. That is, the fitness of an individual is evaluated by *number of hits*.\n",
    "## Define the indiviudal class, a subclass of *gep.Chromosome*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from deap import creator, base, tools\n",
    "\n",
    "creator.create(\"FitnessMax\", base.Fitness, weights=(1,))  # to maximize the objective (fitness)\n",
    "creator.create(\"Individual\", gep.Chromosome, fitness=creator.FitnessMax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register the individual and population creation operations\n",
    "In DEAP, it is recommended to register the operations used in evolution into a *toolbox* to make full use of DEAP functionality. In *geppy*, a `geppy.Toolbox` class is also provided which inherits `deap.base.Toolbox`. For The configuration of individuals in this problem is:\n",
    "+ head length of each gene is 5\n",
    "+ each chromosome (individual) is composed of two genes.\n",
    "+ the linking function is chosen to be *OR*. In practice, we may need to try different linking functions.\n",
    "\n",
    "Generally, more complicated problems require a larger head length and longer chromosomes formed with more genes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = 5   # head length\n",
    "n_genes = 2   # number of genes in a chromosome\n",
    "toolbox = gep.Toolbox()\n",
    "toolbox.register('gene_gen', gep.Gene, pset=pset, head_length=h)\n",
    "toolbox.register('individual', creator.Individual, gene_gen=toolbox.gene_gen, n_genes=n_genes, linker=operator.or_)\n",
    "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)\n",
    "\n",
    "# compile utility: which translates an individual into an executable function (Lambda)\n",
    "toolbox.register('compile', gep.compile_, pset=pset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Define the fitness evaluation function\n",
    "In DEAP, the single objective optimization problem is just a special case of more general multiobjective ones. Since *geppy* is built on top of DEAP, it conforms to this convention. **Even if the fitness only contains one measure, keep in mind that DEAP stores it as an iterable.** \n",
    "\n",
    "Knowing that, you can understand why the evaluation function must return a tuple value (even if it is a 1-tuple). That's also why we set ``weights=(1,)`` when creating the ``FitnessMax`` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(individual):\n",
    "    \"\"\"Evalute the fitness of an individual\"\"\"\n",
    "    func = toolbox.compile(individual)  # a lambda function\n",
    "    n_correct = 0\n",
    "    for (a, b, c, d), y in zip(X, Y):\n",
    "        prediction = func(a, b, c, d)\n",
    "        if prediction == y:\n",
    "            n_correct += 1\n",
    "    return n_correct,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "toolbox.register('evaluate', evaluate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Register genetic operators\n",
    "Compared with GP and other genetic algorithms, GEP has its own set of genetic operators aside from common mutation and crossover. For details, please check the tutorial [Introduction to gene expression programming](https://geppy.readthedocs.io/en/latest/intro_GEP.html).\n",
    "\n",
    "In the following code, the selection operator is ``tools.selRoulette`` provided by DEAP, while all other operators are specially designed for GEP in *geppy*. To learn about the details regarding operator design and registration, please check the [conventions for operator design & registration](https://geppy.readthedocs.io/en/latest/index.html) tutorial. In short, we mainly focus on two kinds of operators with modification power: general mutations starting with *mut* and general crossover operators starting with *cx*. For such kinds of operators, a probability for their application is also needed. You can use as many operators as you like as long as they follow the conventions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "toolbox.register('select', tools.selRoulette)\n",
    "\n",
    "## general mutations whose aliases start with 'mut'\n",
    "# We can specify the probability for an operator with the .pbs property\n",
    "toolbox.register('mut_uniform', gep.mutate_uniform, pset=pset, ind_pb=2 / (2 * h + 1))\n",
    "toolbox.pbs['mut_uniform'] = 0.1\n",
    "# Alternatively, assign the probability along with registration using the pb keyword argument.\n",
    "toolbox.register('mut_invert', gep.invert, pb=0.1)\n",
    "toolbox.register('mut_is_ts', gep.is_transpose, pb=0.1)\n",
    "toolbox.register('mut_ris_ts', gep.ris_transpose, pb=0.1)\n",
    "toolbox.register('mut_gene_ts', gep.gene_transpose, pb=0.1)\n",
    "\n",
    "## general crossover whose aliases start with 'cx'\n",
    "toolbox.register('cx_1p', gep.crossover_one_point, pb=0.1)\n",
    "toolbox.pbs['cx_1p'] = 0.4   # just show that the probability can be overwritten\n",
    "toolbox.register('cx_2p', gep.crossover_two_point, pb=0.2)\n",
    "toolbox.register('cx_gene', gep.crossover_gene, pb=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Statistics to be inspected\n",
    "We often need to monitor of progress of an evolutionary program. DEAP offers two classes to handle the boring work of recording statistics. Details are presented in [Computing statistics](http://deap.readthedocs.io/en/master/tutorials/basic/part3.html). In the following, we are intereted in the average/standard deviation/min/max of all the individuals' fitness in each generation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy \n",
    "\n",
    "stats = tools.Statistics(key=lambda ind: ind.fitness.values[0])\n",
    "stats.register(\"avg\", numpy.mean)\n",
    "stats.register(\"std\", numpy.std)\n",
    "stats.register(\"min\", numpy.min)\n",
    "stats.register(\"max\", numpy.max)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Launch evolution\n",
    "We make use of *geppy*'s builtin algorithm ``gep_simple`` here to perform the standard GEP evolution. A special class from DEAP, `HallOfFame`, is adopted to store the best individuals ever found. Besides, it should be noted that in GEP [*elitism*](https://en.wikipedia.org/wiki/Genetic_algorithm#Elitism) is highly recommended because some genetic operators in GEP are destructive and may destroy the best individual we have evolved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this section is just for reproduction purpose since evolutionary algorithms all involve randomness\n",
    "import random\n",
    "random.seed(123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gen\tnevals\tavg \tstd   \tmin\tmax\n",
      "0  \t50    \t8.38\t2.6145\t2  \t14 \n",
      "1  \t45    \t9.14\t2.1167\t6  \t14 \n",
      "2  \t40    \t9.96\t2.23571\t6  \t14 \n",
      "3  \t36    \t10.22\t2.28289\t6  \t14 \n",
      "4  \t36    \t10.22\t2.02277\t6  \t14 \n",
      "5  \t36    \t10.3 \t1.86815\t6  \t14 \n",
      "6  \t39    \t9.98 \t2.12123\t4  \t14 \n",
      "7  \t34    \t10.42\t1.97069\t6  \t14 \n",
      "8  \t34    \t10.16\t2.0036 \t5  \t14 \n",
      "9  \t38    \t10.52\t2.55531\t2  \t14 \n",
      "10 \t34    \t10.46\t2.30833\t6  \t14 \n",
      "11 \t34    \t10.78\t2.46   \t6  \t14 \n",
      "12 \t39    \t11.64\t2.25175\t6  \t16 \n",
      "13 \t40    \t11.14\t2.89834\t2  \t16 \n",
      "14 \t37    \t11.84\t2.38629\t6  \t16 \n",
      "15 \t39    \t11.28\t2.49832\t6  \t16 \n",
      "16 \t36    \t11.62\t2.62975\t6  \t16 \n",
      "17 \t40    \t12.2 \t2.498  \t6  \t16 \n",
      "18 \t37    \t12.3 \t2.69258\t6  \t16 \n",
      "19 \t35    \t12.46\t2.55507\t6  \t16 \n",
      "20 \t32    \t12.26\t2.77712\t2  \t16 \n",
      "21 \t29    \t12.46\t2.94761\t4  \t16 \n",
      "22 \t34    \t12.48\t2.98154\t2  \t16 \n",
      "23 \t30    \t12.64\t2.95134\t2  \t16 \n",
      "24 \t37    \t12.26\t2.89005\t4  \t16 \n",
      "25 \t34    \t12.3 \t3.18904\t4  \t16 \n",
      "26 \t38    \t12.68\t2.98288\t2  \t16 \n",
      "27 \t40    \t12.68\t2.67163\t4  \t16 \n",
      "28 \t34    \t13.36\t2.05679\t10 \t16 \n",
      "29 \t36    \t13.08\t2.80599\t2  \t16 \n",
      "30 \t39    \t13.06\t2.75978\t6  \t16 \n",
      "31 \t39    \t12.94\t2.56445\t6  \t16 \n",
      "32 \t31    \t12.8 \t2.5923 \t6  \t16 \n",
      "33 \t29    \t13.3 \t2.33452\t7  \t16 \n",
      "34 \t41    \t13.18\t2.551  \t6  \t16 \n",
      "35 \t36    \t13.56\t2.17403\t6  \t16 \n",
      "36 \t37    \t13.36\t2.55155\t6  \t16 \n",
      "37 \t39    \t12.86\t2.73503\t4  \t16 \n",
      "38 \t43    \t13.12\t2.55844\t6  \t16 \n",
      "39 \t34    \t13.14\t2.72037\t4  \t16 \n",
      "40 \t39    \t12.94\t2.42   \t7  \t16 \n",
      "41 \t37    \t12.1 \t3.06757\t2  \t16 \n",
      "42 \t39    \t12.42\t2.82906\t8  \t16 \n",
      "43 \t36    \t12.44\t2.66203\t8  \t16 \n",
      "44 \t36    \t11.88\t2.84   \t4  \t16 \n",
      "45 \t41    \t11.68\t3.30115\t4  \t16 \n",
      "46 \t36    \t12.14\t3.2864 \t4  \t16 \n",
      "47 \t31    \t12.46\t2.97463\t4  \t16 \n",
      "48 \t32    \t12.78\t2.73708\t8  \t16 \n",
      "49 \t41    \t12.92\t2.81311\t7  \t16 \n",
      "50 \t36    \t12.54\t3.26931\t4  \t16 \n"
     ]
    }
   ],
   "source": [
    "# size of population and number of generations\n",
    "n_pop = 50\n",
    "n_gen = 50\n",
    "\n",
    "pop = toolbox.population(n=n_pop)\n",
    "hof = tools.HallOfFame(1)   # only record the best individual ever found in all generations\n",
    "\n",
    "# start evolution\n",
    "pop, log = gep.gep_simple(pop, toolbox,\n",
    "                          n_generations=n_gen, n_elites=2,\n",
    "                          stats=stats, hall_of_fame=hof, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, after few generations, the maximum fitness is 16. Remember that we only have 16 samples in total. Thus, it takes 20 generations to find a perfect model. Now let's print the best individual to investigate what we have found."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "or_(\n",
      "\tand_(a, d),\n",
      "\tnot_(c)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "best = hof[0]\n",
    "print(best)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above output shows: \n",
    "+ the linking function is `or_`\n",
    "+ the two genes encode two functions respectively: `not_(c)` and `and_(c, and_(and_(c, a), d))`\n",
    "\n",
    "Since the gene has a head domain of length 5, the length of the tail is $5*(2-1)+1=6$. That is, the whole length of a gene is 11. If you are interested in the content of a gene, you can check it with `repr`. It is clearly demonstrated that in GEP though each gene has a fixed length, the expression tree (or sub-tree in a multigenic chromosome) translated from a gene can have various sizes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- Content of the two genes: \n",
      "<class 'geppy.core.entity.Gene'> [and_, a, d, c, a, b, d, d, b, c, a]\n",
      "<class 'geppy.core.entity.Gene'> [not_, c, and_, and_, d, d, c, b, b, d, a]\n",
      "- K-expression of the two genes\n",
      "[and_, a, d]\n",
      "[not_, c]\n"
     ]
    }
   ],
   "source": [
    "# [optional] check the 11 primitives of each gene\n",
    "print('- Content of the two genes: ')\n",
    "for gene in best:\n",
    "    print(repr(gene))\n",
    "#[optional] we can also check the K-expression (only including primitives that are expressed) of a gene\n",
    "print('- K-expression of the two genes')\n",
    "for gene in best:\n",
    "    print(gene.kexpression)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *[Optional]* Post-processing: simplification and visualization\n",
    "## Symbolic simplification of the final solution\n",
    "After the GEP evolution finishes, we can get the best individual (i.e., the best solution) best. Sometimes, the expression of the individual may involve a lot of redundancies. For example, x∗x+3∗y−y+(10−2)/4 is just x∗x+2∗y+2. \n",
    "However, the GEP evolution cannot realize this fact and thus the result we get are generally not simplified. \n",
    "\n",
    "In this problem, the final solution *best* seems more complicated than the true function $f$. Thus, it would be quite helpful if we can get a simplified version of the model we found by removing all the redundancies for better comparison and verification.*geppy* has provided a convenient function `simplify()` to perform symbolic simplification of the individual (solution) by leveraging the `sympy` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Symplified best individual: \n",
      "~c | (a & d)\n"
     ]
    }
   ],
   "source": [
    "symplified_best = gep.simplify(best)\n",
    "print('Symplified best individual: ')\n",
    "print(symplified_best)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clearly, after simplification the best individual evolved by GEP is just the ideal function $f$. More importantly, GEP can perform implicit variable (feature) selection effectively: though we provide it with four inputs, GEP only picks the three useful inputs `a, c, d`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization\n",
    "If you are interested in the expression tree corresponding to the individual, i.e., the genotype/phenotype system, *geppy* supports tree visualization by the `graph` and the `export_expression_tree` functions:\n",
    "\n",
    "- `graph` only outputs the nodes and links information to describe the tree topology, with which you can render the tree with tools you like;\n",
    "- `export_expression_tree` implements tree visualization with data generated by `graph` internally using the `graphviz` package. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "rename_labels = {'and_': '&', 'or_': '|', 'not_': '~'}   # we want use symbol labels instead of words in the tree graph\n",
    "gep.export_expression_tree(best, rename_labels, 'data/bool_tree.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARMAAAD7CAYAAABJyXDjAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVhV9b4G8HczCUgxCojmjIgyZJhDEpAexboCXofjkGLqSeE6PGqJmJaznnCqvCYnS71ZYDklOPWo6EZFZVATwQFCGTI3DkwhsGHv7/3DgydTAXWv/Vsbvp/nOf9oZ6+XN3pZe2AtBRERGGPsxUwzEp2AMdY48JgwxnSCx4QxphMmogMwpisajQYqlQoqlQpFRUXQarUoLS2FRqOBhYUFzM3NYW5uDltbW7i4uMDW1lZ05EaFx4QZnKqqKqSmpuLixYtIT09HRkYGsrOzoVKpoNFoGvw4FhYWaN26Nbp06YJu3brB09MTPj4+cHNzkzB946Xgd3OY3BERUlNTceDAASiVSpw5cwYVFRWwt7eHl5cXunbtCldXV7i4uKBly5ZwcnKCra0tjI2NYWVlBVNTU5SXl0OtVqOiogJFRUX4/fffcfPmTRQUFODy5cvIyMjA5cuXUVlZiZYtW8LPzw8BAQEIDg6Gi4uL6AoMwTQeEyZbZ86cQUxMDH766Sfk5+fDzc0Nb731Fvz9/fHmm2+iVatWOj2eRqPBpUuXoFQqoVQqcezYMRQXF6NXr14YNmwYxowZw8PydDwmTF7++OMPbNu2Df/617/wyy+/oHv37hgxYgSGDBkCd3d3vWaprq6GUqnErl27sHv3bhQVFSE4OBhhYWH429/+ptcsBmAaiDEZKC0tpRUrVpCDgwM1b96cJk2aRCkpKaJjPVRZWUmxsbEUEBBACoWCevToQfHx8aTVakVHk4upPCZMKI1GQxs3biR7e3t6+eWXaf78+XT37l3RseqUlpZGISEhpFAoqFevXpScnCw6khxM5c+ZMGHS0tLQu3dvzJgxAxMnTsT169exbNky2NnZiY5Wp9deew0//fQT0tLSYGFhgd69eyMsLAzFxcWiownFY8L0joiwatUq9OnTBxYWFjh//jyioqJkPyJ/1b17dyQkJOD//u//sHfvXnTv3h2nT58WHUsYHhOmV8XFxXjnnXcwf/58rFixAsePH0e3bt1Ex3puCoUCY8eORXp6Ojw8PODn54e1a9eKjiUEf2iN6c1vv/2Gd955ByUlJUhKSkKPHj1ER9IZBwcHxMXF4fPPP8eHH36IGzdu4LPPPoORUdP5ec1jwvSioKAAvr6+sLGxwenTp9GyZUvRkXROoVBg5syZ6NSpE0aOHIni4mJs3bq1yQxK0/gqmVC3b99Gv3790K5dOyQmJjbKIfmzwYMHIyEhAXFxcZg2bZroOHrDY8IkVV1djSFDhsDc3Bx79+7Fyy+/LDqSXvTq1Qu7d+/GN998g88++0x0HL3gMWGSWrRoEW7cuIGDBw/C2tpadBy96tevH7Zs2YK5c+fi3LlzouNIjj9OzySTlJSEgIAAHDx4EP379xcdR5jJkycjMTERv/zyC5o1ayY6jlT4so1MGkSEmTNnYurUqU16SADgs88+Q3V1NTZs2CA6iqR4TJgktm/fjpycHCxcuFBoDoVCIfT4AGBpaYmVK1di+fLlKCoqEh1HMjwmTBKLFy/GjBkzYGNjIzqKLIwYMQKOjo5Yv3696CiS4TFhOnfq1ClkZWXhvffeEx1FNhQKBSZNmoQtW7ZAq9WKjiMJHhOmc9999x38/f3Rpk0b0VFkZdy4ccjNzUViYqLoKJLgMWE6p1QqERAQIDqG7Dg5OcHNzY3HhLGGuHfvHq5cuYI+ffqIjiJLffv2xalTp0THkASPCdOp69evg4jQoUMH0VFkqX379rh+/broGJLgMWE6de/ePQCAvb294CTy5OjoiDt37oiOIQkeE6ZT5eXlAAArKyvBSeTJ0tISZWVlomNIgseE6VTt50pqz1DYo+7evdtoz9p4TJhOtWjRAsCDyw6wx925c+dhR40NjwnTKVdXV5ibmyM9PV10FFm6ePEivL29RceQBI8J0ykzMzP07Nmz0b79+aKSkpIa7dvmPCZM5wIDA/Hzzz+Dr27xqNTUVNy5cweDBg0SHUUSPCZM50JDQ5GTk4OTJ0+KjiIrmzdvhr+/Pzp27Cg6iiR4TJjOtW7dGkFBQYiOjhYdRTZKSkoQExOD8PBw0VEkw1daY5LIyMhA9+7dkZycjFdffVVYDoVCIYunW/PmzcPhw4eRkpIii2usSGAajwmTzOTJk5GRkYHExEQYGxuLjiNMZmYmfHx8sG/fvsZ81Tm+bCOTzvLly5GTk4OlS5eKjiJMZWUlRo8ejaFDhzbmIQHAr5kwCbVo0QJbt27F8uXLERcXJzqO3hER/vGPf6CsrAwbN24UHUdyPCZMUoGBgZg7dy5Gjx7d5G7q/dFHH2HPnj348ccfm8T9gnhMmOSWLl2KsWPH4u233260Fwb6MyLCvHnzsHbtWuzatatR3VO5LjwmTHIKhQJffvklgoKCEBgYiJ07d4qOJJnq6mpMmDAB69atw/fff99oP6D2JDwmTC+MjY3x7bffYsaMGRg5ciTmzZuHmpoa0bF0Ki8vD2+99Rb27t2LgwcPYvjw4aIj6RWPCdMbhUKBTz/9FN988w3Wr1+PN998Ezk5OaJj6cSuXbvQvXt3lJSUICkpCW+99ZboSHrHY8L07r333kNqaioqKyvh4eGBJUuWoLKyUnSs55KTk4OgoCAMHz4cI0aMQHJyMtzd3UXHEoLHhAnRpUsXJCcnY/HixVi1ahU8PT3x3XffQaPRiI7WILdv38bcuXPh4eGBnJwcJCQkIDo6GhYWFqKjiUOMCZaXl0ehoaFkYmJCbm5utGXLFqqsrBQd64ny8/MpIiKCrKysyNHRkdauXUtqtVp0LDmYymPCZCMrK4vGjx9PZmZmZG9vT7Nnz6bMzEzRsaimpob2799PwcHBZGxsTM7OzrR69WoqLy8XHU1OpvLv5jDZKSwsxObNm7Fp0ybk5OTAw8MDQ4cOxX//93/Dy8sLRkbSPzu/f/8+jh07hl27diE+Ph53797F3/72N0yZMgXBwcEwNTWVPIOB4V/0Y/JFRDh79ix2796NXbt2IScnB3Z2dvD19YWfnx+8vb3h4eEBZ2fnFzqORqNBdnY20tPTkZaWhsTERKSkpECr1cLX1xfDhg1DSEgI3+60bjwmzHBcuXIFSqUSq1evxt27d1FUVATgwT16OnfuDGdnZ7Ru3RpOTk546aWXYG5uDktLSzRr1gxlZWWoqalBaWkpysrKUFBQgFu3biE/Px/Xrl1DVVUVTExM0Lx5c/j5+WHixInw9fWFg4OD4K/aYPCYMMNy6dIleHt7IyYmBgMHDkR6ejoyMzNx9epVqFQqFBQUQKVSoaSkBGq1GuXl5VCr1bCysoKpqSmsrKxgbW39cHReeeUVuLu7o1u3bnB3d8eYMWOQlZWF8+fP6+XpVCPCY8IMy/DhwyX9jz0jIwNeXl6IiYnByJEjdf74jRiPCTMc58+fh4+PD/bs2YOQkBDJjjN27FikpaXh0qVLTfqiTs+Ix4QZjqCgIKhUKpw9e1bSSx9mZWWha9eu+OabbxAaGirZcRoZHhNmGM6cOYM+ffrg4MGDevlN3EmTJuH48eO4cuUKvw3cMDwmzDAMGDAAFRUVert9xvXr19GlSxesX78ekydP1ssxDRxfA5bJn1KpxJEjR7Bs2TK9HbN9+/aYOHEili1bhqqqKr0d15DxmQmTPT8/P5iZmeHIkSN6Pe5vv/2GTp06ISoqCtOnT9frsQ0Qn5kwefv5559x4sQJIVe4b9WqFcLCwrBy5UpUVFTo/fiGhs9MmGwREXr37g0HBwfs379fSAaVSoWOHTti0aJF+PDDD4VkMBB8ZsLkKz4+HikpKViyZImwDE5OTpg2bRqioqJQVlYmLIch4DFhskREWLhwIUJCQuDj4yM0S0REBKqqqvDFF18IzSF3PCZMlnbu3ImLFy8KPSupZWdnh1mzZmHNmjUoLi4WHUe2eEyY7Gi1WixatAh///vf4enpKToOAGDWrFlQKBRYs2aN6CiyxWPCZCcmJgbXrl3DokWLREd5yNraGh9++CE+//xz3LlzR3QcWeJ3c5is1NTUwN3dHX379sXWrVtFx3lEeXk5OnTogPHjxyMqKkp0HLnhd3OYvGzduhW5ublYuHCh6CiPad68OSIjI7FhwwbcunVLdBzZ4TMTJhtqtRqdO3fGoEGDEB0dLTrOE1VUVMDV1RVDhw7ld3cexWcmTD42bdoElUqFBQsWiI7yVBYWFpg3bx6++uor5Ofni44jK3xmwmShoqICnTp1wvDhw/H555+LjlOn2jOowMBA/Otf/xIdRy74zITJw5dffoni4mJ89NFHoqPUy8zMDAsWLMCWLVtw/fp10XFkg89MmHB//PEHOnbsaFDvksj5XSdB+MyEibd+/XpUVFQgIiJCdJQGMzExwcKFC/H999/j6tWrouPIAp+ZMKFKSkrQoUMH/M///I+Qywy8CK1WC09PT3h5eSE2NlZ0HNH4zISJtW7dOhARPvjgA9FRnpmRkREWLVqEH3/8Eenp6aLjCMdnJkyYe/fuoX379oiIiMD8+fNFx3kuRITXXnsN7dq1w549e0THEYnPTJg4UVFRaNasGWbMmCE6ynNTKBRYsmQJ9u7di7S0NNFxhOIzEyZEY7uCWe/evWFnZ4cDBw6IjiIKn5kwMf75z3/i5ZdfxtSpU0VH0YklS5bg4MGDSEpKEh1FGD4zYXpXe9X3VatWYdq0aaLj6Iy/vz+MjY2RkJAgOooIfGbC9G/ZsmVwdHTE+++/LzqKTi1duhTHjh3DsWPHREcRgs9MmF419jvlDRgwAPfv38epU6dER9E3vj0o06+JEydCqVQ22nv41t4T+cCBA3j77bdFx9EnHhOmP1lZWejatSu++eYbhIaGio4jmaCgINy6dQvJyclQKBSi4+gLjwnTn7FjxyItLQ2XLl2CsbGx6DiSOX/+PHx8fLBnzx6EhISIjqMvPCZMPzIyMuDl5YWYmBiMHDlSdBzJDR8+HFlZWTh//jyMjJrE+xw8Jkw/mtp/XE1tPMFjwvShiZ72N5mndf/GY8KkFxQUBJVKhbNnzzalFySbzAvO/8ZjwqRV+1bpwYMHMWjQINFx9G7SpEk4fvx4o30r/E94TJi0BgwYgIqKCpw8eVJ0FCFu3LgBNze3RvshvT/hj9Mz6SiVShw5cgTLli0THUWYdu3a4R//+AeWLVuGqqoq0XEkxWcmTDJ+fn4wMTFpqr/49tBvv/0GV1dXfPrpp5g+fbroOFLhMxMmjZ9//hknTpxo0mcltVq1aoWwsDCsWLEC9+/fFx1HMnxmwiTBFwt6VO3FoBYuXIg5c+aIjiMFPjNhuhcfH4/k5GSDu9q8lJycnDBt2jRERUWhrKxMdBxJ8JgwnSIifPLJJwgJCYGPj4/oOLISEREBtVot+9ufPi9+msMe0mg0UKlUUKlUKCoqglarRWlpKTQaDSwsLGBubg5zc3PY2trCxcUFtra2jz3Gjh07MGrUKFy4cAGenp4Cvgp5W7RoEb744gvk5OTAxsbmkb/TRf8C8edMmqKqqiqkpqbi4sWLSE9PR0ZGBrKzs6FSqaDRaBr8OBYWFmjdujW6dOmCbt26wcPDA4sWLUKPHj34plRPUXvTsaCgIPTq1Uun/Xt6esLHxwdubm4SfgVPxWPSFBARUlNTceDAASiVSpw5cwYVFRWwt7eHl5cXunbtCldXV7i4uKBly5ZwcnKCra0tjI2NYWVlBVNTU5SXl0OtVqOiogJFRUX4/fffcfPmTRQUFODy5cvIyMjA5cuXUVlZiRYtWqBfv34ICAhAcHAwXFxcRFcg1F/7P3XqFNRqtWT9t2zZEn5+fvrun8ekMTtz5gxiYmLw008/IT8/H25ubnjrrbfg7++PN998E61atdLp8TQaDS5dugSlUgmlUoljx46huLgYvXr1wrBhwzBmzJgmNSxNrP9pINaolJWV0Zdffkne3t4EgLp3704rVqygzMxMvWdRq9V0+PBhCgsLI0dHRzI1NaVhw4bR4cOH9Z5FX5pw/1N5TBqJ0tJSWrFiBTk4OFDz5s1p0qRJlJKSIjrWQ5WVlRQbG0sBAQGkUCioR48eFB8fT1qtVnQ0neD+eUwMnkajoY0bN5K9vT29/PLLNH/+fLp7967oWHVKS0ujkJAQUigU1KtXL0pOThYd6blx/w/xmBiy1NRUev3118nU1JTmzJkj+2/ivzp37hwFBASQkZERTZkyhYqKikRHeibc/yN4TAyRVqulqKgoMjU1JT8/P7p06ZLoSM9Nq9XStm3byNnZmdq1a0dJSUmiI9WL+38iHhNDU1RURIMGDSJTU1NatWpVo3nN4fbt2zR48GAyMTGhNWvWiI7zVNz/U/GYGJKCggLy8vKitm3byurFPV3RarW0bt06MjY2punTp5NGoxEd6RHcf514TAxFfn4+tW3blry9venmzZui40gqPj6eLC0tady4cbIZFO6/XjwmhqCwsJBcXV3J39+fSkpKRMfRizNnzpC1tTWFh4eLjsL9NwyPidyp1Wp64403yNPTk4qLi0XH0aujR4+SmZkZrVu3TlgG7r/B/fOYyN1HH31ELi4uVFBQIDqKEN9//z2ZmZlRWlqakONz/w3un8dEzk6dOkWmpqZ05MgR0VGEev/998nNzY0qKyv1elzu/4EG9j+VL44kU0SEmTNnYurUqejfv7/oOEJ99tlnqK6uxoYNG/R2TO7/Pxrcv362jT2rmJgYsre3N7hPhUrlhx9+IDs7O7p3755ejsf9P6oB/fOZiVwtXrwYM2bMeOxqXPqSkJCA8PBwKBQKfPzxxygsLBSSo9aIESPg6OiI9evX6+V4ovuXmwb1r891Yw1z8uRJMjIyotzcXCHHj4uLIwB09OhRIiK6cOEChYWFCcnyZ6tWraJ27dpJ/tkT0f3LVT39T+WLI8lQeHg4rl69KuzmVcHBwYiPj8efvzUUCgVEf6uoVCq0bNkSCQkJCAgIkOw4ovuXq3r651tdyJFSqZT0P5b6xMfHA3hwvVIAyMvLQ1xcnLA8tZycnODm5obExERJjyO6f7mqr38eE5m5d+8erly5gj59+gjL8NVXXwF4cNPtkpIS3Lx5E0FBQcLy/Fnfvn1x6tQpyR5fDv3LWV3985jIzPXr10FE6NChg7AMgYGBAIDo6GiUlJSgd+/eAB481fnz/0Ro3749rl+/Ltnjy6F/Oaurfx4Tmbl37x4AwN7eXsjxf/nlF+zYsQOrV69GdHQ0kpKSHv5dbm4uACAuLk7Y6yeOjo64c+eOZI8vuv/CwkIEBwcjODj4ia/ZiBrxWnX1b6LnLKwe5eXlAAArKyu9H3vNmjVo1aoVPvjgA5SUlECpVGL06NF444030KZNG2RnZyM2NlboUx5LS0tJb68psn8A2Lt3L5YuXYp27dohLS0N4eHhGDFiBHx8fHDjxo2HT0FFqat/PjORmdrPNdT+hNSX7du348MPP8SoUaMAANbW1g/vFbxp0ybk5eXh2LFjD/9elLt370p61iCq/1qTJ0+Gt7c3rK2t0a9fP/zzn/9EYWEhxo0bh+joaISEhAjJVauu/vnMRGZatGgBALh9+zYcHR31dtzRo0c/9mfe3t6IjY3F6NGjcefOHWzcuPHh3+Xl5aFNmzZ6y1frzp07DzuSgqj+a/316aO1tTVGjRolfMRr1dU/n5nIjKurK8zNzZGenq7X465evRrAg6c6tW8Jl5SUwNHREWFhYYiOjn74HP7MmTPYsWOHXvPVunjxIry9vSV7fFH9G4q6+ucxkRkzMzP07NlT0rc/n+SDDz5AbGwslEolbGxsEBwcjIMHD6JTp07YuHEjTp8+jR07dkChUGD//v0YN26cXvPVSkpKkvRtW1H9G4o6+9fXR3FZwy1fvpxcXV0bzcWKdSUlJYUUCgVlZ2dLehzu/8nq6Z9/0U+OQkNDkZOTg5MnT4qOIiubN2+Gv78/OnbsKOlxuP8nq69/HhMZat26NYKCghAdHS06imyUlJQgJiYG4eHhkh+L+39cQ/rnX/STqYyMDHTv3h3Jycl49dVXRccRbt68eTh8+DBSUlL08sEt7v9RDeh/Go+JjE2ePBkZGRlITEyEsbGx6DjCZGZmwsfHB/v27dPrVc+4/wca2P80fgFWxgoLC8nZ2ZkWLlwoOoowFRUV5OXlRWPGjNH7sbn/Z+qfLygtd4cOHSITExPau3ev6Ch6p9Vq6d1336X27dsLu18N99/g/nlMDMH8+fPJ0tLSIG7qrUuRkZFkaWkp/Fac3H+D+ucxMQRarZYmT55M1tbWpFQqRceRnFarpcjISDIzM6ODBw+KjsP9NwyPiaGoqamhsWPHkrm5Oe3YsUN0HMmo1WoaP348NWvWTFZfJ/dfLx4TQ6LVaikiIoKMjIwoMjKSqqurRUfSqdzcXOrbty/Z2NhQQkKC6DiP4f7rxGNiiLZs2ULNmzen3r1706+//io6jk7s3LmT7OzsyMPDgzIzM0XHqRP3/0Q8Jobq8uXL9Oqrr5KFhQUtXryYKioqREd6Lr/++isNHjyYANCUKVPo/v37oiM1CPf/GB4TQ6ZWqykqKoqsrKyoU6dOtG3bNqqpqREdq0EKCwspIiKCLCwsqGvXrrJ8WlMf7v8RPCaNQV5eHoWGhpKJiQm5ubnRli1b9H6T74bKz8+niIgIsrKyIkdHR1q7di2p1WrRsV4I909EPCaNS1ZWFg0bNoyMjIzIzs6OZs+eLYvXH2pqamj//v0UHBxMxsbG5OzsTKtXr6by8nLR0XQqKyuLxo8fT2ZmZmRvb9/U+ucxaUyysrKoTZs25OnpSQsWLKAOHToQAPLw8KBPPvmEzp8/L/mtNWuVl5fTvn37aMKECeTg4EAKhYIGDBhAO3fuNPgzkfqoVCpauXJlU+ufbw/aWFy5cgX9+/dHq1at8PPPP8PW1hZEhLNnz2L37t3YtWsXcnJyYGdnB19fX/j5+cHb2xseHh5wdnZ+oWNrNBpkZ2cjPT0daWlpSExMREpKCrRaLXx9fTFs2DCEhIQIuWasSE2sf/6t4cYgMzMT/fr1Q6dOnbB//35YW1s/8Z+7cuUKlEolTpw4gcTEROTn5wN4cI+Yzp07w9nZGa1bt4aTkxNeeuklmJubw9LSEs2aNUNZWRlqampQWlqKsrIyFBQU4NatW8jPz8e1a9dQVVUFExMTeHl5wc/PD/7+/vD19YWDg4M+q5C1Rt4/j4mhu3DhAgYOHIiuXbti3759z3S/l6KiIqSnpyMzMxNXr16FSqVCQUEBVCoVSkpKoFar8ccff6C6uhpWVlYwNTWFlZUVrK2tH37Tv/LKK3B3d0e3bt3g7u4OMzMzCb/axqWR9c9jYsjS0tIwcOBAdO/eHXFxcbC0tNT5MWJiYvDuu+8Ku4NfU2dA/U/jyzYaqNOnT6N///7o2bMn4uPjJRkSxp4Fj4kBOnHiBAIDA+Hv74+9e/fCwsJCdCTGeEwMzdGjRzFo0CAEBgZix44d/BoFkw0eEwNy6NAhBAUFYciQIdi+fTsPCZMVHhMDERcXh5CQEIwcORLffvttk77AMZMnHhMDsHPnTowYMQITJkzA5s2beUiYLPGYyFxMTAxGjRqFKVOmYOPGjXq5Zwxjz4PHRMa2bt2K0NBQzJ49G1988QUPCZM1HhOZ+uqrrzBx4kRERkYiKipKdBzG6sVjIkMbNmxAWFgYli5dimXLlomOw1iD8JjIzJo1azB9+nR8+umnmD9/vug4jDUYj4mMrFy5EnPmzMHatWsxZ84c0XEYeyYmogOwBxYuXIilS5diw4YNCA8PFx2HsWfGYyIDkZGRWL16NTZt2oRJkyaJjsPYc+ExEYiIMHv2bKxfvx5bt27F2LFjRUdi7LnxmAhCRJg6dSq+/vprxMTE4O9//7voSIy9EB4TATQaDaZMmYJt27bhxx9/xJAhQ0RHYuyF8ZjomUajwXvvvYedO3diz549eOedd0RHYkwneEz0qLq6GmPHjsW+ffsQFxeHAQMGiI7EmM7wmOiJWq3GyJEjceTIERw4cAD+/v6iIzGmUzwmelBVVYXhw4fjxIkTOHToEPr27Ss6EmM6x2Misfv372Po0KFITk7G4cOH8frrr4uOxJgkeEwkVF5ejqCgIFy8eBFHjhzBa6+9JjoSY5LhMZFIaWkp/uu//gvXrl1DQkICvLy8REdiTFI8JhIoLi7GoEGDkJeXh+PHj8Pd3V10JMYkx2OiY3fv3sXAgQNx+/ZtHD9+HJ07dxYdiTG94DHRocLCQgwYMAClpaVQKpVo37696EiM6Q2PiY78/vvv6N+/P6qrq6FUKtGmTRvRkRjTK744kg7k5+cjICAARMRDwposHpMXdOPGDfj7+8PMzAzHjx+Hi4uL6EiMCcFj8gKys7Ph7+8PGxsbHDt2DE5OTqIjMSYMj8lzunr1Kvz9/eHk5ISjR4/CwcFBdCTGhOIxeQ6ZmZkICAhA+/btcfjwYdja2oqOxJhwPCbP6MKFCwgICICrqysOHToEa2tr0ZEYkwUek2eQlpaG/v37w8vLC4cOHYKVlZXoSIzJBo9JA50+fRr9+/dHz549ER8fD0tLS9GRGJMVHpMGOHHiBAIDA+Hv74+9e/fCwsJCdCTGZIfHpB5Hjx7FoEGDEBgYiB07dsDMzEx0JMZkicekDocOHUJQUBCGDBmC7du385AwVgcek6eIi4tDSEgIRo4ciW+//RbGxsaiIzEmazwmT7Bz506MGDECEyZMwObNm3lIGGsAHpO/iImJwahRozBlyhRs3LgRCoVCdCTGDAKPyZ9s3boVoaGhmD17Nr744gseEsaeAY/Jv23atAmTJk1CZGQkoqKiRMdhzODwmADYsGEDpkyZgiVLlmDZsmWi4zBmkGR1pTWNRgOVSgWVSoWioiJotVqUlpZCo9HAwsIC5ubmMDc3h62tLVxcXHTyC3Zr1qzBnDlzsHLlSsydO1cHX/xmZ4kAAAuuSURBVIXhelL/KSkpAIB9+/ZJ0j/7D0PvX0FEpO+DVlVVITU1FRcvXkR6ejoyMjKQnZ0NlUoFjUbT4MexsLBA69at0aVLF3Tr1g2enp7w8fGBm5tbg/7/K1euxPz587F27VrMnDnzeb8cgyOX/puqRtr/NL2MCREhNTUVBw4cgFKpxJkzZ1BRUQF7e3t4eXmha9eucHV1hYuLC1q2bAknJyfY2trC2NgYVlZWMDU1RXl5OdRqNSoqKlBUVITff/8dN2/eREFBAS5fvoyMjAxcvnwZlZWVaNmyJfz8/BAQEIDg4OAnXv1s4cKFWLp0KTZs2IDw8HCpKxBKjv03JU2k/2kgCZ0+fZqmT59Or7zyCgEgNzc3CgsLo9jYWCooKND58WpqaujChQv0+eef09ChQ8nW1pYUCgX17t2bVq1aRb/99hsREc2dO5eMjY3p66+/1nkGOZFr/01FE+t/qs7HpKysjL788kvy9vYmANS9e3dasWIFZWZm6vpQ9VKr1XT48GEKCwsjR0dHMjU1pYEDB1KzZs1o27Ztes+jD3Lvf9iwYXT48GG9Z9GXJty/7saktLSUVqxYQQ4ODtS8eXOaNGkSpaSk6OrhX1hlZSXFxsZSQEAAKRQK6tGjB8XHx5NWqxUdTSe4f7G4fx2MiUajoY0bN5K9vT29/PLLNH/+fLp7964uwkkmLS2NQkJCSKFQUK9evSg5OVl0pOfG/YvF/T/0YmOSmppKr7/+OpmamtKcOXNkX+JfnTt3jgICAsjIyIimTJlCRUVFoiM9E+5fLO7/Ec83JlqtlqKiosjU1JT8/Pzo0qVLLxJCKK1WS9u2bSNnZ2dq164dJSUliY5UL+5fLO7/iZ59TIqKimjQoEFkampKq1atajTPeW/fvk2DBw8mExMTWrNmjeg4T8X9i8X9P9WzjUlBQQF5eXlR27ZtZfXikq5otVpat24dGRsb0/Tp00mj0YiO9AjuXyzuv04NH5P8/Hxq27YteXt7082bN589qQGJj48nS0tLGjdunGy+obl/sbj/ejVsTAoLC8nV1ZX8/f2ppKTk+VMakDNnzpC1tTWFh4eLjsL9C8b9N0j9Y6JWq+mNN94gT09PKi4ufrGEBubo0aNkZmZG69atE5aB++f+RXnG/usfk48++ohcXFwk+fivIfj+++/JzMyM0tLShByf++f+RXqG/usek1OnTpGpqSkdOXJEd+kM0Pvvv09ubm5UWVmp1+Ny/w9w/2I1sP+nj4lWq6XXX3+dZs6cqft0Bqa8vJw6dOig17csuf//4P7FamD/Tx+TmJgYsre3N7hPJUrlhx9+IDs7O7p3755ejie3/lUqFcXGxlJQUJCQ4zf1/kVrQP9PHxM3NzdavHixNMkMkFarpS5duuitE7n1HxYWRgDoweU59K+p9y9aA/p/8picPHmSjIyMKDc3V7p0BmjVqlXUrl07yT/7INf+RY4JEfcvWj39T33iBaW/++47+Pv7o02bNvq4QpPBGDduHHJzc5GYmCjpcbj/J+P+xaqv/yeOiVKpREBAgJS56lVSUoJNmzZBoVBAoVDg448/RmFhodBMTk5OcHNzk/ybWS79b9++HQqFAsHBwbh27ZrQPEDT6h949N+BQqHApk2bhOapr//HxuTevXu4cuUK+vTpI3m4ukRGRmLy5MlQqVTIzc3FsmXLsHDhQqGZAKBv3744deqUZI8vl/7HjRsHpVKJ4uJixMXF4dy5c0Lz1Goq/QMP/h1kZGSAiEBEOHfuHD7++GOhmerq/7ExuX79OogIHTp0kDxYXRwcHBAWFgZHR8eHp5vR0dFCMwFA+/btcf36dckeXw79x8fHIz4+HrNmzYK1tTUA4O233xaW58+aQv8AsH37dsTHx2P69OkP/2z8+PH45ZdfBKaqu//H7ptz7949AIC9vb20qeqxdOlSAEBeXh527NghNMufOTo64s6dO5I9vhz6P3DgAACgc+fOD/+sdlREawr9Aw/ueQ08+Hpr9e7dG3FxcaIiAai7/8fOTMrLywEAVlZW0qZqgE2bNmHatGkICgoSHeUhS0tLlJWVSfb4cuhfDmeAT9MU+gcenB3KUV39PzYmNjY2AP6z0KJs374dkydPxv/+7/8+8hNStLt370r6U0su/ctVU+m/9geo6Kc1f1VX/4+NSYsWLQAAt2/fljZVPUaPHg0Asnt77s6dOw87koIc+v/qq68AyO8bGWga/QP/GZPo6GiUlJQAePCUX/QN4+rq/7ExcXV1hbm5OdLT0yUPVpfaMvPy8h55W1L028MXL16Et7e3ZI8vh/4DAwMBAB9//DHy8vIAAAkJCQ//XuQ3dFPoHwBCQkIQFBSE6Oho2NjYQKFQYOXKlZg1a5bQXHX1/9iYmJmZoWfPnpK+/dYQtS/Abtq0CTY2NliwYAHCwsJQWVkpNFdSUpKkbxvKof82bdogNzcXrVq1Qtu2bREeHg4PDw8EBQUhNjYWixcvFpatKfQPPHih8+uvv8aCBQsAAAsWLMCsWbOEP+Wvs/8nfS52+fLl5Orq2mgulqsrKSkppFAoKDs7W9LjcP9Pxv2LVU//T/44fWhoKHJycnDy5EkpR87gbN68Gf7+/ujYsaOkx+H+n4z7F6ve/p+2QkOGDKExY8ZIN3MGpri4mKytremHH37Qy/G4/0dx/2I1oP+nX4Lg0qVLZGpqSufPn5cmnYGJjIwkHx8fvZ36cv+P4v7FakD/dV+28f3336c33niDampqdJ/OgGRkZJC5ubneL9/H/T/A/YvVwP7rHpPCwkJydnamhQsX6jScIamoqCAvLy8hp7zcP/cv2jP0X//V6Q8dOkQmJia0d+9e3aQzIFqtlt59911q3769sPulcP/cvyjP2H/DbsI1f/58srS0NIibSutSZGQkWVpaCr8VJPfP/YvwjP03bEy0Wi1NnjyZrK2tSalUvlhCA6DVaikyMpLMzMzo4MGDouNw/zLIw/3Xq+H3Gq6pqaGxY8eSubk57dix4/lSGgC1Wk3jx4+nZs2ayerr5P7F4v7r1fAxIXqwWBEREWRkZESRkZFUXV39bEllLjc3l/r27Us2NjaUkJAgOs5juH+xuP86PduY1NqyZQs1b96cevfuTb/++uvzPITs7Ny5k+zs7MjDw4MyMzNFx6kT9y8W9/9EzzcmRESXL1+mV199lSwsLGjx4sVUUVHxvA8l1K+//kqDBw8mADRlyhS6f/++6EgNwv2Lxf0/5vnHhOjB86uoqCiysrKiTp060bZt2wzmAz6FhYUUERFBFhYW1LVrV1meVteH+xeL+3/Ei41Jrby8PAoNDSUTExNyc3OjLVu26P0m0w2Vn59PERERZGVlRY6OjrR27VpSq9WiY70Q7l8s7p+IdDUmtbKysmj8+PFkZmZG9vb2NHv2bFk8/62pqaH9+/dTcHAwGRsbk7OzM61evZrKy8tFR9Mp7l+sJt6/bseklkqlopUrV1KHDh0IAHl4eNAnn3xC58+fl/zWjrXKy8tp3759NGHCBHJwcCCFQkEDBgygnTt3GvxPwvpw/2I10f6nKoiIpLr+ARHh7Nmz2L17N3bt2oWcnBzY2dnB19cXfn5+8Pb2hoeHB5ydnV/oOBqNBtnZ2UhPT0daWhoSExORkpICrVYLX19fDBs2DCEhIbK7nqzUuH+xmlj/0yQdk7+6cuUKlEolTpw4gcTEROTn5wN4cI+Szp07w9nZGa1bt4aTkxNeeuklmJubw9LSEs2aNUNZWRlqampQWlqKsrIyFBQU4NatW8jPz8e1a9dQVVUFExMTeHl5wc/PD/7+/vD19YWDg4O+vjzZ4/7FauT963dM/qqoqAjp6enIzMzE1atXoVKpUFBQAJVKhZKSEqjVapSXl0OtVsPKygqmpqawsrKCtbX1w9JfeeUVuLu7o1u3bnB3d4eZmZmoL8fgcP9iNbL+xY4JY6zRmPbEa8Ayxtiz4jFhjOkEjwljTCdMAMwWHYIxZvBO/j8W3KpYU9rzSwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# show the above image here for convenience\n",
    "from IPython.display import Image\n",
    "Image(filename='data/bool_tree.png') "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}