Lea3_Tutorial_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lea3_Tutorial_2.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1",
        "colab_type": "text"
      },
      "source": [
        "![leas3_title_2.png](http://bitbucket.org/repo/BpoAoj/images/2132380690-leas3_title_2.png)\n",
        "\n",
        "---\n",
        "\n",
        "# Introduction\n",
        "\n",
        "The present Jupyter Notebook tutorial is the second part of the tutorial for the Lea library. It covers advanced topics in probability, including standard probability distributions, joint probability distributions, bayesian reasonning and Markov chains. It assumes that you have installed Lea 3 and that you are familiar with the techniques presented in the [first part of the tutorial](http://mybinder.org/v2/gh/piedenis/lea_tutorials/master?filepath=Lea3_Tutorial_1.ipynb).\n",
        "\n",
        "The main topics are mostly independent, so you could choose to jump to your preferred subject without much trouble.\n",
        "\n",
        "In some sections, we shall use some examples found in the excellent [\"Artificial Intelligence: A Modern Approach\" book](http://aima.cs.berkeley.edu/) of Stuart Russell and Peter Norvig (second edition). The reader is invited to refer to this book, named the \"AIMA book\" in the following, for more details on presented methods and results. _These examples are reproduced with the kind permissions of Stuart Russell and Peter Norvig._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FWd6GUCpCNhW"
      },
      "source": [
        "## Prerequisites\n",
        "\n",
        "\n",
        "Before running the examples below, you have to import the `lea` module:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EAAgkvxE_Tn9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import lea"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e0mmcJ-4lyoC",
        "colab_type": "text"
      },
      "source": [
        "This shall import the latest Lea 3.x, the baseline for the present tutorial.\n",
        "\n",
        "Later, when you are used to Lea functions, you could also use the alternative form:\n",
        "\n",
        "```from lea import *```\n",
        "\n",
        "and then, you can omit the `lea.` prefix in all the examples.\n",
        "\n",
        "**Note**: if the import fails, then it means that the lea module is not installed on your Jupyter server. In such case, execute the following `pip` command:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9hvHh-p6_c6T",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install lea"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ft_i09jtClWx"
      },
      "source": [
        "and retry the `import lea` statement above. Also, if you have module dependency issues with some examples, you may probably have to install the missing module(s) using using similar `pip install ...` statements for [matplotlib](http://matplotlib.org), [SymPy](http://www.sympy.org) or [pandas](http://pandas.pydata.org). These libraries can be installed at any time, independently each from each other, without reinstalling Lea."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uYqBi9SMCzsv",
        "colab_type": "text"
      },
      "source": [
        "# Standard probability distributions\n",
        "\n",
        "Beside general-purpose functions presented in the first part of the tutorial, Lea provides several functions for defining standard probability distributions, as well as gambling devices.\n",
        "\n",
        "Note that all these functions admit [an optional `prob_type` argument](#setting-probability-type-by-passing-declaration) for setting a specific probability representation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2",
        "colab_type": "text"
      },
      "source": [
        "## Bernoulli distribution\n",
        "\n",
        "The method `bernoulli(p)` defines a [Bernoulli distribution](http://en.wikipedia.org/wiki/Bernoulli_distribution): it gives the value $1$ with probability $p$ and $0$ with probability $1 - p$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.bernoulli(0.25)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5",
        "colab_type": "text"
      },
      "source": [
        "## Binomial distribution\n",
        "\n",
        "The method `binomial(n,p)` defines a [binomial distribution](http://en.wikipedia.org/wiki/Binomial_distribution): it gives the number of successes among a number $n$ of independent experiments, each having probability $p$ of success. For example, suppose a biased coin comes up heads with probability $0.3$ when tossed. What is the probability of achieving $0, 1,$ ... or $6$ heads after six tosses?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.binom(6,0.3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gYx8C281q5QI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.binom(6,0.3).plot()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7",
        "colab_type": "text"
      },
      "source": [
        "Note that the binomial distribution is a generalization of the Bernoulli distribution since `binomial(1,p)` is equivalent to `bernoulli(p)`. Also, this distribution can be obtained also by using the [`times(n)` method](https://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_1#markdown-header-repeating-operations): the binomial distribution above is exactly the same as"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.bernoulli(0.3).times(6)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "10",
        "colab_type": "text"
      },
      "source": [
        "## Poisson distribution\n",
        "\n",
        "The method `poisson(m)` defines a [Poisson distribution](http://en.wikipedia.org/wiki/Poisson_distribution) having mean $m$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "11",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.poisson(2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WvArnzxJ_vnI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.poisson(2).plot()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "12",
        "colab_type": "text"
      },
      "source": [
        "Because the Poisson distribution has, in theory, an infinite number of possible values, Lea provides an approximation where values with a probability below $1^{-20}$ are dropped. The threshold can be changed by issuing the needed precision as a second argument, e.g. `poisson(2,1e-30)`. Note that, since the highest values are dropped, the remaining probabilities are all slightly overestimated in order to keep a probability total equal to $1$."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "13",
        "colab_type": "text"
      },
      "source": [
        "## Convenience functions and objects\n",
        "\n",
        "Beside the standard probability distributions seen above, Lea provides a couple of convenience functions and prebuilt distributions related to gambling. These functions and objects are not loaded by default when importing lea; these are in a separated module called `lea.leaf` that you must import:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "14",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from lea.leaf import *"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "15",
        "colab_type": "text"
      },
      "source": [
        "Here is a table providing details on the functions provided in this module:\n",
        "\n",
        "| **function**     | **equivalent to**    | **description** |\n",
        "|:-----------------|:---------------------|:----------------|\n",
        "| `die(nb_faces=6)`                          | `lea.interval(1,nb_faces)`    | a fair die with faces marked from 1 to `nb_faces` (default: 6) |\n",
        "| `dice(nb_dice,nb_faces=6)`                 | `lea.interval(1,nb_faces).times(nb_dice)`    | total of nb_dice independent fair dice with faces marked from 1 to `nb_faces` (default: 6) |\n",
        "| `dice_seq(nb_dice,nb_faces=6,sorted=True)` | `lea.interval(1,nb_faces).draw(nb_dice,sorted=sorted,replacement=True)`    | tuple with `nb_dice` elements showing a throw of `nb_dice` independent fair dice with faces marked from 1 to `nb_faces` (default: 6); if `sorted` is `True` (default), then the combinations of dice which are the same apart from their order of throw are considered equal; the particular value used is chosen to be in order from smallest to largest value; if `sorted` is `False`, then all `nb_dice**nb_faces` combinations are produced, with equal probabilities |\n",
        "| `D6`                                      | `lea.interval(1,6,prob_type='r')`    | a fair die with 6 faces (fraction probability type) |\n",
        "| `flip`                                    | `lea.event('1/2',prob_type='r')`   | a True/False variable with 50-50 chances (fraction probability type) |\n",
        "| `card_suite`                              | `lea.vals(*'SHDC',prob_type='r')` | random one-character symbol representing a card suite among Spades, Hearts, Diamonds and Clubs (fraction probability type) |\n",
        "| `card_rank`                               | `lea.vals(*'A23456789TJQK',prob_type='r')` | random one-character symbol representing a card rank among Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack, Queen and King (fraction probability type) |\n",
        "| `card`                                    | `leaf.card_rank + leaf.card_suite`    | random two-characters symbol representing a card chosen in a deck of 52 cards, `leaf.card` is the concatenation of `card_suite` and `leaf.card_rank` (fraction probability type) |\n",
        "\n",
        "Note that the prebuilt objects use _fractions_ for probabilities. On the other side, the three function work with [the default probability type](#setting-probability-type-by-permanent-declaration); they also admit [an optional `prob_type` argument](#setting-probability-type-by-passing-declaration), which allows choosing a probability type different from the default one. For example, here is how you could see the probabilities of throwing two fair dice, with fractions or with floating-point numbers:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "16",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dice(2,prob_type='r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "17",
        "colab_type": "text"
      },
      "source": [
        "Here is how you could draw 13 cards from a deck of cards:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "18",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "card.random_draw(13)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "19",
        "colab_type": "text"
      },
      "source": [
        "Note also that `leaf.card`, `leaf.card_suite` and `leaf.card_rank` are interdependent:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "20",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "card.given(card_suite=='S',card_rank.is_none_of(*'JQKA'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V4a1YnYFFb7Y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "card_rank.given(card.is_any_of('8S','KS','8H','KC'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YCOn8RYbFbo1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "card_suite.given(card.is_any_of('8S','KS','8H','KC'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "21",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "# Advanced constructor arguments\n",
        "\n",
        "The `pmf` and `new` method accept several optional arguments, which may be useful in some occasions. The following table provides the description of these arguments\n",
        "\n",
        "| argument           | default        | description     |\n",
        "|:-------------------|:---------------|:----------------|\n",
        "| `sorting`          | `True`         | if `True`, then the values for displaying the distribution or getting the values will be sorted if possible (i.e. no exception on sort); otherwise, the order of values is unspecified unless ordered=True |\n",
        "| `ordered`          | `False`        | if `True`, then the values for displaying the distribution or getting the values will follow the order in which you provide them (requires that `sorting=False` and that the given pmf is an iterable or a collections.OrderedDict) |\n",
        "| `normalization`    | `True`         | if `True`, then each element of the given ps is divided by the sum of all ps before being stored (in such case, it's not mandatory to provide true probabilities; these could be simple counters, for example) |\n",
        "| `prob_type`        | `-1`           | allows converting the given probabilities: -1: no conversion, other codes [here](#setting-probability-type-by-passing-declaration) |\n",
        "| `check`            | `True`         | if `True`, then the constructor may make some checks on the given data and report errors, if any (exception); if you guarantee the given dat, you can make a slight optimization by setting the `check=False` |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QlN62HXV9SqC",
        "colab_type": "text"
      },
      "source": [
        "# Changing probability representation\n",
        "\n",
        "In the first tutorial, we have seen that probabilities are stored and displayed as floating-point numbers. This is a classical/conservative representation, which is used in the majority of statistical/probability packages. Other representations may however be more suitable. Lea proposes actually four choices for representing probabilities, namely floats, decimals, fractions and symbols:\n",
        "\n",
        "* **float**: based on Python builtin float type\n",
        "* **decimal**: based on Python built-in [Decimal](http://docs.python.org/3.6/library/decimal.html) class\n",
        "* **fraction**: based on Python built-in [Fraction](http://docs.python.org/3.6/library/fractions.html) class\n",
        "* **symbol**: based on SymPy's classes (requires installation of the [SymPy](http://www.sympy.org), a Python library for symbolic mathematics).\n",
        "\n",
        "The representation as floating-point numbers is clearly the most standard/conservative representation; it's easy to use and it integrates seamlessly with many functions and libraries. However, floating-point numbers present several drawbacks due to its rounding errors. In particular, it's not the most suited when dealing with probability distributions calculated by combinatorial (e.g. cards and dice games). The three other probability representations provide alternatives that can better fit your type of problems.\n",
        "\n",
        "There are several ways to specify the type used for representing probability in a given distribution: by automatic detection, by passing declaration, by permanent declaration."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "22",
        "colab_type": "text"
      },
      "source": [
        "## Setting probability type by automatic detection\n",
        "\n",
        "Lea is able to choose the probability representation based on the way you express it. This is probably the simplest way to work when you use Lea in an interactive session. The rules are quite simple:\n",
        "\n",
        "* If the probabilities are written as number literals or expressions, then the float type is used\n",
        "* If the probabilities are written as strings, then conversions are tempted until success in the following order: Decimal, Fraction, SymPy's Symbol.\n",
        "\n",
        "Here is a table giving examples of the different possible cases:\n",
        "\n",
        "| if you type ...    | you get ...                              |\n",
        "|:-------------------|:-----------------------------------------|\n",
        "| `0.333`            | 0.333 as a float instance                |\n",
        "| `'0.333'`          | 0.333 as a [Decimal](http://docs.python.org/3.6/library/decimal.html) instance              |\n",
        "| `1/3`              | 0.333333... as a float instance          |\n",
        "| `'1/3'`            | 1/3 as a [Fraction](http://docs.python.org/3.6/library/fractions.html) instance               |\n",
        "| `'p'`              | _p_ as a Sympy Symbol instance           |\n",
        "\n",
        "To illustrate this, here are some samples showing how a Boolean probability distribution and a probability distribution with the 3 possible outcomes \"A\", \"B\" and \"C\", can be represented."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "23",
        "colab_type": "text"
      },
      "source": [
        "### Probabilities as floating-point numbers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "24",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event(2/3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZZzBfBgpGSLi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 1/6, \"B\": 1/3, \"C\": 1/2})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "25",
        "colab_type": "text"
      },
      "source": [
        "Note that the same probability distributions can be obtained also by the following constructions:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "26",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event(0.6666666666666666)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tJP7BppwGYwr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 1, \"B\": 2, \"C\": 3})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N_CFShQbGYjg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 0.16666666666666666, \"B\": 0.33333333333333334, \"C\": 0.5})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "28",
        "colab_type": "text"
      },
      "source": [
        "### Probabilities as decimal numbers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "29",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event('0.666666666666666666666')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P_iab-5SBzVL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": '0.1666666666666666666', \"B\": '0.3333333333333333334', \"C\": '0.5'})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "30",
        "colab_type": "text"
      },
      "source": [
        "Note that we intentionally increased the number of decimals to highlight the difference with floating-point numbers. The decimal representation allows representing probability values precisely, keeping the given decimals unaltered and controlling rounding errors (see details\n",
        "in [Python's Decimal class doc](http://docs.python.org/3.6/library/decimal.html))."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31",
        "colab_type": "text"
      },
      "source": [
        "### Probabilities as fractions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "32",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event('2/3')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wBMlVxhFB-xQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": '1/6', \"B\": '1/3', \"C\": '1/2'})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "33",
        "colab_type": "text"
      },
      "source": [
        "The fraction representation is another way to avoid rounding errors appearing with floating-point numbers. It is respectful of the very origins of probability theory, combinatory and â€¦ gambling! Also, contrarily to float and decimals, fractions offer unlimited precision (see details\n",
        "in [Python's Fraction class doc](http://docs.python.org/3.6/library/fractions.html)).\n",
        "\n",
        "Note that the probabilities are displayed using the common denominator, so as to ease comparisons; when extracting a single probability value however, the fraction is reduced."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "34",
        "colab_type": "text"
      },
      "source": [
        "### Probabilities as variables (symbolic representation)\n",
        "_Reminder: the following requires the [SymPy](http://www.sympy.org) library._ If absent from your Jupyter server, you may install by executing `!pip install sympy`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "35",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event('p')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ekAf8BMrG1YQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 'pA', \"B\": 'pB', \"C\": 'pC'})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "36",
        "colab_type": "text"
      },
      "source": [
        "You can see that the probability variables have been normalized, to ensure that the sum equals 1. Other way to construct symbolic probability distributions, without requiring normalization, shall be presented is the [third tutorial - symbolic computation](http://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_3#markdown-header-symbolic-computation-with-probabilities)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "37",
        "colab_type": "text"
      },
      "source": [
        "<a id=\"setting-probability-type-by-passing-declaration\"></a>\n",
        "## Setting probability type by passing declaration\n",
        "\n",
        "As alternative to automatic detection that we have seen above, you can specify the required type _explicitly_ in the constructor. This is done by specifying the optional `prob_type` argument on Lea constructors; the simplest way to define this argument is to specify a one-character code, among `'f'`, `'d'`, `'r'`, `'s'` and `'x'`.\n",
        "\n",
        "| **`prob_type`**   | **probability type**  |\n",
        "|:------------------|:----------------------|\n",
        "| `'f'`             | float                 |\n",
        "| `'d'`             | Decimal               |\n",
        "| `'r'`             | Fraction              |\n",
        "| `'s'`             | Symbol                |\n",
        "| `'x'`             | -automatic- (default) |\n",
        "\n",
        "The `'x'` code is the default: it makes automatic probability type detection, as explained before. Here are some examples, using the other codes:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DgzkARonHmpv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event(1/3, prob_type='d')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "38",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event(1/3, prob_type='r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wocw6qK2Hsea",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event('0.6666666666666667', prob_type='r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ueW-vW_THsPS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 1/6, \"B\": 1/3, \"C\": 1/2}, prob_type='r')\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "USC_7cqSHsA2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 1, \"B\": 2, \"C\": 3}, prob_type='r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "39",
        "colab_type": "text"
      },
      "source": [
        "Note that what `prob_type` specifies is the target probability type, towards which the given probability values will be converted. If you provide float numbers in input, as in the examples above, you get results values of the target type that reveal the inner float representation (which can be somehow surprising!).\n",
        "\n",
        "The `prob_type` argument is also required if you want to change the probability type of a distribution that is specified by a list of values (since you don't provide any probability explicitly):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "40",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.vals(\"A\", \"B\", \"B\", \"C\", \"C\", \"C\", prob_type='r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "41",
        "colab_type": "text"
      },
      "source": [
        "For symbolic type, the conversion to a symbol is applied only if the probability is provided as a string. For example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "42",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event(1/3, prob_type='s')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eosqAjzbH7Es",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.event('1/3', prob_type='s')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "43",
        "colab_type": "text"
      },
      "source": [
        "In the first case, `1/3` is a float, not a string, so it is kept as a float. In the second case, `'1/3'` is a string, then it is converted to a symbol; note that it has been surrounded by parentheses, in order to avoid any misinterpretations in expression (this happens only if the string is not a valid variable name, according to Python's `isidentifier` method)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "44",
        "colab_type": "text"
      },
      "source": [
        "<a id=\"setting-probability-type-by-permanent-declaration\"></a>\n",
        "## Setting probability type by permanent declaration\n",
        "\n",
        "If you want to avoid specifying `prob_type` over and over, then you may invoke `lea.set_prob_type(...)` static method to set it once for all (well, more precisely: until the next call to `lea.set_prob_type` or until the end of your program/session). This sets the default probability type, the argument being interpreted as seen before for the `prob_type` argument.\n",
        "\n",
        "Suppose that you want to work only with probability fractions. You'll do this:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "45",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.set_prob_type('r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46",
        "colab_type": "text"
      },
      "source": [
        "Then, every Lea constructor shall use fractions:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "47",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.pmf({\"A\": 1, \"B\": 2, \"C\": 3})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E8Ztea80IAwu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.vals(\"A\", \"B\", \"B\", \"C\", \"C\", \"C\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "48",
        "colab_type": "text"
      },
      "source": [
        "To set back the default behavior (automatic type detection), set the `'x'` code:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "49",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.set_prob_type('x')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "51",
        "colab_type": "text"
      },
      "source": [
        "## Converting from one probability type to another\n",
        "\n",
        "It's sometimes useful to convert the probabilities of given probability distribution to a given type. Typically, you will like to convert fractions to floating-point numbers in order to verify results found in the literature. This is easy to do: just invoke the `new(prob_type=...)` method, where argument is the type code define in the [table](#setting-probability-type-by-passing-declaration) above, and you'll get a cloned probability distribution with the required type.\n",
        "\n",
        "Here is an example of conversion from fractions to floats:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "52",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "flip = lea.vals('Head', 'Tail' ,prob_type='r')\n",
        "flip"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NxN5CIuQCb9w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "flip.new(prob_type='f')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "53",
        "colab_type": "text"
      },
      "source": [
        "All conversions are feasible, **excepting**\n",
        "\n",
        "* conversions of symbolic probabilities,\n",
        "* conversions from fractions to decimals.\n",
        "\n",
        "For the last case, a workaround consists in chaining a conversion to float with a conversion to decimal, i.e. `.new(prob_type='f').new(prob_type='d')`.\n",
        "\n",
        "If you need to convert an individual probability, as returned for example by the `P` function, you have to call builtin Python functions or types:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "54",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.P(flip == 'Tail')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iQ2fU5aPIP8K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "float(lea.P(flip == 'Tail'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "55",
        "colab_type": "text"
      },
      "source": [
        "Since such conversion to floating-point number is quite common, Lea provides a convenience function, `Pf`, which performs this conversion directly on the result of `P`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "56",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.Pf(flip == 'Tail')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "57",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "# Joint probability distributions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "58",
        "colab_type": "text"
      },
      "source": [
        "## Calculating joint probability distributions (`joint` method)\n",
        "\n",
        "Given a set of Lea probability distributions, the `joint(d1,d2,...)` method allows you to build a new distribution that represents the joint distribution of the given probability distributions `d1`, `d2`, .... Basically, it builds up tuples giving all possible combinations of values of given distributions, with the associated probability. If the arguments are independent variables, then the returned joint distribution is a cartesian product, which gives all possible combinations. Here is a first example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "59",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "die1 = lea.interval(1, 6, prob_type='r')\n",
        "die2 = die1.new()\n",
        "lea.joint(die1, die2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "60",
        "colab_type": "text"
      },
      "source": [
        "The `lea.joint` method has unlimited number of arguments. If there are some dependencies between the given arguments, the tuples are built up consistently, such that they contain only possible combinations:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "61",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(die1 ,die1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fwgM50pArTSf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(die1, 7-die1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "62",
        "colab_type": "text"
      },
      "source": [
        "These results are again a direct application of the concept of [referential consistency](https://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_1#markdown-header-referential-consistency), which is ubiquitous in Lea.\n",
        "\n",
        "Calculating joint probability distribution may seem uninteresting for most of real use cases but, actually, it is used behind the scene for most of Lea's calculations. Also, it could be helpful, for a user point of view, in order to understand (or verify) how Lea calculates a probability distribution from atomic events.\n",
        "\n",
        "For instance, imagine you have executed the following query to give the probability distribution of the first die, knowing the fact that the total of the two dice does not exceed 4:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "63",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dice = die1 + die2\n",
        "die1.given(dice <= 4)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "64",
        "colab_type": "text"
      },
      "source": [
        "Now, if you want to understand why you get this distribution, then the following joint will provide some clues:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "65",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(die1, die2, dice).given(dice <= 4)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "66",
        "colab_type": "text"
      },
      "source": [
        "This joint distribution shows all combinations of dice consistent with the condition. The value of `die1` is the first element of the tuples, so you can now easily check the given results by adding atomic probabilities 1/6 together."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "67",
        "colab_type": "text"
      },
      "source": [
        "## Defining joint probability distributions\n",
        "\n",
        "So far, we have seen how a joint probability distribution can be _calculated_. In the present section, we shall see how a joint probability distribution (JPD) can be _constructed_ from scratch.\n",
        "\n",
        "Let us briefly explain the motivation for constructing a JPD. So far, we have seen different techniques to build random variables mutually dependent, by defining arithmetic/logical rules explaining how each variable is derived from others. This assumes that the rules are known and are expressible in mathematical formula. As we have seen, this works fine in many \"simple\" situations, like those found in gambling. However, our world is usually more complex than that: in many situations, the rules are unknown or too complex to model for a human being. In such case, one approach is collecting data and trying to find correlations in these data. This approach, studied in statistics field, is probably one of the first steps towards the now popular research field called \"Machine Learning\". Constructing a JPD from collected data is a basic way to express dependencies between random variables: the idea is to identify a set of variables and build a table thereof giving the probability of each possible combination of these variables. When the actual probabilities are missing (which is often the case), these are replaced by frequencies of occurrences; of course, providing as many data as possible should tend to improve the model (at least, this explains the hype with that \"Big Data\" thing!).\n",
        "\n",
        "To illustrate the definition of JPD on a simple example, we shall use the example of 3 boolean variables Toothache / Cavity / Catch, given in the [AIMA book](http://aima.cs.berkeley.edu/) referred in the introduction (section 13.4 in the second edition). We assume that there are some dependencies between these three variables, but in contrast to all previous examples, we assume that it is impossible to express these dependencies as formulas with logical operators. What we have however is a table indicating the 2 x 2 x 2 = 8 probabilities of each True/False combinations. We shall first make a temporary JPD with 3-tuples: for instance `(True,False,False)` shall mean \"toothache AND no Cavity AND no Catch\". For this purpose, we call the usual `pmf` function:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "68",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(T, F) = (True, False)\n",
        "_jpd = lea.pmf({ (T,T,T): '0.108', (T,F,T): '0.012', (F,T,T): '0.072', (F,F,T): '0.008',\n",
        "                 (T,T,F): '0.016', (T,F,F): '0.064', (F,T,F): '0.144', (F,F,F): '0.576' })\n",
        "_jpd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "69",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "Incidentally, we have put the given probabilities between quotes in order to use decimal numbers (this is not mandatory, this is just to avoid unpleasant rounding errors occurring with floating-point numbers, as explained [here](#setting-probability-type-by-automatic-detection)). The `_jpd` distribution captures the given JPD data but it misses the name of the underlying variables (Toothache / Cavity / Catch). The method `as_Joint(â€¦)` allows defining a new JPD, which is aware of the variable names:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "70",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "jpd = _jpd.as_joint('toothache', 'catch', 'cavity')\n",
        "jpd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "71",
        "colab_type": "text"
      },
      "source": [
        "Note that a header with field names is now displayed at the top. Technically, what `as_joint(...)` does is converting Python usual tuples into [named tuples](http://docs.python.org/3.6/library/collections.html#collections.namedtuple). If you want to see the true inner representation of these named tuple, you can do this:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "72",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print (jpd.as_string(tabular=False))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "73",
        "colab_type": "text"
      },
      "source": [
        "Note: in [\"Indexing and slicing\"](http://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_3#markdown-header-indexing-and-slicing), we present another way to proceed; the technique we use here is detailed in [\"Object attributes and methods\" section](http://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_3#markdown-header-object-attributes-and-methods).\n",
        "\n",
        "Now, `jpd` is a JPD ready to be used: it can be queried to get probabilities of given variables or logical combination of variables."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_p532d-aK85s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P = lea.P"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "74",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.cavity)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oqIt28hIKykU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.cavity | jpd.toothache)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RjbVmhqoKyVb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.cavity & ~jpd.toothache)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "75",
        "colab_type": "text"
      },
      "source": [
        "The resulting distributions are calculated by summing the atomic probabilities given in the joint distribution. This kind of calculation is referred to as a _marginalisation_.\n",
        "\n",
        "You can check that the variables defined in the joint distribution are actually dependant, as P(catch AND cavity) is different from P(catch) x P(cavity):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "76",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.catch & jpd.cavity)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zv8pLnwzLE-k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.catch) * P(jpd.cavity)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "77",
        "colab_type": "text"
      },
      "source": [
        "Since variables are dependent, the calculation of conditional probabilities provides useful results like\n",
        "\n",
        "* the probability of cavity for a patient having toothache,\n",
        "* the probability of cavity for a patient having toothache and a probe being caught."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "78",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.cavity.given(jpd.toothache))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mMeLTa33LLXT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.cavity.given(jpd.toothache & jpd.catch))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "79",
        "colab_type": "text"
      },
      "source": [
        "You can verify the correctness of these values in the [AIMA book](http://aima.cs.berkeley.edu/). The correctness of these calculations results again from the [referential consistency](http://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_1#markdown-header-referential-consistency) enforced in Lea.\n",
        "\n",
        "The conditional probability method can be used also to produce \"reduced\" or \"filtered\" joint distributions from the initial joint distribution, on the basis of known facts or assumptions:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "80",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "jpd.given(jpd.cavity, ~jpd.toothache)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6uKqYWtbLPvT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "jpd.given(jpd.cavity)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "81",
        "colab_type": "text"
      },
      "source": [
        "In order to remove the 'fixed' attributes, the following techniques can be used:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "82",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(jpd.catch.given(jpd.cavity, ~jpd.toothache))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NN7-V4OPLU1X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(jpd.toothache, jpd.catch).given(jpd.cavity).as_joint('toothache', 'catch')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "83",
        "colab_type": "text"
      },
      "source": [
        "Now, for those who are tired to type `jpd.` over and over, there's some good news! You can assign to your own variables and rely to the lazy evaluation of Lea. For instance, if you make the following assignments:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "84",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "toothache = jpd.toothache\n",
        "catch = jpd.catch\n",
        "cavity = jpd.cavity"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "85",
        "colab_type": "text"
      },
      "source": [
        "then, you should be able to redo ALL the examples given above WITHOUT any `jpd.` prefix. This assignment trick is actually something that can be used for any Lea expression. For the special case of joint distributions, the convenience function `make_vars` is handy to avoid typing boilerplate statements. Here is how you shall call it:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "86",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.make_vars(jpd, globals())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "87",
        "colab_type": "text"
      },
      "source": [
        "This call does the same as the 3 assignments seen above (take care however of possible variable overwriting!). The method accepts also optional prefix and suffix arguments, which shall be concatenated to the names of variable created. Example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "88",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.make_vars(jpd, globals(), '_', '2')\n",
        "P(_toothache2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "89",
        "colab_type": "text"
      },
      "source": [
        "The technique of joint distribution is very flexible, allowing defining any possible dependencies between variables. However, it requires defining many probability values and it scales not well as the number of variables grows. We shall see later in the tutorial a most efficient alternative, namely the [Bayesian networks](#bayesian-networks).\n",
        "\n",
        "Beyond boolean random variables, JPD can also be used for numerical values or other Python objects, for which you can evaluate expressions containing usual comparison operators. We shall now elaborate this topic, showing how Lea can be used to read larger set of data and get statistics on them. First, we shall see how to build a JPD from a given CSV file or from a given [pandas](http://pandas.pydata.org/) dataframe. Then, we shall show advanced techniques to extract data from the JPD, including filtering, marginalisation, grouping, etc."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "90",
        "colab_type": "text"
      },
      "source": [
        "## Building JPD from dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "91",
        "colab_type": "text"
      },
      "source": [
        "### Building JPD from a CSV file (`read_csv_file` method)\n",
        "\n",
        "For the example, we shall assume that we have a database of 50 patients within a CSV file. The fields are: given name, surname, gender, title, birthday, blood_type, weight in kg, height in cm, smoker (Y/N). Here are Python statements to create a sample **patients.csv** file (after execution, this file will be created on your Jupyter server so you can read it back later in your session):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "92",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patients_content = \"\"\"\n",
        "Elaine,McLaughlin,female,Ms.,12-01-1984,O+,44.9,141.0,Y\n",
        "Alan,Music,male,Mr.,30-12-1966,A-,61.3,181.0,Y\n",
        "Debra,Roberts,female,Mrs.,01-12-1927,O+,79.6,168.0,N\n",
        "Martine,Albright,female,Mrs.,05-01-1975,O+,46.0,156.0,N\n",
        "Terry,Jordan,male,Mr.,28-12-1963,A-,96.9,181.0,N\n",
        "Joshua,Spinney,male,Mr.,19-12-1952,O+,106.4,183.0,N\n",
        "Tawanna,Short,female,Ms.,02-02-2012,O+,93.4,175.0,N\n",
        "Frances,Moore,female,Ms.,07-01-1978,B+,91.6,164.0,N\n",
        "Jon,Overbey,male,Mr.,12-01-1985,A+,60.9,164.0,N\n",
        "Milton,Silva,male,Mr.,04-12-1932,O+,88.9,177.0,N\n",
        "Damon,Kiser,male,Mr.,09-12-1938,A+,97.2,181.0,N\n",
        "Ruby,Nunez,female,Mrs.,30-12-1966,O-,23.6,126.0,Y\n",
        "David,Seguin,male,Mr.,21-01-1996,A+,28.8,118.0,N\n",
        "Lillian,Wooley,female,Ms.,28-12-1964,A-,54.5,156.0,N\n",
        "Lena,Sharp,female,Ms.,02-01-1971,AB+,58.4,170.0,N\n",
        "Felecia,Decker,female,Mrs.,20-12-1953,O+,95.8,172.0,N\n",
        "Belva,Dry,female,Mrs.,07-12-1936,AB+,76.9,151.0,N\n",
        "Lawrence,McGovern,male,Mr.,06-01-1976,A+,16.8,91.0,Y\n",
        "Shelley,Bailey,female,Ms.,14-12-1945,A+,91.2,157.0,N\n",
        "Mildred,Heaps,female,Mrs.,01-01-1970,O+,67.8,162.0,N\n",
        "Melinda,Edie,female,Ms.,20-12-1953,A+,34.5,145.0,N\n",
        "Angela,Vaz,female,Mrs.,11-12-1941,AB+,30.3,126.0,N\n",
        "Gilberto,Schultz,male,Mr.,02-01-1971,B+,63.5,175.0,N\n",
        "Dianne,Zubia,female,Ms.,28-01-2006,O+,69.1,157.0,N\n",
        "Truman,Conner,male,Mr.,10-12-1940,O-,105.4,176.0,N\n",
        "Dorothy,Keegan,female,Mrs.,14-12-1945,A+,54.7,174.0,N\n",
        "Pamela,Guzman,female,Ms.,11-12-1941,B-,70.9,154.0,Y\n",
        "John,Davenport,male,Mr.,04-01-1974,B+,96.9,177.0,N\n",
        "Dorothy,Christen,female,Ms.,05-12-1933,B+,98.0,171.0,N\n",
        "Mary,Baird,female,Mrs.,26-01-2003,O+,61.3,170.0,N\n",
        "Luis,Boyett,male,Mr.,17-01-1991,O+,103.4,170.0,N\n",
        "Tricia,Lombardi,female,Mrs.,13-12-1944,A+,50.5,155.0,Y\n",
        "Joshua,West,male,Mr.,17-01-1991,A+,14.9,93.0,Y\n",
        "Jimmie,Martinez,male,Mr.,03-12-1930,B+,114.0,171.0,N\n",
        "Carla,Moore,female,Ms.,24-12-1958,B-,61.5,154.0,N\n",
        "Megan,Johanson,female,Mrs.,16-12-1948,B+,85.3,159.0,N\n",
        "Jenna,Tipton,female,Ms.,06-01-1977,O+,80.1,155.0,Y\n",
        "Goldie,Maestas,female,Mrs.,06-12-1934,O-,83.4,161.0,N\n",
        "Jeffrey,Evatt,male,Mr.,26-12-1961,O+,99.6,183.0,N\n",
        "Lloyd,Carroll,male,Mr.,16-12-1947,B-,98.6,189.0,N\n",
        "Dennis,Akins,male,Mr.,07-01-1978,B+,66.2,174.0,N\n",
        "Naomi,Paulus,female,Mrs.,04-02-2015,B+,73.4,167.0,N\n",
        "Dallas,Iverson,male,Mr.,03-01-1973,A+,66.5,177.0,N\n",
        "Thomas,Boren,male,Mr.,10-12-1939,A+,67.9,188.0,N\n",
        "Daniel,Helms,male,Mr.,22-12-1956,A+,87.0,174.0,N\n",
        "Muriel,Labrecque,female,Ms.,30-01-2009,B+,48.3,155.0,Y\n",
        "Bernice,Humphries,female,Mrs.,26-01-2003,O+,23.2,123.0,N\n",
        "Kimberly,Victory,female,Ms.,10-01-1982,A+,100.8,167.0,N\n",
        "Pam,Beach,female,Mrs.,12-12-1942,O+,79.8,155.0,N\n",
        "Rita,Hines,female,Ms.,06-01-1977,A+,51.5,166.0,N\n",
        "\"\"\"[1:]\n",
        "with open('patients.csv','w') as f:\n",
        "    f.write(patients_content)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "93",
        "colab_type": "text"
      },
      "source": [
        "Note that these data are fictional; these have been created thanks to the excellent online tool [Fake Name Generator](http://www.fakenamegenerator.com/).\n",
        "\n",
        "Here is how you can read this file and build a joint probability distribution from it:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "94",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient = lea.read_csv_file('patients.csv',\n",
        "                            col_names=('given_name','surname','gender','title','birthday','blood_type','weight{f}','height{f}','smoker{b}'),\n",
        "                            prob_type='d')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "95",
        "colab_type": "text"
      },
      "source": [
        "Note that the given column names shall be valid Python identifiers (starts with a letter, no blank, etc). The special suffix with curly braces used for the three last names are NOT part of the name. By default (no suffix), all read values are stored as strings in the built JPD; a curly braces suffix indicates the corresponding read values shall be converted as follows:\n",
        "\n",
        "|  **suffix**  | **conversion type**\n",
        "|--------------|----------------------\n",
        "| `{s}`        | string (note that this suffix can be omitted)\n",
        "| `{i}`        | integer\n",
        "| `{f}`        | float\n",
        "| `{b}`        | boolean\n",
        "\n",
        "For converting string to boolean, the following rules are applied (case insensitive):\n",
        "\n",
        "* `'1'`, `'t'`, `'true'`, `'y'`, `'yes'` --> Python's `True`,\n",
        "* `'0'`, `'f'`, `'false'`, `'n'`, `'no'` --> Python's `False`\n",
        "\n",
        "As we shall see, putting `{f}` on weight and height shall allow to make comparisons and calculations and these attributes; putting `{b}` on smoker shall ease the queries by removing references to 'Y'/'N' (see below).\n",
        "\n",
        "Now, `patient` is a JPD with named attributes, which you can display in tabular format"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "96",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "97",
        "colab_type": "text"
      },
      "source": [
        "You can access the attributes using standard Python syntax, as shown in the previous section. For example, to get distribution of genders and titles (marginalisation):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s3WDmr6wsQut",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient.gender"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "98",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient.title"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "99",
        "colab_type": "text"
      },
      "source": [
        "We'll see more elaborated queries in a next section."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "101",
        "colab_type": "text"
      },
      "source": [
        "## Handling of CSV formats\n",
        "\n",
        "The `read_csv_file` method as used above will work fine provided that your CSV file as a format similar to the format of our _patient.csv_ file example. Here are indications on how to handle other CSV format.\n",
        "\n",
        "Some CSV files contain a first line to be interpreted as a header containing field names.\n",
        "```\n",
        "given_name,surname,gender,title,birthday,blood_type,weight,height,smoker\n",
        "Elaine,McLaughlin,female,Ms.,12-01-1984,O+,44.9,141.0,Y\n",
        "Alan,Music,male,Mr.,30-12-1966,A-,61.3,181.0,Y\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lVSmmcCOhkaV",
        "colab_type": "text"
      },
      "source": [
        "Here are Python statementrs to create such file on your Jupyter session (**patients2.csv**):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rcjV_77RZZ8m",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patients_content2 = 'given_name,surname,gender,title,birthday,blood_type,weight,height,smoker\\n' \\\n",
        "                    + patients_content\n",
        "with open(\"patients2.csv\",\"w\") as f:\n",
        "    f.write(patients_content2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "103",
        "colab_type": "text"
      },
      "source": [
        "In such case, all you have to do is to omit the `col_names` argument in the call to `read_csv_file`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "104",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient2 = lea.read_csv_file('patients2.csv', prob_type='d')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "105",
        "colab_type": "text"
      },
      "source": [
        "Then, the names found in the first row shall be used to set the attribute names. Note that the curly braces suffixes could be added in the CSV file, with the same semantics as described above.\n",
        "\n",
        "Another thing to know is that `read_csv_file` relies on Python's standard [csv module](http://docs.python.org/2/library/csv.html) and it supports the same arguments (with same default) as `csv.reader` function, namely `dialect` and `**fmtparams`. So, assuming for instance that your CSV file is tab-delimited, you can use:\n",
        "```\n",
        "my_data = lea.read_csv_file('data.csv', colNames=..., delimiter='\\t')\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "107",
        "colab_type": "text"
      },
      "source": [
        "A last point concerns possible problems of CSV file's _encoding_. If such problem occurs, you'll have to open the CSV file yourself with the right encoding and pass the file object to `Lea.read_csv_file` (instead of a filename). For example, in Python 3,\n",
        "```\n",
        "with open('exoticData.csv','r',encoding='utf-8') as f\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "110",
        "colab_type": "text"
      },
      "source": [
        "### Building JPD from a pandas dataframe\n",
        "\n",
        "If you do data analysis using [pandas library](http://pandas.pydata.org/), you would be tempted to build a Lea JPD from a pandas \"dataframe\". To do this, you could manage to extract raw tuples from the data frames and use Lea `from_joint` method. Actually, Lea provides a dedicated method to do that, `Lea.from_pandas_df`. In the following example, we build first a pandas dataframe by reading the above-mentioned patients2.csv file (this time using pandas method, not Lea's)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "111",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "patients_df = pd.read_csv('patients2.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cupsxAE_5YVv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patients_df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "112",
        "colab_type": "text"
      },
      "source": [
        "You could of course create or transform such dataframe using other pandas functions. We then build a Lea JPD from that dataframe, using `read_pandas_df` function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "113",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient3 = lea.read_pandas_df(patients_df, prob_type='d')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FHV1--mldrUZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient3"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "114",
        "colab_type": "text"
      },
      "source": [
        "The resulting JPD shall be the same as before, except that all the fields of the JPD are strings, as it is the field format used in the present dataframe. For other dataframe formats, the fields in Lea JPD will mimic the dataframe's."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "115",
        "colab_type": "text"
      },
      "source": [
        "## Data mining on JPD\n",
        "\n",
        "As explained in [Defining joint probability distributions](#defining-joint-probability-distributions), all JPD attributes are accessed by using the standard dot syntax. Remember that, to work with simpler expressions, you could assign variables:\n",
        "```\n",
        "given_name = patient.given_name\n",
        "surname = patient.surname\n",
        "gender = patient.gender\n",
        "...\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "117",
        "colab_type": "text"
      },
      "source": [
        "or do these assignments automatically in one shot with:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "118",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.make_vars(patient,globals())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "119",
        "colab_type": "text"
      },
      "source": [
        "From now on, you can use the `patient`'s attributes, without the `patient.` prefix. That's what we'll assume in the following section. Should you prefer to be safer with your namespace, simply keep prefixing these attribute variables with `patient.`.\n",
        "\n",
        "We can now start to query the JPD data. In all examples below, the returned probabilities have to be interpreted primarily as frequencies in the given 50 patients dataset; if the sample is large enough, and enough representative of the whole population, then these figures could be interpreted as probabilities.\n",
        "\n",
        "Let's calculate the distribution of genders and titles."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "120",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gender"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g4mDXHnxeZBy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "title"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "121",
        "colab_type": "text"
      },
      "source": [
        "By using, the joint operation, we can see that genders and titles are dependent of each other, as we expect:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "122",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(gender,title)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "123",
        "colab_type": "text"
      },
      "source": [
        "Let's calculate the distribution of heights. To avoid too long display, we shall group with a precision of 10 cm, using Python's integer division."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "124",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "height_by_ten = (height//10) * 10\n",
        "height_by_ten"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "125",
        "colab_type": "text"
      },
      "source": [
        "Then, let's map these values into category labels:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "126",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "height_cat = height_by_ten.map(lambda h: 'H-%03d-%03d'%(h,h+9))\n",
        "height_cat"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "127",
        "colab_type": "text"
      },
      "source": [
        "We could check who is in the category 'H-120-129' and how tall they are (using English sentences, as an academic exercise!):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "128",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(given_name+' is '+(height/100.).map(str)+' m high').given(height_cat=='H-120-129')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "129",
        "colab_type": "text"
      },
      "source": [
        "Note that there seem to be only women in this category... This raises another question: how are heights distributed by gender?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "130",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "female = gender=='female'\n",
        "male = gender=='male'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FaxVu6PMennf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "height_cat.given(female)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bohYs1MEenYI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "height_cat.given(male)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "131",
        "colab_type": "text"
      },
      "source": [
        "You can note on this example how `height_cat`, `male` and `female` variables are interdependent. Another approach, more integrated, is to use the joint operation:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "132",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(gender,height_cat)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "133",
        "colab_type": "text"
      },
      "source": [
        "Imagine now that you have to calculate the [body mass index (BMI)](http://en.wikipedia.org/wiki/Body_mass_index) of the patients. Lea's arithmetic shall do the job quite easily (note that we round the BMI to limit the number of distinct values):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "134",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bmi = (weight/(height/100.)**2).map(round)\n",
        "bmi.plot()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xpwJrvtze65z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bmi.mean_f"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "135",
        "colab_type": "text"
      },
      "source": [
        "We want to split patients into \"underweight\", \"normal\" and \"obese\" categories. For this purpose, we shall define 3 boolean variables, which are mutually exclusive."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "136",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "underweight = bmi < 18.5\n",
        "obese = bmi >= 30\n",
        "normal = ~underweight & ~obese"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1-Ot9ZZ4fTr1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P = lea.P\n",
        "P(underweight)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ESprCfKVfVNG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(obese)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vth1KLDvfU9r",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(normal)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "137",
        "colab_type": "text"
      },
      "source": [
        "Note that we could have also defined a variable `cat_bmi`, mapping the BMI values into strings, using `if` / `elif` construct or, simpler, the Python's [`bisect`](http://docs.python.org/2/library/bisect.html) module).\n",
        "\n",
        "As a final example, let's select the female patients who are underweight and aged 18 years or more; for these patients, we want to display the full name, age, height, weight and BMI. Let's first build the full name and age; to ease things, we shall calculate the approximate age, for the year 2018, without considering the birth day and month."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "138",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "full_name = title + ' ' + given_name + ' ' + surname\n",
        "age = 2018 - birthday[-4:].map(int)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "139",
        "colab_type": "text"
      },
      "source": [
        "Now, using these two new variables, we can answer the initial request:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "140",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(full_name,age,height,weight,bmi).given(female,underweight,age>=18)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "141",
        "colab_type": "text"
      },
      "source": [
        "To conclude this section, let us mention that it's possible to change to order of displayed rows according to some attribute(s). By default, the rows are sorted as Python tuples: the first column is the main sort key, the second one, etc. If you want to change the order, you may then use the `lea.joint` function to reorder attributes as needed. An alternative to avoid this consists in using the `sort_by` method, which takes any number of attributes as arguments. For example here is how you can order underweight patients by their blood type, then their gender, then their BMI:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "142",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "patient.given(underweight).sort_by(blood_type,gender,bmi)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "143",
        "colab_type": "text"
      },
      "source": [
        "You may note that the method can use any attribute, whether displayed or not, whether direct or derived.\n",
        "\n",
        "WARNING: The `sort_by` method returns a distribution that has lost any dependency relationship. It has to be the last call before displaying the results. As a counter-example, the following construct `patient.sort_by(blood_type,gender,bmi).given(underweight)` will display the full list of patients, ignoring the filtering condition on `underweight`.\n",
        "\n",
        "\n",
        "# Bayesian reasoning\n",
        "\n",
        "So far, we have seen that the `given` method allows _calculating_ conditional probabilities. However, there is a whole class of probability problems where conditional probabilities P(A|B) are _given_. The general formula\n",
        "\n",
        "$$ P(A \\;|\\; B) = \\dfrac{P(A \\wedge B)}{P(B)} $$\n",
        "\n",
        "allows you, when applied with care and patience, answering most problems. We have seen how to define dependant random variables through joint distributions. Basically, this boils down to define $P(A \\wedge B \\;\\wedge\\; ...)$ for each conjunction of 'atomic' events $A, B,$ ... . This method allows modeling very accurately the dependences and to answer any query, including conditional probabilities. However, it requires the knowledge of the probability of each atomic combination, which could be difficult to know. Also, it has the drawback that combinatorial explosion entails the need to enter huge buckets of probability values.\n",
        "\n",
        "As alternative to joint distributions, Lea provides a more effective approach using _conditional probability tables_ (CPT). The idea is to reduce the degrees of freedom of joint distributions by modeling only causal relationships. In the following, we shall see that this approach is general and powerful when dealing with problems where conditional probabilities are given. We will see that the CPT technique is the cornerstone for Bayes reasoning."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "144",
        "colab_type": "text"
      },
      "source": [
        "## Introducing CPT with a beetle\n",
        "\n",
        "We shall start with the [\"beetle\" example (Wikipedia)](http://en.wikipedia.org/wiki/Bayes%27_theorem#Example):\n",
        "\n",
        "\"_An entomologist spots what might be a rare subspecies of beetle, due to the pattern on its back. In the rare subspecies, 98% have the pattern, or P(Pattern|Rare) = 98%. In the common subspecies, 5% have the pattern. The rare subspecies accounts for only 0.1% of the population. How likely is the beetle having the pattern to be rare, or what is P(Rare|Pattern)?_\"\n",
        "\n",
        "The random variables and their dependency can be represented by the following graphical model.\n",
        "\n",
        "![bn1.png](https://bitbucket.org/repo/BpoAoj/images/953499668-bn1.png)\n",
        "\n",
        "To solve this, we need to define two boolean probability distributions, namely `rare` and `pattern`. The first one is unconditional and easy to define:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "145",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "rare = lea.event(0.001)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "146",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "For `pattern`, we cannot (without manual calculations) set it so easily. We need to set two conditional probabilities\n",
        "* $P(pattern \\;|\\; rare) = 98\\%$\n",
        "* $P(pattern \\;|\\; \\overline{rare}) = 5\\%$\n",
        "\n",
        "Lea provides a special construct to set these probabilities (requires Lea 2.3+):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "147",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "pattern_if_rare     = lea.event(0.98)\n",
        "pattern_if_not_rare = lea.event(0.05)\n",
        "pattern = rare.switch({ True  : pattern_if_rare,\n",
        "                        False : pattern_if_not_rare })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "148",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "This `switch` method defines a special Lea instance, which represents a CPT. The idea is that `pattern` probability shall depend on the value of `rare`, according to the given dictionary: the keys of this dictionary shall be the possible values of `rare`, namely `True` and `False`. Let us check that our definition is in line with what we need:\n",
        "\n",
        "* What is the probability of having pattern for a rare beetle?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "149",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P = lea.P\n",
        "P(pattern.given(rare))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "150",
        "colab_type": "text"
      },
      "source": [
        "* What is the probability of having pattern for a common beetle?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "151",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(pattern.given(~rare))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "152",
        "colab_type": "text"
      },
      "source": [
        "OK, `pattern` gives back the data we put in it; this does not bring any new information, it is just a sanity test. Now, let us come back to the initial question:\n",
        "\n",
        "* What is the probability to be rare for a beetle having the pattern?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "153",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(rare.given(pattern))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "154",
        "colab_type": "text"
      },
      "source": [
        "This result is really what Bayes inference is about. It can be checked by manual calculations using the Bayes theorem.\n",
        "\n",
        "Once the CPT is defined, other calculations can be done easily:\n",
        "\n",
        "* What is the probability to be rare for a beetle NOT having the pattern?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "155",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(rare.given(~pattern))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "156",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability for any beetle to have the pattern?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "157",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(pattern)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "158",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability for a beetle to be rare AND to have the pattern?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "159",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(rare & pattern)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "160",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "It is even possible to build a joint distribution giving all possible conjunctions (AND), by using the joint operation:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "162",
        "colab_type": "text"
      },
      "source": [
        "This first example shows you the general process: you set up the input probabilities, conditional or not, in a declarative manner. Then, you type queries is a natural way and Lea apply conditional probability rules behind the scene. This technique can be used to solve the intriguing [Monty Hall problem](http://bitbucket.org/piedenis/lea/wiki/Lea3_Examples#markdown-header-the-monty-hall-problem)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "161",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(rare,pattern)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "163",
        "colab_type": "text"
      },
      "source": [
        "## CPT with non-boolean probability distributions\n",
        "\n",
        "The previous example use boolean probability distributions, which is common with conditional probabilities. However, depending on the problem at hand, other types of distribution can be handled. To illustrate this point we shall revisit the previous problem with 2 variables, `kind` and `aspect`, which refer to string probability distributions.\n",
        "\n",
        "![bn2.png](https://bitbucket.org/repo/BpoAoj/images/3868465651-bn2.png)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "164",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "kind = lea.pmf({ 'rare'   : 0.001,\n",
        "                 'common' : 0.999 })\n",
        "aspect_if_rare   = lea.pmf({ 'pattern'    : 0.98,\n",
        "                             'no pattern' : 0.02 })\n",
        "aspect_if_common = lea.pmf({ 'pattern'    : 0.05,\n",
        "                             'no pattern' : 0.95 })\n",
        "aspect = kind.switch({ 'rare'   : aspect_if_rare,\n",
        "                       'common' : aspect_if_common })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "165",
        "colab_type": "text"
      },
      "source": [
        "Note: an alternative, more general, construction shall be presented later (see [`cpt` method](#cpt-with-boolean-expressions-cpt-method)).\n",
        "\n",
        "Now, `aspect` is a new CPT that gives probability distribution of 'pattern' vs 'no pattern', according to the value of `kind`. Now, the question \"what is the probability to be rare for a beetle having the pattern?\" can be restated in one of the following manners:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "166",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "kind.given(aspect == 'pattern')\n",
        "P((kind == 'rare').given(aspect == 'pattern'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "167",
        "colab_type": "text"
      },
      "source": [
        "In the present example, using booleans or string attributes to model the problem is a matter of taste. However, in many situations, models go beyond binary choices and cannot be represented by boolean distributions. For example, imagine that the entomologist wants split the kind 'common' into 'common\\_A' and 'common\\_B' as follows:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "168",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "kind2 = lea.pmf({ 'rare'     : 0.001,\n",
        "                  'common_A' : 0.342,\n",
        "                  'common_B' : 0.657 })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "169",
        "colab_type": "text"
      },
      "source": [
        "Also, he wants to split the aspect 'pattern' into 'stripped' and 'mottled', with given conditional probabilities:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "170",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "aspect2_if_rare     = lea.pmf({ 'stripped'   : 0.78,\n",
        "                                'mottled'    : 0.20,\n",
        "                                'no pattern' : 0.02 })\n",
        "aspect2_if_common_A = lea.pmf({ 'stripped'   : 0.01,\n",
        "                                'mottled'    : 0.04,\n",
        "                                'no pattern' : 0.95 })\n",
        "aspect2_if_common_B = lea.pmf({ 'stripped'   : 0.03,\n",
        "                                'mottled'    : 0.02,\n",
        "                                'no pattern' : 0.95 })\n",
        "aspect2 = kind2.switch({ 'rare'     : aspect2_if_rare,\n",
        "                         'common_A' : aspect2_if_common_A,\n",
        "                         'common_B' : aspect2_if_common_B } )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "171",
        "colab_type": "text"
      },
      "source": [
        "Here are some examples of queries."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "172",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "aspect2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mXFxxnUEl89K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "kind2.given(aspect2 == 'stripped')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g2Ok0nKtl8vF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "kind2.given(aspect2 != 'no pattern')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n70a9LKJl8Se",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((kind2[:3] == 'com').given(aspect2 != 'no pattern'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ka07EYlZmJyQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((kind2 == 'rare').given(aspect2 != 'no pattern'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "173",
        "colab_type": "text"
      },
      "source": [
        "Note the consistency of the last result with the first beetle model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "174",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((kind == 'rare').given(aspect == 'pattern'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "175",
        "colab_type": "text"
      },
      "source": [
        "This consistency is due to the fact that the entomologist has carefully made the model refinement so that (see above):\n",
        "$$\n",
        "\\begin{align}\n",
        "P(common) &= P(common\\_A) + P(common\\_B)\\\\\n",
        "P(pattern) &= P(stripped) + P(mottled)\n",
        "\\end{align}\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "176",
        "colab_type": "text"
      },
      "source": [
        "<a id=\"bayesian-networks\"></a>\n",
        "## Bayesian networks\n",
        "\n",
        "The technique to build CPT can be used to define _Bayesian networks_ (BN), which model causality chains between uncertain events. There is no new syntax here but you shall see how multiple CPT can be connected together to define complex networks.\n",
        "\n",
        "We shall use the well-known \"burglary Bayesian network\" of J. Pearl, explained in [AIMA book](http://aima.cs.berkeley.edu/). You can find also good descriptions of this network on the Internet with the following keywords: _burglar_, _Bayes_ (note that that the classical, yet simpler, example of \"rain-sprinkler-grass\" is covered in [Lea examples page](https://bitbucket.org/piedenis/lea/wiki/Examples#markdown-header-bayesian-network-the-rain-sprinkler-grass-example)).\n",
        "\n",
        "Here is the graphical model of this BN.\n",
        "\n",
        "![bn3.png](https://bitbucket.org/repo/BpoAoj/images/2515863782-bn3.png)\n",
        "\n",
        "Here is how to model this Bayesian network in Lea:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "177",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "burglary   = lea.event(0.001)\n",
        "earthquake = lea.event(0.002)\n",
        "alarm = lea.joint(burglary,earthquake).switch({ (True ,True ) : lea.event(0.950),\n",
        "                                                (True ,False) : lea.event(0.940),\n",
        "                                                (False,True ) : lea.event(0.290),\n",
        "                                                (False,False) : lea.event(0.001) })\n",
        "john_calls = alarm.switch({ True  : lea.event(0.90),\n",
        "                            False : lea.event(0.05) })\n",
        "mary_calls = alarm.switch({ True  : lea.event(0.70),\n",
        "                            False : lea.event(0.01) })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "178",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "We have not done more than building three CPT, using the syntax explained in the previous sections. The sole new technique used here is for the `alarm` CPT, which depends on two variables, viz. `burglary` and `earthquake`: the `joint` method is used to join these two variables so as to produce the four combinations as tuples containing booleans.\n",
        "\n",
        "Now, we are ready to query the network. Let us first make \"forward\" derivations (i.e from causes to effects).\n",
        "\n",
        "* What is the probability that Mary calls if the alarm is triggered? (this is a given!)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "179",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(mary_calls.given(alarm))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "180",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability that Mary calls if there is a burglary?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "181",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(mary_calls.given(burglary))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "182",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "Now, we can also query the Bayes network in a \"backward\" manner (i.e. from effects to cause)\n",
        "\n",
        "* What is the probability that there is a burglary if alarm is triggered?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "183",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(burglary.given(alarm))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "184",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability that the alarm is triggered if Mary calls?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "185",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(alarm.given(mary_calls))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "186",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability that there is a burglary if Mary OR John calls (or both)?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "187",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(burglary.given(mary_calls | john_calls))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "188",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability that there is a burglary if Mary AND John call?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "189",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(burglary.given(mary_calls & john_calls))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "190",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "It is also possible to get unconditional probabilities of events or conjunction of events\n",
        "\n",
        "* What is the probability that the alarm is triggered?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "191",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(alarm)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "192",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability that there is a burglary and Mary calls?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "193",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(burglary & mary_calls)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "194",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "* What is the probability that there is neither burglary nor earthquake, the alarm triggers and both John and Mary call?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "195",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(~burglary & ~earthquake & alarm & john_calls & mary_calls)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "196",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "Note that the last result can be checked in the [AIMA book](http://aima.cs.berkeley.edu/).\n",
        "\n",
        "As an academic exercise, you can very easily \"flatten\" the network to build a joint table giving the probabilities of each conjunction of events. This boils down to calculate the joint probability distribution between the 5 variables."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "197",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(burglary,earthquake,alarm,john_calls,mary_calls)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "198",
        "colab_type": "text"
      },
      "source": [
        "We see here the interest of using Bayesian networks, defined with only 10 probability values for the causal dependencies, instead of 32 for the joint table. Let's stress that this calculated table takes into account the dependencies between the 5 variables. It may be used to verify or explain the query results given above (marginalization)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "199",
        "colab_type": "text"
      },
      "source": [
        "## Advanced features for Bayesian networks\n",
        "\n",
        "One of the drawbacks of Bayesian networks using the simple CPT technique presented is that it requires specifying probability data for each combination of influencing variables. Since the number of cases grows exponentially with the number of influencing variables, this could become difficult to specify.\n",
        "\n",
        "The present section presents several ways to reduce the number of cases to specify in CPT, by means of factorization.\n",
        "\n",
        "Note that the special topic of calculating CPT from data shall be covered later, in [a specific section dedicated to Machine Learning](http://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_3#markdown-header-building-a-bayesian-network-from-a-joint-table)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "200",
        "colab_type": "text"
      },
      "source": [
        "### Specifying a default entry\n",
        "\n",
        "One of the simplest ways to reduce the enumeration of cases in CPT is to have a default entry that groups all the cases that have not been explicitly given.\n",
        "\n",
        "Let us revisit the burglary network. Consider a new model of alarm device for which the two cases \"burglary without earthquake\" and \"earthquake without burglary\" cannot be distinguished, these being associated to a probability equal to 0.9. Instead of writing this:\n",
        "```\n",
        "alarm2 = lea.joint(burglary,earthquake).switch({ (True ,True ) : lea.event(0.950),\n",
        "                                                 (True ,False) : lea.event(0.900),\n",
        "                                                 (False,True ) : lea.event(0.900),\n",
        "                                                 (False,False) : lea.event(0.001) })\n",
        "```\n",
        "you could put the common 0.900 probability as a second argument of the `switch` method, just after the dictionary:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "203",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "alarm2 = lea.joint(burglary,earthquake).switch({ (True ,True ) : lea.event(0.950),\n",
        "                                                 (False,False) : lea.event(0.001) },\n",
        "                                                 lea.event(0.900)  )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q68JJY52kDTM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(alarm2.given(burglary != earthquake))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "204",
        "colab_type": "text"
      },
      "source": [
        "### Building CPT with one condition (`if_` method)\n",
        "\n",
        "For variables depending on one Boolean variable, the switch method can be abbreviated using the `if_` convenience function: it mimics the if-then-else constructions found in programming languages (the underscore is put to avoid a clash with Python's `if` keyword). For instance, the definition"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "205",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "john_calls = alarm.switch({ True  : lea.event(0.90),\n",
        "                            False : lea.event(0.05) })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "206",
        "colab_type": "text"
      },
      "source": [
        "can be rewritten as follows :"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "207",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "john_calls = lea.if_( alarm, lea.event(0.90),\n",
        "                             lea.event(0.05) )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "208",
        "colab_type": "text"
      },
      "source": [
        "This is no more than syntactic sugar: the `if_` construct builds a CPT in a somehow more expressive way than the `switch` method."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "209",
        "colab_type": "text"
      },
      "source": [
        "## Reducing the size of CPT\n",
        "\n",
        "All the examples seen above use the `switch` method to build up CPT. This allows specifying a set of clauses, each of which is triggered when the decision variables are equal to a specific combination of values. Although the `switch` method works fine and is efficient (through Python dictionaries), it is limited in the way the conditions are expressed since it requires an enumeration of all possible values. This could be a burden when the decision variable has a large domain, for example if it joins multiple variables or if the domain covers a big range of values. More importantly, such explicit CPT may be intractable due to memory usage. In several cases however, it happens that the CPT\n",
        "\n",
        "* returns the same probability distribution for different values (see examples below), possibly if some condition is verified (_context-specific independence_),\n",
        "* or returns probability distributions that can be calculated by a deterministic function (e.g. noisy-or, noisy-max models).\n",
        "\n",
        "In such situations, it's possible to avoid the definition of an explicit CPT by exploiting the properties/assumptions of the probabilistic model. For this purpose, we present below three techniques, all available in Lea:\n",
        "\n",
        "1. CPT defined by function\n",
        "2. CPT with Boolean expressions\n",
        "3. Cascaded CPT"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "211",
        "colab_type": "text"
      },
      "source": [
        "### CPT defined by function (`switch_func` method)\n",
        "\n",
        "***Note: the technique shown in the present section requires Lea 3.1 at least.***\n",
        "\n",
        "Let's imagine a world where the temperatures are distributed uniformly from -10Â°C to 29Â°C, as integer numbers:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "212",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temperature = lea.interval(-10,29)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "213",
        "colab_type": "text"
      },
      "source": [
        "To measure the temperature in this world, in place of a precise thermometer, we've got a device that indicates COLD, MEDIUM or HOT depending on the temperature: unfortunately, this device is very imprecise and the best that we can do is to model its indicator as three probability distributions, which depend of the range of temperatures:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "214",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "ind_if_temperature_below_0          = lea.pmf({ 'COLD': 0.75, 'MEDIUM': 0.25               })\n",
        "ind_if_temperature_between_0_and_19 = lea.pmf({ 'COLD': 0.25, 'MEDIUM': 0.60,  'HOT': 0.15 })\n",
        "ind_if_temperature_above_20         = lea.pmf({               'MEDIUM': 0.20,  'HOT': 0.80 })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "215",
        "colab_type": "text"
      },
      "source": [
        "So, we see that the indicator roughly follows the temperature value, with some randomness. The indicator shall then be specified as a CPT linking any temperature to one of the three above-specified distributions. If we use the `switch` technique seen above, there are 40 entries to specify:\n",
        "```\n",
        "indicator = temperature.switch({ -10: ind_if_temperature_below_0,\n",
        "                                 ...\n",
        "                                  -1: ind_if_temperature_below_0,\n",
        "                                   0: ind_if_temperature_between_0_and_19,\n",
        "                                 ...\n",
        "                                  19: ind_if_temperature_between_0_and_19,\n",
        "                                  20: ind_if_temperature_above_20,\n",
        "                                 ...\n",
        "                                  29: ind_if_temperature_above_20 })\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "217",
        "colab_type": "text"
      },
      "source": [
        "There is much redundancy here; it's a case of context-specific independence: the indicator is independent of the temperature as soon as the actual temperature has defined which of the three ranges is applicable. Things could be improved a bit by grouping one of the third as a [default entry](#specifying-a-default-entry), but it remains unsatisfactory for the modeler point of view.\n",
        "\n",
        "The `switch_func` method offers a way to remove the need to provide such redundant CPT. The idea is to provide a method that emulates the CPT lookup on the fly. So instead of relying on a dictionary as above, you provide a Python function that returns the probability distribution of the indicator for any temperature given in argument."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "218",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def indicator_func(temperature_value):\n",
        "    if temperature_value < 0:\n",
        "        return ind_if_temperature_below_0\n",
        "    if temperature_value < 20:\n",
        "        return ind_if_temperature_between_0_and_19\n",
        "    return ind_if_temperature_above_20\n",
        "    \n",
        "indicator = temperature.switch_func(indicator_func)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "219",
        "colab_type": "text"
      },
      "source": [
        "Here are some queries, which check the forward dependency from the temperature to the indicator:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "220",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "indicator.given(temperature == -5)\n",
        "indicator.given(temperature <= 0)\n",
        "indicator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "221",
        "colab_type": "text"
      },
      "source": [
        "And here are queries that infer the temperature from the indicator:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "222",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature < 0).given(indicator == 'COLD'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ChwYXS-0nAiQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature < 0).given(indicator == 'MEDIUM'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n9PQzZlNnATS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature < 0).given(indicator == 'HOT'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7lX3oz12nADp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature <= 0).given(indicator == 'HOT'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ngspRkWUm_2g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature > 0).given(indicator == 'HOT'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pDQEFNHFm_oE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature <= 0).given(indicator != 'COLD'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1s8JCPg6m_Xd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P((temperature <= 0).given(indicator.is_any_of('MEDIUM','HOT')))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "223",
        "colab_type": "text"
      },
      "source": [
        "As explained before, you can obtain some understanding on the underlying calculations by making a joint between the variables:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "224",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lea.joint(indicator,temperature)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "225",
        "colab_type": "text"
      },
      "source": [
        "For this `switch_func` construct, it's important to note that, internally, the `indicator_func` function is stored *in place of* the CPT. The probabilistic inference algorithm is smart enough to call this function when needed, instead of expanding the full CPT. So, this construct is very efficient in term of memory usage.\n",
        "\n",
        "We shall see later how the `switch_func` can be used to build up noisy-or / noisy-max models (see [here](#noisy-or-noisy-max-models)). Also, it can be used to solve the [Monty Hall problem](http://bitbucket.org/piedenis/lea/wiki/Lea3_Examples#markdown-header-the-monty-hall-problem), more easily than with the `switch` method."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "227",
        "colab_type": "text"
      },
      "source": [
        "### CPT with Boolean expressions (`cpt` method)\n",
        "\n",
        "The `cpt` function offers an alternate solution to the avoid CPT redundancies. The idea is to specify a list of entries made up of Boolean conditions with associated Lea instances. It's a bit like you would do in Python with a sequence of `if` constructs (see `indicator_func` above). Revisiting the example given in the previous section, here is the syntax to build the `indicator` variable using `cpt`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "228",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "indicator = lea.cpt( (temperature < 0                        , ind_if_temperature_below_0          ),\n",
        "                     ((0 <= temperature) & (temperature < 20), ind_if_temperature_between_0_and_19 ),\n",
        "                     (20 <= temperature                      , ind_if_temperature_above_20         ))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "229",
        "colab_type": "text"
      },
      "source": [
        "With this construct, the example queries given in the previous section produce the very same results.\n",
        "\n",
        "Note that the method expects that the given conditions cover all possible cases and are mutually exclusive (so, it's not exactly comparable to an `if` ...`elif` ... `else` construct). If this is not respected, then an exception is raised.\n",
        "\n",
        "Similarly to the `switch` method, the `cpt` method accepts a default clause: it is represented by condition set to `None`. This special value is a placeholder for any case not covered by the other condition(s). To illustrate this, here is a definition of `indicator` that is exactly equivalent to the previous one."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "230",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "indicator = lea.cpt( (temperature < 0  , ind_if_temperature_below_0          ),\n",
        "                     (None             , ind_if_temperature_between_0_and_19 ),\n",
        "                     (20 <= temperature, ind_if_temperature_above_20         ))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "231",
        "colab_type": "text"
      },
      "source": [
        "As you see, `None` can be specified at any place of `lea.cpt`'s arguments, not specially the last one. Of course, no more than one `None` is authorized (an exception is raised, otherwise)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "233",
        "colab_type": "text"
      },
      "source": [
        "## Cascaded CPT\n",
        "\n",
        "In several CPT presenting context-specific independence, it's possible to remove redundancies by building CPT \"in cascade\". Here is a very basic example, where `z` is a CPT depending of `x` and `y`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "234",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = lea.event('1/3')\n",
        "y = lea.event('1/4')\n",
        "z = lea.joint(x,y).switch({ (False, False): lea.event('1/5'),\n",
        "                            (False, True ): lea.event('1/7'),\n",
        "                            (True , False): lea.event('1/2'),\n",
        "                            (True , True ): lea.event('1/2') })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "235",
        "colab_type": "text"
      },
      "source": [
        "You see that if `x` is True then `z` is a 50-50 distribution, which does not depend of `y`. The redundancy can then be removed by embedding a CPT governed by `y` only within a CPT governerd by `x` only, as follows:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "236",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "z1 = x.switch({ False: y.switch({ False: lea.event('1/5'),\n",
        "                                  True : lea.event('1/7')}),\n",
        "                True : lea.event('1/2') })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "237",
        "colab_type": "text"
      },
      "source": [
        "Equivalently, a similar construct can be done using `cpt`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "238",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "z2 = lea.cpt( ( ~x, lea.cpt( (~y, lea.event('1/5')),\n",
        "                             ( y, lea.event('1/7')))),\n",
        "              (  x, lea.event('1/2')) )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "239",
        "colab_type": "text"
      },
      "source": [
        "You can now check by various queries that `z`, `z1` and `z2` represent the same logical CPT. For instance,"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "240",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P = lea.P\n",
        "print (P(z.given(y)))\n",
        "print (P(z1.given(y)))\n",
        "print (P(z2.given(y)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "241",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "Of course, all these examples are quite contrived and the gain is negligible; you can anyway extrapolate the benefit of these techniques on bigger CPT, having prohibitive combinatorial."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "243",
        "colab_type": "text"
      },
      "source": [
        "### Noisy-or / noisy-max models\n",
        "\n",
        "The noisy-or / noisy-max models provide practical alternatives to naive CPT, provided that some assumptions can be made on the causal dependencies (we won't detail these here). Lea provides several means to build up such models. For the example, we shall define a symptom boolean random variable (`fever`), which is modeled as a noisy-or of disease boolean random variables (`cold`, `flu` and `malaria`) (see figures and explanations in [AIMA book](http://aima.cs.berkeley.edu/) or in [this presentation](http://www.blutner.de/Intension/Noisy%20OR.pdf).\n",
        "\n",
        "Let's first define the input data, which are the conditional probabilities of fever if the occurrence of one single disease:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "244",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "prob_fever_if_sole_cold    = 0.4\n",
        "prob_fever_if_sole_flu     = 0.8\n",
        "prob_fever_if_sole_malaria = 0.9"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "245",
        "colab_type": "text"
      },
      "source": [
        "then,  a priori probability distributions of these diseases (arbitrary values, not found in afore-mentioned reference):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "246",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "cold    = lea.event(0.082)\n",
        "flu     = lea.event(0.025)\n",
        "malaria = lea.event(0.001)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "247",
        "colab_type": "text"
      },
      "source": [
        "Now, the following construct define the `fever` variable as a noisy-or of the three diseases:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "248",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fever_by_cold    = lea.if_(cold   , lea.event(prob_fever_if_sole_cold)   , False)\n",
        "fever_by_flu     = lea.if_(flu    , lea.event(prob_fever_if_sole_flu)    , False)\n",
        "fever_by_malaria = lea.if_(malaria, lea.event(prob_fever_if_sole_malaria), False)\n",
        "fever = fever_by_cold | fever_by_flu | fever_by_malaria"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "249",
        "colab_type": "text"
      },
      "source": [
        "The `fever_by_cold` variable can be interpreted as \"having fever due to a cold\": in case of cold, it has a probability 0.4 to be true, if there is no cold, it is surely false (i.e. either there is no fever or the fever is not caused by cold). The `fever` is obtained by ORing the three 'noised' variables. This uses the dedicated Lea's  construct, which simply uses the classical OR to combine all possible values. Note that this noisy-or model is very different to:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "250",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "light_fever = cold | flu | malaria"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YOKja7sf6le7",
        "colab_type": "text"
      },
      "source": [
        "which models a symptom that occurs certainly (probability 1) as soon as at least one disease occurs, e.g. a light fever. Note that you would get the same result if you had set `prob_fever_if_sole_cold` = `prob_fever_if_sole_flu` = `prob_fever_if_sole_malaria` = 1. In such special case, the noisy-or boils down to a classical or.\n",
        "One way to check that `fever` is correctly defined is by checking one by one each entry of the equivalent CPT, which can be calculated easily from the 3 given probabilities. Here is how this check can be done:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CHe8TLIk6p-h",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "diseases = lea.joint(cold,flu,malaria)\n",
        "for (c,f,m) in diseases.support:\n",
        "     print (c, f, m, P(fever.given(cold==c,flu==f,malaria==m)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bQ1yDpPv7m4s",
        "colab_type": "text"
      },
      "source": [
        "You can check that values match the ones found in the afore-mentioned references. You may then execute queries, as described earlier in the tutorial:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tQiOPjkQ7s8G",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(fever.given(~cold,flu,malaria))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "00qvaYQ_7x9v",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(malaria.given(fever))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wd21GHCq7xp4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(flu.given(~fever,cold))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rFMVwa1y7-UU",
        "colab_type": "text"
      },
      "source": [
        "Noisy-max models may be define in a similar way, using `lea.max_of` method. These may use non-boolean random variables, provided that their values are ordered. You may note that the definition of `fever` above could be replaced by"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E-RfDY6E8dk_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fever = lea.max_of(fever_by_cold, fever_by_flu, fever_by_malaria)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GwmelmTg8pvQ",
        "colab_type": "text"
      },
      "source": [
        "without changing anything since Python booleans are ordered (`False < True`). This tends to confirm that noisy-max is a generalization of noisy-or.\n",
        "Before concluding the section, let's mention that there exists an alternate way to make the noisy-or / noisy-max models, which relies on the [switch_func method](#cpt-defined-by-function-switch_func-method). The idea is to define a function that mimics the CPT, by calculating the probability distributions expected for values of causing variables passed as argument. This function shall of course use the assumptions made for the noisy model at hand. Here is how to proceed to redo the fever model:\n",
        "\n",
        "**Note: the technique shown below requires Lea 3.1 at least.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "256",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def fever_event(disease_val):\n",
        "    (cold_val, fever_val, malaria_val) = disease_val\n",
        "    inv_prob = 1.\n",
        "    if cold_val:\n",
        "        inv_prob *= 1. - prob_fever_if_sole_cold\n",
        "    if fever_val:\n",
        "        inv_prob *= 1. - prob_fever_if_sole_flu\n",
        "    if malaria_val: inv_prob *= 1. - prob_fever_if_sole_malaria\n",
        "    return lea.event(1.-inv_prob)\n",
        "    \n",
        "diseases = lea.joint(cold,flu,malaria)\n",
        "fever2 = diseases.switch_func(fever_event)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "257",
        "colab_type": "text"
      },
      "source": [
        "You may check that replacing `fever` by `fever2` in the query above give the very same results. For instance,\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vrMiv2MOt7XQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(fever2.given(~cold,flu,malaria))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "laQ0T-BluAPH",
        "colab_type": "text"
      },
      "source": [
        "To conclude, all the models seen here do not necessitate the definition of large CPT. Furthermore, the inference algorithm used does not require to calculate such CPT internally, so these models are very efficient in memory usage.\n",
        "\n",
        "Note: for another application of noisy-or, you may check the Jupyter notebook [\"Bayesian Reasoning with Lea: the COVID-19 Case\"](https://mybinder.org/v2/gh/piedenis/lea_mini_tutorials/master?filepath=Lea_COVID19.ipynb).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "259",
        "colab_type": "text"
      },
      "source": [
        "### Prior probabilities (`revised_with_cpt` method)\n",
        "\n",
        "The methods seen so far for building CPT miss an important class of problems of conditional probability: those mixing a conditional probability and a prior, unconditional, probability.\n",
        "\n",
        "We shall take here an example of the [AIMA book](http://aima.cs.berkeley.edu/). It models a causal dependency between the meningitis and the stiff neck. The assumptions are the following:\n",
        "\n",
        "* the probability that a patient has meningitis is 1/50,000 (prior probability)\n",
        "* the probability that a patient has stiff neck is 1/20 (prior probability)\n",
        "* the meningitis causes stiff neck 50 % of the times (conditional probability)\n",
        "\n",
        "OK, the probability of meningitis is easily tackled:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "260",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "meningitis = lea.event(1/50000)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "261",
        "colab_type": "text"
      },
      "source": [
        "For stiff neck however, should we try the `if_` syntax, we are stuck on the \"else\" argument:\n",
        "```\n",
        "stiffneck = Lea.if_( meningitis, lea.event(1/2),\n",
        "                                 ?? WHAT HERE ??  )\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "263",
        "colab_type": "text"
      },
      "source": [
        "The problem is that the probability of stiff neck if NO meningitis is unknown, so we cannot fill in the second part of the CPT definition. By careful calculations, we could try to find this value such that the unconditional probability of stiff neck is equal to 1/20. Fortunately, this is not needed: Lea provides a special method ` revised_with_cpt`, to define CPT taking into account prior probability. Here is how to define `stiffneck` from the given data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "264",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "stiffneck = lea.event(1/20).revised_with_cpt((meningitis,lea.event(1/2)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "265",
        "colab_type": "text"
      },
      "source": [
        "This says: \"Without any information, the probability of stiff neck for a given patient is 1/20; however, if the patient has got meningitis, then this probability becomes 1/2\". Let us check these statements:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "266",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(stiffneck)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GR7DiV_M-I7R",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(stiffneck.given(meningitis))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "267",
        "colab_type": "text"
      },
      "source": [
        "Note that `stiffneck` object is a true CPT. Behind the scene, Lea has calculated the probability of stiff neck if no meningitis; then, it was able to fill in the missing data. You can get this value:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "268",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(stiffneck.given(~meningitis))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "269",
        "colab_type": "text"
      },
      "source": [
        "In the first statement, you could replace the \"?? WHAT HERE ??\" by `0.0499909998199964` and get the same CPT. Of course, this is not needed since Lea has made the work for you!\n",
        "\n",
        "Now, we are able to compare the probability of meningitis, prior and after stiff neck information:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "270",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(meningitis)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sptq96Sh-Wjt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(meningitis.given(stiffneck))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "271",
        "colab_type": "text"
      },
      "source": [
        "We see that the given information has multiplied the probability of meningitis by ten. This result is in line with the Bayes rule, since\n",
        "\n",
        "$$\n",
        "\\begin{align}\n",
        "P(meningitis \\;|\\; stiffneck) & = \\dfrac{P(stiffneck \\;|\\; meningitis) P(meningitis)} { P(stiffneck)} \\\\\n",
        " & = \\dfrac {\\frac{1}{2} \\frac{1}{50000}} {\\frac{1}{20}} = \\dfrac {1}{5000} = 0.0002\n",
        "\\end{align}\n",
        "$$\n",
        "\n",
        "We can check that also using Lea:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "272",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "P(stiffneck.given(meningitis)) * P(meningitis) / P(stiffneck)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "273",
        "colab_type": "text"
      },
      "source": [
        "As noted before, the previous expression is NOT the way the conditional probability is computed in Lea: a most general algorithm is used, which is also most efficient.\n",
        "\n",
        "You should take care that some problems have no solutions. In such cases, Lea raises an exception with an error message giving the possible range for the prior probability. Example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "274",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "stiffneck2 = lea.event(1/200000).revised_with_cpt((meningitis,lea.event(1/2)))  # raise lea.Error exception!"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "275",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "Prior probabilities work also with non-boolean distributions. Let us revisit [the beetle example modeled with attribute](#cpt-with-non-boolean-probability-distributions). Imagine that the problem is restated as follows: the prior probability of a beetle with pattern is 5.093 %, in the rare subspecies, 98 % have the pattern; the rare subspecies accounts for only 0.1 % of the population. Note that, considering the initial problem, we have removed one data (conditional probability of pattern for common beetle) and added one data (prior probability pattern).\n",
        "\n",
        "Here are the statements to model these data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "276",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "kind = lea.pmf({ 'rare'   : 0.001,\n",
        "                 'common' : 0.999 })\n",
        "aspect0 = lea.pmf({ 'pattern'    : 0.05093,\n",
        "                    'no pattern' : 0.94907 })\n",
        "aspect1 = aspect0.revised_with_cpt(( kind == 'rare', lea.pmf({ 'pattern'    : 0.98,\n",
        "                                                               'no pattern' : 0.02 })))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "277",
        "colab_type": "text"
      },
      "source": [
        "Now, what is probability of pattern in common subspecies?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "278",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "aspect1.given(kind == 'common')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "279",
        "colab_type": "text"
      },
      "source": [
        "This is striking: we get back the same probability of 5% as the initial problem! Why is it so? The answer lies in the given value 0.05093 for prior probability of pattern. Actually, this value has been chosen on purpose, based on unconditional probability of pattern, as calculated in the initial problem (i.e. `aspect`, not `aspect1`):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "280",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "aspect"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "281",
        "colab_type": "text"
      },
      "source": [
        "This demonstrates the consistency between the two methods to define CPT.\n",
        "\n",
        "\n",
        "# Markov chains\n",
        "\n",
        "Lea allows defining [Markov chains](http://en.wikipedia.org/wiki/Markov_chain). We shall show here how to model the [stock market Markov chain example](http://en.wikipedia.org/wiki/Markov_chain#Example) given on Wikipedia.\n",
        "\n",
        "To execute the examples shown in this section, you have to import the `markov` module present in the Lea package:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "282",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from lea import markov"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "284",
        "colab_type": "text"
      },
      "source": [
        "## Definition from transition probability matrix\n",
        "\n",
        "The states defining the Markov chain can be any Python object. We shall use here the most simple option : strings. The three market states of the example shall be named \"BULL\", \"BEAR\" and \"STAG\". Here is how you define these states in Lea, along with the probability transition matrix (note that we omit the prompts, so you can copy-paste easily the multline statement):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "285",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market = markov.chain_from_matrix(\n",
        "( 'BULL', 'BEAR', 'STAG'  ),\n",
        "( 'BULL', ( 0.900 , 0.075 , 0.025  )),\n",
        "( 'BEAR', ( 0.150 , 0.800 , 0.050  )),\n",
        "( 'STAG', ( 0.250 , 0.250 , 0.500  )))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "286",
        "colab_type": "text"
      },
      "source": [
        "Note that the rows of the matrix represent _from_ state and columns represent _to_ states. For instance, the first row indicates the probabilities of transition starting from \"BULL\" : 90 % to stay \"BULL\", 7.5 % to become \"BEAR\" and 2.5 % to become \"STAG\".\n",
        "\n",
        "We have here a Markov chain, named `market`, which captures the set of states and all probabilities of transition from state to state. These probabilities can be displayed by using the same methods as shown for simple distributions:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "287",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "288",
        "colab_type": "text"
      },
      "source": [
        "Take care however that `market` is NOT a Lea instance (actually, it's a `markov.Chain`). Apart displaying itself, the operations you can do on this object are minimal: getting its inner states or given probability distributions of these states. We shall see this in the following."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "289",
        "colab_type": "text"
      },
      "source": [
        "## Querying the Markov chain\n",
        "\n",
        "To make queries, we need to be able to define an initial state. Here is how to proceed:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "290",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(bull_state,bear_state,stag_state) = market.get_states()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "291",
        "colab_type": "text"
      },
      "source": [
        "Each of these 3 variables is Lea instance representing a certain distribution, i.e. having a probability 1. For instance, the `bear_state` object represents the state BEAR:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "292",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bear_state"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "293",
        "colab_type": "text"
      },
      "source": [
        "You may also use the `state' attribute, which defines a uniform probability distribution over the states:\n",
        "\n",
        "***Note: In Lea version previous to 3.1, write `_state` instead of `state`.***"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "294",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market.state"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "295",
        "colab_type": "text"
      },
      "source": [
        "To get the \"next state\" distribution (i.e. the state at _next week_, in the example), use the `next_state()` method:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "296",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market.state.next_state()\n",
        "bear_state.next_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "297",
        "colab_type": "text"
      },
      "source": [
        "You can check (again) on this last example that the given probability transition matrix has been set up correctly. Note that the object returned by `next_state(â€¦)` is a Lea instance, on which you can apply usual techniques seen in the tutorial.\n",
        "\n",
        "To get the state probability distribution after _n_ steps, instead of chaining _n_ times the `next_state()` method, you can simply pass _n_ as argument:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "298",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bear_state.next_state(3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "299",
        "colab_type": "text"
      },
      "source": [
        "You can check these results with Wikipedia's.\n",
        "\n",
        "To define an initial non-uniform probability distribution, you need to pass the state probability distribution to the `.get_state` method :"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "300",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fp_state = market.get_state(lea.pmf({ 'BULL': 0.6250,\n",
        "                                      'BEAR': 0.3125,\n",
        "                                      'STAG': 0.0625 }))\n",
        "fp_state"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "301",
        "colab_type": "text"
      },
      "source": [
        "In the present case, this distribution is a bit special: it is actually the \"fixed point\"; you can verify that it does not change on next step (if you ignore rounding errors!):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "302",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fp_state.next_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "303",
        "colab_type": "text"
      },
      "source": [
        "Note that you can also reach this distribution, by starting from any state and making a large number of transitions (e.g. `bear_state.next_state(1000)`).\n",
        "\n",
        "Another way to define a non-trivial probability distribution is by specifying a condition on the state:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "304",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market.state_given(market.state != 'STAG')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SAmA_bYnAp4L",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market.state_given(market.state[0] == 'B')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "306",
        "colab_type": "text"
      },
      "source": [
        "## Definition from a sequence of states\n",
        "\n",
        "There exists another way to define a Markov chain: the empirical way. Instead of setting a probability matrix, which could be hard to define, it is possible to define a Markov chain by providing a sequence of states supposed to be long enough to be representative of all transitions. Typically, this could be a sample sequence of observed states for a given system.\n",
        "\n",
        "This sort of definition is done through the `markov.chain_from_seq` method. Here is an example, using a very short sequence:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "307",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market2 = markov.chain_from_seq(('BEAR','BEAR','BULL','BULL','BULL','BEAR','STAG','BULL','STAG','BEAR'))\n",
        "market2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "308",
        "colab_type": "text"
      },
      "source": [
        "You can verify that the transition probabilities are exactly the frequencies of transition in the given sequence. In particular, there is no transition from \"STAG\" to \"STAG\"."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "309",
        "colab_type": "text"
      },
      "source": [
        "## Generating random sequence of states\n",
        "\n",
        "Starting from a given state of a given Markov chain, it is possible to generate a random sequence of states, obeying the transition probabilities of that Markov chain. This is done by calling the `random_seq(n)` method where `n` is the size of the sample sequence. Example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "310",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bear_state.random_seq(40)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "311",
        "colab_type": "text"
      },
      "source": [
        "Now, to check the likelihood of the generated sequence, we can use the method `markov.chain_from_seq ` seen above, which can be seen also as a transition frequency calculator. Let's use a random sequence of 100,000 states this time:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "312",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "state_seq_sample = bear_state.random_seq(100000)\n",
        "markov.chain_from_seq(state_seq_sample)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "313",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "We see that the frequencies of transition are close to the probabilities defined for the `market` Markov chain, as expected."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "314",
        "colab_type": "text"
      },
      "source": [
        "## Matrix extraction / absorbing Markov chains\n",
        "\n",
        "***Note: what follows requires Lea 3.1 at least.***\n",
        "\n",
        "For advanced treatments on Markov chains, you may desire to obtain characteristic data as matrices.\n",
        "\n",
        "The probability transition matrix can be retrieved by using the `matrix` method. If you have installed [NumPy package](http://www.numpy.org), then you can obtain this matrix as a numpy array using the `as_array` optional argument:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "315",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market.matrix()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6bT2GArfBaOH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "market.matrix(as_array=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "316",
        "colab_type": "text"
      },
      "source": [
        "If your Markov chain is [absorbing](https://en.wikipedia.org/wiki/Absorbing_Markov_chain), than you may want to get\n",
        "several information on this absorbing MC. The method `absorbing_mc_info()` serves this purpose, it returns a 6-tuple:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "317",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(is_absorbing,transient_states,absorbing_states,q_matrix,r_matrix,n_matrix) = market.absorbing_mc_info()\n",
        "print (is_absorbing,transient_states,absorbing_states,q_matrix,r_matrix,n_matrix)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "318",
        "colab_type": "text"
      },
      "source": [
        "Since the market MC is not absorbing, the returned `is_absorbing` is `False` and the other elements give no interesting data. Consider now the following MC:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "319",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dw = markov.chain_from_matrix(\n",
        "(         'S0', 'S1', 'S2', 'S3', 'S4' ),\n",
        "( 'S0', ( 1.0 , 0.0 , 0.0 , 0.0 , 0.0 )),\n",
        "( 'S1', ( 0.5 , 0.0 , 0.5 , 0.0 , 0.0 )),\n",
        "( 'S2', ( 0.0 , 0.5 , 0.0 , 0.5 , 0.0 )),\n",
        "( 'S3', ( 0.0 , 0.0 , 0.5 , 0.0 , 0.5 )),\n",
        "( 'S4', ( 0.0 , 0.0 , 0.0 , 0.0 , 1.0 )))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "320",
        "colab_type": "text"
      },
      "source": [
        "This MC is absorbing because 'S0' and 'S4' are absorbing and because the three other states can reach these two absorbing states. These information, as well as the Q and R matrices, can be retrieved as follows:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "321",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(is_absorbing,transient_states,absorbing_states,q_matrix,r_matrix,n_matrix) = dw.absorbing_mc_info()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Vy0gqRPB0Bo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "is_absorbing"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nbqbpTFLBzqs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "transient_states"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YJTuzPsqB4Js",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "absorbing_states"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OOQB8S1QB_eZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "q_matrix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qSIll9EkCDb1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(n_matrix)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "322",
        "colab_type": "text"
      },
      "source": [
        "The last argument `n_matrix` is supposed to contain the fundamental matrix $N = (I-Q)^{-1}$. To be actually calculated, you need [NumPy package](http://www.numpy.org) and you have to put the `as_array` optional argument: then, the 3 matrices $Q$, $R$ and calculated $N$ are returned as numpy arrays:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "323",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(is_absorbing,transient_states,absorbing_states,q_matrix,r_matrix,n_matrix) = dw.absorbing_mc_info(as_array=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4dqtTWzJCkCk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "q_matrix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eh8eVkXjCnha",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "r_matrix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EPFcsqC_CpRo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "n_matrix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "324",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "# What's next?\n",
        "\n",
        "Thank you for reading the present tutorial page! We hope that you found it enough clear and informative.\n",
        "\n",
        "To learn more, you are invited to jump to\n",
        "\n",
        "* [tutorial [3/3]](http://bitbucket.org/piedenis/lea/wiki/Lea3_Tutorial_3) - plotting, drawing without replacement, machine learning, information theory, MC estimation, symbolic calculation, ...\n",
        "* [Examples page](http://bitbucket.org/piedenis/lea/wiki/Lea3_Examples)\n",
        "\n",
        "For enhancements or bugs, please [create issues on bitbucket Lea page](http://bitbucket.org/piedenis/lea/issues).\n",
        "\n",
        "Feel free also to send an e-mail to [pie.denis@skynet.be](mailto:pie.denis@skynet.be) (English or French spoken). Use cases of Lea in _your_ projects/researches are especially appreciated! A \"references\" page listing the projects using Lea could greatly improve the reputation of the library."
      ]
    }
  ]
}