Lecture 4 - Intro to Stats & Probability.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border:5px outset; width:75%; margin:auto\">\n",
    "<h1 style=\"text-align:center\">Contents Overview</h1>\n",
    "<ul>\n",
    "    <li><h5><a href=\"#c0\">0: Preface</a></h5></li>\n",
    "    <li><h5><a href=\"#c1\">1: Introduction to Statistics</a></h5></li>\n",
    "    <li><h5><a href=\"#c2\">2: Descriptive Statistics</a></h5></li>\n",
    "    <li><h5><a href=\"#c3\">3: Introduction to Probability Theory</a></h5></li>\n",
    "    <li><h5><a href=\"#c4\">4: Random Variables & Common Distributions</a></h5></li>\n",
    "    <li><h5><a href=\"#c5\">5: Other Learning Resources</a></h5></li>\n",
    "</ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border: 5px ridge; padding:5%\"> \n",
    "<h1 style=\"text-align:right\">Contents Breakdown</h1>\n",
    "<hr style=\"border:2px solid black\" />\n",
    "<hr style=\"border:1px solid gray\" />\n",
    "\n",
    "<h3><a href=\"#c0\">Preface</a></h3>\n",
    "\n",
    "<hr style=\"border:1px solid #F0F0F0\" />\n",
    "\n",
    "<h3><a href=\"#c1\">Chapter 1: Introduction to Statistics</a></h3>\n",
    "<ul>\n",
    "    <li><h4><a href=\"#s1.1\">&sect;1.1 Basic Definitions & Concepts</a></h4></li>\n",
    "    <li><h4><a href=\"#s1.2\">&sect;1.2 Types of Variables & Data</a></h4></li>\n",
    "    <li><h4><a href=\"#s1.3\">&sect;1.3 Assumptions & Occam's Razor</a></h4></li>\n",
    "</ul>\n",
    "\n",
    "<hr style=\"border:1px solid #F0F0F0\" />\n",
    "\n",
    "<h3><a href=\"#c2\">Chapter 2: Descriptive Statistics</a></h3>\n",
    "<ul>\n",
    "    <li><h4><a href=\"#s2.1\">&sect;2.1 Describing Qualitative Data</a></h4></li>\n",
    "    <li><h4><a href=\"#s2.2\">&sect;2.2 Describing Quantitative Data</a></h4></li>\n",
    "    <li><h4><a href=\"#s2.3\">&sect;2.3 Measures of Central Tendency</a></h4></li>\n",
    "    <li><h4><a href=\"#s2.4\">&sect;2.4 Measures of Variability</a></h4></li>\n",
    "    <li><h4><a href=\"#s2.5\">&sect;2.5 Measures of Relative Standing</a></h4></li>\n",
    "    <li><h4><a href=\"#s2.6\">&sect;2.6 Methods for Detecting Outliers</a></h4></li>\n",
    "    <li><h4><a href=\"#s2.7\">&sect;2.7 Distorting the Truth</a></h4></li>\n",
    "</ul>\n",
    "\n",
    "<hr style=\"border:1px solid #F0F0F0\" />\n",
    "\n",
    "<h3><a href=\"#c3\">Chapter 3: Introduction to Probability Theory</a></h3>\n",
    "<ul>\n",
    "    <li><h4><a href=\"#s3.1\">&sect;3.1 Events & Sample Spaces</a></h4></li>\n",
    "    <li><h4><a href=\"#s3.2\">&sect;3.2 Basic Set Theory</a></h4></li>\n",
    "    <li><h4><a href=\"#s3.3\">&sect;3.3 Conditional Probability</a></h4></li>\n",
    "    <li><h4><a href=\"#s3.4\">&sect;3.4 Mutual Exclusivity & Independence of Events</a></h4></li>\n",
    "    <li><h4><a href=\"#s3.5\">&sect;3.4 Additive Rule</a></h4></li>\n",
    "    <li><h4><a href=\"#s3.6\">&sect;3.6 Multiplicative Rule</a></h4></li>\n",
    "    <li><h4><a href=\"#s3.7\">&sect;3.7 Bayes' Theorem</a></h4></li>\n",
    "</ul>\n",
    "\n",
    "<hr style=\"border:1px solid #F0F0F0\" />\n",
    "\n",
    "<h3><a href=\"#c4\">Chapter 4: Random Varaibles & Common Distributions</a></h3>\n",
    "<ul>\n",
    "    <li><h4><a href=\"#s4.1\">&sect;4.1 Discrete & Continuous</a></h4></li>\n",
    "    <li><h4><a href=\"#s4.2\">&sect;4.2 Probability Distribution Function</a></h4></li>\n",
    "    <li><h4><a href=\"#s4.3\">&sect;4.3 Cumulative Distribution Function</a></h4></li>\n",
    "    <li><h4><a href=\"#s4.4\">&sect;4.4 Expectation Values</a></h4></li>\n",
    "    <li><h4><a href=\"#s4.5\">&sect;4.5 Mean, Variance, Skewness, Kurtosis</a></h4></li>\n",
    "    <li><h4><a href=\"#s4.6\">&sect;4.6 Common Discrete Distributions</a></h4></li>\n",
    "    <ul>\n",
    "        <li><h5><a href=\"#s4.6.1\">&sect;4.6.1 Uniform</a></h5>\n",
    "        <li><h5><a href=\"#s4.6.2\">&sect;4.6.2 Bernoulli</a></h5>\n",
    "        <li><h5><a href=\"#s4.6.3\">&sect;4.6.3 Binomial</a></h5>\n",
    "        <li><h5><a href=\"#s4.6.4\">&sect;4.6.4 Poisson</a></h5>\n",
    "        <li><h5><a href=\"#s4.6.5\">&sect;4.6.5 Geometric</a></h5>\n",
    "        <li><h5><a href=\"#s4.6.6\">&sect;4.6.6 Hypergeometric</a></h5>\n",
    "    </ul>\n",
    "    <li><h4><a href=\"#s4.7\">&sect;4.7 Common Continuous Distributions</a></h4></li>\n",
    "    <ul>\n",
    "        <li><h5><a href=\"#s4.7.1\">&sect;4.7.1 Uniform</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.2\">&sect;4.7.2 Normal</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.3\">&sect;4.7.3 Log-Normal</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.4\">&sect;4.7.4 Logistic</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.5\">&sect;4.7.5 Exponential</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.6\">&sect;4.7.6 Chi-Square</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.7\">&sect;4.7.7 Student's t</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.8\">&sect;4.7.8 Beta</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.9\">&sect;4.7.9 Gamma</a></h5>\n",
    "        <li><h5><a href=\"#s4.7.10\">&sect;4.7.10 Weibull</a></h5>\n",
    "    </ul>\n",
    "</ul>\n",
    "\n",
    "<hr style=\"border:1px solid #F0F0F0\" />\n",
    "\n",
    "<h3><a href=\"#c5\">Chapter 5: Other Learning Resources</a></h3>\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:2px solid black\" />\n",
    "<h1><a id=\"c0\">Preface</a></h1>\n",
    "<div style=\"padding: 5% 5% 5% 5%\">\n",
    "    <p>This lecture is a beginner's introduction to statistics and probability. It covers the basic definitions and concepts of the field. Examples will be provided to aid in understanding, however, statistical computations in Python will be reserved for Lantern's Statistical Models & Probability course.</p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:2px solid black\" />\n",
    "<h1><a id=\"c1\">Chapter 1: Introduction to Statistics</a></h1>\n",
    "<div style=\"padding: 5% 5% 5% 5%\">\n",
    "    <p>Statistics is a branch of applied mathematics that deals with and is centered around all things data. This includes data collection, organization & classification, analysis, interpretation, and presentation. Given that numerous other fields, such as physical science, medicine, economics, finance, politics, and marketing, are all data-drive, statistics plays major role in their process as well.</p>\n",
    "    <p>Statistics is typically divided into two sub-branches (though in practice both are typically applied in complete data analysis situations):</p>\n",
    "    <ul>\n",
    "        <li><b>Descriptive statistics</b> summarizes data (from a sample) and its properties through the use of numerical and graphical methods. This includes graphs, measures of central tendency and variability.</li>\n",
    "        <li><b>Inferential statistics</b> draws conclusions and makes predictions about the nature and population of a data sample. This includes hypothesis tests, correlation testing, regression analysis.</li>\n",
    "    </ul>\n",
    "    <img src=\"images/4.1_branches.jpg\" width=650 />\n",
    "    <p>Note that the term \"statistics\" is also used to refer to numbers that are used to describe data or relationships (e.g. the average is a statistic).</p>\n",
    "    <p>When working with data, statistics has any number of important uses. It helps us avoid collecting biased data, overgenalizing, interpreting results incorrectly, and allows us to clarify our process and predictions with quantitative statements. One of the most important, it allow us to discrminate between variations in data that are significant and scientifically interesting, and those variations that are just noise and reflect background hetergeneity (e.g. variations cause by spatial or temporal differences). If the variations are bigger than we'd expect by chance than we could deem the results to be statistically significant and when there is no significance we want to confirm as much.</p>\n",
    "    <p>One pitfall that students sometimes fall into when analyzing data is the feeling that they need to report a significant result and not doing so will be a failure of analysis. However there are not right and wrong results (assuming there's no fault in the analysis), we just want to know the true nature of the data even if that result is just confirming what we already know.</p>\n",
    "    <h3><a id=\"s1.1\">&sect;1.1 Basic Definitions & Concepts</a></h3><!--################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>Whenever we wish to begin studying a new subject there is new terminology involved that we must first familiarize ourselves with. Understanding new terms and their specific usage will help us when studying new concepts and trying to formulate our analysis for others comprehension.</p>\n",
    "        <p>A large portion of this lecture revolves around stating the definitions and concepts, for various sub-topics, that we will require to lay the foundation of statistics. The following are some to get us started. \n",
    "        <ul>\n",
    "            <li><b>Experimental unit</b>: an object or entity that a researcher wants to describe and make inferences about based on the sample.</li>\n",
    "            <li><b>Population</b>: a set of units that we wish to study.</li>\n",
    "            <li><b>Variable</b>: a characteristic/property of a population unit.</li>\n",
    "            <li><b>Response variable</b>: the variable whose variation is dependent on other variables and we want to understand.</li>\n",
    "            <li><b>Explanatory variables</b>: the independent variables that we manipulate in a study to explain variations in the response variable.</li>\n",
    "            <li><b>Sample</b>: a subset of the population that is chosen to respresent and explain the whole population.</li>\n",
    "            <li><b>Statistical inference</b>: an estimate/prediction/generalization about a population based on sample information.</li>\n",
    "            <li><b>Measure of reliability</b>: a statement (usually quantitative) about the degree of uncertainty associated with a statistical inference.</li>\n",
    "        </ul>\n",
    "        <img src=\"images/4.1_population_sample.png\" width=550 />\n",
    "        <p><em>Example: \"Cola-wars\" is a popular term for the intense competition between Coca-Cola and Pepsi displayed in their marketing campaigns. Suppose, as part of a Pepsi marketing campaign, 1,000 cola consumers are given a blind taste test (i.e. a taste test in which two brand names are disguised). Each consumer is asked to stat a preference for brand A or brand B.</em></p>\n",
    "        <ul>\n",
    "            <li>Our population for this example is the collection of all cola consumers.</li>\n",
    "            <li>The variable of interest (response variable) of the population is their cola preference.</li>\n",
    "            <li>The sample is the 1,000 cola consumers selected for the blind taste test (they are meant to represent the whole population).</li>\n",
    "            <li>The statistical inference that we want to make is to generalize the cola preferences of the 1,000 sample consumers to the population of all cola consumers.</li>\n",
    "            <li>The estimate from the sample will not exactly mirror the preferences of the population. Just because 56% of our sample preferred Pepsi doesn't mean that 56% of the cola population prefer Pepsi. However we can use statistical reasoning and one of several methods to ensure that the generated estimages are within a specified limit of true percentages. For example, if we estimate our sample results are within 5% of the true results (with a very high likelihood) then that 5% would be our measure of reliability and we could say the preference for Pepsi si between 51% and 61% (this is an example of a confidence interval).</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s1.2\">&sect;1.2 Types of Variables & Data</a></h3><!--##################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>When we are working with data/variables, or planning an experiment, it is important to recognize that there are different types and sub-types of data based on their range of possible values. The type of data will determine the limitations that analysis can be performed on the variable in question, and may also provide some insight in regards to the reliability of the collected data. the two main divisions of data are quantitative and qualitative.</p>\n",
    "        <p><b>Quantitative Data</b>: [aka numerical data] consists of numerical measurement values with magnitudes that can be placed in a meaningful order with consistent intervals.</p>\n",
    "        <ul>\n",
    "            <li>Has two sub-types:</li>\n",
    "            <ul>\n",
    "                <li><b>Continuous</b>: variables that can take on any value in an interval (e.g. decimals like 3.999),</li>\n",
    "                <li><b>Discrete</b>: variables that can take on only particular values of a predetermined set, usually integers (like 0, 1, 2, etc.)</li>\n",
    "            </ul>\n",
    "            <li>Has two levels of meaurement (both applicable for continuous and discrete data).</li>\n",
    "            <ul>\n",
    "                <li><b>Interval</b>: equal intervals among levels where the differences make sense but ratios don't and the zero point is arbitrary (e.g. temperature in Celcius: the difference between $10^oC$ and $20^oC$ is same as the difference between $20^oC$ and $30^oC$ but $20^oC$ isn't twice as hot as $10^oC$ and $0^oC$  is not the point of absence of heat)</li>\n",
    "                <li><b>Ratio</b>: satifies the condition of the interval level with the addition of ratios and a true zero point (e.g. volume in Litres: $0L$ of water is the absence of water and $2L$ is twice as much as $4L$ of water).</li>\n",
    "            </ul>\n",
    "            <li>Continuous data is always a type of quantitative data.</li>\n",
    "            <li>Discrete data is typically quantitative data.</li>\n",
    "            <li>Quantitative data can be grouped into numerical categories to mimic qualitative properties (i.e. for graphing and binning purposes).</li>\n",
    "            <li>Can be used with all measures of central tendency and variability.</li>\n",
    "        </ul>\n",
    "        <p><b>Qualitative Data</b>: [aka categorical data] consists of names or labels with no logical order, or with a logical order but inconsistent differences between groups measurements that can only be classified into one of a group of categories.</p>\n",
    "        <ul>\n",
    "            <li>Has two levels of measurement:</li>\n",
    "            <ul>\n",
    "                <li><b>Nominal</b>: no natural ranking or ordering of the data exists,</li>\n",
    "                <li><b>Ordinal</b>: provides an order, but can't get a precise mathematical difference between levels.</li>\n",
    "            </ul>\n",
    "            <li>May be numerical but have no order (e.g. ID #s, jersey #s).</li>\n",
    "            <li>May not be numerical but still have order (e.g. low &lt; medium &lt; high).</li>\n",
    "            <li>Can only be used with mode [of central tendency measurements].</li>\n",
    "        </ul>\n",
    "        <img src=\"images/4.1_levels_of_measurement.png\" width=650 />\n",
    "    </div>\n",
    "    <h3><a id=\"s1.3\">&sect;1.3 Assumptions & Occam's Razor</a></h3><!--###############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>As you study tests and models in statistics you will the assumptions associated with each of them. When these assumptions are violated the results of any analysis perform (which required that assumption) can be misleading or completely erroneous. Common assumptions are things like: normally distributed data, independent errors, and homoscedasticity of variance.</p>\n",
    "        <p>Almost all of the most commonly used statistical tests (not covered in this lecture) rely on the data following a specific distribution function. Such tests are called <b>parametric</b> tests. When one of the key assumptions of such a test is violated, a <b>non-parametric</b> test can be used instead, which doesn't rely on a specific probability distribution function. The advantage of using non-parametric tests is that they are more robust than parametric tests (i.e. can be used in a broader range of situations given the fewer conditions). The advantage of using parametric tests is that the extra conditions give it more statistical power leading to better rejections of null hypotheses and lower p-values (these concepts are covered in more detail in the Statistical Models & Probability course). An alternate approach to our data having an incorrect distribution function can be to transform it (e.g. transforming from log-normal data to normal data).</p>\n",
    "        <p>Lastly, I want to introduce the concept of <b>parsimony</b>, also commonly known as Occam's Razor. The concept states that if we are given a set of equally good explanations for a given phenomenon, then the correct explanation is the simplest explanation. This can be applied to statistical modelling through the ideas that:</p>\n",
    "        <ul>\n",
    "            <li>models should have as few parameters as possilbe,</li>\n",
    "            <li>linear models should be preferred to non-linear models,</li>\n",
    "            <li>experiments relying on few assumptions should be preferred to those relying on many,</li>\n",
    "            <li>models should be pared down until they are minimal adequate,</li>\n",
    "            <li>simple explanations should be preferred to complex explanations.</li>\n",
    "        </ul>\n",
    "        <p>In general, a variable should be retained in a model only if it causes significant increases in deviance when it's removed from the current model. However, we should also be careful not to oversimplify in our haste as any models we use should fit our data well and, as stated above, some complexity can help provide statistical power to our tests.</p>\n",
    "    </div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:2px solid black\" />\n",
    "<h1><a id=\"c2\">Chapter 2: Descriptive Statistics</a></h1>\n",
    "<div style=\"padding: 5% 5% 5% 5%\">\n",
    "    <p>As mentioned in the introduction, descriptive statistics is the branch which is primarily concerned with describing and quantifying our sample data. The following are some commonly used methods and statistics for describing data.</p>\n",
    "    <h3><a id=\"s2.1\">&sect;2.1 Describing Qualitative Data</a></h3><!--################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Class</b>: one of the categories into which qualitative data can be classified</li>\n",
    "            <li><b>Class frequency</b>: the number of observations in the dataset from a particular class</li>\n",
    "            <li><b>Class relative frequency</b>: the class frequency per total observations in the set $$\\large \\text{class rel. freq.} = \\frac{\\text{class freq.}}{n}$$</li>\n",
    "            <li><b>Class percentage</b>: the class relative frequency as a percentage $$\\large \\text{class %} = (\\text{class rel. freq.})\\times 100$$</li>\n",
    "        </ul>\n",
    "        <p>Graphical methods:</p>\n",
    "        <ul>\n",
    "            <li><b>Bar graphs</b>: the categories/classes are represented by bars where the height is either class frequency, class relative frequency, or class percentage</li>\n",
    "            <li><b>Pie chart</b>: the classes are represented by slices of a pie (circle). The size of each slice is proportional to the class relative frequency</li>\n",
    "            <li><b>Pareto diagram</b>: a bar graph with classes organized by descending height with a curve for the cumulative total</li>\n",
    "        </ul>\n",
    "        <p><em>Example: A pet store has an inventory of 12 puppies, 7 kittens, 31 fish, 17 rabbits, and 2 lizards.</em></p>\n",
    "        <ul>\n",
    "            <li>The classes are: puppies, kittens, fish, rabbits, and lizards.</li>\n",
    "            <li>The classes frequencies are: 12, 7, 31, 17, 2 (respectively).</li>\n",
    "            <li>The class relative frequencies are: 0.174, 0.101, 0.449, 0.246, 0.029 (respectively).</li>\n",
    "            <li>The class percentages are: 17.4%, 10.1%, 44.9%, 24.6%, 2.9% (respectively).</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s2.2\">&sect;2.2 Describing Quantitative Data</a></h3><!--################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>As mentioned in section 1.2, quantitative data can be binned to emulate qualitative data. As a result any descriptive methods for qualitative data can also be applied to quantitative data under the right circumstances. The reverse is not true however. The numerical methods outlined in the remainer of this chapters applied primarily to quanitative data.</p>\n",
    "        <p>The graphical methods for quantitative data are also extended. In addition to bar graphs and pie charts we can also use scatter plots, histograms, kernel-density-estimation (KDE) plots, cumulative frequency plots, and even higher dimensional graphs such as 3D scatter plots and curved surfaces.\n",
    "    </div>\n",
    "    <h3><a id=\"s2.3\">&sect;2.3 Measures of Central Tendency</a></h3><!--#######################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Central Tendency</b>: the tendency of the data to cluster, or center, about certain numerical values</li>\n",
    "            <li><b>Arithmetic Mean</b>: the sum of measurements divided by the number of measurements</li>\n",
    "            <li>Sample mean: $$\\Large \\bar{x} = \\sum^n_{i=1}\\frac{x_i}{n}$$</li>\n",
    "            <li>Population mean: $$\\Large \\bar{\\mu} = \\sum^N_{i=1}\\frac{x_i}{N}$$</li>\n",
    "            <li>The sample mean is used to estimate the population mean ($\\bar{x}$ is an estimator/approximator for $\\mu$)</li>\n",
    "            <li>For our estimator we'd need to know the reliability of the inference (how accurately it estimates $\\mu$). This depends on two factors:</li>\n",
    "            <ul>\n",
    "                <li>The size of the sample (law of large numbers)</li>\n",
    "                <li>The variability of the data (more variability = less accurate estimate)</li>\n",
    "            </ul>\n",
    "            <li><b>Geometric Mean</b>: the product of measurements taken to the nth root where n is the number of measurements $$\\Large \\bar{x}_{Geo} = \\sqrt[n]{\\prod_{i=1}^n x_i}$$</li>\n",
    "            <ul>\n",
    "                <li>most appropriate for series that exhibit serial correlation/autocorrelation (e.g. yields on bonds, stock returns, market risk premiums)</li>\n",
    "                <li>more accurate measurement of true return for volatile compounding numbers</li>\n",
    "            </ul>\n",
    "            <li><b>Harmonic Mean</b>: the number of measurements, $n$, divided by the sum of the inverse measurements $$\\Large \\bar{x}_{Har} = \\frac{n}{\\sum_{i=1}{n}\\frac{1}{x_i}}$$</li>\n",
    "            <ul>\n",
    "                <li>the harmonic mean is the reciprocal of the arithmetic mean of the reciprocals, i.e. $\\bar{x}_{Har} = \\large(\\frac{\\sum_{i=1}{n}{x_i^{-1}}}{n}\\large)^{-1}$</li>\n",
    "                <li>most appropriate for situations when the average of rates is desired</li>\n",
    "            </ul>\n",
    "        </ul>\n",
    "        <p><a href=\"https://machinelearningmastery.com/arithmetic-geometric-and-harmonic-means-for-machine-learning/\">https://machinelearningmastery.com/arithmetic-geometric-and-harmonic-means-for-machine-learning/</a></p>\n",
    "        <p><em>Example: Suppose you invested savings in the financial market for five years. What would be your average return if your returns per year were: 90%, 10%, 20%, 30%, -90%?</em></p>\n",
    "        <ul>\n",
    "            <li>Arithmetic mean: $(90+10+20+30-90)/5 = 12%$</li>\n",
    "            <li>12% average return is impressive but not entirely accurate. Annual investment returns are not independent, rather massive loses or gains in a year will carry forward to the next year.</li>\n",
    "            <li>Geometric mean: (the percentage need to converted to rates) $\\sqrt[5]{1.9\\times 1.1\\times 1.2\\times 1.3 \\times 0.1} = 0.799 \\rightarrow -20%$ average return.</li>\n",
    "            <li>This makes more sense since if I start with $\\$$100 over those 5 years I would have $\\$ $32.60 which is a net loss and not a profit.</li>\n",
    "        </ul>\n",
    "        <ul>\n",
    "            <li><b>Mean</b>: the center of the data (average measurement)</li>\n",
    "            <li><b>Median</b>: the middle of the data (half of measurements are above and half below)</li>\n",
    "            <ul>\n",
    "                <li>if $n$is odd, median is the middle number of ordered measurements</li>\n",
    "                <li>if $n$ is even, median is the mean of the middle two numbers of ordered measurements</li>\n",
    "            </ul>\n",
    "            <li><b>Mode</b>: the location of most concentration of the data (measurement that occurs most frequently)</li>\n",
    "            <ul>\n",
    "                <li>not typically the most useful metric; mean and median are favoured</li>\n",
    "                <li>modal class is the measurement class with the largest relative frequency and may be a better metric in class based data</li>\n",
    "                <li>the mode is located in the modal class</li>\n",
    "                <li>can be used for non-numerical data (unlike mean and median)</li>\n",
    "                <li>data can be bimodal (or multi-modal)</li>\n",
    "                <li>mode is useful for discrete values (e.g. the average number of legs people have is less than two but the mode is two)</li>\n",
    "                <li>the number of customers per hour of day might have two peaks (lunch time, 12pm, and dinner time, 5pm) then the mean would be inaccurate for the busiest time (e.g. 2:30pm) and mode would be a better measure</li>\n",
    "            </ul>\n",
    "            <li><b>Skewed</b>: when one side of a distribution has more extreme observations than the other</li>\n",
    "        </ul>\n",
    "        <p>Detecting Skewness by Comparing the Mean and Median</p>\n",
    "        <img src=\"images/4.2_skewness.png\" width=750 />\n",
    "        <ul>\n",
    "            <li>if Mean &lt; Median, then leftward skewed (negative skew)</li>\n",
    "            <li>if Mean &gt; Median, then rightward skewed (positive skew)</li>\n",
    "            <li>if Mean = Median, then symmetrical</li>\n",
    "        </ul>\n",
    "        <p>Mean vs Median vs Mode</p>\n",
    "        <ul>\n",
    "            <li>median is less sensitive to extremely large/small measurements than mean</li>\n",
    "            <li>mode is the highest point of the distribution</li>\n",
    "            <li>median is the middle point of the distribution</li>\n",
    "        </ul>\n",
    "        <p>Note: parameters are numbers that summarize population data (e.g. $\\mu, \\sigma$) and statistics are numbers that summarize sample data and estimate the population (e.g. $\\bar x, s $)</p>\n",
    "        <p><b>Bimodal Data</b></p>\n",
    "        <ul>\n",
    "            <li>example of multimodal data which has many values that are similarly common</li>\n",
    "            <li>typically results from two or more underlying groups all being measured together</li>\n",
    "            <li>e.g. two peaks of busiest business hours: lunchtime customers and dinnertime customers</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s2.4\">&sect;2.4 Measure of Variability</a></h3><!--###############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Variability</b>: the spread of the data (departures from the mean)</li>\n",
    "            <ul>\n",
    "                <li>the variability with central tendency helps us visualize the shape of the data and its extreme values</li>\n",
    "            </ul>\n",
    "            <li><b>Range</b>: the difference of the largest and smallest measurements</li>\n",
    "            <li><b>Variance</b>: the sum of the squared distances from [measurements to] the mean, divided by the number of measurements</li>\n",
    "            <li>Sample variance: $$\\Large s^2 = \\frac{1}{n-1}\\sum^n_{i=1}(x_i-\\bar{x})^2$$</li>\n",
    "            <li>Sample standard deviation: $$\\Large s = \\sqrt{s^2}$$</li>\n",
    "            <li>Population variance: $$\\Large \\sigma^2 = \\frac{1}{N}\\sum^N_{i=1}(x_i-\\mu)^2$$</li>\n",
    "            <li>Sample standard deviation: $$\\Large \\sigma = \\sqrt{\\sigma^2}$$</li>\n",
    "            <ul>\n",
    "                <li>Note: standard deviation is expressed in the original $\\text{units}$ (variance is $\\text{units}^2$)</li>\n",
    "                <li>Sample variance uses a denominator of $n-1$ due to the bias that arises otherwise (https://en.wikipedia.org/wiki/Bias_of_an_estimator#Sample_variance)</li>\n",
    "                <li>this can also be considered coming from the degrees of freedom [see below] (i.e. we use one degree of freedom to calculate the mean and are left with $n-1$ when we calculate the variance).</li>\n",
    "            </ul>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s2.5\">&sect;2.5 Measure of Relative Standing</a></h3><!--###############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Measures of Relative Standing</b>: descriptive measurements of the relationship of a measurement to the rest of the data</li>\n",
    "            <li><b>pth Percentile</b>: [for any set of n measurements in order] a number such that $p\\%$ of the measurements fall below that number and $(100-p)\\%$ fall above</li>\n",
    "            <ul><li>E.g. You scored an 80 on a test and want to know how you fared in comparison to the rest of the class. If your teacher tells you that you scored in the 90th percentile it means that 90% of the grades were lower than yours and 10% were higher.</li></ul>\n",
    "            <li><b>z-Score</b>: represents the distance between a given measurement $x$ and the mean, expressed in standard deviations</li>\n",
    "            <li><b>Sample z-Score</b> for a measurement $x$ is $$\\Large z = \\frac{x-\\bar{x}}{s}$$</li>\n",
    "            <li><b>Population z-Score</b> for a measurement $x$ is $$\\Large z = \\frac{x-\\mu}{\\sigma}$$</li>\n",
    "        </ul>\n",
    "        <p>Interpretations of z-Scores for Mound-Shaped Distributions of Data</p>\n",
    "        <ol>\n",
    "            <li>Approx. 68% of the measurements will have a z-score between -1 and 1</li>\n",
    "            <li>Approx. 95% of the measurements will have a z-score between -2 and 2</li>\n",
    "            <li>Approx. 99.7% of the measurements will have a z-score between -3 and 3</li>\n",
    "        </ol>\n",
    "        <img src=\"images/4.2_standard_normal.png\" width=750 />\n",
    "    </div>\n",
    "    <h3><a id=\"s2.6\">&sect;2.6 Methods for Detecting Outliers</a></h3><!--#############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Outlier</b>: an observation that is unusually large or small relative to the data values we want to describe. Often caused by one of several things:</li>\n",
    "            <ul>\n",
    "                <li>invalid measurement (e.g. malfunctioned measuring device; misrecorded measurement; computer encoding error)</li>\n",
    "                <li>misclassified measurement (i.e. measurement belongs to a different population than the rest of our sample)</li>\n",
    "                <li>rare event (i.e. measurement is recorded properly and belongs to same population but distribution is extremely skewed)</li>\n",
    "            </ul>\n",
    "            <li>Two useful methods for detecting outliers are boxplots (graphical) and z-scores (numerical)</li>\n",
    "            <li><b>Lower Quartile</b> $Q_L$: the 25th percentile of the dataset</li>\n",
    "            <li><b>Median</b> $M$: the 50th percentile of the dataset</li>\n",
    "            <li><b>Upper Quartile</b> $Q_U$: the 75th percentile of the dataset</li>\n",
    "            <li><b>Interquartile Range</b>: the distance between the lower and upper quartiles $$\\Large \\text{IQR}=Q_U-Q_L$$</li>\n",
    "        </ul>\n",
    "        <p>Elements of a Box Plot</p>\n",
    "        <ol>\n",
    "            <li>A rectangle (*box*) with ends (*hinges*) drawn at the lower and upper quartile ($Q_L$ and $Q_U$). The median ($M$) shown in the box by a line, and the mean ($\\bar{x}$) shown by a marker.</li>\n",
    "            <li>The points at distances 1.5 times the $IQR$ from each hinge mark the *inner fences* of the dataset. Lines (*whiskers*) are drawn from each hinge to the most extreme measurement inside the inner fence.</li>\n",
    "            <ul>\n",
    "                <li>Lower inner fence = $Q_L$ - $1.5\\times\\text{(IQR)}$</li>\n",
    "                <li>Upper inner fence = $Q_U$ + $1.5\\times\\text{(IQR)}$</li>\n",
    "            </ul>\n",
    "            <li>A second pair of fences (*outer fences*) appear at a distance of $3\\times\\text{(IQR)}$ from the hinges.</li>\n",
    "            <ul>\n",
    "                <li>Typically, one marker type is used for measurements between inner and outer fences and another type is used for measurments beyond the outer fences</li>\n",
    "                <li>Lower outer fence = $Q_L$ - $3\\times\\text{(IQR)}$</li>\n",
    "                <li>Upper outer fence = $Q_U$ + $3\\times\\text{(IQR)}$</li>\n",
    "            </ul>\n",
    "        </ol>\n",
    "        <img src=\"images/4.2_boxplot.jpg\" width=550 />\n",
    "        <p>Aids to the Interpretation of Box Plots</p>\n",
    "        <ol>\n",
    "            <li>Examine the length of the box.</li>\n",
    "            <ul>\n",
    "                <li>IQR is a measure of the sample's variability and is especially useful for the comparison of two samples</li>\n",
    "            </ul>\n",
    "            <li>Visually compare the lengths of the whiskers.</li>\n",
    "            <ul>\n",
    "                <li>If one is clearly longer, the distribution of the data is probably skewed in the direction of the longer whisker</li>\n",
    "            </ul>\n",
    "            <li>Analyze any measurements that lie beyond the fences.</li>\n",
    "            <ul>\n",
    "                <li>Less than 5% should fall beyond the inner fences, even for very skewed distributions</li>\n",
    "                <li>Measurements beyond the outer fences are probably outliers, with one of the following explanations:</li>\n",
    "                <ol>\n",
    "                    <li>The measurement is incorrect</li>\n",
    "                    <li>The measurement belongs to a population different from the population that the rest of the sample was drawn from</li>\n",
    "                    <li>The measurement is correct and from the same population (generally accepted as the explanation only after carefully ruling out all others)</li>\n",
    "                </ol>\n",
    "            </ul>\n",
    "        </ol>\n",
    "        <p>Rules of Thumb for Detecting Outliers'</p>\n",
    "        <ul>\n",
    "            <li>Box Plots: Observations falling between the inner and outer fences are deemed *suspect outliers*. Observations falling beyond the outer fence are deemed *highly suspect outliers*.</li>\n",
    "            <li>z-Scores: Observations with z-scores greater than 3 in absolute value are considered *outliers*. For some highly skewed datasets, observations with z-socres greater than 2 in absolute value *may be outliers*.</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s2.7\">&sect;2.7 Distorting the Truth</a></h3><!--###############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>Misleading statistics occur when a fault, whether deliberate or not, is presented in one of the key aspects of research:</p>\n",
    "        <ol>\n",
    "            <li>Collecting data: using small sample sizes that project big numbers but have little statistical significance.</li>\n",
    "            <li>Organizing: omitting findings that contradict the point the researcher is trying to prove.</li>\n",
    "            <li>Presenting: manipulating visual/numerical data to influence perception.</li>\n",
    "        </ol>\n",
    "        <img src=\"images/4.2_misleading_stats.jpeg\" width=650 />\n",
    "        <p>Generating Data:</p>\n",
    "        <ul>\n",
    "            <li>Bad/Biased Sampling: working with data that isn't randomly sampled from the full population (e.g. asking homeowners about the cost of living in a city, but not asking renters)</li>\n",
    "            <li>Leading/Loaded Questions: asking survey questions that assume a position or uses certain language to lead a response (e.g. What do you like about X? vs. What do you think about X?)</li>\n",
    "        </ul>\n",
    "        <p>Processing Data:</p>\n",
    "        <ul>\n",
    "            <li>Discarding Unfavourable Data: only using a subset of the data that agrees with the point your trying to make (e.g. reducing the range of your dataset)</li>\n",
    "            <li>Data Dredging/p-Hacking: analysing your data without a clearly defined focus & hypothesis; instead testing a wide range of hypothesis until a significant result is found which will likely be a false positive</li>\n",
    "        </ul>\n",
    "        <p>Presenting Data:</p>\n",
    "        <ul>\n",
    "            <li>Graphing: truncating axes or altering the scale, omitting data points, inapropriately scaling size (e.g. increasing width proportional to height in a bar graph)</li>\n",
    "            <li>Conclusions: misinterpreting the results/casuality (e.g. presenting correlation as causation, misinterpreting survivorship bias)</li>\n",
    "            <li>Hiding Context: omitting information that may give context, omitting measures of variability, obscuring details with excess data</li>\n",
    "            <li>Language: using specific language and word order (e.g. the average person has two legs (not entirely incorrect) vs. the average number of legs per person is two [on average, people have two legs] (incorrect)</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:2px solid black\" />\n",
    "<h1><a id=\"c3\">Chapter 3: Introduction to Probability Theory</a></h1>\n",
    "<div style=\"padding: 5% 5% 5% 5%\">\n",
    "    <p>Probability is a branch of mathematics dealing with predictions and likelihood of future events. For that reason, probability plays a big role in the area of inferential statistics and making preditions from sample data. The following are some basic definitions and formulations that are used throughout probability and statistics.</p>\n",
    "    <h3><a id=\"s3.1\">&sect;3.1 Events & Sample Spaces</a></h3><!--################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Experiment</b>: an act or process of observation that leads to a single outcome that cannot be predicted with certainty.</li>\n",
    "            <li><b>Sample Point</b> of <b>Simple Event</b>: the most basic outcome of an experiment (which cannot be decomposed into yet simpler outcomes).</li>\n",
    "            <li><b>Sample Space</b>: the collection of all of an experiment's sample points</li>\n",
    "            <li><b>Probability Rules for Sample Points</b>: Let $p_i$ represent the probability of sample point $i$. Then</li>\n",
    "            <ol><li>All sample point probabilities must lie between 0 and 1 (i.e. $0 \\leq p_i \\leq 1$)</li>\n",
    "                <li>The probabilities of all the sample points within a sample space must sum to 1 (i.e. $\\sum p_i = 1$)</li>\n",
    "            </ol>\n",
    "            <li><b>Event</b>: a specific collection of sample points (simple events)</li>\n",
    "            <li><b>Probability of an Event</b>: is calculated by summing the probabilities of the sample points in the sample space for the event</li>\n",
    "            <ol><li>Define the experiment (describe the process used to make an observation and the type of observation that will be recorded)</li>\n",
    "                <li>List the sample points</li>\n",
    "                <li>Assign Probabilities to the sample points</li>\n",
    "                <li>Determine the collection of sample points contained in the event of interest</li>\n",
    "                <li>Sum the sample point probabilities to get the probability of the event</li>\n",
    "            </ol>\n",
    "        </ul>\n",
    "        <p><em>Example: Experiment of two tosses of a fair coin:</em></p>\n",
    "        <ul><li>Suppose our event $A$ is that at least one heads and one tails (irrespective of sequence) show up</li>\n",
    "            <li>Sample points: heads ($H$) and tails ($T$) in combinations of 2</li>\n",
    "            <li>Sample space: {$HH, HT, TH, TT$}</li>\n",
    "            <li>Sample point probabilities: $P(HH) = 1/4$, $P(HT) = 1/4$, $P(TH) = 1/4$, $P(TT) = 1/4$</li>\n",
    "            <li>Sample points that lead to event $A$: {$HT, TH$}</li>\n",
    "            <li>Probability of event $A$: $P(A) = P(HT) + P(TH) = 1/4 + 1/4 = 1/2$</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s3.2\">&sect;3.2 Basic Set Theory</a></h3><!--################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>When working with probabilities and experiement events we are often concerned with sets of points (e.g. sample space or events) and the combinations/relations between these sets. The following concepts are the fundamentals of basic set theories which are used to formulate probability equations for events and combination of events.</p>\n",
    "        <ul>\n",
    "            <li><b>Compound Events</b>: events that can be viewed as a composition of two or more other events</li>\n",
    "            <li><b>Union</b>: of two sets/events A and B is the set/event that occurs if either A or B or both occur on a single performance of the experiment. Denoted $A\\cup B$</li>\n",
    "            <img src=\"images/4.3_union.png\" />\n",
    "            <li><b>Intersection</b>: of two sets/events A and B is the set/event that occurs if both A and B occur on a single performance of the experiment. Denoted $A\\cap B$</li>\n",
    "            <img src=\"images/4.3_intersection.png\" />\n",
    "            <li><b>Difference of Sets</b>: of two sets/events A and B is the set/event that occurs in one set but not the other. Denoted $B\\setminus A$</li>\n",
    "            <img src=\"images/4.3_difference.png\" />\n",
    "            <li><b>Complement of a Set</b>: is the collection of points outside of the set, i.e. the event consisting of all sample points that are not in the given event. Denoted $A^c$</li>\n",
    "            <li><b>Rule of Complements</b>: $P(A) + P(A^c) = 1$</li>\n",
    "            <img src=\"images/4.3_complement.png\" />\n",
    "        </ul>\n",
    "        <p><em>Example: Consider a fair dice toss experiment and suppose we define the following events $A$ and $B$</em></p>\n",
    "        <ul><li>$A = \\{\\text{Roll an even number}\\} = \\{2,4,6\\}$</li>\n",
    "            <li>$B = \\{\\text{Roll a number less than 4}\\} = \\{1,2,3\\}$</li></ul>\n",
    "        <p><em>What is the union? Intersection? Differences? Complements?</em></p>\n",
    "        <ul>\n",
    "            <li>Union (points in either $A$ or $B$): $A \\cup B = \\{1,2,3,4,6\\}$</li>\n",
    "            <li>Intersection (points in both $A$ and $B$): $A \\cap B = \\{2\\}$</li>\n",
    "            <li>Set difference of $A$ and $B$ (points in $A$ but not $B$): $A \\setminus B = \\{4,6\\}$</li>\n",
    "            <li>Set difference of $B$ and $A$ (points in $B$ but not $A$): $B \\setminus A = \\{1,3\\}$</li>\n",
    "            <li>Complement of $A$ (points not in $A$): $A^c = \\{1,3,5\\}$</li>\n",
    "            <li>Complement of $B$ (points not in $B$): $B^c = \\{4,5,6\\}$</li>\n",
    "            <li>$(A \\cup B)^c = A^c \\cap B^c = \\{5\\}$</li>\n",
    "            <li>$(A \\cap B)^c = A^c \\cup B^c = \\{1,3,4,5,6\\}$</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s3.3\">&sect;3.3 Conditional Probability</a></h3><!--###########################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Conditional Probability</b>: when we have additional knowledge that might affect the outcome of an experiment, so we may need to alter the probability of an event of interest.</li>\n",
    "            <li>To find the conditional probability that event A occurs given that event B has occured (or will 100% occur), divide the probability that both occur by the probability that B occurs (i.e. that ratio of the intersection and B as a whole) $$\\large P(A|B) = \\frac{P(A \\cap B)}{P(B)}$$</li>\n",
    "        </ul>\n",
    "        <img src=\"images/4.3_conditional_prob.png\" width=550 />\n",
    "    </div>\n",
    "    <h3><a id=\"s3.4\">&sect;3.4 Mutual Exclusivity & Independence of Events</a></h3><!--#################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Mutually Exclusive</b>: events A and B are mutually exclusive if $A \\cap B$ contains no sample points, i.e. A and B have no sample points in common $$\\large P(A \\cap B) = 0$$</li>\n",
    "            <li><b>Independent Events</b>: A and B are independent events if the occurance one does not alter the probability of the other occuring, i.e. $$\\large P(A|B) = P(A)$$ and $$\\large P(B|A) = P(B)$$</li>\n",
    "            <ul><li>Probability of Intersection of Two Independent Events: A and B are independent if and only if the probability of the intersection is equal to the product of probabilities of A and B $$\\large P(A\\cap B) = P(A)P(B)$$</li></ul>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s3.5\">&sect;3.5 Additive Rule</a></h3><!--###############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p><b>Additive Rule of Probability</b>: the probability of the union of events A and B is the sum of the probability of events A and the probability of event B, minus the probability of the intersection of events A and B (to account for double counting) $$\\large P(A \\cup B) = P(A) + P(B) - P(A \\cap B)$$</p>\n",
    "        <ul><li>If A and B are mutually exclusive then $$\\large P(A \\cup B) = P(A) + P(B)$$</li></ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s3.6\">&sect;3.6 Multiplicative Rule</a></h3><!--#############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p><b>Mulitplication Rule of Probability</b>: the probability that both event A and event B occur is equal to the probability that A occurs times the probability that B occurs, given that A has occurred (conditional probability), and vice versa. I.e. $$\\large P(A \\cap B) = P(A)P(B|A) = P(B)P(A|B)$$</p>\n",
    "        <ul><li>If events A and B are independent then $$\\large P(A \\cap B) = P(A)P(B)$$</li></ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s3.7\">&sect;3.7 Bayes' Theorem</a></h3><!--###############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p><b>Bayes' Rule</b> is a way of finding a conditional probability when we know certain other probabilities, i.e. $$\\large P(B|A) = \\frac{P(A|B)P(B)}{P(A)}$$</p>\n",
    "        <p><em>Example: You're planning a picnic, but it's a cloudy morning. You know that 50% of all rainy days start off cloudy, 40% of mornings are cloudy, and the chance of rain this month is around 10%. What is the chance of rain today?</em></p>\n",
    "        <ul>\n",
    "            <li>Probability of rain this month: $P(\\text{rain}) = 0.1$\n",
    "            <li>Probability of a cloudy morning: $P(\\text{cloundy}) = 0.4$\n",
    "            <li>Probability of a rainy day starting cloudy: $P(\\text{cloudy|rain}) = 0.5$\n",
    "            <li>Probability of today (a cloudy day) will be rainy: $$ P(\\text{rain|cloudy}) = \\frac{P(\\text{rain}) \\times P(\\text{cloudy|rain})}{P(\\text{cloudy})} = \\frac{0.1\\times 0.5}{0.4} = 0.125$$</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:2px solid black\" />\n",
    "<h1><a id=\"c4\">Chapter 4: Random Variables & Common Distributions</a></h1>\n",
    "<div style=\"padding: 5% 5% 5% 5%\">\n",
    "    <p>When you study a sample dataset as a precursor to predictive models, you will want to also determine the distribution of the original population that the data was sampled from. Fitting a few relevant distributions to your sample and determining the best fitting ones using measures of fit will likely be your course of action (that process is outside of the scope of this lecture).</p>\n",
    "    <p>Before we fit any distributions it's important that we understand so more terminology, properties, and familiarize ourselves with a handful of commonly used distributions.</p>\n",
    "    <ul>\n",
    "        <li><b>Random Variable</b>: a variable that assumes numerical values associated with the random outcomes of an experiment, where one (and only one) numerical value is assigned to each sample point.</li>\n",
    "        <li>Random variables are effectively the same as the response and explanatory variables mentioned above, however, random variables are purely numerical. This means if we wish to work with qualitative data we must first appropriately transform the data into a usable numerical scale. Furthermore, there are really only two distributions that are compatible with qualitative data: binomial and poisson.</li>\n",
    "    </ul>\n",
    "    <img src=\"images/4.4_random_variable.svg\" width=350 />\n",
    "    <h3><a id=\"s4.1\">&sect;4.1 Discrete & Continuous</a></h3><!--################################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p><b>Discrete</b>: random variables that can assume a countable number of values</p>\n",
    "        <p><b>Continuous</b>: random varialbes that can assume values corresponding to any of the points contained in an interval</p>\n",
    "        <img src=\"images/4.4_discrete_vs_cont.png\" width=550 />\n",
    "    </div>\n",
    "    <h3><a id=\"s4.2\">&sect;4.2 Probability Distribution Function</a></h3><!--#########################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>When we talk about a distribution of a random variable or sample data what we're really talking about is a probability distribution. When we plot the distribution of occurances of specific values (or value ranges called bins) we end up with a graph that maps out the likelihoods, or probabilities, of any random data point belonging to the speicific value or bin. I.e. the probability distribution is a graph, table, or formula that specifies the probability associated with each possible value that a random variable can assume.</p>\n",
    "        <p>Given the different mathematical nature of discrete and continuous variables, the formulations of functions and properties are slightly different. For continuous random variables we use the continuous equivalance such as integrals instead of summations. Additionally, the concepts discussed in this chapter refer to the theoretical distributions whereas in practice we will be working with datasets that are discretizations of continuous variables. For the concepts herein I will denote which type that the formulation is for.</p>\n",
    "        <p><b>Probability mass function</b> [discrete]: for discrete random variable $X$ we define the probability mass function, or $\\text{pmf}$, as $$\\large p_X(x) = P(X=x)$$</p>\n",
    "        <ul><li>Requirements for the $\\text{PMF}$ to be a valid distribution of discrete $X$:</li>\n",
    "            <ol>\n",
    "                <li>$0 \\leq p_X(x) \\leq 1$ for all values of $x$</li>\n",
    "                <li>$\\sum_x p_X(x) = 1$</li>\n",
    "            </ol>\n",
    "        </ul>\n",
    "        <p><b>Probability density function</b> [continuous]: for continuous random variable $X$ we define the probability density function, or $\\text{pdf}$, to be the continuous and smooth curve $$\\large f_X(x)$$</p>\n",
    "        <ul><li>Requirements for the $\\text{PDF}$ to be a valid distribution of continuous $X$:</li>\n",
    "            <ol>\n",
    "                <li>$f_X(x) \\geq 0$ for all $x$</li>\n",
    "                <li>$\\int_\\Omega f_X(x)dx = 1$ where $\\Omega$ is the range of $x$ values that $f_X(x)$ is defined for.</li>\n",
    "            </ol>\n",
    "            <li>Furthermore, to calculate the probability of an event for a continuous variable we must integrate over the range of values that represent the event. E.g. the probability of $X$ taking on a value in the interval $(a,b)$ is given by $$P(X\\in (a,b)) = \\int_a^b f_X(x)dx$$ and for a singluar point the probability goes to zero (because there's no weight associated with the singular point), i.e. $$P(X=x) = 0$$ for all $x$.</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s4.3\">&sect;4.3 Cumulative Distribution Function</a></h3><!--####################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p><b>Cumulative Distribution function</b> [discrete]: is defined as the cumlination of probabilities up to some point $X=x$ given by $$\\large F_X(x) = \\sum_{y \\leq x} p_X(y)$$</p>\n",
    "        <ul>\n",
    "            <li>Properties of $\\text{CDF}$:</li>\n",
    "            <ul>\n",
    "                <li>$F_X(x) \\leq F_X(y)$ if $x \\leq y$</li>\n",
    "                <li>$\\lim_{x\\to -\\infty} F_X(x) = 0$</li>\n",
    "                <li>$\\lim_{x\\to\\infty} F_X(x) = 1$</li>\n",
    "            </ul>\n",
    "        </ul>\n",
    "        <p><b>Culumative Distribution function</b> [continuous]: is a non-decreasing differentiable function $F_X$ defined on the range of possible values $x\\in (a,b)$ such that $$\\large F_X(x) = \\int_a^xf_X(x)dx$$</p>\n",
    "        <ul>\n",
    "            <li>Properties of $\\text{CDF}$:</li>\n",
    "            <ul>\n",
    "                <li>$F_X(x) \\leq F_X(y)$ if $x \\leq y$</li>\n",
    "                <li>$\\lim_{x\\to a} F_X(x) = 0$</li>\n",
    "                <li>$\\lim_{x\\to b} F_X(x) = 1$</li>\n",
    "                <li>$\\frac{d}{dx}F_X(x) = f_X(x)$</li>\n",
    "            </ul>\n",
    "            <li>The $\\text{CDF}$ is useful for computing the probabilities of ranges of values</li>\n",
    "            <li>$P(X < c) = \\int_a^c f_X(x)dx = F_X(c)$</li>\n",
    "            <li>$P(X > c) = \\int_c^b f_X(x)dx = 1 - F_X(c)$</li>\n",
    "            <li>$P(X\\in (c,d)) = \\int^d_cf_X(x)dx = F_X(d) - F_X(c)$</li>\n",
    "        </ul>\n",
    "        <img src=\"images/4.4_pmf_cdf.gif\" width=550 />\n",
    "    </div>\n",
    "    <h3><a id=\"s4.4\">&sect;4.4 Expectation Values</a></h3><!--#################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <p>The expected value of a random variable $X$ is the weighted average of values of the variable. This simplifies to the arithmetic mean in the case where all values of $X$ have equal probabilities. Intuitively the expected value can be thought of the value we most expect to get when sampling a point from a distribution (i.e. the centre of the distribution).</p> \n",
    "        <p><b>Expectation Operator</b> [discrete]: $$\\large E[X] = \\sum_x xp(x)$$ and more generally, $$\\large E[g(X)] = \\sum_x g(x)p_X(x)$$</p>\n",
    "        <p><b>Expectation Operator</b> [continuous]: $$\\large E[X] = \\int_\\Omega x f_X(x)dx$$ and more generally, $$\\large E[g(X)] = \\int_\\Omega g(x)f_X(x)dx$$</p>\n",
    "        <ul>\n",
    "            <li>The expectaion operator obeys the associative properties:</li>\n",
    "            <ul>\n",
    "                <li>For random variables $X$ and $Y$ we have $E[X + Y] = E[X] + E[Y]$</li>\n",
    "                <li>For random variable $X$ and constants $a$ and $b$ we have $E[aX + b] = aE[X] + b$</li>\n",
    "            </ul>\n",
    "            <li>The <b>$n$-th moment</b> of random variable $X$ refers to the expectation where $g(X)=X^n$, namely $$E[X^n] = \\sum_x x^np_X(x)$$ for discrete $X$ and $$E[X^n] = \\int_\\Omega x^nf_X(x)dx$$ for continuous $X$.</li>\n",
    "            <li>The <b>$n$-th moment about the mean</b> of random variable $X$, typically called the <b>$n$-th central moment</b>, refers to the expectation where the moment is centred about the mean, i.e. $g(X) = (X-\\mu)^n$, namely $$E[(X-\\mu)^n] = \\sum_x (x-\\mu)^np_X(x)$$ for discrete $X$ and $$E[(X-\\mu)^n]= \\int_\\Omega (x-\\mu)^nf_X(x)dx$$ for continuous $X$.</li>\n",
    "        </ul>\n",
    "    </div>\n",
    "    <h3><a id=\"s4.5\">&sect;4.5 Mean, Variance, Skew, Kurtosis</a></h3><!--#############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <ul>\n",
    "            <li><b>Mean</b> or <b>Expected Value</b>: of the population is given by the first moment, namely $$\\large E[X] = \\mu = \\sum_xxp_X(x)$$ $$\\large E[X] = \\mu = \\int_\\Omega xf_X(x)dx$$ for $X$ discrete and continuous respectively.</li>\n",
    "            <li><b>Variance</b>: of the population is given by the second central moment, namely $$\\large E[(X-\\mu)^2] = Var[X] = \\sigma^2 = \\sum_x(x-\\mu)^2p_X(x)$$ $$\\large E[(X-\\mu)^2] = Var[X] = \\sigma^2 = \\int_\\Omega (x-\\mu)^2f_X(x)dx$$ for $X$ discrete and continuous respectively.</li>\n",
    "            <li><b>Skewness</b>: of the population is given by the third standardized  moment, namely $$\\large E\\Big[\\Big(\\frac{X-\\mu}{\\sigma}\\Big)^3\\Big] = \\sum_x\\Big(\\frac{x-\\mu}{\\sigma}\\Big)^3p_X(x) = \\frac{\\mu_3}{\\sigma^3}$$ $$E\\Big[\\Big(\\frac{X-\\mu}{\\sigma}\\Big)^3\\Big] = \\int_\\Omega \\Big(\\frac{x-\\mu}{\\sigma}\\Big)^3f_X(x)dx = \\frac{\\mu_3}{\\sigma^3}$$ for $X$ discrete and continuous respectively.</li>\n",
    "            <li><b>Kurtosis</b>: of the population is given by the fourth standardized  moment, namely $$E\\Big[\\Big(\\frac{X-\\mu}{\\sigma}\\Big)^4\\Big] = \\sum_x\\Big(\\frac{x-\\mu}{\\sigma}\\Big)^4p_X(x) = \\frac{\\mu_4}{\\sigma^4}$$ $$E\\Big[\\Big(\\frac{X-\\mu}{\\sigma}\\Big)^4\\Big] = \\int_\\Omega \\Big(\\frac{x-\\mu}{\\sigma}\\Big)^4f_X(x)dx = \\frac{\\mu_4}{\\sigma^4}$$ for $X$ discrete and continuous respectively.</li>\n",
    "            <li><b>Excess Kurtosis</b>: is defined as three less than the kurtosis, namely $$\\frac{\\mu_4}{\\sigma^4} - 3$$ This is done since the noraml distribution has a kurtosis of $3$ and provides a baseline for comparing the kurtosis values with it.</li>\n",
    "            <ul>\n",
    "                <li><b>Mesokurtic</b> is used to describe a distribution with zero excess kurtosis (i.e. matching the normal distribution).</li>\n",
    "                <li><b>Leptokurtic</b> is used to describe distributions with positive excess kurtosis which corresponds with fatter tails.</li>\n",
    "                <li><b>Platykurtic</b> is used to describe distributions with negative excess kurtosis which corresponds with thinner tails.\n",
    "        </ul>\n",
    "        <img src=\"images/4.4_kurtosis.png\" width=650 />\n",
    "    </div>\n",
    "    <h3><a id=\"s4.6\">&sect;4.6 Common Discrete Distributions</a></h3><!--##########################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <h5><a id=\"s4.6.1\">&sect;4.6.1 Uniform</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\text{unif}(a,b)$</li>\n",
    "                <li>Description: A symmetric distribution wherein a finite number of values are equally likely to be observed.</li>\n",
    "                <li>PMF: $p_X(x) = \\frac{1}{n}$</li>\n",
    "                <li>Mean: $\\mu = \\frac{a+b}{2}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{(b-a+1)^2-1}{12}$</li>\n",
    "                <li>Skewness: $0$</li>\n",
    "                <li>Excess Kurtosis: $-\\frac{6(n^2+1)}{5(n^2-1)}$</li>\n",
    "                <li>Support: $x\\in {a,a+1,...,b-1,b}$</li>\n",
    "                <li>Applications: used when all outcomes are equally probable</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_uniform_discrete.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.6.2\">&sect;4.6.2 Bernoulli</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $B(1,p)$</li>\n",
    "                <li>Description: A binary probability distribution of a random variable which takes the value 1 with probability $p$ and the value 0 with probability $q=1-p$.</li>\n",
    "                <li>PMF: $p_X(x) = p^x(1-p)^{1-x}$</li>\n",
    "                <li>Mean: $\\mu = p$</li>\n",
    "                <li>Variance: $\\sigma^2 = p(1-p)$</li>\n",
    "                <li>Skewness: $\\frac{1-2p}{\\sqrt{p(1-p)}}$</li>\n",
    "                <li>Excess Kurtosis: $\\frac{1-6p(1-p)}{p(1-p)}$</li>\n",
    "                <li>Support: $x\\in {0,1}$</li>\n",
    "                <li>Applications: modelling probability of event occurring or not occurring</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_bernoulli.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.6.3\">&sect;4.6.3 Binomial</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{B}(n,p)$</li>\n",
    "                <li>Description: The probability distribution of the number of successes in a sequence of $n$ independent experiments, each asking a yes-no question with probabilities $p$ and $1-p$ respectively.</li>\n",
    "                <li>PMF: $p_X(x) = \\binom{n}{x}p^x(1-p)^{n-x}$</li>\n",
    "                <li>Mean: $\\mu = np$</li>\n",
    "                <li>Variance: $\\sigma^2 = np(1-p)$</li>\n",
    "                <li>Skewness: $\\frac{1-2p}{\\sqrt{np(1-p)}}$</li>\n",
    "                <li>Excess Kurtosis: $\\frac{1-6p(1-p)}{np(1-p)}$</li>\n",
    "                <li>Support: $x\\in {0,1,...,n}$</li>\n",
    "                <li>Applications: risk analysis</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_binomial.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.6.4\">&sect;4.6.4 Poisson</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Pois}(\\lambda)$</li>\n",
    "                <li>Description: A probability distribution expressing the probability of a given number of events occurring in a fixed interval of time or space, if these events occur with a known constant mean rate (and independently of the time since the last event).</li>\n",
    "                <li>PMF: $p_X(x) = \\frac{\\lambda^x e^{-\\lambda}}{x!}$</li>\n",
    "                <li>Mean: $\\mu = \\lambda$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\lambda$</li>\n",
    "                <li>Skewness: $\\lambda^{-1/2}$</li>\n",
    "                <li>Excess Kurtosis: $\\lambda^{-1}$</li>\n",
    "                <li>Support: $x\\in {0,1,2,...}$</li>\n",
    "                <li>Applications: analyzing amount of variation from average number of occurances</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_poisson.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.6.5\">&sect;4.6.5 Geometric</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Geo}(p)$</li>\n",
    "                <li>Description: The probability distribution of the number of Bernoulli trials needed to get one success.</li>\n",
    "                <li>PMF: $p_X(x) = (1-p)^{x-1}p$</li>\n",
    "                <li>Mean: $\\mu = \\frac{1}{p}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{1-p}{p^2}$</li>\n",
    "                <li>Skewness: $\\frac{2-p}{\\sqrt{1-p}}$</li>\n",
    "                <li>Excess Kurtosis: $6+\\frac{p^2}{1-p}$</li>\n",
    "                <li>Support: $x \\in {1,2,3,...}$</li>\n",
    "                <li>Applications: modelling populations, return on investment research, econometrics</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_geometric.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.6.6\">&sect;4.6.6 Hypergeometric</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Hypergeo}(N,K,n)$</li>\n",
    "                <li>Description: A probability distribution describing the probability of $x$ successes in $n$ draws (without replacement), from finite population of size $N$ that contains exactly $K$ success elements.</li>\n",
    "                <li>PMF: $p_X(x) = \\frac{\\binom{K}{x} \\binom{N-K}{n-x}}{\\binom{N}{n}}$</li>\n",
    "                <li>Mean: $\\mu = n\\frac{K}{N}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{nK(N-K)(N-n)}{N^2(N-1)}$</li>\n",
    "                <li>Skewness: $\\frac{(N-2K)(N-1)^{1/2}(N-2n)}{(N-2)\\sqrt{nK(N-K)(N-n)}}$</li>\n",
    "                <li>Excess Kurtosis: <em>ommited for brevity</em></li>\n",
    "                <li>Support: $x \\in {0,1,...,n}$</li>\n",
    "                <li>Applications: sampling without replacement</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_hypergeometric.png\" width=450 />\n",
    "        </div>\n",
    "    </div>\n",
    "    <h3><a id=\"s4.7\">&sect;4.7 Common Continuous Distributions</a></h3><!--#############################################-->\n",
    "    <div style=\"padding: 1% 5% 0% 5%\">\n",
    "        <h5><a id=\"s4.7.1\">&sect;4.7.1 Uniform</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\text{unif}(a,b)$</li>\n",
    "                <li>Description: A symmetric distribution describing an experiment where there is an arbitrary outcome within certain bounds.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{1}{b-a}$</li>\n",
    "                <li>Mean: $\\mu = \\frac{a+b}{2}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{(b-a)^2}{12}$</li>\n",
    "                <li>Skewness: $0$</li>\n",
    "                <li>Excess Kurtosis: platykurtic $-1.2$</li>\n",
    "                <li>Support: $x\\in [a,b]$</li>\n",
    "                <li>Applications: reference, inverse transform sampling</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_uniform_cont.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.2\">&sect;4.7.2 Normal</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{N}(\\mu,\\sigma^2)$</li>\n",
    "                <li>Description: A common symmetric distribution for real-valued random variables.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{1}{\\sigma\\sqrt{2\\pi}} e^{-\\frac{(x-\\mu)^2}{2\\sigma^2}}$</li>\n",
    "                <li>Mean: $\\mu = \\mu$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\sigma^2$</li>\n",
    "                <li>Skewness: $0$</li>\n",
    "                <li>Excess Kurtosis: $0$</li>\n",
    "                <li>Support: $x\\in \\mathBB{R}$</li>\n",
    "                <li>Applications: found everywhere in nature and natrual occuring processes, central limit theorem</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_normal.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.3\">&sect;4.7.3 Log-Normal</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{LogNormal}(\\mu,\\sigma^2)$</li>\n",
    "                <li>Description: A probability distribution of a random variable whose logarithm is normally distributed.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{1}{x\\sigma\\sqrt{2\\pi}} e^{-\\frac{(\\ln x-\\mu)^2}{2\\sigma^2}}$</li>\n",
    "                <li>Mean: $\\mu = e^{(\\mu + \\frac{\\sigma^2}{2})}$</li>\n",
    "                <li>Variance: $\\sigma^2 = (e^{\\sigma^2} - 1)e^{2\\mu + \\sigma^2}$</li>\n",
    "                <li>Skewness: $(e^{\\sigma^2}+2)\\sqrt{e^{\\sigma^2}-1}$</li>\n",
    "                <li>Excess Kurtosis: $e^{4\\sigma^2}+2e^{3\\sigma^2}+3e^{2\\sigma^2}-6$</li>\n",
    "                <li>Support: $x\\in (0,\\infty)$</li>\n",
    "                <li>Applications: naturally occuring and strictly postive quantities (stock proces, FX rates, precipitation)</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_log_normal.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.4\">&sect;4.7.4 Logistic</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Log}(\\mu,s)$</li>\n",
    "                <li>Description: A distribution similar to the normal distribution but with heavier tails. Beneficial for its simplified CDF.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{e^{-(x-\\mu)/s}}{s(1+e^{-(x-\\mu)/s})^2}$</li>\n",
    "                <li>CDF: $f_X(x) = \\frac{1}{1+e^{-(x-\\mu)/s}}$</li>\n",
    "                <li>Mean: $\\mu = \\mu$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{s^2\\pi^2}{3}$</li>\n",
    "                <li>Skewness: $0$</li>\n",
    "                <li>Excess Kurtosis: $6/5$</li>\n",
    "                <li>Support: $x\\in \\mathBB{R}$</li>\n",
    "                <li>Applications: logistic regressions & neural networks</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_logistic.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.5\">&sect;4.7.5 Exponential</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Exp}(\\lambda)$</li>\n",
    "                <li>Description: A distribution that models the time between events (waiting time) in a Poisson process.</li>\n",
    "                <li>PDF: $f_X(x) = \\lambda e^{-\\lambda x}$</li>\n",
    "                <li>Mean: $\\mu = \\frac{1}{\\lambda}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{1}{\\lambda^2}$</li>\n",
    "                <li>Skewness: $2$</li>\n",
    "                <li>Excess Kurtosis: $6$</li>\n",
    "                <li>Support: $x\\in [0,\\infty)$</li>\n",
    "                <li>Applications: product reliability modelling & wait time modelling</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_exponential.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.6\">&sect;4.7.6 Chi-Square</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\chi^2_k$</li>\n",
    "                <li>Description: A distribution with k-degrees of freedom that models the sum of squares of k indpendent standard normal random variables.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{1}{2^{k/2}\\Gamma(k/2)}x^{k/2-1}e^{-x/2}$ where $\\Gamma(k/2) = \\int_0^\\infty x^{k/2-1}e^{-x}dx$</li>\n",
    "                <li>Mean: $\\mu = k$</li>\n",
    "                <li>Variance: $\\sigma^2 = 2k$</li>\n",
    "                <li>Skewness: $\\sqrt{8/k}$</li>\n",
    "                <li>Excess Kurtosis: $\\frac{12}{k}$</li>\n",
    "                <li>Support: $x\\in [0,\\infty)$</li>\n",
    "                <li>Applications: hypothesis testing, confidence interval construction</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_chi_square.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.7\">&sect;4.7.7 Student's t</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $t_\\nu$</li>\n",
    "                <li>Description: A distribution arising when estimating the mean of normally distributed population with small sample sizes. If sample of $n$ observations is used then $\\nu=n-1$</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{\\Gamma(\\frac{\\nu+1}{2}) \\Big( 1+\\frac{x^2}{\\nu} \\Big)^{-\\frac{\\nu+1}{2}}}{\\sqrt{\\nu\\pi}\\Gamma(\\frac{\\nu}{2})}$ where $\\nu=$ degrees of freedom</li>\n",
    "                <li>Mean: $\\mu = 0$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{\\nu}{\\nu-2}$</li>\n",
    "                <li>Skewness: $0$</li>\n",
    "                <li>Excess Kurtosis: $\\frac{6}{\\nu-4}$</li>\n",
    "                <li>Support: $x\\in \\mathBB{R}$</li>\n",
    "                <li>Applications: hypothesis testing & confidence interval construction</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_student_t.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.8\">&sect;4.7.8 Beta</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Beta}(\\alpha,\\beta)$</li>\n",
    "                <li>Description: A distribution that models the distribution of probabilities $x$ (probability of success).</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{x^{\\alpha-1}(1-x)^{\\beta-1}}{B(\\alpha,\\beta)}$ where $B(\\alpha,\\beta) = \\frac{\\Gamma(\\alpha)\\Gamma(\\beta)}{\\Gamma(\\alpha+\\beta)}$ is the normalization factor</li>\n",
    "                <li>Mean: $\\mu = \\frac{\\alpha}{\\alpha \\beta}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{\\alpha}{\\beta^2}$</li>\n",
    "                <li>Skewness: <em>omitted for brevity</em></li>\n",
    "                <li>Excess Kurtosis: <em>omitted for brevity</em></li>\n",
    "                <li>Support: $x\\in [0,1]$</li>\n",
    "                <li>Applications: Bayesian statistics & task duration modelling</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_beta.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.9\">&sect;4.7.9 Gamma</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Gamma}(\\alpha,\\beta)$</li>\n",
    "                <li>Description: A distribution that is a generalization of the exponential distribution. Used to model the wait time until k-th event occurs.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{\\beta^\\alpha}{\\Gamma(\\alpha)}x^{\\alpha-1}e^{-\\beta x}$</li>\n",
    "                <li>Mean: $\\mu = \\frac{\\alpha}{\\beta}$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\frac{\\alpha}{\\beta^2}$</li>\n",
    "                <li>Skewness: $\\frac{2}{\\sqrt{\\alpha}}$</li>\n",
    "                <li>Excess Kurtosis: $\\frac{6}{\\alpha}$</li>\n",
    "                <li>Support: $x \\in (0,+\\infty)$</li>\n",
    "                <li>Applications: queuing models, financial services modelling, climatology</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_gamma.png\" width=450 />\n",
    "        </div>\n",
    "        <h5><a id=\"s4.7.10\">&sect;4.7.10 Weibull</a></h5>\n",
    "        <div style=\"padding: 1% 5% 0% 5%\">\n",
    "            <ul>\n",
    "                <li>Notation: $\\mathcal{Weibull}(k,\\lambda)$</li>\n",
    "                <li>Description: A distribution that models the wait time for an event, where the event becomes more/less likely with time.</li>\n",
    "                <li>PDF: $f_X(x) = \\frac{k}{\\lambda}\\Big(\\frac{x}{\\lambda}\\Big)^{k-1}e^{-(x/\\lambda)^k}$</li>\n",
    "                <li>Mean: $\\mu = \\lambda\\Gamma(1+1/k)$</li>\n",
    "                <li>Variance: $\\sigma^2 = \\lambda^2(\\Gamma(1+2/k) - (\\Gamma(1+1/k))^2)$</li>\n",
    "                <li>Skewness: <em>omitted for brevity</em></li>\n",
    "                <li>Excess Kurtosis: <em>omitted for brevity</em></li>\n",
    "                <li>Support: $x \\in [0,+\\infty)$</li>\n",
    "                <li>Applications: survival analysis & reliability analysis</li>\n",
    "            </ul>\n",
    "            <img src=\"images/4.4_weibull.png\" width=450 />\n",
    "        </div>\n",
    "    </div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:2px solid black\" />\n",
    "<h1><a id=\"c5\">Chapter 5: Other Learning Resources</a></h1>\n",
    "<div style=\"padding: 5% 5% 5% 5%\">\n",
    "    <p>Students taking the Statistical Models & Probability course are expected to know and understand the topics covered in this lecture. The first lecture will briefly review the topics with respect to calculating the values in Python. If you need any clarifications or extra explanations the first lecture will be a good opportunity to ask and get up to speed.</p>\n",
    "    <p>If you wish to get an alternate explanation of some of the topics covered check out the <a href=\"https://youtube.com/playlist?list=PL8dPuuaLjXtNM_Y-bUAhblSAdWRnmBUcr\">YouTube playlist by CrashCourse</a></p>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}