Conditional-Probability-and-Bayes-Rule.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conditional Probability and Bayes Rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to understand Conditional Probability we first need to understand some basics of Set Theory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set Theory:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set:\n",
    "A Collection of numbers without repeating a single number more than a single time.\n",
    "Set is represented as \n",
    "A = {1,2,3,4,5}\n",
    "\n",
    "Here A is a set.\n",
    "\n",
    "Now let's understand some operations on Sets which will help us to understand conditional probability much better"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Union of Set\n",
    "\n",
    "So Union is an Operation and it need operands and operands are sets.\n",
    "So it is operated on two sets and gives another set that is called union of both the sets.\n",
    "\n",
    "Union contains elements of both the sets on which is is applied on.\n",
    "\n",
    "For Example,\n",
    "A = {1,2,3,4}\n",
    "B = {5,6,7,8}\n",
    "\n",
    "A U B = {1,2,3,4,5,6,7,8}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Intersection of Sets\n",
    "\n",
    "Intersection contains only common elements between two sets.\n",
    "\n",
    "It is similar to intersection of line where the point where two lines intersect is a common point between two lines\n",
    "\n",
    "For Example,\n",
    "A = {1,2,3,4}\n",
    "B = {2,3}\n",
    "\n",
    "A ꓵ B = {2,3}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conditional Probability\n",
    "\n",
    "It is a type of Probability that is calculated when another event which is given and has already occured\n",
    "\n",
    "For Example,\n",
    "Given that it rained yesterday, what is the probability of having rain today.\n",
    "\n",
    "\n",
    "And its Relation is \n",
    "\n",
    "Conditional Probability = P of (A ꓵ B) / P (B)\n",
    "\n",
    "### We already discussed the way it is derieved in class\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiplication Theorem on Conditional Probability:\n",
    "\n",
    "According to it Intersection Probability of two sets can also be calculated by this relation which is not easy to find easily.\n",
    "\n",
    "A ꓵ B = P(A) x P(B|A) \n",
    "\n",
    "A ꓵ B = P(B) x P(A|B)\n",
    "\n",
    "#### Notice the difference between two relations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conditional Probability of Independent Events\n",
    "\n",
    "Independent Events: When there are two events and Occurence of one Event doesn't affect the probability of another event.\n",
    "\n",
    "It gives us a special relation,\n",
    "\n",
    "Here P(A|B) becomes P(A)\n",
    "and  P(B|A) becomes P(B)\n",
    "\n",
    "Hence, using Multiplication theorem\n",
    "\n",
    "P(A ꓵ B) = P(A) x P(B)\n",
    "\n",
    "#### This relation will be very useful when we will be finding Estimates using Maximum Likelihood Estimation "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bayes Rule\n",
    "\n",
    "It is a theorem describing how the conditional probability of each of a set of possible causes for a given observed outcome can be computed from knowledge of the probability of cach cause and the conditional probability of the outcome of each cause.\n",
    "\n",
    "Its Relation,\n",
    "\n",
    "Let's say there are 5 events A1,A2,A3 and there is an already occured event B\n",
    "\n",
    "So Conditional Probability of one of these events will be:\n",
    "\n",
    "P(A1|B) = P(B|A1) x P(A1) / P(B)\n",
    "\n",
    "or \n",
    "\n",
    "P(A1|B) = P(B|A1) x P(A1) / P(B|A1) x P(A1) + P(B|A2) x P(A2) + P(B|A3) x P(A3)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}