II_07_Equations_for_the_inverse_Cramer_rule_Volume_of_a_box.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This notebook is part of lecture 20 *Cramer's rule, inverse, and volume of a box* in the OCW MIT course 18.06 by Prof Gilbert Strang [1]\n",
    "+ Created by me, Dr Juan H Klopper\n",
    "    + Head of Acute Care Surgery\n",
    "    + Groote Schuur Hospital\n",
    "    + University Cape Town\n",
    "    + <a href=\"mailto:juan.klopper@uct.ac.za\">Email me with your thoughts, comments, suggestions and corrections</a> \n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/InteractiveResource\" property=\"dct:title\" rel=\"dct:type\">Linear Algebra OCW MIT18.06</span> <span xmlns:cc=\"http://creativecommons.org/ns#\" property=\"cc:attributionName\">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>.\n",
    "\n",
    "+ [1] <a href=\"http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm\">OCW MIT 18.06</a>\n",
    "+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: #ddd;\n",
       "    color: #000000;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(200,100,0);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML, Image\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sympy import init_printing, symbols, eye, Matrix, Rational\n",
    "from warnings import filterwarnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "init_printing(use_latex = 'mathjax')\n",
    "filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Equations for the inverse of a matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cramer's rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The volume of a box"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deriving an equation for the inverse of a matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The equation for the inverse of a matrix\n",
    "$$ {A}^{-1}=\\frac{1}{\\left| A \\right|}{C}^{T} $$\n",
    "* With arithmetic alteration we have the following\n",
    "$$ \\therefore {A}{C}^{T}=\\left|A\\right|I $$\n",
    "* Writing out the left-hand side we have\n",
    "$$ \\begin{bmatrix} {a}_{11} & {a}_{12} & \\dots & {a}_{1n} \\\\ {a}_{21} & {a}_{22} & \\dots & {a}_{2n} \\\\  \\vdots & \\vdots & \\dots & \\vdots \\\\ {a}_{n1} & {a}_{n2} & \\dots & {a}_{nn} \\end{bmatrix} \\begin{bmatrix} { C }_{ 11 } & { C }_{ 21 } & \\dots  & { C }_{ n1 } \\\\ { C }_{ 12 } & { C }_{ 22 } & \\dots  & { C }_{ n2 } \\\\ \\vdots  & \\vdots  & \\dots  & \\vdots  \\\\ { C }_{ 1n } & { C }_{ 2n } & \\dots  & { C }_{ nn } \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* From the previous lecture we had the equation for the determinant (using cofactors), which correlates with the above (looking at row 1 times column 1 (*i*=1)\n",
    "$$ \\left|A\\right|={a}_{i1}{C}_{i1} + {a}_{i2}{C}_{i2} \\dots {a}_{in}{C}_{in}$$\n",
    "* Alas we have to get |A| only on the main diagonal for the right-hand side above\n",
    "* It follows, though that i.e. for position row 1, column 2 we do get a zero\n",
    "* Look at the 2&#215;2 matrix\n",
    "$$ { \\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix} }^{ -1 }=\\frac { 1 }{ \\left| A \\right|  } { C }^{ T }\\\\ { \\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix} }^{ -1 }=\\frac { 1 }{ \\left| A \\right|  } { \\begin{bmatrix} d & -c \\\\ -b & a \\end{bmatrix} }^{ T }\\\\ { \\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix} }^{ -1 }=\\frac { 1 }{ \\left| A \\right|  } { \\begin{bmatrix} d & -b \\\\ -c & a \\end{bmatrix} } $$\n",
    "* So for AC<sup>T</sup> we would have the following (note, though what happens if we try and get row 1, column 2\n",
    "$$ \\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix}{ \\begin{bmatrix} d & -b \\\\ -c & a \\end{bmatrix} }=\\begin{bmatrix} ad-bc & -ab+ab \\\\ cd-cd & ad-bc \\end{bmatrix}=\\begin{bmatrix} ad-bc & 0 \\\\ 0 & ad-bc \\end{bmatrix}=\\begin{bmatrix} \\left| A \\right|  & 0 \\\\ 0 & \\left| A \\right|  \\end{bmatrix}=\\left| A \\right| \\begin{bmatrix} 1 & 0 \\\\ 0 & 1 \\end{bmatrix} $$\n",
    "* ... and that's so cool!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cramer's rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* From A**x**=**b** we have **x**=A<sup>-1</sup>**b**, which gives us the following\n",
    "$$ \\mathbf { x } =\\frac { 1 }{ \\left| A \\right|  } { C }^{ T }\\mathbf { b } $$\n",
    "$$ \\therefore \\begin{bmatrix} { x }_{ 1 } \\\\ { x }_{ 2 } \\\\ \\vdots  \\\\ { x }_{ n } \\end{bmatrix}=\\frac { 1 }{ \\left| A \\right|  } \\begin{bmatrix} { C }_{ 11 } & { C }_{ 21 } & \\dots  & { C }_{ n1 } \\\\ { C }_{ 12 } & { C }_{ 22 } & \\dots  & { C }_{ n2 } \\\\ \\vdots  & \\vdots  & \\dots  & \\vdots  \\\\ { C }_{ 1n } & { C }_{ 2n } & \\dots  & { C }_{ nn } \\end{bmatrix}\\begin{bmatrix} { b }_{ 1 } \\\\ { b }_{ 2 } \\\\ \\vdots  \\\\ { b }_{ n } \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This is difficult to see, but we successively replace each column in A with the column vector **b**, whic creates a bunch of new matrices B<sub>j</sub>, such that the following applies\n",
    "$$  \\begin{bmatrix} { x }_{ 1 } \\\\ { x }_{ 2 } \\\\ \\vdots  \\\\ { x }_{ n } \\end{bmatrix}=\\begin{bmatrix} \\frac { \\left| { B }_{ 1 } \\right|  }{ \\left| A \\right|  }  \\\\ \\frac { \\left| { B }_{ 2 } \\right|  }{ \\left| A \\right|  }  \\\\ \\vdots  \\\\ \\frac { \\left| { B }_{ n } \\right|  }{ \\left| A \\right|  }  \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The volume of a box (parallelepiped )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Consider a box in three dimensions (each side is a parallelogram)\n",
    "* Make one corner coincide with origin\n",
    "* The vector coordinate of the three sides that emanate from this corner become the rows of a square matrix, 3&#215;3 in this case\n",
    "* The volume is then the determinant of this matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Consider a square box of sides of unit length one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  0  0⎤\n",
       "⎢       ⎥\n",
       "⎢0  1  0⎥\n",
       "⎢       ⎥\n",
       "⎣0  0  1⎦"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = eye(3)\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$1$$"
      ],
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.det()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This proves the first property of determinants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What about the orthogonal matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Here we have the three edges being orthonormal\n",
    "* We know that Q<sup>T</sup>Q = I"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\left| { Q }^{ T }Q \\right| =\\left| I \\right| \\\\ \\left| { Q }^{ T } \\right| \\left| Q \\right| =\\left| I \\right| \\\\ \\because \\quad \\left| { Q }^{ T } \\right| =\\left| Q \\right| \\\\ \\therefore \\quad \\left| Q \\right| \\left| Q \\right| =\\left| I \\right| \\\\ { \\left| Q \\right|  }^{ 2 }=\\left| I \\right| =1\\\\ \\therefore \\quad \\left| Q \\right| =\\pm 1 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A rectangular box (edges square)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Doubling an edge doubles the volume\n",
    "* This is akin to a single row multiplied by a scalar\n",
    "* Thus determinant will increase by this scalar (multiplication)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Area of  parallelogram and a triangle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The area of a parallelogram is just the determinant of a 2&#215;2 matrix with the rows being row vectors of two sides from the origin\n",
    "* The area of a triangle is simply half of this"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* For the triangle that is not at the origin, with three angles at (*x*<sub>1</sub>, *y*<sub>1</sub>), (*x*<sub>2</sub>, *y*<sub>2</sub>) and (*x*<sub>3</sub>, *y*<sub>3</sub>), we simply subtract values along the axes from each other\n",
    "* That is akin to getting the determinant of this matrix\n",
    "$$ \\begin{vmatrix} { x }_{ 1 } & { y }_{ 1 } & 1 \\\\ { x }_{ 2 } & { y }_{ 2 } & 1 \\\\ { x }_{ 3 } & { y }_{ 3 } & 1 \\end{vmatrix} $$\n",
    "* Simple row reduction is equivalent to moving the triangle by the subtraction above\n",
    "$$ \\begin{vmatrix} { x }_{ 1 } & { y }_{ 1 } & 1 \\\\ { x }_{ 2 }-{ x }_{ 1 } & { y }_{ 2 }-{ y }_{ 1 } & 0 \\\\ { x }_{ 3 }-{ x }_{ 1 } & { y }_{ 3 }-{ y }_{ 1 } & 0 \\end{vmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example problem 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Calculate the volume of the tetrahedron with vertices being the following vectors\n",
    "    * **a**<sub>1</sub>=(2,2,-1), **a**<sub>2</sub>=(1,3,0), **a**<sub>3</sub>=(-1,1,4)\n",
    "* Also calculate the volume if **a**<sub>3</sub>=(-201,-199,104)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}2 & 2 & -1\\\\1 & 3 & 0\\\\-1 & 1 & 4\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡2   2  -1⎤\n",
       "⎢         ⎥\n",
       "⎢1   3  0 ⎥\n",
       "⎢         ⎥\n",
       "⎣-1  1  4 ⎦"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = Matrix([[2, 2, -1], [1, 3, 0], [-1, 1, 4]])\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The volume of a tetrahedron is a third times the area of the (any) base and the height from the (chosen) base\n",
    "* The volume of a parallelepiped is the area of the base times the height\n",
    "* If we keep the base of the two the same and the apex the same, we note that the base is twice the area of the triangle that forms the base of the tetrahedron\n",
    "* We thus have that the volume of the tetrahedron is a 6<sup>th</sup> of the volume of the parallelepiped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$12$$"
      ],
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.det()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$2$$"
      ],
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Rational(1, 6) * A.det()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}2 & 2 & -1\\\\1 & 3 & 0\\\\-201 & -199 & 104\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡ 2     2    -1 ⎤\n",
       "⎢               ⎥\n",
       "⎢ 1     3     0 ⎥\n",
       "⎢               ⎥\n",
       "⎣-201  -199  104⎦"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A_new = Matrix([[2, 2, -1], [1, 3, 0], [-201, -199, 104]])\n",
    "A_new"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$2$$"
      ],
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Rational(1, 6) * A_new.det()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* By the second part of the third property of determinants we know, though, that a constant multiple of a row subtracted from another (one of the elementary row operations) does not change the determinant\n",
    "* In this case we subtracted 100 times row 1 from row 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* In effect, the height is not changing; the apex is moving away parallel to **a**<sub>1</sub>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}