08_Lecture_I_05_LU_decomposition_of_A.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This notebook is part of lecture 4 *Factorization into LU* in the OCW MIT course 18.06 [1]\n",
    "+ Created by me, Dr Juan H Klopper\n",
    "    + Head of Acute Care Surgery\n",
    "    + Groote Schuur Hospital\n",
    "    + University Cape Town\n",
    "    + <a href=\"mailto:juan.klopper@uct.ac.za\">Email me with your thoughts, comments, suggestions and corrections</a> \n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/InteractiveResource\" property=\"dct:title\" rel=\"dct:type\">Linear Algebra OCW MIT18.06</span> <span xmlns:cc=\"http://creativecommons.org/ns#\" property=\"cc:attributionName\">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>.\n",
    "\n",
    "+ [1] <a href=\"http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm\">OCW MIT 18.06</a>\n",
    "+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(245,179,64);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML, Image\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![SegmentLocal](KRG_elegant_logo_for_light_BG.png \"segment\")\n",
    "![SegmentLocal](UCT_logo.gif \"segment\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Library import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sympy import init_printing, Matrix, symbols, eye, Rational\n",
    "init_printing()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LU decomposition of a matrix A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will decompose the matrix $A$ into and upper and lower triangular matrix, such that multiplying these will return $A$.  this is shown in (1) where $L$ is the lower triangular matrix and $U$ is the upper triangular matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ A = LU \\tag{1}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Turning a matrix of coefficients into _U_pper triangular form"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the following matrix of coefficients shown in (2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix} 1 & -2 & 1 \\\\ 3 & 2 & -2 \\\\ 6 & -1 & -1 \\end{bmatrix}  \\tag{2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to convert this into upper triangular form.  A generic $ 3 \\times 3$ upper triangular matrix is shown in (3)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix} a_{11} & a_{12} & a_{13} \\\\ 0 & a_{22} & a_{23} \\\\ 0 & 0 & a_{3,3} \\end{bmatrix} \\tag{3} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that all entries _below_ the main diagonal are $0$.  This is an _upper triangular matrix_."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get our matrix in (2) into upper triangular form, successive elementary row operation follow, which remember, is nothing other than matrix multiplication of the elementary matrices.  An elementary matrix is an identity matrix on which one elementary row operation was performed.  below, we create the $3 \\times 3$ matrix in (2) and save it as `sympy` matrix object named `A`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHkAAABLCAMAAABjltjdAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC4ElEQVRoBe2a63KEIAyFAW8tXsv7P2wB3TUHEeKs7nam8qeAyfkkYg2wQhpflHhX0TNQCGmK0pbqXWDhaKUyjly/DbqCyoAsdbNezNXqtu0GmbNy1yOyQG5U2xk+uXYzozDZpxSXBbIV6g+Qlb/JcXSDypSI7Cvk0d+l4sySk8lKu5Eqw3jSJ5PnCGs7R7PlCrI0fZYbnT6vPGdP1C0DHJu4R8hNp9eyPNyWBX6VHBlcyQOfT54Kezc143/v2TNMOrDoGf/1TibXurVlfqsjT4J2ZclqGI1WnNfEyXbzZ7ajiGg9KhvO7ajnJZ03+ZKw7oje0d4JzCXdd7QvCeuO6F+Jdl0U7ZDNnwU/w8fxgh+MuXHf+dJMaL9pcTP80BH9gNw7aGNyH58DGT7A0Q/I1eginSUfyPCBjH5A9naTKcF+2ziQ4YMz+m3Ilc6BFzVWhg/kufHwC8iyV4qxWLEavAx/S376BWRrOWXntpfjZfhb8tNvSxbzRNj6QA8nw0+vDBz5y3zPqtIHemAMmpvhw93aBvH7IbsVjfELYgaZneEHZOoH0R79xlQ+2vwMH8ngB+TSvVD59/lAhg9k9AOyqOyaIf/FYGf4wA1XBkgOTC9t3uRLwxuI39EOAnJp8472peENxP9OtAtbeOmQH0Jkrz4YGjaJPY651jbtZW0zOcH4Xj2iaAvtkdy5RcYwUPN0PbLPlXQg9kCeDhwlzACilCQ+LhJ7IHecI4mHiP9LlKB/r0HsgWw6WfRtfi256hKltTNRI/aUbNdyLh0ac2tJokyUSO9+ldgj2R9JtCNj83YRJ0r7OHKF2FOyMP4wptw7+4pk7kSJ6C/VtD2QtV85l9nF5ApJklezZ43YA3lIj/npv1aI0tqZqBF7IFfLc064BpeIUnAl3iT2QBaDPR5oPjC37W0W9nSA/8GI7tXHB+t7wR7HnPA6/dJNPj2kCcH/G+1P/ZKl8b8sefcvWewGxS9L5ybTY1kkkAAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & -2 & 1\\\\3 & 2 & -2\\\\6 & -1 & -1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  -2  1 ⎤\n",
       "⎢         ⎥\n",
       "⎢3  2   -2⎥\n",
       "⎢         ⎥\n",
       "⎣6  -1  -1⎦"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = Matrix([[1, -2, 1], [3, 2, -2], [6, -1, -1]])\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `eye()` function returns an identity matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFgAAABLCAMAAADDCbAzAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACN0lEQVRYCe2Z0XKDIBBFUUlsNWrK/39sAQvuRZYdUpuZzuBDQ7zZ47qhk9NGdcYfvbromHaeUp0ZtD1uF3GVg+neOPD9KubB0QjupvHIzqthWOZCEzSm4LGfH6YEfjyVGieWDDEF2w6XElhP7h6G9Xwn/gzGNeDV75snd22Ma8Bmdp3djB1I7sC4AjyGSp3DqiSuAN/N4IidWbLgJH4F7PkneATvcQV43Du+MR0ncQVY7TN+lt+8ENeAV7+DNbvdIK4B680Ndn6cxrufwLgGrCb3K711DBhjAPfrZqY+v5scbRyWpWe5GAOYa+WV8w0cp9ZG0UYRJxAXb9oV1DjitemiLDS0HDoG46C8fS0JDZRTMBrHGWzPlIQGyykYjaMajOUUHLyBERJ3pVLHWE7AiXHUdpyUE3AUA/4jpNRxUp4D54XE30FhFBF8EpbEOOpH4YnBZ1zHH+bTUwQhca8pdJz4zBf5U0EQEgmM5WTGCo3D30T6o9QxllMwGkfK9M9LYCwHsCAkShAaKAdwtskXTzZwHFwbRRtFnEBcvGlXUOOI1yYLKSc+Ax2DcRBeWJZz9BkKRuMItONRyuEDhoLROA5gWEk5C0bjCLjjUco5cGIcB/BnJeX2ZeQDhowiigEjLFIugxlhiWAmZ8GSsEg5C06Mw74uOWShyc9YoXEkVPtUyrk3TxQWNJLzhVkwGkemUPgPCw8G48iAhRx8huzjDOkXpxo4Du+/juKPvmAZ/TciF3/BorX6BphxJ5MVW0+5AAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  0  0⎤\n",
       "⎢       ⎥\n",
       "⎢0  1  0⎥\n",
       "⎢       ⎥\n",
       "⎣0  0  1⎦"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eye(3)  # Identity matrix of size 3 x 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to get a $-3$ in the first _pivot_ (the $1$ in row $1$, column $1$) to get rid of the $3$ in row $2$, column $1$ (we call the resulting elementary matrix `E21`, referring to the row $2$, column $1$).  Then we add the new row $1$ to row $2$.  Row $1$ of the identity matrix is then $\\left( -3,0,0 \\right) $ (but we leave it as $\\left( 1,0,0 \\right)$ in `E21`).  Adding this to row $2$ leaves $ \\left( -3,1,0 \\right) $."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see why this is so, if we multiply row $1$ by $-3$ we have $ \\left( -3, 0, 0 \\right) $.  Adding this to row $2$, which is $ \\left( 0,1,0 \\right) $, results in $ \\left( -3,1,0 \\right) $ and hence `E21` being what we see below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGkAAABLCAMAAABEOFk1AAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACeElEQVRYCe2Z7XKEIAxFUXBp8bO8/8MWcBUSgbCytjOt/tBgMjkmol6VNdotLbtqUSuAsUZzYZbuKhCz2UWrLelxGcQnFglSo6QPSlucj33mMEN3lCTbftAlpGFhTKokCrijJFPDWEISylbLp0TN0F1Fmtx0XVIHBd1VJN3bajptehhboLuGJLdUIsZhyF1DemhuEY0eoyTkfgvJAQ+0nbS6a0hyralL1ITcNSS2nqclPyM2dxVpcleSSM5y4K4iidmenH44nKJ1B3RXkZiyd6O5SZCgO05qp1mrNj55w7SSj2ObBDHgjpPCbO+yb1JNJ+/u3d2DHfh/M+LBeT9lVXRegZLKcmuwtNpHpB505jZOKNAiZelgo4VInXr+2JicAi1Tlo7UzbZzp0llytKR3GrRcTm3RuRq2uTgKjzJWd6pHCjXvUJl+aypGdvMM9UEZWra9d767CZrYpuKerLRpoQUUZZyUH7Z5cGcUlkWmiFFlOWH/kRH+hw2jjalL6gsCQnPr8Tbp2VJ7ao5TYLCM3ueZvcmdrZ7rExZuv4JO8HPX08lytJxzKrr++wdllCgt7LcGlm9zc696uxhgpsUduNV++7eqx0L43+9e6EkDA8M21RcIDzjNQFJiLMH43wcFJ5REpSEQWZk0nHBwz9KgpIQpQ+GdBxFgpIwSI1MOo4gIUmI0vthQRxBQpLQp0ZWQVwpKf4tcuftpHQcQUKScM+MjYI4goQkIQb4MfHN0gRSJCgJfWZs0XEUCUpCnN+P6TiKBCWhz3ywiG+WdPfgx8ZDfr8DSEe/e7OA8IzejbbIt25vUk07/273furPsXR/dq/+c2xeML8BXKkn9DzBUVYAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\-3 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1   0  0⎤\n",
       "⎢        ⎥\n",
       "⎢-3  1  0⎥\n",
       "⎢        ⎥\n",
       "⎣0   0  1⎦"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E21 = Matrix([[1, 0, 0], [-3, 1, 0], [0, 0, 1]])\n",
    "E21"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we left-multiply $A$ by the elementary matrix `E21`.  In matrix notation, we would write $E_{21} A$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHkAAABLCAMAAABjltjdAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAADMUlEQVRoBe2a65ajIAyAEajsAl4m7/+wy0WdRKGh2+rsOSu/EGI+CBESWtFBKlJcVWwGCtGB0qE8rgKLSNMSIrm/DPoN0jtyZ813J1frnfNjx0nF/oJaQjbSeWgn99EzFLCrVFZLyEHR8AJZpkFOU5wUUwpq3yFPaZSyxUs+TJY2zlRCw0p/mJwtbIOPsuUMcgcDyy26zzvrnIjWNYBLjvsK2Xj7XZbFdU3gd8mFyek28OfJswqj6Rv23k97WBfBYmjY9T5M7q0LJX/VhZXATSxZjhNY2fKZRLU+H7MeI4r1otq9bxffPKXxJp9i1orS29oVw5zSfFv7FLNWlF5q7XSmm+Vo25GVGhx/6Bk1KMeG2ce5TuBDcLEcbZTsZyGMZdEpHJBB9sViLVi1nqmErFMYq0ZGodZRwPBn1F4PyVYJeUxdM5dm5KF9lAzJjA9g7PiAMZjMpZnvp/X0uTpns5I5nRJgcMzwSkOQehjcmpFga/eQ4qqG2F0BeNYPj+xoqn7NPYvkNIDji1vLrPoRVg1ba2NlTG4sBCabPOcHl6+kIHuGaf0+ashSZhD8Y8k9I/kX/M4v53WeOQ/LyJ4TOwxoTN+hW3LPL3xbMaYvWTNflVmSR8k54h49LVlvbsfWFjql/47bIqbsXOOrPpbdZ1o2KkIWNu6e0+r3+zGvzw8fV1i3huXra6KP28Wwugclh6NgkBw43PTIkFv8xYkR7pKkW/2SkrfhXVC5yRcYeUPc1t5McUHltvYFRt4Q/461VSj8xr0NvHBXv/WVKkiezrm34RxoumaKast39SVgbqPylOzjOZbjg7oC3FO458LdhzqSJ2Q2xn+m6dBXaqiRfctPEkQh0kTaaw9InswZfBdyyVeOfKSpBiPtSB6TDfgY1E0vZA9IEyHUHpA8Jaeo0q2RUu111I40odZ6FcljsoAUlupa9lCI3JGmI+65PCHbFO9qaA+kn5KPY8F3+4ScU57qnJ9rKvQem9BICfmxrPPxjVoL0lQTIe1InpDFGNIA8wO+HUanQizefmAU7+rJHOkDkadzpoLnPt3kc+1Ltf+/1v6pf7KY9M+Sq//JEm5P/wAImycXRVgtogAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & -2 & 1\\\\0 & 8 & -5\\\\6 & -1 & -1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  -2  1 ⎤\n",
       "⎢         ⎥\n",
       "⎢0  8   -5⎥\n",
       "⎢         ⎥\n",
       "⎣6  -1  -1⎦"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E21 * A # The resulting matrix after multiplication by E21"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Good, we now have a leading $0$ in row $2$.  We follow the same steps to get rid of the leading $6$ in row $3$, column $1$.  Multiplying row $1$ (of the identity matrix) by $-6$ and adding this new row to row $3$ yields elementary matrix `E31`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGkAAABLCAMAAABEOFk1AAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACkUlEQVRYCe2Z3bajIAyFUWmZ8bfD+z/sAFbNhiS6Fu1czNGbIrH7MxHrbjSNT1trvrX1K8CYxnc2bI9vgUxUt62PpOfXIIewFUhN746D5FHXTYNymjTMklw7jP4KaXwZ43oRBWGWFHKYrpBsH7PtZiFnDFeR5rRcX9JJYbiK5IeYzcOHGnIbhmtIbpOyHMdk4RrS03cR0fiJJWXhj5ASsKDtpDVcQ3JrTg8hpyxcQzLrdXrpK2ILV5HmdCdZcZVDuIpkl3hxhrG4ROsEhqtIpo+/RksjkDDMk9p58X3LL14q67ppakWQgTBPomqfGt+kmkre1burhxX4QSuCWkIsQranO1Aqw1cPLGGmTXbPHCjIsCS0hES6HGoOFGVYElrCUp7MaCSUYUmbjxMcIwGprhplOFJmCalwMVZyymQ40m7Uzh+6Wk6ZjEriHSPkpeS0k2RnmVlCUM53FFImE3P65X+jwIljpAcrpMx4/uH+fZ44xqsklOGuk0FLSIWLsZYTyrAktISFOp3QSCjDk8ASUuFsfOJAQYYnZYIf2b1JNWW8q3dXDyvwE1dEFzb5L7Oh1hGLte4R46lX79mHNnCb+oWcjgHrWByBxlMnjbGBtz7QCp0wgdaRO4I8VFSS2I18i6J1rCGNqVvDKaxzaB25467m5McmdKnFjn1mHStIzo+x77lIpnk3dLIDvZiT88E4hV7XIrTOd5LsQCWSG/tjC3eRT+vbeqF+mXWsqJ7pU+fOer53HM4kdbG3pmQNadZzet9pYs8yoKXq5Wf1eF+nfH7bR+u4zdLPqyQzh2vtxLVn0DpSwja+TDLdMCg9SWxKburHJxhP9dfo+M4HRjeppoj/b/X+1Ztjl97sfvvNsbXmLwWMJ8N/15XxAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\-6 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1   0  0⎤\n",
       "⎢        ⎥\n",
       "⎢0   1  0⎥\n",
       "⎢        ⎥\n",
       "⎣-6  0  1⎦"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E31 = Matrix([[1, 0, 0], [0, 1, 0], [-6, 0, 1]])\n",
    "E31"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now for the left-multiplication of $E_{21} A$ by `E31` so that we have $E_{31} E_{21} A$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHkAAABLCAMAAABjltjdAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAADOUlEQVRoBe2a24KcIAxAERBa8Lb5/48tFzHgoGE6jn2oPGVFcyAEkrDDOgiNs7uaiUDGOhDStf4uMPM0ycGT1W1QBMkduTMaOylJWTtMHfWW76+oLcia2wHaycp7hgBylepqC7JTNL5B5mGQ8+wnRbSK2k/Icxglb/GSi8nc+JlyaFjpi8nRwsb5KNm+Qe5gJLlV9/lknQPR2AZwzXHfIevBYFsX1zaBPyVXJifbwNeTF+FGoxrO3qs9rPNgNjaceheTlbGuxV1dWYn8EUnm0wyGt2wTr3aIYXbIEVW5qnbv29Uvv/LwIX/FrAdKH2sfGOYrjx9rf8WsB0pvtXaI6XoNbTuyEKOlg54Wo7Bkmv061xkGl1ysoa0kDwtj2pDokA5w9+6bzRgwIsXUgixDGismQqGU/gVNx6i9nqJaLchT6FqoMiMO7VIyBDP2QNixh8mZzIaZ76d1+vfhnHUiUzo5wGiJ4dWGwOU42lSR5NZWEPKqhtxdAAykH76yvalUqj2r5DCA1w+3J4tQEyQN29NGYQpuzFhO1nHOPVWvhCR7gTntjyNkrTJw/rHWnp78C37Hj+M6L5SHRaSiXnsZ0BT2oV1rz5/8tmIKO1kSu0qvxSOnHHGPnteqNz7Prc1kKP8tdUTM0bmmd30sus+8HlQFmRl/es7J7/djTn/3g19h2ZqWp8+Y8sfFmNyjJLtQMHIK7G56uKst/iJiuLskbpNfluRteDcID/kGI2+Ix9qbKW4QHmvfYOQNcae1lUwnp8fvyA2ZPl7Po7RN5FRY4o0OxAvxkkxm+ng9j9IpLe8Ug4801gdE1wpyU6aPV1so5fqP5XhTGWLlntyU6SMPpWNa3hPnmpLuYs4p3z5NpZGHUq6fkEUK/zm5LdNHHkoELetWW9GWk9syfeShlKkmRL6lMlXyaaaPPJQIHHYr/KdHTm7L9JGHEqrOpUqmLzCvzcmsKdNHHko571Q22zKX+7kp00ceSqe0rFND2lP7k6Ql00ceSpnyU7GPVg3vFNZuyvSRh9IpLutcYs1YIdOZPl7Po5SpJkQ14yFVzpn48NLuh3ypOQll/6+1/9UvWXT4Zcndv2Rxt6d/AJhoKP/ZXl4RAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & -2 & 1\\\\0 & 8 & -5\\\\0 & 11 & -7\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  -2  1 ⎤\n",
       "⎢         ⎥\n",
       "⎢0  8   -5⎥\n",
       "⎢         ⎥\n",
       "⎣0  11  -7⎦"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E31 * E21 * A # This got rid of the leading 6 in row 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the $11$ in row $3$, column $2$ is the _pivot_ and we also need to get rid of the $8$ in row $2$, column $2$.  Unfortunately, we have an $8$ and an $11$ to deal with.  We will have to do two elementary row operations.  First $-11$ times row $2$ of the identity matrix $ \\therefore \\left( 0,-11,0 \\right) $,  added to $8$ times row $3$ $ \\therefore \\left( 0,0,8 \\right) $) such that we have $ \\left( 0,-11,8 \\right) $.  Below it is save as `E32`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHMAAABLCAMAAAB0tEgUAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC7ElEQVRoBe2aS2KrMAxFjXHCK/96/4t9GAPytQWIT9MOYNAqQtJBApIbE5XZcdPq57fSo5TKbG6G7fXzSOU4RlvHfH8ARwgTMbOyoJ07Vp43lfRow1hgFrqqrZhZ90oVpQwKscAc+mrETFO6MeTtzjDG3Rh7ntmO13kvOkaMPc+0lWvhZYcJ724Ye5pZzHXMLlFFsaeZb5s7WGabfWYUe505ore5C9PHnmYWvs+XpM8o9jRT+fPZH7iG5tjzzHa8M43sXoHY80zTuZNY1dun0u/F2PNMVbr3vi6TMDEWmbrtbKkFV78DFXnTaBkSY5EpOubLQQ/z8ghXCzyzXR3N5R3PbC+PcLXAH5htKH1Xj3PZcUCAh4WxT5C+S2neOCTAoTAwUfryqNArFuBYGJgofcPqvC1mYmFgzpJVIpPdUYiZWDhkRtKX7y30SplR4ZC5yFChUBD3GRVmmQKZPLYr7XNhppo6kr7hFHlbyowKuz7/2S9f84hMdhlSZqS/v8Pv9kdk8iEmFg7Pp0Lp63vf+ivuEwsDE6XvFs3vEzOxMDKPyGR1RIBDYWTut3ZHxMO8Y4p8jWe2/Fzu8D6zvWOKfI0/MNtQ+vIHiV5S1WRhhH81vN/m1bzgj32C9OVywUeqmiwIoBfjuque9CQwUfpSxrpFn2ZkMdFmXHAtpvUrYKL0ZXITF5HISoLmFXSWidKXyU1cRCIrCXKr2e3waKGalpfDPiPpy+QmLiKRlQQNDm1tU81fD0LmIkOlmjpUfttMlVtbz49FWKZUU8uZff5urZ1ulpAZSd9kRkVd0uYXF6k7spI8pYy7V3rb+edFITOSvkxu4iISWUmQUh72nta0gYnSl8lNXEQiKwkannV4n/YXLjBR+qa5qYdIZKVRqvOXT+v/AROlL5ObuIhEVhI03J+1O5Nmuh+QCdKXyY1cpKrJikKml5muqpX3eD7hbi/2eXd1vt7D5Odyh/f3Zvv530QU4y8VPvebiEGp/Ad+1Cob3R+EDwAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & -11 & 8\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1   0   0⎤\n",
       "⎢         ⎥\n",
       "⎢0   1   0⎥\n",
       "⎢         ⎥\n",
       "⎣0  -11  8⎦"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E32 = Matrix([[1, 0 , 0], [0, 1, 0], [0, -11, 8]])\n",
    "E32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHkAAABLCAMAAABjltjdAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC4ElEQVRoBe2a65ajIAyAEVB2QMXJ+z/schEqik2cqrPnrPyiGPNBCJBQWQOhcHZX0RHIWANCutLeBWaeJjl4cncb9AWSK3Kj1eshVuuM6ccGk/LPK2oLsuKmBzq5854hAJ2lutqC7BQNB8g8dNJaPyikVNR+Qrahl5ziJSeTufYj5UCY6ZPJ0cLa+ShariA3MKDcqvt8Ms+BqA0BXHPcI2TV61eZJ9eQwJ+SK4OTNPD55Em43nSEvfdsD2s8mA2EXe9kcqeNK3FVV2Zi2YSS+WhBc8oy8Wr7eMz2S0S1XlW79u3qm5c0PuRLzLqj9LH2jmEuaX6sfYlZd5Teau1wpqv5aFuRhRgMfugpMQiDhtnbsVroXXAxH20luZ8YUxpFh3CAO9mDRWvQIp2pBVmGMFaMiEIpvYDCz6i1niJbLchjeDRhaUbs2qlkCGZsAbFjC6MzmQkjXw/r7e/dMatExnRygMEg3at1gcthMCkjWVq7gxBXEWJ3AdCjfrhle1N1KfeskkMHti/mlkl0IyQNuZVYGYMbM7YkqzjmFstXQpA9gU3rYw9Zywycf8y5pyf/ga/4cpznCfOwiOwwsU2HxrAOzZx7fi9vK8awkiWyqtScPHLMEddoO2e9sX1pbSZD+m+wLcJG5xqP+lh0HztvVAWZab972uT36z6n323vZ1hSw/L0Guv8djEk9yjJ7igYOAZ2Nz3c5RY/ODHcXRI3yS9Lcu7eDZWHfIORM+KxdjbFDZXH2jcYOSP+GWvTIn1GFMvjy5XF3X45ZmKkTxTLvLlS3u0XZGqkT0sI1mD/e3EvVpCJkT5R7Ag5xdtIKE0UO0AmRvpEsRp4z9rESJ8o9iPy+0g/k9+LHSETI32imM9zX38B6Bjd7fk2MdInitUGvUemRfqMKHaETIz0iWJHyMRInypWQe9ZmxEjfaLYhlzc7Re750b0yoaHfKV117r/X2v/1pcsKnxZcveXLO729C/TuyeYbKYzmwAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & -2 & 1\\\\0 & 8 & -5\\\\0 & 0 & -1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  -2  1 ⎤\n",
       "⎢         ⎥\n",
       "⎢0  8   -5⎥\n",
       "⎢         ⎥\n",
       "⎣0  0   -1⎦"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "U = E32 * E31 * E21 * A\n",
    "U # We call is U for upper triangular"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix is now in upper triangular form, achived by the elementary matrixes shown in (4)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ { E }_{ 32 } { E }_{ 31 } { E }_{ 21 } A=U \\tag{4} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating the _L_ower triangular from"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, to reverse the process above, we would have to do the multiplication shown in (5)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ { \\left( { E }_{ 21 } \\right)  }^{ -1 }{ \\left( { E }_{ 31 } \\right)  }^{ -1 }{ \\left( { E }_{ 32 } \\right)  }^{ -1 }\\left( { E }_{ 32 } \\right) \\left( { E }_{ 31 } \\right) \\left( { E }_{ 21 } \\right) A=A \\tag{5} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The inverse of a matrix can be calculated using the `sympy` method `.inv()`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check this with a Boolean logic, using the `==` symbol, which checks if the left- and right-hand sides re equal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E21.inv() * E31.inv() * E32.inv() * E32 * E31 * E21 * A == A # The Boolean double equal signs asks: Is the\n",
    "# left-hand side equal to the right-hand side?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be back with the identity matrix just multiplying the inverse elementary matrices and the elementary matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFgAAABLCAMAAADDCbAzAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACN0lEQVRYCe2Z0XKDIBBFUUlsNWrK/39sAQvuRZYdUpuZzuBDQ7zZ47qhk9NGdcYfvbromHaeUp0ZtD1uF3GVg+neOPD9KubB0QjupvHIzqthWOZCEzSm4LGfH6YEfjyVGieWDDEF2w6XElhP7h6G9Xwn/gzGNeDV75snd22Ma8Bmdp3djB1I7sC4AjyGSp3DqiSuAN/N4IidWbLgJH4F7PkneATvcQV43Du+MR0ncQVY7TN+lt+8ENeAV7+DNbvdIK4B680Ndn6cxrufwLgGrCb3K711DBhjAPfrZqY+v5scbRyWpWe5GAOYa+WV8w0cp9ZG0UYRJxAXb9oV1DjitemiLDS0HDoG46C8fS0JDZRTMBrHGWzPlIQGyykYjaMajOUUHLyBERJ3pVLHWE7AiXHUdpyUE3AUA/4jpNRxUp4D54XE30FhFBF8EpbEOOpH4YnBZ1zHH+bTUwQhca8pdJz4zBf5U0EQEgmM5WTGCo3D30T6o9QxllMwGkfK9M9LYCwHsCAkShAaKAdwtskXTzZwHFwbRRtFnEBcvGlXUOOI1yYLKSc+Ax2DcRBeWJZz9BkKRuMItONRyuEDhoLROA5gWEk5C0bjCLjjUco5cGIcB/BnJeX2ZeQDhowiigEjLFIugxlhiWAmZ8GSsEg5C06Mw74uOWShyc9YoXEkVPtUyrk3TxQWNJLzhVkwGkemUPgPCw8G48iAhRx8huzjDOkXpxo4Du+/juKPvmAZ/TciF3/BorX6BphxJ5MVW0+5AAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  0  0⎤\n",
       "⎢       ⎥\n",
       "⎢0  1  0⎥\n",
       "⎢       ⎥\n",
       "⎣0  0  1⎦"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E21.inv() * E31.inv() * E32.inv() * E32 * E31 * E21"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We left-multiply both sides of (4) by these inverse elementary matrices as shown in (6)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ { \\left( { E }_{ 21 } \\right)  }^{ -1 }{ \\left( { E }_{ 31 } \\right)  }^{ -1 }{ \\left( { E }_{ 32 } \\right)  }^{ -1 }\\left( { E }_{ 32 } \\right) \\left( { E }_{ 31 } \\right) \\left( { E }_{ 21 } \\right) A={ \\left( { E }_{ 21 } \\right)  }^{ -1 }{ \\left( { E }_{ 31 } \\right)  }^{ -1 }{ \\left( { E }_{ 32 } \\right)  }^{ -1 }U \\tag{6} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The multiplication of these inverse elementary matrices results in a _l_ower triangular matrix, $L$, such that we have (1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGYAAABLCAMAAAC1MwK4AAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC8ElEQVRYCe2Z27arIAxFw0U5W0R7+P+P3QFLBVoS6GXsl/pQOmSZKQFxgSB8PCR85FiO6ADCK43H9BEKhNBa+oCZP0M4o+oSIxZz1jX+KeUsf1+lKscYaVfPYtYLgFk4TqXKMXj3jsXoJTRSbY2mXk/XqmHMFgfkhbmdWjWM8Tbc8OQxdcRRq0YxJgXQBATuVKOY2asQX3hHYe5UT2MirYW6YZJqFGOO1kx0a+5Uoxg4+ubSNQRO1TBmi0+M5gZ0pRrG6D10iF1b3XKcr1XDGFjCZLMLGlOrCozcdr9IcqgiQjknOUqtKjDMHb5Q/cU8kbxv0r5JKw3UE/nouuQvRtqslN14j9vjGYGwg8G0aOaF1ecZgbCDLhCMZ14lPZ4RKDs47SFhb8EwdhDg4kkHFsYV74CvjuE0jfVImxaW0oFh7KBwkn8xdmBuPi29ievWYNJod9yVtBumYQcxyM54o56+eWgH//mf0LMgopHY2OZ0D4EzM//POc342I63YGrTmPfNHldI70gaUHZQh7H8nueGtIOTtfzU2eUZv3YwjvbXfvKR9lok8uovhkzP48rBpBl9vMJT+ThoOFsqxjDCubgzlMo2pVKMYXAuipizbIOSMir+BqPwIJev6R5T+Uxr5gUdlLym5XGAFD6Vj1XhbKEokrYG23m8kVqXp4tT2dIRGG7LL4RM4VP5BGaN+yTtC9+E8avAvV9qRTCr1Vv0Xqls31KlyPoGTXp4S++sT2sHb9YUGHQ5uLm0s1vRzWjNigwDPg5l7ZtpOz4qZL/NsDfNVZFjljgtan5J0IzerMgxG9eaZhS2IsdM175hLtJOK+4TAS4uC1WOgQ0NvOFGmkOEYT4RoJ0vVQUGlLXs+iYutdn1aaUqMUy6YrXDnehog0lxpRrH4GLt2PImMZVqHCOkkMeeN8WpVOOY0C0Tu0FcqYYxcxxkipn4atUwBtYw5Ulu3qtUByZOQZ2fV2frHP94nqrb51UTv4B+9PMqPgG/ntsqZM/6uhwAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\3 & 1 & 0\\\\6 & \\frac{11}{8} & \\frac{1}{8}\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1   0     0 ⎤\n",
       "⎢            ⎥\n",
       "⎢3   1     0 ⎥\n",
       "⎢            ⎥\n",
       "⎣6  11/8  1/8⎦"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = E21.inv() * E31.inv() * E32.inv()\n",
    "L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check if this is so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A == L * U # Checking this with a Boolean question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUQAAABLCAMAAAA/FQ6VAAAAPFBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAo1xBWAAAAE3RSTlMAzUSZq1TvEGYy3Xa7iSJQQDB8rw0HegAAAAlwSFlzAAAOxAAADsQBlSsOGwAABuFJREFUeAHtnNcWrCgQRQ2Yw9wZ/v9fhyBQRVCkteGu1f3SiICHbZHBqtK/upHOiYof0Tfedszygfcek1nlPg8+va2iNtFtZL8DqS/ow378aSOh91LNrXJTuKDsrldoJ7rDG19yj7chZlbZjw6ZwfhBiJPfap3owmNv23qd/PewryfZTyB6ksPPg1dPqWxcWyO1fpCGOJC2pso+9e2gY+cGvtHLasCfbDJEf3JfUElW6yETyLuGyMIsNyASwbvvraR9l55kkyFmU7nTDuetNoZYpULsBXDiWjl+FL/KCPFBleuMctYBQ0yGSESahEbUihkhPqhyogukWEOmqZYoE5xjWtmMEJ9UucKqCyP9CCJOCr4o6M4N8SGVHawVCWo+PoI4txBWyJ0b4lMqqWlKBooa61iIQz2b31ERtlEMv9mwvKoSNAAjtMoquWFhZjfGMfwmRE9heE5lQzeV/opK8wcQO57kHjEcy1qcn1RJVdMyUNg2f2CJk3gtS8QYJyfER1WuqkM3GpsUthlbJypDVv/73LKf7Icpv8B/RojPqlwUOwJ72izXBiJZezoT1KMMUOHetZwWNA1WKKw32fRhnze50LMfVjmp9rm3uscGYlDJCzfSIb4gJpiko5JKeLtVJQJLDKb1wg1H3sUzCnnVtSzGHe4lwuJ8kY9Hb/+lEFs5fm5V3aiYFPKOlZzAfyEqRyqWCWrc1f5ZYuCtCW+nvEyyp9irro6KXMg7VnIC/4WoHGTLQvF45WeJgbcmvB1LrCifhN6PRtpELeQdG0FeVykqZ948d3YP59fF8b406elaYk3Z0umi+tw6ainvWAvyOkpRSXjnZrO7iT9L9L406elaYsv7OER2dEBE8473bWvXy3XkKn45HDyFOVE8Vx4ObF+VolIY4UrtiVQtb+B3rBlbOy/sOnbR3o6K4yVDzKxy49VhHYa48G744NSZNowbi/YoKo6XDDGzyoU3zLU96jN1YtPzonwJ8cZyOIKI4yVDzKxy5BDnMESR5Y434ae/G8vhKB0cLxmiSDOfypGP+2a8jM8k6TqRy2vmK4YiEzydw3HzT8X7CGJGlR0frPSOpQGI00JIxK4Qxi1uOdwFrON9ADGrSgGRnkFkmUar/C4E5RO3HK5Cm38d7wOILLV8KhtuidSeCcPFmZtqxPJdzKL9+TL6ZxDzqWR7xwcG8R9jF8L1h/4rfSZRkleHshWeXcYuh9sxQbz/btappaj8IyFaWxW1JbKpMm6DERCjl8MtiDBeqiXmVskskZfWEMSqFzPf18U5fjkcQ0TxMMTrGkQ3fxlV8uyIOnEONywj791c98BuLIcjiDgegng9TNLlpcqnUuTmCmLVsB0N1xMQ0Yv2CKG92I8gVvPlqQVtiRlV8gwJiGJSEWXPyEPeL19giFV3VaCLUMmYdHzEsp6PWF5GZ5K3IOotayYEdpUCUQz71ouxM5b+3hWGOFxuACoFopjFIeGpsPeIeVLGEK8H7KVAFPOJYnobZaoUeUiUc1GKypYvryy/NRbnBYU9cHnh4URJ7pxJ11LecTgr/E4pKkXDfGwmAYpLkQckeZylqBRbmY7NJEBmKfKAJI+zFJVyK5PYTAJVliIPanLdhag85j/EZhIoshB5UJLHXYjKozYk9gwElLexX9z6gMjnrSPwLAYI77Z7HnLAqxCVo2yXF7u3beTtM1szjTpTwXN38wi8FT4dYlaVmxzw6VME6jUbiDXfArGig38qkP/fcz7FH/DwBeHTIWZVqaas7Q2KGmIXs7qCIAEoyD90AcInQ8yrcj4YzdYHBzTEWh1cCzFw/AEU557PA4RPhphV5aAO921Wy6Ih0nralvZ6V5jBA6AYzxMXCJ8MMavKTu2pa5TjyK2CyObo+WRKb6/BxEE5CWVuPQAxr0qiV6isr7AYiGIVs9WfbjKZD7kAlFAQ5A/Cp1riMeLKpNIc6WvxIQwFsZK7uUfr/KSm4FmMB1B0MO04D58KMavKyRRicH6c51hDnMWB0dGqMjUUj+MU4nn4ZIg5VYq9kUe+cHnWEOXHc4KWeA7Fc9f1AtCTIeZUCXsGCyqxGmJz1Ilu5kM+AEooCPIH4ZMhZlTJPj8IsmOKNvPUEKuVhRkKb53ZkCqbSvwpnA2OTQzEamOfI4iff7h3BJ6NyuHB/mRLZKdIMqkckO2x3e3ALgFEYKxvOz+A+LY0kD5SuVhDPVA5geIMYr/uRPIinlbAq+7twyu9McUC5P0VEKHhScGdMc0fxPA7BOVl8PSfzcc9fxCjIBpgILg+a/GDCKhYTmOJjXdKYVK+mT8DbskOXeZWyVdOPL/m+FLaID4e/u2PlYsNrx5VIa/MKgfI8H9cOmtCztE0RwAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left ( \\left[\\begin{matrix}1 & -2 & 1\\\\3 & 2 & -2\\\\6 & -1 & -1\\end{matrix}\\right], \\quad \\left[\\begin{matrix}1 & -2 & 1\\\\3 & 2 & -2\\\\6 & -1 & -1\\end{matrix}\\right]\\right )$$"
      ],
      "text/plain": [
       "⎛⎡1  -2  1 ⎤  ⎡1  -2  1 ⎤⎞\n",
       "⎜⎢         ⎥  ⎢         ⎥⎟\n",
       "⎜⎢3  2   -2⎥, ⎢3  2   -2⎥⎟\n",
       "⎜⎢         ⎥  ⎢         ⎥⎟\n",
       "⎝⎣6  -1  -1⎦  ⎣6  -1  -1⎦⎠"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A, L * U # They are identical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Doing this in one go using sympy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `sympy` library provides the `.LUdecomposition()` method for rectangular matrices.  It returns three values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "L, U, _ = A.LUdecomposition()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGQAAABLCAMAAACxxtKFAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC40lEQVRYCe2Z3ZqjIAyG+VN2RbTD/V/sBhCnMCaBbn3mpB6UPjXNSwLCRxQypEuJG64l+xZCBm3gmm5giOjYqBAh8x3+v32aBiIX+30T+aa1d1y3KpsKYpVbAwtZH0LYhabUNhUEeu5ZiFligHpDwkw/NzbjkC3NwgfZmcZmHBJc7OwUIGno1dgMQ2xxYFCEaG2GIXPQ0bsMHoe0Nq9DEusadEIOm2GIzZFMVCStzTBE5DF5dAz8aTMO2dITYugpXNuMQ8weB8Kt18ORf21sxiFiicvKLilIY1ND1LaHRRGTM3q22ntFMxqbGkL27vWbH8hQ7j7p+qRrKANDxr8wu2at3cbL1Q4FSIi7KEQMuR3B+tijAAlx56N/G8itIg4GqwApcTftMVVvgDDiTohHIBRVjKMjkqLNDgH4Y3ZNC8tg08WIO+kVt+l1RHLqrmOP/REJpItSuV3pOiGIuAMnO6l2IoWbXVfi7k/4mzooZJIHGxsKB2kF4NfTcc6GFMMbII0ArMZkT+eb/0+XoMSdibP3Hc8JKe4m5/gFskcB1gKwSlce/vd/fiBDOf2k6850WZP35tL2scbGRHqfyjel7WOANhkrqj0SBBafo+3C/ApEw0UdOksEpUUjeRaAdSTzAppIUZkozkt7DWkEYA1Zo4TMO871v8+xoCHw5+fds4LQ5biELc5Li/SFgKypooH+L94ozkuLGqORhFVCNZZQ9bNegwMxVVoUgUcCUjvuvzuruwjf5y0sElAr0cbtbGn4dIV/wSAipMlrApawXNx/+sQZ+Oxa0upneFlPOC+30Eg2JpLioKdFIdMxJowT442mS/XgAIWIDWS45WaXB4AlS/WxkzhEaOfY80k6HjPnyloAVssKk6bjtofKcBK0feZg9QIEMpEL0HdCpJIqnzx7KS9EEodjYkq2NX4cMqeJpUcWuHGIWOPSpkbWtwwZepk5O+/5h/FI2Pky06Y3jje+zIQJ/w+ZWClKOSOG8wAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\3 & 1 & 0\\\\6 & \\frac{11}{8} & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1   0    0⎤\n",
       "⎢          ⎥\n",
       "⎢3   1    0⎥\n",
       "⎢          ⎥\n",
       "⎣6  11/8  1⎦"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & -2 & 1\\\\0 & 8 & -5\\\\0 & 0 & - \\frac{1}{8}\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  -2   1  ⎤\n",
       "⎢           ⎥\n",
       "⎢0  8    -5 ⎥\n",
       "⎢           ⎥\n",
       "⎣0  0   -1/8⎦"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "U # Note the difference from the U above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the subtle difference between this $U$ and the one calculated above.  It simply has row $3$ divided by $8$.  It makes no difference as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHkAAABLCAMAAABjltjdAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC4ElEQVRoBe2a63KEIAyFAW8tXsv7P2wB3TUHEeKs7nam8qeAyfkkYg2wQhpflHhX0TNQCGmK0pbqXWDhaKUyjly/DbqCyoAsdbNezNXqtu0GmbNy1yOyQG5U2xk+uXYzozDZpxSXBbIV6g+Qlb/JcXSDypSI7Cvk0d+l4sySk8lKu5Eqw3jSJ5PnCGs7R7PlCrI0fZYbnT6vPGdP1C0DHJu4R8hNp9eyPNyWBX6VHBlcyQOfT54Kezc143/v2TNMOrDoGf/1TibXurVlfqsjT4J2ZclqGI1WnNfEyXbzZ7ajiGg9KhvO7ajnJZ03+ZKw7oje0d4JzCXdd7QvCeuO6F+Jdl0U7ZDNnwU/w8fxgh+MuXHf+dJMaL9pcTP80BH9gNw7aGNyH58DGT7A0Q/I1eginSUfyPCBjH5A9naTKcF+2ziQ4YMz+m3Ilc6BFzVWhg/kufHwC8iyV4qxWLEavAx/S376BWRrOWXntpfjZfhb8tNvSxbzRNj6QA8nw0+vDBz5y3zPqtIHemAMmpvhw93aBvH7IbsVjfELYgaZneEHZOoH0R79xlQ+2vwMH8ngB+TSvVD59/lAhg9k9AOyqOyaIf/FYGf4wA1XBkgOTC9t3uRLwxuI39EOAnJp8472peENxP9OtAtbeOmQH0Jkrz4YGjaJPY651jbtZW0zOcH4Xj2iaAvtkdy5RcYwUPN0PbLPlXQg9kCeDhwlzACilCQ+LhJ7IHecI4mHiP9LlKB/r0HsgWw6WfRtfi256hKltTNRI/aUbNdyLh0ac2tJokyUSO9+ldgj2R9JtCNj83YRJ0r7OHKF2FOyMP4wptw7+4pk7kSJ6C/VtD2QtV85l9nF5ApJklezZ43YA3lIj/npv1aI0tqZqBF7IFfLc064BpeIUnAl3iT2QBaDPR5oPjC37W0W9nSA/8GI7tXHB+t7wR7HnPA6/dJNPj2kCcH/G+1P/ZKl8b8sefcvWewGxS9L5ybTY1kkkAAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & -2 & 1\\\\3 & 2 & -2\\\\6 & -1 & -1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  -2  1 ⎤\n",
       "⎢         ⎥\n",
       "⎢3  2   -2⎥\n",
       "⎢         ⎥\n",
       "⎣6  -1  -1⎦"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L * U # Back to A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What's special about L?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our methods only works when no row interchange happens.  It also actually only works when doing the conventional subtracting the scalar multiplication of a row from another row, leaving the positive scalar.  (This is opposed to the negatives I often use in my head, allowing me to add the two rows)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the $3$ (in row $2$, column $1$) and the $6$ (in row $3$, column $1$).  They are the row multiplications we have to do for `E21` and `E31`.  The \n",
    "$\\frac{11}{8}$ is what we did for `E32` (we just did it in two steps so as not to use fractions)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Row exchanges"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, we have to allow row exchanges, i.e. if the pivot contains a $0$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an example, from a $ 3 \\times 3 $ identity matrix we could have the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFgAAABLCAMAAADDCbAzAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACN0lEQVRYCe2Z0XKDIBBFUUlsNWrK/39sAQvuRZYdUpuZzuBDQ7zZ47qhk9NGdcYfvbromHaeUp0ZtD1uF3GVg+neOPD9KubB0QjupvHIzqthWOZCEzSm4LGfH6YEfjyVGieWDDEF2w6XElhP7h6G9Xwn/gzGNeDV75snd22Ma8Bmdp3djB1I7sC4AjyGSp3DqiSuAN/N4IidWbLgJH4F7PkneATvcQV43Du+MR0ncQVY7TN+lt+8ENeAV7+DNbvdIK4B680Ndn6cxrufwLgGrCb3K711DBhjAPfrZqY+v5scbRyWpWe5GAOYa+WV8w0cp9ZG0UYRJxAXb9oV1DjitemiLDS0HDoG46C8fS0JDZRTMBrHGWzPlIQGyykYjaMajOUUHLyBERJ3pVLHWE7AiXHUdpyUE3AUA/4jpNRxUp4D54XE30FhFBF8EpbEOOpH4YnBZ1zHH+bTUwQhca8pdJz4zBf5U0EQEgmM5WTGCo3D30T6o9QxllMwGkfK9M9LYCwHsCAkShAaKAdwtskXTzZwHFwbRRtFnEBcvGlXUOOI1yYLKSc+Ax2DcRBeWJZz9BkKRuMItONRyuEDhoLROA5gWEk5C0bjCLjjUco5cGIcB/BnJeX2ZeQDhowiigEjLFIugxlhiWAmZ8GSsEg5C06Mw74uOWShyc9YoXEkVPtUyrk3TxQWNJLzhVkwGkemUPgPCw8G48iAhRx8huzjDOkXpxo4Du+/juKPvmAZ/TciF3/BorX6BphxJ5MVW0+5AAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  0  0⎤\n",
       "⎢       ⎥\n",
       "⎢0  1  0⎥\n",
       "⎢       ⎥\n",
       "⎣0  0  1⎦"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eye(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exchanging rows $1$ and $2$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFgAAABLCAMAAADDCbAzAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAwdpnNVN2773xf50j7AAAACXBIWXMAAA7EAAAOxAGVKw4bAAACNElEQVRYCe2Z4XKCQAyEA5ylRUV77/+w5dDksieXDJQ60xn4cxmXfIb11FWoifPR0k7H+OARNbEL03HaiUsJFtqYwB97MTMnALjrhrP5JM3Y596XSrcD+HIl6scquW/Pl2iAoV2Dw5hm6G4vk+QHBgOM7Rp8m/fF1eglC4ztGhzPabRTnAypHRYY2xW4ZyXUsGRNXLQr8EfsErGJwyZw0b4EnvnLcMMKAT/aFbh/THzaOHHRrsD08Pj6uxeP2zX4Nu/gsHm7QbsGh3sy9nxZtnd+1PCYsF2DaUxv6XuzEYztAO67YWgNbnu7x7Gt7kZoB7Ax6mrpAItlhxWHFeKAFO/aFXYgIZ1IZLZcaFlP7AUSgkSSeVyBrMHTCdbnLWEiYZysKK8BYyIRIBcorwFz7qgEGpRXgItEwoPyWsgrwBIclr9CCnkLeDnQCPglsKRrsnZFkUjYAl4LOU38Gb9YtcBeoMG88w0/FcyJyQk0KK/wuEgkfJGyGoHF9pgwkQhQCsg7MLETSAgSifCkABnAcsoOxQEWEw8rDivEASnetCt04pDnVoWnq7wDE0PiUDwubR3zjgZj4mBaXj0dPnU1GBNHBnLl6VUwJg7G5dXTa+AicWTgs/L06TT1XayskGCwHEjI033wciDJ4IpeBReJ42lAXjy9CvYCiavXPPYCiatXwZg4sgdceXoV7AUSV69st+lvG+cfFkeHvKP2MV/xPusBFh//qxV/dIOln++I7HyDJQT6ATUqJvO2xTVAAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}0 & 1 & 0\\\\1 & 0 & 0\\\\0 & 0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0  1  0⎤\n",
       "⎢       ⎥\n",
       "⎢1  0  0⎥\n",
       "⎢       ⎥\n",
       "⎣0  0  1⎦"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix([[0, 1, 0], [1, 0, 0], [0, 0, 1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUQAAABLCAMAAAA/FQ6VAAAAPFBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAo1xBWAAAAE3RSTlMAzUSZq1TvEGYy3Xa7iSJQQDB8rw0HegAAAAlwSFlzAAAOxAAADsQBlSsOGwAAB9FJREFUeAHtXNm24ygMtOPdiTM94///12ExUBLCJumbhj4neQkIIYpiNcJuGv9rbza47ObX+4RPB1Zb4GvFFEb5XCcJb+dYW/ZtVr+DUkn1h2W6tLnfX7NaGuXm6ELYw+ioXfYnJvyh8PwyiYVRjnPEzBRkSOIi99oouxE8u659LHIalQpm3yfxuW3d43rc5KOjWEk+QHmL+1rf+qyexKnv2t31T5+cDDx1B9/2y+rIZgFesgRMCCg7JZ73AROFcC46npXmQ5T9g+kuUHcPT+ncXyCxN3yPIzMtRQWzCE/KwmUe5V3zN+2hE3BNG38BHTFA8yHKJ2+4FjB4eMqYUFtSBEZGQ3gf93JUsmHBLMKLM8QSj/I26r5/SeIL6EhhNB9B+ViJ5gAdsfHwlIpQW5IRI72x2e8Zs6JglsBDs4kwomyaYY+neZrxBXQkI81HUC77HVVb5BThCbXFfEJ4zVllBbMEnmCXixBlc1uvODyyZ6HjRam4y0dRPnDqopQiPKG2QhEgoqYggQQFsxQe0RYjgHK5931G91dm8tDF5fl8FOWAs2JPlg+A99JwNmWveq28/P0siao4Up108Xno4vw+HyURZ+JpJ4t1LolTu4bf0RO6LA6ltmHw4oowCaJUSXb+Zzo8moPuvFYMJSwAbJOF8IQuw5FhfM7j8EdJXEz7PXBkISQI56KDLCYI+RiJt31zyg8ymt9endWg0iafGY9jQtsweA5b8t819bQb9BkkZqNjZWI+jnJ3S8u049qsZ99Ag1BbVgREF9Ms94xnHMEshwd2xaBHOZqTgOvhnI+OlkfycZQPt6GbQ5802T08FRNqS4uA2HPt1M/uq0AsBQWzHJ6UDWUe5ax3N9f7xBfQYTENzcdR3h13Pe60lQEPr+kf4772ZEdJCqCR1h4LwrMPTfcx0SyH57UTgYDyppru+gAiGx0rj+bjKBf3pDSy7XGAx+x9NMrhXRVWCcrdkvdkUyL0xKuK/GT6X0pia4fxQHeJOJx/kqQrW38piZ19fu7c3OiqWclAcXAS/5WgnHezOWj5NrUSeAnynLgSlIvdKY5uq1MZPAcn8V8JiWqvrwHaHT9ArQQeIJKCtaDc9aPJ81ikA9Ba4AVEUqgWlKtenge+w/lucaQ2O2TxHqLVZ+p3t+f2WWtpYw9IDNSCstebm41vE789UWw0K4x7Yqf3OL3d6EDG0MaF3OKAJR0MKNXjQcErBqYTPnZ+kOrhTTqFndgKtaLubUEhIaL54jZOZDvEAWXZKwabng7bNInF3OLn9NlUT6KKCidrSRPUGZ9U0wmC2bip73phbvlTX5gTy7nFT+tmE98lkTrjTwvKInHWJK5pEk0J18ed1L19Cosk0nxxGxPlKPIuibTUyCwKMklUHoKVuvGVEYRXyi2OdZHDiFKorZzJS50z3gukgGA2bupBP6yM0QUMgFfMLS7VicoApTR5UWUe8854nkDi+STuZyQqm4Xc4qQ2UuS3SPTOeMmyl2WReNM9cecnYXQ4l3KL+5qkArkknjvjU9aNPItEdXd8UiT+wyz92v+1kj/qFv/POisYlnTUo1QqQm3TGdXWl++ME8qC2RjlL0siv2Pq2risWzxRMS92KLVAqK3XiwPojI9TQSKYjRcW1RP1aE2R2BR0i19fAHibROKMB8riYBaJZk5c0wtLObf49e1hMnMLtY0pOSTUGZ9U0wmC2bgnXpHYlHOLr5dHCqEnincBkuxQZ3xSTZ3MSDcXEiSaQ0ViK8Aj4g9HGLzhakBXgVJxMmhP1eP8ieXD1AXzjER/ZS1o0FAtJM6WRI63BnjT5QWgGlDqZjWnOH36KIw2/YdjtCde32OvhURznmiOtwlBtcAjoKJILSg77V65f30sUQOlBXS8aD0zkoevty/NWZQSk2gW5uMyCajXMlAAkhCsBaW5ynRcJgGYtcADSEKwFpT2KpO5TIIoa4GHmOJwJSiPUxpzmQRBVgIPIQnhSlAes2HPTyAQ3qZ+ea/NmXq+5EVXOUA/nrIF5kBUCcrZrst3vtsO8J6ruvGU9U6Frp38Rj3UmwWp/vskFkW5WWepf4vA1TGQ2Ooj4Ad58c8pyf/C4ZGseEhB/30Si6J0b3LxC4qexIG+rnZKh00EUjK0yYnd2ySWRbkeHK3wGpquuiexdS+uZRGilUqQWBTl5F7u29jK4knc22W7d5dfFwGKS5BYFOXg7tTdXOBgw5Gozuj1YcrIfTBAGg8WILEsyt57qNhXWAKJxovZ+U83ccrieBESS6IMr/R19CUMR2Jjb3PP7P1JT53gFj8l8Vz/7YWlJMolDGJ4f1wz5ElczQujM5syPYlC4JTEc/23SSyJ0tzgPOpFx7Mn0X48J9kTz0kRUmMRkP42iSVR4s7gTkasJ/F2zDZx5VMSICWlQuSg/zaJBVGqzw9CdcLQVkJPYvNQOlPlq7N6pCqGkn4KZ8Nnk0Bis6nPEeSfP4jubmgqHiT6b/dE9RZJIZQT6Xvqo2/QL4FEXu0Pxn+DxA+i4qYJyjt71IPJCYYzN/HJOIGXUVAFTT3yK3pj6IoVwPsrSMSOZwEPoWt+SUy3IYyXSdg/h497fknMIjEQBur+E4RfEoEVFgw98SYeKSxOWvgz4Ax2KloapfacCL/b8YmwyXw8/E9/rNxcyxVQpUSFUU7I4f/CwmtCeNpNOAAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left ( \\left[\\begin{matrix}1 & -2 & 1\\\\3 & 2 & -2\\\\6 & -1 & -1\\end{matrix}\\right], \\quad \\left[\\begin{matrix}3 & 2 & -2\\\\1 & -2 & 1\\\\6 & -1 & -1\\end{matrix}\\right]\\right )$$"
      ],
      "text/plain": [
       "⎛⎡1  -2  1 ⎤  ⎡3  2   -2⎤⎞\n",
       "⎜⎢         ⎥  ⎢         ⎥⎟\n",
       "⎜⎢3  2   -2⎥, ⎢1  -2  1 ⎥⎟\n",
       "⎜⎢         ⎥  ⎢         ⎥⎟\n",
       "⎝⎣6  -1  -1⎦  ⎣6  -1  -1⎦⎠"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A, Matrix([[0, 1, 0], [1, 0, 0], [0, 0, 1]]) * A # Showing row exchange"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By the way, how many permutations of row exchanges are there?  This answer is $n!$, where $n$ is the number of rows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example problem 01"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Perform LU decomposition of:\n",
    "$$  \\begin{bmatrix} 1 & 0 & 1 \\\\ a & a & a \\\\ b & b & a \\end{bmatrix} $$\n",
    "\n",
    "2. For which values of $a$ and $b$ does $L$ and $U$ exist?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "a, b = symbols('a b')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFoAAABLCAMAAADH/GAOAAAAOVBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACXHtMAAAAEnRSTlMAMquJZkQiEEAw3ZnNdrvvVHxhCJaFAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACGElEQVRYCe2Z4Y6EIAyEK6DcgajH+z/sQc1G27UQzHrJJfhrcZbPOtsNI8IQ8VDwscPtRIAhapOO8WNkyDijYkZPn6MeJMPQg7OH+P5J6+BLdZynE7RVfo4l9LwBWCex2XSCTlWGEtq4fB96eb+b15nz9Cb0gk20Fa5+Gx19rm6MyRbhuIu2L7QRwNTPFkOmqDNziOE5NF7hEn/fEGSOD1QNu9fbAz8jLNjR5onmM2s22M+XNuPJu14DuPxHX4c7aLWs0Sm5t6wOQclkOp33tVxQs9LRzLJuSDeEOcCGvUO6IcwBNqQdMnkdJnklgCadoMc1hdBFXlXb9DParjlxeYyj7OZw2Kif0R7rdWI2atQz+it+Y1Vrjhk2is80jfrP8cBhMS3ma10frfrJEIuBa5lhu36iaNVPaJhT2xnnQTK7UT+jrQp6mFSQMkyjfkZfW3z7bEcz67oh3RDmABv+WYfYZRXXmFRTRa5UPeO+BPvSMazIxxfTJ25I4SE5z6vIJfQoL+d5WkUmZF61Llmdtp3KchE9qxC8vP9TkWkEol7bONu0pl+vjXm1L8pAIxBF75th4lZKRWYRiKJxP2IPBdS3fVSTaUSiaGxbebOtIrMIRNERQ5/YgGWZRyCGTqFsd+zKD8iZTZZ5BKJolape5L96RWYRiKKtL3U11GQakSj60oa7JzuaOfefDXnslY/FNzQff+VjDPwCPuYkDmyDunMAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0 & 1\\\\a & a & a\\\\b & b & a\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  0  1⎤\n",
       "⎢       ⎥\n",
       "⎢a  a  a⎥\n",
       "⎢       ⎥\n",
       "⎣b  b  a⎦"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = Matrix([[1, 0, 1], [a, a, a], [b, b, a]])\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "L,U, _ = A.LUdecomposition()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAABLCAMAAAAf+VFoAAAAPFBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAo1xBWAAAAE3RSTlMAzUSZq1TvEGYy3Xa7iSJQQDB8rw0HegAAAAlwSFlzAAAOxAAADsQBlSsOGwAACTZJREFUeAHtXOmytCYQ1XHfJl/i+79rWAR6ARpmtCqpmvlxL9ILx0ODSKtN43/tyxaX0/x6L7ijsFqnda4eQSJAwEDf6xTT7xw5y3nM6ncxF1P9oE57nPuzzvIRJAIEAvRwrECzYXQMLucbCm4sz9VcPYVEOCkAdJyZ7hTqIFdLPAa9+XFsXfZ8kAKA4D3kChBJTu92GQD64pHTt75Bj3Dqu/Z00ebFsNAOTTOtGbKwAoAAvSTLHonSyPca6pKkPyjI+oNA+x2aaSRnmJ4gwi3L1bxqNwd1FnwTBQghKKVLHonUa7hL0g6dRPIHgb5PFRDw14awUryFQMlztZuJb0jzSRQgBNh6qlyKhHRJyh2qz54ZArqbiPDGAwirCq7OTnt4UeK924YoIAhBK1kq5Yp0SdIfFJRztZwbNGwhdaUIm8lRwS8V1jlVeIorh4MMFniGrFzOVbOPwBozV8zV+zy0E2wN3DZU4SGuaJdACMlyBVcDHDg9mnHquTKURVB5ri6Fh7jyzaCxEsEDqyq4as4wm08nupIVczXZuHrh8QwAUYWnuUr1GYDkizVc9efi7GYYY3pMlV4H7TyBQtT5tP+JwkNc0S7BGBJHNVy9bFBoTzsaghVc7SYeZ2wOsRGFh7i6LreZPoOYrnINV83pZvfphFfBmriajYsujGaKiSg8xRXpEooielzF1e4G4RwizHgtH4PNqu9xRj+YOSis8BRXpEs4jEhNFVebo6iHC1HlNXDV7+O59pmry3RsW5+hqsEKT3El9xknq4qrxV0JR7JTErjiLXxX8zlXQq/hLikAKfhjQE/L0ZtMVyCuChqtUmEQBOvnek1omAFt7dgb8OoKjkHBY7WYQRA8/He46uwasnPzlgP+HML/L1fzabZUWrwS/cWVihnWqYtdYY1u7fCLK8cA52qyk/tJl92/Mci5ak594/e+Loee0991MDIGm1VfCAe6ZPhxFeOqPdWG5uaWpD6wfmMwMgZ7vVo46PLqF1exuOr0oqG3KwcfVZVrhmyaTTlFiTt2KQatxorPRXisNVDHgZqQ2u1+HFAsRyil2ZoGJ+44BNBspAiQIM4jqrVVeX8c6KGnqvYLrhTA7O16QxJ3HEL+FANXmPO8VYlU8MeBbvoS2NJbnLr5Ks8VSdxxCPnz8lwRzvNWBVLJHwc6a67WJ7myMeuTrRxC/rw8V4TzvFWBVPLHgc76JmdlWRiPUC1Uu2N7Z/b6hDFIE3ccQv68PBLCed6qQCr540AHvWQf9SIL/TzC5jWqhT3JXCBVgSuauOMQiDdy6JBQzomaOpQ7FdqI/jhQw9WZ5so+k9XhzAVsU5Wz85Xn6stcqveTCvGCToW4RX+cq5eOq5NuyYS5vTN31Ws2Q5nliibuOAR4Brzs4sqfWwJKSadC75K/yLpdPbo6Ka7+gm5U+c/5t60ZdfpvIpkLopzliibu/iE7+8QXO3RIKOdUsaRToY3kr+FA/1iu6NMlrjetRyEW8lyRxJ3gC56OKTsklHOqWNKpyMbO7enkKweq4krP7WmutGRvmyHk7FGL+iDPFUnccQjMH6rwXBHOkZIOfT02a5zn/cV8mflqTc/trZpK57VrErOEQZzniiTu8Olkn0Q1zj1XhHPGVUGnIpu8v0+4mvrtWN79lk6WCmk21eUo2Yq4mthmEDodfeC5IpxTxZJOxTY4H45l6ggBNVITV2YTCykHhKj6hgMMYRWeB4dcYc4pFLlTmQXqQyqNcDXodfueW7czJ99VYK6aQRqFz/WacB4EqNI29zj7d/eDQqNYTCDk5kFj+B/iyuwz9Pb6CU7qOYSYqym1CvdYnkPim4gXMFCtY/avzOYosngOIYZA70MRCHPwHBLeFqrBQLWo01vt22+/HdFkDjhXZvgN7NL9XG9yCBwmrHkOCWwlUuZAzSXwytQDg+cQcgig2UjxOSSRxmAVB2qe+rgy9UCzBuHSose9gZdYkUOIaYW6GiTB6oYSB2qf+jCZeui/CmEvXviBaw4BCCPFKiQR+4+rGFAVUXoxaDL10GsVwtyDtdCpKTMITANXVCHBpt8dMaDXTNXTm+cahOpdnK2Xlt8eN4PgJfECQJLP5+GUbdyZUDvtY9gAZkBnewXc6GI0IET20bY2dU+3mTfjomJVCRPTDELK6KoPSIR8niAWmrnEbTgPBvSwdzf+eWTnMSBU2cNg78To/663bdLvApDENIOAfPEDj0TI5wli7jhaAzbyGND9EtIHsDxC5RHYR/2P6qkk/7GCqAbc4GIQohah0iMR8nmCODjMlV7gmT0GdL2EK3hXSTvzCPUrp8JcpOUj3VhFiG7hSsjnCWKEJ3lwhOmK7clM7oWcg0zugCtoH21ETVdzfoV1B1dCPk8QK+Al6cO237bu6nYaV4N7mOjlChcbgCtoH+Vq6TZhgXUHVz5HFd+ZEMRqeMg5YbVp307qj00tUK56PxeN7o0vS0fgCtlHuZIrb+Uq3i+eq7hYvV2ltzTyOWH1LKiebq6NT8pVeA2nw09tB66QvUxLVOMOroR8niBW+ymahpATnto1/FwywbzZd3mi89USRh547VKfbuAK2UeYsN8e0n8jwqvqDq6k/KCd25PpvqL0oVkduSd6SFxtfgiqKxkahIErZJ+mIyu5hSshn5cXl6UPzQXORiDL47SAnw3l4QNXyD7LSFp4C1dCPi8vnkxUCDnhRlPgP0CE40p9WgqcXxiPqhJwBe2Bek3xFq6E/KAgLkof9mr+391dCuYKf5LhgGvOwBWyryEI6N7DVT4/SFK2oHVTLEofTp1fXZExOKFIup4IuNoIXCF7iqDoGCemcXfJDgISWfdWDQR0I/c1sPdrEE7dvB2ZR0PwGSAIWBQ9qkESdfBpJQI6upHpvI1h+qpAOK2Lmt6Ee0bXBAntUJ0sVSBJ+vhIALmCYWSdDSHQKhB2ivIFXFAFYBCCoGrEFUhK3JXrAKATuVvWTsKn1ioQ6gvtFvtSYhwWgBBXILUVSIjll4cAaOAF+FxdHrgcobnV3OepdBACCKDhdLEcSdrHR5IA9OU/jwkdLa624guo6v5dfenPkQy90TL+xCmVJo4rkCQ81FcToKsaOZHf67qeTeZzpEXfYn1187AdJVxZnyWaAFkFEmD1XREDnSBV/wKJk29QbyeWvAAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$$\\left ( \\left[\\begin{matrix}1 & 0 & 0\\\\a & 1 & 0\\\\b & \\frac{b}{a} & 1\\end{matrix}\\right], \\quad \\left[\\begin{matrix}1 & 0 & 1\\\\0 & a & 0\\\\0 & 0 & a - b\\end{matrix}\\right]\\right )$$"
      ],
      "text/plain": [
       "⎛⎡1  0  0⎤               ⎞\n",
       "⎜⎢       ⎥  ⎡1  0    1  ⎤⎟\n",
       "⎜⎢a  1  0⎥  ⎢           ⎥⎟\n",
       "⎜⎢       ⎥, ⎢0  a    0  ⎥⎟\n",
       "⎜⎢   b   ⎥  ⎢           ⎥⎟\n",
       "⎜⎢b  ─  1⎥  ⎣0  0  a - b⎦⎟\n",
       "⎝⎣   a   ⎦               ⎠"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L, U"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L * U == A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For existence it is clear that $ a \\ne 0 $.  Not only for the division by $0$, but because we will have a row of $0$'s and a $0$ in the pivot position, row $2$, column $2$.  Furthermore, $a \\ne b$, for the same reasons."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hints and tips"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARQAAABLCAMAAACGAYjdAAAAPFBMVEX///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAo1xBWAAAAE3RSTlMAzUSZq1TvEGYy3Xa7iSJQQDB8rw0HegAAAAlwSFlzAAAOxAAADsQBlSsOGwAACCxJREFUeAHtnG23oygMgG21arV2Ztf//19X0IS8QGIdvWfO6d4PVyBp8hgBkdhWFf7d7lvxOce/BiUnF4bV/mdWT4fiFO+hy/HUGITnPPXLH8Qop/xHbcF438yf2TgdSlBMePqE69FiqJ7zmwguKvYfB+UKKELR9upMO9KWC8oz372EnWkaa4OdigmOMJKvUigbhnrJ2aJyQnHXXaG5pc9T/7G1a+rbjB0pKcrS7VFV3VCMChMTHGkmW0coD4Z5yZhickrRvIT2cyYzCPonSuOOoPRD+MAkbYMVLqY4oGEdKZQFw71oi1xOKd7zck3p3410lIr6ByWLA3Reca56lMLHxRQHDFhHCmXBcC/aIpczile8qPiRB+0oh4My18HgXQYcvHAxwwEV47g3KNyLNsjljOI5j1T/xmJE/YOWdXE2nQ786Wk8aAgxwwEvxpFCGTDCizIo5Jzi1RJ9ESLqH7QMDlB5z1MoCmMgrYSY46BWsUChDBjhRZkTck7xoJ284dMA9Q9WDQ5QQX8xNtCKRyHmOKhVLFAoA0Z4UeaEXFDMaWrtZn7DoP7BqsEBKt3aU+58ZIJ0GT4xWCAWOKhWKlAoA0Z4UdaEXFA08xM+0dNeszRS/6BjcIBKtc4prA+ibClwscChitkyhbJguBdtissFxX29cOFTLz56DgflFTtcL6whFxcLHNQqFfYGhXvR1rhcUsww1XYzu/cc7yl9tFinccmRuFjicF1d2xsU7kXb4XJJ8YLx06c+s9qg/sGq1WNBpxrCMr/FYYntW4GJJY5UlnUKZcIwL9LKUmdySTFCLBq2cls+Rv2vVptXOw8NW9pk3FXdNI5NMSZcLHFy9mhbgnJgHAib4gn3n1Y+xCf/lOrk8vGgnAmiKOY1GG85pWR6ypkcmy2F4/i45kopits6bB5ilZIbPg7wEbHCcYz8UFDqdZVVw9yCVNf4R/Nr4S8NSj/HR/2bWLp9d095riuVFm7NeCm/uad060w7qxXoNwelmsNW7Xu7CWE/+e7hUw3h9vNQd+SvviVXt3nZIxthDZe6ylcPnybcjCe1TPnunlKHe3Kz3phTPzHmlPc01S+SC6EfWsv701TH1ykuhQ1RFZJh28nETvJad13o+ZWGTxd26+WOFPngR2mqw0FxKDyIqpgMW89kCtPJbX9QxrDM6/QcROJiPdSX01DEQLGIV8qnsCAqj2IMN56bWuUX55R7G4bO4aAYaahiKJIAg+JTmEHxKPoQlGF/UCLhI9yyin8Wz9olIVd2ePj4FBbEtlNcpujDOn/Qm+94UTLnfh+smCw3+GIe3kxDZTyJJgZlUxgQfkruERazrb7yzD9je46NsbO2qBo8mHFZN/D+oKd4FAaEn5KLQZk/Ccpy2sUMRoyewYNBWXNlfxAUj8KASEEpUdxDT5nVzgFOtN1tSH+49dqWR4jZU+w0VAyp9U90X4vCCopLsbxb1y1B+SVZfs//yKZYf8bAvHQUk7bFw9NQ/8qd4WQkW0Ion8KCECk5TfF7DYp4VaW4ol22GsIsejgodhoqGwnSCD1lB4UZFI9i6Slhot0blKqNO3VWx7Um2spMQxVvWhAXCMoOCjMoJsXiLM4pw/6Jtg934+PrFCsNZa+TQ2AwKD6FGRSLIvj5NCjVva7NB8JP0lTi7jN4b19iUDwKB8JOhm1BiZsqIUTpL/lPbaeXRFCqhzOAroGSFGEkLCva12cr2tOCI3HWdUPZ/E8FJS7zXx8++5SxP5OIoHReovqnghKfkpv9WwefnbWjLYJiPlEFUz8VlLifErff+Alc45/7WDarRINTvQZKU9Rhe3b8f4+WXY44ch560+iai8Jch21N0eBUr4HSFPHGsyVPKdI1/qmHpaxxhIKoXgOlKWJqfUueUoRr/FMPS1njCAVRvQZKU6yp9Zg8ZQTX+Gcu/tqgbM+bMXnKiL85KNts0qgnwkxQaAKJBZBXPDWSptIdl5uSNQLleHHEy5InPWcpin6974xq9Ub8b2gsgSRxU91W42kqhZPMZEsJyvbCk13alEMxrQt8fEsSDST/WxNPIKGeLPhq5KH+cFAcL444MFsUsIWmXlBRQeEJJBkLrPtqFg6ayRcQyvHiiINxi2LYdqAH+TVL9A94PI0Freroq1k4yhxvQCjHiyMORg2KDl7On+RMi/43LJHG4rCptkPNwEl28iWAcrw44mjboHjAOxh3KAAN+Ic6Zmzsx/sdagYOOCsdAcrx4oijdYOiwR3rFr7PsQGBf+BDR/ZG0A41AweclY4A5XhxxNG6QZFeya/FS6PgH/hEAgma5XGHmoEjrck6QDleHHG0WqZ4pkFDvg8VPwT+kWudvOyE6aLsq5Vx0FepgFCOF0cczJcp4rsvG4EYP+gfAHkCCVrV0Vcr4yhjsgGhHC+OOJgtU9zIRDLyb/ygfwDjCSRoVUdfrYyjjMkGhHK8OOJgtkix/BwJcZuGUmhE/6jBvk2FrargqhVxlCnVkKAcL454MVyk4F9FntibBMk/oHnfttr0HDWWpjq8zOfZLCBMRweiKlN0vG9sc/ZmWQcluTytdDwopyEshhjFKJb2pEflhs+ZHJsthrPD/jVXilG08QcsCEtLpphr/BNnochwhCxXvQaKUrCOEREepOtc41+cKcURomz1GihC0clHwAWD/PjONf7FqRIcIclXr4EiFCQAiSC9C3qN/+QplgiOkOSr10Alinv69TIC8MTW039njngJRf6zc0JYqp4OJSiG/Bcx7vCzW138XbiLf/wuvpBUikCm/XSo9SQhsd/RmPwHBvFuj8/Mc1YAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$$\\left ( \\left[\\begin{matrix}1 & 0 & 0\\\\-3 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right], \\quad \\left[\\begin{matrix}1 & 0 & 0\\\\3 & 1 & 0\\\\0 & 0 & 1\\end{matrix}\\right]\\right )$$"
      ],
      "text/plain": [
       "⎛⎡1   0  0⎤  ⎡1  0  0⎤⎞\n",
       "⎜⎢        ⎥  ⎢       ⎥⎟\n",
       "⎜⎢-3  1  0⎥, ⎢3  1  0⎥⎟\n",
       "⎜⎢        ⎥  ⎢       ⎥⎟\n",
       "⎝⎣0   0  1⎦  ⎣0  0  1⎦⎠"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E21, E21.inv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To take the inverse of an elementary matrix, simply change the sign of the off-diagonal elements and multiply each element by 1 over the determinant (more about the determinant later).  The determinant is easy to do for these $ n=3 $ square matrices, since the top row is $ \\left( 1,0,0 \\right) $."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\begin{pmatrix}\\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & -11 & 8\\end{matrix}\\right], & \\left[\\begin{matrix}1 & 0 & 0\\\\0 & 1 & 0\\\\0 & \\frac{11}{8} & \\frac{1}{8}\\end{matrix}\\right]\\end{pmatrix}$$"
      ],
      "text/plain": [
       "⎛⎡1   0   0⎤, ⎡1   0     0 ⎤⎞\n",
       "⎜⎢         ⎥  ⎢            ⎥⎟\n",
       "⎜⎢0   1   0⎥  ⎢0   1     0 ⎥⎟\n",
       "⎜⎢         ⎥  ⎢            ⎥⎟\n",
       "⎝⎣0  -11  8⎦  ⎣0  11/8  1/8⎦⎠"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E32, E32.inv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By keeping track of the elementary matrices it is easy to get $L$ and $U$.  It is also easy to get the inverses of $L$ and $U$.  This means it is easy to calculate the values of a column vector $ \\mathbf{x} $ when we have $ A \\mathbf{x} = \\mathbf{b} $ as shown in (7)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{align} Ax &= LUx=b\\\\ Ux &= { L }^{ -1 }b\\\\ x &= { U }^{ -1 }{ L }^{ -1 }b \\end{align} \\tag{7} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}