I_14_Exam_review.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This notebook is part of *Exam review 1* in the OCW MIT course 18.06 by Prof Gilbert Strang [1]\n",
    "+ Created by me, Dr Juan H Klopper\n",
    "    + Head of Acute Care Surgery\n",
    "    + Groote Schuur Hospital\n",
    "    + University Cape Town\n",
    "    + <a href=\"mailto:juan.klopper@uct.ac.za\">Email me with your thoughts, comments, suggestions and corrections</a> \n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/InteractiveResource\" property=\"dct:title\" rel=\"dct:type\">Linear Algebra OCW MIT18.06</span> <span xmlns:cc=\"http://creativecommons.org/ns#\" property=\"cc:attributionName\">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>.\n",
    "\n",
    "+ [1] <a href=\"http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm\">OCW MIT 18.06</a>\n",
    "+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: #ddd;\n",
       "    color: #000000;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(200,100,0);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML, Image\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sympy import init_printing, symbols, Matrix\n",
    "from warnings import filterwarnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "init_printing(use_latex='mathjax')\n",
    "filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exam review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Consider three non-zero vectors in &#8477;<sup>7</sup>\n",
    "    * What is the dimension of the subspace that they can span?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* One, two, or three\n",
    "* They can't span a subspace of higher dimension as there are only three vectors\n",
    "* Zero cannot be an answer, because they are all non-zero vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Part 1 \n",
    "    * Consider a 5&#215;3 matrix in echelon form with three pivots\n",
    "        * What's the nullspace\n",
    "*  Part 2 \n",
    "    * Consider a 10&#215;3 matrix of form the form below and calculate its rank and echelon form\n",
    "$$ \\begin{bmatrix} R \\\\ 2R \\end{bmatrix} $$\n",
    "* Part 3\n",
    "    * Give the row-reduced form of the matrix\n",
    "$$ \\begin{bmatrix} U & U \\\\ U & 0 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Part 1\n",
    "    * The nullspace can only be the zero vector\n",
    "$$ \\begin{bmatrix} 0 \\\\ 0 \\\\ 0 \\end{bmatrix} $$\n",
    "    * With three columns all with pivots, we have *n* - *r* = 3 - 3 = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Part 2\n",
    "    * Row reduction will take us to\n",
    "$$ \\begin{bmatrix} R \\\\ 0 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Part 3\n",
    "$$ \\begin{bmatrix} U & U \\\\ 0 & -U \\end{bmatrix} $$\n",
    "    * In reduced row echelon form\n",
    "$$ \\begin{bmatrix} U & 0 \\\\ 0 & U \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Consider\n",
    "$$ {A} \\overline {x} = \\begin{bmatrix} 2 \\\\ 4 \\\\ 2 \\end{bmatrix} $$\n",
    "* With\n",
    "$$ {x} = \\begin{bmatrix} 2 \\\\ 0 \\\\ 0 \\end{bmatrix} + {c}\\begin{bmatrix} 1 \\\\ 1 \\\\ 0 \\end{bmatrix} + {d}\\begin{bmatrix} 0 \\\\ 0 \\\\ 1 \\end{bmatrix} $$\n",
    "* Part 1\n",
    "    * What is the dimension of the rowspace of A and the nullspace of A\n",
    "* Part 2\n",
    "    * For what values of **b** can A**x** = **b** be solved?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Part 1\n",
    "    * Well, the size of the matrix must be 3&#215;3\n",
    "    * The dimension of the nullspace is 2 (because two non-pivot columns)\n",
    "    * With *n* - *r* = 2, we have *r* = 1, which must hold for the rowspace of A\n",
    "* Part 2\n",
    "    * Looking only at the particular solution we must have\n",
    "$$ 2\\begin{bmatrix} { a }_{ 11 } \\\\ { a }_{ 21 } \\\\ { a }_{ 31 } \\end{bmatrix}=\\begin{bmatrix} 2 \\\\ 4 \\\\ 2 \\end{bmatrix}\\\\ \\therefore \\quad \\begin{bmatrix} { a }_{ 11 } \\\\ { a }_{ 21 } \\\\ { a }_{ 31 } \\end{bmatrix}\\quad =\\quad \\begin{bmatrix} 1 \\\\ 2 \\\\ 1 \\end{bmatrix}\\\\ \\therefore \\quad \\begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } & { a }_{ 13 } \\\\ { a }_{ 21 } & { a }_{ 22 } & { a }_{ 23 } \\\\ { a }_{ 31 } & { a }_{ 32 } & { a }_{ 33 } \\end{bmatrix}=\\begin{bmatrix} 1 & -1 & 0 \\\\ 2 & -2 & 0 \\\\ 1 & -1 & 0 \\end{bmatrix} $$\n",
    "* So, how did I get the last two columns?\n",
    "    * Well, they cannot be independent of the first column and the last column must have all zeros to set up the first variable solution fr *x*<sub>3</sub>, i.e. *x*<sub>3</sub> = *d*\n",
    "    * Adding column 2 to column 1 to get all zeros must result in what you seen for column 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}2\\\\4\\\\2\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡2⎤\n",
       "⎢ ⎥\n",
       "⎢4⎥\n",
       "⎢ ⎥\n",
       "⎣2⎦"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = Matrix([[1, -1, 0], [2, -2, 0], [1, -1, 0]])\n",
    "x_vect = Matrix([2, 0, 0])\n",
    "x_vect_null_1 = Matrix([1, 1, 0])\n",
    "x_vect_null_2 = Matrix([0, 0, 1])\n",
    "A * x_vect + A * x_vect_null_1 + A * x_vect_null_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\begin{bmatrix}\\left[\\begin{matrix}1\\\\1\\\\0\\end{matrix}\\right], & \\left[\\begin{matrix}0\\\\0\\\\1\\end{matrix}\\right]\\end{bmatrix}$$"
      ],
      "text/plain": [
       "⎡⎡1⎤, ⎡0⎤⎤\n",
       "⎢⎢ ⎥  ⎢ ⎥⎥\n",
       "⎢⎢1⎥  ⎢0⎥⎥\n",
       "⎢⎢ ⎥  ⎢ ⎥⎥\n",
       "⎣⎣0⎦  ⎣1⎦⎦"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.nullspace()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* It can only be solve for scalar multiples of\n",
    "$$ \\begin{bmatrix} 1 \\\\ 2 \\\\ 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question and solution 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* If the nullspace of a square matrix is the zero vector only, does the nullspace of the transpose also only contain the zero vector\n",
    "    * Yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Consider the matrix space of all 5&#215;5 matrices; do the invertible 5&#215;5 matrices form a subspace\n",
    "    * No, as the set of invertible matrices would not contain the zero matrix\n",
    "    * Also if I add two invertible matrices, I don't know if the resultant matrix is invertible\n",
    "    * The singular ones won't work either as adding two we also don't know if the resultant matrix is invertible"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* If B<sup>2</sup> = **0**, is B = **0**?\n",
    "    * No, i.e\n",
    "$$ {B} = \\begin{bmatrix} 0 & 1 \\\\ 0 & 0 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0 & 0\\\\0 & 0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0  0⎤\n",
       "⎢    ⎥\n",
       "⎣0  0⎦"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = Matrix([[0, 1], [0, 0]])\n",
    "B ** 2 # Could also use B * B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B == B * B # Checking by Boolean statement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* A system of *n* unknowns in *n* equations is solvable for every **b** if the columns of the matrix of coefficients are independent?\n",
    "    * Yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ B=\\begin{bmatrix} 1 & 1 & 0 \\\\ 0 & 1 & 0 \\\\ 1 & 0 & 1 \\end{bmatrix}\\begin{bmatrix} 1 & 0 & -1 & 2 \\\\ 0 & 1 & 1 & -1 \\\\ 0 & 0 & 0 & 0 \\end{bmatrix} $$\n",
    "* Calculate the basis of the nullspace of B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* B will have to be a 3&#215;4 matrix\n",
    "* The last row will be all zeros"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\begin{pmatrix}\\left[\\begin{matrix}1 & 1 & 0\\\\0 & 1 & 0\\\\1 & 0 & 1\\end{matrix}\\right], & \\left[\\begin{matrix}1 & 0 & -1 & 2\\\\0 & 1 & 1 & -1\\\\0 & 0 & 0 & 0\\end{matrix}\\right]\\end{pmatrix}$$"
      ],
      "text/plain": [
       "⎛⎡1  1  0⎤, ⎡1  0  -1  2 ⎤⎞\n",
       "⎜⎢       ⎥  ⎢            ⎥⎟\n",
       "⎜⎢0  1  0⎥  ⎢0  1  1   -1⎥⎟\n",
       "⎜⎢       ⎥  ⎢            ⎥⎟\n",
       "⎝⎣1  0  1⎦  ⎣0  0  0   0 ⎦⎠"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B1 = Matrix([[1, 1, 0], [0, 1, 0], [1, 0, 1]])\n",
    "B2 = Matrix([[1, 0, -1, 2], [0, 1, 1, -1], [0, 0, 0, 0]])\n",
    "B1, B2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* It is important to note that of we multiply an invertible matrix by another matrix (assuming multiplication is possible by the shape of the matrices), then the invertible one (B1 above), plays no part in the nullspace\n",
    "    * N(CD) = N(D) if C is invertible\n",
    "* We therefor only have to look at B2\n",
    "* It has 2 pivot columns, i.e. rank is *r* = 2\n",
    "* It will therefor have 2 independent variables, making the nullspace 2-dimensional"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\begin{bmatrix}\\left[\\begin{matrix}1\\\\-1\\\\1\\\\0\\end{matrix}\\right], & \\left[\\begin{matrix}-2\\\\1\\\\0\\\\1\\end{matrix}\\right]\\end{bmatrix}$$"
      ],
      "text/plain": [
       "⎡⎡1 ⎤, ⎡-2⎤⎤\n",
       "⎢⎢  ⎥  ⎢  ⎥⎥\n",
       "⎢⎢-1⎥  ⎢1 ⎥⎥\n",
       "⎢⎢  ⎥  ⎢  ⎥⎥\n",
       "⎢⎢1 ⎥  ⎢0 ⎥⎥\n",
       "⎢⎢  ⎥  ⎢  ⎥⎥\n",
       "⎣⎣0 ⎦  ⎣1 ⎦⎦"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = B1 * B2\n",
    "B.nullspace()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}