I_12_Matrix_spaces.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This notebook is part of lecture 11 *Matrix spaces, rank 1, small world graphs* in the OCW MIT course 18.06 by Prof Gilbert Strang [1]\n",
    "+ Created by me, Dr Juan H Klopper\n",
    "    + Head of Acute Care Surgery\n",
    "    + Groote Schuur Hospital\n",
    "    + University Cape Town\n",
    "    + <a href=\"mailto:juan.klopper@uct.ac.za\">Email me with your thoughts, comments, suggestions and corrections</a> \n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/InteractiveResource\" property=\"dct:title\" rel=\"dct:type\">Linear Algebra OCW MIT18.06</span> <span xmlns:cc=\"http://creativecommons.org/ns#\" property=\"cc:attributionName\">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>.\n",
    "\n",
    "+ [1] <a href=\"http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm\">OCW MIT 18.06</a>\n",
    "+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: #ddd;\n",
       "    color: #000000;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(200,100,0);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML, Image\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#import numpy as np\n",
    "from sympy import init_printing, Matrix, symbols\n",
    "#import matplotlib.pyplot as plt\n",
    "#import seaborn as sns\n",
    "#from IPython.display import Image\n",
    "from warnings import filterwarnings\n",
    "\n",
    "init_printing(use_latex = 'mathjax') # Pretty Latex printing to the screen\n",
    "#%matplotlib inline\n",
    "filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Matrix spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## New *vector* / matrix spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Square matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Consider *M* to be all 3&#215;3 matrices (with real elements)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Subspaces would be:\n",
    "    * Upper or lower triangular matrices\n",
    "    * Symmetric matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Basis would be:\n",
    "$$ \\begin{bmatrix} 1 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 1 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 1 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 1 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 1 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 1 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 1 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 1 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The dimension would be 9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* For upper and lower triangular matrices the dimensions would be 6 and the basis:\n",
    "$$ \\begin{bmatrix} 1 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 1 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 1 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 1 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 1 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* For symmetric matrices the dimension would also be six ( Knowing the diagonal and entries on one of the two sides)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* These are unique cases where the bases for the subspaces are contained in the basis of the 3&#215;3 matrix *M*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other square matrices that are subspaces of *M*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The intersection of symmetric and upper triangular matrices (that is symmetric AND upper triangular, *S*&#8745;*U*)\n",
    "    * This is a diagonal matrix\n",
    "    * The dimension is 3\n",
    "    * The basis is\n",
    "$$ \\begin{bmatrix} 1 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 1 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ 0 & 0 & 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The union of symmetric and upper triangular matrices (that is symmetric OR upper triangular, *S*&#8746;*U*)\n",
    "    * It is **NOT** a subspace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The addition (sum) of symmetric and upper triangular matrices\n",
    "    * It **IS** a subspace\n",
    "    * It is actually all 3&#215;3 matrices\n",
    "    * The dimension is 9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This gives the equation: dim(*S*) + dim(*U*) = dim(*S*&#8745;*U*) + dim(*S*+*U*) = 12"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example problem 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Show that the set of 2&#215;3 matrices whose nullspace contains the column vector below is a vector subspace and find a basis for it\n",
    "$$ \\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* In essence we have to show the following\n",
    "$$ A\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix}\\\\ \\begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } & { a }_{ 13 } \\\\ { a }_{ 21 } & { a }_{ 22 } & { a }_{ 23 } \\end{bmatrix}\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix} $$\n",
    "* ... and ...\n",
    "$$ B\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This can be shown by addition:\n",
    "$$ \\left( A+B \\right) \\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix}=A\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}+B\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix} $$\n",
    "* Therefor (by virtue of the fact that addition remains in the nullspace) the set is vector subspace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* We also need to look at scalar multiplication (if we multiply a matrix in the set by a scalar, does it remain in the set)\n",
    "$$ \\left( cA \\right) \\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=c\\left( A\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}= \\right) c\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example problem 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Find a basis for the nullspace above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Let's look at the first row:\n",
    "$$ \\begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } & { a }_{ 13 } \\end{bmatrix}\\begin{bmatrix} 2 \\\\ 1 \\\\ 1 \\end{bmatrix}=\\begin{bmatrix} 0 \\\\ 0 \\end{bmatrix}\\\\ 2{ a }_{ 11 }+{ a }_{ 12 }+{ a }_{ 13 }=0\\\\ { a }_{ 13 }=-2{ a }_{ 11 }-{ a }_{ 12 } $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* From this we can make the following row vectors\n",
    "$$ \\begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } & { a }_{ 13 } \\end{bmatrix}=\\begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } & \\left( -2{ a }_{ 11 }-{ a }_{ 12 } \\right)  \\end{bmatrix}\\\\ =\\quad \\begin{bmatrix} { a }_{ 11 } & 0 & { -2a }_{ 11 } \\end{bmatrix}+\\begin{bmatrix} 0 & { a }_{ 12 } & { -a }_{ 12 } \\end{bmatrix}\\\\ ={ a }_{ 11 }\\begin{bmatrix} 1 & 0 & -2 \\end{bmatrix}+{ a }_{ 12 }\\begin{bmatrix} 0 & 1 & -1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* From this we can construct 4 basis:\n",
    "$$ \\begin{bmatrix} 1 & 0 & -2 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 1 & -1 \\\\ 0 & 0 & 0 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 1 & 0 & -2 \\end{bmatrix},\\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 1 & -1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example problem 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* What about the set of those whose column space contains the following column vector?\n",
    "$$ \\begin{bmatrix} 2 \\\\ 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Well, any subspace must contain the zero matrix\n",
    "$$ \\begin{bmatrix} 0 & 0 & 0 \\\\ 0 & 0 & 0 \\end{bmatrix} $$\n",
    "* It does not contain the above column vector, which is therefor not a subspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}