3-dictionaries.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Week 5 Lecture - Dictionaries\n",
    "\n",
    "\n",
    "* What is a Dictionary\n",
    "* Data in / Data Out\n",
    "* Looping\n",
    "* Making a word counter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is a Dictionary?\n",
    "\n",
    "![Dictionary Picture](http://www.trytoprogram.com/images/python_dictionary.jpg)\n",
    "\n",
    "*Image used without permission from [Trytoprogram](http://www.trytoprogram.com/python-programming/python-dictionary/)*\n",
    "\n",
    "Similar to a Python list, a Python Dictionary is a data structure that behaves as a *container* or *collection* for other data values. \n",
    "A a list because it contains a values and each value has an index.\n",
    "\n",
    "Not like a list because the indicies are not implicit, they can be more than numbers, and there is no order.\n",
    "\n",
    "In a dicitonary, data is stored in *key*/*value* pairs. The *key* is the index and and *value* is the actual data.\n",
    "When you create an *item* in a dictionary (another term for the key/value pairs) then you store teh data in the dictionary.\n",
    "To get the data back out we use the key to *lookup* the value in the dictionary. In this way the dictionary behaves like a english language dictionary. The *key* is like the word and the *value* is like the definition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new dictionary\n",
    "english2spanish = dict()\n",
    "# another way to do the same thing using the literal syntax\n",
    "english2spanish = {}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have an empty dictionary with zero items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(english2spanish)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Putting Data in a Dictionary\n",
    "If you have an empty dictionary, you can add items using the "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add values with the assignment operator\n",
    "english2spanish[\"one\"] = \"uno\"\n",
    "\n",
    "# add items with the update method\n",
    "english2spanish.update({\"two\":\"dos\"})\n",
    "\n",
    "print(english2spanish) # look at the contents of the entire dictionary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at the syntax of what Python spit out. Curly braces with two string values separated by a colon. The curly braces mean we are looking at a dictionary (as opposed to a list which uses square brackets `[]`) and the colon always separates the key/value pairs (the *item*). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add another item to the dictionary\n",
    "english2spanish[\"three\"] = \"tres\"\n",
    "\n",
    "# print dictionary contents\n",
    "print(english2spanish)\n",
    "#display the length of the dict\n",
    "len(english2spanish)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now our `english2spanish` dictionary has three items! \n",
    "\n",
    "![The count counting](https://media.giphy.com/media/FHzemFzwkyRfq/giphy.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting data out of a dictionary\n",
    "We can use the familiar *indexing syntax* (square brackets) to get a value based on its key. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# lookup the value associated with the key\n",
    "english2spanish[\"one\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you can't look up by value\n",
    "english2spanish[\"uno\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you try to look up a key that does not exist using the index syntax, Python will yell at you.\n",
    "\n",
    "Use the `get()` method to look up keys with less errors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# getting data with get\n",
    "english2spanish.get(\"one\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# getting data that does not exist\n",
    "english2spanish.get(\"seven\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look! no error. Sometimes this is preferred behavior when you are processing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structuring Data in Data Structures\n",
    "\n",
    "Dictionary keys must be basic data types (ints and strings are most common, but floats work too). The values can be anything, even more complicated data structures like lists and dictionaries. This allows you to create complicated, multidimentional data structures for storing your data in Python. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a dictionary with data already loaded up\n",
    "floaty_keys = {4.5:\"four point five\", 2.1:\"Two point one\", \"one point 2\":1.2, \"4.2\":4.2}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# grab the value with the float key 4.5\n",
    "floaty_keys[4.5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the value with the string key \"one point 2\"\n",
    "floaty_keys[\"one point 2\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the value with the string key \"4.2\"\n",
    "floaty_keys.get(\"4.2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dictionary values can be any Python data type or data structure. Here is an example of storing tabular data in lists with the key as the column name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# put some tabular data into a dictionary of lists\n",
    "user_data = {\n",
    "    \"ages\":[12,23,234,95],\n",
    "    \"name\":[\"Bobby\", \"Karl\", \"Dracula\", \"Henry\"],\n",
    "    \"occupation\":[\"Boy\", \"Barista\", \"Sandwich Artist by Night\", \"Taxi Driver\"]\n",
    "}\n",
    "user_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the occupation data\n",
    "user_data[\"occupation\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the third item in the occupation list\n",
    "user_data[\"occupation\"][2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way of storing this data would be to have a dictionary of dictionaries. Note how the key must be explicit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1: {'name': 'Bobby', 'age': 12, 'occupation': 'Boy'},\n",
       " 2: {'name': 'Karl', 'age': 23, 'occupation': 'Barista'},\n",
       " 3: {'name': 'Dracula', 'age': 234, 'occupation': 'Sandwich Artist by Night'},\n",
       " 4: {'name': 'Henry', 'age': 95, 'occupation': 'Taxi Driver'}}"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a dictionary of dictionaries to hold all the data\n",
    "data_as_dictionary = {\n",
    "    1:{\"name\":\"Bobby\", \"age\":12, \"occupation\":\"Boy\"},\n",
    "    2:{\"name\":\"Karl\", \"age\":23, \"occupation\":\"Barista\"},\n",
    "    3:{\"name\":\"Dracula\", \"age\":234, \"occupation\":\"Sandwich Artist by Night\"},\n",
    "    4:{\"name\":\"Henry\", \"age\":95, \"occupation\":\"Taxi Driver\"}\n",
    "}\n",
    "data_as_dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the data dictionary for user 3\n",
    "data_as_dictionary[3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the occupation of user 3\n",
    "data_as_dictionary[3]['occupation']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The cell above when the first expression is evaluated\n",
    "{'name': 'Dracula', 'age': 234, 'occupation': 'Sandwich Artist by Night'}[\"occupation\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can even make a list of dictionaries which will behave very similarly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list_of_dictionaries = [\n",
    "    {\"name\":\"Bobby\", \"age\":12, \"occupation\":\"Boy\"},\n",
    "    {\"name\":\"Karl\", \"age\":23, \"occupation\":\"Barista\"},\n",
    "    {\"name\":\"Dracula\", \"age\":234, \"occupation\":\"Sandwich Artist by Night\"},\n",
    "    {\"name\":\"Henry\", \"age\":95, \"occupation\":\"Taxi Driver\"}\n",
    "]\n",
    "list_of_dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the item at index position 3 from the list\n",
    "list_of_dictionaries[3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the value with the key name from the dictionary at index position of the list called list_of_dictionaries\n",
    "list_of_dictionaries[3]['name']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looping over dictionaries\n",
    "\n",
    "While dictionaries are not sequential like lists, you can loop over all the items. The order will be equivalent to the insertion order (but remember there is no implicit numerical index).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ages\n",
      "name\n",
      "occupation\n"
     ]
    }
   ],
   "source": [
    "# Looping over a dictionary will go over all the keys\n",
    "for item in user_data:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ages\n",
      "[12, 23, 234, 95]\n",
      "name\n",
      "['Bobby', 'Karl', 'Dracula', 'Henry']\n",
      "occupation\n",
      "['Boy', 'Barista', 'Sandwich Artist by Night', 'Taxi Driver']\n"
     ]
    }
   ],
   "source": [
    "# Looping over a dictionary will go over all the keys\n",
    "for item in user_data:\n",
    "    print(item)\n",
    "    print(user_data[item])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Putting it all together - Stylometrics\n",
    "\n",
    "\n",
    "Don't be afraid of the $5 word, it just a technique for analyzing texts (usually to [determine authorship](https://www.latimes.com/science/sciencenow/la-sci-sn-shakespeare-play-linguistic-analysis-20150410-story.html)). Computational Stylometics does a lot of fancy statistics, but much of it is based on *counting words*.\n",
    "\n",
    "Given the text below, we want to write a program that counts all the words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#swoon\n",
    "romeo = \"\"\"\n",
    "But, soft! what light through yonder window breaks?\n",
    "It is the east, and Juliet is the sun.\n",
    "Arise, fair sun, and kill the envious moon,\n",
    "Who is already sick and pale with grief,\n",
    "That thou her maid art far more fair than she:\n",
    "Be not her maid, since she is envious;\n",
    "Her vestal livery is but sick and green\n",
    "And none but fools do wear it; cast it off.\n",
    "It is my lady, O, it is my love!\n",
    "O, that she knew she were!\n",
    "She speaks yet she says nothing: what of that?\n",
    "Her eye discourses; I will answer it.\n",
    "I am too bold, 'tis not to me she speaks:\n",
    "Two of the fairest stars in all the heaven,\n",
    "Having some business, do entreat her eyes\n",
    "To twinkle in their spheres till they return.\n",
    "What if her eyes were there, they in her head?\n",
    "The brightness of her cheek would shame those stars,\n",
    "As daylight doth a lamp; her eyes in heaven\n",
    "Would through the airy region stream so bright\n",
    "That birds would sing and think it were not night.\n",
    "See, how she leans her cheek upon her hand!\n",
    "O, that I were a glove upon that hand,\n",
    "That I might touch that cheek!\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computational Thinking\n",
    "\n",
    "If we want to count the words in the text above, we need to do the following things.\n",
    "\n",
    "1. Normalize the text by removing punctuation and converting to lowercase.\n",
    "2. Split the string of text into a list of words\n",
    "3. Loop over the list and count each instance of a word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert everything to lowercase\n",
    "romeo.lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have everything in lowercase, but we need to remove the punctuation. Now, we could use the `replace()` string method and manually identify and remove each punctuation mark, but that would make for some ugly code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ugly approach to removing punctuation\n",
    "romeo.replace(\".\",\"\").replace(\",\",\"\").replace(\"!\",\"\").replace(\"'\",\"\") #and so on"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wouldn't it be nice it we could do this all in one shot? Fortunately, we can but it is a bit complicated.\n",
    "\n",
    "\n",
    "```\n",
    "Replace each character in the string using the given translation table.\n",
    "\n",
    "table\n",
    "    Translation table, which must be a mapping of Unicode ordinals to\n",
    "    Unicode ordinals, strings, or None.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remember the ord()\n",
    "print(\"Period:\", ord(\".\"))\n",
    "print(\"Comma:\", ord(\",\"))\n",
    "print(\"Explaination\", ord(\"!\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "translation_table = {46:\"\",\n",
    "                     44:\"\",\n",
    "                     33:\"\"}\n",
    "romeo.translate(translation_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the `maketrans()` function to automatically translate characters into their ordinal values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "punctuation_dictionary = {\n",
    "    \".\":\"\",\n",
    "    \"!\":\"\",\n",
    "    \":\":\"\",\n",
    "    \",\":\"\",\n",
    "    \"?\":\"\",\n",
    "    \";\":\"\",\n",
    "    \",\":\"\"\n",
    "}\n",
    "translation_table = romeo.maketrans(punctuation_dictionary)\n",
    "translation_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use this table to remove the punctuation from our string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "romeo.translate(translation_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But, that was still a lot of typing and it looks like we missed the apostrophe...ugh. More typing means more bugs...\n",
    "\n",
    "If Pythong is *actually* batteries included, then wouldn't this already be a solved problem?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get a list of all the punctuation from the standard library\n",
    "from string import punctuation\n",
    "print(punctuation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make our translation table programmatically with the excludes argument\n",
    "translation_table = romeo.maketrans(\"\", \"\", punctuation)\n",
    "translation_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test our our punctuation remover\n",
    "romeo.translate(translation_table)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yes! Now we have almost solved the first step. Now make everything lowercase, fortunately that is easy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# normalize the text\n",
    "romeo_normalized = romeo.translate(translation_table).lower()\n",
    "romeo_normalized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can do computational thinking step 2: split the string of words into a list. Also an easy task thanks to the string method `split()` which will automatically split on whitespace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#split text into a list of words\n",
    "romeo_list = romeo_normalized.split()\n",
    "romeo_list[0:10] #look at the first 10 words in the list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, now we can do the final step, which is loop over each word and count them up in a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a counter\n",
    "word_counter = {}\n",
    "\n",
    "# loop over each wor\n",
    "for word in romeo_list:\n",
    "    # check to see if we have encountered the word\n",
    "    if word not in word_counter:\n",
    "        # have not seen this word before, so create a key with value 1\n",
    "        word_counter[word] = 1\n",
    "    else: \n",
    "        # we have seen this word before, so increment the value by 1\n",
    "        word_counter[word] += 1\n",
    "\n",
    "print(word_counter)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}