week5-dict-counting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# counting everything\n",
    "\n",
    "The basic counting idiom in python is just a formalization of how we count things in regular life. Arrange them into a set of things of the general type to be counted, then go through them one by one, making a note of each that meets some criterion. \n",
    "\n",
    "That's a fancy way of saying, if you want to count words, you need words; if you want to do letters, you want letters. Don't try to count words with a pile of letters; transform them into words first.\n",
    "\n",
    "Counting many words as once takes advantage of the name-based strucutre of a dictionary to keep all of the separate counts divided up, but it's fundamentally not that different from counting instances of one word. Noteably, you don't loop over the dictionary or digest your text into a new format. As before, we almost certainly just want a list of all the words in order (actually, they don't have to be in order for this, as we'll see later in the course—all that matters is that we have an easy way to go through them all).\n",
    "\n",
    "Let's review how this works by fleshing out the following function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_word(word, text: str):\n",
    "    '''\n",
    "    counts all occurances of word in text and returns the count as an integer\n",
    "    for this function, a word means a space separated string of characters; no need to lowercase or strip punctuation\n",
    "    '''\n",
    "    count = 0\n",
    "    return count\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With a dictionary, we do something pretty similar, except that for each word, we have to make sure it's in the dictionary, then add to our count. There are actually two ways to do this, depending on how we handle the case where we have a new word and it's not in the dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_all_words(text: str):\n",
    "    '''\n",
    "    counts all of the words in text and returns a dict of counts\n",
    "    '''\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our return value here is a bit less transparent than in the case of `count_word()`. We could further process the dictionary, but if we want to print it out, we need to know a few dictionary methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'fish': 14, 'whale': 6, 'shark': 3, 'shrimp': 4}\n",
      "fish\n",
      "whale\n",
      "shark\n",
      "shrimp\n"
     ]
    }
   ],
   "source": [
    "d = {'fish' : 14, 'whale' : 6, 'shark' : 3, 'shrimp' : 4}\n",
    "print(dict)\n",
    "\n",
    "for count in dict:\n",
    "    print(count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we loop over a dictionary, we just get the keys; it's the equivalent of looping over `dict.keys()`. That might suggest we try looping over `dict.values()` but that won't print the keys, which is probably even less helpful. Of course, you could loop over the values and for each of them, `print(f'{key} : {dict[key]}')` but there's an easier way. `dict.items()` returns a list of what in python are called tuples. A **tuple** (a double, triple etc) is kind of like a list, except you can't change it in any way. It's a useful way to package together information that doesn't make much sense without it's other parts, or to show that the individual pieces won't be updated in any way.\n",
    "\n",
    "tuples are especially useful in python because of what's called unpacking, where python will automatically map from separate things onto collections of things. Consider this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fish\n",
      "14\n"
     ]
    }
   ],
   "source": [
    "word, count = ['fish', 14]\n",
    "print(word)\n",
    "print(count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It works with tuples exactly like with lists, which brings us around to how to actually make a tuple. Tuples come in parentheses, with items divided by commas—a lot likie lists. In actual code, there's not much point in making them (think about making lists: how often have you made a list that's not an empty list? And, why would an empty tuple be less useful than an empty list?) but if you want, you can do it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('word', 'count')\n"
     ]
    }
   ],
   "source": [
    "my_tuple = ('word', 'count')\n",
    "print(my_tuple)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, for the dictionary case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('fish', 14)\n",
      "('whale', 6)\n",
      "('shark', 3)\n",
      "('shrimp', 4)\n"
     ]
    }
   ],
   "source": [
    "for item in d.items():\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "these are tuples, and this isn't a terrible way to print out the dictionary, but we can do the unpacking thing to do a bit better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "word:fish -> count:14\n",
      "word:whale -> count:6\n",
      "word:shark -> count:3\n",
      "word:shrimp -> count:4\n"
     ]
    }
   ],
   "source": [
    "for key, value in d.items():\n",
    "    print(f'word:{key} -> count:{value}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we start writing things like this to file, rather than printing them to screen, we can skip this step, because usually what we want for this kind of data is a list containing rows of data, where each row is another list, or something like it—which includes a tuple. `dict.items()` is dressed up a bit, but that's basically what it is, and you can write it directly into a csv with a column for each of words and counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_items([('fish', 14), ('whale', 6), ('shark', 3), ('shrimp', 4)])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d.items()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_file('filename')\n",
    "with open(filename, 'R', encoding='UTF-8') as f:\n",
    "    text = f.read\n",
    "    return count_all_words(text)\n",
    "ygb_counts = count_file('hawthorne_young_goodman_brown.txt')\n",
    "print(ygb_counts.items()[:100])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}