simlang_11_lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simulating Language 11, Learned signalling (lab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simulation in this notebook implements the learning of a signalling system. In the previous simulations, an individual agent’s signalling system was provided innately, and didn’t change in its lifetime. Populations of agents evolved through natural selection according to the fitness function we specified, to be ‘optimal’ in some way for communication.\n",
    "\n",
    "In this simulation, we’re ignoring evolution, and instead allowing the weights in an agent’s signalling system to change through learning, as a result of their experiences.\n",
    "\n",
    "The first section of the code is similar to the code we used in our first simulation, when we introduced the following:\n",
    "- a signalling system is represented as a list of lists - you can think of this as a matrix or as a neural network.\n",
    "- how to produce a signal to express a meaning, using winner-take-all;\n",
    "- how to decide which meaning a received signal is expressing, using winner-take-all;\n",
    "- communication as a measure of how well the speaker’s meaning matches the hearer’s meaning after being transmitted via a signal.\n",
    "\n",
    "These should all be very familiar by now. We have made one major change to the code though. In the earlier models we had separate matrices for production and reception. From now on we are going to use a model where we just have a single matrix which handles both processes. There are some small changes to the code to accomplish this.\n",
    "\n",
    "*Identify the changes required to go from a two-matrix model to a one-matrix model, and figure out why they have been made.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.display import set_matplotlib_formats\n",
    "set_matplotlib_formats('svg', 'pdf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def wta(items):\n",
    "    maxweight = max(items)\n",
    "    candidates = []\n",
    "    for i in range(len(items)):\n",
    "        if items[i] == maxweight:\n",
    "            candidates.append(i)\n",
    "    return random.choice(candidates)\n",
    "\n",
    "def reception_weights(system, signal):\n",
    "    weights = []\n",
    "    for row in system:\n",
    "        weights.append(row[signal])\n",
    "    return weights\n",
    "\n",
    "def communicate(speaker_system, hearer_system, meaning):\n",
    "    speaker_signal = wta(speaker_system[meaning])\n",
    "    hearer_meaning = wta(reception_weights(hearer_system, speaker_signal))\n",
    "    if meaning == hearer_meaning: \n",
    "        return 1\n",
    "    else: \n",
    "        return 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### New code\n",
    "\n",
    "It's really remarkable how little extra code is needed to build a model of learning. Here it is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learn(system, meaning, signal):\n",
    "    system[meaning][signal] += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simple, huh? In learning, agents store the association between the meaning and signal. We need one simple function to implement learning. The function learn takes three arguments and is just two lines of code. The arguments are:\n",
    "1. a signalling system\n",
    "2. a meaning \n",
    "3. a signal\n",
    "\n",
    "The function finds the appropriate cell in the signalling system matrix indexed by the meaning and signal, and adds one to the value of the weight in this cell.\n",
    "Make sure you understand how this learning function works, what the parameters mean, and how the function updates the correct cell in the matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try out the function. Start with an \"empty\" agent that doesn't know anything about meanings and signals:\n",
    "\n",
    "```python\n",
    "s = [[0, 0, 0], \n",
    "     [0, 0, 0],\n",
    "     [0, 0, 0],\n",
    "     [0, 0, 0]]\n",
    "```\n",
    "\n",
    "(This agent has four meanings, and three signals).\n",
    "\n",
    "We can train this agent with commands like these:\n",
    "\n",
    "```python\n",
    "learn(s,0,2)\n",
    "learn(s,1,1)\n",
    "learn(s,0,2)\n",
    "learn(s,3,0)\n",
    "```\n",
    "\n",
    "And look at the resulting matrix using:\n",
    "\n",
    "```python\n",
    "print(s)\n",
    "```\n",
    "\n",
    "Make sure you understand how and why the matrix has changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "Rather than input each learning episode individually (which is a bit laborious), we can give an agent a list of meaning-signal pairs, and learn them all through the single function train. This function goes through each item in the list, and learns each meaning-signal pair individually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(system, ms_pair_list):\n",
    "    for pair in ms_pair_list:\n",
    "        learn(system, pair[0], pair[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make sure you understand how this works, create a signalling system, then provide it with a list of learning exposures, and check that the system has learnt from the data you have given it. Does it do what you expect?\n",
    "\n",
    "Hint: you'll need to use the `wta` function to find out what your trained agent would produce/understand. For example, if you had an agent called `network`, then: `wta(network[1])` would tell you what signal that agent would produce for meaning 1, and `wta(reception_weights(network, 2))` would tell you what meaning that agent would understand for signal 2. (Make sure you understand why these are the right things to type in!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning in a population\n",
    "\n",
    "The next part of the code allows us to go from a single agent to a population (if we wish). `pop_learn` takes a list of signalling systems, a list of utterances (i.e. `[meaning, signal]` pairs), and some number of learning episodes. For the number of learning episodes specified, it trains a random individual in the population on a random utterance picked from the list of data.\n",
    "\n",
    "The reason we need a function like `pop_learn` might not be immediately obvious, but will be clear when we come to the next notebook! For the time being, you can use this function to train a single agent by simply building a population that has a single agent in it. Alternatively, you can use it to look at whether two or more agents may end up speaking similar languages when exposed to utterances picked at random from a set of training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pop_learn(population, data, no_learning_episodes):\n",
    "    for n in range(no_learning_episodes):\n",
    "        ms_pair = random.choice(data)\n",
    "        learn(random.choice(population), ms_pair[0], ms_pair[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try the following code out:\n",
    "\n",
    "```python\n",
    "p = [[[0, 0], [0, 0]]]\n",
    "pop_learn(p, [[0, 0], [1, 1]], 100)\n",
    "print(p)\n",
    "```\n",
    "\n",
    "Why are there three square brackets at the start of the variable “p”? Try different data. How can we use this way of training to model different frequencies of different types of utterance?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a way for a population to learn from some data, but how about getting them to produce data, in order to evaluate how well they have learnt? `pop_produce` carries out this function. It takes a population and a required number of productions, and returns a list of utterances (meaning-signal pairs) generated by individuals picked randomly from the population:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pop_produce(population, no_productions):\n",
    "    ms_pairs = []\n",
    "    for n in range(no_productions):\n",
    "        speaker = random.choice(population)\n",
    "        meaning = random.randrange(len(speaker))\n",
    "        signal = wta(speaker[meaning])\n",
    "        ms_pairs.append([meaning, signal])  \n",
    "    return ms_pairs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we have a population-based version of our Monte Carlo measure of communicative accuracy: `ca_monte_pop`. This takes a population and a number of trials, and return a Monte Carlo estimate of the chance that a random communication between members of the population will be successful - note that it just returns a single value, rather than a list of values (which is different from previous implementations of Monte Carlo evaluation)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ca_monte_pop(population, trials):\n",
    "    total = 0.\n",
    "    for n in range(trials):\n",
    "        speaker = random.choice(population)\n",
    "        hearer = random.choice(population)\n",
    "        total += communicate(speaker, hearer, random.randrange(len(speaker)))\n",
    "    return total / trials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Questions\n",
    "\n",
    "Answering questions 1-3 involves playing with the model - for question 4, you can just think about it (although you can have a go at coding your ideas up if you like).\n",
    "\n",
    "1. How good is this model of learning? How can you test it? What does \"good\" even mean in this context? (Hint: this is a deeper and more important question than it first appears!)\n",
    "2. Can you write some code to test how well an agent has learnt a particular language?\n",
    "3. Learning is implemented as a frequency count of associations. Are there other reasonable ways of updating the matrix? How else might you change weights in response to observations? \n",
    "4. In answering questions 1-3, you have probably been training agents on data that *you* provided. In a proper model of language learning, where would this data come from? Could you use the code above to model this?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}