worksheet_inference2/worksheet_inference2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a797fc967c4f7d1e3e270fed595dc508",
     "grade": false,
     "grade_id": "cell-f863750069537ead",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# Worksheet 12 - The Bootstrap\n",
    "\n",
    "### Lecture and Tutorial Learning Goals:\n",
    "\n",
    "After completing this week's lecture and tutorial work, you will be able to:\n",
    "- Explain why we don't have a sampling distribution in practice/real life.\n",
    "- Define bootstrapping.\n",
    "- Use R to create a bootstrap distribution to approximate a sampling distribution.\n",
    "- Contrast bootstrap and sampling distributions.\n",
    "\n",
    "This worksheet covers parts of [the Inference chapter](https://datasciencebook.ca/inference.html) of the online textbook. You should read this chapter before attempting the worksheet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d9ff9c9e4d7e25d62de14220b4956e9b",
     "grade": false,
     "grade_id": "cell-55b5050a0a343a4f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "### Run this cell before continuing.\n",
    "library(tidyverse)\n",
    "library(repr)\n",
    "library(infer)\n",
    "library(cowplot)\n",
    "options(repr.matrix.max.rows = 6)\n",
    "source('tests.R')\n",
    "source(\"cleanup.R\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ea58748ba8abd00b4d778ad8dc9e00ce",
     "grade": false,
     "grade_id": "cell-b0098728d66708d2",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.1** True/False:\n",
    "<br> {points: 1}\n",
    "\n",
    "In real life, we typically take many samples from the population and create a sampling distribution when we perform estimation. True or false?\n",
    "\n",
    "*Assign your answer to an object called `answer1.1`. Your answer should be in lowercase letters and is surrounded by quotes (e.g. `\"true\"` or `\"false\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "febbd354b2f2ca03729f132e8e79ecb7",
     "grade": false,
     "grade_id": "cell-03cc711477e4baf7",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "29db99141228bbcb8cbac44b35a4566d",
     "grade": true,
     "grade_id": "cell-e8aa24da1ef5f535",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "d8b3b78a644062149e179ad24bc6cf17",
     "grade": false,
     "grade_id": "cell-05b19510ad97aeb3",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.2** Ordering\n",
    "<br> {points: 1}\n",
    "\n",
    "Correctly re-order the steps for creating a bootstrap sample from those listed below. \n",
    "\n",
    "1. record the observation's value\n",
    "2. repeat the above the same number of times as there are observations in the original sample \n",
    "3. return the observation to the original sample\n",
    "4. randomly draw an observation from the original sample (which was drawn from the population)\n",
    "\n",
    "Create your answer by reordering values below in the `answer1.2` vector with the correct order for the steps above for creating a bootstrap sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "519eebab5b974ad14e6c859787608837",
     "grade": false,
     "grade_id": "cell-dbdb7facfc6bda43",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# answer1.2 <- c(1, 2, 3, 4) # reorder these values!\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "33ab78646661e4ea7b4af234a4044164",
     "grade": true,
     "grade_id": "cell-8a257d7535ce46fe",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "d31aab764945c44f50688abc038e91dd",
     "grade": false,
     "grade_id": "cell-a612ade2b8ba83fc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.3** Multiple choice\n",
    "<br> {points: 1}\n",
    "\n",
    "From the list below, choose the correct explanation of a bootstrap distribution for a point estimate:\n",
    "\n",
    "A. a list of point estimates calculated from many samples drawn with replacement from the population\n",
    "\n",
    "B. a list of point estimates calculated from many samples drawn without replacement from the population\n",
    "\n",
    "C. a list of point estimates calculated from bootstrap samples drawn with replacement from a single sample (that was drawn from the population)\n",
    "\n",
    "D. a list of point estimates calculated from bootstrap samples drawn without replacement from a single sample (that was drawn from the population)\n",
    "\n",
    "*Assign your answer to an object called `answer1.3`. Your answer should be an uppercase letter and is surrounded by quotes. (e.g. `\"F\"`)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "0dacd29a14dcafbb822d33871c4cfd39",
     "grade": false,
     "grade_id": "cell-fe8959defed4a439",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "9708366699b1ce6e00d18495886928ce",
     "grade": true,
     "grade_id": "cell-59dbb6a427a133c5",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "9e1159baefde49908391f825cae581a2",
     "grade": false,
     "grade_id": "cell-1df2a1ba39ccc80d",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.4** Multiple choice\n",
    "<br> {points: 1}\n",
    "\n",
    "From the list below, choose the correct explanation of why, when performing estimation, we want to report a plausible **range** for the true population quantity we are trying to estimate along with the point estimate:\n",
    "\n",
    "A. The point estimate is our best guess at the true population quantity we are trying to estimate\n",
    "\n",
    "B. The point estimate will often not be the exact value of the true population quantity we are trying to estimate\n",
    "\n",
    "C. The value of a point estimate from one sample might very well be different than the value of a point estimate from another sample.\n",
    "\n",
    "D. B & C\n",
    "\n",
    "F. A & C\n",
    "\n",
    "E. None of the above\n",
    "\n",
    "*Assign your answer to an object called `answer1.4`. Your answer should be an uppercase letter and is surrounded by quotes (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "28e5fb0455502bd31008937e083d5807",
     "grade": false,
     "grade_id": "cell-fd7e89338bbb71ca",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2225bc6706869eacd7f38d98fb6f1e14",
     "grade": true,
     "grade_id": "cell-9448b3731eccbe8c",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.4()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "cdcf658b805ec55ff277da8d8dbb9462",
     "grade": false,
     "grade_id": "cell-76363f7f1f56807e",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "###  Continuing with our virtual population of Canadian seniors from last worksheet\n",
    "\n",
    "Here we re-create the virtual population (ages of all Canadian seniors) we used in the last worksheet. It was bounded by realistic values ($\\geq$ 65 and $\\leq$ 117):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b30d4d8c75fd62e9906b1b8d010e81da",
     "grade": false,
     "grade_id": "cell-522aafd1e9430ef3",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell to simulate a finite population\n",
    "set.seed(4321) # DO NOT CHANGE\n",
    "can_seniors <- tibble(age = (rexp(2000000, rate = 0.1)^2) + 65) |> \n",
    "    filter(age <= 117, age >= 65)\n",
    "can_seniors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "5561245d433c1e817a99ae2a8ba5d19c",
     "grade": false,
     "grade_id": "cell-557951d5cb618c2c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's remind ourselves of what this population looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "fa54f2d5d00e770f4ff592e2aec57ed0",
     "grade": false,
     "grade_id": "cell-ac4865a9c4709bbb",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell \n",
    "options(repr.plot.width = 8, repr.plot.height = 7)\n",
    "pop_dist <- ggplot(can_seniors, aes(age)) + \n",
    "    geom_histogram(binwidth = 1) +\n",
    "    xlab(\"Age (years)\") +\n",
    "    ggtitle(\"Population distribution\") +\n",
    "    theme(text = element_text(size = 20), plot.margin = margin(10, 100)) # last x value was getting cut off\n",
    "pop_dist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c195a28b9d8c3412d6f580ae62d5e25b",
     "grade": false,
     "grade_id": "cell-f2e798c845edc3ee",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Estimate the mean age of Canadian Seniors\n",
    "\n",
    "Let's say we are interested in estimating the mean age of all Canadian Seniors. Given that we have the population (we created it) we could just calculate the mean age from this population data. However in real life, we usually only have one small-ish sample from the population. Also, from our examination of sampling distributions, we know that different random samples will give us different point estimates. We also know that the point estimates from different random samples will vary around the true population quanitity we are trying to estimate, and the sample size affects how close it is.\n",
    "\n",
    "What about in real life though, when we only have one sample? Can we say how close? Or at least give some plausible range of where we would expect the population quanitity we are trying to estimate to fall? Yes! We can do this using one method called **bootstrapping**! Let's explore how to create a bootstrap distribution from a single sample using R. And then we will discuss how the bootstrap distribution relates to the sampling distribution, and what it can tell us about the true population quantity we are trying to estimate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "5c79b5dab23b2d6c7cf9cf3d3f3ab343",
     "grade": false,
     "grade_id": "cell-9282c2797da1931b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's draw a single sample of size 40 from the population and visualize it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e0bf0c2cb869152a84bf682768bd45e7",
     "grade": false,
     "grade_id": "cell-01de9069dec4f4e5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell \n",
    "set.seed(12345) \n",
    "one_sample <- can_seniors |> \n",
    "    rep_sample_n(40) |> \n",
    "    ungroup() |> # ungroup the data frame \n",
    "    select(age) # drop the replicate column \n",
    "one_sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b215227439420b58a83c8f2f0931c054",
     "grade": false,
     "grade_id": "cell-3f8eadc7f353c3fb",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "> ### An aside\n",
    "> To make this work with the `infer` package functions, we need to drop the first column (named `replicate`) as when we are bootstrapping we will be generating this column again. We also need to ungroup this data frame too...\n",
    ">\n",
    "> This is because in real life you would only have one sample (and hence no replicate column and the data would be ungrouped)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f3bfd7b5e52bd2aa0b69b66d41680375",
     "grade": false,
     "grade_id": "cell-1468c684182d5b3b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 8, repr.plot.height = 7)\n",
    "# run this cell \n",
    "one_sample_dist <- ggplot(one_sample, aes(age)) + \n",
    "    geom_histogram(binwidth = 1) +\n",
    "    xlab(\"Age (years)\") +\n",
    "    ggtitle(\"Distribution of one sample\") +\n",
    "    theme(text = element_text(size = 20))\n",
    "one_sample_dist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "3c1038bf12a8855d23fb0eca6812bb96",
     "grade": false,
     "grade_id": "cell-aaf103d19b9fba0d",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.5** \n",
    "<br> {points: 1}\n",
    "\n",
    "Use `summarise` to calculate the mean age (our point estimate of interest) from the random sample you just took (`one_sample`):\n",
    "\n",
    "Name this data frame `one_sample_estimates` which has a single column named `mean`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "5a31064545a044eb58ac989026152ecf",
     "grade": false,
     "grade_id": "cell-443a7d1dcaf35e9a",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "one_sample_estimates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "195ade455a8ba601a49b256622d5fc6a",
     "grade": true,
     "grade_id": "cell-8e9e83895e2967dd",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "9f9a64ea9ac7e0422d39631edf94ae4b",
     "grade": false,
     "grade_id": "cell-5575d82939196fc9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.6** \n",
    "<br> {points: 1}\n",
    "\n",
    "To generate a single bootstrap sample in R, we can use the `rep_sample_n` function as we did when we were creating our sampling distribution, however this time we change the argument for `replace` from its default value of `FALSE` to `TRUE`.\n",
    "\n",
    "Use `rep_sample_n` to take a single bootstrap sample from the sample you drew from the population. Name this bootstrap sample `boot1`.\n",
    "\n",
    "Use `4321` as the seed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "33a5a3eb8f8d0e6842cac2a440af70d9",
     "grade": false,
     "grade_id": "cell-239c20c4566f1f3c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "set.seed(4321)  # DO NOT CHANGE!\n",
    "# ... <- ... |>\n",
    "#    rep_sample_n(size = ..., replace = ..., reps = ...)\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "boot1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "55890f2548a34f988236080df9693494",
     "grade": true,
     "grade_id": "cell-af716b8959aee1ac",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.6()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b2bd40833acfd81d6a8ed328661931e6",
     "grade": false,
     "grade_id": "cell-7760b102574e0887",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.7** Multiple choice\n",
    "<br> {points: 1}\n",
    "\n",
    "Why do we change `replace` to `TRUE`?\n",
    "\n",
    "A. Taking a bootstrap sample involves drawing observations from the original population without replacement\n",
    "\n",
    "B. Taking a bootstrap sample involves drawing observations from the original population with replacement\n",
    "\n",
    "C. Taking a bootstrap sample involves drawing observations from the original sample without replacement\n",
    "\n",
    "D. Taking a bootstrap sample involves drawing observations from the original sample with replacement\n",
    "\n",
    "*Assign your answer to an object called `answer1.7`. Your answer should be an uppercase letter and is surrounded by quotes (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "9ce28d85cc67d60e7aecb7ffddd7b3a6",
     "grade": false,
     "grade_id": "cell-ba91c2de68609fe7",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "c5f3aaac6be510e847b75087dbcc7122",
     "grade": true,
     "grade_id": "cell-1b79e9b905b70215",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.7()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "2727ed76f31497eb9208c7a9ea39eeb4",
     "grade": false,
     "grade_id": "cell-93ea2b7d26e0e137",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.8** \n",
    "<br> {points: 1}\n",
    "\n",
    "Visualize the distribution of the bootstrap sample you just took (`boot1`) that was just created by plotting a histogram using `binwidth = 1` in the `geom_histogram` argument. Name the plot `boot1_dist` and give the plot (using `ggtitle`) and the x-axis a descriptive label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "8a6d9047d5ffccf0ea9af3fb48439b92",
     "grade": false,
     "grade_id": "cell-389271affd1fc5af",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 8, repr.plot.height = 7)\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "boot1_dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f1b03417aaa0fae40d39644333323390",
     "grade": true,
     "grade_id": "cell-3b7b4b4c4cc4971e",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.8()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c691f81e9028349e41fe15d875710e53",
     "grade": false,
     "grade_id": "cell-84ed6755705e4eee",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's now compare our bootstrap sample to the original random sample that we drew from the population:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "85a0ce5a6e58e250fb33edab5fc5e68b",
     "grade": false,
     "grade_id": "cell-afd154dc6f0c3e39",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this code cell\n",
    "options(repr.plot.width = 15, repr.plot.height = 7)\n",
    "plot_grid(one_sample_dist, boot1_dist, ncol = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b795147ba6a03c8d95909af6369648c5",
     "grade": false,
     "grade_id": "cell-fa64eb1f98987c99",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Earlier we calculate the mean of our original sample (`one_sample_estimates`). What is the mean of our bootstrap sample and how does it compare to the mean of our original sample? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "edb9d11243d19069931373a88bba6243",
     "grade": false,
     "grade_id": "cell-77b326e307cf1cf6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell\n",
    "one_sample_estimates\n",
    "\n",
    "boot1  |> \n",
    "    summarise(mean = mean(age))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a6a6f6afe4ff8ff4ffe61ab9de6af88b",
     "grade": false,
     "grade_id": "cell-c33dd58cb3c572af",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We see that original sample distrbution and the bootstrap sample distribution are of similar shape, but not identical. They also have different means. The difference of the frequency of the values in the bootstrap sample (and the difference of the value of the mean) comes from sampling from the original sample with replacement. Why sample with replacement? If we didn't we would end up with the original sample again. What we are trying to do with bootstrapping is to mimic drawing another sample from the population, without actually doing that. \n",
    "\n",
    "Why are we doing this? As mentioned earlier, in real life we typically only have one sample and thus we cannot create a sampling distribution that we can use to tell us about how we might expect our point estimate to behave if we took another sample. What we can do instead, is to use our sample as an estimate of our population, and sample from that with replacement (i.e., bootstrapping) many times to create many bootstrap samples. We can then calculate point estimates for each bootstrap sample and create a bootstrap distribution of our point estimates and use this as a proxy for a sampling distribution. We can finally use this bootstrap distribution of our point estimates to suggest how we might expected our point estimate to behave if we took another sample."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "d85c7cf5780e4d947daf78a973fd6f0f",
     "grade": false,
     "grade_id": "cell-f5b5d685693ffe7f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.9** \n",
    "<br> {points: 1}\n",
    "\n",
    "What do 6 different bootstrap samples look like? Use `rep_sample_n` to create a single data frame with 6 bootstrap samples of size 40 drawn from the original sample we drew from the population, `one_sample`. Name the data frame `boot6`.\n",
    "\n",
    "Set the seed as `1234`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b4a19d22d00ea9316b9fd2a1c8447b5c",
     "grade": false,
     "grade_id": "cell-6607b2dec92a2913",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "set.seed(1234)\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "head(boot6)\n",
    "tail(boot6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "c9f31d5e5bff9a93f70ed96a169c3dde",
     "grade": true,
     "grade_id": "cell-e57717e45df34f9a",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.9()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "6e41b8e06ffd13a87849b8e97830c4b1",
     "grade": false,
     "grade_id": "cell-558b102565ddf4ae",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.0** \n",
    "<br> {points: 1}\n",
    "\n",
    "Now visualize the six bootstrap sample distributions from `boot6` by faceting the `replicate` column. Name the plot object `boot6_dist` and give the plot (using `ggtitle`) and the x-axis a descriptive label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "a072040386a99e108151756fae2927a9",
     "grade": false,
     "grade_id": "cell-d1d36a4806776c6c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 15, repr.plot.height = 7)\n",
    "# ... <- ggplot(...) +\n",
    "#   ...(binwidth = 1) +\n",
    "#   facet_wrap(facets = vars(...)) +\n",
    "#   xlab(...) +\n",
    "#   ggtitle(...) +\n",
    "#   theme(text = element_text(size = 20))\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "boot6_dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "bff296c3af5da0c5cc92d1387d033852",
     "grade": true,
     "grade_id": "cell-38e6f9e544c5ce17",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.0()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e29a3cb5b825474caa170c374fc196c9",
     "grade": false,
     "grade_id": "cell-8c48012a69725219",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.1** \n",
    "<br> {points: 1}\n",
    "\n",
    "Calculate the mean of these 6 bootstrap samples using `group_by` and `summarize` the data into a column called `mean`. Name the data frame `boot6_means`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "a4598b10d8ff39dbe4e0107dbabdb338",
     "grade": false,
     "grade_id": "cell-b9692d80805c724a",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "boot6_means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4db30bd8c604c043807577d2d4792ca7",
     "grade": true,
     "grade_id": "cell-f2cfe994fece602d",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "dee11ff2078e4157511a721a42aa87a0",
     "grade": false,
     "grade_id": "cell-00909da5655c00a5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.2** \n",
    "<br> {points: 1}\n",
    "\n",
    "Let's now take 1000 bootstrap samples of size 40 from the original sample we drew from the population (`one_sample`) using `rep_sample_n`. Name the data frame `boot1000`.\n",
    "\n",
    "Set the seed as 1234."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "17ebd4223e1d427eb54c73441c18161a",
     "grade": false,
     "grade_id": "cell-cd89ca0c9e456e1b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "set.seed(1234)\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "head(boot1000)\n",
    "tail(boot1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6796142cbc5097af6e106f9d120ae52b",
     "grade": true,
     "grade_id": "cell-343d880b6f801822",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a56759fb2b6ecbef7697e415f0b81f66",
     "grade": false,
     "grade_id": "cell-85473b296ef14d1c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.3** \n",
    "<br> {points: 1}\n",
    "\n",
    "Calculate the mean of these 1000 bootstrap samples using `group_by` and `summarize` the data into a column called `mean`. Name the data frame `boot1000_means`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e9a6104422fdd4f0909795213eacba41",
     "grade": false,
     "grade_id": "cell-51d5f2afbb677369",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "head(boot1000_means)\n",
    "tail(boot1000_means)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "abcd2d6ef3d011a47e87b71a5367e5b2",
     "grade": true,
     "grade_id": "cell-bcf16788d57cef52",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "4ef71a42a34f6009ea4c8c53e172d7f7",
     "grade": false,
     "grade_id": "cell-75683253957d46bd",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.4** \n",
    "<br> {points: 1}\n",
    "\n",
    "Visualize the distribution of the bootstrap sample point estimates (`boot1000_means`) you just calculated by plotting a histogram using `binwidth = 1` in the `geom_histogram` argument. Name the plot `boot_est_dist` and give the plot (using `ggtitle`) and the x-axis a descriptive label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b6ac5d456d342a8b3ea1ceaf25603a61",
     "grade": false,
     "grade_id": "cell-21f093a752cc5d8c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 8, repr.plot.height = 7)\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "boot_est_dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "c6e6d9081b7bfc146d2439b62ecf574c",
     "grade": true,
     "grade_id": "cell-13a4f6d6fbcd8cbb",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.4()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ab6b0dcf3ba1c60460060f2f68a0dc2e",
     "grade": false,
     "grade_id": "cell-16863c73a0e61db1",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "How does the bootstrap distribution above compare to the sampling distribution? Let's visualize them side by side:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7aad1ca02b848b5bc34ef552c08d46d4",
     "grade": false,
     "grade_id": "cell-a86b551fb1260606",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell\n",
    "\n",
    "# create sampling distribution histogram\n",
    "set.seed(4321)\n",
    "samples <- rep_sample_n(can_seniors, size = 40, reps = 1500)\n",
    "sample_estimates <- samples |> \n",
    "    group_by(replicate) |> \n",
    "    summarise(mean = mean(age))\n",
    "sampling_dist <-  ggplot(sample_estimates, aes(x = mean)) +\n",
    "    geom_histogram(binwidth = 1) +\n",
    "    xlab(\"Mean Age (years)\") +\n",
    "    ggtitle(\"Sampling distribution of the sample means\") +\n",
    "    annotate(\"text\", x = 85, y = 200, label = paste(\"mean = \", round(mean(sample_estimates$mean), 1)), cex = 5) +\n",
    "    theme(text = element_text(size = 20))\n",
    "\n",
    "# annotate distribution mean to the bootstrap dist\n",
    "boot_est_dist <- boot_est_dist + \n",
    "    annotate(\"text\", x = 84, y = 160, label = paste(\"mean = \", round(mean(boot1000_means$mean), 1)), cex = 5)\n",
    "\n",
    "# plot bootstrap distribution beside sampling distribution\n",
    "options(repr.plot.width = 8, repr.plot.height = 7)\n",
    "plot_grid(boot_est_dist, sampling_dist, ncol = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "981aa4e76d780e72437a5024940a6544",
     "grade": false,
     "grade_id": "cell-2529f74ae181fd44",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Reminder: the true population quantity we are trying to estimate, the population mean, is about 79.3 years (as we see below). We know this because we created this population and calculated this value. In real life we wouldn't know this value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run this cell\n",
    "can_seniors  |> \n",
    "    summarize(mean(age))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c78b3c8fa60412e9f266b2c8367d4375",
     "grade": false,
     "grade_id": "cell-15147e6c224ccb72",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.5** True/False\n",
    "<br> {points: 1}\n",
    "\n",
    "The mean of the bootstrap distribution is the same value as the mean of the sampling distribution of the sample means. True or false?\n",
    "\n",
    "*Assign your answer to an object called `answer2.5`. Your answer should be in lowercase letters and is surrounded by quotes (e.g. `\"true\"` or `\"false\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e1de14e35ecb59a90145de4119c40c47",
     "grade": false,
     "grade_id": "cell-0e3ed3ca2495a0bc",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "bcd8ea6ebef1c201cd0eb09552b40e30",
     "grade": true,
     "grade_id": "cell-3ea212ad593b9f57",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c4e2a1c54cac55436bf9cf5c6f129fb7",
     "grade": false,
     "grade_id": "cell-8f170c7bb0c58227",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.6** True/False\n",
    "<br> {points: 1}\n",
    "\n",
    "The mean of the bootstrap distribution is not the same value as the mean of the sampling distribution because the bootstrap distribution was created from samples drawn from a single sample, whereas the sampling distribution was created from samples drawn from the population. True or false?\n",
    "\n",
    "*Assign your answer to an object called `answer2.6`. Your answer should be in lowercase letters and is surrounded by quotes (e.g. `\"true\"` or `\"false\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "64660e8fdc0245014f693ca58bd0c496",
     "grade": false,
     "grade_id": "cell-230cf28d4590a63d",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2a49b70ce59f34cb0b00965a14c1ab35",
     "grade": true,
     "grade_id": "cell-b167494acba2816d",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.6()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "fc1fe3e9d9984ee497744c4dd9c7ed73",
     "grade": false,
     "grade_id": "cell-c569dabb88890bfc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.7** True/False\n",
    "<br> {points: 1}\n",
    "\n",
    "The shape and spread (i.e. width) of the bootstrap distribution is a poor approximation of the  shape and spread of the sampling distribution of the sample means. True or false?\n",
    "\n",
    "*Assign your answer to an object called `answer2.7`. Your answer should be in lowercase letters and is surrounded by quotes (e.g. `\"true\"` or `\"false\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b0f8e9cb17ebe6e95b6c30e5deea0c5a",
     "grade": false,
     "grade_id": "cell-f82d46022821b073",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7cfc7f258f2e1e6dae0541dea62f6fa8",
     "grade": true,
     "grade_id": "cell-2ab1350547ae451d",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.7()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "5744a22ea4f1d9c9154caf73fd584499",
     "grade": false,
     "grade_id": "cell-8399b964d743f33c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.8** True/False\n",
    "<br> {points: 1}\n",
    "\n",
    "In real life, where we only have one sample and cannot create a sampling distribution, the distribution of the bootstrap sample estimates (here means) can suggest how we might expect our point estimate to behave if we took another sample. True or false?\n",
    "\n",
    "*Assign your answer to an object called `answer2.8`. Your answer should be in lowercase letters and is surrounded by quotes (e.g. `\"true\"` or `\"false\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ec231afb1a2a657a5a9fb9f00247e918",
     "grade": false,
     "grade_id": "cell-59cfd94899e457a7",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "5d1013b0eb48689ab2f877f4cca971ee",
     "grade": true,
     "grade_id": "cell-4d2a0325f4fd2406",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.8()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e1922c201ead42beb140513403972276",
     "grade": false,
     "grade_id": "cell-3af745f41b79a813",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Using the bootstrap distribution to calculate a plausible range for point estimates\n",
    "\n",
    "Once we have created a bootstrap distribution, we can use it to suggest a plausible range where we might expect the true population quantity to lie. One formal name for a commonly used plausible range is called a confidence interval. Confidence intervals can be set at different levels, an example of a commonly used level is 95%. When we report a point estimate with a 95% confidence interval as the plausible range, formally we are saying that if we repeated this process of building confidence intervals more times with more samples, we’d expect ~ 95% of them to contain the value of the population quantity.\n",
    "\n",
    "> How do you choose a level for a confidence interval? You have to consider the downstream application of your estimation and what the cost/consequence of an incorrect estimate would be. The higher the cost/consequence, the higher a confidence level you would want to use. You will learn more about this in later Statistics courses.\n",
    "\n",
    "To calculate an approximate 95% confidence interval using bootstrapping, we essentially order the values in our bootstrap distribution and then take the value at the 2.5th percentile as the lower bound of the plausible range, and the 97.5th percentile as the upper bound of the plausible range. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f52ef91c806996c82e16ceadeb692649",
     "grade": false,
     "grade_id": "cell-4e8bd702d14d0973",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell\n",
    "boot1000_means |> \n",
    "    select(mean) |> \n",
    "    pull() |> \n",
    "    quantile(c(0.025, 0.975))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "845327498fa40c0e47d4466296a41291",
     "grade": false,
     "grade_id": "cell-671ea52cb570ec80",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Thus, to finish our estimation of the population quantity that we are trying to estimate, we would report the point estimate and the lower and upper bounds of our confidence interval. We would say something like this:\n",
    "\n",
    "Our sample mean age for Canadian seniors was measured to be 77.8 years, and we’re 95% “confident” that the true population mean for Canadian seniors is between (73.7, 82.0). \n",
    "\n",
    "Here our 95% confidence interval does contain the true population mean for Canadian seniors, 79.3 years - pretty neat! However, in real life we would never be able to know this because we only have observations from a single sample, not the whole population."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "161310c3988771ee591d791bfa0efb53",
     "grade": false,
     "grade_id": "cell-9453844511483837",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.9** True/False\n",
    "<br> {points: 1}\n",
    "\n",
    "Assuming we knew the true population quantity we are trying to estimate (so we could verify this):\n",
    "\n",
    "For any sample we take, if we use bootstrapping to calculate the 95% confidence intervals, the true population quantity we are trying to estimate would always fall within the lower and upper bounds of the confidence interval. True or false?\n",
    "\n",
    "*Assign your answer to an object called `answer2.9`. Your answer should be in lowercase letters and is surrounded by quotes (e.g. `\"true\"` or `\"false\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "3ca6fca52a24b2e2fd616cbbf0e84bed",
     "grade": false,
     "grade_id": "cell-91e462917a53edad",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "dd44b7097d2ceac232b7491acc8b0902",
     "grade": true,
     "grade_id": "cell-e0a24858ef65cea9",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.9()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "673706940aa4886e8b25ef66041f5ab6",
     "grade": false,
     "grade_id": "cell-0cd6dc906d5624fc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "source(\"cleanup.R\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.3.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}