One Sample z- and t-tests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One-Sample t-tests\n",
    "One sample t-tests are used to compare the mean of some data with a known value.  Because it is unlikely that you will be comparing your sample data with a known population mean that has a known population standard error, we will focus on t- and not z-tests."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "-- Attaching packages --------------------------------------- tidyverse 1.2.1 --\n",
      "v ggplot2 3.2.1     v purrr   0.3.2\n",
      "v tibble  2.1.3     v dplyr   0.8.3\n",
      "v tidyr   0.8.3     v stringr 1.4.0\n",
      "v readr   1.3.1     v forcats 0.4.0\n",
      "-- Conflicts ------------------------------------------ tidyverse_conflicts() --\n",
      "x dplyr::filter() masks stats::filter()\n",
      "x dplyr::lag()    masks stats::lag()\n"
     ]
    }
   ],
   "source": [
    "library(tidyverse)\n",
    "library(DescTools)\n",
    "library(plotrix)\n",
    "library(lsr)\n",
    "library(pwr)\n",
    "library(readxl)\n",
    "\n",
    "options(repr.plot.width=5, repr.plot.height=5) ## set options for plot size within the notebook -\n",
    "# this is only for jupyter notebooks, you can disregard this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use the mtcars dataset for this example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "head(mtcars)  ## remind ourselves of the setup of the mtcars dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One-Sample t-test\n",
    "We want to know if the average mpg of the cars in our sample `mtcars` is lower than the current national average of 24.7 mpg.\n",
    "\n",
    "How would we run a hypothesis test for this?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1 - Formulate Hypothesis\n",
    "\n",
    "Population mean is $\\mu$, which specified in our question - 24.7mpg\n",
    "\n",
    "Sample mean is $\\bar{x}$ which we will calculate from our data.\n",
    "\n",
    "$H_0 : \\bar{x} = 24.7$\n",
    "\n",
    "$H_A : \\bar{x} < 24.7$\n",
    "\n",
    "**Note:** Given our $H_A$ we're running a one-tailed test."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Step 2 - Prepare and Check Conditions\n",
    "\n",
    "Set alpha ->>> $\\alpha = 0.05$\n",
    "\n",
    "Random and independent sample ->>> assumed for now\n",
    "\n",
    "Sample is <10% of the population? ->>> Yes\n",
    "\n",
    "Let's take a look at our distribution and summary statistics first to get an idea of our sample data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## IS OUR SAMPLE DISTRIBUTION NORMAL?\n",
    "qqnorm(mtcars$mpg) ## calculate QQ values\n",
    "qqline(mtcars$mpg, col=\"red\")  ## create QQ plot\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(mtcars$mpg) ## summary statistics - mean, median, min, max, etc.\n",
    "sd(mtcars$mpg) ## standard deviation\n",
    "std.error(mtcars$mpg) ## standard error\n",
    "\n",
    "## show that the standard error is sd/sqrt(n)\n",
    "sd(mtcars$mpg)/sqrt(length(mtcars$mpg))\n",
    "\n",
    "## what is our n?\n",
    "length(mtcars$mpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our sample data deviates from normality a bit in the tails, but it's actually not that bad in terms of approximating a normal distribution.\n",
    "\n",
    "So we can already see that our sample mean is 20.09, which is lower than 24.7, but is the difference statistically significant?\n",
    "\n",
    "### Step 3 - Calculate t-statistic and p-value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## using t.test to calculate one-sample t-statistic - t.test(VECTOR, mu, alternative = \"one.sided\" or\" less\" or \"greater\",\n",
    "                                                                                                # conf= 1- alpha)\n",
    "\n",
    "mpg_less_test <- t.test(mtcars$mpg, mu = 24.7, alternative = \"less\", conf = 0.95)\n",
    "mpg_less_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**So what we see in the output is:**\n",
    "\n",
    "our t-statistic/t-value is -4.3263\n",
    "\n",
    "we have 31 degrees of freedom (n-1)\n",
    "\n",
    "It tells us in words what our H_A is\n",
    "\n",
    "and it gives us a 95% CI.  This is the 95% CI related to the \"less than\" hypothesis, therefore it starts at -Inf, because we're looking at that side of the distribution.  \n",
    "\n",
    "If we wanted a 95% CI as a range, we'd run a two-sided test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## rerunning t.test to obtain centered 95% CI\n",
    "\n",
    "t.test(mtcars$mpg, mu = 24.7, alternative = \"two.sided\", conf = 0.95)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This gives a 95% CI centered around our sample mean, but it also is testing a different alternative hypothesis, whether the mean is different from 24.7 in any direction.\n",
    "\n",
    "### Going back to our one-sided test, lets graph our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dof <- mpg_less_test$parameter\n",
    "tvalue <- mpg_less_test$statistic\n",
    "tcrit <- qt(0.05, df = dof)\n",
    "\n",
    "ggplot(data.frame(x = c(-5, 5)), aes(x = x)) + \n",
    "    stat_function(fun = dt, args = list(0.05,df = dof)) +  ## dt is the density function for t\n",
    "     stat_function(fun = dt, \n",
    "                  args = list(0.05, df = dof),\n",
    "                  xlim = c(-5,tcrit),  \n",
    "                  geom = \"area\",\n",
    "                  alpha=0.2) +\n",
    "    labs(x = \"t\",y=\"\") + \n",
    "    geom_vline(xintercept = tvalue, color = \"red\") +\n",
    "    annotate(geom=\"text\", x=tcrit, y=-0.004, label=paste(\"critical t = \", round(tcrit, digits = 3)),\n",
    "             color=\"blue\", fontface = 2, size = 4bel=paste(\"t-value =\", round(tvalue, digits = 2)),\n",
    "             color=\"red\", fontface = 2, size = 4)) +  ## x and y tell ggplot the coords to place your words\n",
    "    annotate(geom=\"text\", x=tvalue+1.5, y=0.2, la"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Effect Size\n",
    "\n",
    "We now want to know how big the difference is between our sample mean and the population mean.  For this we will calculate Cohen's d."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cohensD(mtcars$mpg, 24.7) ## cohen's d function, two arguments, our vector of data, and our population mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our \"rule of thumb\" for interpreting Cohen's d is:\n",
    "\n",
    "| Cohen's d | Effect Size |\n",
    "|:---------:|:-----------:|\n",
    "| 0.20 | Small |\n",
    "| 0.50 | Medium|\n",
    "| 0.80 | Large |\n",
    "\n",
    "So our value of 0.765 is consistent with a decent effect size (e.g. difference between sample mean and null hypothesis mean).  Would you consider a difference of about 4.7 mpg a difference that would be important in the real world?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Power\n",
    "\n",
    "Finally, we want to see how much power our test has.  To conduct a power analysis we need 5 of the 6 pieces of information:\n",
    "\n",
    "- n: sample size\n",
    "- d: effect size (in this case cohensD)\n",
    "- sig.level: alpha\n",
    "- power: power (1-$\\beta$)\n",
    "- alternative: type of alternative hypothesis (two.sided, less, or greater)\n",
    "- type: right now we're only looking at one.sample tests\n",
    "\n",
    "When we run the power function we set one of these things to NULL, to calculate that value based on the other inputs.  Typically we will either be wanting to calculate a sample size *a priori* or to calculate the power of an analysis after you conduct the analysis.\n",
    "\n",
    "We want to get the power for our mpg analysis, so we know our arguments will be:\n",
    "\n",
    "- n = 32\n",
    "- d = 0.765\n",
    "- sig.level = 0.05\n",
    "- power = NULL\n",
    "- alternative = \"two-sided\" - Let's look at the more \"conservative\" version of the test.\n",
    "- type = \"one.sample\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x <- pwr.t.test(d = 0.765, n = 32, sig.level=0.05, power = NULL, alternative = \"two.sided\", type = \"one.sample\")\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our power is extremely high.  If we only wanted to have a power of 0.80 we wouldn't have needed as large of a sample, given our effect size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pwr.t.test(d = 0.765, n = NULL, sig.level=0.05, power = 0.80, alternative = \"two.sided\", type = \"one.sample\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One Sample t-test - Proportions\n",
    "\n",
    "At the most basic level a proportion is a mean of a vector of 0/1 observations.  Given this, we can also do t-tests to calculate the difference in proportions.\n",
    "\n",
    "For this we'll use the small_gss dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>id_</th><th scope=col>year</th><th scope=col>abdefect</th><th scope=col>abnomore</th><th scope=col>abhlth</th><th scope=col>abpoor</th><th scope=col>abrape</th><th scope=col>absingle</th><th scope=col>abany</th><th scope=col>realinc</th><th scope=col>...</th><th scope=col>childs</th><th scope=col>age</th><th scope=col>educ</th><th scope=col>sex</th><th scope=col>partyid</th><th scope=col>spkath</th><th scope=col>spkrac</th><th scope=col>spkcom</th><th scope=col>spkmil</th><th scope=col>sexnow</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>1                 </td><td>2016              </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>131676.7          </td><td>...               </td><td>3                 </td><td>47                </td><td>16                </td><td>Male              </td><td>Independent       </td><td>Allowed           </td><td>Allowed           </td><td>Allowed           </td><td>Allowed           </td><td>Not applicable    </td></tr>\n",
       "\t<tr><td>2                 </td><td>2016              </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td> 25740.0          </td><td>...               </td><td>0                 </td><td>61                </td><td>12                </td><td>Male              </td><td>Ind,near dem      </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td></tr>\n",
       "\t<tr><td>3                 </td><td>2016              </td><td>No                </td><td>No                </td><td>Yes               </td><td>No                </td><td>Yes               </td><td>No                </td><td>No                </td><td> 38610.0          </td><td>...               </td><td>2                 </td><td>72                </td><td>16                </td><td>Male              </td><td>Not str republican</td><td>Allowed           </td><td>Allowed           </td><td>Allowed           </td><td>Allowed           </td><td>Not applicable    </td></tr>\n",
       "\t<tr><td>4                 </td><td>2016              </td><td>No                </td><td>No                </td><td>Yes               </td><td>No                </td><td>Yes               </td><td>No                </td><td>No                </td><td>131676.7          </td><td>...               </td><td>4                 </td><td>43                </td><td>12                </td><td>Female            </td><td>Not str republican</td><td>Not allowed       </td><td>Not allowed       </td><td>Not allowed       </td><td>Allowed           </td><td>Not applicable    </td></tr>\n",
       "\t<tr><td>5                 </td><td>2016              </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>Yes               </td><td>131676.7          </td><td>...               </td><td>2                 </td><td>55                </td><td>18                </td><td>Female            </td><td>Not str democrat  </td><td>Allowed           </td><td>Not allowed       </td><td>Allowed           </td><td>Not allowed       </td><td>Not applicable    </td></tr>\n",
       "\t<tr><td>6                 </td><td>2016              </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td> 31590.0          </td><td>...               </td><td>2                 </td><td>53                </td><td>14                </td><td>Female            </td><td>Not str democrat  </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td><td>Not applicable    </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllllllllllllllll}\n",
       " id\\_ & year & abdefect & abnomore & abhlth & abpoor & abrape & absingle & abany & realinc & ... & childs & age & educ & sex & partyid & spkath & spkrac & spkcom & spkmil & sexnow\\\\\n",
       "\\hline\n",
       "\t 1                  & 2016               & Yes                & Yes                & Yes                & Yes                & Yes                & Yes                & Yes                & 131676.7           & ...                & 3                  & 47                 & 16                 & Male               & Independent        & Allowed            & Allowed            & Allowed            & Allowed            & Not applicable    \\\\\n",
       "\t 2                  & 2016               & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable     &  25740.0           & ...                & 0                  & 61                 & 12                 & Male               & Ind,near dem       & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable    \\\\\n",
       "\t 3                  & 2016               & No                 & No                 & Yes                & No                 & Yes                & No                 & No                 &  38610.0           & ...                & 2                  & 72                 & 16                 & Male               & Not str republican & Allowed            & Allowed            & Allowed            & Allowed            & Not applicable    \\\\\n",
       "\t 4                  & 2016               & No                 & No                 & Yes                & No                 & Yes                & No                 & No                 & 131676.7           & ...                & 4                  & 43                 & 12                 & Female             & Not str republican & Not allowed        & Not allowed        & Not allowed        & Allowed            & Not applicable    \\\\\n",
       "\t 5                  & 2016               & Yes                & Yes                & Yes                & Yes                & Yes                & Yes                & Yes                & 131676.7           & ...                & 2                  & 55                 & 18                 & Female             & Not str democrat   & Allowed            & Not allowed        & Allowed            & Not allowed        & Not applicable    \\\\\n",
       "\t 6                  & 2016               & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable     &  31590.0           & ...                & 2                  & 53                 & 14                 & Female             & Not str democrat   & Not applicable     & Not applicable     & Not applicable     & Not applicable     & Not applicable    \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| id_ | year | abdefect | abnomore | abhlth | abpoor | abrape | absingle | abany | realinc | ... | childs | age | educ | sex | partyid | spkath | spkrac | spkcom | spkmil | sexnow |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 1                  | 2016               | Yes                | Yes                | Yes                | Yes                | Yes                | Yes                | Yes                | 131676.7           | ...                | 3                  | 47                 | 16                 | Male               | Independent        | Allowed            | Allowed            | Allowed            | Allowed            | Not applicable     |\n",
       "| 2                  | 2016               | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     |  25740.0           | ...                | 0                  | 61                 | 12                 | Male               | Ind,near dem       | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     |\n",
       "| 3                  | 2016               | No                 | No                 | Yes                | No                 | Yes                | No                 | No                 |  38610.0           | ...                | 2                  | 72                 | 16                 | Male               | Not str republican | Allowed            | Allowed            | Allowed            | Allowed            | Not applicable     |\n",
       "| 4                  | 2016               | No                 | No                 | Yes                | No                 | Yes                | No                 | No                 | 131676.7           | ...                | 4                  | 43                 | 12                 | Female             | Not str republican | Not allowed        | Not allowed        | Not allowed        | Allowed            | Not applicable     |\n",
       "| 5                  | 2016               | Yes                | Yes                | Yes                | Yes                | Yes                | Yes                | Yes                | 131676.7           | ...                | 2                  | 55                 | 18                 | Female             | Not str democrat   | Allowed            | Not allowed        | Allowed            | Not allowed        | Not applicable     |\n",
       "| 6                  | 2016               | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     |  31590.0           | ...                | 2                  | 53                 | 14                 | Female             | Not str democrat   | Not applicable     | Not applicable     | Not applicable     | Not applicable     | Not applicable     |\n",
       "\n"
      ],
      "text/plain": [
       "  id_ year abdefect       abnomore       abhlth         abpoor        \n",
       "1 1   2016 Yes            Yes            Yes            Yes           \n",
       "2 2   2016 Not applicable Not applicable Not applicable Not applicable\n",
       "3 3   2016 No             No             Yes            No            \n",
       "4 4   2016 No             No             Yes            No            \n",
       "5 5   2016 Yes            Yes            Yes            Yes           \n",
       "6 6   2016 Not applicable Not applicable Not applicable Not applicable\n",
       "  abrape         absingle       abany          realinc  ... childs age educ\n",
       "1 Yes            Yes            Yes            131676.7 ... 3      47  16  \n",
       "2 Not applicable Not applicable Not applicable  25740.0 ... 0      61  12  \n",
       "3 Yes            No             No              38610.0 ... 2      72  16  \n",
       "4 Yes            No             No             131676.7 ... 4      43  12  \n",
       "5 Yes            Yes            Yes            131676.7 ... 2      55  18  \n",
       "6 Not applicable Not applicable Not applicable  31590.0 ... 2      53  14  \n",
       "  sex    partyid            spkath         spkrac         spkcom        \n",
       "1 Male   Independent        Allowed        Allowed        Allowed       \n",
       "2 Male   Ind,near dem       Not applicable Not applicable Not applicable\n",
       "3 Male   Not str republican Allowed        Allowed        Allowed       \n",
       "4 Female Not str republican Not allowed    Not allowed    Not allowed   \n",
       "5 Female Not str democrat   Allowed        Not allowed    Allowed       \n",
       "6 Female Not str democrat   Not applicable Not applicable Not applicable\n",
       "  spkmil         sexnow        \n",
       "1 Allowed        Not applicable\n",
       "2 Not applicable Not applicable\n",
       "3 Allowed        Not applicable\n",
       "4 Allowed        Not applicable\n",
       "5 Not allowed    Not applicable\n",
       "6 Not applicable Not applicable"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "small_gss <- read_xls(\"small_gss.xls\")\n",
    "head(small_gss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll look at the proportion of female respondents who report that they support abortion for any reason `abany`.  We'll compare our sample mean to the value of 61% support as reported by Pew - https://www.pewforum.org/fact-sheet/public-opinion-on-abortion/\n",
    "\n",
    "Before we can conduct our analysis I'm going to do some quick data cleaning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>abany</th><th scope=col>n</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>No            </td><td>1733          </td></tr>\n",
       "\t<tr><td>Not applicable</td><td>1751          </td></tr>\n",
       "\t<tr><td>Yes           </td><td>1601          </td></tr>\n",
       "\t<tr><td>NA            </td><td> 130          </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       " abany & n\\\\\n",
       "\\hline\n",
       "\t No             & 1733          \\\\\n",
       "\t Not applicable & 1751          \\\\\n",
       "\t Yes            & 1601          \\\\\n",
       "\t NA             &  130          \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| abany | n |\n",
       "|---|---|\n",
       "| No             | 1733           |\n",
       "| Not applicable | 1751           |\n",
       "| Yes            | 1601           |\n",
       "| NA             |  130           |\n",
       "\n"
      ],
      "text/plain": [
       "  abany          n   \n",
       "1 No             1733\n",
       "2 Not applicable 1751\n",
       "3 Yes            1601\n",
       "4 NA              130"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>abany</th><th scope=col>n</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>0  </td><td>975</td></tr>\n",
       "\t<tr><td>1  </td><td>875</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       " abany & n\\\\\n",
       "\\hline\n",
       "\t 0   & 975\\\\\n",
       "\t 1   & 875\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| abany | n |\n",
       "|---|---|\n",
       "| 0   | 975 |\n",
       "| 1   | 875 |\n",
       "\n"
      ],
      "text/plain": [
       "  abany n  \n",
       "1 0     975\n",
       "2 1     875"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "small_gss  %>% count(abany) #pre-recoding counts\n",
    "df <- small_gss %>% ## save recoded version as df\n",
    "    filter (abany != \"Not applicable\")  %>% #filter out rows where abany was missing (because the question wasn't asked)\n",
    "    filter (sex == \"Female\")  %>%    ## we've defined our population as women only, so we'll subset to just those observations\n",
    "    mutate (abany = ifelse(abany == \"Yes\", 1, 0)) ## using ifelse to convert chr yes/no to numerical 1/0 \n",
    "                                                ## the format of the function is ifelse(test, valiftrue, valiffalse)\n",
    "df %>%count(abany)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1 - Formulate Hypothesis\n",
    "\n",
    "Population proportion is $p_u$, which specified in our question - 0.61 (61%)\n",
    "\n",
    "Sample proportion is $p_s$ which we will calculate from our data.\n",
    "\n",
    "$H_0 : p_s = 0.61$\n",
    "\n",
    "$H_A : p_s \\neq 0.61$\n",
    "\n",
    "Given our $H_A$ we're running a two-tailed test."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2 - Prepare and Check Conditions\n",
    "\n",
    "Set alpha ->>> $\\alpha = 0.05$\n",
    "\n",
    "Random and independent sample ->>> assumed for now\n",
    "\n",
    "Sample is <10% of the population? ->>> Yes\n",
    "\n",
    "Let's take a look at our distribution and summary statistics first to get an idea of our sample data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.472972972972973"
      ],
      "text/latex": [
       "0.472972972972973"
      ],
      "text/markdown": [
       "0.472972972972973"
      ],
      "text/plain": [
       "[1] 0.472973"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "0.0116109071036525"
      ],
      "text/latex": [
       "0.0116109071036525"
      ],
      "text/markdown": [
       "0.0116109071036525"
      ],
      "text/plain": [
       "[1] 0.01161091"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "0.0116077685964716"
      ],
      "text/latex": [
       "0.0116077685964716"
      ],
      "text/markdown": [
       "0.0116077685964716"
      ],
      "text/plain": [
       "[1] 0.01160777"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "1850"
      ],
      "text/latex": [
       "1850"
      ],
      "text/markdown": [
       "1850"
      ],
      "text/plain": [
       "[1] 1850"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mean(df$abany) ## mean of 0/1 variable is proportion saying \"yes\"\n",
    "p <- mean(df$abany) ## save prop to use below\n",
    "\n",
    "std.error(df$abany) ## standard error\n",
    "## show that the standard error is square root of p_s(1-p_s)/n\n",
    "sqrt(p*(1-p)/length(df$abany))\n",
    "\n",
    "## what is our n?\n",
    "length(df$abany)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we can already see that our sample proportion is 0.47, which seems much different from 0.61, but is the difference statistically significant?\n",
    "\n",
    "(SPOILER ALERT: it has to be, like really...)\n",
    "\n",
    "### Step 3 - Calculate t-statistic and p-value\n",
    "\n",
    "The correct way to compare a one-sample proportion with a given probability to test against, is by using `prop.test()`.\n",
    "\n",
    "#### prop.test()\n",
    "`prop.test()` takes 5 arguments:\n",
    "\n",
    "- x = the total number of \"yes\" or \"success\" or ones\n",
    "- n = the total number of observations\n",
    "- p = the probability to test against (default is 0.5)\n",
    "- alternative = the type of test to conduct (\"two.sided\", \"less\", or \"greater\") depending on your alternative hypothesis\n",
    "- correct = optional to use Yates' continuity correction - default is TRUE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "\t1-sample proportions test with continuity correction\n",
       "\n",
       "data:  x out of n, null probability p\n",
       "X-squared = 145.44, df = 1, p-value < 2.2e-16\n",
       "alternative hypothesis: true p is not equal to 0.61\n",
       "95 percent confidence interval:\n",
       " 0.4500326 0.4960266\n",
       "sample estimates:\n",
       "       p \n",
       "0.472973 \n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# prop.test(x, n, p = NULL, alternative = \"two.sided\", correct = TRUE)\n",
    "x = sum(df$abany) ## summing a 0/1 variable gives you the count of \"yesses\"\n",
    "n = length(df$abany) ## length of a vector gives you the number of values in that vector\n",
    "p = 0.61 ## the probability we want to test against\n",
    "prop.test(x, n, p = p, alternative = \"two.sided\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our output shows a chi-squared value, not a t-value!  That's because under the hood R is calculating this using the chi-square distribution because we're looking at frequencies (counts).  Our extremely low p-value indicates that we can reject the null hypothesis (no surprise).\n",
    "\n",
    "#### t.test() with 0/1 data\n",
    "\n",
    "Given that our proportion is a mean of a 0/1 variable, we can simply use our t-test function and obtain similar results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "\tOne Sample t-test\n",
       "\n",
       "data:  df$abany\n",
       "t = -11.802, df = 1849, p-value < 2.2e-16\n",
       "alternative hypothesis: true mean is not equal to 0.61\n",
       "95 percent confidence interval:\n",
       " 0.4502011 0.4957448\n",
       "sample estimates:\n",
       "mean of x \n",
       " 0.472973 \n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "prop_t <- t.test(df$abany, mu = 0.61, alternative = \"two.sided\", conf = 0.95) ### remember we can save our results\n",
    "prop_t ## and then print them"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see almost identical results, but this time we have a t-value and our alternative hypothesis references a mean of 0.61.\n",
    "\n",
    "In that block I saved the results to an object called `prop_t`.  Remember when we did the chi-square test that that object held all of the information outputted from the test.  Let's look at the structure of that object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "List of 10\n",
      " $ statistic  : Named num -11.8\n",
      "  ..- attr(*, \"names\")= chr \"t\"\n",
      " $ parameter  : Named num 1849\n",
      "  ..- attr(*, \"names\")= chr \"df\"\n",
      " $ p.value    : num 4.83e-31\n",
      " $ conf.int   : num [1:2] 0.45 0.496\n",
      "  ..- attr(*, \"conf.level\")= num 0.95\n",
      " $ estimate   : Named num 0.473\n",
      "  ..- attr(*, \"names\")= chr \"mean of x\"\n",
      " $ null.value : Named num 0.61\n",
      "  ..- attr(*, \"names\")= chr \"mean\"\n",
      " $ stderr     : num 0.0116\n",
      " $ alternative: chr \"two.sided\"\n",
      " $ method     : chr \"One Sample t-test\"\n",
      " $ data.name  : chr \"df$abany\"\n",
      " - attr(*, \"class\")= chr \"htest\"\n"
     ]
    }
   ],
   "source": [
    "str(prop_t) ## look at the structure of the t.test output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get all of the \"pieces\" of the result as parts of the object - it's a list where we have key:value pairs.  We can use those keys to obtain the single pieces of information we want."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<strong>t:</strong> -11.8015781027067"
      ],
      "text/latex": [
       "\\textbf{t:} -11.8015781027067"
      ],
      "text/markdown": [
       "**t:** -11.8015781027067"
      ],
      "text/plain": [
       "        t \n",
       "-11.80158 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong>df:</strong> 1849"
      ],
      "text/latex": [
       "\\textbf{df:} 1849"
      ],
      "text/markdown": [
       "**df:** 1849"
      ],
      "text/plain": [
       "  df \n",
       "1849 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>0.45020110681115</li>\n",
       "\t<li>0.495744839134796</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 0.45020110681115\n",
       "\\item 0.495744839134796\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 0.45020110681115\n",
       "2. 0.495744839134796\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 0.4502011 0.4957448\n",
       "attr(,\"conf.level\")\n",
       "[1] 0.95"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "prop_t$statistic # t value/statistic\n",
    "prop_t$parameter # degrees of freedom\n",
    "prop_t$conf.int # confidence interval"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Effect Size - Comparing Proportions\n",
    "\n",
    "To compare our proportions, we should look at Cohen's H, which is a non-directional magnitude of difference in proportions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "-0.27589488211097"
      ],
      "text/latex": [
       "-0.27589488211097"
      ],
      "text/markdown": [
       "-0.27589488211097"
      ],
      "text/plain": [
       "[1] -0.2758949"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### The function is ES.h(ps,pu) - remember ps is sample prop. and pu is population (universe) prop.\n",
    "\n",
    "ES.h(mean(df$abany),0.61)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.274381117578083"
      ],
      "text/latex": [
       "0.274381117578083"
      ],
      "text/markdown": [
       "0.274381117578083"
      ],
      "text/plain": [
       "[1] 0.2743811"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cohensD(df$abany, 0.61)  ## try the Cohen's D, which is the mean effect size."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We calculated Cohen's H, for proportions, and also Cohen's D to compare (given that the mean of 0/1 data is a proportion).  We got very similar results in terms of absolute magnitude.  The negative sign for the Cohen's H result is because it's a uni-directional test.  However I wouldn't rely on using Cohen's D for proportion data unless you find that they are virtually identical."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Power\n",
    "\n",
    "We can also calculate the power of a comparison of proportions.  All of the arguments are similar to those above, except notice we're using a different formula (pwr.<b>p</b>.test, not pwr.<b>t</b>.test) and we have <b>h</b> as the effect size, not <b>d</b>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "     proportion power calculation for binomial distribution (arcsine transformation) \n",
       "\n",
       "              h = 0.2758949\n",
       "              n = 1850\n",
       "      sig.level = 0.05\n",
       "          power = 1\n",
       "    alternative = two.sided\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "h1 <- ES.h(mean(df$abany), 0.61)\n",
    "n1 <- length(df$abany)\n",
    "\n",
    "\n",
    "pwr.p.test(h = h1, n= n1, sig.level=0.05, power=NULL, alternative=\"two.sided\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our difference is so big that our power is virtually 1.  Which means we have pretty much no probability of Type II error.  Let's compare to using the `pwr.t.test()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "     One-sample t test power calculation \n",
       "\n",
       "              n = 1850\n",
       "              d = 0.2743811\n",
       "      sig.level = 0.05\n",
       "          power = 1\n",
       "    alternative = two.sided\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d1 <- cohensD(df$abany, 0.61)\n",
    "\n",
    "pwr.t.test(d = d1, \n",
    "           n = n1, ## same n1 we set in the block above\n",
    "           sig.level=0.05, \n",
    "           power = NULL, \n",
    "           alternative = \"two.sided\", \n",
    "           type = \"one.sample\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get essentially the same result."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### BONUS CONTENT: Power Plots\n",
    "\n",
    "Power plots show us the range of sample sizes for a range of levels of power.  \"Optimal\" sample size line is the number of observations you would need to achieve the power specified in the function call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAMAAACJuGjuAAAAPFBMVEUAAAAAAIszMzNNTU1o\naGh8fHyMjIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enr6+vw8PD/AAD///+z979qAAAA\nCXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2diZqjIBCEyZpMMjM5x/d/1/UWUJTbBqq+\n3cQolq38o9iishqCAogdHQCUpwAWFEQACwoigAUFEcCCgghgQUEEsKAgAlhQEAEsKIgAFhRE\nAcBinPy7z0tZG/u5rg9vFgwl9erLU/poQm6u6MoMLH7s5tJjVKI+WP1vgKXjG3ojZQhWVgJY\n4QSwQvj2xg927b8e7de1/XpeK1Zdn3zh55U1o7oi9e3M2OUxWHxX7Hyv65/m66cf0wxe7vMC\nHhc2/B6WOh9+p2GuzLQg4TjN23LxrcQ+WzH2ObOvacH3ZsL1s1wBftKwwParH5xKDtEMBbgN\n1Iy59ZsgOYXeY1X9JmS3bmTVbOeh9cVtrAc3qpoHGfvuhh/X7utnHnMbF/DLpt/DUpdgcWXm\nBUlgzbZ8fIvYOSvGvrjl3rrxVS2vgDBJBmsuKYDFb6AGPHljpaLQYH0PuLRb9s6+62dTi5/6\n02zu11T2zH7rttbObfHvbqZLZ1Hd24qp+q9zN6Ypea/Ys1/As/uDf17mDb88FPJluAWJBWdb\nPr5l7LNVU+WfyeDZRvq5tKCJKyBMksCSSw6fwgZqInvUn6+uTGIKDdarPZ682LndTNfmY/j7\nb4Zvi7J1W/WfaUz/h/oZ/l77+uhprIZquvWlP/MxaQkWX4abKhacbfn4lrHzVo/Z4NruTpvx\nZ3kFhEkSWHLJaQ/HbaBpE+xsbYIK3ni/dH+cv+2fZ/uHdx4q8dVu6kFfTZPid9qBPe/fl2Fr\nfzinrj6Gv9yv81g3clpjCRZfhluQUJCzFeJbxD5bCXU9MLJcAWHSoo0llhw+hQC4WVJTcLB+\nmz+6M6ubzXRvD0RcE6geK+nVtTa65nnbih5JEbdqVx8DjGc21s8+WHwZbkFCQcl2mnsRuwIs\n/oe0Asud5DR2sar8WgtzAyzOd67ra3vs+GKfa/vHuwZW04Lp2ufNjuGnabvcfl87YF3YotYW\nixVqa9K0oDWwLkyqV1Xssi33Q14BNVjLVQVYOr6T8Y39NO2M5n939r52KOz0vLaN5DObT9RX\nwBraUuOhsGtuKxY7Di/K9AsSCnK2Ynxi7LyVUNfVfLyTV6BSHgqXq4pDoY7vZPxoTpmfzcnO\nuWuI3vrkkNB452YZdmAqsIbW89h4v/Zmz/msaQnWssxiZ8fbivGJsfNWQl1fhzZ2tVwBYdJn\nnLS+qt2nEADAWvrOxudh23VpnmbXf+vPpp9cgTYLcGt3HOf2JKo58VeB1WYe7tV4XHp2qaRn\nxacb5ixGP8yX4RYkFpxtpfiE2Hkroa7vrHoOOQV5BbhJF/b1GSb1eyy+5GtcUyEAgLX0nY37\nVM3XsItSJ0irV9fw6PVYB+tLzD7e54xlrzPXkB+GuTLcgviCvK0U32rsU352Up8FbYsuVmCe\n1C/9xrWxppJn7oRATJDKGzMZRQDr02UCf8ZdxMolnUd3paWb/tNOfNy5rJPYNGnqZbra0eh1\nG672DHqe+50LP8yVmRfEF+Rtpfik2Gcrqa5/GrsevsUKzJMezdC3cFY4luyjGTzFSzorC0tC\nSYUcaAOnWG/kldQ2BVjpKKltCrDSUVLbFGClI2xTKIgAFhREAAsKIoAFBRHAgoIIYEFBBLCg\nIAJYUBABLCiI/IP1btV/BlFAa0Jhn07hvE1kaA2wonuXETbAiu5dRtgAK7p3GWEDrOjeZYQN\nsKJ7lxE2wIruXUbYACu6N9INAIuCNcACWIdbEwobYEX3LiNsgBXdu4ywAVZ07zLCBljRvcsI\nG2BF9y4jbIAV3RvpBoBFwRpgAazDrQmFDbCie5cRtgVY/WPKqkbiNzcRYB1jTShsc7AGjoYP\n7ruSuAJY0a0JhW0MVlUDLLLWhMI232NtgSVwBbCiWxMKOxBY/1rpGkJbOp2OjsBRbmB1Ta9q\nasN3sgHd6I8inDWhsMvJY62DVS8Ph6p4mPyDLcsoy6tWla0M7RlsiE4NUfFODazVEXqTFasK\nsChYewSrqgHWwdaEwj4ArPYtHtzX+L//uSI2Tpo+ht+1YDf+GIqNbtMYw+1iPgsBa0Jh24K1\nyLxPo+eCqngEStj8NQEwvltmLr82S2/N5BICrvMYQ9GpISreCVwr5HYwIlgKTUUYV5TxYK2Z\nigOmolNDVLwTAav7ZgBrX+WkG/yANVK138YaikyfIlhyW603HQ+lAMubEgFLPhROE7rvRRtr\nA6yFD7//ywAsKt45gLVa3hysPPZYVLxTAktuY22AxSOCs8IjvFMAiw3NIN02ltxaGvNYY2ON\nCVMYV+SNPNZB1keA5Utl1BAVb4BF3buMsAFWdG+kGwAWBWuABbAOtyYUNsCK7l1G2AAruncZ\nYQOs6N5lhA2wonuXETbAiu5dRtgAK7o30g0Ai4I1wLIBiyl7HyzGLQvVO9OnZTDlL6Xo1BAV\n76TA2ugso1H9emAxYRrbKKn29iqAlQVYYle/rW5fG95eBbDCgTXeKNh1lppuM+Sn8T2Tpw5X\nXI/l9qsW/LTAWv5QbReNMpbKCyxV0+IQsIT+nWKv0LnjH1+Y78c+l6r5IgKHUod5kSW0sTxa\n85taLH4AWHNdi0yt9FZ/S0At+ykvfikXNm2J9YLSdtEpZKecwBL/iIXiJMF6hwNLXVIQHbBI\npxuIgTVFw93vINwI+Jb2MQJ7i/UxBcv8xMCrsgHr7++PGFji0LJFJRYafvEndEywlvZf+9Ql\nBRYVb4Gfv1bD2FTbWMIIJjssfykXtltU2C77RWyVJlgTQRNT0/j1SA4Ay/6sUBipOCtULuw9\nlkqrjUXDezgMCExtR3IEWBPmfEtczGNxfwlueSxpYTgrtJO6MaWK5BCw/CjJGkoz7I1WuioS\ngBXbO7Ww+zaVIVcAK753UumGuVFlxhXAiu9NHayZIKGpnsRFaD8qA6zY3sMx708+AQRY1L1p\nhz3mFVytAVZ0b9phK0//ABZ1b9Jhq/MKAIu6N9mwt/MKAIu6N82wd/MKAIu6N5l0gyKv4MMa\nYB3gTQUsVV7Bg/X7YLCYYnhL3I665i4wm15x2BMdsEJ5K/MK7tZd8cTA4ru91FL/G58qBqwQ\n1l3xtMASOurVUo9BnwJYDtZd8UPA4rpYTf2v5lsJN9ZXAZZ3rnIHq21XmfaDMYzkCLC4TqHz\neySWHUm7740e7BxYnltYmYM1tNYN+8EYRnIAWPyORtXZXacHu7TH8kpWvmDp9y42thaKHwBW\nK8ZUYE1TV2fjS9USg3meFXpNN9hSpWEtFw8IllINN93y2LhUfmCaujon9yn/irkGEXU6OVt0\nW7TR35+zlc3SvTuqQDc5FOq1sd7igAfR2WO5e/cb0GFnpbbeKE4cLNWsnbXePFbKCCzj+yD0\nrbeKHwXW2MaazgrHSdPUVW0kSNF4Lx0s/l2FXB5r/Bynrou7S5C7pGPa1X9X+YBlfueWtvVm\n8SPA4uSyvscnhKhZL71t7tzStN4pDrBie8cL+88qE6plvV8cYMX2jpTHcj0L3LDWKn4wWC4C\nWGpvz1S9ARZ973DW4yHP986qE8Ci7h3M2k8iVCWARd07lLWntIJKAIu6N8ACWElZAyxfAlic\nzLuEGgpgUfcOkW6YHmFsEY+mABZ1b/9gzeeBdMIGWNG9fVs7PB3NRACLurdfazFrRSdsgBXd\n26e1nAylEzbAiu7twZqNL4kI4K0SwKLu7W49XrsJ4a0UwKLu7Wy9kQmlEzbAiu7tnG4AWBbx\nGMUezppQ2AALYMWxHl4/GcR7QwCLurebdddkV167oRM2wIru7WK914mPTtgAK7q3vfV+31A6\nYQOs6N621jpdjumEDbCie9tZB3m0sYkAFnVvm3SD7g0SdMIGWNG9Tayb078GLP37boiE/QZY\nB3gbWPdPYje4nYtG2F1xgBXbW9/a/PYIEmH3xQFWbG+ABbAOtgZYACuE9dZFQVdvcwEs6t66\n1psXBR29LQSwqHvrWU/nguHeV2gmgEXdW8eaSzEALIDlzTrW6yMCWgOs6N671g4PuKKztQFW\ndO89a5fnptHZ2gAruve2tdvz+OhsbYAV3XvL2vUxj3S2NsCK7r1q3SesnJ8eSmdrA6zo3mvW\nG90YkG4AWNbWWxcFARbAsrb29vhQOlsbYEX3BlgAK5a1eTcGfW9fAljUvRfWw6vfgnj7E8Ci\n7i1b+3xDCZ2tbQtW1Wkc6L778QDL0Nrrm2/obG1bsCaKqmmQ+wWwdK33U+0Fphuq6QNgWVpr\n7K4KBauahyuJK4C1Z3342yoDWjuA1e+wxiaWCNa/VsakFqa/owOII0uwxo9qQIvbadmAbvRH\nEc46QthB3ldJZ2u7gsUPyYdDm3iMYg9nHT5sEq9BDWhtD1a1GARY+tZhuCK0tV3BqsRBgLWn\nLsUe6LXNlLa2D7DmHRbA2lPf8cpkjuLSDSNOlZBzR+N9UxbdGIoDa1828RjFHs6aEFhGorO1\nAVZUb4AFsILIW8crhehsbYAV09tfxyuF6GxtgBXR+y+c9SA6WxtgRfMek1eJhW1pDbBieU/J\nK0NrpBsA1pbmpCjAAlj+xCXbUwrb3hpgxfAWrg2mE7aLNcCK4C1eG0wmbCdrgBXeW7rmnErY\nbtYAK7i33JchkbAdrQFWYO9l16skwna2BlhhvVe6XiHdALBc1HcVdbcGWACLl7qrKOmwvVkD\nrDDeGx2vKIftzxpghfEGWAAriDfAAlhBvDe6ilIO2581wArivdVVlHDYHq0BVgjvzRsHkW4A\nWJbaviEVYAEsK+3dP080bM/WAMu39+798zTD9m0NsDx77z+XgWTY3q0BlldvncfIEAw7gDXA\n8umt9RgZemGHsAZYHr31Hk9ELuwg1gDLn7fmY6+QbgBYJtJ+Sh/AAlgG0n9KH6mwg1kDLD/e\nBk9/pBR2OGuA5eqt7ILsbm0uOlsbYDl6mz+tlkTYwa0Blpu3xbMfKYQd3hpguXkDLFXxbbCa\nbQawthQerNzSDe3m6sbZk2UTj1Hs4awN21hhrHtlBhabmbImyyYeo9jDWWt7/5k/rZZC2OGt\nlWDNowCWWjbvxCEQdgRrgOXibfWupePDjmGtBIutjwdYnOze4XV42FGsVWDVTBTAWsry3XBH\nhx3HWgmWB9nEYxR7OGsdb9t3DgIsgLUl63dZlp1uAFg7sn9HKsACWEq5vHsXh0KApZLTK50B\nFqTQ39EBJCXssXS9HV9Bjz0WwFqVI1cAq+bz7wBrlCtXAAtgrcmZK6QbANaK3LkCWABLVNvx\nygNXOBQCLEHmXUW1rf2JztbeBstNNvEYxR7Oeult0bld19qj6GxtgKXpDbAMiwMsPW+AZVgc\nYGl6o41lVhxg6Xlb3I6ja72tLNMNAGuUjzyDwnpHACtnsDxyhUMhwJrkk6uywXK+QycrsLxy\nVTZYN4A1yy9XZYPVkPWw5Sk3sDxzVThY9bUCWJ18c1U6WHV1A1jvAFwh3eAsm3iMYg9nTShs\ngJUlWP53WIX8PQCsTe8AXAEsgBWEK4AFsIJwBbCKBysMVwCrdLACcQWwCgcrFFfFpxvaa4QF\n36UTjCuAVSxYvm4gVKj0Q2GxYPU9OsJ4twJYZYLl73YchQAWwAoigAWwgghglQkW2lherAGW\nrP4GQjphI92QB1h/Ab2trLMDy1028RjFHsBzzF8lFnYEb4BF3buMsAGWoCnhnlbYMbwBloPm\nCzlJhR3FG2DZi7tAmFLYcbwBlrX4C88JhR3J2xtYl2thYAkdGuiEnV26oXLehdnEYxS7Vzex\nowydsLMD63m5vQoCS+qAlUrY8bw9Zt5LetqM3LEvkbAjegMsGy06jKYRdkxvnBVaaNkROYmw\no3oDLHOtdHBPIey43h7B+vlqDoOXZ/Zgrd04kUDYkb29gfU5d+0rZv9oP5t4jGIPZ00o7OzS\nDVd2a/tj/bLLKjVVq+G7/92PTg+s1Tu96ISdHVhjRz/FWWHFfVXjP4GrRMBav4OQfNjRvQGW\nvjbuTKUc9jHevg+FN7Z6zbDivyuerXTA2robh3DYB3n7a7xXfXq0Wr2wMzWx1sH612ptPkJy\nTP9Cmlpu4e/mvPB8+6yW5oAaDoIDWtxOywZ0oz8Kt9k3byCkG/ZR3t72WPd9Fnmw6uXh0CYe\no9jdZgdYIa3VYDF23WMrbbASaWNll274ajf71+/6kXB5KJzOCtMB62/jTno6YWcHVl0/22s6\n7PKrAotvvCcI1uYjsOiGfZS354vQN9V505xxF3Lu6TTetx+tRjbsw7x9gvW4Vc154fcqWBqy\niccodpeZdx7ZRzXs47z9nRW2VCnbWKmDtfcoSKJhH+jtswepIoeVAVi7jxilGfaR3t7AevR7\nLIcbKmziMYrdflaAFdpaDdbI1tn6vYU28RjFbj3n/jOR6YSdYbqh0ec7x5spNJ61TSfsDMF6\nfl+YRv49ObB0nuFOMOyDvf11m6mcqKILlta7AeiFfbR3zGuFSYKl984JcmEf7h2zdwPAslHp\nYNWf20Z/rGTB0nxJDrWwj/f2BtZrswdpqmDpvnyJWNgEvP013tmlQep1We/znihY2i/1ohN2\ndumGMX+VUx5L/2VxdMIGWADrcGtCYavByvBQaPB2S0ph0/BG410pk7emEgqbiDfSDSoZvY2X\nTthUvP1ehHaSTTxGsRuVNnvLM5mwyXgDrBU1R3TDt4eTCJuUt+dD4cW6xzshsCxemUoh7F7Z\npRvyabzbvIyXQNiDsgPrkk26IW2wqHj7TpB+0k+QAqwDrNVgfbE+0aB4VGRCYKXdxqLi7a/x\nfm0fmPy6XJJvY20+pMHV21ylg8UEpQyWYabByJuWNaGwCwDLhisCYVPzRoJUkhVXx4c9Kbt0\nQyZg2XF1eNizABbAOtyaUNi5g2XJ1dFhE/QGWLxsuSJUQ1S8ARYna64I1RAVb4A1y54rQjVE\nxdv3tULhqaIAy4OKBqtyTY5SAMuBK0I1lFe64Yfj6idRsFy4IlRDeYFVu9xPSAMsJ64I1RAV\nbzTeBwGsQ603wPppWu0PVtl3ereJxyj2rYluXBGqISre3sBqmll9v/ckXyDgyBWhGqLi7Q2s\nM3s0/3+eLMV0gytXhGqIirfPxvudndN8KIgzV4RqiIq3N7Aq9rqyZ9vKAlheVXq6oX3Ce9Xu\nsNJ7gYA7V4RqKDuw6hur7s2Oy5qrw8DywBWhGqLijTyWD64I1RAV77LBMn/6h763LwGs9s29\ndXtzYTJgWdyZqu3tTaWD9Tl3PRsYeyQDls299Lre/lQ6WNfmdLAB69f+FvvocuzmAwXRWu+G\n8b+dbEA3+qOQR2S5x8ou3ZAgWFm2sbIDazgU3lJ6PpbF0z+0vb2p9DbWJ70n+vnJNKx7J2FN\nKGw1WHX9ndjjuD1yRaiGqHiXmyD1yRWhGqLiDbBCeCdiTShsNVip3VfolStCNUTF2w9Y6d1X\n6JcrQjWUV7ohufsKPXNFqIbyAqtO7L5C31wRqiEq3mU23gFWcO8iwfLOFaEaouJdIlj+uSJU\nQ1S8CwQrAFeEaoiKN8Dy6p2WNaGwMwArBFeEaii7dEMqYAXhilANAaxjwArDFaEaouINsPx5\nJ2dNKOzUwQrEFaEaouJdFlihuCJUQ1S8iwIrGFeEaoiKd0lgheOKUA1R8QZYfrzJWCPdEB2s\ngFwRqiGAFRusvzKOKVS8ywCre1xRGTVExbsIsPpb6cuoISreJYA13uQRwHoUnRqi4g2wvIhO\nDVHxBlheRKeGqHiXAFZRbSykGyKC1T+uCGC5epuoBLCGzGgZYFHxLgCsMeNeRg1R8QZYXkSn\nhqh45w/WdImwjBqi4p09WPOl5zJqiIo3wPIiOjVExTt3sLi+MmXUENINUcDi+2ABLFdvE+UN\nltC3rwywqHgDLC+iU0NUvLMGS+yMXEYNUfHOGSypk3sZNUTFO2Ow5JsnyqghKt4Ay4vo1BAV\n73zBWtztVUYNId0QGKzlXYQAy9XbRADLi+jUEBXvXMFaue25jBqi4h0LrKrq395UDQPDu5z4\nNzrZxKPS2u30ZdQQFe9IYFXjRzX95n55B2v1MQ1l1BAVb4DlRXRqiIp3zDZWNZNUCb98g7X+\nXJkyaoiKd2ywxiaWCNa/VuaGCv15c0pQp9PRETjKHKzp2Dc23ivxfaw2oK9K8SCsMv70C8xj\nVdKAfDi0iWdNqgeslQEWFe94YFXyUAiwugdhqWJ3s94UnRqi4h0NLC5vNQ/6Bqt/SIOfVTUS\nnRqi4h0tQTp/iYM+wRoeK+NnVY1Ep4aoeMfKY80ZdyHn7rfxDrDoeGd1rRBg0fHOCiy0sYpM\nN0QA62+DK4Dl7G2irMDafkVAGWBR8c4JrJ1XT5RRQ1S8AZYX0akhKt4ZgbX3rpwyaoiKdz5g\n7b6DqYwaouINsLyITg1R8c4GrP2XxpVRQ0g3+AVL42WEAMvV20SZgKXzkssywKLiDbC8iE4N\nUfHOAyytt/KWUUNUvLMAS+9tz2XUEBXvHMDSfIt4GTVExRtgeRGdGqLinQFYmlwVUkNIN/gC\nS5crgOXsbSKA5UV0aoiKd/JgaXNVSA1R8U4dLH2uCqkhKt6Jg2XAVSE1RMUbYHkRnRqi4p02\nWCZcFVJDVLyTBsuIq0JqCOkGgBXEGmC5g2XGVSFgUfFOFaytB2GpYjcsT8S7jLCpgLX5kAY/\nq0rFu4ywiYC1/VgZP6tKxbuMsAFWdO8ywgZY0b3LCJsIWGhjKYV0gxNYmw/CUsVuOgMNb4AV\nESzTTIOBtZ3o1BAV7yTBsuKqkBqi4g2wvIhODVHxThEsO64KqSEq3gmCZclVITVExRtgeRGd\nGqLinR5YtlwVUkNIN1iCZc0VwHL2NlFqYNlzVQhYVLwBlhfRqSEq3omB5cBVITVExTstsFy4\nKqSGqHgnBZYTV4XUEBVvgOVFdGqIindKYLlxVUgN6acbmoK1urBJ1mKteK2asK4jwXLkCmDJ\nJTe9fYGlqZBg7ekv+hLz1slhqmvxDUXfY7nusArZY6l16ndmp2Gg+Tx1h8JhxDC5G//mdkGn\nxQSu+Gl2nZfQhz37nHrJJcS1PAwsZ65KB2ukZcJmbGOd5LE9Ddxs4gS+uDCbyOOmz0KHgeXO\nVeFgncZPfqDmiVlM5eaTJ6wVP62Un75PgsdSACu6d1ywuq/TCljihGzA8sAVwHrrgDXCs2hj\nCRNWwZqbUm+hjTV/CiXEtTwGLB9cFQKWqnVseyhcm21rjyWELRRXNa/64gArtndiYE17rEVx\ngodCL1wVApZSI0PLs8L3WyJl0cZaTJjGnnjnN0cNP58wch38+GBZPAhLocLBkvNY7zmPxX90\njSBFG2ueMGOTah7L5iENCpUO1iC+Zp291a0mQpd01uKxeqyMn1Wl4g2wAFYS1gCrFcAqJOzY\nYKGNhdu/woD19sYVwHL2NhF5sPypDLCoeAMs6t5lhA2wonuXETbAiu5dRtgAK7p3GWEDrOje\nZYQNsKJ7x0o3TFea964Xm1vrFAdYsb0jgcXNx/V0MFu2wlqrOMCK7R0n7NNbBGujT56ptV5x\ngBXb25P16b3S3byepy676LkJYFH39gWWwMtpeV+DDBbaWNqxh7NOIOzVY9ui8T4XVPci1hPA\nou59EFhrxU0EsKh7AyyARcF69/avoZSyjQWwTAWwdPZY+7cA6glgUfeOfCjkd1xovOvFHs46\ngbBNwUK6QT/2cNYI29UaYEX3LiNsgBXdu4ywAVZ07zLCBljRvWP1x/IsgEXdG2ABrNSsCYUN\nsKJ7lxE2wIruXUbYACu6dxlhA6zo3mWEDbCie5cRNsCK7h0k3XAy9N40XZt42rE+SYMAK7o3\niTyWYUcH5QsoloZDUYAV3ZtE2MY9aLDHIu/trT8WvxeZn9wuT576ba2/l3D5uPixT6Cwkxrv\nf+XnOi1mP81FAVZ0b19gifcV9v9rrhP8MHoGa7rJ/vTm7gYbB6Rb8kX7ESxhLvENFtwYgHWI\nd5AepBNYi8niwBI3cYA3ewsja7WpMDAWB1ixvXMAq/vi39DjH6yqUf/d/wRYkayXYC0bXTpg\njbeNaYE13WO2fAuiVN4VrGr46P8JXAEsL9aa9xWO3id5stYe660J1mLhAIuON0Ww6LWxRLAE\nrgBWUGtFG2sdLPV7CSeKdMASCy/ebRgBrH+trAwhTZ24z+G7bf8sJ3djT2OR4eNUT2WHgWnO\nYa5aGikUnkyl2Rfla1ewusZ7NbXhO82gh1HZe6zY3pEvQk9g1cvDoU08RrGHs0bYrtYAK7p3\nGWF7BKuqAdbB1oTCBljRvUl0mzFX7I5+Y+Z9pAuNd8/WpYK1IZt4jGIPZ42wXa0BVnTvMsIG\nWNG9ywgbYEX3LiNsgBXdu4ywAVZ07zLCBljRvZFuAFgUrAEWwDrcmlDYACu6dxlhA6zo3mWE\nDbCie5cRNsCK7l1G2AAruncZYQOs6N5INwAsCtYAa12J3gSGsL0JYPFC2N4EsHghbG8CWLwQ\ntjcFAgsqXQALCiKABQURwIKCCGBBQQSwoCAKAtZ0A34qGh8UMD6xN4nw5XBphR0CLP4hR0mo\nmiOukglfDpdY2ACrDRVgeRfAapUiWJ0AFm0BLO8CWK1SBWt6yD69sAFWK4DlXQCrVaJgUQ4b\nYLWiXENqcQ/rpBc2wGqVJFgV90UvbGTeW6WYea+qIU6aYeNaIRREAAsKIoAFBRHAgoIIYEFB\nBLCgIAJYUBABLCiIABYURADLp9jG5tyalqHKWtvQAliTylrb0CoMni1hS+zru2Lnn3bg8cVY\ndas7gL7YV/06s6/P8OvyqgewPlfGrp/F3M00NmhRJkMBrF3dOhgaNu49FrcWkgYx9ntuPq7t\nrwYTVn0GsKq20HkxtwCWWCZHAaxdMfaqH6yq6zP7retny0XL02+L2G//6/KpLz1wzR6qHbix\nH3nu8TB5YRe5TI4CWLuq2PU+DL7u35cepVf7MeyjGHs2k9odUPvr3G3S5kApzT2A1XIll8lR\nAGtX9+a4dW5bUAGEScEAAADwSURBVA0Uw5Gsg2T66JEZh+bjnTh33/7quJLL5Kic182bnmdW\nPer62rTC7y9DsKa52zGvqm2TASxo1M8E0GcNrFd7KLzwh8LVuRuubt2IlTK5Kf81dFbFHk2T\nvWt+P9pj2QpYl3b8d//r1sLz2x3xpLnrkSu5TI4CWLvqEwbf48AqWG26Yfj16VIJbXtenJtP\nN0hlchTA2tetYtV3O3BtEHqsHgov7Do10F9dscXcQh5LKpOhAJa7sm6E2wrbxF0Aa0XYJu4C\nWCvCNnEXwFoRtgkURAALCiKABQURwIKCCGBBQQSwoCACWFAQASwoiP4D2gd0+sWGDKQAAAAA\nSUVORK5CYII=",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "power_plot_data <- pwr.t.test(d=0.30, n=NULL, sig.level=0.05, power = 0.8, alternative = \"two.sided\")\n",
    "plot(power_plot_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}