Lecture7_twosamplet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 7 - Two-sample t-tests\n",
    "This notebook is the conceptual examples we will work through in Lecture.  This will not focus on the actual \"doing\" of the analysis in a practical application.  Lab 5 will focus on that and show you examples of how to use the appropriate code to conduct your analyses in HW 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# LIBRARIES\n",
    "library(tidyverse)\n",
    "library(magrittr) ## for pipe operators\n",
    "library(scales) ## for scaling functions for ggplot2\n",
    "\n",
    "#### plot size options for Jupyter Notebooks ONLY\n",
    "options(repr.plot.width  = 8,\n",
    "        repr.plot.height = 6)\n",
    "#### do not use these options for RStudio\n",
    "\n",
    "bold.14.text <- element_text(face = \"bold\", size = 14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "## DATA\n",
    "cah <- read_csv(\"201710-CAH_PulseOfTheNation_Raw.csv\")\n",
    "## variable names currently full questions - need to rename\n",
    "new_names <- c(\"income\", \"gender\", \"age\", \"age_cat\", \"polaffil\", \"trump\", \"educ\", \"race\", \"whtnat\", \"whtnat_rep\",\n",
    "              \"love_us\", \"love_us_dem\", \"helppoor\", \"helppoor_rep\", \"racist\", \"racist_dem\", \"friendtrump\", \"civilwar\",\n",
    "              \"hunting\", \"kale\", \"therock\", \"trumpvader\")\n",
    "colnames(cah) <- new_names\n",
    "cah_oct <- cah %>% drop_na(income) %>% filter(!gender %in% c(\"DK/REF\", \"Other\"))\n",
    "glimpse(cah_oct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two-Sample t-test - Conceptual Example\n",
    "We're going to again look at income in the CAH sample. This time we are going to compare the average income within two samples - these two samples are defined by the gender variable - the male sample vs. the female sample.  Note that these are two subgroups that come from the same sample survey, but they are treated as two samples.  This is how the majority of analysts conduct t-tests.\n",
    "\n",
    "How would we run a hypothesis test for this?\n",
    "\n",
    "### Step 1 - Formulate Hypothesis\n",
    "\n",
    "$H_0 : \\mu_{female} = \\mu_{male}$\n",
    "\n",
    "$H_A : \\mu_{female} \\neq \\mu_{male}$\n",
    "\n",
    "**Note:** Given our $H_A$ we're running a two-tailed test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cah_oct %>%\n",
    "  ggplot( aes(x=income/1000, fill=gender)) +\n",
    "    geom_density(alpha=0.6) +\n",
    "    scale_fill_manual(values=c(\"red\", \"#404080\")) +\n",
    "    labs(fill=\"Gender\",\n",
    "         y = \"Density\",\n",
    "         x = \"Income in $1000\",\n",
    "         title = \"Distribution of Income by Gender\") +\n",
    "    theme(text = bold.14.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "meantab <- cah_oct %>% \n",
    "                group_by(gender)  %>% \n",
    "                summarize(freq = n(),\n",
    "                          mean = mean(income),\n",
    "                          stddev = sd(income),\n",
    "                          var = stddev^2)\n",
    "meantab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2 - Prepare and Check Conditions\n",
    "\n",
    "Set alpha ->>> $\\alpha = 0.05$\n",
    "\n",
    "Random and independent sample ->>> Yes\n",
    "\n",
    "Sample is <10% of the population? ->>> Yes\n",
    "\n",
    "Sampling distribution is normally distributed? ->>> Yes, given Central Limit Theorem\n",
    "\n",
    "**Are the variances of each sample equal? ->>>**\n",
    "\n",
    "The variance of each group/sample ($s_x^2$) need to be relatively equal to each other.  If the variance is equal, they have a ratio of one.\n",
    "\n",
    "We can check this assumption via hypothesis test:\n",
    "\n",
    "$H_0: var1 = var2$\n",
    "\n",
    "$H_A: var1 \\neq var2$\n",
    "\n",
    "### $\\frac{\\sigma_1^2}{\\sigma_2^2} = F$\n",
    "\n",
    "We compare the calculated F ratio to an F-distribution.  \n",
    "\n",
    "The F distribution is defined by two degrees of freedom, the numerator dof and the denominator dof.  \n",
    "\n",
    "Numerator df = n of first(numerator) group - 1<br>\n",
    "Denominator df = n of second(denominator) group - 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# observed F - the ratio of the variances of the samples\n",
    "# variance of income in female sample / variance of income in male sample\n",
    "meantab\n",
    "meantab$var[1] / meantab$var[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# critical F\n",
    "qf(0.05, df1 = meantab$freq[1]-1, df2 = meantab$freq[2]-1, lower.tail = FALSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# p-value\n",
    "2*pf(1.06429, df1 = meantab$freq[1]-1, df2 = meantab$freq[2]-1, lower.tail = FALSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# checking the results with var.test()\n",
    "var.test(cah_oct$income ~ cah_oct$gender)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Calculate t-statistic and p-value\n",
    "\n",
    "We know that our variances are not significantly different, so we can treat them as equal. Let's now see if the mean of income among female respondents is significantly different than the mean of male respondents. \n",
    "\n",
    "We're doing a two-tailed test.\n",
    "\n",
    "The formula is:\n",
    "![](tformula.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## calculate the denominator - the se_diff\n",
    "se_diff = sqrt(meantab$var[1]/meantab$freq[1] + \n",
    "               meantab$var[2]/meantab$freq[2])\n",
    "\n",
    "# calculate observed t-value\n",
    "t_obs = (meantab$mean[1] - meantab$mean[2]) / se_diff\n",
    "t_obs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# critical t-value, lower.tail because observed t is negative\n",
    "qt(0.025, df = sum(meantab$freq) - 2, lower.tail = TRUE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# p-value\n",
    "2*pt(t_obs, df = sum(meantab$freq) - 2, lower.tail = TRUE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check with t.test()\n",
    "t.test(cah_oct$income ~ cah_oct$gender)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusion:\n",
    "1. The observed t-value does not exceed the critical t-value.\n",
    "2. The p-value is greater than alpha = 0.05.\n",
    "3. There is no significant difference between mean of income for males and females.\n",
    "4. There is no evidence to indicate that males and females earn different amounts of money.\n",
    "\n",
    "## Two-Sample t-test - Proportions\n",
    "Now we'll look at an example of testing proportions.  We'll look at the proportion of individuals who reported eating kale, again by gender.\n",
    "\n",
    "How would we run a hypothesis test for this?\n",
    "\n",
    "### Step 1 - Formulate Hypothesis\n",
    "\n",
    "$H_0 : p_{female} = p_{male}$\n",
    "\n",
    "$H_A : p_{female} \\neq p_{male}$\n",
    "\n",
    "**Note:** Given our $H_A$ we're running a two-tailed test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cah2 <- cah %>% filter(kale != \"DK/REF\" & !gender %in% c(\"DK/REF\", \"Other\")) %>% \n",
    "                mutate(prefkale = ifelse(kale == \"Yes\", 1, 0))\n",
    "summary(cah2$prefkale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proptab <- cah2 %>% \n",
    "            group_by(gender)  %>% \n",
    "            summarize(freq = n(),\n",
    "                      prop = mean(prefkale),\n",
    "                      sd = sqrt(prop*(1-prop)),\n",
    "                      var = (sd^2), # variance of observations - sd squared\n",
    "                      se = sd / sqrt(freq)) %>% # standard error - sqrt of variance of proportion \n",
    "            select(-sd)\n",
    "proptab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proptab %>% ggplot(aes(x = gender, y = prop, fill = gender)) +\n",
    "                geom_bar(stat = \"identity\", position = position_dodge()) +\n",
    "                geom_errorbar(aes(ymin = prop - 1.96*se, ymax = prop + 1.96*se), \n",
    "                                  width = 0.3, position = position_dodge(0.9), size = 1) +\n",
    "                labs(title = \"Proportion of Respondents Who Eat Kale by Gender\",\n",
    "                     subtitle = \"With 95% Confidence Interval\",\n",
    "                     x = \"Gender\",\n",
    "                     y = \"Proportion\") +\n",
    "                theme(legend.position = \"none\", text = bold.14.text) +\n",
    "                scale_fill_manual(values=c(\"#52C87d\", \"#26d5b8\")) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2 - Prepare and Check Conditions\n",
    "\n",
    "Set alpha ->>> $\\alpha = 0.05$\n",
    "\n",
    "Random and independent sample ->>> Yes\n",
    "\n",
    "Sample is <10% of the population? ->>> Yes\n",
    "\n",
    "Sampling distribution is normally distributed? ->>> Yes, given Central Limit Theorem\n",
    "\n",
    "**Are the variances of each sample equal? ->>>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# observed F\n",
    "proptab$var[1] / proptab$var[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# critical F\n",
    "qf(0.05, df1 = proptab$freq[1]-1, df2 = proptab$freq[2]-1, lower.tail = FALSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# p-value\n",
    "2*pf(0.9907, df1 = proptab$freq[1]-1, df2 = proptab$freq[2]-1, lower.tail = TRUE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compare to var.test\n",
    "var.test(cah2$prefkale ~ cah2$gender)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Calculate t-statistic and p-value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check with t.test()\n",
    "t.test(cah2$prefkale ~ cah2$gender)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusions:\n",
    "1. The observed t-value does not exceed the critical t-value.\n",
    "2. The p-value is greater than alpha = 0.05.\n",
    "3. There is no significant difference between proportion of men and women who eat kale.\n",
    "\n",
    "### Two 0/1 variables? Chi-square test.\n",
    "Since we have two variables with two levels, we could have instead run chi-square test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kale <- table(cah2$prefkale, cah2$gender)\n",
    "chisq.test(kale)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two-Sample test - Non-parametric\n",
    "t-tests are relatively robust to parametric violations (deviation from a normal distribution), especially at large n. But in some conditions it's appropriate to use a non-parametric test.\n",
    "When it might be appropriate:\n",
    "- DV is ordinal and cannot appoximate numeric\n",
    "- DV is numeric but highly skewed\n",
    "\n",
    "For non-paired data we use the Mann-Whitney U Test.\n",
    "\n",
    "### Step 1 - Formulate Hypothesis\n",
    "Instead of comparing means we're comparing medians.\n",
    "\n",
    "$H_0: $ There is no difference in medians between the two groups.\n",
    "\n",
    "$H_A: $ There is a significant difference in medians between the two groups\n",
    "\n",
    "**Note:** Given our $H_A$ we're running a two-tailed test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cah3 <- cah %>% drop_na(whtnat_rep) %>% filter(kale != \"DK/REF\") \n",
    "summary(cah3$whtnat_rep)\n",
    "table(cah3$kale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "medtab <- cah3 %>% \n",
    "            group_by(kale)  %>% \n",
    "            summarize(freq = n(),\n",
    "                      median = median(whtnat_rep),\n",
    "                      mean = mean(whtnat_rep)) \n",
    "medtab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cah3 %>%\n",
    "  ggplot(aes(x=whtnat_rep, fill=kale)) +\n",
    "    geom_histogram(bins = 15, alpha = 0.9) +\n",
    "    scale_fill_manual(values=c(\"#52C87d\", \"#26d5b8\"))  +\n",
    "    labs(fill=\"Eats Kale\",\n",
    "         y = \"Frequency\",\n",
    "         x = \"% Republicans Agree with White Nationalists\",\n",
    "         title = \"Guesses of % of Republicans that agree with White Nationalists\",\n",
    "         subtitle = \"By if they eat kale\") +\n",
    "    theme(text = bold.14.text) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cah3  %>%  ggplot(aes(sample = whtnat_rep)) +\n",
    "  geom_qq_line(color = \"purple\", size = 1) +\n",
    "  geom_qq(color = \"#1d9a86\") +\n",
    "  labs(title = \"QQ Plot of Guesses\")+\n",
    "    theme(axis.text.x = bold.14.text, \n",
    "                      text = bold.14.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2 - Prepare and Check Conditions\n",
    "\n",
    "Set alpha ->>> $\\alpha = 0.05$\n",
    "\n",
    "Random and independent sample ->>> Yes\n",
    "\n",
    "Sample is <10% of the population? ->>> Yes\n",
    "\n",
    "Sampling distribution is normally distributed? ->>> Doesn't matter - we're running a non-parametric test\n",
    "\n",
    "Are the variances of each sample equal? ->>> Doesn't matter for this test\n",
    "\n",
    "\n",
    "### Step 3 - Calculate Mann-Whitney U"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the wilcox.test with paired = FALSE conducts Mann-Whitney\n",
    "wilcox.test(cah3$whtnat_rep ~ cah3$kale, paired = FALSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusion:\n",
    "1. The p-value is lower than alpha = 0.05.\n",
    "3. There is a significant difference between medians .\n",
    "4. There is evidence that suggests that depending on whether or not a person eats kale impacts their guess of proportion of Republicans that support White Nationalists.\n",
    "\n",
    "## New Version of Effect Size\n",
    "\n",
    "### R-squared\n",
    "Finally we'll turn to R-squared to see what proportion of the variance in our DV is explained by the IV (the groups).  \n",
    "\n",
    "For t-values, $r^2$ is calculated using this formula:\n",
    "\n",
    "# $$  r^2 = \\frac{t^2}{t^2 + df}  $$\n",
    "\n",
    "where $t$ is your t-value(statistic) and $df$ is your degrees of freedom.\n",
    "\n",
    "$r^2$ ranges from 0 to 1 where 0 means there is no variation explained by the IV and 1 means all of the variation is explained by the IV.\n",
    "- $r^2 \\approx$ 0.1, little to no effect\n",
    "- $r^2 \\approx$ 0.3, weak effect\n",
    "- $r^2 \\approx$ 0.5, moderate effect\n",
    "- $r^2 \\approx$ 0.6 to 1, strong effect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "k <- t.test(cah2$prefkale ~ cah2$gender)\n",
    "str(k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#rsquared of proportion of kale by gender\n",
    "rsq <- k$statistic^2 / (k$statistic^2 + k$parameter)\n",
    "names(rsq) <- \"r-squared\"\n",
    "rsq\n",
    "percent(rsq, accuracy = .01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# rsquared of mean of guesses of white nationalist support by kale\n",
    "w <- t.test(cah3$whtnat_rep ~ cah3$kale)\n",
    "rsq2 <- w$statistic^2 / (w$statistic^2 + w$parameter)\n",
    "names(rsq2) <- \"r-squared\"\n",
    "rsq2\n",
    "percent(rsq2, accuracy = .01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two-Sample t-test - Paired\n",
    "Finally, we'll look at a paired t-test.  We need to use this if our observations are not independent, either because the individuals are paired (spouses), or because the responses are paired within individuals (pre- and post-test, for example).\n",
    "\n",
    "How would we run a hypothesis test for this?\n",
    "\n",
    "### Step 1 - Formulate Hypothesis\n",
    "\n",
    "$H_0 : \\mu_{pretest} = \\mu_{posttest}$\n",
    "\n",
    "$H_A : \\mu_{pretest} \\neq \\mu_{posttest}$\n",
    "\n",
    "**Note:** Given our $H_A$ we're running a two-tailed test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load the .rds file\n",
    "anes <- readRDS(\"anes2.rds\")\n",
    "glimpse(anes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "anes_long <- anes %>% \n",
    "                select(ft_pre_rep, ft_post_rep) %>% \n",
    "                mutate(\"pre-election\" = as.numeric(ft_pre_rep), \"post-election\" = as.numeric(ft_post_rep))  %>% \n",
    "                select(-ft_pre_rep, -ft_post_rep) %>% \n",
    "                gather(key = \"time\", value = \"ft\") %>% \n",
    "                mutate(time = factor(time)) %>% \n",
    "                mutate(time = fct_relevel(time, \"pre-election\"))\n",
    "pre_mean <- mean(anes$ft_pre_rep)\n",
    "post_mean <- mean(anes$ft_post_rep)\n",
    "anes_long %>%\n",
    "  ggplot(aes(x=ft, fill=time)) +\n",
    "    geom_density(alpha=0.6) +\n",
    "    scale_fill_manual(values=c(\"#26d5b8\", \"#ff5733\")) +\n",
    "    labs(fill=\"\",\n",
    "         y = \"Density\",\n",
    "         x = \"Feeling Thermometer Rating\",\n",
    "         title = \"Distribution of FT Rating toward 2016 Republican Candidate\",\n",
    "         subtitle = \"Pre-election vs. Post-election\") +\n",
    "    theme(text = bold.14.text, legend.position = \"top\") + \n",
    "    geom_vline(xintercept = post_mean, color = \"#ff5733\", size = 2) +\n",
    "    geom_vline(xintercept = pre_mean, color = \"#26d5b8\", size = 2) +\n",
    "    annotate(geom=\"text\", x=67, y=.0175, \n",
    "             label=paste0(\"Pre-election Mean = \", round(pre_mean, digits = 1)),\n",
    "             color = \"#26d5b8\", size = 6, fontface = 2)+\n",
    "    annotate(geom=\"text\", x=67, y=.015, \n",
    "             label=paste0(\"Post-election Mean = \", round(post_mean, digits = 1)),\n",
    "             color = \"#ff5733\", size = 6, fontface = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2 - Prepare and Check Conditions\n",
    "\n",
    "Set alpha ->>> $\\alpha = 0.05$\n",
    "\n",
    "Random and independent sample ->>> No - paired, therefore paired test.\n",
    "\n",
    "Sample is <10% of the population? ->>> Yes\n",
    "\n",
    "Sampling distribution is normally distributed? ->>> We can rely on the Central Limit Theorem\n",
    "\n",
    "**Are the variances of each time period equal? ->>> Let's check**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# checking the results with var.test()\n",
    "var.test(anes$ft_pre_rep, anes$ft_post_rep)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Calculate t-statistic and p-value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use the options pooled and paired for paired data\n",
    "t.test(anes$ft_pre_rep, anes$ft_post_rep, pooled=F, paired=T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# effect size - rsquared\n",
    "ft <- t.test(anes$ft_pre_rep, anes$ft_post_rep, pooled=F, paired=T)\n",
    "rsq3 <- ft$statistic^2 / (ft$statistic^2 + ft$parameter)\n",
    "names(rsq3) <- \"r-squared\"\n",
    "rsq3\n",
    "percent(rsq3, accuracy = .01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusion:\n",
    "1. The p-value is lower than alpha = 0.05.\n",
    "3. There is a significant difference between means.\n",
    "4. There is a statistically significant difference in support for Trump pre-election vs. support for Trump post-election.\n",
    "5. The time period of the survey explains about 8% of the variance in responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}