rtweet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# rtweet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The examples in this notebook are adapted from the official [`rtweet` documentation](https://docs.ropensci.org/rtweet/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load package"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(rtweet)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create authentication token"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to collect data with `rtweet` you need to create an access token. The [`rtweet` documentation section](https://docs.ropensci.org/rtweet/articles/auth.html) on this topic provides a detailed explanation of how to obtain and where to find the necessary permissions and information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB**: You should treat all information relating to your API key like a password and never share it or post it publicly anywhere. Although nobody except you should be able to access your personal instance of this notebook (and your edits will also not be persistent if you do not have/use a *GESIS Notebooks* user account), if you want to be extra cautious, you can delete your API access information from the following cell after running it once (and save the notebook again after that)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "token <- create_token(app = \"app_name\",\n",
    "                      consumer_key = \"consumer_api_key\",\n",
    "                      consumer_secret = \"consumer_api_key_secret\",\n",
    "                      access_token = \"access_token\",\n",
    "                      access_secret = \"access_token_secret\",\n",
    "                      set_renv = FALSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Search tweets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Search for 1000 tweets using the rstats hashtag, excluding retweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rtweets <- search_tweets(q = \"#rstats\",\n",
    "                    n = 1000,\n",
    "                    include_rts = FALSE,\n",
    "                    token = token\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Note*: If you want to know how to use an `R` function, you can simply run `?function_name`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "?search_tweets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the first 10 tweets that were collected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "head(rtweets,\n",
    "    n = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which variables are included in the data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "names(rtweets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get trends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Discover what is currently trending in Cologne, Germany."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cologne_trends <- get_trends(\"cologne\",\n",
    "                            token = token)\n",
    "\n",
    "cologne_trends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stream tweets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stream all tweets mentioning \"COVID-19\" or \"Corona\" for 5 seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cov_stream <- stream_tweets(q = \"COVID-19, Corona\",\n",
    "                   timeout = 5,\n",
    "                   token = token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the first 10 tweets that were collected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "head(cov_stream,\n",
    "    n = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use the `stream_tweets` function to continuously collect tweets by specific users if you provide a comma separated list of user IDs or screen names to the query (q) argument."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accounts followed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The file [twitter_accounts.csv](./data/twitter_accounts.csv) in the `data` folder of this repository contains the Twitter screen names of [*GESIS - Leibniz Institute for the Social Sciences*](https://www.gesis.org/en/home) and the [*Social Data Science Lab*](http://socialdatalab.net/) at *Cardiff University* which we will use in the following examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "accounts_df <- read.csv(\"./data/twitter_accounts.csv\")\n",
    "accounts = as.character(accounts_df$Screen.Name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieve a list of all the accounts the specified users follow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "followed <- get_friends(accounts,\n",
    "                       token = token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get information about those followed accounts (including their most recent tweet). Note: It might take a moment to to run the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "followed_info <- lookup_users(followed$user_id,\n",
    "                             token = token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at the information for the first 10 accounts in the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "head(followed_info,\n",
    "    n = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which variables are included?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "names(followed_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get timelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the most recent 100 tweets by a set of users through the REST API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmls <- get_timelines(accounts,\n",
    "                      n = 100,\n",
    "                      token = token)\n",
    "\n",
    "head(tmls,\n",
    "    n = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can save the results as a `CSV` file. Before you do that, you should unnest the list columns in the dataframe. You can run `?rtweet::flatten` if you want to know more about that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmls_flat <- rtweet::flatten(tmls)\n",
    "\n",
    "write.csv(tmls_flat,\n",
    "         file = \"./data/tweets_rtweet.csv\",\n",
    "         row.names = FALSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get favorites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the 50 most recently \"favorited\" (liked) tweets for a specific set of users."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "favs <- get_favorites(accounts,\n",
    "                      n = 100,\n",
    "                      token = token)\n",
    "\n",
    "head(favs,\n",
    "    n = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outlook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, you can do much more with `rtweet` than what is shown in these examples. You can find some good examples in the [*21 Recipes for Mining Twitter Data with rtweet*](https://rud.is/books/21-recipes/) by Bob Rudis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are also [plans for including functionalities for the Twitter API v2 in `rtweet`](https://github.com/ropensci/rtweet/issues/445)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}