ad_impression_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ad Impression Analyzer for Twitter\n",
    "Released 2022-11-23 under an MIT license. \n",
    "\n",
    "This notebook looks at targetting data for twitter ads.  To use it, you need to be able to run Jupyter notebooks, and also requires you to request, download, and unzip your twitter archive.  You can request your twitter archive in your Twitter settings.  Delivery, for me, took about a week. \n",
    "\n",
    "Once you have that downloaded and unzipped, put this file in your main archive directory and run all the cells below.  \n",
    "\n",
    "Once you get to the section called \"Let's actually do it\", then you can run those cells as many times as you'd like to see the different results.  \n",
    "\n",
    "[Example output shows that CVS is targeting sick people and those interested in the weather. Interesting.](cvs.png \"Example output shows that CVS is targeting sick people and those interested in the weather. Interesting. \")\n",
    "\n",
    "Please note, I am not a professional programmer, so this code likely has issues.  Mainly I'm releasing it after a few folk on Twitter (@jeffemtman) requested it.  If you see issues in my code, please LMK or help me improve it. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open file and process data\n",
    "file we're looking for is called 'ad-impressions.js'.  It's in the \"data\" folder.  It's almost JSON and we can use a regular expression to dump the one thing that keeps it from being parsed right by Python's json library.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import random\n",
    "from textwrap import indent\n",
    "from collections import OrderedDict\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# path to your file called \"ad-impressions.js\"\n",
    "# If you've placed this in the main directory of your twitter archive, then the default path below shouldn't need changing\n",
    "file_path = \"data/ad-impressions.js\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# UTF 8 encoding is important.  otherwise you'll get unicode errors. \n",
    "with open(file_path, encoding=\"utf-8\") as f: \n",
    "    \n",
    "    # There's a variable name up top that needs to be avoided. Load the file as a string.\n",
    "    string_data = f.read()\n",
    "\n",
    "    # Nasty regex to discard the variable.  Then convert to JSON\n",
    "    data = json.loads(re.sub('^(.*)(?= )', \"\", string_data))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make a list of all the ads\n",
    "This is called \"library\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library = []\n",
    "for entry in data: \n",
    "    ad_clump = entry['ad']['adsUserData']['adImpressions']['impressions']\n",
    "\n",
    "    #print(len(ad_clump))\n",
    "    for i in ad_clump: \n",
    "        library.append(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make a list and count of all advertisers\n",
    "Here you can see all the advertisers you've had and how many times their ads have been served to you in the data's period. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions\n",
    "4 main functions here. \n",
    "- `collateAdvertisers()` Create a dictionary with all advertisers and how many ads they've served you.\n",
    "- `printAd()` which formats the ad's data in a more visually friendly way\n",
    "- `randomAd()` which randomly selects an ad from your library\n",
    "- `lookupAdvertiser()` which lets you see all ads from a specific advertiser (use in conjunction with the list of advertisers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def collateAdvertisers(print_output = False):\n",
    "    '''Returns a sorted dictionay of all advertisers and how many ads they've served you'''\n",
    "    advertisers = {}\n",
    "    for ad in library: \n",
    "        advertiser = (ad['advertiserInfo']['advertiserName'])\n",
    "        \n",
    "        if advertiser in advertisers.keys(): \n",
    "            advertisers[advertiser] += 1\n",
    "        \n",
    "        else: \n",
    "            advertisers[advertiser] = 1\n",
    "\n",
    "\n",
    "    # Sort the output from highest count to lowest\n",
    "    advertisers_sorted = OrderedDict(sorted(advertisers.items(), key=lambda t: t[1])[::-1])\n",
    "\n",
    "    if print_output == True: \n",
    "        for k,v in advertisers_sorted.items(): \n",
    "            print(v, \"\\t\", k)\n",
    "\n",
    "    return advertisers_sorted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def printAd(ad): \n",
    "    '''Prints out relevant info on an ad.'''\n",
    "    date = ad[\"impressionTime\"].split(' ')[0] #discard the time\n",
    "    device = ad['deviceInfo']['osType']\n",
    "    advertiser = ad['advertiserInfo']['advertiserName']\n",
    "    handle = ad['advertiserInfo']['screenName']\n",
    "    display_loc = ad['displayLocation']\n",
    "    criteria = \"\"\n",
    "\n",
    "    for c in ad['matchedTargetingCriteria']: \n",
    "        try:\n",
    "            criteria += \"\\t\" + c['targetingType'] + \" = \" + c['targetingValue'] + \"\\n\"\n",
    "        except KeyError as e: \n",
    "            criteria += \"\\t\" + c['targetingType'] + \" = \" + \"NO AVAILABLE INFO\" + \"\\n\"\n",
    "\n",
    "    print(f\"\"\"\n",
    "👁️ Advertiser: {advertiser} ({handle})\n",
    "         Served on {date} on {device} \n",
    "         Display location: {display_loc}\n",
    "    \"\"\")\n",
    "\n",
    "    print(\"🔵 Tweet Text:\")\n",
    "\n",
    "    if \"promotedTweetInfo\" in ad:\n",
    "        print(indent(ad[\"promotedTweetInfo\"][\"tweetText\"], \"\\t\")) #indents multiline tweets\n",
    "\n",
    "        if len(ad[\"promotedTweetInfo\"][\"urls\"]) > 0: \n",
    "            print(\"\\tAdditional URLS:\", ad[\"promotedTweetInfo\"]['urls'])\n",
    "\n",
    "    else: \n",
    "        print(\"\\tNo Tweet Text Provided.\")\n",
    "\n",
    "    print(\"\\n🎯 Targetting:\\n\", criteria)\n",
    "\n",
    "    print(\" - - ✂ - - - - ✂ - - - - ✂ - - - - ✂ - - - - ✂ - - - - ✂ - - - - ✂ - - \")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lookupAdvertiser(lookup):\n",
    "    '''Allows you to look up all ads from a specific company. Requires exact text match, otherwise nothing will be returned'''\n",
    "\n",
    "    for ad in library: \n",
    "        if (ad['advertiserInfo']['advertiserName'].lower()) == lookup.lower(): \n",
    "            printAd(ad)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def randomAd(print_raw=False):\n",
    "    '''returns random ad from the library'''\n",
    "    rando_ad = random.choice(library)\n",
    "    if print_raw == True: \n",
    "        for k,v in rando_ad.items(): \n",
    "            print(k, v)\n",
    "\n",
    "    return rando_ad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's actually do it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Print the list of advertisers\n",
    "Let's see who's advertising to us and how many times they've done it in the period covered by our data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a dictionary called \"advertisers\" that holds all this data\n",
    "# can choose to print this data out, or hide it by swapping \"print_output\" to False\n",
    "advertisers = collateAdvertisers(print_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Print out some random ads\n",
    "each time the next cell is run, you'll see however many ads you've requested via the \"how_many\" variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "how_many = 3\n",
    "\n",
    "for i in range (how_many): \n",
    "    our_ad = randomAd()\n",
    "    printAd(our_ad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### See all ads from a specific advertiser\n",
    "Note that `the_company` has to be an exact match.  Helpful to copy-paste from the output of `collateAdvertisers()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "the_company = \"CVS Pharmacy\"\n",
    "\n",
    "lookupAdvertiser(the_company)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.13 64-bit (microsoft store)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "92c77c1d503d24070c81093e913e273e62f1270ebe97d9273b6b4b3b51eeb4b6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}