py4e_ch13_web_services.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 13 Web Services\n",
    "\n",
    "Web Services and APIs (Application Programming Interfaces) are your access to the world of data on the internet. While HTTP pages are still great sources of data, more and more, data are accessed via APIs and your scripts can interact with APIs to get data, publish data, and do all kinds of things.\n",
    "\n",
    "## eXtensible Markup Language - XML\n",
    "XML is a more generic markup language than HTML. While HTML is specific to web pages, XML is used for any number of things--it is extensible for any data!\n",
    "\n",
    "The example in the text:\n",
    "```xml\n",
    "<person> \n",
    "    <name>Chuck</name> \n",
    "    <phone type=\"intl\">\n",
    "     +1 734 303 4456\n",
    "    </phone>\n",
    "    <email hide=\"yes\"/> \n",
    "</person>\n",
    "```\n",
    "XML can be viewed as a tree stucture with parent and children nodes.\n",
    "\n",
    "Kind of like HTML, we could write parser scripts to parse XML documents. In general, this may be easier than HTML as XML tends to be a bit more strictly defined, but it still a pain--don't do it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: Chuck\n",
      "Attr: yes\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.ElementTree as ET\n",
    "\n",
    "data = '''\n",
    "<person>\n",
    "  <name>Chuck</name>\n",
    "  <phone type=\"intl\">\n",
    "     +1 734 303 4456\n",
    "   </phone>\n",
    "   <email hide=\"yes\"/>\n",
    "</person>'''\n",
    "\n",
    "tree = ET.fromstring(data)\n",
    "print('Name:', tree.find('name').text)\n",
    "print('Attr:', tree.find('email').get('hide'))\n",
    "\n",
    "# Code: http://www.py4e.com/code3/xml1.py\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## JavaScript Object Notation - JSON\n",
    "\n",
    "The JSON format is rabily growing in popularity, and as noted in the text, has large similarities to Python dictionaries. Here's a JSON:\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"name\" : \"Chuck\",\n",
    "  \"phone\" : {\n",
    "    \"type\" : \"intl\",\n",
    "    \"number\" : \"+1 734 303 4456\"\n",
    "   },\n",
    "   \"email\" : {\n",
    "     \"hide\" : \"yes\"\n",
    "   } \n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And of course there is a json module to use!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Application Programming Interfaces - APIs\n",
    "\n",
    "An API is written by a data/service provider as a way for programs to interact with their data/services. Typically, they aren't meant for people to work with directly, but for your scripts to request information and get results back in a parseable format.\n",
    "\n",
    "The text goes over a couple of APIs here: [Google's GeoCode API](https://developers.google.com/maps/documentation/geocoding/start) for converting a place name to a latitude/longitude coordinate, and [Twitter's API](https://developer.twitter.com/en/docs.html) for retriving tweets.\n",
    "\n",
    "The Twitter API is introduced in the text specifically to show how to manage APIs that require authentication. If data is the new currency, nobody wants to give data away for free! Unfortunately, this can make your life a pain! Google's GeoCode API used to provide free access for limited numbers of searches...not anymore! You *may* be able to get some free searches after registering, but certainly no more un-authenticated searches!\n",
    "\n",
    "The same has happened with weather data--the last time I taught this course, there were sites with free APIs for getting weather data, most of those have now gone to authenticated only.\n",
    "\n",
    "As more and more sites try to monitize thier data, authentication will be more important, and often require payment for access.\n",
    "\n",
    "That said, authentication can be a challenge in a course. I do not want to force all of you to sign up for a Google API or Twitter account. This limits what can be done.\n",
    "\n",
    "Luckily, there are still some sites with interesting data and free APIs to access the data...\n",
    "\n",
    "I found a nice list of [Public APIs](https://github.com/toddmotto/public-apis) for a wide variety of data.\n",
    "\n",
    "And what better data than beer recipes! Here's the [Punk API](https://punkapi.com/documentation/v2) documentation page.\n",
    "\n",
    "I've copied the `geojson.py` file and copied it to `../Examples/beerjson.py`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a beer name: pumpkin\n",
      "Retrieving https://api.punkapi.com/v2/beers?beer_name=pumpkin\n",
      "Retrieved 2053 characters\n",
      "Beer name: Pumpkin King\n",
      "Tagline: Spicy Citrus Pumpkin Ale.\n",
      "Enter a beer name: \n"
     ]
    }
   ],
   "source": [
    "import urllib.request, urllib.parse, urllib.error\n",
    "import json\n",
    "\n",
    "serviceurl = 'https://api.punkapi.com/v2/beers'\n",
    "\n",
    "while True:\n",
    "    beer_name = input('Enter a beer name: ')\n",
    "    if len(beer_name) < 1: break\n",
    "\n",
    "    url = serviceurl + \"?\" + urllib.parse.urlencode(\n",
    "        {'beer_name': beer_name})\n",
    "\n",
    "    print('Retrieving', url)\n",
    "    uh = urllib.request.urlopen(url)\n",
    "    data = uh.read().decode()\n",
    "    print('Retrieved', len(data), 'characters')\n",
    "\n",
    "    try:\n",
    "        js = json.loads(data)\n",
    "    except:\n",
    "        js = None\n",
    "\n",
    "    # Comment out because it's kind of long   \n",
    "    # print(json.dumps(js, indent=4))\n",
    "    \n",
    "    for beer in js:\n",
    "       print(\"Beer name:\", beer['name'])\n",
    "       print(\"Tagline:\", beer['tagline'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Retrieving http://api.dataatwork.org/v1/jobs/autocomplete?begins_with=software\n",
      "Retrieved 6785 characters\n",
      "Software Publisher\n",
      "Software Installer\n",
      "Software Trainer\n",
      "Software Quality Engineer\n",
      "Software Tester\n",
      "Software Project Manager\n",
      "Software Consultant\n",
      "Software Programmer\n",
      "Software Applications Architect\n",
      "Software Applications Designer\n",
      "Software Applications Engineer\n",
      "Software Applications Specialist\n",
      "Software Computer Specialist\n",
      "Software Designer\n",
      "Software Development Specialist\n",
      "Software Engineering Supervisor\n",
      "Software QA Tester\n",
      "Software Quality Assurance Specialist\n",
      "Software Requirements Engineer\n",
      "Software Writer\n",
      "Software Engineering Systems Information Technology Specialist\n",
      "Software Quality Assurance Engineer (SQA Engineer)\n",
      "Software Quality Tester\n",
      "Software Test Analyst\n",
      "Software Test Engineer\n",
      "Software Testing Specialist\n",
      "Software Development Project Manager\n",
      "Software Technician\n",
      "Software Analyst\n",
      "Software Design Engineer\n",
      "Software Systems Engineer\n",
      "Software Development Engineer\n",
      "Software Architect\n",
      "Software Developer\n",
      "Software Engineer\n"
     ]
    }
   ],
   "source": [
    "import urllib.request, urllib.parse, urllib.error\n",
    "import json\n",
    "\n",
    "serviceurl = 'http://api.dataatwork.org/v1/jobs/autocomplete'\n",
    "\n",
    "while True:\n",
    "    job_name = input('Enter a job name: ')\n",
    "    if len(job_name) < 1: break\n",
    "        \n",
    "        #autocomplete?begins_with=\"software\"\n",
    "\n",
    "    url = serviceurl + \"?\" + urllib.parse.urlencode(\n",
    "        {'begins_with': job_name})\n",
    "    #testing api\n",
    "    #url='https://api.punkapi.com/v2/beers/random'\n",
    "    print('Retrieving', url)\n",
    "    uh = urllib.request.urlopen(url)\n",
    "    data = uh.read().decode()\n",
    "    print('Retrieved', len(data), 'characters')\n",
    "\n",
    "    try:\n",
    "        js = json.loads(data)\n",
    "    except:\n",
    "        js = None\n",
    "\n",
    "   # print(js)\n",
    "    for job in js:\n",
    "       print(job['suggestion'])\n",
    "   "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "Python 3.8.6 64-bit",
   "display_name": "Python 3.8.6 64-bit",
   "metadata": {
    "interpreter": {
     "hash": "1baa965d5efe3ac65b79dfc60c0d706280b1da80fedb7760faf2759126c4f253"
    }
   }
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}