2. Python Web Scraping Essentials.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recap: Session 1: Python for Beginners"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    Everything in Python is an object. Objects have attributes and methods. \n",
    "Real life example: a car as an object, has attributes such as color, make, year of manufacture, transmission mode etc., and methods such as accelerating, reverse, braking."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key areas\n",
    "- Strings\n",
    "- Integers\n",
    "- Floats\n",
    "- Booleans & Logicals\n",
    "- Variables\n",
    "- Lists & Arrays*   ``` [1,2,3]```  ```array([1,2,3])```\n",
    "- Dictionaries*    ```{'name':['Michael','Joseph'], 'phone':[12323, 342121]}```\n",
    "- Tuple*   ```('Your_name', 1234242312, '12 street, james town')```\n",
    "- Set*     ```set([1,2,3])``` or ```{1,2,3}```\n",
    "- \"Built-in Methods\"\n",
    "- None\n",
    "- Logicals and Booleans\n",
    "- Conditionals\n",
    "- Loops\n",
    "- Input keyword\n",
    "\n",
    "    *FYI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Objects and methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides using basic types that come with the language, programmers can design their own types (classes), and create **objects** out of them.\n",
    "Usually, objects have built-in methods that can be _called_ with a dot notation:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "greeting = \"hello\"\n",
    "greeting.capitalize() # => \"Hello\"\n",
    "\n",
    "# BeautifulSoup has its own types:\n",
    "\n",
    "scraped = BeautifulSoup(html, \"html.parser\")\n",
    "type(scraped) # => bs4.BeautifulSoup \n",
    "\n",
    "link = scraped.article.h3.a\n",
    "type(link) # => bs4.element.Tag\n",
    "```\n",
    "\n",
    ".h1, .find(), find_all(), select() — are all methods that can be called on various BeautifulSoup objects "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Lists, loops, Dictionaries\n",
    "Python has a built-in List type for storing a collection of values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "numbers = [1, 5, 10, 15] # a list of ints\n",
    "names = [\"Ann\", \"Bob\", \"Charlie\"] # a list of strings\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In real-world programs the elements of the list are rarely known beforehand, so it is very common to start with an empty list and then populate it with elements. Lists have an ```append()``` method that takes any value as an argument."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "numbers = []\n",
    "numbers.append(1)\n",
    "numbers.append(2)\n",
    "# Now the list is [1, 2]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To do something with each element of the list, we need to iterate over it. Python has a special for... in... construct for that:\n",
    "\n",
    "```python\n",
    "names = [\"Ann\", \"Bob\", \"Charlie\"] \n",
    "#    1.       2.\n",
    "for name in names:\n",
    "    print(name) # we can only use a `name` variable here, in a nested block of code.\n",
    "    \n",
    " # 1. We can refer to each individual element on every iteration by that name\n",
    " # 2. The list we are iterating upon\n",
    "```\n",
    "\n",
    "This will print all the names in the list, individually.\n",
    "\n",
    "It does not matter how we name first variable in the for... in..., but the second one has to be the one that holds a list.\n",
    "\n",
    "```python\n",
    "for pizza in names:\n",
    "    print(pizza)    \n",
    "# Same result!\n",
    "```\n",
    "\n",
    "Another common built-in data structure is called a **Dictionary**.\n",
    "\n",
    "```book = {\"War and Peace\": \"A famous novel by Leo Tolstoi\"}```\n",
    "Dictionaries are used to store “key-value” information. Here, “War and Peace” is the key, and the description is the value. So it works like a real… dictionary! 📖\n",
    "\n",
    "You can have Lists of Dictionaries. Here’s how we can build one\n",
    "\n",
    "```python\n",
    "food_prices = []\n",
    "\n",
    "# Note the curly braces: \n",
    "food_prices.append({\"Apple\": 1})\n",
    "food_prices.append({\"Coco\": 5})\n",
    "Now the food_prices list will look like this: [{\"Apple: 1\"}, {\"Coco\": 5}]\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Session 2: Python Web Scraping Essentials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's import the libraries we'll need"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's set the URL for the page we're scraping, make a request to get the HTML and parse it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"http://books.toscrape.com/index.html\"\n",
    "response = requests.get(url)\n",
    "html = response.content\n",
    "scraped = BeautifulSoup(html, 'html.parser')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}