Week 6
Getting Data Part 2: Web Scraping

Updating your local environment

There are a few new packages we'll need for web scraping this week so we'll need to update your Python environment. The environment.yml in this repository contains all of the necessary packages. To update your local environment:

Step 1. Download the environment.yml file in this repository to your computer.

Important: Make sure you download the raw version of the file from GitHub and that the file extension on your computer is .yml.

Step 2. From either the Anaconda Prompt (Windows) or Terminal app (MacOS):

conda env update --file environment.yml --name musa-620

where musa-620 should be the same name of the environment you have been using.

New Packages

• Requests
• Beautiful Soup
• Selenium

Useful Links and Reference Materials

• Insights into Housing Markets from Scraping Craigslist
  • and accompanying academic article
• HTML tutorial
  • sections: introduction, basics, elements, attributes
• Tutorial on Web Scraping with Requests and BeautifulSoup
• CSS Selectors
• 101 Web Scraping Exercises