Code from the Latino Hackathon on International Open Data Day (21-02-2015)
We worked to identify foreclosed single-family houses in Cook County that have been bought by Invitation Homes, a subsidiary of the global private equity firm Blackstone.
We can find the properties via the Cook County Recorder of Deeds website, albeit through a clumsily manual process. This document describes the manual process involved.
Trying to improve upon this manual process, I worked on scraping from this site. It's an ASP.NET site apparently created by a Xerox contract in 2010 for the Cook County Recorder of Deeds, and this type of site is notoriously difficult to scrape.
I'm using the web browser automation library selenium with the headless browser PhantomJS to create a new session ID. I'm then simulating the POST requests of the real website using the Python library requests, based on monitoring the actual POST requests using the Developer Tools in Google Chrome. I'm using cookies also pulled from an actual session in Google Chrome.