This project provides code for crawling through and scraping the search engines of Google and Bing. These programs serve the purpose to show the online visibility of selected companies. Based on these analyses they can reevaluate and develop new marketing strategies.
The Bing Search Crawler and Scraper is identical to the GoogleCrawler with different functions and classes for using the webbrowser, searching, clicking through pages and collect the details of ads and generic search results.
This is a newer version of my Google Crawling and Scraping program applied on heat pumps.
I've implemented some classes and functions to make the code much better readable and more efficient.
I have applied this program on 15 popular German Brokers and selected 5 keywords.
It uses ChromeDriver and Selenium to crawl through the google search engine and scrapes all the relevant data with BeautifulSoup.
Please note that this code is written for German search results. If you want to use it for your purposes, you maybe need to change some terms and paths. You also have to install a driver and all the required packages first to make it work.