Skip to content

In this repository i will expalin how to scrap websites using python programming language with BeautifulSoup and requestsmodulues

Notifications You must be signed in to change notification settings

rajat4665/web-scraping-with-python

Repository files navigation

web scraping with python
Introduction:

Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites using its HTML structure, In this post, I will explain basic fundaments of web scraping using python and also explore it by a live demonstration with two python libraries Beautifulsoup and requests respectively.

What you will learn from this post:

  • basic understanding of web scraping
  • how to extract data from a website using classes and HTML tags
  • how to use requests module to get data
  • how to use Beautifulsoup
Requirements:
  • python3
  • requests
  • bs4

Install required dependencies :

  • clone or download it from here
  • install requirements.txt file
  • pip install -r requirements.txt

How to run this code

  • there are two source code files, one is .py extention and another is .ipynb extention
  • one can run Scraping with BeautifulSoup.py file in python by run this cammand in terminal "python3 Web Scraping with BeautifulSoup.py"
  • one can run Scraping with BeautifulSoup.ipynb file in jupyter notebook /li>
  • one can install juypyter notebook by this command "pip3 install jupyter"
  • CLI scraping tool is underdevelopment only beta version is available now
----------------------------------------------------------------------------------------

HAPPY CODING