Scraper

Python web scrapers built using Selenium, BS4 and Playwright

Installation

Clone the git repository:

git clone https://github.com/nirantak/scraper.git && cd scraper
cp -nv .env.sample .env  # copy and update the env variables

Install necessary dependencies

python3 -m venv .venv
source .venv/bin/activate
pip install -U pip wheel setuptools
pip install -U -r requirements.txt
playwright install

See scrapers/README.md for usage instructions.

Samples present in demo/.

Name		Name	Last commit message	Last commit date
Latest commit History 29 Commits
.github		.github
demo		demo
out		out
scrapers		scrapers
.editorconfig		.editorconfig
.env.sample		.env.sample
.flake8		.flake8
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
.python-version		.python-version
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt