Skip to content

A news feed scraping API with Request-html and FastAPI

Notifications You must be signed in to change notification settings

hericlibong/NewsScraper_API

Repository files navigation

News Feed Sraper API with FastAPI

python Python

Presentation

In this repo I demonstrate how to easily build your own scraping API using FastAPI and the Request-html library. And how to build a basic news feed updated in real time.
The abstracts of articles and their links have been collected from Newswire, an online press collector and distributor.
The information concerns the latest news from Africa by regions.

The API has been enriched with some endpoints and several search parameters accessible on the Swagger interface delivered with FastAPI

Prerequisites

  • Python

versions3.10 or 3.8

Install and run

open a new folder with terminal

mkdir my_new_folder

create a virtual environment

virtualenv venv

... activate it

source venv/bin/activate
  • Clone the repo
  • open NewsScraper_API folder
  • install dependencies
pip install -r requirements.txt

and run main.py with uvicorn

uvicorn main:app --reload

To launch the documentation interface, you can use the following endpoints:

docs: Swagger UI, which allows you to test the API and see the automatically generated documentation. redoc: ReDoc, an alternative to Swagger UI

http://localhost:8000/docs
http://localhost:8000/redoc

About

A news feed scraping API with Request-html and FastAPI

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages