1.Description 2. Project Structure 3. Getting Started
This web scraper grabs all of the articles and related articles off of the google news web page, stores it into JSON and adds the articles (without duplicates) to a database.
📂 makescraper
├── README.md
└── scrape.go
-
Create an empty repository to store this file.
-
Run each command line-by-line in your terminal to set up the project:
$ git clone git@github.com:omarsagoo/makescraper_v1.git $ cd makescraper_v1 $ git remote rm origin $ git remote add origin git@github.com:YOUR_GITHUB_USERNAME/YOUR_REPO_NAME.git $ go mod download
-
Run this command to start scraping:
$ go build && ./makescraper_v1