urlscrapy

A web scrapper to extract the URLs embedded on the Website.

Installation

go install -v github.com/thevillagehacker/urlscrapy@latest

or

git clone https://github.com/thevillagehacker/urlscrapy.git
cd urlscrapy
go build

urlscrapy binary will be created and Move the binary to the required folder and add the path to the environment variables.

Usage

go run urlscrapy.go -u https://example.com

or

urlscrapy -u https://example.com

Update

• Can now pipe the collected URLs to stdin to other tools to check for status codes and others.

Example

urlscrapy -u https://example.com | httpx -<flags>