Spider Trap

Inspiration

This project was inspired by Spider Trap

Description

A simple tool for catching web crawlers.

Build

cargo build --release

Run

cargo run

cargo run -- --port 8080 --directories ./web-directories.txt --min-links 5 --max-links 10 --min-length 3 --max-length 20

Usage

Usage: spidertrap-rs [OPTIONS]

Options:
      --port <PORT>              [default: 8080]
      --directories <DIRECTORIES>
      --min-links <MIN_LINKS>    [default: 5]
      --max-links <MAX_LINKS>    [default: 10]
      --min-length <MIN_LENGTH>  [default: 3]
      --max-length <MAX_LENGTH>  [default: 20]
  -h, --help                     Print help