Skip to content

Commit

Permalink
Added OSENV and built binary
Browse files Browse the repository at this point in the history
  • Loading branch information
NinjaAung committed Aug 31, 2022
1 parent 90e066a commit 6661687
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 5 deletions.
2 changes: 1 addition & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ func findWebsite(cid string, apiKey string) string {
func pushToSheet(flagged []Businesses, empty []Businesses, review []Businesses) {
ctx := context.Background()

b, err := ioutil.ReadFile("supersite-scraping.json")
b, err := ioutil.ReadFile(os.Getenv("JWT_CONFIG"))
errCheck(err)
conf, err := google.JWTConfigFromJSON(b, "https://www.googleapis.com/auth/spreadsheets")
errCheck(err)
Expand Down
28 changes: 24 additions & 4 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,30 @@

Scrap file to check if website is "broken" according to these patterns:

- Up for sell by domain name provider
- Url dosen't work 404
- Url is forwarded 301
- Website has minimal details
- Owned by domains
- Not a lot of content in body
- Website 404
- Other possible page errors

## How To Use

How to use will be updated
1. Place supersite application in folder with csvs

```env
|- Folder
|- supersite
|- livermore.csv
|- oakland.csv
|- sf.csv
```

2. Create an `.env` file with these items

```env
API_KEY= // Google Places API key from Google Console
SPREADSHEET_ID= // Spreadsheet you want to update
JWT_TOKEN= // JSON file from google console
```

3. run `./supersite` in terminal and it should start working
Binary file added supersite
Binary file not shown.

0 comments on commit 6661687

Please sign in to comment.