Response code 403 (Forbidden)

[wiktorpikosz]

Hey,

I used codemanki/cloudscraper but author decided to archived his project and I looking for a alternative.

I tracked a lot of websites and cloudscraper usually works fine but for one website is a problem. But when the project was archived, it isn't good prognostic for future. So I tested your solution.

When I try load it: https://vitals.com - I get error:

{
"name": "HTTPError",
"timings": {
"start": 1589285452689,
"socket": 1589285453194,
"lookup": 1589285453195,
"connect": 1589285453195,
"secureConnect": 1589285453195,
"upload": 1589285453197,
"response": 1589285453359,
"end": 1589285453395,
"phases": {
"wait": 505,
"dns": 1,
"tcp": 0,
"tls": 0,
"request": 2,
"firstByte": 162,
"download": 36,
"total": 706
}
}
}

In the stack is:
HTTPError: Response code 403 (Forbidden) at PromisableRequest.request.once (/root/cloudflare-bot/node_modules/got/dist/source/as-promise/index.js:124:28) at process._tickCallback (internal/process/next_tick.js:68:7)

Could you tell me, if your library should work fine for this example and I'm doing wrong something or your library don't support it.

[sayem314]

I can't even use that site in the browser.

[wiktorpikosz]

Thanks for your fast answer!

I don't have this problem:

Maybe you have a ban, because has tested something on your localhost :D

You can change IP? VPN?

[sayem314]

Yes, VPN worked. Their site is very well protected, I will try to make it work but no guarantee. You can always use puppeteer tho.

[wiktorpikosz]

Ok, thanks!

Yeah, but pupeeteer uses headless and needs instance of chromium if I understand their description. It's barier for me.

[msjonker]

I've run into the same error with a site that I am scraping, and the way I got around it was to enable http2 in the got request.