It's possible to avoid "403 Client Error: Forbidden for url" with url function?

[Jorman]

Hi, I'm trying to configure a very simple url, is a simple site where you find used items, no log-in required, and nothing special is needed. If I try a simple wget with the url, it downloads the page, but if I use the url in the urlwatch configuration it returns error
403 Client Error: Forbidden for url
The only way I found to not get error is to configure urlwatch with "navigate" instead of "url", but of course it is much slower. Is there any way to understand why "url" mode doesn't work with this site?

If you want to try this is my configuration:

name: "Test"
navigate: "https://www.subito.it/annunci-italia/vendita/auto/suzuki/?q=suzuki+vitara"
filter:
  - css:
      selector: 'div.ItemListContainer_container__SjEc1 > p'
diff_filter:
  - grep: '^[@+]'

Any ideas?

[thp]

It depends on what the server does, e.g. maybe it checks user-agent or some other headers (it's probably not your IP address if wget works). You can override the user-agent header.

[trevorshannon]

Sometimes you can also check the Google cache instead of the url directly. You can't do this very often (seems like a few times per day is ok) and it will not always be as immediately up-to-date as the direct url, but can help.

https://webcache.googleusercontent.com/search?q=cache:https://www.subito.it/annunci-italia/vendita/auto/suzuki/