Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Response 429 (Too Many Requests) if I select Open Street Maps #7

Open
DiHLoS opened this issue Apr 28, 2020 · 3 comments
Open

Comments

@DiHLoS
Copy link

DiHLoS commented Apr 28, 2020

...
127.0.0.1 - - [29/Apr/2020 00:15:04] "POST /download-tile HTTP/1.1" 200 -
HIT: https://a.tile.openstreetmap.org/{z}/{x}/{y}.png
RETURN: 429
...

You need find a way for add header at request 'urllib.request.urlretrieve', like this: {"User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0"} ?
Your code should seem like a Webbrowser for OSM servers, not a script.

@AliFlux
Copy link
Owner

AliFlux commented Apr 28, 2020

I'm not sure if it goes against OSM terms and conditions

@DiHLoS
Copy link
Author

DiHLoS commented Apr 28, 2020

I made a small example. I try to download the OSM tile in two ways:

  • requests.get (without any headers)
  • requests.get (with headers)

Only the second method works with OSM.
See code and result below.

Code:

import requests

url = "https://a.tile.openstreetmap.org/1/0/0.png"

print("Try download OSM tile without 'headers'...")
r = requests.get(url)
print(r)
if r.status_code == 200:
	with open("0.png", "wb") as f:  # saves '0.png'
		f.write(r.content)

print("Try download OSM tile with 'headers'...")
r = requests.get(url, headers={"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0"})
print(r)
if r.status_code == 200:
	with open("0_.png", "wb") as f:  # saves '0_.png'
		f.write(r.content)

Result:

Try download OSM tile without 'headers'...
<Response [429]>
Try download OSM tile with 'headers'...
<Response [200]>

If you change url to another tile server, for example: https://maps.wikimedia.org/osm-intl/1/0/0.png, both download methods are works properly:

Try download OSM tile without 'headers'...
<Response [200]>
Try download OSM tile with 'headers'...
<Response [200]>

OSM server does not allow to download the tile if no headers is specified...

ethosgr added a commit to ethosgr/MapTilesDownloader that referenced this issue Dec 15, 2020
Added Headers to fix issue AliFlux#7 .
@JaffaKetchup
Copy link

Note that it is against OSM tiles server rules to bulk download tiles like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants