Load and display OSM elements with addr:housenumber from overpass.api.de #27
Comments
|
Is this worth it? You can already see all the housenumbers on the default "Osm-carto" view. Adding this might help with other views. It could also mean that you can edit existing numbers. However, implementing the ability to edit existing numbers will be more difficult I think |
IpswichMapper
added
enhancement
|
@stephan75 thoughts? |
|
When writing this feature request I had in mind certain POIs where the OSM standard carto style only displays its POI icon, but NOT the house number, which is definitively tagged on that node or building. |
|
Unfortunately I have no android programming / coding skills, MOVED TODO LIST TO FIRST ISSUE COMMENT Maybe we can get some initial hints from @westnordost who is the developer of StreetComplete? |
|
I don't think this will be very difficult to implement (maybe it will take some time). I've already worked with Overpass API to create the "dropdown" feature when you try and manually enter a housenumber (i.e., when you click on the street name in the keypad and start typing out the street, a dropdown will show displaying the names of all the streets that are close by). I've also worked with boundingboxes already to create the "proof of concept" for downloading map tiles. All you have to do is parse the XML and display them as markers and the map. I have other issues though, so as if I add the ability to "edit" these, then the .osm files generated will contain already existing nodes. Will this cause issues when merging? And also, large requests to Overpass API can take relatively long to parse too. Also, would it clutter the map? |
|
I've written a Java library to query tgr overpass api. In case you are interested:
https://github.com/westnordost/osmapi-overpass
…On April 7, 2021 11:35:59 PM GMT+02:00, IpswichMapper ***@***.***> wrote:
I don't think this will be very difficult to implement (maybe it will
take some time). I've already worked with Overpass API to create the
"dropdown" feature when you try and manually enter a housenumber (i.e.,
when you click on the street name in the keypad and start typing out
the street, a dropdown will show displaying the names of all the
streets that are close by). I've also worked with boundingboxes already
to create the "proof of concept" for downloading map tiles.
All you have to do is parse the XML and display them as markers and the
map.
I have other issues though, so as if I add the ability to "edit" these,
then the .osm files generated will contain already existing nodes. Will
this cause issues when merging? And also, large requests to Overpass
API can take relatively long to parse too.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#27 (comment)
--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
|
|
This is what I wrote for the query: boundingBox = mapView.boundingBox
Thread {
val queryText = "https://overpass-api.de/api/interpreter?data=" +
"<query type='node'>" +
"<has-kv k='addr:housenumber' regv='.+'/>" +
"<bbox-query s='${boundingBox.latSouth}' w='${boundingBox.lonWest}' " +
"n='${boundingBox.latNorth}' e='${boundingBox.lonEast}' />" +
"</query> <print />"
val query = URL(queryText)
val result = query.readText()
}.start()seems good enough, however, it isn't working reliably at the moment because osmdroid's "onZoom()" function doesn't activate properly when using a "DelayedMapListener" (which is a listener that activies only after you have finished zooming or scrolling). See osmdroid/osmdroid#1712 Is your library more efficient than the normal Kotlin API code I have shown above? Another thing, how can I make this efficient / avoid redownloading housenumbers? For example, if I were to scroll the map so that half of it still the old boundingBox, using a normal DelayedMapListener all the housenumbers would have to be downloaded (there is quite a bit of delay for this). Is there a way to only download sections that you haven't downloaded already? I have checked off all the boxes that I did today and added some more detail to your checklist. |
The library is not about efficiency but convenience: It does the querying and the parsing of the data for you. If I remember correctly, it has built-in support for parsing the "out geom" result as well, so you directly get the geometry for each returned element too. |
val myDataHandler = object : MapDataWithGeometryHandler {
fun handle(bounds: BoundingBox) {
/* here you get returned the bounding box of your request */
}
fun handle(node: Node) {
/* this is called for every node in the overpass result */
}
fun handle(way: Way, geometry: List<LatLon>) {
/* this is called for every way in the overpass result, geometry is attached */
}
fun handle(relation: Relation, bounds: BoundingBox, nodeGeometries: Map<Long, LatLon>, wayGeometries: Map<Long, List<LatLon>>) {
/* this is called for every relation in the overpass result, geometry is attached */
}
}
// get all housenumbers on Malta
overpassApi.queryElementsWithGeometry(
"[bbox:13.8,35.5,14.9,36.3]; nwr['addr:housenumber']; out meta geom;",
myDataHandler
)What you do in |
How would you get the values for each node? For example, in the Plus it gives me an error when building when I add this dependancy: Here are my dependancies: Even though your library isn't mentioned in the duplicate class StackTrace, removing that library fixes the problem for some reason. |
Should be not problem to use XML in the query. The library just passes the query string through to Overpass and does not do anything with it.
Right, it seems that on Android, certain classes (XmlPullParser, JetBrains annotations) are already defined, so you need to remove them from that dependency. implementation 'de.westnordost:osmapi-overpass:1.3' {
// xml pull parser already included in Android
exclude group: 'net.sf.kxml', module: 'kxml2'
// NonNull etc annotations are already available in Android
exclude group: 'org.jetbrains', module: 'annotations'
exclude group: 'com.intellij', module: 'annotations'
exclude group: 'org.intellij', module: 'annotations'
} |
|
In an old version of StreetComplete, I found this block configurations {
// it's already included in Android
all*.exclude group: 'net.sf.kxml', module: 'kxml2'
cleanedAnnotations
compile.exclude group: 'org.jetbrains', module:'annotations'
compile.exclude group: 'com.intellij', module:'annotations'
compile.exclude group: 'org.intellij', module:'annotations'
}This excludes the given dependencies not only for the specified dependency but for all dependencies. Please if anything of that works, let me know, then I can update the README.md in that repos. |
|
I just tried using the OSMAPI library that you have for uploading/downloading OSM data (using APIv0.6). It says to add this to your code: (Note: So it seems like Also, another thing that is confusing me is branches. I made the above |
This does not happen. What made you think this is what happened? If you switch branches, I recommend re-sync the project with the gradle files and running hte clean task.
Okay, just as a test, what about only implementation 'de.westnordost:osmapi-core:1.4' {
exclude group: 'net.sf.kxml', module: 'kxml12'
}? And if this gives an error, what about dependencies {
implementation 'de.westnordost:osmapi-core:1.4'
}
configurations {
all*.exclude group: 'net.sf.kxml', module: 'kxml2'
} |
Turns out you have to commit changes in order for them to be tied to that branch. (Other option is to use
Note, even with implementation the actual library needs to be in brackets if you want to do that: That didn't work, however, the bug still showed up. This, however, even with implementation the actual library needs to be in brackets if you want to do that: That didn't work, however, the bug still showed up. This, however, worked fine: |
|
Seems like, in your documention, this: compile ('de.westnordost:osmapi:3.11')
{
exclude group: 'net.sf.kxml', module: 'kxml2' // already included in Android
}needs to be replaced with this: dependancies {
implementation 'de.westnordost:osmapi:3.11'
}
configurations {
all*.exclude group: 'net.sf.kxml', module: 'kxml2'
} |
|
Hm? |
|
In the documentation for OSMAPI (the api v0.6 version), it says to use the "exclude" command to remove the However, that doesn't work, but rather |
|
Got it working. However, on my first run, I got rate limited (it says Not only this, but the downloading really lagged the app. I changed the zoomLevel from |
|
@stephan75 I don't think continuous download of housenumbers when you are zoomed in is possible. Not only would it send far too many requests to the server, it would also lead to duplication of markers and would lag the app. I think a better solution is to be able to download already existing markers in the "DownloadTilesFragment". This can be accessed from the Imagery icon in the top right of the map. When downloading tiles, I could add an option to also download existing housenumbers in that area. |
On your first request? Well, the request limiting is per IP. So if you made requests earlier, you have to wait. Note the other methods of the OverpassMapDataDao. You can actually query how long more you have to wait until you are allowed to do the next request. But in any way, you should specify the correct user agent (in this case, "SwiftAddress") Anyway, check how many requests you do. Maybe you do too many requests.
You are downloading in a background thread, right? OverpassMapDataDao doesn't do any threading on its own. |
Sorry, I meant my first run of the app. I was using a "DelayedMapListener" to call the function. That means that everytime I stop scrolling or stop zooming, it calls the function.
You are downloading in a background thread, right? OverpassMapDataDao doesn't do any threading on its own. Placing the markers has to happen on a main thread, however. Since then I increased the zoom level, and that issue went away. That being said, do you think it is possible to have a continuous download like @stephan75 wants? Streetcomplete only downloads data when you tell it to. Would it be possible to use the overpass API to download data continuously when you scroll the map? |
That's too much. You should memorize which areas you already downloaded and memorize the results. StreetComplete does this like this:
iD does it the same, and I assume that every other proper editor that allows free scrolling (OsmInEdit, Osm Go, .... for example) does it the same. Maybe not persisting it into a DB, but at least persisting the information into memory.
No, StreetComplete downloads automatically areas around the user's current location. The logic is functionally the same as the "download everything that comes into view" as it is done for iD etc. |
|
Though in all fairness, I have to tell you that it is almost certain that I will implement a view that looks something like this
in StreetComplete within the next 3-4 months or so, as described in detail here streetcomplete/StreetComplete#2461 |
So tasks are downloaded automatically? I think I got confused there because when I used to use it, I would always tell it to download manually tasks in an area. When they are downloaded automatically, isn't it in the area surrounding your current location? I'm pretty sure it doesn't start downloading tasks when you zoom in. Could you create code where housenumbers are downloaded when you zoom in enough using the techniques you have shown? |
Yes, around your GPS location. You can additionally download areas manually, if you want to download areas not around your location in advance.
Correct.
What do you mean exactly? If I can theoretically? |
Sorry, bad phrasing. I essentially meant, "can it be done". I'm not asking you to do it for me 😅. |
|
Yes, of course. val downloadedTiles = HashSet<Tile>()
@Synchronized fun download(bbox: Bounds) {
val tiles = bbox.asTilesAtZoomLevel(16)
if (downloadedTiles.containsAll(tiles)) return
val bboxExpandedToTileEdges = tiles.toBbox()
doTheDownloadNow(bboxExpandedToTileEdges )
downloadedTiles.putAll(tiles)
}
fun onMapViewHasChangedInAnyWay() {
val bbox = getDisplayedBoundingBox()
download(bbox)
} |
|
Thanks for all the help. Currently I'm trying to login to the "sandbox api". How do you get an OAuthConsumer? val provider: OAuthProvider = DefaultOAuthProvider(
"https://api06.dev.openstreetmap.org/oauth/request_token",
"https://api06.dev.openstreetmap.org/oauth/access_token",
"https://api06.dev.openstreetmap.org/oauth/authorize"
)
val connection = OsmConnection("https://api06.dev.openstreetmap.org/", null, null)You do you get this "consumer" after the user has entered a username and password? https://github.com/mttkay/signpost/blob/master/docs/GettingStarted.md This guide does not mention what role the username and password play. |
|
Not as simple as that. You need to go through the OAuth process. The Sandbox Api is the same as the normal api. So you'd need to create an account on the dev instance of openstreetmap. Have a look at: However, you don't need to be logged in at OpenStreetMap.org to do an overpass query. So the "OsmConnection" you use for the overpass-api can have |
This is for uploading data to OSM. Which is something I was the app to eventually do.
I did, but I'm still confused as to where this What is inState, what is |
This comment has been minimized.
This comment has been minimized.
|
Oh wait - You sign in through the OSM Website. That makes sense then, you are just opening the URL and getting back the oauth tokens |
|
It is not entered in the app. If you want to know the details, read up on how OAuth 1 works. The |
Having a continuous download from overpass-api when dragging or zooming the map view was not my my prime aim, to be honest. I was indeed more thinking about how StreetComplete does it. Most minimalistic solution: only download housenumbers when the user has done a certain action, maybe by pressing a button like "download housenumber data" or similar. Maybe there are also some users who want to use the SwiftAddress app in the fields without an online connection. PS: I am really very very amazed about the efforts of you two concerning this feature request. Many thanks! This coding things are really out of my scope. But keep on! |
|
@stephan75 Is it okay if I only add a "download housenumbers" dialog in the "imagery" tab? Trying to implement this automatically will be a lot of effort. |
|
Yes it is okay when that download starts by a user action. No need for automatic download, I think. |
What about the following feature request:
When the map view of SwiftAddress is zoomed in enough, enable a button that downloads all OSM elements from overpass-api.de that contain a tagging with addr:housenumber=*
and then display each element from that data collection on the map view.
In this way, a mapper can see where buildings are present in the OSM data which already have a tag addr:housenumber, and where this tag is missing.
Maybe as a start, try this method only for OSM nodes ... and extend this method to ways and relations later.
ToDo steps moved from a following comment to have github's progress bar:
The text was updated successfully, but these errors were encountered: