Different geometry decisions (point/line/polygon) on database re-import

[sommerluk]

Currently there is no database re-import planned. And apparently also no urgent need to do so. Anyway it might be useful to have a list of geometry decisions (point/line/polygon) that we might maybe change if in the future we would make a database re-import.

• barrier=toll_booth as line geometry (reference [DO NOT MERGE] Treat barrier=toll_booth as area #3249 and Render barrier=toll_booth areas #3244)

[kocio-pl]

Does it cover import issues like #3606?

[jeisenbe]

allotments=plot should imported as a polygon, so that we have the option of solving #432 "render ref numbers for allotments=plot"

[HolgerJeromin]

Healthcare #3639

[bhousel]

In case it helps, iD's areaKeys list is published here: https://github.com/osmlab/id-area-keys

The main file in there areaKeys.json contains a dictionary of keys that imply polygon, and subkeys of exceptions that imply linestring.

[kocio-pl]

I just asked OWG about possible database reload on openstreetmap/chef#211.

[matthijsmelissen]

@kocio-pl I think this might be a bit too hasty. Wouldn't it be better to first discuss within our own repository if and when a database reload is desired, before asking externally?

[kocio-pl]

Since this is about synchronizing efforts and we're dependent on them, it's good to know both sides IMO.

[jeisenbe]

boundary=national_park and boundary=protected_area should be added to local_polygon_values

[sommerluk]

boundary=national_park and boundary=protected_area should be added to local_polygon_values

Cross-reference #1141

[sommerluk]

@bhousel Thanks!

[jeisenbe]

"Golf=*" as polygon:
'rough', 'fairway', 'driving_range', 'water_hazard', 'green', 'bunker'

Possibly 'tee'?

The other values should remain as lines

[jeisenbe]

The key "emergency" should probably be imported as polygons.

[jeisenbe]

For reference, here are the list of keys that the ID editor considers to be areas (polygons) when drawn as closed ways, with the exceptions.

"advertising" (except "billboard")
"aerialway": (Except "cable_car", "chair_lift", "drag_lift", "gondola", "goods", "magic_carpet", "mixed_lift":, "platter", "rope_tow", "t-bar")
"aeroway": (Except "runway", taxiway")
"allotments"
"amenity": (except "bench")
"area:highway"
"attraction": (except "dark_ride", "river_rafting", "summer_toboggan", "train", "water_slide" )
"bridge:support"
"building"
"camp_site"
"club"
"craft"
"emergency" (except "designated", "destination", "no", "official", "private", "yes")
"golf": (except "hole", "lateral_water_hazard", "water_hazard")
"healthcare"
"historic"
"industrial"
"internet_access"
"junction" (except "circular", "roundabout")
"landuse"
"leisure" (except "slipway", "track")
"man_made" (except "breakwater", "crane", "cutline", "embankment", "groyne", "pier", "pipeline")
"military":
"natural": (except "cliff", "coastline", "ridge", "tree_row")
"office":
"piste:type": (except "downhill", "hike", "ice_skate", "nordic", "skitour", "sled", "sleigh")
"place"
"playground" (except "balancebeam", "slide", "zipwire")
"power": (except "cable", "line", "minor_line")
"public_transport": (except "platform")
"residential"
"seamark:type"
"shop":
"tourism" (except "artwork")
"traffic_calming": (except "bump", "cushion", "dip", "hump", "rumble_strip")
"waterway" (except "canal", "dam", "ditch", "drain", "river", "stream", "weir")

[chadrockey]

@jeisenbe golf:cartpath and golf:path are also lines that should be excluded.

lateral_water_hazard and water_hazards are now exclusively areas and should be removed from the exceptions list.

[jeisenbe]

@chadrockey - the list is from the ID Editor, and I've posted it just for information. I agree that water hazards should be polygons.

I think you can ask for this to be fixed in the ID Editor Github repository, with this link: https://github.com/openstreetmap/iD/issues/new

[chadrockey]

@jeisenbe The changes I mentioned were already made in the iD editor and are set to be released soon.

[pnorman]

I've been thinking about the best way to accumulate changes we want to make for the next major release with schema changes, etc. The best route is similar to what we did before with a long-running dev branch kept up to date with the minor releases that we can merge when we feel it's done.

[jeisenbe]

According to this comment, it's possible a database reload could happen soon. Should I submit a PR that includes the new geometry decisions now? Or do we need to wait? openstreetmap/chef#211 (comment)

[pnorman]

I'll make a new branch to PR against after v4.21.0. I don't see any rush since we're going to need to test any changes, and part of testing is full planet imports, so we'll take longer than setting up the new server.

[jeisenbe]

part of testing is full planet imports

Why is this required? I would think that testing with an large extract should have the same results, or is this to make sure that performance is not too slow during the full planet import after the Lua transformations are changed?

[pnorman]

I would think that testing with an large extract should have the same results, or is this to make sure that performance is not too slow during the full planet import after the Lua transformations are changed?

It depends on the nature of the changes, but if I were reviewing it or submitting it, I'd want to check how it works on the full dataset. There's also not that much difference between import time of a large extract like Europe or NA and the full planet.

[jeisenbe]

Re: golf; does anyone know if golf=out_of_bounds is meant to be mapped as a linestring (open way) or a polygon (closed way)? There is no wiki page. It's used <50 times, but it's mentioned on the main golf=* page - http://wiki.openstreetmap.org/wiki/Key:golf

[jeisenbe]

These are my plans so far on what keys and tags to import as polygons vs linestrings.

Even if we do not end up rendering all of the features, they should be available in the database so that other map styles based off of this style can use these features.

Keys to be imported as polygons:

• Craft
• Club - needs disucssion
• Golf* (except the tags listed below)
• Healthcare
• Playground - declined

Tags to be imported as polygon features:

• Allotments = plot
• Barrier = toll_booth
• Boundary = aboriginal_lands, national_park, protected_area
• Emergency = ambulance_station, control_centre, evacuation_centre, landing_site, lifeguard, lifeguard_base, lifeguard_tower, mountain_rescue, water_rescue_station - EDIT: see shorter list below

Tags to be imported as linestring features:

• Golf - golf=hole, cartpath, path
• Playground - balancebeam, slide, zipwire

Discussion

I reviewed the wiki pages and Taginfo for each of these keys. Craft, Club, and Healthcare are all often used to map features as an area with closed ways, and there do not appear to be any features that should be mapped as linestrings.

The emergency key is a mix of access restrictions and features. I believe it would be best to only render the features that are clearly unique (ignoring duplicates like emergency=water_tank and emergency=fire_water_pond), and which are used over 100 times and documented, so I selected the list above after reviewing taginfo and the wiki pages. See http://taginfo.openstreetmap.org/keys/emer…

The Golf key has a number of features that are usually mapped as areas, but golf=path, golf=cartpath and golf=hole are only used for linestrings. While it is unlikely that we will render golf=path or =cartpath, it is simplest to import all the golf features as polygons, and list these as exceptions

See below for Playground.

There are a few other features in the list from the ID editor (above). I don't think we need to import "area:highway" because rendering this has already been rejected (if I recall correctly?), and it's not necessary to import "industrial" or "residential" because these ought to be tagged with "landuse".

I haven't reviewed these keys yet, but I think they will need separate discussions

• advertising
• attraction
• camp_site
• internet_access
• piste:type
• seamark:type

[jeisenbe]

The Key "playground" is a little trickier.

Both JOSM and ID suggest that these should all be mapped as nodes or closed ways.

I downloaded 11,035 ways (there are 11, 101 according to taginfo) and 8,642 are closed ways, 2353 are unclosed. Thus 78% are closed, 22% not. Of the unclosed ways 1,870 have exactly 2 nodes, which suggests they were intentionally mapped in this way (and not a mistake).

The most common tags on unclosed ways are slide (810 features compared to 403 on closed ways), zipwire (593 vs 84), swing (223 open vs 495 closed), balancebeam (129 vs 23), structure (126 open vs 1730 closed), and climbingframe (103 vs 506 closed). So zipwires, slides and balance beams are usually tagged as open ways, but the rest are more commonly tagged on closed ways.

My (limited) understanding of the lua transformations is that the unclosed ways will still be imported as linestrings, and only the closed ways will be imported as polygons. Is this correct? In this case, there should be little harm in including all values in the key as polygons, right?

But we should exclude "balancebeam", "slide", and "zipwire" from the polygon transformation, as ID has done.

[jeisenbe]

[Advertising=column is] already rendered as a node

I did not realize this. In that case we might want to include it, even though only 3% of the features (303 out of 11k) are mapped as an area according to taginfo.

[pnorman]

I'm wondering if we should have general logic for =no not matching the rules for . This would cover building=no

[jeisenbe]

general logic for =no not matching the rules

That sounds good. What is the syntax for that?

[jeisenbe]

I've looked into advertising=column with more detail. The suggestion to map as an area seems to be a mistake. 98% are mapped as nodes. I checked a sample of the 222 features mapped as closed ways (another 80 or so are mapped as unclosed ways), and almost all are 1 to 2 meter diameter circles, so mapping as an area is not adding any information that could not be provided with width=1.5. The main text of the page has only mentioned mapping as a node. Since this tagging is quite rare, we do not need to encourage it.

[pnorman]

That sounds good. What is the syntax for that?

It would need to go inside the loops searching through the objects tags:

openstreetmap-carto/openstreetmap-carto.lua

Lines 395 to 397 in 295d65d

	if k == ptag and not (linestring_values[k] and linestring_values[k][v]) then
	return 1
	end

Right now for tags that match and don't have a polygon override it's an area, a v ~= 'no' check would need to be added.

[jeisenbe]

I also noticed that waterway=tidal_channel should be added to the list of waterway linestrings

[jeisenbe]

Any thoughts on importing attraction=, club= and emergency= as polygons when mapped as closed ways?

For attraction= there are a number which are mapped as lines, so we could import just these as polygons:
attraction=animal, =amusement_ride, =carousel, =big_wheel, and attraction=yes

Club= is always an area or node, though we probably won't render it. Should it still get imported properly as a polygon?

The emergency= features are all nodes and areas. There is a long list of rather rare tags, so perhaps the whole key should be imported as a polygon, even if these will not be rendered?

[jeisenbe]

It appears we can make a goal of finishing this discussion for the next release in a month.

Who has a server with sufficient RAM to import the whole planet.osm.pbf with the new schema_changes, to confirm that performance is acceptable? @pnorman?

[pnorman]

Who has a server with sufficient RAM to import the whole planet.osm.pbf with the new schema_changes, to confirm that performance is acceptable? @pnorman?

What specifically are we looking to performance test? I don't see anything that needs it

[jeisenbe]

Perhaps I misunderstood your comment above:

"we're going to need to test any changes, and part of testing is full planet imports"

#3611 (comment)

(And also #3611 (comment))

My understanding was that we ought to test the full planet import prior to merging schema_changes to master?

[jeisenbe]

Declined:

• [ ] attraction=animal, =amusement_ride, =carousel, =big_wheel, =maze as polygons. Unfortunately there is a "long tail" of features which could be imported as polygons, such as swing_carousel, drop_tower, winery, formal_garden, kiddie_ride, etc, but also these are intermingled with linear features like pirate_ship, dark_ride, boat_ride, river_rafting etc, so importing the whole key as polygon features would be imprecise.

Since we do not have a current plan to render these features, they will have to wait till another year.

[jeisenbe]

It would need to go inside the loops searching through the objects tags:

openstreetmap-carto/openstreetmap-carto.lua

Lines 395 to 397 in 295d65d

	if k == ptag and not (linestring_values[k] and linestring_values[k][v]) then
	return 1
	end

Right now for tags that match and don't have a polygon override it's an area, a v ~= 'no' check would need to be added.

I tried changing the line to:

            if k == ptag and not (linestring_values[k] and linestring_values[k][v] and v ~= 'no') then
                return 1

Using v4.24.0 as the basis for rendering, this still renders the hedge fill for barrier=hedge + landuse=no closed ways, but renders nothing at all for barrier=hedge + building=no, whether or not area=yes is used. Perhaps I have the syntax wrong?

Ideally, we would like to ignore <polygon_key>="no", but still import the feature properly if there is another polygon key or an area=yes on the feature.

(In practice, I think only "building=no" is very common)

[Adamant36]

No winery huh? Bummer.

[jeisenbe]

Do you mean attraction=winery? This is a rather rare, and undocumented, tag.

But I've submitted a PR to import craft= as a polygon, and craft=winery is common: https://taginfo.openstreetmap.org/tags/craft=winery

[Adamant36]

Yeah, I got it confused with craft though. My bad. It makes way more sense as a craft key.

[jeisenbe]

One more thing: should barrier=hedge closed ways be imported as polygons?

Discussion on the tagging list shows that some mappers are unhappy with #3844 which removed the area rendering for barrier=hedge, and there is not yet an alternative way of mapping hedges as areas (apparently some mappers disgree with use of natural=scrub for these areas, though landuse=forest is sometimes used for tall hedges).

In a comment, @imagico suggested the possibility of rendering barrier=hedge areas as landcover, with lower priority than other landuse, so that issue #971 would not reappear on features like leisure=playground + barrier=hedge.

See: https://github.com/imagico/osm-carto-alternative-colors/tree/hedge_polygons_as_landcover

However, currently this would only render barrier=hedge areas which are tagged with area=yes or mapped as multipolygons or tagged with another key which is interpreted as a polygon. It would be more consistent to import barrier=hedge areas as polygons if we are considering rendering them again.

(We could also consider this for barrier=wall and city wall areas, but these are less common and were never supported by this style).

[pnorman]

One more thing: should barrier=hedge closed ways be imported as polygons?

I'm against this. The first reason is that none of the other barrier values default to being a polygon. The second is that this adds another situation where you can't tell if something is an area or not based on tags. The third is it's against what all data consumers and editing software expects. The fourt is it doesn't reflect current usage. I sampled 32 closed ways without area tags on them, and 6% were probably intended to be areas, 84% were clearly lines, and the remainder I couldn't tell by looking at surrounding OSM data and imagery.

[imagico]

In conclusion i concur with @pnorman (as indicated before - me making a branch implementing that does not mean i am in favor of it, i just want to demonstrate that this would be the way we should implement it if we decide to do so) but i disagree with reason four - you cannot just look at the features without an area tag. In terms of closed ways with a barrier=hedge tag the ones meant to be 2d representations outnumber the 1d representations probably somewhere between 3:1 and 4:1. That most of these currently have an area=yes tag is largely due to us having incentivized that in the past.

But i agree on the other points and especially think we should be very careful in special casing new key/value combinations, especially if we render them both as 1d and 2d features. This makes life more complicated for all data users and mappers and the whole idea of dual use of tags for both 1d and 2d representations is bad practice we should avoid to support when not unavoidable due to extensive legacy use.