Using jq to filter buildings

[jaskiratr]

The following command also sets default height = 3.
jq --compact-output '.features | .[] | select(.properties.building != null) | select(.properties.height != null) , (select(.properties.height == null) | .properties.height = 3)' output.geojson > buildings.json

@auchers This may be used to replace the MongoDB workflow.

[jaskiratr]

1. Download OSM tiles
2. Convert OSM files to geojson
3. Extract buildings into a json array.

find -name '*.geojson' -exec jq --compact-output '.features | .[] \
| select(.properties.building != null) | (select(.properties.height != null) \
| try (.properties.height = (.properties.height | tonumber) )) , (select(.properties.height == null)\
| .properties.height = 3)' {} > buildings.json  +

Note This will reject features that have non-numeric height like height: "170 m" but height: "170" and height: 170 will work fine

[jaskiratr]

Nevermind, the following extracts the number from a string. Hence it will parse height: "170 m" > height: 170

find -name '*.geojson' -exec jq --compact-output '.features | .[] \
| select(.properties.building != null)  \
| (select(.properties.height != null) \
| ((.properties.height = try (.properties.height | tostring | \
capture("(?<num>[0-9]+)") | .num | tonumber)))), \
(select(.properties.height == null) | .properties.height = 3)' {} > buildings.json  +

[jaskiratr]

Tested on jqplay.org
Filter

.[] | select(.properties.building != null) | (select(.properties.height != null) | ((.properties.height = try (.properties.height | tostring | capture("(?<num>[0-9]+)")| .num | tonumber) catch .)   )) , (select(.properties.height == null) | .properties.height = 1)

Sample JSON

[
{
    "type": "Feature",
    "properties": {"party": "Green", "building": "yes", "height" : "10"}
},
 {
    "type": "Feature",
    "properties": {"party": "Republican", "building": "yes", "height" : 15 }
}, {
    "type": "Feature",
    "properties": {"party": "Democrat", "building": "yes", "height" : "20 m"}
},
{
    "type": "Feature",
    "properties": {"party": "Salty", "building": "yes"}
}]

Output

{"type":"Feature","properties":{"party":"green","building":"yes","height":10}}
{"type":"Feature","properties":{"party":"Republican","building":"yes","height":15}}
{"type":"Feature","properties":{"party":"Democrat","building":"yes","height":20}}
{"type":"Feature","properties":{"party":"Salty","building":"yes","height":1}}

[auchers]

In order to load the buildings into mongodb I had to turn the set of objects (output by jq) into an array of objects. I did that by wrapping the filter with '[]'. My final command was:

find . -name '*.geojson' -exec jq --compact-output '[.features | .[] | select(.properties.building != null) | (select(.properties.height != null) | ((.properties.height = try (.properties.height | tostring | capture("(?<num>[0-9]+)") | .num | tonumber)))), (select(.properties.height == null) | .properties.height = 3)]' {} > buildings.json +

[jaskiratr]

@auchers what command are you using for mongoimport?
It shouldn't require --jsonArray flag.

[auchers]

Yup — that’s the one. I’ll try it without. Thanks!

…

On Wed, Aug 29, 2018 at 12:42 AM Jaskirat Randhawa ***@***.***> wrote: @auchers <https://github.com/auchers> what command are you using for mongoimport? It shouldn't require --jsonArray flag. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG3fdh7szwZQbfEqFGf36fVC4S5Juyiaks5uVhubgaJpZM4WMRcA> .

[auchers]

Found a slight, but significant, bug here. This filter only includes objects that have a building property. This is a problem because many of the stratified building extrusions in Open Street Maps don't have this property (See image below)

Filter

.[] | select(.properties.building != null) | (select(.properties.height != null) | ((.properties.height = try (.properties.height | tostring | capture("(?<num>[0-9]+)")| .num | tonumber) catch .)   )) , (select(.properties.height == null) | .properties.height = 1)

Sample JSON:

[
{
    "type": "Feature",
    "properties": {"party": "Green", "building": "yes", "height" : "10"}
},
 {
    "type": "Feature",
    "properties": {"party": "Republican", "building": "yes", "height" : 15 }
}, {
    "type": "Feature",
    "properties": {"party": "Democrat", "building": "yes", "height" : "20 m"}
},
{
    "type": "Feature",
    "properties": {"party": "Salty", "building": "yes"}
},
{
    "type": "Feature",
    "properties": {"party": "Salty", "height": 30}
}]

Output

{"type":"Feature","properties":{"party":"Green","building":"yes","height":10}}
{"type":"Feature","properties":{"party":"Republican","building":"yes","height":15}}
{"type":"Feature","properties":{"party":"Democrat","building":"yes","height":20}}
{"type":"Feature","properties":{"party":"Salty","building":"yes","height":1}}
# missing the last object that doesn't have a 'building' property

[jaskiratr]

Sample JSON:

[{
   "type": "Feature",
   "properties": {"party": "A", "building:part": "yes", "height" : "10"}
},
{
   "type": "Feature",
   "properties": {"party": "B", "building:part": "yes"}
},
{
   "type": "Feature",
   "properties": {"party": "C", "building": "yes", "height" : 15 }
}, {
   "type": "Feature",
   "properties": {"party": "D", "building": "yes", "height" : "20 m"}
},
{
   "type": "Feature",
   "properties": {"party": "E", "building": "yes"}
}]

Following command adds a default height to building:part if it is missing

.[] | select(.properties.building != null or .properties["building:part"] != null) | (select(.properties.height != null) | ((.properties.height = try (.properties.height | tostring | capture("(?<num>[0-9]+)")| .num | tonumber) catch .)   )) , (select(.properties.height == null) | .properties.height = 1)

Following would leave building:part as it is.

.[] | select(.properties.building != null or .properties["building:part"] != null) | (select(.properties.height != null) | ((.properties.height = try (.properties.height | tostring | capture("(?<num>[0-9]+)")| .num | tonumber) catch .)   )) , (select(.properties.building != null and .properties.height == null) | .properties.height = 1), (select(.properties["building:part"] != null and .properties.height == null) )