Skip to content

Geospatial data picker via fast Api Rest interface, not only another simple elevation service.

License

Notifications You must be signed in to change notification settings

opengeo-tech/GeoPicker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

GeoPicker

geopicker

Geospatial dataset picker via fast Api Rest interface written in NodeJs for GDAL bindings and Fastify

Scope

It is basically an advanced elevation service and Geopicker has been specially designed to offer the widest range of formats and methods of data requests that is possible, to adapt to any context of use by the client. Each endpoint and the parameters it accepts have been designed on the basis of the functioning of already existing services, gathering a complete and coherent collection of APIs. At present the index.html page contains a large implementation of browser side requests using LeafletJs as basemap and jQuery.

Features

  • Large API Rest: ergonomic endpoints suitable for any type of use case
  • Validation: full validation of endpoint and parameters via JSON-Schema which allows output optimization
  • Configuration: friendly configs and to help devs in many deployment contexts
  • Formats: support for different geospatial input and output formats
  • Compression: configurable output compression if client accept encoding: deflate,gzip

and includes some other additional functions:

  • Densify: add more interpolated points in input coordinates, this improves the display on an elevation graph, adding intermediate positions at a minimum fixed distance.
  • Simplify: unlike densify it removes points that are too close together from coordinates.
  • Precision: alter the coordinates precision digits to lossily reduce the size of a geometry.
  • Height: add the vertical distance from the ground, if input has elevation add a fourth coordinate with this value.
  • Metadata: get on the fly metadata informations for a geometry (ex. length, direction, bbox, centroid, middlepoint)

API Rest endpoints

The API is work in progress. This basic structure can be extended starting from the environment variable PREFIX which by default /

(βœ”οΈ Work ❌ TODO 🚧 Work in Progress)

Status Method Path Return Description
βœ”οΈ GET / html default demo map page if enabled by env var DEMO_PAGE=true
βœ”οΈ GET /status object service status, versions, datasets
βœ”οΈ GET /datasets array list available datasets and their attributes
βœ”οΈ GET /datasets/:datasetId object search dataset by id
βœ”οΈ GET /datasets/:lon/:lat array search dataset contains lon,lat
βœ”οΈ GET /:datasetId object show attributes of a certain dataset by id
βœ”οΈ GET /:datasetId/:lon/:lat array get single location value of dataset, densify not supported
βœ”οΈ GET /:datasetId/:locations array locations is a string (format: `lon,lat
βœ”οΈ POST /:datasetId/lonlat arrays accept array or object in body
βœ”οΈ POST /:datasetId/locations arrays accept array or object of locations in body (format is [[lon,lat],[lon,lat],[lon,lat]])
βœ”οΈ POST /:datasetId/geometry object geojson Point or LineString in body (support feature/geometry/f.collection)
βœ”οΈ GET /metadata/:locations object return info about direction, length, centroid, middlepoint of locations
βœ”οΈ POST /metadata/geometry object return info about direction, length, centroid, middlepoint of geometry

Global Parameters

Status Parameter Default Description
βœ”οΈ precision input rounded to digits decimal precision
βœ”οΈ format input output format conversion
βœ”οΈ densify input enable densification of points in the result
βœ”οΈ simplify input enable simplication geometry of the result
❌ height false add vertical distance from the ground(only input has elevation)

Some behaviors to know about parameters are that:

  • precision and densify parameters is only supported by endpoints and formats that return coordinates
  • datasetId can have the value default to referring the main dataset defined in config
  • from version v1.6.1 /<datasetId>/... is the same of /datasets/<datasetId>/... /datasets/ is implicit.

Formats

If the format parameter is not specified the default behavior is to output the same format as the input

  • input format can be specified by Content-type: header in request
  • output format can be specified by format parameter

the support for various input and output formats is summarized in the table

Value In Out Description
input βœ”οΈ βœ”οΈ means the same format as the input data
array βœ”οΈ 🚧 each location is Array and a Z dimension as value [lon,lat,val]
json βœ”οΈ 🚧 each location is Object having lon,lat and val attributes
geojson βœ”οΈ 🚧 standard GeoJSON objects Feature, Geometry with a Z dimension in coordinates as value
polyline 🚧 βœ”οΈ Encoded Polyline Algorithm
gpx 🚧 βœ”οΈ GPS eXchange Format is an XML textual format
csv ❌ ❌ Comma-separated values is an textual format
kml ❌ ❌ Keyhole Markup Language is an XML format for Google Earth

each endpoint has its own default format, for example endpoint /dataset/lon/lat return a simple array of one value.

Usage

Running by official Docker image:

docker run -v "/$(pwd)/tests/data:/data" -e DEMO_PAGE=true -p 9090:9090 stefcud/geopicker

Running from source code in development mode, requirements: nodejs 16.x > and glibc 2.28 (Ubuntu 20.x > ):

npm install
cd server && npm install && cd -
npm run dev

Browse the demo page: http://localhost:9090/

Configuration

Full configuration options can be found in docs config

Development

some useful tools for contributors npm run <scriptname>

  • start run in production mode
  • dev run in development mode
  • docker-up run in local docker-compose container
  • bench run benchmarks
  • npm publish . build and publish new docker image

Requests Example

Get single location exchanging a few bytes:

 $ curl "http://localhost:9090/default/11.123/46.123"

[195]
$ curl "http://localhost:9090/default/11.123/46.123?format=gpx"

output is a waypoint in GPX format:

<gpx version="1.1" creator="Geopicker">
<metadata/>
<wpt lat="46.123" lon="11.123">
<name/>
<desc/>
<ele>400</ele>
</wpt>
</gpx>

Post a json object and receive the same decorated with the result(still works with longitude,latitude):

$ curl -X POST -d '{"lon": 11.123, "lat": 46.123"}' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/lonlat"

{"lon": 11.123,"lat": 46.123,"val":195}

Get many stringified locations in one time(designed for not too long LineString):

curl "http://localhost:9090/elevation/11.1,46.1|11.2,46.2|11.3,46.3"

[195,1149,1051]

Post a very long LineString saving bytes:

$ curl -X POST -d '[[10.9998,46.0064],[10.9998,46.0065],[10.9999,46.0066],[11.0000,46.0067]]' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/locations"

[[10.9998,46.0064,900],[10.9998,46.0065,898],[10.9999,46.0066,898],[11.0000,46.0067,900]]

Post anyone GeoJSON geometry, the same input geometry is always returned which has a third dimension:

$ curl -X POST -d '{"type":"LineString","coordinates":[[11.1,46.1],[11.2,46.2],[11.3,46.3]]}' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/geometry"

{"type":"LineString","coordinates":[[11.1,46.1,195],[11.2,46.2,1149],[11.3,46.3,1051]]}

Post a GeoJSON geometry of 2 points and Densify the linestring adding point each 400 meters, this will help you build a less angular elevation graph:

$ curl -X POST -d '{"type":"LineString","coordinates":[[11,46],[11.01,46.01]]}' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/geometry?densify=400&precision=5"

output contains 3 additional interpolated locations with reduced precision digits:

{
    "type": "Feature",
    "geometry": {
        "type": "LineString",
        "coordinates": [
          [11, 46, 897],
          [11.003, 46.003, 968],
          [11.006, 46.006, 1029],
          [11.009, 46.009, 1122],
          [11.01, 46.01, 1187]
        ]
    }
}

Get the elevation value between two locations every 100 meters

curl "http://localhost:9090/elevation/11,46|11.01,46.01?densify=100"

[925,858,909,963,968,1001,1018,997,1025,1062,1064,1102,1115,1163,1187]

Benchmarks

benchmarks scripts: tests/benchmarks.js using AutoCannon

cd tests && npm install && cd -
npm run bench

The results testing a dataset of 2x2km geotiff

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”
β”‚ Stat    β”‚ 2.5% β”‚ 50%  β”‚ 97.5% β”‚ 99%  β”‚ Avg     β”‚ Stdev   β”‚ Max  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€
β”‚ Latency β”‚ 0 ms β”‚ 0 ms β”‚ 0 ms  β”‚ 1 ms β”‚ 0.02 ms β”‚ 0.16 ms β”‚ 6 ms β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”˜
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Stat      β”‚ 1%      β”‚ 2.5%    β”‚ 50%     β”‚ 97.5%   β”‚ Avg      β”‚ Stdev   β”‚ Min     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Req/Sec   β”‚ 18111   β”‚ 18111   β”‚ 22783   β”‚ 23471   β”‚ 22175.28 β”‚ 1473.21 β”‚ 18099   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Bytes/Sec β”‚ 4.02 MB β”‚ 4.02 MB β”‚ 5.05 MB β”‚ 5.21 MB β”‚ 4.92 MB  β”‚ 327 kB  β”‚ 4.01 MB β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Req/Bytes counts sampled once per second.
# of samples: 11

244k requests in 11.01s, 54.1 MB read

Roadmap

for details see the descriptions in the Roadmap issues

Status Goal
βœ”οΈ Swagger Documentation Interface
🚧 manage multiple datasets
🚧 ES6 modules
🚧 extend benchmarks for any endpoints
βœ”οΈ enable densify function
🚧 enable simply function
❌ unit testing
❌ support vector format in datasets, such as shapefile
❌ limit access by api key
❌ caching responses
❌ interfaces: websocket, jsonrpc
❌ command line interface

Copyright

Created by Stefano Cudini @zakis Distributed under the BSD 2-Clause license.