Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to Lua transforms and change database schema #2533

Merged
merged 101 commits into from May 10, 2017
Merged

Switch to Lua transforms and change database schema #2533

merged 101 commits into from May 10, 2017

Commits on May 16, 2016

  1. Database reload

    **This is a pull-request against the lua branch**
    
    This PR proposes a database-reload. It changes the documentation on the use of
    osm2pgsql, and adds a .lua file for preprocessing.
    
    This database import aims to be backwards compatible with older versions of
    the style.
    
    This resolves (at least part of) #1504.
    
     #Hstore
    Adding the hstore option to osm2pgsql allows the database to use all keys.
    
    This PR proposes using hstore for new keys, and using traditional columns for
    old keys. This allows us to stay compatible with old versions of the style.
    
    According to [@pnorman's benchmarks](http://www.paulnorman.ca/blog/2014/03/osm2pgsql-and-hstore/),
    using hstore without dropping columns would lead to a 9% size increase and a
    0.38% speed decrease. Given the benefits of having all columns available, I
    would consider this acceptable.
    
     # Make polygon/linestring decision based on value
    This is based on the following unmerged PR to osm2pgsql:
    osm2pgsql-dev/osm2pgsql#346
    
    It makes the polygon/linestring decision based on the tag value, in addition to
    the tag key. In particular, highway=services and junction=yes are now treated as
    polygon, while leisure=track, man_made=embankment, man_made=breakwater,
    man_made=groyne, natural=cliff, natural=tree_row, historic=citywalls,
    waterway=derelict_canal, waterway=ditch, waterway=drain, waterway=river,
    waterway=stream, waterway=wadi, waterway=weir, power=line, and power=minor_line
    are now treated as linestring.
    
    This resolves #1362, resolves #137, resolves #268, and resolves #892.
    
     # Rendering order
    The new .lua file creates a osmcarto_z_order value in the database, which stores
    the rendering order we use. This will allow us to simplify the road queries.
    
     # Recommend -G flag for multipolygons
    This resolves #59.
    
     # Import ele on buildings
    This resolves #101.
    
     # Delete more meta-tags from imports
    The .lua file drops some more meta-information from imports that is of no
    need for rendering.
    
    This is based on the following unmerged PR to osm2pgsql:
    * osm2pgsql-dev/osm2pgsql#532
    matthijsmelissen committed May 16, 2016
    Configuration menu
    Copy the full SHA
    d48106e View commit details
    Browse the repository at this point in the history

Commits on May 17, 2016

  1. Configuration menu
    Copy the full SHA
    e0aa01d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1ec1cfc View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    02edb0a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e5660e6 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    d524487 View commit details
    Browse the repository at this point in the history
  6. Code simplification

    matthijsmelissen committed May 17, 2016
    Configuration menu
    Copy the full SHA
    d276c90 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2016

  1. Typo

    matthijsmelissen committed May 22, 2016
    Configuration menu
    Copy the full SHA
    1b82cb6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    343d1c9 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2016

  1. Configuration menu
    Copy the full SHA
    8d0489e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d34bebc View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    21d9b80 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    118c5cf View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    6798824 View commit details
    Browse the repository at this point in the history
  6. Convet layer to integer and user layer instead of z_order

    * Convert layer to integer and use it in queries
    * Use layer instead of z_order in queries where possible
    matthijsmelissen committed May 30, 2016
    Configuration menu
    Copy the full SHA
    ae3f38e View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2016

  1. Configuration menu
    Copy the full SHA
    04956b4 View commit details
    Browse the repository at this point in the history
  2. Use luadoc strings

    pnorman committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d0b1f53 View commit details
    Browse the repository at this point in the history
  3. Only handle area=yes

    pnorman committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    73496d8 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    a43cb7a View commit details
    Browse the repository at this point in the history
  5. Name polygon function isarea

    pnorman committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    659840c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    10bff06 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2016

  1. Configuration menu
    Copy the full SHA
    5175195 View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2016

  1. Configuration menu
    Copy the full SHA
    be2eb57 View commit details
    Browse the repository at this point in the history
  2. Add testing framework

    pnorman committed Jun 16, 2016
    Configuration menu
    Copy the full SHA
    4499350 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2016

  1. Configuration menu
    Copy the full SHA
    3621bd7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    dc36874 View commit details
    Browse the repository at this point in the history
  3. Use spaces, not tabs

    matthijsmelissen committed Jun 18, 2016
    Configuration menu
    Copy the full SHA
    f2030f2 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2016

  1. Regenerate .mml

    matthijsmelissen committed Jun 26, 2016
    Configuration menu
    Copy the full SHA
    2ec88f3 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2016

  1. Merge remote-tracking branch 'math1985/lua_changes' into lua

    Conflicts:
    	openstreetmap-carto.style
    	project.mml
    pnorman committed Jul 1, 2016
    Configuration menu
    Copy the full SHA
    b5e4dbd View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2016

  1. Use 4sp for Lua indents

    pnorman committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    7216ac7 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2016

  1. Merge pull request #2217 from pnorman/lua_spacing

    Use 4sp for Lua indents
    matthijsmelissen committed Jul 8, 2016
    Configuration menu
    Copy the full SHA
    d28339a View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2016

  1. Merge tag 'v2.41.0' into lua

    Tag v2.41.0
    
    Conflicts:
    	project.mml
    pnorman committed Jul 13, 2016
    Configuration menu
    Copy the full SHA
    3ae8ca8 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2016

  1. Fix usage of ele

    pnorman committed Jul 19, 2016
    Configuration menu
    Copy the full SHA
    73a2cb8 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2016

  1. Merge tag 'v2.42.0' into lua

    Tag 2.42.0
    
    Conflicts:
    	project.mml
    matthijsmelissen committed Aug 3, 2016
    Configuration menu
    Copy the full SHA
    3dda54d View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2016

  1. Drop sport from polygon_keys

    Sport is a non-physical key, so it should always be combined with
    another, physical key. That physical key should define whether the
    object is a polygon or line. Since an object with a polygon key
    and a line key will be treated as polygon, treating sport as line
    by default is the sane default.
    
    For example:
    * sport=running, highway=footway should be treated as line. The
      current PR accomplishes this, before it would be a polygon.
    * sport=soccer, leisure=pitch should be treated as polygon. Nothing
      changes here.
    
    This resolves ... #854 but only after it gets merged into master.
    matthijsmelissen committed Aug 8, 2016
    Configuration menu
    Copy the full SHA
    3652b34 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #2274 from math1985/lua-sport

    Drop sport from polygon_keys
    pnorman committed Aug 8, 2016
    Configuration menu
    Copy the full SHA
    cbf2653 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2016

  1. Merge tag 'v2.43.0' into lua

    Tag v2.43.0
    
    Conflicts:
    	project.mml
    	project.yaml
    pnorman committed Sep 6, 2016
    Configuration menu
    Copy the full SHA
    0fb174b View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2016

  1. Merge tag 'v2.44.0' into lua

    Tag v2.44.0
    
    Conflicts:
    	project.mml
    pnorman committed Sep 26, 2016
    Configuration menu
    Copy the full SHA
    cf690e5 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2016

  1. Merge tag 'v2.44.1' into lua

    Tag v2.44.1
    
    Conflicts:
    	project.mml
    pnorman committed Oct 12, 2016
    Configuration menu
    Copy the full SHA
    9eb43a5 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2016

  1. Configuration menu
    Copy the full SHA
    8435825 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    aea093e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9e6e0d7 View commit details
    Browse the repository at this point in the history
  4. Add missing project.mml

    pnorman committed Nov 14, 2016
    Configuration menu
    Copy the full SHA
    2282dc6 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    afcafb2 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    2d424b4 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    60544b9 View commit details
    Browse the repository at this point in the history
  8. Merge pull request #2438 from pnorman/lua_location

    Use correct location for Lua script
    matthijsmelissen committed Nov 14, 2016
    Configuration menu
    Copy the full SHA
    1b6c1f0 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2016

  1. Configuration menu
    Copy the full SHA
    f56dc94 View commit details
    Browse the repository at this point in the history
  2. Remove osmcarto_z_order

    pnorman committed Nov 15, 2016
    Configuration menu
    Copy the full SHA
    6397f86 View commit details
    Browse the repository at this point in the history
  3. Only have route col for ways

    pnorman committed Nov 15, 2016
    Configuration menu
    Copy the full SHA
    db2d105 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4c437f1 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    bcdf12f View commit details
    Browse the repository at this point in the history
  6. Remove capital column

    pnorman committed Nov 15, 2016
    Configuration menu
    Copy the full SHA
    2aec867 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    ac5976a View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2016

  1. Configuration menu
    Copy the full SHA
    f6ba2a7 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #2455 from math1985/linestring-tags

    Treat leisure=slipway, natural=ridge,arete as linestring by default
    pnorman committed Nov 21, 2016
    Configuration menu
    Copy the full SHA
    c55384e View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2016

  1. fix svg regressions (fixes #2479, #2481)

    nebulon42 authored and kocio-pl committed Dec 2, 2016
    Configuration menu
    Copy the full SHA
    fea4ff2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0ae8b27 View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2016

  1. Merge tag 'v2.45.0' into lua

    Tag v2.45.0
    pnorman committed Dec 4, 2016
    Configuration menu
    Copy the full SHA
    42cf572 View commit details
    Browse the repository at this point in the history
  2. Merge tag 'v2.45.1' into lua

    pnorman committed Dec 4, 2016
    Configuration menu
    Copy the full SHA
    d7a4d4f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2502625 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1c7a001 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4986fbf View commit details
    Browse the repository at this point in the history
  6. Exclude SSY rail

    pnorman committed Dec 4, 2016
    Configuration menu
    Copy the full SHA
    d8f8223 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2016

  1. Merge pull request #2486 from pnorman/lua_loading_docs

    Add documentation about needing hstore
    pnorman committed Dec 5, 2016
    Configuration menu
    Copy the full SHA
    de4c547 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #2487 from pnorman/lua_rail_roads

    Lua rail roads
    pnorman committed Dec 5, 2016
    Configuration menu
    Copy the full SHA
    ab03d93 View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2016

  1. Configuration menu
    Copy the full SHA
    a7e7be8 View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2016

  1. Merge pull request #2499 from pnorman/lua/building_part

    Consider building:part a polygon tag
    pnorman committed Dec 9, 2016
    Configuration menu
    Copy the full SHA
    214a673 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2016

  1. Configuration menu
    Copy the full SHA
    2714ced View commit details
    Browse the repository at this point in the history
  2. Test lua isarea()

    pnorman committed Dec 20, 2016
    Configuration menu
    Copy the full SHA
    5161010 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2016

  1. Merge tag 'v3.0.0' into lua/master

    Tag v3.0.0
    pnorman committed Dec 22, 2016
    Configuration menu
    Copy the full SHA
    3f24b7d View commit details
    Browse the repository at this point in the history

Commits on Dec 24, 2016

  1. Rewrite relation handling code

    This code takes a stricter view on what a multipolygon is, and adds
    test coverage for relation handling.
    pnorman committed Dec 24, 2016
    Configuration menu
    Copy the full SHA
    6e6b6f8 View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2016

  1. Merge pull request #2527 from pnorman/lua/multipolygons

    Rewrite relation handling code
    pnorman committed Dec 27, 2016
    Configuration menu
    Copy the full SHA
    a1fda3d View commit details
    Browse the repository at this point in the history

Commits on Jan 2, 2017

  1. Merge tag 'v3.0.1' into lua

    Tag v3.0.1
    pnorman committed Jan 2, 2017
    Configuration menu
    Copy the full SHA
    e29b281 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2017

  1. Run transform tests in Travis

    pnorman committed Jan 19, 2017
    Configuration menu
    Copy the full SHA
    7c0bd30 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0f4d11f View commit details
    Browse the repository at this point in the history
  3. Add SQL style notes on hstore

    pnorman committed Jan 19, 2017
    Configuration menu
    Copy the full SHA
    8c524ce View commit details
    Browse the repository at this point in the history
  4. Luadoc fixes

    pnorman committed Jan 19, 2017
    Configuration menu
    Copy the full SHA
    3c8fc0f View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8aee71b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e4a4645 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    cf7e614 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    884134d View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    be75078 View commit details
    Browse the repository at this point in the history
  10. Simplify deleting tags

    pnorman committed Jan 19, 2017
    Configuration menu
    Copy the full SHA
    a9170aa View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    19a1894 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    55bc1cc View commit details
    Browse the repository at this point in the history
  13. Fix spelling of superseeded

    pnorman committed Jan 19, 2017
    Configuration menu
    Copy the full SHA
    75cf488 View commit details
    Browse the repository at this point in the history
  14. Merge pull request #2552 from pnorman/lua/hstore_style

    Add SQL style notes on hstore
    nebulon42 committed Jan 19, 2017
    Configuration menu
    Copy the full SHA
    31ef4fe View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2017

  1. Fix equaltables doc

    pnorman committed Jan 24, 2017
    Configuration menu
    Copy the full SHA
    03f3ad9 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    96f9a3f View commit details
    Browse the repository at this point in the history
  3. Remove unused variable

    pnorman committed Jan 24, 2017
    Configuration menu
    Copy the full SHA
    1fa3b0e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    76d239b View commit details
    Browse the repository at this point in the history
  5. Comment formatting

    pnorman committed Jan 24, 2017
    Configuration menu
    Copy the full SHA
    1f8413d View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2017

  1. Merge tag 'v3.1.0' into lua/master

    Tag v3.1.0
    pnorman committed Jan 29, 2017
    Configuration menu
    Copy the full SHA
    d274b62 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1b87f4a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    10e1cd1 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2017

  1. 1 Configuration menu
    Copy the full SHA
    cb23299 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2017

  1. Merge pull request #2610 from pnorman/lua/no_old_style_mps

    Remove support for old-style multipolygons
    pnorman committed Apr 17, 2017
    Configuration menu
    Copy the full SHA
    799aebf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    77cc9c4 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b6b155b View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2017

  1. Merge tag 'v3.2.0' into lua/master

    Tag v3.2.0
    pnorman committed Apr 18, 2017
    Configuration menu
    Copy the full SHA
    a002f47 View commit details
    Browse the repository at this point in the history