Skip to content

v0.2.48..v0.2.49 changeset README.md

Jump to bottom
Garret Voltz edited this page Oct 2, 2019 · 1 revision 
diff --git a/README.md b/README.md
index 8d714e3..1260aa6 100644
--- a/README.md
+++ b/README.md
@@ -11,7 +11,7 @@ _Conflation_:
 
 1. Fancy word for merge
 
-[Hootenanny](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Introduction.asciidoc) is an open source conflation tool developed with machine learning techniques to facilitate automated and semi-automated conflation of critical Foundation GEOINT features in the topographic domain.  In short, it merges multiple maps into a single seamless map.
+[Hootenanny](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Introduction.asciidoc) is an open source conflation tool developed with machine learning techniques to facilitate automated and semi-automated conflation of critical Foundation GEOINT features in the topographic domain. In short, it merges multiple maps into a single seamless map.
 
 Hootenanny conflation occurs at the dataset level, where the user’s workflow determines the best reference dataset, source content, geometry, and attributes to transfer to the output map.  Hootenanny's internal processing leverages the key value pair structure of OpenStreetMap (OSM) for improved utility and applicability to broader user groups.  Normalized attributes can be used to aid in feature matching, and OSM’s free tagging system allows the map to include an unlimited number of attributes describing each feature.
 
@@ -22,34 +22,37 @@ Hootenanny conflation occurs at the dataset level, where the user’s workflow d
 * Create up-to-date routable transportation networks from multiple sources
 
 # Conflatable Feature Types
-* Area polygons
-* Building polygons
+* Areas
+* Buildings
 * Points of Interest (POIs)
-* Transportation polylines (roads and railways)
-* Utility polylines (power lines)
-* Waterway polylines
-
-Additional feature types can be made conflatable quickly via custom script by using Hootenanny's [Generic Conflation capability](https://github.com/ngageoint/hootenanny/blob/master/docs/developer/HootenannyConflatingANewFeatureTypeWithGenericConflation.asciidoc).
-
-# Types of [Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Introduction.asciidoc)
-## Default
-* **[Reference Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/user/OldDocs.asciidoc)** - Conflate the best geometry and tag parts of map B into map A, favoring map A's data. Use this type of conflation when you want conflated output based on the best state of both 
-input datasets.
-## Advanced
-* **[Attribute Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/AttributeConflation.asciidoc)** - Conflate map A with B where only tags are transferred from B to A and few or no changes are made to A's geometry. Use this type of conflation when the first 
-dataset's geometry is superior to a second dataset but the attributes of the second dataset are superior to that of the first dataset.
-* **[Differential Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/DifferentialConflation.asciidoc)** - Conflate map A with B where the only data added to the output from B is in areas that don't overlap with A. Use this type of conflation when you only
-want to fill in holes in your dataset with data from another source without modifying the first dataset. Optionally, you can configure to 
-overwrite tags in A from B even when there is overlap or to output the unmodified map A data as well.
-* **[Horizontal Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/commands/cookie-cut.asciidoc)** (aka Cookie Cutter Conflation) - Define a specific region in map A that should not be modified and stitch in data from map B around or inside of it. Use this type of
-conflation if you have a specific region of your dataset that you would like to replace with data from another dataset.
-
-There are a wide range of [configuration options](https://github.com/ngageoint/hootenanny/blob/master/conf/core/ConfigOptions.asciidoc) 
-available to customize the conflation workflows.
+* Power Lines
+* Railways
+* Rivers
+* Roads
+
+Additional feature types can be made conflatable via Javascript by using Hootenanny's [Generic Conflation capability](https://github.com/ngageoint/hootenanny/blob/master/docs/developer/HootenannyConflatingANewFeatureTypeWithGenericConflation.asciidoc).
+
+# [Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Introduction.asciidoc) Workflows
+* **[Reference Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/user/OldDocs.asciidoc)** (default) - _Keep the best of both_ - Conflate the best geometry and tag parts of map B into map A, favoring map A's data. Use this type of conflation when you want conflated output based on the best state of both input datasets.
+* **[Attribute Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/AttributeConflation.asciidoc)** - _Transfer attributes over to geometries_ - Conflate map A with B where only tags are transferred from B to matching features in A and no changes are made to A's geometries. Use this type of conflation when the first dataset's geometry is superior to a second dataset, but the attributes of the second dataset are superior to that of the first dataset.
+* **[Differential Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/DifferentialConflation.asciidoc)** - _Only add new features_ - Conflate map A with B where the only data added to the output from B is in areas that don't overlap with A. Use this type of conflation when you want to fill in holes in your dataset with data from another source without modifying any data in the first dataset.
+* **[Differential Tag Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/DifferentialConflation.asciidoc)** - _Only add new tags_ - Conflate map A with B where only tags are transferred from B to matching features in A and entire features are added from B to A in areas where B does not overlap with A. This workflow provides a mix of capabilities from both [Attribute Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/AttributeConflation.asciidoc) and regular [Differential Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/DifferentialConflation.asciidoc).
+* **[Horizontal Conflation](https://github.com/ngageoint/hootenanny/blob/master/docs/commands/cookie-cut.asciidoc)** (aka Cookie Cutter Conflation) - _Replace a section_ - 1) Define a region in map A and replace data in that region with data in the same region from map B OR 2) Define a region in map A to preserve and replace data outside of it with data outside of the region from map B. Use this type of conflation if you have a specific region of your dataset that you would like to replace with data from another dataset.
 
 # [Conflation Algorithms](https://github.com/ngageoint/hootenanny/blob/master/docs/algorithms/ConflationAlgsOverview.md)
 
-# Tag Schemas
+# Feature Filtering
+
+Hootenanny has the capability to let you selectively pick the features that are conflated from your data to save you from some pre-conflation data wrangling. Some examples:
+* conflate only buildings
+* conflate only restaurant buildings
+* conflate only restaurant buildings with "Subway" in the name
+
+[Command line example](https://github.com/ngageoint/hootenanny/blob/master/docs/user/CommandLineExamples.asciidoc#conflate-only-restaurants-see-the-user-guide-feature-filtering-section-for-more-filter-examples)
+
+Any of Hootenanny's many [available filters](https://github.com/ngageoint/hootenanny/blob/master/docs/user/CommandLineExamples.asciidoc#list-all-entities-that-can-operate-on-data) can be specified to perform feature filtering during conflation.
+
+# Attribute Translation
 Hootenanny leverages the OSM key value pair tag concept to support translation between various data schemas and supports automated schema conversion between: 
 * Topographic Data Store (TDS) v6.1/v4.0 
 * Multi-National Geospatial Co-Production Program (MGCP)
@@ -57,9 +60,19 @@ Hootenanny leverages the OSM key value pair tag concept to support translation b
 * OSM 
 * [others](https://github.com/ngageoint/hootenanny/tree/master/translations)
 
-Users can define their own [custom translations](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Hootenanny-id.asciidoc#translations).
+Users can define their own [custom translations](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Hootenanny-id.asciidoc#translations) via Javascript or Python.
 
-# [Supported Data Formats](https://github.com/ngageoint/hootenanny/tree/master/docs/user/SupportedDataFormats.asciidoc)
+# When To Use
+No automated map conflation technology is perfect. If you are conflating a relatively small number of features, you may be best served to conflate them manually yourself, given you are looking for perfectly conflated output and want to avoid any potential time spent configuring conflation software options to get the best conflated output. 
+
+For larger datasets, Hootenanny can be used standalone or as an inital step in conjunction with a crowd sourced [campaign](https://tasks.hotosm.org/) to conflate new data into your dataset. You will find that the conflation automation provided by Hootenanny saves effort overall, and that most inaccuracies in the conflated output are a small subset of the input data which end up being flagged for human review so they may later be manually corrected.
+
+# Scalability
+Hootenanny currently does not strive to conflate data at the global level. An earlier implementation of Hootenanny supported a map/reduce architecture that was capable of global conflation for some data types but was shelved due to general lack of interest and the maintenance costs to support the capability (so the conflation algorithms are capable of supporting distributed computing, with some limitations). Hootenanny generally can scale well running on a single machine from the larger city level up to the smaller country level, depending on the density of the data being conflated and the RAM available on the machine.
+
+# Configuration
+
+There are a wide range of [configuration options](https://github.com/ngageoint/hootenanny/blob/master/conf/core/ConfigOptions.asciidoc) available to customize the conflation and translation workflows.
 
 # Web User Interface
 [Hootenanny's](https://github.com/ngageoint/hootenanny-ui) [web user interface](https://github.com/ngageoint/hootenanny/blob/master/docs/user/Hootenanny-id.asciidoc) is built upon the open source 
@@ -100,7 +113,37 @@ new hoot.UnifyingConflator().apply(map)
 hoot.saveMap(map, "output.osm");

-# Additional Feature Summary +# Additional Features + +In addition to running conflation jobs with map data, Hootenanny also provides finer-grained capabilities: +* Add missing type tags to features based on their name +* Align two maps more closely with each other +* Apply data transformation operations to a map +* Calculate the geospatial extent of a map +* Clean map data +* Compare maps with each other +* Compute bounding tiles based on node density +* Convert maps between different geodata formats +* Crop a map to a geospatial extent +* Derive large changesets between maps and push them through the OpenStreetMap API with automatic conflict resolution +* Detect spoken languages in a map's tag data +* Explore tag data +* Filter a subset of features before conflation based on tag content and schema relationships +* Grab a random section of data from a map for testing against +* Crop a map to a geospatial extent +* Identify road intersections in a map +* Locate phone numbers geographically +* Perturb map data for testing purposes +* Plot node density +* Simplify ways by removing unnecessary nodes +* Snap unconnected ways to neighboring ways by type +* Process map postal address data +* Replace OpenStreetMap data completely within a bounding box +* Sort map data +* Translate feature tags using standardized or user defined schemas +* Translate feature tags to English + +# Supported Data Formats

Documentation

Support

  • FAQ -* If you have any support questions, please create an issue in this repository.

+Don't hesitate to ask for help if something isn't conflating how you expect it to. If you have any support questions, please create an issue in this repository. + +As there are lot of different conflation scenarios out in the wild, there is no one-size fits all conflation workflow or algorithm. Hootenanny attempts to capture most scenarios with the default configuration options, but sometimes you will need to modify configuration options specific to the data you are conflating to get the best results. + +Additionally, the addition of new software features to the user interface may lag their initial availability from the CLI by multiple development cycles. If you find a conflation feature you want to use mentioned in the CLI documentation but can't find it available in the UI, let us know.

Development

Clone this wiki locally