Frequently Asked Questions

Q: Does Hootenanny require any specialized training or expert knowledge to use?

You do not need to know anything necessarily about GIS or conflation to use the software but some background on Openstreetmap is typically needed to fully understand the data lifecycle within Hootenanny. Knowledge of Topographic Data Store (TDS), Multinational Geospatial Co-Production Program (MGCP) or other supported data schemas may also be helpful.

It is also important background for users i.e. data stewards, to know the source data and how they are derived (automatic feature extraction, heads-up digitizing, etc..) since conflation processes will assume they are appropriate datasets to merge.

In general, the Hootenanny UI is designed with ease of use in mind but more technical knowledge will certainly be useful for interpreting the results of a conflation output.

Q: Please describe Hootenanny's ability to conflate attributes from non-authoritative sources into authoritative source layers where authoritative attribution is missing.

Hootenanny supports tag merging from non-authoritative into authoritative but it is not specific to cases where attribution is absent.

Q: Please describe Hootenanny's multi-user handling of a dataset? How does Hootenanny handle re-integrating the chunks and edge matching?

Hootenanny is designed to handle discrete conflation tasks and generate a set of reviews that can be queued out to multiple users with access to the same server instance. Hootenanny may be used with Tasking Manager to handle multiple work cells that are merged back into a central environment.

Q: How does Hootenanny handle matching datasets of different scale, example 100k to 500k?

Hootenanny relies on the data steward to determine whether input datasets are appropriate candidates for conflation. If for example, one input layer is a CADRG derived roads layer created from 1:100K RPF data and another is digitized from 0.5 meter high resolution commercial imagery, it will probably be inappropriate to conflate these two source datasets together because it would create an invalid product and also Hootenanny would likely not merge many features due to the misalignment of most of the feature geometry. That said, there are some use cases where it makes sense to merge data derived from two different input spatial or precision/accuracy data sources. To meet those needs, Hootenanny includes a cookie-cutter and horizontal conflation option which allows users to conflate a high quality (in terms of precision, currency, resolution, etc) urban dataset against a lower quality, coarser resolution peri-urban or rural dataset that is general larger in coverage area. Hootenanny will "cut" out a buffered region of the high resolution data and match it to the lower quality "rural" data to create a seamless product that incorporates both input sources. For more background on this methodology see the Cookie Cutter examples in the Hoot User Interface Guide and Hoot User Guide).

Q: How do you preserve unmatched road network connectivity? In a situation where a primary road is matched in both datasets, is connectivity preserved with unmatched secondary roads present only in non-authoritative data connected to matched primary road?

To preserve unmatched road network connectivity when conflating two input road networks, the secondary dataset intersection in the non-authoritative data is snapped to the authoritative road thus preserving connectivity between the features.

Q: How does Hootenanny handle specified null values for transfer of attributes between authoritative and non-authoritative sources?

Null, default and empty values are dropped during import. If an attribute is not present at export but required by a particular translation schema, it is created with a default OSM/TDS value and included in the output dataset.

Q: Does Hootenanny support user defined data exclusion?

Hootenanny does not currently implement a user-defined data exclusion AOI or ROI but this could be a possible customization to the product to implement using a forked version.

Q: How does Hootenanny preserve and process data within a user specified schema?

Hootenanny does all processing using an enhanced version of the OSM schema (http://wiki.openstreetmap.org/wiki/Map_Features). This is why we load and translate the data before conflation and then write it out as an OSM file or translate it to something else (TDS, MGCP etc). That said we do have requests to support user-defined outbound schema export so it could be possible to maintain the user-specified schema provided there are outbound translation rules i.e. what geometry/tag combinations get put in what layers in the exported file (see https://github.com/ngageoint/hootenanny/issues/208).

Q: Does Hootenanny allow users to specify work areas i.e. "Active Region" for focused conflation within AOI.

Hootenanny uses the entire layer as conflation inputs however it is possible within the UI to “clip” a region using either a defined bounding box or visible map extent to generate a new input that is a subset of the original input dataset.

Q: Describe the ability to report on change detection from pre-conflated to post-conflated datasets

Statistics are generated from a given conflation of two datasets. We have business logic to capture and report some of these changes and present those to end users but the scope would of course depend on the requirements.

Q: Describe the ability to prevent unmatched data from shifting during output of final product?

There can be very very minor shifts in the output format due to projection, but that is usually well below the 9th decimal place. See the Algorithms document for additional background on this.

Q: Describe how Hootenanny ingests, conflates and exports against the TDS schema?

Hootenanny supports the Topographic Data Store (TDS) schema (see the translations directory for versions). To ingest TDS data, simply select the appropriate TDS translation version you wish to use. Since all data is stored in the same database structure, no modifications will occur except for those that occur when merging features during conflation. Once conflated, users can select to either export to the same TDS schema or pick an alternative translation such as MGCP or OSM. Other currently unsupported schemas such as NAS can be added to Hootenanny with additional development.

Q: Does Hootenanny support the ability to allow for custom user hot keys, for production purposes?

Yes, this can be done since we use hot keys for a number of custom operations. Users would need to create their own forked version to develop from in order to modify or add new user hot keys.

Q: Is there an ability to work in the SDE environment?

We currently work in a PostgreSQL/PostGIS environment because it provides the most permissive licensing for open source distribution and meets the functional and performance needs of the software (http://www.postgresql.org/about/licence/).

Q: What kind of load time do you have in loading data into the tool? What is the process of getting data into the tool?

In general the time required to load data will depend on the file size of the source data and the resources allocated to that Hootenanny server instance (RAM, CPU, disk I/O, network speed, etc.). It is not uncommon for 50MB + osm, fgdb, shp, etc. file to take upwards of a couple of minute load time via the UI but the upload time will increase several fold with larger datasets (see Section 18.5.2 Stress Test Files in the Hoot Algorithms Guide for more background). Users can perform single file ingests (including .zip or .gdb files containing multiple feature types) or bulk file ingests via a simple interface in the User Interface. Alternatively more advanced users can run ingest scripts from command line.

Q: Please describe how attribute conflict detection is handled within Hootenanny?

We have built in conflict detection algorithms such that features with similar attributes/tags and geometry are either automatically merged or flagged for review for the user to step through and validate iteratively based on the relationship of the attributes/tags within a known schema e.g. OSM, MGCP (Multinational Geospatial Co-Production Program), etc. In the case of POI conflation for example, if two POIs fall into a distance threshold (i.e. within 500m) and share common tags such as name, amenity, etc., they are flagged for review. The user then is presented each review in a pairwise manner and can decide to retain both features as separate, edit/delete either, or merge them into a single feature. We can obviously provide more background as needed on this.

Q: Which OSM schema is the hoot schema extended from?

Its based on a subset of the public OSM database v0.6, available on openstreetmap.org (circa 2013) that has been customized with optimizations for performance that are specifically related to the way the database is accessed from the Hootenanny User Interface.

Q: How does the Hootenanny OSM database differ from the standard OSM database?

• Hootenanny stores a separate map (dataset) per database table to facilitate better performance when deleting maps.
• Hootenanny stores node coordinates with a higher precision (0.01 nanodegrees vs the standard OSM database precision of 100 nanodegrees).
• Hootenanny stores element tags in an HStore data structure which resides as a database field in each type of element table.

Q: Can Hooteannny write data to a standard OSM database?

Yes it can. Either via Rails Port of CGIMap (production) or directly to the database via SQL (testing purposes only). See the User documentation for details.

Q: How can I customize the Hootenanny OSM API database for personal use?

Hootenanny uses liquibase changesets to manage its OSM database. To customize the database, you can write your own changesets to be applied after the Hootenanny changesets are applied.

Q: What tools are available to migrate data into the hoot schema? Can we use our own tools to insert data directly?

There are command line and web service interfaces to do this for a variety of data types. The web services wrap the command line interface and may not be 100% in sync with what's available from the command line. The web user interface based on iD editor uses the web services directly, so at a minimum, anything that can be done from hoot's iD instance you could do from your own tool capable of invoking the services.

Q: Does Hootenanny use database tables to perform translations?

No, Hootenanny uses JavaScript translation files internally to translate data. The translations are stored in a "translations" folder at the root of the application.

Q: Does the Hootenanny OSM database use PostGIS?

The Hootenanny OSM database does not use PostGIS.