complexfeature parsing and building support

Motivation: Add support for ComplexFeatures. GeoTools should be able to parse a complex WFS response and construct a set of Feature objects from it.
Contact: Adam Brown
Tracker: https://osgeo-org.atlassian.net/browse/GEOT-4147
Merged pull request: https://github.com/geotools/geotools/pull/841
Tagline: complex features FTW
Location: https://github.com/Adam-Brown/geotools

Description

GeoTool's lack of support for parsing XML into complex features limits its utility as a GIS toolkit.

I propose to add this functionality by augmenting the codebase such that there are complex-compatible analogues or alternatives to all the classes necessary for WFS-based communications. Existing type hierarchies will be modifed and added to to provide this new functionality. In making these changes I will strive for maximum reuse of existing code and will follow the coding patterns and API conventions currently employed; as such, the code for performing a complex feature request will be much like the code for a simple feature request. Breaking changes will be eschewed.

Central to this work is a new XmlComplexFeatureParser which will use an AttributeBuilder and a ComplexFeatureBuilder to create objects that represent the content of WFS XML responses.

The addition of the following classes has had varying levels of impact on their encompassing type hierarchies:

WFSContentDataAccess
WFSDataAccessFactory
XmlComplexFeatureParser
ComplexFeatureBuilder

The proposed type hierachies can be seen alongside their previous forms here: ChangedTypeHierarchies.pdf (ChangedTypeHierarchies Seegrid Wiki).

Resources

GML Consumption Library Use cases

Status

Voting has started:

Tasks

✅ WFSDataAccessFactory integrated into 'Factory' hierarchy.
✅ WFSContentDataAccess integrated into 'DataAccess' hierarcy.
✅ A new GetParser<...> hierarchy created to supplant existing 'GetFeatureParser' hierarchy. XmlComplexFeatureParser integrated into this.
✅ A new FeatureBuilder<...> hierarchy created to generalise functionality from 'SimpleFeatureBuilder' so that it can be used by a new ComplexFeatureBuilder class.
✅ Create an AttributeBuilder.
✅ Create a ComplexFeatureBuilder.
✅ Complete the XmlComplexFeatureParser.
Update documentation. (in progress)

API Changes

Public API for making WFS request

BEFORE (Example of SIMPLE feature request)

// 1. Configure:
String getCapabilitiesURL = "http://services.auscope.org/geodesy/wfs?REQUEST=GetCapabilities";
Map<String, Serializable> connectionParameters = new HashMap<String, Serializable>();
connectionParameters.put("WFSDataStoreFactory:GET_CAPABILITIES_URL", getCapabilitiesURL);
connectionParameters.put("WFSDataStoreFactory:TIMEOUT", 0);

// 2. Find suitable DataStore:
DataStore dataStore = DataStoreFinder.getDataStore(connectionParameters);

// 3. Declare the typeName you're interested in (this could be done by iterating through the typeNames in dataStore.getTypeNames()):
String typeName = "ngcp:GnssStation";

// 4. Get the FeatureSource:
FeatureSource<SimpleFeatureType, SimpleFeature> featureSource = dataStore.getFeatureSource(typeName);

// 5. Get the (Simple)FeatureType (AKA schema):
SimpleFeatureType schema = dataStore.getSchema(typeName);

// 6. Create a Query
String geomName = schema.getGeometryDescriptor().getLocalName();
FilterFactory2 ff = CommonFactoryFinder.getFilterFactory2(GeoTools.getDefaultHints());
Object polygon = JTS.toGeometry(new Envelope(-30, -32, 115, 116)); 
Intersects filter = ff.intersects(ff.property(geomName), ff.literal(polygon));
// STATIONNO could have been extracted from schema.getDescriptors(), see note on 3, above.
Query query = new Query(typeName, filter, new String[] { "STATIONNO", geomName });
query.setCoordinateSystem(schema.getGeometryDescriptor().getCoordinateReferenceSystem());

// 7. Get the features and their corresponding types:
FeatureCollection<SimpleFeatureType, SimpleFeature> features = featureSource.getFeatures(query);

// 8. Iterate over and interact with the items:
FeatureIterator<SimpleFeature> iterator = features.features();
try {
    while (iterator.hasNext()) {
        Feature feature = (Feature) iterator.next();
        
        for (Property property : feature.getProperties()) {
            System.out.println(property);
        }
    }

} finally {
    iterator.close();
}

AFTER (Example of COMPLEX feature request)

// 1. Configure:
String getCapabilitiesURL = "http://nvclwebservices.vm.csiro.au/geoserverBH/wfs?REQUEST=GetCapabilities";
Map<String, Serializable> connectionParameters = new HashMap<String, Serializable>();
connectionParameters.put("WFSDataStoreFactory:WFS_GET_CAPABILITIES_URL", getCapabilitiesURL);
connectionParameters.put("WFSDataStoreFactory:TIMEOUT", 0);
connectionParameters.put("WFSDataStoreFactory:PROTOCOL", false);
/** #A: Require a Factory with this level of compliance. **/
connectionParameters.put("WFSDataStoreFactory:GML_COMPLIANCE_LEVEL", 2);
connectionParameters.put("WFSDataStoreFactory:MAXFEATURES", 2);
/** #B: Specify the location of the folder to be used by app-schema-resolver. **/
connectionParameters.put("WFSDataStoreFactory:SCHEMA_CACHE_LOCATION", "C:/Adams/GitHub_GeoTools_GeoTools/schema_cache");

// 2. Find suitable DataAccess:
/** #C: Notice that these classes are the non-Simple forms. **/
DataAccess<FeatureType, Feature> dataAccess = DataAccessFinder.getDataStore(connectionParameters);

// 3. Declare the type you're interested in (this could be done by iterating through the typeNames in dataStore.getTypeNames()).
Name nameToRetrieve = new NameImpl("urn:cgi:xmlns:CGI:GeoSciML:2.0", ":", "Borehole");

// 4. Get the FeatureSource (WFSContentComplexFeatureSource):
FeatureSource<FeatureType, Feature> featureSource = dataAccess.getFeatureSource(nameToRetrieve);

// 5. Get the FeatureType (AKA schema):
FeatureType schema = dataAccess.getSchema(nameToRetrieve);

// 6. Create Query using the schema.
Query query = new Query("gsml:Borehole");
query.setCoordinateSystem(schema.getGeometryDescriptor().getCoordinateReferenceSystem());

// 7. Get the features and their corresponding types:
FeatureCollection<FeatureType, Feature> features = featureSource.getFeatures(query);

// 8. Iterate over the features and display them:
FeatureIterator<Feature> iterator = features.features();

try {
    while (iterator.hasNext()) {
        Feature feature = iterator.next();
        for (Property property : feature.getProperties()) {
            System.out.println(property);
        }
    }
} finally {
    iterator.close();
}

Take note of the comments enclosed by double asterix, i.e; /** comment **/. They point out some important differences between the two code samples. Each one has letter identifier preceded by a hash. I will hearafter refer back to these identifiers to elaborate more on the code they annotate:

#A: The 'getDataStore' method iterates through a collection of factories until it finds one that meets the requirements of the connectionParameters passed in. I added a new optional GML_COMPLIANCE_LEVEL parameter to be passed through to impose a restriction on the type of DataAccess object that can be passed back. I.e. one that supports complex features.
#B: When dealing with complex features we can utilise the app-schema-cache to cache schemata.
#C: I created WFSDataAccessFactory which conforms to GML_COMPLIANCE_LEVEL 2. It creates and returns a WFSContentDataAccess object. WFSContentDataAccess borrows extensively from WFSContentDataStore with notable exceptions being its lack of reliance on simple types in methods such as getSchema, updateSchema, etc.

Comparison and Contrast between SimpleFeatureBuilder, ComplexFeatureBuilder usage

The builders are used by their corresponding XML feature parsers to associate attributes with names.

SimpleFeatureBuilder

SimpleFeatureBuilder

this.builder = new SimpleFeatureBuilder(this.targetType);

// Repeat for each attribute
builder.set(descriptor.getLocalName(), attributeValue);

// Once you've added all the attributes you build the feature itself
SimpleFeature feature = builder.buildFeature(fid);

The attributeValue, above, is created by a static Converters.convert(...) method which takes a raw text value and a binding (Java Class object).

ComplexFeatureBuilder

SimpleFeatureBuilder

this.builder = new ComplexFeatureBuilder(this.targetType);

// Repeat for each property
builder.append(descriptor.getLocalName(), propertyValue);

// Once the properties are added you build the complex feature itself
Feature feature = builder.buildFeature(fid);

In the case of ComplexFeatures the propertyValue, above, is constructed with an AttributeBuilder and bound to a Java Class object.

Excerpt from ComplexFeatureBuilderTests

Excerpt from ComplexFeatureBuilderTests

@Test
public void test_append_validNameInvalidValueClass_throwsIllegalArgumentException() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    
    // Act
    try {
        builder.append(LOCATION, londonBridge); // Passing in londonBridge instead of a location.
        fail("Expected IllegalArgumentException but it wasn't thrown.");
    }
    catch (IllegalArgumentException iae) {
        String expectedMessage = "The value provided contains an object of 'class java.lang.String' but the method expects an object of 'class com.vividsolutions.jts.geom.Geometry'.";
        if (iae.getMessage().compareTo(expectedMessage) != 0) {
            fail("Expected IllegalArgumentExceptionMessage to say: '" + expectedMessage + "' but got: '" + iae.getMessage() + "'");
        }
        
        // Assert (This is the expected exception).
        return; 
    }
    catch (Exception e) {
        fail("Expected IllegalArgumentException but it wasn't thrown; got " + e.getClass() + " instead. " + e.getMessage());
    }
}
@Test
public void test_append_validNameButNullValue_throwsIllegalArgumentException() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    // Act
    try {
        builder.append(BRIDGE_NAME, null); // Passing a null reference for a non-nillable type.
        fail("Expected IllegalArgumentException but it wasn't thrown.");
    }
    catch (IllegalArgumentException iae) {
        String expectedMessage = "The value provided is a null reference but the property descriptor 'AttributeDescriptorImpl urn:Bridge:Test:1.1:bridgeName <string:String> 0:1' is non-nillable.";
        if (iae.getMessage().compareTo(expectedMessage) != 0) {
            fail("Expected IllegalArgumentExceptionMessage to say: '" + expectedMessage + "' but got: '" + iae.getMessage() + "'");
        }
        // Assert (This is the expected exception).
        return; 
    }
    catch (Exception e) {
        fail("Expected IllegalArgumentException but it wasn't thrown; got " + e.getClass() + " instead. " + e.getMessage());
    }
}
@Test
public void test_append_validNameValidValue_valueShouldBeAddedToTheMap() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    // Act
    builder.append(BRIDGE_NAME, londonBridge);
    Object actualValue = builder.values.get(BRIDGE_NAME).get(0);
    // Assert    
    Assert.assertSame(londonBridge, actualValue);
}
@Test
public void test_append_exceedMaxOccursLimit_throwsIndexOutOfBoundsException() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    builder.append(BRIDGE_NAME, londonBridge);
    // Act
    try {
        builder.append(BRIDGE_NAME, londonBridge); // Add it once too many times.
        fail("Expected IndexOutOfBoundsException but it wasn't thrown.");
    }
    catch (IndexOutOfBoundsException iae) {
        String expectedMessage = "You can't add another object with the name of 'urn:Bridge:Test:1.1:bridgeName' because you already have the maximum number (1) allowed by the property descriptor.";
        if (iae.getMessage().compareTo(expectedMessage) != 0) {
            fail("Expected IndexOutOfBoundsException to say: '" + expectedMessage + "' but got: '" + iae.getMessage() + "'");
        }
        // Assert (This is the expected exception).
        return; 
    }
    catch (Exception e) {
        fail("Expected IndexOutOfBoundsException but it wasn't thrown; got " + e.getClass() + " instead. " + e.getMessage());
    }
}
@Test
public void test_buildFeature_validInput_buildsFeature() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    AttributeImpl description = new AttributeImpl("description", DESCRIPTION_DESCRIPTOR, null);
    builder.append(BRIDGE_NAME, londonBridge);
    builder.append(LOCATION, location);
    builder.append(DESCRIPTION, description);
    // Act
    Feature feature = builder.buildFeature("id");
    // Assert
    assertNotNull(feature);
    assertSame(londonBridge, feature.getProperty(BRIDGE_NAME));
    assertSame(location, feature.getProperty(LOCATION));
    assertSame(description, feature.getProperty(DESCRIPTION));
}
@Test
public void test_buildFeature_missingDescription_descriptionDefaultsToNull() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    builder.append(BRIDGE_NAME, londonBridge);
    builder.append(LOCATION, location);
    // Act
    Feature feature = builder.buildFeature("id");
    // Assert
    assertNull(feature.getProperty(DESCRIPTION).getValue());
}
@Test
public void test_buildFeature_noLocationSet_throwsIllegalStateException() {
    // Arrange
    ComplexFeatureBuilder builder = new ComplexFeatureBuilder(BRIDGE_TYPE);
    // Deliberately not setting location
    builder.append(BRIDGE_NAME, londonBridge);
    // Act
    try {
        Feature feature = builder.buildFeature("id");
        fail("expected an exception");
    }
    catch (IllegalStateException ise) {
        String expectedMessage = "Failed to build feature 'urn:Bridge:Test:1.1:Bridge'; its property 'urn:Bridge:Test:1.1:location' requires at least 1 occurrence(s) but number of occurrences was 0.";
        if (ise.getMessage().compareTo(expectedMessage) != 0) {
            fail("Expected IllegalStateException to say: '" + expectedMessage + "' but got: '" + ise.getMessage() + "'");
        }
        // Assert (This is the expected exception).
        return;
    }
    catch (Exception e) {
        fail("Expected IllegalStateException but it wasn't thrown; got " + e.getClass() + " instead. " + e.getMessage());
    }
}

Documentation Changes

New documentation will be required to cover the steps involved in processing a complex WFS response.
http://docs.geotools.org/latest/userguide/library/data/wfs.html
- Add details of new connection parameter 'GML_COMPLIANCE_LEVEL'.
- Add a new example of a WFS request showing the retrieval and consumption of a complex version.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly