Skip to content

1) Image Segmentation

Jump to bottom
Sophie Crommelinck edited this page Oct 23, 2019 · 9 revisions

Description:

Image segmentation groups similar pixels into segments. The segmentation delivers closed contours capturing the outlines of visible objects (Figure 1). In this wiki, the use of Multiscale Combinatorial Grouping (MCG) is described. MCG is an open source method developed by Berkley University, which we have modified for georeferenced remote sensing imagery. The source code is pre-complied to be used in Matlab under Linux.

Figure 1: Image segmentation delivers closed contours (right) capturing the outlines of visible objects in a remote sensing image (left).

After applying MGC image segmentation, we convert the MCG raster to vector lines (Figure 2) and apply line filtering (Figure 3). These steps can be conducted in any GIS such as the free and open source QGIS or the commercial ESRI ArcGIS.

Figure 2: Remote sensing image (left). MCG image segmentation result in raster format (middle). MCG lines derived from raster to vector conversion (right).

Figure 3: MCG line filtering reducing the line count by 80%.

Input:

  • RGB orthoimage raster .tif
  • World file .tfw

Output:

  • image segmentation lines without attributes .shp

Steps:

  1. Install Matlab & MCG image segmentation
  2. Prepare input orthoimage
  3. Run MCG
  4. Raster to vector conversion
  5. Line filtering (optional)

1. Install Matlab & MCG image segmentation

2. Prepare input orthoimage

Depending on the memory capacity of your computer, you might have to clip your RGB orthoimage raster (.tif) into tiles that do not exceed 10,000 x 10,000 pixels. The tiles must be quadratic.

  • Clip orthoimage into quadratic tiles of max. 10,000 x 10,000 pixels

    • Open your orthoimage in QGIS

    • Right click your orthoimage | Export | Save As... | Enable Create VRT | Define a file name with ... | Set VRT Tiles max columns and max rows to 10000 or lower | Click OK

      • Alternatively, you can clip your orthoimage in the OSGeo4W shell with gdalwarp -crop_to_cutline -cutline clip.shp orthoimage.tif orthoimage_clipped.tif

  • Create a world file for each tiled orthoimage

    • Open OSGeo4W shell
      • Select the Windows start button > type OSGeo4W
      • Change the shells's working directory to your tiled orthoimages
        • cd path_to_folder

      • Create a .tfw file describing the georeference of your orthoimage. The orthoimage will not be changed.
        • gdal_translate -co "TFW=YES" orthoimage.tif orthoimage_translated.tif

3. Run MCG

  • Save your RGB orthoimage raster (.tif) and the corresponding world file (.tfw) both having the same name to \MCG\pre-trained\demos\data

  • Open Matlab

  • Add the path of the downloaded MCG folder in Matlab

    • Right click on the MCG folder | Add to Path | Selected Folders and Subfolders

  • Change the EPSG code in \MCG\pre-trained\scripts\geotiffwrapper.m to that of your orthoimage

    • A.ProjectedCSTypeGeoKey = your EPSG code

    • You can check the EPSG code of your orthoimage by opening it in QGIS and reading the EPSG code from the bottom right corner

      or by right clicking your orthoimage | Properties | Information

  • Set all parameters under %%Predefine variables%% in \MCG\pre-trained\demos\mcg.m

    • k: value between 0 and 1 that regulates over- and under-segmentations.

      • Low values produce more boundaries and thus more over-segmentation. For our test data of 0.25 m GSD, we set k = 0.1 to produce over-segmentation. This setting creates outlines around the majority of visible objects. Tests with higher values (k = 0.3 and k = 0.5) resulting in less over-segmentation show that visible object outlines are partly missed, while irrelevant lines around small objects are still produced. For high-resolution imagery of 0.05 cm GSD, we set k = 0.3 or k = 0.4.

    • ext: appendix added to output boundary map files. You can chose any text to later identify your results.

    • myDir: path to input data directory

    • outDir: path to store output data

  • Run \MCG\pre-trained\demos\mcg.m

    • Right click on mcg.m | run

    • MCG will be applied to each orthoimage stored in myDir

  • Merge MCG raster tiles (optional)

    • Note: only required in case you tiled your orthoimage before applying MCG
    • Open Search (Ctrl + F) and search for Mosaic To New Raster
    • Set the MCG raster tiles as Input Rasters | Define an Output Location | Define Raster Dataset Name with Extension | Set Number of Bands to 1

4. Raster to vector conversion

The result of mcg.m are binary rasters, in which the value 1 is assigned to boundary pixels and 0 to not boundary pixels. These rasters need to converted to a vector line format ([Figure 2](Figure 2)) and ([Figure 4](Figure 4)).

Figure 4: Raster to vector conversion, which converts the binary MCG boundary map (black and white pixels) to a vector line layer (red lines).

Convert raster to vector

  • Open ArcGIS

  • Open Search (Ctrl + F) and search for Raster to Polyline

  • Run Raster to Polyline on the MCG raster, changing only the Input raster variable

5. Line filtering (optional)

To reduce the number of irrelevant lines produced through over-segmentation, lines can be simplified through filtering: lines around areas smaller than 30 m2 are merged to the neighboring segments, which reduces the line count by roughly 80%. According to our visual inspections, this post-processing removes artefacts in the segmentation results and keeps outlines of large objects being more relevant for cadastral mapping.

Simplify segment lines

  • Download the ArcGIS toolbox from Github that contains our tool SimplifySegmentation
    • SimplifySegmentation is a tool created with the Model Builder that combines different ArcGIS toolbox functionalities:

  • Add the downloaded toolbox in ArcGIS
    • right click in ArcToolbox window | Add Toolbox...

    • Load the downloaded toolbox

  • Open the tool SimplifySegmentation, which should now appear as a new toolbox in your ArcToolbox

  • Run SimplifySegmentation

    • Select lines to be simplified as Input Lines

    • Set path to store simplified lines as Output Lines

Open source alternatives (optional)

The steps described in ArcGIS can also be performed in free GIS software such as QGIS or the OSGeo4W.

Merge rasters

  • Download the Python gdal_merge script to your gdal libraries directory
  • Replace the following parameters output.tif, tile1.tif, tile2.tif with your file names and run the following in the OSGeo4W Shell
import subprocess
merge_command = ["python", "gdal_merge.py", "-o", "output.tif", "tile1.tif", "tile2.tif", "tile3.tif"]
subprocess.call(merge_command)

Convert raster to vector

  • Refer to instructions on Stack Exchange, in which we propose to apply the following steps:
    1. Replace all no data values
    processing.runalg('grass:r.mapcalculator',
                {"amap": gPb_rlayer,
                 "formula": "if(A>0, 1, null())",
                 "GRASS_REGION_PARAMETER": "%f,%f,%f,%f" % (xmin, xmax, ymin, ymax),
                 "GRASS_REGION_CELLSIZE_PARAMETER": 1,
                 "outfile": mapcalc})
    1. Thin raster layer to thin non-null cells
    processing.runalg('grass7:r.thin',
                {"input": mapcalc,
                 "GRASS_REGION_PARAMETER": "%f,%f,%f,%f" % (xmin, xmax, ymin, ymax),
                 "output": thinned})
    1. Raster to vector conversion
    processing.runalg('grass7:r.to.vect',
                {"input": thinned,
                 "type": 0,
                 "GRASS_OUTPUT_TYPE_PARAMETER": 2,
                 "GRASS_REGION_PARAMETER": "%f,%f,%f,%f" % (xmin, xmax, ymin, ymax),
                 "output": centerlines})

Simplify segment lines

  • We created the same SimplifySegmentation tool with the Processing Modeler in QGIS that we created before in ArcGIS. Download the QGIS toolbox from Github. However, we observed that it runs less stable for larger data sets.

General notes:

  • In comparison to the , we have modified the following scripts:
    • \MCG\pre-trained\demos\mcg.m -> to call MCG image segmentation
    • \MCG\pre-trained\scripts\geotiffwrapper.m -> to define EPSG code in input raster data
    • \MCG\pre-trained\scripts\geotiffwrite.m -> to write EPSG code from input to output raster

References:

  • The original MGC source code developed by Berkley University that we have modified for georeferenced remote sensing imagery is publically available under the GNU General Public License

Clone this wiki locally