Shoreline detection using a modified Hollistic Edge Detection (HED) Convolutional Neural Network (CNN) approach. The algorithm identifies from images the pixels most likely to represent the shoreline, at the intersection between the water and sand.
Written by Joshua Simmons using PyTorch.
Note: while this model shows promise, it is still under development to improve performance. As can be seen below, while it shows excellent performance in some images, it is not currently robust enough for use in production. The Shoreline_detection_predict_unseen.ipynb notebook can be run on provided images as an example.
Figure 1 - CNN identified shoreline 'edges' overlaid in magenta
Shoreline detection methods are well established in the field of coastal imaging and seek to divide wet from dry pixels (Harley et al., 2019). Existing methods have tended used pixel hue and intensity (Plant and Holman, 1997; Aarninkhof et al., 2003), RGB colour channels (e.g., Turner et al., 2004; Plant et al, 2007), and even machine learning techniques which seek to classify on a pixel-by-pixel basis (Hoonhout et al, 2015). Many applications rely on the well validated threshold techniques based on the distribution of the RGB channels with excellent results (e.g., Harley et al., 2019)
However, when undertaking automated processing over years of image data, there are many situations that can present challenges to these types of algorithms. Figures 2 and 3 show this algorithm failing due to glare and shadow effects. Even small improvements in accuracy can save engineers large amounts of time when checking of shoreline data.
Figure 2 - Failure of an RGB based shoreline detection algorithm due to glare
Figure 3 - Failure of an RGB based shoreline detection algorithm due to shadows
The CNN developed here is used to identify those pixels most likely to be shoreline pixels with the aim of identifying an accurate region of interest. Once this is obtained, established and validated shoreline detection algorithms can search within this narrow band to find the precise shoreline. This repository seeks to test the usefulness of such an approach.
This code adapts the HED CNN architecture developed by Xie and Tu (2015) and is adapted from a PyTorch implementation by buntyke. The HED architecture outputs "side response" activation maps at various levels through the CNN, as shown in Figure 4. These are combined and thresholded to produce the final output as shown in Figure 5. OpenCV is used for some simple noise reduction on the final output.
Figure 4 - CNN activations maps from the side outputs with increasing deepness
Figure 5 - CNN model output combining information from the side outputs and after noise reduction
- The root folder contains the jupyter notebook files detailed below
- ./figures contains the printed outputs of the model
- ./functions contains the functions called by the jupyter notebooks implementing the HED CNN
- ./models contains pretrained model weights for the shoreline detection CNN
- ./test_sites contains some unseen trial images on which the model can be tested
The CNN model provided was trained on images from six sites monitored as a part of the Water Research Laboratory (UNSW Syney) coastal imaging program. Training images used were time-averaged and oblique. Checked shorelines form an existing database were used to train the model and were provided as vectors of U (width), V (height) pixel coordinates.
The file ./functions/data_preprocessing.py provides functions which load the images (X data) and shorelines (Y data), pre-process them to reduce the size to a manageable 320x480 and augment them if required.
Shoreline_detection_create_data.ipynb notebook was used to create the data for training.
A weighted binary cross-entropy loss was used as the ratio of shoreline pixels to image size was small. A regularised implementation of this loss is also available which penalises the total number of pixels selected in an attempt to sharpen the edge detected.
Shoreline_detection_create_data.ipynb notebook was used to undertake model training.
A demonstration of the use of the trained shoreline detection network can be run using the example notebook Shoreline_detection_predict_unseen.ipynb. The CNN is used to predict the shoreline across a range of unseen images images. Some images are taken from the Narrabeen coastal imaging station (one of the beaches used to train the model). Other images are from different beaches provided by coastsnap and include both time-averaged and still images at multiple sites.
Anaconda environment with package requirements provided as environment.yml. Tensorflow should be done separately depending on your GPU configuration.
- Improved training:
- Training could be improved with more images (current training was limited by available GPU)
- Checkpoint models need to be examined to check training is progressing as intended and the model solution is not getting stuck.
- Broaden the band around the training data shoreline so that the algorithm has an easier time converging on a good solution.
- Post-processing:
- The model output can be easily post-processed to remove small misclassified pixels.
- This could be done with an adaptive thresholding and openCV blob tools (as we know we are looking for long continous and smooth shoreline edges)
- Repository
- move the large files to a data host and implement a call to download these files within the code
This software is provided "as is", without warranty of any kind. If you encounter a problem/bug or have a question or suggestion, please use the "issues" tab on github. Otherwise, this software is unsupported.