Skip to content

Latest commit

 

History

History
142 lines (95 loc) · 5.34 KB

README.md

File metadata and controls

142 lines (95 loc) · 5.34 KB

Unsupervised person localization in wilderness search and rescue

Table of contents

Introduction

This is part of the exercise class "UE Computer Vision, Oliver Bimber / Indrajit Kurmi, 2021W" at the JKU Austria.

In this lab project, we had to implement an unsupervised person localization algorithm.

Challenge

challenge.png

Data extraction

data_method.png

Methods and solution

0. Image pre-processing

img.png

img_1.png

1. Color channels approach

Method

img_2.png

img_3.png

img.png

img_1.png

  • OpenCV functions cv2.findContours() and cv2.boundingRect()
  • Pad by 2 pixels in each direction
  • Merge overlapping bounding boxes
  • Pad by 7 pixels in x and 4 pixels in y
  • Choose the biggest bounding box of blue and red image, respectively
  • Remove if x<24 or y<18
  • If no detections -> lower threshold for binary image and start from 1
  • Merge detections overlapping between the two images
  • Pad detections to a minimum size of 38x30 px

Advantages and disadvantages

  • Advantage:
    • can distinguish people from other objects by detecting movement
  • Disadvantages:
    • bias towards detecting people wearing blue or red – problems finding people with green clothing
    • cannot detect people that are not moving or moving too little

2. Autoencoder approach

The autoeconder is implemented in anomaly_detection_autoencoder_SAR_JKU.ipynb

Initial idea

Going through various research papers on anomaly detection, we decided to try out an Autoencoder approach for this task

  • Autoencoder -> encoder-decoder system to reconstruct the input as the output.

  • Train a convolutional autoencoder so that it will reconstruct an image from the normal data with a smaller reconstruction error, but reconstruct an image from the anomaly data with a larger reconstruction error

  • Our solution decides if an image is from the normal data or from the anomaly data based on a threshold of the reconstruction error.

  • the model is encouraged to learn to precisely reproduce the most frequently observed characteristics

  • when facing anomalies, the model should worsen its reconstruction performance.

  • after training, the autoencoder will accurately reconstruct normal data, while failing to do so with unfamiliar anomalous data

  • reconstruction error (the error between the original data and its low dimensional reconstruction) is used as an anomaly score to detect anomalies

  • we are aware that autoencoding models can, be very good at reconstructing anomalous examples and consequently not able to reliably perform anomaly detection

Model Architecture

img_4.png

  • Base is a Convolutional autoencoder for image denoising from official Keras docs
  • Adapted loss for Structural Similarity Index (SSIM)
  • Decided for that because
    • Relatively straight forward to tune
    • Simple architecture
    • Sufficient for our image detection problem

Implementation and findings

Over the course of the implementation it became apparent, that

  • properly pre-processed images improve the performance of the autoencoder a lot
  • a deep convolutional autoencoder is sufficient to reproduce the images properly
  • the autoencoder should be trained with color images as the color provides most of the information for the task
  • the biggest challenge is the length of training as
    • too short training shows too many reconstruction errors
    • too long training reconstructs anomalies
  • as well as the threshold for finding the most useful SSIM differences

img_5.png Reconstruction worked well

img_6.png

Visualization of activation layers over RBG channels, showing stronger activations for red and blue channel

img_8.png

Indication of finding the anomalies as desired.

img_9.png

Finding the proper threshold for SSIM differences

img.png


The project was implemented over the course of a semester at university.

In the end we implemented the whole pipeline to fit the corresponding grading criteria.