Skip to content

Latest commit

 

History

History
85 lines (58 loc) · 7.29 KB

README.md

File metadata and controls

85 lines (58 loc) · 7.29 KB

Optical Flow and Deep Learning

A set of projects that illustrate different approaches to Optical Flow.

Projects

Optical Flow explores three techniques to tackle the tracking problem:

  1. Feature_Tracking.ipynb illustrates how to detect and track features across consecutive images.

  2. Sparse_Optical_Flow.ipynb illustrates how to use sparse optical flow on images and videos.

  3. Dense_Optical_Flow.ipynb illustrates how to use dense optical flow on images and videos.

FlowNet illustrates Deep Learning for Optical Flow by implementing the FlowNet algorithm using PyTorch and training the models on the KITTI dataset. The goal is to output the optical flow of two images.

RAFT explores the RAFT deep network architecture for optical flow.

Here is the same video of the skateboarder as used above to illustrate sparse and dense optical flow, this time using RAFT:

Visual SLAM shows an example of Visual SLAM (Simultaneous Localization and Mapping) using visual features.

Datasets

The KITTI Vision Benchmark Suite

The stereo 2015 / flow 2015 / scene flow 2015 benchmark consists of 200 training scenes and 200 test scenes (4 color images per scene, saved in lossless png format).

Geiger, A, P. Lenz, R. URtasun, 2015. Optical Flow Evaluation 2015. The KITTI Vision Benchmark Suite: A project of Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago.

HD1K Benchmark Suite

The HD1K Benchmark Suite is an autonomous driving dataset and benchmark for optical flow. The public training dataset contains:

  • More than 1000 frames at 2560x1080 with diverse lighting and weather scenarios
  • reference data with error bars for optical flow
  • evaluation masks for dynamic objects
  • specific robustness evaluation on challenging scenes

The "Flying Chairs" Dataset

The "Flying Chairs" are a synthetic dataset with optical flow ground truth. It consists of 22872 image pairs and corresponding flow fields. Images show renderings of 3D chair models moving in front of random backgrounds from Flickr. Motions of both the chairs and the background are purely planar.

Scene Flow Datasets: FlyingThings3D, Driving, Monkaa

The Scene Flow Datasets collection contains more than 39000 stereo frames in 960x540 pixel resolution, rendered from various synthetic sequences. Mayer, et al. (2016) reference and describe these datasets.

MPI Sintel Flow Dataset

The MPI Sintel Flow Dataset is a data set for the evaluation of optical flow derived from the open source 3D animated short film, Sintel.

References

Papers

Courses

Software Implementations