This repository provides a Colab Notebook that shows how to use Spatial Transformer Networks (STN) inside CNNs build in Keras. The Colab Notebook has been obtained from forking Spatial-Transformer-Networks-with-Keras.
I have used utility functions mostly from this repository to demonstrate an end-to-end example. As such please install stn from pypi:
pip3 install stn
STNs allow a (vision) network to learn the optimal spatial transformations for maximizing its performance. In other words, we can expect when STNs are incorporated inside a network, it would learn how much to rotate or crop (or any affine transformations) the given input images so as to make itself more invariant to these changes.
Here's a demonstration:
Demo.mov
Notice how the STN module is able to figure out transformations for the dataset that may be helpful to boost its performance. Here are the original images for reference:
This repository has also been updated to use a STN as a Keras Layer.