adaptative-dropout-pytorch

Pytorch implementation of Adaptive Dropout a.k.a Standout.

• Unfortunately I wasn't able to achieve the results reported in the paper, Regular dropout always get aproximately 98.70, while the standout version I was able to achieve only 98.51 after tweaking a lot the knoobs, i didn't make grid search was all empirical and cloning the paper parameters, furthermore I did not make any unsupervized pretraining, maybe this tecnique is very sensitive to this step! Hope someone can achieve the reporteded results from this code.

References:

-Papers: https://papers.nips.cc/paper/5032-adaptive-dropout-for-training-deep-neural-networks.pdf

-Code:

https://github.com/gngdb/adaptive-standout

https://github.com/pytorch/examples