[IDEA] Chapter 17: More complete solution to Exercise 10

[vaharoni]

Exercise 10 of Chapter 17 seems very interesting and relevant. Unfortunately, I have been struggling with it for quite some time. I pretrained around 15 different autoencoders on 40K+ CIFAR-10 samples using a bunch of tricks from chapter 14 (while keeping them undercomplete), and many of them visually seem to properly reconstruct the images. However, when transferring the encoder's layers and weights, I was unable to materially outperform a rather naive convolutional network trained only on 500 sample without the encoder, and I utilized quite a few tricks from throughout the book.

As such, I would have loved to see a full solution to the exercise. Currently, the available solution only shows how to perform denoising, and skips the transfer learning part (as well as the part about visualizing "images that most activate each neuron in the coding layer" - I assume for testing that the encoder doesn't only enable reconstruction, but also learns useful latent representations which sounds very useful).

Thank you 🙏