Disentangled Representation Learning

Disentangled representations learning is an unsupervised learning technique that disentangles each feature into narrowly defined variables and encodes them as separated dimensions. Its goal is to mimic the quick intuition process of a human, using both “high” and “low” dimension reasoning. This is a collection of milestone papers on disentangled representation learning since 2016.

• [2016] InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets [paper]
• [NIPS2016] Disentangling factors of variation in deep representations using adversarial training [paper]
• [ICLR2017] Deep Variational Information Bottleneck [paper]
• [ICLR2017] Beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework [paper] [code]
• [NIPS2017] The Beta-VAE’s Implicit Prior [paper]
• [NIPS2017] Understanding Disentangling in Beta-VAE [paper]
• [NIPS2017] Disentangling the independently controllable factors of variation by interacting with the world [paper]
• [NIPS2017] Disentangled Representations for Manipulation of Sentiment in Text [paper]
• [NIPS2017] Quantifying the Effects of Enforcing Disentanglement on Variational Autoencoders [paper]
• [NIPS2017] Semantically Decomposing the Latent Spaces of Adversarial Generative Network [paper]
• [NIPS2017] Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data [paper] [code]
• [NIPS2017] Learning Disentangled Representations with Semi-Supervised Deep Generative Models [paper]
• [Workshop on NIPS2018] Recent Advances in Autoencoder-Based Representation Learning [paper]
• [ICML2018] Disentangling by Factorising [paper]
• [NIPS2018] Isolating Sources of Disentanglement in VAEs [paper] [code]
• [ICML2018] Fixing a Broken ELBO [paper]
• [2018] Auto-Encoding Total Correlation Explanation [paper]
• [2018] On the Latent Space of Wasserstein Auto-Encoders [paper]
• [Workshop on ICLR2018] Learning Disentangled Representations with Wasserstein Auto-encoders [paper]
• [ICLR2018] A Framework for the Quantitative Evaluation of Disentangled Representations [paper]
• [Workshop on ICLR2018] Rethinking Style and Content Disentanglement in Variational Auto-encoders [paper]
• [ICML2018] Disentangled Sequential Autoencoder [paper]
• [ICML2018] Learning Independent Causal Mechanisms [paper]
• [ICLR2019] Learning deep representations by mutual information estimation and maximization [paper] [code]
• [ICML2019] Disentangling Disentanglement in Variational Autoencoders [paper]
• [AISTATS2019] Structured Disentangled Representations [paper]
• [2019] Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations [paper]

Variational Neural Models

• [ICLR2014] Auto-Encoding Variational Bayes [paper]
• [ICLR2014] Stochastic Backpropagation and Approximate Inference [paper]
• [ICLR2015] Importance Weighted Autoencoders [paper]
• [Workshop on ICLR2015] Variational Recurrent Auto-Encoders [paper]
• [NIPS2015] A Recurrent Latent Variable Model for Sequential Data [paper] [code]
• [ICML2015] Learning to Generate with Memory [paper]
• [ICLR2016] Generating Sentences from a Continuous Space [paper]
• [ICLR2016] Hierarchical Variational Models [paper]
• [NIPS2016] Ladder Variational Autoencoders [paper]
• [NIPS2016] Improved Variational Autoencoders with Inverse Autoregressive Flow [paper]
• [Workshop on ICLR2017] Re-interpreting Importance Weighted Autoencoders [paper]
• [ICML2017] Improved Variational Autoencoders for Text Modeling using Dilated Convolutions [paper]
• [AAAI2017] Variational Autoencoder for Semi-Supervised Text Classification [paper]
• [ICLR2017] Variational Lossy Autoencoder [paper]
• [ICLR2017] Beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework [paper] [code]
• [ICLR2017] On the Quantitative Analysis of Decoder-based Generative Models [paper]
• [NIPS2017] InfoVAE: Information Maximizing Variational Autoencoders [paper]
• [ICML2018] Tighter Variational Bounds are Not Necessarily Better [paper]
• [NAACL2019] Stochastic Wasserstein Autoencoder for Probabilistic Sentence Generation [paper] [code]

Neural Dialogue Generation

• [Workshop in ICML2015] A Neural Conversational Model [paper]
• [2016] A Persona-Based Neural Conversation Model [paper]
• [2016] A Diversity-Promoting Objective Function for Neural Conversation Models [paper]
• [2016] How NOT To Evaluate Your Dialogue System: An Empirical Study ofUnsupervised Evaluation Metrics for Dialogue Response Generation [paper]
• [2016] A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues [paper]
• [2016] Deep Reinforcement Learning for Dialogue Generation [paper]
• [2016] Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models [paper]
• [ACL2016] A Persona-Based Neural Conversation Model [paper]
• [ACL2017] A Conditional Variational Framework for Dialog Generation [paper]
• [IJCAI2017] Exploring Personalized Neural Conversational Models [paper]
• [SIGIR2017] Personalized Response Generation via Domain adaptation [paper]
• [AAAI2017] Topic Aware Neural Response Generation [paper]
• [ACL2018] Personalizing Dialogue Agents: I have a dog, do you have pets too? [paper]
• [ACL2018] Generating Informative Responses with Controlled Sentence Function [paper]
• [ACL2018] Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation [paper] [code]
• [ICLR2018] Towards Interpretable Chit-Chat: Open Domain Dialogue Generation With Dialogue Acts [paper]
• [AAAI2018] Improving Variational Encoder-Decoders in Dialogue Generation [paper]
• [AAAI2018] Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory [paper]
• [SIGIR2018] Chat More: Deepening and Widening the Chatting Topic via A Deep Model [paper]
• [WWW2018] Hierarchical Variational Memory Network for Dialogue Generation [paper]
• [NAACL2018] Automatic Dialogue Generation with Expressed Emotions [paper]
• [ICLR2019] DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder [paper]
• [Workshop on ACL2019] Augmenting Neural Response Generation with Context-Aware Topical Attention [paper]

Techniques for KL Vanishing

• [ICLR2016] Generating Sentences from a Continuous Space [paper]
• [2016] Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders [paper]
• [AAAI2018] Improving Variational Encoder-Decoders in Dialogue Generation [paper]
• [NIPS2016] Improved Variational Autoencoders with Inverse Autoregressive Flow [paper]
• [NAACL2019] Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing [paper] [code]