This project is part of the lecture on Bayesian Machine Learning tought by Rémi BARDENET. The idea is to pick a paper from a list of given papers, and read it with a critical mind. For instance, we will:
- (1) explain the contents of the paper
- (2) emphasize the strong and weak points of the paper
- (3) apply it to real data of our choice when applicable.
This repository contains our Python implementation for this project, The notebook file (notebook.iypnb) contains all the code and our comments. But you can also check the Python files.
This project is done by Zakaria Boulkhir & me. For more insights check out our report.
M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 681–688, 2011.
@inproceedings{welling2011bayesian,
title={Bayesian learning via stochastic gradient Langevin dynamics},
author={Welling, Max and Teh, Yee W},
booktitle={Proceedings of the 28th international conference on machine learning (ICML-11)},
pages={681--688},
year={2011},
organization={Citeseer}
}
During this project, we worked with both artificial and real data. For the Mixture of Gaussians we will generate a number of samples we will be working with. For the Logistic Regression experiment, we will use the a9a dataset from the UCI adult dataset. Finally for the ICA experiments, we will use both a synthetic and real data.
To show that our method works well, we start by applying it on a very basic and simple example with two parameters. This first example is the mixture of Gaussians.
python mixture_of_gaussians.py
For this second example, we apply stochastic gradient Langevin algorithm to a Bayesian logistic regression model. We will be using the data from the UCI dataset, more specificaly, the a9a dataset which consists of 32561 observations and 123 features.
python logistic_regression.py
The last experiment concerns ICA algorithm based on stochastic (natural) gradient optimization (Amari et al., 1996).
To try our implementation in your own machine you need to install the following requirements:
pip install -r requirements.txt