Machine Learning Models and Methods for Econometricians

by Mattias Villani, Division of Statistics and Machine Learning, Linköping University

---

09.00-09.15 Coffee/Tea

---

09.15-10.00 Gaussian Process Regression

Reading:

• Slides
• Chapters 2.1-2.5 in Gaussian Processes for Machine Learning.
• Chapter on kernels from David Duvenaud's PhD thesis.

Software:

• Kernlab - R package for GPs. Quick demo.
• GPML - Matlab suite for GPs.
• Matlab's GP fitting in the Statistics and Machine Learning Toolbox. Very quick demo.
• GPy - extensive Python suite for GPs.

Code snippets:

• R code for simulating from a GP.

Extras:

• Interactive web app for playing around with GPs

---

10.00-10.15 Break

---

10.15-11.00 Gaussian Process Classification and Optimization

Reading:

• Slides
• Chapters 3.1-3.4.1 and 3.7 in Gaussian Processes for Machine Learning.
• Practical Bayesian Optimization of Machine Learning Algorithms (NIPS2012)

Software:

• GPML - Matlab suite for GPs.
• GPy - extensive Python suite for GPs.
• Spearmint - Python library for GPO
• Bayesian Optimization - Python library for GPO
• BayesOpt in the Statistics and Machine Learning Toolbox
• RStan

11.00-11.15 Coffee/Tea break

---

11.15-12.00 Distributed and Subsampling MCMC

Reading:

• Slides Distributed MCMC
• Slides Subsampling MCMC

---

12.00-13.15 Lunch

---

13.15-14.00 Text as data - Topic Models

Reading:

• Slides
• Probabilistic Topic Models - a non-technical intro to topic models
• Latent Dirichlet Allocation - the original paper
• Topic Models - a survey of the field

Software:

• Mallet R package CRAN version or more refined GitHub repo
• gensim - Python module.

---

14.00-14.15 Break

---

14.15-15.00 Variational Bayes

Reading:

• Slides
• Explaining Variational Approximations (Amer Stat)
• Variational Inference: A Review for Statisticians (JASA2017)

Software:

Edward for variational inference - a probabilistic programming language (think Stan)

---

15.00-15.15 Coffee/Tea break

---

15.15-16.00 Deep Learning

Reading:

• Slides
• Chapters 1,4,5 and 6 of Deep Learning - the main book for deep learning.

Software:

• Google's TensorFlow

Code for analyzing the Bank Marketing Data from the UCI repository:

• classification.R using the mxnet package. R-bloggers about mxnet
• classification.py and as python notebook
• classification.m

---