Hierarchical Inducing Point Gaussian Process for Inter-domain Observations

This repo contains the codes for the AISTATS 2021 paper Hierarchical Inducing Point Gaussian Process for Inter-domain Observations.

Abstract

We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy ob-servations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or differentiation, inference is still closed form.However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP(HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points tothe millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in generalGP settings.

Requirements

Create a new conda enviroment with python=3.7 as follows

conda create -n hipgp python=3.7

To activate the conda environment, run:

conda activate hipgp

Ensure that you are in the root directory:

cd hipgp

To install the package requirements run the following

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Finally, install ziggy and ziggy.misc-- the packages for hipgp implementation, by running

python setup.py install

Datasets and Experiments

First,

cd experiments-hip-gp

1. Effect of the Preconditioner (Sec 5.1 )

python run_solve_kn_experiment.py

2. Speedup over Cholesky Decomposition (Sec 5.2)

python run_pcg_vs_cholesky.py

3.Synthetic Derivative Observations (Sec 5.3)

Run the jupyter notebook GP-with-Derivatives.ipynb

In order to run the notebook, you would probably need to create a kernel by

pip install ipykernel
python -m ipykernel install --user --name hipgp

4. Spatial Analysis: UK Housing Prices (Sec 5.4)

Download the 2018 uk housing data from the following sources:

• UK land registry monthly price data: https://ckan.publishing.service.gov.uk/dataset/land-registry-monthly-price-paid-data

  This file includes POSTCODE, which are pretty granular in the UK.

• UK Post Code lat/long data: https://www.freemaptools.com/download-uk-postcode-lat-lng.htm

• UK shapefiles: https://gadm.org/download_country_v3.html

Put these data files under hipgp/experiments-hipgp/uk-price-paid-data

An example command to run the experiment

python run_ukhousing_experiment.py --fit-models --mf-model --nobs=1000 --ntest=100 --num-inducing-x=10 --num-inducing-y=10

5. Inferring Interstellar Dust Map (Sec 5.5)

This repo contains a small sample (5k data points) of the whole dataset,

under hipgp/experiemnts-hipgp/domain-data.

An example command to run the experiment

python run_domain_experiment.py --nobs=1000 --ntest=1000 --nx=10 --nz=5 --mf-model --fit-models --lr=1e-6 --epochs=2

The whole domain simulation dataset comes from the following sources:

https://ui.adsabs.harvard.edu/abs/2016ApJ...827L..23W/abstract

https://ui.adsabs.harvard.edu/abs/2018MNRAS.480..800H/abstract

https://ui.adsabs.harvard.edu/abs/2020ApJS..246....6S/abstract

6. Empirical analysis on preconditioner (Appendix C.1)

Run preconditioner-analysis.ipynb

7. UCI 3d road dataset (Appendix C.3)

The full set of UCI regression datasets can be downloaded at https://d2hg8soec8ck9v.cloudfront.net/datasets/uci_data.tar.gz. Download the tar file and then put the 3droad.mat file under hipgp/experiments-hipgp/uci-data/.

An example command to run the experiment

python run_3droad_experiment.py --nobs=1000 --ntest=1000 --num-inducing-x=20 --num-inducing-y=20 --mf-model --fit-models --lr=1e-6 --epochs=2