Generalize implementation of Gaussian expectations of Gaussian kernels

[j-wilson]

PR type: Enhancement

Summary

Proposed changes
Extend the existing implementation of Gaussian expectations of Gaussian kernels $E_{p(x)}[k1(Z1, x) k2(x, Z2)]$ to support different kernels and different inducing locations.

Minimal working example

import tensorflow as tf
from gpflow.kernels import SquaredExponential
from gpflow.expectations import expectation
from gpflow.inducing_variables import InducingPoints
from gpflow.probability_distributions import Gaussian

N = 4  # num. uncertain test locations
M = 3  # num. inducing locations per kernel
D = 2  # dimensionality of input domain

mx = tf.random.uniform([N, D],  dtype=tf.float64)
Lxx = tf.random.normal([N, D, D],  dtype=tf.float64)
Sxx = tf.matmul(Lxx, Lxx, transpose_b=True)
px = Gaussian(mx, Sxx)

k1 = SquaredExponential(lengthscales=0.01 + tf.random.uniform([D], dtype=tf.float64))
Z1 = InducingPoints(tf.random.uniform([M, D], dtype=tf.float64))

k2 = SquaredExponential(lengthscales=0.01 + tf.random.uniform([D], dtype=tf.float64))
Z2 = InducingPoints(tf.random.uniform([M, D], dtype=tf.float64))

eK = expectation(px, (k1, Z1), (k2, Z2))  # [N, M, M]

PR checklist

• New features: code is well-documented
• detailed docstrings (API documentation)
• notebook examples (usage demonstration)
• The bug case / new feature is covered by unit tests
• Code has type annotations
• I ran the black+isort formatter (make format)
• I locally tested that the tests pass (make check-all)

Release notes

Fully backwards compatible: yes

[vdutor]

I didn't double check all the maths, but if the tests pass I'm happy to accept this PR.

[vdutor]

you'll have to run make format to make the formatting checks pass in the tests.

[vdutor]

You'll have to run make format to make the format checkers happy in the tests.

[st--]

@j-wilson I just noticed this PR is still not merged despite @vdutor having approved it: it seems you just need to get the tests to pass (and merge latest develop into it)! you may want to double-check that you're using the right version of black (see tests_requirements.txt)

[j-wilson]

@j-wilson I just noticed this PR is still not merged despite @vdutor having approved it: it seems you just need to get the tests to pass (and merge latest develop into it)! you may want to double-check that you're using the right version of black (see tests_requirements.txt)

@st-- Just merged with the current develop branch and reran make format and make check-all after installing packages from tests_requirements.txt. All tests passed, save for one in .../notebooks/theory/upper_bound.pct.py.