Gaussian-Process-for-Regression

• Implementation of Gaussian Process (GP) for regreesion with the exponential-quadratic kernel function.
• We have 100 pairs of samples (x,t) in 'gp.mat' for training & testing.
  Consider the 60% as traing data, 40% as testing.
• Given that the input x is limited in [0,2], predict the target t using GP.

Implmentation process

1. Calculate the kernel matrix from the first 60 samples using the following equation:
   

2. Calculate the Covariance matrix
   
   has values when n = m, so is equal to

3. Calculate the kernel vecotr suing testing & training input x using the following equation:
   

4. Calaulate the variance c of test set
   

5. Obtain the prediction of tset set
   
   represent the t in train set

   • Mean & Variance of predict distribution
     

Investigate the Predict distribution

• The procedure is similar to the above implementation:
  Replace the input for test set with all the value in x-domain like [0, 0.01, 0.02, ... 2]
• Visualization:
  

Fine-tuning with different combinations of hyperparameters

• Evaluation parameter is root-mean-square (RMS)
  

• Comparison Table:
  

  It was found that the results of prediction are more accurate for Train & Test set when using the parameters {1,32,5,5}.

• Further Analysis:
  To validate the individual influence of theta_0, theta_1, theta_2 and theta_3, we try the combination like [1,0,0,0], [0,1,0,0], [0,0,1,0] and [0,0,0,1] for GP Regression.
  
  It was found that the prediction will be degraded to linear regression if the other three values are all zero.

Best Combination for given dataset - ARD

• automatic relevance determination (ARD)