ChangeLog.txt

2016-06-08 Version 4.7

New features
  - Simple Bayesian Optimization demo

Improvements
  - Improved use of PSIS
  - More options added to gp_monotonic
  - Monotonicity now works for additive covariance functions with selected
    variables
  - Possibility to use gpcf_squared.m-covariance function with derivative
    observations/monotonicity
  - Default behaviour made more robust by changing default jitter from
    1e-9 to 1e-6
  - LA-LOO uses the cavity method as the default (see Vehtari et al
    (2016). Bayesian leave-one-out cross-validation approximations for
    Gaussian latent variable models. JMLR, accpeted for publication)
  - Selected variables -option works now better with monotonicity

Bugfixes
 - small error in derivative observation computation fixed
 - several minor bug fixes


2015-07-09 Version 4.6

New features

 - Use Pareto smoothed importance sampling (Vehtari & Gelman, 2015) for 
   - importance sampling leave-one-out cross-validation
     (gpmc_loopred.m)
   - importance sampling integration over hyperparameters
     (gp_ia.m)
   - importance sampling part of the logistic Gaussian process density
     estimation (lgpdens.m)
   - references:
      - Aki Vehtari and Andrew Gelman (2015). Pareto smoothed importance
        sampling. arXiv preprint arXiv:1507.02646.
      - Aki Vehtari, Andrew Gelman and Jonah Gabry (2015). Efficient
        implementation of leave-one-out cross-validation and WAIC for
        evaluating fitted Bayesian models.

 - New covariance functions
    - gpcf_additive creates a mixture over products of kernels for each dimension
      reference: Duvenaud, D. K., Nickisch, H., & Rasmussen, C. E. (2011).
      Additive Gaussian processes. In Advances in neural information
      processing systems, pp. 226-234.
    - gpcf_linearLogistic corresponds to logistic mean function 
    - gpcf_linearMichelismenten correpsonds Michelis Menten mean function 

Improvements
 - faster EP moment calculation for lik_logit

Several minor bugfixes

2014-07-22 Version 4.5

New features
 - Input dependent noise and signal variance.

   Tolvanen, V., Jylänki, P. and Vehtari, A. (2014). Expectation Propagation for
   Nonstationary Heteroscedastic Gaussian Process Regression. In Proceedings of IEEE
   International Workshop on Machine Learning for Signal Processing, accepted for
   publication. Preprint arXiv:1404.5443

 - Sparse stochastic variational inference model.

   Hensman, J., Fusi, N. and Lawrence, N. D. (2013). Gaussian processes
   for big data. arXiv preprint arXiv:1309.6835.

 - Option 'autoscale' in the gp_rnd.m to get split normal approximated samples from the
   posterior predictive distribution of the latent variable.
   
   Geweke, J. (1989). Bayesian Inference in Econometric Models Using Monte Carlo
   Integration. Econometrica, 57(6):1317-1339.
   
   Villani, M. and Larsson, R. (2006). The Multivariate Split Normal Distribution
   and Asymmetric Principal Components Analysis. Communications in
   Statistics - Theory and Methods, 35(6):1123-1140.

Improvements
 - New unit test environment using the Matlab built-in test framework (the old Xunit
   package is still also supported).
 - Precomputed demo results (including the figures) are now available in the folder
   tests/realValues.
 - New demos demonstrating new features etc.
    - demo_epinf, demonstrating the input dependent noise and signal variance model
    - demo_svi_regression, demo_svi_classification
    - demo_modelcomparison2, demo_survival_comparison

Several minor bugfixes

2014-04-11 Version 4.4

New features
 - Monotonicity constraint for the latent function. 

   Riihimäki and Vehtari (2010). Gaussian processes with monotonicity
     information.  Journal of Machine Learning Research: Workshop and
     Conference Proceedings, 9:645-652.

 - State space implementation for GP inference (1D) using Kalman filtering.
   For the following covariance functions
     - Squared-Exponential 
     - Matérn-3/2 & 5/2 
     - Exponential
     - Periodic
     - Constant

   Särkkä, S., Solin, A., Hartikainen, J. (2013). 
     Spatiotemporal learning via infinite-dimensional Bayesian filtering and 
     smoothing. IEEE Signal Processing Magazine, 30(4):51-61.   
   Simo Sarkka (2013). Bayesian filtering and smoothing. Cambridge 
     University Press.
   Solin, A. and Särkkä, S. (2014). Explicit link between periodic 
     covariance functions and state space models. AISTATS 2014. 

Improvements
 - GP_PLOT function for quick plotting of GP predictions
 - GP_IA now warns if it detects multimodal posterior distributions
 - much faster EP with log-Gaussian likelihood (numerical integrals -> 
   analytical results)
 - faster WAIC with GP_IA array (numerical integrals -> analytical results)
 - New demos demonstrating new features etc.
    - demo_minimal, minimal demo for regression and classification
    - demo_kalman1, demo_kalman2
    - demo_monotonic, demo_monotonic2

Bug fixes
 - Periodic covariance function works with selectedVariables
 - Survival likelihoods (log-Gaussian, log-Logistic and Weibull) work now with
   empty z for uncensored data
 - Return parameters in correct order from gp_pak if using hyperhyperpriors
 - Other bug fixes

2013-11-26 Version 4.3.1

Improvements:
 - Updated cpsrf and psrf to follow BDA3: split each chain to two
   halves and use Geyer's IPSE for n_eff
 - Multi-latent models for Octave

2013-10-14 Version 4.3

Improvements:
 - lgpdens.m: better default estimation using importance and rejection sampling, better default priors
 - Robust-EP for zero truncated negative-binomial likelihood
 - If moment computations in EP return NaN, return NaN energy (handled
   gracefully by fminlbfgs and fminscg)
 - gp_cpred.m: new option 'target'
 - gp_ia.m: Changed Hessian computation stepsize to 1e-3
 - gpstuff_version.m: function for returning current GPstuff version
 - gpia_jpreds.m: a new function
 - demo_survival_weibull.m -> demo_survival_aft.m

Bug fixes:
 - build suitesparse path correctly if it includes spaces
 - gp_avpredcomp.m: fixed for Cox-PH
 - gp_cpred.m: fixed for Cox-PH
 - esls.m: don't accept a step to a point with infinite log likelihood
 - gp_ia.m: removed some redundant computation
 - gp_rnd.m: works now for multilatent models also
 - bugfixes for setrandstream
 - other bugfixes 


2013-06-13 Version 4.2

Improvements
 - Cross-validation much faster if no bias-corrections are needed (computes
   only the necessary predictions)
 - Marginal posterior corrections with loopred (Laplace) and cross-validation
 - More robust computation of marginal posterior corrections (utilize 
   log distributions)
 - More robust density estimation in lgpdens (default parameters changed)

Bug fixes
 - Mex files now in correct folders if compiled with SuiteSparse (covariance matrix
   computation now much faster)
 - Fixed bug with default marginal posterior correction when using gp_predcm
 - Fixed conditions in likelihood functions for grid approximation of 
   predictions with marginal posterior corrections
 - Fixed outputs of gpmc_preds with multilatent models (thanks to Mahdi
   Biparva for pointing this out)
 - and some minor bug fixes

2013-04-24 Version 4.1

New features:
  - Multinomial probit classification with nested-EP. Jaakko Riihimäki, 
    Pasi Jylänki and Aki Vehtari (2013). Nested Expectation Propagation 
    for Gaussian Process Classification with a Multinomial Probit 
    Likelihood. Journal of Machine Learning Research 14:75-109, 2013.
  - Marginal posterior corrections for latent values. Cseke & Heskes 
    (2011). Approximate Marginals in Latent Gaussian Models. Journal of 
    Machine Learning Research 12 (2011), 417-454
     - Laplace: cm2 and fact
     - EP: fact

Improvements
 - lgpdens ignores now NaNs instead of giving error
 - gp_cpred has a new option 'target' accpeting values 'f' or 'mu'
 - unified gp_waic and gp_dic
   - by default return mlpd
   - option 'form' accetps now values 'mean' 'all' 'sum' and 'dic'
 - improved survival demo demo_survival_aft (accalerated failure time)
   - renamed and improved from demo_survival_weibull
 - rearranged some files to more logical directories
 - bug fixes 

New files
 - gp_predcm: marginal posterior corrections for latent values.
 - demo_improvedmarginals: demonstration of marginal posterior corrections
 - demo_improvedmarginals2: demonstration of marginal posterior corrections
 - lik_multinomprobit: multinomial probit likelihood
 - demo_multiclass_nested_ep: demonstration of nested EP with multinomprobit


2013-03-12 Version 4.0

New features:
  - Multilatent models: multinomial, softmax, Cox-PH, density
    estimation, density regression, input dependent noise, input
    dependent overdispersion in Weibull, zero-inflated negative binomial
  - Survival models: Cox-PH, Log-Gaussian, Log-logistic, diagnostic criteria
  - Quantile regression
  - PASS-GP active set selection for classification
  - optional memory save in gradient computation
  - approximative gradient for EP-LOO
  - Octave compatibility. Please download Octave specific version of GPstuff
    to use GPstuff with Octave. Following features of v4.0 work only with 
    Matlab:
     - Inputdependent multilatent models
     - Zero-Inflated Negative-Binomial model
     - Cox proportional hazard model
     - Compactly Supported (PPCS*) covariance functions
     - Kronecker speedup for density estimation

Improvements:
  - much faster parallel-EP (now default)
  - faster sequential EP

New functions & files
  - demo_inputdependentnoise: input dependent noise in Gaussian model
  - demo_inputdependentweibull: input dependent overdispersion in Weibull
  - demo_lgpdens: density estimation, density regression
  - demo_loopred: leave-one-out cross-validation approximations
  - demo_mcmc: different MCMC methods demonstrated
  - demo_memorysave: memory save in gradient computation
  - demo_modelcomparison2: additional model comparsion demo
  - demo_multiclass: multi-class classification
  - demo_multinom: multinomial model
  - demo_passgp: PASS-GP active set selection for classification
  - demo_quantilegp: Quantile regression
  - demo_survival_comparison: survival model diagnostic criteria
  - demo_survival_coxph: Gaussian process Cox-PH model
  - demo_zinegbin: zero-inflated negative binomial
  - gpep_loog.m: approximate gradient for EP-LOO
  - gp_kfcv_cdf.m: K-fold cross validation to predict CDF for GP model
  - gp_kfcve.m: mean negative log k-fold-cv predictive density.
  - gpla_looe.m: Laplace Leave-one-out energy (negative preditive density)
  - gp_predcdf.m: Predictive distribution CDF estimation
  - lgpdens_cum.m: Bayesian Bootstrap density estimation integration
  - lgpdens.m: Density estimation with Gaussian Processes
  - lik_coxph.m: Cox proportaional hazard likelihood
  - lik_inputdependentnoise.m: Input-dependent noise likelihood
  - lik_inputdependentweibull.m: Input-dependent Weibull likelihood
  - lik_lgpc.m: Logistic likelihood for conditional density estimation
  - lik_lgp.m: Logistic likelihood for density estimation
  - lik_loggaussian.m: Log-Gaussian likelihood
  - lik_loglogistic.m: Log-logistic likelihood
  - lik_multinom.m: Multinomial likelihood
  - lik_qgp.m: Quantile-GP regression likelihood
  - lik_softmax.m: Softmax (multiclass) likelihood
  - lik_zinegbin.m: Zero-Inflated Negative-Binomial likelihood
  - passgp.m: Pass-GP routine 
  - pred_coxphhs.m: Hazard and survival functions for Cox-Ph likelihood
  - pred_coxph.m: Returns useful values for Cox-PH likelihood
  - pred_coxphp.m: Integrate model (cox-ph) with respect to time

And some bug fixes 

2012-10-29 Version 3.4.1 published.

Bug fixes
 - LOO-CV predictions fixed (gp_loopred, gpmc_loopred, gpla_e, gpep_e)
 - k-fold-cv for PIC sparse approximation fixed (gp_kfcv)
 - other bug fixes (gp_pred, gp_e, prior_t)

Improved functions
 - LOO-CV added to demo_modelassessment1, demo_modelassessment2,
   xunit tests
 - DTC,VAR,SOR sparse approximations for Laplace (gpla_g, gpla_pred)
 - improved robustnes of optimisation functions (fminscg, fminlbfgs)
 - new function for setting random stream (setrandstream)
 - many demos made to display less clutter

2012-10-08 Version 3.4 published. 

Improved functions
 - GP_IA     improved autoscale and display options
 - GP_KFCV   optional return values for cvpreds (f,lp,y)
 - GP_LOOEG  approximate gradients for EP-LOO (no implicit terms)
 - GP_OPTIM  new options 'lambda' and 'lambdalimit' used by fminscg
 - GP_SET    new option 'savememory' for memory saving in gradient calculations
 - FMINSCG   new options 'lambda' and 'lambdalimit'

Other changes
 - SuiteSparse v 3.4 included in the distribution package
 - gp/demos renamed to gp/demodata
 - Few bug fixes and several documentation fixes

2012-06-20 Version 3.3 published. Some new functions and bug fixes. 

New functions
 - GP_CPRED      Conditional predictions using specific covariates
 - GPCF_SCALED   Create a scaled covariance function
 - SURROGATE_SLS Markov chain Monte Carlo sampling using Surrogate 
                 data Slice Sampling

 - PRIOR_INVT    Inverse Student-t prior structure
 - PRIOR_INVUNIF Inverse uniform prior structure
 - PRIOR_LOGT    Student-t prior structure for the logarithm of the parameter

 - HMC_NUTS      No-U-Turn Sampler (NUTS)

 - ADDLOGS       Add numbers represented by their logarithms.
 - LOGITINV      Inverse of the logit transformation
 - MAPCOLOR      Returns a colormap ranging from blue through gray to red
 - SUMLOGS       Sum of vector where numbers are represented by
                 their logarithms.

 - TEST_ALL      Unit testing with xunit package (requires xunit
                 from Mathowrks File Exchange)

Improved functions
 - Robust-EP method improved
 - GP_PRED       Makes predictions using training data if no test
                 data is given
 - GP_LOOPRED    Supports now sparse approximations FIC, PIC,
                 CS+FIC and latent method Laplace
 - GP_MC         By default use Surrogate Slice Sampler for
                 hyperparameters and Elliptical Slice Sampler for
                 latent values. Supports now also NUTS.
 - SLS           Added Shrinking-rank SLS, Covariance-matching SLS

Several bug and documentation fixes

2012-03-16 Version 3.2.1 published. Many bug fixes.

2011-10-11 Version 3.2 published. New observation models and new inference algorithms (e.g. EP for Student-t model) added plus bug fixes.

2011-04-15 Version 3.1 published. New functionalities and major update to argument syntax making the package easier to use.