Optimization in memory usage of gKDR

[yyimingucl]

Hi MOGP developing team,

I met a RAM exceeding problems when I use gKDR for a 40k dimensional dataset and the it was quite slow to solve a 641d problem. I am raising the issue for a potential optimisation after I compare the gKDR in mogp and its original implementation by matlab.

I noticed that we compute the matrix R jointly with all samples but this will lead to a problem when the sample size is large. From the original implementation of gKDR, the author uses a loop instead and compute the R incrementally which largely reduce the required memory.

The matlab code looks like (all the notations are same with mogp):
R = 0;
for i = 1:N
% Derivative of k(X_i,x) w.r.t x
Hi = ((repmat(X(i,:),N,1) - X)./SGX/SGX).*repmat(Kx(:,i),1,M);
%Hi = squeeze(Xij(i,:,:)).*repmat(Kx(:,i),1,M);
Ri = Hi'FHi;
R = R+Ri;
end

Best,
Yiming

[ots22]

Hi @yyimingucl,

Thanks for opening this issue - I agree with your diagnosis and proposed solution and will try to push a fix fairly soon.

Oliver

[yyimingucl]

Hi @yyimingucl,

Thanks for opening this issue - I agree with your diagnosis and proposed solution and will try to push a fix fairly soon.

Oliver

Many thanks Oliver

Yiming

[ots22]

Hi @yyimingucl, the PR above introduces these changes from the branch gkdr-mem.