Numerical Instability Termination with Exact NES

[sushantveer]

Thanks a lot to the pybrain team for developing this amazing package!

I was trying to use the ExactNES optimization algorithm which efficiently computes the exact inverse of the Fisher Information Matrix. The ExactNES optimizer seems to terminate prematurely with the message: Numerical Instability. Stopping.

This is a simple test script for reproducing the error:

import numpy as np
from pybrain.optimization import ExactNES

# Set seed for repeatability
np.random.seed(0)

# Loss function
def sphere(x):
    return np.sum(x**2)

# Problem parameters
dim = 2
x0 = np.ones(dim)

# Initialize the optimizer
optim = ExactNES(sphere, x0)
optim.minimize = True
optim.verbose = True
optim.maxLearningSteps = 1000

# Train
x_best, best_val = optim.learn()
print("Best Value:", best_val)

This gives the following output:

('Evals:', 100)
('Step:', 0, 'best:', -0.03253752948627375)
Numerical Instability. Stopping.
('Evals:', 100)
('Step:', 1, 'best:', -0.03253752948627375)
Best Value: 0.03253752948627375

I must also mention a couple of minor changes I had to make in the source code without which I was getting a type error (`TypeError: 'float' object cannot be interpreted as an integer'). The explicit changes are:

1. Explicitly typecasted xdim + xdim * (xdim + 1) / 2 to int(xdim + xdim * (xdim + 1) / 2) in

   pybrain/pybrain/optimization/distributionbased/ves.py

   Line 50 in dcdf32b

   self.numDistrParams = xdim + xdim * (xdim + 1) / 2

2. Explicitly typecasted dim * (dim + 1) / 2 to int(dim * (dim + 1) / 2) in

   pybrain/pybrain/utilities.py

   Line 560 in dcdf32b

   res = zeros(dim * (dim + 1) / 2)

System Details: Ubuntu 16.04; running in a conda environment with Python 3.7.3.

Please let me know if any further details are necessary.