Visualization

[Daniele-Dondi]

I've seen your docs on visualization, and everything is explained with 1D function.
However,
in the program description you shown the visualization of a 2D function.
My question is:
How to implement visualization for 2D functions?
We just iterate the optimizer._gp.fit(x_obs, y_obs) two times?
Thank you

[till-m]

Hi @Daniele-Dondi,

if I'm understanding your question right, you would like to plot some 2D optimization results?
See the code here (cell 13) and also #18 for some examples.

[Daniele-Dondi]

I'm sorry because I wasn't clear enough. I will try to explain better:

In the readme.md file,
in the section:
Bayesian Optimization in Action

you are showing an animated gif:
https://github.com/bayesian-optimization/BayesianOptimization/blob/master/static/bayesian_optimization.gif

I wish to know how you created the topleft graph (Gaussian Process Predicted Mean).

Later,
in the visualization.html (http://bayesian-optimization.github.io/BayesianOptimization/visualization.html)

You showed a code able to plot the guess for a function target(x)
def target(x):
return np.exp(-(x - 2)**2) + np.exp(-(x - 6)2/10) + 1/ (x2 + 1)

by defining a procedure named plot_gp(optimizer, x, y)

and you are showing the change step after step calling the code:

optimizer.maximize(init_points=0, n_iter=1)
plot_gp(optimizer, x, y)

My question is:
What if my target function having two variables instead of one like in your example?
How can I modify the code plot_gp?
Thank you for you patience

[till-m]

Hi @Daniele-Dondi,

the code used to create the animated gif is discussed in #18. If you want to generally plot 2D-optimization results (i.e. no animation), see the code I linked in the constraints notebook, specifically the bottom left cell (axs[1, 0]) should do what you want.

[Daniele-Dondi]

Thank you,
I'm trying to extract that section from your code.

[Daniele-Dondi]

I inserted your function in the step to step optimizer.
Annoingly all the figures could be visualized only at the end.
Anyway it is saving correctly at each step.
I post here my code:

from bayes_opt import BayesianOptimization
from bayes_opt.logger import JSONLogger
from bayes_opt.event import Events
from bayes_opt.util import load_logs
from bayes_opt import UtilityFunction
import numpy as np
import matplotlib.pyplot as plt


def black_box_function(x,y):
    """Function with unknown internals we wish to maximize.

    This is just serving as an example, for all intents and
    purposes think of the internals of this function, i.e.: the process
    which generates its output values, as unknown.
    """
    value=x ** 3 - (y - 1) ** 2 -x**2 -x*y
    if value<0: value=0
    return value

def plot_target_estimation(pbounds, optimizer, next_point, cycle): 

    # Setup the grid to plot on
    num_points=300
    x = np.linspace(pbounds['x'][0]-0.1, pbounds['x'][1]+0.1, num_points) 
    y = np.linspace(pbounds['y'][0]-0.1, pbounds['y'][1]+0.1, num_points) 
    xy = np.array([[x_i, y_j] for y_j in y for x_i in x]) 
    X, Y = np.meshgrid(x, y)  
    
    fig, axs = plt.subplots(constrained_layout=True, figsize=(4,4))

    # Extract & unpack the optimization results
    max_ = optimizer.max
    res = optimizer.res
    x_ = np.array([r["params"]['x'] for r in res])
    y_ = np.array([r["params"]['y'] for r in res])

    Z_est = optimizer._gp.predict(xy).reshape(num_points,num_points)
    axs.contourf(X, Y, Z_est, cmap=plt.cm.coolwarm) 
    axs.set_title('Target estimated, cycle n.'+str(cycle+1))
    axs.scatter(x_, y_, c='red', s=80, edgecolors='black') #add evaluated points
    axs.scatter(next_point['x'], next_point['y'], c='white', s=80, edgecolors='black') #add future point
    fig.savefig("Cycle "+str(cycle+1))
    return fig


utility = UtilityFunction(kind="ucb", kappa=2.5, xi=0.0)
figures=[]


# Bounded region of parameter space
pbounds = {'x': (2, 4), 'y': (-3, 3)}
MaxIterations=10

optimizer = BayesianOptimization(
    f=None,
    pbounds=pbounds,
    verbose=2,
    random_state=1
)

logger = JSONLogger(path="./logs.log")
optimizer.subscribe(Events.OPTIMIZATION_STEP, logger)

for cycle in range(MaxIterations):
     next_point = optimizer.suggest(utility)
     print("Next point to probe is:", next_point)
     figures.append(plot_target_estimation(pbounds, optimizer, next_point, cycle))
     ask=input('Press ENTER to perform the measurement')
     target = black_box_function(**next_point)
     optimizer.register(params=next_point, target=target)
     print(target, next_point)


for figure in figures: #show all the figures
    figure.show()

print('\nMAXIMUM:',optimizer.max)

[till-m]

Hi @Daniele-Dondi,

glad to see things worked out for you. I will close this issue. Feel free to make a new issue, if you run into other problems.