Optuna throws None value from Objective function

[Chidu2000]

Trial 4 failed with parameters: {'ACTOR_LR_RATE': 0.00576513613836057, 'CRITIC_LR_RATE': 0.016747063526072883, 'S_INFO': 6, 'S_LEN': 9, 'TRAIN_SEQ_LEN': 74, 'MODEL_SAVE_INTERVAL': 76, 'BUFFER_NORM_FACTOR': 11, 'CHUNK_TIL_VIDEO_END_CAP': 49, 'M_IN_K': 1017, 'REBUF_PENALTY': 4, 'SMOOTH_PENALTY': 1, 'RAND_RANGE': 1161} because of the following error: The value None could not be cast to float.

[nzw0301]

could you share minimal reproducible codes with us?

[nzw0301]

I think your objective function returns None, not float.

[Chidu2000]

This is the objective function,

def objective(trial):
ACTOR_LR_RATE = trial.suggest_float("ACTOR_LR_RATE", 1e-4, 1e-2)
CRITIC_LR_RATE = trial.suggest_float("CRITIC_LR_RATE", 1e-3, 1e-1)
S_INFO = trial.suggest_int("S_INFO", 6, 8) # bit_rate, buffer_size, next_chunk_size, bandwidth_measurement(throughput and time), chunk_til_video_end
S_LEN = trial.suggest_int("S_LEN", 8, 9) # take how many frames in the past
A_DIM = 6
NUM_AGENTS = 3
TRAIN_SEQ_LEN = trial.suggest_int("TRAIN_SEQ_LEN", 50, 100)
MODEL_SAVE_INTERVAL = trial.suggest_int("MODEL_SAVE_INTERVAL", 50, 100)
VIDEO_BIT_RATE = [300, 750, 1200, 1850, 2850, 4300]
HD_REWARD = [1, 2, 3, 12, 15, 20]
BUFFER_NORM_FACTOR = trial.suggest_int("BUFFER_NORM_FACTOR", 10.0, 12.0)
CHUNK_TIL_VIDEO_END_CAP = trial.suggest_int("CHUNK_TIL_VIDEO_END_CAP", 48.0, 50.0)
M_IN_K = trial.suggest_int("M_IN_K", 1000.0, 1100.0)
REBUF_PENALTY = trial.suggest_int("REBUF_PENALTY", 4.3, 5)
SMOOTH_PENALTY = trial.suggest_int("SMOOTH_PENALTY", 1, 2)
DEFAULT_QUALITY = 1 # default video quality without agent
RANDOM_SEED = 42
RAND_RANGE = trial.suggest_int("RAND_RANGE", 1000, 1200)

TOTALEPOCH=100
IS_CENTRAL=True
NO_CENTRAL=False

time=datetime.now()
np.random.seed(RANDOM_SEED)
torch.manual_seed(RANDOM_SEED)


# assert len(VIDEO_BIT_RATE) == A_DIM

# create result directory
if not os.path.exists(SUMMARY_DIR):
    os.makedirs(SUMMARY_DIR)

# inter-process communication queues
net_params_queues = []
exp_queues = []
for _ in range(NUM_AGENTS):
    net_params_queues.append(mp.Queue(1))
    exp_queues.append(mp.Queue())

# create a coordinator and multiple agent processes
# (note: threading is not desirable due to python GIL)
    
all_cooked_time, all_cooked_bw, _,= load_trace.load_trace(TRAIN_TRACES)

best_hyperparams = (ACTOR_LR_RATE, CRITIC_LR_RATE, S_INFO, S_LEN, A_DIM, NUM_AGENTS, TRAIN_SEQ_LEN, MODEL_SAVE_INTERVAL, VIDEO_BIT_RATE, HD_REWARD, BUFFER_NORM_FACTOR, CHUNK_TIL_VIDEO_END_CAP, M_IN_K,\
                        REBUF_PENALTY, SMOOTH_PENALTY, DEFAULT_QUALITY, RANDOM_SEED, RAND_RANGE, TOTALEPOCH, IS_CENTRAL, NO_CENTRAL)

coordinator = mp.Process(target=central_agent,
                         args=(net_params_queues, exp_queues,arglist.model_type,best_hyperparams))
coordinator.start()


agents = []
for i in range(NUM_AGENTS):
    agents.append(mp.Process(target=agent,
                             args=(i, all_cooked_time, all_cooked_bw,
                                   net_params_queues[i],
                                   exp_queues[i],arglist.model_type,best_hyperparams)))
for i in range(NUM_AGENTS):
    agents[i].start()

# wait unit training is done
coordinator.join()
for i in range(NUM_AGENTS):
    agents[i].join()

print(str(datetime.now()-time))

def main(arglist):

study = optuna.create_study(direction="maximize")  # Create a new study.
study.optimize(objective, n_trials=100)

Here, is the agent function which unpacks the best_hyperparameters,

def agent(agent_id, all_cooked_time, all_cooked_bw, net_params_queue, exp_queue, model_type,best_hyperparams):
(ACTOR_LR_RATE, CRITIC_LR_RATE, S_INFO, S_LEN, A_DIM, NUM_AGENTS, TRAIN_SEQ_LEN, MODEL_SAVE_INTERVAL, VIDEO_BIT_RATE, HD_REWARD, BUFFER_NORM_FACTOR, CHUNK_TIL_VIDEO_END_CAP, M_IN_K,
REBUF_PENALTY, SMOOTH_PENALTY, DEFAULT_QUALITY, RANDOM_SEED, RAND_RANGE, TOTALEPOCH, IS_CENTRAL, NO_CENTRAL) = best_hyperparams
torch.set_num_threads(1)

This is the code on which I'm conducting my Optuna analysis, the study returns values though, but asserts it as a Fail.

[nzw0301]

As I said the objective function returns None because your objective function does not have return.

[Chidu2000]

I am doing multi-parameter tuning in my objective function, but the Optuna expects to return a single value.
I am packing multiple values inside a tuple and then returning it, but it still throws an error.

Trial 0 failed with parameters: {'ACTOR_LR_RATE': 0.0008129823403034368, 'CRITIC_LR_RATE': 0.037952993255706884, 'S_INFO': 8, 'S_LEN': 8, 'TRAIN_SEQ_LEN': 100, 'MODEL_SAVE_INTERVAL': 52, 'BUFFER_NORM_FACTOR': 11, 'CHUNK_TIL_VIDEO_END_CAP': 48, 'M_IN_K': 1033, 'REBUF_PENALTY': 4, 'SMOOTH_PENALTY': 1, 'RAND_RANGE': 1019} because of the following error: The number of the values 12 did not match the number of the objectives 1.

[Chidu2000]

For your inference,

best_hyperparams = (ACTOR_LR_RATE, CRITIC_LR_RATE, S_INFO, S_LEN, TRAIN_SEQ_LEN, MODEL_SAVE_INTERVAL, BUFFER_NORM_FACTOR, CHUNK_TIL_VIDEO_END_CAP, M_IN_K,
REBUF_PENALTY, SMOOTH_PENALTY, RAND_RANGE)

return best_hyperparams

[nzw0301]

I think your usage of optuna seems strange. I highly recommend you to read the docs.

[Chidu2000]

Is it even possible to do multi-parameter optimization from Optuna and return them all at once?

[Chidu2000]

I figured out that instead of returning the tuple with hyperparameters, a return 0 works.