Inconsistent Dynamic Parameter Handling in Optuna Ask-Tell Interface with Conditional Trials

[Djkmarco]

Expected behavior

I'm encountering unexpected behavior when using the Optuna ask-tell interface with dynamic search spaces and conditional trials. The issue manifests as the dynamic parameter being treated as a separate parameter at each iteration, even though it's intended to be the same parameter across iterations.
Within the ask-tell interface, I anticipate that the ask-tell interface would recognize the dynamic parameter as a single entity throughout the optimization process. This implies that the value provided by the tell method should be used for subsequent ask calls with the same parameter name, regardless of the trial iteration.
This behavior isn't confined to the Optuna dashboard; it also affects core Optuna functionality. For instance, if I reload a study and attempt a new optimization using study.optimize, the conditional trial's dynamic parameter isn't carried over to subsequent trials.

Environment

• Optuna version: 3.6.1
• Python version: 3.10.11
• OS :Windows-10-10.0.22631-SP0

Error messages, stack traces, or logs

No Error Messages

Steps to reproduce

Example 1

import optuna
from numpy import random

study = optuna.create_study(sampler=optuna.samplers.RandomSampler(seed=42),
                            direction='minimize',
                            study_name='nsga3',
                            storage='sqlite:///example.db',
                            load_if_exists=True)

for i in range(10):
    trial = study.ask()
    x = trial.suggest_float('x', -10, 10)
    y = trial.suggest_int('y', 0, i)
    result = (x - 2) ** 2 + y
    study.tell(trial, result)

Example 2

import optuna
from numpy import random

study = optuna.create_study(sampler=optuna.samplers.RandomSampler(seed=42),
                            direction='minimize',
                            study_name='nsga3',
                            storage='sqlite:///example.db',
                            load_if_exists=True)

for i in range(10):
    x = optuna.distributions.IntDistribution(-10, 10)
    y = optuna.distributions.IntDistribution(0, i)
    distribution = {'x': x, 'y': y}
    trial = study.ask(distribution)
    result = (trial.params['x'] - 2) ** 2 + trial.params['y']
    study.tell(trial, result)

Example 3

import optuna
from numpy import random


storage = optuna.storages.InMemoryStorage()

study = optuna.create_study(sampler=optuna.samplers.RandomSampler(seed=42),
                            direction='minimize',
                            study_name='nsga3',
                            storage=storage)

for i in range(3):
    x = optuna.distributions.IntDistribution(-10, 10)
    y = optuna.distributions.IntDistribution(0, i)
    distribution = {'x': x, 'y': y}
    trial = study.ask(distribution)
    result = (trial.params['x'] - 2) ** 2 + trial.params['y']
    study.tell(trial, result)

study2 = optuna.create_study(sampler=optuna.samplers.RandomSampler(seed=42),
                             direction='minimize',
                             study_name='nsga3',
                             storage=storage,
                             load_if_exists=True)

for j in range(10):
    x = optuna.distributions.IntDistribution(-10, 10)
    y = optuna.distributions.IntDistribution(0, 10)
    distribution = {'x': x, 'y': y}
    trial = study2.ask(distribution)
    result = (trial.params['x'] - 2) ** 2 + trial.params['y']
    study2.tell(trial, result)

study3 = optuna.copy_study(from_study_name='nsga3', from_storage=storage, to_storage="sqlite:///example.db")

Evidence this affects following steps

import optuna

storage = optuna.storages.InMemoryStorage()

partial_sampler = optuna.samplers.PartialFixedSampler({'y':0}, optuna.samplers.NSGAIIISampler(seed=42,
                                                                                              population_size=5,
                                                                                              mutation_prob=0,
                                                                                              crossover_prob=0.9,
                                                                                              crossover=optuna.samplers.nsgaii.UniformCrossover()))

study = optuna.create_study(sampler=partial_sampler,
                            direction='minimize',
                            study_name='nsga3',
                            storage=storage)

for i in range(10):
    x = optuna.distributions.IntDistribution(-10, 10)
    y = optuna.distributions.IntDistribution(0, i)
    distribution = {'x': x, 'y': y}
    trial = study.ask(distribution)
    result = (trial.params['x'] - 2) ** 2 + trial.params['y']
    study.tell(trial, result)

study2 = optuna.create_study(sampler=optuna.samplers.NSGAIIISampler(seed=42,
                                                                    population_size=5,
                                                                    mutation_prob=0,
                                                                    crossover_prob=0.9,
                                                                    crossover=optuna.samplers.nsgaii.UniformCrossover(),
                                                                    ),
                             direction='minimize',
                             study_name='nsga3',
                             storage=storage,
                             load_if_exists=True)

for j in range(10):
    x = optuna.distributions.IntDistribution(-10, 10)
    y = optuna.distributions.IntDistribution(0, 10)
    distribution = {'x': x, 'y': y}
    trial = study2.ask(distribution)
    result = (trial.params['x'] - 2) ** 2 + trial.params['y']
    study2.tell(trial, result)

study3 = optuna.copy_study(from_study_name='nsga3', from_storage=storage, to_storage="sqlite:///example.db")

Gives results:

Expected y for trials from 10 to 19 to be 0 (as mutation probability is 0)

Setting in the first loop:

y = optuna.distributions.IntDistribution(0, 10)

correctly returns:

Additional context (optional)

It is important ask-tell interface can deal with conditional search space, as it is one of the most usefull application of such interface itself.
Thank you for your assistance!