Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Seeding is not applied to configspace #83

Open
eddiebergman opened this issue May 7, 2024 · 3 comments · May be fixed by #84
Open

[Bug] Seeding is not applied to configspace #83

eddiebergman opened this issue May 7, 2024 · 3 comments · May be fixed by #84
Assignees
Labels
bug Something isn't working

Comments

@eddiebergman
Copy link
Contributor

eddiebergman commented May 7, 2024

Please see this reproducible example and output, I believe it's becuase the configspace is never seeded:

from __future__ import annotations

from ConfigSpace import ConfigurationSpace

from dehb import DEHB

cs = ConfigurationSpace({"a": (1.0, 10.0)})


def f(config, fidelity):
    print(config, fidelity)
    return {"fitness": config["a"] ** 2, "cost": 1.0, "info": {}}


D = DEHB(cs, f=f, seed=1, min_fidelity=1, max_fidelity=100, n_workers=1)
D.run(1)

D = DEHB(cs, f=f, seed=1, min_fidelity=1, max_fidelity=100, n_workers=1)
D.run(1)
@Bronzila
Copy link
Collaborator

Bronzila commented May 7, 2024

Hey,
thanks for the issue! :)
Yes this is most certainly due to us not reseeding the ConfigurationSpace, however do you think this is our job to do or shouldn't the user seed the ConfigurationSpace before passing it in? Just wondering how other libraries handle this.

@eddiebergman
Copy link
Contributor Author

Most people don't know that they need to seed the space and I imagine they would expect if they pass a seed to DEHB then it should reproduce. Otherwise, the seed passed to DEHB has no meaning, (unless you happened to know you should also do this other thing).

Every library I've worked on that uses ConfigSpace sets the seed themselves on it for this reason.

@Bronzila
Copy link
Collaborator

Bronzila commented May 7, 2024

That makes sense. Thanks for the PR, I will aim to review it by the end of the week! :)

@Bronzila Bronzila self-assigned this May 7, 2024
@Bronzila Bronzila added bug Something isn't working and removed bug Something isn't working labels May 7, 2024
@Bronzila Bronzila added the bug Something isn't working label May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants