Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include DAC-BO as a benchmark #135

Open
4 tasks
benjamc opened this issue Nov 15, 2023 · 1 comment
Open
4 tasks

Include DAC-BO as a benchmark #135

benjamc opened this issue Nov 15, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@benjamc
Copy link
Contributor

benjamc commented Nov 15, 2023

Include the configuration of BO (SMAC) as a benchmark.
Environment already exists here: https://github.com/automl/SAWEI/blob/AutoMLConf23/dacbo/env.py

  • Check if everything runs with current DACBench version
  • Check action space
  • Create instance set
  • Port SAWEI to policy format /check, run as baseline
@benjamc benjamc added the enhancement New feature or request label Nov 15, 2023
@TheEimer
Copy link
Contributor

Thanks for opening! I think the most reasonable plan would be:

  • @TR0l1 (after SGD is through) makes a suggestion of how to unify the versions and update to the current DACBench interface (should basically be gymnasium update, I think) - this can happen on the dev branch since I think including it in the next update would be good anyway
  • @benjamc reviews the code and makes a suggestion as to search spaces and instance sets
  • I fit it those suggestions and do a final code review for the benchmark
  • @benjamc makes a separate PR for the SAWEI policy at some point

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants