Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Multi-Objective Reporting and Pruning #5268

Closed
5 tasks
bionicles opened this issue Feb 22, 2024 · 2 comments
Closed
5 tasks

Feature Request: Multi-Objective Reporting and Pruning #5268

bionicles opened this issue Feb 22, 2024 · 2 comments
Labels
feature Change that does not break compatibility, but affects the public interfaces.

Comments

@bionicles
Copy link

bionicles commented Feb 22, 2024

Motivation

I just switched from single to multi objective and thought more information would be helpful to prune but unfortunately it is unsupported on the reporting side and then the pruning side has nothing to build on.

Description

Idea: all objectives are multi-objective, even single objective ones, and all of these operate the same way the single objective case works, then there's no special case here, report can insert a row per objective, the pruner can always work on the hypervolume history of reports, which is just the history if it's single objective. The stupidest idea could be to just independently prune each objective the exact same way as before and then

return all(pruned_for_objective(name, trial) for name in objective_names)
that could work to eliminate trials if and only if the pruner prunes them in each objective simultaneously, if the trials make progress on a subobjective, that keeps em alive

Alternatives (optional)

I could make my own custom pruner and just use the intermediate values from my own database but then i need to decide how to make a multiobjective pruner!

Additional context (optional)

  • test multiobjective reporting
  • remove the conditional preventing multiobjective reporting
  • report multi-objective data
  • test multiobjective pruning
  • map the pruner over the objective series columns and filter by
    e.g.
    pruned_trials_ser = trials_df[trials_df.map(pruner).all(axis='columns')]
    ( no progress on all objectives simultaneously )
@bionicles bionicles added the feature Change that does not break compatibility, but affects the public interfaces. label Feb 22, 2024
@nzw0301
Copy link
Member

nzw0301 commented Feb 22, 2024

Seems duplicated with #3450. The core team and I attempted to implement this but the PR was closed due to #3450 (comment).

@nabenabe0928
Copy link
Collaborator

I will close this PR because this PR relates to #3450 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Change that does not break compatibility, but affects the public interfaces.
Projects
None yet
Development

No branches or pull requests

3 participants