Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Systematic testing of settings and combinations #283

Open
JuliaS92 opened this issue Jul 21, 2021 · 1 comment
Open

Systematic testing of settings and combinations #283

JuliaS92 opened this issue Jul 21, 2021 · 1 comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@JuliaS92
Copy link

Is your feature request related to a problem? Please describe.
Currently it is rather complicated to trace which settings are used where and which combinations cause which behaviour.

Describe the solution you'd like
There should be a workflow, that tests two things:

  1. If you change a setting, it is actually passed on to the executing functions and changes the result. This can potentially be automated by changing each setting to it's min and max value based on the default settings. Both have to be different from the default result and different from one another.
  2. Which workflows are valid and and which ones crash. For the major workflow steps all combinations of selecting or unselecting them should be iterated autoamtically and they should either raise appropriate errors or run through smoothly.
    All of this could be done on a very minimal dataset (e.g. two short gradient files).
@JuliaS92 JuliaS92 added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 21, 2021
@straussmaximilian
Copy link
Member

That's an excellent suggestion.
We had started with the ParameterScreen at some point: https://github.com/MannLabs/alphapept/blob/master/sandbox/ParameterScreen.ipynb

The idea at that time was to screen through all parameters for some test sets and define sane min max values - and also find out what makes things slow w/o having a performance gain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
No open projects
Development

No branches or pull requests

2 participants