Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stopping criterion utility #333

Open
TimotheeMathieu opened this issue Jul 10, 2023 · 3 comments
Open

Stopping criterion utility #333

TimotheeMathieu opened this issue Jul 10, 2023 · 3 comments
Labels
enhancement New feature or request good first issue Good for newcomers Marathon To do during Marathon

Comments

@TimotheeMathieu
Copy link
Collaborator

Implement a tool to stop the algorithm when some value (in the writer) go above some threshold. Typically for either early stopping or for stopping after a certain number of episodes.

@riiswa
Copy link
Collaborator

riiswa commented Jul 10, 2023

Should we modify the signature of the fit method to be somth likedef fit(self, budget=100, stop_callback: Callable[[Writer], bool]) ?

@KohlerHECTOR KohlerHECTOR added enhancement New feature or request good first issue Good for newcomers Marathon To do during Marathon labels Jul 13, 2023
@KohlerHECTOR KohlerHECTOR added this to To do in Marathon rlberry Jul 13, 2023
@riiswa riiswa moved this from To do to In progress in Marathon rlberry Jul 24, 2023
@riiswa riiswa self-assigned this Jul 24, 2023
@riiswa riiswa moved this from In progress to Skipped in Marathon rlberry Jul 24, 2023
@riiswa riiswa removed their assignment Jul 24, 2023
@riiswa
Copy link
Collaborator

riiswa commented Jul 24, 2023

I think there is not a simple way to solve this PR. Each agent have its own implementation of the fit method, so the early stopping criterion should be handled in each agents...

@TimotheeMathieu
Copy link
Collaborator Author

My idea was the following:

  • Have an optional argument in the fit that default to None, as you said it can be something like stop_callback: None or Callable[[Writer], bool]. This is optional, users can implement an agent without using it, the same way as users can use the budget or not (see ValueIteration for instance).
  • Have a prototype of callback function that can be used for anyone and is well documented.
  • Have the function implemented in all rlberry agents to show how it is done.

This should be sufficient, we don't need to have something automatic, we only need to make it simple to use and have it implmented in the rlberry agents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers Marathon To do during Marathon
Projects
Development

No branches or pull requests

3 participants