Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Re-score existing validation predictions #550

Open
psinger opened this issue Dec 21, 2023 · 0 comments
Open

[FEATURE] Re-score existing validation predictions #550

psinger opened this issue Dec 21, 2023 · 0 comments
Labels
type/feature Feature request

Comments

@psinger
Copy link
Collaborator

psinger commented Dec 21, 2023

馃殌 Feature

Add a functionality to re-score the existing validation predictions of an experiment using a different (or even same) metric.

Motivation

I regularly run into the use-case that I want to re-score an existing experiment with a different metrics such as different GPT endpoint. As generation is very expensive, it would be helpful to allow to directly re-use the previous experiments outputs and only re-score.

@psinger psinger added the type/feature Feature request label Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature Feature request
Projects
None yet
Development

No branches or pull requests

1 participant