Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support disambiguation of scores from metrics that returns same score names #802

Open
yoavkatz opened this issue May 6, 2024 · 2 comments

Comments

@yoavkatz
Copy link
Member

yoavkatz commented May 6, 2024

Today, If metrics, returns the same score (e.g. "f1") then the first metric score is returned, and the second metric is ignored.

metrics ["metrics.bert_score.distilbert_base_uncased","metrics.bert_score.deberta_base_mnli"],

It would be good to be able to add a prefix:

metrics ["metrics.bert_score.distilbert_base_uncased[score_prefix="distilbert_]","metrics.bert_score.deberta_base_mnli[score_prefix="deberta_]"],

The score prefix will be appended to any score returned by the metrics (except score_name, and score which are reserved names).

@elronbandel
Copy link
Member

I think @arielge also dealt with it. @arielge if you have more to add to this issue feel free.

@arielge
Copy link
Member

arielge commented May 6, 2024

Indeed, I encountered this issue in the context of overwrite args - #737, but nothing to add otherwise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants