Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SWE-Agent: Implement the hallucinations metric from the main branch in the workflow branch #298

Open
naddeoa opened this issue Apr 21, 2024 · 0 comments

Comments

@naddeoa
Copy link
Collaborator

naddeoa commented Apr 21, 2024

A lot of things have changed between the main branch and the workflow branch. It reimplements all of the metrics from the main branch with a different Workflow, Metric based interface. One of the metrics that have not been ported over yet is Hallucination:

This needs to be ported to the workflow branch and implemented like the other metrics in the workflow brach. Some examples of how that looks:

The metric python module is full of other metrics too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant