Skip to content

Issues: confident-ai/deepeval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

42 Open 145 Closed
42 Open 145 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Suggestion for Future Improvement: Enable Function Calling for Consistent JSON Output
#790 opened May 24, 2024 by BarendPotijk
RagasMetric breaks when used with dataset
#782 opened May 21, 2024 by haarisedhi102
1
Synthesizer generates goldens but a dataset loads as test cases
#779 opened May 21, 2024 by penguine-ip
Dramatically simpler and more reliable cache
#775 opened May 18, 2024 by prescod
Remove unconditional, uncontrollable output statements
#769 opened May 15, 2024 by prescod
Corrupted cache can make it impossible to run deepeval
#768 opened May 15, 2024 by prescod
.generate_raw_response(prompt) gives an attribute error
#767 opened May 14, 2024 by rtzy7
2
Cant use HuggingFace Model for evaluation
#743 opened May 6, 2024 by Kraebs
4
No test cases found error
#739 opened May 5, 2024 by GantaVenkataKousik
4
Score is not cached
#734 opened Apr 30, 2024 by prescod
Return type of evaluate is not documented.
#733 opened Apr 29, 2024 by prescod
Show me the prompt!
#731 opened Apr 29, 2024 by prescod
2
Hallucination test is really a test for faithful summarization, not of hallucination-absence
#730 opened Apr 29, 2024 by prescod
Support batching for custom models
#702 opened Apr 19, 2024 by alexkreidler
8
Writing of any files like cache or temp to running directory - Disable or Provide Config to set location
#671 opened Apr 9, 2024 by kubre
1
Add browser opening flag to config
#645 opened Mar 31, 2024 by agamm
1
Make evaluate() run test cases concurrently
#642 opened Mar 29, 2024 by penguine-ip
@penguine-ip
temp_test_run.json contains partial entries when running in bulk
#605 opened Mar 16, 2024 by piseabhijeet
1
Consideration behind a "stateful" metric UX
#597 opened Mar 14, 2024 by Peilun-Li
@penguine-ip
9
Store computed metrics in a dataframe
#595 opened Mar 13, 2024 by piseabhijeet
1
Custom dataset evaluation giving ValueError: Input, actual output, and retrieval context cannot be None
#568 opened Mar 6, 2024 by piseabhijeet
13
Summarization metric generates <1 score for exactly the same input and output
#565 opened Mar 5, 2024 by Peilun-Li
1
Make multithreading optional
#538 opened Feb 27, 2024 by AndresPrez
6
Evaluation Dataset Generation enhancement New feature or request help wanted Extra attention is needed
#530 opened Feb 26, 2024 by penguine-ip
@penguine-ip
1
Support popular benchmarks enhancement New feature or request help wanted Extra attention is needed
#508 opened Feb 22, 2024 by penguine-ip
3
ProTip! Follow long discussions with comments:>50.