Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating a New Project with a bad key results in null values #601

Open
cfrenchi opened this issue Mar 7, 2024 · 1 comment
Open

Creating a New Project with a bad key results in null values #601

cfrenchi opened this issue Mar 7, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@cfrenchi
Copy link

cfrenchi commented Mar 7, 2024

Describe the bug
After creating a New Project with a bad key for OpenAI GPT-3.5-Turbo, the results are returned as Null. I was expecting to get a bad key response or error. Perhaps rather than"null", the error code or reason is what is captured from the request? Either an invalid_api_key or if/when rate limits are hit a 429?

To Reproduce
Steps to reproduce the behavior:

  1. Create a New Project and fill in all the fields and add a jsonl file for your dataset
  2. Select an OpenAI LLM for Eval
  3. Enter in the key sk (Should there be a key-length check here and warning?)
  4. Click Next
  5. Select any/all metrics
  6. Click Finish
  7. The Evaluations screen shows "null" for your metric score

Should scores be null based on a key failure? Should there be an error reason shown? Null is vague. Is my key wrong or is my system?

Expected behavior
If a bad/expired key is used, the evaluation to reflect that there is a key problem and not present "null"

Screenshots
If applicable, add screenshots to help explain your problem.
image
Seeing null for a bad key is confusing. Where is the error actually happening?

Additional context
Entering in a bad key or having an expired key might lead to confusion when "null" starts to present itself. Should an error code or reason be presented instead?

@cfrenchi cfrenchi added the bug Something isn't working label Mar 7, 2024
@Dominastorm
Copy link
Collaborator

Thanks for raising this. This is currently in our pipeline, and should be fixed in the next dashboard release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants