Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MODEL EVALUATION REQUEST] Phi-3-mini-4k-instruct-dansk #427

Open
1 of 8 tasks
emillykkejensen opened this issue May 2, 2024 · 1 comment
Open
1 of 8 tasks

[MODEL EVALUATION REQUEST] Phi-3-mini-4k-instruct-dansk #427

emillykkejensen opened this issue May 2, 2024 · 1 comment
Assignees
@saattrupdan
Labels
model evaluation request Request to evaluate a model and add it to the leaderboard(s) small model (<7B) This model has less than 7B parameters, so can be evaluated on an RTX 4090 GPU or smaller.

Comments

@emillykkejensen
Copy link

Model ID

emillykkejensen/Phi-3-mini-4k-instruct-dansk

Model type

Decoder model (e.g., GPT)

Model languages

  • Danish
  • Swedish
  • Norwegian (Bokmål or Nynorsk)
  • Icelandic
  • Faroese
  • German
  • Dutch
  • English

Merged model

Not a merged model
@emillykkejensen emillykkejensen added the model evaluation request Request to evaluate a model and add it to the leaderboard(s) label May 2, 2024
@saattrupdan saattrupdan self-assigned this May 2, 2024
@saattrupdan saattrupdan added the small model (<7B) This model has less than 7B parameters, so can be evaluated on an RTX 4090 GPU or smaller. label May 2, 2024
@saattrupdan
Copy link
Member

This requires Phi3 to be supported in vLLM. This vLLM PR merged that functionality into main, so waiting for the next release to benchmark it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@saattrupdan saattrupdan

Labels
model evaluation request Request to evaluate a model and add it to the leaderboard(s) small model (<7B) This model has less than 7B parameters, so can be evaluated on an RTX 4090 GPU or smaller.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@emillykkejensen @saattrupdan