feat: llama3 in studio #23307
Draft
feat: llama3 in studio #23307
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
No changes detected in Branching Preview Branches by Supabase. |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Initial work to Bring-your-own-AI™ to Supabase studio. Requires an OpenAI-compatible API. Tested with Llama3 via Groq.
Changes
In addition to the existing
OPENAI_API_KEYenv var, there are now 2 more:OPENAI_BASE_URL: point this to your openai-compatible API, eg https://api.groq.com/openai/v1OPENAI_MODEL: choose the language model, currently supports llama3, gpt-3.5, and gpt-4In order for clippy to continue to work, we also added these env vars:
OPENAI_EMBEDDING_API_KEYOPENAI_EMBEDDING_BASE_URLOPENAI_EMBEDDING_MODELThese are necessary if we want to be able to eg. use groq + llama3 for the language model and openai + text-embedding-ada-002 for the embedding model.
Other
Also did some work on the tokenizer to support open models like llama3 (we use the tokenizer to count tokens before sending them to the model).