-
-
Notifications
You must be signed in to change notification settings - Fork 332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adopt BNF (via llamacpp) and/or Dspy #1070
Comments
So I'm gonna try to take a crack at this myself It would seem. I'm adding "grammar_json_functions" and "grammar" attribute to the "ChatCompletion" class in Example BNF grammar call to llama.cpp api: So i'm thinking a prompt say like "Break into steps.txt" might look like: Primary objective: {user_input} {introduction} Instructions:
{"grammar": "numbered_list ::= (task_number task_description)+; task_number ::= INTEGER; task_description ::= /[^"]+/;"} Any thoughts/advice? |
Hey @kratosok ! I actually haven't used the grammars. I found an approach the worked before those started being implemented and stuck with it. I am open to moving to grammars since they're trending towards being the standard for that, but I don't currently have any data formatting issues with AGiXT that I am aware of. |
Feature/Improvement Description
I've been using Localai (as an openai replacement) to serve multiple local models for inferencing and came across the idea of BNF grammars in llamacpp. I recall some talk here about the challenges of getting consistent responses from opensource models.
Adopting BNF grammars to constrain responses (See BNF grammars in llamacpp and other projects), or the new DSPy work from Stanford (You might be able to provide a good working conversation flow/examples from say GPT4, to give the DSPy compiler targets to learn how to use any given open source model the same or a similar type response) might be the solution to reliable responses.
One or both of these approaches would push AGIxt to the next level of usability and reliability, in my opinion.
Proposed Solution
See the BNF conversation here:
https://www.reddit.com/r/LocalLLaMA/comments/16wf3w5/ggml_bnf_grammar_the_key_to_autonomous_agents/
Or:
https://github.com/stanfordnlp/dspy
Acknowledgements
The text was updated successfully, but these errors were encountered: