Modifiy post processing in chat #698
Conversation
Remove spurious fragments from the suggestion.
While most models (especially instruct and infill models do not require
any pre-processing), some models such as gpt-4 which only have chat APIs
may require removing spurious fragments (such as backticks).
This function uses heuristics and request data to remove such fragments.
It comments out all lines that are not code, irrespective of programming language.
for more information, see https://pre-commit.ci
srdas
added
enhancement
There was a problem hiding this comment.
There are many languages where comments are written differently. This would generate invalid code for these languages.
Why not just modify the previous logic to strip the triple backtick suffix? Why do you want to ignore language? There is a very different use case of generating suggestions for markdown where having triple backtick is desired and of generating Python (or other) code.
|
In general, if an LLM wants to generate comments, it should generate comments as valid language-specific syntax. It is not responsibility of jupyter-ai to transform any spurious text to comments. If jupyter-ai were to do that, one would need to write (and maintain) a very extensive multi-language logic to handle it. Instead, a LLM should be prompted differently (to only return non-code as comments with syntax matching given language). If a given LLM cannot do that, user should be advised that this is not a good LLM for code completions (ideally we would have two lists, one for chat and one for code completion; some models would be on both, but many are fine tuned for one or the other task). |
|
Closing this PR as discussed here: #726 (review) |
Fixes #686
Removed spurious fragments from the suggestion, like backticks
Changes made to packages/jupyter-ai/jupyter_ai/tests/completions/test_handlers.py
Result looks like the following:
