Document how to create completions using full notebook content #777
Conversation
| prefix = request.prefix | ||
| suffix = request.suffix.strip() | ||
|
|
||
| server_ydoc = self.settings.get("jupyter_server_ydoc", None) |
There was a problem hiding this comment.
The problem is that the provider does not have access to settings. We would need to expose the jupyter_server_ydoc either:
- on provider instances as an optional attribute,
- as an argument to
generate_inline_completionsandstream_inline_completions
if this is a use-case we want to support. Any thoughts?
There was a problem hiding this comment.
Alternatively, jupyter-ai could take the responsibility of extracting the document (YNotebook) and pass it to generate_inline_completions/stream_inline_completions. This way the providers would be restricted to accessing collaborative model of the document they were invoked on (rather than of any document user has open).
There was a problem hiding this comment.
Checking back on this one :)
There was a problem hiding this comment.
If both solutions are fine and either would be accepted, that would be helpful to know too - more than radio silence. I just do not want to work on a PR to learn that the maintainers would not accept it anyways.
There was a problem hiding this comment.
@krassowski Hey Mike, sorry for missing this! I generally don't check in on draft PRs, as I thought the convention was that only open PRs are ready for review. Let me take a look.
There was a problem hiding this comment.
The problem is that the provider does not have access to settings.
I think it makes sense to define settings as an optional instance attribute on BaseProvider, e.g. settings: Optional[Dict[str, any]]. There are other use-cases for chat language models for this as well, so this is a very reasonable addition. This attribute will be useless in the magics (as the server context is not available there), but I think that's fine. The type hint provided here should be sufficient to convey that info to devs. We can also include more details in a docstring for that attribute.
Alternatively, jupyter-ai could take the responsibility of extracting the document
I think we can explore this option later, but this settings API you are proposing should be implemented first to test completion with notebook context. If generated output turns out to be significantly better with notebook context, then I agree this makes sense to implement within Jupyter AI server extension itself rather than in the providers.
jupyter_server_ydoc
I thought this package was renamed entirely to jupyter_collaboration after v1? IIRC jupyter_server_ydoc is essentially the 0.x major version of jupyter_collaboration.
There was a problem hiding this comment.
BTW, in the future, feel free to email me if you'd like to call my attention on anything! My email should be on my GitHub profile.
I get tons of GitHub notifications due to all the repos & orgs I'm in, and GitHub doesn't clearly distinguish between direct pings and random comments on old PRs/issues.
There was a problem hiding this comment.
I thought this package was renamed entirely to
jupyter_collaborationafter v1?
Yes, it was.
IIRC
jupyter_server_ydocis essentially the 0.x major version ofjupyter_collaboration.
It was. Based on regular discussions in jupyter-server calls and in public issues, the jupyter_server_ydoc was resurrected in jupyterlab/jupyter-collaboration#280 to allow for exactly this use case, as well as for using the server-side execution with shared model without forcing users to enable RTC. It is a part of the jupyter-collaboration monorepo, and pre-releases of version 1.0 are on PyPI, e.g. https://pypi.org/project/jupyter-server-ydoc/1.0.0a2/ (so jupyter-collaboration v3 will just depend on jupyter_server_ydoc along few others).
Closes #708