Auto-generate .pyi files for Model initializers?

[bryevdv]

This discussion is intended to be a constructive version of #13869 The OP in that thread gave no indication of where their actual issue was. I am going to assume that perhaps it was VSCode:

I ran a quick experiment. I manually added the following sources.pyi file to a conda env with Bokeh 3.4.1 installed, to see what might happen in VSCode:

class ColumnDataSource:
    def __init__(
        self,
        *,
        data,
        default_values,
        name,
        selected,
        selection_policy,
        syncable=True,
        tags=[],
    ) -> None: ...

The results in VSCode was markedly better:

Tab completion seems to work as expected as well.

---

This stub is basically the signature that gets dynamically generated for IPython already, just with things like this edited out:

selected=<Instance: Selection()>

We have all this information and can basically produce almost exactly the format needed for these .pyi entries.

So my question: Would it make sense to auto-generate these signatures as part of the build to include in the released packages? In fact if we do that could we also remove the code that supports jupyter and just rely on this for that, too? (I have not run that experiment yet). If that is the case it would be even nicer, as that would reduce import-time work.

Edit: Another alternative could be to generate the stub files manually, and actually check them in to the repo, but have tests in CI that verify the stubs are kept up to date.

It seems to me this might be a reasonable compromise to improve user experience in a meaningful way in IDEs. Another route would be to auto-generate actual code for the Models, or at least for initializers somehow (i.e. real __init__ defs with all the actual real params), but IME that would be an enormous lift.

cc @bokeh/dev

[bryevdv]

There is a relevant discussion regarding a separate question of stubs for sampledata shims here as well: #13874

[bryevdv]

Thinking about it further, maybe generating actual __init__ signatures to check in would not be that bad? And then we have tests that maintain the concrete signature is up to date with the model properties. It would be a semi-manual process to update existing signatures, and the very first time might be especially a bit of a pain. OTOH I think our Python models do not actually change very often these days, so it shouldn't have to happen very often.

[bryevdv]

Well, I guess one issue with actual concrete signatures is that they would have to have real defaults everywhere to make args optional, and these defaults would get legitimately "set" i.e. all the code for setting every property will always get executed, and maybe (probably) that is undesirable. (And dynamic or very complicated defaults, etc. would need some sort of special treatment)