Use plugin to support Django models

[JukkaL]

We'd also need stubs for Django. Django is very popular so having the plugin live in the mypy repo would be reasonable.

[JukkaL]

This was originally discussed at #1240.

[ilevkivskyi]

@dmoisset You might be interested in this.

[decentral1se]

Hi folks, there is some interest coming into helping out on Django stubs (we're organising over at https://gitter.im/mypy-django/Lobby if you would like to chat) although I am pretty confused as to what direction to kick off with. Some questions ...

1. Is what you are proposing in this ticket that if stubs existed, they could be included in this repository and wired in using some internal plugin architecture of mypy?

2. If Django stubs were to exist, would you suggest that they live in https://github.com/python/typeshed? From some reading, I see that https://www.python.org/dev/peps/pep-0561/ provides a new way to distribute stubs which may be preferable.

Thanks!

PS. @dmoisset, we're basing our initial investigation off your great start, please join the chat if you're around to hand out some advice, it would be much appreciated!

[emmatyping]

Hi @lwm,

Regardless of what solution is chosen, I strongly recommend, and hope we can all agree, that keeping the stubs and the plugin together will make everything simpler.

Here are the two best options I see:

1. There is a django-stubs package which maintains stubs and a plugin for django.

2. The stubs are put in typeshed and a plugin is added to mypy itself.

I recommend option 1, the stubs and plugin are put in a django-stubs package. #5358 will help make loading the plugin. While many people use Django, it would reduce the maintenance work by the mypy team and allow for faster iteration on the stubs.

[decentral1se]

Thanks @ethanhs, happy to go with your recommended approach. I'm asking a heck of a lot of questions but I'll keep going - what is meant by 'a plugin for django'? I'm not quite sure what is the purpose of the plugin and why we need it.

EDIT: I've added typeddjango/django-stubs#5 to track more specifics if needed.

[emmatyping]

@lwm the idea is that Django models do some rather dynamic things that mypy does not, nor will not in the near future, support. Therefore, if there is a plugin for mypy which knows about how these models work, people can still get some type safety when using mypy in conjunction with Django. There is a plugin for attrs which lives in the mypy sources which you may be interested in.

[decentral1se]

Thanks @ethanhs, I'll give that existing plugin a look over and see how I go 👍

[emmatyping]

Great, let me know (here or on Gitter) if you have questions. There is usually someone around who can answer your questions on the typing Gitter.

[willmcgugan]

Hi,

I've been looking in to typing Django models myself, which would be a massive benefit for my client's project. The plugin approach seems the clear winner, and I would love to see that in mypy. But I understand its quite challenging and might take some time.

In the meantime, I've been trying to put together another solution that might work for us (and other projects). It's just a days work, but I've fudged together a script which can import Django models, inspect them and output .pyi files. It's not quite finished, but in theory it should cover all the field attributes and the dynamic attributes Django adds.

I was wondering if anyone has tried such an approach and if its worth persevering?

Somewhat related is an issue I had with the inspect lib, which I use to extract method signatures from the models. The Signature object turns forward references in to something like the following:

-> Union[_ForwardRef('DeviceMemory'), NoneType]

Which mypy rejects, because of the use of parenthesis in _ForwardRef. Is that a bug in the inspect module? Replacing that with a regex, feels like a hack--I'm hoping there is a workaround.

[ilevkivskyi]

@willmcgugan This is an interesting approach, somewhere in between stubgen (a basic static stub generator, comes with mypy) and PyAnnotate (collecting types from function calls at runtime). I am not aware of anyone else doing this, but there is a project https://github.com/TypedDjango/django-stubs, so you might want to coordinate your efforts with them.

Regarding your question, this is caused by the fact that all string literals that appear as type parameters are internally wrapped in a special class typing._ForwardRef. What you see is its standard repr(). There are several possible solutions, the best one is probably to use typing.get_type_hints() instead (it requires a namespace argument where to evaluate the forward references).

[ilevkivskyi]

I think this can be closed, there is a Django plugin actively developed in a separate repo: https://github.com/mkurnikov/django-stubs. All further discussion about supporting various Django features could continue there.