Add missing `Type Hints` to torch. issue #7318 #8845

kimdwkimdw · 2018-06-25T08:37:27Z

#7318
Trying to fix #7318.
I fixed all issues except PY2.

torch/_torch_docs.py

@@ -68,7 +69,7 @@ def parse_kwargs(desc):
        returned tensor. Default: ``False``.
 """)

-add_docstr(torch.abs,
+add_docstr(_C.abs,


tools/autograd/templates/python_torch_functions.cpp

+  {"saddmm", (PyCFunction)THPVariable_sspaddmm, METH_VARARGS | METH_KEYWORDS, NULL},
+  {"sparse_coo_tensor", (PyCFunction)THPVariable_sparse_coo_tensor, METH_VARARGS | METH_KEYWORDS, NULL},
+  {"spmm", (PyCFunction)THPVariable_mm, METH_VARARGS | METH_KEYWORDS, NULL},
+  {"tensor", (PyCFunction)THPVariable_tensor, METH_VARARGS | METH_KEYWORDS, NULL},


tools/autograd/gen_python_functions.py

@@ -186,7 +186,7 @@ def should_bind(declaration):

    py_torch_functions = group_declarations_by_name(declarations, should_bind)

-    env = create_python_bindings(py_torch_functions, has_self=False)
+    env = create_python_bindings(py_torch_functions, has_self=False, is_module=True)


torch/__init__.py

@@ -186,6 +186,7 @@ def set_default_dtype(d):
 from .tensor import Tensor
 from .storage import _StorageBase

+tensor = _C.tensor  # remove duplicate name, and fix to shortcut function


https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-macos-10.13-py3-test1/1965/console

…ional

zou3519 · 2018-06-25T15:24:17Z

Thank you for the PR, @kimdwkimdw! What is the general strategy for this PR? From what I can tell it looks like you're removing METH_STATIC, but why does that affect how pycharm can read the code?

kimdwkimdw · 2018-06-25T15:55:04Z

@zou3519 , Thank you for comment!

First, here is my concept of gist,
https://gist.github.com/kimdwkimdw/50c18b5cf72c69c2d01bb4146c8a2b5c

From 0.4.0, all methods in torch_functions goes to _C._VariableFunctions class.
With PyModule_AddObject, all methods are instance methods. Thats why we need below lines.

# torch/__init__.py
for name in dir(_C._VariableFunctions):
    globals()[name] = getattr(_C._VariableFunctions, name)

The shorthand of above is that all methods are not imported with pythonic way.

I want to import all methods via below code.

# torch/__init__.py
from torch._C import *

https://docs.python.org/3/c-api/structures.html#METH_STATIC
With METH_STATIC, C Extension will generate kind of static methods. However, we don't need @staticmethod (https://docs.python.org/3/library/functions.html#staticmethod) for method forwarding. We just only need module methods.

When we import module methods like below,

# torch/__init__.py
from torch._C import *

IDE will run __init__.py internally, and generate stub files following PEP-0484

There is an important sentence.

Modules and variables imported into the stub are not considered exported from the stub unless the import uses the import ... as ... form or the equivalent from ... import ... as ... form.
...
..
..
Stub files may be incomplete. To make type checkers aware of this, the file can contain the following code:  def __getattr__(name) -> Any: .

I read all codes related in pytorch-0.3.0, pytorch-0.4.0, pytorch-master.
I think that this solution is not the best solution, but it is working and helpful for now.

tools/autograd/gen_python_functions.py

@@ -186,7 +189,10 @@ def should_bind(declaration):

    py_torch_functions = group_declarations_by_name(declarations, should_bind)

-    env = create_python_bindings(py_torch_functions, has_self=False)
+    if PY2:


zou3519 · 2018-06-26T14:30:35Z

@kimdwkimdw I'm not very familiar with what we're doing with METH_STATIC and what PyModule_AddFunctions is doing. Someone (probably me) will have to sit down and read the specs for these things for a bit, I will get back to you soon after I understand this a little more

kimdwkimdw · 2018-06-26T16:19:05Z

@zou3519 👍 I'll think about better solution and substitute for PY2.

zou3519 · 2018-07-09T17:15:50Z

Sorry for the delay in reviewing, @kimdwkimdw. I will get to it soon

zou3519 · 2018-07-17T19:41:36Z

@kimdwkimdw I wonder if making _C._VariableFunctions a module and then creating the torch_functions on the _C._VariableFunctions module will fix the problem for both Python 2 and Python 3. I'll try to look more into it, but what do you think?

i.e.

#if PY_MAJOR_VERSION >= 3
    m = PyModule_Create(&moduledef);
#else
    m = Py_InitModule3("themodulename",

kimdwkimdw · 2018-07-20T05:33:01Z

@zou3519 I'm not sure. But, I don't think that can solve its problem easily.

Originally, pytorch 0.3.x have all torch functions in torch._C with PyModule_Create and Py_InitModule3.
However, pytorch 0.4.0 divided torch._C into multiple objects like _C._VariableFunctions.

I think we can think multiple solutions

Move all code to torch._C gracefully for Python 2 like pytorch 0.3.0.
All torch functions and other functions are added with PyModule_AddObject, not Module.

Too lots of code change are needed

Make multiple C Extensions and connect them. Then we can use your suggestion

#if PY_MAJOR_VERSION >= 3
    m = PyModule_Create(&moduledef);
#else
    m = Py_InitModule3("themodulename",

Also too lots of code change are needed

Merge it, and wait for deprecation of PY2. https://pythonclock.org/ lol.

zou3519 · 2018-08-14T16:32:02Z

I tried (2), it's a little painful to get it to work.
For (3), RHEL announced support for Py2 until 2022 or 2024 so it is not going away anytime soon.

For (1), we have this problem that we're dumping all functions into torch._C. It would be nice to do some namespacing and put the variable functions into torch._C.VariableFunctions or something, where VariableFunctions is not a class.

HWiese1980 · 2018-10-08T08:49:48Z

Is this going to be merged anytime soon?

sproshev · 2018-10-23T16:40:09Z

Generated /torch/_C/__init__.py is much longer (7600 LOC vs 700) after applying this PR.

We're using https://github.com/JetBrains/intellij-community/blob/master/python/helpers/generator3.py to generate stubs for binaries. I have not investigated yet what is the difference between doing so for released pytorch and pytorch with changes in this PR applied.

niushencheng · 2018-11-10T10:49:28Z

anyone can help solve this issue? I almost give up pytorch for the boken type hints

fmassa · 2018-11-11T01:24:19Z

@t-vi has a pending PR in #12500 that is a step further to have type hints work for PyTorch.

MarkVdBergh · 2018-12-11T18:12:36Z

I'm having the same type hints problem with pytorch 1.0.0
Will this solution work with that version?

t-vi · 2018-12-13T08:10:52Z

I have a workshop coming up where people want to use them, so I'll refresh my PR for the typehints before early next week. I'll send a link for the PYI for 1.0 when I have it.

ezyang · 2019-01-30T20:28:03Z

This PR was subsumed by @t-vi's PR #12500, which has been merged to master. I'm going to close this PR.

@kimdwkimdw, thank you for your hard work here, and your patience. It might still be worth incorporating some of the ideas from this patch when we (some day in the unforseeable future) become Python 3 only. For example, it seems like a generally good idea to bind functions as functions, not static methods. I've filed #16573 to keep track of this.

Add missing Type Hints to torch. issue pytorch#7318

6cebcdb

kimdwkimdw requested review from apaszke, colesbury, ezyang, gchanan, soumith and zdevito as code owners June 25, 2018 08:37

kimdwkimdw mentioned this pull request Jun 25, 2018

Broken Type Hints in PyTorch 0.4.0, related to IDEs(eq. PyCharm) #7318

Closed

apaszke reviewed Jun 25, 2018

View reviewed changes

reversed commit _C -> torch add_docstr in _torch_docs.py

9650c42

kimdwkimdw commented Jun 25, 2018

View reviewed changes

kimdwkimdw added 4 commits June 25, 2018 19:24

resolve conflict in torch/__init__.py#L82 (__all__)

086ae9f

Fix issues about flake8 in travis-ci.org

e6d7c33

fix error with 'torch' has no attribute '_* from _C

cb83f16

https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-macos-10.13-py3-test1/1965/console

fix failure of builds about private method in torch_functions_addit…

8c0f11e

…ional

fix tests for PY2

c26dcca

kimdwkimdw commented Jun 26, 2018

View reviewed changes

tools/autograd/gen_python_functions.py

@@ -186,7 +189,10 @@ def should_bind(declaration):

py_torch_functions = group_declarations_by_name(declarations, should_bind)

env = create_python_bindings(py_torch_functions, has_self=False)

if PY2:

This comment was marked as off-topic.

Sign in to view

zou3519 self-assigned this Jul 9, 2018

ezyang mentioned this pull request Jan 30, 2019

Stop generating VariableFunctions as a class with static functions; instead, generate it as a module with functions #16573

Closed

ezyang closed this Jan 30, 2019

ezyang added the open source label Jun 24, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add missing `Type Hints` to torch. issue #7318 #8845

Add missing `Type Hints` to torch. issue #7318 #8845

kimdwkimdw commented Jun 25, 2018 •

edited

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

zou3519 commented Jun 25, 2018

kimdwkimdw commented Jun 25, 2018 •

edited

This comment was marked as off-topic.

zou3519 commented Jun 26, 2018

kimdwkimdw commented Jun 26, 2018

zou3519 commented Jul 9, 2018

zou3519 commented Jul 17, 2018 •

edited

kimdwkimdw commented Jul 20, 2018

zou3519 commented Aug 14, 2018

HWiese1980 commented Oct 8, 2018

sproshev commented Oct 23, 2018

niushencheng commented Nov 10, 2018

fmassa commented Nov 11, 2018

MarkVdBergh commented Dec 11, 2018

t-vi commented Dec 13, 2018

ezyang commented Jan 30, 2019

Add missing Type Hints to torch. issue #7318 #8845

Add missing Type Hints to torch. issue #7318 #8845

Conversation

kimdwkimdw commented Jun 25, 2018 • edited

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

zou3519 commented Jun 25, 2018

kimdwkimdw commented Jun 25, 2018 • edited

This comment was marked as off-topic.

zou3519 commented Jun 26, 2018

kimdwkimdw commented Jun 26, 2018

zou3519 commented Jul 9, 2018

zou3519 commented Jul 17, 2018 • edited

kimdwkimdw commented Jul 20, 2018

zou3519 commented Aug 14, 2018

HWiese1980 commented Oct 8, 2018

sproshev commented Oct 23, 2018

niushencheng commented Nov 10, 2018

fmassa commented Nov 11, 2018

MarkVdBergh commented Dec 11, 2018

t-vi commented Dec 13, 2018

ezyang commented Jan 30, 2019

Add missing `Type Hints` to torch. issue #7318 #8845

Add missing `Type Hints` to torch. issue #7318 #8845

kimdwkimdw commented Jun 25, 2018 •

edited

kimdwkimdw commented Jun 25, 2018 •

edited

zou3519 commented Jul 17, 2018 •

edited