Circular import + reexport = "error: Module '...' has no attribute '..."?

[jstasiak]

This is a minimal test case replicating what I'm experiencing in a larger project:

% tree
.
└── asd
    ├── __init__.py
    ├── one.py
    └── two.py

2 directories, 6 files

% cat asd/__init__.py 
from asd.one import One
from asd.two import Two

% cat asd/one.py 
class One:
    pass

% cat asd/two.py 
from asd import One

class Two:
    pass

Python can work with it:

% python -c 'from asd import Two; print(Two)'
<class 'asd.two.Two'>

mypy, however, doesn't like it:

% mypy .
asd/two.py:1: error: Module 'asd' has no attribute 'One'

It fails all the same when I change __init__.py to use import .. as .. (I saw this pattern mentioned in #3981):

% cat asd/__init__.py 
from asd.one import One as One
from asd.two import Two as Two

Apologies if there's already an issue filled for this, I searched for one briefly and haven't found any.

mypy 0.521, CPython 3.6.2

[ilevkivskyi]

I see the same error on master. There are few other issues related to import cycles. Maybe we can process situation like this or #3277 in the third pass (similar to forward references). @JukkaL what do you think?

[JukkaL]

Yes, creating some sort of forward references for imported names seems like a potential way to fix this. Another thing that might work is improving the processing order of modules within cycles.

We should also make sure that any fix will also work in fine-grained incremental checking.

[bbarker]

Is there a workaround? I tried doing something like the following in database/__init__.py:

from browse.services.database.models import dbx
#...
db: SQLAlchemy = dbx

However, in my test code I still get errors like:

tests/test_database_service.py:22: error: Module has no attribute "db"  

for code like:

    def setUp(self) -> None:
        # ...
        from browse.services import database
        self.database_service = database

        #
        self.database_service.db.init_app(mock_app) #accessing db here causes the error

Edit: I found the discussion on forward references ... so I should look into that next.

[bbarker]

@JukkaL After looking into it more, I now realize you probably meant that forward references might provide a fix internally to mypy - I couldn't see how to use them myself to avoid the error I mentioned above. Is there currently a workaround?

[JukkaL]

@bbarker Yes, I was talking how we could fix this in mypy internally. Your workaround should work, if you access the attribute directly and not through a "module alias":

from browse.services.database import db
db.init_app(...)

You seem to be assigning a module to an instance attribute, which isn't supported right now (#4291).

[carljm]

We're running into this problem at Instagram, too. It's a high priority for us since it's causing type errors we can't easily work around, so I'm looking into it. It seems like we already have a sort of internal "forward reference" for imported names, in the form of node.kind == UNBOUND_IMPORTED. In the problematic cases, we are visiting an import in semantic analysis pass 2 where the name we are importing is still of kind UNBOUND_IMPORTED in the module we are importing it from. Looking into whether we can just let UNBOUND_IMPORTED propagate in pass 2 (instead of immediately erroring) and then resolve it in pass 3.

(It's hard to workaround in our case because the error is raised in generated code, and the code generator has no way to tell whether this particular module will be involved in an import cycle, so it doesn't know whether to generate a # type: ignore comment on the import line; and we consider unused # type: ignore to be an error, too.)