Prevent importing of local modules when starting a new Notebook

[pya]

Problem

A new Notebook starts in the current working directory. This means a file named logging.py in my working directory shadows the module logging of the standard library. Accordingly, I get this error message on the console from which I started the server:

concurrent/futures/_base.py", line 43, in <module>
    LOGGER = logging.getLogger("concurrent.futures")
AttributeError: module 'logging' has no attribute 'getLogger'

The Notebook cannot connect to a kernel. After deleting logging.py, creating a new Notebook works again without problem. So the obvious solutions is not name own modules with names of module of the standard library, which is alway a good idea. There are few problems with this solutions:

1. Notebooks are often used by beginners who may not be aware of this name shadowing problem.
2. The error message appears only on the console. So you need to look at the terminal to find the problem.
3. Starting a Notebook in JupyterHub has the same problem. Since there is no visible terminal, you need to check the log file to find the error message.

Proposed Solution

These maybe solutions:

1. Temporally modify sys.path so that the current directory is not in the search path. Set sys.path back to its original state after all relevant imports.
2. If this is not feasible, maybe checking that imported module are not from the current directory can help. For example, for module mod do a assert Path(mod.__file__).parent != Path().resolve(). The error message needs to be shown in the UI.

Additional context

Tested with JupyterLab version 3.3.4.

[agoose77]

This is mainly an ipykernel discussion, for that is where code (and the module search path) is executed (considered). In particular, it's a Python-specific problem that JupyterLab can't really do too much to solve (in my opinion).

[krassowski]

I think that we should try to address it in JupyterLab, as this problem affects many novice users (based on frequency of this question on StackOverflow)discouraging them from using JupyterLab.

While we do not explicitly handle the PYTHONPATH in JupterLab today (although we do in JupterLab Desktop), maybe we should. If we go this route, changes would probably need to be implemented in both ipykernel and JupyterLab. There are other, related requests to allow handling of PYTHONPATH in JupyterLab, for example: #11619 asking about a way to set the PYTHONPATH to JupyterLab root (which is a feature present in VSCode, but not in JupyterLab).

Other editors also try to make the experience better for novice users (although there are conflicting opinions on how to handle it best), for example Spyder: spyder-ide/spyder#17066.

[agoose77]

Thanks @krassowski. What I meant here was that, as it stands, JupyterLab can't do anything - it doesn't intrinsically care about what kernel the user is running. If we want to special-case something for ipykernel, then we might be able to first modify ipykernel to accept an option, and then JupyterLab to provide it to the kernel environment.

[krassowski]

There is another one - when you open a debugger with ipykernel having a file named code.py in jupyter root you will see this (and kernel die, but no message on the front):

  File "/3.9.5/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/3.9.5/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/ipykernel_launcher.py", line 15, in <module>
    from ipykernel import kernelapp as app
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 51, in <module>
    from .ipkernel import IPythonKernel
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 19, in <module>
    from .debugger import Debugger, _is_debugpy_available
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/ipykernel/debugger.py", line 22, in <module>
    from debugpy.server import api  # noqa
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/debugpy/server/__init__.py", line 7, in <module>
    import debugpy._vendored.force_pydevd  # noqa
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/debugpy/_vendored/force_pydevd.py", line 28, in <module>
    pydevd_constants = import_module('_pydevd_bundle.pydevd_constants')
  File "/3.9.5/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_constants.py", line 379, in <module>
    from _pydev_bundle._pydev_saved_modules import thread, threading
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/debugpy/_vendored/pydevd/_pydev_bundle/_pydev_saved_modules.py", line 91, in <module>
    import code as _code;    verify_shadowed.check(_code, ['compile_command', 'InteractiveInterpreter'])
  File "/3.9.5/envs/ipython/lib/python3.9/site-packages/debugpy/_vendored/pydevd/_pydev_bundle/_pydev_saved_modules.py", line 75, in check
    raise DebuggerInitializationError(msg)
_pydev_bundle._pydev_saved_modules.DebuggerInitializationError: It was not possible to initialize the debugger due to a module name conflict.

i.e.: the module "code" could not be imported because it is shadowed by:
/path/to/root/code.py
Please rename this file/folder so that the original module from the standard library can be imported.

This is actually hard to recover from (cf #13069, other than removing the code.py file).

[Carreau]

I don't believe this is in JupyterLab, it is either in Jupyter Client, or in ipykernel.
Former one if we explicitly pass CWD as a place to start executing – which I doubt we do,
In the second one we can filter CWD (as we already do, but we filter without expanding the path, so we get false negatives).

[krassowski]

Given a number of reports I do think we need to do something about PYTHONPATH (for ipykernel or in general) as this issue recurs frequently, with just some of the examples reports:

• logging.py: Prevent importing of local modules when starting a new Notebook #12914 (this issue)
• code.py: Cannot start up a kernel when debugger initialization fails (cannot disable debugger) #13069
• decorator.py: ipython module name conflicts with user created file leading to import error #15268
• html.py: Under certain conditions, the kernel fails to start #15559

This is particularly erogenous for new users, and exacerbates with time as more dependencies get added to the kernels and any server-side code which may get broken in that way (but on startup rather than on kernel launch).