External file loading problem with pdpipe import

[shaypal5]

Discussed in #97

^{Originally posted by Dranikf March 14, 2022}
Will start with example. I create test.py file in some dir, it may looks like this:

from sys import path
import pdpipe

path.append("<path to external folder>/some_external_folder")
import temp

In ".../some_external_folder" i have added temp.py file, which contains 'hello world' printing. When I try to run first file i have a error:

Traceback (most recent call last):
  File "<first file folder path>/test.py", line 6, in <module>
    import temp
ModuleNotFoundError: No module named 'temp'

But when i rem import pdpipe program starts. What can cause this behavior and how to correct it?

[shaypal5]

Clues:

1. https://justus.science/blog/2015/04/19/sys.modules-is-dangerous.html
2. sys.modules: dictionary changed size during iteration python/cpython#89516
3. https://www.oreilly.com/library/view/python-cookbook/0596001673/ch14s02.html
4. This package also looks at sys.modules directly on init:
   https://github.com/searxng/searxng/pull/
5. https://tenthousandmeters.com/blog/python-behind-the-scenes-11-how-the-python-import-system-works/

[shaypal5]

Update: Even when I delete the only piece of my code using sys, the error presists. :/

See this method:

pdpipe/pdpipe/core.py

Line 62 in ebd4153

def __load_stage_attributes_from_module__(module_name: object) -> None:

[DavidKatz-il]

This one is working fine

import pdpipe
from sys import path

path.append("<path to external folder>/some_external_folder")

import temp

This one is raising ModuleNotFoundError

from sys import path
import pdpipe

path.append("<path to external folder>/some_external_folder")

import temp

This is due to the import of the nltk package here.
The above error was not raised after removing this import.

[DavidKatz-il]

import sys

# Hack to keep NLTK's "tokenize" module from colliding with the "tokenize" in
# the Python standard library.
OLD_SYS_PATH = sys.path[:]
sys.path = [p for p in sys.path if p and "nltk" not in str(p)]
import inspect

sys.path = OLD_SYS_PATH

https://github.com/nltk/nltk/blob/develop/nltk/decorators.py#L18-L26