Error in fnn test

[carlos-hoyos]

I wrote a fnn test:

tau=3
x = np.linspace(0, 10, 1000)
dim = np.arange(1, max_dim)
f1, f2, f3 = dimension.fnn(x, tau=tau, dim=dim)
print(f1)

Program doesn't reach print(f1), instead gives the following error after getting stuck for several seconds:

File "C:\Users\Carlos\PycharmProjects\parkinson.venv\Lib\site-packages\nolitsa\dimension.py", line 187, in fnn
r = utils.parallel_map(_fnn, dim, (x,), {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\PycharmProjects\parkinson.venv\Lib\site-packages\nolitsa\utils.py", line 247, in parallel_map
pool = Pool(processes=processes)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 215, in init
self._repopulate_pool()
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 306, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 329, in _repopulate_pool_static
w.start()
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 337, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\popen_spawn_win32.py", line 45, in init
prep_data = spawn.get_preparation_data(process_obj._name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 164, in get_preparation_data
_check_not_importing_main()
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 140, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

    To fix this issue, refer to the "Safe importing of main module"
    section in https://docs.python.org/3/library/multiprocessing.html

[manu-mannattil]

I'm not sure. It seems to work for me, and I get `[0. 0. 0. 0. 0. 0. 0. 0. 0.]` as the output. Could you try running the examples in the examples/fnn directory?

[carlos-hoyos]

Running the example henon.py I get the same errors. Could it be that I'm running the program in Windows 11?

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 122, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 131, in _main
prepare(preparation_data)
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 246, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 297, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 286, in run_path
File "", line 98, in _run_module_code
File "", line 88, in _run_code
File "C:\Users\Carlos\PycharmProjects\parkinson.venv\henon.py", line 18, in
f1, f2, f3 = dimension.fnn(x, tau=1, dim=dim, window=10, metric='cityblock')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\PycharmProjects\parkinson.venv\Lib\site-packages\nolitsa\dimension.py", line 187, in fnn
r = utils.parallel_map(_fnn, dim, (x,), {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\PycharmProjects\parkinson.venv\Lib\site-packages\nolitsa\utils.py", line 247, in parallel_map
pool = Pool(processes=processes)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 215, in init
self._repopulate_pool()
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 306, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 329, in _repopulate_pool_static
w.start()
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 337, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\popen_spawn_win32.py", line 45, in init
prep_data = spawn.get_preparation_data(process_obj._name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 164, in get_preparation_data
_check_not_importing_main()
File "C:\Users\Carlos\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 140, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

    To fix this issue, refer to the "Safe importing of main module"
    section in https://docs.python.org/3/library/multiprocessing.html

[manu-mannattil]

Yes, I think so. The multiprocessing module seems to have issues with Windows [1]. Let me see if there's a Windows-specific fix. [1] https://stackoverflow.com/q/20360686

[manu-mannattil]

I don't have Windows to test this, but could you try running the following two versions of your script?

import numpy as np
from nolitsa import dimension

if __name__ == '__main__':
    x = np.linspace(0, 10, 1000)
    print(dimension.fnn(x, tau=3, dim=np.arange(1, 5)))

and

import numpy as np
from nolitsa import dimension

if __name__ != '__main__':
    exit()

x = np.linspace(0, 10, 1000)
print(dimension.fnn(x, tau=3, dim=np.arange(1, 5)))

[carlos-hoyos]

The first one works! Thank you.

[manu-mannattil]

That's good to know. The issues arises because Python's multiprocess module works differently on Windows and *nix. So, to use NoLiTSA functions that make use of multiprocessing on Windows, you'll have to wrap your entire code in a if __name__ == '__main_' block. I would've preferred a cleaner workaround, but I don't know if there's one. I'll leave this issue open in case someone else has a better fix.