The debugger seems to break down when a multiprocessing Pool is created and map function is applied.

[NathanSenV]

Could someone explain to me why this won't work. I'm unfamiliar with the technicalities and hope to find a fix to this challenge.

[afshin]

Could you please post a snippet of code that will trigger this behavior? This is likely a behavior of the underlying kernel itself (I'm assuming you are using xeus-python) and not an issue with the debugger front-end.

cc: @JohanMabille

EDIT: It may actually have both front-end and back-end implications.

[NathanSenV]

Thanks for the quick reponse. I guess this may indeed be behaviour triggered by the kernel itself. I prefer not to share the actual code here, but I can share the structure below. Basically we create a Pool of N processors and create a list of tuples as tasks to be performed on a thread. Each list item tuple has function applied to itself. Results are put back into the list and stored in resultsMP.

p = Pool(processes=cpu_count())
resultsMP = p.map(partial(function, parameterA=parameterA), list_to_loop_over)))
p.close()
p.join()

[NathanSenV]

But honestly I don't know what would be expected behaviour. I'm not very familiar with debugging in parallel mode. Building an interface for this would be a whole new challenge I can imagine. Maybe for now it should just jump over any multi processing (?)

[JohanMabille]

The current implementation of xeus-python does not support debugging in multiprocess. Some work has been done in ptvsd to get it work, but this would require some addition "hacking" to get it work in xeus-python. It should work out of the box when we switch to debugpy, so I think it might be a better solution to wait for that switch instead of implementing a hack that will be soon obsolete.

See microsoft/ptvsd#57 and microsoft/ptvsd#1706 for more details about this issue in ptvsd.

I haven't get into the details yet, but enabling this may require additional development in the frontend too, to distinguish the processes.

EDIT: see jupyter-xeus/xeus-python#315 for the switch to debugpy in xeus-python