Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement Request: Ability to call chain manager by external function / module #9

Open
AdityaSavara opened this issue Oct 13, 2020 · 0 comments

Comments

@AdityaSavara
Copy link

AdityaSavara commented Oct 13, 2020

I was able to get the test_mpi.py example to work, but I have not been able to get the chain manager to work when trying to use it inside a function or importing it. This functionality is desirable so that people can use custom run files or make other packages that use zeus as a dependency (which is what I am doing).

I am providing example files and how to reproduce the behaviour.


In the attached files 4.0MPITest.zip
This works:
mpiexec -n 5 python test_mpi.py

These below 3 do not work. They run, but they do not do have the correct behavior, they do not actually seem to initiate walkers and they do not export the expected ".npy" files.

mpiexec -n 5 python test_mpi_by_function1.py
mpiexec -n 5 python test_mpi_by_function2.py
mpiexec -n 5 python test_mpi_by_function3.py

The below 4th case runs for a bit , then 'stalls' or crashes with message that it can't pickle the object. This is when using zeus 2.0 (I don't think the below 4th case crashed before zeus 2.0, but I am not sure anymore).
mpiexec -n 5 python test_mpi_by_function4.py


Proposed solution: Unknown. But it would be nice if a clever way can be found to allow chain manager to work inside functions and when imported elsewhere. At present, it is rather puzzling to me that it doesn't work.

At the moment I am just going to use MPI externally. For example, if I have 8 processors, I will do 8 entirely separate EnsembleSampling and then combine the results. This is not ideal for a variety of reasons. I thought about generating a custom python file each time and running mpi from the command line, but that is not a suitable solution for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant