Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error running ERPYes100Trials.param file #315

Open
carolinafernandezp opened this issue Feb 24, 2022 · 8 comments
Open

Error running ERPYes100Trials.param file #315

carolinafernandezp opened this issue Feb 24, 2022 · 8 comments

Comments

@carolinafernandezp
Copy link

Hello, I was following the online ERP tutorial and after I click on run simulation I get the following error

From GUI:
Failed running sim using /home/taylorariko/hnn_out/param/ERPNo100Trials.param. Saved data/figures in: /home/taylorariko/hnn_out/data/ERPNo100Trials

From Terminal:
Warning: could not read file: /home/taylorariko/hnn_out/data/ERPNo100Trials/spk.txt
Starting simulation (8 cores). . .
INFO: Failed starting mpiexec, retrying with 4 cores
Simulation exited with return code 134. Stderr from console:
Invalid MIT-MAGIC-COOKIE-1 key--------------------------------------------------------------------------
mpiexec was unable to find the specified executable file, and therefore
did not launch the job. This error was first reported for process
rank 0; it may have occurred for other processes as well.

@jasmainak
Copy link
Collaborator

Do you have MPI installed ? What do you get if you do:

$ which mpiexec

?

@carolinafernandezp
Copy link
Author

Yes it's installed, I get:
/usr/bin/mpiexec
And when I check the version I get:
mpiexec (OpenRTE) 4.0.3

@jasmainak
Copy link
Collaborator

I see that you also installed hnn-core. Maybe try running this example first:

https://jonescompneurolab.github.io/hnn-core/stable/auto_examples/howto/plot_simulate_mpi_backend.html#sphx-glr-auto-examples-howto-plot-simulate-mpi-backend-py

to see if that works or if you see the same error there?

@rythorpe
Copy link
Contributor

I suspect this might be an mpi4py or python environment issue.

  1. Is it possible that you have two versions of OpenMPI installed, one at /usr/bin and another within your Anaconda environment (assuming you're using Anaconda)?
  2. Do you have mpi4py installed in your Anaconda environment? If not, try
$ conda install mpi4py

@carolinafernandezp
Copy link
Author

carolinafernandezp commented Mar 1, 2022

I did a full wipe of the computer and reinstalled the hnn GUI, so the only thing installed in the computer at the moment is HNN (anaconda is not installed).

I also made sure that mpi4py is installed.

But I'm getting the same error:

Warning: could not read file: /home/joshuabruce/hnn_out/data/default/dpl.txt
Warning: could not read file: /home/joshuabruce/hnn_out/data/default/spk.txt
Warning: could not read file: /home/joshuabruce/hnn_out/data/default/dpl.txt
Warning: could not read file: /home/joshuabruce/hnn_out/data/default/spk.txt
Starting simulation (8 cores). . .
INFO: Failed starting mpiexec, retrying with 4 cores
Simulation exited with return code 134. Stderr from console:
Invalid MIT-MAGIC-COOKIE-1 key--------------------------------------------------------------------------
mpiexec was unable to find the specified executable file, and therefore
did not launch the job. This error was first reported for process
rank 0; it may have occurred for other processes as well.

NOTE: A common cause for this error is misspelling a mpiexec command
line parameter option (remember that mpiexec interprets the first
unrecognized command line token as the executable).

Node: pop-os
Executable: nrniv

Warning: could not read file: /home/joshuabruce/hnn_out/data/default/dpl.txt
Warning: could not read file: /home/joshuabruce/hnn_out/data/default/spk.txt
Warning: could not read file: /home/joshuabruce/hnn_out/data/default/dpl.txt
Warning: could not read file: /home/joshuabruce/hnn_out/data/default/spk.txt

Any ideas on what could be going on?

@jasmainak
Copy link
Collaborator

I would really suggest trying to run the hnn-core example that uses MPI first. That way you can rule out problems due to something specific in the GUI code. I don't know if anyone is actively maintaining the GUI code, so it's easier for us to start with HNN-core to troubleshoot the problem.

See:

https://jonescompneurolab.github.io/hnn-core/stable/parallel.html

What OS are you using?

@darcywaller
Copy link

Hi @jasmainak and @rythorpe, I wanted to follow up on an error I just encountered and found a workaround for which generated nearly identical error messages to the one described here.

I've noticed that HNN displays buggy behavior when trying to load a simulation that has previously been canceled manually or by killing the terminal running HNN GUI during the run, which I always assumed is due to corruption of the associated simulation output files in the hnn_out/data folder, which it still tries to load the next time the associated parameter file is loaded. We solved this by renaming the associated folder in hnn_out/data, so it didn't attempt to load the output files anymore and were able to get it to load in the parameter file again (albeit with no ouput yet until the sim is run). I don't know if you think this constitutes an issue with the manual shutdown option of an ongoing simulation that you would like me to open an issue for or not (if it doesn't exist already).

@jasmainak
Copy link
Collaborator

jasmainak commented Jun 9, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants