Integrate hnn-core for running simulations

[blakecaldwell]

We have been internally discussing this WIP for a few months; time to put it up on github.

The hnn-core project is a more flexible way to run NEURON (Python) simulations than was originally coded in run.py. It allows users to create, modify, and read native Python objects representing the parameters, network construction, simulation settings, and simulation results. There has been significant effort put into documentation, testing, and best-practices as part of hnn-core.

Our plan is to remove certain sections of code in HNN and migrate them to hnn-core. This will begin with the code reading the parameter files, running the simulation, and saving spiking and dipole data. Then HNN will import the hnn_core module to read parameter files, running the simulation, saving dipoles, and saving spiking data. Lastly, code in HNN that has been replaced by code in hnn_core as part of this first step needs to be removed.

Code changes part of this integration will not change the GUI functionality. Ideally, the transition will be transparent.

Work remaining:

• Updating existing param files that use -1 for an input time (see warn user of invalid poisson input timing specification hnn-core#180)
• Add 'spec_cmap' parameter to hnn-core (Add 'spec_cmap' paramter for use by HNN GUI hnn-core#184)
• Print simulation status with MPI (Print simulation progress messages to screen with MPIBackend hnn-core#112)
• Update HNN documentation to use pip install neuron and reference a tagged HNN release @blakecaldwell
• Somatic voltages (depends on: Implement mechanism to record somatic voltages for all cells in a simulation hnn-core#181) @ntolley
• Somatic currents (depends on: Make cell currents a model output hnn-core#187) @ntolley
• Print status messages from hnn-core to the simulation log window (part of [MRG] hnn-core integration #232) @blakecaldwell
• Save rawdpl files (depends on: access raw dipole for each trial hnn-core#183) @kohl-carmen
• Handle empty spike files (depends on: read_spikes doesn't handle an empty file properly hnn-core#178) @rythorpe
• Tonic inputs (depends on: MRG ENH: start adding tonic input hnn-core#209) @jasmainak
• Integrate hnn-core required functionality into [MRG] hnn-core integration #232 @blakecaldwell
  • Almost complete. A few tasks left in [MRG] hnn-core integration #232
• Verify trial seeding and results between HNN and hnn-core @rythorpe (depends on: multiple trials with consistent seeding hnn-core#113)
• Add toggle between old and updated calcium dynamics (ENH: Sarah's calcium dynamics code hnn-core#110) @samikane
• Resolve reliability issue with MPI simulations in hnn-core (Passing Network to mpi_child instead of Params causes intermittent crashes hnn-core#285)
• Run through tutorials with the new code to verify functionality (not formal tests)
  • ERP tutorial
  • Alpha/Beta tutorial
  • Gamma tutorial
  • Optimization tutorial @blakecaldwell
• Merge [MRG] hnn-core integration #232

Things pushed to another PR:

• Add back tooltips to the dialogs (Documenting HNN parameters hnn-core#214)
• Set global variables through the GUI
• Convert optimization to scipy instead of nlopt

[blakecaldwell]

A large part of this is done in #232, but I will need some help to complete this. I do not plan on working on this after the end of the month. @jasmainak @rythorpe @ntolley feel free to take issues on this list. I will finish up #112 as indicated.

[jasmainak]

I am going to be somewhat busy until the mid of next month ... trying to resist touching code, but happy to review PRs!

[blakecaldwell]

@jasmainak @stephanie-r-jones I have done as much as possible for this in PR #232. The items on the checklist relate to functionality missing from hnn-core. Once they are added, minor changes will be needed in #232 to complete integration.

Two aspects may take significant time and probably should begin ASAP:

• Somatic current/voltage recording in hnn-core (hnn-core #181)
• Making make sure that the hnn-core simulations are giving the same or comparable results to old HNN simulations. Significant progress has already been made in checking dipole results in hnn-core. Multiple trials need to be validated (hnn-core #113) Checking the other forms of model output can also be done in hnn-core.

[blakecaldwell]

@rythorpe what do you think about result comparisons between hnn-core and HNN? Do the model results need to be checked for consistency?

[blakecaldwell]

Thanks for the work on somatic voltages @ntolley! I'll integrate that into #232 and check off some of my todo's there.

It looks like jonescompneurolab/hnn-core#183 is almost ready. Thanks @kohl-carmen!

Certifying hnn-core results as the new standard for HNN might be the most time-consuming task remaining. I defer to @rythorpe for a strategy for that.

[rythorpe]

@rythorpe what do you think about result comparisons between hnn-core and HNN? Do the model results need to be checked for consistency?

Comparing the seeding behavior between hnn-gui and hnn-core is turning out to be a bit more complicated than I expected. As I mentioned here, the discrepancy is hard for me to diagnose particularly when it comes to adding rhythmic inputs. I'll try to give an update this weekend.

[blakecaldwell]

While #232 is merged, this issue can remain open. Besides testing and fixing any new bugs, there is one issue in hnn-core that seriously affects the stability of parallel simulations. So a resolution to jonescompneurolab/hnn-core#285 is another prerequisite to making this a release