05_pointnet_modeling.ipynb gives error "Exception: Could not find any network files. Unable to build network."

[mariakesa]

Hello,

I ran the notebook "05_pointnet_modeling.ipynb" and I get the error:

---

OSError Traceback (most recent call last)
in
2
3 configure = pointnet.Config.from_json('sim_ch05/simulation_config.json')
----> 4 configure.build_env()
5 network = pointnet.PointNetwork.from_config(configure)
6 sim = pointnet.PointSimulator.from_config(configure, network)

~/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/core/simulation_config.py in build_env(self, force)
95
96 self._set_logging()
---> 97 self.io.setup_output_dir(self.output_dir, self.log_file, self.overwrite_output)
98 self.copy_to_output()
99 self.env_built = True

~/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/pointnet/io_tools.py in setup_output_dir(self, config_dir, log_file, overwrite)
57
58 def setup_output_dir(self, config_dir, log_file, overwrite=True):
---> 59 super(NestIOUtils, self).setup_output_dir(config_dir, log_file, overwrite=True)
60 if n_nodes > 1 and rank == 0:
61 io.log_info('Running NEST with MPI ({} cores)'.format(n_nodes))

~/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/core/io_tools.py in setup_output_dir(self, output_dir, log_file, overwrite)
80 if os.path.exists(output_dir):
81 if overwrite:
---> 82 shutil.rmtree(output_dir)
83 os.makedirs(output_dir)
84 else:

~/anaconda3/lib/python3.7/shutil.py in rmtree(path, ignore_errors, onerror)
496 os.rmdir(path)
497 except OSError:
--> 498 onerror(os.rmdir, path, sys.exc_info())
499 else:
500 try:

~/anaconda3/lib/python3.7/shutil.py in rmtree(path, ignore_errors, onerror)
494 _rmtree_safe_fd(fd, path, onerror)
495 try:
--> 496 os.rmdir(path)
497 except OSError:
498 onerror(os.rmdir, path, sys.exc_info())

OSError: [Errno 39] Directory not empty: '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/output'

This happens although the directory is empty.

Help would be appreciated!

[mariakesa]

I was able to get rid of the Directory not empty error by pressing Ctrl+H in the folder and deleting the hidden files, but I still get the following error:

2021-09-22 19:45:08,416 [INFO] Created log file
INFO:NestIOUtils:Created log file
2021-09-22 19:45:08,418 [ERROR] Could not find any network files. Unable to build network.
ERROR:NestIOUtils:Could not find any network files. Unable to build network.

Exception Traceback (most recent call last)
in
3 configure = pointnet.Config.from_json('sim_ch05/simulation_config.json')
4 configure.build_env()
----> 5 network = pointnet.PointNetwork.from_config(configure)
6 sim = pointnet.PointSimulator.from_config(configure, network)
7 sim.run()

~/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/core/simulator_network.py in from_config(cls, conf, **properties)
225
226 if not config.with_networks:
--> 227 network.io.log_exception('Could not find any network files. Unable to build network.')
228
229 # TODO: These are simulator specific

~/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/core/io_tools.py in log_exception(self, message)
120
121 self.barrier()
--> 122 raise Exception(message)
123
124

Exception: Could not find any network files. Unable to build network.

Any help would be appreciated!

[mariakesa]

This is my config file

{'manifest': {'workingdir': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk', 'configdir': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05', 'configfname': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/simulation_config.json', 'time': '20-03-24', 'date': '2021-09-22', 'datetime': '2021-09-22_20-03-24', 'BASE_DIR': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05', 'OUTPUT_DIR': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/output'}, 'target_simulator': 'NEST', 'run': {'tstart': 0.0, 'tstop': 3000.0, 'dt': 0.01}, 'output': {'log_file': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/output/log.txt', 'output_dir': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/output', 'spikes_file': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/output/spikes.h5', 'quiet_simulator': True}, 'reports': {}, 'node_sets_file': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/node_sets.json', 'inputs': {'LGN_spikes': {'input_type': 'spikes', 'module': 'h5', 'input_file': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/lgn_spikes.h5', 'node_set': 'LGN'}}, 'config_path': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/simulation_config.json', 'config_dir': '/media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05', 'node_sets': {'biophysical_nodes': {'model_type': 'biophysical'}, 'point_nodes': {'model_type': 'point_process'}}}

[mariakesa]

So I manually included the following lines into the simulation_config.json:

{
"networks": {
"nodes": [
{
"nodes_file": "$BASE_DIR/network/V1_nodes.h5",
"node_types_file": "$BASE_DIR/network/V1_node_types.csv"
},
{
"nodes_file": "$BASE_DIR/network/LGN_nodes.h5",
"node_types_file": "$BASE_DIR/network/LGN_node_types.csv"
}
],
"edges": [
{
"edges_file": "$BASE_DIR/network/V1_V1_edges.h5",
"edge_types_file": "$BASE_DIR/network/V1_V1_edge_types.csv"
},
{
"edges_file": "$BASE_DIR/network/LGN_V1_edges.h5",
"edge_types_file": "$BASE_DIR/network/LGN_V1_edge_types.csv"
}
]
},

"components": {
"morphologies_dir": "$BASE_DIR/components/morphologies",
"synaptic_models_dir": "$BASE_DIR/components/synaptic_models",
"mechanisms_dir":"$BASE_DIR/components/mechanisms",
"point_neuron_models_dir": "$BASE_DIR/components/point_neuron_models_dir"
}
}

The error of "Could not find any network files." disappeared, but now I get this error:

Exception: Could not find node dynamics_params file /media/maria/DATA1/Documents/documents/NeuroPixelsDataAnalysis/bmtk/sim_ch05/components/point_neuron_models_dir/472363762_point.json.

Thank you for any help that you would be able to provide:-)

[kaeldai]

Hi @mariakesa - can you look to see if you have a directory called sim_ch05/components/point_neuron_models_dir/ or is it called sim_ch05/components/point_neuron_models/ (eg no _dir ending)? When I try running build_env() it should create a folder point_neuron_models/ with the correct json file.

[mariakesa]

Hi:-)

Yeah, there's a directory like that! And there's a file too, "472363762_point.json". But I guess it doesn't have the right contents?

Here's what's inside:

{
"I_e": 0.0,
"tau_m": 44.9,
"C_m": 239.0,
"t_ref": 3.0,
"E_L": -78.0,
"V_th": -43.0,
"V_reset": -55.0
}

[kaeldai]

That looks correct. The problem seems to be in the simulation_config.json is not pointing to the correct directory, I think the line in the config json should be changed from:

"point_neuron_models_dir": "$BASE_DIR/components/point_neuron_models**_dir**"

I think should be changed to

"point_neuron_models_dir": "$BASE_DIR/components/point_neuron_models"

[mariakesa]

Thanks that solved that error:-)

But there's a next one:-D

2021-09-22 21:03:43,488 [INFO] Created log file
2021-09-22 21:03:43,507 [INFO] Batch processing nodes for V1/0.
2021-09-22 21:03:43,549 [INFO] Batch processing nodes for LGN/0.
2021-09-22 21:03:43,621 [INFO] Setting up output directory
2021-09-22 21:03:43,622 [INFO] Building cells.
2021-09-22 21:03:43,636 [INFO] Building recurrent connections
Traceback (most recent call last):
File "run_pointnet.py", line 14, in
main('simulation_config.json')
File "run_pointnet.py", line 9, in main
sim = pointnet.PointSimulator.from_config(configure, network)
File "/home/maria/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/pointnet/pointsimulator.py", line 236, in from_config
graph.build_recurrent_edges()
File "/home/maria/anaconda3/lib/python3.7/site-packages/bmtk-0.0.9-py3.7.egg/bmtk/simulator/pointnet/pointnetwork.py", line 143, in build_recurrent_edges
nest.Connect(nest_srcs, nest_trgs, conn_spec='one_to_one', syn_spec=edge.nest_params)
File "/usr/lib/python3.6/dist-packages/nest/ll_api.py", line 246, in stack_checker_func
return f(*args, **kwargs)
File "/usr/lib/python3.6/dist-packages/nest/lib/hl_api_connections.py", line 210, in Connect
sps(pre)
File "pynestkernel.pyx", line 221, in pynestkernel.NESTEngine.push
File "pynestkernel.pyx", line 317, in pynestkernel.python_object_to_datum
ValueError: Buffer dtype mismatch, expected 'int' but got 'long'

[kaeldai]

Do you know what version of NEST you have installed?

[mariakesa]

Sure!

          -- N E S T --

Copyright (C) 2004 The NEST Initiative

Version: nest-2.20.1

[aitsam12]

I am also getting the similar error.

File "/usr/lib/python3.6/dist-packages/nest/lib/hl_api_connections.py", line 312, in Connect
sps(syn_spec)
File "pynestkernel.pyx", line 221, in pynestkernel.NESTEngine.push
File "pynestkernel.pyx", line 295, in pynestkernel.python_object_to_datum
File "pynestkernel.pyx", line 317, in pynestkernel.python_object_to_datum
ValueError: Buffer dtype mismatch, expected 'int' but got 'double'

My NEST version is also nest-2.20.1

[brainmodelingtools]

Can you let us know how and when you installed BMTK as well so I can try to replicate the error? And are you also running tutorial 05?

[aitsam12]

I recently installed NEST-2.20.1.
I am using PyNN.
My task is to convert ANN to SNN with SNNtoolbox. When I use the NEST simulator with PyNN then i am getting above error.

Thank you in advance

[xpliu16]

We are not able to support PyNN or SNNtoolbox, but one thought is whether any properties in the syn_specs being passed to Nest are supposed to be ints (e.g., receptor port) but are being sent as floats? If you're still having issues, good places to turn for Nest help would be:

• https://github.com/nest/nest-simulator/issues
• https://www.nest-simulator.org/mailinglist/hyperkitty/list/users@nest-simulator.org/
  If you figure out the cause of this error, it would be helpful to post back what you find.

Thanks!