Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NEST Desktop target #802

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
41 changes: 39 additions & 2 deletions doc/running.rst
@@ -1,6 +1,28 @@
Running NESTML
##############

Running NESTML causes several processing steps to occur:

1. The model is parsed from file and checked (syntax, consistent physical units, and so on).
2. Code is generated from the model by one of the "code generators" selected when NESTML is invoked.
3. If necessary, the code is compiled and built by the "builder" that belongs to the selected code generator.

Currently, the following code generators are supported:

.. list-table::
:header-rows: 1
:widths: 10 30 30

* - Name
- NESTML features supported
- Notes
* - NEST
- neurons, synapses, vectors, guards
- NEST 2 and NEST 3 are supported
* - NEST Desktop
- neurons
- Model ``json`` files are generated

Running NESTML from Python
--------------------------

Expand Down Expand Up @@ -30,7 +52,7 @@ The following default values are used, corresponding to the command line default
- *no default*
* - target_platform
- str
- "NEST"
- "NEST", "NEST_DESKTOP"
* - target_path
- str
- None
Expand Down Expand Up @@ -113,7 +135,7 @@ This will generate, compile, build, and install the code for a set of specified
* - ``--target_path``
- (Optional) Path to target directory where generated code will be written into. Default is ``target``, which will be created in the current working directory if it does not yet exist.
* - ``--target_platform``
- (Optional) The name of the target platform to generate code for. Default is ``NEST``.
- (Optional) The name of the target platform to generate code for. The available targets are ``NEST`` and ``NEST_DESKTOP``. Default is ``NEST``.
* - ``--logging_level``
- (Optional) Sets the logging level, i.e., which level of messages should be printed. Default is ERROR, available are [DEBUG, INFO, WARNING, ERROR, NO]
* - ``--module_name``
Expand Down Expand Up @@ -163,3 +185,18 @@ To generate code that is compatible with particular release versions of NEST Sim
- The default is the empty string, which causes the NEST version to be automatically identified from the ``nest`` Python module.
- ``"v2.20.2"``: Latest NEST 2 release.
- ``"master"``: Latest NEST GitHub master branch version (https://github.com/nest/nest-simulator/).

NEST Desktop target
~~~~~~~~~~~~~~~~~~~

The aim of the NEST Desktop as target is to generate ``json`` files for the neuron models. The resulting file contains details about the state variables, parameters and their initial values defined in their respective ``.nestml`` files. The ``json`` files are used to load them in the NEST Desktop user interface.

For example, for the neuron model ``iaf_psc_exp``, the ``json`` file will be generated by running the ``generate_target`` function with ``target_platform`` option set to ``NEST_DESKTOP``.

.. code-block:: python

from pynestml.frontend.pynestml_frontend import generate_target

generate_target(input_path="/home/nest/work/pynestml/models/neurons/iaf_psc_exp.nestml",
target_platform="NEST_DESKTOP",
target_path="/tmp/nestml_target")
5 changes: 4 additions & 1 deletion pynestml/codegeneration/code_generator.py
Expand Up @@ -28,7 +28,7 @@

import os

from jinja2 import Template, Environment, FileSystemLoader
from jinja2 import Template, Environment, FileSystemLoader, TemplateRuntimeError

from pynestml.exceptions.invalid_path_exception import InvalidPathException
from pynestml.exceptions.invalid_target_exception import InvalidTargetException
Expand Down Expand Up @@ -98,6 +98,9 @@ def setup_template_env(self):
raise Exception("A list of module template files/directories is missing.")
self._module_templates.extend(self.__setup_template_env(module_templates, templates_root_dir))

def raise_helper(self, msg):
raise TemplateRuntimeError(msg)

def __setup_template_env(self, template_files: List[str], templates_root_dir: str) -> List[Template]:
"""
A helper function to set up the jinja2 template environment
Expand Down
7 changes: 0 additions & 7 deletions pynestml/codegeneration/nest_code_generator.py
Expand Up @@ -23,8 +23,6 @@

import datetime

from jinja2 import TemplateRuntimeError

import odetoolbox

import pynestml
Expand Down Expand Up @@ -157,9 +155,6 @@ def __init__(self, options: Optional[Mapping[str, Any]] = None):

self._ode_toolbox_printer = UnitlessExpressionPrinter(ODEToolboxReferenceConverter())

def raise_helper(self, msg):
raise TemplateRuntimeError(msg)

def set_options(self, options: Mapping[str, Any]) -> Mapping[str, Any]:
# insist on using the old Archiving_Node class for NEST 2
if self.get_option("nest_version").startswith("v2"):
Expand Down Expand Up @@ -477,9 +472,7 @@ def _get_neuron_model_namespace(self, neuron: ASTNeuron) -> Dict:
"""
Returns a standard namespace with often required functionality.
:param neuron: a single neuron instance
:type neuron: ASTNeuron
:return: a map from name to functionality.
:rtype: dict
"""
namespace = dict()

Expand Down
71 changes: 71 additions & 0 deletions pynestml/codegeneration/nest_desktop_code_generator.py
@@ -0,0 +1,71 @@
# -*- coding: utf-8 -*-
#
# nest_desktop_code_generator.py
#
# This file is part of NEST.
#
# Copyright (C) 2004 The NEST Initiative
#
# NEST is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# NEST is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with NEST. If not, see <http://www.gnu.org/licenses/>.
import os
from typing import Sequence, Union, Optional, Mapping, Any, Dict

from pynestml.codegeneration.code_generator import CodeGenerator
from pynestml.frontend.frontend_configuration import FrontendConfiguration
from pynestml.meta_model.ast_neuron import ASTNeuron
from pynestml.meta_model.ast_synapse import ASTSynapse


class NESTDesktopCodeGenerator(CodeGenerator):
"""
Code generator for NEST Desktop
"""
_default_options = {
"templates": {
"path": "",
"model_templates": {
"neuron": ["@NEURON_NAME@.json.jinja2"],
}
}
}

def __init__(self, options: Optional[Mapping[str, Any]] = None):
self._target = "NEST_DESKTOP"
super().__init__(self._target, options)
self.setup_template_env()

def generate_code(self, models: Sequence[Union[ASTNeuron, ASTSynapse]]) -> None:
"""
Generate the .json files for the given neuron and synapse models
:param models: list of neuron models
"""
if not os.path.isdir(FrontendConfiguration.get_target_path()):
os.makedirs(FrontendConfiguration.get_target_path())
neurons = [model for model in models if isinstance(model, ASTNeuron)]
synapses = [model for model in models if isinstance(model, ASTSynapse)]
self.generate_neurons(neurons)
self.generate_synapses(synapses)

def _get_neuron_model_namespace(self, neuron: ASTNeuron) -> Dict:
"""
Returns a standard namespace with often required functionality.
:param neuron: a single neuron instance
:return: a map from name to functionality.
"""
from pynestml.codegeneration.nest_tools import NESTTools
namespace = dict()
namespace["neuronName"] = neuron.get_name()
namespace["neuron"] = neuron
namespace["parameters"] = NESTTools.get_neuron_parameters(neuron.get_name())
return namespace
62 changes: 60 additions & 2 deletions pynestml/codegeneration/nest_tools.py
Expand Up @@ -20,8 +20,10 @@
# along with NEST. If not, see <http://www.gnu.org/licenses/>.

import multiprocessing as mp
import os
import sys

from pynestml.frontend.pynestml_frontend import generate_nest_target
from pynestml.utils.logger import Logger
from pynestml.utils.logger import LoggingLevel

Expand All @@ -43,9 +45,9 @@ def _detect_nest_version(user_args):
nest_version = "v3.0"
elif "prepared" in nest.GetKernelStatus().keys(): # "prepared" key was added after v3.3 release
nest_version = "master"
elif "tau_u_bar_minus" in neuron.get().keys(): # added in v3.3
elif "tau_u_bar_minus" in neuron.get().keys(): # added in v3.3
nest_version = "v3.3"
elif "tau_Ca" in vt.get().keys(): # removed in v3.2
elif "tau_Ca" in vt.get().keys(): # removed in v3.2
nest_version = "v3.1"
else:
nest_version = "v3.2"
Expand All @@ -56,6 +58,30 @@ def _detect_nest_version(user_args):
return nest_version


def _get_model_parameters(model_name: str, queue: mp.Queue):
try:
import nest
input_path = os.path.join(os.path.realpath(os.path.join(os.path.dirname(__file__), os.path.join(
os.pardir, os.pardir, "models", "neurons", model_name + ".nestml"))))
target_path = "target"
suffix = "_nestml"
module_name = "nest_desktop_module"
generate_nest_target(input_path=input_path,
target_path=target_path,
suffix=suffix,
module_name=module_name,
logging_level="INFO")
# Install the nest module and query all the parameters
nest.Install(module_name)
n = nest.Create(model_name + suffix)
parameters = n.get()

except ModuleNotFoundError:
parameters = {}

queue.put(parameters)


class NESTTools:
r"""Helper functions for NEST Simulator"""

Expand All @@ -81,3 +107,35 @@ def detect_nest_version(cls) -> str:
Logger.log_message(None, -1, "The NEST Simulator version was automatically detected as: " + nest_version, None, LoggingLevel.INFO)

return nest_version

@classmethod
def get_neuron_parameters(cls, neuron_model_name: str) -> dict:
r"""
Get the parameters for the given neuron model. The code is generated for the model and installed into NEST.
The parameters are then queried by creating the neuron in NEST.
:param neuron_model_name: Name of the neuron model
:return: A dictionary of parameters
"""
# This function internally calls the nest_code_generator which calls the detect_nest_version() function that
# uses mp.Pool. If a Pool is used here instead of a Process, it gives the error "daemonic processes are not
# allowed to have children". Since creating a Pool inside a Pool is not allowed, we create a Process
# object here instead.
_queue = mp.Queue()
p = mp.Process(target=_get_model_parameters, args=(neuron_model_name, _queue))
p.start()
p.join()
parameters = _queue.get()
p.close()

if not parameters:
Logger.log_message(None, -1,
"An error occurred while importing the `nest` module in Python. Please check your NEST "
"installation-related environment variables and paths, or specify ``nest_version`` "
"manually in the code generator options.",
None, LoggingLevel.ERROR)
sys.exit(1)
else:
Logger.log_message(None, -1, "The model parameters were successfully queried from NEST simulator",
None, LoggingLevel.INFO)

return parameters
@@ -0,0 +1,17 @@
{
"id": "{{neuronName}}",
"elementType": "neuron",
"label": "{{neuronName}}}}",
pnbabu marked this conversation as resolved.
Show resolved Hide resolved
"recordables": [
{%- for variable in neuron.get_state_symbols() %}
"{{variable.get_symbol_name()}}"
{%- if not loop.last %},{%- endif %}
{%- endfor %}
],
"params": [
{%- for variable in neuron.get_parameter_symbols() %}
{%- set last = loop.last %}
{%- include "ParameterDeclaration.jinja2" %}
{%- endfor %}
]
}
@@ -0,0 +1,10 @@
{%- set name = variable.get_symbol_name() %}
{%- set type_symbol = variable.get_type_symbol().print_symbol() %}
{%- set default_value = parameters[variable.get_symbol_name()] %}
{
"id": "{{name}}",
"label": "{{name}}",
"unit": "{{type_symbol}}",
"value": "{{default_value}}",
"input": "arrayInput"
}{%- if not last %},{%- endif %}
4 changes: 2 additions & 2 deletions pynestml/frontend/frontend_configuration.py
Expand Up @@ -36,7 +36,7 @@
help_input_path = 'One or more input path(s). Each path is a NESTML file, or a directory containing NESTML files. Directories will be searched recursively for files matching \'*.nestml\'.'
help_target_path = 'Path to a directory where generated code should be written to. Standard is "target".'
help_install_path = 'Path to the directory where the generated code will be installed.'
help_target = 'Name of the target platform to build code for. Default is NEST.'
help_target_platform = 'Name of the target platform to build code for. The available targets are NEST and NEST_DESKTOP. Default is NEST.'
help_logging = 'Indicates which messages shall be logged and printed to the screen. Standard is ERROR.'
help_module = 'Indicates the name of the module. Optional. If not indicated, the name of the directory containing the models is used'
help_log = 'Indicates whether a log file containing all messages shall be stored. Standard is NO.'
Expand Down Expand Up @@ -97,7 +97,7 @@ def parse_config(cls, args):
type=str, help=help_input_path, required=True)
cls.argument_parser.add_argument(qualifier_target_path_arg, metavar='PATH', type=str, help=help_target_path)
cls.argument_parser.add_argument(qualifier_install_path_arg, metavar='PATH', type=str, help=help_install_path)
cls.argument_parser.add_argument(qualifier_target_platform_arg, choices=get_known_targets(), type=str.upper, help=help_target, default='NEST')
cls.argument_parser.add_argument(qualifier_target_platform_arg, choices=get_known_targets(), type=str.upper, help=help_target_platform, default='NEST')
cls.argument_parser.add_argument(qualifier_logging_level_arg, metavar='{DEBUG, INFO, WARNING, ERROR, NONE}', choices=[
'DEBUG', 'INFO', 'WARNING', 'WARNINGS', 'ERROR', 'ERRORS', 'NONE', 'NO'], type=str, help=help_logging, default='ERROR')
cls.argument_parser.add_argument(qualifier_module_name_arg, metavar='NAME', type=str, help=help_module)
Expand Down
6 changes: 5 additions & 1 deletion pynestml/frontend/pynestml_frontend.py
Expand Up @@ -45,7 +45,7 @@


def get_known_targets():
targets = ["NEST", "autodoc", "none"]
targets = ["NEST", "autodoc", "NEST_DESKTOP", "none"]
targets = [s.upper() for s in targets]
return targets

Expand Down Expand Up @@ -89,6 +89,10 @@ def code_generator_from_target_name(target_name: str, options: Optional[Mapping[
assert options is None or options == {}, "\"autodoc\" code generator does not support options"
return AutoDocCodeGenerator()

if target_name.upper() == "NEST_DESKTOP":
from pynestml.codegeneration.nest_desktop_code_generator import NESTDesktopCodeGenerator
return NESTDesktopCodeGenerator(options)

if target_name.upper() == "NONE":
# dummy/null target: user requested to not generate any code
code, message = Messages.get_no_code_generated()
Expand Down
3 changes: 2 additions & 1 deletion setup.py
Expand Up @@ -52,7 +52,8 @@
"codegeneration/resources_nest/point_neuron/*.jinja2",
"codegeneration/resources_nest/point_neuron/common/*.jinja2",
"codegeneration/resources_nest/point_neuron/directives/*.jinja2",
"codegeneration/resources_nest/point_neuron/setup/*.jinja2"]},
"codegeneration/resources_nest/point_neuron/setup/*.jinja2",
"codegeneration/resources_nest_desktop/*.jinja2"]},
data_files=data_files,
entry_points={
"console_scripts": [
Expand Down