update the slowly changing parameters faster and more consistently

[robertoostenveld]

Initially we implemented many modules with get/set, but later switched to patch.setvalue which does simultaneous pub/sub and get/set. As a consequence, when the value in one module changes and is sent to Redis, another module can in principle be notified of this immediately. However, we have many modules that only once or infrequently check the value of parameters that can be hardcoded in the ini, but that can also come from Redis.

For example delaytrigger only checks the delay and output value upon startup

eegsynth/module/delaytrigger/delaytrigger.py

Line 137 in 32a3fa7

delay = patch.getfloat("delay", item[0])

and in the outputmidi module the UpdateParameters function is called upon each iteration (typically every 50 ms)

eegsynth/module/outputmidi/outputmidi.py

Line 317 in 32a3fa7

UpdateParameters()

whereas in the synthesizer module the ControlThread thread runs constantly without delays (that must be hard on the CPU)

eegsynth/module/synthesizer/synthesizer.py

Line 101 in 32a3fa7

while self.running:

It would be good to make this more consistent and to come up with a generic implementation that does not make the code of individual modules more complex, but that does take care of more consistent, faster, and more efficient updating of Redis values between modules.

[stephenwhitmarsh]

A somewhat specific example is the recordsignal module in a live recording situation where it starts/stops-writes according to a Redis value ; due to it's polling of changes to Redis values (when to record), and the use of a blocksize, the resultant duration of recordings are variable with multiples of blocksize, depending when a changing Redis value is noticed.

Except for a complicated with multiple threads which I can't fully oversee, perhaps the easiest would be to create another module ft2file that writes the content of the ft buffer to file at button-press (with sub rather than get'), rather than continuously writing blocks of data.
The ini settings could define how much time will be written, i.e. a constant duration of the recording until 'button-press'. A start-stop could also be implemented - where the start position of the recording is set based on the current timestamp in the FT buffer.
I guess there are other possiblities as well. If there is no solution that is more responsive but while staying consistent with the current functionality, we can bump this to a new issue.

[robertoostenveld]

With recordsignal we can write the signals to either EDF or WAV.

EDF requires the block size to be specified in the header (see https://www.edfplus.info/specs/edf.html), hence it cannot change on the fly.

WAV writes the data in a multiplexed format (one sample for all channels, and that repeated for all samples) which is technically independent of the specified block size. The only consideration there is that you don't want to read/write the data in too short (e.g., 1-sample) blocks, as that puts more overhead on the network (getting data) and on the disk (writing data).

Given the topic of this issue, I suggest that further discussions that are specific to writing signals to disk are continued in another issue.

[robertoostenveld]

I suggest we also move the discussion about switching on and off certain behaviour of a module (such as saving to disk) to another thread. Note that with delaytrigger we can already easily implement the behaviour of a useless box.

Edit: see also #393.

[robertoostenveld]

I have been thinking about this: the EEGsynth patch object now contains a few functions like patch.getfloat(...) that are called by the module. Each time these getfloat functions are called, the ini file is read (which does not change anyway) or the value is get from Redis. To keep track of changes to the patched value, getfloat needs to be called repeatedly, or the module has to subscribe to Redis (with pub/sub).

Getting the value repeatedly is resource (cpu) intensive and updates to the value are always a bit too late. Implementing pub/sub in each module is code-wise complex, as it requires a thread.

Imagine that the patch object would have its own internal thread that keeps track of the values using pub/sub, and that a call to patch.getfloat(...) from within a module would only copy the value that is caches inside the patch object. In that way, calling getfloat would not be resource intense any more, and whenever you call it, you know it has the latest value.

It would look like this:

patch = EEGsynth.patch(config, r) # this connects to the ini file and redis
patch.start() # from now on it monitors all values of interest

patch.getfloat('section', 'key') # this gets the value the first time, but also registers it as a value of interest

# if applicable, the patch subscribes to section.key on Redis 
# the internal thread now keep an eye out for any changes

patch.getfloat('section', 'key') # this returns the most recent value from memory

and at the end there should be a patch.stop().

The advantage is that the code in the modules need fewer global variables, that the modules become more resource efficient (since no polling of Redis with getfloat()) and that the modules can always use the latest values.