Lua API questions and comments

[mfisher31]

I know you're trying to get 1.0 out the door, so I want to give you some quick comments, today I started to convert a longish midi script to LUA...its not done, will take me a few days, but it brought up questions and comments I wanted to send to you as I go... In a few days I can send you this much longer script for reference, let me get it finished and hopefully working first. Its gonna be about 500-600 lines of lua.

so round 1 Lua api questions:

1. You don't seem have PolyPressure correctly setup in the MidiMessage api. Unless I'm misunderstanding something or missing it, but anyway, PolyPressure has a pitch component in each message, its basically per pitch after touch.

2. question about the input:swap operation, how fast is it? is it copying all the contents of all the messages or copying pointers or what is it doing exactly? Furthermore I need to know if I could potentially declare output globally instead of inside the process() function, to avoid malloc, but that only works if in between calls to process the midi events are copied internally by value or something. if the guts of the Lua api is still using that MidiBuffer by reference, then obviously I can't do that. Just need some clarity about what I can or can't do...and the docs should probably point out any danger there. Prefer to avoid any kind of new or malloc in the realtime thread if possible, but copying by reference has its advantages too.

3. when a MidiMessage is inserted into a buffer with output:insert(msg, time) is the event copied in or is a reference pointer copied in? the reason I ask is in case I can reuse the MidiMessage object over and over to avoid having to call MidiMessage.new(). If its copied in by value, then yes I can reuse a single MidiMessage object repeatedly to end up with a buffer full of events and minimal malloc; but if its copied in by reference then obviously: no I would need to take care not to do anything that would overwrite that MidiMessage that is being referenced. Just want to make sure I use a design pattern that will minimize memory allocation and minimize large copying of data, so just wondering what the MidiBuffer.insert does, by value or by ref

4. When inserting a MidiMessage to the MidiBuffer, there is frame argument, which is the sample timestamp. The question is, if the MidiMessage has a time() value already in it, can we insert without having to re-specify the time again? I found myself having to redundantly specify time a lot in various places, which is ripe for coding errors. My preference would be, that we can say output:insert(msg) and if the msg has its timestamp configured, it will be used. if its nil, then either error or perhaps default to something sensible like inserting it with the timestamp of the process block or something like that. Or optionally, Call insert with the time specified as it currently is. And further to that, can we specify a time in the future beyond the process block or do we need care to avoid that?

5. Further to all of that, can you clarify if the timestamp of each event is a number relative to the start of the process block or a number relative to the start of the sequence. In each case where timestamp is used in any of the API, please indicate if its the sample position relative to start or relative to process block

6. Regarding to the position structure, it appears that is passed into the process function as an argument. when I am doing complicated scripting I have to call a lot of functions and pass things around and its kind of a PITA to have to pass the position table around to everything so that at some lower level when I actually need it there it is. It would be nice if we could query for it at any time by calling an API function like getPosition() or something like that. If not, se la vie I could probably just stash it in a global or something, but anyway, an Api function to retrieve it would be useful.

7. regarding parameters, I haven't gotten to that yet, will probably have more questions about that after I get there, but its kind of the same as item 6 above, it seems the current parameter values are passed in to proces() which is good for automation purposes we can know the values match what the host is expecting those to be at a certain point in time matching the process block. However, same thing, pain to pass down around through a big chain of complex functions so that I can get later. Would like it if somehow I can access globally. Maybe I just need to stash those values globally myself and you don't worry about it, or maybe there is some way for us to get the current values of the more automatically at any time without having to pass that passed in value down through a function stack all over the place. Am I making sense?

8. I didn't immediately see a function anywhere to handle when a parameter has actually changed. In some other scripters, there is a callback whenever one of the parameters is changed by automation (or by default GUI). Anyway, I will get into the parameter stuff tomorrow and maybe it will all make more sense when I do.

That's all I have for now, I'll surely have more comments and questions in a few days.

[mfisher31]

@steveschow - FYI I moved this issue over to GitHub.

1. You don't seem have PolyPressure correctly setup in the MidiMessage api. Unless I'm misunderstanding something or missing it, but anyway, PolyPressure has a pitch component in each message, its basically per pitch after touch.

I'm not sure what poly pressure is. The MidiMessage comes directly from juce which has no functions for it.

2. question about the input:swap operation, how fast is it? is it copying all the contents of all the messages or copying pointers or what is it doing exactly? Furthermore I need to know if I could potentially declare output globally instead of inside the process() function, to avoid malloc, but that only works if in between calls to process the midi events are copied internally by value or something. if the guts of the Lua api is still using that MidiBuffer by reference, then obviously I can't do that. Just need some clarity about what I can or can't do...and the docs should probably point out any danger there. Prefer to avoid any kind of new or malloc in the realtime thread if possible, but copying by reference has its advantages too.

It's near instant and swaps pointers at the c++ level. No copying of data. No dynamic allocation.

3. when a MidiMessage is inserted into a buffer with output:insert(msg, time) is the event copied in or is a reference pointer copied in? the reason I ask is in case I can reuse the MidiMessage object over and over to avoid having to call MidiMessage.new(). If its copied in by value, then yes I can reuse a single MidiMessage object repeatedly to end up with a buffer full of events and minimal malloc; but if its copied in by reference then obviously: no I would need to take care not to do anything that would overwrite that MidiMessage that is being referenced. Just want to make sure I use a design pattern that will minimize memory allocation and minimize large copying of data, so just wondering what the MidiBuffer.insert does, by value or by ref

All of the insert* functions copy the data to the buffer, which may or may not be pre-allocated. Buffers coming in the process function will be pre-allocated. See MidiBuffer:reserve()

4. When inserting a MidiMessage to the MidiBuffer, there is frame argument, which is the sample timestamp. The question is, if the MidiMessage has a time() value already in it, can we insert without having to re-specify the time again? I found myself having to redundantly specify time a lot in various places, which is ripe for coding errors. My preference would be, that we can say output:insert(msg) and if the msg has its timestamp configured, it will be used. if its nil, then either error or perhaps default to something sensible like inserting it with the timestamp of the process block or something like that. Or optionally, Call insert with the time specified as it currently is. And further to that, can we specify a time in the future beyond the process block or do we need care to avoid that?

It's the same rules as JUCE. MidiMessage:time() is ignored by MidiBuffer which manages a raw buffer operating in audio frames. The time method is for end-user use and places where MidiMessages are flying in somewhere outside of a MidiBuffer. The MidiBuffer ignores it because it doesn't maintain a list of MidiMessages. It is a raw data block of uint8_t's segmented by frame indexes.

5. Further to all of that, can you clarify if the timestamp of each event is a number relative to the start of the process block or a number relative to the start of the sequence. In each case where timestamp is used in any of the API, please indicate if its the sample position relative to start or relative to process block.

Things going in and out of MidiBuffer ignore the MidiMessage:time(). Use the frame parameter from the buffer iterators which go from 1 to buffer_size inclusive.

6. Regarding to the position structure, it appears that is passed into the process function as an argument. when I am doing complicated scripting I have to call a lot of functions and pass things around and its kind of a PITA to have to pass the position table around to everything so that at some lower level when I actually need it there it is. It would be nice if we could query for it at any time by calling an API function like getPosition() or something like that. If not, se la vie I could probably just stash it in a global or something, but anyway, an Api function to retrieve it would be useful.

I was actually pondering this kind of thing as well. I'm thinking a regular Lua module that could be used in other places too. The tricky part is that scripts running in the audio thread can't really synchronize with the UI thread + mutexes... because of Lua's garbage collector.

-- Imagine this "transport" or something else is set before instantiating the script or an import
local transport = require ('el.transport')
local pos = transport.position()

A new module like this could also have all kinds of transport and time specific helpers. The DSP script's are intended to be self-contained. The script file itself could be considered the object and anything in the global scope is not visible anywhere else. Until I can make up my mind storing position in the global scope and updated in process would effectively be the same thing done in a potential el.whatever module.

7. regarding parameters, I haven't gotten to that yet, will probably have more questions about that after I get there, but its kind of the same as item 6 above, it seems the current parameter values are passed in to proces() which is good for automation purposes we can know the values match what the host is expecting those to be at a certain point in time matching the process block. However, same thing, pain to pass down around through a big chain of complex functions so that I can get later. Would like it if somehow I can access globally. Maybe I just need to stash those values globally myself and you don't worry about it, or maybe there is some way for us to get the current values of the more automatically at any time without having to pass that passed in value down through a function stack all over the place. Am I making sense?

Yeah, it makes sense. Let me think about that as it's an API breaking change. Glad to have not tagged 1.0.0 yet. The API is bare-bones for speed reasons. It's perfectly fine inside a DSP script to store the params array globally. Just make sure to update it in process.

8. I didn't immediately see a function anywhere to handle when a parameter has actually changed. In some other scripters, there is a callback whenever one of the parameters is changed by automation (or by default GUI). Anyway, I will get into the parameter stuff tomorrow and maybe it will all make more sense when I do.

if last_param ~= current_param then
-- it changed
end

That's all I have for now, I'll surely have more comments and questions in a few days.

[mfisher31]

Forgot to mention that there isn't much in the backend doing things like value change detection. The reason is that once it's baked in C++ and final... the extra bloat can't be removed. I look at it like simple change detection should be handled by the developer however they see fit.

[mfisher31]

And also for the avoidance of parameter passing. The backend, in my opinion shouldn't be handling things like this. I wouldn't mind coming up with some helpers here and there to aid in setting up basic things though. e.g. modules that can be required in somehow and not force bloat to all others.

[steveschow]

Since then I have spent some
Time converting my event chaser to c++ juce and I think I am understanding better the relationship between MidiMesaage, MidiMessageMetadata and MidiPipe, etc. and some other things which I see you are trying to stick as closely as possible to the actual juce api. I think it's worthwhile to point out in the Lua dics some of these details or reference the juce api docs. One thing I try to avoid in realtime thread is using new or doing anything that might do so under the covers. It's a little more obsfucated when wrapped by lua and juce. Just want to make sure this is avoided; copying midi messages by value into buffer is better alternative

Polypressure is what Logic Pro calls per-note after touch rather then channel pressure. I could not see how to determine the after touch pitch for those

Biggest problem is it crashes very easy.

[steveschow]

So parameters are useful for non gui reasons to use daw automation. Not sure the best way to supply that I haven't dug into your approach quite yet. After I finish my juce eventchaser plugin will come back to element lua again and try some more things

[mfisher31]

One thing I try to avoid in realtime thread is using new or doing anything that might do so under the covers

Element will never be doing any of these things under the hood.

[steveschow]

That's good to know but I was referring more to careful lua and juce use

[mfisher31]

Oh, gotcha. Yeah it is possible to on the end-user side be allocating memory.

[steveschow]

Basically I was not really knowledgeable before about JUCE midi Api. I know a little more now. That's why i was asking some dumb questions about how and when the JUCE api might be allocating memory even without me calling "new". After working with JUCE in C++ a little bit, I understand it better now. i think it would be good for Element docs to explain in simple terms about that API a little bit and also some pitfalls to avoid in the realtime thread, such as calling "new MidiMessage", etc. Try to use the stack and know that Midi Messages will be copied by value back to a MidiBuffer, etc.. it wasn't entirely clear to me before since I really hadn't coded anything in Juce midi before. its making more sense to me now. but Lua users can easily end up creating memory allocation if they are not careful with new, and perhaps even some JUCE calls that might be doing it under the covers.

[mfisher31]

It's Ok, I see no dumb questions here. The docs can definitely be better.