Native plugin data parsers

[demberto]

Check out Walkthrough: Implementing a plugin data parser to learn how plugin data parsers are implemented.

Currently, these native plugins have a preset parser (or are on my to-do list):

Generators

• 3xOSC
• BooBass
• Fruit Kick
• Plucked

Effects

• Fruity Balance
• Fruity Blood Overdrive
• Fruity Center
• Fruity Fast Dist
• Fruity Free Filter
• Fruity Mute 2
• Fruity Notebook 2
• Fruity Send
• Fruity Soft Clipper
• Fruity Stereo Enhancer
• Soundgoodizer

Notes for implementers

Technically, all plugins with just knobs and sliders should be easy to implement, Most data is stored in 4 byte unsigned integers but 0-100% or whatever the extremes of the knobs/sliders are, they are pretty small values in terms of the available range.

Once you have implemented a parser for a plugin, please make sure that you add the relevant docstrings and docs. You can choose to add an image of the plugin in docs/img folder of the repo, but don't add your own bit.ly links to docstrings, that will be done by me.

If you have any suggestions, please let me know or if you have PRs mention this issue.

[ttaschke]

@demberto
I tried looking into Fruity Blood Overdrive, but I am confused about how to interpret the plugin parameters.

Based on this and looking at FruityFastDist as an example I planned to create the parameters "pre_band", "color", "pre_amp", "x100", "post_filter", "post_gain" with all of them StructProp[int], except of x100 which probably will be an enum.IntEnum (0,1).

However, when looking at the parameter values with FLPEdit this does not look at all like what I expected.

First of all, I have no idea what the 3 represents, as I expected this to be the value for pre_band which should have been 0 or some similar minimum value.
I also expected the other values to be 0, 128, 512 or anything similar representing minimum or maximum values as per the current plugin settings.

I also tried to set all knobs to 100% to get a feeling for the range, but these values also do not make immediate sense to me.

In comparison to this Fruity Fast Distortion's parameters fit very neatly with their values interpreted as int.

I have the feeling I am missing something fundamental about event values here. 🤔
Any pointers for how to interpret this would be a great help.

[demberto]

Ahh I remember Fruity Blood Overdrive, I had tried implementing it once earlier. IL probably made the representations closer to the one used by VST plugins.

VST plugins store an 8 or 10 (to indicate they are a VST) in the first 4 bytes. I think redesigned native plugins store a 3.

That's the reason it never changes. Why they did it, IDK. The internal name "Fruity Wrapper" is enough to indicate a VST plugin is loaded, same goes for native plugins.

I also tried to set all knobs to 100% to get a feeling for the range, but these values also do not make immediate sense to me.

Yes the values will be like that. If you check the docstrings on the minimum, maximum and default values of plugin parameters, you will find out that ranges like 128, 256 and 1024 are used a lot. Its indeed a space waste to use a 4 byte integer for such small values.

Match the list of parameters to the values you got starting from the first [16, 39, 0, ... and you will immediately notice that you have cracked it 😉. All knobs have the same range and x100 being a button is stored as a 0 or 1 but in a space of 4 bytes. Hence remaining bytes will be empty.

To get a 4 byte integer, you do:

So, finally 36 bytes are divided into:

• 4 bytes "native plugin marker", always 3 (the next time you see a 3 at top, just ignore it)
• 6 params - 4 bytes each = 24 bytes
• Remaining 4 bytes will always be zero ig, maybe they are used when used in Patcher, but that's another topic.

I hope I helped. This is a time consuming job.

[ttaschke]

Thanks for the explanation. The first 4 bytes for the plugin type really threw me off..
With this knowledge and the explanation of the params type I will be able to give this another go.

[ttaschke]

I wanted to try out the mixer/insert API with an already implemented plugin, before testing Blood Overdrive, but somehow I am not able to access plugins or slots in general.

This is the simple example that I used:

And here is how I tried to access the first slot in Channel 1:

>>> project.mixer[1]
Insert 'check_here' #1
>>> project.mixer[1][0]
Empty unnamed slot
>>> project.mixer[1][1]
Empty unnamed slot

All the slots are Empty unnamed and additionally I get an infinite loop when trying to iterate over the channel's slots (https://github.com/demberto/PyFLP/issues/530).
Apparently the slots are not parsed correctly? Or am I doing something wrong here?

Note: I am using the most recent master for testing

[demberto]

Damn the amount of bugs I need to fix sigh

[ttaschke]

@demberto
I am making progress on implementing the parser for Blood Overdrive, but wanted to ask your advice about how to best represent "redesigned native plugins".

As a simple approach I first just used StructEventBase and set the plugin marker (3) and the last 2x 4 bytes that are always 0 as kind of dummy props.

class FruityBloodOverdriveEvent(StructEventBase):
    STRUCT = _FruityBloodOverdriveStruct

class _FruityBloodOverdriveStruct(StructBase):
    PROPS = dict.fromkeys(("plugin_marker", "pre_band", "color", "pre_amp", "x100", "post_filter", "post_gain", "unknown_event_1", "unknown_event_2"), "I")

This works, but is probably not the most elegant approach...

Another approach would be to add a NativePluginEvent class and:

• A) Extend StructEventBase and add logic to parse the first and last two bytes separately while setting the other params via PROPS

or

• B) Implement NativePluginEvent similar to VSTPluginEvent (based on DataEventBase) and implement all related parsing there from zero

What would be the best approach for this based on your thinking for the design of the Events API?

[ttaschke]

I see. I actually did not think about backwards compatibility for the older plugin version until you mentioned it.
Is there a specific backward range of FL Studio versions that should be supported by new PyFLP features? Or just the goal to cover as much backwards as possible by covering for example both old and new plugin versions in this case?

Btw, how do you manage different FL Studio versions for testing in general? Do you just have several versions installed locally (I wonder if this brakes anything...)?
For Blood Overdrive it seems like the new vector-GUI version was added in 20.5, so I wanted to try out 20.4, but IL does not offer it anymore at https://forum.image-line.com/viewtopic.php?f=200&t=194258. I plan to try out FL Studio 12.5.1 later instead.

[demberto]

I intend to mimic FL's own parser, which is like the ideal implementation. If you see the EventID enums, you will notice certain members whose names start from _. Those are deprecated events not used anymore by FL. However the properties scan for them as well.

Ofcourse its not a priority. The goal is to accommodate as many events and their orderings as possible. The WhatsNew.rtf(changelog of FL) is a very important source of information, for determining which events, classes and properties were added when. I recommend you to check it out, its a goldmine of information. You also have FLPEdit's sources. Its author did quite some reversing himself and you will find comments about that a lot.

I don't have any other version than FL 20.8.4. I gathered the information by looking at my projects in FLPEdit saved in older versions.

[ttaschke]

I created a draft PR for Blood Overdrive to be able to talk about it more concrete:
#63

The code in the PR adds logic for just parsing the recent version of Blood Overdrive. Maybe adding this first as a base might make sense?
I also added comments for how I understood your recommendations in #42 (reply in thread), but I do not feel like I have a complete grasp about how this would look like in the end yet. Any feedback there would be great.

[ttaschke]

Regarding looking at older versions of the plugin:
I tried installing an older one and saving a .flp with the old Blood Overdrive, but I could not open the file with FLPEdit. The same goes for some older .flps of mine that I found (FL Studio 20.0 - 20.5ish). So not much luck here...

Ideal would be to have several different FL versions of https://github.com/demberto/PyFLP/blob/master/tests/assets/FL%2020.8.4.flp, but this is hard to realize, I guess.

Speaking of the test .flp.. I was thinking about how to do updates to this, but given the fixed version and things that should stay fixed like the project settings, probably updates should be done centrally by you?

In this case this would be adding Blood Overdrive with default settings to "Plugin Test":

[demberto]

Ideal would be to have several different FL versions of https://github.com/demberto/PyFLP/blob/master/tests/assets/FL%2020.8.4.flp, but this is hard to realize, I guess.

I thought about the same initially. However, the issue is not about the FL Studio version itself but the layout of structures and ordering of the events used in them. If we have an FLP for older versions, it begs the question, how many and which old versions? There are certain versions of FL in which its own serializer is fucked up. The changelog is helpful but not complete and hard to navigate through. Eventually as more details about the format are discovered, this might be worth a shot, so that we ensure PyFLP stays compatible with older versions while adding features for new.

Speaking of the test .flp.. I was thinking about how to do updates to this, but given the fixed version and things that should stay fixed like the project settings, probably updates should be done centrally by you?

Yea, as it is we have little control over our test data. It can probably work if you download the same version of FL, tho I am not sure how strings stored in Fruity Notebook 2, for example are affected.

I have some fresh ideas on how test FLPs can be bifurcated for the purpose they are used for.