Addition: Implementing Save and Load Functionality for VST3 Plugin States

[Myuqou]

Problem:

• Some VST3 plugins don't expose all UI parameters to the host, hindering
  programmatic manipulation without manual intervention. This is evident in
  plugins like Native Instrument's Kontakt and Plogue's Sforzando, where
  instrument or effect selection is confined to the editor window, posing
  challenges for automated loading.

• Additionally, issues arise with plugins featuring presets incompatible with
  existing load methods, or being overly complicated in setting their parameters,
  as evidenced in previous issue reports:
  Save and Load Presets Automatically for VST3 Plugins? #187
  Loading non .vstpreset files with load_preset function #245
  VST plugin Program presets not accessible #257
  Cannot load .fxp file for Serum #277).

Solution:

• Implemented save_plugin_state and load_plugin_state methods in ExternalPlugin
  to address the above issues. Leveraging JUCE's getStateInformation and
  setStateInformation, these methods facilitate saving and loading plugin
  configurations to/from disk in much the same way that a DAW would.

• The save_plugin_state method also makes a copy of the accessible parameters
  which the load_plugin_state reapplies twice, similar to how Pedalboard's
  reinstantiatePlugin method works.

• The binary state information is converted to a base64 string using JUCE, and
  the recorded parameters are also converted to a string. The two data types are
  combined, being separated by a pipe character, which is not used in the base64
  encoding. The resulting character string is then saved to disk.

Using the new methods in Python is as simple as this example:

plugin = load_plugin('path/to/plugin/plugin.vst3')
plugin.show_editor()   // Manually configure the plugin to the desired state
plugin.save_plugin_state("path/to/your/directory/plugin.sav")

Later, to restore the plugin to the previously saved state:

plugin.load_plugin_state("path/to/your/directory/plugin.sav")

• Note that the extension on the save file does not have to be .sav, the user
  can name the file whatever they want. The data is just a character string
  stored in what is essentially a text file.

Results:

I tested the new methods on the following VST3 plugins and they work flawlessly:

• Plogue's Sforzando
• Native Instrument's Kontact 7
• Native Instrument's Guitar Rig 7
• Blue Cat Free Amp
• Vital (the wavetable synth)

I do not have access to Serum, so I can't verify that this will address
Issue #277, but Vital is the OSS counterpart of Serum and I was able to load a
previously saved state, which included instrument presets, without any issue.

Potential Problems:

• I am a beginner level, self taught programmer who has no experience working
  on team projects, so I don't know if I followed all of the best practices when
  writing this code. Reviewers should keep that in mind in case I messed
  something up.

• I noticed that Pedalboard's reinstantiatePlugin method goes through a number
  of steps when loading a state back into the plugin after resetting it. I only
  incorporated the part where, after the state is set, the parameters are
  reassigned twice. I didn't run into any problems, but I am not familiar with
  the situations that reinstantiatePlugin is trying to address so someone with
  more experience need to verify that there isn't going to be an issue there.

• The names of the parameter keys do not update if you switch instruments on an
  existing plugin instance by loading a plugin state. I am not sure how to fix
  this, but a best practice would be to load the parameter settings on a fresh
  instance of the plugin.

• There is currently no safeguard or error warning when if a user tries to load
  a state into the wrong plugin. It's currently up to the user to keep their
  saved state files organized. I am not sure how to fix this at this time.

• Unfortunately, I couldn't test this on a Mac, so I can't guarantee
  compatibility with AU plugins. In theory, it should work, but I don't want to
  implement a feature I can't test. Updating the code for AU plugins should be
  as simple as adding two lines for the pybind11 bindings.

• I did not add any type hints or updates to the documentation as I don't feel
  entirely confident about getting it right without causing more confusion.

[psobot]

Hi @Myuqou!

Thanks for this contribution - this looks really great!

I have a couple high-level suggestions before I think we can merge this:

• These new methods look like they work just fine - but they require the user to save the plugin state to a file. Pedalboard's design philosophy is to integrate with Python as much as possible; in keeping with that philosophy, it might be more intuitive to change these save/load methods into a .def_property, so the plugin's state becomes a property that users can do what they like with:

  # Instead of:
  my_plugin.save_plugin_state("to/filename")
  
  # ...we could use a property:
  len(my_plugin.state) # 12,345 bytes
  with open("my_filename.sav", "wb") as f:
      f.write(my_plugin.state)
  
  # ...which also allows people to do what they want with the
  # plugin state, without requiring them to write to disk:
  my_plugin.state = some_function(my_plugin.state)  # modify the plugin's internal state

• It would be great to add a test to test_external_plugins.py to ensure that these methods (or properties) behave as expected.

Let me know if you'd like to make these changes or if you'd prefer me to do so; I'd be happy to make these changes in this PR if you like.

[Myuqou]

As tempted as I am to take on the challenge, it would probably be a better idea for you to make the changes if you have the time as you are more familiar with the program.

I wasn't sure if capturing the parameters was even a necessary step as in every test I conducted I was able to simply save and load the state information without running into any problems. If the copy of the parameters is not needed then that would make it much easier to just grab the state information and pass it back to Python.

[brresnic]

just wanted to chime in that I'm looking forward to using this feature! ...might even just dev off of this branch, since this is so useful

[0xdevalias]

@psobot Let me know if you'd like to make these changes or if you'd prefer me to do so; I'd be happy to make these changes in this PR if you like.

@Myuqou As tempted as I am to take on the challenge, it would probably be a better idea for you to make the changes if you have the time as you are more familiar with the program.

@psobot Curious if you were still up for making the changes required to land this? Would be an awesome feature to have in pedalboard!

Or alternatively, perhaps this PR?

• Add binary state field to VST3 and AU plugins #297

From a quick skim of that PR, it looks like it's closer (def_property_readonly + def set_state ) to the suggested method (def_property).

I'm not sure if the 'separated read/write' approach in that PR is better than just a pure def_property as suggested here.

[0xdevalias]

Given #297 just got merged, wondering if this PR is still relevant?

Apologies that this took so long - I've just made a couple changes and merged this, and it should be available as of v0.9.6 (released later today).

I did rename this property to raw_state instead of just state, as the state data is often (but not always) encoded in a format that I hope we can parse and expose as a .state parameter later. (i.e.: if the state of a VST3 is valid XML, Pedalboard could unwrap and parse that XML directly to make the client code simpler.)

Originally posted by @psobot in #297 (comment)

Docs:

• https://spotify.github.io/pedalboard/reference/pedalboard.html#pedalboard.VST3Plugin.raw_state

  • A bytes object representing the plugin’s internal state.
    For the VST3 format, this is usually an XML-encoded string prefixed with an 8-byte header and suffixed with a single null byte.

• https://spotify.github.io/pedalboard/reference/pedalboard.html#pedalboard.AudioUnitPlugin.raw_state

  • A bytes object representing the plugin’s internal state.
    For the Audio Unit format, this is usually a binary property list that can be decoded or encoded with the built-in plistlib package.

[0xdevalias]

I was reading a bit more about saving/loading state for VST3, and it sounds like there are 2 different kinds of state:

• https://www.kvraudio.com/forum/viewtopic.php?t=597225

  • Vst3 has 2 sets of states, a processor state, which is the state of the audio processor (all parameter values etc) , and a controller state, which is the state of the gui and everything that is unique to that and not part of the audio processing.

    Iirc, a host calls Processor::SetState() with the processor state and then EditController::SetComponentState() with the same state, and finally EditController::SetState() with the controller state. Maybe not exactly in that order, but you get the idea.

• https://steinbergmedia.github.io/vst3_dev_portal/pages/Technical+Documentation/API+Documentation/Index.html

  • A VST 3 audio effect or instrument basically consists of two parts: a processing part and an edit controller part. The corresponding interfaces are:

    • Processor: Steinberg::Vst::IAudioProcessor + Steinberg::Vst::IComponent
    • Controller: Steinberg::Vst::IEditController

  • https://steinbergmedia.github.io/vst3_dev_portal/pages/Technical+Documentation/API+Documentation/Index.html#persistence

    • Persistence

      The host stores and restores the complete state of the processor and of the controller in project files and in preset files:

      • Steinberg::Vst::IComponent::getState + Steinberg::Vst::IComponent::setState store and restore the DSP model.
      • Steinberg::Vst::IEditController::getState + Steinberg::Vst::IEditController::setState store and restore any GUI settings that are not related to the processor (like scroll positions etc).
      • Restore: When the states are restored, the host passes the processor state to both the processor and the controller (Steinberg::Vst::IEditController::setComponentState). A host must always pass that state to the processor first. The controller then has to synchronize its parameters to this state (but must not perform any IComponentHandler callbacks). After restoring a state, the host rescans the parameters (asking the controller) in order to update its internal representation.

Am I right in assuming that previously we were able to access the 'processor state' (param values), and as of #297 we can now access the 'controller state'?

---

Edit: Looking at the code in the PR (Ref) it calls pluginInstance->getStateInformation and pluginInstance->setStateInformation; which appear to be JUCE methods:

• https://docs.juce.com/master/classAudioProcessor.html#a5d79591b367a7c0516e4ef4d1d6c32b2

  • getStateInformation()
virtual void AudioProcessor::getStateInformation (juce::MemoryBlock & destData)
    The host will call this method when it wants to save the processor's internal state.

    This must copy any info about the processor's state into the block of memory provided, so that the host can store this and later restore it using setStateInformation().

    Note that there's also a getCurrentProgramStateInformation() method, which only stores the current program, not the state of the entire processor.

    See also the helper function copyXmlToBinary() for storing settings as XML.

• https://docs.juce.com/master/classAudioProcessor.html#a6154837fea67c594a9b35c487894df27

  • setStateInformation()
virtual void AudioProcessor::setStateInformation (const void * data, int sizeInBytes)
    This must restore the processor's state from a block of data previously created using getStateInformation().

    Note that there's also a setCurrentProgramStateInformation() method, which tries to restore just the current program, not the state of the entire processor.

    See also the helper function getXmlFromBinary() for loading settings as XML.

Those docs also mention these following different methods:

• https://docs.juce.com/master/classAudioProcessor.html#aa8f9774ef205e4b19174f2de7664928f

  • getCurrentProgramStateInformation()
virtual void AudioProcessor::getCurrentProgramStateInformation(juce::MemoryBlock &destData)
    The host will call this method if it wants to save the state of just the processor's current program.

    Unlike getStateInformation, this should only return the current program's state.

    Not all hosts support this, and if you don't implement it, the base class method just calls getStateInformation() instead. If you do implement it, be sure to also implement setCurrentProgramStateInformation.

• https://docs.juce.com/master/classAudioProcessor.html#ade2c2df3606218b0f9fa1a3a376440a5

  • setCurrentProgramStateInformation()
virtual void AudioProcessor::setCurrentProgramStateInformation(const void * data, int sizeInBytes)
    The host will call this method if it wants to restore the state of just the processor's current program.

    Not all hosts support this, and if you don't implement it, the base class method just calls setStateInformation() instead. If you do implement it, be sure to also implement getCurrentProgramStateInformation.

• https://docs.juce.com/master/classAudioProcessor.html#a6d0c1c945bebbc967d187c0f08b42c4b

  • copyXmlToBinary()
static void AudioProcessor::copyXmlToBinary(const XmlElement & xml, juce::MemoryBlock & destData)
    Helper function that just converts an xml element into a binary blob.

    Use this in your processor's getStateInformation() method if you want to store its state as xml.

    Then use getXmlFromBinary() to reverse this operation and retrieve the XML from a binary blob.

• https://docs.juce.com/master/classAudioProcessor.html#a80c616e54758a0a411d27d6d76df956c

  • getXmlFromBinary()
static std::unique_ptr< XmlElement > AudioProcessor::getXmlFromBinary(const void * data, int sizeInBytes)
    Retrieves an XML element that was stored as binary with the copyXmlToBinary() method.

    This might return nullptr if the data's unsuitable or corrupted.

Looking at JUCE's code we can see the implementations for:

• AudioPluginAudioProcessor's getStateInformation / setStateInformation: https://github.com/juce-framework/JUCE/blob/4f43011b96eb0636104cb3e433894cda98243626/examples/CMake/AudioPlugin/PluginProcessor.h#L42-L43
  • https://github.com/juce-framework/JUCE/blob/4f43011b96eb0636104cb3e433894cda98243626/examples/CMake/AudioPlugin/PluginProcessor.cpp#L168-L181

    • You should use this method to store your parameters in the memory block.
      You could do that either as raw data, or use the XML or ValueTree classes
      as intermediaries to make it easy to save and load complex data.

  • VST3PluginInstance / AudioPluginInstance (getStateInformation / setStateInformation): https://github.com/juce-framework/JUCE/blob/4f43011b96eb0636104cb3e433894cda98243626/modules/juce_audio_processors/format_types/juce_VST3PluginFormat.cpp#L3055-L3109
    • getStateInformation seems to appendStateFrom with both IComponent and IEditController
    • setStateInformation also seems to have handling for both IComponent and IEditController
    • so I think this JUCE state method may handle both of them in a single call, which is convenient
• setComponentState, which calls Vst::EditController::setComponentState: https://github.com/juce-framework/JUCE/blob/master/modules/juce_audio_plugin_client/juce_audio_plugin_client_VST3.cpp#L1034-L1060
• EditController (which has setComponentState, getState, setState, etc): https://github.com/juce-framework/JUCE/blob/master/modules/juce_audio_processors/format_types/VST3_SDK/public.sdk/source/vst/vsteditcontroller.h#L68
• PresetFile::restoreComponentState, which calls editController->setComponentState: https://github.com/juce-framework/JUCE/blob/master/modules/juce_audio_processors/format_types/VST3_SDK/public.sdk/source/vst/vstpresetfile.cpp#L500-L509
• etc

This tutorial may also be interesting for futher/deeper reading:

• https://docs.juce.com/master/tutorial_audio_processor_value_tree_state.html

[psobot]

Apologies for the delay - #297 has now been merged, exposing a .raw_state property as of Pedalboard v0.9.6.