Classification wrappers that expose the same API for the following Deep Learning Embedded Runtimes:
- TensorFlow Lite 2.7.0
- Torchscript/Pytorch C++
- ONNX Runtime
- RTNeural
We also provide an improved wrapper for TFlite 2.5.3 with support for 2d input. However, that version of the API is different and not compatible with the others
Each folder contains scripts and library binaries to compile libraries for the Elk Audio OS
/** Dynamically allocate an instance of a classifier object (do not use in real time threads!) */
ClassifierPtr createClassifier(const std::string &filename, bool verbose = false);
/** Feed a feature array (C Array) to the model, perform inference and return the prediction */
int classify(ClassifierPtr cls, const float featureVector[], size_t numFeatures, float outputVector[], size_t numClasses);
template<std::size_t IN_SIZE, std::size_t OUT_SIZE>
int classify(ClassifierPtr cls, std::array<float,IN_SIZE>& featureArray, std::array<float,OUT_SIZE>& outputArray);
/** Free the classifier memory (do not use in real time threads) */
void deleteClassifier(ClassifierPtr cls);
The classify(...) function is meant to be called from real-time threads (e.g., audio thread) so it is meant to be real-time safe, if the interpreter at hand allows for rt-safe inference.
RT-safety was tested for:
- TensorFlow Lite
- TorchScript (Yes, but classify must be called once during the first call of the audio processing function, otherwise it allocates memory. See this)
- Onnx Runtime (Yes, but same as TorchScript. See this)
- RtNeural
The createClassifier(...) and deleteClassifier(...) functions will mess with memory allocation and are definitely not meant to be called from real-time thread, but only once on program start and end.
The interactive notebook tensorflow_model_conversion.ipynb contains utilities to convert a TensorFlow model to the formats accepted by each runtime.
Domenico Stefani, Simone Peroni 2022