Using AI deck 1.1 for model, how to use Python on GAP8

[cli48]

Hello,

I'm trying to use the AI deck 1.1 to run a google media pipeline CNN for hand detection. I have a TFlite model, BUT I require the GAP8 to run a python script to do data pre-processing as the model is deployed. Both examples in the aideck-gap8 examples solely rely on model deployment and do not have any seeable python code to preprocess input data before feeding to the model.

In short, I need help with how to get python on the GAP8, how to build custom libraries like OpenCV, NumPy and run a python script on the GAP8 while the model is deployed. The following is what we are attempting to build on the GAP8. If anyone has a solution or a similar project that would like to share their experience, it would be much appreciated.

https://github.com/Ahayat10/HandGesture/tree/main

Thanks,
Calvin Li

[knmcguire]

Hi! You will need to contact the gap8 manufacturer Greenwaves for this (we use their gap_sdk). However to our knowledge, this is not an chip that supports python or micropython. You can contact them for that request, however the standard procedure is to use gapflow in order to convert your model to fit on the aideck's gap8.

You can take a look at this tutorial for some inspiration: https://www.bitcraze.io/documentation/repository/aideck-gap8-examples/master/ai-examples/classification-demo/

[Jool99]

Hi!

We also tried to port mediepipe hands to the gap8 the "normal" way by taking the smallest tflite and adjusting, fusing, quantizing and autotiling it with Greenwaves nntool, but it was too large from what we saw.

[gemenerik]

If I may suggest an alternative course of action; I took a look at the TFlite model in the attached repository and I think you should perfectly be able to get that to run using our normal workflow.

I require the GAP8 to run a python script to do data pre-processing as the model is deployed. Both examples in the aideck-gap8 examples solely rely on model deployment and do not have any seeable python code to preprocess input data before feeding to the model.

As @knmcguire said, the GAP8 does not support Python. I would recommend you program your preprocessing operations in C. We have some image preprocessing examples. Depending on what kind of operations you want to apply to your input data, you can consider including a lightweight image processing library.

[Jool99]

I'm so sorry if my last comment set you off track!

With "google media pipeline CNN for hand detection" I thought you meant this:

https://github.com/google/mediapipe/blob/master/docs/solutions/models.md

And we got stuck on nntool not supporting PRELU.

[gemenerik]

I think I was mistaken instead. Although there are some very small tflite models in the repository, there indeed seems to be a reference to "mediapipe" models in the code. The models you linked to indeed seem to be pretty big and unsupported ops may be a concern