[Onnx] Integrate an interface to easily load exported ONNX models [will be updated]

[felixdittrich92]

🚀 The feature

We need to:

• ensure all exported models can provide the information we need (for example: which postprocessor, vocab ,etc.)
• setup build without TF and PT -> Onnxruntime should be enough
• build an equivalent to the current ocr_predictor
• make the predictor configurable about CPU / GPU (runtime configs)
• add ability to export models in detection/recognition reference scripts
• replace documentation section: preparing your model for inference with onnx saving / loading for prod

To solve before:

• [models] Replace current tf mobilenetv3 implementation with tf.keras.applications implementation #978

Other related Issues:
#789 #790

Related discussion:
#981

Motivation, pitch

As a user after exporting my models in onnx format i want to easily load these in docTR

Something like:

#-> same way as ocr_predictor

predictor = onnx_ocr_predictor(
    det_model='path/db_mobilenet_v3_small.onnx, 
    reco_model='path/crnn_mobilenet_v3_small.onnx,
    provider='gpu' (default='cpu')
)

#-> same way as ocr_predictor

[frgfm]

Hey Felix :)

Thanks for the suggestion!
So I have experimented with that and there are a few things I think we should consider:

• exporting to ONNX require way more than loading it : the original DL backend framework, the architecture, the parameter values and whatever dependencies allow the export
• loading ONNX is made for light environments & fast inference : we dont need the original DL backend framework, only ONNX and the archi+param file

So my point is that loading ONNX file with docTR is doable but it will be significantly heavier than loading in a separate lighter environments 👍

That being said, even if it's heavier, we could do it for developers to play with it. If so, we need to add a new supported backend 🤷‍♂️

What do you think?

[felixdittrich92]

Hi @frgfm 👋,

in short yes this would be the plan 😅
a bit more in detail what i have had in mind: #981

[felixdittrich92]

Add link in docs then we can close

https://github.com/felixdittrich92/OnnxTR