Mismatched type issue

[deephog]

Hi, do you know what may cause this kind of mismatched type issue?

[PINTO0309]

I have never done INT8 quantization of ONNX before running trtexec.

[PINTO0309]

https://github.com/PINTO0309/simple-onnx-processing-tools/releases/tag/1.0.41

[PINTO0309]

https://github.com/PINTO0309/simple-onnx-processing-tools

[deephog]

Two further questions:

1. What do you recommend as a more practical way to quantize a model and convert it to tensorrt, if I am working on a Stereo-Depth model that usually involve tricky layers such as "gridSample", "Where", "Gather" and etc..

2. And also, to this specific issue, forget about the quantization part, I checked the onnx file of the model after simplification, the layer that reports a mismatched type has an input node of INT64 and a output node of INT64, how come itself is expecting a flaot32? And if it is caused by either ONNX generation of Pytorch or ONNX-simplifier, and I wanted to manually revise the properties using your tool, what should I revise, If I couldn't even find a "float32" to work on?

Thanks!

[PINTO0309]

What do you recommend as a more practical way to quantize a model and convert it to tensorrt, if I am working on a Stereo-Depth model that usually involve tricky layers such as "gridSample", "Where", "Gather" and etc..

I think the only way to do this is to replace all of them with OPs that support TensorRT. Before converting to ONNX.

And also, to this specific issue, forget about the quantization part, I checked the onnx file of the model after simplification, the layer that reports a mismatched type has an input node of INT64 and a output node of INT64, how come itself is expecting a flaot32? And if it is caused by either ONNX generation of Pytorch or ONNX-simplifier, and I wanted to manually revise the properties using your tool, what should I revise, If I couldn't even find a "float32" to work on?

I don't know anything about it because I don't own the ONNX file before quantization.

[PINTO0309]

https://github.com/megvii-research/Sparsebit

[deephog]

Thank you for the suggestion, I will try their own tool to see if it works better.

Actually now quantization is no longer the immediate issue, I found that the model works very differently even under FP16. It is reasonable since a disparity estimation model will definitely be more sensitive to minor weight value changes.

Do you know any good method of doing FP16 aware training? Pytorch seems only to provide mixed precision training which is of no use to this situation.

[PINTO0309]

I am not aware of any way to train regular means using methods other than mixing precision that avoids the effects of batch normalization, etc.
https://pytorch.org/docs/stable/notes/amp_examples.html