New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mnn推理时间异常 #2850
Labels
question
Further information is requested
Comments
测试用哪个模型?有参考的开源模型吗? |
|
这是用benchmark跑的结果
|
1.好的,我这样编译和推理试一下
|
1.关于1打开bf16解了一些编译问题,测试发现: bf16更慢也是能感受到的,不是单次误差,测过多次loop=50,都是bf16的推理时间更长。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
两个现象:
1.fp32和fp16的推理时间一样
2.int8的推理时间大于fp16/fp32
armv8,linux aarch64
推理代码上,唯一的区别就是fp32是Precision_High,fp16和int8是Precision_Low。
可能的原因是什么呢?谢谢
The text was updated successfully, but these errors were encountered: