Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

鲲鹏920环境session推理采用fp16推理结果错误 #2855

Open
zhenjing opened this issue Apr 29, 2024 · 2 comments
Open

鲲鹏920环境session推理采用fp16推理结果错误 #2855

zhenjing opened this issue Apr 29, 2024 · 2 comments
Labels
question Further information is requested

Comments

@zhenjing
Copy link

代码:
MNN::ScheduleConfig sConfig;
sConfig.type = static_cast(MNN_FORWARD_CPU);
sConfig.numThread = threadNum;
BackendConfig bConfig;
bConfig.precision = static_castBackendConfig::PrecisionMode(precision);
sConfig.backendConfig = &bConfig;
auto session = net->createSession(sConfig);

当precision=2时,推理结果是错误,其他精度可正常推理。
若采用module方式,启用fp16推理结果正常,速度可加速。

看文档这么写: 后端 CPU precision 为 Low 时,根据设备情况开启 FP16 计算 precision 为 Low_BF16 时,根据设备情况开启 BF16 计算
session应该怎么开启fp16?

@zhenjing
Copy link
Author

版本:2.8.4

@jxt1234
Copy link
Collaborator

jxt1234 commented May 8, 2024

大概率是 session api 的调用代码问题。fp16 的输入输出必须用 copyFromHost / copyToHost ,不能直接访问 tensor 的 host 指针。建议都用 Module API.

@jxt1234 jxt1234 added the question Further information is requested label May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants