Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

how to do Post-training integer quantization with int16 activation #279

Closed
FengWu-PKU opened this issue Jan 29, 2024 · 4 comments
Closed

how to do Post-training integer quantization with int16 activation #279

FengWu-PKU opened this issue Jan 29, 2024 · 4 comments
Labels
question Further information is requested

Comments

@FengWu-PKU
Copy link

模型在int8 PTQ后精度下降很多，请问现在支持int16 activation吗，谢谢
@FengWu-PKU
Copy link
Author

以及钉钉群的二维码过期吗，能否麻烦更新一下

@peterjc123 peterjc123 added the question Further information is requested label Jan 29, 2024
@peterjc123
Copy link
Collaborator

以及钉钉群的二维码过期吗，能否麻烦更新一下

二维码已经更新了

@peterjc123
Copy link
Collaborator

模型在int8 PTQ后精度下降很多，请问现在支持int16 activation吗，谢谢

Int16量化我们是支持的（详见 #47 ），但是一般来说后端的实现还不太成熟，可能导致模型推理要比int8慢很多，一般来说还是建议进一步分析是哪些层导致量化的损失，然后在转换时跳过这些层的量化。

@FengWu-PKU
Copy link
Author

好的，十分感谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@peterjc123 @FengWu-PKU