Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MNN 在 iOS平台上编译mnn_2.8.0_ios_llm.zip release版本的方法 #2845

Open
xingjinglu opened this issue Apr 25, 2024 · 2 comments
Open

Comments

@xingjinglu
Copy link

xingjinglu commented Apr 25, 2024

现在是在mac上,通过 project/ios/下的项目直接打开进行编译。现在的问题:
1)编译出来的MNN库大小是8.1MB,比 2.8.0 release的版本的要大;
2)执行llm推理时,会存在占用内存过大被杀死的问题;

但是使用下面版本的MNN库,就可以正常图例qwen1.8b-int8模型,能否提供一下编译mnn_2.8.0_ios_llm.zip 的方法。
https://github.com/alibaba/MNN/releases/download/2.8.0/mnn_2.8.0_ios_llm.zip

@xingjinglu xingjinglu changed the title MNN 在 iOSrelease版本的 MNN 在 iOS平台上编译mnn_2.8.0_ios_llm.zip release版本的方法 Apr 25, 2024
@xingjinglu
Copy link
Author

解决了,修改一下 project/ios/ xcode项目的配置即可,具体如下:

1)# 预处理编译选项 Preprocessor Macros
$(inherited)
MNN_CODEGEN_REGISTER=1
MNN_METAL_ENABLED=1
ENABLE_ARMV82=1
MNN_COREML_ENABLED=1
USE_LZ4_FLAG=1
MNN_USE_SPARSE_COMPUTE=1
MNN_LOW_MEMORY=1

2) 添加下面文件
source/backend/arm82/asm/arm64/low_memory/
source/backend/cpu/arm/arm64/low_memory

3)针对下面的上面的文件设置编译选项:-march=armv8.2-a+fp16

具体可以参考 https://github.com/xingjinglu/MNN/tree/memory_compress/project/ios/MNN.xcodeproj

@jxt1234
Copy link
Collaborator

jxt1234 commented Apr 29, 2024

一般参考 package_scripts/ios/buildiOS.sh 用脚本编译,额外加上编译参数 -DMNN_LOW_MEMORY=ON

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants