Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Mac上运行llm_demo报错:./llm_demo model/qwen-1.8b-int4 #2834

Open
luocf opened this issue Apr 18, 2024 · 3 comments
Open

【Mac上运行llm_demo报错:./llm_demo model/qwen-1.8b-int4 #2834

luocf opened this issue Apr 18, 2024 · 3 comments

Comments

@luocf
Copy link

luocf commented Apr 18, 2024

model name : Qwen_1.8b

hw.cpufamily: 458787763 , size = 4
The device support i8sdot:1, support fp16:1, support i8mm: 0
load tokenizer
load tokenizer Done
[ 10% ] load model/qwen-1.8b-int4/lm.mnn model ... Done!
[ 13% ] load model/qwen-1.8b-int4/embedding.mnn model ... Done!
[ 20% ] load model/qwen-1.8b-int4/block_0.mnn model ... Done!
[ 24% ] load model/qwen-1.8b-int4/block_1.mnn model ... Done!
[ 27% ] load model/qwen-1.8b-int4/block_2.mnn model ... Done!
[ 31% ] load model/qwen-1.8b-int4/block_3.mnn model ... Done!
[ 34% ] load model/qwen-1.8b-int4/block_4.mnn model ... Done!
[ 38% ] load model/qwen-1.8b-int4/block_5.mnn model ... Done!
[ 41% ] load model/qwen-1.8b-int4/block_6.mnn model ... Done!
[ 45% ] load model/qwen-1.8b-int4/block_7.mnn model ... Done!
[ 48% ] load model/qwen-1.8b-int4/block_8.mnn model ... Done!
[ 52% ] load model/qwen-1.8b-int4/block_9.mnn model ... Done!
[ 55% ] load model/qwen-1.8b-int4/block_10.mnn model ... Done!
[ 58% ] load model/qwen-1.8b-int4/block_11.mnn model ... Done!
[ 62% ] load model/qwen-1.8b-int4/block_12.mnn model ... Done!
[ 65% ] load model/qwen-1.8b-int4/block_13.mnn model ... Done!
[ 69% ] load model/qwen-1.8b-int4/block_14.mnn model ... Done!
[ 72% ] load model/qwen-1.8b-int4/block_15.mnn model ... Done!
[ 76% ] load model/qwen-1.8b-int4/block_16.mnn model ... Done!
[ 79% ] load model/qwen-1.8b-int4/block_17.mnn model ... Done!
[ 83% ] load model/qwen-1.8b-int4/block_18.mnn model ... Done!
[ 86% ] load model/qwen-1.8b-int4/block_19.mnn model ... Done!
[ 90% ] load model/qwen-1.8b-int4/block_20.mnn model ... Done!
[ 93% ] load model/qwen-1.8b-int4/block_21.mnn model ... Done!
[ 97% ] load model/qwen-1.8b-int4/block_22.mnn model ... Done!
[100% ] load model/qwen-1.8b-int4/block_23.mnn model ... Done!

Q: 你好!

A: Reshape error: 28160 -> 26624
Compute Shape Error for onnx::Cast_30
code=3 in onForward, 372
Segmentation fault: 11

@jxt1234
Copy link
Collaborator

jxt1234 commented Apr 23, 2024

  1. 现在建议用完整模型,即合并为同一模型的用法
  2. mnn 版本是什么?

@luocf
Copy link
Author

luocf commented Apr 28, 2024

模型用的是这个:qwen-1.8b-int4
mnn版本用是github上最新版本:
commit 976d1d7 (HEAD -> master, origin/master, origin/HEAD)
Merge: 2d926e4 ed68e16
Author: jxt1234 jxt1234@zju.edu.cn
Date: Mon Apr 8 16:17:33 2024 +0800

Merge pull request #2812 from juju812/patch-2

[bugfix] add return value to lambda of std::async

@luocf
Copy link
Author

luocf commented Apr 28, 2024

》〉现在建议用完整模型,即合并为同一模型的用法
请问,如何合并为一个模型?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants