3 Billion Model is throwing error

[okpatil4u]

1.5 billion model works as intended. But 3 billion model refuses to convert on my system.

(mps) ➜  verbaflow git:(main) ✗ ./verbaflow -model-dir models/nlpodyssey/rwkv-4-pile-3b-instruct download
(mps) ➜  verbaflow git:(main) ✗ ./verbaflow -model-dir models/nlpodyssey/rwkv-4-pile-3b-instruct convert
5:28PM FTL error="model conversion failed: failed to convert embeddings: only BFloat16Storage is supported, actual *pytorch.FloatStorage"

Similarly, 1.5 billion model works well with original BlinkDL/RWKV-LM code, but 3 billion model throws dimension mismatch error. Is current 3B model working correctly at your end ?

[matteo-grella]

Thank you @okpatil4u for giving the 3B model a try. It appears that the 3B model has an extra conversion that renders it incompatible with the current converter. Rest assured that I will be uploading a revised version of the 3B model within the week. If you require it sooner, please let me know, and I can provide you with straightforward instructions on how to do it yourself.

May I ask what architecture you are using? I'm interested in hearing about the performance of the 1B5 version with your setup.

[okpatil4u]

Thanks Matteo, no need to hurry. I am using Mac M1 Max. I have been trying to get RWKV working on Core ML, but it seems that RNNs are not supported. But it was interesting to see Verbaflow built on Golang.

This is a pretty cool project !