You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am seeing 100% RAM usage in my laptop when running this. Can you please let me know if its an issue or the minimum RAM requirement is higher than 8 GB?
#324
Hi @devashish234073, if you look at https://github.com/mlc-ai/web-llm/blob/main/examples/simple-chat/src/gh-config.js, there is a field called vram_required_MB for each model. I would say it is an optimistic estimation and the actual usage should be higher than this field. Looking at llama 7b q4f32 specifically, it is indeed around 8GB. I would suggest perhaps using smaller models (e.g. the 3B ones), and use f16 if your browser/device supports that.
The text was updated successfully, but these errors were encountered: