Increase CPU usage cap from 400% to higher number, better if configurable

[tuobulatuo]

Hi Song Lab Team,

I noticed that the CPU usage is capped at 4 threads (400%). Can you please make it a configurable number? For example, if I have a machine with 10 CPUs, I can use all of them.

Thanks!
Alex

[RaymondWang0]

Hi @tuobulatuo, thanks for your valuable suggestion. We've made a quick fix to support a configurable number of threads for matrix multiplication on CPU platforms, as shown in README. You can now put this in the command line to configure different numbers of threads:

./chat <model_name> <precision> <num_threads>

I hope this helps. Thanks!