llama.cpp integration with DLL

[amakropoulos]

Describe the feature

LLM for Unity uses the llamafile server for the LLM functionality.
This approach can't be used for mobile integrations due to security limitations.
Purpose of this feature is to instead integrate llama.cpp directly as DLL.
This feature doesn't necessarily mean to replace llamafile for PCs as this will need quite some testing and optimisation on different OS+CPUs

[otdavies]

How is this task going?

[amakropoulos]

It's progressing well, it should be done in the next 1-2 weeks

[amakropoulos]

is there something blocking you at the moment?

[otdavies]

I was just curious and prefer integrated dlls when possible. The tooling is impressive as is, I tested it with llama 3 this evening. I will keep an eye on this task. Let me know if you need any assistance testing etc.

[amakropoulos]

I would love some testing, thank you!
I'll send you when I'm at that stage.

[SubatomicPlanets]

Is there any other difference this will make except adding Android support? What about performance and file size?