-
Notifications
You must be signed in to change notification settings - Fork 344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Does it support the new GGMLv3 quantization methods? #286
Comments
Hi there! Yes, it's supported, but only on the latest version ( |
My apologies, should've tried the main branch instead of just trying the release 😅 |
No worries - I'll keep this up for now and pin it for people's reference until we get it out the door :) |
@philpax have you considered making some |
Hi there! Yeah, I've considered it, but the main blocker is #221 - I don't want to cut a release where the interface is going to be radically different in the next release. I'm hoping to have this all closed out within the next week or two, especially with GGUF on the horizon, but I've been quite busy. |
Tried using the cli application to see how far it had come from being llama-rs, and noticed that an error popped up using one of the newer WizardLM uncensored models using the GGMLv3 method,
Am I using it the wrong way or is it not supported yet?
The text was updated successfully, but these errors were encountered: