Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Whisper support #652

Open
Lyrcaxis opened this issue Apr 6, 2024 · 7 comments
Open

Whisper support #652

Lyrcaxis opened this issue Apr 6, 2024 · 7 comments

Comments

@Lyrcaxis
Copy link
Contributor

Lyrcaxis commented Apr 6, 2024

Hi, is anyone interested in Speech-To-Text support? (whisper.cpp)

If someone is willing to create the native handles for it I'd be glad to contribute the mid-level & high-level bindings for it.

@AsakusaRinne
Copy link
Collaborator

Hi, speech-to-text is an interesting topic. However I don't think we would include the full binding of whisper.cpp in LLamaSharp because there is already a binding for it (whisper.net). It will be great if you would like to add an example to LLamaSharp to show how to use it to implement the workflow of audio-text-response with whisper.net and LLamaSharp. :)

@Lyrcaxis
Copy link
Contributor Author

Lyrcaxis commented Apr 6, 2024

Neat I wasn't aware of that library. I'll give it a try over the weekend and post back!

@AsakusaRinne
Copy link
Collaborator

I'm looking forward to that!

@Lyrcaxis
Copy link
Contributor Author

Lyrcaxis commented Apr 7, 2024

I do have a satisfying example where speech is automatically being recognized and sent to the LLM to invoke responses, resulting in speech-text chat, but it requires adding the Whisper.NET & NAudio packages to the solution -- how should this be approached for the LlamaSharp example?

@AsakusaRinne
Copy link
Collaborator

Is Whisper.NET & NAudio cross-platform? To me it's okay to add the two packages to LLama.Example project of this repo as long as they are cross-platform (working on Windows/Linux/MAC)

@Lyrcaxis
Copy link
Contributor Author

Lyrcaxis commented Apr 7, 2024

👌Yup, both are cross platform! I'll add them and make a PR later today, then!

@AsakusaRinne
Copy link
Collaborator

No rush for that. :) Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants