Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with koboldcpp #4

Open
pbz134 opened this issue May 8, 2024 · 1 comment
Open

Error with koboldcpp #4

pbz134 opened this issue May 8, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@pbz134
Copy link

pbz134 commented May 8, 2024

Describe the bug
When trying to connect the program to koboldcpp's API (localhost:5001/v1) it throws an error.

To Reproduce
Install infinite-sides as normal, then launch a .gguf model with koboldcpp and try to connect infinite-sides to the API address of koboldcpp.

Expected behavior
infinite-sides should connect to koboldcpp and display the currently running model in the selection menu.

Screenshots
grafik

Desktop (please complete the following information):

  • OS: Windows
@pbz134 pbz134 added the bug Something isn't working label May 8, 2024
@Fus3n
Copy link
Owner

Fus3n commented May 9, 2024

Currently Infinite sides only support ollama or any other software that supports ollama's API, so you might need to install ollama and models through it, i could add support for others but that might break some parts of the app.
If you want custom models i think ollama has modelfiles you can check that out,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants