Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add documentation for google vertex ai backend #93

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 26 additions & 2 deletions docs/reference/providers/backend.md
Expand Up @@ -2,14 +2,15 @@

A Backend (also called Provider) is a service that provides access to the AI language model. There are many different backends available for K8sGPT. Each backend has its own strengths and weaknesses, so it is important to choose the one that is right for your needs.

Currently, we have a total of 8 backends available:
Currently, we have a total of 9 backends available:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to update this once the Hugging Face Docs is merged


- [OpenAI](https://openai.com/)
- [Cohere](https://cohere.com/)
- [Amazon Bedrock](https://aws.amazon.com/bedrock/)
- [Amazon SageMaker](https://aws.amazon.com/sagemaker/)
- [Azure OpenAI](https://azure.microsoft.com/en-us/products/cognitive-services/openai-service)
- [Google Gemini](https://ai.google.dev/docs/gemini_api_overview)
- [Google Vertex AI](https://cloud.google.com/vertex-ai)
- [LocalAI](https://github.com/go-skynet/LocalAI)
- FakeAI

Expand Down Expand Up @@ -110,13 +111,36 @@ Google [Gemini](https://blog.google/technology/ai/google-gemini-ai/#performance)
- To use Google Gemini API in K8sGPT, obtain [the API key](https://ai.google.dev/tutorials/setup).
- To configure Google backend in K8sGPT with `gemini-pro` model (see all [models](https://ai.google.dev/models) here) use auth command:
```bash
k8sgpt auth add --backend google --model gemini-pro --password "<Your API KEY>"
k8sgpt auth add --backend googlevertexai --model gemini-pro --password "<Your API KEY>"
```
- Run the following command to analyze issues within your cluster with the Google provider:
```bash
k8sgpt analyze --explain --backend google
```

## Google Gemini via Vertex AI

Google [Gemini](https://blog.google/technology/ai/google-gemini-ai/#performance) allows generative AI capabilities with multimodal approach (it is capable to understand not only text, but also code, audio, image and video).

- To use [Google Vertex AI](https://cloud.google.com/vertex-ai?#build-with-gemini) you need to be authorized via [Google Cloud SDK](https://cloud.google.com/sdk/install).
The [Vertex AI API](https://console.cloud.google.com/apis/library/vertex-ai.googleapis.com) needs to be enabled.

> Note: Vertex AI Gemini API is currently available in these [regions](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini?hl=de#http_request)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this expected to change? Maybe write something more along the lines of:
"Verify that the Vertex AI Gemini API is available in your region"

This makes it sound more timeless independent to how things change on the Google side :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree to some degree, the only issue I have is, you can basically use the other regions to debug clusters not in this region. I added a more clarification hope this fits!


- Open a terminal or command prompt and run the following command to authenticate using your Google Cloud credentials:
```bash
gcloud auth application-default login
```

- To configure Google backend in K8sGPT with `gemini-pro` model (see all [models](https://ai.google.dev/models) here) use auth command:
```bash
k8sgpt auth add --backend googlevertexai --model "gemini-pro" --providerRegion "us-central1" --providerId "<your project id>"
```
- Run the following command to analyze issues within your cluster with the Google provider:
```bash
k8sgpt analyze --explain --backend googlevertexai
```

## LocalAI

LocalAI is a local model, which is an OpenAI compatible API. It uses llama.cpp and ggml to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.
Expand Down