Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add documentation for google vertex ai backend #93

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 26 additions & 2 deletions docs/reference/providers/backend.md
Expand Up @@ -2,14 +2,15 @@

A Backend (also called Provider) is a service that provides access to the AI language model. There are many different backends available for K8sGPT. Each backend has its own strengths and weaknesses, so it is important to choose the one that is right for your needs.

Currently, we have a total of 8 backends available:
Currently, we have a total of 10 backends available:

- [OpenAI](https://openai.com/)
- [Cohere](https://cohere.com/)
- [Amazon Bedrock](https://aws.amazon.com/bedrock/)
- [Amazon SageMaker](https://aws.amazon.com/sagemaker/)
- [Azure OpenAI](https://azure.microsoft.com/en-us/products/cognitive-services/openai-service)
- [Google Gemini](https://ai.google.dev/docs/gemini_api_overview)
- [Google Vertex AI](https://cloud.google.com/vertex-ai)
- [LocalAI](https://github.com/go-skynet/LocalAI)
- FakeAI

Expand Down Expand Up @@ -110,13 +111,36 @@ Google [Gemini](https://blog.google/technology/ai/google-gemini-ai/#performance)
- To use Google Gemini API in K8sGPT, obtain [the API key](https://ai.google.dev/tutorials/setup).
- To configure Google backend in K8sGPT with `gemini-pro` model (see all [models](https://ai.google.dev/models) here) use auth command:
```bash
k8sgpt auth add --backend google --model gemini-pro --password "<Your API KEY>"
k8sgpt auth add --backend googlevertexai --model gemini-pro --password "<Your API KEY>"
```
- Run the following command to analyze issues within your cluster with the Google provider:
```bash
k8sgpt analyze --explain --backend google
```

## Google Gemini via Vertex AI

Google [Gemini](https://blog.google/technology/ai/google-gemini-ai/#performance) allows generative AI capabilities with multimodal approach (it is capable to understand not only text, but also code, audio, image and video).

- To use [Google Vertex AI](https://cloud.google.com/vertex-ai?#build-with-gemini) you need to be authorized via [Google Cloud SDK](https://cloud.google.com/sdk/install).
The [Vertex AI API](https://console.cloud.google.com/apis/library/vertex-ai.googleapis.com) needs to be enabled.

> Note: Vertex AI Gemini API is currently available in these [regions](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini?hl=de#http_request), verify if those are working for your environment

- Open a terminal or command prompt and run the following command to authenticate using your Google Cloud credentials:
```bash
gcloud auth application-default login
```

- To configure Google backend in K8sGPT with `gemini-pro` model (see all [models](https://ai.google.dev/models) here) use auth command:
```bash
k8sgpt auth add --backend googlevertexai --model "gemini-pro" --providerRegion "us-central1" --providerId "<your project id>"
```
- Run the following command to analyze issues within your cluster with the Google provider:
```bash
k8sgpt analyze --explain --backend googlevertexai
```

## LocalAI

LocalAI is a local model, which is an OpenAI compatible API. It uses llama.cpp and ggml to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.
Expand Down