Demo: Integrating Azure OpenAI and Azure Kubernetes Service to build Your Own Intelligent Apps

• Last Updated: 20230912
• TODO
  • GitHub Workflow and Terraform setup
    • Use the Azure login action with OpenID Connect
    • Apply Terraform to create resource group via GitHub Workflows
    • Destroy resource group via GitHub Workflows
  • Create AKS
    • VNET
    • Public AKS
    • Enable Azure Service Mesh
  • Surrounding Services
    • Azure Log Analytics Workspace
    • Azure Monitor managed service for Prometheus
    • Azure Managed Grafana
    • Azure OpenAI (AOAI): The service should be provisioned by manual, and get the API key
    • Workload Identiy with OIDC
    • Azure Key Vault Secret Provider
  • Put a Customer GPT Service on AKS + GitOps
    • Flux Podinfo
    • Canary Deployment with Azure Service Mesh
    • AOAI Application: pichuang/chatgpt-lite
  • Static Code Analysis
    • aquasecurity/tfsec: Security scanner for your Terraform code
    • stackrox/kube-linter: Static analysis tool that checks Kubernetes YAML files and Helm charts to ensure the applications represented in them adhere to best practices.
    • Microsoft Security DevOps GitHub action

Prerequisites

1. Register provider and feature

   #!/bin/bash
   az account set --subscription "YOUR_SUBSCRIPTION_ID"
   az account show
   az provider register --namespace "Microsoft.Dashboard" --wait
   az provider register --namespace "Microsoft.Kubernetes"
   az feature register --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"
   az feature register --namespace "Microsoft.ContainerService" --name "NodeOsUpgradeChannelPreview"
   az feature register --namespace "Microsoft.ContainerService" --name "AKS-ExtensionManager"
   az feature register --namespace "Microsoft.ContainerService" --name "EnableAPIServerVnetIntegrationPreview"
   sleep 600

2. Create a new Resource Group

   • Setup a Storage Account for Terraform State
   • Setup User Managed Identity
   • Setup a Key Vault for saving AOAI secret

Components Version

Azure Service	Support Agreement	Version
Azure Kubernetes Service	GA	1.27.3
Azure Service Mesh (a.k.a Istio Service Mesh)	Preview	1.17
GitOps Flux v2	GA	v2.0.1
Azure Monitor managed service for Prometheus	GA
Azure Managed Grafana	GA	v9.5.6 (859a2654d3)
Azure AI services - Azure OpenAI (AOAI)	GA	gpt-35-turbo (0301)
Azure Key Vault Secrets Provider	GA
Microsoft Defender for Cloud - DevOps security	Preview	v1.7.2

OSS Project	Version
blrchen/chatgpt-lite	latest
aquasecurity/tfsec	lastet (v.1.28.1)
stackrox/kube-linter	v1.0.4

Architecture

References

• pichuang/k8s-deployment-strategies-azure-edition
• terraform-github-actions/GitHub Actions Workflows for Terraform
• Use the Azure login action with OpenID Connect
• Using OIDC with Terraform in GitHub Actions
• stefanprodan/podinfo
• microsoft/sample-app-aoai-chatGPT
• Microsft Build: Integrating Azure AI and Azure Kubernetes Service to build intelligent apps
• k8sgpt-ai/k8sgpt
• Empowering AI: Building and Deploying Azure AI Landing Zones with Terraform
• Building a Private ChatGPT Interface With Azure OpenAI
• mckaywrigley/chatbot-ui
• flux/mozilla-sops/#azure
• Yidadaa/ChatGPT-Next-Web
• External Secrerts Operator - GitOps using FluxCD
• 4-5. Cloud Native New Year Wrap Up

Seminar Information

• Name: DevDays Asia 2023
• Agenda: Integrating Azure OpenAI and Azure Kubernetes Service to build Your Own Intelligent Apps
• Spearker: Phil Huang @pichuang
• Date: Wed., Sep. 13, 2023