GPT Deploy: One line to generate them all 🧙🚀

Turn your natural language descriptions into fully functional, deployed AI-powered microservices with a single command! Your imagination is the limit!

gptdeploy-tiny.mp4

This project streamlines the creation and deployment of AI-powered microservices. Simply describe your task using natural language, and the system will automatically build and deploy your microservice. To ensure the microservice accurately aligns with your intended task a test scenario is required.

Quickstart

Requirements

• OpenAI key with access to GPT-3.5 or GPT-4

Installation

pip install gptdeploy
gptdeploy configure --key <your openai api key>

If you set the environment variable OPENAI_API_KEY, the configuration step can be skipped. Your api key must have access to gpt-4 to use this tool. We are working on a way to use gpt-3.5-turbo as well.

Generate Microservice

gptdeploy generate \
--description "<description of the microservice>" \
--test "<specification of a test scenario>" \
--model <gpt-3.5 or gpt-4> \
--path </path/to/local/folder>

To generate your personal microservice two things are required:

• A description of the task you want to accomplish.
• A test scenario that ensures the microservice works as expected.
• The model you want to use - either gpt-3.5 or gpt-4. gpt-3.5 is ~10x cheaper, but will not be able to generate as complex microservices.
• A path on the local drive where the microservice will be generated.

The creation process should take between 5 and 15 minutes. During this time, GPT iteratively builds your microservice until it finds a strategy that make your test scenario pass.

Be aware that the costs you have to pay for openai vary between $0.50 and $3.00 per microservice (using GPT-4).

Run Microservice

Run the microservice locally in docker. In case docker is not running on your machine, it will try to run it withouth docker. With this command a playground opens in your browser where you can test the microservice.

gptdeploy run --path <path to microservice>

Deploy Microservice

If you want to deploy your microservice to the cloud a Jina account is required. When creating a Jina account, you get some free credits, which you can use to deploy your microservice ($0.025/hour). If you run out of credits, you can purchase more.

gptdeploy deploy --microservice_path <path to microservice>

Delete Microservice

To save credits you can delete your microservice via the following commands:

jc list # get the microservice id
jc delete <microservice id>

Examples

In this section you can get a feeling for the kind of microservices that can be generated with GPT Deploy.

Extract and summarize news articles given a URL

gptdeploy generate \
--description "Extract text from a news article URL using Newspaper3k library and generate a summary using gpt." \
--test "input: 'http://fox13now.com/2013/12/30/new-year-new-laws-obamacare-pot-guns-and-drones/' output: assert a summarized version of the article exists" \
--model gpt-4 \
--path microservice

Chemical Formula Visualization

gptdeploy generate \
--description "Convert a chemical formula into a 2D chemical structure diagram" \
--test "C=C, CN=C=O, CCC(=O)O" \
--model gpt-4 \
--path microservice

2d rendering of 3d model

gptdeploy generate \
--description "create a 2d rendering of a whole 3d object and x,y,z object rotation using trimesh and pyrender.OffscreenRenderer with os.environ['PYOPENGL_PLATFORM'] = 'egl' and freeglut3-dev library" \
--test "input: https://graphics.stanford.edu/courses/cs148-10-summer/as3/code/as3/teapot.obj output: assert the image is not completely white or black" \
--model gpt-4 \
--path microservice

Product Recommendation

gptdeploy generate \
--description "Generate personalized product recommendations based on user product browsing history and the product categories fashion, electronics and sport" \
--test "Test that a user how visited p1(electronics),p2(fashion),p3(fashion) is more likely to buy p4(fashion) than p5(sports)" \
--model gpt-4 \
--path microservice

Hacker News Search

gptdeploy generate \
--description "Given a search query, find articles on hacker news using the hacker news api and return a list of (title, author, website_link, first_image_on_the_website)" \
--test "searching for GPT gives results" \
--model gpt-4 \
--path microservice

Animal Detector

gptdeploy generate \
--description "Given an image, return the image with bounding boxes of all animals (https://pjreddie.com/media/files/yolov3.weights, https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg)" \
--test "https://images.unsplash.com/photo-1444212477490-ca407925329e contains animals" \
--model gpt-4 \
--path microservice

Meme Generator

gptdeploy generate \
--description "Generate a meme from an image and a caption" \
--test "Surprised Pikachu: https://media.wired.com/photos/5f87340d114b38fa1f8339f9/master/w_1600%2Cc_limit/Ideas_Surprised_Pikachu_HD.jpg, TOP:When you discovered GPTDeploy" \
--model gpt-4 \
--path microservice

Rhyme Generator

gptdeploy generate \
--description "Given a word, return a list of rhyming words using the datamuse api" \
--test "hello" \
--model gpt-4 \
--path microservice

Word Cloud Generator

gptdeploy generate \
--description "Generate a word cloud from a given text" \
--test "Lorem ipsum dolor sit amet, consectetur adipiscing elit." \
--model gpt-4 \
--path microservice

3d model info

gptdeploy generate \
--description "Given a 3d object, return vertex count and face count" \
--test "https://raw.githubusercontent.com/polygonjs/polygonjs-assets/master/models/wolf.obj" \
--model gpt-4 \
--path microservice

Table extraction

gptdeploy generate \
--description "Given a URL, extract all tables as csv" \
--test "http://www.ins.tn/statistiques/90" \
--model gpt-4 \
--path microservice

Audio to mel spectrogram

gptdeploy generate \
--description "Create mel spectrograms from audio file" \
--test "https://cdn.pixabay.com/download/audio/2023/02/28/audio_550d815fa5.mp3" \
--model gpt-4 \
--path microservice

Text to speech

gptdeploy generate \
--description "Convert text to speech" \
--test "Hello, welcome to GPT Deploy!" \
--model gpt-4 \
--path microservice

Your browser does not support the audio element.

Heatmap Generator

gptdeploy generate \
--description "Create a heatmap from an image and a list of relative coordinates" \
--test "https://images.unsplash.com/photo-1574786198875-49f5d09fe2d2, [[0.1, 0.2], [0.3, 0.4], [0.5, 0.6], [0.2, 0.1], [0.7, 0.2], [0.4, 0.2]]" \
--model gpt-4 \
--path microservice

QR Code Generator

gptdeploy generate \
--description "Generate QR code from URL" \
--test "https://www.example.com" \
--model gpt-4 \
--path microservice

Mandelbrot Set Visualizer

gptdeploy generate \
--description "Visualize the Mandelbrot set with custom parameters" \
--test "center=-0+1i, zoom=1.0, size=800x800, iterations=1000" \
--model gpt-4 \
--path microservice

Markdown to HTML Converter

gptdeploy generate --description "Convert markdown to HTML" --test "# Hello, welcome to GPT Deploy!"

Technical Insights

The graphic below illustrates the process of creating a microservice and deploying it to the cloud elaboration two different implementation strategies.

graph TB

    description[description: generate QR code from URL] --> make_strat{think a}

    test[test: https://www.example.com] --> make_strat[generate strategies]

    make_strat --> implement1[implement strategy 1]

    implement1 --> build1{build image}

    build1 -->|error message| implement1

    build1 -->|failed 10 times| implement2[implement strategy 2]

    build1 -->|success| registry[push docker image to registry]

    implement2 --> build2{build image}

    build2 -->|error message| implement2

    build2 -->|failed 10 times| all_failed[all strategies failed]

    build2 -->|success| registry[push docker image to registry]

    registry --> deploy[deploy microservice]

    deploy --> streamlit[generate streamlit playground]

    streamlit --> user_run[user tests microservice]

1. GPT Deploy identifies several strategies to implement your task.
2. It tests each strategy until it finds one that works.
3. For each strategy, it generates the following files:

• microservice.py: This is the main implementation of the microservice.
• test_microservice.py: These are test cases to ensure the microservice works as expected.
• requirements.txt: This file lists the packages needed by the microservice and its tests.
• Dockerfile: This file is used to run the microservice in a container and also runs the tests when building the image.

4. GPT Deploy attempts to build the image. If the build fails, it uses the error message to apply a fix and tries again to build the image.
5. Once it finds a successful strategy, it:

• Pushes the Docker image to the registry.
• Deploys the microservice.
• Generates a Streamlit playground where you can test the microservice.

6. If it fails 10 times in a row, it moves on to the next approach.

🔮 vision

Use natural language interface to generate, deploy and update your microservice infrastructure.

✨ Contributers

If you want to contribute to this project, feel free to open a PR or an issue. In the following, you can find a list of things that need to be done.

next steps:

• check if windows and linux support works
• add video to README.md
• bug: it can happen that the code generation is hanging forever - in this case aboard and redo the generation
• new user has free credits but should be told to verify account

Nice to have:

• smooth rendering animation of the responses
• if the user runs gptdeploy without any arguments, show the help message
• don't show this message: 🔐 You are logged in to Jina AI as florian.hoenicke (username:auth0-unified-448f11965ce142b6). To log out, use jina auth logout.
• put the playground into the custom gateway (without rebuilding the custom gateway)
• hide prompts in normal mode and show them in verbose mode
• tests
• clean up duplicate code
• support popular cloud providers - lambda, cloud run, cloud functions, ...
• support local docker builds
• autoscaling enabled for cost saving
• add more examples to README.md
• support multiple endpoints - example: todolist microservice with endpoints for adding, deleting, and listing todos
• support stateful microservices
• The playground is currently printed twice even if it did not change. Make sure it is only printed twice in case it changed.
• allow to update your microservice by providing feedback
• support for other large language models like Open Assistent
• for cost savings, it should be possible to insert less context during the code generation of the main functionality - no jina knowledge is required
• use gptdeploy list to show all deployments
• gptdeploy delete to delete a deployment
• gptdeploy update to update a deployment
• test param optional - but how would you test the pdf extractor without a pdf?
• section for microservices built by the community
• test feedback for playground generation (could be part of the debugging)
• should we send everything via json in the text attribute for simplicity?
• fix release workflow
• after the user specified the task, ask them questions back if the task is not clear enough or something is missing

Proposal:

• just generate the non-jina related code and insert it into an executor template
• think about strategies after the first approach failed?