[prompty] Prompty supports image as input

[lalala123123]

Description

prompty

---
name: Basic Prompt with Image
description: A basic prompt that uses the GPT-3 chat API to answer questions
model:
    api: chat
    configuration:
      type: azure_openai
      azure_deployment: gpt-4-vision-preview
      connection: azure_open_ai_connection
    parameters:
      temperature: 0.2
sample:
  image: "data:image/jpg;path:../datas/logo.jpg"
  question: what is it
---
system:
As an AI assistant, your task involves interpreting images and responding to questions about the image.
Remember to provide accurate answers based on the information present in the image.
Directly give the answer, no more explanation is needed.

# user:
{{question}}
![image]({{image}})

prompty = Prompty.load(source="path/to/prompty.prompty")

# Local image path
prompty(image="data:image/jpg;path:path/to/image.jpg")
# image url
prompty(image="data:image/jpg;url:http://link.to.image.jpg")

# or image obj
from promptflow.contracts.multimedia import Image

image = Image(image_buffer, mime_type="image/png")
prompty(image=image)

Trace:

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

All Promptflow Contribution checklist:

• The pull request does not introduce [breaking changes].
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.
• Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[github-actions]

promptflow SDK CLI Azure E2E Test Result zhrua/prompty_image_input

  4 files    4 suites   4m 11s ⏱️
243 tests 203 ✅  40 💤 0 ❌
972 runs  812 ✅ 160 💤 0 ❌

Results for commit 08b4d02.

[github-actions]

SDK CLI Global Config Test Result zhrua/prompty_image_input

6 tests   6 ✅  1m 12s ⏱️
1 suites  0 💤
1 files    0 ❌

Results for commit 08b4d02.

[github-actions]

promptflow-core test result

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 08b4d02.

[github-actions]

Executor Unit Test Result zhrua/prompty_image_input

796 tests   796 ✅  3m 43s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 08b4d02.

[github-actions]

Executor E2E Test Result zhrua/prompty_image_input

246 tests   240 ✅  5m 2s ⏱️
  1 suites    6 💤
  1 files      0 ❌

Results for commit 08b4d02.

[github-actions]

SDK CLI Test Result zhrua/prompty_image_input

    4 files      4 suites   1h 4m 37s ⏱️
  784 tests   740 ✅  44 💤 0 ❌
3 136 runs  2 960 ✅ 176 💤 0 ❌

Results for commit 08b4d02.