Include an image in a message into a thread

[leandrosardi]

I want to create and run a message into a thread, including the URL to an image.

Is it possible to do something like this in the current version?

Example:

            mid = @@openai_client.messages.create(
                'thread_id': @@openai_thread_id,
                'parameters': {
                    'role': "user", # Required for manually created messages
                    'content': "What do you see in this picture?", # Required.
                    'image_url': 'https://static.vecteezy.com/system/resources/thumbnails/039/391/155/small/500-social-media-stories-v2-abstract-shopping-nft-e-commerce-memories-bundle.png',
                },
            )["id"]

            @@openai_message_ids << mid

            # run the assistant
            response = @@openai_client.runs.create(
                thread_id: @@openai_thread_id,
                parameters: {
                    assistant_id: @@openai_assistant_id,
                }
            )
            run_id = response['id']

[willywg]

If you are referring to GPT-4 with vision, it's currently not possible with the Assistant API. However, you can use it with the Chat Completions API.

See: https://platform.openai.com/docs/guides/vision

Plese note that the Assistants API does not currently support image inputs.

[willywg]

@leandrosardi now it's possible. Look at this: https://platform.openai.com/docs/assistants/how-it-works/creating-image-input-content