New (Python) example: simple drawings (rectangle, ellipse, arrow) #238

gvanrossum · 2024-04-18T00:38:22Z

For me, this works best with openai model gpt-3.5-turbo; gpt-4 seems worse (?!).

Here's a sample session (no longer valid without history support)

PATH=/opt/homebrew/opt/ccache/libexec:/opt/homebrew/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/guido/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/bash:/Users/guido/TypeChat/python/.venv/bin:/Library/Frameworks/Python.framework/Versions/3.13/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/Users/guido/.nvm/versions/node/v19.8.1/bin:/opt/homebrew/opt/ccache/libexec:/opt/homebrew/bin

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
(.venv) ~/TypeChat/python$ PYTHONPATH=src python examples/drawing/demo.py 
~> draw three squares of side 50 in a diagonal
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw three squares of side 50 in a diagonal\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
ReadTimeout('') raised from within internal TypeChat language model.
~> draw two squares of side 50 in a diagonal
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    }
  ]
}
~> make it three squares
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    },
    {
      "type": "Box",
      "x": 100,
      "y": 100,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    }
  ]
}
~> label them according to size
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 100,\n      "y": 100,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nlabel them according to size\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "Small",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "Medium",
      "style": null
    },
    {
      "type": "Box",
      "x": 100,
      "y": 100,
      "width": 50,
      "height": 50,
      "text": "Large",
      "style": null
    }
  ]
}
~> oops, label them according to position
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 100,\n      "y": 100,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nlabel them according to size\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "Small",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "Medium",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 100,\n      "y": 100,\n      "width": 50,\n      "height": 50,\n      "text": "Large",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\noops, label them according to position\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

retrying (ReadTimeout('')) ...
{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "First",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "Second",
      "style": null
    },
    {
      "type": "Box",
      "x": 100,
      "y": 100,
      "width": 50,
      "height": 50,
      "text": "Third",
      "style": null
    }
  ]
}
~>

The idea is that you can write a small amount of JavaScript that reads the JSON and renders in an HTML5 canvas, or some Python that renders it in Tkinter, or anything else that strikes your fancy. I would like to use this to draw diagrams representing the stack and memory layout for an interpreter, to be used in internal docs.

DanielRosenwasser · 2024-04-18T01:07:23Z

There's some stuff here that makes intuitive sense, but I'll need to check in with others (CC @ahejlsberg)

I would think that the best format for incorporating chat history would be something like

System: The assistant is a bot that responds in JSON according to a schema.
...: In-between messages alternating between user/assistant
User: Some request
System: Translate the prior request with JSON that satisfies Type in the following schema:

That gives background for the current convo, plus reinforces the current task at hand. This PR is pretty close to that.

gvanrossum · 2024-04-22T15:49:59Z

I wonder how to proceed. Clearly this PR two things (adding history and adding a new example) and that's not great. Your fixes to the translator (#240) broke my PR and I would rather not try to merge it back. I propose to add my example without chat history and start a separate discussion on chat history -- okay?

(Just append the inputs, separated by newlines.)

gvanrossum added 5 commits April 17, 2024 17:01

Install python-dotenv and tabulate

bb9e32f

Add chat history to drawing demo

de03aa9

[WIP] Hack on translation a bit

2edaedd

New API for chat history

09ea859

gvanrossum added 4 commits April 18, 2024 17:48

Improved the schema with ChatGPT's help. Added Arrow and Ellipse.

c52aa53

rendering program using Tkinter

5d07a2a

Add a simple rendering function (most of it written by ChatGPT)

f46a836

Added dashed and dotted arrows (some help from ChatGPT)

131ad4f

DanielRosenwasser mentioned this pull request Apr 20, 2024

Actually test the Python translator #240

Merged

Merge commit '66fd7bb' into drawing

3f44129

gvanrossum added 2 commits April 22, 2024 13:20

Rip oput history, for now

b90faa7

Make pyright happy

02e28e4

gvanrossum changed the title ~~A sketch for how chat history could be added to translate()~~ New example: simple drawings (rectangle, ellipse, arrow) Apr 22, 2024

Merge branch 'main' into drawing

3e1f3e6

gvanrossum changed the title ~~New example: simple drawings (rectangle, ellipse, arrow)~~ New (Python) example: simple drawings (rectangle, ellipse, arrow) Apr 22, 2024

gvanrossum added 2 commits April 22, 2024 13:58

Keep pydantic also happy

e9ac5cf

Add (flawed) history based on prompt_preamble

9e14f42

This was referenced Apr 22, 2024

Conversations rather than one-offs #39

Open

Proposed alternate prompting scheme #241

Draft

gvanrossum added 8 commits April 22, 2024 16:37

Prune history

75c2efe

Add temporary logging

970d50b

Alternate prompting scheme from microsoft#241

7560273

Revert changes to translator.py

439f521

Merge branch 'main' into drawing

b0fdd8d

Use Anders' suggestion for history

d385e40

(Just append the inputs, separated by newlines.)

Switch schema to dataclasses

42ce16a

Black formatting

c6da914

gvanrossum added 6 commits May 2, 2024 16:06

Remove unused imports

390558e

Clarify Failure output

e42b2dd

Revert verbose logging

11edfb7

Reuse drawing window

3f099dc

Don't crash if we cannot import readline (e.g. on Windows)

5c6e15f

Add crude progress messages

0894ef6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New (Python) example: simple drawings (rectangle, ellipse, arrow) #238

New (Python) example: simple drawings (rectangle, ellipse, arrow) #238

gvanrossum commented Apr 18, 2024 •

edited

DanielRosenwasser commented Apr 18, 2024 •

edited

gvanrossum commented Apr 22, 2024

New (Python) example: simple drawings (rectangle, ellipse, arrow) #238

Are you sure you want to change the base?

New (Python) example: simple drawings (rectangle, ellipse, arrow) #238

Conversation

gvanrossum commented Apr 18, 2024 • edited

DanielRosenwasser commented Apr 18, 2024 • edited

gvanrossum commented Apr 22, 2024

gvanrossum commented Apr 18, 2024 •

edited

DanielRosenwasser commented Apr 18, 2024 •

edited