Releases: OpenInterpreter/open-interpreter
Local II Preview
Better local model support, bug fixes, faster load, new features. Official release + blog post coming soon.
v0.2.4
Fixed a potential recurring computer
import that occurs when teaching new skills to the 01
, along with several great fixes from the community:
What's Changed
- 01 by @KillianLucas in #1104
- fix for Files property missing on Computer module by @bars0um in #1107
- check that the message content is a string before calling strip() by @hackley in #1117
- Support OpenAI's new models. by @Steve235lab in #1099
New Contributors
Full Changelog: v0.2.3...v0.2.4
v0.2.3
interpreter --local
is really good now thanks to @tyfiero. Try it!
What's Changed
- The first development to main merge by @KillianLucas in #1077
- Better local messaging by @KillianLucas in #1078
- Change docs as --local no longer uses LM Studio by @aj47 in #1074
- Update by @KillianLucas in #1081
- Interactive local mode. Fixes errors from 0.2.1 change by @tyfiero in #1089
- Clear messages on %reset instead of instantiating a new Interpreter by @MikeBirdTech in #1095
- does not prompt openapi key when running --api_base by @lorenzejay in #1093
- Fix markdown in README_JA.md: Correct code block language and command⦠by @ishibashi-futos in #1091
New Contributors
- @aj47 made their first contribution in #1074
- @lorenzejay made their first contribution in #1093
- @ishibashi-futos made their first contribution in #1091
Full Changelog: v0.2.2...v0.2.3
The New Computer Update Part II (quick fixes)
Quick fixes for computer.mail
and computer.sms
.
The New Computer Update Part II
The New Computer Update Part II introduces our first local vision model, a suite of native Mac integrations, 5x launch speed, and dozens of other requested features.
Mac Control
The new calendar
, contacts
, browser
, mail
, and sms
Computer API modules let Open Interpreter control your Mac's native applications.
For example, Open Interpreter can execute the following code to get a contact, then send a text and an email with an attachment.
LLM-first Web Browser
The browser
lets Open Interpreter quickly browse the web by querying a web-enabled language model.
Point Model
We have developed a model called point
which is capable of locating visual controls precisely. It was designed to run locally on consumer hardware.
We leverage existing open-source models to "semantically search the screen" for text and icons. Language models can then call on this composite model to 'point' at text or icons on the screen:
While this model is capable of understanding simple interfaces with discrete icons, we intend to explore more general solutions in the next few weeks.
And more:
- 5X launch speed
- Experimental Docker support
- Export conversation to a Jupyter notebook using %jupyter
- Experimental one-click installers for Mac, Linux, and Windows
- Profiles preview (a feature similar to custom GPTs)
β¦
- Added all docs and folders by @tyfiero in #873
- Add safety and docker support documentation by @tyfiero in #882
- Fix cli crash during screenshot generation by @double-thinker in #884
- Add documentation for local models by @tyfiero in #889
- Fix config edit mac os by @MikeBirdTech in #927
- Add reference to open-interpreter-termux in README by @MikeBirdTech in #906
- Fix feedback from input api key by @Notnaton in #932
- Fix azure openai coding error by @MonsterCoder in #924
- Fixed %tokens command by @CyanideByte in #896
- update documentation to include the parameters by @Notnaton in #908
- Fix: NameError: name 'computer' is not defined by @MikeBirdTech in #937
- Update ROADMAP.md by @eltociear in #861
- Fix small typo in README.md by @jameynakama in #918
- Update README.md by @codesmith-emmy in #929
- Update local model docs by @MikeBirdTech in #931
- Profiles~ by @Notnaton in #933
- Introduced a new documentation structure, added demos, and more by @tyfiero in #948
- create new cli entry point and update poetry script by @MikeBirdTech in #947
- Added settings by @tyfiero in #949
- Update All Settings page for Open Interpreter Docs by @tyfiero in #967
- Move the entry point to start_terminal_interface by @Notnaton in #960
- Try to get some error logs from Markdown() by @Notnaton in #969
- Add missing llm. by @Notnaton in #975
- Fix the conversation restoration problem in version 0.2.0 New Computer by @Steve235lab in #977
- Remove getpass add prompt_toolkit by @Notnaton in #991
- [WIP] Fix broken unit tests. by @Steve235lab in #995
- chore: update aifs by @shivenmian in #997
- added string to multiple python file generation for skill library by @akuten1298 in #996
- add docs to computer by @birbbit in #1007
- Option to pass paths to docs.search by @birbbit in #1008
- Load skills without overwriting them by @birbbit in #1009
- Litellm custom provider by @Notnaton in #955
- Jupyter export magic command by @tyfiero in #986
- Docs for magic commands updated by @CyanideByte in #1004
- Add doc strings to all computer API functions for the LLM to use in the skill library by @tyfiero in #1011
- Fix broken link in lm-studio.mdx by @jorgepvenegas in #1019
- Update README.md by @MikeBirdTech in #1017
- Boost performance by lazy loading modules by @tyfiero in #1020
- Quick fix to sync the computer api between processes by @tyfiero in #1022
- fix parsing of a function by @birbbit in #1024
- Make messages accessible to interpreter by @birbbit in #1025
- Correct punctuation in the document. by @Dragonchu in #1033
- ROADMAP.md by @matthewrygh in #1029
- Resolve #1034 Migrate from appdirs to platformdirs by @lombad in #1038
- Multi-line input support. by @Steve235lab in #1048
- Fixed local.py not working on Windows. Added GPU acceleration. by @CyanideByte in #1043
- Rename the app directory to open-intepreter, and migrate all files from older versions by @tyfiero in #1060
- Expand computer api with texting, mail, contacts, and calendar by @tyfiero in #1061
- Fix profile migration bug by @tyfiero in #1066
- fix some error when start command on macos by @982945902 in #1062
New Contributors
- @double-thinker made their first contribution in #884
- @MonsterCoder made their first contribution in #924
- @jameynakama made their first contribution in #918
- @codesmith-emmy made their first contribution in #929
- @Steve235lab made their first contribution in #977
- @shivenmian β€οΈ made their first contribution in #997
- @akuten1298 made their first contribution in #996
- @birbbit made their first contribution in #1007
- @jorgepvenegas made their first contribution in #1019
- @Dragonchu made their first contribution in #1033
- @matthewrygh made their first contribution in #1029
- @lombad made their first contribution in #1038
- @982945902 made their first contribution in #1062
- Full Changelog*: v0.2.0...v0.2.1
The New Computer Update
β The New Computer Update
This is the most significant upgrade to Open Interpreter since 0.1.0
. Almost every component has been rewritten to support our project's objectiveβbuilding a standard interface between language models and computers.
Note: This update is not backwards compatible. If you use Open Interpreter in Python, please read our migration guide.
1. The Computer API
In 2023, Open Interpreter started building the world's first real-time code execution environment for language models.
Now, we're creating an API for language models to use in that environment, starting with basic I/O controls like display
, mouse
, and keyboard
:
computer.display.view() # Returns a screenshot to vision models
computer.mouse.click("On-screen text") # Locates text on-screen and clicks it
computer.mouse.move(icon="Description of icon") # Locates icon on-screen and hovers over it
computer.clipboard.view() # Returns the contents of the user's clipboard
# Full reference: https://docs.openinterpreter.com/computer-api/reference
We are also launching a free preview of the hosted tools that power computer.mouse
at api.openinterpreter.com.
2. OS
Mode
You can instruct Open Interpreter to use the Computer API to control your computer graphically:
interpreter --os
Even local vision models running via .llamafile
, LM-Studio
, or Jan.ai
are supported.
3. LMC Messages
To support the incoming L
anguage M
odel C
omputer architecture, the new messaging format that powers Open Interpreter extends OpenAI's messages format to include additional information, and a new role called computer
:
[
{
"role": "assistant",
"type": "code",
"format": "python",
"content": "plot = create_plot_from_data('data')\ndisplay_as_image(plot)\ndisplay_as_html(plot)"
},
{
"role": "computer",
"type": "image",
"format": "base64.png",
"content": "base64"
},
{
"role": "assistant",
"type": "message",
"content": "Plot generated successfully."
}
]
Read about LMC Messages
here.
And more...
In addition to these major updates, 0.2.0
comes with a suite of fixes and enhancement from our growing open-source community:
- Fixes crash UnboundLocalError active_block by @CyanideByte in #818
- Inserts dummy api key if missing by @CyanideByte in #808
- Fix README_JA.md by @tegnike in #810
- return empty dict if config file is empty by @sbendary25 in #811
- Add pyautogui mouse click functions and parameters by @Arrendy in #843
- Fixed the error when using Azure OpenAI API by @wsbao in #840
- Add package mismatches info to system_debug_info by @tegnike in #800
- Update keyboard control functions for better input handling by @Arrendy in #845
- implement mouse position function by @Arrendy in #850
- Fixed another a few bugs in using OpenAI API/Azure OpenAI API/OpenAI compatible custom API by @wsbao in #848
- Added new docs for litellm hosted models by @tyfiero in #858
- Added refreshed docs for terminal arguments and python streaming responses by @tyfiero in #864
- Add os docs by @tyfiero in #868
- Fixed the case where UnicodeDecodeError by @Moonlight-YS in #863
New Contributors
- @tegnike made their first contribution in #810
- @sbendary25 made their first contribution in #811
- @Arrendy made their first contribution in #843
- @wsbao made their first contribution in #840
- @tyfiero made their first contribution in #858
- @Moonlight-YS made their first contribution in #863
Full Changelog: v0.1.17...v0.2.0
This is only the beginning. Happy 2024. β
The New Computer Update (Preview)
β
The New Computer Update is the most significant update to Open Interpreter since v0.1.0
.
Ahead of its release, we're soft-launching this preview to get feedback on the update.
pip install --upgrade open-interpreter
interpreter --os
v0.1.17
Minor fix: gpt-4-1106-preview
β gpt-4
We decided to make gpt-4
no longer default to gpt-4-1106-preview
, which we've found performs worse on code interpretation tasks.
Important Note: This was not published from the repository. The last release, v0.1.16
, was simply patched to not switch to gpt-4-1106-preview
. The updates in the repo (and the commit associated with this release) are therefore out of sync with the binaries below.
We'll be back in sync by the next release of 0.2.0
, the New Computer update.
v0.1.16
Critical fixes and cool new magic commands in this one.
- New magic commands:
%% {insert shell code}
for custom shell code running and%info
added by @CyanideByte - Significantly better debugging with more system information in errors, thank you @Notnaton!
- Create multiple agents / instances and watch them communicate in Python
- Azure now works with the
config
thanks to @hargunmujral in #786 - @CyanideByte made it so if the user can't access
gpt-4
, we give a great error, and let them switch togpt-3.5
- @pratss10 added some helpful documentation (with screenshots!) for setting up
gpt-4
vision
now works via the Python package- Brand new German
README
thanks to @iongpt! - We're significantly more organized β one folder for the
terminal_interface
, another forcore
- Much better
shell
output processing - Finally, a proper ROADMAP!
Thank you to everyone that made it happen!
Cannot go without thanking the extraordinary @ericrallen for this release as well, who practically single-handedly brought our issues below 100 (from nearly 300 just a few weeks ago). Have a safe trip to Japan Eric!
New Contributors
- @joshuavial made their first contribution in #767
- @pratss10 made their first contribution in #773
- @hargunmujral made their first contribution in #786
- @iongpt made their first contribution in #738
Full Changelog: v0.1.15...v0.1.16
v0.1.15
Quick fixes to resolve some common issues:
- Streaming responses in Python missing flag issue has been fixed (
end_of_message
wasn't always being emitted) - Added more useful info to system info by @Notnaton in #750
- chore: popen using utf8 encoding by @fatelei in #742
- MOD: close subprocess and its file handles when reset by @coderfengyun in #476
New Contributors
- @fatelei made their first contribution in #742
- @coderfengyun made their first contribution in #476
Full Changelog: v0.1.14...v0.1.15