[Feat] A competitive Web Browsing agent #1856
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Example logs: |
xingyaoww
approved these changes
May 21, 2024
There was a problem hiding this comment.
LGTM! This mainly adds a browsing agent to the agent hub and tweaked a little bit about browser env. I think we can approve it to unblock the integration of BrowserGym.
EDIT: I also locally tested and confirmed the sample command works on my end!
PS: When we figure out a way to do task decomposition, CodeAct can eventually delegate tasks to this BrowserAgent for complex web browsing tasks!
yufansong
approved these changes
May 21, 2024
There was a problem hiding this comment.
Leave some nits. Mostly LGTM. I would be appricate it if you can add more comments or simply elaborate your design and some parameter setting. Then other people can add more work on your codebase. I don't want to block our integration progress and AP it. I can help for some follow up refactor or nits if you have no time.
|
Thanks! @yufansong I added some comments for things that are not clear. Hope it's good for now -- since I changed the BrowserOutputObservation a bit, the integration tests are failing for some, would you mind taking a look how to fix those? EDIT: NVM, just fixed those, should be ready to go |
|
Sad, our project test coverage reduced by 5.87%... let me see if there's anything we could do to test this. |
|
I've made some progress in creating an integration test for this agent! Will create a PR in a day. |
li-boxuan
reviewed
May 22, 2024
| ) | ||
|
|
||
|
|
||
| class SystemPrompt(PromptElement): |
There was a problem hiding this comment.
@frankxu2004 this prompt (along with many other prompts in this file) seems unused? Is it by intention?
There was a problem hiding this comment.
Yes, basically this whole prompt.py file is not currently used. Currently the agent is a simplified version for ease of understanding. However I included here with the intention of incorporating a more complex agent using more comprehensive information as next steps. Here it's still useful as it provides others of building blocks of prompts and understanding what possible information to include as context for LLMs.
There was a problem hiding this comment.
These PRs are mostly for chasing the neurips paper deadline so not all features are implemented yet.
There was a problem hiding this comment.
I see, sounds fair. I am just having a bit trouble reproducing poetry run python ./opendevin/core/main.py -i 5 -t "tell me the usa's president using google search" -c BrowsingAgent -m gpt-4o-2024-05-13... I tried like 5 times and only succeeded once.
There was a problem hiding this comment.
that's a bit weird, what error are you seeing? do you have logs
There was a problem hiding this comment.
Currently the agent does not return AgentFinishAction, so to the eyes of the frame, it's always error in the end. Maybe I should add this Finish thing
There was a problem hiding this comment.
There was a problem hiding this comment.
Yeah, sometimes it's like this. I improved the agent a bit and fixed some issues here #1993
super-dainiu
pushed a commit
to super-dainiu/OpenDevin
that referenced
this pull request
May 23, 2024
* initial attempt at a browsing only agent * add browsing agent * update * implement agent * update * fix comments * remove unnecessary things from memory extras * update image processing --------- Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims at enabling a competitive browsing agent for #1470.
Now I transplanted the simplified demo agent used in WebArena in our agent hub.
To test, it works best with GPT-4 LLMs such as GPT-4o.