Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sweep: modify sweepai/utils/diff_test.py to use pytest instead of unittest #2953

Open
wwzeng1 opened this issue Jan 16, 2024 · 1 comment
Open
Labels
sweep Assigns Sweep to an issue or pull request.

Comments

@wwzeng1
Copy link
Contributor

wwzeng1 commented Jan 16, 2024

No description provided.

@wwzeng1 wwzeng1 added the sweep Assigns Sweep to an issue or pull request. label Jan 16, 2024
Copy link
Contributor

sweep-nightly bot commented Jan 16, 2024

Sweeping

25%

💎 Sweep Pro: I'm using GPT-4. You have unlimited GPT-4 tickets. (tracking ID: None)

Tip

I can email you when I complete this pull request if you set up your email here!


Actions (click)

  • ↻ Restart Sweep

Step 1: 🔎 Searching

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I think are relevant in decreasing order of relevance (click to expand). If some file is missing from here, you can mention the path in the ticket description.

import unittest
from unittest.mock import patch
from sweepai.utils.diff import (
format_contents,
is_markdown,
match_string,
revert_whitespace_changes,
)
class TestDiff(unittest.TestCase):
def test_revert_whitespace_changes(self):
original_file_str = " line1\n line2\n line3"
modified_file_str = "line1\n line2\n line3"
expected_output = " line1\n line2\n line3"
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_revert_whitespace_changes_more_whitespace(self):
original_file_str = "line1\nline2\nline3"
modified_file_str = " line1\n line2\n line3"
expected_output = "line1\nline2\nline3"
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_revert_whitespace_changes_non_whitespace_changes(self):
original_file_str = "line1\nline2\nline3"
modified_file_str = "line4\nline5\nline6"
expected_output = "line1\nline2\nline3"
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_revert_whitespace_changes_same_files(self):
original_file_str = "line1\nline2\nline3"
modified_file_str = "line1\nline2\nline3"
expected_output = "line1\nline2\nline3"
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_revert_whitespace_changes_empty_files(self):
original_file_str = ""
modified_file_str = ""
expected_output = ""
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_revert_whitespace_changes_whitespace_only_files(self):
original_file_str = " \n \n "
modified_file_str = " \n \n "
expected_output = " \n \n "
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_format_contents(self):
file_contents = "line1\nline2\nline3"
expected_output = "line1\nline2\nline3"
self.assertEqual(format_contents(file_contents), expected_output)
@patch("sweepai.utils.diff.find_best_match")
def test_match_string(self, mock_find_best_match):
original = ["line1", "line2", "line3"]
search = ["line2"]
mock_find_best_match.return_value = 1
self.assertEqual(match_string(original, search), 1)
def test_is_markdown(self):
filename = "test.md"
self.assertTrue(is_markdown(filename))
if __name__ == "__main__":

=======
def test_revert_whitespace_changes_empty_files(self):
print("Testing revert_whitespace_changes with empty original and modified files")
>>>>>>> MODIFIED
<<<<<<< ORIGINAL
def test_revert_whitespace_changes_whitespace_only_files(self):
=======
def test_revert_whitespace_changes_whitespace_only_files(self):
print("Testing revert_whitespace_changes with files containing only whitespace")
>>>>>>> MODIFIED
<<<<<<< ORIGINAL
def test_format_contents(self):
=======
def test_format_contents(self):
print("Testing format_contents simplicity")
>>>>>>> MODIFIED
<<<<<<< ORIGINAL
@patch("sweepai.utils.diff.find_best_match")
def test_match_string(self, mock_find_best_match):
=======
@patch("sweepai.utils.diff.find_best_match")
def test_match_string(self, mock_find_best_match):
print("Testing match_string function with a mock for find_best_match")
>>>>>>> MODIFIED
<<<<<<< ORIGINAL
def test_is_markdown(self):
=======
def test_is_markdown(self):
print("Testing is_markdown function for markdown file determination")
>>>>>>> MODIFIED`;
const mockFileContents = String.raw`import unittest
from unittest.mock import patch
from sweepai.utils.diff import (
format_contents,
is_markdown,
match_string,
revert_whitespace_changes,
)
class TestDiff(unittest.TestCase):
def test_revert_whitespace_changes(self):
original_file_str = " line1\n line2\n line3"
modified_file_str = "line1\n line2\n line3"
expected_output = " line1\n line2\n line3"
self.assertEqual(

import unittest
from sweepai.utils.comment_utils import check_comments_presence
class TestCommentUtils(unittest.TestCase):
def test_check_comments_presence(self):
# Test Python file with a comment
self.assertEqual(
check_comments_presence("test.py", "# This is a comment"), True
)
# Test Python file without a comment
self.assertEqual(
check_comments_presence("test.py", 'print("Hello, World!")'), False
)
# Test JavaScript file with a comment
self.assertEqual(
check_comments_presence("test.js", "// This is a comment"), True
)
# Test JavaScript file without a comment
self.assertEqual(
check_comments_presence("test.js", 'console.log("Hello, World!");'), False
)
# Test unsupported file type with a comment
self.assertEqual(
check_comments_presence("test.txt", "# This is a comment"), False
)
# Test unsupported file type without a comment
self.assertEqual(check_comments_presence("test.txt", "Hello, World!"), False)
if __name__ == "__main__":

import { mock } from "node:test";
jest.mock("openai", () => {
return {
OpenAI: jest.fn().mockImplementation(() => ({
chat: {
completions: {
create: jest.fn().mockResolvedValue({
choices: [{ message: { content: "mocked response" } }],
}),
},
},
})),
};
});
const { parseRegexFromOpenAI } = require("./route.ts");
const expectedAnswer = String.raw`import unittest
from unittest.mock import patch
from sweepai.utils.diff import (
format_contents,
is_markdown,
match_string,
revert_whitespace_changes,
)
class EnhancedDiffTest(unittest.TestCase):
def test_revert_whitespace_changes(self):
print("Testing revert_whitespace_changes with typical input")
original_file_str = " line1\n line2\n line3"
modified_file_str = "line1\n line2\n line3"
expected_output = " line1\n line2\n line3"
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)
def test_revert_whitespace_changes_more_whitespace(self):
print("Testing revert_whitespace_changes with more whitespace in modified file")
original_file_str = "line1\nline2\nline3"
modified_file_str = " line1\n line2\n line3"
expected_output = "line1\nline2\nline3"
self.assertEqual(
revert_whitespace_changes(original_file_str, modified_file_str),
expected_output,
)

import pytest
from sweepai.utils.utils import check_syntax
@pytest.mark.parametrize(
"file_path, code, expected_validity, expected_message",
[
("file.tsx", "let x = 1;", True, ""),
("file.tsx", "let x = ;", False, "Invalid syntax found within or before the lines 0-0, displayed below:\nlet x = ;"),
("file.py", "x = 1", True, ""),
],
)
def test_check_syntax(file_path, code, expected_validity, expected_message):
validity, message = check_syntax(file_path, code)
assert validity == expected_validity

<snippet file_path=”test.py” start_line=”1” end_line=”32”>
import pytest
...
</snippet>
...
</relevant_snippets>
<repo_tree>
.gitignore
jp-app/
|- App.xaml
|- App.xaml.cs
| |- namespace jp_app
| |- class App
| |- method App (ISystemLanguageService systemLanguageService)
| |- method OnStart ()
</repo_tree>
Repo name: sweepai/sweep: an AI junior dev
Username: kevinlu1248
Query: Sweep: Use os agnostic temp directory for windows
```
### External Search
On top of the code snippet search, we also perform external search:
1. **Public websites**: leaving a link to a publicly accessible site will make Sweep read the contents of the page.
2. **Documentation search**: GPT-4, unfortunately, has only been trained until 2022 so it won’t have access to the latest docs, so we set up search indices for docs that update once a day. The current list of docs that get indexed can be found at https://github.com/sweepai/sweep/blob/main/sweepai/pre_indexed_docs.py, and we’ll write another post describing how we fetch the docs soon. Feel free to create a PR to add docs you would like to see Sweep use!
## Planning
![](/sweep-core-algo/plan.png)
Sweep then curates a list of files it wants to modify and those it wants to create by:
1. Determining what the root causes of the issues are. E.g. the bug has to do with `/tmp` not existing on Windows.
2. Determining in natural language an implementation plan using chain-of-thought prompting. E.g. replace all occurrences of `/tmp` with `tempfile.TemporaryDirectory()`.
3. Specify a list of files to modify, like the following:
```xml
<modify_file>
* filename_3: instructions_3
* filename_4: instructions_4
...
</modify_file>
<create_file>
* filename_1: instructions_1
* filename_2: instructions_2
...

```python
@patch("os.path.join")
- @patch("open")
+ @patch("builtins.open")
def test_get_file_contents(self, mock_open, mock_join):
```
We ran the tests again, and finally we have unit tests that actually work! 😀
## Visualizing Sweep's process
Sweep updates you with a flowchart showing you the code it's written and the tests it's run.
Sweep starts off by writing the original `github_utils_tests.py` file. <br></br>
Then Sweep runs the tests using `python sweepai/utils/github_utils_test.py`. (this command is configurable)
![image](/flowchart/flowchart_2.png)
Sweep found the error, and suggests a new modification to the tests.
![image](/flowchart/flowchart_3.png)
Finally, we used the test feedback to fix the code and run the tests again. If there was a bug in the original `github_utils.py`, we would have fixed it here.
![image](/flowchart/flowchart_4.png)
## Becoming an AI junior developer
The benefits of this approach seem small at first: “Ok, now I don’t need to fix the tests”.
But if you just wanted to write unit tests, you might as well use ChatGPT.
With this approach, Sweep can not only write the test, but:
1. find bugs in the original code
2. *fix the original code*
3. run the tests again to confirm the bugfix
One step closer to becoming an AI developer.
<style>{`
pre {
white-space: pre-wrap !important;
word-wrap: break-word !important;
}
code {
white-space: pre-wrap !important;
}


Step 2: ⌨️ Coding

Working on it...


🎉 Latest improvements to Sweep:
  • New dashboard launched for real-time tracking of Sweep issues, covering all stages from search to coding.
  • Integration of OpenAI's latest Assistant API for more efficient and reliable code planning and editing, improving speed by 3x.
  • Use the GitHub issues extension for creating Sweep issues directly from your editor.

💡 To recreate the pull request edit the issue title or description.
Something wrong? Let us know.

This is an automated message generated by Sweep AI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sweep Assigns Sweep to an issue or pull request.
Projects
None yet
Development

No branches or pull requests

1 participant