Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sweep: add a /version endpoint to api.py that tells you which version of sweep we're running #3151

Open
6 tasks done
wwzeng1 opened this issue Feb 23, 2024 · 1 comment · May be fixed by #3156
Open
6 tasks done

Sweep: add a /version endpoint to api.py that tells you which version of sweep we're running #3151

wwzeng1 opened this issue Feb 23, 2024 · 1 comment · May be fixed by #3156
Labels
sweep Assigns Sweep to an issue or pull request.

Comments

@wwzeng1
Copy link
Contributor

wwzeng1 commented Feb 23, 2024

use the generated current day hour and minute when the docker image is built

Checklist
  • Create sweepai/utils/version.pyfd22c71 Edit
  • Running GitHub Actions for sweepai/utils/version.pyEdit
  • Modify sweepai/api.py2e62f0d Edit
  • Running GitHub Actions for sweepai/api.pyEdit
  • Modify Dockerfileabcce8a Edit
  • Running GitHub Actions for DockerfileEdit
@wwzeng1 wwzeng1 added the sweep Assigns Sweep to an issue or pull request. label Feb 23, 2024
Copy link
Contributor

sweep-nightly bot commented Feb 23, 2024

🚀 Here's the PR! #3156

See Sweep's progress at the progress dashboard!
💎 Sweep Pro: I'm using GPT-4. You have unlimited GPT-4 tickets. (tracking ID: None)

Tip

I can email you next time I complete a pull request if you set up your email here!


Actions (click)

  • ↻ Restart Sweep

Step 1: 🔎 Searching

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I think are relevant in decreasing order of relevance (click to expand). If some file is missing from here, you can mention the path in the ticket description.

sweep/sweepai/api.py

Lines 91 to 344 in 6430b99

app = FastAPI()
events = {}
on_ticket_events = {}
security = HTTPBearer()
logger.bind(application="webhook")
def auth_metrics(credentials: HTTPAuthorizationCredentials = Security(security)):
if credentials.scheme != "Bearer":
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Invalid authentication scheme.",
)
if credentials.credentials != "example_token": # grafana requires authentication
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token."
)
return True
if not IS_SELF_HOSTED:
Instrumentator().instrument(app).expose(
app,
should_gzip=False,
endpoint="/metrics",
include_in_schema=True,
tags=["metrics"],
dependencies=[Depends(auth_metrics)],
)
def run_on_ticket(*args, **kwargs):
tracking_id = get_hash()
with logger.contextualize(
**kwargs,
name="ticket_" + kwargs["username"],
tracking_id=tracking_id,
):
return on_ticket(*args, **kwargs, tracking_id=tracking_id)
def run_on_comment(*args, **kwargs):
tracking_id = get_hash()
with logger.contextualize(
**kwargs,
name="comment_" + kwargs["username"],
tracking_id=tracking_id,
):
on_comment(*args, **kwargs, tracking_id=tracking_id)
def run_on_button_click(*args, **kwargs):
thread = threading.Thread(target=handle_button_click, args=args, kwargs=kwargs)
thread.start()
global_threads.append(thread)
def run_on_check_suite(*args, **kwargs):
request = kwargs["request"]
pr_change_request = on_check_suite(request)
if pr_change_request:
call_on_comment(**pr_change_request.params, comment_type="github_action")
logger.info("Done with on_check_suite")
else:
logger.info("Skipping on_check_suite as no pr_change_request was returned")
def terminate_thread(thread):
"""Terminate a python threading.Thread."""
try:
if not thread.is_alive():
return
exc = ctypes.py_object(SystemExit)
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(
ctypes.c_long(thread.ident), exc
)
if res == 0:
raise ValueError("Invalid thread ID")
elif res != 1:
# Call with exception set to 0 is needed to cleanup properly.
ctypes.pythonapi.PyThreadState_SetAsyncExc(thread.ident, 0)
raise SystemError("PyThreadState_SetAsyncExc failed")
except SystemExit:
raise SystemExit
except Exception as e:
logger.exception(f"Failed to terminate thread: {e}")
# def delayed_kill(thread: threading.Thread, delay: int = 60 * 60):
# time.sleep(delay)
# terminate_thread(thread)
def call_on_ticket(*args, **kwargs):
global on_ticket_events
key = f"{kwargs['repo_full_name']}-{kwargs['issue_number']}" # Full name, issue number as key
# Use multithreading
# Check if a previous process exists for the same key, cancel it
e = on_ticket_events.get(key, None)
if e:
logger.info(f"Found previous thread for key {key} and cancelling it")
terminate_thread(e)
thread = threading.Thread(target=run_on_ticket, args=args, kwargs=kwargs)
on_ticket_events[key] = thread
thread.start()
global_threads.append(thread)
# delayed_kill_thread = threading.Thread(target=delayed_kill, args=(thread,))
# delayed_kill_thread.start()
def call_on_check_suite(*args, **kwargs):
kwargs["request"].repository.full_name
kwargs["request"].check_run.pull_requests[0].number
thread = threading.Thread(target=run_on_check_suite, args=args, kwargs=kwargs)
thread.start()
global_threads.append(thread)
def call_on_comment(
*args, **kwargs
): # TODO: if its a GHA delete all previous GHA and append to the end
def worker():
while not events[key].empty():
task_args, task_kwargs = events[key].get()
run_on_comment(*task_args, **task_kwargs)
global events
repo_full_name = kwargs["repo_full_name"]
pr_id = kwargs["pr_number"]
key = f"{repo_full_name}-{pr_id}" # Full name, comment number as key
comment_type = kwargs["comment_type"]
logger.info(f"Received comment type: {comment_type}")
if key not in events:
events[key] = SafePriorityQueue()
events[key].put(0, (args, kwargs))
# If a thread isn't running, start one
if not any(
thread.name == key and thread.is_alive() for thread in threading.enumerate()
):
thread = threading.Thread(target=worker, name=key)
thread.start()
global_threads.append(thread)
def call_on_merge(*args, **kwargs):
thread = threading.Thread(target=on_merge, args=args, kwargs=kwargs)
thread.start()
global_threads.append(thread)
@app.get("/health")
def redirect_to_health():
return health_check()
@app.get("/", response_class=HTMLResponse)
def home():
return "<h2>Sweep Webhook is up and running! To get started, copy the URL into the GitHub App settings' webhook field.</h2>"
@app.get("/ticket_progress/{tracking_id}")
def progress(tracking_id: str = Path(...)):
ticket_progress = TicketProgress.load(tracking_id)
return ticket_progress.dict()
def init_hatchet() -> Hatchet | None:
try:
hatchet = Hatchet(debug=True)
worker = hatchet.worker("github-worker")
@hatchet.workflow(on_events=["github:webhook"])
class OnGithubEvent:
"""Workflow for handling GitHub events."""
@hatchet.step()
def run(self, context: Context):
event_payload = context.workflow_input()
request_dict = event_payload.get("request")
event = event_payload.get("event")
run(request_dict, event)
workflow = OnGithubEvent()
worker.register_workflow(workflow)
# start worker in the background
thread = threading.Thread(target=worker.start)
thread.start()
global_threads.append(thread)
return hatchet
except Exception as e:
print(f"Failed to initialize Hatchet: {e}, continuing with local mode")
return None
# hatchet = init_hatchet()
def handle_github_webhook(event_payload):
# if hatchet:
# hatchet.client.event.push("github:webhook", event_payload)
# else:
run(event_payload.get("request"), event_payload.get("event"))
def handle_request(request_dict, event=None):
"""So it can be exported to the listen endpoint."""
with logger.contextualize(tracking_id="main", env=ENV):
action = request_dict.get("action")
try:
# Send the event to Hatchet
handle_github_webhook(
{
"request": request_dict,
"event": event,
}
)
except Exception as e:
logger.exception(f"Failed to send event to Hatchet: {e}")
# try:
# worker()
# except Exception as e:
# discord_log_error(str(e), priority=1)
logger.info(f"Done handling {event}, {action}")
return {"success": True}
@app.post("/")
def webhook(
request_dict: dict = Body(...),
x_github_event: Optional[str] = Header(None, alias="X-GitHub-Event"),
):
"""Handle a webhook request from GitHub."""
with logger.contextualize(tracking_id="main", env=ENV):
action = request_dict.get("action", None)
logger.info(f"Received event: {x_github_event}, {action}")

# Deploying your own Sweep instance via Docker
We use the&nbsp;[Elastic License V2](https://www.elastic.co/licensing/elastic-license) which allows commercial usage for Sweep.
This is a guide for self-hosting Sweep. If you would like to use our hosted version, please visit https://github.com/apps/sweep-ai.
## 1. Downloading Sweep
Open [Docker Desktop](https://www.docker.com/products/docker-desktop/) and run the following code in your terminal (hover over code for copy button):
```sh filename="terminal"
git clone --depth 1 --single-branch https://github.com/sweepai/sweep
cd sweep
touch .env
docker compose pull
```
1. Clone the [sweepai/sweep](https://github.com/sweepai/sweep) repository
2. Create a `.env` file in the root directory
3. Pull the docker image.
In the `.env`, configure the repository that Sweep should work on.
```sh filename=".env"
REPO=your-username/your-repo
```
We're halfway done! Pulling the image will take a few minutes, so let's complete the next steps for now.
## 2. GitHub PAT
To allow Sweep to access your repo, create a new [GitHub Personal Access Token (PAT) here](https://github.com/settings/tokens/new?description=Sweep%20Self-hosted&scopes=repo,workflow). Sweep will run entirely locally, and no code will be shared outside of OpenAI's API.
Add this to your `.env` which should now look like this:
```sh filename=".env"
REPO=your-username/your-repo
GITHUB_PAT=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
<details>
<summary>
Link not working?
</summary>
If the link doesn't work, you can go [here](https://github.com/settings/tokens/new) and manually set your preferred token name (e.g. Sweep Self-Hosted), and select the following permissions:
* repo (ALL)
* workflow
</details>
## 3. OpenAI API Token
Please create an OpenAI API token at https://platform.openai.com/account/api-keys.
```sh
OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
After following the above sections, your `.env` should look like this:
```sh filename=".env"
REPO=your-username/your-repo
GITHUB_PAT=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
<details>
<summary>
(Optional) Disabling Telemetry
</summary>
We log user events(not user code) to improve the product, such as the # of pull requests created and merged. To disable telemetry, you can add the following line to your `.env`:
```sh filename=".env"
POSTHOG_API_KEY=none
```
</details>
---
## 4. Deploying Sweep Locally
Once `docker compose pull` from Step 1 is complete, you can run Sweep locally with the following command:
```sh filename="terminal"
docker compose up # Add -d to run in background
```
## 5. Creating a Pull Request with Sweep
Finally, you can use Sweep by creating a new issue on the repository with a title prefixed with `Sweep:`, like `Sweep: add type hints to BaseIndex.tsx`!
For more details on using Sweep see [how to use Sweep](https://docs.sweep.dev/#-how-to-use-sweep).


Step 2: ⌨️ Coding

Create sweepai/utils/version.py with contents:
• Create a new file named `version.py` in the `sweepai/utils` directory.
• Inside `version.py`, define a global variable `__version__` initialized to `None`. This variable will hold the version string.
• Add a function `get_version()` that returns the value of `__version__`.
• At the top of the file, add logic to read the version information from an environment variable, for example, `SWEEP_VERSION`, and assign it to `__version__`. If the environment variable is not set, `__version__` should default to a value indicating that the version is unknown, such as `"unknown"`.
• Ensure that `version.py` is included in the `__init__.py` file within the `sweepai/utils` directory to make it importable.
  • Running GitHub Actions for sweepai/utils/version.pyEdit
Check sweepai/utils/version.py with contents:

Ran GitHub Actions for fd22c717795f99eceaaf31d65c5da4e1a85dbded:

Modify sweepai/api.py with contents:
• After the existing `/` endpoint, add a new endpoint for `/version`.
• Define a new function `get_version()` that uses the `get_version()` function from `sweepai/utils/version.py` to retrieve the version information.
• The `get_version()` function should be decorated with the `@app.get("/version")` decorator to create the new endpoint.
• The function should return a JSON response with the version information, for example, `{"version": utils.version.get_version()}`.
--- 
+++ 
@@ -259,6 +259,11 @@
 def home():
     return "

Sweep Webhook is up and running! To get started, copy the URL into the GitHub App settings' webhook field.

" + +@app.get("/version") +def get_version(): + from sweepai.utils.version import get_version as get_sweep_version + return {"version": get_sweep_version()} @app.get("/ticket_progress/{tracking_id}") def progress(tracking_id: str = Path(...)):
  • Running GitHub Actions for sweepai/api.pyEdit
Check sweepai/api.py with contents:

Ran GitHub Actions for 2e62f0d35a4a22287ba5e059b543a8f16426413b:

Modify Dockerfile with contents:
• In the Dockerfile, add a new instruction to set an environment variable `SWEEP_VERSION` with the value generated based on the current day, hour, and minute. This can be done using shell commands within the `RUN` instruction or by using the `ARG` and `ENV` instructions.
• The value should be in a format that represents the version, such as `YYYYMMDDHHMM`, and can be generated using a command like `date +%Y%m%d%H%M`.
• Ensure that the `SWEEP_VERSION` environment variable is available at runtime so that it can be read by the `sweepai/utils/version.py` file.
--- 
+++ 
@@ -61,6 +61,9 @@
 ENV PYTHONPATH=.
 COPY bin/startup.sh /app/startup.sh
 COPY redis.conf /app/redis.conf
+
+# Set the SWEEP_VERSION environment variable to the current date and time during image build
+ENV SWEEP_VERSION=$(date +%Y%m%d%H%M)
 RUN chmod u+x /app/startup.sh
 
 EXPOSE $PORT
  • Running GitHub Actions for DockerfileEdit
Check Dockerfile with contents:

Ran GitHub Actions for abcce8aa48502912d0cd6b7b2d50238a09d3b484:


Step 3: 🔁 Code Review

I have finished reviewing the code for completeness. I did not find errors for sweep/add_a_version_endpoint_to_apipy_that_tel.


🎉 Latest improvements to Sweep:
  • New dashboard launched for real-time tracking of Sweep issues, covering all stages from search to coding.
  • Integration of OpenAI's latest Assistant API for more efficient and reliable code planning and editing, improving speed by 3x.
  • Use the GitHub issues extension for creating Sweep issues directly from your editor.

💡 To recreate the pull request edit the issue title or description.
Something wrong? Let us know.

This is an automated message generated by Sweep AI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment