suite: handle cancelled requests

[VallariAg]

Current behaviour: When a request is sent to /suite and the teuthology-suite command is being executed, then if the client request is cancelled (due to timeout or any other reason), the command continues to schedule.

Expected behaviour: If the request is cancelled, the scheduling command should also be cancelled/stopped.

[dikwickley]

hi @VallariAg i would like to work on this issue.

[dikwickley]

The main issue here is there is no way to know at the FastAPI server if the client has disconnected.
After going through tiangolo/fastapi#8805 and tiangolo/fastapi#8699
i found this https://github.com/RedRoserade/fastapi-disconnect-example/blob/main/app.py.

A simple example that uses polling to check if the user has cancelled the request or not.

This works pretty well with only caveat that we would need to use uvicorn instead of gunicorn.
What do you think @VallariAg

[VallariAg]

Hey @dikwickley, this issue would need more discussions before we can plan on working on this.
We might make the long-running functions execute asynchronously which should also handle this issue.

[dikwickley]

@VallariAg makes sense. I had created a PR that does just that #29