Add debug setup for inference server & worker #3575
Conversation
0xfacade
requested review from
andreaskoepf,
melvinebenezer,
yk,
olliestanley and
AbdBarho
as code owners
|
❌ pre-commit failed. |
| @@ -231,12 +231,14 @@ services: | |||
| TRUSTED_CLIENT_KEYS: "6969" | |||
| ALLOW_DEBUG_AUTH: "True" | |||
| API_ROOT: "http://localhost:8000" | |||
| DEBUG: "True" | |||
There was a problem hiding this comment.
If I understand correctly, the compose file is only meant for local development, so setting this here shouldn't be a problem?
| "request": "attach", | ||
| "connect": { | ||
| "host": "localhost", | ||
| "port": 5679 |
There was a problem hiding this comment.
Note the different ports for server and worker
| # Uncomment to wait here until a debugger is attached | ||
| # debugpy.wait_for_client() | ||
|
|
||
| uvicorn.run("main:app", host="0.0.0.0", port=port, reload=is_debug) |
There was a problem hiding this comment.
This method of starting the server is only used for development - the docker image for production still invokes the uvicorn command. I could change that to also use python main.py instead for consistency, if desired.
|
❌ pre-commit failed. |
…o inference_worker_debug
|
❌ pre-commit failed. |
| @@ -29,7 +29,7 @@ | |||
| exclude: build|stubs|^bot/templates/$|openassistant/templates|docs/docs/api/openapi.json|scripts/postprocessing/regex_pii_detector.py | |||
|
|
|||
| default_language_version: | |||
| python: python3 | |||
| python: python3.10 | |||
There was a problem hiding this comment.
@olliestanley This is what was causing pre-commit to fail on my machine. python3 is interpreted as python3.7, which is not new enough for some of the syntax, or isort. When I am more specific, like here, it works. Should this be examined some more?
There was a problem hiding this comment.
I suspect this wouldn't be an issue if you ran the commands from inside a Python 3.10 virtual environment (I guess that's why others haven't had any similar issues) but I don't see any reason we can't make this config change if it helps make things easier
There was a problem hiding this comment.
I'm on a rolling release Linux system with Python 3.10.10. It worked correctly inside a Ubuntu Docker container with the same Python version.. so it's probably some other dependency that broke this
|
@olliestanley From my side, this is ready to be merged, now that I've figured out why pre-commit was failing (see other message). |
I'm happy for it to be merged but it touches repo root files etc so will need approval from @andreaskoepf or @yk If one of them is reading this - would also be good to get Florian collaborator access to the repo |
I couldn't find an existing way to attach a debugger to the inference server or worker. In this PR, I enable both to be debugged using debugpy, the Python debugger that integrates easily with VS Code.
Open issues: