uvicorn waits 30 seconds before terminating with SIGINT

[pawel-jaworski]

When uvicorn is terminated with CTRL^C while there is an open TCP connection with TLS (keep-alive + https), it hangs for 30 seconds before shutting down.

To reproduce:

1. Setup SSL cert with mkcert:

mkcert localhost

2. Create a simple app from README, log timestamps, setup SSL cert and set timeout_keep_alive to 600 seconds:

# main.py
import uvicorn
from uvicorn.config import LOGGING_CONFIG


async def app(scope, receive, send):
    assert scope['type'] == 'http'

    await send({
        'type': 'http.response.start',
        'status': 200,
        'headers': [
            [b'content-type', b'text/plain'],
        ],
    })
    await send({
        'type': 'http.response.body',
        'body': b'Hello, world!',
    })


def run():
    log_fmt = "%(asctime)s [%(name)s] %(levelprefix)s %(message)s"
    LOGGING_CONFIG["formatters"]["default"]["fmt"] = log_fmt
    LOGGING_CONFIG["formatters"]["access"]["fmt"] = log_fmt
    uvicorn.run(app, port=5000, timeout_keep_alive=600, ssl_keyfile="localhost-key.pem", ssl_certfile="localhost.pem")


if __name__ == '__main__':
    run()

3. Run the server:

python main.py

4. Open the browser and go to: https://localhost:5000
5. Keep the browser tab open
6. Close the server with CTRL^C

Logs:

2024-05-08 13:52:34,459 [uvicorn.error] INFO:     Started server process [26125]
2024-05-08 13:52:34,459 [uvicorn.error] INFO:     Waiting for application startup.
2024-05-08 13:52:34,459 [uvicorn.error] INFO:     ASGI 'lifespan' protocol appears unsupported.
2024-05-08 13:52:34,459 [uvicorn.error] INFO:     Application startup complete.
2024-05-08 13:52:34,459 [uvicorn.error] INFO:     Uvicorn running on https://127.0.0.1:5000 (Press CTRL+C to quit)
2024-05-08 13:52:40,854 [uvicorn.access] INFO:     127.0.0.1:51712 - "GET / HTTP/1.1" 200
2024-05-08 13:52:40,896 [uvicorn.access] INFO:     127.0.0.1:51712 - "GET /favicon.ico HTTP/1.1" 200
^C2024-05-08 13:52:45,399 [uvicorn.error] INFO:     Shutting down
2024-05-08 13:52:45,500 [uvicorn.error] INFO:     Waiting for connections to close. (CTRL+C to force quit)
2024-05-08 13:53:15,476 [uvicorn.error] INFO:     Finished server process [26125]

Python, uvicorn & OS:

python: 3.12.2
uvicorn: 0.28.1
OS: Ubuntu 20.04

Notes on reproduction:

• hitting CTRL^C again when waiting for connections to close, does not do anything
• reproduced it in Google Chrome and Brave, couldn't reproduce it in Firefox
• we originally observed it in fastapi service with gunicorn behind nginx

Debugging details:

• worker is hanging at:
  https://github.com/encode/uvicorn/blob/0.28.1/uvicorn/server.py#L303
  because there's active connection, so _waiters is not None:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/base_events.py#L400
• few lines before, when connection.shutdown() is called:
  https://github.com/encode/uvicorn/blob/0.28.1/uvicorn/server.py#L263
  close() is called on transport instance:
  https://github.com/encode/uvicorn/blob/0.28.1/uvicorn/protocols/http/h11_impl.py#L339
• for HTTPS connections, transport is an instance of _SSLProtocolTransport class, so when _start_shutdown is called:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/sslproto.py#L112
  inside close() method, shutdown is scheduled with 30 seconds delay:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/sslproto.py#L620
• after 30 seconds (ssl_shutdown_timeout), _detach() is called:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/base_events.py#L297
  which calls _wakeup(), which in turn sets _waiters to None:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/base_events.py#L305
• this unblocks wait_closed:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/base_events.py#L401
• for HTTP connection, transport is an instance of _SelectorSocketTransport, so when close is called,
  _detach() is called inside _call_connection_lost:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/selector_events.py#L915
• but because _call_connection_lost is scheduled with call_soon instead of call_later, it executes without a delay:
  https://github.com/python/cpython/blob/v3.12.1/Lib/asyncio/selector_events.py#L871

Expected behaviour:

When server is closed with SIGINT and force_exit is set to True, we shouldn't wait for graceful connection shutdown.

Proposed solution:

When force_exit is True, when shutting down connections, we should call transport.abort() instead of transport.close():
https://github.com/encode/uvicorn/blob/0.28.1/uvicorn/protocols/http/h11_impl.py#L339

Other solutions:

• set ssl_shutdown_timeout to a low value (or 0) when force_exit=True: probably not that straightforward
• we could set timeout_graceful_shutdown to a low value, but this would affect graceful shutdowns (with SIGTERM) as well, unless depending on force_exit