Adding ASGI support?

[arcivanov]

Django Daphne is under development to support ASGI, but Daphne server is extremely new and not yet suitable for production (e.g. no SSL support, maturation pains).

Is there any interest in adding ASGI support in GUnicorn to allow seamless WSGI/ASGI use?

[benoitc]

@arcivanov I will have a closer look at this protocol to see what needs to be done for it :)

Will update this ticket ASAP.

[japrogramer]

any updates?

[tilgovi]

@benoitc @berkerpeksag and I chatted over e-mail about it. No work has started, yet, that I know about. I have not had time yet to read more about ASGI, so I cannot estimate the complexity of implementing this. If anyone wants to contribute work toward this, I would enthusiastically help, discuss and review.

[andrewgodwin]

I'm in the process of overhauling the ASGI spec to make a lot more sense for servers anyway, so I'd caution about doing anything until that's finished (new spec is being drafted here, but is not final at all: http://channels.readthedocs.io/en/2.0/asgi.html - it's basically a lot more like WSGI)

[notedit]

@andrewgodwin cool can not wait to have a try gunicorn with asgi

[sirex]

@andrewgodwin what is the ASGI status?

I recently tried to test an ASGI app and here is what I got:

import json

def app(scope):
    async def channel(receive, send):
        message = await receive()

        if scope['method'] == 'POST':
            response = message
        else:
            response = scope

        await send({
            'type': 'http.response.start',
            'status': 200,
            'headers': [
                [b'Content-Type', b'application/json'],
            ],
        })
        await send({
            'type': 'http.response.body',
            'body': json.dumps(response, default=bytes.decode).encode(),
        })
        await send({
            'type': 'http.disconnect',
        })
    return channel

> daphne app:app
2018-03-31 22:28:10,823 INFO     Starting server at tcp:port=8000:interface=127.0.0.1
2018-03-31 22:28:10,824 INFO     HTTP/2 support enabled
2018-03-31 22:28:10,824 INFO     Configuring endpoint tcp:port=8000:interface=127.0.0.1
2018-03-31 22:28:10,825 INFO     Listening on TCP address 127.0.0.1:8000
127.0.0.1:43436 - - [31/Mar/2018:22:28:17] "GET /" 200 347
127.0.0.1:43440 - - [31/Mar/2018:22:28:22] "POST /" 200 43
127.0.0.1:43446 - - [31/Mar/2018:22:28:42] "POST /" 200 54

> http -b get :8000/
{
    "type": "http"
    "http_version": "1.1",
    "method": "GET",
    "path": "/",
    "query_string": "",
    "root_path": "",
    "scheme": "http",
    "headers": [
        ["host", "localhost:8000"],
        ["user-agent", "HTTPie/0.9.9"],
        ["accept-encoding", "gzip, deflate"],
        ["accept", "*/*"],
        ["connection", "keep-alive"]
    ],
    "client": ["127.0.0.1", 43360],
    "server": ["127.0.0.1", 8000],
}

> http -b -f post :8000/ foo=bar
{
    "body": "foo=bar",
    "type": "http.request"
}

> http -b -j post :8000/ foo=bar
{
    "body": "{\"foo\": \"bar\"}",
    "type": "http.request"
}

https://github.com/django/asgiref/blob/master/specs/www.rst

[andrewgodwin]

@sirex ASGI is stable now. I'm not sure if you are saying that daphne does or does not adhere to the spec?

[benoitc]

is the link on django the last about the ASGI "spec", or is there anything more that describe the flow ?

[sirex]

@andrewgodwin I was trying to demostrate, that if Daphne supports ASGI, then ASGI must be ready for other frameworks/servers to adopt.

@benoitc there are two ASGI specification pages:

https://github.com/django/asgiref/blob/master/specs/asgi.rst - general ASGI specification.

https://github.com/django/asgiref/blob/master/specs/www.rst - specifications describing HTTP and WebSockets protocols and compatibility with WSGI.

[benoitc]

@sirex thanks. Though these specs are not very helpful when it's about implementing.

Anyway I have found https://channels.readthedocs.io/en/latest/ which is okish. I think i would prefer a pep, but we can propose something supporting asgi as soon as we removed the python 2 support.

btw Ideleted last comment as it's not related to that discussion.

[andrewgodwin]

I haven't moved to make ASGI a PEP since I want to make sure it has enough general support first (and that process takes a lot of time and effort), but that is the intention and it should be written like one.

[tomchristie]

we can propose something supporting asgi as soon as we removed the python 2 support.

If you do start thinking about implementation then it'd be worth starting with uvicorn, which glues together gunicorn, uvloop, and httptools. It might well make sense for that work to get rolled back into gunicorn as a supported worker class at some point.

(It might be nice for uvicorn to then just become a more minimal option, without the process monitoring etc. that gunicorn provides.)

[jordaneremieff]

The ASGI spec documentation is now available here: https://asgi.readthedocs.io/en/latest/.

[tomchristie]

Okay, have this covered now. (As a third-party worker class.)

Uvicorn 0.2 no longer depends on gunicorn for running it directly, but includes a two gunicorn worker classes for running ASGI apps.

For local dev, or cases where you don't care enough about process supervision you can just run uvicorn directly. For production we document using a gunicorn class as the preferred option.

http://www.uvicorn.org/#running-with-gunicorn

Run ASGI app from gunicorn, using uvloop and httptools:

gunicorn app:App -w 4 -k uvicorn.workers.UvicornWorker

Run ASGI app from gunicorn, using asyncio and h11 (for pypy compatibility):

gunicorn app:App -w 4 -k uvicorn.workers.UvicornH11Worker

If there any desire to bringing either of the protocol implementations into the core gunicorn package then am very open to considering that.

[waghanza]

Now that gunicorn does not support python 2 anymore, it's technically doable to add ASGI support

[tilgovi]

@tomchristie I'm interested in what that would look like. As an alternative, maybe extracting the protocols as a separate package or packages?

[tomchristie]

I'm interested in what that would look like. As an alternative, maybe extracting the protocols as a separate package or packages?

I guess either a worker class in gunicorn that depended on uvicorn for the h11 and/or httptools protocols. Or else duplicate the implementation here.

Duplicating a limited part of the implementation to give Gunicorn built-in ASGI HTTP support with an h11 dependency might be a good baseline?

maybe extracting the protocols as a separate package or packages?

There's not really much to extract from uvicorn - If we dropped click and just used argparse, and tweaked our setup.py dependencies a little then it'd have zero hard dependencies, with various options for different HTTP implementations, WS implementations, Event loops.

[tilgovi]

I'd like first-class ASGI support, whether that means adding extras dependencies in Gunicorn, a stub worker that tells you what to install, documentation, or something else.

[japrogramer]

yes, please. about to switch to daphne from gunicorn .. if gunicorn could just support asgi
that would save me so much time.

[tilgovi]

@japrogramer http://www.uvicorn.org/#running-with-gunicorn

[johnthagen]

Django 3.0 (set to release December 2019) will have ASGI support out of the box, so that will be another great reason for gunicorn to support it directly. We'll probably see a big up-tick in user interest after that is released.

[benoitc]

@johnthagen that's on the pipe. However, we need to clear up a little the tickets and make a release this month before starting .

[mirekphd]

@benoitc any progress on this issue?

[Ahleroy]

Also interested in the ASGI support so we do not need to use both GUnicorn and UVicorn.