Question about custom event handling

[jrobbins-LiveData]

The Mangum doc shows this example of how one might handle a custom event:

def handler(event, context):
    if event.get("some-key"):
        # Do something or return, etc.
        return

    asgi_handler = Mangum(app)
    response = asgi_handler(event, context) # Call the instance with the event arguments

    return response

I need to handle an incoming AWS EventBridge event. I want to invoke the same method that my HTTP API handler is also invoking -- and it is an async method.

Here's what I am using

def handler(event: LambdaEvent, context: LambdaContext) -> dict:
    if "requestContext" in event:
        logger.debug("HTTP API")
        response = fastapi_handler(event, context)
    else:
        logger.debug("Not HTTP API")
        loop = asyncio.get_event_loop()
        response = loop.run_until_complete(app_logic())
    return response

My question(s): Is this the correct pattern to use? In the Mangum source, I see code like this, so I am trying to fit in. My code seems to work, but I read in the Python docs that get_event_loop() is

Deprecated since version 3.10: Deprecation warning is emitted if there is no running event loop. In future Python releases, this function will be an alias of get_running_loop().

Which made me wonder if I were doing this correctly?

[jordaneremieff]

Hi @jrobbins-LiveData,

I'm not sure exactly what your example code is attempting to do. Does thefastapi_handler here refer to the Mangum class or some event handling for something else?

In any case, the get_event_loop in Mangum is likely just hanging around from the original implementation that supported 3.6, but get_running_loop is supported since 3.7 and should probably be changed since 3.6 support was dropped.

[jrobbins-LiveData]

Hi @jordaneremieff . Sorry for the lack of clarity.

app = FastAPI()
fastapi_handler = Mangum(app)

If I replace get_event_loop with get_running_loop, I'm wondering who (which function call) creates the asyncio loop?

[jrobbins-LiveData]

@jordaneremieff this is still an issue for us.

If a lambda handler function wants to either call a Mangum(app) or run a coroutine directly, some code has to be responsible for creating the asyncio loop. If the first call to my lambda handler after coldstart is, say, from EventBridge, instead of ApiGateway, I cannot call get_running_loop because there is not yet a running loop.

If I call get_running_loop in the handler at coldstart, it raises RuntimeError: no running event loop.

[jordaneremieff]

Hi @jrobbins-LiveData,

Coincidentally I've actually run into this issue myself because I was trying to develop a local dev server for testing the adapter using asyncio.Protocol, but I ended up hitting the RuntimeError: no running event loop problem as well. I eventually shelved the idea for the time being, but I did come across https://github.com/erdewit/nest_asyncio which has been mentioned as a way to workaround this limitation.

I'm going to convert this discussion into an issue now that I understand the problem correctly, but at this point I'm not certain the best way to resolve it - open to reviewing a PR to address this, otherwise I'll revisit when I have time.

[jrobbins-LiveData]

Hi @jordaneremieff,

It appears that the trend for asyncio has been to make the simple use-cases simpler. There's a lot of emphasis on getting up and running with asyncio.run(), for example. But I think that due to the AWS Lambda architecture, this is not our use-case. As the comment to this SO post correctly points out:

I think this is wrong, at least if you are reusing global resources across Lambda invocations. When I use asyncio.run, I get "event loop closed" errors on subsequent runs. I guess this is because those global resources are tied to the previous event loop? –
mariaines
Jul 21, 2021 at 22:00

I think the solution might be to have the Mangum class explicitly create a new event loop in its __init__ method. (There could be an option to opt-out of this behavior if a given user wants to own the responsibility for creating the Lambda's asyncio loop.)

This would allow Mangum's internals to switch away from the to-be-deprecated get_event_loop() and allow for my use-case (with some hand-waving):

app = FastAPI()
fastapi_handler = Mangum(app)

def handler(event, context):
    if 'requestContext' in event:
        logger.debug('HTTP API')
        response = fastapi_handler(event, context)
    else:
        logger.debug('Not HTTP API')
        # somehow get the loop from Mangum; via the `asgi` `scope`?
        response = loop.run_until_complete(sync_lastpass_to_cognito())

[jrobbins-LiveData]

I just tested in a Lambda (without editing Mangum) by putting this code up top in my Lambda file, and my use-case works.

loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)


app = FastAPI()
fastapi_handler = Mangum(app)

Ideally we could put that loop creation code inside Mangum (easily done) and provide a way for the user to access that loop from Mangum. I don't know the right way to do that ... ought it simply be a loop property on the Mangum instance, or ought it be in the scope or somewhere else?