Gevent has more compatibility issue than I thought...

[dongzeli95]

This is more of a general discussion regarding whether to use gevent with flask socketio in production. We are using gunicorn + gevent + flask socketio for our python backend service.

As I use gevent with flask socketio more and more, I found that a lot of third party SDKs from GCP or Azure doesn't have good supports regarding gevent or eventlet due to its monkey patch nature.

For references:
googleapis/google-cloud-python#12535
googleapis/python-storage#1268

A lot of the python SDK on the market doesn't support gevent really well and it creates high CPU usages and performance issues.

Setting up flask socketio behind a nginx isn't exactly trivial effort either as documented here.
I am thinking about switching to python-socketio for a try for having more concurrency and hopefully solve the performance issues under load for SDK usages.

I've also read a paper benchmarking websocket performance over multiple languages and frameworks and the conclusion was that python should be avoided at all costs when dealing with websocket connections. Not sure if that's still the case today.

At this stage, neither migrating to python-socketio nor adopting a new language is a trivial task. Could someone give me some ideas and advise? Thank you so much!

[miguelgrinberg]

With Flask-SocketIO you have three concurrency options:

• threads (async_mode='threading', and the --threads option in Gunicorn)
• gevent
• eventlet

Using threads is going to give you the biggest compatibility with third-party libraries libraries. Gevent and Eventlet provide higher scalability than threads, but the cost is that the asynchronous programming model is harder to set up, as you need to make sure that nothing in the application blocks for too long. The Eventlet project is currently on life support, so even though I used to like it more than Gevent at this point I cannot recommend it anymore.

A lot of the python SDK on the market doesn't support gevent really well

I think this is a bit of an exaggeration. Google libraries have always had issues though, and Google never had interest in supporting Gevent. But many other libraries work well with Gevent's monkey patching of the standard library. As long as you stay away from CPU intensive tasks it shouldn't be too hard to get a working application.

I am thinking about switching to python-socketio for a try for having more concurrency

What do you mean? Flask-SocketIO uses python-socketio. If you drop Flask-SocketIO and Flask and use python-socketio directly you will have access to asyncio concurrency, but this is another asynchronous platform, subject to most of the same issues Gevent and Eventlet have in terms of blocking and CPU use. I suggest you do a more careful analysis of your options with pyhton-socketio, I don't think you'll find better options unless you are interested in using asyncio for some specific reason.

and the conclusion was that python should be avoided at all costs when dealing with websocket connections

Python is a slow language, and WebSocket has nothing to do with this. If you are obsessed with getting the best performance you should be using C/C++, Go, Rust, etc. WebSocket implementations in Python are not to blame, in my opinion.

Could someone give me some ideas and advise?

I think you should try a multi-threaded web server and drop Gevent/Eventlet.

[dongzeli95]

Thank you so much! Good suggestion on the whether or not to depend on python-socketio directly. It saves us a lot of time by eliminating trial and error, ultimately resulting in a limited ROI.

I will give async_mode=threading a try! Hopefully will do some similar load tests and report back. It seems like the only viable option so far.

One question, for a single-core CPU machine with 1GB memory on linux, what's the realistic number of threads I can configure it for? eg:

gunicorn -w 1 --threads 100 module:app

I know this probably need more load test and benchmarking but just wanted to have a rough idea. Thank you so much!

[miguelgrinberg]

It really depends on the application and how much work those threads are going to do. The only way to know is to measure it.

[dongzeli95]

@miguelgrinberg We are switching to threading and it's working great so far. One minor issue though, we are seeing a lot of gunicorn worker timeout logs on our service:

2024-04-30 01:49:29.996 PDT
[2024-04-30 08:49:29 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:2942)
2024-04-30 01:49:30.194 PDT
[2024-04-30 08:49:30 +0000] [2942] [INFO] Worker exiting (pid: 2942)
2024-04-30 01:49:31.696 PDT
[2024-04-30 08:49:31 +0000] [1] [ERROR] Worker (pid:2942) was sent SIGKILL! Perhaps out of memory?

The instance is not out of memory so I am guessing this is related to the change switching to threading instead of gevent.
Any suggestions where to look for the culprit? Thank you so much!

[dongzeli95]

Actually trying this preload option. The timeout error doesn't seem to happen locally though.

[miguelgrinberg]

@dongzeli95 have you made any changes to the Socket.IO ping interval or timeout, or are you running with the defaults? If you changed them, then you need to adjust the Gunicorn timeout accordingly. I recommend you use the defaults, there is really no reason to depart from that 99.9% of the times.

[shrinath-gupta-ril]

Hi @dongzeli95 since you are using gunicorn and gevent in production can you let me know who are you able to see access logs of gunicorn when using the geventwebsocket.gunicorn.workers.GeventWebSocketWorker worker because I'm not able to see the access log if I use this worker
Similar issue on gunicorn github issues

[miguelgrinberg]

If you are using a recent version of the Socket.IO server you don't have to use gevent-websocket. Try uninstalling it, and then start Gunicorn with -k gevent.