Why gevent response time is much less than the real response time while high concurrency

[XGiton]

• gevent version: gevent==20.9.0（pypi）.
• Python version: Python 3.6.9
• Operating System: Ubuntu 18.04

Description:

I built a flask application and use gevent monkey_patch() to support high concurrency, and use nginx upstream for reverse proxy. everything is ok but occurred lots of 499 http status code responses yesterday.

The 499 status code is returned by nginx because of connection closed by client with setting timeout=5, but all request's response time lte 1s in flask app request log, and CPU usage not full, only reach 60%, same as usual.

So I wrote a demo to test that scene as below:

import click
import json
from gevent import monkey
monkey.patch_all()

from time import sleep
from flask import Flask, jsonify, request
from gevent.pywsgi import WSGIServer


def create_app():
    flask_app = Flask(__name__)

    @flask_app.route('/test/<q_id>', methods=['GET'])
    def test(q_id):
        # for timeout easily
        sleep(0.1)
        # for high cpu usage
        for i in range(1000):
            json.dumps(request.cookies.to_dict())
        return jsonify(result='ok'), 200

    return flask_app


@click.command()
@click.option('--port', help='port')
def main(port):
    port = int(port)
    app = create_app()
    http_server = WSGIServer(('0.0.0.0', port), app)
    http_server.serve_forever()


if __name__ == '__main__':
    main()

I run this app and send lost of concurrency requests with timeout=0.3 on another server. several minutes later, much 499 status code appeared in nginx log like below screenshot:

And this is flask app log, request with id=x-333:

And CPU usage is 60%, so why the response time in nginx is gte 0.3s, but only 0.146s in gevent + flask app?