Incorrect assumption in werkzeug.serving about socket behavior

[d42]

As we all know socket.send() doesn't guarantee all of the data getting sent

and this line:

werkzeug/werkzeug/serving.py

Line 165 in 03faf05

self.wfile.write(data)

combined with gevent and the greatest webscale technology, known as docker has led me to just that behavior with responses getting truncated :3

It seems like the socket.SocketIO should care about it, but it doesn't

[untitaker]

While you're probably correct (haven't done any research), please note that Werkzeug's server is only intended for development. Running it in Docker and with gevent leads me to believe that you don't use it for that purpose.

[d42]

I've been actually using docker for development and part of the code used gevent monkeypatches quite frivulously.
I probably wouldn't even notice it if it wouldn't break dontpanic :3

[odony]

I've experienced this bug in Python 3.5, but only when setting the underlying socket to non-blocking mode (e.g with settimeout). It turns out this is in fact caused by an upstream bug in Python 3: issue24291. The issue is fixed in Python 3.6 (see issue26721)

A workaround that has been implemented in other web servers is to wrap the wfile in an io.BufferedWriter (e.g in CPython's wsgiref in 3.5) or to change the wbufsize to enable buffering by default (e.g. see this gevent issue)

Not sure if werkzeug wants to implements a workaround too, or considers it a non-werkzeug issue.

[RonnyPfannschmidt]

based on the provided information i believe this indeed is either a gevent bug or a stdlib bug, werkzeug uses write + flush on a supposed buffered file, which in turn should always work, now i wonder if this also happens on plain python, or only on gevent

[odony]

@RonnyPfannschmidt yes, I get it without gevent too, I just need to use a non-blocking socket in combination with werkzeug's WSGIRequestHandler. Gevent was just another example of the many projects who are indirectly affected by the upstream bug in Python 3.5 :-)

[davidism]

Given the circumstances this happens in, as well as the fact that it's fixed in Python 3.5 (3.4 goes EOL in March), I'm closing this.