"cannot send more responses than requests" if connection is reset mid transfer

[ariscop]

version 0.4.15
Steps to reproduce:

createWebServer(i)
.add(new StaticFileHandler([path])
.start().get();

curl [url] | dd of=/dev/null bs=[smaller than file size] count=1

[wherget]

I'm seeing this as well. It doesn't break anything (apparently), but the console is being flooded by stack traces.

[ariscop]

it will consume a thread though, actually you could easily bring down a webbit server by repeatedly resetting connections.

[wherget]

I'm currently invesitgating this. I'll issue a pull request when (if) I have a fix ready.

[ghost]

Any news ?

[aslakhellesoy]

If there were any news they would appear right here in this issue :-)

[wherget]

My apologies for not sharing this sooner, what I changed in wherget/webbit@298b570 doesn't quite catch all cases.

It's still something to do with correcly avoiding unsolicited error resonses, but I haven't been quite able to pin down the exact situation where a channel is still writable but accepts no further replies (I'm thinking keep-alive, maybe ). Neither have I found a nice solution, and I keep thinking of using a ChannelLocal that tracks "demand for a response" as a last resort.
Maybe http://github.com/viva/webbit is on the right track along this...

[ghost]

Thanks alot :) I will check both

[wherget]

I think I just stumbled across something, just jotting it down here until I have a creative Idea for a fix.
There might still be a problem in NettyHttpResponse.error(...): there's a call to exceptionHandler.uncaughtException(...) which might later down the line try to write another failure response to the channel, which will fail, which will do the same thing over and over again.

[chtheis]

I got the same problem. It looks like an IOException is thrown when the server wants to write the response. The server will then try to write an error response (some 5xx code) to the client, which in turn throws an IllegalStateExeption, which will cause the server to write an error response to the client, which in turn throws ...
Just like l337r00l commented. As I understand isWritable might not be enough: a Channel might be writeable (or connectable) even if it is not (yet) connected.

I have a workaround where I test for
if (!ctx.getChannel().isOpen())
return;

[duelinmarkers]

@chtheis, could you expand on that workaround? Where do you do that?

[wherget]

Same as mine (298b570) I guess, but instead of isWritable() he used isOpen(); I've got a nagging feeling though that it should be (isOpen() && isConnected()), see this StackOverflow question.

[chtheis]

Yes, the same as yours but with isOpen instead of isWriteable.
I'm not sure it that is enough, once I saw the repeating error again, but I could not reproduce it.
The StackOverflow question deals with connectionless or not connected sockets where you would use "sendto" instead of a simple "send". I don't think it would makea difference in our case.

[chtheis]

And I think it is just a workaround. Somewhere deeper in the code a write to the channel fails and that write shall not trigger an error message to be sent to the client but drop the connection. But I couldn't pin down where that would happen, not even if it is in Netty or in Webbit.

[nmussy]

I don't think any fixes have been found for that one, but I have the same issue when I try to open a wss on a non-SSL initialized WebServer. It creates an absurd amount of those exceptions:

org.webbitserver.WebbitException: cannot send more responses than requests on [id: 0x201c2e7f, /127.0.0.1:58808 :> /127.0.0.1:8887]
    at org.webbitserver.WebbitException.fromException(WebbitException.java:36)
    at org.webbitserver.netty.NettyHttpResponse.error(NettyHttpResponse.java:171)
    at org.webbitserver.netty.NettyHttpChannelHandler$3.run(NettyHttpChannelHandler.java:94)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: cannot send more responses than requests
    at org.jboss.netty.handler.codec.http.HttpContentEncoder.writeRequested(HttpContentEncoder.java:101)
    at org.jboss.netty.channel.Channels.write(Channels.java:712)
    at org.jboss.netty.channel.Channels.write(Channels.java:679)
    at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:246)
    at org.webbitserver.netty.NettyHttpResponse.write(NettyHttpResponse.java:208)
    at org.webbitserver.netty.NettyHttpResponse.flushResponse(NettyHttpResponse.java:196)
    at org.webbitserver.netty.NettyHttpResponse.error(NettyHttpResponse.java:169)
    ... 8 more

Those are definitely related, but I can't find a viable solution to this, i.e. catching the connection to the wrong port, and redirecting it, or at least dropping the connection.

Edit: Same thing happens the other way around, when I try to connect to a secure WebSocket with a ws:// URL. It works perfectly with the right port/protocol combination, but it's a huge issue if someone mixes those up.

[nmussy]

Well, never mind, @lostdj's fork seems to be doing the trick. It's still throwing exceptions, but not in an infinite loop. 😄

[lostdj]

@nmussy, actually I didn't fix it, they did:
#125
#126

[nmussy]

That's odd, webbit/webbit's snapshot still had the same issue. It might have been the change in netty's version. Anyway, it's working now, regardless of whomever I should be thankful to. 👍

[lostdj]

No, those and other https://github.com/webbit/webbit/pulls requests wasn't merged, thus they are not in webbit's master. Apparently this project is abandoned and ignored completely.

[zhegexiaohuozi]

Waiting for fix

[frankyen389]

I have the same issue, do you have any solution for this?

10:02:48,122 ERROR stderr org.webbitserver.WebbitException: cannot send more responses than requests on [id: 0xdd66ffce, /192.168.251.35:12019 :> /172.17.4.172:8082]
10:02:48,123 ERROR stderr at org.webbitserver.WebbitException.fromException(WebbitException.java:36)
10:02:48,123 ERROR stderr at org.webbitserver.netty.NettyHttpResponse.error(NettyHttpResponse.java:171)
10:02:48,123 ERROR stderr at org.webbitserver.netty.NettyHttpChannelHandler$3.run(NettyHttpChannelHandler.java:94)
10:02:48,123 ERROR stderr at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
10:02:48,123 ERROR stderr at java.util.concurrent.FutureTask.run(FutureTask.java:266)
10:02:48,123 ERROR stderr at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
10:02:48,123 ERROR stderr at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
10:02:48,123 ERROR stderr at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
10:02:48,123 ERROR stderr at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
10:02:48,123 ERROR stderr at java.lang.Thread.run(Thread.java:745)
10:02:48,124 ERROR stderr Caused by: java.lang.IllegalStateException: cannot send more responses than requests
10:02:48,124 ERROR stderr at org.jboss.netty.handler.codec.http.HttpContentEncoder.writeRequested(HttpContentEncoder.java:100)
10:02:48,124 ERROR stderr at org.jboss.netty.channel.Channels.write(Channels.java:704)
10:02:48,124 ERROR stderr at org.jboss.netty.channel.Channels.write(Channels.java:671)
10:02:48,124 ERROR stderr at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248)
10:02:48,124 ERROR stderr at org.webbitserver.netty.NettyHttpResponse.write(NettyHttpResponse.java:208)
10:02:48,124 ERROR stderr at org.webbitserver.netty.NettyHttpResponse.flushResponse(NettyHttpResponse.java:196)
10:02:48,124 ERROR stderr at org.webbitserver.netty.NettyHttpResponse.error(NettyHttpResponse.java:169)
10:02:48,124 ERROR stderr ... 8 more
10:02:48,124 ERROR stderr

[ariscop]

It's been over a year since there was a commit to master, webbit seems pretty dead

[gabereiser]

This issue is with netty and can easily be fixed in your own fork: https://github.com/webbit/webbit/blob/master/src/main/java/org/webbitserver/netty/NettyHttpChannelHandler.java

Webbit is dead, there are no new commits and pull requests are ignored. The source code is there so it's pretty much self-support right now.

[RedShift1]

Same problem occurs when sending a few newlines.

[DefinitlyEvil]

i got it too!

[Arisstath]

Still getting this issue, it doesn't seem that anything is breaking though.

[RedShift1]

I switched to Undertow (http://undertow.io/) if there are people here still searching an alternative.

[DefinitlyEvil]

@RedShift1 thanks good to know!