grpc-js: Close http2 sessions that are dropped by their subchannels

[murgatroid99]

This should fix the connection leak described in #1085.

[Legogris]

@murgatroid99 : Any estimate for when this will be released? We have severly degraded performance after upgrading to 0.6.9 and I want to know if this fixes it.

Also, what is the recommended version to run at this point in time? The last version that looks stable so far is 0.5.4, but I haven't tried every version in-between yet.

[murgatroid99]

I was going to bundle this change with a feature release because I did not consider it high priority. This bug should have only a very minor impact for regular usage of the library. Version 0.6.9 is currently the recommended version; if you have a problem with it I suggest filing an issue.

[Legogris]

@murgatroid99 I'm running this on a host that does a large number of outbound http connections - with 0.6.9 a large portion of those requests start timing out, and I would like to see if it's related to these danlging sessions.

[murgatroid99]

I think it's very unlikely to be related. Your problem sounds similar to one reported on earlier 0.6.x versions that according to all other reports was fixed in 0.6.9 (#1064, googleapis/nodejs-pubsub#770). Can you verify that every instance of @grpc/grpc-js in npm ls is version 0.6.9, and if so can you file an issue with more information about your problem?

[Legogris]

@murgatroid99 Verified. I suspect that this is a duplicate of #1085. It manifests as outgoing HTTP connections failing (these connections don't involve grpc-js or PubSub) when publishing messages on PubSub. These issues are not present when using 0.5.4.

Where can I submit logs?

[Legogris]

@murgatroid99 I think I have a satisfactory explanation; Since we have a global http agent configured with maxSockets set to a finite value, the dangling sockets end up blocking the other http requests made from the app. Even with separate agents, a lot of dangling sockets will become an issue and it would be great to see this released soon.

EDIT: As the maxSockets is per origin, this does not fully explain it after all (it does explain failing requests to PubSub, though). Grasping at straws here, but maybe it can have to do with the maxFreeSockets setting of the http agents.

For reference, we're using https://www.npmjs.com/package/agentkeepalive with the following settings:

  maxSockets: 1000,
  maxFreeSockets: 10,
  timeout: 10000,
  freeSocketTimeout: 30000,

[murgatroid99]

OK, I see now how these might be connected. I'll publish a new version of grpc-js soon.

[murgatroid99]

Version 0.6.10 is out now