Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The gRPC server bidirectional streams do not receive exceptions when the client loses connectivity #28

Open
doctorpangloss opened this issue Dec 27, 2022 · 1 comment
Assignees
Labels
bug Something isn't working
Milestone

Comments

@doctorpangloss
Copy link

Version

4.3.7

Context

Using request.exceptionHandler(...), bidirectional streams can respond to ungraceful stream closure, which could be killing an app, a browser crashing, a phone running out of battery, network disconnection or other failure.

This package does not correctly receive exceptions the way Netty does. It lacks the configuration that the Netty server has for keep alives. This appears to be the root issue.

Steps to reproduce

  1. Generate a bidi streaming method stub using the original Netty-based Vertx package's jprotoc based code generator. You will have a method of the signature:
void bidiCall(ReadStream<X> request, WriteStream<Y> response);
  1. Bind it to the Netty-based VertxServer. Configure it with timeouts:
var builder = VertxServerBuilder.forPort(vertx, port);
var nettyServerBuilder = builder.nettyBuilder();
nettyServerBuilder
.maxConnectionIdle(Long.MAX_VALUE, TimeUnit.NANOSECONDS)
.maxConnectionAge(29, TimeUnit.DAYS)
.keepAliveTime(400, TimeUnit.MILLISECONDS)
.keepAliveTimeout(8000, TimeUnit.MILLISECONDS)
.permitKeepAliveTime(100, TimeUnit.MILLISECONDS)
.permitKeepAliveWithoutCalls(true);
builder.build();
  1. Proxy the server with ToxiProxy.
  2. Connect to the bidi RPC endpoint.
  3. Observe you are connected.
  4. Disconnect the network using ToxiProxy.
  5. After a short while, observe a StatusRuntimeException with status Status.CANCELLED is delivered to request.exceptionHandler.
  6. Bind the same service to a GrpcServer.
  7. Proxy it with ToxiProxy.
  8. Connect to the bidi RPC endpoint.
  9. Observe you are connected.
  10. Disconnect the network using ToxiProxy.
  11. Observe no exception is received nor the endHandler called in a short while.

Extra

This is reproduced in the tests of my own open source program. I can share that one. Otherwise I will need help authoring tests in this package, since Maven in my IntelliJ does not appear to generate the protos / reference them correctly and it's impossible to code. Additionally I suggest adding TestContainers as a dependency for using ToxiProxy.

@doctorpangloss doctorpangloss added the bug Something isn't working label Dec 27, 2022
@doctorpangloss doctorpangloss changed the title The gRPC server bidirectional streams do not receive exceptions when the client loses connectivity, root issue is config options The gRPC server bidirectional streams do not receive exceptions when the client loses connectivity Jun 26, 2023
@vietj vietj added this to the 4.4.5 milestone Jun 26, 2023
@doctorpangloss
Copy link
Author

I ended up using the KeepAliveManager from io.grpc. You will need to deal with adapting ScheduledFuture.

@vietj vietj self-assigned this Jul 4, 2023
@vietj vietj modified the milestones: 4.4.5, 4.4.6 Aug 30, 2023
@vietj vietj modified the milestones: 4.4.6, 4.5.0 Sep 12, 2023
@vietj vietj modified the milestones: 4.5.0, 4.5.1 Nov 15, 2023
@vietj vietj modified the milestones: 4.5.1, 4.5.2 Dec 13, 2023
@vietj vietj modified the milestones: 4.5.2, 4.5.3 Jan 30, 2024
@vietj vietj modified the milestones: 4.5.3, 4.5.4 Feb 6, 2024
@vietj vietj modified the milestones: 4.5.4, 4.5.5 Feb 22, 2024
@vietj vietj modified the milestones: 4.5.5, 4.5.6 Mar 14, 2024
@vietj vietj modified the milestones: 4.5.6, 4.5.7, 4.5.8 Mar 21, 2024
@vietj vietj modified the milestones: 4.5.8, 4.5.9 May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants