Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connection may persist on broken pipe #78

Open
gdfast opened this issue Nov 8, 2018 · 2 comments · May be fixed by #113
Open

Connection may persist on broken pipe #78

gdfast opened this issue Nov 8, 2018 · 2 comments · May be fixed by #113
Assignees
@jgaa
Labels
bug

Comments

@gdfast
Copy link

gdfast commented Nov 8, 2018 •
edited

I was doing some testing of my app in which I have restc_cpp as the client to an HTTP server (in this case, InfluxDB). In testing my app's resilience, I restarted the HTTP server and noticed that the app didn't resume successfully sending HTTP requests to the server. Instead, it kept trying to reuse the Connection, and the Connection kept being persisted.

I have been able to reproduce this in unit tests by:

  • Starting up the HTTP server and successfully sending it a requests
  • Terminating the HTTP server for some time and attempting more requests
  • Starting the HTTP server again and attempting a request

I need to dive into the code further to try and see what's going wrong, because I know the Connection logic is such that ~ReplyImpl() is usually good about closing a Connection when necessary. But I wanted to get the conversation going and enlist some help.

I'm wondering if either ConnectionPoolImp::GetConnection or ConnectionPoolImpl::OnRelease would be an appropriate place to destroy a connection with a defunct socket based on the following logs I'm seeing:

Nov 08 17:49:48.754 [ATCS|REST] TRACE (00007f9d85e58700) Reusing connection from cache {Connection f6b26435-3b20-4f70-8f01-e1dea96d2599 {Socket socket# e {std exception: remote_endpoint: Transport endpoint is not connected}}{Socket (unused/closed)}}

Nov 08 17:49:48.754 [ATCS|REST] WARN  (00007f9d85e58700) Write failed with exception type: N5boost6system12system_errorE, message: Broken pipe
Nov 08 17:49:48.755 [ATCS|REST] TRACE (00007f9d85e58700) Canceled timer SendRequestPayload
Nov 08 17:49:48.755 [ATCS|REST] TRACE (00007f9d85e58700) Recycling {Entry {Key http://127.0.0.1:8086} {Connection f6b26435-3b20-4f70-8f01-e1dea96d2599 {Socket socket# e {std exception: remote_endpoint: Transport endpoint is not connected}}{Socket (unused/closed)}}} after use

Nov 08 17:49:48.756 [ATCS|REST] TRACE (00007f9d85e58700) Reusing connection from cache {Connection f6b26435-3b20-4f70-8f01-e1dea96d2599 {Socket socket# e {std exception: remote_endpoint: Transport endpoint is not connected}}{Socket (unused/closed)}}

The problem may also be that Cache Cleanup does not reap this connection

Nov 08 17:49:48.605 [ATCS|REST] TRACE (00007f9d85e58700) Cleaning cache...
Nov 08 17:49:48.606 [ATCS|REST] TRACE (00007f9d85e58700) Keeping << {Connection f6b26435-3b20-4f70-8f01-e1dea96d2599 {Socket socket# e {std exception: remote_endpoint: Transport endpoint is not connected}}{Socket (unused/closed)}} expieres in 3a seconds

In IoTimer::Create the broken socket is also apparent:

Nov 08 17:49:48.754 [ATCS|REST] TRACE (00007f9d85e58700) Created timer SendRequestPayload for {Connection f6b26435-3b20-4f70-8f01-e1dea96d2599 {Socket socket# e {std exception: remote_endpoint: Transport endpoint is not connected}}{Socket (unused/closed)}}

test_logs.txt
@jgaa jgaa self-assigned this Nov 10, 2018
@jgaa jgaa added the bug label Nov 10, 2018
@jgaa
Copy link
Owner

jgaa commented Nov 10, 2018

Than you. I'll take a look at the issue.

@gdfast
Copy link
Author

gdfast commented Nov 12, 2018

Thanks @jgaa, much appreciated!

@patnashev patnashev linked a pull request Apr 1, 2021 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jgaa jgaa

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Lingering connections fix. patnashev/restc-cpp
2 participants
@jgaa @gdfast