Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spanner: Retry PDML on "Received unexpected EOS on DATA frame from server" #2657

Closed
thiagotnunes opened this issue Jul 29, 2020 · 0 comments · Fixed by #2678
Closed

spanner: Retry PDML on "Received unexpected EOS on DATA frame from server" #2657

thiagotnunes opened this issue Jul 29, 2020 · 0 comments · Fixed by #2678
Assignees
@hengfengli
Labels
api: spanner Issues related to the Spanner API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@thiagotnunes
Copy link
Contributor

This bug is related to the Spanner client library.

For long lived transactions (>= 30 minutes), in the case of large PDML changes, it is possible that the gRPC connection is terminated with an error "Received unexpected EOS on DATA frame from server".

In this case, we need to retry the transaction either with the received resume token obtained on reading the stream or from scratch. This will ensure that the PDML transaction continues to execute until it is successful or a hard timeout is reached.

We have already implemented such change in the Java client library, for more information see this PR: googleapis/java-spanner#360.

In order to test the fix, we can use a large spanner database. Please speak to @thiagotnunes for more details.
@thiagotnunes thiagotnunes added triage me I really want to be triaged. api: spanner Issues related to the Spanner API. labels Jul 29, 2020
@codyoss codyoss changed the title spanner-client: Retry PDML on "Received unexpected EOS on DATA frame from server" spanner: Retry PDML on "Received unexpected EOS on DATA frame from server" Jul 29, 2020
@skuruppu skuruppu added priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed triage me I really want to be triaged. labels Jul 30, 2020
@hengfengli hengfengli mentioned this issue Aug 4, 2020
Merged
@skuruppu skuruppu removed their assignment Aug 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hengfengli hengfengli

Labels
api: spanner Issues related to the Spanner API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(spanner): update pdml to retry EOS internal error hengfengli/google-cloud-go
3 participants
@thiagotnunes @hengfengli @skuruppu