Skip to content

Commit

Permalink
fix: retry non-idempotent long-running RPCs (#141)
Browse files Browse the repository at this point in the history
RPCs returning a long-running operation, such as CreateDatabase, CreateBackup and RestoreDatabase, are non-idempotent and cannot be retried automatically by gax. This means that these RPCs sometimes fail with transient errors, such as UNAVAILABLE or DEADLINE_EXCEEDED. This change introduces automatic retries of these RPCs using the following logic:
1. Execute the RPC and wait for the operation to be returned.
2. If a transient error occurs while waiting for the operation, the
   client library queries the backend for the corresponding operation.
   If the operation is found, the resumes the tracking of the existing
   operation and returns that to the user.
3. If no corresponding operation is found in step 2, the client library
   retries the RPC from step 1.

Fixes GoogleCloudPlatform/java-docs-samples#2571
  • Loading branch information
olavloite committed Apr 13, 2020
1 parent c8be7b5 commit 4669c02
Show file tree
Hide file tree
Showing 7 changed files with 1,001 additions and 224 deletions.
Expand Up @@ -21,6 +21,7 @@
import com.google.api.gax.longrunning.OperationSnapshot;
import com.google.api.gax.longrunning.OperationTimedPollAlgorithm;
import com.google.api.gax.retrying.RetrySettings;
import com.google.api.gax.rpc.StatusCode;
import com.google.api.gax.rpc.TransportChannelProvider;
import com.google.api.gax.rpc.UnaryCallSettings;
import com.google.cloud.NoCredentials;
Expand Down Expand Up @@ -291,7 +292,8 @@ private Builder() {
.setRetrySettings(longRunningRetrySettings);
databaseAdminStubSettingsBuilder
.updateBackupSettings()
.setRetrySettings(longRunningRetrySettings);
.setRetrySettings(longRunningRetrySettings)
.setRetryableCodes(StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE);
}

Builder(SpannerOptions options) {
Expand Down Expand Up @@ -581,6 +583,7 @@ public Builder setEmulatorHost(String emulatorHost) {
return this;
}

@SuppressWarnings("rawtypes")
@Override
public SpannerOptions build() {
// Set the host of emulator has been set.
Expand Down

0 comments on commit 4669c02

Please sign in to comment.