Create specific exception classes for datastore errors

[przemekpastuszka]

Hello guys,

I'm in a process of migration from AppEngine NDB to Cloud NDB and I find it hard to identify and handle exceptional cases that can occur. While AppEngine NDB had lots of meaningful exception classes like TransactionFailedError, I cannot find any equivalent in Cloud NDB (although migration notes suggest there will be "shims" created for that).

Here are some examples of the errors I'm getting right now in my app after migration:
_Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.ABORTED, too much contention on these datastore entities. please try again. entity groups: [....])>

_Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, {"created":"@1588764808.925861830","description":"Transport closed","file":"third_party/apphosting/python/grpcio/v1_0_0/src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":1031})>

_Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.INVALID_ARGUMENT, The referenced transaction has expired or is no longer valid.)>

<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Received RST_STREAM with error code 7)>

The biggest problems are:

• the exception class starts with underscore which suggests it's a private class and as such it should never be thrown outside
• status code ABORTED or INVALID_ARGUMENT is uninformative (alone it doesn't tell me whether it's a transient error I can retry on). I need to match against the detail string to understand what happened

[przemekpastuszka]

Looks like there are exceptions already defined in google.api_core.exceptions. Moreover, there's a method that is able to translate from _Rendezvous to those exceptions - from_grpc_error.

I will probably use that method to parse exceptions on my side, but ideally those should be thrown by python-ndb itself.

[przemekpastuszka]

I am also getting the google.api_core.exceptions.RetryError with _Rendezvous as a cause:
RetryError: Maximum number of 3 retries exceeded while calling <function rpc_call at 0x3e467886aa28>, last exception: <_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Received RST_STREAM with error code 7)>

[chrisrossi]

@rtshadow Thanks for the from_grpc_error tip!

[przemekpastuszka]

@chrisrossi, thanks for the fix. Unfortunately, with 1.3.0 I'm still getting RetryError: Maximum number of 3 retries exceeded while calling <function rpc_call at 0x3e77fe6630c8>, last exception: 503 Received RST_STREAM with error code 7

I believe you should unwrap the underlying exception from RetryError.