feat: AsyncIO Integration [Part 3] #29

lidizheng · 2020-05-18T22:19:01Z

Children PR of #26 and #28.

This PR includes AsyncIO version of:

LRO client
gRPC wrappers & helpers

Related #23

google/api_core/grpc_helpers_async.py

busunkim96 · 2020-05-27T02:08:20Z

google/api_core/grpc_helpers_async.py

+class FakeUnaryUnaryCall(_WrappedUnaryUnaryCall):
+    """Fake implementation for unary-unary RPCs.
+
+    It is a dummy object for response message. Supply the intended response
+    upon the initialization, and the coroutine will return the exact response
+    message.
+    """
+
+    def __init__(self, response=object()):
+        self.response = response
+        self._future = asyncio.get_event_loop().create_future()
+        self._future.set_result(self.response)
+
+    def __await__(self):
+        response = yield from self._future.__await__()
+        return response
+
+
+class FakeStreamUnaryCall(_WrappedStreamUnaryCall):
+    """Fake implementation for stream-unary RPCs.
+
+    It is a dummy object for response message. Supply the intended response
+    upon the initialization, and the coroutine will return the exact response
+    message.
+    """
+
+    def __init__(self, response=object()):
+        self.response = response
+        self._future = asyncio.get_event_loop().create_future()
+        self._future.set_result(self.response)
+
+    def __await__(self):
+        response = yield from self._future.__await__()
+        return response
+
+    async def wait_for_connection(self):
+        pass


Are these intended for use by tests only? I'd prefer they be named Mock... instead of Fake... but don't feel too strongly about it.

I considered Mock... naming, but they are not magic mock objects. It might misdirect users' expectation of its behavior. I'm happy to change if you have better naming.

Let's leave it as is then. @software-dov do you have any thoughts here?

+1 to not mock naming, although I'm a little confused as to why they don't live in the unit test file.

I think leaving them here makes it easier for dependent libraries to use it in their own tests.

(HttpMock in the apiary library for instance)

I think the 'correct' think to do then would be to have a testlib. I am in general not a fan of mixing test code and production code, but it's not a hill I'm willing to die on.

lidizheng

@busunkim96 Thank you for the suggestions. PTALA.

google/api_core/grpc_helpers_async.py

lidizheng · 2020-05-27T17:25:21Z

google/api_core/grpc_helpers_async.py

+class FakeUnaryUnaryCall(_WrappedUnaryUnaryCall):
+    """Fake implementation for unary-unary RPCs.
+
+    It is a dummy object for response message. Supply the intended response
+    upon the initialization, and the coroutine will return the exact response
+    message.
+    """
+
+    def __init__(self, response=object()):
+        self.response = response
+        self._future = asyncio.get_event_loop().create_future()
+        self._future.set_result(self.response)
+
+    def __await__(self):
+        response = yield from self._future.__await__()
+        return response
+
+
+class FakeStreamUnaryCall(_WrappedStreamUnaryCall):
+    """Fake implementation for stream-unary RPCs.
+
+    It is a dummy object for response message. Supply the intended response
+    upon the initialization, and the coroutine will return the exact response
+    message.
+    """
+
+    def __init__(self, response=object()):
+        self.response = response
+        self._future = asyncio.get_event_loop().create_future()
+        self._future.set_result(self.response)
+
+    def __await__(self):
+        response = yield from self._future.__await__()
+        return response
+
+    async def wait_for_connection(self):
+        pass


I considered Mock... naming, but they are not magic mock objects. It might misdirect users' expectation of its behavior. I'm happy to change if you have better naming.

software-dov · 2020-06-03T17:36:32Z

google/api_core/gapic_v1/method_async.py

+    if client_info is not None:
+        user_agent_metadata = [client_info.to_grpc_metadata()]
+    else:
+        user_agent_metadata = None


Style nit: what about

metadata = [client_info.to_grpc_metadata()] if client_info is not None else None

?

Good idea. Updated to the suggested style.

software-dov · 2020-06-03T17:41:45Z

google/api_core/grpc_helpers_async.py

+class FakeUnaryUnaryCall(_WrappedUnaryUnaryCall):
+    """Fake implementation for unary-unary RPCs.
+
+    It is a dummy object for response message. Supply the intended response
+    upon the initialization, and the coroutine will return the exact response
+    message.
+    """
+
+    def __init__(self, response=object()):
+        self.response = response
+        self._future = asyncio.get_event_loop().create_future()
+        self._future.set_result(self.response)
+
+    def __await__(self):
+        response = yield from self._future.__await__()
+        return response
+
+
+class FakeStreamUnaryCall(_WrappedStreamUnaryCall):
+    """Fake implementation for stream-unary RPCs.
+
+    It is a dummy object for response message. Supply the intended response
+    upon the initialization, and the coroutine will return the exact response
+    message.
+    """
+
+    def __init__(self, response=object()):
+        self.response = response
+        self._future = asyncio.get_event_loop().create_future()
+        self._future.set_result(self.response)
+
+    def __await__(self):
+        response = yield from self._future.__await__()
+        return response
+
+    async def wait_for_connection(self):
+        pass


+1 to not mock naming, although I'm a little confused as to why they don't live in the unit test file.

lidizheng · 2020-06-03T18:54:58Z

@software-dov PTALA.

busunkim96 · 2020-06-04T00:54:11Z

@lidizheng It looks like the unit tests need to be tweaked:

func = <function _wrap_unary_errors.<locals>.error_remapped_callable at 0x7fb165893790>
default_retry = <google.api_core.retry_async.AsyncRetry object at 0x7fb166304160>
default_timeout = <google.api_core.timeout.ExponentialTimeout object at 0x7fb1663046d0>
client_info = <google.api_core.gapic_v1.client_info.ClientInfo object at 0x7fb1664eeeb0>

    def wrap_method(
            func,
            default_retry=None,
            default_timeout=None,
            client_info=client_info.DEFAULT_CLIENT_INFO,
    ):
        """Wrap an async RPC method with common behavior.

        Returns:
            Callable: A new callable that takes optional ``retry`` and ``timeout``
                arguments and applies the common error mapping, retry, timeout,
                and metadata behavior to the low-level RPC method.
        """
        func = grpc_helpers_async.wrap_errors(func)

        metadata = [client_info.to_grpc_metadata()] if client_info is not None else None

        return general_helpers.wraps(func)(_GapicCallable(
>           func, default_retry, default_timeout, metadata=user_agent_metadata))
E       NameError: name 'user_agent_metadata' is not defined

software-dov

LGTM. Just one minor optional tweak and a vote from my end that the test classes be moved into a standalone test library.

google/api_core/operations_v1/__init__.py

* LRO client * gRPC wrappers & helpers * With unit tests & docs

googlebot added the cla: yes This human has signed the Contributor License Agreement. label May 18, 2020

lidizheng marked this pull request as ready for review May 18, 2020 22:34

busunkim96 requested review from busunkim96 and software-dov May 27, 2020 00:28

busunkim96 reviewed May 27, 2020

View reviewed changes

lidizheng force-pushed the aio-part-3 branch from 421ef12 to 11f63d6 Compare May 27, 2020 17:28

lidizheng commented May 27, 2020

View reviewed changes

busunkim96 mentioned this pull request May 27, 2020

chore: release 1.18.0 #33

Merged

lidizheng force-pushed the aio-part-3 branch from 11f63d6 to 9a8906e Compare June 2, 2020 22:09

software-dov reviewed Jun 3, 2020

View reviewed changes

software-dov mentioned this pull request Jun 3, 2020

asyncio support googleapis/google-cloud-python#3103

Closed

kamyar mentioned this pull request Jun 3, 2020

FR: Async requests firebase/firebase-admin-python#104

Open

lidizheng force-pushed the aio-part-3 branch from 9a8906e to cc8dc51 Compare June 3, 2020 18:53

lidizheng force-pushed the aio-part-3 branch from cc8dc51 to 7778770 Compare June 4, 2020 17:07

busunkim96 approved these changes Jun 4, 2020

View reviewed changes

software-dov approved these changes Jun 4, 2020

View reviewed changes

google/api_core/operations_v1/__init__.py Outdated Show resolved Hide resolved

feat: Third batch of AsyncIO integration

7f319ab

* LRO client * gRPC wrappers & helpers * With unit tests & docs

lidizheng force-pushed the aio-part-3 branch from 7778770 to 7f319ab Compare June 4, 2020 17:37

busunkim96 merged commit 7d8d580 into googleapis:master Jun 4, 2020

busunkim96 mentioned this pull request Jun 17, 2020

WIP: AsyncIO integration #22

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: AsyncIO Integration [Part 3] #29

feat: AsyncIO Integration [Part 3] #29

lidizheng commented May 18, 2020

busunkim96 May 27, 2020

lidizheng May 27, 2020

busunkim96 May 27, 2020

software-dov Jun 3, 2020

busunkim96 Jun 3, 2020

software-dov Jun 4, 2020

lidizheng left a comment

lidizheng May 27, 2020

software-dov Jun 3, 2020

lidizheng Jun 3, 2020

software-dov Jun 3, 2020

lidizheng commented Jun 3, 2020

busunkim96 commented Jun 4, 2020

software-dov left a comment

feat: AsyncIO Integration [Part 3] #29

feat: AsyncIO Integration [Part 3] #29

Conversation

lidizheng commented May 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lidizheng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lidizheng commented Jun 3, 2020

busunkim96 commented Jun 4, 2020

software-dov left a comment

Choose a reason for hiding this comment