feat: async changes to resumable upload/download #153

anibadde · 2020-08-03T15:58:30Z

AsyncIO functionality for resumable media

Current State: All unit tests for asynchronous functionality are passing. The system tests are close to complete, though a timeout context manager error means that we are currently not able to run the upload and download tests simultaneously and need to run each in separate sessions to avoid that bug.

Related Branch for auth: https://github.com/googleapis/google-auth-library-python/tree/async

google/async_resumable_media/_download.py

tritone

Some initial comments-- this definitely needs some cleanup.

google/async_resumable_media/__init__.py

google/async_resumable_media/_download.py

google/async_resumable_media/_helpers.py

* fix(docs-presubmit): changes needed for docs-presubmit build * docs: update docs build (via synth) By manually running synthtool * add docfx nox session

anibadde

Review Comments

google/async_resumable_media/_download.py

google/async_resumable_media/_helpers.py

google/async_resumable_media/_download.py

* refactor: use named consts for session Python versions Closes googleapis#150.

Co-authored-by: Tres Seaver <tseaver@palladion.com>

Prepare for upcoming version, which will namespace the module to avoid conflict with the ICRAR version. See: googleapis/python-crc32c#29

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Tres Seaver <tseaver@palladion.com>

google-cla · 2020-08-06T17:35:09Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-08-06T22:17:10Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-08-07T03:25:17Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-08-07T04:07:51Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-08-07T05:44:07Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

…media-python into async

google-cla · 2020-09-25T01:58:40Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-09-25T19:22:12Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

crwilcox · 2020-09-28T22:36:02Z

Merging. PR to follow to move to now internal surfaces on auth. Also it is now on the main branch of auth 🎉

crwilcox · 2020-09-28T22:39:21Z

Thanks @anibadde for all your work on this!

Merge long-lived async branch * refactor: use named consts for session Python versions * chore(deps): pin 'google-crc32c < 0.2dev' (#160) * feat: async changes to resumable upload/download (#153) * chore(deps): use google-auth >= 1.22.0, 2.0dev which includes google.auth.default_async, aiohttp_requests (#176) * fix: move `async_resumable_media` to `_async_resumable_media` (#178) Co-authored-by: Chris Cotter <cjcotter@google.com> Co-authored-by: Takashi Matsuo <tmatsuo@google.com> Co-authored-by: Tres Seaver <tseaver@palladion.com> Co-authored-by: Anirudh Baddepudi <43104821+anibadde@users.noreply.github.com> Co-authored-by: Anirudh Baddepuddi <anibadde@google.com>

feat: async changes to resumable upload/download

72c5dce

google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Aug 3, 2020

crwilcox previously requested changes Aug 3, 2020

View reviewed changes

google/async_resumable_media/_download.py Outdated Show resolved Hide resolved

google/async_resumable_media/_download.py Outdated Show resolved Hide resolved

google/async_resumable_media/_download.py Show resolved Hide resolved

tritone reviewed Aug 3, 2020

View reviewed changes

anibadde and others added 4 commits August 3, 2020 19:26

fix: _download fixes

ff6675c

fix: More comment fixes

a2dbfff

fix: checksum changes

b9b7c29

docs: update docs build (via synth) (googleapis#155)

1c33de4

* fix(docs-presubmit): changes needed for docs-presubmit build * docs: update docs build (via synth) By manually running synthtool * add docfx nox session

anibadde commented Aug 5, 2020

View reviewed changes

anibadde and others added 8 commits August 5, 2020 10:37

fix: system tests and cleanup

faa0d4d

tests: harden systest teardowns vs transient GCS errors (googleapis#158)

c98855e

* refactor: use named consts for session Python versions Closes googleapis#150.

chore(deps): update ubuntu docker tag to v20.10 (googleapis#156)

0731354

Co-authored-by: Tres Seaver <tseaver@palladion.com>

fix: gzip debugging in progress

a2d3641

fix: cleanup

ce6facf

chore(deps): pin 'google-crc32c < 0.2dev' (googleapis#160)

52a322d

Prepare for upcoming version, which will namespace the module to avoid conflict with the ICRAR version. See: googleapis/python-crc32c#29

fix: full coverage

934f95f

chore: release 0.7.1 (googleapis#157)

f15d7d0

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Tres Seaver <tseaver@palladion.com>

google-cla bot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Aug 6, 2020

fix: gzip system tests

67e7169

fix: cleanup and system tests fix

c57a35b

fix: system tests (to fix multidict issue and context manager)

8517480

modify to use new auth surface

c533b33

anibadde added 2 commits August 7, 2020 01:02

fix: local changes

d01f986

Merge branch 'async' of https://github.com/AniBadde/google-resumable-…

f07c289

…media-python into async

google-cla bot added the cla: no This human has *not* signed the Contributor License Agreement. label Sep 25, 2020

crwilcox added cla: yes This human has signed the Contributor License Agreement. kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Sep 25, 2020

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 25, 2020

fix: aiohttp requires 3.6

2e527dc

crwilcox added the kokoro:run Add this label to force Kokoro to re-run the tests. label Sep 25, 2020

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Sep 25, 2020

google-cla bot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Sep 25, 2020

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 25, 2020

crwilcox added the kokoro:run Add this label to force Kokoro to re-run the tests. label Sep 25, 2020

chore(deps): pin to commit of dev google-auth

9a03e35

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Sep 25, 2020

google-cla bot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Sep 25, 2020

crwilcox closed this Sep 25, 2020

crwilcox reopened this Sep 26, 2020

crwilcox merged commit 190a62c into googleapis:async Sep 28, 2020

crwilcox mentioned this pull request Sep 29, 2020

feat: add _async_resumable_media experimental support #179

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: async changes to resumable upload/download #153

feat: async changes to resumable upload/download #153

anibadde commented Aug 3, 2020 •

edited by crwilcox

tritone left a comment

anibadde left a comment

google-cla bot commented Aug 6, 2020

google-cla bot commented Aug 6, 2020

google-cla bot commented Aug 7, 2020

google-cla bot commented Aug 7, 2020

google-cla bot commented Aug 7, 2020

google-cla bot commented Sep 25, 2020

google-cla bot commented Sep 25, 2020

crwilcox commented Sep 28, 2020

crwilcox commented Sep 28, 2020

feat: async changes to resumable upload/download #153

feat: async changes to resumable upload/download #153

Conversation

anibadde commented Aug 3, 2020 • edited by crwilcox

tritone left a comment

Choose a reason for hiding this comment

anibadde left a comment

Choose a reason for hiding this comment

google-cla bot commented Aug 6, 2020

google-cla bot commented Aug 6, 2020

google-cla bot commented Aug 7, 2020

google-cla bot commented Aug 7, 2020

google-cla bot commented Aug 7, 2020

google-cla bot commented Sep 25, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

google-cla bot commented Sep 25, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

crwilcox commented Sep 28, 2020

crwilcox commented Sep 28, 2020

anibadde commented Aug 3, 2020 •

edited by crwilcox