Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(blob.py): auto-populate standard headers for non-chunked downloads #204

Merged
merged 13 commits into from Jul 23, 2020
Merged

Conversation

william-silversmith
Copy link
Contributor

@william-silversmith william-silversmith commented Jul 15, 2020

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #24 🦕

This PR autopopulates the following fields for non-chunked downloads based on the server header response:

blob.content_encoding
blob.content_type
blob.cache_control
blob.storage_class
blob.content_language
blob.md5_hash
blob.crc32c

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 15, 2020
@william-silversmith william-silversmith changed the title Automatically Populate Standard Headers for Non-Chunked Downloads fix(blob.py): auto-populate standard headers for non-chunked downloads Jul 15, 2020
Copy link
Contributor

@tseaver tseaver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@william-silversmith Thanks very much for the patch! In addition to tweaking the merge conflict, we need to add unit tests and / or assertions to show that the various properties are set / cleared based on the response payload.

@tseaver tseaver changed the title fix(blob.py): auto-populate standard headers for non-chunked downloads feat(blob.py): auto-populate standard headers for non-chunked downloads Jul 21, 2020
@william-silversmith
Copy link
Contributor Author

@tseaver would these unit tests be sufficient? I can get more involved if necessary. I had trouble invoking _do_download, it was interacting strangely with "MagicMock" and dying on closing a file stream.

@tseaver tseaver added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Jul 23, 2020
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jul 23, 2020
@tseaver tseaver added the automerge Merge the pull request once unit tests and other checks pass. label Jul 23, 2020
@tseaver
Copy link
Contributor

tseaver commented Jul 23, 2020

@william-silversmith Thanks, that new test was sufficient.

@william-silversmith
Copy link
Contributor Author

I'm a little confused because 806-808 have to be evaluated for the crc32c and md5_hash tests to pass.

google/cloud/storage/blob.py Outdated Show resolved Hide resolved
@google-cla
Copy link

google-cla bot commented Jul 23, 2020

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Jul 23, 2020
@tseaver
Copy link
Contributor

tseaver commented Jul 23, 2020

@googlebot I consent

@google-cla
Copy link

google-cla bot commented Jul 23, 2020

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@william-silversmith
Copy link
Contributor Author

@googlebot I fixed it.

@google-cla
Copy link

google-cla bot commented Jul 23, 2020

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@tseaver tseaver added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Jul 23, 2020
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jul 23, 2020
@william-silversmith
Copy link
Contributor Author

I could try force pushing a double quote update that removes you as an author if that would be okay with you?

@tseaver
Copy link
Contributor

tseaver commented Jul 23, 2020

@william-silversmith Sure, WFM.

@google-cla google-cla bot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Jul 23, 2020
@tseaver tseaver added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 23, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 23, 2020
@gcf-merge-on-green gcf-merge-on-green bot merged commit d8432cd into googleapis:master Jul 23, 2020
@tseaver
Copy link
Contributor

tseaver commented Jul 23, 2020

W00t! Thanks for persevering, @william-silversmith .

@william-silversmith
Copy link
Contributor Author

Excited thank you!

cojenco pushed a commit to cojenco/python-storage that referenced this pull request Oct 13, 2021
…ds (googleapis#204)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-storage/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes googleapis#24  🦕

This PR autopopulates the following fields for non-chunked downloads based on the server header response:
```
blob.content_encoding
blob.content_type
blob.cache_control
blob.storage_class
blob.content_language
blob.md5_hash
blob.crc32c
```
cojenco pushed a commit to cojenco/python-storage that referenced this pull request Oct 13, 2021
…ds (googleapis#204)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-storage/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes googleapis#24  🦕

This PR autopopulates the following fields for non-chunked downloads based on the server header response:
```
blob.content_encoding
blob.content_type
blob.cache_control
blob.storage_class
blob.content_language
blob.md5_hash
blob.crc32c
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Merge the pull request once unit tests and other checks pass. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Storage: Capture relevant headers to blob properties during download
4 participants