Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add flag to allow disabling auto decompression by client #850

Merged
merged 14 commits into from Oct 4, 2019
Merged

feat: add flag to allow disabling auto decompression by client #850

merged 14 commits into from Oct 4, 2019

Conversation

AVaksman
Copy link
Contributor

Fixes #827

  • Tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Sep 11, 2019
@codecov
Copy link

codecov bot commented Sep 11, 2019 •
edited

Codecov Report

Merging #850 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #850      +/-   ##
==========================================
+ Coverage   95.26%   95.27%   +<.01%     
==========================================
  Files          11       11              
  Lines        1205     1206       +1     
  Branches      300      300              
==========================================
+ Hits         1148     1149       +1     
  Misses         29       29              
  Partials       28       28
Impacted Files Coverage Δ
src/file.ts 98.16% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 302ad88...489b4cd. Read the comment docs.

@AVaksman AVaksman added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 11, 2019
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 11, 2019
stephenplusplus
stephenplusplus reviewed Sep 11, 2019
View reviewed changes
src/file.ts Outdated
@@ -359,6 +359,7 @@ export interface CreateReadStreamOptions {
validation?: 'md5' | 'crc32c' | false | true;
start?: number;
end?: number;
returnCompressed?: boolean;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about decompress, default true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

stephenplusplus
stephenplusplus reviewed Sep 11, 2019
View reviewed changes
src/file.ts Outdated
@@ -1113,8 +1113,8 @@ class File extends ServiceObject<File> {
* NOTE: Byte ranges are inclusive; that is, `options.start = 0` and
* `options.end = 999` represent the first 1000 bytes in a file or object.
* NOTE: when specifying a byte range, data integrity is not available.
* @property {booblean} [returnCompressed] Disable auto decompression of the
* received data. By default this option set to `false`.
* @property {booblean} [decompress] Disable auto decompression of the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you put [decompress=true], that'll tip off the docs site that the default value is true. Also, I think it's just called a boolean 😅.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! 😹

@AVaksman AVaksman added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 11, 2019
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 11, 2019
@AVaksman AVaksman added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 12, 2019
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 12, 2019
@AVaksman AVaksman added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 12, 2019
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 12, 2019
@AVaksman AVaksman added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 17, 2019
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 17, 2019
@AVaksman AVaksman marked this pull request as ready for review September 17, 2019 17:44
@JustinBeckwith
Copy link
Contributor

@frankyn @jkwlui mind taking a look?

frankyn
frankyn requested changes Sep 25, 2019
View reviewed changes
src/file.ts Outdated Show resolved Hide resolved
src/file.ts
@@ -1120,6 +1121,10 @@ class File extends ServiceObject<File> {
* NOTE: Byte ranges are inclusive; that is, `options.start = 0` and
* `options.end = 999` represent the first 1000 bytes in a file or object.
* NOTE: when specifying a byte range, data integrity is not available.
* @property {boolean} [decompress=true] Disable auto decompression of the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this from the perspective of the GCS service?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, nodejs-sotage API always requests that the data should be served as is (compressed if applicable) i. e. sending appropriate headers

const headers = {
  'Accept-Encoding': 'gzip',
  'Cache-Control': 'no-store',
} as Headers;

This flag is only from the perspective of nodejs-storage client to allow this library to not transcode the data if received from the server in compressed form and return to user compressed..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@frankyn ping!

@bcoe bcoe changed the title feat: add flag to allow disabling auto decomression by client feat: add flag to allow disabling auto decompression by client Sep 30, 2019
@stephenplusplus
Copy link
Contributor

@AVaksman this also fixes #709, right?

jkwlui
jkwlui reviewed Oct 3, 2019
View reviewed changes
test/file.ts Outdated Show resolved Hide resolved
frankyn
frankyn approved these changes Oct 4, 2019
View reviewed changes
Copy link
Member

@frankyn frankyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pending @jkwlui's last comment. Otherwise LGTM. Thanks for the ping.

@AVaksman
Copy link
Contributor Author

AVaksman commented Oct 4, 2019

@AVaksman this also fixes #709, right?

@stephenplusplus No, the bottom line of #709 is that server does not respect cache-control: no transform header from JSON API (only XML) and does not serve doubly compressed data (for example jpg + gzip), but rather automatically removes the top level compression (gzip) and serves the data singly compressed (just .jpg).

This PR cover the case when user is downloading txt.gzip file. Upon receiving content-encoding: gzip, the library would auto decompress the data and return to user (just .txt). Here we allowing the user to request from the library not to auto-decompress the data upon receipt but rather pass it to the user as is (.txt.gzip).

@jkwlui jkwlui merged commit 9ebface into googleapis:master Oct 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stephenplusplus stephenplusplus stephenplusplus left review comments

@frankyn frankyn frankyn approved these changes

@jkwlui jkwlui jkwlui approved these changes

Assignees
No one assigned
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Allow disabling automatic decompression when downloading files
7 participants
@AVaksman @JustinBeckwith @stephenplusplus @frankyn @jkwlui @googlebot @kokoro-team