Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: Reader uses the xml api but sends json preconditions #4470

Closed
fejta opened this issue Jul 20, 2021 · 0 comments · Fixed by #4479
Closed

storage: Reader uses the xml api but sends json preconditions #4470

fejta opened this issue Jul 20, 2021 · 0 comments · Fixed by #4479
Assignees
Labels
api: storage Issues related to the Cloud Storage API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@fejta
Copy link

fejta commented Jul 20, 2021

  • NewRangerReader uses the XML api to download files.
  • conditionsQuery() uses the JSON API to configure preconditions
  • NewRangerReader calls conditionsQuery() rather than setting the expected precondition headers.

ObjectHandle.NewReader() calls NewRangeReader:

func (o *ObjectHandle) NewReader(ctx context.Context) (*Reader, error) {
return o.NewRangeReader(ctx, 0, -1)
}

This eventually sets req.URL.RawQuery = conditionsQuery(gen, o.conds):

// We wait to assign conditions here because the generation number can change in between reopen() runs.
req.URL.RawQuery = conditionsQuery(gen, o.conds)

conditionsQuery() appends the applicable JSON API URL parameters:

switch {
case conds.GenerationMatch != 0:
appendParam("ifGenerationMatch=", conds.GenerationMatch)
case conds.GenerationNotMatch != 0:
appendParam("ifGenerationNotMatch=", conds.GenerationNotMatch)
case conds.DoesNotExist:
appendParam("ifGenerationMatch=", 0)
}
switch {
case conds.MetagenerationMatch != 0:
appendParam("ifMetagenerationMatch=", conds.MetagenerationMatch)
case conds.MetagenerationNotMatch != 0:
appendParam("ifMetagenerationNotMatch=", conds.MetagenerationNotMatch)
}

However, the readHost is not the json api, it is the xml api:

readHost = "storage.googleapis.com"

The xml API expects an x-goog-if-generation-match header. It does not recognize the JSON API's url parameters.

Thus the following will always work and never return the expected preconditon error:

client := storage.NewClient()
r, err := client.Bucket("whatever").Object("some/object").If(&storage.Conditions{GenerationMatch: 666}).NewReader(context.Background())
if err == nil {
  panic("should have failed")
}
gerr, ok := err.(*googleapi.Error)
if !ok {
  panic("should have returned a googleapi error")
}
if gerr.Code != http.PreconditionFailed {
  panic("should have return a precondition failed error")
}
// congrats! This bug no longer repros.
@fejta fejta added the triage me I really want to be triaged. label Jul 20, 2021
@product-auto-label product-auto-label bot added the api: storage Issues related to the Cloud Storage API. label Jul 20, 2021
@tritone tritone added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p2 Moderately-important priority. Fix may not be included in next release. and removed triage me I really want to be triaged. labels Jul 21, 2021
gcf-merge-on-green bot pushed a commit that referenced this issue Jul 23, 2021
This updates `Reader` to send precondition request headers to fit the XML API used for downloads. 

- adds a new function `setConditionsHeaders()` that parses `Conditions` and sets the corresponding precondition headers
- revises `NewRangerReader` to call `setConditionsHeaders()`
- if an object version is specified via `generation`, include `generation` as query string parameters
- updates tests

Fixes #4470
gcf-merge-on-green bot pushed a commit that referenced this issue Aug 31, 2021
🤖 I have created a release \*beep\* \*boop\*
---
### [1.16.1](https://www.github.com/googleapis/google-cloud-go/compare/storage/v1.16.0...storage/v1.16.1) (2021-08-30)


### Bug Fixes

* **storage/internal:** Update encryption_key fields to "bytes" type. fix: Improve date/times and field name clarity in lifecycle conditions. ([a52baa4](https://www.github.com/googleapis/google-cloud-go/commit/a52baa456ed8513ec492c4b573c191eb61468758))
* **storage:** accept emulator env var without scheme ([#4616](https://www.github.com/googleapis/google-cloud-go/issues/4616)) ([5f8cbb9](https://www.github.com/googleapis/google-cloud-go/commit/5f8cbb98070109e2a34409ac775ed63b94d37efd))
* **storage:** preserve supplied endpoint's scheme ([#4609](https://www.github.com/googleapis/google-cloud-go/issues/4609)) ([ee2756f](https://www.github.com/googleapis/google-cloud-go/commit/ee2756fb0a335d591464a770c9fa4f8fe0ba2e01))
* **storage:** remove unnecessary variable ([#4608](https://www.github.com/googleapis/google-cloud-go/issues/4608)) ([27fc784](https://www.github.com/googleapis/google-cloud-go/commit/27fc78456fb251652bdf5cdb493734a7e1e643e1))
* **storage:** retry LockRetentionPolicy ([#4439](https://www.github.com/googleapis/google-cloud-go/issues/4439)) ([09879ea](https://www.github.com/googleapis/google-cloud-go/commit/09879ea80cb67f9bfd8fc9384b0fda335567cba9)), refs [#4437](https://www.github.com/googleapis/google-cloud-go/issues/4437)
* **storage:** revise Reader to send XML preconditions ([#4479](https://www.github.com/googleapis/google-cloud-go/issues/4479)) ([e36b29a](https://www.github.com/googleapis/google-cloud-go/commit/e36b29a3d43bce5c1c044f7daf6e1db00b0a49e0)), refs [#4470](https://www.github.com/googleapis/google-cloud-go/issues/4470)

This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants