Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add preconditions and retry config support to ACL patch operationss #586

Merged
merged 9 commits into from Sep 15, 2021

Conversation

cojenco
Copy link
Contributor

@cojenco cojenco commented Sep 13, 2021

Add preconditions and retry config support to the ACL patch operations. The PATCH API requests for ACL in python-storage utilize storage.buckets.patch or storage.objects.patch

  • ACL patch methods in ACL (_save /save /save_predefined /clear)
  • ACL patch methods in Bucket (make_public /make_private)
  • ACL patch methods in Blob (make_public /make_private)
  • Add and update tests
  • Update doscstrings

Note: IAM configurations storage.buckets.getIamPolicy storage.buckets.setIamPolicy are highly encouraged to control access instead

Fixes #582

@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/python-storage API. label Sep 13, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 13, 2021
@cojenco cojenco marked this pull request as ready for review September 13, 2021 23:23
@cojenco cojenco requested review from a team as code owners September 13, 2021 23:23
Copy link
Contributor

@tritone tritone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these!

Copy link
Contributor

@tseaver tseaver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We patch ACLs using the objectAccessControls.pach and defaultObjectAccessControls.pach methods, which do not AFAICT take the metageneration / generation into account: instead, they rely on the resource's Etag for concurrency. Note the absence of the if_*_match parameters on those docs page.

@tseaver tseaver added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Sep 14, 2021
@tseaver
Copy link
Contributor

tseaver commented Sep 14, 2021

At a minimum, if the back-end docs are incomplete, and the back-end actually supports these semantics, we need to add one or more system tests which exercise them.

@cojenco
Copy link
Contributor Author

cojenco commented Sep 14, 2021

@tseaver We patch ACLs here using storage.buckets.patch and storage.objects.patch, instead of ACL specific endpoints (such as storage.object_acl.patch). See here that we're using bucket and object paths for patching resources.

You are right about storage.object_acl.patch storage.default_object_acl.patch relying on the resource's Etag for concurrency.

That is actually one of the considerations why storage.buckets.patch and storage.objects.patch are used here instead. If you recall our previous discussion, the If-Match header that specifies an entity tag does not work for patch operations (internal tracking bug).

Since we are using storage.buckets.patch and storage.objects.patch under the hood, it makes sense to expose the related if_*_match parameters.

@tseaver
Copy link
Contributor

tseaver commented Sep 14, 2021

My bad -- I had talked myself into "remembering" the need to back out some similar changes for ACLs in past PRs. You are indeed correct that we do not use the ACL specific APIs, and so should be free to use the if-*-match headers.

@tseaver tseaver removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Sep 14, 2021
@tseaver
Copy link
Contributor

tseaver commented Sep 14, 2021

I do think we should add a system test which exercises the new headers, though.

@cojenco cojenco merged commit 4333caf into googleapis:main Sep 15, 2021
cojenco added a commit to cojenco/python-storage that referenced this pull request Sep 15, 2021
…nss (googleapis#586)

* add preconditions and retry config support to ACL patch operations

* update existing unit tests

* add unit tests

* add preconditions and retry config to bucket make public/private

* add preconditions and retry config to blob make public/private

* update docstrings

* add system tests acl with metegeneration match

* revise to use permitted group email
@cojenco cojenco deleted the update-acl branch September 17, 2021 21:00
cojenco added a commit to cojenco/python-storage that referenced this pull request Oct 13, 2021
…nss (googleapis#586)

* add preconditions and retry config support to ACL patch operations

* update existing unit tests

* add unit tests

* add preconditions and retry config to bucket make public/private

* add preconditions and retry config to blob make public/private

* update docstrings

* add system tests acl with metegeneration match

* revise to use permitted group email
cojenco added a commit to cojenco/python-storage that referenced this pull request Oct 13, 2021
…nss (googleapis#586)

* add preconditions and retry config support to ACL patch operations

* update existing unit tests

* add unit tests

* add preconditions and retry config to bucket make public/private

* add preconditions and retry config to blob make public/private

* update docstrings

* add system tests acl with metegeneration match

* revise to use permitted group email
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/python-storage API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose preconditions to ACL methods
3 participants