Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bigtable): Managed Backups wrappers #57

Merged
merged 27 commits into from Jul 21, 2020

Conversation

mf2199
Copy link
Contributor

@mf2199 mf2199 commented Jun 18, 2020

A draft/beta implementation of Managed Backups' wrappers for Google Cloud Bigtable.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jun 18, 2020
Copy link
Collaborator

@kolea2 kolea2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all of the work here! Left a few comments - also, are there any integration tests?

google/cloud/bigtable/backup.py Outdated Show resolved Hide resolved
google/cloud/bigtable/backup.py Outdated Show resolved Hide resolved
self.name + "/clusters/" + cluster_id,
filter_=filter_,
order_by=order_by,
page_size=page_size,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the default here if not specified?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to bigtable_table_admin.pb2.py,

      page_size:
          Number of backups to be returned in the response. If 0 or
          less, defaults to the server’s maximum allowed page size.

In the list_backups method of bigtable_table_admin_client.py, this parameter defaults to None, so here we follow the same logic. But I agree, it should probably default to 0 rather than none-type, will change.

google/cloud/bigtable/backup.py Show resolved Hide resolved
google/cloud/bigtable/backup.py Outdated Show resolved Hide resolved
google/cloud/bigtable/backup.py Outdated Show resolved Hide resolved
google/cloud/bigtable/instance.py Outdated Show resolved Hide resolved
@mf2199
Copy link
Contributor Author

mf2199 commented Jun 24, 2020

@kolea2 Added the integration test. To avoid repetitive LROs, everything is placed into a single comprehensive test method inside TestTableAdminAPI class. But please let me know if you'd rather see several smaller tests covering each individual Backup function separately.

@mf2199
Copy link
Contributor Author

mf2199 commented Jun 29, 2020

@kolea2 PTAL. Replaced some string composition code with 'canonical' methods from bigtable_table_admin_client; added __eq__ and __ne__ convenience methods commonly found in other classes (e.g. Table); as well as some other refactoring. Also, giving another thought to the reload method that was removed in compliance with other Backups libraries, maybe we ought to have it back, as this is another quite common method, at least in Python APIs. Just a thought.

@kolea2
Copy link
Collaborator

kolea2 commented Jul 8, 2020

Also, giving another thought to the reload method that was removed in compliance with other Backups libraries, maybe we ought to have it back, as this is another quite common method, at least in Python APIs. Just a thought.

SGTM for the reload method

# See the License for the specific language governing permissions and
# limitations under the License.

"""A user-friendly container for Cloud Bigtable Backup."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure container is the correct word here? wrapper?. Also a Cloud Bigtable Backup or Cloud Bigtable Backups perhaps?

And it seems we mix google cloud bigtable and cloud bigtable. Maybe best to be consistent?

Copy link
Contributor Author

@mf2199 mf2199 Jul 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed about the wrapper vs container as well as Google Cloud Bigtable. Not sure about the article though, since it implies a definition of notion here, rather than "a" (i.e. particular) Backup class. I've put it in for now, just must say that it's debatable.

google/cloud/bigtable/backup.py Outdated Show resolved Hide resolved
backup_id = match.group("backup_id")
cluster_id = match.group("cluster_id")

match = re.compile(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a const regex above. Should this be done similarly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. We couldn't decide whether to declare re as constants, hence the two ended up in different representations) It's fixed now.

"""
if not self._expire_time:
raise ValueError('"expire_time" parameter must be set')
# TODO: Consider implementing a method that sets a default value of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create an issue to track this as a feature request?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, that's the idea. I'd just wait until the initial release is merged, but please let know if you'd rather see it sooner.

google/cloud/bigtable/backup.py Outdated Show resolved Hide resolved
:param backup_id: The ID of the Backup to be created.

:type cluster_id: str
:param cluster_id: (Optional) The ID of the Cluster. Required for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems odd we have an optional but required parameter. Can we get this by retrieving the backup? This kind of seems required? What is the use case gained by making this optional? Is this just to make it easier to create a client side object?

tests/system.py Outdated Show resolved Hide resolved
tests/system.py Outdated Show resolved Hide resolved
tests/system.py Outdated Show resolved Hide resolved
@kolea2 kolea2 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 17, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 17, 2020
@kolea2
Copy link
Collaborator

kolea2 commented Jul 20, 2020

@mf2199 lgtm - would you be able to take a look at the build error?

Edit: when the tests are fixed, please mark as ready for review (from draft) as well. Thanks!

@mf2199
Copy link
Contributor Author

mf2199 commented Jul 21, 2020

@kolea2 Done. According to the log, one of the errors was due to the datetime.timestamp method not existing in Python 2, the support of which is currently being dropped. The Python 3 tests run OK. I've added the backwards compatibility at the expense of somewhat less elegant code.

@mf2199 mf2199 marked this pull request as ready for review July 21, 2020 02:23
@kolea2 kolea2 merged commit a351734 into googleapis:master Jul 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants