Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

samples: add samples #37

Merged
merged 23 commits into from
Nov 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
92 changes: 92 additions & 0 deletions samples/snippets/create_lite_subscription_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#!/usr/bin/env python

# Copyright 2020 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This application demonstrates how to create a subscription with the Pub/Sub
Lite API. For more information, see the root level README.md and the
documentation at https://cloud.google.com/pubsub/lite/docs/subscriptions.
"""

import argparse


def create_lite_subscription(
project_number, cloud_region, zone_id, topic_id, subscription_id
):
# [START pubsublite_create_subscription]
from google.api_core.exceptions import AlreadyExists
from google.cloud.pubsublite import AdminClient
from google.cloud.pubsublite.types import (
CloudRegion,
CloudZone,
SubscriptionPath,
TopicPath,
)
from google.cloud.pubsublite_v1 import Subscription

# TODO(developer):
# project_number = 1122334455
# cloud_region = "us-central1"
# zone_id = "a"
# topic_id = "your-topic-id"
# subscription_id = "your-subscription-id"

client = AdminClient(cloud_region)

location = CloudZone(CloudRegion(cloud_region), zone_id)
topic_path = TopicPath(project_number, location, topic_id)
subscription_path = SubscriptionPath(project_number, location, subscription_id)

subscription = Subscription(
name=str(subscription_path),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: str(), it might be a library usability issue, but isn't it better if we can just pass the SubscriptionPath as the argument? Is it possible to do it in the constructor? Feel free to ignore, or file a new issue and fix later.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type is a proto, so unfortunately not unless we want to wrap the proto.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so nevermind.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/googleapis/python-pubsublite/blob/master/google/cloud/pubsublite/types/paths.py looks like a handwritten file to me.

The generator usually produces methods to construct and decompose paths (see examples in AutoML). https://aip.dev/client-libraries/4231

Did that not happen for pubsublite?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It produces methods but not types. These methods (like

def subscription_path(project: str, location: str, subscription: str,) -> str:
) produce strings as their output: meaning there is no way to type check arguments of path type. If these produced wrapper objects I would have used them.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They also don't type check the current restriction (that we're trying to remove in the backend) that only project numbers be used in the path.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those methods did get generated for pubsublite but they aren't used.

Topic paths are created by a publisher client in Pub/Sub but by an admin client in Pub/Sub Lite.

# This looks cumbersome.
client = AdminClient(cloud_region)
topic_path = client.topic_path(project_number, location, topic_id)
with PublisherClient() as publisher_client:
    api_future = publisher_client.publish(topic_path, data.encode("utf-8"))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, thanks for the explanation. 👍

Was there a decision made that pubsublite is not going to support project IDs? https://google.aip.dev/cloud/2510 I haven't been looking at all these PRs so apologies if this is digging up an old conversation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its the difference between "must" and "should". Its nontrivial work to convert between the two and it has not been addressed until this quarter. I've sent you the internal link from the elysium team that clarifies this guidance.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The discussion was in internal b/162623801. Java currently accepts both. Python/Go will wait for this change on the server side.

topic=str(topic_path),
delivery_config=Subscription.DeliveryConfig(
# Possible values for delivery_requirement:
# - `DELIVER_IMMEDIATELY`
# - `DELIVER_AFTER_STORED`
# You may choose whether to wait for a published message to be successfully written
# to storage before the server delivers it to subscribers. `DELIVER_IMMEDIATELY` is
# suitable for applications that need higher throughput.
delivery_requirement=Subscription.DeliveryConfig.DeliveryRequirement.DELIVER_IMMEDIATELY,
),
)

try:
response = client.create_subscription(subscription)
print(f"{response.name} created successfully.")
except AlreadyExists:
print(f"{subscription_path} already exists.")
# [END pubsublite_create_subscription]


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("project_number", help="Your Google Cloud Project Number")
parser.add_argument("cloud_region", help="Your Cloud Region, e.g. 'us-central1'")
parser.add_argument("zone_id", help="Your Zone ID, e.g. 'a'")
parser.add_argument("topic_id", help="Your topic ID")
parser.add_argument("subscription_id", help="Your subscription ID")

args = parser.parse_args()

create_lite_subscription(
args.project_number,
args.cloud_region,
args.zone_id,
args.topic_id,
args.subscription_id,
)
96 changes: 96 additions & 0 deletions samples/snippets/create_lite_topic_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
#!/usr/bin/env python

# Copyright 2020 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This application demonstrates how to create a topic with the Pub/Sub
Lite API. For more information, see the root level README.md and the
documentation at https://cloud.google.com/pubsub/lite/docs/topics.
"""

import argparse


def create_lite_topic(project_number, cloud_region, zone_id, topic_id, num_partitions):
# [START pubsublite_create_topic]
from google.api_core.exceptions import AlreadyExists
from google.cloud.pubsublite import AdminClient
from google.cloud.pubsublite.types import CloudRegion, CloudZone, TopicPath
from google.cloud.pubsublite_v1 import Topic
from google.protobuf.duration_pb2 import Duration

# TODO(developer):
# project_number = 1122334455
# cloud_region = "us-central1"
# zone_id = "a"
# topic_id = "your-topic-id"
# num_partitions = 1

client = AdminClient(cloud_region)

location = CloudZone(CloudRegion(cloud_region), zone_id)
topic_path = TopicPath(project_number, location, topic_id)
topic = Topic(
name=str(topic_path),
partition_config=Topic.PartitionConfig(
# A topic must have at least one partition.
count=num_partitions,
# Set throughput capacity per partition in MiB/s.
capacity=Topic.PartitionConfig.Capacity(
# Set publish throughput capacity per partition to 4 MiB/s. Must be >= 4 and <= 16.
publish_mib_per_sec=4,
# Set subscribe throughput capacity per partition to 4 MiB/s. Must be >= 4 and <= 32.
subscribe_mib_per_sec=8,
),
),
retention_config=Topic.RetentionConfig(
# Set storage per partition to 30 GiB. This must be in the range 30 GiB-10TiB.
# If the number of byptes stored in any of the topic's partitions grows beyond
# this value, older messages will be dropped to make room for newer ones,
# regardless of the value of `period`.
per_partition_bytes=30 * 1024 * 1024 * 1024,
# Allow messages to be retained for 7 days.
period=Duration(seconds=60 * 60 * 24 * 7),
),
)

try:
response = client.create_topic(topic)
print(f"{response.name} created successfully.")
except AlreadyExists:
print(f"{topic_path} already exists.")
# [END pubsublite_create_topic]


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("project_number", help="Your Google Cloud Project Number")
parser.add_argument("cloud_region", help="Your Cloud Region, e.g. 'us-central1'")
parser.add_argument("zone_id", help="Your Zone ID, e.g. 'a'")
parser.add_argument("topic_id", help="Your topic ID")
parser.add_argument(
"num_partitions", type=int, help="Number of partitions in the topic"
)

args = parser.parse_args()

create_lite_topic(
args.project_number,
args.cloud_region,
args.zone_id,
args.topic_id,
args.num_partitions,
)
63 changes: 63 additions & 0 deletions samples/snippets/delete_lite_subscription_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env python

# Copyright 2020 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This application demonstrates how to delete a subscription with the Pub/Sub
Lite API. For more information, see the root level README.md and the
documentation at https://cloud.google.com/pubsub/lite/docs/subscriptions.
"""

import argparse


def delete_lite_subscription(project_number, cloud_region, zone_id, subscription_id):
# [START pubsublite_delete_subscription]
from google.api_core.exceptions import NotFound
from google.cloud.pubsublite import AdminClient
from google.cloud.pubsublite.types import CloudRegion, CloudZone, SubscriptionPath

# TODO(developer):
# project_number = 1122334455
# cloud_region = "us-central1"
# zone_id = "a"
# subscription_id = "your-subscription-id"

client = AdminClient(cloud_region)

location = CloudZone(CloudRegion(cloud_region), zone_id)
subscription_path = SubscriptionPath(project_number, location, subscription_id)

try:
client.delete_subscription(subscription_path)
print(f"{subscription_path} deleted successfully.")
except NotFound:
print(f"{subscription_path} not found.")
# [END pubsublite_delete_subscription]


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("project_number", help="Your Google Cloud Project Number")
parser.add_argument("cloud_region", help="Your Cloud Region, e.g. 'us-central1'")
parser.add_argument("zone_id", help="Your Zone ID, e.g. 'a'")
parser.add_argument("subscription_id", help="Your subscription ID")

args = parser.parse_args()

delete_lite_subscription(
args.project_number, args.cloud_region, args.zone_id, args.subscription_id,
)
63 changes: 63 additions & 0 deletions samples/snippets/delete_lite_topic_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env python

# Copyright 2020 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This application demonstrates how to delete a topic with the Pub/Sub
Lite API. For more information, see the root level README.md and the
documentation at https://cloud.google.com/pubsub/lite/docs/topics.
"""

import argparse


def delete_lite_topic(project_number, cloud_region, zone_id, topic_id):
# [START pubsublite_delete_topic]
from google.api_core.exceptions import NotFound
from google.cloud.pubsublite import AdminClient
from google.cloud.pubsublite.types import CloudRegion, CloudZone, TopicPath

# TODO(developer):
# project_number = 1122334455
# cloud_region = "us-central1"
# zone_id = "a"
# topic_id = "your-topic-id"

client = AdminClient(cloud_region)

location = CloudZone(CloudRegion(cloud_region), zone_id)
topic_path = TopicPath(project_number, location, topic_id)

try:
client.delete_topic(topic_path)
print(f"{topic_path} deleted successfully.")
except NotFound:
print(f"{topic_path} not found.")
# [END pubsublite_delete_topic]


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("project_number", help="Your Google Cloud Project Number")
parser.add_argument("cloud_region", help="Your Cloud Region, e.g. 'us-central1'")
parser.add_argument("zone_id", help="Your Zone ID, e.g. 'a'")
parser.add_argument("topic_id", help="Your topic ID")

args = parser.parse_args()

delete_lite_topic(
args.project_number, args.cloud_region, args.zone_id, args.topic_id,
)
63 changes: 63 additions & 0 deletions samples/snippets/get_lite_subscription_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env python

# Copyright 2020 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This application demonstrates how to get a subscription with the Pub/Sub
Lite API. For more information, see the root level README.md and the
documentation at https://cloud.google.com/pubsub/lite/docs/subscriptions.
"""

import argparse


def get_lite_subscription(project_number, cloud_region, zone_id, subscription_id):
# [START pubsublite_get_subscription]
from google.api_core.exceptions import NotFound
from google.cloud.pubsublite import AdminClient
from google.cloud.pubsublite.types import CloudRegion, CloudZone, SubscriptionPath

# TODO(developer):
# project_number = 1122334455
# cloud_region = "us-central1"
# zone_id = "a"
# subscription_id = "your-subscription-id"

client = AdminClient(cloud_region)

location = CloudZone(CloudRegion(cloud_region), zone_id)
subscription_path = SubscriptionPath(project_number, location, subscription_id)

try:
response = client.get_subscription(subscription_path)
print(f"{response.name} exists.")
except NotFound:
print(f"{subscription_path} not found.")
# [END pubsublite_get_subscription]


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("project_number", help="Your Google Cloud Project Number")
parser.add_argument("cloud_region", help="Your Cloud Region, e.g. 'us-central1'")
parser.add_argument("zone_id", help="Your Zone ID, e.g. 'a'")
parser.add_argument("subscription_id", help="Your subscription ID")

args = parser.parse_args()

get_lite_subscription(
args.project_number, args.cloud_region, args.zone_id, args.subscription_id,
)