Skip to content
This repository has been archived by the owner on Feb 1, 2024. It is now read-only.

Commit

Permalink
Browse files Browse the repository at this point in the history
docs: add samples from python-docs-samples/tasks (#36)
* Add samples for Cloud Tasks [(#1068)](GoogleCloudPlatform/python-docs-samples#1068)

* Add samples for Cloud Tasks

* Respond to tasks sample review

* Update app engine queues samples

* Address review feedback

* Address review issues and convert pull queue sample to not use API key auth

* Reform pull queues to match appengine queues changes to auth, command line input, readme

* flake8 and fix comment

* Fix Tasks sample test issues.

* Remove queue.yaml, now unused

* Add required flag on certain arguments to command-line tools

* Auto-update dependencies. [(#1116)](GoogleCloudPlatform/python-docs-samples#1116)

* Auto-update dependencies. [(#1133)](GoogleCloudPlatform/python-docs-samples#1133)

* Auto-update dependencies.

* Fix missing http library

Change-Id: I99faa600f2f3f1f50f57694fc9835d7f35bda250

* Update Task Queue samples for beta

* Use full import to get client

* Auto-update dependencies. [(#1186)](GoogleCloudPlatform/python-docs-samples#1186)

* Auto-update dependencies. [(#1217)](GoogleCloudPlatform/python-docs-samples#1217)

* Added "Open in Cloud Shell" buttons to README files [(#1254)](GoogleCloudPlatform/python-docs-samples#1254)

* Add comments and region tags to Cloud Tasks samples [(#1271)](GoogleCloudPlatform/python-docs-samples#1271)

* Standardize on CamelCase, reword confusing endpoint name [(#1288)](GoogleCloudPlatform/python-docs-samples#1288)

* Auto-update dependencies. [(#1309)](GoogleCloudPlatform/python-docs-samples#1309)

* Rename pull to lease and fix name/parent confusion [(#1311)](GoogleCloudPlatform/python-docs-samples#1311)

* Updated region tags for pull queue samples [(#1329)](GoogleCloudPlatform/python-docs-samples#1329)

* Auto-update dependencies. [(#1320)](GoogleCloudPlatform/python-docs-samples#1320)

* Auto-update dependencies. [(#1355)](GoogleCloudPlatform/python-docs-samples#1355)

* Auto-update dependencies. [(#1359)](GoogleCloudPlatform/python-docs-samples#1359)

* Auto-update dependencies.

* Update Cloud Tasks Samples [(#1529)](GoogleCloudPlatform/python-docs-samples#1529)

* passing create task

* Passing tests

* updates to region tags

* update region tags [(#1532)](GoogleCloudPlatform/python-docs-samples#1532)

* update Tasks Sample for App Engine [(#1541)](GoogleCloudPlatform/python-docs-samples#1541)

* update gcloud command for creating queues

* deploys and runs

* update license

* passing tests

* Fix run command [(#1563)](GoogleCloudPlatform/python-docs-samples#1563)

* Updated library for TTS GA [(#1552)](GoogleCloudPlatform/python-docs-samples#1552)

* update gcloud command for Cloud Tasks [(#1566)](GoogleCloudPlatform/python-docs-samples#1566)

* update gcloud command

* update pull queue command

* update pull queue command

* Update Cloud Tasks Push Queue Sample [(#1698)](GoogleCloudPlatform/python-docs-samples#1698)

* deleted pull queues

* updated samples

* fix dependency versions

* [Cloud Tasks] Move samples to new folder [(#2114)](GoogleCloudPlatform/python-docs-samples#2114)

* Move samples to keep consistent with other langauges

* Ad system tests as well

* [Cloud Tasks] Add task with authentication sample [(#2113)](GoogleCloudPlatform/python-docs-samples#2113)

* Add task with authentication sample

* Fix linting

* Fix linting

* Fix spacing

* Update tests with service account

* Move samples and update READMEs

* Update version and linting

* Update task sample comments [(#2156)](GoogleCloudPlatform/python-docs-samples#2156)

* Update task comments

* Update readme

* Update queue name

* update gcloud [(#2208)](GoogleCloudPlatform/python-docs-samples#2208)

* Add protobuf dep and install instructions [(#2250)](GoogleCloudPlatform/python-docs-samples#2250)

* Add Migration Guide Snippets for Cloud Tasks [(#2316)](GoogleCloudPlatform/python-docs-samples#2316)

* Migration guide

* remove app and update migraitonn

* snippets for migration guide - tests added

* lint

* remove print statements

* Styling changes

* Travis trigger

* Update create_http_task.py [(#2187)](GoogleCloudPlatform/python-docs-samples#2187)

Updates `create_http_task.py` to have missing `in_seconds` variable

* Adds updates for samples profiler ... vision [(#2439)](GoogleCloudPlatform/python-docs-samples#2439)

* Update Cloud Tasks library version [(#2516)](GoogleCloudPlatform/python-docs-samples#2516)

* Update to new library

* update library version

* Adds Task name while creating Task [(#2543)](GoogleCloudPlatform/python-docs-samples#2543)

* Task Name in Creating HTTP Task with Token [(#2700)](GoogleCloudPlatform/python-docs-samples#2700)

* chore(deps): update dependency google-cloud-tasks to v1.5.0 [(#3168)](GoogleCloudPlatform/python-docs-samples#3168)

* chore(deps): update dependency googleapis-common-protos to v1.51.0 [(#3171)](GoogleCloudPlatform/python-docs-samples#3171)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [googleapis-common-protos](https://togithub.com/googleapis/googleapis) | minor | `==1.6.0` -> `==1.51.0` |

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* Remove dependency googleapis-common-protos [(#3955)](GoogleCloudPlatform/python-docs-samples#3955)

* Update dependency googleapis-common-protos to v1.52.0

* Update requirements.txt

* Update requirements.txt

* Update requirements.txt

Co-authored-by: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](GoogleCloudPlatform/python-docs-samples#4022)

* [tasks] testing: use fixtures for the queue [(#4049)](GoogleCloudPlatform/python-docs-samples#4049)

fixes #4045
fixes #4044

I don't know why these tests started to fail, but anyways we'd better
use fixtures and temporary queues.

* add python snippets and tests for creating, listing, and deleting queues [(#4012)](GoogleCloudPlatform/python-docs-samples#4012)

* add python snippets and tests for creating, listing, and deleting queues

* fix grammar

* update licenses

* apply suggested fixes and format with black

* refine delete_queue_test with fixture for setup

* utilize fixtures and match format of create_http_task_test

* utilize fixtures in list_queues_test and create_queue_test

* make create_queue_test call the right function

* still attempt to delete queue after test runs in case of failure

* attempt to delete queue in case of failure, using try/except approach

* add print when NotFound is caught

* fix import

Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* docs(tasks): service_account_email parameter example [(#4183)](GoogleCloudPlatform/python-docs-samples#4183)

* docs(tasks): serviceAccountEmail parameter example

* docs(tasks): parameter name camel => snake case

* docs(tasks): comment order = function param order

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* Update dependency pytest to v6 [(#4390)](GoogleCloudPlatform/python-docs-samples#4390)

* tasks: added json content-type request [(#4473)](GoogleCloudPlatform/python-docs-samples#4473)

- added json payload compatibility
- fix imports and code block used on https://cloud.google.com/tasks/docs/creating-http-target-tasks#python

## Description

Fixes #<ISSUE-NUMBER>

Note: It's a good idea to open an issue first for discussion.

## Checklist
- [ ] I have followed [Sample Guidelines from AUTHORING_GUIDE.MD](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md)
- [ ] README is updated to include [all relevant information](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#readme-file)
- [ ] **Tests** pass:   `nox -s py-3.6` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup))
- [ ] **Lint** pass:   `nox -s lint` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup))
- [ ] These samples need a new **API enabled** in testing projects to pass (let us know which ones)
- [ ] These samples need a new/updated **env vars** in testing projects set to pass (let us know which ones)
- [ ] Please **merge** this PR for me once it is approved.
- [ ] This sample adds a new sample directory, and I updated the [CODEOWNERS file](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/.github/CODEOWNERS) with the codeowners for this sample

* docs: add samples from python-docs-samples/tasks

* chore: update templates

Co-authored-by: Andrew Gorcester <andrew.gorcester@gmail.com>
Co-authored-by: Andrew Gorcester <gorcester@google.com>
Co-authored-by: DPE bot <dpebot@google.com>
Co-authored-by: michaelawyu <chenyumic@google.com>
Co-authored-by: ellenevans <35748459+ellenevans@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: michaelawyu <michael.a.w.yu@hotmail.com>
Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com>
Co-authored-by: Alex Voorhees <alex@cloudbakers.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Sarath Kaul <kaul.sarath@gmail.com>
Co-authored-by: Sarath Kaul <sarath.kaul@searce.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: Aaron Johnson <aaronmjohnson29@gmail.com>
Co-authored-by: Adam Ross <adamross@google.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: Joab Leite S. Neto <leitejoab@gmail.com>
  • Loading branch information
21 people committed Aug 31, 2020
1 parent a88e42b commit 9d022f7
Show file tree
Hide file tree
Showing 30 changed files with 1,709 additions and 27 deletions.
8 changes: 8 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
@@ -0,0 +1,8 @@
# Code owners file.
# This file controls who is tagged for review for any given pull request.
#
# For syntax help see:
# https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners#codeowners-syntax


/samples/**/*.py @averikitsch @googleapis/python-samples-owners
3 changes: 2 additions & 1 deletion .gitignore
Expand Up @@ -46,6 +46,7 @@ pip-log.txt
# Built documentation
docs/_build
bigquery/docs/generated
docs.metadata

# Virtual environment
env/
Expand All @@ -57,4 +58,4 @@ system_tests/local_test_setup

# Make sure a generated file isn't accidentally committed.
pylintrc
pylintrc.test
pylintrc.test
8 changes: 7 additions & 1 deletion .kokoro/build.sh
Expand Up @@ -36,4 +36,10 @@ python3.6 -m pip uninstall --yes --quiet nox-automation
python3.6 -m pip install --upgrade --quiet nox
python3.6 -m nox --version

python3.6 -m nox
# If NOX_SESSION is set, it only runs the specified session,
# otherwise run all the sessions.
if [[ -n "${NOX_SESSION:-}" ]]; then
python3.6 -m nox -s "${NOX_SESSION:-}"
else
python3.6 -m nox
fi
98 changes: 98 additions & 0 deletions .kokoro/docker/docs/Dockerfile
@@ -0,0 +1,98 @@
# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from ubuntu:20.04

ENV DEBIAN_FRONTEND noninteractive

# Ensure local Python is preferred over distribution Python.
ENV PATH /usr/local/bin:$PATH

# Install dependencies.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-transport-https \
build-essential \
ca-certificates \
curl \
dirmngr \
git \
gpg-agent \
graphviz \
libbz2-dev \
libdb5.3-dev \
libexpat1-dev \
libffi-dev \
liblzma-dev \
libreadline-dev \
libsnappy-dev \
libssl-dev \
libsqlite3-dev \
portaudio19-dev \
redis-server \
software-properties-common \
ssh \
sudo \
tcl \
tcl-dev \
tk \
tk-dev \
uuid-dev \
wget \
zlib1g-dev \
&& add-apt-repository universe \
&& apt-get update \
&& apt-get -y install jq \
&& apt-get clean autoclean \
&& apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/* \
&& rm -f /var/cache/apt/archives/*.deb


COPY fetch_gpg_keys.sh /tmp
# Install the desired versions of Python.
RUN set -ex \
&& export GNUPGHOME="$(mktemp -d)" \
&& echo "disable-ipv6" >> "${GNUPGHOME}/dirmngr.conf" \
&& /tmp/fetch_gpg_keys.sh \
&& for PYTHON_VERSION in 3.7.8 3.8.5; do \
wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \
&& wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \
&& gpg --batch --verify python-${PYTHON_VERSION}.tar.xz.asc python-${PYTHON_VERSION}.tar.xz \
&& rm -r python-${PYTHON_VERSION}.tar.xz.asc \
&& mkdir -p /usr/src/python-${PYTHON_VERSION} \
&& tar -xJC /usr/src/python-${PYTHON_VERSION} --strip-components=1 -f python-${PYTHON_VERSION}.tar.xz \
&& rm python-${PYTHON_VERSION}.tar.xz \
&& cd /usr/src/python-${PYTHON_VERSION} \
&& ./configure \
--enable-shared \
# This works only on Python 2.7 and throws a warning on every other
# version, but seems otherwise harmless.
--enable-unicode=ucs4 \
--with-system-ffi \
--without-ensurepip \
&& make -j$(nproc) \
&& make install \
&& ldconfig \
; done \
&& rm -rf "${GNUPGHOME}" \
&& rm -rf /usr/src/python* \
&& rm -rf ~/.cache/

RUN wget -O /tmp/get-pip.py 'https://bootstrap.pypa.io/get-pip.py' \
&& python3.7 /tmp/get-pip.py \
&& python3.8 /tmp/get-pip.py \
&& rm /tmp/get-pip.py

CMD ["python3.7"]
45 changes: 45 additions & 0 deletions .kokoro/docker/docs/fetch_gpg_keys.sh
@@ -0,0 +1,45 @@
#!/bin/bash
# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# A script to fetch gpg keys with retry.
# Avoid jinja parsing the file.
#

function retry {
if [[ "${#}" -le 1 ]]; then
echo "Usage: ${0} retry_count commands.."
exit 1
fi
local retries=${1}
local command="${@:2}"
until [[ "${retries}" -le 0 ]]; do
$command && return 0
if [[ $? -ne 0 ]]; then
echo "command failed, retrying"
((retries--))
fi
done
return 1
}

# 3.6.9, 3.7.5 (Ned Deily)
retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D

# 3.8.0 (Łukasz Langa)
retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
E3FF2839C048B25C084DEBE9B26995E310250568

#
21 changes: 19 additions & 2 deletions .kokoro/docs/common.cfg
Expand Up @@ -11,12 +11,12 @@ action {
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-tasks/.kokoro/trampoline.sh"
build_file: "python-tasks/.kokoro/trampoline_v2.sh"

# Configure the docker image for kokoro-trampoline.
env_vars: {
key: "TRAMPOLINE_IMAGE"
value: "gcr.io/cloud-devrel-kokoro-resources/python-multi"
value: "gcr.io/cloud-devrel-kokoro-resources/python-lib-docs"
}
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
Expand All @@ -28,6 +28,23 @@ env_vars: {
value: "docs-staging"
}

env_vars: {
key: "V2_STAGING_BUCKET"
value: "docs-staging-v2-staging"
}

# It will upload the docker image after successful builds.
env_vars: {
key: "TRAMPOLINE_IMAGE_UPLOAD"
value: "true"
}

# It will always build the docker image.
env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/docs/Dockerfile"
}

# Fetch the token needed for reporting release status to GitHub
before_action {
fetch_keystore {
Expand Down
17 changes: 17 additions & 0 deletions .kokoro/docs/docs-presubmit.cfg
@@ -0,0 +1,17 @@
# Format: //devtools/kokoro/config/proto/build.proto

env_vars: {
key: "STAGING_BUCKET"
value: "gcloud-python-test"
}

env_vars: {
key: "V2_STAGING_BUCKET"
value: "gcloud-python-test"
}

# We only upload the image in the main `docs` build.
env_vars: {
key: "TRAMPOLINE_IMAGE_UPLOAD"
value: "false"
}
39 changes: 24 additions & 15 deletions .kokoro/publish-docs.sh
Expand Up @@ -18,26 +18,16 @@ set -eo pipefail
# Disable buffering, so that the logs stream through.
export PYTHONUNBUFFERED=1

cd github/python-tasks

# Remove old nox
python3.6 -m pip uninstall --yes --quiet nox-automation
export PATH="${HOME}/.local/bin:${PATH}"

# Install nox
python3.6 -m pip install --upgrade --quiet nox
python3.6 -m nox --version
python3 -m pip install --user --upgrade --quiet nox
python3 -m nox --version

# build docs
nox -s docs

python3 -m pip install gcp-docuploader

# install a json parser
sudo apt-get update
sudo apt-get -y install software-properties-common
sudo add-apt-repository universe
sudo apt-get update
sudo apt-get -y install jq
python3 -m pip install --user gcp-docuploader

# create metadata
python3 -m docuploader create-metadata \
Expand All @@ -52,4 +42,23 @@ python3 -m docuploader create-metadata \
cat docs.metadata

# upload docs
python3 -m docuploader upload docs/_build/html --metadata-file docs.metadata --staging-bucket docs-staging
python3 -m docuploader upload docs/_build/html --metadata-file docs.metadata --staging-bucket "${STAGING_BUCKET}"


# docfx yaml files
nox -s docfx

# create metadata.
python3 -m docuploader create-metadata \
--name=$(jq --raw-output '.name // empty' .repo-metadata.json) \
--version=$(python3 setup.py --version) \
--language=$(jq --raw-output '.language // empty' .repo-metadata.json) \
--distribution-name=$(python3 setup.py --name) \
--product-page=$(jq --raw-output '.product_documentation // empty' .repo-metadata.json) \
--github-repository=$(jq --raw-output '.repo // empty' .repo-metadata.json) \
--issue-tracker=$(jq --raw-output '.issue_tracker // empty' .repo-metadata.json)

cat docs.metadata

# upload docs
python3 -m docuploader upload docs/_build/html/docfx_yaml --metadata-file docs.metadata --destination-prefix docfx --staging-bucket "${V2_STAGING_BUCKET}"

0 comments on commit 9d022f7

Please sign in to comment.