test: require 100% unit test coverage (via synth) (#359)

* changes without context

        autosynth cannot find the source of changes triggered by earlier changes in this
        repository, or by version upgrades to tools such as linters.

* feat: add RPC Priority request options

PiperOrigin-RevId: 364449524

Source-Author: Google APIs <noreply@google.com>
Source-Date: Mon Mar 22 17:39:37 2021 -0700
Source-Repo: googleapis/googleapis
Source-Sha: 6598bb829c9e9a534be674649ffd1b4671a821f9
Source-Link: googleapis/googleapis@6598bb8

* chore(deps): update precommit hook pycqa/flake8 to v3.9.0

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [pycqa/flake8](https://gitlab.com/pycqa/flake8) | repository | minor | `3.8.4` -> `3.9.0` |

---

### Release Notes

<details>
<summary>pycqa/flake8</summary>

### [`v3.9.0`](https://gitlab.com/pycqa/flake8/compare/3.8.4...3.9.0)

[Compare Source](https://gitlab.com/pycqa/flake8/compare/3.8.4...3.9.0)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/synthtool).

Source-Author: WhiteSource Renovate <bot@renovateapp.com>
Source-Date: Tue Mar 23 17:38:03 2021 +0100
Source-Repo: googleapis/synthtool
Source-Sha: f5c5904fb0c6aa3b3730eadf4e5a4485afc65726
Source-Link: googleapis/synthtool@f5c5904

* test(python): use constraints files to check dependency lower bounds

Use a constraints file when installing dependencies for system and unit tests nox sessions.

https://pip.pypa.io/en/stable/user_guide/#constraints-files
> Constraints files are requirements files that **only control which version of a requirement is installed, not whether it is installed or not**. Their syntax and contents is nearly identical to Requirements Files. There is one key difference: Including a package in a constraints file does not trigger installation of the package.

```
testing
├── constraints-3.10.txt
├── constraints-3.11.txt
├── constraints-3.6.txt
├── constraints-3.7.txt
├── constraints-3.8.txt
└── constraints-3.9.txt
```

Going forward, one constraints file (currently 3.6) will be populated with every library requirement and extra listed in the `setup.py`. The constraints file will pin each requirement to the lower bound. This ensures that library maintainers will see test failures if they forget to update a lower bound on a dependency.

See googleapis/python-bigquery#263 for an example

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Tue Mar 23 10:52:02 2021 -0600
Source-Repo: googleapis/synthtool
Source-Sha: 86ed43d4f56e6404d068e62e497029018879c771
Source-Link: googleapis/synthtool@86ed43d

* build(python): update docfx job to use new plugin

Source-Author: Dan Lee <71398022+dandhlee@users.noreply.github.com>
Source-Date: Tue Mar 30 19:36:37 2021 -0400
Source-Repo: googleapis/synthtool
Source-Sha: 4501974ad08b5d693311457e2ea4ce845676e329
Source-Link: googleapis/synthtool@4501974

* chore: Add license headers for python config files

Source-Author: Anthonios Partheniou <partheniou@google.com>
Source-Date: Tue Apr 6 11:32:03 2021 -0400
Source-Repo: googleapis/synthtool
Source-Sha: 5b5bf6d519b2d658d9f2e483d9f6f3d0ba8ee6bc
Source-Link: googleapis/synthtool@5b5bf6d

* chore: add constraints file check for python samples

This is the sibling PR to GoogleCloudPlatform/python-docs-samples#5611
and this is the issue opened for it GoogleCloudPlatform/python-docs-samples#5549

If you look at the files in [this example repo](https://github.com/leahecole/testrepo-githubapp/pull/31/files), you'll see that renovate successfully opened a PR on three constraints files in `samples` directories and subdirectories, and properly ignored `constraints` files at the root level

cc @tswast

TODO:

- [x]   update renovate to check for samples/constraints.txt dependency updates
- [x]  run lint locally to double check that I'm not introducing lint error

Source-Author: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Source-Date: Fri Apr 9 22:50:04 2021 -0700
Source-Repo: googleapis/synthtool
Source-Sha: 0a071b3460344886297a304253bf924aa68ddb7e
Source-Link: googleapis/synthtool@0a071b3

.github/header-checker-lint.yml

@@ -1,6 +1,6 @@
 {"allowedCopyrightHolders": ["Google LLC"],
  "allowedLicenses": ["Apache-2.0", "MIT", "BSD-3"],
- "ignoreFiles": ["**/requirements.txt", "**/requirements-test.txt"],
+ "ignoreFiles": ["**/requirements.txt", "**/requirements-test.txt", "**/__init__.py", "samples/**/constraints.txt", "samples/**/constraints-test.txt"],
  "sourceFileExtensions": [
  	"ts", 
  	"js", 

.pre-commit-config.yaml

@@ -1,3 +1,17 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
 # See https://pre-commit.com for more information
 # See https://pre-commit.com/hooks.html for more hooks
 repos:

docs/conf.py

@@ -1,4 +1,17 @@
 # -*- coding: utf-8 -*-
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 #
 # google-cloud-pubsub documentation build configuration file
 #

noxfile.py

@@ -18,6 +18,7 @@
 
 from __future__ import absolute_import
 import os
+import pathlib
 import shutil
 
 import nox
@@ -30,6 +31,8 @@
 SYSTEM_TEST_PYTHON_VERSIONS = ["3.8"]
 UNIT_TEST_PYTHON_VERSIONS = ["3.6", "3.7", "3.8", "3.9"]
 
+CURRENT_DIRECTORY = pathlib.Path(__file__).parent.absolute()
+
 # 'docfx' is excluded since it only needs to run in 'docs-presubmit'
 nox.options.sessions = [
     "unit",
@@ -84,13 +87,15 @@ def lint_setup_py(session):
 
 def default(session):
     # Install all test dependencies, then install this package in-place.
-    session.install("asyncmock", "pytest-asyncio")
 
-    session.install(
-        "mock", "pytest", "pytest-cov",
+    constraints_path = str(
+        CURRENT_DIRECTORY / "testing" / f"constraints-{session.python}.txt"
     )
+    session.install("asyncmock", "pytest-asyncio", "-c", constraints_path)
 
-    session.install("-e", ".")
+    session.install("mock", "pytest", "pytest-cov", "-c", constraints_path)
+
+    session.install("-e", ".", "-c", constraints_path)
 
     # Run py.test against the unit tests.
     session.run(
@@ -117,6 +122,9 @@ def unit(session):
 @nox.session(python=SYSTEM_TEST_PYTHON_VERSIONS)
 def system(session):
     """Run the system test suite."""
+    constraints_path = str(
+        CURRENT_DIRECTORY / "testing" / f"constraints-{session.python}.txt"
+    )
     system_test_path = os.path.join("tests", "system.py")
     system_test_folder_path = os.path.join("tests", "system")
 
@@ -141,8 +149,10 @@ def system(session):
 
     # Install all test dependencies, then install this package into the
     # virtualenv's dist-packages.
-    session.install("mock", "pytest", "google-cloud-testutils", "psutil")
-    session.install("-e", ".")
+    session.install(
+        "mock", "pytest", "google-cloud-testutils", "psutil", "-c", constraints_path
+    )
+    session.install("-e", ".", "-c", constraints_path)
 
     # Run py.test against the system tests.
     if system_test_exists:
@@ -171,7 +181,7 @@ def cover(session):
     test runs (not system test runs), and then erases coverage data.
     """
     session.install("coverage", "pytest-cov")
-    session.run("coverage", "report", "--show-missing", "--fail-under=99")
+    session.run("coverage", "report", "--show-missing", "--fail-under=100")
 
     session.run("coverage", "erase")
 
@@ -203,9 +213,7 @@ def docfx(session):
     """Build the docfx yaml files for this library."""
 
     session.install("-e", ".")
-    # sphinx-docfx-yaml supports up to sphinx version 1.5.5.
-    # https://github.com/docascode/sphinx-docfx-yaml/issues/97
-    session.install("sphinx==1.5.5", "alabaster", "recommonmark", "sphinx-docfx-yaml")
+    session.install("sphinx", "alabaster", "recommonmark", "gcp-sphinx-docfx-yaml")
 
     shutil.rmtree(os.path.join("docs", "_build"), ignore_errors=True)
     session.run(

renovate.json

@@ -2,5 +2,8 @@
   "extends": [
     "config:base",  ":preserveSemverRanges"
   ],
-  "ignorePaths": [".pre-commit-config.yaml"]
+  "ignorePaths": [".pre-commit-config.yaml"],
+  "pip_requirements": {
+    "fileMatch": ["requirements-test.txt", "samples/[\\S/]*constraints.txt", "samples/[\\S/]*constraints-test.txt"]
+  }
 }

samples/snippets/noxfile.py

@@ -172,10 +172,16 @@ def blacken(session: nox.sessions.Session) -> None:
 def _session_tests(session: nox.sessions.Session, post_install: Callable = None) -> None:
     """Runs py.test for a particular project."""
     if os.path.exists("requirements.txt"):
-        session.install("-r", "requirements.txt")
+        if os.path.exists("constraints.txt"):
+            session.install("-r", "requirements.txt", "-c", "constraints.txt")
+        else:
+            session.install("-r", "requirements.txt")
 
     if os.path.exists("requirements-test.txt"):
-        session.install("-r", "requirements-test.txt")
+        if os.path.exists("constraints-test.txt"):
+            session.install("-r", "requirements-test.txt", "-c", "constraints-test.txt")
+        else:
+            session.install("-r", "requirements-test.txt")
 
     if INSTALL_LIBRARY_FROM_SOURCE:
         session.install("-e", _get_repo_root())

synth.metadata

@@ -3,8 +3,8 @@
     {
       "git": {
         "name": ".",
-        "remote": "git@github.com:plamut/python-pubsub.git",
-        "sha": "0a662a6daad0517fb0e01732c9dd7f9d1852924c"
+        "remote": "https://github.com/googleapis/python-pubsub.git",
+        "sha": "9f451a1fe0ac5cb2fb13d72b0436e0b521a4fecb"
       }
     },
     {
@@ -19,21 +19,21 @@
       "git": {
         "name": "synthtool",
         "remote": "https://github.com/googleapis/synthtool.git",
-        "sha": "c84c8f156e09702e1c8946bfb9746e6f5892cf27"
+        "sha": "0a071b3460344886297a304253bf924aa68ddb7e"
       }
     },
     {
       "git": {
         "name": "synthtool",
         "remote": "https://github.com/googleapis/synthtool.git",
-        "sha": "c84c8f156e09702e1c8946bfb9746e6f5892cf27"
+        "sha": "0a071b3460344886297a304253bf924aa68ddb7e"
       }
     },
     {
       "git": {
         "name": "synthtool",
         "remote": "https://github.com/googleapis/synthtool.git",
-        "sha": "c84c8f156e09702e1c8946bfb9746e6f5892cf27"
+        "sha": "0a071b3460344886297a304253bf924aa68ddb7e"
       }
     }
   ],
@@ -47,5 +47,121 @@
         "generator": "bazel"
       }
     }
+  ],
+  "generatedFiles": [
+    ".coveragerc",
+    ".flake8",
+    ".github/CONTRIBUTING.md",
+    ".github/ISSUE_TEMPLATE/bug_report.md",
+    ".github/ISSUE_TEMPLATE/feature_request.md",
+    ".github/ISSUE_TEMPLATE/support_request.md",
+    ".github/PULL_REQUEST_TEMPLATE.md",
+    ".github/header-checker-lint.yml",
+    ".github/release-please.yml",
+    ".github/snippet-bot.yml",
+    ".gitignore",
+    ".kokoro/build.sh",
+    ".kokoro/continuous/common.cfg",
+    ".kokoro/continuous/continuous.cfg",
+    ".kokoro/docker/docs/Dockerfile",
+    ".kokoro/docker/docs/fetch_gpg_keys.sh",
+    ".kokoro/docs/common.cfg",
+    ".kokoro/docs/docs-presubmit.cfg",
+    ".kokoro/docs/docs.cfg",
+    ".kokoro/populate-secrets.sh",
+    ".kokoro/presubmit/common.cfg",
+    ".kokoro/presubmit/presubmit.cfg",
+    ".kokoro/publish-docs.sh",
+    ".kokoro/release.sh",
+    ".kokoro/release/common.cfg",
+    ".kokoro/release/release.cfg",
+    ".kokoro/samples/lint/common.cfg",
+    ".kokoro/samples/lint/continuous.cfg",
+    ".kokoro/samples/lint/periodic.cfg",
+    ".kokoro/samples/lint/presubmit.cfg",
+    ".kokoro/samples/python3.6/common.cfg",
+    ".kokoro/samples/python3.6/continuous.cfg",
+    ".kokoro/samples/python3.6/periodic-head.cfg",
+    ".kokoro/samples/python3.6/periodic.cfg",
+    ".kokoro/samples/python3.6/presubmit.cfg",
+    ".kokoro/samples/python3.7/common.cfg",
+    ".kokoro/samples/python3.7/continuous.cfg",
+    ".kokoro/samples/python3.7/periodic-head.cfg",
+    ".kokoro/samples/python3.7/periodic.cfg",
+    ".kokoro/samples/python3.7/presubmit.cfg",
+    ".kokoro/samples/python3.8/common.cfg",
+    ".kokoro/samples/python3.8/continuous.cfg",
+    ".kokoro/samples/python3.8/periodic-head.cfg",
+    ".kokoro/samples/python3.8/periodic.cfg",
+    ".kokoro/samples/python3.8/presubmit.cfg",
+    ".kokoro/test-samples-against-head.sh",
+    ".kokoro/test-samples-impl.sh",
+    ".kokoro/test-samples.sh",
+    ".kokoro/trampoline.sh",
+    ".kokoro/trampoline_v2.sh",
+    ".pre-commit-config.yaml",
+    ".trampolinerc",
+    "CODE_OF_CONDUCT.md",
+    "CONTRIBUTING.rst",
+    "LICENSE",
+    "MANIFEST.in",
+    "docs/_static/custom.css",
+    "docs/_templates/layout.html",
+    "docs/conf.py",
+    "docs/multiprocessing.rst",
+    "google/cloud/pubsub_v1/proto/pubsub.proto",
+    "google/cloud/pubsub_v1/proto/schema.proto",
+    "google/pubsub/__init__.py",
+    "google/pubsub/py.typed",
+    "google/pubsub_v1/__init__.py",
+    "google/pubsub_v1/py.typed",
+    "google/pubsub_v1/services/__init__.py",
+    "google/pubsub_v1/services/publisher/__init__.py",
+    "google/pubsub_v1/services/publisher/async_client.py",
+    "google/pubsub_v1/services/publisher/client.py",
+    "google/pubsub_v1/services/publisher/pagers.py",
+    "google/pubsub_v1/services/publisher/transports/__init__.py",
+    "google/pubsub_v1/services/publisher/transports/base.py",
+    "google/pubsub_v1/services/publisher/transports/grpc.py",
+    "google/pubsub_v1/services/publisher/transports/grpc_asyncio.py",
+    "google/pubsub_v1/services/schema_service/__init__.py",
+    "google/pubsub_v1/services/schema_service/async_client.py",
+    "google/pubsub_v1/services/schema_service/client.py",
+    "google/pubsub_v1/services/schema_service/pagers.py",
+    "google/pubsub_v1/services/schema_service/transports/__init__.py",
+    "google/pubsub_v1/services/schema_service/transports/base.py",
+    "google/pubsub_v1/services/schema_service/transports/grpc.py",
+    "google/pubsub_v1/services/schema_service/transports/grpc_asyncio.py",
+    "google/pubsub_v1/services/subscriber/__init__.py",
+    "google/pubsub_v1/services/subscriber/async_client.py",
+    "google/pubsub_v1/services/subscriber/client.py",
+    "google/pubsub_v1/services/subscriber/pagers.py",
+    "google/pubsub_v1/services/subscriber/transports/__init__.py",
+    "google/pubsub_v1/services/subscriber/transports/base.py",
+    "google/pubsub_v1/services/subscriber/transports/grpc.py",
+    "google/pubsub_v1/services/subscriber/transports/grpc_asyncio.py",
+    "google/pubsub_v1/types/__init__.py",
+    "google/pubsub_v1/types/pubsub.py",
+    "google/pubsub_v1/types/schema.py",
+    "mypy.ini",
+    "noxfile.py",
+    "renovate.json",
+    "samples/AUTHORING_GUIDE.md",
+    "samples/CONTRIBUTING.md",
+    "samples/snippets/noxfile.py",
+    "scripts/decrypt-secrets.sh",
+    "scripts/fixup_pubsub_v1_keywords.py",
+    "scripts/readme-gen/readme_gen.py",
+    "scripts/readme-gen/templates/README.tmpl.rst",
+    "scripts/readme-gen/templates/auth.tmpl.rst",
+    "scripts/readme-gen/templates/auth_api_key.tmpl.rst",
+    "scripts/readme-gen/templates/install_deps.tmpl.rst",
+    "scripts/readme-gen/templates/install_portaudio.tmpl.rst",
+    "setup.cfg",
+    "testing/.gitignore",
+    "tests/unit/gapic/pubsub_v1/__init__.py",
+    "tests/unit/gapic/pubsub_v1/test_publisher.py",
+    "tests/unit/gapic/pubsub_v1/test_schema_service.py",
+    "tests/unit/gapic/pubsub_v1/test_subscriber.py"
   ]
 }