Skip to content

Commit

Permalink
fix: populate default timeout and retry after client-side timeout (#…
Browse files Browse the repository at this point in the history
…896)

This addresses internal issue 195337762 where sometimes query job creation can
take longer than expected and retrying the API call can be faster than waiting
for the first query job request to fail.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes #889
Towards #779

 🦕
  • Loading branch information
tswast committed Aug 25, 2021
1 parent 1cb3e55 commit b508809
Show file tree
Hide file tree
Showing 14 changed files with 282 additions and 219 deletions.
123 changes: 62 additions & 61 deletions google/cloud/bigquery/client.py

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions google/cloud/bigquery/retry.py
Expand Up @@ -29,6 +29,7 @@
exceptions.BadGateway,
requests.exceptions.ChunkedEncodingError,
requests.exceptions.ConnectionError,
requests.exceptions.Timeout,
auth_exceptions.TransportError,
)

Expand Down Expand Up @@ -59,6 +60,13 @@ def _should_retry(exc):
pass ``retry=bigquery.DEFAULT_RETRY.with_deadline(30)``.
"""

DEFAULT_TIMEOUT = 5.0 * 60.0
"""The default API timeout.
This is the time to wait per request. To adjust the total wait time, set a
deadline on the retry object.
"""

job_retry_reasons = "rateLimitExceeded", "backendError"


Expand Down
4 changes: 0 additions & 4 deletions noxfile.py
Expand Up @@ -160,10 +160,6 @@ def snippets(session):
if os.environ.get("RUN_SNIPPETS_TESTS", "true") == "false":
session.skip("RUN_SNIPPETS_TESTS is set to false, skipping")

# Sanity check: Only run snippets tests if the environment variable is set.
if not os.environ.get("GOOGLE_APPLICATION_CREDENTIALS", ""):
session.skip("Credentials must be set via environment variable.")

constraints_path = str(
CURRENT_DIRECTORY / "testing" / f"constraints-{session.python}.txt"
)
Expand Down

0 comments on commit b508809

Please sign in to comment.