-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: add missing latency check #763
Changes from 7 commits
3555a5b
b1bd9fc
c48349b
256d9f6
f7cc202
ec73fc8
36dd4df
3a5b149
9d129bd
b2c1353
6043722
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -411,7 +411,20 @@ def run_query(self, query, max_results=None, progress_bar_type=None, **kwargs): | |
timeout_ms = job_config_dict.get("jobTimeoutMs") or job_config_dict[ | ||
"query" | ||
].get("timeoutMs") | ||
timeout_ms = int(timeout_ms) if timeout_ms else None | ||
|
||
if timeout_ms: | ||
timeout_ms = int(timeout_ms) | ||
# Having too small a timeout_ms results in individual | ||
# API calls timing out before they can finish. | ||
# ~300 milliseconds is rule of thumb for bare minimum | ||
# latency from the BigQuery API. | ||
minimum_latency = 400 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe we can define There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would like to leave this as is. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you explain why? Thank you. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see this particular variable being used widely throughout the code (now or in the future). I appreciate having the variable and the comment about the variable close together and in this case close to where they are being used. I feel it improves the readability/maintainability of the code. Especially since this edit is in response to a flakybot failing test. If it continues to fail, then whoever troubleshoots this is likely to see the comment in situ and may then tweak the duration to better match real world conditions. |
||
if timeout_ms < minimum_latency: | ||
raise QueryTimeout( | ||
f"Query timeout must be at least 400 milliseconds: timeout_ms equals {timeout_ms}." | ||
) | ||
else: | ||
timeout_ms = None | ||
|
||
self._start_timer() | ||
job_config = bigquery.QueryJobConfig.from_api_repr(job_config_dict) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: remove the added empty line?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case, if it passes linting and black... I am fine with it being as it is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure about this - if there isn't a good reason to change it, I don't think we should.