Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: add missing latency check #763

Merged
merged 11 commits into from May 8, 2024
Merged

bug: add missing latency check #763

merged 11 commits into from May 8, 2024

Conversation

chalmerlowe
Copy link
Collaborator

@chalmerlowe chalmerlowe commented Apr 23, 2024

There is a failing flakybot test. This PR provides a possible mitigation.
During a recent PR, we appear to have introduced a situation that occurs when we attempt to perform certain actions without recognizing that there will be some latency in the system. This adds checks to account for that latency.

Fixes #762

@chalmerlowe chalmerlowe requested review from a team as code owners April 23, 2024 16:43
@chalmerlowe chalmerlowe self-assigned this Apr 23, 2024
Copy link

conventional-commit-lint-gcf bot commented Apr 23, 2024

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot
https://conventionalcommits.org/

@product-auto-label product-auto-label bot added size: xs Pull request size is extra small. api: bigquery Issues related to the googleapis/python-bigquery-pandas API. labels Apr 23, 2024
@chalmerlowe chalmerlowe added do not merge Indicates a pull request not ready for merge, due to either quality or timing. and removed api: bigquery Issues related to the googleapis/python-bigquery-pandas API. labels Apr 23, 2024
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-pandas API. label Apr 24, 2024
@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels May 3, 2024
Copy link
Collaborator Author

@chalmerlowe chalmerlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed two octothorpes.

noxfile.py Outdated Show resolved Hide resolved
noxfile.py Outdated Show resolved Hide resolved
@chalmerlowe chalmerlowe removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 6, 2024
setup.py Outdated Show resolved Hide resolved
@chalmerlowe chalmerlowe changed the title bug: specify a particular version of bigquery to debug bug: add missing latency check May 6, 2024
noxfile.py Outdated
@@ -347,17 +348,21 @@ def prerelease(session):
session.run("python", "-m", "pip", "freeze")

# Run all tests, except a few samples tests which require extra dependencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove the added empty line?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, if it passes linting and black... I am fine with it being as it is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this - if there isn't a good reason to change it, I don't think we should.

# API calls timing out before they can finish.
# ~300 milliseconds is rule of thumb for bare minimum
# latency from the BigQuery API.
minimum_latency = 400
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can define minimum_latency as a constant at top of the file, for easy access and reuse in the future (see example).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to leave this as is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain why? Thank you.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Linchin

I don't see this particular variable being used widely throughout the code (now or in the future).

I appreciate having the variable and the comment about the variable close together and in this case close to where they are being used. I feel it improves the readability/maintainability of the code. Especially since this edit is in response to a flakybot failing test. If it continues to fail, then whoever troubleshoots this is likely to see the comment in situ and may then tweak the duration to better match real world conditions.

noxfile.py Show resolved Hide resolved
@Linchin Linchin self-requested a review May 7, 2024 17:03
noxfile.py Outdated Show resolved Hide resolved
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels May 7, 2024
@chalmerlowe chalmerlowe merged commit 6bfd1ee into main May 8, 2024
26 checks passed
@chalmerlowe chalmerlowe deleted the flaky-test-debug branch May 8, 2024 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tests.system.test_gbq.TestReadGBQIntegration: test_timeout_configuration failed
2 participants