Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve __call__ performance benchmarking #616

Open
daniel-sanche opened this issue Feb 15, 2024 · 2 comments
Open

improve __call__ performance benchmarking #616

daniel-sanche opened this issue Feb 15, 2024 · 2 comments
Labels
type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@daniel-sanche
Copy link
Contributor

daniel-sanche commented Feb 15, 2024

In #527, we improved the performance of _GapicCallable's call method, and added a simple benchmark unit test to prevent performance from regressing too much

The test was very simple, using timeit to measure the time to run 10,000 calls, and dailing if it was slower than expected. This doesn't take performance of the worker into account, so the test may flake in different environments. To prevent this, we left a large buffer, but this means smaller regressions won't be caught by the test

@parthea suggested opening this issue to investigate improving the performance tests in the future. If we could test the performance of the old revision and the suggested change as part of the same test, we could make much better assertions on the performance of the library

CC @ohmayr

@daniel-sanche
Copy link
Contributor Author

Related: #542 (comment)

@parthea parthea added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Mar 28, 2024
@vchudnov-g
Copy link
Contributor

See also my suggestion for capturing tighter limits per platform: have a dict of platform to expected performance threshold, and have the failing assertion print out both the time and the current plaatform so we can add that pair to the table for future runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants