Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dataset.list_tables max_results parameter #173

Closed
OmriBromberg opened this issue May 12, 2021 · 2 comments · Fixed by #174
Closed

dataset.list_tables max_results parameter #173

OmriBromberg opened this issue May 12, 2021 · 2 comments · Fixed by #174
Assignees
@jimfulton
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. status: blocked Resolving the issue is dependent on other work. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@OmriBromberg
Copy link
Contributor

I'm using great_expectations to test my data in BigQuery and the library introspects the entire database/schema for tables, meaning they use the function sa.inspect(engine).get_table_names(), because I have alot of tables this process is very very slow, I want to change the max_results parameter passed on to dataset.list_tables operation, when tested it increased performance dramatically
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label May 12, 2021
@meredithslota meredithslota added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label May 13, 2021
@jimfulton jimfulton mentioned this issue May 16, 2021
Merged
@jimfulton jimfulton self-assigned this May 17, 2021
@jimfulton jimfulton mentioned this issue May 24, 2021
Closed
@jimfulton jimfulton added the status: blocked Resolving the issue is dependent on other work. label May 25, 2021
@jimfulton
Copy link
Contributor

max_results doesn't really do what we want. We really want page_size.

See googleapis/python-api-core#194

@jimfulton
Copy link
Contributor

api_core has been updated. There's a bigquery PR: googleapis/python-bigquery#686

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jimfulton jimfulton

Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. status: blocked Resolving the issue is dependent on other work. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Feature/list tables page size OmriBromberg/python-bigquery-sqlalchemy
3 participants
@jimfulton @OmriBromberg @meredithslota