Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Cluster By in table definition #734

Open
pythonruss opened this issue Jan 20, 2023 · 3 comments
Open

Support Cluster By in table definition #734

pythonruss opened this issue Jan 20, 2023 · 3 comments
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@pythonruss
Copy link

There needs to be a dialect construct for adding a cluster by to a table definition

Here's what they did for the snowflake sqlalchemy package link

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label Jan 20, 2023
@pythonruss
Copy link
Author

BTW anyone who runs into this as a workaround:

from sqlalchemy_bigquery import dialect as bq_dialect
from sqlalchemy import Table
from sqlalchemy.schema import CreateTable
from sqlalchemy.ext.compiler import compiles

@compiles(CreateTable, "bigquery")
def compile_create_table(create, compiler, **kw):
    preparer = compiler.preparer
    stmt = compiler.visit_create_table(create, **kw)
    bigquery_opts = create.element.dialect_options["bigquery"]
    cluster_by = bigquery_opts.get("clusterby")
    if cluster_by:
        columns = ", ".join([
            compiler.sql_compiler.process(col, include_table=False, **kw)
            for col in cluster_by
        ])
        stmt = f"{stmt} CLUSTER BY {columns}\n\n"
    return stmt


    dialect = bq_dialect
    dialect.construct_arguments = [
        (Table, {"clusterby": None } )
        ]

@meredithslota meredithslota added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Mar 2, 2023
@jlynchMicron
Copy link

Possibly duplicate of: #395

@leblancfg
Copy link

To anyone coming here trying to extend the dialect to support partitioning and/or clustering: it looks like @pythonruss' code doesn't work as a workaround and made other parts of my existing code break. Would make a good forked branch or PR, but doesn't extend the dialect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

4 participants