Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing support for UPDATE + JOIN in BigQuery dialect. #1010

Open
brian-pond opened this issue Jan 18, 2024 · 0 comments
Open

Missing support for UPDATE + JOIN in BigQuery dialect. #1010

brian-pond opened this issue Jan 18, 2024 · 0 comments
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@brian-pond
Copy link

brian-pond commented Jan 18, 2024

Environment

  • Python 3.11 on a Debian 11 host.
  • sqlalchemy version 1.4.50
  • sqlalchemy-bigquery version 1.9

Steps to reproduce

Assume a BigQuery dataset has two tables: tbl_foo and tbl_bar.
You want to UPDATE values in tbl_foo, using a JOIN against tbl_bar.

update_stmt = (
    update(tbl_foo)
    .where(tbl_foo.c.id == tbl_bar.c.foo_id)
    .where(tbl_bar.c.my_group == "some group")
    .values(somefield="new-value")

engine = create_engine('bigquery://', credentials_path='/path/to/keyfile.json')
with engine.connect() as conn:
    result = conn.execute(update_stmt)
)

The compiled SQL should look something like this:

UPDATE tbl_foo
SET somefield = "new-value"
FROM tbl_bar
WHERE tbl_foo.id = tbl_bar.foo_id
AND tbl_bar.my_group = "some_group"

BigQuery does indeed support this (^) SQL syntax. I successfully ran this statement in a browser with Google Cloud and BigQuery Explorer.

However, when the Python shown above is executed, this error is thrown:

NotImplementedError: This backend does not support multiple-table criteria within UPDATE

I did some investigation. The root cause is that base SQLCompiler class has a function named update_from_clause that looks like this (GitHub).

Built-in Alchemy dialects (like PostgreSQL) overrride update_from_clause function, as shown in code here (GitHub).

Issue : The BigQuery dialect offered by sqlalchemy-bigquery does not override this function.


NOTE: I tried to override update_from_clause in my development environment, adding it to ../sqlalchemy_bigquery/base.py.
But this did not succeed. At runtime, Python didn't recognize my change, and continued to throw the NotImplementedError error from the original SQLCompiler class. 😕

To be fair, I don't exactly understand how sqlalchemy-bigquery is patching and overriding the vanilla sqlalchemy code (some kind of guerilla patching during import?). I tried deleting all my bytecode and __pycache__, but that didn't seem to help.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label Jan 18, 2024
@Linchin Linchin added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Apr 30, 2024
@Linchin Linchin self-assigned this Apr 30, 2024
@Linchin Linchin added the priority: p2 Moderately-important priority. Fix may not be included in next release. label Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants