Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: avoid adding dummy WHERE clause into UPDATE and DELETE queires #516

Merged
merged 12 commits into from Nov 18, 2020

Conversation

IlyaFaer
Copy link
Contributor

No description provided.

@IlyaFaer IlyaFaer added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. do not merge Indicates a pull request not ready for merge, due to either quality or timing. api: spanner Issues related to the googleapis/python-spanner-django API. labels Sep 21, 2020
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 21, 2020
@IlyaFaer
Copy link
Contributor Author

@c24t, looks like it's our way to raise exceptions on lack of WHERE statement, but not to break all the Django tests. What do you think?

@IlyaFaer IlyaFaer removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Sep 21, 2020
@IlyaFaer IlyaFaer marked this pull request as ready for review September 21, 2020 11:29
@IlyaFaer IlyaFaer requested a review from c24t September 21, 2020 11:29
@mf2199 mf2199 requested a review from a team as a code owner September 29, 2020 02:29
@mf2199 mf2199 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 29, 2020
Comment on lines 506 to 511
"ALL",
"AND",
"ANY",
"ARRAY",
"AS",
"ASC",
"ASSERT_ROWS_MODIFIED",
"AT",
"BETWEEN",
"BY",
"CASE",
"CAST",
"COLLATE",
"CONTAINS",
"CREATE",
"CROSS",
"CUBE",
"CURRENT",
"DEFAULT",
"DEFINE",
"DESC",
"DISTINCT",
"DROP",
"ELSE",
"END",
"ENUM",
"ESCAPE",
"EXCEPT",
"EXCLUDE",
"EXISTS",
"EXTRACT",
"FALSE",
"FETCH",
"FOLLOWING",
"FOR",
"FROM",
"FULL",
"GROUP",
"GROUPING",
"GROUPS",
"HASH",
"HAVING",
"IF",
"IGNORE",
"IN",
"INNER",
"INTERSECT",
"INTERVAL",
"INTO",
"IS",
"JOIN",
"LATERAL",
"LEFT",
"LIKE",
"LIMIT",
"LOOKUP",
"MERGE",
"NATURAL",
"NEW",
"NO",
"NOT",
"NULL",
"NULLS",
"OF",
"ON",
"OR",
"ORDER",
"OUTER",
"OVER",
"PARTITION",
"PRECEDING",
"PROTO",
"RANGE",
"RECURSIVE",
"RESPECT",
"RIGHT",
"ROLLUP",
"ROWS",
"SELECT",
"SET",
"SOME",
"STRUCT",
"TABLESAMPLE",
"THEN",
"TO",
"TREAT",
"TRUE",
"UNBOUNDED",
"UNION",
"UNNEST",
"USING",
"WHEN",
"WHERE",
"WINDOW",
"WITH",
"WITHIN",
}
raise ProgrammingError(
"Cloud Spanner requires a WHERE clause when executing DELETE or UPDATE query"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies if I misunderstood something here.

I believe this clause is added by default because Django will construct DELETE and UPDATE queries that don't have the WHERE clause as this is not a requirement in other databases. It ensure that customers can smoothly transition from their existing applications without having to write raw queries with the WHERE clause.

The clause wasn't added just to get the tests to pass.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skuruppu, hiding this useful restriction doesn't look right. But I see the point in smooth transition as well. Do you mean that only for Django? Or we should consider that any user can have queries without WHERE, so this dummy should be kept for the whole connection API? (we have spanner_dbapi directory, which seems to be intended to be default connection API - it's from where I'm proposing to erase the dummy, and django_spanner directory, where Django-specific parts are held).

The main thing that makes me nervous is that queries can be big and branched (for example, UNIONS are little bit hard to interpret, as WHERE clause can be relative to the last of the union parts, but also can be relative to the result of the UNION on the whole, and they both will be at the about same place). I'm not sure if there is a way to check 100% of variants with several regexes, and insert this dummy correctly. So, I think it'll be exploding things from time to time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it may be good to notify the user about this restriction from the connection API. Is it possible to instead modify the Django implementation to add the WHERE clause in the generated query? I think this is where it matters the most in order to support users migrating from other databases.

@mf2199 mf2199 added kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Sep 29, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 29, 2020
@c24t
Copy link
Contributor

c24t commented Oct 22, 2020

Talking about this PR in person: following @skuruppu's comment, these changes should move to django-specific code.

@IlyaFaer
Copy link
Contributor Author

IlyaFaer commented Nov 10, 2020

@c24t, I've pushed the changes, it seems to be ready for merge now.

Copy link
Contributor

@c24t c24t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but please rename one of those ensure_where_clause funcs.

Note that this will have to be split up across python-spanner and spanner-django now because of googleapis/python-spanner#160.

django_spanner/utils.py Show resolved Hide resolved
tests/unit/spanner_dbapi/test_parse_utils.py Show resolved Hide resolved
django_spanner/utils.py Outdated Show resolved Hide resolved
google/cloud/spanner_dbapi/parse_utils.py Outdated Show resolved Hide resolved
django_spanner/utils.py Show resolved Hide resolved
google/cloud/spanner_dbapi/parse_utils.py Outdated Show resolved Hide resolved
@c24t
Copy link
Contributor

c24t commented Nov 18, 2020

@IlyaFaer just pushed some minor fixes for my own comments, if it LGTY we can merge.

@IlyaFaer
Copy link
Contributor Author

@c24t, yeah, I'm okay with this changes

@c24t
Copy link
Contributor

c24t commented Nov 18, 2020

Note that we need to rip out spanner_dbapi to avoid having to keep making these changes in two places.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner-django API. cla: yes This human has signed the Contributor License Agreement. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants