Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't prepare statements unless explicitly specified #149

Open
mewa opened this issue Dec 29, 2021 · 3 comments
Open

Don't prepare statements unless explicitly specified #149

mewa opened this issue Dec 29, 2021 · 3 comments

Comments

@mewa
Copy link

mewa commented Dec 29, 2021

This is linked to #148 but since it explores a different/wider aspect of this issue, I opted for creating another issue.

The root cause in #148 is that statements are prepared, which combined with a large number of values results in too large header sizes.

As I explored it, it got me thinking about whether we should be preparing each and every query before executing it. Especially since currently it seems everything is happening implicitly, without the knowledge of the user. Any thoughts?

@hashhar
Copy link
Member

hashhar commented Dec 29, 2021

If I am reading the code correctly the prepared statements path is only used when params are provided to execute. If not plain SQL is sent as it is.

Are you observing different behaviour?

@mewa
Copy link
Author

mewa commented Dec 29, 2021

I had to dig into the stack traces since we're not calling execute directly, but yes, the params are provided.

For context, we're using:

  1. pandas - which in turn calls sqlalchemy
  2. sqlalchemy - which relies on sqlalchemy-trino dialect to communicate with Trino
  3. finally, sqlalchemy-trino then calls the Trino client

I think the issue here is that the client is doing more than it's asked for. In my opinion, deciding which statement should be prepared (or not) is the end-user's responsibility.

@hashhar
Copy link
Member

hashhar commented Dec 29, 2021

There is no way today (or before the PR which added prepared statements support) to be able to pass parameters to a query without using prepared statements.

If you want to avoid prepared statements you need to stop passing params by constructing the SQL as a string and then calling cursor.execute(sql) instead of cursor.execute(sql, params).

I'm not sure if there's a way to have both:

  1. Ability for client to handle params.
  2. Not use prepared statements when params are passed.

What solution are you suggesting?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants