New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pandas 2.0 conflicts with the SQL connectors #1085
Comments
@gventuri |
@ArslanSaleem, do you have any insight on which connectors do not support sqlalchemy 2.0 or above? |
@ArslanSaleem it looks like snowflake-connector-python[pandas] does not require sqlalchemy at all, and the latest version of snowflake-sqlalchemy seems to allow 2.0.29. As @YarShev , please do let us know which snowflake connector is not working for you. Please also share python and operating system versions. |
@sfc-gh-mvashishtha xref- snowflakedb/snowflake-sqlalchemy#380 - it looks like the latest release on Apr 11 (EDIT: That release got yanked - no reason provided) might have addressed it.
not sure if this was the core issue, if not apologies! |
@asishm @sfc-gh-mvashishtha from what I see, it might be a problem only from pandas >= 2.2. Can we assume we can safely migrate to pandas > 2 < 2.2 for the time being? @ArslanSaleem would that work? |
@gventuri, pandas added support for SQLAlchemy 2.0 as of pandas 2.0. |
@sfc-gh-mvashishtha, @ArslanSaleem, given that, can we assume there is no an issue with the latest version of snowflake-sqlalchemy? |
@YarShev There is. It's not possible to use pandas 2.2 and snowflake-sqlalchemy in the same project.
Someone tried to add support for 2.0 on their own almost 1 year ago, and sent this PR to the snowflake-sqlalchemy repo: snowflakedb/snowflake-sqlalchemy#414, but it was completely ignored and it's now closed. There are many issues open in snowflake-sqlalchemy about this lack of support for >=2.0, with the main one being snowflakedb/snowflake-sqlalchemy#380 In that same issue on Mar 9, someone from the snowflake team said (regarding support for sqlalchemy>=2.0):
Hopefully we'll see support for sqlalchemy 2 by the end of April, but I wouldn't bet on it. If |
@rafaelclp then what would you recommend? We could also consider installing Snowflake from that feature branch for the short term hoping they'll eventually fix it? |
@ArslanSaleem is only Snowflake blocking the upgrade as far as you know? |
This is just my opinion, so hardly a recommendation. My reasoning is that people using that connector are already stuck with older pandas versions and a 1 year old sqlalchemy version, so I don't see why they couldn't afford to be stuck with older pandas-ai versions as well. That's all. Edit: if someone absolutely must use the latest version of pandas-ai, I think your own suggestion is the best option indeed, just install snowflake-sqlalchemy from the PR that adds support for sqlalchemy 2.0 while waiting for it to be merged. In any case, I was wrong in betting against support for sqlalchemy 2 in snowflake-sqlalchemy by the end of April: snowflakedb/snowflake-sqlalchemy#469. It's almost here! |
System Info
OS version: any
Python version: any
pandasai: any
馃悰 Describe the bug
Our current setup is facing issues when attempting to upgrade to pandas versions beyond 2.0, as this update seems to interfere with the functionality of our SQL connectors. It's essential for us to delve into the root cause of this problem to identify a viable solution. This will enable us to successfully transition to a more recent version of pandas, as well as to the latest version of modin.
The text was updated successfully, but these errors were encountered: