Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for exporting to google bigquery (to_gbq) #2142

Open
andhuang-CLGX opened this issue Sep 29, 2021 · 5 comments
Open

Support for exporting to google bigquery (to_gbq) #2142

andhuang-CLGX opened this issue Sep 29, 2021 · 5 comments

Comments

@andhuang-CLGX
Copy link

andhuang-CLGX commented Sep 29, 2021

Not sure if it's more appropriate here:
googleapis/python-bigquery-pandas#400

But basically:
https://geopandas.org/docs/reference/api/geopandas.GeoDataFrame.html lists to_gbq, but I think it just inherits pandas-gbq; is there any plans to support a geometry column to bigquery; right now I think the geometry column gets converted to string in the schema

@martinfleis
Copy link
Member

Thanks for the suggestion! I think that is on us to override to_gbq to ensure that geometry gets converted properly. There are no specific plans on that but contributions are welcome :).

@martinfleis
Copy link
Member

Dropping this here as a reference https://cloud.google.com/bigquery/docs/geospatial-data

It seems that we need to ensure lat/lon in WGS84 and then we can go via WKT/WKB as we do with PostGIS.

@martinfleis
Copy link
Member

Looking into the pandas implementation, it actually is probably not something to be implemented here. pandas uses pandas_gbq to take care of complete IO. If the geometry support is implemented there, we don't have to do anything.

@jorisvandenbossche
Copy link
Member

Yes, I think it would be nice if this could be supported by https://github.com/googleapis/python-bigquery-pandas (they could check if they receive a GeoDataFrame without having a hard dependency on geopandas).

@andhuang-CLGX
Copy link
Author

andhuang-CLGX commented Sep 30, 2021

I tried uploading thru gbq with no changes to my geopandas dataframe (e.g. the geometry column is automatically converted to WKT) and it just works as long as the table_schema explicitly specifies the geometry column as "GEOGRAPHY" type.

Used the list comprehension described here with an added if clause is geometry type
https://stackoverflow.com/a/21898044/16707045

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants