Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong output using "to_dataframe" for year above 2099 #201

Closed
davidedironza opened this issue Jul 30, 2020 · 1 comment
Closed

Wrong output using "to_dataframe" for year above 2099 #201

davidedironza opened this issue Jul 30, 2020 · 1 comment
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. duplicate This issue or pull request already exists

Comments

@davidedironza
Copy link

Hi

The bigquery client library is not able to convert the datetime '9999-12-31T00:00:00' from bigquery to dataframe correctly.

from google.cloud import bigquery
sql_statement = f"""SELECT cast('9999-12-31T00:00:00' as datetime)"""
client_bq = bigquery.Client()
df = client_bq.query(sql_statement).to_dataframe()
print(df)

Output
1816-03-29 05:56:08.066277376

Why this is not possible?

Cheers
Davide

@busunkim96 busunkim96 transferred this issue from googleapis/google-cloud-python Jul 30, 2020
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jul 30, 2020
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Jul 30, 2020
@plamut
Copy link
Contributor

plamut commented Jul 31, 2020

Running the query directly in the BigQuery cloud works fine, thus the problem seems to be in result conversion.

I actually wasn't able to reproduce this with a "nice" incorrect result, I actually got a loud exception, the same as reported in #168:

pyarrow.lib.ArrowInvalid: Casting from timestamp[us] to timestamp[ns] would result in out of bounds timestamp: 253402214400000000

The difference can probably be explained by different installed dependency versions (would you mind sharing them? Thanks!), but all in all it seems like a duplicate of #168, thus closing it to keep the discussion in a single place.

@plamut plamut added duplicate This issue or pull request already exists and removed triage me I really want to be triaged. labels Jul 31, 2020
@plamut plamut closed this as completed Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

3 participants