Python to construct CASE WHEN update SQL statement

[elad2109]

I try to update 2K rows in BQ

def update_bq_ads_status_failed(self, update_ads):
        affected_rows = 0
        for update_ads_chunk in split(update_ads, _UPDATE_CHUNK_SIZE):
            ad_ids = [item["ad_id"] for item in update_ads_chunk]
            removal_errors = [item["removal_error"] for item in update_ads_chunk]

            update_removal_error = ""
            for ad_id, removal_error in zip(ad_ids, removal_errors):
                update_removal_error = update_removal_error + \
                                       f''' WHEN ad_id = '{ad_id}' Then '{removal_error}' '''
            affected_rows += self.update_bq_ads_status(f"""
                            UPDATE '{table_full_name}' 
                            SET status = 'Failed Removing'  
                SET removal_error = CASE {update_removal_error} END 
                WHERE ad_id IN {str(ad_ids)}
                """)
        return affected_rows

I'm getting this error. I know it's too vague and not possible to debug like this.

timeout=300.0, headers={'X-Server-Timeout': '300.0',
'Accept-Encoding': 'gzip', 'Content-Type': 'application/json',
'X-Goog-API-Client': 'gl-python/3.8.10 grpc/1.39.0 gax/2.0.0
gapic/2.26.0 gccl/2.26.0', 'User-Agent': 'gl-python/3.8.10 grpc/1.39.0
gax/2.0.0 gapic/2.26.0 gccl/2.26.0'})), last exception: ('Connection
aborted.', RemoteDisconnected('Remote end closed connection without
response'))

I'm trying to eliminate errors. Is my BQ update syntactically correct?

What's the BQ update timeout?

[busunkim96]

It looks like the library version in x-goog-api-client is 2.26.0, which is the latest version of [google-cloud-bigquery](https://pypi.org/project/google-cloud-bigquery/. The latest release of google-api-python-client is 2.20.0.

@elad2109 Could you confirm which library you are using (google-api-python-client or google-cloud-bigquery?

[elad2109]

It looks like the library version in x-goog-api-client is 2.26.0, which is the latest version of [google-cloud-bigquery](https://pypi.org/project/google-cloud-bigquery/. The latest release of google-api-python-client is 2.20.0.

@elad2109 Could you confirm which library you are using (google-api-python-client or google-cloud-bigquery?

Sorry for asking, is there any easy way cmd to run to verify?

[busunkim96]

@elad2109 Ah no worries, a pip freeze (assuming you're using pip) will show the packages you currently have installed.

Alternatively, you can check the imports in your code:
from google.cloud import bigquery for google-cloud-bigquery

from googleapiclient import discovery for google-api-python-client

[elad2109]

My code has from google.cloud import bigquery

[tswast]

It looks like the default timeout of 5 minutes (60 * 5 seconds) might not be long enough. The default timeout was added in #896 in response to a request from the backend engineering folks.

It seems to me we should remove the default timeout, though the logic to add the X-Server-Timeout header if a timeout is explicitly set should remain.