Thread safe of httplib2 and credentials -> update Documentation

[DopeforHope]

So right now I'm using your library with multiple threads.
Due to the fact that httplib2 is not thread safe I tried to use your examples.

# Create a new Http() object for every request
def build_request(http, *args, **kwargs):
  new_http = httplib2.Http()
  return apiclient.http.HttpRequest(new_http, *args, **kwargs)
service = build('api_name', 'api_version', requestBuilder=build_request)

# Pass in a new Http() manually for every request
service = build('api_name', 'api_version')
http = httplib2.Http()
service.stamps().list().execute(http=http)

But either way I have the problem that the threads are not authenticated. In the documentation is nothing like this mentioned.
I tried just building an own httplib2.Http() object for every thread at the beginning of it and also tried the approach of overriding the default requestBuilder of the service which I used across all threads but nothing worked due to authentification issues.

In the end it worked for me that I just executed the following authenticate_and_build_service() function for every thread. This function basically return a new service object via the build('drive', 'v3', credentials=creds).

def authenticate_and_build_service():
    # If modifying these scopes, delete the file server_token.pickle.
    scopes = ['https://www.googleapis.com/auth/drive']

    creds = None
    # The file server_token.pickle stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('server_token.pickle'):
        with open('server_token.pickle', 'rb') as token:
            creds = pickle.load(token)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'server_credentials.json', scopes)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('server_token.pickle', 'wb') as token:
            pickle.dump(creds, token)

    return build('drive', 'v3', credentials=creds)

Nevertheless I'm not sure if it has any downsides of building a service object for every thread and it would be nice if someone could comment on this and maybe even update the documentation.

I also didn't find a way in the execute() function to specify credentials.

[busunkim96]

@DopeforHope Thanks for the detailed report! It looks like our docs need to be updated to explain how to authorize the httplib2 instances.

[iMoses]

@busunkim96 please do so

[mescanne]

Is this really relevant?

https://github.com/googleapis/google-api-python-client/blob/master/googleapiclient/discovery.py#L237

On build() it looks like it calls build_http() which also creates a new httplib2.Http() object. Each thread for me has a separate build() call. (Each client => one build() => one thread)

[hendolim]

Hi, I experienced the same problem. Unfortunately on my case, I cannot build a new service object for every thread. So I wish to pass in a new http client on execute instead.
There's no docs anywhere on getting the http client authorized using the oauth2 service_account Credentials object. Please let me know if anyone was able to get this working

[hendolim]

After digging up the source code, I found this in _auth.py:

def authorized_http(credentials):
    """Returns an http client that is authorized with the given credentials.

    Args:
        credentials (Union[
            google.auth.credentials.Credentials,
            oauth2client.client.Credentials]): The credentials to use.

    Returns:
        Union[httplib2.Http, google_auth_httplib2.AuthorizedHttp]: An
            authorized http client.
    """
    from googleapiclient.http import build_http

    if HAS_GOOGLE_AUTH and isinstance(credentials, google.auth.credentials.Credentials):
        if google_auth_httplib2 is None:
            raise ValueError(
                "Credentials from google.auth specified, but "
                "google-api-python-client is unable to use these credentials "
                "unless google-auth-httplib2 is installed. Please install "
                "google-auth-httplib2."
            )
        return google_auth_httplib2.AuthorizedHttp(credentials, http=build_http())
    else:
        return credentials.authorize(build_http())

So I think I can just do something like this:

new_http_client = httplib2.Http()
authorized_htpp = google_auth_httplib2.AuthorizedHttp(credentials, http=new_http_client)
gcpservice.execute(http=authorized_htpp)

UPDATE: This works for me. Although I'd still love to hear from google engineers if this is a recommended approach