Cloud functions Export/Import - A Forbidden error was returned while attempting to retrieve an access token for the Compute Engine built-in service account.

[39ro]

I am trying to generate a snapshot of my Firestore database, while running the following code inside a Cloud Function, I receive a "Forbidden Error".
From my side, I can't see nothing else in my request that need to be added.

const {auth} = require('google-auth-library');

async function main() {
    const client = await auth.getClient({
            scopes: [
                'https://www.googleapis.com/auth/datastore',
                'https://www.googleapis.com/auth/cloud-platform'
            ]
        }
    );
    const projectId = await auth.getProjectId();
    const url = `https://firestore.googleapis.com/v1beta1/projects/${projectId}/databases/(default):exportDocuments`;
    return client.request({
        url: url,
        method: 'post',
        data: {
            outputUriPrefix: 'gs://my-awesome-storage.appspot.com'
        }
    });
}

Could someone please clarify what it actually means?

Forbidden error was returned while attempting to retrieve an access token for the Compute Engine built-in service account. This may be because the Compute Engine instance does not have the correct permission scopes specified. The caller does not have permission

• https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases/exportDocuments#http-request

Environment details

• google-auth-library version: 2.0.1

[JustinBeckwith]

Greetings! This is a bit of a tricky one. When I try this out for myself, everything seems to work. Can you double check the permissions on the service account you're using to run your cloud function? Is it using the default App Engine service account?

[torbjornvatn]

@JustinBeckwith I'm having the exact same issue and my function uses the default App Engine service account which has the Project Editor role enabled.
When I run the function in the emulator using firebase functions:shell with my personal account logged in to gcloud it works just fine

[acoulton]

I also this evening spent a while trying to figure out the other variant of this error when running in CloudFunctions:

Error: A Not Found error was returned while attempting to retrieve an accesstoken for the Compute Engine built-in service account. This may be because the Compute Engine instance does not have any permission scopes specified.

It took me a while to realise the issue was actually nothing at all to do with the accesstoken : I had actually entered the wrong API url. Fixing that URL solved the problem.

It looks like https://github.com/googleapis/google-auth-library-nodejs/blob/master/src/auth/computeclient.ts#L92 is potentially firing on any 403/404 rather than just ones related to getting a token?

[torbjornvatn]

I don't enter the URL manually though, but rather call the export endpoint like this

 firestore.projects.databases.exportDocuments({
        name: `projects/${projectId}/databases/${projectId}`,
        requestBody: {"outputUriPrefix": `gs://${backupBucket}`}
 })

And I've added these scopes:

scopes: [
            'https://www.googleapis.com/auth/cloud-platform',
            'https://www.googleapis.com/auth/datastore',
            'https://www.googleapis.com/auth/compute'
        ],

And as mentioned it works just fine when running the function locally

[torbjornvatn]

I also tried changing the name part to name: 'projects/${projectId}/databases/(default)' to use the default database feature, but got the same error

[javiermuniz]

@JustinBeckwith I'm having the same issue when running from functions, oddly when I download the json file, set env, and make the same call locally I'm getting "The caller does not have permission"

This is using the json key file from the App Engine service account. I've tried using the client.request() method as well as the exportDocuments method within the firestore namespace in googleapis. Both yield the same result.

[javiermuniz]

@torbjornvatn the PrIoject Editor role, and therefore the default App Engine service account, is not enough to perform backups. Try going into IAM and adding the Cloud Datastore Import Export Admin role to your App Engine service account. This fixed the issue for me.

[JustinBeckwith]

Greetings folks! I suspect the problem y'all are running into is independent of this library. It sounds like a case of trying to use an API that the service account doesn't have access to. If you're still running into issues - please do let me know! Otherwise - I'm closing this out for now.

[acoulton]

@JustinBeckwith the root cause is the absence of permissions on the service account, or in my case just using the wrong URL for an API.

But per my comment above the problem is that this library at https://github.com/googleapis/google-auth-library-nodejs/blob/master/src/auth/computeclient.ts#L92 discards any error message returned from the API.

Instead for all 403 or 404 it just returns a generic message about an error while attempting to retrieve an access token for the Compute Engine built-in service account.

This causes confusion and means the user doesn't see the actual error returned by the API (which I think in the case of a 403 usually mentions the permission that is required). I spent a lot of time trying to debug permissions things before I realised I'd just got a typo in the URL.

[jpotterm]

I'll second what @acoulton said. The discarding of any error messages from the API makes mistakes really hard to debug.

[jpotterm]

A workaround I found is to use the getRequestHeaders function from this library and then send the request through something like axios instead. That will let you see the error message.

[JustinBeckwith]

Thanks for sticking with me here folks. I submitted #668, which should do a better job of surfacing the underlying exception. I think this is the last of the exceptions we were eating and not re-throwing in the code base.

[acoulton]

Brilliant, thanks @JustinBeckwith

[DB-Alex]

@torbjornvatn the PrIoject Editor role, and therefore the default App Engine service account, is not enough to perform backups. Try going into IAM and adding the Cloud Datastore Import Export Admin role to your App Engine service account. This fixed the issue for me.

I needed the "Cloud Datastore Owner" role in order to get it working