"Error: The incoming JSON object does not contain a client_email field" after upgrading to 7.0.2 #1451

serbanstef · 2019-06-28T15:58:58Z

[REQUIRED] Environment info

firebase-tools: 7.0.2

Platform: Oracle Linux Server 7.6, Node 10.15

[REQUIRED] Test case

emulate https function with "firebase-admin": "^8.2.0", "firebase-functions": "^3.0.2"

[REQUIRED] Steps to reproduce

run any https function with firebase functions:shell

[REQUIRED] Expected behavior

no error (firebase-tools 7.0.0 throws no error)

[REQUIRED] Actual behavior

Sent request to function.

firebase > ⚠  Error: The incoming JSON object does not contain a client_email field
    at JWT.fromJSON (/riderequest/functions/node_modules/google-auth-library/build/src/auth/jwtclient.js:165:19)
    at GoogleAuth.fromJSON (/riderequest/functions/node_modules/google-auth-library/build/src/auth/googleauth.js:294:16)
    at GoogleAuth.getClient (/riderequest/functions/node_modules/google-auth-library/build/src/auth/googleauth.js:476:52)
    at GrpcClient._getCredentials (/riderequest/functions/node_modules/google-gax/build/src/grpc.js:107:40)
    at GrpcClient.createStub (/riderequest/functions/node_modules/google-gax/build/src/grpc.js:223:34)
    at new FirestoreClient (/riderequest/functions/node_modules/@google-cloud/firestore/build/src/v1/firestore_client.js:128:39)
    at ClientPool.Firestore._clientPool.pool_1.ClientPool [as clientFactory] (/riderequest/functions/node_modules/@google-cloud/firestore/build/src/index.js:315:26)
    at ClientPool.acquire (/riderequest/functions/node_modules/@google-cloud/firestore/build/src/pool.js:61:35)
    at ClientPool.run (/riderequest/functions/node_modules/@google-cloud/firestore/build/src/pool.js:114:29)
    at Firestore.request (/riderequest/functions/node_modules/@google-cloud/firestore/build/src/index.js:957:33)
⚠  Your function was killed because it raised an unhandled error.

The text was updated successfully, but these errors were encountered:

noelmansour · 2019-06-28T16:54:50Z

Relevant stackoverflow post: https://stackoverflow.com/questions/56789424/the-incoming-json-object-does-not-contain-a-client-email-field

samtstern · 2019-06-28T21:12:04Z

@noelmansour thanks so it seems like something between 7.0.1 and 7.0.2 caused this, I will look at the changes made.

samtstern · 2019-06-28T21:19:48Z

Looks like that error comes from this library: googleapis/google-auth-library-nodejs#137

https://github.com/googleapis/google-auth-library-nodejs/blob/37bb8c7cd0a6501103274284d9cddd6816cc881e/src/auth/jwtclient.ts#L253

mschristensen · 2019-06-30T19:38:16Z

Am also experiencing this issue since 7.0.2

jrthib · 2019-07-01T02:26:57Z

Just upgraded to 7.0.2, seeing this as well. Checked my account credentials and the client_email field is present.

samtstern · 2019-07-01T17:26:15Z

Could someone share the code of a function they're trying to run in the emulator that fails with this error? The simpler the better.

wceolin · 2019-07-01T18:23:26Z

@samtstern The following code fails for me:

import * as admin from 'firebase-admin';
admin.initializeApp();

import * as functions from 'firebase-functions';

export const testFunction = functions.https.onRequest(async (req, res) => {
  const request = await admin.firestore().doc('test/123').get();

  return request.ref.set({ test: 'random' })
    .then(() => res.send('ok!'))
    .catch((err) => res.status(500).send('error'));
});

It only works if I set a service account when calling admin.initializeApp():

// this works
admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
});

samtstern · 2019-07-01T18:29:26Z

@wceolin thanks for that! Can you run the following inside your functions directory?

$ npm list @google-cloud/firestore

This is my result (and everything works for me):

functions@ /tmp/tmp.ZkMEM0XGPF/functions
└─┬ firebase-admin@8.2.0
  └── @google-cloud/firestore@2.2.3

wceolin · 2019-07-01T18:45:43Z

I got the same result: 🤔

samtstern · 2019-07-01T18:50:59Z

So I am walking my way up the error stack trace and in this function:
https://github.com/googleapis/gax-nodejs/blob/e1be4ebcc2287c61d5f1884033449e3b4e143242/src/grpc.ts#L141

We have this branching:

 async _getCredentials(opts: ClientStubOptions) {
    if (opts.sslCreds) {
      return opts.sslCreds;
    }
    const grpc = this.grpc;
    const sslCreds = grpc.credentials.createSsl();
    const client = await this.auth.getClient();
    const credentials = grpc.credentials.combineChannelCredentials(
      sslCreds,
      grpc.credentials.createFromGoogleCredential(client)
    );
    return credentials;
  }

In my code opts.sslCreds is defined so I hit this branch:

    if (opts.sslCreds) {
      return opts.sslCreds;
    }

It's the other branch which tries to assemble credentials and causes problems. Now I need to figure out why my code goes one way and yours goes another way.

samtstern · 2019-07-01T18:52:25Z

@wceolin could you reach into your node_modules folder and edit this file functions/node_modules/google-gax/build/src/grpc.js to have a console.log() that prints the options object in the createStub function and let me know what you see?

Thank you for your help so far!

noelmansour · 2019-07-01T18:53:09Z

@samtstern Not sure if this helps, but I did a bunch of tweaking with my package.json and node_modules over the weekend. I no longer have the issue on 7.0.2.

The npm list @google-cloud/firestore output matches yours.

I nuked my node_modules directory over the weekend so I wonder if that's related.

samtstern · 2019-07-01T18:55:49Z

@noelmansour thanks for that note, explains why I can't reproduce in a clean setup.

wceolin · 2019-07-01T19:05:40Z

@noelmansour I also tried deleting my node_modules, yarn.lock, removing dependencies but it didn't work for me. I'm gonna try creating a project from scratch to see what happens.

@noelmansour this is what I got from the options object:

{
  "clientConfig": {},
  "port":443,
  "servicePath":"firestore.googleapis.com",
  "credentials":{},
  "projectId":"my-test-project",
  "firebaseVersion":"8.2.0",
  "libName":"gccl",
  "libVersion":"2.2.3 fire/8.2.0",
  "scopes":[
    "https://www.googleapis.com/auth/cloud-platform", 
    "https://www.googleapis.com/auth/datastore"
  ]
}

samtstern · 2019-07-01T19:08:45Z

@wceolin thanks that's extremely helpful because this is what mine looks like (and what it should look like):

{
   "clientConfig":{},
   "port":8080,
   "servicePath":"localhost",
   "credentials":{},
   "projectId":"fir-dumpster",
   "firebaseVersion":"8.2.0",
   "libName":"gccl",
   "libVersion":"2.2.3 fire/8.2.0",
   "service":"firestore.googleapis.com",
   "sslCreds":{
      "callCredentials": {}
   },
   "customHeaders":{
      "Authorization":"Bearer owner"
   },
   "scopes":[
      "https://www.googleapis.com/auth/cloud-platform",
      "https://www.googleapis.com/auth/datastore"
   ]
}

As you can see your client is still trying to access production, and then it fails because the functions emulator is not authorized to do so.

Are you also running the firestore emulator? Is your goal to write to the firestore emulator or do you actually want to write to production firestore?

samtstern · 2019-07-01T19:09:53Z

OH I WAS FINALLY ABLE TO REPRODUCE THIS! If I run firebase emulators:start --only functions then I try to write to Firestore I get this error.

I see this log message:

⚠  The Cloud Firestore emulator is not running so database operations will fail with a 'default credentials' error.

Which is no longer exactly right (the error is about client_email) but the detection is still correct. Do others here see this warning as well?

noelmansour · 2019-07-01T19:13:29Z

Here's some more info that may help. Over the weekend, I had also explicitly set the firestore emulator env variable in my .zshrc:

export FIRESTORE_EMULATOR_HOST="localhost:8080"

When I remove it, I get the error. And this is with the command firebase emulators:start

samtstern · 2019-07-01T19:18:04Z

@noelmansour can you run that command with --debug and attach the logs here as a txt file?

wceolin · 2019-07-01T19:20:01Z

@samtstern sometimes I was using the emulator to serve an HTTP function locally which sets some data in production (usually to do some data migration). I'm using:

firebase serve --only functions --project myProjectAlias

I'm not sure if it was supposed to work like that (allowing us to write data to production) but it used to work. 😅

samtstern · 2019-07-01T19:23:06Z

@wceolin it wasn't supposed to work like that with bare initializeApp() in versions after 6.8.0. I also have that use case but we want a find a way to make that opt-in so that the default is to protect production.

But I am glad to know where this error comes from now!

noelmansour · 2019-07-01T19:28:26Z

@samtstern Just to confirm, run firebase emulators:start --debug with the FIRESTORE_EMULATOR_HOST not set, correct?

samtstern · 2019-07-01T19:30:42Z

Correct!

…

On Mon, Jul 1, 2019, 12:28 PM Noel Mansour ***@***.***> wrote: @samtstern <https://github.com/samtstern> Just to confirm, run firebase emulators:start --debug with the FIRESTORE_EMULATOR_HOST not set, correct? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1451?email_source=notifications&email_token=ACATB2WV27VPIODNLGFS4TLP5JLGBA5CNFSM4H4GTWI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY7DO2Q#issuecomment-507393898>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACATB2WG4EAD5KU7YRB5AUTP5JLGBANCNFSM4H4GTWIQ> .

noelmansour · 2019-07-01T19:36:52Z

Here it is.

FYI I removed the project-id from this line (don't think it matters):
[2019-07-01T19:32:42.859Z] >>> HTTP REQUEST GET https://mobilesdk-pa.googleapis.com/v1/projects/<project-id>:getServerAppConfig

debug.txt

noelmansour · 2019-07-01T19:39:58Z

Also worth mentioning, without exporting the env variable, firebase-tools@7.0.1 gives me this error:

⚠  Error: Getting metadata from plugin failed with error: Header field "authorization" must have only a single value
    at Http2CallStream.call.on (/Users/noel/dev/snowble/functions/node_modules/@grpc/grpc-js/build/src/call.js:68:41)
    at emitOne (events.js:121:20)
    at Http2CallStream.emit (events.js:211:7)
    at process.nextTick (/Users/noel/dev/snowble/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:71:22)
    at _combinedTickCallback (internal/process/next_tick.js:132:7)
    at process._tickCallback (internal/process/next_tick.js:181:9)
⚠  Your function was killed because it raised an unhandled error.

samtstern · 2019-07-01T20:53:20Z

Thanks @noelmansour for those logs. Would you mind showing your functions code as well?

noelmansour · 2019-07-01T21:23:59Z

It's a little tricky because I'd rather not share all my functions, and I don't have time right now to verify that in isolation this will repro the error, but here's the hello function I've been running:

index.ts:

require('./common'); // this should always be first in this file

export * from './debug'
// other exports for other functions

common.ts:

import * as admin from 'firebase-admin';

export const app = admin.initializeApp();

debug.ts:

import * as admin from 'firebase-admin';
import * as functions from 'firebase-functions';
import {app} from "./common";

export const hello = functions.https.onRequest(async (req, resp) => {
    const firestore = app.firestore();
    const users = await firestore.collection('users').get();
    console.log('empty users collection? ' + users.empty);
    resp.sendStatus(200);
});

package.json:

{
  "name": "functions",
  "scripts": {
    "lint": "tslint --project tsconfig.json",
    "build": "tsc",
    "serve": "npm run build && firebase serve --only functions",
    "shell": "npm run build && firebase functions:shell",
    "emulators": "npm run build && firebase emulators:start",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "8"
  },
  "main": "lib/index.js",
  "dependencies": {
    "@google-cloud/tasks": "^1.1.0",
    "@types/jsonwebtoken": "^8.3.2",
    "@types/request-promise": "^4.1.42",
    "firebase-admin": "^8.2.0",
    "firebase-functions": "^3.0.2",
    "request": "^2.88.0",
    "request-promise": "^4.2.4"
  },
  "devDependencies": {
    "tslint": "^5.12.0",
    "typescript": "^3.2.2"
  },
  "private": true
}

samtstern · 2019-07-01T21:36:55Z

@noelmansour thank you! That just revealed a bug to me. These two things should be identical but are not:

Option 1

admin.initializeApp();
const firestore = admin.firestore();

Option 2

const app = admin.initializeApp();
const firestore = app.firestore();

DavidAPears · 2019-07-09T09:59:31Z

Firebase Console > Settings (Gear icon) > Users and permissions > Service accounts

Generate new key.

Add json to your project folder.
var admin = require("firebase-admin");

var serviceAccount = require("path/to/serviceAccountKey.json");

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL: "https://lts-profile.firebaseio.com"
});
The Service account json file makes your firebase services and features available to your project.
It should always be private. Include file in .gitignore

Thanks phtn, appreciated - I followed your instructions but I now get the error

Error: Error occurred while parsing your function triggers.

Weirdly when I run 'firebase deploy' everything is working on: 'https://europe-west1-myapp.cloudfunctions.net/api/... - just NOT on localhost?

DavidAPears · 2019-07-09T12:09:52Z

A combo of these two responses worked! Thanks again for helping.

https://stackoverflow.com/questions/44626919/where-is-or-what-is-the-serviceaccountkey-json-is-the-node-js-sample-of-firebase?rq=1

https://stackoverflow.com/questions/42176453/google-app-engine-node-cannot-find-module-firebase-admin/42634321#42634321

phtn · 2019-07-10T06:26:04Z

serviceAccountKey.json should be in /functions directory.

the firebase deploy --only functions will complain if it's outside.

DavidAPears · 2019-07-10T08:34:41Z

Thanks again (for being so patient!). My serviceAccountKey.json is in the /functions folder (See screen-grab on link below).

It appears there are three ways to initilizeApp...I have tried all three of these individually and none of them seem to work - I have a feeling this is going to be something crazy simple I am missing.

D

Screen Grab:

ryanhornberger · 2019-07-11T14:04:31Z

We are having the same issue on our end. We're initializing firebase under this model:

const firebaseInstance = admin.initializeApp()
const firestoreInstance = firebaseInstance.firestore()

Let me know if we can provide anything to help track this down.

ryanhornberger · 2019-07-11T14:24:17Z

Everyone on this thread. We discovered a solution. We downgraded firebase-tools from 7.0.2 to 7.0.1 and received an entirely different error (something about not being able to load default credentials).

So we continued by running:

gcloud auth application-default login

This fixed our issue

samtstern · 2019-07-11T16:01:59Z

@ryanhornberger you could work around your issue by doing:

const firebaseInstance = admin.initializeApp()
const firestoreInstance = admin.firestore() // I changed this line

I have a fix for that coming in #1459

ryanhornberger · 2019-07-11T17:40:19Z

@samtstern thank you!

kwangdev · 2019-07-14T01:47:17Z

Everyone on this thread. We discovered a solution. We downgraded firebase-tools from 7.0.2 to 7.0.1 and received an entirely different error (something about not being able to load default credentials).

So we continued by running:
gcloud auth application-default login
This fixed our issue

This works for me. I initially upgraded firebase-tools to 7.1, I got the 'same error' when I run 'firebase serve' in my local. However it's working fine if I deployed to firebase. Downgraded to 7.0.1 works for me in my local.

westlakem · 2019-07-14T17:51:47Z

Same issue with 7.1

samtstern · 2019-07-15T20:13:09Z

We are working on a fix, but it's not ready yet:
#1479

Hipheckts · 2019-07-16T19:28:53Z

I'm stuck with this please, any way out?

zcaalock · 2019-07-16T19:39:20Z

downgrading to 7.0.1 did`t help for me

samtstern · 2019-07-18T00:32:06Z

A fix for this was merged into #1479 and will be included in the next release (so 7.2.0)

noelmansour · 2019-07-19T19:28:34Z

Hey @samtstern, not sure if it's something I'm doing incorrectly, or perhaps a couple of new issues, but I've noticed a couple of things running the tools off the latest master (dad143c).

First, I get the following warning when running firebase emulators:exec "npm run test":

⚠ The Cloud Firestore emulator is not running, so calls to Firestore will affect production.

Even though earlier in the command I can see that it starts the emulator:

~/d/s/functions ❯❯❯ firebase emulators:exec "npm run test"                                                                                                                           ✘ 130
i  Starting emulators: ["functions","firestore"]
✔  functions: Using node@8 from host.
✔  functions: Emulator started at http://localhost:5001
i  firestore: Logging to firestore-debug.log
✔  firestore: Emulator started at http://localhost:8080
i  firestore: For testing set FIRESTORE_EMULATOR_HOST=localhost:8080

It's not actually updating the production firestore DB. Maybe just a false negative?

The other issue is that I'm still getting the error. I didn't set $GOOGLE_APPLICATION_CREDENTIALS, but according to https://firebase.google.com/docs/functions/local-emulator#set_up_admin_credentials_optional there should already be sufficient permission if only accessing firestore:

Cloud Firestore and Realtime Database triggers already have sufficient credentials, and do not require additional setup.

I'm initializing the admin sdk without any parameters using admin.initializeApp(); Perhaps something I'm doing incorrectly?

samtstern · 2019-07-19T19:45:06Z

@noelmansour thanks for reporting this (and for trying master!) Can you show the code of your main functions file?

Almost certainly a false negative, but an important one that we need to fix.

noelmansour · 2019-07-19T19:47:20Z

It's pretty minimal as it's just exporting from other files:

import * as admin from 'firebase-admin';

// this should happen before any function runs
admin.initializeApp();

export * from './authFunctions';
export * from './fitbit';
export * from './sleep'
export * from './scheduler';
export * from './debug'
export {scheduleFunction} from "./scheduler";

noelmansour · 2019-07-19T19:51:57Z

Actually, it seems to be an issue with my test setup. When running firebase emulators:start I don't see the warning.

samtstern · 2019-07-19T19:53:18Z

@noelmansour what do your tests look like? Also could you file a new issue to discuss this so we don't spam everyone on this thread>

noelmansour · 2019-07-19T22:43:32Z

Yes, of course. My apologies. #1530 opened

Immanuel-john · 2019-07-22T06:07:11Z

Everyone on this thread. We discovered a solution. We downgraded firebase-tools from 7.0.2 to 7.0.1 and received an entirely different error (something about not being able to load default credentials).
So we continued by running:
gcloud auth application-default login
This fixed our issue
This works for me. I initially upgraded firebase-tools to 7.1, I got the 'same error' when I run 'firebase serve' in my local. However it's working fine if I deployed to firebase. Downgraded to 7.0.1 works for me in my local.

Could you pls tell how to downgrade firebase version? My current version is 7.0.2 then i tried installing npm i firebase-tools@7.0.0 -g this. but still my version is 7.0.2 . how to downgrade pls help?

samtstern · 2019-07-24T17:28:45Z

The full fix for this issue has been released in 7.2.0:

npm install -g firebase-tools@7.2.0

If you are still experiencing a similar bug on that version, please open a new issue.

quickresolve · 2019-12-12T09:24:31Z

Having this same issue with Firebase functions. Already tried npm install -g firebase-tools@7.2.0 resulting in the same error.

quickresolve · 2019-12-12T09:33:28Z

Having this same issue with Firebase functions. Already tried npm install -g firebase-tools@7.2.0 resulting in the same error.

The following solution worked for me:

// Create Firebase-adminsdk key

// Providing a service account object inline
admin.initializeApp({
    credential: admin.credential.cert({
        projectId: "<PROJECT_ID>",
        clientEmail: "foo@<PROJECT_ID>.iam.gserviceaccount.com",
        privateKey: "-----BEGIN PRIVATE KEY-----<KEY>-----END PRIVATE KEY-----\n"
    })
});

samtstern added emulator-suite emulators: functions labels Jun 28, 2019

samtstern self-assigned this Jul 1, 2019

samtstern closed this as completed in #1479 Jul 18, 2019

noelmansour mentioned this issue Jul 19, 2019

Firestore emulator: False negative warning about affecting production when running tests #1530

Closed

This was referenced Jan 18, 2022

[Snyk] Security upgrade node-fetch from 2.6.1 to 3.1.1 XirdigH/firebase-tools#28

Open

[Snyk] Security upgrade node-fetch from 2.6.1 to 3.1.1 Manny27nyc/firebase-tools#17

Open

[Snyk] Security upgrade node-fetch from 2.6.1 to 3.1.1 majacQ/firebase-tools#23

Open

adi9090 mentioned this issue Mar 21, 2023

Error: The incoming JSON object does not contain a client_email field r0adkll/upload-google-play#176

Closed

"Error: The incoming JSON object does not contain a client_email field" after upgrading to 7.0.2 #1451

"Error: The incoming JSON object does not contain a client_email field" after upgrading to 7.0.2 #1451

Comments

serbanstef commented Jun 28, 2019 • edited by samtstern

[REQUIRED] Environment info

[REQUIRED] Test case

[REQUIRED] Steps to reproduce

[REQUIRED] Expected behavior

[REQUIRED] Actual behavior

noelmansour commented Jun 28, 2019

samtstern commented Jun 28, 2019 • edited

samtstern commented Jun 28, 2019 • edited

mschristensen commented Jun 30, 2019

jrthib commented Jul 1, 2019

samtstern commented Jul 1, 2019

wceolin commented Jul 1, 2019

samtstern commented Jul 1, 2019

wceolin commented Jul 1, 2019

samtstern commented Jul 1, 2019

samtstern commented Jul 1, 2019

noelmansour commented Jul 1, 2019

samtstern commented Jul 1, 2019

wceolin commented Jul 1, 2019 • edited

samtstern commented Jul 1, 2019

samtstern commented Jul 1, 2019 • edited

noelmansour commented Jul 1, 2019 • edited

samtstern commented Jul 1, 2019

wceolin commented Jul 1, 2019

samtstern commented Jul 1, 2019

noelmansour commented Jul 1, 2019

samtstern commented Jul 1, 2019 via email

noelmansour commented Jul 1, 2019 • edited

noelmansour commented Jul 1, 2019

samtstern commented Jul 1, 2019

noelmansour commented Jul 1, 2019

samtstern commented Jul 1, 2019

DavidAPears commented Jul 9, 2019

DavidAPears commented Jul 9, 2019

phtn commented Jul 10, 2019

DavidAPears commented Jul 10, 2019 • edited

ryanhornberger commented Jul 11, 2019

ryanhornberger commented Jul 11, 2019

samtstern commented Jul 11, 2019

ryanhornberger commented Jul 11, 2019

kwangdev commented Jul 14, 2019

westlakem commented Jul 14, 2019

samtstern commented Jul 15, 2019

Hipheckts commented Jul 16, 2019

zcaalock commented Jul 16, 2019

samtstern commented Jul 18, 2019

noelmansour commented Jul 19, 2019 • edited

samtstern commented Jul 19, 2019 • edited

noelmansour commented Jul 19, 2019

noelmansour commented Jul 19, 2019

samtstern commented Jul 19, 2019

noelmansour commented Jul 19, 2019

Immanuel-john commented Jul 22, 2019

samtstern commented Jul 24, 2019

quickresolve commented Dec 12, 2019

quickresolve commented Dec 12, 2019

serbanstef commented Jun 28, 2019 •

edited by samtstern

samtstern commented Jun 28, 2019 •

edited

samtstern commented Jun 28, 2019 •

edited

wceolin commented Jul 1, 2019 •

edited

samtstern commented Jul 1, 2019 •

edited

noelmansour commented Jul 1, 2019 •

edited

noelmansour commented Jul 1, 2019 •

edited

DavidAPears commented Jul 10, 2019 •

edited

noelmansour commented Jul 19, 2019 •

edited

samtstern commented Jul 19, 2019 •

edited