Keeping long term stream open with custom tokens

[peterz0310]

Hi, I'm working on a project where I need a RTDB stream to be open and listening at all times. I am getting a custom token from my API and logging in just fine, then beginning a stream. It works perfectly for exactly 55 minutes each time until (I think this is the problem) the token expires, then no more changes on the node are detected. I've tried setting config.signer.preRefreshSeconds = 30 * 60, and then the same thing happens at 30 minutes. I'm not notified of a timeout or anything on the stream, it just stops working. Am I doing something wrong? Is there a way to handle this problem without a significant amount of downtime? Or do I just have to end the stream, get a new token, and restart the stream? If there's a solution in the README I couldn't find it. Thanks in advance for the help.

[mobizt]

When custom token expired before the 60 minutes after setCustomToken was called, library can't do anything as it is not sync between the time to live of token and the time + 60 min since setCustomToken was called.

Unfortunately, the function setCustomToken does not provide the expire period as parameter which you can't change the default 60 minutes expire period.

Normally, the stream time out should be occurred unless your code stops at some points.

We can't refresh the token because of no refresh token provided then config.signer.preRefreshSeconds does nothing.

You have to call setCustomToken with refresh token that get return from googleapis when you exchanged your custom token with ID token.

This is better if you provide the refresh token to setCustomToken instead of custom token as it will refresh depends on config.signer.preRefreshSeconds option.

The refresh token itself is never expired and you can use it many times.

[mobizt]

I recommend you set custom token with setCustomToken only one time, and when token is ready (Firebase.ready), you save the refresh token from Firebase.getRefreshToken() as file.

After 30 minutes or whenever you want, call setCustomToken again with the refresh token and set config.signer.preRefreshSeconds to 30 minutes or any.

The token is now will be refresh automatically.

After your device reboot, you can read the refresh token from file and set it to setCustomToken which custom token will not be required anymore.

[peterz0310]

Thanks that makes sense. I really appreciate the help. I tried to implement this, but it still loses connection to the stream after 30 minutes. I must be setting something up wrong. My program has a 'setup' mode and a 'normal' mode. When you first set the device up, it:

1. Gets a custom token
2. Gets the refresh token and saves it with the preferences library
3. Restarts to enter 'normal' mode

In normal mode, it gets the token from storage, and sets it with setCustomToken (no more custom token requested from API). This successfully logs in and works until the time comes to refresh the token, which is when the stream stops being responsive. I might have to do something to manage the stream during these refreshes, I'm not sure. Here's some of my simplified code in case you can spot anything:

void initializeStream() {
  Firebase.RTDB.setStreamCallback(&fbdo, streamCallback, streamTimeoutCallback);
  fbdo.keepAlive(5, 5, 1);
  Serial.println("Initializing Firebase Stream...");
  if (!Firebase.RTDB.beginStream(&fbdo, deviceNode + "request_id")) {
    Serial.println("Could not begin stream. REASON:" + fbdo.errorReason());
  } else {
    Serial.println("Firebase Stream initialized successfully.");
  }
}

...

void setup() {
  Serial.begin(115200);

  preferences.begin("led-controller", false);

  if (refreshToken.length() < 1) {
    ownerUID = preferences.getString("ownerUID", "");
    refreshToken = preferences.getString("refreshToken", "");
  }

  deviceStatus = getDeviceStatus();

  if (deviceStatus != "not_set_up") {

    deviceNode = "/" + ownerUID + "/" + deviceID + "/";

    WiFi.begin();
    Serial.println("Trying to connect to WiFi...");

    unsigned long startTime = millis();
    while (WiFi.status() != WL_CONNECTED && millis() - startTime < 10000) {
      delay(500);
      Serial.print(".");
    }

    if (WiFi.status() != WL_CONNECTED) {
      Serial.println("Failed to connect, restarting");
      ESP.restart();
    }

    Serial.println("Connected to WiFi");
    Serial.print("IP Address: ");
    Serial.println(WiFi.localIP());

    config.api_key = API_KEY;
    config.database_url = DATABASE_URL;

    Firebase.reconnectNetwork(true);
    fbdo.setBSSLBufferSize(4096, 1024);
    config.token_status_callback = tokenStatusCallback;
    config.signer.preRefreshSeconds = 30 * 60;
    Firebase.setCustomToken(&config, refreshToken);

    Firebase.begin(&config, &auth);
    
    initializeStream();

  } else {
    Serial.println("Setup!");

    WiFiManager wifiManager;

    wifiManager.setConfigPortalTimeout(180);

    wifiManager.setAPCallback([](WiFiManager* myWiFiManager) {
      Serial.println("Entered config mode");
      Serial.print("Connecting to WiFi... Status: ");
      Serial.println(WiFi.status());
    });

    Serial.println("Trying to autoconnect...");

    if (!wifiManager.autoConnect("LED_Controller")) {
      Serial.println("Failed to connect, restarting");
      delay(3000);
      ESP.restart();
    }

    Serial.println("Connected to WiFi");
    Serial.print("IP Address: ");
    Serial.println(WiFi.localIP());

    config.api_key = API_KEY;
    config.database_url = DATABASE_URL;

    Firebase.begin(&config, &auth);

    if (Firebase.signUp(&config, &auth, "", "")) {
      Serial.println("Signed up anonymously!");
    } else {
      Serial.println("Sign up failed.");
    }

    fetchOwnerUID();
    if (ownerUID.length() > 0) {
      if (fetchCustomToken()) {
        while (!Firebase.ready()) {
          delay(100);
        }
        refreshToken = Firebase.getRefreshToken();
        Serial.println("Refresh token: " + refreshToken);
        preferences.putString("refreshToken", refreshToken);
        Serial.println("Restarting...");
        ESP.restart();
        }
    }
  }
}

Thanks again for your time.

[mobizt]

If you use WiFi manager to control WiFi connection, you have to remove or don't use Firebase.reconnectNetwork function or pass false as the parameter because it conflicts with internal library network reconnection.

I recommend that you don't use it and should update the ESP32 Arduino core because bugs in the core WiFi library.

[peterz0310]

Does it still conflict even when it's not in use and there's no instance started? It's only used in my code for the initial setup portion, then the standard WiFi library handles the rest after a restart, including everything with the refresh tokens.

[mobizt]

As the name implies, the wifiManager.autoConnect and Firebase.reconnectNetwork do the same thing to stop, restart, monitoring the network status and try it over and over again when it is not connected.

As I said before, stream time out should be (always be) happened unless there is something blocking in your code or third-party library and also this library will not raise the network connection failure error status for stream because stream usually run repeatedly in loop and may produce too much network connection errors in the result. The stream time out should be taken places for all cases of network, internet and server connection lost.

It is not possible that you don't see the stream time out error if your code run normally unless it stops or freezes.

No matter what you use WiFiManager or not, inside the Firebase.reconnectNetwork, it will call WiFi.reconnect which has a bug in old ESP32 Arduino core and you should update.

After code inspection, I see that you are using the library incorrectly.

while (!Firebase.ready()) {
          delay(100);
        }

You have to place Firebase.ready() in loop as I wrote this in the comment in all library examples which you should read it carefully.

I also wrote the comment of Firebase.reconnectNetwork in all examples comment too, please take a look.

[peterz0310]

Thanks so much. I updated the Arduino core, made the changes you recommended, and it worked. I was making the mistake of not including just Firebase.ready() in the loop, because I didn't understand the point of having a function there that (to me) just returned a boolean. But it was needed. The examples helped massively too.

Thanks for all the hard work you do with this library, we appreciate it :)