🔥 Hot Fix - Firebase - live Updates issues

[jonsnowpt]

🐛 Introduction:

Real-time updates on fixtures incidents, scores, and other events should update in real-time.

📝 Details:

Currently, when using the PWA live score updates on the live score table, fixtures and other platform parts with real-time updates stop updating. Making it necessary to refresh the page or restart the PWA.

🔍 Steps to Reproduce:

Use the mobile version of the platform with the PWA and Safari browser. These issues were detected on both versions.

🤔 Expected Behavior:

Real-time updates when the browser is opened or when coming back to the browser to check updates, the same for the PWA version.

💥 Actual Behavior:

PWA stops updating, and in the mobile version in the browser, the same happens with less intensity.

[migbash]

NOTES

@jonsnowpt require more information as cannot replicate the issue.

@jonsnowpt need more recordings of the issue.

---

NOTES

1. Potential issue with the data failing to update when the user is switching between apps on mobile. ⏸️ pending | recordings of DEV deployment.

[jonsnowpt]

@migbash

Added the video recordings and logs to Slack, which does not seem to be catching any issues, but it still gets stuck in updating the real-time data.

[jonsnowpt]

@migbash

Sentry has a few errors with my IP; maybe it helps to sort out this issue:

IP: 95.136.72.210

[jonsnowpt]

@migbash

Detected error:

Blocked a frame with origin "https://scores.betarena.com" from accessing a frame with origin "https://betarena-ios.firebaseapp.com". Protocols, domains, and ports must match.

[jonsnowpt]

@migbash

I changed a setting on Cloudflare and for now, this error is not being shown.

[migbash]

NOTES

Addition of Content Security Policy for the application has been added, as the initially thought issue, due to it's effect on Safari strict security. Improving the Ensure CSP is effective against XSS attacks point of Lighthouse.

[jonsnowpt]

@migbash

The issue persists, and we lose the WebSocket connection.

There are several reasons why Firebase WebSockets might stop working. Here are some possible causes:

Network issues: Firebase WebSockets rely on a stable network connection between the client and the Firebase servers. If there is a network outage, poor network performance, or a disruption in connectivity, the WebSocket connection may stop working.

Server issues: Firebase WebSockets rely on the Firebase Realtime Database servers to function properly. If there are any issues with the servers, such as maintenance, downtime, or an outage, the WebSocket connection may stop working.

Connection limits: Firebase imposes connection limits for Realtime Database and Firestore, which can cause WebSocket connections to stop working if these limits are exceeded. If you have multiple clients connecting to the same database, it's important to manage the connections properly and ensure that you are not exceeding the connection limits.

Authentication issues: If there are any issues with the Firebase Authentication service, this can affect WebSocket connections. For example, if a user's authentication token expires or becomes invalid, the WebSocket connection may stop working.

Client-side issues: Issues with the client-side code, such as bugs or incorrect configuration settings, can also cause WebSocket connections to stop working.

To troubleshoot this issue, you can try the following:

Check the Firebase status page to see if there are any known issues or outages.

Check your network connection and ensure that there are no issues with connectivity.

Check your client-side code and ensure that it is properly configured and functioning as expected.

Monitor the Firebase connection limits to ensure that you are not exceeding them.

Check the Firebase Authentication service and ensure that there are no issues with authentication.

If none of these solutions work, you can contact Firebase support for further assistance.

In our case, the probable culprit should be a code issue.

I have confirmed that we are within the connection limits.

[jonsnowpt]

#1108 (comment)

@migbash

Has this been sent to PROD?

[jonsnowpt]

@migbash

Maybe this technique might be useful:

Keep connections alive using server-side techniques: To keep Firebase WebSocket connections alive from the server-side, you can use techniques such as sending keep-alive messages or using WebSocket ping/pong frames. By sending periodic messages to the client, you can ensure that the WebSocket connection stays open and prevent it from being closed due to inactivity.

Here is an example of how to use server-side techniques to keep Firebase WebSocket connections alive. In this example, we use a Cloud Function to send keep-alive messages to the client every 30 seconds:

const functions = require('firebase-functions');
const admin = require('firebase-admin');

admin.initializeApp();

exports.keepAlive = functions.database.ref('/keepAlive').onUpdate((change, context) => {
  // Get the Firebase Realtime Database instance
  const database = admin.database();

  // Get the current timestamp
  const timestamp = new Date().getTime();

  // Update the "lastSeen" value for each connected client
  database.ref('/status').once('value', (snapshot) => {
    const status = snapshot.val();

    for (const key in status) {
      if (status.hasOwnProperty(key)) {
        const client = status[key];

        // Only update the "lastSeen" value if the client is connected
        if (client.connected) {
          database.ref(`/status/${key}/lastSeen`).set(timestamp);
        }
      }
    }
  });

  // Schedule the next keep-alive message in 30 seconds
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve();
    }, 30000);
  });
});


`

In this code, we define a Cloud Function that is triggered whenever the /keepAlive value is updated in the Firebase Realtime Database. When this function is triggered, it updates the lastSeen value for each connected client in the /status node of the database. This keeps the WebSocket connection alive by sending messages to the client every 30 seconds.

Note that this code assumes that you have a /status node in your Firebase Realtime Database that tracks the status of each connected client, and that you are updating this node whenever clients connect or disconnect. You may need to adjust the code to fit your specific use case.

[jonsnowpt]

Firebase updates do not work correctly when closing the devices and getting back to them later.

RPReplay_Final1682780619.MP4

[jonsnowpt]

[migbash]

NOTES

• WebSockets timeout and long wait time before fallback to long polling firebase/firebase-js-sdk#540
• [firebase_auth]: signInWithPopup fails with "Blocked a frame with origin from accessing a cross-origin frame." firebase/firebase-js-sdk#4202
• Connection fails and never recovers firebase/firebase-js-sdk#6537
• https://stackoverflow.com/questions/12421993/firebase-websocket-is-closed-before-the-connection-is-established

---

NOTES

❗️ Potential cause for issues:

► Blocked a frame with origin "https://betarena-scores-platform.herokuapp.com" from accessing a frame with origin "https://betarena-ios.firebaseapp.com". Protocols, domains, and ports must match.

🔍 CAUSE

firebase/auth module is responsible for this errors:

Proof

Proof never load erroneous - iframes | 2

• Firebase v9 loads a large iframe.js file but only on mobile firebase/firebase-js-sdk#4946
• [firebase_auth]: signInWithPopup fails with "Blocked a frame with origin from accessing a cross-origin frame." firebase/firebase-js-sdk#4202

// NOTE: Initialize Firebase Authentication and get a reference to the service;
export async function initAuth
(
): Promise < Auth >
{
  const { getAuth } = await import ('firebase/auth');
  const auth = getAuth(app);
  return auth;
}

NO VIABLE SOLUTION as of yet.
❗️ NOTE: does not inflict any potential issue on the platform.
Only happens in strict-er browsers with Content-Security Policies that are tougher, such as: Safari.

---

2. (pending 1. testing completion)

Data from the Realtime DB happens to be wrong / out of date, at times.

Realtime Incoming data

Expected (example) of actual data on other instances of the scores platform

RESOURCES

• https://stackoverflow.com/questions/69288486/firebase-realtime-db-query-returns-an-old-datasnapshot-and-not-new-keys-unle
• https://stackoverflow.com/questions/55041761/clear-firebase-cache-with-keepsyncedtrue-not-working
• https://stackoverflow.com/questions/66608771/firebase-get-vs-once-what-is-the-difference

(alternative) A second solution is to use the new get method (introduced in early 2021), which doesn't have this problematic behavior. Note that this method always tries to first fetch the value from the server, so it will take longer to completely. If your value never changes, it might still be better to use addListenerForSingleValueEvent (but you probably wouldn't have ended up on this page in that case).

Stackoverflow (source)

alternative Unfortunately the offline behaviour in the Realtime Database is "build" in without much control on how it works. With Firestore you could detect with the metadata if data is comming from the cache or server. Maybe a migration to that database would be a solution.

Stackoverflow (source)

---

3. (2. continued)

Realtime DB looses connection with the Firebase.

RESOURCES

• Safari dropping web socket connection due to inactivity when page not in focus socketio/socket.io#2924
• Database emulator query.once hangs firebase/firebase-tools#4579

explanation It is expected that apps will lose their socket connections when there is no visible activity. That's the way the Firebase SDK works. Also, Android will aggressively pause background applications so they don't consume data and battery when the user isn't actively using it.

Stackoverflow (source)

explanation This is and iOS feature that protects users against code draining their battery...
Stackoverflow (source)

REPLICATE (CONSISTENTLY)

• https://discussions.apple.com/message/25254749#25254749
• https://stackoverflow.com/questions/22585558/how-can-one-suppress-automatic-refresh-of-open-browser-windows-on-the-iphone-ip

In Safari or another browser app, touch and hold on a piece of text (a single word or letter is fine), then lift your finger off the screen. The word or letter will be selected and the familiar little pop-up that says "Copy | Define" appears. Touch "Define" and the screen that shows the word's definition or says it couldn't find a definition pops up. Now, do not touch anything else; not "Done", "Search the Web", or anything else. Now press the Home button (i.e., the one on the bottom portion of the front face panel of the iPhone with the square in it). You will be switched back to the springboard with all your app icons and from there can do anything else you want. Then, touch the browser icon for the browser you were in and you'll switch back into it. The definition screen covering the browser window will still be there. Touch "Done", and the screen drops away. Blessedly, the open browser window(s) will not reload.

---

Inquiry made to:

• https://discord.com/channels/626375663313748008/1109377198328123393
• https://firebase-community.slack.com/archives/C3U4MSE0Y/p1684567253537699
• https://stackoverflow.com/questions/76294230/realtime-db-is-not-responding-to-requests-firebase
• Firebase Support Dev Team (email)

[jonsnowpt]

@migbash

Still not working.

Desktop issues also:

[jonsnowpt]

@migbash

It stopped and Im unable to click anything, the script was broken.

https://github.com/Betarena/scores/assets/37311649/2e26e323-174d-4f1c-9dae-ef400f9cc073

[meropis]

Some more information I'd like to put forward if it helps is that:

1. In order to help with debugging, this error persists for any browser that uses the webkit renderer. You can install a browser such as epiphany on linux where you can debug and test far more easily than loading up an IOS emulator etc;
2. If you are running a localhost development environment you'll find that when the error is thrown, if you have an existing auth object saved you can simply hot reload the environment (I use vite) and the auth will work correctly, even in an iframe;
3. CSP errors are fixable in most cases by using window.postMessage to make calls to external windows e.g. iframes. This is apparently not a possibility for this error after testing on my end. It is possible that someone with more experience may be able to find a way to pass the auth redirect back using postMessage and avoiding the CSP error, but I was unable to implement this.
   I hope some of that will help in some way.

[jonsnowpt]

Thanks @meropis

It seems that the issue was on iOS and Firebase. In the new iOS 17 the issue ceased to exist.