Firestore QuerySnapshot.documentChanges(includeMetadataChanges: false) always returns DocumentChange with only metadata changes

[fanwgwg]

Description

In Firestore's QuerySnapshot#documentChanges API at

firebase-ios-sdk/Firestore/Source/Public/FirebaseFirestore/FIRQuerySnapshot.h

Line 68 in 03ce722

- (NSArray<FIRDocumentChange *> *)documentChangesWithIncludeMetadataChanges:

, it allows us to specify whether to include document changes with only metadata changes. However, from my observation, no matter this is set to true or false, it always returns document change even if there is only metadata change. This make it impossible for us to know which document change actually comes from the server.

I've tested the same piece of code in both Android and iOS. In Android, this behaviour works as expected, when I do querySnapshot.getDocumentChanges(MetadataChanges.EXCLUDE), it will return 0 documentChange if there is only metadata changed.

Reproducing the issue

For both Android and iOS, I'm using the exact same setup, listening to same collection with the same query, with includeMetadataChanges set to true.

Here's my code in iOS that processes the received QuerySnapshot and its log output when I write a new document locally:

Here's my code in Android that processes the received QuerySnapshot and its log output when I write a new document locally:

Firebase SDK Version

10.23.1

Xcode Version

15.2

Installation Method

Swift Package Manager

Firebase Product(s)

Firestore

Targeted Platforms

iOS

Relevant Log Output

No response

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

Replace this line with the contents of your Package.resolved.

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

Replace this line with the contents of your Podfile.lock!

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[fanwgwg]

I've done some more testing and found an even stranger behaviour with snapshot listeners. By default FIRQuery.addSnapshotListener shouldn't include metadata changes, however, the current behaviour of the SDK seems to be always including metadata changes.

Code on iOS and its output when I write 1 document locally:

Code on Android and its output when I write 1 document locally:

[MarkDuckworth]

It's difficult for me to determine what is happening in your screenshots because I can't see the full application or document changes. Can you provide a minimal reproducible example?

However, I tested the following and did not find any issues.

• coll.addSnapshotListener() defaults to not raising a snapshot for metadata-only changes.
• coll.addSnapshotListener(includeMetadataChanges: true) causes snapshots to be raised for metadata-only as well as document changes.
• querySnap.documentChanges(includeMetadataChanges: true) returns an array of documents that have changed since the last snapshot, and includes documents where only the metadata has changed.
• querySnap.documentChanges(includeMetadataChanges: false) returns an array of documents that have changed since the last snapshot, and excludes documents where only the metadata has changed.

Creating and sharing a minimal reproducible example can help us more easily diagnose the behavior you are experiencing.

[fanwgwg]

@MarkDuckworth Thanks for getting back. I've done some more testing and I think this might be related to precision loss for Timestamp/Date field when Firestore compares a local/cache document and a server document.

I've tried that when I update the document locally, the snapshot listener always include the metadata change only if the update contains a Date field, as long as I don't update any Date field, the snapshot listener works as expected and respects the includeMetadataChanges ` setting.

After some more digging when includeMetadataChanges is set to false, I think the reason why it still returns a metadata change is that from Firestore's point of view, the document has actually changed, because for some mysterious reason (which I assume is related to the parsing/conversion of Date and Timestamp of Firestore), there would be a a very small
difference for theDate field between a local/cache document and a server document.

Here is a basic setup:

• When I update the name field only, the snapshot listener works as expected. No metadata change is delivered.
• When I update the name field + any of the Date field, the snapshot listener will always include the metadata change.

[MarkDuckworth]

Firestore has microsecond precision, so it seems expected that the backend truncates the nanosecond-precision value you provided 161365032 to a microsecond-precision value 161365000. The SDK, storing nanoseconds, detects this difference and raises a snapshot.

What's not clear to me is how you are setting this nanosecond precision value in the SDK? I'm seeing that the SDK also truncates nano-second precision values to micro-seconds when I attempt to write a date value.

[fanwgwg]

Got it, thanks for the explanation.

Here's how I'm updating the timestamp:

self.firestoreManager.updateFSFolder(
          folderId,
          fields: [
            "name": name.trimmed,
            "lastUpdatedTimestamp": Date.now,
          ]
        )

Here's the updateFSFolder method:

  func updateFSFolder(_ id: String, fields: [String: Any]) {
    db.collection("folder").document(id).updateData(fields)
  }

[MarkDuckworth]

When I initially tested I was using Timestamp(), which was giving microsecond precision values, and therefore I was not experiencing this snapshot behavior. When I test with Date.now, I see the reported behavior.

To get a different behavior from Firestore, it might be worth generating a date or timestamp with microsecond behavior. Also, I would consider using FieldValue.serverTimestamp() to generate your timestamp from the server and not based on the client device's clock.

It's unlikely we will modify the current SDK behavior to change the snapshot behavior you are seeing. Such changes could have implications for other current and future users.

If you have a path forward, then I will close this issue.

[fanwgwg]

Thank you for your response. While I understand that changing the current behavior of microsecond precision is challenging (and I'm not proposing to change that), however, I would suggest that the iOS SDK should correctly truncate any passed Date value to microsecond precision. At the very least, we should not expect any timestamp with nanosecond precision from the SDK's own snapshot listener.

Using FieldValue.serverTimestamp() is not always an option, especially for apps designed with document conflict resolution and offline usage in mind. For example, when resolving document conflicts, we need to consider the time when the document was last modified by the user, not the time when the document reaches the server. This is because users might be offline or have a slower internet connection, and their changes should not be forfeited due to these circumstances.

Hope this makes sense!