Error with Cal importer and missing transactions of the same charge with time proximity

[Moshet]

Hi,

I've noticed that a few transactions are missing in my export file. After investigating it, I understood that the missing transactions are ones of same charged amount and that took place shortly after another transaction in the same business:

1. 2 Bit transactions of the same amount, processed almost at the same time: only one of them is documented (happened on Amex, showed on Amex website as 2 separated transactions)
2. Same for cash withdrawal of the same amount in a couple of minutes interval (happened on Isracard, showed on Isracard website as one double transaction)
3. Same in Cal (happened in gas station in one visit)

Another thing, I have a recurring error with the Cal importer, it says "failed to fetch process date" but completes the export anyway, just without one of my credit cards.

Hope you can fix it soon:)

Thanks!

[baruchiro]

Hi, here is how we calculate the hash of transactions, and with this, we identify "duplicate" transactions:

caspion/src/backend/transactions/transactions.ts

Line 20 in 0eeaa39

return unifyHash(`${date}_${chargedAmount}_${description}_${memo}_${companyId}_${accountNumber}`);

To support hash changes and failures, we process the hash with this function:

caspion/src/backend/transactions/transactions.ts

Lines 6 to 10 in 0eeaa39

	const unifyHash = (hash: string) => hash
	.replace(/`/g, "'")
	.replace(/00\dZ/, '000Z') // Leumi: Last part of the date is sometimes 000Z, 002Z, 003Z...
	// eslint-disable-next-line no-control-regex
	.replace(/[\u0000-\u001F\u007F-\u009F\u200E]/g, ''); // Special characters

The question is, if the date of the two transactions is exactly the same?

[Moshet]

The date is the same and time difference between the transactions is few seconds-few minutes

[brafdlog]

In order to prevent duplicates, we use the data we have which is what @baruchiro showed.
If all these parameters are equal between two transactions I don't see a way to distinguish that from a duplicate transaction.
For cases where you know this happens, changing the memo/description of one transaction in the destination (google sheets/ynab) is a workaround.
If you have a suggestion on how to implement this better I will be glad to hear

[Moshet]

Every transaction has its own unique ID right? Maybe different transaction IDs can help distinguishing between different transaction. Distinguishing between transactions only by date and charged amount is basically incorrect.
If there's no such unique ID, then I suggest showing all transactions, even if they are suspiciously duplicated, because users would also like to know about duplicate transactions in order to contact the credit card issuer.