Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repeatedly synchronizing entire Invoices table #30

Open
brylie opened this issue Aug 26, 2020 · 7 comments
Open

Repeatedly synchronizing entire Invoices table #30

brylie opened this issue Aug 26, 2020 · 7 comments

Comments

@brylie
Copy link

brylie commented Aug 26, 2020

We are using Stitch-managed version of Singer with the tap-chargebee enabled and noticed that we were hitting our row limit.

The Chargebee tap is replicating all of the rows in the Invoices table repeatedly, instead of the new rows only. What can we do to prevent the tap-chargebee from replicating the entire table, and all related tables, each night?

@brylie
Copy link
Author

brylie commented Aug 26, 2020

Specifically, does this tap support INCREMENTAL_UPDATE?

@cb-nandita
Copy link

@brylie,
Hope you are doing well.
Could you please check and confirm if there are any errors in the Stitch logs? Are all the syncs successful ?

@brylie
Copy link
Author

brylie commented Dec 15, 2020

I will re-enable the Invoices table sync.

@brylie
Copy link
Author

brylie commented Dec 17, 2020

I re-enabled the invoices table sync in Stitchdata.com and the row count skyrocketed:

image

There do not appear to be any errors in the replication log. I will check again tomorrow to see if we have the same row replication issue.

@brylie
Copy link
Author

brylie commented Dec 17, 2020

The weird thing above is it says there were 7.6 M rows loaded into the data warehouse that day, while the bar chart shows 37.5 M total rows loaded. I don't understand the discrepancy.

@brylie
Copy link
Author

brylie commented Dec 21, 2020

Here is the usage increase after re-enabling the Invoices table sync:

image

@brylie
Copy link
Author

brylie commented Dec 21, 2020

The spike in rows loaded seems to have passed, and things are back to the previous, upwards cumulative trend.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants