Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gsheet: updated connection returns 0 row #68

Open
yamhirotoCS opened this issue Dec 8, 2023 · 0 comments
Open

Gsheet: updated connection returns 0 row #68

yamhirotoCS opened this issue Dec 8, 2023 · 0 comments

Comments

@yamhirotoCS
Copy link

using provider 0.3.7.

Whenever I update some variables related to a gsheet source.
there isn't any Terraform fail : resource and connection get updated successfully.
but the sync returns 0.
The only way to make it work is to first get it destroyed (by commenting my code) and recreate it from scratch. Then it will sync data correctly.

Here is the log I get, I can see the right row counts but it doesn't do anything right after.

2023-12-08 09:42:29 source > Row counts: {'MY_GSHEET': 2128, 'Importrange_References': 4}
2023-12-08 09:42:29 source > Finished syncing spreadsheet 12hW9u2QSLNjwdAdNoJWGSCMFaODArnEHBHi9STiodkY
2023-12-08 09:42:29 platform > Total records read: 0 (0 bytes)
2023-12-08 09:42:29 platform > Schema validation was performed to a max of 10 records with errors per stream.
2023-12-08 09:42:29 platform > readFromSource: done. (source.isFinished:true, fromSource.isClosed:false)
2023-12-08 09:42:29 platform > processMessage: done. (fromSource.isDone:true, forDest.isClosed:false)
2023-12-08 09:42:29 platform > thread status... heartbeat thread: false , replication thread: true
2023-12-08 09:42:29 platform > writeToDestination: done. (forDest.isDone:true, isDestRunning:true)
2023-12-08 09:42:29 platform > thread status... timeout thread: false , replication thread: true

whereas when I recreate it 👍

2023-12-08 02:09:07 source > Starting syncing spreadsheet 12hW9u2QSLNjwdAdNoJWGSCMFaODArnEHBHi9STiodkY
2023-12-08 02:09:08 source > Available sheets: ['MY_GSHEET', 'Importrange_References']
2023-12-08 02:09:10 source > Row counts: {'MY_GSHEET': 2128, 'Importrange_References': 4}
2023-12-08 02:09:10 source > Syncing sheet MY_GSHEET
2023-12-08 02:09:11 source > Fetching range MY_GSHEET!2:202
2023-12-08 02:09:13 source > Fetching range MY_GSHEET!203:403
2023-12-08 02:09:14 source > Fetching range MY_GSHEET!404:604
2023-12-08 02:09:15 destination > 2023-12-08 02:09:15 INFO i.a.c.i.b.IntegrationCliParser(parseOptions):126 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2023-12-08 02:09:16 source > Fetching range MY_GSHEET!605:805
2023-12-08 02:09:16 destination > 2023-12-08 02:09:16 INFO i.a.c.i.b.a.AdaptiveDestinationRunner$Runner(getDestination):79 - Running destination under deployment mode: OSS
2023-12-08 02:09:17 destination > 2023-12-08 02:09:16 INFO i.a.c.i.b.a.AdaptiveDestinationRunner$Runner(run):100 - Starting destination: io.airbyte.integrations.destination.snowflake.SnowflakeDestination
2023-12-08 02:09:17 source > Fetching range MY_GSHEET!806:1006
2023-12-08 02:09:17 destination > 2023-12-08 02:09:17 INFO i.a.c.i.b.IntegrationCliParser(parseOptions):126 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2023-12-08 02:09:17 destination > 2023-12-08 02:09:17 INFO i.a.c.i.b.IntegrationRunner(runInternal):132 - Running integration: io.airbyte.integrations.destination.snowflake.SnowflakeDestination
2023-12-08 02:09:17 destination > 2023-12-08 02:09:17 INFO i.a.c.i.b.IntegrationRunner(runInternal):133 - Command: WRITE
2023-12-08 02:09:17 destination > 2023-12-08 02:09:17 INFO i.a.c.i.b.IntegrationRunner(runInternal):134 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2023-12-08 02:09:19 source > Fetching range MY_GSHEET!1007:1207
2023-12-08 02:09:19 destination > 2023-12-08 02:09:19 WARN c.n.s.JsonMetaSchema(newValidator):278 - Unknown keyword pattern_descriptor - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-12-08 02:09:19 destination > 2023-12-08 02:09:19 WARN c.n.s.JsonMetaSchema(newValidator):278 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-12-08 02:09:19 destination > 2023-12-08 02:09:19 WARN c.n.s.JsonMetaSchema(newValidator):278 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-12-08 02:09:19 destination > 2023-12-08 02:09:19 WARN i.a.c.i.b.DestinationConfig(initialize):36 - Singleton was already initialized.
2023-12-08 02:09:20 source > Fetching range MY_GSHEET!1208:1408
2023-12-08 02:09:21 source > Fetching range MY_GSHEET!1409:1609
2023-12-08 02:09:23 source > Fetching range MY_GSHEET!1610:1810
2023-12-08 02:09:24 source > Fetching range MY_GSHEET!1811:2011
2023-12-08 02:09:24 source > Finished syncing spreadsheet 12hW9u2QSLNjwdAdNoJWGSCMFaODArnEHBHi9STiodkY
2023-12-08 02:09:27 platform > Total records read: 1615 (964 KB)
2023-12-08 02:09:27 platform > Schema validation was performed to a max of 10 records with errors per stream.
2023-12-08 02:09:27 platform > readFromSource: done. (source.isFinished:true, fromSource.isClosed:false)
2023-12-08 02:09:27 platform > thread status... heartbeat thread: false , replication thread: true

To reproduce my error:

  • create a gsheet source + connection with the terraform provider
  • update names_conversion = false to names_conversion=true
  • apply the update
  • try to sync.

It's might be related to this error as well :
#31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant