Metabase Sync as Target

[tjworks]

This is not a general target. It is intended for Tapdata + Metabase integration.

User Story 1:

• With the help of Tapdata, Max synced some data from CRM system to a MySQL database for analytical purpose, lets say Mysql database is called "MySQL_Warehouse"
• Max also has a metabase installation for BI dashboards. He added MySQL_Warehouse as a database in metabase
• Metabase would scan the MySQL db and collect the schema, so Max can create dashboards.
• In Tapdata, Max decided to add a new data type, Contracts, into MySQL_Warehosue
• After successful task completion in Tapdata, MySQL_Warehouse now has a new table "Contract"
• Max now needs to go to Metabase, Administrative interface, find the MySQL_Warehouse, and click "Referesh Schema"

This Metabase Sync Target would replace above last step, whenever there is a schema change to a table or database, this target will automatically send an api call to Metabase to trigger a schema refresh. Specifically:

If a new table is added to a database, and this database is in Metabase already, then call this api: https://www.metabase.com/docs/latest/api/database#post-apidatabaseidsync

if an existing table is changed(DDL events), then call this api: (needs double check)
https://www.metabase.com/docs/latest/api/table#post-apitableidrescan_values

This target would most likely listening on Tapdata's meta db, MetadataInstances collection for changes.

User Story 2:

• Will created a question in Metabase, named "Cloud Registrations By Day"
• One day Will noticed the Chart is missing data for last couple of days
• Will opens "Data Console", found "Metabase Service", then locate the question
• Will click on the question, then click "Trace"
• Tapdata highlights "CloudUsers" table in the MDM pane, indicates this is the base table used to draw metabase chart
• Will click on the "CloudUsers " table, it shows the sync status is error, now he knows why the data is missing

Requirements:

• retrieve metabase questions as tables
• UI: display questions in the Services section