[Feature][Transform] Add split multiple source table in transform

[liunaijie]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Description

Now we supported read multiple table in source, and the result_table_name can only be one table. so the transform/sink can't only choose one table that needed. we need implement the split function to split the source multile tables.

One user case is:
read multiple table in source, and we need do some different table transform with different table, and write it to one/multiple sinks.

the example config is look like this:

source  {
  read_tables = [tableA, tableB, tableC]
}

transform {
   Sql {
       source_table_name = "tableA"
        result_table_name = "tableA_sql"
       query = "select xxxx from tableA"
   }
 
   Sql {
       source_table_name = "tableB"
        result_table_name = "tableB_sql"
       query = "select xxxx from tableB"
   }

   Sql {
       source_table_name = "tableC"
        result_table_name = "tableC_sql"
       query = "select xxxx from tableC"
   }

}


sink {
   
  LocalFile {
      source_table_name = "tableA_sql"
  }
 
  MySQL {
      source_table_name = "tableB_sql"
  }
 
  Hive {
      source_table_name = ["tableA_sql", "tableB_sql", "tableC_sql"]
  }

}

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[EricJoy2048]

Here is a pr for this feature #5646. But the contributor is busy with other work and can not do it continue.