[Bug] Combine mode sink only supports FIXED bucket mode, but table is DYNAMIC #3321
Open
2 tasks done
Labels
bug
Something isn't working
Search before asking
Paimon version
0.8.0
Compute Engine
flink
Minimal reproduce step
sync database from kafka topic, when the table bucket is -1,the error:java.lang.UnsupportedoperationException: Combine mode Sink only supports FIxED bucket mode, but org t main is DYNAMIC;
my command is:
flink run-application -t yarn-application
-Dyarn.application.name="paimon-load-realtime-orc-test"
-Dpipeline.name='paimon-canal-databasesync-orc-test'
-Djobmanager.memory.process.size=4096m
-Dtaskmanager.memory.process.size=4096m
-Dexecution.checkpointing.interval=60000
-Dexecution.checkpointing.timeout=60000
-Dexecution.checkpointing.tolerable-failed-checkpoints=3
-Dstate.backend.type='rocksdb'
-Dexecution.checkpointing.checkpoints-after-tasks-finish.enabled='true'
-Dpipeline.operator-chaining='false'
-Dstate.savepoints.dir='hdfs://mycluster/flink/savepoints'
-Dstate.checkpoints.dir='hdfs://mycluster/flink/checkpoints'
-Dstate.checkpoints.num-retained=100
-Dcluster.evenly-spread-out-slots='true'
/opt/bigdata/flink/lib/paimon-flink-action-0.8.0.jar
kafka-sync-database
--warehouse hdfs://mycluster/user/paimon/warehouse
--database paimon_odata
--table-prefix ""
--table-suffix ""
--kafka-conf properties.bootstrap.servers='node1:9092,node2:9092,node3:9092'
--kafka-conf topic="new_tables_canal"
--kafka-conf properties.security.protocol=SASL_PLAINTEXT
--kafka-conf properties.sasl.mechanism=PLAIN
--kafka-conf properties.sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="client" password="";'
--kafka-conf properties.group.id=paimon-canal-orc-test
--kafka-conf scan.startup.mode=earliest-offset
--kafka-conf value.format=canal-json
--catalog-conf metastore=hive
--catalog-conf uri=thrift://node2:9083
--table-conf bucket=-1
--table-conf changelog-producer=lookup
--table-conf sink.parallelism=32
--including_tables org_t_main
What doesn't meet your expectations?
cant sync table or database,the error:
Anything else?
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: