Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spark load fail when use hudi table for origin table #45317

Open
blanklin030 opened this issue May 8, 2024 · 0 comments
Open

spark load fail when use hudi table for origin table #45317

blanklin030 opened this issue May 8, 2024 · 0 comments
Labels
type/bug Something isn't working

Comments

@blanklin030
Copy link
Contributor

Steps to reproduce the behavior (Required)

LOAD LABEL db.linyu_20240424_hudi2sr (
DATA FROM TABLE hudi_table
INTO TABLE linyu001_hudi_2_sr_test
TEMPORARY PARTITION(temp__p20210102)
SET (
`_hoodie_commit_time` = `_hoodie_commit_time`,
 `_hoodie_commit_seqno` = `_hoodie_commit_seqno`, `_hoodie_commit_seqno`),
 `_hoodie_record_key` =`_hoodie_record_key`,
 `_hoodie_partition_path` = `_hoodie_partition_path`,
 `_hoodie_file_name` = `_hoodie_file_name`,
 `dt` = `dt`,
 `id` = `id`,
 `name` =  `name`,
 `age` =  `age`,
 `ts` = `ts`
) 
WHERE (`dt` = '20210102')
)WITH RESOURCE 'xxxxx' (
  "spark.yarn.tags" = "h2s_foit_linyu20240422abc001",
  "spark.dynamicAllocation.enabled" = "true",
  "spark.executor.memory" = "3g",
  "spark.executor.memoryOverhead" = "2g",
  "spark.streaming.batchDuration" = "5",
  "spark.executor.cores" = "1",
  "spark.yarn.executor.memoryOverhead" = "4g",
  "spark.speculation" = "false",
  "spark.dynamicAllocation.minExecutors" = "10",
  "spark.dynamicAllocation.maxExecutors" = "100"
) PROPERTIES (
  "timeout" = "72000",
  "spark_load_submit_timeout" = "36000"
)

Expected behavior (Required)

return load success

Real behavior (Required)

Unexpected exception: Source table hudi_table is not HiveTable ### The error may exist in

StarRocks version (Required)

2.5.12

@blanklin030 blanklin030 added the type/bug Something isn't working label May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant