Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [Module Name] Can not print Job Statistic Information when Using Spark Engine #6851

Open
3 tasks done
mayunqisparkedu opened this issue May 14, 2024 · 1 comment
Open
3 tasks done
Labels
Usage Error The problems cause by usage error

Comments

@mayunqisparkedu
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

Can not print Job Statistic Information when Using Spark Engine

SeaTunnel Version

2.3.4

SeaTunnel Config

env {
  execution.parallelism = 1
  job.mode = "BATCH"
}

source {
  jdbc {
    url = ""
    driver = "com.mysql.cj.jdbc.Driver"
    connection_check_timeout_sec = 100
    user = ""
    password = ""
    query = "select * from t_table"
  }
}

transform {

}

sink {


  CosFile {
    path="/tmp/hive"
    bucket = "cosn://****"
    secret_id = ""
    secret_key = ""
    region = "ap-beijing"
    file_format_type = "parquet"
  }


}

Running Command

./bin/start-seatunnel-spark-3-connector-v2.sh --master local --deploy-mode client --config ./config/mysql_cos.conf

Error Exception

seatunnel job success done,but there no Job Statistic Information  in logs

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@EricJoy2048
Copy link
Member

Job metrics information only supported when use seatunnel zeta engine.

@EricJoy2048 EricJoy2048 added Usage Error The problems cause by usage error and removed bug labels May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Usage Error The problems cause by usage error
Projects
None yet
Development

No branches or pull requests

2 participants