Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When kill the DBT Cli, Job still running on Databricks #644

Open
gaoshihang opened this issue Apr 22, 2024 · 1 comment
Open

When kill the DBT Cli, Job still running on Databricks #644

gaoshihang opened this issue Apr 22, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@gaoshihang
Copy link

gaoshihang commented Apr 22, 2024

Describe the bug

When we run "dbt run" to submit jobs to Databricks cluster, then we kill the DBT CLI, what we want is job killed on Databricks, but actually they will still running on Databricks.

It seems like dbt-databricks adaptor doesn't support query cancellation.

image
image

And in DBT fail-fast feature, it also has problem, related to this issue: dbt-labs/dbt-core#10057

Expected behavior

When the DBT CLI exits, all submitted jobs will be killed too.

@gaoshihang gaoshihang added the bug Something isn't working label Apr 22, 2024
@gaoshihang
Copy link
Author

Hi @benc-db , could you help to look at this issue? thank you very much! I think query cancellation is an important part in DBT.

@gaoshihang gaoshihang changed the title When kill the DBT Cli, Job still on Databricks When kill the DBT Cli, Job still running on Databricks Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant