You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ config(
pre_hook=['
function custom_df(input)
# do some logic
return output
spark.udf.register('custom_df', custom_df)
']
) }}
select custom_df( x ) from {{ ref('my_table') }}
or
{{ config(
pre_hook=['dbfs:/scripts/init_functions.py']
) }}
select custom_df( x ) from {{ ref('my_table') }}
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Describe the feature
A clear and concise description of what you want to happen.
Using Spark UDF From DBT will be helpful.
As discussing in dbt-spark, Something like using pre_hook will be helpfull.
dbt-labs/dbt-spark#135 (comment)
or
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Using DBT Python models. Ideally want to use udf from DBT SQL model
https://docs.getdbt.com/docs/build/python-models
https://docs.databricks.com/en/workflows/jobs/how-to/use-dbt-in-workflows.html#advanced-use-dbt-python-models-in-a-workflow
Additional context
Please include any other relevant context here.
Who will this benefit?
What kind of use case will this feature be useful for? Please be specific and provide examples, this will help us prioritize properly.
Some one who want to use UDF.In our company proceed migration from Oracle PL/SQL to Databricks. If we can use udf some function will be easy to migrate.
https://www.databricks.com/blog/how-migrate-your-oracle-plsql-code-databricks-lakehouse-platform
Are you interested in contributing this feature?
Let us know if you want to write some code, and how we can help.
Yes
The text was updated successfully, but these errors were encountered: