Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run_query doesn't support array inside named_struct #559

Open
ashwintastic opened this issue Jan 19, 2024 · 0 comments
Open

run_query doesn't support array inside named_struct #559

ashwintastic opened this issue Jan 19, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@ashwintastic
Copy link

ashwintastic commented Jan 19, 2024

Describe the bug

A clear and concise description of what the bug is. What command did you run? What happened?
Array inside struct not working throught DBT

Steps To Reproduce

In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.

This query works fine when in directly run it into databricks but not from dbt macro run_query

select col1, col2, col3, named_struct( 'K1',col3, 'K2', col2 ) map from table --col2 is an Array , col3 is string

It throws this error
Object of type ndarray is not JSON serializable> in macro run_query (macros\etc\statement.sql)

Expected behavior

A clear and concise description of what you expected to happen.

If this query runs well in databricks why it is not working throught DBT

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

The output of dbt --version:

Core:
  - installed: 1.6.6

  The latest version of dbt-core could not be determined!
  Make sure that the following URL is accessible:
  https://pypi.org/pypi/dbt-core/json

Plugins:
  - databricks: 1.6.6 - Update available!
  - spark:      1.6.1 - Update available!

The operating system you're using:

Windows 11

The output of python --version:
Python 3.10.6

Additional context

Add any other context about the problem here.

I downgraded from 1.7 to 1.6 because slowness issue in 1.7

@ashwintastic ashwintastic added the bug Something isn't working label Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant