Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dbt-hive can only connect to default database #73

Open
hyewonee-0 opened this issue Nov 8, 2022 · 6 comments
Open

dbt-hive can only connect to default database #73

hyewonee-0 opened this issue Nov 8, 2022 · 6 comments

Comments

@hyewonee-0
Copy link

Hi,

I am trying to connect database specified in profiles.yml.
but it seems dbt-hive that connecting default database, not using schema property.

Is there a way to connect specific database using dbt-hive?

@tovganesh
Copy link
Contributor

You should be able to use a custom schema as follows:

dbt_hive_demo:
  outputs:
    dev_cia_cdp:
      auth_type: ldap
      host: hostname
      http_path: path
      password: pwd
      port: 443
      schema: my_custom_schema
      threads: n
      type: hive
      use_http_transport: true
      use_ssl: true
      username: name
  target: dev_cia_cdp
  

@hyewonee-0
Copy link
Author

Hello @tovganesh , I added custom schema but it does not working.

And I think dbt-hive does not using custom schema property when connecting.

auth_type = "kerberos"
hive_conn = impala.dbapi.connect(
host=credentials.host,
port=credentials.port,
auth_mechanism="GSSAPI",
kerberos_service_name=credentials.kerberos_service_name,
use_http_transport=credentials.use_http_transport,
use_ssl=credentials.use_ssl,
)

I am using kerberos for auth mechanism and authenticated before running dbt debug.

Here's the profiles.yml I am using.

dbt_dev:
  outputs:
    dev:
      type: hive
      schema: schema_name
      host: hostname
      port: port
      auth_type: KERBEROS
      kerberos_service_name: hive
      username: username
      use_ssl: False
      use_http_transport: False

  target: dev

Am I missing something? or Do I have to add other properties to connect hive?
Thanks for helping.

@tovganesh
Copy link
Contributor

tovganesh commented Nov 8, 2022

The connection is created as a default connection, and later each identifier qualified with schema.model name when actual models are run.
Curious if this is causing an issue to you? Is the connection itself not been established when running dbt debug ? What error do you see?

@tovganesh
Copy link
Contributor

hi @hyewonee-0 could you update on the above?

@tovganesh
Copy link
Contributor

Hey @hyewonee-0 could you be able to provide some more details as to what error you are seeing? I would also help if you can send redacted jdbc string of the warehouse you are trying to connect.

@Janzenzzc
Copy link

Hi @tovganesh ,
I refer to the official configuration, running dbt-debug is OK.

image
image
image

But running dbt-run is getting an error, and I haven't found the solution yet, can you help me, thank you

image
image

my hive version is 1.1.0-cdh5.13.3, Is the version too low reason?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants