-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
edr send-report ignores custom schema #1462
Comments
Hey @adamcunnington-mlg, you are right, the edr commands are using only the values from your profiles.yml. My suggestion is to have 2 targets for the elementary profile you create:
Then when you run edr on prod pass the relevant target that points to the right schema 🙂 |
Thanks for the response. As I said in slack, for benefit of those looking at this issue, please can I suggest:
|
@adamcunnington-mlg Yep, came across this problem today as well using Prefect. Should definitely been explained in the docs. |
Describe the bug
I use a pretty typical custom schema setup (a pattern described in dbt docs) whereby dbt invocations against the target
prod
use the schema set in my project's yml files (the custom schema) but in non-prod, the target schema is used (default schema) and the custom schema is ignored.My target schema is TRANSFORMED_elementary. I never want this to be used in prod.
My custom schema is DBT_ELEMENTARY.
In prod, the dbt invocations reflect this and edr tables correctly build in the DBT_ELEMENTARY dataset but edr send-report seems to not. I don't know if this effects other edr commands too. When the edr send-report command runs, it tries to populated tables in TRANSFORMED_elementary that don't exist. I thought this might be because the target name wasn't respected (but I guess you use the default set for the elementary profile) and so I adapted the custom schema macro logic to explicitly use the custom schema (DBT_ELEMENTARY) when target.profile_name = elementary but it has no effect.
My conclusion is edr send-report only uses the values from profiles.yml and so it's bypassing/ignoring my override within the dbt macro. That doesn't sound quite right though because surely edr send-report is running a dbt run command under the hood?
Please confirm my understanding. Is there a workaround? Seems like a big omission so I assume I'm missing something!
The text was updated successfully, but these errors were encountered: