Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSO dataset cleaner #1357

Open
ravenac95 opened this issue May 2, 2024 · 1 comment
Open

OSO dataset cleaner #1357

ravenac95 opened this issue May 2, 2024 · 1 comment

Comments

@ravenac95
Copy link
Member

Describe the feature you'd like to request

Currently, when things are removed from the models, we don't have a job that removes those views/tables from the dataset. We should clean anything that is no longer tracked in dbt. This will also mean that we need to move any source tables (from cloudquery or other kinds of data collection methods) into datasets that are not oso.

Describe the solution you'd like

We should have something that periodically cleans the models in the oso dataset.

Describe alternatives you've considered

N/A

@davidgasquez
Copy link

I remember doing something like this in my previous setup!

You can run a macro to clean models not present in dbt.

https://discourse.getdbt.com/t/clean-your-warehouse-of-old-and-deprecated-models/1547/16

There was also a package which had a dbt-something ls --outdated or similar but can't find right now.

@ryscheng ryscheng added this to the (c) Data warehouse milestone May 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog
Development

No branches or pull requests

3 participants