Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Question: DQ over time #554

Open
jonathanapp opened this issue Apr 8, 2024 · 0 comments
Open

Question: DQ over time #554

jonathanapp opened this issue Apr 8, 2024 · 0 comments
Labels
question Further information is requested

Comments

@jonathanapp
Copy link

We have years of historical data in addition to (daily) updated data streams.
Can I use Deequ to view metrics over time? For us, data quality is a trajectory, not a point. Based on the examples I see, I'd have to create hundreds (thousands, actually) of DataFrames for each day of data and run the analysis for each. Is there no way to run the metrics for all my data and disaggregate by date?
TIA.
@jonathanapp jonathanapp added the question Further information is requested label Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jonathanapp