Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anomaly checks when fails #531

Open
dinjazelena opened this issue Jan 20, 2024 · 0 comments
Open

Anomaly checks when fails #531

dinjazelena opened this issue Jan 20, 2024 · 0 comments
Labels
question Further information is requested

Comments

@dinjazelena
Copy link

Hey, so when we use anomaly checks which compares DataFrame metrics to previous DataFrame.
Lets say we have batch jobs with pydeequ checks, and one of the checks failed from anomaly check. I go back repair it, but then when i rerun batch job again, it would compare it to failed metric and fail again.

How can i avoid this, or is there option to compare only to baseline DataFrame?

to sum it up:

I have monthly jobs with anomaly checks with lets say relative changes of +-20%, if it fails, job fails, i repair, but then it would compare new run to failed metric and it would fail again.

@dinjazelena dinjazelena added the question Further information is requested label Jan 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant