Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job sensor is aiming for inaccurate window of upstream #640

Open
arinda-arif opened this issue Oct 18, 2022 · 0 comments
Open

Job sensor is aiming for inaccurate window of upstream #640

arinda-arif opened this issue Oct 18, 2022 · 0 comments

Comments

@arinda-arif
Copy link
Contributor

Describe the bug
A daily job that depends on another daily job, does not have an accurate sensor aiming for the same data window. For example, given this case:

Job A

  • Window size: 24h
  • Truncate to: d
  • Offset: 24h
  • Schedule: every 22.00

Job B

  • Window size: 24h
  • Truncate to: d
  • Offset 0
  • Schedule: every 23.00
  • Have dependency to job A

Job B will have a sensor that is pointing to job A, with the same window. However, let's say on 2nd October run, job B sensor is checking on job A 1st October run, instead of 2nd. The impact is if Job A on 2nd October takes longer to finish (for example 23.30), job B will not wait for it to be finished as the 1st October run has been finished.

To Reproduce
Create 2 jobs with details as mentioned in the bug description.

Expected behavior
Sensors should be created accurately. Daily jobs depend on another daily job should wait for an equal data window.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant