Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wpt.fyi experimental runs check too lenient #25

Open
foolip opened this issue Oct 29, 2019 · 2 comments
Open

wpt.fyi experimental runs check too lenient #25

foolip opened this issue Oct 29, 2019 · 2 comments
Assignees

Comments

@foolip
Copy link
Collaborator

foolip commented Oct 29, 2019

@stephenmcgruer noticed in web-platform-tests/wpt#19975 (comment) that https://ecosystem-infra-rotation.appspot.com/ isn't reflecting the current problem with old Edge Dev results. (I assumed that it did but didn't check.)

The problem is here:
https://github.com/foolip/ecosystem-infra-rotation/blob/3a52deb7efc4caeede433b58e850623d681737aa/webapp/components/ecosystem-infra-rotation.html#L116-L125

Because the run end time is used, the check passes because the now most recent Edge Dev run started quite long after the commit itself.

Unfortunately, we don't store the commit time. web-platform-tests/wpt.fyi#310 was about this and mostly resolved by web-platform-tests/wpt.fyi#349, but it doesn't really help if the run itself starts very late, as was the case here.

Another issue here is that each run is checked individually so this check can pass even if there aren't any aligned runs.

@stephenmcgruer stephenmcgruer self-assigned this Oct 30, 2019
@foolip
Copy link
Collaborator Author

foolip commented Nov 12, 2019

Also need to check for WebKitGTK runs.

@stephenmcgruer
Copy link
Contributor

We added WebKitGTK in #28

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants