Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't nag for uplifts when a bug is fixed in version N and gets verified in version N+2 #2294

Open
dholbert opened this issue Dec 6, 2023 · 0 comments

Comments

@dholbert
Copy link

dholbert commented Dec 6, 2023

We landed on an interesting bugbot edge case in https://bugzilla.mozilla.org/show_bug.cgi?id=1853455#c11 . Here's what happened:

  1. A fix landed in Firefox 120, which set the status firefox120: fixed
  2. Some time passed
  3. QA verified the fix in version 122, and manually set the status firefox122: verified. So at this point we have:
firefox120: fixed
firefox121: [unset]
firefox122: verified
  1. Bugbot interprets those status flags as meaning that the fix is present in 122 and 120 but maybe not 121, and so bugbot worries that 121 (current beta) could be affected, which would be a valid concern if this were a fix that we landed on trunk in the 122 cycle and uplifted to 120 but we forgot about 121.
  2. Bugbot sets status firefox121: affected (incorrectly)
  3. Bugbot nags to see if we need an uplift to 121, since 121 is now flagged as affected. This doesn't really make sense since the patch hit trunk in version 120, so uplift to 121 is nonsensical. (Only after reconstructing the history as I have above does it become clear why bugbot is concerned with hypothetical "uplift" here.)

If things remain as they are, we will potentially hit this same issue over and over and over again, any time that a fix lands in version N and QA verifies it in version N+2 (without explicitly setting a status for version N+1).

One mitigation here could be to change QA practices to verify the fix in different versions instead or in addition, but I'm not sure that's really a good use of time. But I think we can improve bugbot to avoid this:

SUGGESTED FIX: in steps 4-5 here, bugbot should perhaps not worry about intermediate versions (121 in this case) if the Milestone flag is set to a lower version (120 in this case). The milestone indicates when the patch hit trunk, and so fixed version-status can be presumed to apply to all subsequent versions. And if version N is the "milestone" and version N+2 gets a verified status, bugbot should infer that that's simply the version that QA tested when verifying the fix, rather than inferring that we cherrypicked the fix to only land on certain versions and skipped intermediate ones.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant