Closed Bug 1036325 Opened 5 years ago Closed 5 years ago

Support bisection for failing intermittent mochitests

Categories

(Testing :: Mochitest, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla34

People

(Reporter: vaibhav1994, Assigned: vaibhav1994, Mentored)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

This bug is in continuation of bug 1014125 . We need to support bisection for failing intermittent tests. For this, we need to run the failing test 10 times independently and ensure it passes every time.
Assignee: nobody → vaibhavmagarwal
Depends on: 1036372
Blocks: 1036372
No longer depends on: 1036372
Attached patch intermittent.patch (obsolete) — Splinter Review
Try Run: https://tbpl.mozilla.org/?tree=Try&rev=05d069406c9d
Attachment #8461136 - Flags: review?(jmaher)
Comment on attachment 8461136 [details] [diff] [review]
intermittent.patch

Review of attachment 8461136 [details] [diff] [review]:
-----------------------------------------------------------------

looking great, and thanks for ensuring you have tested this!

::: testing/mochitest/bisection.py
@@ +259,5 @@
> +            self.summary.append("\t\tThe test %s failed." % self.contents['testsToRun'][0])
> +            self.failcount += 1
> +            self.contents['loop'] = 1
> +            self.repeat -= 1
> +            if self.failcount < 3:

right here is a hardcoded assumption which we have determined in a closed box as the right solution.  Could we make this a variable, say self.maxFailures

set it up on line 16, and then add a comment explaining how it works above where it is defined?
Attachment #8461136 - Flags: review?(jmaher) → review+
Fixed the nits. Carrying over the r+
Attachment #8461136 - Attachment is obsolete: true
Attachment #8461570 - Flags: review+
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/dc5dfa48e98d
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla34
You need to log in before you can comment on or make changes to this bug.