Closed Bug 843738 Opened 8 years ago Closed 8 years ago

Extremely frequent Android ARMv6 Reftest-2 TEST-UNEXPECTED-FAIL | <some test> | application ran for longer than allowed maximum time

Categories

(Core :: Canvas: WebGL, defect)

ARM
Android
defect
Not set
critical

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: RyanVM, Unassigned)

References

Details

(Keywords: intermittent-failure)

The landing of bug 716859 has led to Android ARMv6 Reftest-2 failing with roughly 80% frequency due to running for longer than the harness permits. This suite contains the WebGL tests among other things.

From my looking into this, it appears that the suite has become bimodal with the passing runs typically finishing in around 60 minutes (normal for this suite) and the orange runs being killed around 80 minutes. There are occasionally green runs that finish just short of 80 minutes.

There are a few possible options for where we go from here:
1.) Change the harness to work around this. We can increase the timeout threshold or change the chunking.
2.) Back out bug 716859.
3.) Disable the webgl reftests on Android (not sure if we can disable on just ARMv6).

I would prefer option 2 as it's an obvious regression from the code that landed in that bug. However, Jeff and Vlad have voiced their extreme displeasure with the idea of backing this out (again).

I'm personally opposed to making changes to the test harness to work around this as it seems like we're papering over a real regression and it removes any real incentive to investigate and fix whatever regressed.

Therefore, I'm proposing we go with Option 3 for now until the cause of the regression can be located and fixed.
Depends on: 843733
Depends on: 843728
No longer depends on: 843733
In bug 843728, jmaher proposed landing the temporary increase for now with a 2-26 backout deadline.
I'm strongly against increasing the timeout.
I've relanded bug 716859 with webgl disabled on ARMv6
Resolving WFM keyword:intermittent-failure bugs last modified >3 months ago, whose whiteboard contains none of:
{random,disabled,marked,fuzzy,todo,fails,failing,annotated,time-bomb,leave open}

There will inevitably be some false positives; for that (and the bugspam) I apologise. Filter on orangewfm.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.