1298319 - Intermittent toolkit/mozapps/extensions/test/mochitest/test_bug609794.html | application ran for longer than allowed maximum time

Status: RESOLVED DUPLICATE of bug 1204281

(Toolkit :: Add-ons Manager, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=34727586&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/Kafdq-HrTKKdR4BcEucCLw/runs/0/artifacts/public%2Flogs%2Flive_backing.log

Comment 1

[BMO Automation]

Bulk assigning P3 to all open intermittent bugs without a priority set in Firefox components per bug 1298978.

Comment 2

[Andy McKay]

Hasn't happened for a month.

Comment 3

[Phil Ringnalda (:philor)]

That turns out not to be the case: you need to look at https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298319&entireHistory=true&tree=trunk to see when it happens, rather than at bug comments, since the stupid commenting bot was stupidly rewritten to stupidly only comment in failures that happen 5 or more times per week, as though it somehow wasn't stupid to say that less than that just didn't ever happen.

And the resolution you wanted there was "worksforme" rather than "wontfix" which says that even if someone wrote a patch which would fix the test failure, you would not accept it.

Comment 4

[:shell escalante]

Hi Phil,
Following up on your comment 3 (which made me laugh - bugs are too boring sometimes).  We triage these intermittents all the time and really do look at the tree herder count to determine priority.  

Unless we see a high count/re-occurrence in the bug, we primarily look at orange factors all up for any of our failures that are high on that list.  

Is there someone who determines what tree herder updates in the bugs?  and if that should be changed to not only flag 5 in a week?  I don't know enough about the frequency to know what drives that logic - but we use that logic to make "worksforme" decisions.  

1/4 of our incoming bugs are triggered by treeherder - and most are 1 time notices that don't re-occur (according to bugzilla).  So unless they stand out in orange factor all up dashboard - we don't note them.

Comment 5

[Phil Ringnalda (:philor)]

In https://bugzilla.mozilla.org/userprefs.cgi?tab=settings toggle on "When viewing a bug, show its corresponding Orange Factor page" so that you get a "Orange Factor: 3 failures on trunk in the past week (link)" stuck in the bug page for intermittent-failure bugs, where the link goes to https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298319 which has an "entire history" link at the top. For things which failed at least once in the last week, you don't even need to click through, you already know; for things that haven't, two clicks and a page-down gets you to the most recent ones.

Bug 1179821 was what decided that <5 per week never happened. The reason you'll find me doing reopens like in this bug when it turns out that <5 per week actually *do* happen is because treeherder's bug suggestions for a failure come in two classes: the visible by default open-active where the bug is open and has been modified in the last 5 months, and closed-inactive where either the bug is closed or was last modified more than 5 months ago, for which you have to know to click the "Show/Hide more" link to see suggestions, and then probably have to open the bug to see that it is resolved wontfix because our crappy notifications made someone think it had stopped happening.

Comment 6

[Geoff Brown [:gbrown]]

We resolved many 2016-09 Android "application ran for longer..." bugs by increasing the number of test chunks for Android mochitests in bug...