1174373 - Unhide b2g desktop Gip(a) when its failure rate meets visibility requirements

Status: RESOLVED WONTFIX

(Tree Management Graveyard :: Visibility Requests, defect)

Description

[Phil Ringnalda (:philor)]

Because, unlike the tests on every other platform, the TaskCluster-run tests on b2g do not report a failure when a test fails in the first run on a push, they retry, and they do not report a failure when a test fails on the second run on a push, they retry, and they only report a failure when a test (the same test or a different one, the same failure or a different one, doesn't matter) fails on the third run on a push, it's more difficult to determine the failure percentage for b2g tests.

The most generous measure would be to only say that there was a failure when there was a failure on the third run.

The most reasonable measure would be to say that there was a failure when there was a failure on the first run.

The most likely measure would be to say that there was a failure when there was a failure, that failing three runs on a push means failing three times.

By the most generous measure, Gip(a) is failing 50% of the time. By the most reasonable measure, it is failing 90% of the time. By the most likely measure, it is failing 80% of the time.

None of those are within miles of the 1% to 5% that are actually acceptable according to https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy#Low_intermittent_failure_rate

Hidden on all trunk trees.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

WONTFIX in favor of bug 1180903.