790689 - Temporarily treat reftest failures on the new tegras (>=tegra-286) as hidden

Status: RESOLVED WONTFIX

(Tree Management Graveyard :: TBPL, defect)

Description

[Ed Morley [:emorley]]

Bug 784278 is causing sheriff hassle & meaning developers are starting to ignore android results due to lack of confidence in them. We don't have any way to specify not to run reftests on the new tegras & we would prefer not to pull them from production, since they still cope with other suites ok. The conclusion from the mobile testing meeting was that we should just try to work around it for now, until the root cause is found, fixed, staged/tested & rolled out to production (at least 2-3 weeks+ away).

The hacky solution I thought of, would be to treat any reftest failures on tegras >= tegra-286 as though they are hidden in TBPL; and only show if using &noignore=1.

I've yet to look at the code, but I think we can do something similar to what is already done for hidden jobs in _filterHiddenBuilds().

If hiding only failures is problematic (due to the way it interacts with moving results from the pending/running list to finished), then we'll just hide all reftest jobs from those devices, regardless of result.

Comment 1

[Armen [:armenzg]]

Adding philor to keep him on the loop.

I know it is hacky, we don't want to do it and we will remove is asap.

Comment 2

[Armen [:armenzg]]

Preventing them from running on the releng side might be easier than what I thought (hopefully).

If bug 790698 (releng) is resolved then this one could be closed.
Please coordinate with Callek to see what could happen first.

Comment 3

[Matt Brubeck (:mbrubeck)]

If we do need to hack something on the TBPL side, maybe turning them purple would be better.  If we just hide them, then I worry that people won't notice the missing jobs on their Try pushes, for example.

Comment 4

[Ed Morley [:emorley]]

(In reply to Armen Zambrano G. [:armenzg] from comment #2)
> If bug 790698 (releng) is resolved then this one could be closed.
> Please coordinate with Callek to see what could happen first.

I think we're best off waiting for that bug, seeing as it is potentially more easily fixed in the releng side than we thought :-)

Comment 5

[Ed Morley [:emorley]]

Wontfix now that bug 790698 has worked out :-)