1341406 - Different (non-orange) colour for tc-M-V Valgrind fails on Treeherder

Status: RESOLVED DUPLICATE of bug 1314796

(Tree Management :: Treeherder, defect)

Description

[Julian Seward [:jseward]]

It is possible to run Mochitests on Valgrind on Treeherder, using the
syntax "-b o -p linux64 -u mochitest-valgrind -t none".  This is in
the process of being automated, with twice daily runs -- see bug 1288993.

One problem with this is that the test runs can fail either due to
Valgrind detecting a memory error, or because they would have failed
anyway when run "natively".  Currently treeherder doesn't distinguish
these cases, showing them both as orange.  This means that finding the
memory errors in such runs requires manually inspecting all the oranges.

As an example, here

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c9d10009c46f08470d585382a51c20517abdb93a&exclusion_profile=false&selectedJob=79073957

is a case where we have 16 out of the 40 chunks orange, but only 4 of
them (chunks 2, 6, 13, 14) are valgrind errors.  Having to sift through
all orange chunks twice a day is going to be a bit annoying.

It would be useful if the runs that give valgrind errors could show a
different colour, or otherwise be differently marked.

Comment 1

[Chris Cooper [:coop] (he/him)]

Found in triage.

Ed: is this achievable? Are there any colors left?

Comment 2

[Ed Morley [:emorley]]

There are a handful of Taskcluster states which map to treeherder states (there isn't a perfect mapping; ideally this will be improved in the future).

There is presumably already a taskcluster state that would get you a colour you want, but I have no idea what that is.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

One option would be to write an artifact from the task that contains a color ("colour"?) field, then have Treeherder read that file.

Basically, I think this is a question of interface between tasks and treeherder, with TC (and pulse, artifacts, etc.) only being the communication channel.

Comment 4

[Ed Morley [:emorley]]

I would prefer the job status to be determined by harness/exit code/... rather than out of bound? Treeherder fetching external files would need to be a background task that runs after initial job ingestion, therefore delaying when we can actually display the job (since we need the result to display it).

Comment 5

[Ed Morley [:emorley]]

Treeherder has a mapping of Taskcluster result states to colours. If we want a particular failure mode to appear as an infra failure or something else, it will need to be handled by bug 1314796 rather than in Treeherder.