1779188 - Make it easier to see possible intermittent failure, after retriggering jobs

Status: RESOLVED FIXED

(Tree Management :: Treeherder, enhancement)

Description

[Tooru Fujisawa [:arai]]

While testing a patch, I often hit possible intermittent failure, and in that case what I do is:

1. retrigger failed jobs
2. expand green jobs
3. check if the retriggered job fails or not, for the failed job, by seeing the occurrences of "orange + orange" or "orange + green" pair in the result

step 2 is necessary because I often forget to retrigger some failed jobs, and counting the number of failed jobs for the same job name doesn't fit for the purpose.

it would be nice if this can be automated, or somehow assisted.

what I can think of are:

• (a) add a filter that shows "failed jobs, and all jobs that failed at least once"
• (b) somehow mark failed jobs that doesn't fail in other run
• (c) add a filter that shows "jobs fails and never passes"

Comment 1

[Tooru Fujisawa [:arai]]

Attachment: Mark possible intermittent failure

This is an UI prototype for (b).
Failed jobs where other job for the same job name succeeds are marked with a mitten (just an example).

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/7491