860870 - TBPL should distinguish between "this build/test failed due to bustage" and "this build failed due to a random orange"

Status: RESOLVED INCOMPLETE

(Tree Management Graveyard :: TBPL, defect)

Description

[Justin Lebar (not reading bugmail)]

Right now we "star" build/test failures and annotate them with human-readable text.

As a result, I don't think it's possible at the moment to automatically determine which build/test failures were due to bustage and which build/test failures were due to random orange / infra / ghosts.

If we had this data, and if in particular we knew exactly which push caused the bustage, we could start building a model of which tests are likely to be irrelevant to changes in a given file.  With this information, we might be able to be smarter about which tests we run on m-i and/or try.

Comment 1

[Ed Morley [:emorley]]

This is something that we're factoring into the design of treeherder, since I agree it will be very useful to have. Other than basic maintenance, we're not adding to TBPL any more, but I'll leave this bug open ready to move/morph to be about treeherder, once we have bugzilla components set up.

Comment 2

[Ed Morley [:emorley]]

https://github.com/mozilla/treeherder-service/blob/master/treeherder/model/models.py#L450

Comment 3

[Ed Morley [:emorley]]

Seeing as this bug keeps on being mentioned on the newsgroups, a temporary workaround would be to grep the comment (either from the TBPL DB or OrangeFactor's ES instance) for "bug [0-9]{4-6}" (for intermittents) or anything else for bustage/infra/...

Comment 4

[Ed Morley [:emorley]]

TBPL has been EOLed (bug 1054977).