1219849 - Add a way to get a list of failures attribute to a backed out patch

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder, defect)

Description

[Seth Fowler [:seth] [:s2h]]

In practice I think this would mean getting a list of failures that are starred with a comment "fixed by <some commit>", where <some commit> is the backout.

It would be super cool to update the scripts we use for backing out to automatically add a link to this list of failure to the bug.

Comment 1

[Jonathan Griffin (:jgriffin)]

Yes, I think we need a more deterministic way to track the causes of backouts. This may or may not be inside Treeherder.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

we have failures at the job level which treeherder knows more about and the starring is done on, and there are actual test failures.  Technically treeherder has all the raw test failures, it is just sorting out the intermittent ones in those starred jobs vs the real failures.

Comment 3

[Ed Morley [:emorley]]

hg.mozilla.org now has a bunch of attributes for pushes/commits relating to backouts, and I think is a better place for this than treeherder.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

I am not sure how hg.mozilla.org has this information?  It has information related to backouts, but not the list of jobs that failed?  I don't know what test case, job, or platform that failed.

Comment 5

[Ed Morley [:emorley]]

Ah right, it wasn't clear that was what was being requested from the bug summary or comments.

That said, I'm not sure this is hugely actionable at the moment. Treeherder now does have the test based view that summarises failures, and that uses the graphql API, which anyone (including any backout tool) can use.