1108663 - Treeherder doesn't associate log output with a known intermittent

Status: RESOLVED DUPLICATE of bug 1107987

(Tree Management :: Treeherder, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

Failures filed as bug 1107870 were not associated with bug 1054793 by the treeherder ui, despite identical log output. Activity in the former bug is from before the sheriffs switched to treeherder.

Comment 1

[Chris Manchester (limited bugmail, email directly)]

The earlier bug was closed, so perhaps this is intended. Do we generally file new bugs when old intermittents start failing again?

Comment 2

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I think TBPL was more willing to suggest resolved bugs than treeherder is.

Comment 3

[Phil Ringnalda (:philor)]

Treeherder is mildly willing to, but it seems to suggest them behind a "show/hide more" link that might as well be in white-on-white Sanskrit, for all the likelihood that we'll see it and click it in any case other than "damn it, I've starred that a thousand times, I know it exists, where is it?." It does seem possible to me that th might fail to suggest closed bugs when there's nothing but closed bugs to be suggested, but I'm not sure and don't have a good test for that, offhand.

Pretty sure that both th and tbpl have a time period for last-activity after which they won't suggest, to reflect the answer to comment 1, which is that we generally file new bugs when the old bug was about some generic message ("test timed out" is more likely to get a new bug than "the frobnitz should be 1.00001, but was 1.00237"), and was truly fixed, and then became freshly broken in a way that just sounds like the previous break, and we generally reopen old bugs when they didn't actually have a chance to get old, but instead were failing once a week and the patch which thought it was going to fix that landed 8 days ago, and then it failed again.

Comment 4

[Ed Morley [:emorley]]

(In reply to Phil Ringnalda (:philor) from comment #3)
> Treeherder is mildly willing to, but it seems to suggest them behind a
> "show/hide more" link that might as well be in white-on-white Sanskrit, for
> all the likelihood that we'll see it and click it in any case other than
> "damn it, I've starred that a thousand times, I know it exists, where is
> it?."

Filed bug 1118555 for making the "show/hide more" link more visible.

> It does seem possible to me that th might fail to suggest closed bugs
> when there's nothing but closed bugs to be suggested, but I'm not sure and
> don't have a good test for that, offhand.

Ah, bug 1107987.

> Pretty sure that both th and tbpl have a time period for last-activity after
> which they won't suggest, to reflect the answer to comment 1, which is that
> we generally file new bugs when the old bug was about some generic message
> ("test timed out" is more likely to get a new bug than "the frobnitz should
> be 1.00001, but was 1.00237"), and was truly fixed, and then became freshly
> broken in a way that just sounds like the previous break, and we generally
> reopen old bugs when they didn't actually have a chance to get old, but
> instead were failing once a week and the patch which thought it was going to
> fix that landed 8 days ago, and then it failed again.

The bugs are currently bucketed as follows:
https://github.com/mozilla/treeherder-service/blob/0d5b02daf52fa9c2637309989798e9518f3a7f80/treeherder/model/derived/refdata.py#L1345

"""
 Retrieves two groups of bugs:
1) "Open recent bugs" (ie bug is not resolved & was modified in last 3 months)
2) "All other bugs" (ie all closed bugs + open bugs that were not modified in the last 3 months).
"""

This is a change from TBPL, in that it only had one bucket, and completely ignored things older than 3 months.

Bug 1107987 really screws with the usefulness of these two buckets - the expected behaviour is that if open_recent is empty, then all_others gets auto-expanded, so you don't even need to click "show/hide more". 

However I'm happy to change the bucketing entirely if people would prefer - I'd just recommend waiting until bug 1107987 lands and see how things compare then - and file a new bug with preferred algorithm.

Making this a dupe of bug 1107987, since whilst bug 1054793 was modified within the last 3 months at the time bug 1107870 was filed, it was resolved fixed, which means it went in the all_others bucket.