1245297 - Decrease the WEEKLY_THRESHOLD in woo_commenter from 5 to a lower value.

Status: RESOLVED WONTFIX

(Tree Management Graveyard :: OrangeFactor, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

At the moment, an intermittent failure needs to be classified at least five times in one week for it to be posted as a comment to the bug. 

This makes it harder to keep closing old intermittent failures that just trailed off because it can catch a lot of current failures if they only happen 4 times a week every week.

Arguably, we should probably be querying orangefactor directly to find stale intermittent bugs and then change only those bugs in bugzilla, but that's a more involved change than just dropping the weekly threshold to a lower number like 1 or 2.

Once we have a better way to query these things, we can bump this threshold back up to 5, but for now, I'd argue a lower threshold is the way to go.



Patch incoming.

Comment 1

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Attachment: Drop the threshold from 5 to 1.

Comment 2

[Ed Morley [:emorley]]

Comment on attachment 8715043 [details] [diff] [review]
Drop the threshold from 5 to 1.

Hi Wes, thank you for filing this and looking into ways to improve the orangefactor commenter behaviour :-)

Before I review this, could you explain the reasoning for the change a bit more, similar to what I asked in bug 1224018 comment 4? 

As mentioned in that bug, even without these changes it may still fine to close out inactive intermittent bugs. For example, a world where we don't file a bug for every intermittent failure, is very similar to one where we are comfortable classifying failures against a closed bug.

As is, it's going to cause a several-fold increase in the amount of bugmail (see the initial analysis I performed in bug 1179821 comment 5).

Possible reasons I can see are:
* "several bugs have similar but incorrect suggestions so need constant expanding of 'show more'"
* "it's hard to tell a closed bug apart from a really-old-so-shouldn't be used closed bug"

...however:
(a) it would be good to have them explained in this bug (and the commit message), particularly since the original thresholds were chosen in an empirical way
(b) there may be alternative ways to fix those original root causes, rather than requiring the threshold be changed

Many thanks :-)

Comment 3

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

NI'ing Dave since this patch came about from a ping from him.

Comment 4

[David Burns :automatedtester]

I am happy for this to be dropped then.