663274 - Restrict OrangeFactor to [orange] bugs

Status: RESOLVED FIXED

(Tree Management Graveyard :: OrangeFactor, defect)

Description

[(no longer active)]

Examples: bug 651665, bug 468736

Comment 1

[Jonathan Griffin (:jgriffin)]

Ehsan, can you explain what you mean by unrelated?

Comment 2

[(no longer active)]

(In reply to comment #1)
> Ehsan, can you explain what you mean by unrelated?

Usually, we denote randomorange bugs by adding [orange] to their status whiteboard.  I'm not sure how Orange Factor finds the bug numbers, but clearly there's something broken there, since these bugs are not randomorange bugs.

Comment 3

[Jonathan Griffin (:jgriffin)]

(In reply to comment #2)
> (In reply to comment #1)
> > Ehsan, can you explain what you mean by unrelated?
> 
> Usually, we denote randomorange bugs by adding [orange] to their status
> whiteboard.  I'm not sure how Orange Factor finds the bug numbers, but
> clearly there's something broken there, since these bugs are not
> randomorange bugs.

Ah!  OrangeFactor lists bugs that have been used in TBPL star comments.  It doesn't restrict them to [orange] bugs, but we could.  Should we do that?

Comment 4

[(no longer active)]

Yes!

Although, I find it kind of strange.  Does this mean that bug 468736 has been mentioned 153 times in TBPL comments?

Comment 5

[Jonathan Griffin (:jgriffin)]

(In reply to comment #4)
> Yes!
> 
> Although, I find it kind of strange.  Does this mean that bug 468736 has
> been mentioned 153 times in TBPL comments?

Indeed it has, in the context of Jetpack SDK tests.

Comment 6

[Phil Ringnalda (:philor)]

That few? Well, I guess it was only recently that I got corrected from what I was using before on the every-single-damn-push two Windows debug failures.

Comment 7

[(no longer active)]

(In reply to comment #5)
> (In reply to comment #4)
> > Yes!
> > 
> > Although, I find it kind of strange.  Does this mean that bug 468736 has
> > been mentioned 153 times in TBPL comments?
> 
> Indeed it has, in the context of Jetpack SDK tests.

Bah.  We definitely don't want those bugs to skew our notion of random oranges in the tree.  Can we also make sure that we only look at non-hidden machines on Tinderbox?

Comment 8

[Jonathan Griffin (:jgriffin)]

Attachment: woo_server.py patch v0.1

This patch causes BugData to return only oranges whose bugs have '[orange]' in their whiteboard field.

This reduces the number of failures tracked by WOO from 1123 to 903 (for mozilla-central), at the time of this writing.

No JS-code changes are needed.

Comment 9

[Mark Côté [:mcote]]

Comment on attachment 539645 [details] [diff] [review]
woo_server.py patch v0.1

Cool cool.

Comment 10

[Jonathan Griffin (:jgriffin)]

I've deployed this change.  Still looking at:

> Can we also make sure that we only look at non-hidden
> machines on Tinderbox?

Comment 11

[Phil Ringnalda (:philor)]

You'll want to just keep on looking until that data migrates off tinderbox, because right now your only way to access it is to scrape the output of tinderbox.m.o/admintree.cgi

Comment 12

[Jonathan Griffin (:jgriffin)]

(In reply to comment #11)
> You'll want to just keep on looking until that data migrates off tinderbox,
> because right now your only way to access it is to scrape the output of
> tinderbox.m.o/admintree.cgi

I was thinking we could parse the hiddenBuilds list from http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/file/tip/js/Config.js; do you think that mechanism of flagging hidden builds will be changing?

Comment 13

[Phil Ringnalda (:philor)]

Yeah: that list never really worked out, nor could it work out, since actually hiding only involves having the sheriffpass and loading a web page, while changing that list involves cloning tbpl, writing a patch, getting review, pushing the patch, filing for a tbpl.m.o update, and getting someone to update the site. From a quick glance, it features some but not all of the currently hidden things, some things which are no longer hidden, and some things that I think no longer exist. That list was updated in mid-May, and again on the first of June, while since last Friday, I've hidden and unhidden a half-dozen things on multiple trees.

That list isn't "hidden on tbpl," it's "someone with a tbpl tree got annoyed by seeing these things as pending/running when they knew they would (or did, at some time in the past) disappear as they finished." Among other things, it doesn't include the Valgrind builds, because they don't show up as pending/running and the only reason that list exists is to hide things that show up there.

What you want is the eventual fruits of bug 630537.

Comment 14

[Jonathan Griffin (:jgriffin)]

(In reply to comment #13)
> 
> What you want is the eventual fruits of bug 630537.

Cool, thanks, I'll wait for that to land.

Comment 15

[Jonathan Griffin (:jgriffin)]

WOO now only displays bugs with a whiteboard of [orange].  The other request about filtering out hidden builders has been split off into a separate bug, bug 668117.