696197 - Compare-talos is completely broken by PGO/non-PGO split and should not be linked to from TBPL

Status: RESOLVED WONTFIX

(Tree Management Graveyard :: TBPL, defect)

Description

[Justin Lebar (not reading bugmail)]

See bug 696196.

I think we should get rid of the compare-talos links on TBPL until we fix this.

Comment 1

[Shawn Wilsher :sdwilsh]

https://bitbucket.org/mconnor/compare-talos is the right place to File compare-talos bugs.

Comment 2

[Phil Ringnalda (:philor)]

No argument from me, periodic PGO nearly completely broke compare-talos, _on those trees that do periodic PGO_ (only nearly completely because if you are interested in PGO results, as you generally should be, then you can overwhelm the average by triggering 10 or 20 or 30 PGO builds on your cset, and 10 or 20 or 30 PGO builds on your previous cset, and I will cheer you on, and laugh and point and drag every single releng person over to see the horrors they've wrought).

However, wontfix, because it didn't break compare-talos on branches that don't do periodic PGO, and it didn't break compare-talos for OSes that don't do PGO on branches that do periodic PGO, and it didn't break comparison between every pair of csets on branches that do periodic PGO.

It would amuse hell out of me if someone wrote a patch to pop up a huge scary interstitial warning saying "compare-talos is broken for anything that runs PGO, ignore the results you're about to get, most of them are meaningless garbage" in cases where the results will be bogus, but I doubt anyone would really go to all the effort required to make it accurate and only trigger in cases where the results really will be bogus, and if they're going to that much trouble, they'd be better served by just fixing compare-talos, if a usable graphserver API exists, or graphserver and compare-talos, if one does not (or buildbot, if it really isn't telling graphserver what is and isn't PGO, but I doubt that's the case).