1229275 - Perfherder doesn't distinguish between linux64 debug builds and linux64 debug static analysis builds

Status: RESOLVED DUPLICATE of bug 1261081

(Tree Management :: Perfherder, defect, P5)

Description

[Chris Manchester (limited bugmail, email directly)]

Attachment: Untitled.jpg

The two series aren't distinguished in the UI, so it's hard to see which is which.

Comment 1

[William Lachance (:wlach)]

No idea why this is offhand, but it is probably fairly easy to fix. Something that could be addressed at the mozlando perfherder hackfest. :)

Comment 2

[Julien Pagès (:parkouss)]

Hmm, should'nt we see an option (e.g, 'static' or something) in https://treeherder.mozilla.org/api/optioncollectionhash/? ?

I can't find a way to distinguish the builds looking at the data available in say, _getSeriesName:
https://github.com/mozilla/treeherder/blob/master/ui/js/perf.js#L27

This information should be provided by OptionCollectionMap I guess - I narrowed down to:
https://github.com/mozilla/treeherder/blob/master/ui/js/models/option_collection.js

which makes a query and retrieve https://treeherder.mozilla.org/api/optioncollectionhash/?.

For the mentioned tests, they both only have the option 'debug' - and there is no static in the above collection.

So - to me - looks like we are not retrieving 'static' or 'analysis' from the buildbot build names and reporting them as options. Here is a build name from buildbot:

"buildername": "OS X 10.7 64-bit fx-team debug static analysis build",

Will, does that make sense to you ? And do you know where that information is ingested in treeherder ?

Comment 3

[Julien Pagès (:parkouss)]

So, maybe we should add 'static' and 'analysis' detection in there as a start:

https://github.com/mozilla/treeherder/blob/master/treeherder/etl/buildbot.py#L393

Ed, some thoughts ?

Comment 4

[Ed Morley [:emorley]]

It was my understanding that these jobs we're more like tests than traditional builds. We normally only give builds a new row if they are running the same tests as another row, but against some kind of different binary. 

Searching inbound I don't see any talos tests being run against this job?

Comment 6

[William Lachance (:wlach)]

I wonder if we should just disable perfherder statistic gathering for these types of builds, I don't think they're very useful.

Comment 7

[Ed Morley [:emorley]]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #6)
> I wonder if we should just disable perfherder statistic gathering for these
> types of builds, I don't think they're very useful.

Comment 8

[William Lachance (:wlach)]

This is probably still a bug, but I'm not sure.