1210543 - Allow fine-grained diffing of talos results between two pushes in PerfHerder

Status: RESOLVED WORKSFORME

(Tree Management :: Perfherder, defect)

Description

[Kartikaya Gupta (email:[e-mail])]

Right now PerfHerder allows you to compare two revisions and see which talos jobs regressed/improved. This is great for detecting regressions, but when it comes to actually debugging and investigating the regression in detail, we need more fine-grained data.

For example, right now I'm trying to look into a tp5o_scroll regression. I did try pushes with mozharness: --spsProfile so I get profile information for the two pushes. However when I go to look at the profile information I find that tp5o_scroll actually loads 51 different webpages and there are separate profiles for each of them. I'm sure that some of these pages regressed more than others, and I would like to know which one regressed the most, so that I can look at that one's profile information. I would like PerfHerder to provide this finer-grained data; if not I have to scrape it manually from the log files for the jobs.

Comment 1

[William Lachance (:wlach)]

Can you not already do this by clicking on "subtests" next to the summary?

e.g. in your case:

https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=mozilla-inbound&originalRevision=2258001851df&newProject=mozilla-inbound&newRevision=30742281c223&originalSignature=6764e9b1ce6291a6d9683810b584f1f3536362e6&newSignature=6764e9b1ce6291a6d9683810b584f1f3536362e6

Comment 2

[Kartikaya Gupta (email:[e-mail])]

Doh I totally missed that. Thanks! That being said - when you generate the link such as in bug 1210376 comment 2, are you adding the "&e10s=1" to the URL manually? I'm not seeing any other way to get the e10s results from the default compare page at https://treeherder.mozilla.org/perf.html#/comparechooser

Comment 3

[William Lachance (:wlach)]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #2)
> Doh I totally missed that. Thanks! That being said - when you generate the
> link such as in bug 1210376 comment 2, are you adding the "&e10s=1" to the
> URL manually? I'm not seeing any other way to get the e10s results from the
> default compare page at
> https://treeherder.mozilla.org/perf.html#/comparechooser

Yeah, known limitation, you need to specify this in the URL string. I've become painfully aware of how annoying this is since I started performance sheriffing. :) Hoping to address it next week via bug 1187000.