1254341 - Make it easy to compare e10s / non-e10s on the same push

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[Mike Conley (:mconley) (:⚙️)]

Sometimes I'll push a patch to try with both e10s and non-e10s talos tests running, and I'll want to compare the results of e10s and non-e10s on the same push.

This is possible using the compare.py tool, but it'd be great if Perfherder had a way for me to do that too. Currently, I can't seem to compare against the same push (at least, not comparing e10s and non-e10s).

Comment 1

[Jim Mathies [:jimm]]

I would love to have this for sorting out scroll regressions independent of apz. Currently they only way I can do the comparison is by manually looking at the numbers.

Ryan, do you know who put the e10s dashboard together for us? Whoever they are they might be able to help here.

Comment 2

[William Lachance (:wlach)]

(In reply to Jim Mathies [:jimm] from comment #1)
> I would love to have this for sorting out scroll regressions independent of
> apz. Currently they only way I can do the comparison is by manually looking
> at the numbers.
> 
> Ryan, do you know who put the e10s dashboard together for us? Whoever they
> are they might be able to help here.

I'm the author and :mconley filed this at my suggestion. It's in my work queue, I should be able to get to it early next week.

Comment 3

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1254341 > mozilla:master

Comment 4

[William Lachance (:wlach)]

Comment on attachment 8739522 [details] [review]
[treeherder] wlach:1254341 > mozilla:master

Some random fixes folded in here as well

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8739522 [details] [review]
[treeherder] wlach:1254341 > mozilla:master

Hey mconley, let me know if this is what you had in mind. You need to use a special incantation to get to this view (there is no UI). If you think it would be really useful I can add it but I doubt more than a few people will want to use it.

Here's an example of its use:

http://wlach.github.io/treeherder/ui/perf.html#/e10s?revision=0e3178883b51&repo=mozilla-inbound

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8739522 [details] [review]
[treeherder] wlach:1254341 > mozilla:master

nice extension of the existing e10s dashboard.

Comment 7

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/63e4045cf9f0d299358dcf25a438fb24acf7e9c4
Bug 1254341 - Use PhSeries to get data in perfherder e10s dashboard

https://github.com/mozilla/treeherder/commit/3b98400ff829efc677fac32fa6b49c4acfd11302
Bug 1254341 - Fix subtest name in perfherder e10s dashboard

https://github.com/mozilla/treeherder/commit/0cfc57f4147e5dd251a52bec15fa45a69dd7e990
Bug 1254341 - Support e10s-vs-base comparisons on a specific push

https://github.com/mozilla/treeherder/commit/93cadc3e63d78ab5b80595647c9181faefeb013a
Merge pull request #1399 from wlach/1254341

Bug 1254341 - Support e10s-vs-base comparisons on a specific push

Comment 8

[William Lachance (:wlach)]

I merged the current patch (with a fix to error handling in case a revision doesn't exist, which tripped us up the first time we tested the feature). Still welcoming feedback before resolving.

Comment 9

[Mike Conley (:mconley) (:⚙️)]

This works beautifully for me, even though discovery is a little hard (I imagine the only way to get here is to craft the URL manually?)

Comment 10

[Mike Conley (:mconley) (:⚙️)]

Comment on attachment 8739522 [details] [review]
[treeherder] wlach:1254341 > mozilla:master

As per above.

Comment 11

[William Lachance (:wlach)]

Yeah, currently you need to craft the URL manually. If there's demand I could add a UI for this feature in the future...

Comment 12

[William Lachance (:wlach)]

Marking this fixed for now, I'll try to get some feedback on how useful this is before possibly working on a UI.