1480667 - 3.09 - 5.58% about_preferences_basic / tp6_google_heavy (linux64, linux64-qr, osx-10-10, windows10-64, windows7-32) regression on push c20648606d7c (Thu Aug 2 2018)

Status: RESOLVED WONTFIX

(Firefox :: Settings UI, defect, P1)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=3535c07b0da95bee0bfb7d5279a08fa9d92e07f1&tochange=c20648606d7c5f03b33259dfe2639f89f15dc375

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  6%  about_preferences_basic windows7-32 opt e10s stylo     136.54 -> 144.16
  5%  about_preferences_basic windows10-64 opt e10s stylo    139.46 -> 146.17
  5%  about_preferences_basic osx-10-10 opt e10s stylo       223.00 -> 233.28
  4%  about_preferences_basic windows7-32 pgo e10s stylo     134.18 -> 140.02
  4%  about_preferences_basic linux64-qr opt e10s stylo      158.93 -> 164.72
  3%  tp6_google_heavy linux64 pgo e10s stylo                379.06 -> 390.79


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=14714

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Johann Hofmann [:johannh]]

Hmm, we added a bunch of stuff to about:preferences, so I'm pessimistic that this is not a WONTFIX.

~5% regression is still a bit of a bummer, I should probably investigate if there's anything we can do to improve this, although, again, I'm not too optimistic. Things might improve once we remove the tracking protection portion of this UI entirely, but really probably just rendering that big blue toggle has a perf impact. I can try to bisect this a little...

So probably WONTFIX, but I can at least try to verify it.

>   3%  tp6_google_heavy linux64 pgo e10s stylo                379.06 -> 390.79

I can't explain this one and I highly doubt whether it's accurate, my patch only touches about:preferences.

Comment 2

[Johann Hofmann [:johannh]]

(In reply to Johann Hofmann [:johannh] - slow to respond, digging out of PTO backlog from comment #1)
> >   3%  tp6_google_heavy linux64 pgo e10s stylo                379.06 -> 390.79
> 
> I can't explain this one and I highly doubt whether it's accurate, my patch
> only touches about:preferences.

Ionuț, is this something you'd want to look into?

Comment 3

[Ionuț Goldan [:igoldan]]

(In reply to Johann Hofmann [:johannh] - slow to respond, digging out of PTO backlog from comment #2)
> (In reply to Johann Hofmann [:johannh] - slow to respond, digging out of PTO
> backlog from comment #1)
> > >   3%  tp6_google_heavy linux64 pgo e10s stylo                379.06 -> 390.79
> > 
> > I can't explain this one and I highly doubt whether it's accurate, my patch
> > only touches about:preferences.
> 
> Ionuț, is this something you'd want to look into?

Sorry, :johannh about this. Somehow I missed that, given the multiple about_preferences_basic above. On first look, it seems  the tp6_google_heavy was caused by a different bug, which originated from the other integration branch.

Thanks for noticing this!

Comment 4

[Ionuț Goldan [:igoldan]]

I'll soon post the Gecko profiles.

Comment 5

[Ionuț Goldan [:igoldan]]

Here are the Gecko profiles for the about_preferences_basic test:


On OS X:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FM4zBuPi8Sei6EjaGN8UbCA%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_about_preferences_basic.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FBpvGmODrS52dqbZtHEc-yA%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_about_preferences_basic.zip


On Windows 10 64bit:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FdOYESq5SR9OP6PtizE7RHw%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_about_preferences_basic.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FW5K7fh4STX2dfHP2C96bJA%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_about_preferences_basic.zip

Comment 6

[Jared Wein [:jaws] (please needinfo? me)]

(marking as P1 since this appears to be actively being worked on)

Comment 7

[Ionuț Goldan [:igoldan]]

:johannh can you provide some updates here?

Comment 8

[Pascal Chevrel (relman team) -> :pascalc]

Johann, any news on this one?

Comment 9

[Johann Hofmann [:johannh]]

Yeah, sorry, I let this stall for a bit because I wasn't able to make any progress here. My initial investigation wasn't conclusive, the profile doesn't reveal anything obvious to me or other people I've asked. 

I've recently done a bit of bisect Talos runs and they're not very enlightening to me and reinforce the feeling that this is just "doing more stuff":

We see a clear speedup when turning the new CB UI off entirely:

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=240e6a058d566a12f2da347c0a59f63b1a3d66a2&framework=1&filter=preferences&selectedTimeRange=604800

I was thinking maybe the blue toggle has some perf impact, but hiding it gives weird results:

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=5b42bc4d273bc9698aaabcf26bc523828d5ead10&framework=1&filter=preference&selectedTimeRange=604800

Hiding the CB categories (avoiding layout etc.) mirrors the pattern that we saw when turning off the CB UI entirely:

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=882323c62b54505a060d7711cad987ade0a5ec98&framework=1&filter=preference&selectedTimeRange=604800

Removing the JS parts instead of hiding the UI has a similar effect:

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=ba8fe2535498b18a5981797268af7f7b785abd7a&framework=1&filter=preference&selectedTimeRange=604800

Thus I assume the slight perf dip is just a mix of JS work to build this UI and extra layout/style/etc. required over the simple old TP UI.

If anyone can think of a better way to approach this, I'm happy to take suggestions and look into it more, until then I'll declare this WONTFIX.

Another general issue is that a lot of this code needs to be so complex right now because we're simultaneously keeping two versions of the same UI around, one of Tracking Protection and one for Content Blocking. This means we need to show/hide the relevant UI on page load. I hope we can remove all that code again soon.