1322105 - 1.68 - 3.49% tp5o (linux64, osx-10-10, windows7-32, windows8-64) regression on push eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e (Sat Dec 3 2016)

Status: RESOLVED DUPLICATE of bug 1321874

(Core :: DOM: Core & HTML, defect)

Description

[Alison Shiue]

Talos has detected a Firefox performance regression from push eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  3%  tp5o summary windows8-64 pgo e10s     261.46 -> 270.57
  3%  tp5o summary linux64 opt e10s         384.68 -> 397.29
  3%  tp5o summary osx-10-10 opt e10s       295.1 -> 303.92
  2%  tp5o summary windows8-64 opt e10s     325.05 -> 332.67
  2%  tp5o summary windows7-32 opt e10s     378.33 -> 384.67


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=4455

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Alison Shiue]

After doing some retriggers, this issue might be caused by the following patch:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=5f494ff3b83be2b5e8529e859d80ca3efacf19f7&tochange=eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e

Hi Ehsan, as you are the patch author, can you take a look at this and determine what is the root cause? Thanks!

Comment 2

[Ben Kelly [:bkelly, not reviewing]]

We might need bug 1321874 if the overhead of a classification lookup is high in e10s mode.

Comment 3

[(no longer active)]

I will have a look at this next week.  This needs some time to focus so unfortunately I can't do anything about this at the work week.  I'm keeping the needinfo on myself.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

thanks :ehsan!

Comment 5

[(no longer active)]

I'm trying to get a sense of whether my fix to bug 1321874 is going to be enough to fix this regression.  I have pushed a patchset with all of the necessary fixes to try, and the comparison against the base revision from mozilla-central looks like the below:

<https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&showOnlyImportant=0>

Looking at tp5o opt e10s, we have:

linux64   subtests · graph 	389.59 	> 	381.64 ± 0.85% 	-2.04% 		2.33 (low) 	1 / 21 	
osx-10-10   subtests · graph 	302.92 	> 	296.29 ± 1.13% 	-2.19% 		1.90 (low) 	1 / 21 	
windows7-32   subtests · graph 	382.22 	> 	370.37 ± 2.53% 	-3.10% 		1.20 (low) 	1 / 21 	

I didn't make a PGO build for this push, and have recently triggered a bunch of more Talos runs on the base central revision because the compare tool indicated that more base runs are needed to increase the confidence.

Eyeballing the above numbers naively it seems that these fixes are sufficient to fix this regression.

Joel, does that seem correct to you?  Is there any other investigation that I need to do?  (Note that the fixes to bug 1321874 are all under review and are slowly landing, but there are 6 or so separate patches on 3 different bugs...)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

doing a lot of retriggers on the base revision- will respond when I have more data- thanks for working on this and I will help out in an hour or so.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

this is looking pretty good:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&filter=tp5o%20summary&showOnlyImportant=0

the win7 has 1 random outlier which is causing the data to not be so reliable, but the rest of the data sans the outlier show a 2%+ improvement.

If we have a path forward this is great- we will wait for the patches to land when they are ready- and if we can get them all in before the next uplift that would be even better :)

Comment 8

[(no longer active)]

(In reply to Joel Maher ( :jmaher) from comment #7)
> this is looking pretty good:
> https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-
> central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=
> try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&filter=t
> p5o%20summary&showOnlyImportant=0
> 
> the win7 has 1 random outlier which is causing the data to not be so
> reliable, but the rest of the data sans the outlier show a 2%+ improvement.

Fantastic!

> If we have a path forward this is great- we will wait for the patches to
> land when they are ready- and if we can get them all in before the next
> uplift that would be even better :)

It seems that the next uplift is on Jan 23, so yeah I would say the patches will definitely land by then.  In the interest of not dropping the ball on this, I'll needinfo myself and will update the bug when all of the patches have landed.

Comment 9

[(no longer active)]

[Tracking Requested - why for this release]:
Performance regression on this release.

Comment 10

[(no longer active)]

Declaring this as fixed per bug 1325467 comment 12.

Comment 11

[Marcia Knous [:marcia]]

Adjusting tracking flag for 53 based on the duplicate status.