Closed Bug 1322105 Opened 5 years ago Closed 5 years ago

1.68 - 3.49% tp5o (linux64, osx-10-10, windows7-32, windows8-64) regression on push eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e (Sat Dec 3 2016)

Categories

(Core :: DOM: Core & HTML, defect)

53 Branch
defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 1321874
Tracking Status
firefox50 --- unaffected
firefox51 --- unaffected
firefox52 --- unaffected
firefox53 + fixed

People

(Reporter: ashiue, Assigned: ehsan.akhgari)

References

Details

(Keywords: perf, regression, talos-regression)

Talos has detected a Firefox performance regression from push eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  3%  tp5o summary windows8-64 pgo e10s     261.46 -> 270.57
  3%  tp5o summary linux64 opt e10s         384.68 -> 397.29
  3%  tp5o summary osx-10-10 opt e10s       295.1 -> 303.92
  2%  tp5o summary windows8-64 opt e10s     325.05 -> 332.67
  2%  tp5o summary windows7-32 opt e10s     378.33 -> 384.67


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=4455

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
After doing some retriggers, this issue might be caused by the following patch:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=5f494ff3b83be2b5e8529e859d80ca3efacf19f7&tochange=eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e

Hi Ehsan, as you are the patch author, can you take a look at this and determine what is the root cause? Thanks!
Blocks: 1318540, 1321868
Flags: needinfo?(ehsan)
We might need bug 1321874 if the overhead of a classification lookup is high in e10s mode.
See Also: → 1321874
I will have a look at this next week.  This needs some time to focus so unfortunately I can't do anything about this at the work week.  I'm keeping the needinfo on myself.
thanks :ehsan!
I'm trying to get a sense of whether my fix to bug 1321874 is going to be enough to fix this regression.  I have pushed a patchset with all of the necessary fixes to try, and the comparison against the base revision from mozilla-central looks like the below:

<https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&showOnlyImportant=0>

Looking at tp5o opt e10s, we have:

linux64   subtests · graph 	389.59 	> 	381.64 ± 0.85% 	-2.04% 		2.33 (low) 	1 / 21 	
osx-10-10   subtests · graph 	302.92 	> 	296.29 ± 1.13% 	-2.19% 		1.90 (low) 	1 / 21 	
windows7-32   subtests · graph 	382.22 	> 	370.37 ± 2.53% 	-3.10% 		1.20 (low) 	1 / 21 	

I didn't make a PGO build for this push, and have recently triggered a bunch of more Talos runs on the base central revision because the compare tool indicated that more base runs are needed to increase the confidence.

Eyeballing the above numbers naively it seems that these fixes are sufficient to fix this regression.

Joel, does that seem correct to you?  Is there any other investigation that I need to do?  (Note that the fixes to bug 1321874 are all under review and are slowly landing, but there are 6 or so separate patches on 3 different bugs...)
Flags: needinfo?(ehsan) → needinfo?(jmaher)
Assignee: nobody → ehsan
Depends on: 1321874
doing a lot of retriggers on the base revision- will respond when I have more data- thanks for working on this and I will help out in an hour or so.
this is looking pretty good:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&filter=tp5o%20summary&showOnlyImportant=0

the win7 has 1 random outlier which is causing the data to not be so reliable, but the rest of the data sans the outlier show a 2%+ improvement.

If we have a path forward this is great- we will wait for the patches to land when they are ready- and if we can get them all in before the next uplift that would be even better :)
Flags: needinfo?(jmaher)
(In reply to Joel Maher ( :jmaher) from comment #7)
> this is looking pretty good:
> https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-
> central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=
> try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&filter=t
> p5o%20summary&showOnlyImportant=0
> 
> the win7 has 1 random outlier which is causing the data to not be so
> reliable, but the rest of the data sans the outlier show a 2%+ improvement.

Fantastic!

> If we have a path forward this is great- we will wait for the patches to
> land when they are ready- and if we can get them all in before the next
> uplift that would be even better :)

It seems that the next uplift is on Jan 23, so yeah I would say the patches will definitely land by then.  In the interest of not dropping the ball on this, I'll needinfo myself and will update the bug when all of the patches have landed.
Flags: needinfo?(ehsan)
[Tracking Requested - why for this release]:
Performance regression on this release.
Flags: needinfo?(ehsan)
Flags: needinfo?(ehsan)
Declaring this as fixed per bug 1325467 comment 12.
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(ehsan)
Resolution: --- → DUPLICATE
Duplicate of bug: 1321874
Component: Untriaged → DOM
Product: Firefox → Core
Adjusting tracking flag for 53 based on the duplicate status.
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.