1520149 - 4.86% raptor-speedometer-geckoview (android-hw-p2-8-0-android-aarch64) regression on push 5ab46880d921f1c948902976e832e995cdaa8104 (Fri Jan 11 2019)

Status: VERIFIED FIXED

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=5ab46880d921f1c948902976e832e995cdaa8104

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

5% raptor-speedometer-geckoview android-hw-p2-8-0-android-aarch64 opt 17.11 -> 16.28

Improvements:

3% raptor-speedometer-geckoview android-hw-p2-8-0-arm7-api-16 opt 21.39 -> 22.01
2% raptor-speedometer-geckoview android-hw-g5-7-0-arm7-api-16 opt 8.48 -> 8.68

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=18714

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Chris Peterson [:cpeterson]]

Note that AArch64 Fennec is only using SpiderMonkey's Baseline JIT. Performance should improve once the Ion JIT supports AArch64.

OTOH, we saw a similar AArch64 regression after updating to clang 7 (bug 1503330) that was fixed by disabling clang's machine outliner (bug 1508547). Perhaps LTO and the machine outliner share some common optimization strategies on AArch64?

Comment 2

[Mike Hommey [:glandium]]

I confirmed on try that this is outlining. The problem is that the flag to disable the outliner only has an effect when compiling, which we're not actually doing when doing LTO: the linker is doing the compilation, and the flag is ignored: either clang doesn't transform it into the right flag for the linker, or the linker ignores it, I don't know which yet.

With the outliner disabled at the compiler level, I get results around 17.8, which is a perf win compared to without LTO.

Comment 3

[Mike Hommey [:glandium]]

(In reply to Mike Hommey [:glandium] from comment #2)

With the outliner disabled at the compiler level

By which I mean, "LLVM compiled with the outliner-enabling patch backed out"

Comment 4

[Mike Hommey [:glandium]]

Filed https://bugs.llvm.org/show_bug.cgi?id=40348

Testing workarounds.

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1520149 - Also disable the machine outliner on Android when LTO is enabled.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c28522aaf391
Also disable the machine outliner on Android when LTO is enabled. r=dmajor

Comment 7

[Ionuț Goldan [:igoldan]]

ni? :davehunt to assess noisiness of raptor-speedometer-geckoview on Android devices + its limited capacity in regards to retriggering/backfilling.

Comment 8

[Mike Hommey [:glandium]]

There is no need to retrigger or backfill for this bug. The landing had the expected effect to restore the performance, and more, since we're now with LTO enabled, which we weren't before bug 1480006.

Comment 9

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/c28522aaf391

Comment 10

[Chris Peterson [:cpeterson]]

You can see the 10% boost to ARM64 Android's Speedometer score on Perfherder:

https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=autoland,1795984,1,10&series=autoland,1795791,1,10&series=autoland,1796140,1,10

https://treeherder.mozilla.org/perf.html#/alerts?id=18793

Comment 11

[Ionuț Goldan [:igoldan]]

I confirm the fix:

== Change summary for alert #18793 (as of Thu, 17 Jan 2019 20:14:49 GMT) ==

Improvements:

10% raptor-speedometer-geckoview android-hw-p2-8-0-android-aarch64 opt 16.19 -> 17.80
6% raptor-tp6-docs-firefox osx-10-10 opt 2,760.79 -> 2,608.53

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=18793

Comment 12

[Ionuț Goldan [:igoldan]]

(In reply to Chris Peterson [:cpeterson] from comment #10)

You can see the 10% boost to ARM64 Android's Speedometer score on Perfherder:

https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=autoland,1795984,1,10&series=autoland,1795791,1,10&series=autoland,1796140,1,10

https://treeherder.mozilla.org/perf.html#/alerts?id=18793

Thanks for verifying this!