Closed Bug 1520149 Opened 1 year ago Closed 1 year ago

4.86% raptor-speedometer-geckoview (android-hw-p2-8-0-android-aarch64) regression on push 5ab46880d921f1c948902976e832e995cdaa8104 (Fri Jan 11 2019)

Categories

(Firefox Build System :: General, defect)

ARM64
Android
defect
Not set
normal

Tracking

(firefox-esr60 unaffected, firefox64 unaffected, firefox65 unaffected, firefox66 verified)

VERIFIED FIXED
mozilla66
Tracking Status
firefox-esr60 --- unaffected
firefox64 --- unaffected
firefox65 --- unaffected
firefox66 --- verified

People

(Reporter: igoldan, Assigned: glandium)

References

(Blocks 1 open bug)

Details

(Keywords: perf, regression, Whiteboard: [geckoview:p2][arm64:m3])

Attachments

(1 file)

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=5ab46880d921f1c948902976e832e995cdaa8104

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

5% raptor-speedometer-geckoview android-hw-p2-8-0-android-aarch64 opt 17.11 -> 16.28

Improvements:

3% raptor-speedometer-geckoview android-hw-p2-8-0-arm7-api-16 opt 21.39 -> 22.01
2% raptor-speedometer-geckoview android-hw-g5-7-0-arm7-api-16 opt 8.48 -> 8.68

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=18714

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Product: Testing → Firefox Build System
Flags: needinfo?(mh+mozilla)

Note that AArch64 Fennec is only using SpiderMonkey's Baseline JIT. Performance should improve once the Ion JIT supports AArch64.

OTOH, we saw a similar AArch64 regression after updating to clang 7 (bug 1503330) that was fixed by disabling clang's machine outliner (bug 1508547). Perhaps LTO and the machine outliner share some common optimization strategies on AArch64?

Hardware: Unspecified → ARM64
See Also: → 1508547
Whiteboard: [geckoview:p2][arm64:m3]

I confirmed on try that this is outlining. The problem is that the flag to disable the outliner only has an effect when compiling, which we're not actually doing when doing LTO: the linker is doing the compilation, and the flag is ignored: either clang doesn't transform it into the right flag for the linker, or the linker ignores it, I don't know which yet.

With the outliner disabled at the compiler level, I get results around 17.8, which is a perf win compared to without LTO.

Flags: needinfo?(mh+mozilla)

(In reply to Mike Hommey [:glandium] from comment #2)

With the outliner disabled at the compiler level

By which I mean, "LLVM compiled with the outliner-enabling patch backed out"

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/c28522aaf391
Also disable the machine outliner on Android when LTO is enabled. r=dmajor

ni? :davehunt to assess noisiness of raptor-speedometer-geckoview on Android devices + its limited capacity in regards to retriggering/backfilling.

Flags: needinfo?(dave.hunt)
Assignee: nobody → mh+mozilla

There is no need to retrigger or backfill for this bug. The landing had the expected effect to restore the performance, and more, since we're now with LTO enabled, which we weren't before bug 1480006.

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66

I confirm the fix:

== Change summary for alert #18793 (as of Thu, 17 Jan 2019 20:14:49 GMT) ==

Improvements:

10% raptor-speedometer-geckoview android-hw-p2-8-0-android-aarch64 opt 16.19 -> 17.80
6% raptor-tp6-docs-firefox osx-10-10 opt 2,760.79 -> 2,608.53

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=18793

(In reply to Chris Peterson [:cpeterson] from comment #10)

You can see the 10% boost to ARM64 Android's Speedometer score on Perfherder:

https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=autoland,1795984,1,10&series=autoland,1795791,1,10&series=autoland,1796140,1,10

https://treeherder.mozilla.org/perf.html#/alerts?id=18793

Thanks for verifying this!

Blocks: 1529656
You need to log in before you can comment on or make changes to this bug.