Closed Bug 1700295 Opened 4 years ago Closed 4 years ago

56.14 - 56.3% unity-webgl (android-hw-p2-8-0-android-aarch64-shippable) regression on push e817161ce652f0e7f2aa026423a6506845b19a8d (Thu March 18 2021)

Categories

(Core :: JavaScript: WebAssembly, defect)

defect

Tracking

()

RESOLVED WONTFIX
Tracking Status
firefox-esr78 --- unaffected
firefox87 --- unaffected
firefox88 --- wontfix
firefox89 --- wontfix

People

(Reporter: Bebe, Unassigned)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Perfherder has detected a browsertime performance regression from push e817161ce652f0e7f2aa026423a6506845b19a8d. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
56% unity-webgl android-hw-p2-8-0-android-aarch64-shippable nocondprof webrender 1,469.56 -> 642.14
56% unity-webgl android-hw-p2-8-0-android-aarch64-shippable nocondprof webrender 1,468.05 -> 643.93

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Flags: needinfo?(jseward)

This is related to the performance gain reported at
https://bugzilla.mozilla.org/show_bug.cgi?id=1678097#c29, last Friday.

I believe this is unrelated to my patches, both here and in bug 1678097.

What seems to have happened is that -- for some reason -- this test
changed score from around 630-640 to about 1460-1470 some time on
Mar 15, which is when 1678097 landed. But then they have changed back
to the original value, 630-640, around when this patch landed. In both
cases, the patches concern SpiderMonkey and I don't see how this has
any relationship to "unity-webgl".

Looking at the graphs
https://treeherder.mozilla.org/perfherder/graphs?timerange=1209600&series=autoland,2935601,1,13 and
https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&series=autoland,2935601,1,13&timerange=1209600
we see the change of values from 630-640 to 1460-1470, and back.

So I am mystified. Is there some other reason -- breakage in perfherder? tests
run on the wrong hardware? etc -- that could account for this change and change back?

Flags: needinfo?(jseward) → needinfo?(fstrugariu)

This test run on Android 8.0 Pixel2 AArch64 Shippable both patches add changes to JS_CODEGEN_ARM64 so I think both patches are related and the second one Bug 1699379 somehow reverts the improvements gained in Bug 1678097.
We can check that this is the patch changed by making a try job with a backout patch.

Flags: needinfo?(fstrugariu) → needinfo?(jseward)

So .. it might be related to my patches after all. I believe this is
related to compilation of asm.js -- an older, downloadable, executable format
that pre-dates WebAssembly.

The affected benchmark is "unity-webgl", on aarch64. This page

https://docs.unity3d.com/Manual/webgl-browsercompatibility.html

says

asm.js AOT compilation

asm.js is a susbset of JavaScript for which a browser can specifically
optimize. Browsers which implement asm.js support may be able to run Unity
WebGL content faster, because Unity uses asm.js.

The first set of patches -- bug 1678097 -- had the side effect of changing
SpiderMonkey's compilation pathway for asm.js on aarch64, so that it would
give significantly higher performance, in line with the performance
improvement seen here. However, that side-effect was unintended -- that
pathway is not considered stable enough for general use at this time. Hence
the small followup change in bug 1699379 two days later, which switches asm.js
back to the original pathway and hence the original performance level.

Flags: needinfo?(jseward)

Hence I suggest we close this bug. The perf improvement was unintended and
the second patch restores performance to original levels. Our understanding
of the issues is consistent with the evidence observed.

Flags: needinfo?(fstrugariu)

Set release status flags based on info from the regressing bug 1699379

Closing this as WONTFIX

Status: NEW → RESOLVED
Closed: 4 years ago
Flags: needinfo?(fstrugariu)
Resolution: --- → WONTFIX
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.