Games using WebGL (created in Unity) get stucks after very short time of gameplay

VERIFIED FIXED

Status

()

defect
P1
critical
VERIFIED FIXED
9 months ago
8 months ago

People

(Reporter: vladimir, Unassigned)

Tracking

63 Branch
Desktop
Unspecified
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox63blocking verified, firefox64 affected, firefox65 affected)

Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36

Steps to reproduce:

Install Chrome ver 63, 64-bit. (Latest version currently). Problem on Win and Mac. (Linux not tested)
Open link https://www.gamearter.com/game/insurgents-2-mp or https://www.gamearter.com/game/fighter-aircraft-pilot.

Launch the game and remain inside. After short time, the game get stuck. If not, it stuck during a part with  heavy particle (like fire or explosion - hit a ground with an airplane to achieve it).


Actual results:

Game get stuck and then Firefox says "A web page is slowing down your browser. What would you like to do?". Browser is completely stucked in this time, there is need to close it.


Expected results:

Should normally work, as in FF ver 62 or 64 (currently in beta) or any other browser.
Hardware: Unspecified → Desktop
I meant Firefox by "Chrome" in the Steps to reproduce.
Ben, do you think this can be a webassembly issue? bug 1502827 seems to suggest it is.
Component: Untriaged → Javascript: Web Assembly
Flags: needinfo?(bbouvier)
Product: Firefox → Core
This sounds a little bit like bug 1488515 even though that was reported against ARM and Android.  I'll boost the priority on that.
Flags: needinfo?(bbouvier)
Reproduces easily - sometimes when the plane crashes, sometimes during takeoff - with the fighter pilot on Mac OS X 10.13.6 High Sierra (older MacBook Pro) with current FF63 release.  Does not yet repro on Linux with FF63.

I can bring up the console after the crash via the hamburger menu but get no content in it so far.
Status: UNCONFIRMED → NEW
Ever confirmed: true
[Tracking Requested - why for this release]:
Seems that we want to fix that asap. Potentially in a dot release.
Severity: normal → critical
Priority: -- → P1
Disabling the baseline compiler does not fix the problem (and I do get the slow script bar).  Console content after killing the script:

Error: Script terminated by timeout at:
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2623]:0x164b52
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2749]:0x1722a0
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2804]:0x1796be
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2737]:0x16fd1e
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2994]:0x184078
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2955]:0x18296f
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[2950]:0x1828ee
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[8190]:0x39b71c
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[8190]:0x39b733
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[8185]:0x39a948
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[8179]:0x3991b7
@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:wasm-function[43532]:0xf83d1c
UnityLoader["7dd274b8f6a9ae4d1fbd7080d9782baf"]/Module.dynCall_v@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170:2:1
browserIterationFunc@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170:2:133740
runIter@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170:2:136832
Browser_mainLoop_runner@blob:https://www.gamearter.com/2f7bbd4d-526c-c849-942c-4c9291d28170:2:135277
2f7bbd4d-526c-c849-942c-4c9291d28170 line 2 > WebAssembly.instantiate:1461074:1
Disabling the optimizing compiler also does not make any difference, here's the stack for that, looks the same to my eyes:

Error: Script terminated by timeout at:
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2623]:0x164b52
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2749]:0x1722a0
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2804]:0x1796be
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2737]:0x16fd1e
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2994]:0x184078
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2955]:0x18296f
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[2950]:0x1828ee
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[8190]:0x39b71c
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[8190]:0x39b733
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[8185]:0x39a948
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[8179]:0x3991b7
@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:wasm-function[43532]:0xf83d1c
UnityLoader["7dd274b8f6a9ae4d1fbd7080d9782baf"]/Module.dynCall_v@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4:2:1
browserIterationFunc@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4:2:133740
runIter@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4:2:136832
Browser_mainLoop_runner@blob:https://www.gamearter.com/070bf70a-a72a-0c40-84ca-ad7da9f37ac4:2:135277
070bf70a-a72a-0c40-84ca-ad7da9f37ac4 line 2 > WebAssembly.instantiate:1461074:1
Funny, after a minute or so the audio starts playing again.
We have about 18M of uncompressed wasm, 400M of wabt disassembly, joy.  Function 2623 does not have any loops (nor is it a leaf) but its caller, 2749, has several.  Presumably the program gets stuck waiting for something to happen - asset load, whatever.

Since the two compilers behave the same (and I was careful about clearing caches and so on, so I think that result is sound) we should assume it's not the code generators, but that the problem lies elsewhere - in the shared wasm infrastructure, in JS, or in the browser.

As this is reproducible on Mac (and Windows, according to the reporter) but not on my Linux system where the audio does not seem to work (because, Linux), we may be looking at some audio effects, but this is just speculation.

Like the reporter, I am not able to repro in FF64 (Dev Edition) or FF65 (Nightly).  In Nightly there is a brief but visible pause when the plane crashes, as if something is being loaded at that point, which is what we'd expect (flames, sounds, whatever).
I think I shall bisect 63 -> 64 to try to get an idea about when the bug went away.  This may take some time.
Actually bisecting 62 -> 63.  Result so far: Nightly 63 Aug 1 works.  Nightly 63 Sep 1 does not.
Initial bisect repeatedly points to this being introduced in the Sep 1 Nightly.
Rev 434349:016e135dee84 is bad.
Rev 434245:ea869706644d is good.
Rev 434260:a7d905753c20 is bad.
Rev 434245:ea869706644d is good.

Every change between those two revs is signed :padenot (hi Paul!) so we're going to kick this over to Web Audio for now.

Detailed steps to reproduce:

1. In a clean profile of the desired build, open https://www.gamearter.com/game/fighter-aircraft-pilot
2. Click "I agree"
3. Click to skip the upgrade to FF64.
4. Click "Play Now"
5. Skip the ad
6. Click "Continue as guest"
7. Click "Free fly"
8. Select "KC-10"
9. Hit the "E" key to start the engine (notice thrust is increasing)
10. Hold the "1" key until throttle reads 100%
11. Hit "X" to release brakes
12. Wait for the plane to hit the end of the runway
13. Hold down the right arrow key to bank into the ocean and crash.

When the build "works" there should be crash sounds and flames.

When the build is broken things will hang and the slow script dialog / bar will appear.
Component: Javascript: Web Assembly → Web Audio
Flags: needinfo?(padenot)
This is because I landed wrong patches on central close to the merge day, they got into beta, karlt made me realize I was wrong, I immediately prepared a beta patch, asked for a+, got it, and nothing happened, all in https://bugzilla.mozilla.org/show_bug.cgi?id=1308436#c114.

I'm pushing the backout to try [0], rebased on mozilla-release. There was some conflicts in tests, should be more or less fine, I'll adjust as needed.

[0]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ce96b3149ee6fbe01095718a418bdf43c8d9b75c
Flags: needinfo?(padenot)
Wrong link, the previous one was on an incorrect base. https://treeherder.mozilla.org/#/jobs?repo=try&revision=3d7fc5076a1fdfafd79cc2095b79c7fdd36cbf2f is the right push.
(In reply to Paul Adenot (:padenot) from comment #16)
> and nothing happened

I guess that is because bug 1308436 got marked firefox63-fixed.
I don't know what the best status to mark on that bug is now.
I guess it doesn't matter because we now have this bug to track it.
Blocks: 1308436
We will probably have a 63.0.2 in November and want to take a patch for this bug. It's indeed better to use this new bug for the uplift request so as to prevent the process bug we had in beta with the missed backout. I am setting the status of this bug to blocking so as that it gets additional relman and QA attention when we uplift.
Thank you for quick fix. I will wait for the 63.0.2 release. Perfect work (y).
Before releasing your patch, take a look at this bug https://bugzilla.mozilla.org/show_bug.cgi?id=1503950 because for me this problem is linked
Flags: needinfo?(padenot)
Flags: needinfo?(lhansen)
Yes, should be the same problem.  I'll let Paul check.
Flags: needinfo?(lhansen)
Yes, same issue, same fix.
Flags: needinfo?(padenot)
Note to Sheriffs:

Please land the backout patch in https://bugzilla.mozilla.org/show_bug.cgi?id=1308436#c114 to mozilla-release for our upcoming 63 dot release and reference this bug, thanks!

For 64/65, I'd like to get confirmation from Paul that the same patch is needed on these branches before committing and closing this bug so let's wait for his comment on this tomorrow as he is not available today.
Flags: needinfo?(padenot)
(In reply to Pascal Chevrel:pascalc from comment #24)
> Note to Sheriffs:
> 
> Please land the backout patch in
> https://bugzilla.mozilla.org/show_bug.cgi?id=1308436#c114 to mozilla-release
> for our upcoming 63 dot release and reference this bug, thanks!
> 
> For 64/65, I'd like to get confirmation from Paul that the same patch is
> needed on these branches before committing and closing this bug so let's
> wait for his comment on this tomorrow as he is not available today.

64 and 65 work well. This uplift was the only one that was needed.
Flags: needinfo?(padenot)
Thanks Paul, closing the bug then
Status: NEW → RESOLVED
Closed: 8 months ago
Resolution: --- → FIXED
I have managed to reproduce this bug using the STR from comment 0, on Ubuntu 16.04 x64 with Firefox 63.0. 

The issue is verified fixed on 63.0.3 (20181114214635) across OSes: Windows 10 x64, macOS 10.12 and Ubuntu 16.04 x64.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.