Closed Bug 1507463 Opened 5 years ago Closed 3 years ago

Crash in libGLESv2_adreno.so@0x34121a

Categories

(Firefox for Android Graveyard :: General, defect, P2)

Unspecified
Android
defect

Tracking

(firefox63 wontfix, firefox64 wontfix, firefox65 wontfix, firefox66 wontfix, firefox67 wontfix, firefox68 wontfix)

RESOLVED INCOMPLETE
Tracking Status
firefox63 --- wontfix
firefox64 --- wontfix
firefox65 --- wontfix
firefox66 --- wontfix
firefox67 --- wontfix
firefox68 --- wontfix

People

(Reporter: marcia, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: crash, regression, Whiteboard: [geckoview:fenix:p2])

Crash Data

This bug was filed from the Socorro interface and is
report bp-aa49168b-212a-417c-a8b9-148ad0181113.
=============================================================

Currently #18 top crash on release: https://bit.ly/2DnHqxe. Appears as to be mostly API 26 crashes. High percentage of Motorola devices:

(78.99% in signature vs 05.56% overall) android_brand = motorola
(78.99% in signature vs 05.68% overall) android_manufacturer = motorola

Comments:
*tried to play video. Went full screen, started playing, wouldn't finish loading (progress bar at @75%) then crash. 
*surfing ESPN 
* was using Instagram. 

Top 10 frames of crashing thread:

0 libGLESv2_adreno.so libGLESv2_adreno.so@0x34121a 
1 libGLESv2_adreno.so libGLESv2_adreno.so@0x1cb427 
2 dalvik-main space (region space) (deleted) dalvik-main space @0x2d400083 
3 dalvik-main space (region space) (deleted) dalvik-main space @0x2d400083 
4 dalvik-main space (region space) (deleted) dalvik-main space @0x2d400083 
5 dalvik-main space (region space) (deleted) dalvik-main space @0x2d400083 
6 libGLESv2_adreno.so libGLESv2_adreno.so@0x44480e 
7 libxul.so std::__ndk1::__tree_iterator<std::__ndk1::__value_type<mozilla::layers::ShaderConfigOGL, mozilla::layers::ShaderProgramOGL*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<mozilla::layers::ShaderConfigOGL, mozilla::layers::ShaderProgramOGL*>, void*>*, int> std::__ndk1::__tree<std::__ndk1::__value_type<mozilla::layers::ShaderConfigOGL, mozilla::layers::ShaderProgramOGL*>, std::__ndk1::__map_value_compare<mozilla::layers::ShaderConfigOGL, std::__ndk1::__value_type<mozilla::layers::ShaderConfigOGL, mozilla::layers::ShaderProgramOGL*>, std::__ndk1::less<mozilla::layers::ShaderConfigOGL>, true>, std::__ndk1::allocator<std::__ndk1::__value_type<mozilla::layers::ShaderConfigOGL, mozilla::layers::ShaderProgramOGL*> > >::find<mozilla::layers::ShaderConfigOGL> android-ndk/sources/cxx-stl/llvm-libc++/include/__tree:2412
8 libxul.so mozilla::layers::CompositorOGL::GetShaderProgramFor android-ndk/sources/cxx-stl/llvm-libc++/include/map:1210
9 libGLESv2_adreno.so libGLESv2_adreno.so@0x1b2217 

=============================================================
Discussed during triage, perhaps QA can try to reproduce if you have one of the affected devices - here are the top crashing devices:

motorola 	moto g(6) 	26 (REL) 	armeabi-v7a 	302 	30.3%
motorola 	Moto Z2 Play 	26 (REL) 	armeabi-v7a 	165 	16.5%
motorola 	XT1635-02 	26 (REL) 	armeabi-v7a 	98 	9.8%
motorola 	XT1710-02 	26 (REL) 	armeabi-v7a 	85 	8.5%
motorola 	XT1635-01 	26 (REL) 	armeabi-v7a 	79 	7.9%
motorola 	XT1650 	26 (REL) 	armeabi-v7a 	24 	2.4%
motorola 	moto g(6) plus 	26 (REL) 	armeabi-v7a 	12 	1.2%
Flags: needinfo?(ioana.chiorean)
See Also: → 1509128
Whiteboard: [geckoview]

ni on Stefan - Can you take a look and see if you are able to reproduce this issue if you have one of the affected devices? Thanks.

Flags: needinfo?(ioana.chiorean) → needinfo?(stefan.georgiev)

I've tried to reproduce the crash using the latest Fennec and Beta build on the ESPN, Instagram and other video platforms but to no avail. I was using different devices but none of them is a Motorola.

What I've used for testing:
-Samsung Note 9
-Sony Experia Z2
-LG Nexus 5
-Asus Nexus 7

Flags: needinfo?(stefan.georgiev)

This is a graphics specific crash it will be device specific. Spending time trying to reproduce with other devices will not be successful. Looks like a Moto G6 would be the best way to try to reproduce.

Jessie, I think Jamie might know if we can do something here.

Flags: needinfo?(jbonisteel)

(In reply to Kevin Brosnan [:kbrosnan] from comment #4)

This is a graphics specific crash it will be device specific. Spending time trying to reproduce with other devices will not be successful. Looks like a Moto G6 would be the best way to try to reproduce.

Probably they don't have this device in Vegas' Office and StefanG is mostly on Desktop (maybe Marcia wanted to ping Stefan Deiac?) so considered to give it a try either way.
We don't have unfortunately a Moto 6 neither the Z2 /Z play which comes next in the list

Hey Jamie - do you, perchance, have access to the devices needed to reproduce this?

Flags: needinfo?(jbonisteel) → needinfo?(jnicol)

I don't I'm afraid. I tried and failed to reproduce on my Pixel 2, which has an Adreno 530 like some of the affected devices.

Should I order a Moto G6?

Flags: needinfo?(jnicol)

Yes, let's go ahead and order one

Phone is ordered. I will update here once it has a arrived and I've tested whether I can reproduce.

Blocks: 1218607
Whiteboard: [geckoview] → [geckoview:fenix:p2]

Phone arrived. I can't reproduce on ESPN or instagram, but could on duckduckgo maps and vrbo.com, both of which were in crash report comments.

This is an OOM crash. Unfortunately that means there probably isn't a single fix. duckduckgo is a zooming problem causing massive layers to be allocated. vrbo layerisation looks an absolute mess but I haven't properly looked in to it yet.

Did this signature spike at a certain point, or just gradually increase as people started using this phone?

I have filed bug 1525570 for the duckduckgo problem.

(In reply to Jamie Nicol [:jnicol] from comment #12)

Phone arrived. I can't reproduce on ESPN or instagram, but could on duckduckgo maps and vrbo.com, both of which were in crash report comments.

This is an OOM crash. Unfortunately that means there probably isn't a single fix. duckduckgo is a zooming problem causing massive layers to be allocated. vrbo layerisation looks an absolute mess but I haven't properly looked in to it yet.

Did this signature spike at a certain point, or just gradually increase as people started using this phone?

I have filed bug 1525570 for the duckduckgo problem.

https://bit.ly/2DUFa03 shows the trend over 6 months. I see some small spikes here and there, but it clear that the G6 along with some other Motorola devices were more prone to this crash (this is over a 6 month period across all versions):

motorola moto g(6) 26 (REL) armeabi-v7a 7997 29.9%
motorola Moto Z2 Play 26 (REL) armeabi-v7a 4433 16.6%
motorola XT1635-02 26 (REL) armeabi-v7a 2779 10.4%
motorola XT1635-01 26 (REL) armeabi-v7a 2581 9.7%
motorola XT1710-02 26 (REL) armeabi-v7a 1495 5.6%

There are other devices in the mix that aren't Motorola as well.

Also another correlation shows (81.99% in signature vs 05.96% overall) adapter_device_id = Adreno (TM) 506 [81.99% vs 11.20% if adapter_vendor_id = Qualcomm]

It's possible that other phones would have a slightly different signature (like those in the metabug in comment 11) due to a different driver version, rather than anything specific about this phone causing the crash.

Browser crash that is #15 overall in 65.0.1 release. Current correlations show almost all users are running on API 26.

(99.22% in signature vs 15.69% overall) Module "boot-oem-services.vdex" = true
(99.22% in signature vs 16.60% overall) Module "boot-qcom.fmradio.vdex" = true
(99.35% in signature vs 18.71% overall) Module "boot-core-oj.vdex" = true
(100.0% in signature vs 20.27% overall) Module "u:object_r:debug_gralloc_prop:s0" = true
(99.35% in signature vs 19.34% overall) Module "system@framework@boot-core-oj.art" = true
(100.0% in signature vs 20.40% overall) Module "libgrallocutils.so" = true
(99.35% in signature vs 19.43% overall) Module "boot-core-oj.oat" = true
(99.22% in signature vs 19.45% overall) Module "system@framework@boot-qcom.fmradio.art" = true
(99.22% in signature vs 19.55% overall) Module "boot-qcom.fmradio.oat" = true
(99.09% in signature vs 20.10% overall) Module "libprotobuf-cpp-full.so" = true
(99.09% in signature vs 20.10% overall) Module "libstagefright_xmlparser@1.0.so" = true
(99.09% in signature vs 21.44% overall) android_version = 26 (REL)

Jamie, there are crashes reported even after bug 1525570 fix was made. Crash reports include 67 beta and 68 too. Is there anything additional that the newer crash reports can reveal to help fix this crash in 67 beta?

Flags: needinfo?(jnicol)

There will be no single fix for this unfortunately, the crash is outside of our code so the best we can do is try to prevent us using too much memory. But we've always struggled with layers using too much memory on android.. In some cases it will be possible to look at the websites which are crashing, and see if we are doing something obviously wrong on that site (like in the duck duck go case). But these things are rarely as straightforward as that.

Flags: needinfo?(jnicol)

One comment from a recent crash report: "FF Android crashes frequently when scrolling or switching orientation. Seems to correlate with large numbers of images on a page, or attempting to scroll while rendering is progressing. Crashes sometimes, but more frequently will present a blank white page (pull to refresh usually reveals a bit of the actual content while pulled down, but doesn't resolve it). Makes FF Android increasingly unpleasant to use."

Currently the top crashing device is moto g(6), followed by several other Motorola devices all running API 26.

Crash Signature: [@ libGLESv2_adreno.so@0x34121a] → [@ libGLESv2_adreno.so@0x102338] [@ libGLESv2_adreno.so@0x1e2950] [@ libGLESv2_adreno.so@0x34121a] [@ libGLESv2_adreno.so@0xe9e24]
We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → INCOMPLETE
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.