Open Bug 1374570 Opened 3 years ago Updated 12 days ago

Crash in Motorola libhwui.so@0x42b46

Categories

(GeckoView :: General, defect, P2, critical)

Unspecified
Android
defect

Tracking

(firefox-esr60 wontfix, firefox-esr68 affected, firefox54 wontfix, firefox55 wontfix, firefox56 wontfix, firefox57 wontfix, firefox58 wontfix, firefox59 wontfix, firefox60 wontfix, firefox61 wontfix, firefox62 wontfix, firefox63 wontfix, firefox64 wontfix, firefox65- wontfix, firefox66 wontfix, firefox67 wontfix, firefox68 wontfix, firefox69 affected, firefox70 affected)

Tracking Status
firefox-esr60 --- wontfix
firefox-esr68 --- affected
firefox54 --- wontfix
firefox55 --- wontfix
firefox56 --- wontfix
firefox57 --- wontfix
firefox58 --- wontfix
firefox59 --- wontfix
firefox60 --- wontfix
firefox61 --- wontfix
firefox62 --- wontfix
firefox63 --- wontfix
firefox64 --- wontfix
firefox65 - wontfix
firefox66 --- wontfix
firefox67 --- wontfix
firefox68 --- wontfix
firefox69 --- affected
firefox70 --- affected

People

(Reporter: gchang, Unassigned)

References

Details

(Keywords: crash, stalled, topcrash)

Crash Data

This bug was filed from the Socorro interface and is 
report bp-bd6aa914-a26d-4c15-bce7-cf8d70170619.
=============================================================
Frame 	Module 	Signature 	Source
Ø 0 	libhwui.so 	libhwui.so@0x42b46 	
Ø 1 	libhwui.so 	libhwui.so@0x435d3 	
Ø 2 	libhwui.so 	libhwui.so@0x4365f 	
Ø 3 	libhwui.so 	libhwui.so@0x4513f 	
Ø 4 	libhwui.so 	libhwui.so@0x56662 	
Ø 5 	libc.so 	libc.so@0x1379b 	
Ø 6 	libutils.so 	libutils.so@0xef55 	
Ø 7 	libandroid_runtime.so 	libandroid_runtime.so@0xf58de 	
Ø 8 	libutils.so 	libutils.so@0xeee3 	
Ø 9 	libandroid_runtime.so 	libandroid_runtime.so@0x5fd21 	
Ø 10 	libandroid_runtime.so 	libandroid_runtime.so@0x5fcd7 	
Ø 11 	libutils.so 	libutils.so@0xeac5 	
Ø 12 	libutils.so 	libutils.so@0xea8b 	
Ø 13 	libc.so 	libc.so@0x137bb 	
Ø 14 	libc.so 	libc.so@0x1379b 	
Ø 15 	libc.so 	libc.so@0x1379b 	
Ø 16 	libc.so 	libc.so@0x1189b 	
Ø 17 	dalvik-main space (deleted) 	dalvik-main space (deleted)@0x1651d4d3

This is #19 topcrash for Firefox for android and there is a spike in the last 3 days.
Adding additional affect branches. This crash seems to only affect two Samsung devices, SM-N9005 and SM-N900U. Comments don't seem to be particularly useful.
In 57 release this signature has 1821 crashes, and there are a few other similar signatures which affects Samsung devices.
Adding 59 and 60 as affected. This is #6 overall browser crash for 58.0.2. It looks as if one Samsung device is generating most of the crashes: SM-N9005
Continues to be quite high in 58.0.2, and 98% of the crashes are GALAXY Note 3 running API 21.
[triage] #8 top crasher.
tracking-fennec: --- → ?
Flags: needinfo?(sdaswani)
Priority: -- → P1
Putting this on Softvision's list.
Flags: needinfo?(sdaswani)
Whiteboard: [Leanplum]
We have tried to reproduce this issue on our Note 3 running Android 5.0 but in the absence of any STRs or actionable stacktraces we could use more info in order to go further with this
Flags: needinfo?(sdaswani)
Marcia, no luck trying to repro - can you suggest next steps?
Flags: needinfo?(sdaswani) → needinfo?(mozillamarcia.knous)
Updating affected branches. I will take a look at the crash reports and see if there are any other clues in the comments.
Flags: needinfo?(mozillamarcia.knous)
I also have an affected Note 3 SM-N9005 device. Firefox v59 and v60 beta crash at least daily. Is there a workaround? I've set "layers.acceleration.disabled;true" and "layers.acceleration.force-enabled;false" in about:config; but Firefox keeps crashing.
Still no news for Note 3 users? Firefox keeps crashing for no reason multiple times per day. Also version 58 crashes, but not so often.
#6 overall top crash on release. Updating flags to reflect that this affects 62/63. SM-N9005 is the primary device crashing.
Let's keep this in triage to follow up on the crash rate for 63 release. 
I think the next step here is to get Vlad access to see the crash reports in the Play Store. 
It is likely too late to fix this on 63 but we could aim to investigate it for 65, and if we arrive at a fix then consider uplift to 64 beta.
63.0.2 has 1824 crashes, almost all from Samsung SM-N9005.
There are a lot of related or possible dupes: bug 1492183, bug 1510984, bug 1235662, bug 1430409, bug 1476077
Crash Signature: [@ libhwui.so@0x42b46] → [@ hwui.so@0x3dff2] [@ hwui.so@0x42b46] [@ hwui.so@0x42f46] [@ hwui.so@0x4830e] [@ hwui.so@0x8201e] [@ hwui.so@0x844a6] [@ hwui.so@0x8699e] [@ hwui.so@0x86d4e] [@ hwui.so@0x86de6] [@ hwui.so@0x870c6] [@ libhwui.so@0x2fe32] [@ libhwui.so@0x42b46…
Keywords: stalled
Flagging this because it is tracking 65 and has no assignee, and it shows up on a list I check weekly.

(In reply to Marcia Knous [:marcia - needinfo? me] from comment #14)
> 63.0.2 has 1824 crashes, almost all from Samsung SM-N9005.

Crashy hwui.so -- anything we should do here? Kevin, NI'ing you in case you want to research it a bit.
Flags: needinfo?(sdaswani)
Flags: needinfo?(kbrosnan)
Clearing my NI since it's on Liz'a radar.
Flags: needinfo?(sdaswani)

This seems very high volume on release.

Whiteboard: [Leanplum] → [Leanplum] [geckoview]

I still see lots of crashes with Samsung devices - SM-N9005 (Samsung Galaxy Note 3) running API 21.

Other devices include:

  • moto g(6)
  • Samsung Galaxy S7
  • Samsung Galaxy S8

ni on Ioana to see if someone can reproduce this one of the affected devices.

Flags: needinfo?(ioana.chiorean)

We do have the Samsung devices (but not the Moto one - i can ask in the company maybe others have it) so we will try to see if we can catch it with them.
Kevin do u have by chance the moto 6?

Flags: needinfo?(ioana.chiorean)

I tested this issue on the RC build 64.0.2 but I wasn't able to reproduce it, with the following devices:
Samsung Galaxy 8(A 8.0), Samsung Galaxy S8+(A 8.0), Samsung Galaxy Note 9(A 8.1.0), Samsung Galaxy S7(A 7.0).
Note that I didn't find a Motorola g(6) and I will try to find one as Ioana said in comment 20.
I will keep an eye out for this issue while testing and if il manage to reproduce it or find any relevant information, ill get back and post them.

I have a G5 not a G6.

Flags: needinfo?(kbrosnan)

Susheel do we get any interesting info out of playstore for these signatures?

Flags: needinfo?(sdaswani)

Sylvestre can you add David to the Play Store for Fennec and Focus so he can take a look at crash sigs? Thanks!

Flags: needinfo?(sdaswani) → needinfo?(sledru)

Sure, you should have permissions now David! (on the 3 fx products: release, beta & nightly)

Flags: needinfo?(sledru)

Looking at the logcats here it's OOM. Most likely just address space exhaustion.

Interesting that the crash volume steadily increased from about 400/day in October to almost 1000/day in December. Perhaps there is an external factor, like the rollout of a new OS update, contributing to this issue.

Whiteboard: [Leanplum] [geckoview]

In looking further at some of the higher volume crashes, here are some data points:

libhwui.so@0x86da6

  • 45% of the crashes are from SM-G965U (Galaxy S9)
  • All of the crashes are Samsung devices running API 26

libhwui.so@0x81fde

  • Largely Motorola devices, all running API 26

libhwui.so@0x86dae

  • All API 26
  • 83% of the crashes are from SM-N950U (Galaxy Note 8)

libhwui.so@0x9dc06

  • All API 27
  • 88% are from SM-N960U (Galaxy Note 9)

Firefox 66.0.2 always crashes on www.foodist.at with Motorola G6 (four attempts, four crashes):
https://crash-stats.mozilla.org/report/index/3874a5e0-dc92-41f6-8db7-084420190402

(In reply to Henrik Gemal from comment #29)

https://www.power.dk/kampagne/powerweekend/#campaign-nav

(In reply to Sören Hentzschel from comment #30)

Firefox 66.0.2 always crashes on www.foodist.at with Motorola G6 (four attempts, four crashes):

Neither power.dk or foodist.at crash on my Moto G5 Plus, so this must be something specific to the Moto G6.

Should the stalled tag remain given this new information?

Updating flags. In 67b11 this was the #6 overall crash.

Hi, I am using Firefox 68.0b5 on a Moto G5S running Android 8.1.0. Since a few days I am experiencing situations in which Firefox stops rendering a page while it is loading. That tab and all other tabs become blank and eventually Firefox crashes. Sometimes I just close the Firefox app before it actually crashes. I think the crash started with the update to beta5. The signature is libhwui.so@0x2fe32, which crash-stats relates to this bug.

I can also always reproduce the crash on www.foodist.at mentioned in the previous comment. The symptoms are the same (all tabs become blank), but the signature is different (libESXGLESv2_adreno.so@0x3476e2).

The largest volume signature in the 68 release is https://mzl.la/2XNX8NB. This is currently the #11 overall top crash for Fennec 68 release.

From that, here are the correlations - high correlation to Motorola:

(100.0% in signature vs 08.69% overall) reason = SIGABRT
(95.27% in signature vs 03.17% overall) Module "u:object_r:product_prop:s0" = true
(95.27% in signature vs 06.40% overall) Module "u:object_r:display_id_prop:s0" = true
(95.27% in signature vs 06.90% overall) Module "Newfont_Light.ttf" = true
(95.27% in signature vs 07.41% overall) android_brand = motorola
(95.27% in signature vs 07.47% overall) android_manufacturer = motorola
(100.0% in signature vs 14.07% overall) android_version = 27 (REL)

Jesse, this medium-volume Android crash looks like a crash in Motorola graphics drivers. I'm not sure what we can do. I hear the Graphics team has had some luck in the past working around Android driver crashes by rearranging our GPU calls by trial and error.

Jamie - could you take a peak when you have a chance?

Flags: needinfo?(jbonisteel) → needinfo?(jnicol)

This crash is caused by OOM. I can reproduce on power.dk and foodist.at on my G6. On those sites we're layerising poorly, meaning we create a lot of textures, run out of memory, and the driver crashes.

The problem with trying to fix layerisation issues is that it's a game of whack-a-mole, changing the heuristics to make one site work better can cause issues on other sites, and even fixing specific sites can have very little effect on the overall numbers because each site has its own unique problems.

With fennec being in maintainence mode, I'm not sure this is worth putting the time in to fix. GeckoView should improve things a lot, presumably due to being multi-process (I can't reproduce a crash on those sites with geckoview). And webrender should improve things further too.

Flags: needinfo?(jnicol)

Some of these signatures currently show up in Fenix as well, see https://bit.ly/2YvNHSw. Will Bug 1530770 help things here?

I forgot to add in my last comment that this crash is #8 overall in the current Fennec 68 release.

See Also: → 1573584
Duplicate of this bug: 1573584

Moving to the GeckoView Bugzilla component because this random libhwui.so crash affects both Fenix/GV and Fennec.

tracking-fennec: ? → ---
Product: Firefox for Android → GeckoView
Version: 54 Branch → unspecified

This Motorola crash signature's volume is still high (about 800 reports per day), but we don't have a clear solution or workaround in hand.

Crash Signature: libhwui.so@0x42b46] [@ libhwui.so@0x81fde] [@ libhwui.so@0x8201e] [@ libhwui.so@0x86d16] [@ libhwui.so@0x86d46] [@ libhwui.so@0x86da6] [@ libhwui.so@0x86dae] [@ libhwui.so@0x9dc06] → libhwui.so@0x2fe86] [@ libhwui.so@0x2febe] [@ libhwui.so@0x30b3a] [@ libhwui.so@0x30c7e] [@ libhwui.so@0x42836] [@ libhwui.so@0x42b46] [@ libhwui.so@0x81fde] [@ libhwui.so@0x8201e] [@ libhwui.so@0x86d16] [@ libhwui.so@0x86d46] [@ libhwui.so@0x8…
Rank: 50
Summary: Crash in libhwui.so@0x42b46 → Crash in Motorola libhwui.so@0x42b46
You need to log in before you can comment on or make changes to this bug.