1218607 - [meta] Firefox on android crashing in [@ libGLESv2_adreno.so@ ...]

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect, P3)

Description

[alessandro]

User Agent: Mozilla/5.0 (Android 5.1.1; Mobile; rv:44.0) Gecko/44.0 Firefox/44.0
Build ID: 20151026030422

Steps to reproduce:

zooming any and out any web pages after two or three time makes nightly 44 on android either crashing or makes web pages all light gray.
it over a month it s happening PLEAE FIX IT

Comment 1

[H. D. Trestini]

Same here. Build 44 Oa2

Comment 2

[Andre Klapper]

Needs a crash ID / stacktrace.

Comment 3

[H. D. Trestini]

Hello. How can i upload the information you are requesting?

Comment 4

[H. D. Trestini]

Attachment: tmp_19664-Submitted Crash Reports-1316348657.pdf

List from about:crashes

Comment 5

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/message/reply/question/etc.)]

[Tracking Requested - why for this release]: Regression

Some crashlog reports:
https://crash-stats.mozilla.com/report/index/3bea5d58-af8c-4bf3-a972-b7c2b2160203
https://crash-stats.mozilla.com/report/index/f0e92290-a1e9-49f2-8386-466ac2160102
https://crash-stats.mozilla.com/report/index/805b68b2-35c8-4237-ba7c-378bb2160102
https://crash-stats.mozilla.com/report/index/7dea52b9-f919-4a49-a2f7-19b5e2160101

Looks like it's top crash on Android.

Comment 6

[Benjamin Kerensa [:bkerensa]]

Milan can someone look at this?

Comment 7

[Jamie Nicol [:jnicol]]

Looks like more OOMs.

Alessandro, is this actually any site, or just some (or even most) that you regularly browse?

Some urls would be helpful.

Comment 8

[Kelsey Gilbert [:jgilbert]]

It looks like it's generally under glDrawArrays, however at least one report is not: (though it is crashing in libGLES2)
https://crash-stats.mozilla.com/report/index/f0e92290-a1e9-49f2-8386-466ac2160102

I echo that this may be OOMs, but in the driver. The driver may be lazily creating a non-trivial buffer.

Comment 9

[H. D. Trestini]

@Jamie: this happens with any site, especially if they contain a lot of graphics, embedded flash videos, etc.  Regards, Hector.

Comment 10

[H. D. Trestini]

Here's another example.  This crash happened a few minutes ago, scrolling through a list of books founs athttp://hundredzeros.com/:

https://crash-stats.mozilla.com/report/index/e1657dea-f3dc-4b4e-920c-f9b982160210

Comment 11

[Jamie Nicol [:jnicol]]

Thanks very much for the url. The reason I'm trying to distinguish between what feels like any site and truly *any* site, is that while I'm sure they're equally frustrating for affected users, knowing specific sites massively helps us find and fix the problems.

On http://hundredzeros.com, I notice that sometimes each of the book "cards" gets its own layer. This seems occur if I scroll as the page is still loading. This causes us to allocate a huge number of textures, and crash. I'll do some sleuthing into why we're giving each card its own layer.

Comment 12

[Jamie Nicol [:jnicol]]

We layerize the cards because javascript is setting the "left" and "top" properties, and we therefore immediately decide the items are animated and give them their own layers. The page actually works fine with javascript disabled - it seems to only set these values once, and it sets them to the existing value anyway.

Here we have a wider problem - we cannot go about creating this many medium-sized layers and thinking we'll get away with it.

But also a more specific problem - should setting those properties once from javascript indicate the frame is animated?

Some of the public methods in ActiveLayerTracker increment the restyle counts by 1 at a time, and we only consider a frame animated at 2 or more. But NotifyInlineStyleRuleModified(), here called by nsDOMCSSAttributeDeclaration::SetPropertyValue(), immediately sets it to 0xFF. Should we reconsider this? Also, should we consider ignoring a property set when its value doesn't actually change?

Comment 13

[Jamie Nicol [:jnicol]]

Filed 1247336 for above issue.

More urls would help find more :)

Comment 14

[u279076]

Most of the URLs appear to be sports related websites (sanitized for privacy):
67 	http://sports.williamhill.com/
65 	about:blank
58 	http://www.t-online.de/ 
43 	http://m.bild.de/
33 	http://mobil.express.de/
22      http://app.handelsblatt.com/
13 	http://m.faz.net/
11 	http://m.heute.at/
 8 	http://www.wordswithfriendscheat.org/
 6 	about:home
 6 	http://www.eurosport.fr/
 6 	http://www.lidl.de/
 6 	http://www.skysports.com/football
 5      https://www.facebook.com/
 4 	http://m.autoanything.com/
 4 	http://m.topky.sk/
 4 	http://www.britishairways.com/
 4 	http://www.marcustheatres.com/
 4 	https://fr.yahoo.com/
 3 	http://digg.com/
 3 	http://eclecticallyvintage.com/
 3 	http://horizon.celestus.fr/game.php
 3 	http://m.sport1.de/
 3 	http://m.sportbild.bild.de/
 3 	http://mobil.mopo.de/
 3 	http://www.allocine.fr/
 3 	http://www.blick.ch/
 3 	http://www.cracked.com/
 3 	http://www.eurosport.de/
 3 	http://www.hollywood.com/
 3 	http://www.meteociel.fr/
 3 	http://www.wrestlingrumors.net/
 3 	http://www.zuhause.de/
 3 	https://play.google.com/store/apps/
 3 	https://web.whatsapp.com/

Comment 15

[H. D. Trestini]

None of those are sites I visited, so I can't give you details.  I will post some for your information soon, after I go through my browser history...

Comment 16

[Kevin Brosnan [Ex-Mozilla]]

If you have URL access https://crash-stats.mozilla.com/search/?product=FennecAndroid&signature=%24libGLESv2_adreno.so&url=%24http&_facets=signature&_facets=url&_columns=url#facet-url provides URLs from all signatures that start with libGLESv2_adreno.so

Comment 17

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Given that this one does not have a fix yet, too late for Fx44.

Comment 18

[H. D. Trestini]

The following crash (https://crash-stats.mozilla.com/report/index/bp-cd2e35f4-2ff8-4789-a331-cce7e2160217) occurred happened after clicking on the Show Other Threads link and zomming in at the following sitehttps://crash-stats.mozilla.com/report/index/51247b54-496e-4c15-9a23-1d1c32160216.

Comment 19

[H. D. Trestini]

(In reply to H. D. Trestini from comment #18)
> The following crash
> (https://crash-stats.mozilla.com/report/index/bp-cd2e35f4-2ff8-4789-a331-
> cce7e2160217) occurred happened after clicking on the Show Other Threads
> link and zomming in at the following
> sitehttps://crash-stats.mozilla.com/report/index/51247b54-496e-4c15-9a23-
> 1d1c32160216.

The message should have read:

The following crash (https://crash-stats.mozilla.com/report/index/bp-cd2e35f4-2ff8-4789-a331-cce7e2160217) occurred after clicking on the Show Other Threads link and zooming in on the page, at the following site: https://crash-stats.mozilla.com/report/index/51247b54-496e-4c15-9a23-1d1c32160216.

Comment 20

[Milan Sreckovic [:milan] (needinfo for best results)]

(In reply to H. D. Trestini from comment #19)
> ...
> https://crash-stats.mozilla.com/report/index/51247b54-496e-4c15-9a23-
> 1d1c32160216.

This last one has a few of these failures: https://dxr.mozilla.org/mozilla-central/source/gfx/layers/client/TiledContentClient.cpp#775 (we fail to get the back buffer.)

Comment 21

[Milan Sreckovic [:milan] (needinfo for best results)]

Jeff, is this actionable on our side?

Comment 22

[Kelsey Gilbert [:jgilbert]]

(In reply to Milan Sreckovic [:milan] from comment #21)
> Jeff, is this actionable on our side?

I cannot reproduce it with my hardware. (2GB Adreno 320) I'll see if I have a lower-spec Adreno to test with.

I do not think this is easily actionable. I think we need to add heuristics to reduce/eliminate layerization when system memory is low.

Comment 23

[Milan Sreckovic [:milan] (needinfo for best results)]

Jamie, is any of your layerization work going to help here?

Comment 24

[James Willcox (:snorp) ([e-mail]) (he/him)]

Jamie is on PTO, but I think I can add a partial answer. The work in bug 1247554 does limit some of our layerization activity, but not all of it. We still need a more general solution that will allow us to put cap on the max number of layer pixels.

Comment 25

[Jamie Nicol [:jnicol]]

we're also uplifting bug 1247336 and bug 1250517 which should help on certain sites. the other layerization fixes I've managed this cycle are unfortunately only fixing regressions on nightly so won't help stable.

I'm going to look into what snorp mentioned in the above comment next. unfortunately it's hard to do accurately. we could also implement something like bug 1247437 as a safety net. hopefully we can get those done next cycle.

Comment 26

[Kelsey Gilbert [:jgilbert]]

(In reply to Jamie Nicol [:jnicol] from comment #25)
> we're also uplifting bug 1247336 and bug 1250517 which should help on
> certain sites. the other layerization fixes I've managed this cycle are
> unfortunately only fixing regressions on nightly so won't help stable.
> 
> I'm going to look into what snorp mentioned in the above comment next.
> unfortunately it's hard to do accurately. we could also implement something
> like bug 1247437 as a safety net. hopefully we can get those done next cycle.

Alright, it sounds like we're investigating paths forward elsewhere. Unassigning myself now since investigation is more or less done.

Comment 27

[Liz Henry (:lizzard) (privacy/fingerprinting team)]

Should we close this bug then? Nothing actionable, work continuing in other bugs?
Wontfix for 46, in the meantime.

Comment 28

[Milan Sreckovic [:milan] (needinfo for best results)]

Making this a meta bug

Comment 29

[u279076]

Dropping the *wanted keywords and status flag since this is now a Meta bug. We can track those things in the dependencies.

Comment 30

[Milan Sreckovic [:milan] (needinfo for best results)]

[Tracking Requested - why for this release]: Doesn't make sense to track a meta bug.

Comment 31

[u279076]

Adding more signatures to this bug.

Comment 32

[u279076]

We've seen a 4x dip in crashes starting on August 4, going from ~400/day to ~130/day placing this around #40 @ 0.71% in Fennec 48 at this point.

Comment 33

[u279076]

Crashes reported over the last week:
> 49.*:    88
> 50.*: 1,290
> 51.*:   108
> 52.*:     3
> 53.*:     1

Comment 34

[Julien Cristau [:jcristau]]

Too late for firefox 52, mass-wontfix.

Comment 35

[Intermittent Failures Robot]

1 failures in 5444 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1218607&startday=2019-11-18&endday=2019-11-24&tree=all

Comment 36

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:bhood, since the bug has high severity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 44

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 45

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 46

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 47

[Jeff Muizelaar [:jrmuizel]]

Will, can we adjust the signature generation so that we don't include the address for libGLESv2_adreno.so like we do dlls with no symbols on Windows? I remember us making this change on Windows but don't remember the details of how we did that.

Comment 48

[Will Kahn-Greene [:willkg] ET needinfo? me]

Jeff: Can you write up a bug for that? I don't remember offhand what we do for DLLs with no symbols. We can probably get someone to look at it in the next week.

Comment 49

[Jeff Muizelaar [:jrmuizel]]

I filed bug 1895527

Comment 50

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.