1264505 - crash in libGLES_mali.so@0x107d8

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect)

Description

[u279076]

This bug was filed from the Socorro interface and is 
report bp-99e300e4-69f1-42f6-bb35-a1abb2160413.
=============================================================
 0 	libGLES_mali.so 	libGLES_mali.so@0x107d8 	
Ø 1 	libGLES_mali.so 	libGLES_mali.so@0x17806 	
Ø 2 	libGLES_mali.so 	libGLES_mali.so@0x12e12 	
Ø 3 	libGLES_mali.so 	libGLES_mali.so@0x17cde 	
Ø 4 	libGLES_mali.so 	libGLES_mali.so@0x52fd6 	
Ø 5 	libGLES_mali.so 	libGLES_mali.so@0x52f52 	
Ø 6 	libGLES_mali.so 	libGLES_mali.so@0x1305e 	
7 	libGLESv2.so 	glGetInternalformativ 	/builds/slave/b2g_m-b44_25_flm-kk_eng_ntly-0/build/frameworks/native/opengl/libs/GLES2/gl3_api.in:737
8 	libxul.so 	mozilla::layers::EGLImageTextureSource::BindTexture 	gfx/gl/GLContext.h
9 	libxul.so 	mozilla::layers::CompositorOGL::DrawQuad 	gfx/layers/opengl/CompositorOGL.cpp
10 	libxul.so 	mozilla::layers::Compositor::DrawQuad 	gfx/layers/Compositor.h
11 	libxul.so 	mozilla::layers::ImageHost::Composite 	gfx/layers/composite/ImageHost.cpp
12 	libxul.so 	libxul.so@0x7cdde5 	
13 	libxul.so 	mozilla::layers::RenderWithAllMasks<mozilla::layers::CanvasLayerComposite::RenderLayer(const IntRect&)::<lambda(mozilla::layers::EffectChain&, const Rect&)> > 	gfx/layers/composite/LayerManagerComposite.h
14 	libxul.so 	mozilla::layers::CanvasLayerComposite::RenderLayer 	gfx/layers/composite/CanvasLayerComposite.cpp
15 	libxul.so 	mozilla::layers::RenderLayers<mozilla::layers::ContainerLayerComposite> 	gfx/layers/composite/ContainerLayerComposite.cpp
16 	libxul.so 	mozilla::layers::ContainerRender<mozilla::layers::ContainerLayerComposite> 	gfx/layers/composite/ContainerLayerComposite.cpp
17 	libxul.so 	mozilla::layers::RenderLayers<mozilla::layers::ContainerLayerComposite> 	gfx/layers/composite/ContainerLayerComposite.cpp
18 	libxul.so 	mozilla::layers::ContainerRender<mozilla::layers::ContainerLayerComposite> 	gfx/layers/composite/ContainerLayerComposite.cpp
19 	libxul.so 	mozilla::layers::LayerManagerComposite::Render 	gfx/layers/composite/LayerManagerComposite.cpp
20 	libxul.so 	mozilla::layers::LayerManagerComposite::UpdateAndRender 	gfx/layers/composite/LayerManagerComposite.cpp
21 	libxul.so 	mozilla::layers::LayerManagerComposite::EndTransaction 	gfx/layers/composite/LayerManagerComposite.cpp
22 	libxul.so 	mozilla::layers::CompositorParent::CompositeToTarget 	gfx/layers/ipc/CompositorParent.cpp
23 	libxul.so 	mozilla::layers::CompositorVsyncScheduler::Composite 	gfx/layers/ipc/CompositorParent.cpp
24 	libxul.so 	MessageLoop::RunTask 	ipc/chromium/src/base/message_loop.cc
25 	libxul.so 	MessageLoop::DeferOrRunPendingTask 	ipc/chromium/src/base/message_loop.cc
26 	libxul.so 	MessageLoop::DoWork 	ipc/chromium/src/base/message_loop.cc
27 	libxul.so 	base::MessagePumpDefault::Run 	ipc/chromium/src/base/message_pump_default.cc
28 	libxul.so 	MessageLoop::Run 	ipc/chromium/src/base/message_loop.cc
29 	libxul.so 	base::Thread::ThreadMain 	ipc/chromium/src/base/thread.cc
30 	libxul.so 	ThreadFunc 	ipc/chromium/src/base/platform_thread_posix.cc
Ø 31 	libc.so 	libc.so@0xd242 	
Ø 32 	libc.so 	libc.so@0xd3de 	
Ø 33 	dalvik-heap (deleted) 	dalvik-heap (deleted)@0x3e219db 	
=============================================================
More reports: https://crash-stats.mozilla.com/report/list?product=FennecAndroid&signature=libGLES_mali.so%400x107d8

Stats:
======
Fennec 46.0b10: #3 @ 3.93% with 51 crashes
Fennec 45.0.2: #3 @ 3.83% with 1843 crashes over 7 days.
Fennec 45.0.1: #4 @ 2.83% with 14969 crashes over 7 days.

Correlations:
=============
Using supersearch it appears this is 100% correlated to Mali-400 MP chipsets.

It looks like this has been a long standing issue as reports go back to Fennec 35 but is quite high in Release/Beta.

Comment 1

[u279076]

Milan, I'm not sure who best to look at this.

Comment 2

[Milan Sreckovic [:milan] (needinfo for best results)]

Jamie, James, is it likely that we need to also "blocklist" this Mali chipset for the problem in bug 1194923, and not just a particular Adreno?

Comment 3

[Jamie Nicol [:jnicol]]

This looks like an unrelated bug to me, so I don't think that would help.

A lot of the comments seem to indicate this can be easily reproduced on youtube or other video sites - do we have any mali 400 devices to verify with? (I think I have a cubieboard 2 devboard lying around somewhere. Might be a bit tricky to get android set up on it but I can try to find it then give it a go.)

Is it a wee bit strange that in all the crash reports the function in libGLESv2.so that gets called by us is glGetInternalformativ() rather than EGLImageTargetTexture2DOES? Perhaps there's some problem with loading that extension on that GPU, or maybe it's just the crash reports are a bit funny?

Comment 4

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Jamie Nicol [:jnicol] from comment #3)
> Is it a wee bit strange that in all the crash reports the function in
> libGLESv2.so that gets called by us is glGetInternalformativ() rather than
> EGLImageTargetTexture2DOES? Perhaps there's some problem with loading that
> extension on that GPU, or maybe it's just the crash reports are a bit funny?

Yeah, that's most likely crash-stats being stupid. The debug id of the libGLESv2.so matches one that we have symbols for from b2g, so it tries to use that. I don't think we're actually calling glGetInternalformativ() here.

Comment 5

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Looks to be Spreadtrum SoCs specifically with Mali 400 GPUs. Might be a driver thing?

Most popular device by far is Huawei Mediapad T1 7.0. Will order one for the London office.

Comment 6

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Filed request RITM0045395 for device

Comment 7

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

This SoC(/driver) seems to have issues in glEGLImageTargetTexture2DOES() when passed the GL_TEXTURE_EXTERNAL_OES texture target.

As a workaround, GL_TEXTURE_2D seems to work. We used to use GL_TEXTURE_2D at [1], but that was changed in bug 1144906. It is not clear why.

[1]  https://dxr.mozilla.org/mozilla-central/rev/b69a5bbb5e40bd426e35222baa600b481e50d265/gfx/layers/opengl/TextureHostOGL.cpp#670

Comment 8

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Jeff, can you remember why it was changed?

Comment 9

[Kelsey Gilbert [:jgilbert]]

cset: https://hg.mozilla.org/mozilla-central/rev/10f2a5e84c91
bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1144906

Comment 10

[Kelsey Gilbert [:jgilbert]]

EXTERNAL is more accurate, because it's a read-only view.

If EXTERNAL is broken here, mark that extension as unsupported for this driver, and add a fallback path for the OES_EGL_image ext.

Comment 11

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Found a bunch more signatures with the same stacks. Not limited to Spreadtrum as previously thought.

Comment 12

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Attachment: 1264505-disable-external.patch

This patch disables OES_EGL_image_external for Mali-400 MP GPUs.

Comment 13

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Attachment: 1264505-unused.patch

Get rid of a seemingly unused caller to EGLImageTargetTexture2D.

Comment 14

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Attachment: 1264505-texture2d.patch

Use GL_TEXTURE_2D if OES_EGL_image is available but OES_EGL_image_external is not.

Comment 15

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8768372 [details] [diff] [review]
1264505-disable-external.patch

Review of attachment 8768372 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/gl/GLContext.cpp
@@ +1744,5 @@
>              MarkExtensionUnsupported(OES_EGL_sync);
>          }
>  
> +        if (Vendor() == GLVendor::ARM &&
> +            Renderer() == GLRenderer::Mali400MP) {

Wrap { to new line for multi-line conditionals.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ff3c444a42ae
Disable OES_EGL_image_external extension on Mali-400 MP - r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/eedb7343813b
Remove unused method SharedSurface_EGLImage::AcquireConsumerTexture - r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/a08e5896b830
Target EGLImage to GL_TEXTURE_2D if OES_EGL_image_external extension is not available - r=jgilbert

Comment 17

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Comment on attachment 8768372 [details] [diff] [review]
1264505-disable-external.patch

Approval Request Comment
[Feature/regressing bug #]: Canvas2D
[User impact if declined]: Crashes -- loads of crashes
[Describe test coverage new/current, TreeHerder]: Little
[Risks and why]: Small chance of breaking canvas elsewhere
[String/UUID change made/needed]: None

Comment 18

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/ff3c444a42ae
https://hg.mozilla.org/mozilla-central/rev/eedb7343813b
https://hg.mozilla.org/mozilla-central/rev/a08e5896b830

Comment 19

[Gerry Chang [:gchang]]

Comment on attachment 8768372 [details] [diff] [review]
1264505-disable-external.patch

Review of attachment 8768372 [details] [diff] [review]:
-----------------------------------------------------------------

This patch fixes a crash. Take it in 48 beta 8 and aurora.

Comment 20

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/3ad93af112af

Comment 21

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/111933946ff4