Closed Bug 930884 Opened 6 years ago Closed 6 years ago

Intermittent PROCESS-CRASH | Shutdown | application crashed [@ 0x0] [@ egl_window_surface_t::~egl_window_surface_t] [@ GLContextEGL::~GLContextEGL]

Categories

(Core :: Graphics: Layers, defect, critical)

Product:
Core
Component:
Graphics: Layers
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla30
Tracking Flags:
Tracking Status
firefox28 --- wontfix
firefox29 --- wontfix
firefox30 --- fixed
firefox-esr24 --- unaffected
b2g-v1.3 --- affected
b2g-v1.4 --- fixed

People

(Reporter: cbook, Assigned: jerry)

References

(
URL
)

Details

(Keywords: crash, intermittent-failure)

Crash Data

Attachments

(3 files, 7 obsolete files)

development/tools/emulator/opengl/system/egl/egl.cpp
407 bytes, text/plain
Details
runreftestb2g.py
23.80 KB, text/x-python
Details
cancel_buffer.patch r=mwu
1.71 KB, patch
Details | Diff | Splinter Review 
b2g_emulator_vm mozilla-inbound opt test mochitest-3 on 2013-10-24 15:07:21 PDT for push 541750e89771

slave: tst-linux64-ec2-105

https://tbpl.mozilla.org/php/getParsedLog.php?id=29625837&tree=Mozilla-Inbound

PROCESS-CRASH | Shutdown | application crashed [@ 0x0]
Return code: 2304
Output exceeded 52428800 bytes, remaining output has been truncated
FYI, pasting the crash stack out of the log is useful for intermittent failures that include a crash:

16:05:26     INFO -  Crash reason:  SIGSEGV
16:05:26     INFO -  Crash address: 0x0
16:05:26     INFO -  Thread 20 (crashed)
16:05:26     INFO -   0  0x0
16:05:26     INFO -       r4 = 0x46fd5490    r5 = 0x00000001    r6 = 0x4036a068    r7 = 0x46d11f30
16:05:26     INFO -       r8 = 0x458ffcec    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffb10    lr = 0x435f2465    pc = 0x00000000
16:05:26     INFO -      Found by: given as instruction pointer in context
16:05:26     INFO -   1  libEGL_emulation.so!egl_window_surface_t::~egl_window_surface_t [egl.cpp : 306 + 0x3]
16:05:26     INFO -       sp = 0x458ffb18    pc = 0x435f25e1
16:05:26     INFO -      Found by: stack scanning
16:05:26     INFO -   2  libEGL_emulation.so!eglDestroySurface [egl.cpp : 713 + 0x7]
16:05:26     INFO -       r4 = 0x46fd54c0    sp = 0x458ffb20    pc = 0x435f44fb
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   3  libEGL.so!eglDestroySurface [eglApi.cpp : 449 + 0xf]
16:05:26     INFO -       r4 = 0x46fd54c0    r5 = 0x46fd54c0    r6 = 0x4036a068    sp = 0x458ffb40
16:05:26     INFO -       pc = 0x403534b5
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   4  libxul.so!mozilla::gl::GLContextEGL::~GLContextEGL [GLLibraryEGL.h:541750e89771 : 187 + 0x7]
16:05:26     INFO -       r4 = 0x474b9000    r5 = 0x422a0c0c    r6 = 0x422a07b8    sp = 0x458ffb58
16:05:26     INFO -       pc = 0x4164e583
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   5  libxul.so!mozilla::gl::GLContextEGL::~GLContextEGL [GLContextProviderEGL.cpp:541750e89771 : 290 + 0x3]
16:05:26     INFO -       r4 = 0x474b9000    r5 = 0x00000000    r6 = 0x422a07b8    sp = 0x458ffb68
16:05:26     INFO -       pc = 0x4164e5b1
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   6  libxul.so!mozilla::detail::GenericRefCounted<(mozilla::detail::RefCountAtomicity)0>::Release() [GenericRefCounted.h : 66 + 0x7]
16:05:26     INFO -       r4 = 0x474b9000    r5 = 0x00000000    r6 = 0x422a07b8    sp = 0x458ffb70
16:05:26     INFO -       pc = 0x4164e503
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   7  libxul.so!mozilla::RefPtr<mozilla::gl::GLContext>::~RefPtr [RefPtr.h : 203 + 0x5]
16:05:26     INFO -       r4 = 0x458ffb84    r5 = 0x00000000    r6 = 0x422a07b8    sp = 0x458ffb78
16:05:26     INFO -       pc = 0x412fc9ab
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   8  libxul.so!mozilla::layers::CompositorOGL::CleanupResources() [CompositorOGL.cpp:541750e89771 : 351 + 0x5]
16:05:26     INFO -       r4 = 0x46d4beb0    r5 = 0x00000000    r6 = 0x422a07b8    sp = 0x458ffb80
16:05:26     INFO -       pc = 0x41640433
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -   9  libxul.so!mozilla::layers::CompositorOGL::Destroy() [CompositorOGL.cpp:541750e89771 : 325 + 0x3]
16:05:26     INFO -       r4 = 0x46d4beb0    r5 = 0x00000000    r6 = 0x422a07b8    sp = 0x458ffb90
16:05:26     INFO -       pc = 0x41640497
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  10  libxul.so!mozilla::layers::LayerManagerComposite::Destroy() [LayerManagerComposite.cpp:541750e89771 : 132 + 0x5]
16:05:26     INFO -       r4 = 0x469cad30    r5 = 0x00000000    r6 = 0x422a07b8    sp = 0x458ffb98
16:05:26     INFO -       pc = 0x4162ae87
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  11  libxul.so!mozilla::layers::CompositorParent::RecvWillStop() [CompositorParent.cpp:541750e89771 : 258 + 0x5]
16:05:26     INFO -       r4 = 0x4366b460    r5 = 0x458ffbac    r6 = 0x422a07b8    sp = 0x458ffba8
16:05:26     INFO -       pc = 0x41633653
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  12  libxul.so!mozilla::layers::PCompositorParent::OnMessageReceived(IPC::Message const&, IPC::Message*&) [PCompositorParent.cpp : 380 + 0x7]
16:05:26     INFO -       r4 = 0x4366b460    r5 = 0x458ffccc    r6 = 0x00000000    sp = 0x458ffbc0
16:05:26     INFO -       pc = 0x4136cf9d
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  13  libxul.so!mozilla::ipc::MessageChannel::DispatchSyncMessage(IPC::Message const&) [MessageChannel.cpp:541750e89771 : 900 + 0x5]
16:05:26     INFO -       r4 = 0x4366b490    r5 = 0x458ffcec    r6 = 0x00000000    r7 = 0x46d11f30
16:05:26     INFO -       r8 = 0x46d11f30    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffcc8    pc = 0x4134ecff
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  14  libxul.so!mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message const&) [MessageChannel.cpp:541750e89771 : 881 + 0x3]
16:05:26     INFO -       r4 = 0x00000001    r5 = 0x458ffcec    r6 = 0x458ffd10    r7 = 0x46d11f30
16:05:26     INFO -       r8 = 0x46d11f30    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffce0    pc = 0x4134fcbf
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  15  libxul.so!mozilla::ipc::MessageChannel::OnMaybeDequeueOne() [MessageChannel.cpp:541750e89771 : 872 + 0x3]
16:05:26     INFO -       r4 = 0x00000001    r5 = 0x458ffcec    r6 = 0x458ffd10    r7 = 0x46d11f30
16:05:26     INFO -       r8 = 0x46d11f30    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffce8    pc = 0x4134fd85
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  16  libxul.so!RunnableMethod<WebCore::ReverbConvolver, void (WebCore::ReverbConvolver::*)(), Tuple0>::Run() [tuple.h:541750e89771 : 383 + 0x5]
16:05:26     INFO -       r4 = 0x458ffdec    r5 = 0x43765640    r6 = 0x458ffd68    r7 = 0x458ffdf8
16:05:26     INFO -       r8 = 0x458ffd60    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffd30    pc = 0x40fe6e3b
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  17  libxul.so!mozilla::ipc::MessageChannel::DequeueTask::Run() [MessageChannel.h : 440 + 0x9]
16:05:26     INFO -       r4 = 0x458ffdec    r5 = 0x43765640    r6 = 0x458ffd68    r7 = 0x458ffdf8
16:05:26     INFO -       r8 = 0x458ffd60    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffd38    pc = 0x4134dfad
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  18  libxul.so!MessageLoop::RunTask(Task*) [message_loop.cc:541750e89771 : 338 + 0x5]
16:05:26     INFO -       r4 = 0x458ffdec    r5 = 0x43765640    r6 = 0x458ffd68    r7 = 0x458ffdf8
16:05:26     INFO -       r8 = 0x458ffd60    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffd40    pc = 0x415ea081
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  19  libxul.so!MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask const&) [message_loop.cc:541750e89771 : 346 + 0x5]
16:05:26     INFO -       r4 = 0x00000001    r5 = 0x458ffd58    r6 = 0x458ffd68    r7 = 0x458ffdf8
16:05:26     INFO -       r8 = 0x458ffd60    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffd50    pc = 0x415eadfb
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  20  libxul.so!MessageLoop::DoWork() [message_loop.cc:541750e89771 : 446 + 0x7]
16:05:26     INFO -       r4 = 0x458ffdec    r5 = 0x458ffd58    r6 = 0x458ffd68    r7 = 0x458ffdf8
16:05:26     INFO -       r8 = 0x458ffd60    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffd58    pc = 0x415eb9b9
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  21  libxul.so!base::MessagePumpDefault::Run(base::MessagePump::Delegate*) [message_pump_default.cc:541750e89771 : 24 + 0x7]
16:05:26     INFO -       r4 = 0x449a89c0    r5 = 0x458ffdec    r6 = 0x458ffd9c    r7 = 0x449a89cc
16:05:26     INFO -       r8 = 0x458ffd88    r9 = 0x449a89d0   r10 = 0x00000000    fp = 0x458ffd90
16:05:26     INFO -       sp = 0x458ffd88    pc = 0x415ebcf7
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  22  libxul.so!MessageLoop::RunInternal() [message_loop.cc:541750e89771 : 220 + 0x5]
16:05:26     INFO -       r4 = 0x458ffdec    r5 = 0x458ffdec    r6 = 0x458ffe7e    r7 = 0x00000000
16:05:26     INFO -       r8 = 0x00000000    r9 = 0x4498bb64   r10 = 0x00100000    fp = 0x00000001
16:05:26     INFO -       sp = 0x458ffdc8    pc = 0x415ea04d
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  23  libxul.so!MessageLoop::Run() [message_loop.cc:541750e89771 : 213 + 0x5]
16:05:26     INFO -       r4 = 0x458ffdec    r5 = 0x458ffdec    r6 = 0x458ffe7e    r7 = 0x00000000
16:05:26     INFO -       r8 = 0x00000000    r9 = 0x4498bb64   r10 = 0x00100000    fp = 0x00000001
16:05:26     INFO -       sp = 0x458ffdd0    pc = 0x415ea0c3
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  24  libxul.so!base::Thread::ThreadMain() [thread.cc:541750e89771 : 160 + 0x5]
16:05:26     INFO -       r4 = 0x4498bb50    r5 = 0x458ffdec    r6 = 0x458ffe7e    r7 = 0x00000000
16:05:26     INFO -       r8 = 0x00000000    r9 = 0x4498bb64   r10 = 0x00100000    fp = 0x00000001
16:05:26     INFO -       sp = 0x458ffde8    pc = 0x415ec70b
16:05:26     INFO -      Found by: call frame info
16:05:26     INFO -  25  libxul.so!ThreadFunc [platform_thread_posix.cc:541750e89771 : 39 + 0x5]
16:05:26     INFO -       r4 = 0x458fff00    r5 = 0x415ee3e5    r6 = 0x4498bb50    r7 = 0x00000078
16:05:26     INFO -       r8 = 0x415ee3e5    r9 = 0x4498bb50   r10 = 0x00100000    fp = 0x00000001
16:05:26     INFO -       sp = 0x458ffee8    pc = 0x415ee3ed

Not sure if this belongs under GFX or layers.
Component: Mochitest → Graphics
Product: Testing → Core
Severity: normal → critical
Crash Signature: egl_window_surface_t::~egl_window_surface_t
Keywords: crash
OS: Linux → Gonk (Firefox OS)
Hardware: x86 → ARM
(In reply to TBPL Robot from comment #26)
> Tomcat
> https://tbpl.mozilla.org/php/getParsedLog.php?id=32020245&tree=B2g-Inbound
> b2g_emulator_vm b2g-inbound opt test mochitest-9 on 2013-12-16 02:26:01
> revision: 3d27a4401833
> slave: tst-linux64-spot-061
> 
> Output exceeded 52428800 bytes, remaining output has been truncated

This was a mis-star. I'm guessing this was fixed by bug 948406.
Summary: Intermittent PROCESS-CRASH | Shutdown | application crashed [@ 0x0] | Output exceeded 52428800 bytes, remaining output has been truncated → Intermittent PROCESS-CRASH | Shutdown | application crashed [@ 0x0]
This looks like something to do with the compositor shutting down.
Component: Graphics → Graphics: Layers
Crash Signature: egl_window_surface_t::~egl_window_surface_t → egl_window_surface_t::~egl_window_surface_t GLContextEGL::~GLContextEGL
Summary: Intermittent PROCESS-CRASH | Shutdown | application crashed [@ 0x0] → Intermittent PROCESS-CRASH | Shutdown | application crashed [@ 0x0] [@ egl_window_surface_t::~egl_window_surface_t] [@ GLContextEGL::~GLContextEGL]
Jerry, please help to fix this one
Assignee: nobody → hshih
Silimiar bug: Bug 776742
Jerry,
1. It happens in emulator (ICS)
2. The crash point has comment: // crash here

It seems the nativeWindow has no cancelBuffer()

Reproduce steps:
1. Enable B2G/OOP reftests by modify runreftestb2g.py
2. reduce reftests to the crash
(I keep only reftests/reftest-sanity.list in reftests/reftest.list and top 10 reftests in reftests/reftest-sanity.list)
Attached file runreftestb2g.py (obsolete) — 
runreftestb2g.py python script to run reftests with B2G/OOP enabled.
Attached file runreftestb2g.py (obsolete) —
Attached file runreftestb2g.py —
Attachment #8390354 - Attachment is obsolete: true
Attachment #8390357 - Attachment is obsolete: true
Jerry, it also happen in B2G/OOP-disabled reftests. It means you do not have to use attach runreftestb2g.py.

nativeWindow->cancelBuffer is NULL.
Flags: needinfo?(hshih)
Thanks, Vincent.
I will check the emulator's implementation.
Flags: needinfo?(hshih)
Jerry,
Please check this: https://tbpl.mozilla.org/?tree=Try&rev=bfe897476327

This prevent original crash but crash somewhere else randomly.
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::gl::GLContext::MakeCurrent(bool)]
Return code: 1

Need your help to verify in other platform porting.
Blocks: B2GRT
(In reply to Vincent Chen [:vichen] from comment #60)
> Jerry,
> Please check this: https://tbpl.mozilla.org/?tree=Try&rev=bfe897476327
> 
> This prevent original crash but crash somewhere else randomly.
> PROCESS-CRASH | Shutdown | application crashed [@
> mozilla::gl::GLContext::MakeCurrent(bool)]
> Return code: 1
> 
> Need your help to verify in other platform porting.

Log in Bug 983489
Attached patch FramebufferNativeWindow.patch (obsolete) — — Splinter Review 
add cancelBufferNoop function into FramebufferNativeWindow.
Attached patch FramebufferNativeWindow.patch (obsolete) — — Splinter Review 
format patch
Attachment #8391061 - Attachment is obsolete: true
See Also: → 776742
Attached patch FramebufferNativeWindow.patch (obsolete) — — Splinter Review 
sorry, format again.
Attachment #8391064 - Attachment is obsolete: true
Attached patch FramebufferNativeWindow.patch (obsolete) — — Splinter Review 
fix typo.
Attachment #8391072 - Attachment is obsolete: true
Attached patch cancel_buffer.patch (obsolete) — — Splinter Review 
update comment
Attachment #8391085 - Attachment is obsolete: true
We call cancelBuffer() here:
b2g/development/tools/emulator/opengl/system/egl/egl.cpp:
egl_window_surface_t::~egl_window_surface_t() {
    ....
    if (buffer) {
        nativeWindow->cancelBuffer(nativeWindow, buffer);
    }
    ....
}

And there is no assignment for cancelBuffer() in:
b2g/frameworks/base/libs/ui/FramebufferNativeWindow.cpp
Attachment #8391106 - Flags: review?(mwu)
Comment on attachment 8391106 [details] [diff] [review]
cancel_buffer.patch

Review of attachment 8391106 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks - looks I forgot to port this over when doing the JB port.

::: widget/gonk/libdisplay/GonkDisplayICS.cpp
@@ +110,5 @@
>      }
>  
>      mFBSurface = new FramebufferNativeWindow();
>  
> +    // ICS FrambufferNativeWindow doesn't set the |cancelBuffer|

missing 'e' in Framebuffer

@@ +112,5 @@
>      mFBSurface = new FramebufferNativeWindow();
>  
> +    // ICS FrambufferNativeWindow doesn't set the |cancelBuffer|
> +    // function pointer.
> +    // It will crash when delete the EGL window surface.

s/delete/deleting/
Attachment #8391106 - Flags: review?(mwu) → review+
fix typo from comment 68
Attachment #8391106 - Attachment is obsolete: true
Please land the attachment 8391661 [details] [diff] [review] to mozilla-central.

try result:
https://tbpl.mozilla.org/?tree=Try&rev=bdfda556c15c
Status: NEW → ASSIGNED
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/55a2f006e3aa
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla30
Duplicate of this bug: 980579
Does this need to be landed on any other branches or are we good here?
status-b2g-v1.4: --- → fixed
status-firefox30: --- → fixed
Flags: needinfo?(hshih)
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #74)
> Does this need to be landed on any other branches or are we good here?

I think we can land to another branches to prevent the ics emulator crashed at this when shutdown.
The ics emulator's FramebufferNativeWindow implementation does not change for a long time.
Flags: needinfo?(hshih)
Please nominate it for b2g28 uplift then :)
status-b2g-v1.3: --- → affected
status-firefox28: --- → wontfix
status-firefox29: --- → wontfix
status-firefox-esr24: --- → unaffected
We landed https://hg.mozilla.org/projects/pine/rev/f8bd5adbe110 (which just checks for child process crashes on test shutdown) on pine and this happens all the time on debug emulators. Opt seems fine for some reason.

E.g: https://tbpl.mozilla.org/php/getParsedLog.php?id=36535254&tree=Pine&full=1#error0
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Andrew Halberstadt [:ahal] from comment #77)
> We landed https://hg.mozilla.org/projects/pine/rev/f8bd5adbe110 (which just
> checks for child process crashes on test shutdown) on pine and this happens
> all the time on debug emulators. Opt seems fine for some reason.
> 
> E.g:
> https://tbpl.mozilla.org/php/getParsedLog.
> php?id=36535254&tree=Pine&full=1#error0

This is a fix for a parent process crash. You should open a new bug for the child process crash.
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → FIXED
See Also: → 987251
Jerry,
The function name is changed to cancelBuffer_deprecate() in JB. Not sure for KK.
Flags: needinfo?(hshih)
JB 4.2.2:
We use SurfaceTextureClient. It has cancelBuffer_DEPRECATED.
http://androidxref.com/4.2.2_r1/search?q=cancelBuffer_DEPRECATED&defs=&refs=&path=&hist=&project=abi&project=bionic&project=bootable&project=build&project=cts&project=dalvik&project=development&project=device&project=docs&project=external&project=frameworks&project=gdk&project=hardware&project=libcore&project=libnativehelper&project=ndk&project=packages&project=pdk&project=prebuilts&project=sdk&project=system

JB 4.3:
We use Surface. It also has cancelBuffer_DEPRECATED.
http://androidxref.com/4.3_r2.1/search?q=cancelBuffer_DEPRECATED&defs=&refs=&path=&hist=&project=abi&project=bionic&project=bootable&project=build&project=cts&project=dalvik&project=developers&project=development&project=device&project=docs&project=external&project=frameworks&project=hardware&project=libcore&project=libnativehelper&project=ndk&project=packages&project=pdk&project=prebuilts&project=sdk&project=system&project=tools

KK:
Finally, We use Surface at KK. It also has cancelBuffer_DEPRECATED.
http://androidxref.com/4.4_r1/search?q=cancelBuffer_DEPRECATED&defs=&refs=&path=&hist=&project=abi&project=art&project=bionic&project=bootable&project=build&project=cts&project=dalvik&project=developers&project=development&project=device&project=docs&project=external&project=frameworks&project=hardware&project=libcore&project=libnativehelper&project=ndk&project=packages&project=pdk&project=prebuilts&project=sdk&project=system&project=tools

So, we don't get the same error at other platform.
Flags: needinfo?(hshih)
You need to log in before you can comment on or make changes to this bug.