1213795 - Nuwa process deadlock in running dom/ipc/tests/test_NuwaProcessDeadlock.html

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Emulator, defect)

Description

[Cervantes Yu [:cyu] [:cervantes]]

A GC timer is fired after Nuwa is ready:

#0  __futex_syscall3 () at bionic/libc/arch-arm/bionic/atomics_arm.S:183
#1  0x4008a264 in _normal_lock (mutex=<optimized out>) at bionic/libc/bionic/pthread.c:951
#2  pthread_mutex_lock (mutex=0x4042afa0) at bionic/libc/bionic/pthread.c:1041
#3  0x4002bb40 in __wrap_pthread_mutex_lock (mtx=0x4042afa0) at mozglue/build/Nuwa.cpp:1308
#4  0x40715648 in PR_Lock (lock=0x4042afa0) at nsprpub/pr/src/pthreads/ptsynch.c:177
#5  0x40b4715e in Lock (this=0x4044dbb8) at ../../dist/include/mozilla/Monitor.h:35
#6  MonitorAutoLock (aMonitor=..., this=0xbeb30d04) at ../../dist/include/mozilla/Monitor.h:78
#7  TimerThread::RemoveTimer (this=0x4044dba0, aTimer=aTimer@entry=0x40431280) at xpcom/threads/TimerThread.cpp:605
#8  0x40b47814 in nsTimerImpl::Cancel (this=0x40431280) at xpcom/threads/nsTimerImpl.cpp:364
#9  0x41060f78 in nsJSContext::KillShrinkGCBuffersTimer () at dom/base/nsJSEnvironment.cpp:2151
#10 0x41062bda in ShrinkGCBuffersTimerFired (aTimer=<optimized out>, aClosure=<optimized out>) at dom/base/nsJSEnvironment.cpp:1823
#11 0x40b47fd2 in nsTimerImpl::Fire (this=0x40431280) at xpcom/threads/nsTimerImpl.cpp:530
#12 0x40b47342 in nsTimerEvent::Run (this=0x404ea028) at xpcom/threads/TimerThread.cpp:267
#13 0x40b45142 in nsThread::ProcessNextEvent (this=0x40411460, aMayWait=<optimized out>, aResult=0xbeb30df7) at xpcom/threads/nsThread.cpp:972
#14 0x40b56520 in NS_ProcessNextEvent (aThread=<optimized out>, aMayWait=aMayWait@entry=false) at xpcom/glue/nsThreadUtils.cpp:297
#15 0x40ca964c in mozilla::ipc::MessagePump::Run (this=0x4040c5c8, aDelegate=0xbeb30f00) at ipc/glue/MessagePump.cpp:95
#16 0x40c98aaa in MessageLoop::RunInternal (this=this@entry=0xbeb30f00) at ipc/chromium/src/base/message_loop.cc:234
#17 0x40c98b5c in RunHandler (this=0xbeb30f00) at ipc/chromium/src/base/message_loop.cc:227
#18 MessageLoop::Run (this=0xbeb30f00) at ipc/chromium/src/base/message_loop.cc:201
#19 0x417159f2 in nsBaseAppShell::Run (this=0x4454fca0) at widget/nsBaseAppShell.cpp:156
#20 0x41aa5cf2 in XRE_RunAppShell () at toolkit/xre/nsEmbedFunctions.cpp:785
#21 0x40c98aaa in MessageLoop::RunInternal (this=this@entry=0xbeb30f00) at ipc/chromium/src/base/message_loop.cc:234
#22 0x40c98b5c in RunHandler (this=0xbeb30f00) at ipc/chromium/src/base/message_loop.cc:227
#23 MessageLoop::Run (this=this@entry=0xbeb30f00) at ipc/chromium/src/base/message_loop.cc:201
#24 0x41aa629a in XRE_InitChildProcess (aArgc=<optimized out>, aArgv=<optimized out>, aGMPLoader=<optimized out>) at toolkit/xre/nsEmbedFunctions.cpp:621
#25 0x000087be in content_process_main (argc=8, argv=0xbeb318a4) at ipc/app/../contentproc/plugin-container.cpp:237
#26 0x4008e722 in __libc_init (elfdata=<optimized out>, onexit=<optimized out>, slingshot=0x87dd <main(int, char**)>, structors=0x9e78 <__FINI_ARRAY__>) at bionic/libc/bionic/libc_init_dynamic.c:114
#27 0x000086e8 in _start ()

I think sleep(5) in some IPC channels reveals this timer bug. The timer shouldn't be fired after Nuwa is ready.

Comment 1

[Cervantes Yu [:cyu] [:cervantes]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c26a3752aa65

Comment 2

[Cervantes Yu [:cyu] [:cervantes]]

Attachment: Don't fire timers in the Nuwa process to fix test case test_NuwaProcessDeadlock.html

Try push log: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b061108b2019

Comment 3

[Cervantes Yu [:cyu] [:cervantes]]

Comment on attachment 8672571 [details]
Don't fire timers in the Nuwa process to fix test case test_NuwaProcessDeadlock.html

Cancel the review since there is crash found mochitest.

Comment 4

[Cervantes Yu [:cyu] [:cervantes]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9898171f1d3f

Comment 5

[Cervantes Yu [:cyu] [:cervantes]]

Attachment: Part 1: Part 1: Don't fire timers in the Nuwa process to fix test case test_NuwaProcessDeadlock.html

Comment 6

[Cervantes Yu [:cyu] [:cervantes]]

Attachment: Part2: Fix protocol cloning of CrossProcessCompositorParent

CrossProcessCompositorParent needs to call OnChannelConnected() in clonging. Also remove cloning CompositorParent because it's used inprocess.

Comment 7

[Cervantes Yu [:cyu] [:cervantes]]

Local overnight test passed without any failure:

cervantes@cyu-dt ~ % grep UNEXPECTED- tmux1.log
cervantes@cyu-dt ~ % grep TEST-OK tmux1.log|grep Nuwa|wc -l
452

Emulator mochitests are green on Treeherder https://treeherder.mozilla.org/#/jobs?repo=try&revision=9898171f1d3f

Comment 8

[Cervantes Yu [:cyu] [:cervantes]]

https://hg.mozilla.org/integration/b2g-inbound/rev/a682a92c03095b9ffaf65afec837f8dd2599bcb5
Bug 1213795, Part 1: Don't fire timers in the Nuwa process to fix test case test_NuwaProcessDeadlock.html. r=khuey

https://hg.mozilla.org/integration/b2g-inbound/rev/319bdc16ecf79c15251acb076d1fb9fdb2fa25d6
Bug 1213795, Part 2: Fix protocol cloning of actor CrossProcessCompositorParent. r=khuey

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a682a92c0309
https://hg.mozilla.org/mozilla-central/rev/319bdc16ecf7