Closed Bug 1168555 Opened 9 years ago Closed 9 years ago

Intermittent mozrunner-startup | application crashed [@ mozilla::ipc::IToplevelProtocol::AddOpenedActorLocked] (Assertion failure: IsSingleThreaded(), at Sandbox.cpp:442)

Categories

(Core :: Security: Process Sandboxing, defect)

Product:
Core
Component:
Security: Process Sandboxing
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla41
Tracking Flags:
Tracking Status
firefox39 --- unaffected
firefox40 --- unaffected
firefox41 --- fixed
firefox-esr31 --- unaffected
firefox-esr38 --- unaffected
b2g-v2.2 --- unaffected
b2g-master --- fixed

People

(Reporter: RyanVM, Assigned: jld)

References

Details

(Keywords: assertion, crash, intermittent-failure)

Attachments

(1 file)

Patch
1.30 KB, patch
kang
: review+
Details | Diff | Splinter Review 
12:06:32 WARNING - PROCESS-CRASH | mozrunner-startup | application crashed [@ mozilla::ipc::IToplevelProtocol::AddOpenedActorLocked]
12:06:32 INFO - Crash dump filename: /tmp/tmpQryCjJ/4d711400-455f-1f6c-6f8842a0-57269ba2.dmp
12:06:32 INFO - Operating system: Android
12:06:32 INFO - 0.0.0 Linux 2.6.29-g41a03df #22 Thu Jun 26 10:59:09 CST 2014 armv7l Android/full/generic:4.0.4.0.4.0.4/OPENMASTER/eng.cltbld.20150526.134140:eng/test-keys
12:06:32 INFO - CPU: arm
12:06:32 INFO - 0 CPUs
12:06:32 INFO - Crash reason: SIGSEGV
12:06:32 INFO - Crash address: 0x0
12:06:32 INFO - Thread 0 (crashed)
12:06:32 INFO - 0 libxul.so!mozilla::ipc::IToplevelProtocol::AddOpenedActorLocked [LinkedList.h : 282 + 0x1c]
12:06:32 INFO - r4 = 0x47396c0c r5 = 0x44bb76ac r6 = 0x00000000 r7 = 0x44bb76b0
12:06:32 INFO - r8 = 0x47396c1c r9 = 0x42a4c18d r10 = 0x42a4a707 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed160d0 lr = 0x40d7e2a3 pc = 0x40d83c84
12:06:32 INFO - Found by: given as instruction pointer in context
12:06:32 INFO - 1 libxul.so!mozilla::ipc::IToplevelProtocol::AddOpenedActor [ProtocolUtils.cpp:2ac2bced5c8f : 89 + 0x7]
12:06:32 INFO - r4 = 0xbed16100 r5 = 0xbed160fc r6 = 0x44bb76ac r7 = 0x47396c0c
12:06:32 INFO - r8 = 0xbed16258 r9 = 0xbed161c8 r10 = 0x003a008a fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed160f8 pc = 0x40d83ce5
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 2 libxul.so!mozilla::dom::PContentParent::OnMessageReceived [PContentParent.cpp : 5669 + 0x9]
12:06:32 INFO - r4 = 0x47396c00 r5 = 0x4741dcd0 r6 = 0x00000000 r7 = 0xbed161b8
12:06:32 INFO - r8 = 0xbed16258 r9 = 0xbed161c8 r10 = 0x003a008a fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed16120 pc = 0x40eea06f
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 3 libxul.so!mozilla::ipc::MessageChannel::DispatchAsyncMessage [MessageChannel.cpp:2ac2bced5c8f : 1279 + 0x5]
12:06:32 INFO - r4 = 0x47396c38 r5 = 0xbed16394 r6 = 0x00000000 r7 = 0x00000000
12:06:32 INFO - r8 = 0xbed1638c r9 = 0x00000000 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed16310 pc = 0x40d817e1
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 4 libxul.so!mozilla::ipc::MessageChannel::DispatchMessage [MessageChannel.cpp:2ac2bced5c8f : 1198 + 0x7]
12:06:32 INFO - r4 = 0xbed16394 r5 = 0x47396c38 r6 = 0x00000000 r7 = 0xbed16388
12:06:32 INFO - r8 = 0xbed1638c r9 = 0x00000000 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed16330 pc = 0x40d85f09
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 5 libxul.so!mozilla::ipc::MessageChannel::OnMaybeDequeueOne [MessageChannel.cpp:2ac2bced5c8f : 1182 + 0x7]
12:06:32 INFO - r4 = 0xbed16394 r5 = 0x47396c38 r6 = 0x00000001 r7 = 0xbed16388
12:06:32 INFO - r8 = 0xbed1638c r9 = 0x00000000 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed16380 pc = 0x40d8b70f
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 6 libxul.so!RunnableMethod<FdWatcher, void (FdWatcher::*)(), Tuple0>::Run [tuple.h:2ac2bced5c8f : 383 + 0x13]
12:06:32 INFO - r4 = 0x474cfb40 r5 = 0x457bc0c0 r6 = 0x474d7ac0 r7 = 0x00000001
12:06:32 INFO - r8 = 0x457bc0cc r9 = 0x00000001 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed163d0 pc = 0x40b6ee47
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 7 libxul.so!mozilla::ipc::MessageChannel::DequeueTask::Run [MessageChannel.h : 446 + 0x5]
12:06:32 INFO - r4 = 0x474cfb40 r5 = 0x457bc0c0 r6 = 0x474d7ac0 r7 = 0x00000001
12:06:32 INFO - r8 = 0x457bc0cc r9 = 0x00000001 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed163d8 pc = 0x40d81c7f
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 8 libxul.so!MessageLoop::RunTask [message_loop.cc:2ac2bced5c8f : 361 + 0x7]
12:06:32 INFO - r4 = 0x474cfb40 r5 = 0x457bc0c0 r6 = 0x474d7ac0 r7 = 0x00000001
12:06:32 INFO - r8 = 0x457bc0cc r9 = 0x00000001 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed163e0 pc = 0x40d6dc05
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 9 libxul.so!MessageLoop::DeferOrRunPendingTask [message_loop.cc:2ac2bced5c8f : 369 + 0x3]
12:06:32 INFO - r4 = 0x00000001 r5 = 0xbed16420 r6 = 0x474d7ac0 r7 = 0x00000001
12:06:32 INFO - r8 = 0x457bc0cc r9 = 0x00000001 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed16400 pc = 0x40d70edf
12:06:32 INFO - Found by: call frame info
12:06:32 INFO - 10 libxul.so!MessageLoop::DoWork [message_loop.cc:2ac2bced5c8f : 456 + 0x3]
12:06:32 INFO - r4 = 0x457bc0c0 r5 = 0xbed16420 r6 = 0x474d7ac0 r7 = 0x00000001
12:06:32 INFO - r8 = 0x457bc0cc r9 = 0x00000001 r10 = 0x00000000 fp = 0xbed164b7
12:06:32 INFO - sp = 0xbed16410 pc = 0x40d72d91
12:06:32 INFO - Found by: call frame info

12:08:42 INFO - 05-26 19:06:08.981 F/MOZ_Assert( 776): Assertion failure: IsSingleThreaded(), at ../../../../gecko/security/sandbox/linux/Sandbox.cpp:442
See Also: → 1169726
That assertion is my doing, and it was added in bug 1151607; bug 1169726 looks like a duplicate with respect to that part of what's going on here.  The child process is crashing before crash reporting is set up, so the logs will have the Android crash dump info but there won't be a minidump (or breakpad stack in the logs) for it.

But it's worse than that — this looks like yet another bug where a child process exiting at an unexpected time causes the parent process to go off the rails — note the linked list assertions in comment #1 and comment #2.  That part of this bug might be a dup of bug 1018985 and/or bug 1026957, but I haven't checked if they look like the same stack.

Bug 1169726 is another case of the child process failing to be single-threaded where I thought it'd be, but the indirect damage is different; it might just be something in IndexedDB not doing the right thing on IPC failure.  I'll look at it.

As for the assertion: I wonder if there's something earlier that does a thread create/join, and the thread isn't gone from procfs by the time IsSingleThreaded runs.
Component: IPC → Security: Process Sandboxing
(In reply to Jed Davis [:jld] {UTC-7} from comment #3)
> But it's worse than that — this looks like yet another bug where a child
> process exiting at an unexpected time causes the parent process to go off
> the rails — note the linked list assertions in comment #1 and comment #2. 
> That part of this bug might be a dup of bug 1018985 and/or bug 1026957, but
> I haven't checked if they look like the same stack.

They're not.  Those are assertion failures for list integrity on shutdown; this is catching an attempt to insert the same actor into a list more than once (or some memory corruption that changes the “not in list” sentinel).
(In reply to Jed Davis [:jld] {UTC-7} from comment #3)
> As for the assertion: I wonder if there's something earlier that does a
> thread create/join, and the thread isn't gone from procfs by the time
> IsSingleThreaded runs.

There is: the ContentProcess object created in mozilla::ipc::ProcLoaderServiceRun.  That's only for the Nuwa process, which currently doesn't have any sandboxing applied to it rather than to individual children (but there is at least one reason why that could be a useful thing to be able to do).  So there's an easy workaround and also a relatively easy fix.

But that was added in bug 977026, which leaves open the question of why this regressed when it did.  My guess: bug 1050122 and using Nuwa on debug builds.  What's failing here is a release assertion, but the reported bugs are all on debug builds; this is timing-sensitive, so maybe the timing just doesn't work out on opt builds.
Assignee: nobody → jld
Attached patch Patch — — Splinter Review 

        Attachment #8620531 -
        Flags: review?(gdestuynder)

    
    

    
  
Comment on attachment 8620531 [details] [diff] [review]
Patch

Review of attachment 8620531 [details] [diff] [review]:
-----------------------------------------------------------------

:( seems necessary for now

        Attachment #8620531 -
        Flags: review?(gdestuynder) → review+

    
    

    
      
  

    
    

    
  
Not running Try because: the patch is simple and just skips an assertion, and I've tested it locally.
Keywords: checkin-needed

    
    

    
  
…and mark this as blocking bug 1050122 because that's very likely the one that exposed the regression being fixed here.
Blocks: 1050122

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/6e2d23f31eeb
Status: NEW → RESOLVED
Closed: 9 years ago

          status-firefox41:
          affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla41

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: