Closed Bug 966802 Opened 10 years ago Closed 10 years ago

Make webrtc not crash with nuwa

Categories

(Core :: IPC, defect)

All
Gonk (Firefox OS)
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla30
blocking-b2g 1.3+
Tracking Status
firefox28 --- wontfix
firefox29 --- wontfix
firefox30 --- fixed
b2g-v1.3 --- fixed
b2g-v1.3T --- fixed
b2g-v1.4 --- fixed

People

(Reporter: fabrice, Assigned: cyu)

References

(Blocks 1 open bug)

Details

(Keywords: crash, reproducible, Whiteboard: [b2g-crash])

Attachments

(1 file)

Original crash in bug 966582

Reproduced at https://bugzilla.mozilla.org/show_bug.cgi?id=950266#c86 :

(In reply to Patrick Wang (ChihKai) (:kk1fff) from comment #86)
> I can reproduce this crash locally, and I got backtrace on content process:
> 
> Program received signal SIGSEGV, Segmentation fault.
> _pr_poll_with_poll (pds=0x402c6020, npds=1, timeout=<value optimized out>)
>     at /home/patrick/w/hgpool/emu1/mcgit/nsprpub/pr/src/pthreads/ptio.c:3814
> 3814	                    in_flags_read = (pds[index].fd->methods->poll)(
> #0  _pr_poll_with_poll (pds=0x402c6020, npds=1, timeout=<value optimized
> out>)
>     at /home/patrick/w/hgpool/emu1/mcgit/nsprpub/pr/src/pthreads/ptio.c:3814
> #1  0x41f4d62e in PR_Poll (pds=0x1f, npds=1076706396, timeout=65535)
>     at /home/patrick/w/hgpool/emu1/mcgit/nsprpub/pr/src/pthreads/ptio.c:4326
> #2  0x405f829e in nsSocketTransportService::Poll (this=<value optimized
> out>, wait=<value optimized out>, 
>     interval=0x402d3cb4) at
> /home/patrick/w/hgpool/emu1/mcgit/netwerk/base/src/nsSocketTransportService2.
> cpp:393
> #3  0x405f85da in nsSocketTransportService::DoPollIteration
> (this=0x42de1a00, wait=true)
>     at
> /home/patrick/w/hgpool/emu1/mcgit/netwerk/base/src/nsSocketTransportService2.
> cpp:820
> #4  0x405f87d6 in nsSocketTransportService::Run (this=0x42de1a00)
>     at
> /home/patrick/w/hgpool/emu1/mcgit/netwerk/base/src/nsSocketTransportService2.
> cpp:684
> #5  0x405d34fe in nsThread::ProcessNextEvent (this=0x40202a20,
> mayWait=<value optimized out>, result=0x402d3d47)
>     at /home/patrick/w/hgpool/emu1/mcgit/xpcom/threads/nsThread.cpp:637
> #6  0x405a3be8 in NS_ProcessNextEvent (thread=0x40202a20, mayWait=false)
>     at /home/patrick/w/hgpool/emu1/mcgit/xpcom/glue/nsThreadUtils.cpp:263
> #7  0x40709f84 in mozilla::ipc::MessagePumpForNonMainThreads::Run
> (this=0x42e20f10, aDelegate=0x402eb1a0)
>     at /home/patrick/w/hgpool/emu1/mcgit/ipc/glue/MessagePump.cpp:303
> #8  0x406ffd3c in MessageLoop::RunInternal (this=0x1000000)
>     at
> /home/patrick/w/hgpool/emu1/mcgit/ipc/chromium/src/base/message_loop.cc:226
> #9  0x406ffdba in MessageLoop::RunHandler (this=0x402eb1a0)
>     at
> /home/patrick/w/hgpool/emu1/mcgit/ipc/chromium/src/base/message_loop.cc:219
> #10 MessageLoop::Run (this=0x402eb1a0) at
> /home/patrick/w/hgpool/emu1/mcgit/ipc/chromium/src/base/message_loop.cc:193
> #11 0x405d3a9c in nsThread::ThreadFunc (arg=<value optimized out>)
>     at /home/patrick/w/hgpool/emu1/mcgit/xpcom/threads/nsThread.cpp:258
> #12 0x41f50348 in _pt_root (arg=<value optimized out>)
>     at
> /home/patrick/w/hgpool/emu1/mcgit/nsprpub/pr/src/pthreads/ptthread.c:205
> #13 0x40017358 in _thread_create_startup (arg=0x402abc00)
>     at /home/patrick/w/hgpool/emu1/mcgit/mozglue/build/Nuwa.cpp:543
> #14 thread_create_startup (arg=0x402abc00) at
> /home/patrick/w/hgpool/emu1/mcgit/mozglue/build/Nuwa.cpp:574
> #15 0x40060e4c in __thread_entry (func=0x402d3f00, arg=0x402abc00,
> tls=<value optimized out>)
>     at bionic/libc/bionic/pthread.c:217
> #16 0x4006099c in pthread_create (thread_out=<value optimized out>,
> attr=0x402abc14, 
>     start_routine=0x40017319 <thread_create_startup>, arg=0x402abc00) at
> bionic/libc/bionic/pthread.c:357
> #17 0x00000000 in ?? ()
> 
> The pds[index].fd has type (PRFileDesc *) and its value is 0x1f, which is
> not accessible.
Blocks a blocker. Maire - Can we get someone to investigate this?
blocking-b2g: --- → 1.3+
Flags: needinfo?(mreavy)
Severity: normal → critical
Keywords: crash, reproducible
Whiteboard: [b2g-crash]
I am looking into this now, but.... is there any evidence that the problem is a defect in the WebRTC code as opposed to something else that is being triggered by WebRTC?
Flags: needinfo?(mreavy)
Thanks for looking at this, Ekr.  I'm assigning the bug to you for now for investigation.  I'm happy to reassign it to someone else (inside or outside the WebRTC team) once you/we know more.
Assignee: nobody → ekr
Reading at the push log here - https://tbpl.mozilla.org/php/getParsedLog.php?id=33908130&tree=B2g-Inbound - looks like we're making it through the test, but we crash on shutdown.
I have an alternate stack which I can get semi-reliably.


(gdb) bt
#0  0x40c6b73e in operator== (aOther=..., this=0xbeaab838) at ../../../layout/style/../base/RestyleTracker.h:178
#1  mozilla::OverflowChangedTracker::Entry::compare (aOne=..., aTwo=...)
    at ../../../layout/style/../base/RestyleTracker.h:180
#2  0x40c6bb9e in mozilla::SplayTree<mozilla::OverflowChangedTracker::Entry, mozilla::OverflowChangedTracker::Entry>::lookup (this=<optimized out>, v=...) at ../../dist/include/mozilla/SplayTree.h:177
#3  0x40cc24e2 in mozilla::SplayTree<mozilla::OverflowChangedTracker::Entry, mozilla::OverflowChangedTracker::Entry>::find (this=0x426c8d78, v=...) at ../../dist/include/mozilla/SplayTree.h:71
#4  0x40cc252e in RemoveFrame (aFrame=0x440a2bf0, this=0x426c8d78) at ../../../layout/base/RestyleTracker.h:72
#5  mozilla::RestyleManager::NotifyDestroyingFrame (this=0x426c8d60, aFrame=0x440a2bf0)
    at ../../../layout/base/RestyleManager.cpp:64
#6  0x40cc757a in nsCSSFrameConstructor::NotifyDestroyingFrame (this=0x438674c0, aFrame=0x440a2bf0)
    at ../../../layout/base/nsCSSFrameConstructor.cpp:1495
#7  0x40cb3a26 in NotifyDestroyingFrame (aFrame=0x440a2bf0, this=0x42eb8c90)
    at ../../../layout/base/nsPresShell.cpp:2058
#8  PresShell::NotifyDestroyingFrame (this=0x42eb8c90, aFrame=0x440a2bf0)
    at ../../../layout/base/nsPresShell.cpp:2053
#9  0x40d0cb24 in nsFrame::DestroyFrom (this=0x440a2bf0, aDestructRoot=<optimized out>)
    at ../../../layout/generic/nsFrame.cpp:647
#10 0x40cf5e22 in nsIFrame::Destroy (this=<optimized out>) at ../../../layout/generic/nsIFrame.h:664
#11 0x40d00630 in RemoveFrame (aOldFrame=0x440a2bf0, aListID=<optimized out>, this=<optimized out>)
    at ../../../layout/generic/nsContainerFrame.cpp:190
#12 nsContainerFrame::RemoveFrame (this=<optimized out>, aListID=<optimized out>, aOldFrame=<optimized out>)
    at ../../../layout/generic/nsContainerFrame.cpp:157
#13 0x40ce2e60 in nsFrameManager::RemoveFrame (this=0x438674c0, aListID=mozilla::layout::kPrincipalList, 
    aOldFrame=0x440a2bf0) at ../../../layout/base/nsFrameManager.cpp:416
#14 0x40ccfa5e in nsCSSFrameConstructor::ContentRemoved (this=0x438674c0, aContainer=0x44077ab0, 
    aChild=0x44077b00, aOldNextSibling=0x0, aFlags=nsCSSFrameConstructor::REMOVE_CONTENT, 
    aDidReconstruct=0xbeaab973) at ../../../layout/base/nsCSSFrameConstructor.cpp:7650
#15 0x40caf15c in PresShell::ContentRemoved (this=0x42eb8c90, aDocument=0x43118000, aContainer=0x44077ab0, 
    aChild=0x44077b00, aIndexInContainer=0, aPreviousSibling=0x0) at ../../../layout/base/nsPresShell.cpp:4334
#16 0x40b03708 in nsNodeUtils::ContentRemoved (aContainer=<optimized out>, aChild=0x44077b00, 
    aIndexInContainer=0, aPreviousSibling=0x0) at ../../../../content/base/src/nsNodeUtils.cpp:183
#17 0x40afba26 in nsINode::doRemoveChildAt (this=0x44077ab0, aIndex=0, aNotify=<optimized out>, aKid=0x44077b00, 
    aChildArray=...) at ../../../../content/base/src/nsINode.cpp:1546
#18 0x40ad3bba in mozilla::dom::FragmentOrElement::RemoveChildAt (this=0x44077ab0, aIndex=0, 
    aNotify=<optimized out>) at ../../../../content/base/src/FragmentOrElement.cpp:1031
#19 0x40ac48d6 in nsContentUtils::SetNodeTextContent (aContent=0x44077ab0, aValue=..., aTryReuse=<optimized out>)
    at ../../../../content/base/src/nsContentUtils.cpp:4148
#20 0x40ad62dc in mozilla::dom::FragmentOrElement::SetInnerHTMLInternal (this=0x44077ab0, aInnerHTML=..., 
    aError=...) at ../../../../content/base/src/FragmentOrElement.cpp:2657
#21 0x4075b00c in mozilla::dom::ElementBinding::set_innerHTML (cx=0x43188c90, obj=<optimized out>, self=
    0x44077ab0, args=<optimized out>) at ElementBinding.cpp:1614
#22 0x40761752 in mozilla::dom::ElementBinding::genericSetter (cx=0x43188c90, argc=<optimized out>, vp=0xbeaabe58)
    at ElementBinding.cpp:2168
#23 0x4127549a in CallJSNative (args=..., 
    native=0x407616a9 <mozilla::dom::ElementBinding::genericSetter(JSContext*, unsigned int, JS::Value*)>, 
    cx=0x43188c90) at ../../../../../js/src/../../js/src/jscntxtinlines.h:220
#24 js::Invoke (cx=0x43188c90, args=..., construct=js::NO_CONSTRUCT)
    at ../../../../../js/src/vm/Interpreter.cpp:466
#25 0x41275eb6 in js::Invoke (cx=0x43188c90, thisv=<optimized out>, fval=..., argc=1, argv=0xbeaac190, rval=...)
    at ../../../../../js/src/vm/Interpreter.cpp:522
#26 0x41275f90 in js::InvokeGetterOrSetter (cx=<optimized out>, obj=0x43032160, fval=..., argc=1, 
    argv=0xbeaac190, rval=...) at ../../../../../js/src/vm/Interpreter.cpp:593
#27 0x41212302 in js::Shape::set (this=<optimized out>, cx=0x43188c90, obj=..., receiver=<optimized out>, 
    strict=false, vp=...) at ../../../../../js/src/../../js/src/vm/Shape-inl.h:121
#28 0x4121780e in js::baseops::SetPropertyHelper<(js::ExecutionMode)0> (cxArg=0x43188c90, obj=..., receiver=..., 
    id=..., defineHow=0, vp=..., strict=false) at ../../../../../js/src/jsobj.cpp:4973
#29 0x4126e37e in SetPropertyOperation (rval=<optimized out>, lval=<optimized out>, pc=<optimized out>, 
    script=..., cx=<optimized out>) at ../../../../../js/src/vm/Interpreter.cpp:336
#30 Interpret (cx=<optimized out>, state=<optimized out>) at ../../../../../js/src/vm/Interpreter.cpp:2447
#31 0x412751f6 in js::RunScript (cx=0x43188c90, state=...) at ../../../../../js/src/vm/Interpreter.cpp:423
#32 0x4127542e in RunScript (state=..., cx=0x43188c90) at ../../../../../js/src/vm/Interpreter.cpp:390
#33 js::Invoke (cx=0x43188c90, args=..., construct=js::NO_CONSTRUCT)
    at ../../../../../js/src/vm/Interpreter.cpp:485
#34 0x41275eb6 in js::Invoke (cx=0x43188c90, thisv=<optimized out>, fval=..., argc=0, argv=0xbeaaca80, rval=...)
    at ../../../../../js/src/vm/Interpreter.cpp:522
#35 0x4122497c in js::DirectProxyHandler::call (this=<optimized out>, cx=<optimized out>, proxy=<optimized out>, 
    args=<optimized out>) at ../../../../../js/src/jsproxy.cpp:465
#36 0x4124fc2a in js::CrossCompartmentWrapper::call (this=0x41a93adc, cx=<optimized out>, wrapper=..., args=...)
    at ../../../../../js/src/jswrapper.cpp:468
#37 0x412284a0 in js::Proxy::call (cx=0x43188c90, proxy=..., args=...) at ../../../../../js/src/jsproxy.cpp:2636
#38 0x41228538 in js::proxy_Call (cx=<optimized out>, argc=<optimized out>, vp=0xbeaaca70)
    at ../../../../../js/src/jsproxy.cpp:3079
#39 0x41275554 in CallJSNative (args=..., 
    native=0x4122851d <js::proxy_Call(JSContext*, unsigned int, JS::Value*)>, cx=0x43188c90)
    at ../../../../../js/src/../../js/src/jscntxtinlines.h:220
---Type <return> to continue, or q <return> to quit---
#40 js::Invoke (cx=0x43188c90, args=..., construct=js::NO_CONSTRUCT)
    at ../../../../../js/src/vm/Interpreter.cpp:459
#41 0x41275eb6 in js::Invoke (cx=0x43188c90, thisv=<optimized out>, fval=..., argc=0, argv=0xbeaacb70, rval=...)
    at ../../../../../js/src/vm/Interpreter.cpp:522
#42 0x411d51fc in JS::Call (cx=0x43188c90, thisv=..., fval=..., argc=0, argv=0xbeaacb70, rval=...)
    at ../../../../../js/src/jsapi.cpp:5004
#43 0x40772cce in mozilla::dom::Function::Call (this=0x436cc3a0, cx=0x43188c90, aThisVal=..., arguments=..., 
    aRv=...) at FunctionBinding.cpp:35
#44 0x409a3aa0 in mozilla::dom::Function::Call<nsCOMPtr<nsISupports> > (this=0x436cc3a0, 
    thisObjPtr=<optimized out>, arguments=..., aRv=..., 
    aExceptionHandling=mozilla::dom::CallbackObject::eReportExceptions)
    at ../../dist/include/mozilla/dom/FunctionBinding.h:58
#45 0x409a3b56 in nsGlobalWindow::RunTimeoutHandler (this=0x42e61e70, aTimeout=<optimized out>, 
    aScx=<optimized out>) at ../../../dom/base/nsGlobalWindow.cpp:11721
#46 0x409a3d9e in nsGlobalWindow::RunTimeout (this=0x42e61e70, aTimeout=0x438e0290)
    at ../../../dom/base/nsGlobalWindow.cpp:11945
#47 0x409a3ea8 in nsGlobalWindow::TimerCallback (aTimer=<optimized out>, aClosure=<optimized out>)
    at ../../../dom/base/nsGlobalWindow.cpp:12191
#48 0x403bba28 in nsTimerImpl::Fire (this=0x426a43c0) at ../../../xpcom/threads/nsTimerImpl.cpp:551
#49 0x403bbae6 in nsTimerEvent::Run (this=<optimized out>) at ../../../xpcom/threads/nsTimerImpl.cpp:635
#50 0x403b9df6 in ProcessNextEvent (result=0xbeaace97, mayWait=<optimized out>, this=0x426024e0)
    at ../../../xpcom/threads/nsThread.cpp:643
#51 nsThread::ProcessNextEvent (this=0x426024e0, mayWait=<optimized out>, result=0xbeaace97)
    at ../../../xpcom/threads/nsThread.cpp:567
#52 0x4038d46a in NS_ProcessNextEvent (thread=<optimized out>, mayWait=<optimized out>)
    at ../../../xpcom/glue/nsThreadUtils.cpp:263
#53 0x404e5c64 in mozilla::ipc::MessagePump::Run (this=0x42601af0, aDelegate=0xbeaacf98)
    at ../../../ipc/glue/MessagePump.cpp:95
#54 0x404dc296 in MessageLoop::RunInternal (this=<optimized out>)
    at ../../../ipc/chromium/src/base/message_loop.cc:226
#55 0x404dc348 in RunHandler (this=0xbeaacf98) at ../../../ipc/chromium/src/base/message_loop.cc:219
#56 MessageLoop::Run (this=0xbeaacf98) at ../../../ipc/chromium/src/base/message_loop.cc:193
#57 0x4091850a in nsBaseAppShell::Run (this=0x431821c0) at ../../../widget/xpwidgets/nsBaseAppShell.cpp:161
#58 0x40edc23e in XRE_RunAppShell () at ../../../toolkit/xre/nsEmbedFunctions.cpp:679
#59 0x404dc296 in MessageLoop::RunInternal (this=<optimized out>)
    at ../../../ipc/chromium/src/base/message_loop.cc:226
#60 0x404dc348 in RunHandler (this=0xbeaacf98) at ../../../ipc/chromium/src/base/message_loop.cc:219
#61 MessageLoop::Run (this=0xbeaacf98) at ../../../ipc/chromium/src/base/message_loop.cc:193
#62 0x40edc5d2 in XRE_InitChildProcess (aArgc=6, aArgv=<optimized out>, aProcess=<optimized out>)
    at ../../../toolkit/xre/nsEmbedFunctions.cpp:516
#63 0x000086ee in main (argc=8, argv=0xbeaad924) at ../../../ipc/app/MozillaRuntimeMain.cpp:137
(gdb) Quit
khuey suggests experimentally turning off the test that seems to trigger this:

https://tbpl.mozilla.org/?tree=Try&rev=cfbcd1a9afb5
(In reply to Jason Smith [:jsmith] from comment #4)
> Reading at the push log here -
> https://tbpl.mozilla.org/php/getParsedLog.php?id=33908130&tree=B2g-Inbound -
> looks like we're making it through the test, but we crash on shutdown.

We don't make it through the test:

15:52:20     INFO -  17790 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":""}]}) throws
15:57:57     INFO -  17791 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":"stun:127.0.0.1"},{"url":"stuns:localhost","foo":""},{"url":"turn:[::1]:3478","username":"p","credential":"p"},{"url":"turns:locB2GRunner TEST-UNEXPECTED-FAIL | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | application timed out after 330.0 seconds with no output

Note the times, and note that the second "TEST-PASS" has a "timed out" error as part of the message - I suspect that test considers that success for some reason.  Then it appears the timeout-test-killer forces a crash.
Actually, in my case it's even more confused

m/src/base/process_util_posix.cc, line 254
###################################### forms.js loaded
############################### browserElementPanning.js loaded
######################## BrowserElementChildPreload.js loaded
###################################### forms.js loaded
############################### browserElementPanning.js loaded
######################## BrowserElementChildPreload.js loaded
0 INFO SimpleTest START
1 INFO TEST-START | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html
2 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection() succeeds
3 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection(1) throws
4 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({}) succeeds
5 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[]}) succeeds
6 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":""}]}) throws
7 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":"stun:127.0.0.1"},{"url":"stuns:localhost","foo":""},{"url":"turn:[::1]:3478","username":"p","credential":"p"},{"url":"turns:localhost:3478?transport=udp","username":"p","credential":"p"}]}) succeeds
8 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":"turns:localhost:3478","username":"p"}]}) throws
9 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":"turns:localhost:3478","credential":"p"}]}) throws
10 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection({"iceServers":[{"url":"http:0.0.0.0"}]}) throws
11 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection() constructor has readable exceptions
12 INFO TEST-INFO | MEMORY STAT vsize after test: 76939264
13 INFO TEST-INFO | MEMORY STAT vsizeMaxContiguous not supported in this build configuration.
14 INFO TEST-INFO | MEMORY STAT residentFast after test: 33501184
15 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 12978352
16 INFO TEST-END | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | finished in 76375ms
*** UTM:SVC TimerManager:notify - notified @mozilla.org/b2g/webapps-update-timer;1
*** UTM:SVC TimerManager:notify - notified timerID: user-agent-updates-timer
*** UTM:SVC TimerManager:registerTimer - id: user-agent-updates-timer
B2GRunner TEST-UNEXPECTED-FAIL | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | application timed out after 330.0 seconds with no output
B2GRunner INFO | checking for crashes in '/data/local/tests/profile/minidumps'
Mochitest INFO | runtestsb2g.py | Running tests: end.
(_virtualenv)ekr@ubuntu-12-04:~/dev/gecko-dev/objdir-gecko-generic/_tests/testing/mochitest$ ^C
(_virtualenv)ekr@ubuntu-12-04:~/dev/gecko-dev/objdir-gecko-generic/_tests/testing/mochitest$
(In reply to Eric Rescorla (:ekr) from comment #6)
> khuey suggests experimentally turning off the test that seems to trigger
> this:
> 
> https://tbpl.mozilla.org/?tree=Try&rev=cfbcd1a9afb5

Hm... test_peerConnection_errorCallbacks.html is also crashing :(
On the Try ekr pushed with that test disabled ( https://tbpl.mozilla.org/?tree=Try&rev=cfbcd1a9afb5 ), there were close to 50% fails on M4, almost all in test_peerConnection_errorCallbacks.html.

On the failures, it finishes the test it appears and starts dumping the memory stats, but in the middle there (mid-line even!) it times out and gets killed.  The odd thing about the successes is that the 3rd memory dump line takes 15-20 seconds or so on the Green M4's, while all the other tests in that log seem to be more or less instantaneous (couldn't find any where the 3 messages weren't in the same second).  so I think something is "odd" even in the successes.
Jesup, let's disable this test and see where we are.
(on try of course)
Ok, I'll push another Try
So this seems to fix the problem, which definitely points
ot these tests as the trigger...

Without knowing anything about this, could the source
of the weirdness be:

11 INFO TEST-PASS | /tests/dom/media/tests/mochitest/test_peerConnection_bug825703.html | mozRTCPeerConnection() constructor has readable exceptions

Maybe that somehow keeps stuff alive past when it should be?
We agreed with ekr to disable the 2 failing webrtc tests for now:
https://hg.mozilla.org/integration/b2g-inbound/rev/322238bbb29b
(In reply to Fabrice Desré [:fabrice] from comment #16)
> We agreed with ekr to disable the 2 failing webrtc tests for now:
> https://hg.mozilla.org/integration/b2g-inbound/rev/322238bbb29b

Wait a second. Why are we turning off tests to resolve this problem? That certainly doesn't sound like the right path to go here - this is legitimate crash, not caused by the tests here. Please back this out.
We can get testing on Nuwa and work on fixing the crashes in parallel.
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #18)
> We can get testing on Nuwa and work on fixing the crashes in parallel.

That's quite risky because if this manifests itself in video/audio calls, then we'll risk impacting the MWC demo for WebRTC on FxOS as a result. We need a finalize a m-c build this week for WebRTC, so introducing a crash risk to a demo certainly isn't a good path to go.
And every day we don't have Nuwa in the tree we're increasing the risk to the 1.3 release by not testing some pretty scary code.  The MWC WebRTC demo can always be built without Nuwa.

There are no good options here.
(In reply to Jason Smith [:jsmith] from comment #19)
> (In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #18)
> > We can get testing on Nuwa and work on fixing the crashes in parallel.
> 
> That's quite risky because if this manifests itself in video/audio calls,
> then we'll risk impacting the MWC demo for WebRTC on FxOS as a result. We
> need a finalize a m-c build this week for WebRTC, so introducing a crash
> risk to a demo certainly isn't a good path to go.

Jason,

As far as I can tell, we can either:

0. Leave the tests orange.
1. Pull these tests.
2. Pull nuwa.

Your preference is for us to do #3? That seems to have the
obvious problem that kyle raises.
Jason, we're going to disable and continue until we find all big issues so we can work them in parallel.  We can turn the tests back on and fix them later...
(In reply to Eric Rescorla (:ekr) from comment #21)
> (In reply to Jason Smith [:jsmith] from comment #19)
> > (In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #18)
> > > We can get testing on Nuwa and work on fixing the crashes in parallel.
> > 
> > That's quite risky because if this manifests itself in video/audio calls,
> > then we'll risk impacting the MWC demo for WebRTC on FxOS as a result. We
> > need a finalize a m-c build this week for WebRTC, so introducing a crash
> > risk to a demo certainly isn't a good path to go.
> 
> Jason,
> 
> As far as I can tell, we can either:
> 
> 0. Leave the tests orange.
> 1. Pull these tests.
> 2. Pull nuwa.
> 
> Your preference is for us to do #3? That seems to have the
> obvious problem that kyle raises.

That's not what I'm arguing for. I'm arguing for protecting the MWC demos so that we don't trash ourselves at MWC. Landing this without knowing if this is going to impact video/audio calls is a terrible idea - it will block testing of the feature until the crash is resolved and put the demo itself at risk for MWC. Marketing has made it clear that they do not want that to happen.

And disabling Nuwa for MWC builds is not something we should assume will be possible, as that requires a custom build for each device in the demo that runs 1.3 or later that will have be flashed on site in Barcelona. We have a limited time to flash, so I wouldn't make the assumption that will happen. It's certainly an option worth discussing, but I wouldn't assume it.

(In reply to Faramarz (:faramarz) from comment #22)
> Jason, we're going to disable and continue until we find all big issues so
> we can work them in parallel.  We can turn the tests back on and fix them
> later...

This does not address the MWC risk very apparent here. I'm following up with Bob offline as this clearly wasn't the right path forward here. Engineering should be communicating with QA before making a risky decision on something like this, especially when we are close to MWC (or better yet, discussing this on drivers). That didn't happen here, which is a clear process failure.
1.3 is way more important than MWC demos. And webrtc is not CS blocker it seems.
(In reply to Fabrice Desré [:fabrice] from comment #24)
> 1.3 is way more important than MWC demos. And webrtc is not CS blocker it
> seems.

That's not an engineering call - that's a product call on the priority on what we should be doing here. If we're posing a risk to an existing demo, then we should get a picture of how well the demo fares with nuwa turned on without the fix included here. That at least establishes awareness of if there are problems here, how many there are, and the severity of those problems. Then we followup with product  raising the risk here with approaches on the table with tradeoffs laid out & mitigation plans with resolution at a high priority. We can then take the path forward that best balances the vision the product team is arguing for here, as they hold the final decision here.
Can we get ETA to fix this bug? Thank you.
Unknown. I will be starting a bunch of trys to track it down, but I doubt before the weekend.
Whiteboard: [b2g-crash] → [b2g-crash], ETA:2/12
fixed the whiteboard formatting
Whiteboard: [b2g-crash], ETA:2/12 → [b2g-crash] [ETA:2/12]
I reproduced this crash quite reliably. nsSocketTransportService::mPollList[0] is just corrupted, while its backup, nsSocketTransportService::mThreadEvent is intact. I am trying to investigate how the corruption came to be (sadly, watchpoint that is fast enough).
I tried to use "poor man's watchpoint" by enlarging PRPollDesc and mprotect() it to see how it is overwritten. Then I got another crash stack reliably (sadly):

#1  0x405a7a5e in PL_DHashTableOperate (table=0x402bffe0, key=0xbeca8920, op=PL_DHASH_LOOKUP) at /home/cervantes/hg/mozilla-central/xpcom/glue/pldhash.cpp:495
#2  0x40bad0d4 in nsTHashtable<nsBaseHashtableET<nsUint64HashKey, nsGlobalWindow*> >::GetEntry (this=0x43d58120, message=<value optimized out>, sourceName=..., sourceLine=..., lineNumber=952, columnNumber=0, flags=2, category=..., aInnerWindowID=4) at ../../../dist/include/nsTHashtable.h:141
#3  nsBaseHashtable<nsUint64HashKey, nsGlobalWindow*, nsGlobalWindow*>::Get (this=0x43d58120, message=<value optimized out>, sourceName=..., sourceLine=..., lineNumber=952, columnNumber=0, flags=2, category=..., aInnerWindowID=4) at ../../../dist/include/nsBaseHashtable.h:111
#4  nsGlobalWindow::GetInnerWindowWithId (this=0x43d58120, message=<value optimized out>, sourceName=..., sourceLine=..., lineNumber=952, columnNumber=0, flags=2, category=..., aInnerWindowID=4) at /home/cervantes/hg/mozilla-central/dom/base/nsGlobalWindow.h:664
#5  nsScriptError::InitWithWindowID (this=0x43d58120, message=<value optimized out>, sourceName=..., sourceLine=..., lineNumber=952, columnNumber=0, flags=2, category=..., aInnerWindowID=4) at /home/cervantes/hg/mozilla-central/js/xpconnect/src/nsScriptError.cpp:135
#6  0x40c03228 in mozilla::dom::AsyncErrorReporter::ReportError (this=<value optimized out>) at /home/cervantes/hg/mozilla-central/dom/base/nsJSEnvironment.cpp:422
#7  0x40c033b8 in ScriptErrorEvent::Run (this=0x431ddc40) at /home/cervantes/hg/mozilla-central/dom/base/nsJSEnvironment.cpp:513
#8  0x40d3cb3c in nsContentUtils::AddScriptRunner (aRunnable=<value optimized out>) at /home/cervantes/hg/mozilla-central/content/base/src/nsContentUtils.cpp:4742
#9  0x40c036a8 in NS_ScriptErrorReporter (cx=<value optimized out>, message=0x43155650 "Error: Permission denied to create wrapper for object of class UnnamedClass", report=0x431d3400) at /home/cervantes/hg/mozilla-central/dom/base/nsJSEnvironment.cpp:597
#10 0x414b7a52 in CallErrorReporter (cx=0x431bcbe0, message=0x43155650 "Error: Permission denied to create wrapper for object of class UnnamedClass", reportp=0x431d3400) at /home/cervantes/hg/mozilla-central/js/src/jscntxt.cpp:898
#11 0x414cc4de in js_ReportUncaughtException (cx=0x431bcbe0) at /home/cervantes/hg/mozilla-central/js/src/jsexn.cpp:871
#12 0x414b2ed6 in ~AutoLastFrameCheck (cx=0x431bcbe0, thisv=..., fval=..., argc=0, argv=0xbeca8c00, rval=...) at /home/cervantes/hg/mozilla-central/js/src/jsapi.cpp:4198
#13 JS::Call (cx=0x431bcbe0, thisv=..., fval=..., argc=0, argv=0xbeca8c00, rval=...) at /home/cervantes/hg/mozilla-central/js/src/jsapi.cpp:5004
#14 0x409bbdda in mozilla::dom::Function::Call (this=<value optimized out>, cx=0x431bcbe0, aThisVal=..., arguments=..., aRv=...) at /home/cervantes/git/emulator/B2G/objdir-gecko/dom/bindings/FunctionBinding.cpp:34
#15 0x40bf3034 in Call<nsCOMPtr<nsISupports> > (this=0x42e8f390, aTimeout=0x43d74ab0, aScx=<value optimized out>) at ../../dist/include/mozilla/dom/FunctionBinding.h:58
#16 nsGlobalWindow::RunTimeoutHandler (this=0x42e8f390, aTimeout=0x43d74ab0, aScx=<value optimized out>) at /home/cervantes/hg/mozilla-central/dom/base/nsGlobalWindow.cpp:11721
#17 0x40bfc98c in nsGlobalWindow::RunTimeout (this=0x42e8f390, aTimeout=0x43d74ab0) at /home/cervantes/hg/mozilla-central/dom/base/nsGlobalWindow.cpp:11945
#18 0x40bfca9c in nsGlobalWindow::TimerCallback (aTimer=<value optimized out>, aClosure=<value optimized out>) at /home/cervantes/hg/mozilla-central/dom/base/nsGlobalWindow.cpp:12191
#19 0x405d81cc in nsTimerImpl::Fire (this=0x43d96980) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsTimerImpl.cpp:551
#20 0x405d827e in nsTimerEvent::Run (this=<value optimized out>) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsTimerImpl.cpp:635
#21 0x405d67dc in nsThread::ProcessNextEvent (this=0x402024e0, mayWait=false, result=0xbeca8eff) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:643
#22 0x405a6ea8 in NS_ProcessNextEvent (thread=0x431ddc40, mayWait=false) at /home/cervantes/hg/mozilla-central/xpcom/glue/nsThreadUtils.cpp:263
#23 0x4070d210 in mozilla::ipc::MessagePump::Run (this=0x40201ac0, aDelegate=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/glue/MessagePump.cpp:95
#24 0x4070d2de in mozilla::ipc::MessagePumpForChildProcess::Run (this=0x40201ac0, aDelegate=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/glue/MessagePump.cpp:283
#25 0x40702e7c in MessageLoop::RunInternal (this=0x1000000) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:226
#26 0x40702efa in MessageLoop::RunHandler (this=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:219
#27 MessageLoop::Run (this=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:193
#28 0x40b71fb8 in nsBaseAppShell::Run (this=0x42ea4f40) at /home/cervantes/hg/mozilla-central/widget/xpwidgets/nsBaseAppShell.cpp:161
#29 0x411932d2 in XRE_RunAppShell () at /home/cervantes/hg/mozilla-central/toolkit/xre/nsEmbedFunctions.cpp:679
#30 0x4070d2ac in mozilla::ipc::MessagePumpForChildProcess::Run (this=0x40201ac0, aDelegate=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/glue/MessagePump.cpp:253
#31 0x40702e7c in MessageLoop::RunInternal (this=0x42ea4f40) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:226
#32 0x40702efa in MessageLoop::RunHandler (this=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:219
#33 MessageLoop::Run (this=0xbeca980c) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:193
#34 0x4119375c in XRE_InitChildProcess (aArgc=-1094018648, aArgv=0xbeca991c, aProcess=1076096000) at /home/cervantes/hg/mozilla-central/toolkit/xre/nsEmbedFunctions.cpp:516
#35 0x00008750 in main (argc=8, argv=0xbeca99a4) at /home/cervantes/hg/mozilla-central/ipc/app/MozillaRuntimeMain.cpp:137
warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

(gdb) fr 0
warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

#0  ?? (warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

) at bionic/linker/linker.c:2072
2072    }
warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

(gdb) info reg
r0             0x402bffe0       1076625376
r1             0xbeca8920       -1094022880
r2             0x0      0
r3             0x0      0
r4             0x402bffe0       1076625376
r5             0x9e3779b9       -1640531527
r6             0xbeca8920       -1094022880
r7             0x0      0
r8             0x431ddc64       1126030436
r9             0x43155650       1125471824
r10            0x1      1
r11            0x42dee000       1121902592
r12            0xbeca8918       -1094022888
sp             0xbeca8908       0xbeca8908
lr             0x405a7a5f       1079671391
pc             0x0      0
cpsr           0x60000010       1610612752
warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)
Log mPollList[0].fd (the crash point is quite deterministic on my local build) in the test to try to narrow down the period that the value is changed in. The value hasn't been changed before end of SimpleTest.finish() function. It seems that the memory corruption happens after test finish.
It appears that the Nuwa-managed threads has a stack size that is too small, and WebRTC can have a deep stack and overruns socket transport service's stack like the following. I got this stack by mprotect() the stack top.

#0  vsnprintf (str=0x402d4848 "\001", n=4096, fmt=0x4182d710 "ICE(%s): no local addresses available", ap=...) at bionic/libc/stdio/vsnprintf.c:47
#1  0x40886d62 in ringbuffer_vlog (facility=<value optimized out>, level=<value optimized out>, format=0x4182d710 "ICE(%s): no local addresses available", ap=...) at /home/cervantes/hg/mozilla-central/media/mtransport/rlogringbuffer.cpp:36
#2  0x410e1b36 in r_vlog (facility=3, level=3, format=<value optimized out>, ap=...) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nrappkit/src/log/r_log.c:370
#3  0x410e1b80 in r_log (facility=<value optimized out>, level=<value optimized out>, format=0x1 <Address 0x1 out of bounds>) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nrappkit/src/log/r_log.c:303
#4  0x410d985c in nr_ice_component_initialize (ctx=0x43d1bd7c, component=0x43d5d40c) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nICEr/src/ice/ice_component.c:412
#5  0x410daa24 in nr_ice_media_stream_initialize (ctx=0x43d1bd7c, stream=<value optimized out>) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:133
#6  0x410da3da in nr_ice_initialize (ctx=0x43d1bd7c, done_cb=<value optimized out>, cb_arg=<value optimized out>) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nICEr/src/ice/ice_ctx.c:560
#7  0x408844c6 in mozilla::NrIceCtx::StartGathering (this=0x43d418d0) at /home/cervantes/hg/mozilla-central/media/mtransport/nricectx.cpp:585
#8  0x406f1c4a in mozilla::runnable_args_m_0<nsRefPtr<mozilla::DataChannelConnection>, void (mozilla::DataChannelConnection::*)()>::Run (this=<value optimized out>) at ../../../dist/include/mtransport/runnable_utils_generated.h:49
#9  0x405d67dc in nsThread::ProcessNextEvent (this=0x40202a90, mayWait=true, result=0x402d8c87) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:643
#10 0x405a6ea8 in NS_ProcessNextEvent (thread=0x3, mayWait=true) at /home/cervantes/hg/mozilla-central/xpcom/glue/nsThreadUtils.cpp:263
#11 0x405d6a24 in nsThread::DispatchInternal (this=0x402024e0, event=0x43d15ee0, flags=<value optimized out>, target=0x0) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:411
#12 0x405d6a7c in nsThread::Dispatch (this=0x40202a90, event=0x1000001, flags=1099093776) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:427
#13 0x405a6f80 in NS_DispatchToMainThread (event=0x43d15ee0, dispatchFlags=1) at /home/cervantes/hg/mozilla-central/xpcom/glue/nsThreadUtils.cpp:182
#14 0x408802ac in nr_stun_get_addrs (aAddrs=<value optimized out>, aMaxAddrs=100, aDropLoopback=<value optimized out>, aCount=0x402dbafc) at /home/cervantes/hg/mozilla-central/media/mtransport/gonk_addrs.cpp:107
#15 0x410e0390 in nr_stun_find_local_addresses (addrs=0x402d8dac, maxaddrs=100, count=0x402dbafc) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nICEr/src/stun/stun_util.c:113
#16 0x410da358 in nr_ice_initialize (ctx=0x43d1ba1c, done_cb=<value optimized out>, cb_arg=0x43d415c0) at /home/cervantes/hg/mozilla-central/media/mtransport/third_party/nICEr/src/ice/ice_ctx.c:534
#17 0x408844c6 in mozilla::NrIceCtx::StartGathering (this=0x43d415c0) at /home/cervantes/hg/mozilla-central/media/mtransport/nricectx.cpp:585
#18 0x406f1c4a in mozilla::runnable_args_m_0<nsRefPtr<mozilla::DataChannelConnection>, void (mozilla::DataChannelConnection::*)()>::Run (this=<value optimized out>) at ../../../dist/include/mtransport/runnable_utils_generated.h:49
#19 0x405d67dc in nsThread::ProcessNextEvent (this=0x40202a90, mayWait=true, result=0x402dbcd7) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:643
#20 0x405a6ea8 in NS_ProcessNextEvent (thread=0x402d8dac, mayWait=true) at /home/cervantes/hg/mozilla-central/xpcom/glue/nsThreadUtils.cpp:263
#21 0x405fbafc in nsSocketTransportService::Run (this=0x42de2c00) at /home/cervantes/hg/mozilla-central/netwerk/base/src/nsSocketTransportService2.cpp:696
#22 0x405d67dc in nsThread::ProcessNextEvent (this=0x40202a90, mayWait=false, result=0x402dbd47) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:643
#23 0x405a6ea8 in NS_ProcessNextEvent (thread=0x40202a90, mayWait=false) at /home/cervantes/hg/mozilla-central/xpcom/glue/nsThreadUtils.cpp:263
#24 0x4070d52c in mozilla::ipc::MessagePumpForNonMainThreads::Run (this=0x402cc430, aDelegate=0x402ef1a0) at /home/cervantes/hg/mozilla-central/ipc/glue/MessagePump.cpp:303
#25 0x407032e4 in MessageLoop::RunInternal (this=0x1000000) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:226
#26 0x40703362 in MessageLoop::RunHandler (this=0x402ef1a0) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:219
#27 MessageLoop::Run (this=0x402ef1a0) at /home/cervantes/hg/mozilla-central/ipc/chromium/src/base/message_loop.cc:193
#28 0x405d6d78 in nsThread::ThreadFunc (arg=<value optimized out>) at /home/cervantes/hg/mozilla-central/xpcom/threads/nsThread.cpp:258
#29 0x41f5a1e0 in _pt_root (arg=<value optimized out>) at /home/cervantes/hg/mozilla-central/nsprpub/pr/src/pthreads/ptthread.c:205
#30 0x40017414 in _thread_create_startup (arg=0x402b1c00) at /home/cervantes/hg/mozilla-central/mozglue/build/Nuwa.cpp:557
#31 thread_create_startup (arg=0x402b1c00) at /home/cervantes/hg/mozilla-central/mozglue/build/Nuwa.cpp:588
#32 0x40060e4c in __thread_entry (func=0x402dbf00, arg=0x402b1c00, tls=<value optimized out>) at bionic/libc/bionic/pthread.c:217
#33 0x4006099c in pthread_create (thread_out=<value optimized out>, attr=0x402b1c14, start_routine=0x400173d5 <thread_create_startup>, arg=0x402b1c00) at bionic/libc/bionic/pthread.c:357
#34 0x00000000 in ?? ()
Assignee: ekr → cyu
Component: WebRTC → IPC
(In reply to Cervantes Yu from comment #33)
> Created attachment 8372223 [details] [diff] [review]
> Fix and protect from stack overflow of the threads cloned by the Nuwa process

Note - this needs to include turning the two preffed off tests back on. We also need to kick off a try run with this patch with the two disabled tests preffed on again.
Test the patch including one offending test case.

https://tbpl.mozilla.org/?tree=Try&rev=1bfeddc7536e
(In reply to Cervantes Yu from comment #35)
> Test the patch including one offending test case.
> 
> https://tbpl.mozilla.org/?tree=Try&rev=1bfeddc7536e

Can we execute this try run with both test cases here?
BTW, these tests seem to have helpfully moved to mochitest-7 even though they were mochitest-4 before.

I also hit a bunch of retriggers.
Blocks: 965140
Comment on attachment 8372223 [details] [diff] [review]
Fix and protect from stack overflow of the threads cloned by the Nuwa process

Review of attachment 8372223 [details] [diff] [review]:
-----------------------------------------------------------------

very nice
Attachment #8372223 - Flags: review?(khuey) → review+
Keywords: checkin-needed
https://hg.mozilla.org/integration/b2g-inbound/rev/727a68ceca5c
Keywords: checkin-needed
Whiteboard: [b2g-crash] [ETA:2/12] → [b2g-crash]
https://hg.mozilla.org/mozilla-central/rev/727a68ceca5c
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla30
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/4180028c15ee

I also re-enabled the two disabled WebRTC tests after a green Try run.
I would not be surprised if the tests fail, since I was seeing then fail.
(In reply to Eric Rescorla (:ekr) from comment #43)
> I would not be surprised if the tests fail, since I was seeing then fail.

(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #42)
> after a green Try run.

FWIW. And on b2g28, they were only ever disabled on B2G, so they've been running green the other platforms there the whole time.
Well, let's see what happens.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: