Closed Bug 892630 Opened 6 years ago Closed 6 years ago

Intermittent test_dataChannel_basicAudio.html | Test timed out, or application timed out after 330 seconds with no output on ASan

Categories

(Core :: WebRTC: Networking, defect, P1, major)

Product:
Core
Component:
WebRTC: Networking
Platform:
x86
Linux
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla29
Tracking Flags:
Tracking Status
firefox27 --- unaffected
firefox28 --- fixed
firefox29 --- fixed
firefox-esr24 --- unaffected
b2g-v1.3 --- fixed
b2g-v1.3T --- fixed
b2g-v1.4 --- fixed

People

(Reporter: RyanVM, Assigned: jesup)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

Attempted fix for IO thread races and event loop spinning
5.31 KB, patch
smaug
: review+
Sylvestre
: approval-mozilla-beta+
Details | Diff | Splinter Review
null ptr fix for bug 892630
1.16 KB, patch
ekr
: review+
Details | Diff | Splinter Review 
https://tbpl.mozilla.org/php/getParsedLog.php?id=25168575&tree=Birch

Ubuntu VM 12.04 birch debug test mochitest-3 on 2013-07-11 04:41:49 PDT for push 7add0d101b2c
slave: tst-linux32-ec2-148

05:02:26     INFO -  WARNING: no real random source present!
05:02:26     INFO -  19891 INFO TEST-INFO | /tests/dom/media/tests/mochitest/test_dataChannel_basicAudio.html | DataChannelWrapper (pcLocal_channel_0): 'onopen' event fired
05:02:26     INFO -  --DOMWINDOW == 45 (0xf096f28) [serial = 1498] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_appFrame.html?remote_browser=true&remote_app=false]
05:02:26     INFO -  --DOMWINDOW == 44 (0x8bcd1d0) [serial = 1437] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_readonly_transactions.html]
05:02:26     INFO -  --DOMWINDOW == 43 (0xba6d630) [serial = 1435] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_put_get_values_autoIncrement.html]
05:02:26     INFO -  --DOMWINDOW == 42 (0xdc1bb60) [serial = 1485] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_transaction_ordering.html]
05:02:26     INFO -  --DOMWINDOW == 41 (0x9d2dbe0) [serial = 1433] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_put_get_values.html]
05:02:26     INFO -  --DOMWINDOW == 40 (0xca100d8) [serial = 1475] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_traffic_jam.html]
05:02:26     INFO -  --DOMWINDOW == 39 (0x9ee4020) [serial = 1441] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_remove_objectStore.html]
05:02:26     INFO -  --DOMWINDOW == 38 (0x88c9ed8) [serial = 1399] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_keys.html]
05:02:26     INFO -  --DOMWINDOW == 37 (0xe641208) [serial = 1488] [outer = (nil)] [url = http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html]
05:02:26     INFO -  --DOMWINDOW == 36 (0xf0ab3c0) [serial = 1439] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_remove_index.html]
05:02:26     INFO -  --DOMWINDOW == 35 (0xbeddc70) [serial = 1445] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_setVersion.html]
05:02:26     INFO -  --DOMWINDOW == 34 (0xd7492a0) [serial = 1443] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_request_readyState.html]
05:02:26     INFO -  --DOMWINDOW == 33 (0xd22b378) [serial = 1453] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_success_events_after_abort.html]
05:02:26     INFO -  --DOMWINDOW == 32 (0xc8d3c48) [serial = 1449] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_setVersion_events.html]
05:02:26     INFO -  --DOMWINDOW == 31 (0xa0e5520) [serial = 1447] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_setVersion_abort.html]
05:02:26     INFO -  --DOMWINDOW == 30 (0x9bf4898) [serial = 1490] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_appFrame.html?remote_browser=false&remote_app=false]
05:02:26     INFO -  --DOMWINDOW == 29 (0xdaf4808) [serial = 1493] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_browserFrame.html?remote_browser=false&remote_app=false]
05:02:32     INFO -  --DOMWINDOW == 28 (0x9b22568) [serial = 1492] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_appFrame.html?remote_browser=false&remote_app=false]
05:02:32     INFO -  --DOMWINDOW == 27 (0xdd1ee68) [serial = 1500] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_appFrame.html?remote_browser=true&remote_app=false]
05:02:32     INFO -  --DOMWINDOW == 26 (0xb5b8c60) [serial = 1495] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_browserFrame.html?remote_browser=false&remote_app=false]
05:02:32     INFO -  --DOMWINDOW == 25 (0xc07e5c0) [serial = 1487] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_unique_index_update.html]
05:02:32     INFO -  --DOMWINDOW == 24 (0xe9e35c0) [serial = 1489] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_webapp_clearBrowserData_inproc_inproc.html]
05:02:32     INFO -  --DOMWINDOW == 23 (0xb923588) [serial = 1497] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_webapp_clearBrowserData_inproc_oop.html]
05:02:33     INFO -  --DOMWINDOW == 22 (0x108237c8) [serial = 1503] [outer = (nil)] [url = http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html]
05:02:33     INFO -  --DOMWINDOW == 21 (0x10717d40) [serial = 1483] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_transaction_lifetimes_nested.html]
05:02:33     INFO -  --DOMWINDOW == 20 (0xa7fb0a0) [serial = 1481] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_transaction_lifetimes.html]
05:02:33     INFO -  --DOMWINDOW == 19 (0xdaa42d8) [serial = 1479] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_transaction_abort_hang.html]
05:02:33     INFO -  --DOMWINDOW == 18 (0xbe9e5e0) [serial = 1477] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_transaction_abort.html]
05:02:40     INFO -  --DOMWINDOW == 17 (0xc1f26d0) [serial = 1502] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/test_webapp_clearBrowserData_oop_inproc.html]
05:05:17     INFO -  [Parent 2246] WARNING: 1 sort operation has occurred for the SQL statement '0x9f5f71c8'.  See https://developer.mozilla.org/En/Storage/Warnings details.: file ../../../storage/src/mozStoragePrivateHelpers.cpp, line 110
05:07:24     INFO -  19892 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/media/tests/mochitest/test_dataChannel_basicAudio.html | Test timed out.
05:07:24     INFO -  args: ['/builds/slave/test/build/tests/bin/screentopng']
05:07:25     INFO -  Xlib:  extension "RANDR" missing on display ":0".
05:07:38     INFO -  SCREENSHOT: <see log>
05:07:38     INFO -  19893 INFO TEST-END | /tests/dom/media/tests/mochitest/test_dataChannel_basicAudio.html | finished in 322129ms
The test case is failed because we enter MediaPipeline::TransportFailed_s(). I'm guessing there is a race condition which causes TransportLayer entering a error state.
@ekr, do you have any insight on this intermittent error?

[Success Log]
06:19:28     INFO -  -335395072[7f67dc000910]: [GSM Task|def] fsm.c:346: SIPCC-FSM: 1/2, fsm_change_state: DEF: HAVE_REMOTE_OFFER -> STABLE
06:19:28     INFO -  -335395072[7f67dc000910]: [GSM Task|def] dcsm.c:397: SIPCC-DCSM: dcsm_update_gsm_state: 2 : DCSM_READY --> DCSM_READY
06:19:28     INFO -  -335395072[7f67dc000910]: [GSM Task|tnp] ui.c:1674: SIPCC-UI_API: 1/2, ui_set_local_description: state=26 call_instance=0
06:19:28     INFO -  -335395072[7f67dc000910]: [GSM Task|def] sm.c:65: SIPCC-GSM: 1/2, sm_process_event: DEF   :(HAVE_REMOTE_OFFER:SETLOCALDESC )
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|def] ccprovider.c:1580: SIPCC-SIP_CC_PROV: ccappUpdateSessionData: Call_STATE:. state=5, attr=0, cause=5, instance=0
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|CC_SIPCCService] CC_SIPCCService.cpp:724: onCallEvent(CCAPI_CALL_EV_STATE, 10002, [CONNECTED|canHold,canEndCall,canSendDigit,canUpdateVideoMediaCap,canMuteAudio,canSetVolume]
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|def] ccapi_call.c:56: SIPCC-SIP_CC_PROV: CCAPI_Call_releaseCallInfo: ref=0x7f67d800a640: count=2
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|CC_SIPCCCallInfo] CC_SIPCCCallInfo.cpp:566: State 26 not handled in generateCapabilities()
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|CC_SIPCCService] CC_SIPCCService.cpp:724: onCallEvent(CCAPI_CALL_EV_STATE, 10002, [UNKNOWN|]
06:19:28     INFO -  -335210752[7f67d8004c00]: [CCAPP Task|def] ccapi_call.c:56: SIPCC-SIP_CC_PROV: CCAPI_Call_releaseCallInfo: ref=0x7f67d800aa90: count=2
06:19:28     INFO -  -671090944[22764a0]: Flow[5d375705e72f97bc:1,rtp(none)]; Layer[dtls]: Lower lower is now open; starting TLS
06:19:28     INFO -  -671090944[22764a0]: Flow[5d375705e72f97bc:1,rtcp(none)]; Layer[dtls]: Lower lower is now open; starting TLS
06:19:28     INFO -  -671090944[22764a0]: Flow[332e5c24089ee6f3:2,rtp(none)]; Layer[dtls]: Lower lower is now open; starting TLS
06:19:28     INFO -  -63338048[1f02300]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:180: PeerConnectionObserverDispatch processing mCallState = 5 (CONNECTED), mFsmState = 0 (IDLE)
06:19:28     INFO -  -63338048[1f02300]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:213: : **** UNHANDLED SIGNALING STATE : 0 (IDLE)
06:19:28     INFO -  -63338048[1f02300]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:303: : **** UNHANDLED CALL STATE : 5 (CONNECTED)
06:19:28     INFO -  -63338048[1f02300]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:180: PeerConnectionObserverDispatch processing mCallState = 26 (SETLOCALDESCSUCCESS), mFsmState = 16 (STABLE)
06:19:28     INFO -  -671090944[22764a0]: Flow[5d375705e72f97bc:2,rtp(none)]; Layer[dtls]: Lower lower is now open; starting TLS

[Fail Log]
06:28:44     INFO -  -1446245568[8df0880]: [GSM Task|def] fsm.c:346: SIPCC-FSM: 1/2, fsm_change_state: DEF: HAVE_REMOTE_OFFER -> STABLE
06:28:44     INFO -  -1446245568[8df0880]: [GSM Task|def] dcsm.c:397: SIPCC-DCSM: dcsm_update_gsm_state: 2 : DCSM_READY --> DCSM_READY
06:28:44     INFO -  -1446245568[8df0880]: [GSM Task|tnp] ui.c:1674: SIPCC-UI_API: 1/2, ui_set_local_description: state=26 call_instance=0
06:28:44     INFO -  -1446245568[8df0880]: [GSM Task|def] sm.c:65: SIPCC-GSM: 1/2, sm_process_event: DEF   :(HAVE_REMOTE_OFFER:SETLOCALDESC )
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|def] ccprovider.c:1580: SIPCC-SIP_CC_PROV: ccappUpdateSessionData: Call_STATE:. state=5, attr=0, cause=5, instance=0
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|CC_SIPCCService] CC_SIPCCService.cpp:724: onCallEvent(CCAPI_CALL_EV_STATE, 10002, [CONNECTED|canHold,canEndCall,canSendDigit,canUpdateVideoMediaCap,canMuteAudio,canSetVolume]
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|def] ccapi_call.c:56: SIPCC-SIP_CC_PROV: CCAPI_Call_releaseCallInfo: ref=0x961f0f0: count=2
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|cpr] cpr_linux_stdio.c:146: cprGetMessage: msgrcv success for queue CCAPPQ
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|CC_SIPCCCallInfo] CC_SIPCCCallInfo.cpp:566: State 26 not handled in generateCapabilities()
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|CC_SIPCCService] CC_SIPCCService.cpp:724: onCallEvent(CCAPI_CALL_EV_STATE, 10002, [UNKNOWN|]
06:28:44     INFO -  -1446061248[8df0580]: [CCAPP Task|def] ccapi_call.c:56: SIPCC-SIP_CC_PROV: CCAPI_Call_releaseCallInfo: ref=0x961ef28: count=2
06:28:44     INFO -  -1311721280[8616998]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:180: PeerConnectionObserverDispatch processing mCallState = 5 (CONNECTED), mFsmState = 0 (IDLE)
06:28:44     INFO -  -1311721280[8616998]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:213: : **** UNHANDLED SIGNALING STATE : 0 (IDLE)
06:28:44     INFO -  -1311721280[8616998]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:303: : **** UNHANDLED CALL STATE : 5 (CONNECTED)
06:28:44     INFO -  -1311721280[8616998]: [main|PeerConnectionImpl] PeerConnectionImpl.cpp:180: PeerConnectionObserverDispatch processing mCallState = 26 (SETLOCALDESCSUCCESS), mFsmState = 16 (STABLE)
06:28:44     INFO -  --DOMWINDOW == 80 (0xa168aa0) [serial = 287] [outer = (nil)] [url = about:blank]
06:28:44     INFO -  --DOMWINDOW == 79 (0xb16f918) [serial = 341] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/third_party_iframe1.html]
06:28:44     INFO -  --DOMWINDOW == 78 (0xb11a658) [serial = 352] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/third_party_iframe2.html]
06:28:44     INFO -  --DOMWINDOW == 77 (0xad8b408) [serial = 348] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/third_party_iframe2.html]
06:28:44     INFO -  --DOMWINDOW == 76 (0x9a923c0) [serial = 378] [outer = (nil)] [url = http://example.org/tests/dom/indexedDB/test/webapp_clearBrowserData_browserFrame.html?remote_browser=false&remote_app=false]
06:28:44     INFO -  --DOMWINDOW == 75 (0xa594978) [serial = 347] [outer = (nil)] [url = http://example.com/tests/dom/indexedDB/test/third_party_iframe1.html]
06:28:44     INFO -  --DOMWINDOW == 74 (0xa052e70) [serial = 351] [outer = (nil)] [url = http://sub1.test2.example.org:8000/tests/dom/indexedDB/test/third_party_iframe1.html]
06:28:44     INFO -  --DOMWINDOW == 73 (0xb1b4c58) [serial = 342] [outer = (nil)] [url = about:blank]
06:28:44     INFO -  --DOMWINDOW == 72 (0x991e450) [serial = 353] [outer = (nil)] [url = about:blank]
06:28:44     INFO -  --DOMWINDOW == 71 (0xabb5638) [serial = 349] [outer = (nil)] [url = about:blank]
06:28:44     INFO -  -1373635776[88848d8]: Flow[800d8570cc63701a:1,rtp(none)]; Layer[dtls]: Lower lower experienced an error
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  -1373635776[88848d8]: Flow[800d8570cc63701a:1,rtcp(none)]; Layer[dtls]: Lower lower experienced an error
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  -1373635776[88848d8]: Flow[04efb51d353141ff:1,rtp(none)]; Layer[dtls]: Lower lower experienced an error
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  -1373635776[88848d8]: Flow[04efb51d353141ff:1,rtcp(none)]; Layer[dtls]: Lower lower experienced an error
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  [Child 2578] WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file ../../../../../media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 293
06:28:44     INFO -  --DOMWINDOW == 70 (0x973c4c0) [serial = 343] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/indexedDB/test/third_party_iframe1.html]

[1] Success Log: https://tbpl.mozilla.org/php/getParsedLog.php?id=27116700&tree=Try&full=1
[2] Fail Log: https://tbpl.mozilla.org/php/getParsedLog.php?id=27117598&tree=Try&full=1
Flags: needinfo?(ekr)
This looked like ICE failed... Try turning on NSPR_LOG_MODULES=mtransport:XXX where XXX is like 5 or 6 (higher numbers are more).
Flags: needinfo?(ekr)
Possibly related to Bug 901448? The ASAN logs all show a UAF in SnowWhiteKiller::~SnowWhiteKiller() for memory that had previously been freed by that same destructor.
Comment 36 looks like that theory may hold water.
Assignee: nobody → rjesup
Severity: normal → major
Priority: -- → P1
I'll note that this also only happens under e10s testing, which means it might be related to Bug 930481 (although that bug has somewhat different behavior that only emerges under OS X 10.6, so I think it's a different, even if related, bug).
Summary: Intermittent test_dataChannel_basicAudio.html | Test timed out. → Intermittent test_dataChannel_basicAudio.html | Test timed out, or application timed out after 330 seconds with no output on ASan
The recent failures all seem to be in memory freed from xpcom/base/nsCycleCollector.cpp:2309
o.mParticipant->DeleteCycleCollectable(o.mPointer);
and the access is on line 2305 of the same loop

Andrew: could this be an issue with the cycle collector itself?  If not, perhaps pointers to where I should look?  Thanks
Flags: needinfo?(continuation)
Oh, yes, that actually looks like something Olli has been investigating in the last few days.
Blocks: snow-white
Flags: needinfo?(continuation)
Depends on: 965893
Assignee: rjesup → nobody
Component: WebRTC → XPCOM
Taking bug back as it appears it's fallout from bug 952220
No longer blocks: snow-white
Component: XPCOM → WebRTC: Networking
Depends on: 952220
Comment on attachment 8368451 [details] [diff] [review]
Attempted fix for IO thread races and event loop spinning

https://tbpl.mozilla.org/?tree=Try&rev=7726ac03c771
Attachment #8368451 - Flags: feedback?(bugs)
Comment on attachment 8368451 [details] [diff] [review]
Attempted fix for IO thread races and event loop spinning

40 respins with no ASAN failures on the Try.  Odds are looking good this solves it, but it is intermittent (~10/day)

There's one FIX comment that I'll revise to point to a new bug before landing.  Fixing that edgecase may need more work; I want to get the lifetime fix in ASAP.
Attachment #8368451 - Flags: feedback?(bugs) → review?(bugs)
Comment on attachment 8368451 [details] [diff] [review]
Attempted fix for IO thread races and event loop spinning

>   NS_IMETHODIMP Run() {
>-    mConnection->ReadBlob(mStream, mBlob);
>+    mConnection.forget().get()->ReadBlob(mStream, mBlob);
Please add a comment here that we're releasing mConnection in ::ReadBlob.
Or actually, I'd prefer if ReadBlob would take already_AddRefed<DataChannelConnection>& aThis
as param so that it is clear that we need to release it.

So, 

DataChannelConnection* connection = mConnection;
connection->ReadBlob(mConnection.forget(), mStream, mBlob);

>+  already_AddRefed<DataChannelConnection> self(this);
>+  RUN_ON_THREAD(mainThread, WrapRunnable(nsRefPtr<DataChannelConnection>(self),
>                                &DataChannelConnection::SendBinaryMsg,
>                                aStream, temp),
Then you don't need already_AddRefed variable here, but just pass aThis to WrapRunnable

That fixed, r=me

And I assume you'll fix the "FIX!"
Attachment #8368451 - Flags: review?(bugs) → review+
Blocks: 966602
Whiteboard: [leave open]
Comment on attachment 8369112 [details] [diff] [review]
null ptr fix for bug 892630

Review of attachment 8369112 [details] [diff] [review]:
-----------------------------------------------------------------

I'm r+ing this on the basis that this change seems clearly safe. I am a bit concerned
about the fact that you seem to be potentially invoking the thread shutdown on different
threads, but that's a question for the previous patch, not this one.
Attachment #8369112 - Flags: review+
(In reply to TBPL Robot from comment #131)
> philor
> https://tbpl.mozilla.org/php/getParsedLog.php?id=33977268&tree=Mozilla-Aurora
> Ubuntu ASAN VM 12.04 x64 mozilla-aurora opt test mochitest-3 on 2014-02-02
> 12:37:50
> 

This Aurora failure is separate and would not be the same problem (and the signature in ASAN is different).  I think a new bug.
Flags: needinfo?(philringnalda)
Flags: needinfo?(philringnalda)
Seems like this is fixed?
Looks like it. Comment 137 was a mis-star and comment 138 is on 28. Can we resolve this and request beta approval on the patch?
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #139)
> Looks like it. Comment 137 was a mis-star and comment 138 is on 28. Can we
> resolve this and request beta approval on the patch?
Flags: needinfo?(rjesup)
status-firefox27: --- → unaffected
status-firefox28: --- → affected
status-firefox29: --- → fixed
Flags: needinfo?(rjesup)
Comment on attachment 8368451 [details] [diff] [review]
Attempted fix for IO thread races and event loop spinning

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 952220

User impact if declined: ASAN failures

Testing completed (on m-c, etc.): On m-c and Aurora for weeks

Risk to taking this patch (and alternatives if risky): Low risk, especially given the bake time.

String or IDL/UUID changes made by this patch: none
Attachment #8368451 - Flags: approval-mozilla-beta?
Attachment #8368451 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
https://hg.mozilla.org/releases/mozilla-beta/rev/eb3b0ef6c34d
Assignee: nobody → rjesup
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox28: affected → fixed
status-firefox-esr24: --- → unaffected
Resolution: --- → FIXED
Whiteboard: [leave open]
Blocks: 986762
You need to log in before you can comment on or make changes to this bug.