Closed
Bug 836379
Opened 11 years ago
Closed 11 years ago
Segfault in srtp.c:1291
Categories
(Core :: WebRTC, defect)
Core
WebRTC
Tracking
()
RESOLVED
INCOMPLETE
People
(Reporter: ianbicking, Assigned: ekr)
References
Details
(Keywords: crash, sec-high, Whiteboard: [webrtc][blocking-webrtc+])
Attachments
(1 file)
281.23 KB,
text/plain
|
Details |
I'm consistently getting a segfault with WebRTC (latest nightly). Here's the backtrace I get from gdb: #0 srtp_get_stream [inlined] () at /Users/ianbicking/src/mozilla-central/netwerk/srtp/src/srtp/srtp.c:1291 #1 0x0000000101272eef in srtp_protect (ctx=0x146adccc0, rtp_hdr=0x144cfd900, pkt_octet_len=0x106780ac4) at /Users/ianbicking/src/mozilla-central/netwerk/srtp/src/srtp/srtp.c:735 #2 0x0cc87cf8a7cde231 in ?? () I believe the point in my code where it is triggered is somewhere around here: https://github.com/mozilla/towtruck/blob/e0aca03c654cd58fbf343748501c7f27bc237b34/app/http/public/towtruck/webrtc.js#L211 I can't be sure because when the browser crashes I'm not sure exactly what the progress is – local video has started, but remote video has not. Setting up the code I'm using is perhaps non-trivial, but it does consistently lead to this crash, always on the computer that answers the RTC connection. Reproduced on Mac and Windows.
Reporter | ||
Comment 1•11 years ago
|
||
If I comment out this line I no longer get a segfault: https://github.com/mozilla/towtruck/blob/e0aca03c654cd58fbf343748501c7f27bc237b34/app/http/public/towtruck/webrtc.js#L231 "conn.addStream(stream)" But since that means the two computers don't have any streams connected to the connection, it doesn't narrow things down too far. Suppressing this callback doesn't help (which surprised me a little): https://github.com/mozilla/towtruck/blob/e0aca03c654cd58fbf343748501c7f27bc237b34/app/http/public/towtruck/webrtc.js#L211
Comment 2•11 years ago
|
||
Callstack please (and local vars if possible - where --full or some such)
Updated•11 years ago
|
Assignee: nobody → rjesup
Whiteboard: [webrtc][blocking-webrtc+]
Reporter | ||
Comment 3•11 years ago
|
||
Callstack: (gdb) bt full #0 srtp_get_stream [inlined] () at /Users/ianbicking/src/mozilla-central/netwerk/srtp/src/srtp/srtp.c:1291 stream = <value temporarily unavailable, due to optimizations> enc_start = <value temporarily unavailable, due to optimizations> prefix_len = <value temporarily unavailable, due to optimizations> est = 0 est = 0 est = 0 est = 0 est = 0 est = 0 status = <value temporarily unavailable, due to optimizations> enc_octet_len = 0 tag_len = 1 auth_tag = <value temporarily unavailable, due to optimizations> auth_start = <value temporarily unavailable, due to optimizations> delta = <value temporarily unavailable, due to optimizations> #1 0x0000000101272eef in srtp_protect (ctx=0x141e86240, rtp_hdr=0x106b12c00, pkt_octet_len=0x106780ac4) at /Users/ianbicking/src/mozilla-central/netwerk/srtp/src/srtp/srtp.c:735 stream = <value temporarily unavailable, due to optimizations> enc_start = <value temporarily unavailable, due to optimizations> prefix_len = <value temporarily unavailable, due to optimizations> est = 0 est = 0 est = 0 est = 0 est = 0 est = 0 status = <value temporarily unavailable, due to optimizations> enc_octet_len = 0 tag_len = 1 auth_tag = <value temporarily unavailable, due to optimizations> auth_start = <value temporarily unavailable, due to optimizations> delta = <value temporarily unavailable, due to optimizations> #2 0x49067ef8bcb0e6f3 in ?? () No symbol table info available.
Comment 4•11 years ago
|
||
Dupe of bug 821887? It's the same stack as I have reported but not on shutdown. The other one is ss so we should mark it too.
Comment 5•11 years ago
|
||
Line numbers are very different, and this has minimal backtrace due to opt build. srtp_protect() is a pretty big routine. We really need a debug build stack to attack it and see *why* we're here. Adding mediapipeline:5 logging (and probably signaling:5 as well) would help too
Reporter | ||
Comment 6•11 years ago
|
||
I've created a debug build, but though it freezes it isn't segfaulting in the same way, so no backtrace. These warning messages do appear: Assertion failure: !other->mOtherDirection, at /Users/ianbicking/src/mozilla-central/media/webrtc/signaling/src/media-conduit/AudioConduit.cpp:123 WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file /Users/ianbicking/src/mozilla-central/media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 279 WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file /Users/ianbicking/src/mozilla-central/media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 279 WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file /Users/ianbicking/src/mozilla-central/media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 279 WARNING: MediaPipeline Transport failed. This is not properly cleaned up yet: file /Users/ianbicking/src/mozilla-central/media/webrtc/signaling/src/mediapipeline/MediaPipeline.cpp, line 279
Reporter | ||
Comment 7•11 years ago
|
||
Reporter | ||
Comment 8•11 years ago
|
||
That is pretty much all the debugging information I can get out of it now.
Assignee | ||
Comment 9•11 years ago
|
||
We really need data further up the stack. What are the contents of the SrtpFlow class?
Reporter | ||
Comment 10•11 years ago
|
||
I'll need further instructions on how to collect that information. After rebuilding with optimizations off I don't seem to be getting a segfault, so there's no opportunity to use backtrace; but maybe there's another way to get that info? I don't do any debugging at this level usually. Also, next week we should have this app on a public URL and you should be able to reproduce it fairly easily from that.
Comment 11•11 years ago
|
||
(In reply to Ian Bicking (:ianb) from comment #6) > I've created a debug build, but though it freezes it isn't segfaulting in > the same way, so no backtrace. These warning messages do appear: > > Assertion failure: !other->mOtherDirection, at > /Users/ianbicking/src/mozilla-central/media/webrtc/signaling/src/media- > conduit/AudioConduit.cpp:123 Ok, *thats* interesting. Why didn't that exit? That's a MOZ_ASSERT() - and they either go to the debugger or exit, they don't just continue on! And the line number is wrong (now); it's at line 129; your rev was probably from around 120236:19e164f7d88d. That's in the timeframe where there was a known missing clobber on inbound and maybe m-c; if you only made incremental builds you may have hit the same problem that caused a gazillion incorrect error reports from inbound.
Reporter | ||
Comment 12•11 years ago
|
||
It should all be a fresh checkout, since I hosed my last checkout while trying to rebuild with different flags. At this point it might be better if I can get things setup so other people can reproduce the error.
Reporter | ||
Comment 13•11 years ago
|
||
Instructions to reproduce, using two computers: Go here: http://towtruck.mozillalabs.com:8082/textarea/ Hit the "Start TowTruck" button. A window titled "Chat" should appear. Click on the "i" on the left side of the Chat window, there will be a link – send that to another computer. When opened on the other computer, the Chat window will automatically appear. On either computer hit the "v" button on the left side of the Chat window, this will start the WebRTC process. Accept video/audio on the first computer. Then you should get a permission prompt for video and audio on the second computer (no action required there). You should see video of yourself on each computer. After a couple seconds as it tries to set up the connection you should get a crash on the second computer (not the one that initiated the conversation). It's happening consistently for me following these instructions, both Windows and Mac.
Assignee | ||
Comment 14•11 years ago
|
||
Do you by any chance have a single-computer repro? I don't have two machines with me here in Boston.
Comment 15•11 years ago
|
||
:ekr: good time to learn how to create firefox profiles ;-)
Assignee | ||
Comment 16•11 years ago
|
||
I already know how to do that, but this says "two computers"
Comment 17•11 years ago
|
||
Ah -- I expect that it should work w/ two profiles. Ian, can you confirm?
Comment 18•11 years ago
|
||
(In reply to Ian Bicking (:ianb) from comment #13) > Instructions to reproduce, using two computers: > > Go here: http://towtruck.mozillalabs.com:8082/textarea/ I tried yesterday and today but I cannot open this page. I always get the following failure: The connection to towtruck.mozillalabs.com:8082 was interrupted while the page was loading.
Reporter | ||
Comment 19•11 years ago
|
||
Yes, with two profiles on the same computer it also crashes (I've had problems getting the camera twice on the same computer, but apparently that works now). I'm not sure what's up with accessing the server, it's working fine for me at the moment, but we just put that server up and there may be problems. I am not on the VPN when I connect. If you are having a problem you could ping wex and me (ianbicking) on #labs.
Assignee | ||
Updated•11 years ago
|
Assignee: rjesup → ekr
Assignee | ||
Comment 20•11 years ago
|
||
Ian, I can't make this even give me the request for audio and video. Would it be possible for you to produce a reduced test case that demonstrates this on a single browser tab?
Assignee | ||
Comment 21•11 years ago
|
||
Minimally, can you provide a stack trace?
Reporter | ||
Comment 22•11 years ago
|
||
The stack trace I put into Comment 3 is the best I was able to do, despite a fair amount of effort to get something better. Let's try to figure out how to reproduce on your computer together on Monday.
Assignee | ||
Comment 23•11 years ago
|
||
I just tried this in two tabs and got the assert listed in c11
Assignee | ||
Comment 24•11 years ago
|
||
I have reproved the assert in bug 840344
Assignee | ||
Comment 25•11 years ago
|
||
Recompiled with optimization and got this backtrace: (gdb) bt #0 webrtc::VoEExternalMediaImpl::SetExternalRecordingStatus (this=0x106041d38, enable=false) at /Users/ekr/dev/mozilla-inbound/media/webrtc/trunk/webrtc/voice_engine/voe_external_media_impl.cc:151 #1 0x000000010260d4e8 in mozilla::WebrtcAudioConduit::~WebrtcAudioConduit (this=0x1403e8c50) at AudioConduit.cpp:68 (gdb)
Assignee | ||
Comment 26•11 years ago
|
||
Please test with the patch from bug 840344 to see if this goes away. I cannot get to the towtruck site.
Flags: needinfo?(ianb)
Reporter | ||
Comment 27•11 years ago
|
||
With the patch I'm not getting a crash, and the connection even establishes itself correctly.
Flags: needinfo?(ianb)
Comment 28•11 years ago
|
||
Closing as incomplete due to it at least no longer being testable or actionable due to the landing of bug 840344, and may well have been collateral damage caused by that bug and the API misuse.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → INCOMPLETE
Updated•10 years ago
|
Group: core-security
You need to log in
before you can comment on or make changes to this bug.
Description
•