1238632 - crash in webrtc::VCMGenericEncoder::SetChannelParameters

Status: RESOLVED WONTFIX

(Core :: WebRTC, defect, P3)

Description

[Martijn Wargers (dead)]

I'm seeing this crash while doing some fuzzing. I haven't been able to get a minimized testcase that was reproducable, but I can reproduce it quite easily.

This bug was filed from the Socorro interface and is 
report bp-982d1a24-dcb6-4701-a059-81b7d2160107.
=============================================================
0 	XUL 	webrtc::VCMGenericEncoder::SetChannelParameters(int, long long) 	media/webrtc/trunk/webrtc/modules/video_coding/main/source/generic_encoder.cc
1 	XUL 	webrtc::(anonymous namespace)::VideoCodingModuleImpl::SetChannelParameters(unsigned int, unsigned char, long long) 	media/webrtc/trunk/webrtc/modules/video_coding/main/source/video_sender.cc
2 	XUL 	webrtc::ViEEncoder::OnNetworkChanged(unsigned int, unsigned char, long long) 	media/webrtc/trunk/webrtc/video_engine/vie_encoder.cc
3 	XUL 	webrtc::BitrateAllocator::AddBitrateObserver(webrtc::BitrateObserver*, unsigned int, unsigned int, unsigned int, int*) 	media/webrtc/trunk/webrtc/modules/bitrate_controller/bitrate_allocator.cc
4 	XUL 	webrtc::ViEEncoder::SetEncoder(webrtc::VideoCodec const&) 	media/webrtc/trunk/webrtc/video_engine/vie_encoder.cc
5 	XUL 	webrtc::ViECodecImpl::SetSendCodec(int, webrtc::VideoCodec const&) 	media/webrtc/trunk/webrtc/video_engine/vie_codec_impl.cc
6 	XUL 	mozilla::WebrtcVideoConduit::ConfigureSendMediaCodec(mozilla::VideoCodecConfig const*) 	media/webrtc/signaling/src/media-conduit/VideoConduit.cpp
7 	XUL 	mozilla::MediaPipelineFactory::GetOrCreateVideoConduit(mozilla::JsepTrackPair const&, mozilla::JsepTrack const&, RefPtr<mozilla::MediaSessionConduit>*) 	media/webrtc/signaling/src/peerconnection/MediaPipelineFactory.cpp
8 	XUL 	mozilla::MediaPipelineFactory::CreateOrUpdateMediaPipeline(mozilla::JsepTrackPair const&, mozilla::JsepTrack const&) 	media/webrtc/signaling/src/peerconnection/MediaPipelineFactory.cpp
9 	XUL 	mozilla::PeerConnectionMedia::UpdateMediaPipelines(mozilla::JsepSession const&) 	media/webrtc/signaling/src/peerconnection/PeerConnectionMedia.cpp
10 	XUL 	mozilla::PeerConnectionImpl::SetSignalingState_m(mozilla::dom::PCImplSignalingState, bool) 	media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

Can you repro it under rr, or in gdb?

Comment 2

[Martijn Wargers (dead)]

I'm on MacOS X, I guess I have to follow the directions here? https://developer.mozilla.org/en-US/docs/Debugging_on_Mac_OS_X
This will probably take a while.

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

I see 8 crashes from you, all on the same day, all look like null-derefs

Comment 4

[Martijn Wargers (dead)]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #2)
> I'm on MacOS X, I guess I have to follow the directions here?
> https://developer.mozilla.org/en-US/docs/Debugging_on_Mac_OS_X
> This will probably take a while.

I tried this, but I couldn't get it working, I get:
mwargers:mozilla-central mwargers$ source ~/.bash_profile
script: topsrcdir=/Users/mwargers/mozilla-central/: No such file or directory
-bash: /Users/mwargers/mozilla-central/.lldbinit: line 12: syntax error near unexpected token `('
-bash: /Users/mwargers/mozilla-central/.lldbinit: line 12: `script topsrcdir = topsrcdir if locals().has_key("topsrcdir") else "."; sys.path.append(os.path.join(topsrcdir, "python/lldbutils")); import lldbutils; lldbutils.init()'

Not sure if that is an actual fatal error, because I could follow the rest of the instructions at:
https://developer.mozilla.org/en-US/docs/Debugging_on_Mac_OS_X

And I could get debugging going for content processes, but now I get stuck at an assertion that happens before the crash:
  static const DisplayItemScrollClip*
  PickInnermost(const DisplayItemScrollClip* aClip1,
                const DisplayItemScrollClip* aClip2)
  {
    MOZ_ASSERT(IsAncestor(aClip1, aClip2) || IsAncestor(aClip2, aClip1),

aClip1	const mozilla::DisplayItemScrollClip *	NULL	
mParent	const mozilla::DisplayItemScrollClip *	NULL	
mScrollableFrame	nsIScrollableFrame *	NULL	
mClip	const mozilla::DisplayItemClip *	NULL	
mIsAsyncScrollable	bool		
mCrossStackingContextParent	const mozilla::DisplayItemScrollClip *	NULL	
mCrossStackingContextDepth	uint32_t		
aClip2	const mozilla::DisplayItemScrollClip *	NULL	
mParent	const mozilla::DisplayItemScrollClip *	NULL	
mScrollableFrame	nsIScrollableFrame *	NULL	
mClip	const mozilla::DisplayItemClip *	NULL	
mIsAsyncScrollable	bool		
mCrossStackingContextParent	const mozilla::DisplayItemScrollClip *	NULL	
mCrossStackingContextDepth	uint32_t		

It has probably nothing to do with this crash, though.

Is there any way of ignoring assertions while running in a debugger?

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

Use a non-Debug build - MOZ_ASSERT is a no-op then.

Comment 6

[Martijn Wargers (dead)]

Ok, I finally managed to get a stack from a non-debug build with debug symbols on my MacOSX10.11.3:

Thread 1Queue : com.apple.main-thread (serial)
#0	0x00000001028908a6 in webrtc::VCMGenericEncoder::SetChannelParameters(int, long long) at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/modules/video_coding/main/source/generic_encoder.cc:165
#1	0x000000010289f8c6 in webrtc::vcm::VideoSender::SetChannelParameters(unsigned int, unsigned char, long long) [inlined] at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/modules/video_coding/main/source/video_sender.cc:284
#2	0x000000010289f868 in webrtc::(anonymous namespace)::VideoCodingModuleImpl::SetChannelParameters(unsigned int, unsigned char, long long) at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/modules/video_coding/main/source/video_coding_impl.cc:151
#3	0x0000000102864fd9 in webrtc::ViEEncoder::OnNetworkChanged(unsigned int, unsigned char, long long) at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/video_engine/vie_encoder.cc:898
#4	0x0000000102757c05 in webrtc::BitrateAllocator::AddBitrateObserver(webrtc::BitrateObserver*, unsigned int, unsigned int, unsigned int, int*) at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/modules/bitrate_controller/bitrate_allocator.cc:108
#5	0x00000001028612c1 in webrtc::ViEEncoder::SetEncoder(webrtc::VideoCodec const&) at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/video_engine/vie_encoder.cc:375
#6	0x0000000102860757 in webrtc::ViECodecImpl::SetSendCodec(int, webrtc::VideoCodec const&) at /Users/mwargers/mozilla-central-opt/media/webrtc/trunk/webrtc/video_engine/vie_codec_impl.cc:214
#7	0x0000000100be55ae in mozilla::WebrtcVideoConduit::ConfigureSendMediaCodec(mozilla::VideoCodecConfig const*) at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/media-conduit/VideoConduit.cpp:716
#8	0x0000000100c0a9e9 in mozilla::MediaPipelineFactory::GetOrCreateVideoConduit(mozilla::JsepTrackPair const&, mozilla::JsepTrack const&, RefPtr<mozilla::MediaSessionConduit>*) at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/MediaPipelineFactory.cpp:813
#9	0x0000000100c04a98 in mozilla::MediaPipelineFactory::CreateOrUpdateMediaPipeline(mozilla::JsepTrackPair const&, mozilla::JsepTrack const&) at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/MediaPipelineFactory.cpp:425
#10	0x0000000100c2ca80 in mozilla::PeerConnectionMedia::UpdateMediaPipelines(mozilla::JsepSession const&) at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/PeerConnectionMedia.cpp:529
#11	0x0000000100c1d44d in mozilla::PeerConnectionImpl::SetSignalingState_m(mozilla::dom::PCImplSignalingState, bool) at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:2790
#12	0x0000000100c18315 in mozilla::PeerConnectionImpl::UpdateSignalingState(bool) [inlined] at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:2870
#13	0x0000000100c18306 in mozilla::PeerConnectionImpl::SetLocalDescription(int, char const*) at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1721
#14	0x0000000101440737 in mozilla::PeerConnectionImpl::SetLocalDescription(int, nsAString_internal const&, mozilla::ErrorResult&) [inlined] at /Users/mwargers/mozilla-central-opt/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.h:391
#15	0x00000001014406cc in mozilla::dom::PeerConnectionImplBinding::setLocalDescription(JSContext*, JS::Handle<JSObject*>, mozilla::PeerConnectionImpl*, JSJitMethodCallArgs const&) at /Users/mwargers/mozilla-central-opt/obj-x86_64-apple-darwin15.3.0/dom/bindings/PeerConnectionImplBinding.cpp:188
#16	0x0000000101a98e8d in mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) at /Users/mwargers/mozilla-central-opt/dom/bindings/BindingUtils.cpp:2731
#17	0x00000001037d6b44 in js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [inlined] at /Users/mwargers/mozilla-central-opt/js/src/jscntxtinlines.h:235
#18	0x00000001037d6a4a in js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:478
#19	0x00000001037cfe92 in Interpret(JSContext*, js::RunState&) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:2802
#20	0x00000001037c27ad in js::RunScript(JSContext*, js::RunState&) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:428
#21	0x00000001037d6e8b in js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:496
#22	0x00000001037d70ee in js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:530
#23	0x00000001036499fd in JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) at /Users/mwargers/mozilla-central-opt/js/src/jsapi.cpp:2892
#24	0x000000010147150e in mozilla::dom::PromiseInit::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JSObject*>, JS::Handle<JSObject*>, mozilla::ErrorResult&) at /Users/mwargers/mozilla-central-opt/obj-x86_64-apple-darwin15.3.0/dom/bindings/PromiseBinding.cpp:47
#25	0x000000010211f29d in mozilla::dom::PromiseInit::Call(JS::Handle<JSObject*>, JS::Handle<JSObject*>, mozilla::ErrorResult&, char const*, mozilla::dom::CallbackObject::ExceptionHandling, JSCompartment*) [inlined] at /Users/mwargers/mozilla-central-opt/obj-x86_64-apple-darwin15.3.0/dist/include/mozilla/dom/PromiseBinding.h:76
#26	0x000000010211f288 in mozilla::dom::Promise::CallInitFunction(mozilla::dom::GlobalObject const&, mozilla::dom::PromiseInit&, mozilla::ErrorResult&) at /Users/mwargers/mozilla-central-opt/dom/promise/Promise.cpp:1129
#27	0x000000010211f136 in mozilla::dom::Promise::Constructor(mozilla::dom::GlobalObject const&, mozilla::dom::PromiseInit&, mozilla::ErrorResult&, JS::Handle<JSObject*>) at /Users/mwargers/mozilla-central-opt/dom/promise/Promise.cpp:1099
#28	0x000000010148bec5 in mozilla::dom::PromiseBinding::_constructor(JSContext*, unsigned int, JS::Value*) at /Users/mwargers/mozilla-central-opt/obj-x86_64-apple-darwin15.3.0/dom/bindings/PromiseBinding.cpp:461
#29	0x0000000100ac9e9f in xpc::DOMXrayTraits::construct(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&, js::Wrapper const&) at /Users/mwargers/mozilla-central-opt/js/xpconnect/wrappers/XrayWrapper.cpp:1699
#30	0x0000000103707b56 in js::Proxy::construct(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) at /Users/mwargers/mozilla-central-opt/js/src/proxy/Proxy.cpp:410
#31	0x00000001037084ea in js::proxy_Construct(JSContext*, unsigned int, JS::Value*) at /Users/mwargers/mozilla-central-opt/js/src/proxy/Proxy.cpp:692
#32	0x00000001037d7340 in js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [inlined] at /Users/mwargers/mozilla-central-opt/js/src/jscntxtinlines.h:235
#33	0x00000001037d7206 in js::CallJSNativeConstructor(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [inlined] at /Users/mwargers/mozilla-central-opt/js/src/jscntxtinlines.h:268
#34	0x00000001037d7206 in InternalConstruct(JSContext*, JS::CallArgs const&) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:567
#35	0x00000001037cfe50 in ConstructFromStack(JSContext*, JS::CallArgs const&) [inlined] at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:594
#36	0x00000001037cfe00 in Interpret(JSContext*, js::RunState&) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:2794
#37	0x00000001037c27ad in js::RunScript(JSContext*, js::RunState&) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:428
#38	0x00000001037d6e8b in js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:496
#39	0x00000001037d70ee in js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) at /Users/mwargers/mozilla-central-opt/js/src/vm/Interpreter.cpp:530
#40	0x00000001036499fd in JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) at /Users/mwargers/mozilla-central-opt/js/src/jsapi.cpp:2892
#41	0x000000010147186b in mozilla::dom::AnyCallback::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&) at /Users/mwargers/mozilla-central-opt/obj-x86_64-apple-darwin15.3.0/dom/bindings/PromiseBinding.cpp:93
#42	0x0000000102126c2e in mozilla::dom::AnyCallback::Call(JS::Handle<JS::Value>, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&, char const*, mozilla::dom::CallbackObject::ExceptionHandling, JSCompartment*) [inlined] at /Users/mwargers/mozilla-central-opt/obj-x86_64-apple-darwin15.3.0/dist/include/mozilla/dom/PromiseBinding.h:224
#43	0x0000000102126bd5 in mozilla::dom::WrapperPromiseCallback::Call(JSContext*, JS::Handle<JS::Value>) at /Users/mwargers/mozilla-central-opt/dom/promise/PromiseCallback.cpp:336
#44	0x000000010212a12b in mozilla::dom::PromiseReactionJob::Run() at /Users/mwargers/mozilla-central-opt/dom/promise/Promise.cpp:106
#45	0x000000010211e1d6 in mozilla::dom::Promise::PerformMicroTaskCheckpoint() at /Users/mwargers/mozilla-central-opt/dom/promise/Promise.cpp:937
#46	0x0000000100401d3c in mozilla::CycleCollectedJSRuntime::AfterProcessTask(unsigned int) at /Users/mwargers/mozilla-central-opt/xpcom/base/CycleCollectedJSRuntime.cpp:1348
#47	0x0000000100b0b109 in XPCJSRuntime::AfterProcessTask(unsigned int) at /Users/mwargers/mozilla-central-opt/js/xpconnect/src/XPCJSRuntime.cpp:3688
#48	0x000000010045ed88 in nsThread::ProcessNextEvent(bool, bool*) at /Users/mwargers/mozilla-central-opt/xpcom/threads/nsThread.cpp:1009
#49	0x00000001004879ee in NS_ProcessPendingEvents(nsIThread*, unsigned int) at /Users/mwargers/mozilla-central-opt/xpcom/glue/nsThreadUtils.cpp:239
#50	0x00000001022545bf in nsBaseAppShell::NativeEventCallback() at /Users/mwargers/mozilla-central-opt/widget/nsBaseAppShell.cpp:97
#51	0x00000001022acd87 in nsAppShell::ProcessGeckoEvents(void*) at /Users/mwargers/mozilla-central-opt/widget/cocoa/nsAppShell.mm:387
#52	0x00007fff8fec75c1 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ ()
#53	0x00007fff8feb941c in __CFRunLoopDoSources0 ()
#54	0x00007fff8feb893f in __CFRunLoopRun ()
#55	0x00007fff8feb8338 in CFRunLoopRunSpecific ()
#56	0x00007fff87586935 in RunCurrentEventLoopInMode ()
#57	0x00007fff8758676f in ReceiveNextEventCommon ()
#58	0x00007fff875865af in _BlockUntilNextEventMatchingListInModeWithFilter ()
#59	0x00007fff94d970ee in _DPSNextEvent ()
#60	0x00007fff95163943 in -[NSApplication _nextEventMatchingEventMask:untilDate:inMode:dequeue:] ()
#61	0x00000001022ac482 in -[GeckoNSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] at /Users/mwargers/mozilla-central-opt/widget/cocoa/nsAppShell.mm:121
#62	0x00007fff94d8cfc8 in -[NSApplication run] ()
#63	0x00000001022ad341 in nsAppShell::Run() at /Users/mwargers/mozilla-central-opt/widget/cocoa/nsAppShell.mm:661
#64	0x0000000102b9dea1 in ::XRE_RunAppShell() at /Users/mwargers/mozilla-central-opt/toolkit/xre/nsEmbedFunctions.cpp:789
#65	0x000000010073da28 in MessageLoop::RunInternal() [inlined] at /Users/mwargers/mozilla-central-opt/ipc/chromium/src/base/message_loop.cc:234
#66	0x000000010073da19 in MessageLoop::RunHandler() [inlined] at /Users/mwargers/mozilla-central-opt/ipc/chromium/src/base/message_loop.cc:227
#67	0x000000010073da19 in MessageLoop::Run() at /Users/mwargers/mozilla-central-opt/ipc/chromium/src/base/message_loop.cc:201
#68	0x0000000102b9da72 in ::XRE_InitChildProcess(int, char **, mozilla::gmp::GMPLoader *) at /Users/mwargers/mozilla-central-opt/toolkit/xre/nsEmbedFunctions.cpp:625
#69	0x0000000100003db8 in content_process_main(int, char**) at /Users/mwargers/mozilla-central-opt/ipc/app/../contentproc/plugin-container.cpp:237
#70	0x0000000100001074 in start ()

This is with the debugger with the one mentioned in comment 2. I hope this is useful to you. If not, please let me know what I should do.

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

Sorry for not replying earlier.

The stack confirms what you hit, but the reason I asked about gdb and rr was so that we could figure out how we got there (rr), or at least look at what variables are in structs (p *this) and the stack (gdb).  I strongly suspect from this that encoder_ in SetChannelParameters is null (and in fact that's pretty much the only pointer dereferenced there).

that can only happen (I believe) if it was null to start with (which I think is very unlikely), or it was trashed (where rr (or ASAN) would help).  It's never modified (it's VideoEncoder* const encoder_;)

ASAN is another possibility.  It would catch any trashing.  And if you're fuzzing, ASAN is often a win.

Or some way for me to hit it myself.

Comment 8

[Martijn Wargers (dead)]

Attachment: crash1.htm

Ok, so it's just a matter of the script that's causing the crash.

Comment 9

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.