1020411 - (CVE-2014-1550) Heap-use-after-free in nsRefPtr<mozilla::MediaInputPort>::operator

Status: VERIFIED FIXED

(Core :: Web Audio, defect)

Description

[Atte Kettunen]

Attachment: repro-file

Tested on:

OS: Ubuntu 12.04

Firefox: ASAN build from https://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux64-asan/1401864017/

Note:

To reproduce this issue you need two html-files. The runner loads the actual repro-file into iframe to trigger the crash. You might need to adjust the timing of "setTimeout(reload,50)" in the runner.

The timing seems to be CPU related and with 50ms timing I got crash reliably with local ASAN opt-build, but was barely able to trigger the crash with the ASAN opt-build from tinderbox.

With ASAN debug-build I only managed to hit 100% CPU load and no crash on my laptop, but Christoph (cdiehl) was able to reproduce also with debug-build. 

Because of the nature of the bug, I'm not able to minimize the repro-file any further.

ASAN-output:

==17451==ERROR: AddressSanitizer: heap-use-after-free on address 0x6140000dea40 at pc 0x7f56fb21f6b8 bp 0x7fff187c8170 sp 0x7fff187c8168
READ of size 8 at 0x6140000dea40 thread T0
    #0 0x7f56fb21f6b7 in nsRefPtr<mozilla::MediaInputPort>::operator->() const /home/attekett/firefox/content/media/MediaStreamGraph.cpp:2490
    #1 0x7f56fb2075c8 in mozilla::MediaStreamGraphImpl::RunInStableState() /home/attekett/firefox/content/media/MediaStreamGraph.cpp:1719
    #2 0x7f56fb220c10 in mozilla::(anonymous namespace)::MediaStreamGraphStableStateRunnable::Run() /home/attekett/firefox/content/media/MediaStreamGraph.cpp:1580
    #3 0x7f56f7898e08 in nsThread::ProcessNextEvent(bool, bool*) /home/attekett/firefox/xpcom/threads/nsThread.cpp:766
    #4 0x7f56f777de09 in NS_ProcessNextEvent(nsIThread*, bool) /home/attekett/firefox/xpcom/glue/nsThreadUtils.cpp:263
    #5 0x7f56f80a23a5 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /home/attekett/firefox/ipc/glue/MessagePump.cpp:95
    #6 0x7f56f80311fe in MessageLoop::RunInternal() /home/attekett/firefox/ipc/chromium/src/base/message_loop.cc:229
    #7 0x7f56fa146e2f in nsBaseAppShell::Run() /home/attekett/firefox/widget/xpwidgets/nsBaseAppShell.cpp:164
    #8 0x7f56fc9f98b6 in nsAppStartup::Run() /home/attekett/firefox/toolkit/components/startup/nsAppStartup.cpp:278
    #9 0x7f56fc869ba8 in XREMain::XRE_mainRun() /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4012
    #10 0x7f56fc86a849 in XREMain::XRE_main(int, char**, nsXREAppData const*) /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4083
    #11 0x7f56fc86b18b in XRE_main /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4297
    #12 0x479fa6 in do_main(int, char**, nsIFile*) /home/attekett/firefox/browser/app/nsBrowserApp.cpp:282
    #13 0x7f570548d76c in ?? ??:0
    #14 0x479448 in _start ??:0

0x6140000dea40 is located 0 bytes inside of 448-byte region [0x6140000dea40,0x6140000dec00)
freed by thread T0 here:
    #0 0x45f64f in __interceptor_free _asan_rtl_
    #1 0x7f56fb279bed in mozilla::dom::AudioNode::DisconnectFromGraph() /home/attekett/firefox/content/media/webaudio/AudioNode.cpp:176
    #2 0x7f56fb27392d in mozilla::dom::AudioNode::Release() /home/attekett/firefox/content/media/webaudio/AudioNode.cpp:44
    #3 0x7f56fb279a0d in ~nsRefPtr /home/attekett/firefox/objdir-ff-asan/content/media/webaudio/../../../dist/include/nsAutoPtr.h:870
    #4 0x7f56fb27392d in mozilla::dom::AudioNode::Release() /home/attekett/firefox/content/media/webaudio/AudioNode.cpp:44
    #5 0x7f56fb28f739 in nsRefPtr<mozilla::dom::AudioNode>::assign_assuming_AddRef(mozilla::dom::AudioNode*) /home/attekett/firefox/objdir-ff-asan/content/media/webaudio/../../../dist/include/nsAutoPtr.h:857
    #6 0x7f56fb2075c8 in mozilla::MediaStreamGraphImpl::RunInStableState() /home/attekett/firefox/content/media/MediaStreamGraph.cpp:1719
    #7 0x7f56fb220c10 in mozilla::(anonymous namespace)::MediaStreamGraphStableStateRunnable::Run() /home/attekett/firefox/content/media/MediaStreamGraph.cpp:1580
    #8 0x7f56f7898e08 in nsThread::ProcessNextEvent(bool, bool*) /home/attekett/firefox/xpcom/threads/nsThread.cpp:766
    #9 0x7f56f777de09 in NS_ProcessNextEvent(nsIThread*, bool) /home/attekett/firefox/xpcom/glue/nsThreadUtils.cpp:263
    #10 0x7f56f80a23a5 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /home/attekett/firefox/ipc/glue/MessagePump.cpp:95
    #11 0x7f56f80311fe in MessageLoop::RunInternal() /home/attekett/firefox/ipc/chromium/src/base/message_loop.cc:229
    #12 0x7f56fa146e2f in nsBaseAppShell::Run() /home/attekett/firefox/widget/xpwidgets/nsBaseAppShell.cpp:164
    #13 0x7f56fc9f98b6 in nsAppStartup::Run() /home/attekett/firefox/toolkit/components/startup/nsAppStartup.cpp:278
    #14 0x7f56fc869ba8 in XREMain::XRE_mainRun() /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4012
    #15 0x7f56fc86a849 in XREMain::XRE_main(int, char**, nsXREAppData const*) /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4083
    #16 0x7f56fc86b18b in XRE_main /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4297
    #17 0x479fa6 in do_main(int, char**, nsIFile*) /home/attekett/firefox/browser/app/nsBrowserApp.cpp:282
    #18 0x7f570548d76c in ?? ??:0

previously allocated by thread T0 here:
    #0 0x45f887 in __interceptor_malloc _asan_rtl_
    #1 0x7f5706847b63 in moz_xmalloc /home/attekett/firefox/memory/mozalloc/mozalloc.cpp:52
    #2 0x7f56fb20e624 in operator new(unsigned long) /home/attekett/firefox/objdir-ff-asan/content/media/../../dist/include/mozilla/mozalloc.h:201
    #3 0x7f56fb28b833 in ConvolverNode /home/attekett/firefox/content/media/webaudio/ConvolverNode.cpp:194
    #4 0x7f56fb27e787 in mozilla::dom::AudioContext::CreateConvolver() /home/attekett/firefox/content/media/webaudio/AudioContext.cpp:342
    #5 0x7f56f902d840 in mozilla::dom::AudioContextBinding::createConvolver(JSContext*, JS::Handle<JSObject*>, mozilla::dom::AudioContext*, JSJitMethodCallArgs const&) /home/attekett/firefox/objdir-ff-asan/dom/bindings/./AudioContextBinding.cpp:750
    #6 0x7f56f9f2c7c6 in mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) /home/attekett/firefox/dom/bindings/BindingUtils.cpp:2350
    #7 0x7f56fe7d4c13 in js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) /home/attekett/firefox/js/src/jscntxtinlines.h:239
    #8 0x7f56fe7c7b75 in Interpret(JSContext*, js::RunState&) /home/attekett/firefox/js/src/vm/Interpreter.cpp:2561
    #9 0x7f56fe7ab56c in js::RunScript(JSContext*, js::RunState&) /home/attekett/firefox/js/src/vm/Interpreter.cpp:402
    #10 0x7f56fe7881cb in js::ExecuteKernel(JSContext*, JS::Handle<JSScript*>, JSObject&, JS::Value const&, js::ExecuteType, js::AbstractFramePtr, JS::Value*) /home/attekett/firefox/js/src/vm/Interpreter.cpp:610
    #11 0x7f56fe7d6d4e in js::Execute(JSContext*, JS::Handle<JSScript*>, JSObject&, JS::Value*) /home/attekett/firefox/js/src/vm/Interpreter.cpp:646
    #12 0x7f56fe3ee424 in Evaluate(JSContext*, JS::Handle<JSObject*>, JS::ReadOnlyCompileOptions const&, JS::SourceBufferHolder&, JS::Value*) /home/attekett/firefox/js/src/jsapi.cpp:4998
    #13 0x7f56fa624675 in nsJSUtils::EvaluateString(JSContext*, JS::SourceBufferHolder&, JS::Handle<JSObject*>, JS::CompileOptions&, nsJSUtils::EvaluateOptions const&, JS::MutableHandle<JS::Value>, void**) /home/attekett/firefox/dom/base/nsJSUtils.cpp:256
    #14 0x7f56fa625514 in nsJSUtils::EvaluateString(JSContext*, JS::SourceBufferHolder&, JS::Handle<JSObject*>, JS::CompileOptions&, void**) /home/attekett/firefox/dom/base/nsJSUtils.cpp:322
    #15 0x7f56faea6a77 in nsScriptLoader::EvaluateScript(nsScriptLoadRequest*, JS::SourceBufferHolder&, void**) /home/attekett/firefox/content/base/src/nsScriptLoader.cpp:1172
    #16 0x7f56faea41dd in nsScriptLoader::ProcessRequest(nsScriptLoadRequest*, void**) /home/attekett/firefox/content/base/src/nsScriptLoader.cpp:1002
    #17 0x7f56fae9f280 in nsScriptLoader::ProcessScriptElement(nsIScriptElement*) /home/attekett/firefox/content/base/src/nsScriptLoader.cpp:812
    #18 0x7f56fae9bb32 in nsScriptElement::MaybeProcessScript() /home/attekett/firefox/content/base/src/nsScriptElement.cpp:140
    #19 0x7f56fa4676a0 in nsCOMPtr<nsIScriptElement>::operator->() const /home/attekett/firefox/objdir-ff-asan/parser/html/../../dist/include/nsIScriptElement.h:220
    #20 0x7f56fa46636f in nsHtml5TreeOpExecutor::RunFlushLoop() /home/attekett/firefox/parser/html/nsHtml5TreeOpExecutor.cpp:484
    #21 0x7f56fa48b53b in nsHtml5ExecutorFlusher::Run() /home/attekett/firefox/parser/html/nsHtml5StreamParser.cpp:126
    #22 0x7f56f7898e08 in nsThread::ProcessNextEvent(bool, bool*) /home/attekett/firefox/xpcom/threads/nsThread.cpp:766
    #23 0x7f56f777de09 in NS_ProcessNextEvent(nsIThread*, bool) /home/attekett/firefox/xpcom/glue/nsThreadUtils.cpp:263
    #24 0x7f56f80a23a5 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /home/attekett/firefox/ipc/glue/MessagePump.cpp:95
    #25 0x7f56f80311fe in MessageLoop::RunInternal() /home/attekett/firefox/ipc/chromium/src/base/message_loop.cc:229
    #26 0x7f56fa146e2f in nsBaseAppShell::Run() /home/attekett/firefox/widget/xpwidgets/nsBaseAppShell.cpp:164
    #27 0x7f56fc9f98b6 in nsAppStartup::Run() /home/attekett/firefox/toolkit/components/startup/nsAppStartup.cpp:278
    #28 0x7f56fc869ba8 in XREMain::XRE_mainRun() /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4012
    #29 0x7f56fc86a849 in XREMain::XRE_main(int, char**, nsXREAppData const*) /home/attekett/firefox/toolkit/xre/nsAppRunner.cpp:4083

SUMMARY: AddressSanitizer: heap-use-after-free ??:0 ??
Shadow bytes around the buggy address:
  0x0c2880013cf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2880013d00: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c2880013d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2880013d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2880013d30: 00 00 00 00 00 00 00 00 00 05 fa fa fa fa fa fa
=>0x0c2880013d40: fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd fd fd
  0x0c2880013d50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2880013d60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2880013d70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2880013d80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2880013d90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Contiguous container OOB:fc
  ASan internal:           fe
==17451==ABORTING

Comment 1

[Atte Kettunen]

Attachment: runner

Comment 2

[Karl Tomlinson (:karlt)]

Perhaps either control messages are out of order or the graph has not taken a reference to the stream early enough.

(In reply to Atte Kettunen from comment #0)
> ==17451==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x6140000dea40 at pc 0x7f56fb21f6b8 bp 0x7fff187c8170 sp 0x7fff187c8168
> READ of size 8 at 0x6140000dea40 thread T0
>     #0 0x7f56fb21f6b7 in nsRefPtr<mozilla::MediaInputPort>::operator->()
> const /home/attekett/firefox/content/media/MediaStreamGraph.cpp:2490
https://hg.mozilla.org/mozilla-central/annotate/49b4b6cf5de8/content/media/MediaStreamGraph.cpp#l2490

>     #1 0x7f56fb2075c8 in mozilla::MediaStreamGraphImpl::RunInStableState()
> /home/attekett/firefox/content/media/MediaStreamGraph.cpp:1719
https://hg.mozilla.org/mozilla-central/annotate/49b4b6cf5de8/content/media/MediaStreamGraph.cpp#l1719

> 0x6140000dea40 is located 0 bytes inside of 448-byte region
> [0x6140000dea40,0x6140000dec00)
> freed by thread T0 here:
>     #0 0x45f64f in __interceptor_free _asan_rtl_
>     #1 0x7f56fb279bed in mozilla::dom::AudioNode::DisconnectFromGraph()
> /home/attekett/firefox/content/media/webaudio/AudioNode.cpp:176
https://hg.mozilla.org/mozilla-central/annotate/49b4b6cf5de8/content/media/webaudio/AudioNode.cpp#l176

>     #2 0x7f56fb27392d in mozilla::dom::AudioNode::Release()
> /home/attekett/firefox/content/media/webaudio/AudioNode.cpp:44
>     #3 0x7f56fb279a0d in ~nsRefPtr
> /home/attekett/firefox/objdir-ff-asan/content/media/webaudio/../../../dist/
> include/nsAutoPtr.h:870
>     #4 0x7f56fb27392d in mozilla::dom::AudioNode::Release()
> /home/attekett/firefox/content/media/webaudio/AudioNode.cpp:44
>     #5 0x7f56fb28f739 in
> nsRefPtr<mozilla::dom::AudioNode>::assign_assuming_AddRef(mozilla::dom::
> AudioNode*)
> /home/attekett/firefox/objdir-ff-asan/content/media/webaudio/../../../dist/
> include/nsAutoPtr.h:857
>     #6 0x7f56fb2075c8 in mozilla::MediaStreamGraphImpl::RunInStableState()
> /home/attekett/firefox/content/media/MediaStreamGraph.cpp:1719
https://hg.mozilla.org/mozilla-central/annotate/49b4b6cf5de8/content/media/MediaStreamGraph.cpp#l1719

> previously allocated by thread T0 here:
>     #0 0x45f887 in __interceptor_malloc _asan_rtl_
>     #1 0x7f5706847b63 in moz_xmalloc
> /home/attekett/firefox/memory/mozalloc/mozalloc.cpp:52
>     #2 0x7f56fb20e624 in operator new(unsigned long)
> /home/attekett/firefox/objdir-ff-asan/content/media/../../dist/include/
> mozilla/mozalloc.h:201
>     #3 0x7f56fb28b833 in ConvolverNode
> /home/attekett/firefox/content/media/webaudio/ConvolverNode.cpp:194
https://hg.mozilla.org/mozilla-central/annotate/49b4b6cf5de8/content/media/webaudio/ConvolverNode.cpp#l194

Comment 3

[Andrew McCreight [:mccr8]]

It looks like Karl is investigating this, so I'm assigning it to him.  Please update as appropriate.

Comment 4

[Andrew McCreight [:mccr8]]

Karl, can you investigate this, or should I look for somebody else?  Thanks.

Comment 5

[Karl Tomlinson (:karlt)]

It seems that while the RunInStableState() list of
controlMessagesToRunDuringShutdown are being run (in order), a new
ControlMessage from MediaStream::Destroy() is (synchronously)
RunDuringShutdown() (out of order).

I guess AppendMessage() and RunInStableState() are assuming that calling RunDuringShutdown() control messages will not cause new messages to be AppendMessage()ed because AppendMessage() is main thread only but ControlMessages usually (when not RunDuringShutdown()) run on the graph thread.  The RunAfterPendingUpdates() ControlMessage violates that
assumption, by running main thread events synchronously from the
ControlMessage.

I'll look at fixing this.

Comment 6

[Karl Tomlinson (:karlt)]

Attachment: correct control message order with RunAfterPendingUpdates() at shutdown

Comment 7

[Karl Tomlinson (:karlt)]

Attachment: correct control message order with RunAfterPendingUpdates() at shutdown v1.1

Adjusted a comment.  (Runnables should be run in order.)

Comment 8

[Karl Tomlinson (:karlt)]

Comment on attachment 8449355 [details] [diff] [review]
correct control message order with RunAfterPendingUpdates() at shutdown v1.1

[Security approval request comment]
How easily could an exploit be constructed based on the patch?
Not easily.  An exploit would require cordinating the timing of different actions, which are not advertised by the patch.

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem? No.

Which older supported branches are affected by this flaw? 26 and newer.

If not all supported branches, which bug introduced the flaw? bug 923301

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be?
I haven't tried applying to older branches yet, but the core code involved hasn't changed.  Other parts of the patch are just for an assertion, and so backports will not be difficult or risky.

How likely is this patch to cause regressions; how much testing does it need?
The patch delays release of an object until the event loop runs.  Such changes should be low risk, even though the lifetimes are complex.

Comment 9

[Al Billings [:abillings - ex-MoCo]]

Comment on attachment 8449355 [details] [diff] [review]
correct control message order with RunAfterPendingUpdates() at shutdown v1.1

sec-approval+ for trunk.

Comment 10

[Karl Tomlinson (:karlt)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/f6c7cc1c3337

Comment 11

[Karl Tomlinson (:karlt)]

https://hg.mozilla.org/mozilla-central/rev/f6c7cc1c3337

Comment 12

[Karl Tomlinson (:karlt)]

Comment on attachment 8449355 [details] [diff] [review]
correct control message order with RunAfterPendingUpdates() at shutdown v1.1

Approval Request Comment
[Feature/regressing bug #]: bug 923301
[User impact if declined]: sec-critical
[Describe test coverage new/current, TBPL]: no testcase landed due to security risk.
[Risks and why]: The patch delays release of an object until the event loop runs.  Such changes should be low risk, even though the lifetimes are complex.
[String/UUID change made/needed]: none.

Comment 13

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8449355 [details] [diff] [review]
correct control message order with RunAfterPendingUpdates() at shutdown v1.1

Taking it because 31 is an ESR.

Comment 14

[Karl Tomlinson (:karlt)]

Attachment: patch for 30 branch

Comment 15

[Karl Tomlinson (:karlt)]

Attachment: patch for 28 branch

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/e12e7d58c028
https://hg.mozilla.org/releases/mozilla-beta/rev/307ac98a19c2
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/70bb28a6d264
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/c4a703996a6e

Comment 18

[Kamil Jozwiak [:kjozwiak]]

Reproduced the original issue from comment #0 using the following builds:
* http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux64-asan/1401864017/

Once I've reproduced the crash five different times (crashed within a minute), I went through verification using the following builds:
* m-c [PASSED]: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux64-asan/1404825288/
* aurora [PASSED]: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-aurora-linux64-asan/1404805201/
* beta [PASSED]: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-beta-linux64-asan/1404763596/

Running the above builds under Ubuntu 13.10 didn't cause any crashes after running the POC on each of the builds for about 10 minutes.