824971 - Heap-use-after-free in mozilla::MediaInputPort::Disconnect

Status: RESOLVED WORKSFORME

(Core :: Audio/Video, defect)

Description

[Abhishek Arya]

Reproduces on trunk. Just need to wait like 10-15 sec for the crash to happen.

>==2645== ERROR: AddressSanitizer: heap-use-after-free on address 0x7fd213057680 at pc 0x7fd23d050cc2 bp 0x7fd2079f6620 sp 0x7fd2079f6618
>READ of size 8 at 0x7fd213057680 thread T33
>    #0 0x7fd23d050cc1 in mozilla::MediaInputPort::Disconnect() src/content/media/MediaStreamGraph.cpp:2139
>    #1 0x7fd23d05efcc in mozilla::(anonymous namespace)::MediaStreamGraphThreadRunnable::Run() src/content/media/MediaStreamGraph.cpp:1531
>    #2 0x7fd23e913c32 in NS_ProcessNextEvent_P(nsIThread*, bool) src/objdir-ff-asan/xpcom/build/nsThreadUtils.cpp:237
>    #3 0x7fd23e9d8dbc in nsThread::ThreadFunc(void*) src/xpcom/threads/nsThread.cpp:265
>    #4 0x7fd2446f375c in _pt_root src/nsprpub/pr/src/pthreads/ptthread.c:156
>    #5 0x415f3a in __asan::AsanThread::ThreadStart()
>0x7fd213057680 is located 0 bytes inside of 224-byte region [0x7fd213057680,0x7fd213057760)
>freed by thread T0 here:
>    #0 0x411232 in __interceptor_free
>    #1 0x7fd23d06fdfa in Release src/content/media/MediaStreamGraph.h:255
>    #2 0x7fd23d06fdfa in ~nsRefPtr src/../../dist/include/nsAutoPtr.h:876
>    #3 0x7fd23d06fdfa in ~nsRefPtr src/../../dist/include/nsAutoPtr.h:874
>    #4 0x7fd23d06fdfa in ~OutputStreamData src/content/media/MediaDecoder.h:385
>    #5 0x7fd23d06fdfa in ~OutputStreamData src/content/media/MediaDecoder.h:385
>    #6 0x7fd23d06fdfa in Destruct src/../../dist/include/nsTArray.h:432
>    #7 0x7fd23d06fdfa in DestructRange src/../../dist/include/nsTArray.h:1297
>    #8 0x7fd23d06fdfa in nsTArray_Impl<mozilla::MediaDecoder::OutputStreamData, nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned int, unsigned int) src/../../dist/include/nsTArray.h:1017
>previously allocated by thread T0 here:
>    #0 0x411312 in malloc
>    #1 0x7fd242b8d148 in moz_xmalloc src/memory/mozalloc/mozalloc.cpp:54
>Thread T33 created by T0 here:
>    #0 0x40d294 in __interceptor_pthread_create
>    #1 0x7fd2446ef9ef in _PR_CreateThread src/nsprpub/pr/src/pthreads/ptthread.c:393
>    #2 0x7fd2446ef44a in PR_CreateThread src/nsprpub/pr/src/pthreads/ptthread.c:476
>    #3 0x7fd23e9d9db9 in nsThread::Init() src/xpcom/threads/nsThread.cpp:331
>Shadow bytes around the buggy address:
>  0x1ffa4260ae80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>  0x1ffa4260ae90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
>  0x1ffa4260aea0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
>  0x1ffa4260aeb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
>  0x1ffa4260aec0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>=>0x1ffa4260aed0:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
>  0x1ffa4260aee0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
>  0x1ffa4260aef0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>  0x1ffa4260af00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>  0x1ffa4260af10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>  0x1ffa4260af20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>Shadow byte legend (one shadow byte represents 8 application bytes):
>  Addressable:           00
>  Partially addressable: 01 02 03 04 05 06 07
>  Heap left redzone:     fa
>  Heap righ redzone:     fb
>  Freed Heap region:     fd
>  Stack left redzone:    f1
>  Stack mid redzone:     f2
>  Stack right redzone:   f3
>  Stack partial redzone: f4
>  Stack after return:    f5
>  Stack use after scope: f8
>  Global redzone:        f9
>  Global init order:     f6
>  Poisoned by user:      f7
>  ASan internal:         fe
>Stats: 564M malloced (457M for red zones) by 577792 calls
>Stats: 53M realloced by 30247 calls
>Stats: 518M freed by 442226 calls
>Stats: 475M really freed by 416400 calls
>Stats: 319M (319M-0M) mmaped; 590 maps, 0 unmaps
>  mmaps   by size class: 8:165807; 9:21483; 10:6643; 11:7905; 12:1664; 13:2432; 14:2368; 15:256; 16:776; 17:460; 18:36; 19:36; 20:33; 21:4;
>  mallocs by size class: 8:454528; 9:53836; 10:13671; 11:26416; 12:3999; 13:9684; 14:10353; 15:755; 16:2638; 17:1636; 18:111; 19:47; 20:114; 21:4;
>  frees   by size class: 8:338453; 9:43056; 10:9572; 11:24217; 12:2825; 13:9276; 14:9827; 15:613; 16:2534; 17:1614; 18:92; 19:44; 20:100; 21:3;
>  rfrees  by size class: 8:319051; 9:40842; 10:8940; 11:23374; 12:2613; 13:8183; 14:8642; 15:584; 16:2360; 17:1596; 18:85; 19:43; 20:84; 21:3;
>Stats: malloc large: 5305 small slow: 14753
>==2645== ABORTING
>

Comment 1

[Abhishek Arya]

Attachment: Testcase (Part 1 of Split Archive)

Splitting archive because of 4 MB attachment limit

Comment 2

[Abhishek Arya]

Attachment: Testcase (Part 2 of Split Archive)

Comment 3

[David Bolter [:davidb] (NeedInfo me for attention)]

Guessing sec-critical. Hoping Chris can take a look.

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

This is Roc's code.

Comment 5

[Daniel Veditz [:dveditz]]

Roc: is this bug on old branches as well or in new code?

Comment 6

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Goes back to 19 at least.

I couldn't reproduce on a regular Windows build. I need to try it in a Linux ASAN build.

Comment 7

[Abhishek Arya]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #6)
> Goes back to 19 at least.
> 
> I couldn't reproduce on a regular Windows build. I need to try it in a Linux
> ASAN build.

Just checked, it reproduces on trunk easily with linux asan build.

Comment 8

[David Bolter [:davidb] (NeedInfo me for attention)]

Matt agreed to try to find a regression window here. Tacking a needinfo on.

Comment 9

[Matt Wobensmith [:mwobensmith][:matt:]]

This crash happens even on a non-ASan build (Mac) with 17.0.2 ESR and 18.0.1 release.

Happy to run on ASan builds of the above, but I think we can confirm comment 5 and comment 6 with this information.

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

Too late for 19b6, we'll keep tracking for 20.

Comment 11

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I cannot reproduce this on trunk with a Linux ASAN build.

Comment 12

[Abhishek Arya]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #11)
> I cannot reproduce this on trunk with a Linux ASAN build.

Verified that it does not reproduce anymore on trunk. Previously it did however reproduce reliably, also confirmed by Matt in c#9.

Comment 13

[Al Billings [:abillings - ex-MoCo]]

Matt, please figure out which releases are still affected (20, 21?).

Comment 14

[Matt Wobensmith [:mwobensmith][:matt:]]

Haven't tried Linux, but on Mac, today's builds:

- ASan m-c 22.0a1 - assertion/errors - no crash
- Aurora 21.0a2 debug - no crash
- Beta 20.0 debug - crash

We don't have ASan builds for every release, but I hope this is helpful. I'm happy to do more of a deep dive on any release if need be.

More details below.

ASan m-c 22.0a1 2013-02-28:

WARNING: Load group requested for media element in inactive document.: file /builds/slave/try-osx64-double-000000000000000000/build/content/html/content/src/nsHTMLMediaElement.cpp, line 3365
WARNING: have unconstrained width; this should only result from very large sizes, not attempts at intrinsic width calculation: '(mFrameType == NS_CSS_FRAME_TYPE_INLINE && !frame->IsFrameOfType(nsIFrame::eReplaced)) || type == nsGkAtoms::textFrame || mComputedWidth != NS_UNCONSTRAINEDSIZE', file /builds/slave/try-osx64-double-000000000000000000/build/layout/generic/nsHTMLReflowState.cpp, line 377
###!!! ASSERTION: Our containing block must not have unconstrained width!: 'aCBSize.width != NS_UNCONSTRAINEDSIZE', file /builds/slave/try-osx64-double-000000000000000000/build/layout/base/nsLayoutUtils.cpp, line 3174
nsLineLayout: HTMLVideo(video)(0)@0x142942940 metrics=1073741824,60!
WARNING: Overflowed nscoord_MAX in conversion to nscoord width: file ../../dist/include/nsRect.h, line 131


Aurora 21.0a2 debug 2013-02-28:
No crash.


Beta 20.0 debug 2013-02-28:

Crash:
https://crash-stats.mozilla.com/report/index/2c88b193-5be3-4512-90f2-d14a52130228

Comment 15

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I guess one of our other fixes fixed this.

Comment 16

[Daniel Veditz [:dveditz]]

sometime between 20 and 21 this got fixed. We need to identify that fix if we can and see if it's appropriate to backport to ESR-17 and the b2g18 branch.

Comment 17

[Matt Wobensmith [:mwobensmith][:matt:]]

I'm still working on this, but an update.

This crash is intermittent and can be hard to reproduce. It often takes a few tries on a given build to see a crash - anywhere between 30 seconds and 10 minutes until the crash occurs. Or, sometimes not at all.

All builds of 19 appear to be affected - including current release - as well as 20.

I'm still spending time on 21/22 to see if they are affected. Currently they seem not to be, but will update once I've gotten sufficient coverage.

Comment 18

[Alex Keybl [:akeybl]]

If a fix-window proves to be too difficult to find, we'll likely wontfix entirely for ESR.

Comment 19

[Matt Wobensmith [:mwobensmith][:matt:]]

Well, in summary, it appears that all builds of 19/20 are affected, and no builds of 21/22 appear to be. 

I'm not sure that makes sense to me, but after spending a lot of time on it, that's all I can come up with.

Comment 20

[Alex Keybl [:akeybl]]

dveditz - do you agree with wontfixing for ESR17 based upon comment 18 and comment 19? Presumably we wouldn't open this bug up until ESR24 in that case.

Comment 21

[Daniel Veditz [:dveditz]]

That approach troubles me. Not so much about ESR given the lack of use, but if we don't know what fixed it there may not, in fact, be a fix. This particular testcase could just be masked.

I guess we just have to hope that if it's still lurking the fuzzers will pop it out on another testcase.

Jesse: does your fuzzer exercise MediaInputPort?

Comment 22

[Jesse Ruderman]

I don't know.  Might improve when cdiehl and I merge his code into my DOM fuzzer.

Comment 23

[u279076]

Dropping QAWANTED since we've been unable to find a fix range narrower than what's indicated in comment 19 after considerable effort. Matt is still free to work on this bug as he chooses but I don't see much benefit in keeping this bug on QA's release tracking queue.