1085175 - (CVE-2014-1593) Stack-buffer-overflow Write in mozilla::FileBlockCache::Read

Status: RESOLVED FIXED

(Core :: Audio/Video, defect)

Description

[Abhishek Arya]

Attachment: stackw.zip

==35313==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f042e1930d1 at pc 0x000000454893 bp 0x7f042e192af0 sp 0x7f042e1922b0
WRITE of size 8200 at 0x7f042e1930d1 thread T83 (Media Decode #1)
    #0 0x454892 in __interceptor_read.part.40 /build/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:352
    #1 0x7f0467295fc7 in pt_Read /build/firefox/src/nsprpub/pr/src/pthreads/ptio.c:1287:13
    #2 0x7f0459e1962f in (anonymous namespace)::interposedRead(PRFileDesc*, void*, int) /build/firefox/src/xpcom/build/NSPRInterposer.cpp:58:10
    #3 0x7f045e2b92ec in mozilla::FileBlockCache::Read(long, unsigned char*, int, int*) /build/firefox/src/content/media/FileBlockCache.cpp:146:16
    #4 0x7f045e2d380c in mozilla::MediaCacheStream::Read(char*, unsigned int, unsigned int*) /build/firefox/src/content/media/MediaCache.cpp:672:10
    #5 0x7f045e32395d in mozilla::ChannelMediaResource::Read(char*, unsigned int, unsigned int*)
    #6 0x7f045e4e8875 in mozilla::webm_read(void*, unsigned long, void*)
    #7 0x7f046047b1f6 in ne_peek_element /build/firefox/src/media/libnestegg/src/nestegg.c:551:10
    #8 0x5d8997ddbd0d9e6f (<unknown module>)

Address 0x7f042e1930d1 is located in stack of thread T83 (Media Decode #1) at offset 49 in frame
    #0 0x7f046047b05f in ne_peek_element /build/firefox/src/media/libnestegg/src/nestegg.c:892

  This frame has 2 object(s):
    [32, 33) 'b.i.i1'
    [48, 49) 'b.i.i' <== Memory access at offset 49 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
Thread T83 (Media Decode #1) created by T82 (Media S~hine #1) here:
    #0 0x439cbe in __interceptor_pthread_create _asan_rtl_
    #1 0x7f0467297030 in _PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:453:14
    #2 0x7f0467296c5a in PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:544:12
    #3 0x7f0459de2805 in nsThread::Init() /build/firefox/src/xpcom/threads/nsThread.cpp:455:19
    #4 0x7f0459de7d94 in nsThreadManager::NewThread(unsigned int, unsigned int, nsIThread**) /build/firefox/src/xpcom/threads/nsThreadManager.cpp:269:17
    #5 0x7f0459de91e0 in nsThreadPool::PutEvent(nsIRunnable*) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:101:3
    #6 0x7f0459deaa0c in nsThreadPool::Dispatch(nsIRunnable*, unsigned int) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:261:5
    #7 0x7f045e3469a4 in mozilla::MediaTaskQueue::DispatchLocked(mozilla::TemporaryRef<nsIRunnable>, mozilla::MediaTaskQueue::DispatchMode) /build/firefox/src/content/media/MediaTaskQueue.cpp:53:17
    #8 0x7f045e30bc08 in mozilla::MediaDecoderStateMachine::EnqueueDecodeMetadataTask() /build/firefox/src/content/media/MediaTaskQueue.cpp:34:10
    #9 0x7f045e31045a in mozilla::MediaDecoderStateMachine::RunStateMachine() /build/firefox/src/content/media/MediaDecoderStateMachine.cpp:2376:14
    #10 0x7f045e364136 in non-virtual thunk to (anonymous namespace)::TimerEvent::Run() /build/firefox/src/content/media/MediaDecoderStateMachineScheduler.cpp:160:10
    #11 0x7f0459dea070 in nsThreadPool::Run() /build/firefox/src/xpcom/threads/nsThreadPool.cpp:220:7
    #12 0x7f0459dea5bc in non-virtual thunk to nsThreadPool::Run()
    #13 0x7f0459de46f6 in nsThread::ProcessNextEvent(bool, bool*) /build/firefox/src/xpcom/threads/nsThread.cpp:830:7
    #14 0x7f0459e3aa66 in NS_ProcessNextEvent(nsIThread*, bool) /build/firefox/src/xpcom/glue/nsThreadUtils.cpp:265:10
    #15 0x7f045a66d29e in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /build/firefox/src/ipc/glue/MessagePump.cpp:368:5
    #16 0x7f045a61c561 in MessageLoop::Run() /build/firefox/src/ipc/chromium/src/base/message_loop.cc:233:3
    #17 0x7f0459de14a6 in nsThread::ThreadFunc(void*) /build/firefox/src/xpcom/threads/nsThread.cpp:350:5
    #18 0x7f046729a4d0 in _pt_root /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:212:5
    #19 0x7f046aa1fe99 in start_thread /build/buildd/eglibc-2.15/nptl/pthread_create.c:308

Thread T82 (Media S~hine #1) created by T0 here:
    #0 0x439cbe in __interceptor_pthread_create _asan_rtl_
    #1 0x7f0467297030 in _PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:453:14
    #2 0x7f0467296c5a in PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:544:12
    #3 0x7f0459de2805 in nsThread::Init() /build/firefox/src/xpcom/threads/nsThread.cpp:455:19
    #4 0x7f0459de7d94 in nsThreadManager::NewThread(unsigned int, unsigned int, nsIThread**) /build/firefox/src/xpcom/threads/nsThreadManager.cpp:269:17
    #5 0x7f0459de91e0 in nsThreadPool::PutEvent(nsIRunnable*) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:101:3
    #6 0x7f0459deaa0c in nsThreadPool::Dispatch(nsIRunnable*, unsigned int) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:261:5
    #7 0x7f045e315989 in mozilla::MediaDecoderStateMachineScheduler::Schedule(long) /build/firefox/src/content/media/MediaDecoderStateMachineScheduler.cpp:129:10
    #8 0x7f045e2e2fde in mozilla::MediaDecoder::InitializeStateMachine(mozilla::MediaDecoder*) /build/firefox/src/content/media/MediaDecoderStateMachine.cpp:3025:10
    #9 0x7f045e14c73d in mozilla::dom::HTMLMediaElement::FinishDecoderSetup(mozilla::MediaDecoder*, mozilla::MediaResource*, nsIStreamListener**, mozilla::MediaDecoder*) /build/firefox/src/content/html/content/src/HTMLMediaElement.cpp:2659:17
    #10 0x7f045e13b388 in mozilla::dom::HTMLMediaElement::InitializeDecoderForChannel(nsIChannel*, nsIStreamListener**) /build/firefox/src/content/html/content/src/HTMLMediaElement.cpp:2607:12
    #11 0x7f045e13a0b3 in mozilla::dom::HTMLMediaElement::MediaLoadListener::OnStartRequest(nsIRequest*, nsISupports*) /build/firefox/src/content/html/content/src/HTMLMediaElement.cpp:349:7
    #12 0x7f045a33ce42 in mozilla::net::nsHttpChannel::CallOnStartRequest() /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:914:14
    #13 0x7f045a348c45 in mozilla::net::nsHttpChannel::ContinueProcessNormal(tag_nsresult) /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:1668:10
    #14 0x7f045a34555a in mozilla::net::nsHttpChannel::ProcessNormal() /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:1603:12
    #15 0x7f045a344a55 in mozilla::net::nsHttpChannel::ProcessResponse() /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:1513:14
    #16 0x7f045a362e4e in mozilla::net::nsHttpChannel::OnStartRequest(nsIRequest*, nsISupports*) /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:5172:20
    #17 0x7f0459f73fb6 in nsInputStreamPump::OnStateStart() /build/firefox/src/netwerk/base/src/nsInputStreamPump.cpp:531:14
    #18 0x7f0459f734c0 in nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) /build/firefox/src/netwerk/base/src/nsInputStreamPump.cpp:433:25
    #19 0x7f0459dac829 in nsInputStreamReadyEvent::Run() /build/firefox/src/xpcom/io/nsStreamUtils.cpp:88:9
    #20 0x7f0459de46f6 in nsThread::ProcessNextEvent(bool, bool*) /build/firefox/src/xpcom/threads/nsThread.cpp:830:7
    #21 0x7f0459e3aa66 in NS_ProcessNextEvent(nsIThread*, bool) /build/firefox/src/xpcom/glue/nsThreadUtils.cpp:265:10
    #22 0x7f045a66c34f in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /build/firefox/src/ipc/glue/MessagePump.cpp:99:21
    #23 0x7f045a61c561 in MessageLoop::Run() /build/firefox/src/ipc/chromium/src/base/message_loop.cc:233:3
    #24 0x7f045dbd214f in nsBaseAppShell::Run()
    #25 0x7f04600884f1 in nsAppStartup::Run() /build/firefox/src/toolkit/components/startup/nsAppStartup.cpp:280:19
    #26 0x7f0460169c9e in XREMain::XRE_mainRun() /build/firefox/src/toolkit/xre/nsAppRunner.cpp:4095:10
    #27 0x7f046016ac86 in XREMain::XRE_main(int, char**, nsXREAppData const*) /build/firefox/src/toolkit/xre/nsAppRunner.cpp:4168:8
    #28 0x7f046016bac6 in XRE_main /build/firefox/src/toolkit/xre/nsAppRunner.cpp:4382:16
    #29 0x4bf702 in main /build/firefox/src/browser/app/nsBrowserApp.cpp:287:12
    #30 0x7f0469a5c76c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226

SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 ??
Shadow bytes around the buggy address:
  0x0fe105c2a5c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe105c2a5d0: f1 f1 f1 f1 04 f3 f3 f3 00 00 00 00 00 00 00 00
  0x0fe105c2a5e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe105c2a5f0: f1 f1 f1 f1 04 f3 f3 f3 00 00 00 00 00 00 00 00
  0x0fe105c2a600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0fe105c2a610: 00 00 00 00 f1 f1 f1 f1 01 f2[01]f3 00 00 00 00
  0x0fe105c2a620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe105c2a630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe105c2a640: f1 f1 f1 f1 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe105c2a650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe105c2a660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  ASan internal:           fe
==35313==ABORTING

Comment 1

[Abhishek Arya]

Just a fyi, you need to load the repro test.html over http.

Comment 2

[Matthew Gregan [:kinetik]]

Thanks for the report.  Which changeset are you testing against?

Comment 3

[Abhishek Arya]

33c0181c4a25+ 20141019160416 - tip of tree trunk.

Comment 4

[Matthew Gregan [:kinetik]]

Not sure what's going on here yet.  It's strange,

    #6 0x7f045e4e8875 in mozilla::webm_read(void*, unsigned long, void*)
    #7 0x7f046047b1f6 in ne_peek_element /build/firefox/src/media/libnestegg/src/nestegg.c:551:10

ne_peek_element calls webm_read only indirectly via ne_read_id and ne_read_vint, both of which simply call ne_bare_read_vint with either MASK_NONE or MASK_FIRST_BIT.  ne_bare_read_vint calls webm_read via ne_io_read, reading a single byte at a time into an unsigned char.  The size of the read is not data controlled.

Further down the callstack: breaking in FileBlockCache::ReadFromFile when aBytesToRead > 1 only shows larger reads originating from sites I'd expect (such as reading frames into heap allocated buffers).

This is on a 32-bit Windows debug build; I'll see if I can make more progress with a 64-bit ASAN build next.

Comment 5

[Matthew Gregan [:kinetik]]

Using the instructions at https://developer.mozilla.org/en-US/docs/Mozilla/Testing/Firefox_and_Address_Sanitizer#Manual_Build, I can't reproduce this with an x86_64 ASAN build on Linux using mozilla-inbound (63f4c2e0a51a) and a current build of clang from upstream (r220427).  Tested both debug/-O1 and opt builds, and loaded the test over both a local HTTP server and a remote one.

So, without being able to find the bug from the provided stacks and unable to reproduce the ASAN report locally, I'm a bit stuck now.  Any ideas?  Do I need to set something special in ASAN_OPTIONS?

Comment 6

[Matthew Gregan [:kinetik]]

Scanning through nestegg.c for cases where ne_io_read is *not* reading into a heap-allocated buffer, the only cases that exist are:

 - ne_io_read_skip: unsigned char buf[8192], read size limited to min(length, sizeof(buf))
 - ne_bare_read_vint and ne_read_uint: read 1 byte into unsigned char, not user controlled

Unless I've missed anything, everything else reads into heap-allocated buffers where the read size is based on the allocation size.

Comment 7

[Abhishek Arya]

(In reply to Matthew Gregan [:kinetik] from comment #5)
> Using the instructions at
> https://developer.mozilla.org/en-US/docs/Mozilla/Testing/
> Firefox_and_Address_Sanitizer#Manual_Build, I can't reproduce this with an
> x86_64 ASAN build on Linux using mozilla-inbound (63f4c2e0a51a) and a
> current build of clang from upstream (r220427).  Tested both debug/-O1 and
> opt builds, and loaded the test over both a local HTTP server and a remote
> one.
> 
> So, without being able to find the bug from the provided stacks and unable
> to reproduce the ASAN report locally, I'm a bit stuck now.  Any ideas?  Do I
> need to set something special in ASAN_OPTIONS?

- Just retested everything on moz-central tip-of-tree with your clang revision r220427.
- Easily reproduces in like 5-10 secs.
- Reproduces only in -O2 release, does not reproduce on -O1.
- Without ASAN_OPTIONS, it does crash with stack-buffer-overflow but no stacktrace since default slow unwinder fails. to get full stack, use ASAN_OPTIONS=fast_unwind_on_fatal=1

=================================================================
==29175==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fb816dad141 at pc 0x000000450e23 bp 0x7fb816dacbb0 sp 0x7fb816dac370
WRITE of size 8200 at 0x7fb816dad141 thread T64 (Media Decode #2)
    #0 0x450e22 in __interceptor_read.part.40 /build/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:352
    #1 0x7fb853334f87 in pt_Read /build/firefox/src/nsprpub/pr/src/pthreads/ptio.c:1287:13
    #2 0x7fb845e9623f in (anonymous namespace)::interposedRead(PRFileDesc*, void*, int) /build/firefox/src/xpcom/build/NSPRInterposer.cpp:58:10
    #3 0x7fb84a317a2c in mozilla::FileBlockCache::Read(long, unsigned char*, int, int*) /build/firefox/src/content/media/FileBlockCache.cpp:146:16
    #4 0x7fb84a3319c1 in mozilla::MediaCacheStream::Read(char*, unsigned int, unsigned int*) /build/firefox/src/content/media/MediaCache.cpp:672:10
    #5 0x7fb84a381d3d in mozilla::ChannelMediaResource::Read(char*, unsigned int, unsigned int*)
    #6 0x7fb84a548ff5 in mozilla::webm_read(void*, unsigned long, void*)
    #7 0x7fb84c504eae in ne_bare_read_vint /build/firefox/src/media/libnestegg/src/nestegg.c:551:10
    #8 0xf7141120c503d892 (<unknown module>)

Address 0x7fb816dad141 is located in stack of thread T64 (Media Decode #2) at offset 33 in frame
    #0 0x7fb84c504ddf in ne_bare_read_vint /build/firefox/src/media/libnestegg/src/nestegg.c:586

  This frame has 1 object(s):
    [32, 33) 'b' <== Memory access at offset 33 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
Thread T64 (Media Decode #2) created by T63 (Media Decode #1) here:
    #0 0x43624e in __interceptor_pthread_create _asan_rtl_
    #1 0x7fb853335ff0 in _PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:453:14
    #2 0x7fb853335c1a in PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:544:12
    #3 0x7fb845e5f435 in nsThread::Init() /build/firefox/src/xpcom/threads/nsThread.cpp:455:19
    #4 0x7fb845e64a74 in nsThreadManager::NewThread(unsigned int, unsigned int, nsIThread**) /build/firefox/src/xpcom/threads/nsThreadManager.cpp:269:17
    #5 0x7fb845e65ebc in nsThreadPool::PutEvent(nsIRunnable*) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:101:3
    #6 0x7fb845e676e9 in nsThreadPool::Dispatch(nsIRunnable*, unsigned int) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:261:5
    #7 0x7fb84a3a640f in mozilla::MediaTaskQueue::Runner::Run() /build/firefox/src/content/media/MediaTaskQueue.cpp:225:19
    #8 0x7fb845e66d47 in nsThreadPool::Run() /build/firefox/src/xpcom/threads/nsThreadPool.cpp:220:7
    #9 0x7fb845e6729c in non-virtual thunk to nsThreadPool::Run()
    #10 0x7fb845e613af in nsThread::ProcessNextEvent(bool, bool*) /build/firefox/src/xpcom/threads/nsThread.cpp:830:7
    #11 0x7fb845eb7266 in NS_ProcessNextEvent(nsIThread*, bool) /build/firefox/src/xpcom/glue/nsThreadUtils.cpp:265:10
    #12 0x7fb8466ef73e in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /build/firefox/src/ipc/glue/MessagePump.cpp:368:5
    #13 0x7fb8466a2c21 in MessageLoop::Run() /build/firefox/src/ipc/chromium/src/base/message_loop.cc:233:3
    #14 0x7fb845e5e0d6 in nsThread::ThreadFunc(void*) /build/firefox/src/xpcom/threads/nsThread.cpp:350:5
    #15 0x7fb853339490 in _pt_root /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:212:5
    #16 0x7fb856ad4181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312

Thread T63 (Media Decode #1) created by T62 (Media S~hine #1) here:
    #0 0x43624e in __interceptor_pthread_create _asan_rtl_
    #1 0x7fb853335ff0 in _PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:453:14
    #2 0x7fb853335c1a in PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:544:12
    #3 0x7fb845e5f435 in nsThread::Init() /build/firefox/src/xpcom/threads/nsThread.cpp:455:19
    #4 0x7fb845e64a74 in nsThreadManager::NewThread(unsigned int, unsigned int, nsIThread**) /build/firefox/src/xpcom/threads/nsThreadManager.cpp:269:17
    #5 0x7fb845e65ebc in nsThreadPool::PutEvent(nsIRunnable*) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:101:3
    #6 0x7fb845e676e9 in nsThreadPool::Dispatch(nsIRunnable*, unsigned int) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:261:5
    #7 0x7fb84a3a4f99 in mozilla::MediaTaskQueue::DispatchLocked(mozilla::TemporaryRef<nsIRunnable>, mozilla::MediaTaskQueue::DispatchMode) /build/firefox/src/content/media/MediaTaskQueue.cpp:53:17
    #8 0x7fb84a369ea8 in mozilla::MediaDecoderStateMachine::EnqueueDecodeMetadataTask() /build/firefox/src/content/media/MediaTaskQueue.cpp:34:10
    #9 0x7fb84a36e72a in mozilla::MediaDecoderStateMachine::RunStateMachine() /build/firefox/src/content/media/MediaDecoderStateMachine.cpp:2376:14
    #10 0x7fb84a3c2bf6 in non-virtual thunk to (anonymous namespace)::TimerEvent::Run() /build/firefox/src/content/media/MediaDecoderStateMachineScheduler.cpp:160:10
    #11 0x7fb845e66d47 in nsThreadPool::Run() /build/firefox/src/xpcom/threads/nsThreadPool.cpp:220:7
    #12 0x7fb845e6729c in non-virtual thunk to nsThreadPool::Run()
    #13 0x7fb845e613af in nsThread::ProcessNextEvent(bool, bool*) /build/firefox/src/xpcom/threads/nsThread.cpp:830:7
    #14 0x7fb845eb7266 in NS_ProcessNextEvent(nsIThread*, bool) /build/firefox/src/xpcom/glue/nsThreadUtils.cpp:265:10
    #15 0x7fb8466ef5bb in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /build/firefox/src/ipc/glue/MessagePump.cpp:339:20
    #16 0x7fb8466a2c21 in MessageLoop::Run() /build/firefox/src/ipc/chromium/src/base/message_loop.cc:233:3
    #17 0x7fb845e5e0d6 in nsThread::ThreadFunc(void*) /build/firefox/src/xpcom/threads/nsThread.cpp:350:5
    #18 0x7fb853339490 in _pt_root /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:212:5
    #19 0x7fb856ad4181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312

Thread T62 (Media S~hine #1) created by T0 here:
    #0 0x43624e in __interceptor_pthread_create _asan_rtl_
    #1 0x7fb853335ff0 in _PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:453:14
    #2 0x7fb853335c1a in PR_CreateThread /build/firefox/src/nsprpub/pr/src/pthreads/ptthread.c:544:12
    #3 0x7fb845e5f435 in nsThread::Init() /build/firefox/src/xpcom/threads/nsThread.cpp:455:19
    #4 0x7fb845e64a74 in nsThreadManager::NewThread(unsigned int, unsigned int, nsIThread**) /build/firefox/src/xpcom/threads/nsThreadManager.cpp:269:17
    #5 0x7fb845e65ebc in nsThreadPool::PutEvent(nsIRunnable*) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:101:3
    #6 0x7fb845e676e9 in nsThreadPool::Dispatch(nsIRunnable*, unsigned int) /build/firefox/src/xpcom/threads/nsThreadPool.cpp:261:5
    #7 0x7fb84a373d49 in mozilla::MediaDecoderStateMachineScheduler::Schedule(long) /build/firefox/src/content/media/MediaDecoderStateMachineScheduler.cpp:129:10
    #8 0x7fb84a3411be in mozilla::MediaDecoder::InitializeStateMachine(mozilla::MediaDecoder*) /build/firefox/src/content/media/MediaDecoderStateMachine.cpp:3026:10
    #9 0x7fb84a1a9a0d in mozilla::dom::HTMLMediaElement::FinishDecoderSetup(mozilla::MediaDecoder*, mozilla::MediaResource*, nsIStreamListener**, mozilla::MediaDecoder*) /build/firefox/src/content/html/content/src/HTMLMediaElement.cpp:2659:17
    #10 0x7fb84a1985b8 in mozilla::dom::HTMLMediaElement::InitializeDecoderForChannel(nsIChannel*, nsIStreamListener**) /build/firefox/src/content/html/content/src/HTMLMediaElement.cpp:2607:12
    #11 0x7fb84a1972b7 in mozilla::dom::HTMLMediaElement::MediaLoadListener::OnStartRequest(nsIRequest*, nsISupports*) /build/firefox/src/content/html/content/src/HTMLMediaElement.cpp:349:7
    #12 0x7fb8463c53c0 in mozilla::net::nsHttpChannel::CallOnStartRequest() /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:920:14
    #13 0x7fb8463ebd84 in mozilla::net::nsHttpChannel::ContinueOnStartRequest2(tag_nsresult) /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:5291:12
    #14 0x7fb8463eb8e3 in mozilla::net::nsHttpChannel::OnStartRequest(nsIRequest*, nsISupports*) /build/firefox/src/netwerk/protocol/http/nsHttpChannel.cpp:5255:12
    #15 0x7fb845ff1736 in nsInputStreamPump::OnStateStart() /build/firefox/src/netwerk/base/src/nsInputStreamPump.cpp:531:14
    #16 0x7fb845ff0c40 in nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) /build/firefox/src/netwerk/base/src/nsInputStreamPump.cpp:433:25
    #17 0x7fb845e292f9 in nsInputStreamReadyEvent::Run() /build/firefox/src/xpcom/io/nsStreamUtils.cpp:88:9
    #18 0x7fb845e613af in nsThread::ProcessNextEvent(bool, bool*) /build/firefox/src/xpcom/threads/nsThread.cpp:830:7
    #19 0x7fb845eb7266 in NS_ProcessNextEvent(nsIThread*, bool) /build/firefox/src/xpcom/glue/nsThreadUtils.cpp:265:10
    #20 0x7fb8466ee7ef in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /build/firefox/src/ipc/glue/MessagePump.cpp:99:21
    #21 0x7fb8466a2c21 in MessageLoop::Run() /build/firefox/src/ipc/chromium/src/base/message_loop.cc:233:3
    #22 0x7fb849c282df in nsBaseAppShell::Run()
    #23 0x7fb84c106d61 in nsAppStartup::Run() /build/firefox/src/toolkit/components/startup/nsAppStartup.cpp:280:19
    #24 0x7fb84c1e903f in XREMain::XRE_mainRun() /build/firefox/src/toolkit/xre/nsAppRunner.cpp:4095:10
    #25 0x7fb84c1ea037 in XREMain::XRE_main(int, char**, nsXREAppData const*) /build/firefox/src/toolkit/xre/nsAppRunner.cpp:4168:8
    #26 0x7fb84c1eae76 in XRE_main /build/firefox/src/toolkit/xre/nsAppRunner.cpp:4384:16
    #27 0x4bc1e2 in main /build/firefox/src/browser/app/nsBrowserApp.cpp:287:12
    #28 0x7fb855afbec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287

SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 ??
Shadow bytes around the buggy address:
  0x0ff782dad9d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff782dad9e0: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 04 f3 f3 f3
  0x0ff782dad9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff782dada00: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 04 f3 f3 f3
  0x0ff782dada10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0ff782dada20: 00 00 00 00 f1 f1 f1 f1[01]f3 f3 f3 00 00 00 00
  0x0ff782dada30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff782dada40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff782dada50: f1 f1 f1 f1 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff782dada60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff782dada70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==29175==ABORTING

Comment 8

[Matthew Gregan [:kinetik]]

Thanks, I can reproduce it locally now.

I changed by opt build from  --enable-optimize="-O2 -gline-tables-only" (from the wiki page) to just --enable-optimize, set ASAN_OPTIONS=fast_unwind_on_fatal=1, and can hit it fairly quickly.  It may be a coincidence, but it seems to occur much faster if I switch to an empty tab, so perhaps I wasn't waiting long enough in my previous attempts (although I left it running for 1-2 minutes).

There's two things going wrong:

1. The ChannelMediaResource's mCacheStream has an mStreamOffset of -9223372036854194184, which is bogus and should not have been permitted.  I haven't tracked down why this happens yet.

2. In http://mxr.mozilla.org/mozilla-central/source/content/media/MediaCache.cpp#2210

    mStreamOffset = -9223372036854194184
    streamWithPartialBlock->mOffset = 601059
    size = 1

resulting in the "int32_t bytes" calculation at line 2220 producing 19435.  This code should not be trying to store values that exceed the range of an int32_t in bytes, nor should bytes ever exceed the requested read size.

Comment 9

[Matthew Gregan [:kinetik]]

(In reply to Matthew Gregan [:kinetik] from comment #8)
> 1. The ChannelMediaResource's mCacheStream has an mStreamOffset of
> -9223372036854194184, which is bogus and should not have been permitted.  I
> haven't tracked down why this happens yet.

nestegg_track_seek is called for track 0, tstamp 67133000000.  The corresponding Cue has a CueClusterPosition of 9223372036855357377 (segment offset is 55, so final value is 9223372036855357432).  The final value is passed to nestegg_offset_seek, which is then passed to ne_io_seek, which takes an int64_t offset (the value had previously been stored in a uint64_t); there's no check that the uint64_t offset will fit in an int64_t's range, so after conversion the seek offset is -9223372036854194184, which is eventually set as the mStreamOffset.

Comment 10

[Matthew Gregan [:kinetik]]

Attachment: bug1085175_v0.patch

Requesting review from roc, since most of the changes are in the MediaCache.

nestegg:
- return an error when attempting to seek to an offset exceeding the range of an int64_t

MediaCache:
- return an error if the newly computed seek offset is negative
- catch overflow when converting bytes to memcpy in a couple of places
- assert that mStreamOffset and mChannelOffset can't become negative in a couple of places

Comment 11

[Matthew Gregan [:kinetik]]

Comment on attachment 8510786 [details] [diff] [review]
bug1085175_v0.patch

[Security approval request comment]
How easily could an exploit be constructed based on the patch?

It looks pretty easy; attacker controlled stack overwrite with data under attacker's control.

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?

Fairly obvious, most of the patch is adding overflow checks.

Which older supported branches are affected by this flaw?

All of them.

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be?

The patch is small, and neither of these pieces of code have seen massive changes, so backporting will be trivial if the attached patch doesn't apply directly.

How likely is this patch to cause regressions; how much testing does it need?

Fairly unlikely, the behavioural changes are all for cases that would've resulted in some type of bogus behaviour and would now be rejected.

Comment 12

[Daniel Veditz [:dveditz]]

Comment on attachment 8510786 [details] [diff] [review]
bug1085175_v0.patch

I'm concerned that the patch makes it clear we're checking for overflow and how to exploit those are pretty well known. We need to land on all the branches but don't want the check-in too far before the next release which isn't until November 25. Might be best to wait until Nov 10 or 11 to check in.

If you're planning vacation for that time it can be a little earlier; wouldn't want it to be much later because that will barely make the last betas as it is.

sec-approval+
a=dveditz

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/4d3c5b18b800

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Backed out for Windows bustage.
https://hg.mozilla.org/integration/mozilla-inbound/rev/d54ee2df4763

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3737879&repo=mozilla-inbound

Comment 15

[Matthew Gregan [:kinetik]]

Ugh, sorry!  I didn't do a try push due to the nature of the bug.  The fix is simple, building it on Win32 now to verify.

Comment 16

[Matthew Gregan [:kinetik]]

Attachment: bug1085175_v0.1.patch

Trivial build fix: add INT64_MAX to nestegg's stdint.h.

Relanded: https://hg.mozilla.org/integration/mozilla-inbound/rev/992ad9a82996

Comment 17

[Matthew Gregan [:kinetik]]

Attachment: bug1085175_v0.1_b2g30.patch

Patch for b2g30, also applies to esr31, should apply with minimal fuzz to everything else.

Comment 18

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Backed out of inbound in https://hg.mozilla.org/integration/mozilla-inbound/rev/1db845973613 for bustage:

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3739458&repo=mozilla-inbound

Comment 19

[Lawrence Mandel [:lmandel] (use needinfo)]

Given that this has been backed out for a second time and that we're very late in Beta, can we sit on this until 35 or do we really need to get this into 34?

Comment 20

[Matthew Gregan [:kinetik]]

Strange, my local Win32 build worked fine.

Relanded with another trivial fix: https://hg.mozilla.org/integration/mozilla-inbound/rev/d829a27e089a

Comment 21

[Matthew Gregan [:kinetik]]

Attachment: bug1085175_v0.2.patch

As landed.

Comment 22

[Matthew Gregan [:kinetik]]

Attachment: bug1085175_v0.2_b2g30.patch

For b2g30 onwards.  Only change is dom -> content in paths.

Comment 23

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d829a27e089a

Comment 24

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/766470b981ee
https://hg.mozilla.org/releases/mozilla-beta/rev/9cd882996cbe
https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/d339f17a4ac6
https://hg.mozilla.org/releases/mozilla-esr31/rev/eb11e249aad3
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/fe54a1b68337

Comment 25

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/9cd882996cbe
https://hg.mozilla.org/releases/mozilla-b2g32_v2_0m/rev/d339f17a4ac6

Comment 26

[Matt Wobensmith [:mwobensmith][:matt:]]

Not able to reproduce crash with normal ASan builds of Fx34 and Fx35 from around the report date. Appears to require custom build parameters based on conversation above. Abhishek, before I dive in, are you able to verify that this has been fixed on your end? Any branch is fine.

Comment 27

[Abhishek Arya]

(In reply to Matt Wobensmith from comment #26)
> Not able to reproduce crash with normal ASan builds of Fx34 and Fx35 from
> around the report date. Appears to require custom build parameters based on
> conversation above. Abhishek, before I dive in, are you able to verify that
> this has been fixed on your end? Any branch is fine.

Checked. It does not reproduce anymore on moz-central trunk.

Comment 28

[Kamil Jozwiak [:kjozwiak]]

I went through the instructions listed under: https://developer.mozilla.org/en-US/docs/Mozilla/Testing/Firefox_and_Address_Sanitizer#Manual_Build

- added "ASAN_OPTIONS=fast_unwind_on_fatal=1" & "ac_add_options --enable-optimize" into .mozconfig (as per comment #3)
- used changeset 33c0181c4a25 as per comment #8x
- as per comment #1, uploaded the test case to a local http server/remote server

Tried going through it several times but couldn't reproduce the original issue either.. I was using llvm r200213 (as per the MDN document mentioned above)

Thanks for checking Abhishek, much appreciated!

Comment 29

[(no longer active)]

Matthew, <https://github.com/kinetiknz/nestegg> needs to updated with this fix as well...

Comment 30

[Matthew Gregan [:kinetik]]

(In reply to :Ehsan Akhgari [Away: 1/29-2/20] (not reading bugmail, needinfo? me!) from comment #29)
> Matthew, <https://github.com/kinetiknz/nestegg> needs to updated with this
> fix as well...

Thanks for the reminder.  I sync patches between nestegg (and cubeb) back and forth between trees on a lazy basis.  In this case, the fix in nestegg is defense-in-depth, but the real bug was in Gecko's media cache.  Anyway, I've pushed the fix now: https://github.com/kinetiknz/nestegg/commit/17ae8cd3cb5d87c2ebdcd48cae4910f5dd86ae93