1164739 - IPC channel listener use-after-free on shutdown when using OpenH264 with WebRTC

Status: RESOLVED DUPLICATE of bug 1169129

(Core :: IPC, defect, P1)

Description

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

This is with a Linux ASan debug build, using https://mozilla.github.io/webrtc-landing/pc_test.html with the H.264 checkbox checked, in e10s mode.  When quitting the browser while the video is happening, the GMP process's IPC channel does a use-after-free of the listener, thusly (I've translated the stacks a little; these are the libxul.so addresses only, run through `addr2line -iCaf`):

==125614==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040000e1d60 at pc 0x7f7009185cc5 bp 0x7f6fff1fc3b0 sp 0x7f6fff1fc3a8
READ of size 8 at 0x6040000e1d60 thread T2 (Gecko_IOThread)

0x00000000027dccc4
IPC::Channel::ChannelImpl::OnFileCanReadWithoutBlocking(int)
/home/jld/src/gecko-dev/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:879
0x0000000002737b67
event_persist_closure
/home/jld/src/gecko-dev/ipc/chromium/src/third_party/libevent/event.c:1301
event_process_active_single_queue
/home/jld/src/gecko-dev/ipc/chromium/src/third_party/libevent/event.c:1345
event_process_active
/home/jld/src/gecko-dev/ipc/chromium/src/third_party/libevent/event.c:1420
event_base_loop
/home/jld/src/gecko-dev/ipc/chromium/src/third_party/libevent/event.c:1621
0x0000000002792206
base::MessagePumpLibevent::Run(base::MessagePump::Delegate*)
/home/jld/src/gecko-dev/ipc/chromium/src/base/message_pump_libevent.cc:349
0x000000000278c618
MessageLoop::RunInternal()
/home/jld/src/gecko-dev/ipc/chromium/src/base/message_loop.cc:233
0x000000000278c0bd
MessageLoop::RunHandler()
/home/jld/src/gecko-dev/ipc/chromium/src/base/message_loop.cc:226
MessageLoop::Run()
/home/jld/src/gecko-dev/ipc/chromium/src/base/message_loop.cc:200
0x00000000027b6dce
base::Thread::ThreadMain()
/home/jld/src/gecko-dev/ipc/chromium/src/base/thread.cc:170
0x00000000027b7983
ThreadFunc(void*)
/home/jld/src/gecko-dev/ipc/chromium/src/base/platform_thread_posix.cc:39

0x6040000e1d60 is located 16 bytes inside of 48-byte region [0x6040000e1d50,0x6040000e1d80)
freed by thread T0 here:

0x0000000002824c74
mozilla::ipc::MessageChannel::Clear()
/home/jld/src/gecko-dev/ipc/glue/MessageChannel.cpp:404
0x0000000002824292
~MessageChannel
/home/jld/src/gecko-dev/ipc/glue/MessageChannel.cpp:356
0x000000000297a2b5
~PGMPParent
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/ipc/ipdl/./PGMPParent.cpp:84
0x00000000067ba1e5
~GMPParent
/home/jld/src/gecko-dev/dom/media/gmp/GMPParent.cpp:74
0x00000000067e4660
mozilla::layers::DeleteOnMainThreadTask<mozilla::gmp::GMPParent>::Run()
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/dom/media/gmp/../../../dist/include/ThreadSafeRefcountingWithMainThreadDestruction.h:47
0x0000000001e5766d
nsThread::ProcessNextEvent(bool, bool*)
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:866
0x0000000001ecaf50
NS_ProcessNextEvent(nsIThread*, bool)
/home/jld/src/gecko-dev/xpcom/glue/nsThreadUtils.cpp:265
0x0000000001e564e2
nsThread::Shutdown()
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:665
0x000000000357de39
mozilla::storage::Connection::shutdownAsyncThread(nsIThread*)
/home/jld/src/gecko-dev/storage/src/mozStorageConnection.cpp:885
0x0000000003588269
void nsRunnableMethodArguments<nsCOMPtr<nsIThread> >::apply<mozilla::storage::Connection, void (mozilla::storage::Connection::*)(nsIThread*)>(mozilla::storage::Connection*, void (mozilla::storage::Connection::*)(nsIThread*))
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/storage/src/../../dist/include/nsThreadUtils.h:615
0x0000000003588010
nsRunnableMethodImpl<void (mozilla::storage::Connection::*)(nsIThread*), true, nsCOMPtr<nsIThread> >::Run()
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/storage/src/../../dist/include/nsThreadUtils.h:809
0x0000000001e5766d
nsThread::ProcessNextEvent(bool, bool*)
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:866
0x0000000001ecaf50
NS_ProcessNextEvent(nsIThread*, bool)
/home/jld/src/gecko-dev/xpcom/glue/nsThreadUtils.cpp:265
0x0000000001e564e2
nsThread::Shutdown()
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:665
0x000000000357de39
mozilla::storage::Connection::shutdownAsyncThread(nsIThread*)
/home/jld/src/gecko-dev/storage/src/mozStorageConnection.cpp:885
0x0000000003588269
void nsRunnableMethodArguments<nsCOMPtr<nsIThread> >::apply<mozilla::storage::Connection, void (mozilla::storage::Connection::*)(nsIThread*)>(mozilla::storage::Connection*, void (mozilla::storage::Connection::*)(nsIThread*))
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/storage/src/../../dist/include/nsThreadUtils.h:615
0x0000000003588010
nsRunnableMethodImpl<void (mozilla::storage::Connection::*)(nsIThread*), true, nsCOMPtr<nsIThread> >::Run()
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/storage/src/../../dist/include/nsThreadUtils.h:809
0x0000000001e5766d
nsThread::ProcessNextEvent(bool, bool*)
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:866
0x0000000001ecaf50
NS_ProcessNextEvent(nsIThread*, bool)
/home/jld/src/gecko-dev/xpcom/glue/nsThreadUtils.cpp:265
0x0000000001e564e2
nsThread::Shutdown()
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:665
0x000000000357de39
mozilla::storage::Connection::shutdownAsyncThread(nsIThread*)
/home/jld/src/gecko-dev/storage/src/mozStorageConnection.cpp:885
0x0000000003588269
void nsRunnableMethodArguments<nsCOMPtr<nsIThread> >::apply<mozilla::storage::Connection, void (mozilla::storage::Connection::*)(nsIThread*)>(mozilla::storage::Connection*, void (mozilla::storage::Connection::*)(nsIThread*))
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/storage/src/../../dist/include/nsThreadUtils.h:615
0x0000000003588010
nsRunnableMethodImpl<void (mozilla::storage::Connection::*)(nsIThread*), true, nsCOMPtr<nsIThread> >::Run()
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/storage/src/../../dist/include/nsThreadUtils.h:809
0x0000000001e5766d
nsThread::ProcessNextEvent(bool, bool*)
/home/jld/src/gecko-dev/xpcom/threads/nsThread.cpp:866
0x0000000001ecaf50
NS_ProcessNextEvent(nsIThread*, bool)
/home/jld/src/gecko-dev/xpcom/glue/nsThreadUtils.cpp:265
0x00000000067cd5e2
mozilla::gmp::GeckoMediaPluginServiceParent::Observe(nsISupports*, char const*, char16_t const*)
/home/jld/src/gecko-dev/dom/media/gmp/GMPServiceParent.cpp:266
0x0000000001d8f445
nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*)
/home/jld/src/gecko-dev/xpcom/ds/nsObserverList.cpp:113
0x0000000001d91f7c
nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*)
/home/jld/src/gecko-dev/xpcom/ds/nsObserverService.cpp:330
0x00000000090e1c48
nsXREDirProvider::DoShutdown()
/home/jld/src/gecko-dev/toolkit/xre/nsXREDirProvider.cpp:891

previously allocated by thread T25 (GMPThread) here:

0x00000000028253cf
operator new(unsigned long)
/home/jld/src/obj.gecko-dev/obj-x86_64-unknown-linux-gnu/ipc/glue/../../dist/include/mozilla/mozalloc.h:186
mozilla::ipc::MessageChannel::Open(IPC::Channel*, MessageLoop*, mozilla::ipc::Side)
/home/jld/src/gecko-dev/ipc/glue/MessageChannel.cpp:432
0x00000000067bd290
mozilla::gmp::GMPParent::LoadProcess()
/home/jld/src/gecko-dev/dom/media/gmp/GMPParent.cpp:151
[...]

Comment 1

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Also important to mention: this occurs after the plugin process crashes with a different use-after-free, which I haven't filed yet.

Comment 2

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Possibly interesting facts: I just reproduced this bug without also seeing bug 1164740 (or any other child crash that isn't caused by the parent crashing first, it looks like).  Also, I didn't see it in non-e10s mode, so there may be something about the way IPC is used to connect the content child to the GMP child.

Comment 3

[Chris Pearce [:cpearce (Not reading bugmail)]]

Jed: do the changes to GMPParent.cpp and GMPParent.h in this rev fix this bug:
https://hg.mozilla.org/try/rev/27c4e6844d8b

Context:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=27c4e6844d8b

I hit what looked like the above failure running `./mach gtest GeckoMediaPlugins.*` (on Windows) with the patch queue above. I think the problem is that we can drop the last refcounted reference to the GMPParent before the child process has completed shutdown, and when messages come in for the GMPParent we get the UAF.

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Chris Pearce (:cpearce) from comment #3)
> Jed: do the changes to GMPParent.cpp and GMPParent.h in this rev fix this
> bug:
> https://hg.mozilla.org/try/rev/27c4e6844d8b

Specifically, this patch:
https://bug1169129.bugzilla.mozilla.org/attachment.cgi?id=8615089

Comment 5

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Yes, it seems to fix the use-after-free.  On some of the test runs I got a three-way deadlock between the main thread, the decoder thread, and the GMP thread, but that looks like it wouldn't have been directly caused by that patch; I'll file a separate bug.