1557739 - Intermittent SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:963:10 in IPC::Channel::Unsound_IsClosed() const

Status: RESOLVED FIXED

(Core :: IPC, defect, P2)

Description

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=250588815&repo=try&lineNumber=9588

[task 2019-06-07T13:00:48.645Z] 13:00:48     INFO - TEST-OK | /encrypted-media/clearkey-mp4-unique-origin.https.html | took 1548ms
[task 2019-06-07T13:00:48.645Z] 13:00:48     INFO - TEST-START | /encrypted-media/clearkey-mp4-update-disallowed-input.https.html
[task 2019-06-07T13:00:48.653Z] 13:00:48     INFO - Closing window 81604378625
[task 2019-06-07T13:00:49.800Z] 13:00:49     INFO - PID 27888 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[task 2019-06-07T13:00:49.801Z] 13:00:49     INFO - PID 27888 | =================================================================
[task 2019-06-07T13:00:49.801Z] 13:00:49    ERROR - PID 27888 | ==29886==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000171170 at pc 0x7f4f74f221dd bp 0x7ffeb00c15d0 sp 0x7ffeb00c15c8
[task 2019-06-07T13:00:49.802Z] 13:00:49     INFO - PID 27888 | READ of size 8 at 0x602000171170 thread T0 (Web Content)
[task 2019-06-07T13:00:50.409Z] 13:00:50     INFO - PID 27888 |     #0 0x7f4f74f221dc in IPC::Channel::Unsound_IsClosed() const /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:963:10
[task 2019-06-07T13:00:50.418Z] 13:00:50     INFO - PID 27888 |     #1 0x7f4f74f9cea2 in Unsound_IsClosed /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/ipc/MessageChannel.h:305:27
[task 2019-06-07T13:00:50.418Z] 13:00:50     INFO - PID 27888 |     #2 0x7f4f74f9cea2 in mozilla::ipc::MessageChannel::Clear() /builds/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:733
[task 2019-06-07T13:00:50.418Z] 13:00:50     INFO - PID 27888 |     #3 0x7f4f74f9bd18 in mozilla::ipc::MessageChannel::~MessageChannel() /builds/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:628:3
[task 2019-06-07T13:00:50.418Z] 13:00:50     INFO - PID 27888 |     #4 0x7f4f74fc3958 in mozilla::ipc::IToplevelProtocol::~IToplevelProtocol() /builds/worker/workspace/build/src/ipc/glue/ProtocolUtils.cpp:680:1
[task 2019-06-07T13:00:50.442Z] 13:00:50     INFO - PID 27888 |     #5 0x7f4f7a8c73bd in mozilla::gmp::GMPContentParent::~GMPContentParent() /builds/worker/workspace/build/src/dom/media/gmp/GMPContentParent.cpp:35:39
[task 2019-06-07T13:00:50.450Z] 13:00:50     INFO - PID 27888 |     #6 0x7f4f73e3f79c in operator() /builds/worker/workspace/build/src/xpcom/ds/PLDHashTable.cpp:304:7
[task 2019-06-07T13:00:50.450Z] 13:00:50     INFO - PID 27888 |     #7 0x7f4f73e3f79c in ForEachSlot<(lambda at /builds/worker/workspace/build/src/xpcom/ds/PLDHashTable.cpp:302:51)> /builds/worker/workspace/build/src/obj-firefox/dist/include/PLDHashTable.h:359
[task 2019-06-07T13:00:50.451Z] 13:00:50     INFO - PID 27888 |     #8 0x7f4f73e3f79c in ForEachSlot<(lambda at /builds/worker/workspace/build/src/xpcom/ds/PLDHashTable.cpp:302:51)> /builds/worker/workspace/build/src/obj-firefox/dist/include/PLDHashTable.h:349
[task 2019-06-07T13:00:50.451Z] 13:00:50     INFO - PID 27888 |     #9 0x7f4f73e3f79c in PLDHashTable::~PLDHashTable() /builds/worker/workspace/build/src/xpcom/ds/PLDHashTable.cpp:302
[task 2019-06-07T13:00:50.473Z] 13:00:50     INFO - PID 27888 |     #10 0x7f4f7a911c5e in ~nsTHashtable /builds/worker/workspace/build/src/obj-firefox/dist/include/nsTHashtable.h:384:43
[task 2019-06-07T13:00:50.473Z] 13:00:50     INFO - PID 27888 |     #11 0x7f4f7a911c5e in ~GMPServiceChild /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:430
[task 2019-06-07T13:00:50.473Z] 13:00:50     INFO - PID 27888 |     #12 0x7f4f7a911c5e in mozilla::gmp::GMPServiceChild::~GMPServiceChild() /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:430
[task 2019-06-07T13:00:50.474Z] 13:00:50     INFO - PID 27888 |     #13 0x7f4f7a91058d in operator() /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/UniquePtr.h:486:5
[task 2019-06-07T13:00:50.474Z] 13:00:50     INFO - PID 27888 |     #14 0x7f4f7a91058d in reset /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/UniquePtr.h:323
[task 2019-06-07T13:00:50.475Z] 13:00:50     INFO - PID 27888 |     #15 0x7f4f7a91058d in operator= /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/UniquePtr.h:296
[task 2019-06-07T13:00:50.476Z] 13:00:50     INFO - PID 27888 |     #16 0x7f4f7a91058d in mozilla::gmp::GeckoMediaPluginServiceChild::Observe(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:361
[task 2019-06-07T13:00:50.477Z] 13:00:50     INFO - PID 27888 |     #17 0x7f4f7a91066c in non-virtual thunk to mozilla::gmp::GeckoMediaPluginServiceChild::Observe(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp
[task 2019-06-07T13:00:50.478Z] 13:00:50     INFO - PID 27888 |     #18 0x7f4f73e637c1 in nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverList.cpp:66:19
[task 2019-06-07T13:00:50.480Z] 13:00:50     INFO - PID 27888 |     #19 0x7f4f73e69450 in nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverService.cpp:291:19
[task 2019-06-07T13:00:50.481Z] 13:00:50     INFO - PID 27888 |     #20 0x7f4f73ff0e1f in mozilla::ShutdownXPCOM(nsIServiceManager*) /builds/worker/workspace/build/src/xpcom/build/XPCOMInit.cpp:633:24
[task 2019-06-07T13:00:50.481Z] 13:00:50     INFO - PID 27888 |     #21 0x7f4f7f94f8ec in XRE_TermEmbedding() /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:223:3
[task 2019-06-07T13:00:50.482Z] 13:00:50     INFO - PID 27888 |     #22 0x7f4f74fc5922 in mozilla::ipc::ScopedXREEmbed::Stop() /builds/worker/workspace/build/src/ipc/glue/ScopedXREEmbed.cpp:90:5
[task 2019-06-07T13:00:50.482Z] 13:00:50     INFO - PID 27888 |     #23 0x7f4f7f95049e in XRE_InitChildProcess(int, char**, XREChildData const*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:756:16
[task 2019-06-07T13:00:50.485Z] 13:00:50     INFO - PID 27888 |     #24 0x5602d36a87b2 in content_process_main /builds/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:56:28
[task 2019-06-07T13:00:50.486Z] 13:00:50     INFO - PID 27888 |     #25 0x5602d36a87b2 in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:267
[task 2019-06-07T13:00:50.523Z] 13:00:50     INFO - PID 27888 |     #26 0x7f4f940ef82f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
[task 2019-06-07T13:00:50.524Z] 13:00:50     INFO - PID 27888 |     #27 0x5602d35c9d68 in _start (/builds/worker/workspace/build/application/firefox/firefox+0x40d68)
[task 2019-06-07T13:00:50.524Z] 13:00:50     INFO - PID 27888 | 0x602000171170 is located 0 bytes inside of 8-byte region [0x602000171170,0x602000171178)
[task 2019-06-07T13:00:50.525Z] 13:00:50     INFO - PID 27888 | freed by thread T2 (Chrome_~dThread) here:
[task 2019-06-07T13:00:50.526Z] 13:00:50     INFO - PID 27888 |     #0 0x5602d36753f2 in free /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:124:3
[task 2019-06-07T13:00:50.527Z] 13:00:50     INFO - PID 27888 |     #1 0x7f4f74fd1a71 in operator delete /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/mozalloc.h:164:10
[task 2019-06-07T13:00:50.527Z] 13:00:50     INFO - PID 27888 |     #2 0x7f4f74fd1a71 in DeleteTask<IPC::Channel>::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/task.h:194
[task 2019-06-07T13:00:50.529Z] 13:00:50     INFO - PID 27888 |     #3 0x7f4f74ef5c75 in RunTask /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:442:9
[task 2019-06-07T13:00:50.531Z] 13:00:50     INFO - PID 27888 |     #4 0x7f4f74ef5c75 in MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask&&) /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:450
[task 2019-06-07T13:00:50.532Z] 13:00:50     INFO - PID 27888 |     #5 0x7f4f74ef6dbb in MessageLoop::DoWork() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:523:13
[task 2019-06-07T13:00:50.534Z] 13:00:50     INFO - PID 27888 |     #6 0x7f4f74efa030 in base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/chromium/src/base/message_pump_libevent.cc:321:31
[task 2019-06-07T13:00:50.535Z] 13:00:50     INFO - PID 27888 |     #7 0x7f4f74ef4762 in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:315:10
[task 2019-06-07T13:00:50.535Z] 13:00:50     INFO - PID 27888 |     #8 0x7f4f74ef4762 in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:308
[task 2019-06-07T13:00:50.536Z] 13:00:50     INFO - PID 27888 |     #9 0x7f4f74ef4762 in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:290
[task 2019-06-07T13:00:50.537Z] 13:00:50     INFO - PID 27888 |     #10 0x7f4f74f1082f in base::Thread::ThreadMain() /builds/worker/workspace/build/src/ipc/chromium/src/base/thread.cc:192:16
[task 2019-06-07T13:00:50.538Z] 13:00:50     INFO - PID 27888 |     #11 0x7f4f74f0545c in ThreadFunc(void*) /builds/worker/workspace/build/src/ipc/chromium/src/base/platform_thread_posix.cc:40:13
[task 2019-06-07T13:00:50.538Z] 13:00:50     INFO - PID 27888 |     #12 0x7f4f9514d6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
[task 2019-06-07T13:00:50.539Z] 13:00:50     INFO - PID 27888 | previously allocated by thread T26 (GMPThread) here:
[task 2019-06-07T13:00:50.541Z] 13:00:50     INFO - PID 27888 |     #0 0x5602d3675773 in __interceptor_malloc /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:146:3
[task 2019-06-07T13:00:50.542Z] 13:00:50     INFO - PID 27888 |     #1 0x5602d36aa42d in moz_xmalloc /builds/worker/workspace/build/src/memory/mozalloc/mozalloc.cpp:52:15
[task 2019-06-07T13:00:50.543Z] 13:00:50     INFO - PID 27888 |     #2 0x7f4f74fd42dd in operator new /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/mozalloc.h:144:10
[task 2019-06-07T13:00:50.544Z] 13:00:50     INFO - PID 27888 |     #3 0x7f4f74fd42dd in MakeUnique<IPC::Channel, const int &, IPC::Channel::Mode &, nullptr_t> /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/UniquePtr.h:617
[task 2019-06-07T13:00:50.544Z] 13:00:50     INFO - PID 27888 |     #4 0x7f4f74fd42dd in mozilla::ipc::OpenDescriptor(mozilla::ipc::TransportDescriptor const&, IPC::Channel::Mode) /builds/worker/workspace/build/src/ipc/glue/Transport_posix.cpp:62
[task 2019-06-07T13:00:50.545Z] 13:00:50     INFO - PID 27888 |     #5 0x7f4f7a8d9716 in mozilla::ipc::Endpoint<mozilla::gmp::PGMPContentParent>::Bind(mozilla::gmp::PGMPContentParent*) /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/ipc/ProtocolUtils.h:781:30
[task 2019-06-07T13:00:50.547Z] 13:00:50     INFO - PID 27888 |     #6 0x7f4f7a911f12 in mozilla::gmp::GMPServiceChild::GetBridgedGMPContentParent(int, mozilla::ipc::Endpoint<mozilla::gmp::PGMPContentParent>&&) /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:445:33
[task 2019-06-07T13:00:50.548Z] 13:00:50     INFO - PID 27888 |     #7 0x7f4f7a956096 in mozilla::gmp::GeckoMediaPluginServiceChild::GetContentParent(mozilla::GMPCrashHelper*, mozilla::gmp::NodeId const&, nsTString<char> const&, nsTArray<nsTString<char> > const&)::$_8::operator()(mozilla::gmp::GMPServiceChild*) const /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:188:50
[task 2019-06-07T13:00:50.552Z] 13:00:50     INFO - PID 27888 |     #8 0x7f4f7a9553b9 in InvokeMethod<(lambda at /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:149:7), void ((lambda at /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:149:7)::*)(mozilla::gmp::GMPServiceChild *) const, mozilla::gmp::GMPServiceChild *> /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/MozPromise.h:503:12
[task 2019-06-07T13:00:50.553Z] 13:00:50     INFO - PID 27888 |     #9 0x7f4f7a9553b9 in InvokeCallbackMethod<false, (lambda at /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:149:7), void ((lambda at /builds/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:149:7)::*)(mozilla::gmp::GMPServiceChild *) const, mozilla::gmp::GMPServiceChild *, RefPtr<mozilla::MozPromise<mozilla::gmp::GMPServiceChild *, mozilla::MediaResult, true>::Private> > /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/MozPromise.h:535
[task 2019-06-07T13:00:50.555Z] 13:00:50     INFO - PID 27888 |     #10 0x7f4f7a9553b9 in mozilla::MozPromise<mozilla::gmp::GMPServiceChild*, mozilla::MediaResult, true>::ThenValue<mozilla::gmp::GeckoMediaPluginServiceChild::GetContentParent(mozilla::GMPCrashHelper*, mozilla::gmp::NodeId const&, nsTString<char> const&, nsTArray<nsTString<char> > const&)::$_8, mozilla::gmp::GeckoMediaPluginServiceChild::GetContentParent(mozilla::GMPCrashHelper*, mozilla::gmp::NodeId const&, nsTString<char> const&, nsTArray<nsTString<char> > const&)::$_9>::DoResolveOrRejectInternal(mozilla::MozPromise<mozilla::gmp::GMPServiceChild*, mozilla::MediaResult, true>::ResolveOrRejectValue&) /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/MozPromise.h:717
[task 2019-06-07T13:00:50.556Z] 13:00:50     INFO - PID 27888 |     #11 0x7f4f7a9541d6 in DoResolveOrReject /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/MozPromise.h:480:7
[task 2019-06-07T13:00:50.557Z] 13:00:50     INFO - PID 27888 |     #12 0x7f4f7a9541d6 in mozilla::MozPromise<mozilla::gmp::GMPServiceChild*, mozilla::MediaResult, true>::ThenValueBase::ResolveOrRejectRunnable::Run() /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/MozPromise.h:393
[task 2019-06-07T13:00:50.558Z] 13:00:50     INFO - PID 27888 |     #13 0x7f4f73f7c2b2 in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:113:25
[task 2019-06-07T13:00:50.559Z] 13:00:50     INFO - PID 27888 |     #14 0x7f4f73f9fe6b in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1176:14
[task 2019-06-07T13:00:50.560Z] 13:00:50     INFO - PID 27888 |     #15 0x7f4f73fa5b98 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
[task 2019-06-07T13:00:50.561Z] 13:00:50     INFO - PID 27888 |     #16 0x7f4f74fb923a in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:303:20
[task 2019-06-07T13:00:50.562Z] 13:00:50     INFO - PID 27888 |     #17 0x7f4f74ef4762 in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:315:10
[task 2019-06-07T13:00:50.563Z] 13:00:50     INFO - PID 27888 |     #18 0x7f4f74ef4762 in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:308
[task 2019-06-07T13:00:50.564Z] 13:00:50     INFO - PID 27888 |     #19 0x7f4f74ef4762 in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:290
[task 2019-06-07T13:00:50.567Z] 13:00:50     INFO - PID 27888 |     #20 0x7f4f73f9a65a in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:455:11
[task 2019-06-07T13:00:50.568Z] 13:00:50     INFO - PID 27888 |     #21 0x7f4f913d505d in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2019-06-07T13:00:50.568Z] 13:00:50     INFO - PID 27888 |     #22 0x7f4f9514d6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
[task 2019-06-07T13:00:50.569Z] 13:00:50     INFO - PID 27888 | Thread T2 (Chrome_~dThread) created by T0 (Web Content) here:
[task 2019-06-07T13:00:50.583Z] 13:00:50     INFO - PID 27888 |     #0 0x5602d365dd4d in pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:210:3
[task 2019-06-07T13:00:50.584Z] 13:00:50     INFO - PID 27888 |     #1 0x7f4f74f02cac in CreateThread /builds/worker/workspace/build/src/ipc/chromium/src/base/platform_thread_posix.cc:123:14
[task 2019-06-07T13:00:50.586Z] 13:00:50     INFO - PID 27888 |     #2 0x7f4f74f02cac in PlatformThread::Create(unsigned long, PlatformThread::Delegate*, unsigned long*) /builds/worker/workspace/build/src/ipc/chromium/src/base/platform_thread_posix.cc:134
[task 2019-06-07T13:00:50.587Z] 13:00:50     INFO - PID 27888 |     #3 0x7f4f74f0ff53 in base::Thread::StartWithOptions(base::Thread::Options const&) /builds/worker/workspace/build/src/ipc/chromium/src/base/thread.cc:97:8
[task 2019-06-07T13:00:50.588Z] 13:00:50     INFO - PID 27888 |     #4 0x7f4f74fb97b7 in mozilla::ipc::ProcessChild::ProcessChild(int) /builds/worker/workspace/build/src/ipc/glue/ProcessChild.cpp:24:7
[task 2019-06-07T13:00:50.590Z] 13:00:50     INFO - PID 27888 |     #5 0x7f4f7f950341 in ContentProcess /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/ContentProcess.h:29:51
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #6 0x7f4f7f950341 in XRE_InitChildProcess(int, char**, XREChildData const*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:682
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #7 0x5602d36a87b2 in content_process_main /builds/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:56:28
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #8 0x5602d36a87b2 in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:267
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #9 0x7f4f940ef82f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 | Thread T26 (GMPThread) created by T0 (Web Content) here:
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #0 0x5602d365dd4d in pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:210:3
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #1 0x7f4f913c7158 in _PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:433:14
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #2 0x7f4f913b0d3e in PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:518:12
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #3 0x7f4f73f9c807 in nsThread::Init(nsTSubstring<char> const&) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:662:8
[task 2019-06-07T13:00:50.591Z] 13:00:50     INFO - PID 27888 |     #4 0x7f4f73fa4d10 in nsThreadManager::NewNamedThread(nsTSubstring<char> const&, unsigned int, nsIThread**) /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp:415:12
[task 2019-06-07T13:00:50.592Z] 13:00:50     INFO - PID 27888 |     #5 0x7f4f73fa881a in NS_NewNamedThread(nsTSubstring<char> const&, nsIThread**, nsIRunnable*, unsigned int) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:139:57
[task 2019-06-07T13:00:50.592Z] 13:00:50     INFO - PID 27888 |     #6 0x7f4f7a9077bc in NS_NewNamedThread<10> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:71:10
[task 2019-06-07T13:00:50.593Z] 13:00:50     INFO - PID 27888 |     #7 0x7f4f7a9077bc in mozilla::gmp::GeckoMediaPluginService::GetThread(nsIThread**) /builds/worker/workspace/build/src/dom/media/gmp/GMPService.cpp:317
[task 2019-06-07T13:00:50.593Z] 13:00:50     INFO - PID 27888 |     #8 0x7f4f7a905def in mozilla::gmp::GeckoMediaPluginService::Init() /builds/worker/workspace/build/src/dom/media/gmp/GMPService.cpp:217:10
[task 2019-06-07T13:00:50.593Z] 13:00:50     INFO - PID 27888 |     #9 0x7f4f7a94729b in mozilla::gmp::GMPServiceCreateHelper::GetOrCreateOnMainThread() /builds/worker/workspace/build/src/dom/media/gmp/GMPService.cpp:108:18
[task 2019-06-07T13:00:50.595Z] 13:00:50     INFO - PID 27888 |     #10 0x7f4f7a9042b7 in mozilla::gmp::GMPServiceCreateHelper::GetOrCreate() /builds/worker/workspace/build/src/dom/media/gmp/GMPService.cpp:76:17
[task 2019-06-07T13:00:50.596Z] 13:00:50     INFO - PID 27888 |     #11 0x7f4f73f24bf2 in mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/obj-firefox/xpcom/components/StaticComponents.cpp:9213:60
[task 2019-06-07T13:00:50.597Z] 13:00:50     INFO - PID 27888 |     #12 0x7f4f73f5a8c1 in CreateInstance /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:224:46
[task 2019-06-07T13:00:50.597Z] 13:00:50     INFO - PID 27888 |     #13 0x7f4f73f5a8c1 in nsComponentManagerImpl::GetServiceLocked((anonymous namespace)::MutexLock&, (anonymous namespace)::EntryWrapper&, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1384
[task 2019-06-07T13:00:50.598Z] 13:00:50     INFO - PID 27888 |     #14 0x7f4f73f4f869 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1571:10
[task 2019-06-07T13:00:50.600Z] 13:00:50     INFO - PID 27888 |     #15 0x7f4f73f632a9 in CallGetService /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:61:43
[task 2019-06-07T13:00:50.600Z] 13:00:50     INFO - PID 27888 |     #16 0x7f4f73f632a9 in nsGetServiceByContractID::operator()(nsID const&, void**) const /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:243
[task 2019-06-07T13:00:50.601Z] 13:00:50     INFO - PID 27888 |     #17 0x7f4f73ddc840 in nsCOMPtr_base::assign_from_gs_contractid(nsGetServiceByContractID, nsID const&) /builds/worker/workspace/build/src/xpcom/base/nsCOMPtr.cpp:82:7
[task 2019-06-07T13:00:50.602Z] 13:00:50     INFO - PID 27888 |     #18 0x7f4f7a93b6e5 in nsCOMPtr /builds/worker/workspace/build/src/obj-firefox/dist/include/nsCOMPtr.h:607:5
[task 2019-06-07T13:00:50.603Z] 13:00:50     INFO - PID 27888 |     #19 0x7f4f7a93b6e5 in mozilla::HaveGMPFor(nsTString<char> const&, nsTArray<nsTString<char> >&&) /builds/worker/workspace/build/src/dom/media/gmp/GMPUtils.cpp:179
[task 2019-06-07T13:00:50.619Z] 13:00:50     INFO - PID 27888 |     #20 0x7f4f7a831bde in mozilla::dom::HavePluginForKeySystem(nsTString<char> const&) /builds/worker/workspace/build/src/dom/media/eme/MediaKeySystemAccess.cpp:94:21
[task 2019-06-07T13:00:50.620Z] 13:00:50     INFO - PID 27888 |     #21 0x7f4f7a816449 in EnsureCDMInstalled /builds/worker/workspace/build/src/dom/media/eme/MediaKeySystemAccess.cpp:106:8
[task 2019-06-07T13:00:50.620Z] 13:00:50     INFO - PID 27888 |     #22 0x7f4f7a816449 in mozilla::dom::MediaKeySystemAccess::GetKeySystemStatus(nsTSubstring<char16_t> const&, nsTSubstring<char>&) /builds/worker/workspace/build/src/dom/media/eme/MediaKeySystemAccess.cpp:120
[task 2019-06-07T13:00:50.620Z] 13:00:50     INFO - PID 27888 |     #23 0x7f4f7a82557c in mozilla::dom::MediaKeySystemAccessManager::Request(mozilla::dom::DetailedPromise*, nsTSubstring<char16_t> const&, mozilla::dom::Sequence<mozilla::dom::MediaKeySystemConfiguration> const&, mozilla::dom::MediaKeySystemAccessManager::RequestType) /builds/worker/workspace/build/src/dom/media/eme/MediaKeySystemAccessManager.cpp:119:7
[task 2019-06-07T13:00:50.622Z] 13:00:50     INFO - PID 27888 |     #24 0x7f4f773bdcee in mozilla::dom::Navigator::RequestMediaKeySystemAccess(nsTSubstring<char16_t> const&, mozilla::dom::Sequence<mozilla::dom::MediaKeySystemConfiguration> const&, mozilla::ErrorResult&) /builds/worker/workspace/build/src/dom/base/Navigator.cpp:1802:33
[task 2019-06-07T13:00:50.690Z] 13:00:50     INFO - PID 27888 |     #25 0x7f4f77ddc79d in requestMediaKeySystemAccess /builds/worker/workspace/build/src/obj-firefox/dom/bindings/NavigatorBinding.cpp:1859:60
[task 2019-06-07T13:00:50.690Z] 13:00:50     INFO - PID 27888 |     #26 0x7f4f77ddc79d in mozilla::dom::Navigator_Binding::requestMediaKeySystemAccess_promiseWrapper(JSContext*, JS::Handle<JSObject*>, mozilla::dom::Navigator*, JSJitMethodCallArgs const&) /builds/worker/workspace/build/src/obj-firefox/dom/bindings/NavigatorBinding.cpp:1875
[task 2019-06-07T13:00:50.698Z] 13:00:50     INFO - PID 27888 |     #27 0x7f4f798fb028 in bool mozilla::dom::binding_detail::GenericMethod<mozilla::dom::binding_detail::NormalThisPolicy, mozilla::dom::binding_detail::ConvertExceptionsToPromises>(JSContext*, unsigned int, JS::Value*) /builds/worker/workspace/build/src/dom/bindings/BindingUtils.cpp:3171:13
[task 2019-06-07T13:00:50.719Z] 13:00:50     INFO - PID 27888 |     #28 0x7f4f7fbc5594 in CallJSNative /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:448:13
[task 2019-06-07T13:00:50.719Z] 13:00:50     INFO - PID 27888 |     #29 0x7f4f7fbc5594 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:540
[task 2019-06-07T13:00:50.720Z] 13:00:50     INFO - PID 27888 |     #30 0x7f4f7fba6789 in CallFromStack /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:599:10
[task 2019-06-07T13:00:50.720Z] 13:00:50     INFO - PID 27888 |     #31 0x7f4f7fba6789 in Interpret(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:3087
[task 2019-06-07T13:00:50.722Z] 13:00:50     INFO - PID 27888 |     #32 0x7f4f7fb90a57 in js::RunScript(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:425:10
[task 2019-06-07T13:00:50.722Z] 13:00:50     INFO - PID 27888 |     #33 0x7f4f7fbc5e71 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:568:13
[task 2019-06-07T13:00:50.722Z] 13:00:50     INFO - PID 27888 |     #34 0x7f4f7fbc7832 in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:611:8
[task 2019-06-07T13:00:50.763Z] 13:00:50     INFO - PID 27888 |     #35 0x7f4f800b7884 in js::fun_apply(JSContext*, unsigned int, JS::Value*) /builds/worker/workspace/build/src/js/src/vm/JSFunction.cpp:1184:10
[task 2019-06-07T13:00:50.764Z] 13:00:50     INFO - PID 27888 |     #36 0x7f4f7fbc5594 in CallJSNative /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:448:13
[task 2019-06-07T13:00:50.764Z] 13:00:50     INFO - PID 27888 |     #37 0x7f4f7fbc5594 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:540
[task 2019-06-07T13:00:50.764Z] 13:00:50     INFO - PID 27888 |     #38 0x7f4f7fba6789 in CallFromStack /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:599:10
[task 2019-06-07T13:00:50.765Z] 13:00:50     INFO - PID 27888 |     #39 0x7f4f7fba6789 in Interpret(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:3087
[task 2019-06-07T13:00:50.765Z] 13:00:50     INFO - PID 27888 |     #40 0x7f4f7fb90a57 in js::RunScript(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:425:10
[task 2019-06-07T13:00:50.765Z] 13:00:50     INFO - PID 27888 |     #41 0x7f4f7fbc5e71 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:568:13
[task 2019-06-07T13:00:50.766Z] 13:00:50     INFO - PID 27888 |     #42 0x7f4f7fbc7832 in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:611:8
[task 2019-06-07T13:00:50.804Z] 13:00:50     INFO - PID 27888 |     #43 0x7f4f7fcf7139 in js::PromiseObject::create(JSContext*, JS::Handle<JSObject*>, JS::Handle<JSObject*>, bool) /builds/worker/workspace/build/src/js/src/builtin/Promise.cpp:2237:15
[task 2019-06-07T13:00:50.805Z] 13:00:50     INFO - PID 27888 |     #44 0x7f4f7fd3e8d8 in PromiseConstructor(JSContext*, unsigned int, JS::Value*) /builds/worker/workspace/build/src/js/src/builtin/Promise.cpp:2158:7
[task 2019-06-07T13:00:50.805Z] 13:00:50     INFO - PID 27888 |     #45 0x7f4f7fbc874a in CallJSNative /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:448:13
[task 2019-06-07T13:00:50.805Z] 13:00:50     INFO - PID 27888 |     #46 0x7f4f7fbc874a in CallJSNativeConstructor /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:464
[task 2019-06-07T13:00:50.806Z] 13:00:50     INFO - PID 27888 |     #47 0x7f4f7fbc874a in InternalConstruct(JSContext*, js::AnyConstructArgs const&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:638
[task 2019-06-07T13:00:50.806Z] 13:00:50     INFO - PID 27888 |     #48 0x7f4f7fba664d in Interpret(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:3078:16
[task 2019-06-07T13:00:50.807Z] 13:00:50     INFO - PID 27888 |     #49 0x7f4f7fb90a57 in js::RunScript(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:425:10
[task 2019-06-07T13:00:50.807Z] 13:00:50     INFO - PID 27888 |     #50 0x7f4f7fbc5e71 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:568:13
[task 2019-06-07T13:00:50.807Z] 13:00:50     INFO - PID 27888 |     #51 0x7f4f7fbc7832 in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:611:8
[task 2019-06-07T13:00:50.808Z] 13:00:50     INFO - PID 27888 |     #52 0x7f4f7fd496f1 in Call /builds/worker/workspace/build/src/js/src/vm/Interpreter.h:98:10
[task 2019-06-07T13:00:50.808Z] 13:00:50     INFO - PID 27888 |     #53 0x7f4f7fd496f1 in PromiseReactionJob(JSContext*, unsigned int, JS::Value*) /builds/worker/workspace/build/src/js/src/builtin/Promise.cpp:1704
[task 2019-06-07T13:00:50.809Z] 13:00:50     INFO - PID 27888 |     #54 0x7f4f7fbc5594 in CallJSNative /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:448:13
[task 2019-06-07T13:00:50.810Z] 13:00:50     INFO - PID 27888 |     #55 0x7f4f7fbc5594 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:540
[task 2019-06-07T13:00:50.810Z] 13:00:50     INFO - PID 27888 |     #56 0x7f4f7fbc7832 in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:611:8
[task 2019-06-07T13:00:50.831Z] 13:00:50     INFO - PID 27888 |     #57 0x7f4f8079a4a7 in JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) /builds/worker/workspace/build/src/js/src/jsapi.cpp:2667:10
[task 2019-06-07T13:00:50.868Z] 13:00:50     INFO - PID 27888 |     #58 0x7f4f7800a4f6 in mozilla::dom::PromiseJobCallback::Call(JSContext*, JS::Handle<JS::Value>, mozilla::ErrorResult&) /builds/worker/workspace/build/src/obj-firefox/dom/bindings/PromiseBinding.cpp:26:8
[task 2019-06-07T13:00:50.885Z] 13:00:50     INFO - PID 27888 |     #59 0x7f4f73dcee33 in Call /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/PromiseBinding.h:91:12
[task 2019-06-07T13:00:50.885Z] 13:00:50     INFO - PID 27888 |     #60 0x7f4f73dcee33 in Call /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/PromiseBinding.h:104
[task 2019-06-07T13:00:50.886Z] 13:00:50     INFO - PID 27888 |     #61 0x7f4f73dcee33 in mozilla::PromiseJobRunnable::Run(mozilla::AutoSlowOperation&) /builds/worker/workspace/build/src/xpcom/base/CycleCollectedJSContext.cpp:239
[task 2019-06-07T13:00:50.886Z] 13:00:50     INFO - PID 27888 |     #62 0x7f4f73daad70 in mozilla::CycleCollectedJSContext::PerformMicroTaskCheckPoint(bool) /builds/worker/workspace/build/src/xpcom/base/CycleCollectedJSContext.cpp:661:17
[task 2019-06-07T13:00:50.907Z] 13:00:50     INFO - PID 27888 |     #63 0x7f4f7bb0541a in LeaveMicroTask /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/CycleCollectedJSContext.h:213:7
[task 2019-06-07T13:00:50.907Z] 13:00:50     INFO - PID 27888 |     #64 0x7f4f7bb0541a in ~nsAutoMicroTask /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/CycleCollectedJSContext.h:361
[task 2019-06-07T13:00:50.907Z] 13:00:50     INFO - PID 27888 |     #65 0x7f4f7bb0541a in mozilla::dom::ScriptLoader::EvaluateScript(mozilla::dom::ScriptLoadRequest*) /builds/worker/workspace/build/src/dom/script/ScriptLoader.cpp:2799
[task 2019-06-07T13:00:50.908Z] 13:00:50     INFO - PID 27888 |     #66 0x7f4f7bafd8bc in mozilla::dom::ScriptLoader::ProcessRequest(mozilla::dom::ScriptLoadRequest*) /builds/worker/workspace/build/src/dom/script/ScriptLoader.cpp:2275:10
[task 2019-06-07T13:00:50.908Z] 13:00:50     INFO - PID 27888 |     #67 0x7f4f7bafa293 in mozilla::dom::ScriptLoader::ProcessInlineScript(nsIScriptElement*, mozilla::dom::ScriptKind) /builds/worker/workspace/build/src/dom/script/ScriptLoader.cpp:1856:10
[task 2019-06-07T13:00:50.908Z] 13:00:50     INFO - PID 27888 |     #68 0x7f4f7bad6fae in mozilla::dom::ScriptLoader::ProcessScriptElement(nsIScriptElement*) /builds/worker/workspace/build/src/dom/script/ScriptLoader.cpp:1579:10
[task 2019-06-07T13:00:50.912Z] 13:00:50     INFO - PID 27888 |     #69 0x7f4f7bad627c in mozilla::dom::ScriptElement::MaybeProcessScript() /builds/worker/workspace/build/src/dom/script/ScriptElement.cpp:118:18
[task 2019-06-07T13:00:50.951Z] 13:00:50     INFO - PID 27888 |     #70 0x7f4f761fe44a in AttemptToExecute /builds/worker/workspace/build/src/obj-firefox/dist/include/nsIScriptElement.h:224:18
[task 2019-06-07T13:00:50.951Z] 13:00:50     INFO - PID 27888 |     #71 0x7f4f761fe44a in nsHtml5TreeOpExecutor::RunScript(nsIContent*) /builds/worker/workspace/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:725
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #72 0x7f4f761f8224 in nsHtml5TreeOpExecutor::RunFlushLoop() /builds/worker/workspace/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:528:7
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #73 0x7f4f7620582f in nsHtml5ExecutorReflusher::Run() /builds/worker/workspace/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:68:16
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #74 0x7f4f73f72831 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #75 0x7f4f73f9fe6b in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1176:14
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #76 0x7f4f73fa5b98 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #77 0x7f4f74fb822a in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:88:21
[task 2019-06-07T13:00:50.952Z] 13:00:50     INFO - PID 27888 |     #78 0x7f4f74ef4762 in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:315:10
[task 2019-06-07T13:00:50.953Z] 13:00:50     INFO - PID 27888 |     #79 0x7f4f74ef4762 in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:308
[task 2019-06-07T13:00:50.954Z] 13:00:50     INFO - PID 27888 |     #80 0x7f4f74ef4762 in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:290
[task 2019-06-07T13:00:50.957Z] 13:00:50     INFO - PID 27888 |     #81 0x7f4f7be63539 in nsBaseAppShell::Run() /builds/worker/workspace/build/src/widget/nsBaseAppShell.cpp:137:27
[task 2019-06-07T13:00:50.962Z] 13:00:50     INFO - PID 27888 |     #82 0x7f4f7f950bcf in XRE_RunAppShell() /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:917:20
[task 2019-06-07T13:00:50.962Z] 13:00:50     INFO - PID 27888 |     #83 0x7f4f74ef4762 in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:315:10
[task 2019-06-07T13:00:50.962Z] 13:00:50     INFO - PID 27888 |     #84 0x7f4f74ef4762 in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:308
[task 2019-06-07T13:00:50.962Z] 13:00:50     INFO - PID 27888 |     #85 0x7f4f74ef4762 in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:290
[task 2019-06-07T13:00:50.963Z] 13:00:50     INFO - PID 27888 |     #86 0x7f4f7f95046f in XRE_InitChildProcess(int, char**, XREChildData const*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:752:34
[task 2019-06-07T13:00:50.968Z] 13:00:50     INFO - PID 27888 |     #87 0x5602d36a87b2 in content_process_main /builds/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:56:28
[task 2019-06-07T13:00:50.968Z] 13:00:50     INFO - PID 27888 |     #88 0x5602d36a87b2 in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:267
[task 2019-06-07T13:00:50.968Z] 13:00:50     INFO - PID 27888 |     #89 0x7f4f940ef82f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
[task 2019-06-07T13:00:50.969Z] 13:00:50     INFO - PID 27888 | SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:963:10 in IPC::Channel::Unsound_IsClosed() const
[task 2019-06-07T13:00:50.970Z] 13:00:50     INFO - PID 27888 | Shadow bytes around the buggy address:
[task 2019-06-07T13:00:50.970Z] 13:00:50     INFO - PID 27888 |   0x0c04800261d0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa 00 00
[task 2019-06-07T13:00:50.971Z] 13:00:50     INFO - PID 27888 |   0x0c04800261e0: fa fa fd fd fa fa fd fd fa fa 00 00 fa fa fd fd
[task 2019-06-07T13:00:50.971Z] 13:00:50     INFO - PID 27888 |   0x0c04800261f0: fa fa fd fa fa fa fd fa fa fa fd fd fa fa fd fd
[task 2019-06-07T13:00:50.972Z] 13:00:50     INFO - PID 27888 |   0x0c0480026200: fa fa fd fd fa fa 00 02 fa fa 01 fa fa fa fd fa
[task 2019-06-07T13:00:50.972Z] 13:00:50     INFO - PID 27888 |   0x0c0480026210: fa fa fd fd fa fa fd fd fa fa 00 fa fa fa 00 00
[task 2019-06-07T13:00:50.974Z] 13:00:50     INFO - PID 27888 | =>0x0c0480026220: fa fa fd fd fa fa 00 fa fa fa 00 00 fa fa[fd]fa
[task 2019-06-07T13:00:50.975Z] 13:00:50     INFO - PID 27888 |   0x0c0480026230: fa fa fd fd fa fa fd fd fa fa 00 00 fa fa 00 00
[task 2019-06-07T13:00:50.975Z] 13:00:50     INFO - PID 27888 |   0x0c0480026240: fa fa 00 00 fa fa fd fd fa fa fd fa fa fa fd fa
[task 2019-06-07T13:00:50.980Z] 13:00:50     INFO - PID 27888 |   0x0c0480026250: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fa
[task 2019-06-07T13:00:50.980Z] 13:00:50     INFO - PID 27888 |   0x0c0480026260: fa fa fd fd fa fa fd fa fa fa fd fd fa fa fd fd
[task 2019-06-07T13:00:50.981Z] 13:00:50     INFO - PID 27888 |   0x0c0480026270: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
[task 2019-06-07T13:00:50.981Z] 13:00:50     INFO - PID 27888 | Shadow byte legend (one shadow byte represents 8 application bytes):
[task 2019-06-07T13:00:50.981Z] 13:00:50     INFO - PID 27888 |   Addressable:           00
[task 2019-06-07T13:00:50.982Z] 13:00:50     INFO - PID 27888 |   Partially addressable: 01 02 03 04 05 06 07
[task 2019-06-07T13:00:50.982Z] 13:00:50     INFO - PID 27888 |   Heap left redzone:       fa
[task 2019-06-07T13:00:50.983Z] 13:00:50     INFO - PID 27888 |   Freed heap region:       fd
[task 2019-06-07T13:00:50.983Z] 13:00:50     INFO - PID 27888 |   Stack left redzone:      f1
[task 2019-06-07T13:00:50.983Z] 13:00:50     INFO - PID 27888 |   Stack mid redzone:       f2
[task 2019-06-07T13:00:50.984Z] 13:00:50     INFO - PID 27888 |   Stack right redzone:     f3
[task 2019-06-07T13:00:50.984Z] 13:00:50     INFO - PID 27888 |   Stack after return:      f5
[task 2019-06-07T13:00:50.985Z] 13:00:50     INFO - PID 27888 |   Stack use after scope:   f8
[task 2019-06-07T13:00:50.985Z] 13:00:50     INFO - PID 27888 |   Global redzone:          f9
[task 2019-06-07T13:00:50.988Z] 13:00:50     INFO - PID 27888 |   Global init order:       f6
[task 2019-06-07T13:00:50.989Z] 13:00:50     INFO - PID 27888 |   Poisoned by user:        f7
[task 2019-06-07T13:00:50.989Z] 13:00:50     INFO - PID 27888 |   Container overflow:      fc
[task 2019-06-07T13:00:50.989Z] 13:00:50     INFO - PID 27888 |   Array cookie:            ac
[task 2019-06-07T13:00:50.990Z] 13:00:50     INFO - PID 27888 |   Intra object redzone:    bb
[task 2019-06-07T13:00:50.990Z] 13:00:50     INFO - PID 27888 |   ASan internal:           fe
[task 2019-06-07T13:00:50.991Z] 13:00:50     INFO - PID 27888 |   Left alloca redzone:     ca
[task 2019-06-07T13:00:50.991Z] 13:00:50     INFO - PID 27888 |   Right alloca redzone:    cb
[task 2019-06-07T13:00:50.992Z] 13:00:50     INFO - PID 27888 |   Shadow gap:              cc
[task 2019-06-07T13:00:50.992Z] 13:00:50     INFO - PID 27888 | ==29886==ABORTING

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:drno, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 2

[Nils Ohlmeier [:drno]]

Bryce could you please have an initial look if this is some kind of obvious problem or adjust the priority if needed?

Comment 3

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Looks like some of the shutdown races we've seen before. Without reproducing, I'd estimate this is caused by a similar issue to some of our other races: the machinery around the GMP has some dual ownership and expects one side to always shutdown before the other. If this doesn't happen then we see UAFs. Holding NI, as this is worth looking into more.

Alexandru, do we have other reports of this? I notice orange factor is not showing anything, is that because this is marked a sec?

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

There were 4 occurrences in the treeherder result database, they are now all classified as this bug:

2019-06-07 https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=250588815&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=f76cdb104443fb74dff6e77313743da491cb7e7a
2019-06-08 https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=250775480&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=9d30902bfedfe2c9a2c932831bae0e8a785e1b2c
2019-06-09 https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=250842740&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=06b2f9ce5796507f3a3333ee0e495a685b68fd63
2019-06-17 (this is in the maple repository used for testing) https://treeherder.mozilla.org/#/jobs?repo=maple&selectedJob=252207506&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=bbcd1c9de49340cc18db96bdd84393a9276a1cf9

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Comment 6

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Looking further into this.

Comment 7

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

The failures form comment #4 look to be happening during /encrypted-media/clearkey-mp4-update-disallowed-input.https.html, but after 1000 local runs of that I was unable to repro. It's possible that there's some extra variables involved, but that's unclear to me at this point. It also looks like we haven't seen any other reports of this for based on the orange factor.

The race here appears between the GMPServiceChild dtor which destroys a hashtable with GMPContentParents and deletion of an IPC channel within the GMPServiceChild. It seems for this bug during destruction of the GMPContentParent the related IPC channel has already been deleted.

I don't know al lot of the IPC machinery for shutdown/deletion. It's possible something else in GMP is causing the deletion of this channel before it should be, but it's not clear from the stack what kicked off the deletion of the channel. An alternate thought I had is that shutdown causes outstanding channels to be closed, in which case I could see us having a race here if that happened before GMP shutdown.

Could anyone recommend someone to talk to about IPC shutdown/deletion machinery?

Marking as stalled, but with more info or if we see another instance can be revived.

Comment 8

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Looks like we have another instance of this.

Poking a bit more:

• DeleteTask<IPC::Channel>::Run() is what is freeing our channel underneath us.
• Transport is a typedef of IPC::Channel
• There's one instance I see of DeleteTask<Transport> and none for Channel.

So the IPC::Channel/Transport is being set up for deletion by ~IToplevelProtocol(). Then later during ~IToplevelProtocol() we try and touch the now deleted memory. It's not clear to me if this is a race in a single call to ~IToplevelProtocol(), or if we have multiple different top level protocols clobbering one another.

The ownership appears to be that ITopLevelProtocol owns a MessageChannel and MessageChannel has a MessageLink. It looks like if we have a ProcessLink subclass of MessageLink it also has a Transport. In this case when we call MessageChannel::Clear (frame #2 on our crashing stack) it calls through to here, which is interacting with what seems likely to be the freed transport.

It's still not clear to me what is allocating and setting up all these transports, so will keep looking at that.

Comment 9

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Removing stalled as this is still happening and it appears there's more digging I can do.

Comment 11

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

I've been looking at this and bug 1558288. I think they're of a similar class of GMP shutdown issue.

There have been some recent developments in terms of using refcounted in IPC per bug 1550560, and there are some examples of usage in bug 1570369. Refcounting IPC machinery is part of the puzzle, but there are also issues of thread shutdown, xpcom shutdown, and ordering within the GMP that remain. I've been testing some modifications, but it's proving to be a wack a mole where different objects end up living too long or short depending on what I modify.

I've reached my current time box for looking at this, so I'm going to mark as stalled again.

Comment 14

[Andrew McCreight [:mccr8]]

Nika has been looking at bug 1608466, and if I'm understanding correctly, it sounds like this might be a problem with IPC, not GMP in particular.

Comment 15

[Andrew McCreight [:mccr8]]

Nika said she could write a patch. I'll move this back to IPC based on Nika's analysis of the issue here.

Comment 16

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

There's also bug 1493656 covering the general problem of unsafety around this call to Unsound_IsClosed, which I was going to “fix” by making it a safe assertion failure instead of a UAF, but we'd still be crashing.

Comment 17

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1557739 - Be more consistent with IPC Transport APIs,

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/66d07ba43d6d

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Given that this is rated sec-moderate and the patch looks pretty big and scary, I'm going to call ESR wontfix. Feel free to nominate for approval (and probably rebase) if you disagree.