1540590 - Intermittent SUMMARY: AddressSanitizer: heap-use-after-free z:\build\build\src\gfx\vr\gfxVRPuppet.cpp:762 in mozilla::gfx::VRSystemManagerPuppet::GetIsPresenting(void)

Status: RESOLVED FIXED

(Core :: WebVR, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: dvarga [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=237252327&repo=autoland

https://queue.taskcluster.net/v1/task/DmpUdBLxQvyh6HswOcjnuQ/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/DmpUdBLxQvyh6HswOcjnuQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

00:41:49 INFO - ###!!! [Child][MessageChannel] Error: (msgtype=0x990002,name=PVRGPU::Msg_StopVRService) Closed channel: cannot send/recv
00:41:49 ERROR - ==7900==ERROR: AddressSanitizer: heap-use-after-free on address 0x11c47da15010 at pc 0x7ffed795fc08 bp 0x003f9b3fe630 sp 0x003f9b3fe678
00:41:49 INFO - READ of size 4 at 0x11c47da15010 thread T2
00:41:49 INFO - [GPU 7900, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
00:41:49 INFO - [GPU 7900, Chrom
00:41:49 INFO - ###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
00:41:49 INFO - e_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
00:41:49 INFO - [Parent 3604, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
00:41:49 INFO - [GPU 7900, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
00:41:49 INFO - #0 0x7ffed795fc07 in mozilla::gfx::VRSystemManagerPuppet::GetIsPresenting(void) z:\build\build\src\gfx\vr\gfxVRPuppet.cpp:762
00:41:49 INFO - #1 0x7ffed797637e in mozilla::gfx::VRSystemManager::NotifyVSync(void) z:\build\build\src\gfx\vr\gfxVR.cpp:48
00:41:49 INFO - #2 0x7ffed795dde7 in mozilla::gfx::VRSystemManagerPuppet::NotifyVSync(void) z:\build\build\src\gfx\vr\gfxVRPuppet.cpp:669
00:41:49 INFO - #3 0x7ffed796c0c5 in mozilla::gfx::VRManager::NotifyVsync(class mozilla::TimeStamp const &) z:\build\build\src\gfx\vr\VRManager.cpp:208
00:41:49 INFO - #4 0x7ffed795dd58 in mozilla::gfx::VRSystemManagerPuppet::Run10msTasks(void) z:\build\build\src\gfx\vr\gfxVRPuppet.cpp:665
00:41:49 INFO - #5 0x7ffed796d7a4 in mozilla::gfx::VRManager::Run10msTasks(void) z:\build\build\src\gfx\vr\VRManager.cpp:337
00:41:49 INFO - #6 0x7ffed796c8d1 in mozilla::gfx::VRManager::RunTasks(void) z:\build\build\src\gfx\vr\VRManager.cpp:274
00:41:49 INFO - #7 0x7ffed4435b14 in nsTimerImpl::Fire(int) z:\build\build\src\xpcom\threads\nsTimerImpl.cpp:559
00:41:49 INFO - #8 0x7ffed44350a5 in nsTimerEvent::Run(void) z:\build\build\src\xpcom\threads\TimerThread.cpp:260
00:41:49 INFO - #9 0x7ffed54975e3 in ?DeferOrRunPendingTask@MessageLoop@@IEAA_N$$QEAUPendingTask@1@@Z z:\build\build\src\ipc\chromium\src\base\message_loop.cc:450
00:41:49 INFO - #10 0x7ffed5498fde in MessageLoop::DoWork(void) z:\build\build\src\ipc\chromium\src\base\message_loop.cc:523
00:41:49 INFO - #11 0x7ffed5469571 in base::MessagePumpForUI::DoRunLoop(void) z:\build\build\src\ipc\chromium\src\base\message_pump_win.cc:203
00:41:49 INFO - #12 0x7ffed546bb99 in base::MessagePumpWin::Run(class base::MessagePump::Delegate *) z:\build\build\src\ipc\chromium\src\base\message_pump_win.h:79
00:41:49 INFO - #13 0x7ffed549635e in MessageLoop::RunHandler(void) z:\build\build\src\ipc\chromium\src\base\message_loop.cc:308
00:41:49 INFO - #14 0x7ffed54a8282 in base::Thread::ThreadMain(void) z:\build\build\src\ipc\chromium\src\base\thread.cc:192
00:41:49 INFO - #15 0x7ffed546d3af in `anonymous namespace'::ThreadFunc z:\build\build\src\ipc\chromium\src\base\platform_thread_win.cc:19
00:41:49 INFO - #16 0x7fff1865e888 in __asan::AsanThread::ThreadStart(unsigned __int64,struct __sanitizer::atomic_uintptr_t *) Z:\task_1553815194\build\src\build\build-clang\build-clang\src\llvm\projects\compiler-rt\lib\asan\asan_thread.cc:264
00:41:49 INFO - #17 0x7fff29d53033 (C:\Windows\System32\KERNEL32.DLL+0x180013033)
00:41:49 INFO - #18 0x7fff25f5f701 in patched_BaseThreadInitThunk z:\build\build\src\mozglue\build\WindowsDllBlocklist.cpp:712
00:41:49 INFO - #19 0x7fff2c8c1460 (C:\Windows\SYSTEM32\ntdll.dll+0x180071460)

Comment 1

[Daosheng Mu[:daoshengmu]]

It has the same result as our recent bugs of Bug 1537692 and Bug 1539119 that are caused by VRGPUChild::ActorDestroy() VRManager->Shutdown();.

It looks like a race condition problem between compositorThread and mainthread when both of them are accessing mVRDisplay. We were usingmTaskTimer->SetTarget(CompositorThreadHolder::Loop()->SerialEventTarget()) to call VRManager::Run10msTasks, but when VR process or thread shutdown, we call vm->Shutdown() at VRGPUChild::ActorDestroy from the main thread. I think we should make ``vm->Shutdown()` be called in the compositor thread as well.

Comment 2

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1540590 - Moving VRManager::Shutdown() task to Compositor thread to avoid race conditions.

Comment 3

[Daosheng Mu[:daoshengmu]]

Please help land this patch to m-c.

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/integration/autoland/rev/d02a00103fe410de75a3943ce8cbff9141533fb7

Comment 5

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/d02a00103fe4