Closed Bug 1522358 Opened 9 months ago Closed 7 months ago

Intermittent Last test finished | application crashed [@ mozilla::gfx::VRDisplayHost::~VRDisplayHost()]

Categories

(Core :: WebVR, defect, P5, critical)

defect

Tracking

()

RESOLVED FIXED
mozilla68
Tracking Status
firefox-esr60 --- unaffected
firefox66 --- unaffected
firefox67 --- wontfix
firefox68 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: daoshengmu)

References

Details

(Keywords: crash, intermittent-failure, regression)

Crash Data

Attachments

(1 file)

Filed by: csabou [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=223636536&repo=autoland

https://queue.taskcluster.net/v1/task/EhW31s6ITWCqDCDtTsxCOA/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/EhW31s6ITWCqDCDtTsxCOA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

00:59:41 INFO - REFTEST PROCESS-CRASH | Last test finished | application crashed [@ mozilla::gfx::VRDisplayHost::~VRDisplayHost()]
00:59:41 INFO - Crash dump filename: c:\users\task_1548289714\appdata\local\temp\tmpaylsje.mozrunner\minidumps\086ca470-3896-43f0-8dac-cd418147ee2d.dmp
00:59:41 INFO - Operating system: Windows NT
00:59:41 INFO - 10.0.15063
00:59:41 INFO - CPU: amd64
00:59:41 INFO - family 6 model 79 stepping 1
00:59:41 INFO - 16 CPUs
00:59:41 INFO -
00:59:41 INFO - GPU: UNKNOWN
00:59:41 INFO -
00:59:41 INFO - Crash reason: EXCEPTION_ILLEGAL_INSTRUCTION
00:59:41 INFO - Crash address: 0x7ffca26e59d0
00:59:41 INFO - Assertion: Unknown assertion type 0x00000000
00:59:41 INFO - Process uptime: 8 seconds
00:59:41 INFO -
00:59:41 INFO - Thread 5 (crashed)
00:59:41 INFO - 0 xul.dll!void mozilla::gfx::VRDisplayHost::~VRDisplayHost() [VRDisplayHost.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 85 + 0x0]
00:59:41 INFO - rax = 0x00007ffca8955020 rdx = 0x0000000000000001
00:59:41 INFO - rcx = 0x000002375fc75000 rbx = 0x0000000000000008
00:59:41 INFO - rsi = 0x000002375fc75000 rdi = 0x0000000000000000
00:59:41 INFO - rbp = 0x0000000000000001 rsp = 0x000000f7c3dff138
00:59:41 INFO - r8 = 0x0000000000000017 r9 = 0x3978ff73726e4778
00:59:41 INFO - r10 = 0xfefefefefefefeff r11 = 0x8080808080808080
00:59:41 INFO - r12 = 0x0000023755aea630 r13 = 0x7fffffffffffffff
00:59:41 INFO - r14 = 0x000000f7c3dff1c0 r15 = 0x00007ffca8955078
00:59:41 INFO - rip = 0x00007ffca26e59d0
00:59:41 INFO - Found by: given as instruction pointer in context
00:59:41 INFO - 1 xul.dll!void nsTArray_Impl<RefPtr<mozilla::gfx::VRDisplayHost>,nsTArrayInfallibleAllocator>::~nsTArray_Impl() [nsTArray.h:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 876 + 0x81]
00:59:41 INFO - rsp = 0x000000f7c3dff140 rip = 0x00007ffca26f2d3b
00:59:41 INFO - Found by: stack scanning
00:59:41 INFO - 2 xul.dll!mozilla::gfx::VRSystemManager::NotifyVSync() [gfxVR.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 53 + 0xa]
00:59:41 INFO - rsp = 0x000000f7c3dff1a0 rip = 0x00007ffca26f49de
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 3 xul.dll!mozilla::gfx::VRSystemManagerPuppet::NotifyVSync() [gfxVRPuppet.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 669 + 0x5]
00:59:41 INFO - rsp = 0x000000f7c3dff1f0 rip = 0x00007ffca26eb81e
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 4 xul.dll!mozilla::gfx::VRManager::NotifyVsync(mozilla::TimeStamp const &) [VRManager.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 206 + 0x6]
00:59:41 INFO - rsp = 0x000000f7c3dff230 rip = 0x00007ffca26f11f0
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 5 xul.dll!mozilla::gfx::VRSystemManagerPuppet::Run10msTasks() [gfxVRPuppet.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 665 + 0xb]
00:59:41 INFO - rsp = 0x000000f7c3dff270 rip = 0x00007ffca26eb7c2
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 6 xul.dll!mozilla::gfx::VRManager::Run10msTasks() [VRManager.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 325 + 0x6]
00:59:41 INFO - rsp = 0x000000f7c3dff2d0 rip = 0x00007ffca26f1854
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 7 xul.dll!mozilla::gfx::VRManager::RunTasks() [VRManager.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 264 + 0x8]
00:59:41 INFO - rsp = 0x000000f7c3dff340 rip = 0x00007ffca26f15a1
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 8 xul.dll!nsTimerImpl::Fire(int) [nsTimerImpl.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 559 + 0xd]
00:59:41 INFO - rsp = 0x000000f7c3dff400 rip = 0x00007ffca110379f
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 9 xul.dll!nsTimerEvent::Run() [TimerThread.cpp:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 260 + 0x8]
00:59:41 INFO - rsp = 0x000000f7c3dff560 rip = 0x00007ffca110337c
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 10 xul.dll!MessageLoop::RunTask(already_AddRefed<nsIRunnable>) [message_loop.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 442 + 0x6]
00:59:41 INFO - rsp = 0x000000f7c3dff5d0 rip = 0x00007ffca180a90f
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 11 xul.dll!MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask &&) [message_loop.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 450 + 0x8]
00:59:41 INFO - rsp = 0x000000f7c3dff650 rip = 0x00007ffca180b43c
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 12 xul.dll!MessageLoop::DoWork() [message_loop.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 523 + 0xb]
00:59:41 INFO - rsp = 0x000000f7c3dff6a0 rip = 0x00007ffca180b77b
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 13 xul.dll!base::MessagePumpForUI::DoRunLoop() [message_pump_win.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 203 + 0x9]
00:59:41 INFO - rsp = 0x000000f7c3dff740 rip = 0x00007ffca17f4909
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 14 xul.dll!base::MessagePumpWin::Run(base::MessagePump::Delegate *) [message_pump_win.h:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 79 + 0x57]
00:59:41 INFO - rsp = 0x000000f7c3dff7f0 rip = 0x00007ffca17f5637
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 15 xul.dll!MessageLoop::RunHandler() [message_loop.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 308 + 0x8]
00:59:41 INFO - rsp = 0x000000f7c3dff850 rip = 0x00007ffca180a4d2
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 16 xul.dll!base::Thread::ThreadMain() [thread.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 192 + 0x5a]
00:59:41 INFO - rsp = 0x000000f7c3dff8a0 rip = 0x00007ffca18146ae
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 17 xul.dll!static unsigned long `anonymous namespace'::ThreadFunc(void *) [platform_thread_win.cc:0d52389e664d34f9f7315e59cb7127c32b90b2e7 : 30 + 0x6]
00:59:41 INFO - rsp = 0x000000f7c3dffac0 rip = 0x00007ffca17f614a
00:59:41 INFO - Found by: call frame info
00:59:41 INFO - 18 kernel32.dll!LdrpLoadResourceFromAlternativeModule + 0x27c
00:59:41 INFO - rsp = 0x000000f7c3dffaf0 rip = 0x00007ffcd9152774
00:59:41 INFO - Found by: call frame info

Daosheng, this looks like a regression from bug 1518895. Please take a look.

Blocks: 1518895
Flags: needinfo?(dmu)

I agree. It is caused by calling VRManager::Destory() as I mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1522576#c0. We will fix it soon.

Blocks: 1522576
Flags: needinfo?(dmu)
Assignee: nobody → dmu

I think we have resolved this in Bug 1523923. We will keep an eye on this.

See Also: → 1523923

There are 35 failures associated to this bug in the last 7 days. These are occurring on windows10-64 and windows10-64-qr debug.

:daoshengmu can you take a look?

Flags: needinfo?(dmu)

np. I am going to investigate this during this week.

Flags: needinfo?(dmu)

I can't reproduce it by ./mach reftest dom/vr/test/reftest/ --verify --repeat=10. However, I think we might need to use a cancelable task for VRManager::TaskTimerCallback and cancel it when the VR service is shutdown.

This patch will help fix other stability bugs as well. The root cause is we were using VRDisplayHostHashMap. This hash table stores weak ptr of VRDisplayHost and these VRDisplayHost ptr will be released when VRService calls VRManager::Shutdown(). So, we should use an array to keep VRDisplayId and check if they are available at runtime. If the id is changed, we need notify to update display info by trigger displaySetChanged.

In terms of VRControllerHostHashMap, we might don't need to change it because we always get length 0 from VRSystemManagerExternal because we handle it at VRDisplayClient. But, to be consistent, I still do adjustments to make it to be mVRControllerIDs array.

Pushed by dmu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/87be6cf9c45b
Requery VRDisplayHost from VRManagers instead of using weakptr from the hash table. r=kip
Status: NEW → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68

I don't see these failures on Beta67, but I believe the regressing change landed in that timeframe. Is there a user-facing issue demonstrated by this failure which would make us want to consider uplifting this patch or can it ride the trains?

Flags: needinfo?(dmu)

If we don't see these failures on Beta 67, that means it only happens in Nightly when we turn on dom.vr.process.enabled. This change also causes an another bug like Bug 1539185. We will fix them in FF 68 and turn on dom.vr.process.enabled for release channel soon. Probably, it is needless to uplift this to cause risks.

Flags: needinfo?(dmu)

If it's still behind a pref in 67, I completely agree. Thanks!

You need to log in before you can comment on or make changes to this bug.