Open Bug 1536369 Opened 6 years ago Updated 2 years ago

Intermittent windows/aarch64 gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw | application crashed [@ mozalloc_abort | mozalloc_handle_oom(unsigned __int64)]

Categories

(Core :: Graphics, defect, P3)

Product:
Core
Component:
Graphics
Type:
defect

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: crash)

#[markdown(off)]
Filed by: egao [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=233751479&repo=try

https://queue.taskcluster.net/v1/task/ZQPpzUf9TM6pXPyP8xMEXw/runs/0/artifacts/public/logs/live_backing.log

Test: MG

Platform: windows10-aarch64

Context:
01:16:55 INFO - TEST-START | gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw
01:16:57 INFO - self.wr_status()=opt-in,d3d11
01:18:07 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/UdcPpO2NQAGjNqayXMYO9Q/artifacts/public/build/target.crashreporter-symbols.zip
01:18:12 INFO - mozcrash Copy/paste: C:\tasks\task_1552525925\build\win32-minidump_stackwalk.exe c:\users\testdr~1\appdata\local\temp\tmpt6rxfc.mozrunner\minidumps\acb6866c-4d9f-4379-aec4-aa5f5e1a4c7c.dmp c:\users\testdr~1\appdata\local\temp\tmp3k0vo3
01:18:21 INFO - mozcrash Saved minidump as C:\tasks\task_1552525925\build\blobber_upload_dir\acb6866c-4d9f-4379-aec4-aa5f5e1a4c7c.dmp
01:18:21 INFO - mozcrash Saved app info as C:\tasks\task_1552525925\build\blobber_upload_dir\acb6866c-4d9f-4379-aec4-aa5f5e1a4c7c.extra
01:18:21 ERROR - PROCESS-CRASH | gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw | application crashed [@ mozalloc_abort]
01:18:21 INFO - Crash dump filename: c:\users\testdr~1\appdata\local\temp\tmpt6rxfc.mozrunner\minidumps\acb6866c-4d9f-4379-aec4-aa5f5e1a4c7c.dmp
01:18:21 INFO - Operating system: Windows NT
01:18:21 INFO - 10.0.17134
01:18:21 INFO - CPU: 0x000c
01:18:21 INFO - 8 CPUs
01:18:21 INFO - GPU: UNKNOWN
01:18:21 INFO - Crash reason: EXCEPTION_ILLEGAL_INSTRUCTION
01:18:21 INFO - Crash address: 0x7fffe957b114
01:18:21 INFO - Process uptime: 1 seconds
01:18:21 INFO - Thread 0 (crashed)
01:18:21 INFO - 0 mozglue.dll!mozalloc_abort [mozalloc_abort.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 33 + 0x0]
01:18:21 INFO - Found by: given as instruction pointer in context
01:18:21 INFO - 1 mozglue.dll!mozalloc_abort [mozalloc_abort.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 26 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 2 mozglue.dll!mozalloc_handle_oom(unsigned __int64) [mozalloc_oom.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 51 + 0x4]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 3 xul.dll!static void gkrust_shared::oom_hook::hook(struct core::alloc::Layout) [lib.rs:cfd502758634e1b8bbb205e53028fff355c8fbbd : 255 + 0x0]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 4 xul.dll!nsDisplayList::PaintRoot(nsDisplayListBuilder *,gfxContext *,unsigned int) [nsDisplayList.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 2685 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 5 xul.dll!nsLayoutUtils::PaintFrame(gfxContext *,nsIFrame *,nsRegion const &,unsigned int,nsDisplayListBuilderMode,nsLayoutUtils::PaintFrameFlags) [nsLayoutUtils.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 3871 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 6 xul.dll!void mozilla::PresShell::Paint(class nsView *, const class nsRegion & const, unsigned int) [PresShell.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 6060 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 7 xul.dll!nsViewManager::ProcessPendingUpdatesForView(nsView *,bool) [nsViewManager.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 396 + 0x210]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 8 xul.dll!void nsViewManager::WillPaintWindow(class nsIWidget *) [nsViewManager.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 666 + 0x44]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 9 xul.dll!void nsView::WillPaintWindow(class nsIWidget *) [nsView.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 990 + 0x0]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 10 xul.dll!nsWindow::OnPaint(HDC__ *,unsigned int) [nsWindowGfx.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 267 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 11 xul.dll!bool nsWindow::ProcessMessage(unsigned int, unsigned __int64 & const, __int64 & const, __int64 *) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 5259 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 12 xul.dll!static __int64 nsWindow::WindowProcInternal(struct HWND__ *, unsigned int, unsigned __int64, __int64) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 4750 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 13 xul.dll!nsWindow::WindowProc(HWND__ *,unsigned int,unsigned __int64,__int64) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 4703 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 14 user32.dll + 0x17174
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 15 user32.dll + 0x16b2c
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 16 user32.dll + 0x2237c
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 17 ntdll.dll!KiUserCallbackDispatcher + 0x1c
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 18 xul.dll!nsWindow::Show(bool) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 1582 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 19 xul.dll!nsresult nsXULWindow::SetVisibility(bool) [nsXULWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 840 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 20 xul.dll!nsresult nsWebShellWindow::OnStateChange(class nsIWebProgress *, class nsIRequest *, unsigned int, nsresult) [nsWebShellWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 613 + 0x50]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 21 xul.dll!void nsDocLoader::DoFireOnStateChange(class nsIWebProgress *, class nsIRequest *, int & const, const nsresult) [nsDocLoader.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 1313 + 0x20]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 22 xul.dll!void nsDocLoader::doStopDocumentLoad(class nsIRequest *, nsresult) [nsDocLoader.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 883 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 23 xul.dll!nsDocLoader::DocLoaderIsEmpty(bool) [nsDocLoader.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 710 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 24 xul.dll!nsDocLoader::OnStopRequest(nsIRequest *,nsresult) [nsDocLoader.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 598 + 0x0]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 25 xul.dll!nsresult mozilla::net::nsLoadGroup::RemoveRequest(class nsIRequest *, class nsISupports *, nsresult) [nsLoadGroup.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 568 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 26 xul.dll!nsresult mozilla::dom::nsUnblockOnloadEvent::Run() [Document.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 7684 + 0x80]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 27 xul.dll!nsThread::ProcessNextEvent(bool,bool *) [nsThread.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 1179 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 28 xul.dll!NS_ProcessNextEvent(nsIThread *,bool) [nsThreadUtils.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 482 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 29 xul.dll!void mozilla::ipc::MessagePump::Run(class base::MessagePump::Delegate *) [MessagePump.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 88 + 0x8]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 30 xul.dll!MessageLoop::RunHandler() [message_loop.cc:cfd502758634e1b8bbb205e53028fff355c8fbbd : 308 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 31 xul.dll!nsBaseAppShell::Run() [nsBaseAppShell.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 137 + 0x34]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 32 xul.dll!nsAppShell::Run() [nsAppShell.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 411 + 0x4]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 33 xul.dll!nsresult nsAppStartup::Run() [nsAppStartup.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 271 + 0x8]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 34 xul.dll!XREMain::XRE_main(int,char * * const,mozilla::BootstrapConfig const &) [nsAppRunner.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 4698 + 0x4cc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 35 xul.dll!int mozilla::BootstrapImpl::XRE_main(int, char * *, const struct mozilla::BootstrapConfig & const) [Bootstrap.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 39 + 0x8c]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 36 firefox.exe!wmain [nsWindowsWMain.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 131 + 0x3d4]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 37 firefox.exe!static int __scrt_common_main_seh() [exe_common.inl : 288 + 0x20]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 38 firefox.exe!wmainCRTStartup [exe_wmain.cpp : 16 + 0x4]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 39 kernel32.dll + 0x15ba0
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 40 ntdll.dll!RtlUserThreadStart + 0x40
01:18:21 INFO - Found by: previous frame's frame pointer

https://phabricator.services.mozilla.com/D23991 will disable this test for the time being until the underlying crash can be fixed.

Nick, as creator of this test maybe this OOM crash is something you are interested in?

Egao, if only one of the tests in that file is failing please only disable that one and not everything.

Severity: normal → critical
Flags: needinfo?(nthomas)
Keywords: crash
Summary: windows/aarch64 - gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw | IOError: Process crashed (Exit code: 0) → Intermittent windows/aarch64 gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw | application crashed [@ mozalloc_abort]

01:18:21 INFO - Operating system: Windows NT
01:18:21 INFO - 10.0.17134
01:18:21 INFO - CPU: 0x000c
01:18:21 INFO - 8 CPUs
01:18:21 INFO - GPU: UNKNOWN
01:18:21 INFO - Crash reason: EXCEPTION_ILLEGAL_INSTRUCTION
01:18:21 INFO - Crash address: 0x7fffe957b114
01:18:21 INFO - Process uptime: 1 seconds
01:18:21 INFO - Thread 0 (crashed)
01:18:21 INFO - 0 mozglue.dll!mozalloc_abort [mozalloc_abort.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 33 + 0x0]
01:18:21 INFO - Found by: given as instruction pointer in context
01:18:21 INFO - 1 mozglue.dll!mozalloc_abort [mozalloc_abort.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 26 + 0x10]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 2 mozglue.dll!mozalloc_handle_oom(unsigned int64) [mozalloc_oom.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 51 + 0x4]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 3 xul.dll!static void gkrust_shared::oom_hook::hook(struct core::alloc::Layout) [lib.rs:cfd502758634e1b8bbb205e53028fff355c8fbbd : 255 + 0x0]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 4 xul.dll!nsDisplayList::PaintRoot(nsDisplayListBuilder *,gfxContext *,unsigned int) [nsDisplayList.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 2685 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 5 xul.dll!nsLayoutUtils::PaintFrame(gfxContext *,nsIFrame *,nsRegion const &,unsigned int,nsDisplayListBuilderMode,nsLayoutUtils::PaintFrameFlags) [nsLayoutUtils.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 3871 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 6 xul.dll!void mozilla::PresShell::Paint(class nsView *, const class nsRegion & const, unsigned int) [PresShell.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 6060 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 7 xul.dll!nsViewManager::ProcessPendingUpdatesForView(nsView *,bool) [nsViewManager.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 396 + 0x210]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 8 xul.dll!void nsViewManager::WillPaintWindow(class nsIWidget *) [nsViewManager.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 666 + 0x44]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 9 xul.dll!void nsView::WillPaintWindow(class nsIWidget *) [nsView.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 990 + 0x0]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 10 xul.dll!nsWindow::OnPaint(HDC *,unsigned int) [nsWindowGfx.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 267 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 11 xul.dll!bool nsWindow::ProcessMessage(unsigned int, unsigned __int64 & const, __int64 & const, __int64 *) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 5259 + 0xc]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 12 xul.dll!static int64 nsWindow::WindowProcInternal(struct HWND *, unsigned int, unsigned __int64, int64) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 4750 + 0x14]
01:18:21 INFO - Found by: previous frame's frame pointer
01:18:21 INFO - 13 xul.dll!nsWindow::WindowProc(HWND *,unsigned int,unsigned __int64,__int64) [nsWindow.cpp:cfd502758634e1b8bbb205e53028fff355c8fbbd : 4703 + 0x10]

Summary: Intermittent windows/aarch64 gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw | application crashed [@ mozalloc_abort] → Intermittent windows/aarch64 gfx\tests\marionette\test_pref_rollout_workaround.py WrPrefRolloutWorkAroundTestCase.test_wr_rollout_workaround_on_qualifying_hw | application crashed [@ mozalloc_abort | mozalloc_handle_oom(unsigned __int64)]

Not sure why I was tagged for ni, forwarding to Bryce.

Flags: needinfo?(nthomas) → needinfo?(bvandyk)

Chris, I imagine this oom crash is not the expected failure mode of the test? Thoughts on if this failure speaks to an issue with the test or with something else? I wonder if something has changed in the painting code such that results in this test triggering an oom.

Flags: needinfo?(bvandyk) → needinfo?(cpearce)

(In reply to Nick Thomas [:nthomas] (UTC+13) from comment #4)

Not sure why I was tagged for ni, forwarding to Bryce.

Sorry, I mixed-up the names. I indeed wanted to needinfo Chris here.

Note, that those are test failures on a new platform, which is about to get integrated. As such it might not be a regression.

I think this is WebRender failing to run on this platform... There's a known issue with WR's Rust code not working correctly there. I think just disabling the test here is fine, as we don't yet care about the functionality this test is testing on ARM64.

Flags: needinfo?(cpearce)

Moving these bugs (intermittent test failures with crashes) out of P5.

Priority: P5 → --
Priority: -- → P3
QA Whiteboard: qa-not-actionable
Severity: critical → S2
Severity: S2 → S3
You need to log in before you can comment on or make changes to this bug.