1759139 - parent process can hang/stop reacting to user input after interaction with Treeherder

Status: RESOLVED WORKSFORME

(Core :: Graphics: WebRender, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Observed by code sheriff Andreea Pavel four times in the first five hours of their shift and twice on Monday European evening by me. Windows 10 and Windows 8.1 and Nightly (in my case the latest 99.0a1 Nightly released that Monday which ran without for several hours without issues and hit this issue twice in 20 minutes).

For both of us the issues started when we interacted with https://treeherder.mozilla.org/jobs?repo=autoland (in my case clicking buttons in the second toolbar at the top) after which neither content nor browser UI reacted to clicks. Andreea reports the window can still be minimized and maximized, I noticed the parent process using the processing power of thread. The Firefox processes needed to be terminated.

Thread snapshots:

0x0000000000000000
wow64win.dll!ZwGdiDrawStream+0xa
wow64win.dll!whNtGdiDrawStream+0x16
wow64.dll!Wow64SystemServiceEx+0xfb
wow64cpu.dll!TurboDispatchJumpAddressEnd+0xb
wow64.dll!RunCpuSimulation+0xa
wow64.dll!Wow64KiUserCallbackDispatcher+0x244
wow64win.dll!whcbfnDWORD+0xbe
ntdll.dll!KiUserCallbackDispatcherContinue
wow64win.dll!ZwUserDispatchMessage+0xa
wow64win.dll!Wow64MsgFncWM_NULL+0x95
wow64win.dll!whNtUserDispatchMessage+0x9a
wow64.dll!Wow64SystemServiceEx+0xfb
wow64cpu.dll!TurboDispatchJumpAddressEnd+0xb
wow64.dll!RunCpuSimulation+0xa
wow64.dll!Wow64LdrpInitialize+0x172
ntdll.dll!LdrpInitializeProcess+0x1591
ntdll.dll!_LdrpInitialize+0x891ce
ntdll.dll!LdrInitializeThunk+0xe
GDI32.dll!_NtGdiDrawStream@12+0xc
GDI32.dll!GdiDrawStream+0x59
uxtheme.dll!CImageFile::DrawBackgroundDS+0x351
uxtheme.dll!CImageFile::DrawImageInfo+0x277
uxtheme.dll!DrawThemeBackground+0x20d
xul.dll!mozilla::widget::nsNativeThemeWin::DrawWidgetBackground+0x3a3
xul.dll!mozilla::nsDisplayThemedBackground::PaintInternal+0x12e
xul.dll!mozilla::nsDisplayThemedBackground::Paint+0x3b
xul.dll!mozilla::nsDisplayList::Paint+0x327
xul.dll!mozilla::FallbackRenderer::EndTransactionWithList+0xfd
xul.dll!mozilla::nsDisplayList::PaintRoot+0x587
xul.dll!nsLayoutUtils::PaintFrame+0xe15
xul.dll!mozilla::PresShell::PaintInternal+0x539
xul.dll!mozilla::PresShell::SyncPaintFallback+0x5f
xul.dll!nsViewManager::Refresh+0x103
xul.dll!nsViewManager::PaintWindow+0x3f
xul.dll!nsView::PaintWindow+0x2b
xul.dll!nsWindow::OnPaint+0x7cd
xul.dll!nsWindow::OnPaint+0x5cd
xul.dll!nsWindow::ProcessMessage+0x156f
xul.dll!nsWindow::WindowProcInternal+0x108
xul.dll!CallWindowProcCrashProtected+0x4d
xul.dll!nsWindow::WindowProc+0x4c
USER32.dll!__InternalCallWinProc@20+0x2b
USER32.dll!UserCallWinProcCheckWow+0x262
USER32.dll!DispatchClientMessage+0xdc
USER32.dll!___fnDWORD@4+0x49
ntdll.dll!_KiUserCallbackDispatcher@12+0x36
xul.dll!nsBaseAppShell::OnProcessNextEvent+0x25f
USER32.dll!_DispatchMessageW@4+0x10
xul.dll!nsBaseAppShell::OnProcessNextEvent+0x25f
xul.dll!nsThread::ProcessNextEvent+0x3c1
xul.dll!mozilla::ipc::MessagePump::Run+0xa7
xul.dll!MessageLoop::RunHandler+0x4f
xul.dll!MessageLoop::Run+0x3f
xul.dll!nsBaseAppShell::Run+0x25
xul.dll!nsAppShell::Run+0x28
xul.dll!nsAppStartup::Run+0x3e
xul.dll!XREMain::XRE_mainRun+0xaf6
xul.dll!XREMain::XRE_main+0x2ac
xul.dll!XRE_main+0x36
xul.dll!mozilla::BootstrapImpl::XRE_main+0x11
firefox.exe!wmain+0x5c2
firefox.exe!__scrt_common_main_seh+0xfa
KERNEL32.DLL!@BaseThreadInitThunk@12+0x24
ntdll.dll!__RtlUserThreadStart+0x2f
ntdll.dll!__RtlUserThreadStart@8+0x1b

0x0000000000000000
uxtheme.dll!CImageFile::SelectCorrectImageFile+0x2
0x0000000000000000
uxtheme.dll!CImageFile::SelectCorrectImageFile+0x2
uxtheme.dll!DrawThemeBackground+0x1f2
xul.dll!mozilla::widget::nsNativeThemeWin::DrawWidgetBackground+0x3a3
xul.dll!mozilla::nsDisplayThemedBackground::PaintInternal+0x12e
xul.dll!mozilla::nsDisplayThemedBackground::Paint+0x3b
xul.dll!mozilla::nsDisplayList::Paint+0x327
xul.dll!mozilla::FallbackRenderer::EndTransactionWithList+0xfd
xul.dll!mozilla::nsDisplayList::PaintRoot+0x587
xul.dll!nsLayoutUtils::PaintFrame+0xe15
xul.dll!mozilla::PresShell::PaintInternal+0x539
xul.dll!mozilla::PresShell::SyncPaintFallback+0x5f
xul.dll!nsViewManager::Refresh+0x103
xul.dll!nsViewManager::PaintWindow+0x3f
xul.dll!nsView::PaintWindow+0x2b
xul.dll!nsWindow::OnPaint+0x7cd
xul.dll!nsWindow::OnPaint+0x5cd
xul.dll!nsWindow::ProcessMessage+0x156f
xul.dll!nsWindow::WindowProcInternal+0x108
xul.dll!CallWindowProcCrashProtected+0x4d
xul.dll!nsWindow::WindowProc+0x4c
USER32.dll!__InternalCallWinProc@20+0x2b
USER32.dll!UserCallWinProcCheckWow+0x262
USER32.dll!DispatchClientMessage+0xdc
USER32.dll!___fnDWORD@4+0x49
ntdll.dll!_KiUserCallbackDispatcher@12+0x36
xul.dll!nsBaseAppShell::OnProcessNextEvent+0x25f
USER32.dll!_DispatchMessageW@4+0x10
xul.dll!nsBaseAppShell::OnProcessNextEvent+0x25f
xul.dll!nsThread::ProcessNextEvent+0x3c1
xul.dll!mozilla::ipc::MessagePump::Run+0xa7
xul.dll!MessageLoop::RunHandler+0x4f
xul.dll!MessageLoop::Run+0x3f
xul.dll!nsBaseAppShell::Run+0x25
xul.dll!nsAppShell::Run+0x28
xul.dll!nsAppStartup::Run+0x3e
xul.dll!XREMain::XRE_mainRun+0xaf6
xul.dll!XREMain::XRE_main+0x2ac
xul.dll!XRE_main+0x36
xul.dll!mozilla::BootstrapImpl::XRE_main+0x11
firefox.exe!wmain+0x5c2
firefox.exe!__scrt_common_main_seh+0xfa
KERNEL32.DLL!@BaseThreadInitThunk@12+0x24
ntdll.dll!__RtlUserThreadStart+0x2f
ntdll.dll!__RtlUserThreadStart@8+0x1b

Comment 1

[Timothy Nikkel (:tnikkel)]

That's weird, SyncPaintFallback is not the main way we paint the window. Something to do with how we paint on Windows? Maybe Sotaro knows of some recent changes in this area that might have affected this.

Comment 2

[Sotaro Ikeda [:sotaro]]

WebRender rendering should not use SyncPaintFallback. When SyncPaintFallback is used, FallbackRenderer is used. On windows, FallbackRenderer is created for tooltip rendering(nsWindow::ShouldUseOffMainThreadCompositing() is false) or when CompositorSession creation is failed.

• https://searchfox.org/mozilla-central/rev/fa69d8b248e6c1df670aa6b019e30ec37e6672be/widget/windows/nsWindow.cpp#7410
• https://searchfox.org/mozilla-central/rev/fa69d8b248e6c1df670aa6b019e30ec37e6672be/widget/windows/nsWindow.cpp#4082

Then, WebRender seemed not related to this bug.

It seemed that DrawThemeBackground() call in nsNativeThemeWin::DrawWidgetBackground() caused the hang

Comment 3

[Sotaro Ikeda [:sotaro]]

FallbackRenderer was added by Bug 1722258, though it is not a recent change. Before FallbackRenderer use, BasicLayerManager was used.

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Is there a way to support you investigating this bug? Another code sheriff experiences this issue with Treeherder.

Comment 5

[Glenn Watson [:gw]]

Is this still relevant?