1681671 - Crash in [@ __memcpy_avx_unaligned | webrender::renderer::Renderer::draw_frame]

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P3)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Bug 1632698 landed earlier this week to fix similar signatures. Andrew, please also take a look at this one (build ID 20201209213504).

Maybe Fission related. (DOMFissionEnabled=1)

Crash report: https://crash-stats.mozilla.org/report/index/356a3a4e-17fa-4d69-a1ce-f3a410201210

Reason: SIGSEGV /SEGV_MAPERR

Top 10 frames of crashing thread:

0 libc.so.6 __memcpy_avx_unaligned 
1 libxul.so webrender::renderer::Renderer::draw_frame gfx/wr/webrender/src/renderer.rs:5915
2 libxul.so webrender::renderer::Renderer::render_impl gfx/wr/webrender/src/renderer.rs:3488
3 libxul.so webrender::renderer::Renderer::render gfx/wr/webrender/src/renderer.rs:3244
4 libxul.so wr_renderer_render gfx/webrender_bindings/src/bindings.rs:614
5 libxul.so mozilla::wr::RendererOGL::UpdateAndRender gfx/webrender_bindings/RendererOGL.cpp:186
6 libxul.so mozilla::wr::RenderThread::UpdateAndRender gfx/webrender_bindings/RenderThread.cpp:492
7 libxul.so mozilla::wr::RenderThread::HandleFrameOneDoc gfx/webrender_bindings/RenderThread.cpp:326
8 libxul.so mozilla::detail::RunnableMethodImpl<mozilla::wr::RenderThread*, void  xpcom/threads/nsThreadUtils.h:1201
9 libxul.so base::MessagePumpDefault::Run ipc/chromium/src/base/message_pump_default.cc:35

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

There are a few similar signatures floating around, we see it in beta too without fission. I'm in the process of fixing some telemetry/crash report annotations which may shed light on whether an NVIDIA or innocent device reset happened before the crash.

Comment 2

[Andrew Osmond [:aosmond] (he/him)]

These signatures began in build 20201108093650:

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e40cc6272439&tochange=d366215bf9af5dd06c27f8b36d875553838d2fe0

But that was a Sunday daily, probably fairly low uptake. Earlier builds would include:

• 20201107100127: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0e95e169ef40&tochange=e40cc6272439b7fa848bdf875bb41d7f4b1a3b71
• 20201106160425: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=41effaf024a5&tochange=0e95e169ef40f27ae4de839644c63249acd3b5b1
• 20201106093443: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=5278acfcd834&tochange=41effaf024a55426f4486278364dea1117ed6571

Bug 1675453 seems of particular note, since it enabled VAAPI by default. We should probably hook hardware video decoding into the gfx blocklist. This would allow us to run a test on nightly to see if it is indeed related without disabling HW acceleration for all users.

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

Reviewing

https://crash-stats.mozilla.org/report/index/6b16cc93-2d3d-49c4-8626-932720201212#tab-metadata

I can see DeviceResetReason is 10, which corresponds to the NVIDIA video memory reset as the last known reset. So it appears our current solution is inadequate for handling it.

Comment 4

[Andrew Osmond [:aosmond] (he/him)]

I think we might need to do:

https://searchfox.org/mozilla-central/rev/95cf843de977805a3951f9137f5ff1930599d94e/gfx/webrender_bindings/RenderThread.cpp#867

for the NVIDIA video memory resets, and implement RenderDMABUFTextureHost::ClearCachedResources.

Comment 5

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1681671 - Ensure we clear cached DMABuf textures during an NVIDIA video memory device reset.

Comment 6

[Andrew Osmond [:aosmond] (he/him)]

This should also fix issues with device resets for non-NVIDIA users as well who experience a full device reset who have DMABuf support turned on.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f78f7ee4f267
Ensure we clear cached DMABuf textures during an NVIDIA video memory device reset. r=sotaro

Comment 8

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f78f7ee4f267