1695379 - WebRender ANGLE Crash in [@ OOM | large | mozalloc_abort | moz_xmalloc | std::vector<T>::_Emplace_reallocate<T>]

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P3)

Description

[Chris Peterson [:cpeterson]]

Crash report: https://crash-stats.mozilla.org/report/index/8221ac78-fb02-405f-8dc5-8351a0210227

I hit this WebRender ANGLE OOM crash three times today in 32-bit Firefox build on 64-bit Windows 10.

MOZ_CRASH Reason: MOZ_CRASH()

Top 10 frames of crashing thread:

0 mozglue.dll mozalloc_abort memory/mozalloc/mozalloc_abort.cpp:33
1 mozglue.dll mozalloc_handle_oom memory/mozalloc/mozalloc_oom.cpp:51
2 mozglue.dll moz_xmalloc memory/mozalloc/mozalloc.cpp:54
3 libglesv2.dll std::vector<gl::ProgramVaryingRef, std::allocator<gl::ProgramVaryingRef> >::_Emplace_reallocate<gl::ProgramVaryingRef> vs2017_15.8.4/VC/include/vector:956
4 libglesv2.dll gl::Program::loadBinary gfx/angle/checkout/src/libANGLE/Program.cpp:1920
5 libglesv2.dll gl::Context::programBinary gfx/angle/checkout/src/libANGLE/Context.cpp:6941
6 libglesv2.dll gl::ProgramBinary gfx/angle/checkout/src/libGLESv2/entry_points_gles_3_0_autogen.cpp:1940
7 xul.dll gleam::gl::{{impl}}::program_binary third_party/rust/gleam/src/gles_fns.rs:1702
8 xul.dll webrender::device::gl::Device::create_program gfx/wr/webrender/src/device/gl.rs:2972
9 xul.dll webrender::renderer::shade::LazilyCompiledShader::get_internal gfx/wr/webrender/src/renderer/shade.rs:201

Comment 1

[Jim Mathies [:jimm]]

This was caused by a 1.63 GB allocation, which doesn't seem sane.

Comment 2

[Jim Mathies [:jimm]]

Hey, can you reproduce this and if so, on what site?

Comment 3

[Jeff Muizelaar [:jrmuizel]]

Can you get a full stack from a debugger? It's difficult to tell what's actually going on here from the crash stack.

Comment 4

[Chris Peterson [:cpeterson]]

(In reply to Jim Mathies [:jimm] from comment #2)

Hey, can you reproduce this and if so, on what site?

(In reply to Jeff Muizelaar [:jrmuizel] from comment #3)

Can you get a full stack from a debugger? It's difficult to tell what's actually going on here from the crash stack.

I haven't seen this crash since I reported it on 2021-02-26. I don't know what site might have triggered it. My crash reports didn't have URLs because they were from the gpu process. But I always have a dozen Google Docs and Sheets tabs open, so that might be related.

Comment 5

[Jeff Muizelaar [:jrmuizel]]

I'll improve the signature and take a look at a minidump.

Comment 6

[Jeff Muizelaar [:jrmuizel]]

This crash is happening here:
https://searchfox.org/mozilla-central/rev/166dfa51ee50207a253fc577b9a596e64f24258c/gfx/angle/checkout/src/libANGLE/Program.cpp#5391

It seems like we must be reading a very large uniformIndexCount out of the stream and that's causing us to OOM.

Comment 7

[Brad Werth [:bradwerth]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #6)

This crash is happening here:
https://searchfox.org/mozilla-central/rev/166dfa51ee50207a253fc577b9a596e64f24258c/gfx/angle/checkout/src/libANGLE/Program.cpp#5391

It seems like we must be reading a very large uniformIndexCount out of the stream and that's causing us to OOM.

This seems exploitable since streams can be manipulated. Should we put a sensible cap here on uniformIndexCount -- in addition to fixing the bug?

Comment 8

[Jeff Muizelaar [:jrmuizel]]

The streams are not exposed to webcontent. We create them, store them to disk and load them.

Comment 9

[Jim Mathies [:jimm]]

https://crash-stats.mozilla.org/search/?signature=%3DOOM%20%7C%20large%20%7C%20mozalloc_abort%20%7C%20moz_xmalloc%20%7C%20std%3A%3Avector%3CT%3E%3A%3A_Emplace_reallocate%3CT%3E&product=Firefox&date=%3E%3D2021-04-01T20%3A42%3A00.000Z&date=%3C2021-04-08T20%3A42%3A00.000Z&_facets=signature&_facets=oom_allocation_size&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-oom_allocation_size

Comment 10

[Sotaro Ikeda [:sotaro]]

Bug 1707471 looks similar problem.