Open Bug 1743190 Opened 2 years ago Updated 22 days ago

unconstrained memory usage in [@ mozilla::gfx::InlineTranslator::TranslateRecording]

Categories

(Core :: Graphics, defect, P1)

Product:
Core
Component:
Graphics
Type:
defect

Tracking

()

Tracking Flags:
Tracking Status
firefox-esr91 --- wontfix
firefox-esr102 --- wontfix
firefox96 --- wontfix
firefox103 --- wontfix
firefox104 --- wontfix
firefox105 --- wontfix
firefox106 --- wontfix
firefox107 --- wontfix
firefox108 --- wontfix

People

(Reporter: tsmith, Assigned: nical)

References

(Depends on 2 open bugs, Blocks 3 open bugs)

Details

(Keywords: csectype-oom, testcase, Whiteboard: [bugmon:bisected,confirmed][fuzzblocker])

Attachments

(8 files)

testcase.html
210 bytes, text/html
Details
heap_profile.txt
143.21 KB, text/plain
Details
testcase_2.html
463 bytes, text/html
Details
testcase_3.html
199 bytes, text/html
Details
testcase_4.html
106 bytes, text/html
Details
oom_tests.zip
1004 bytes, application/x-zip-compressed
Details
testcase_5.html
325 bytes, text/html
Details
testcase_6.html
381 bytes, text/html
Details
Attached file testcase.html —

Found while fuzzing m-c 20211123-ba4d4963c38b (--enable-address-sanitizer --enable-fuzzing)

To help catch this issue ASAN_OPTIONS=hard_rss_limit_mb=10000 was used. See Bug 1715316 for details about fuzzing triggered OOMs.

==437624==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x5598a64199bf bp 0x7fa5ef7808f0 sp 0x7fa5ef7808d0 T48)
==437624==The signal is caused by a WRITE memory access.
==437624==Hint: address points to the zero page.
    #0 0x5598a64199bf in MOZ_Crash /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:256:3
    #1 0x5598a64199bf in mozalloc_abort src/memory/mozalloc/mozalloc_abort.cpp:35:3
    #2 0x5598a64191ad in mozalloc_handle_oom(unsigned long) src/memory/mozalloc/mozalloc_oom.cpp:51:3
    #3 0x5598a64190cb in moz_xmalloc src/memory/mozalloc/mozalloc.cpp:54:5
    #4 0x5598a648b89b in operator new /builds/worker/workspace/obj-build/dist/include/mozilla/cxxalloc.h:33:10
    #5 0x5598a648b89b in allocate /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/ext/new_allocator.h:111:27
    #6 0x5598a648b89b in allocate /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/alloc_traits.h:436:20
    #7 0x5598a648b89b in _M_create /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/basic_string.tcc:153:14
    #8 0x5598a648b89b in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve(unsigned long) /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/basic_string.tcc:293:24
    #9 0x7fa636ef6336 in std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::overflow(int) (/lib/x86_64-linux-gnu/libstdc++.so.6+0x139336)
    #10 0x7fa636efeb59 in std::basic_streambuf<char, std::char_traits<char> >::xsputn(char const*, long) (/lib/x86_64-linux-gnu/libstdc++.so.6+0x141b59)
    #11 0x7fa636ef0823 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (/lib/x86_64-linux-gnu/libstdc++.so.6+0x133823)
    #12 0x7fa636ef0bdb in std::basic_ostream<char, std::char_traits<char> >& std::operator<<<std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*) (/lib/x86_64-linux-gnu/libstdc++.so.6+0x133bdb)
    #13 0x7fa61a95a4d7 in operator<< src/gfx/2d/Logging.h:296:16
    #14 0x7fa61a95a4d7 in RecordedFillGlyphs<MemReader> src/gfx/2d/RecordedEventImpl.h:2423:21
    #15 0x7fa61a95a4d7 in DoWithEvent<MemReader> src/gfx/2d/RecordedEventImpl.h:4061:5
    #16 0x7fa61a95a4d7 in mozilla::gfx::InlineTranslator::TranslateRecording(char*, unsigned long) src/gfx/2d/InlineTranslator.cpp:72:20
    #17 0x7fa61b22b514 in Moz2DRenderCallback src/gfx/webrender_bindings/Moz2DImageRenderer.cpp:427:20
    #18 0x7fa61b22b514 in wr_moz2d_render_cb src/gfx/webrender_bindings/Moz2DImageRenderer.cpp:471:10
    #19 0x7fa627b3cef0 in webrender_bindings::moz2d_renderer::rasterize_blob::_$u7b$$u7b$closure$u7d$$u7d$::h3121e484a1b67f3d src/gfx/webrender_bindings/src/moz2d_renderer.rs:608:16
    #20 0x7fa627b3cef0 in webrender_bindings::moz2d_renderer::autoreleasepool::h94fc726760f09370 src/gfx/webrender_bindings/src/moz2d_renderer.rs:590:9
    #21 0x7fa627b3cef0 in webrender_bindings::moz2d_renderer::rasterize_blob::h7ac12f4789a022e5 src/gfx/webrender_bindings/src/moz2d_renderer.rs:606:18
    #22 0x7fa627ae74ec in core::ops::function::Fn::call::h1d4fdb0a52fa4f92 /builds/worker/fetches/rust/library/core/src/ops/function.rs:70:5
    #23 0x7fa627ae74ec in core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnMut$LT$A$GT$$u20$for$u20$$RF$F$GT$::call_mut::h2daa2bac6a5ab398 /builds/worker/fetches/rust/library/core/src/ops/function.rs:247:13
    #24 0x7fa627ae74ec in core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$mut$u20$F$GT$::call_once::hc26ac72e9f193759 /builds/worker/fetches/rust/library/core/src/ops/function.rs:280:13
    #25 0x7fa627ae74ec in core::option::Option$LT$T$GT$::map::h4bdf95ca64ca0e4f /builds/worker/fetches/rust/library/core/src/option.rs:836:29
    #26 0x7fa627ae74ec in _$LT$core..iter..adapters..map..Map$LT$I$C$F$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::next::hc641fd93fafa78ac /builds/worker/fetches/rust/library/core/src/iter/adapters/map.rs:103:9
    #27 0x7fa627ae74ec in rayon::iter::plumbing::Folder::consume_iter::h89674144a63c178b src/third_party/rust/rayon/src/iter/plumbing/mod.rs:178:21
    #28 0x7fa627ae74ec in _$LT$rayon..iter..map..MapFolder$LT$C$C$F$GT$$u20$as$u20$rayon..iter..plumbing..Folder$LT$T$GT$$GT$::consume_iter::h5ce7eee21e60f4cb src/third_party/rust/rayon/src/iter/map.rs:248:21
    #29 0x7fa627ae74ec in rayon::iter::plumbing::Producer::fold_with::hd549d60b27ae1f60 src/third_party/rust/rayon/src/iter/plumbing/mod.rs:110:9
    #30 0x7fa627ae74ec in rayon::iter::plumbing::bridge_producer_consumer::helper::hbdff87fa1f93f71e src/third_party/rust/rayon/src/iter/plumbing/mod.rs:438:13
    #31 0x7fa627a5f455 in rayon::iter::plumbing::bridge_producer_consumer::helper::_$u7b$$u7b$closure$u7d$$u7d$::h0bdff91ebad9d71a src/third_party/rust/rayon/src/iter/plumbing/mod.rs:418:21
    #32 0x7fa627a5f455 in rayon_core::join::join_context::call_a::_$u7b$$u7b$closure$u7d$$u7d$::h0a90fed1d90eb9f6 src/third_party/rust/rayon-core/src/join/mod.rs:124:17
    #33 0x7fa627a5f455 in _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h11bb5129af55f89e /builds/worker/fetches/rust/library/core/src/panic/unwind_safe.rs:271:9
    #34 0x7fa627a5f455 in std::panicking::try::do_call::h6080a873f99cc4e5 /builds/worker/fetches/rust/library/std/src/panicking.rs:403:40
    #35 0x7fa627a5f455 in std::panicking::try::h40c2f1c016c4dfb3 /builds/worker/fetches/rust/library/std/src/panicking.rs:367:19
    #36 0x7fa627a5f455 in std::panic::catch_unwind::h40c9990754be49bf /builds/worker/fetches/rust/library/std/src/panic.rs:129:14
    #37 0x7fa627a5f455 in rayon_core::unwind::halt_unwinding::h7471fbc0ba16f85d src/third_party/rust/rayon-core/src/unwind.rs:17:5
    #38 0x7fa627a5f455 in rayon_core::join::join_context::_$u7b$$u7b$closure$u7d$$u7d$::h770f75d7ee3fa002 src/third_party/rust/rayon-core/src/join/mod.rs:141:24
    #39 0x7fa627a604e2 in rayon_core::registry::in_worker::h4e106ab7d121351a src/third_party/rust/rayon-core/src/registry.rs:875:13
    #40 0x7fa627ae78f7 in rayon_core::join::join_context::hd698d4c3e8d29183 src/third_party/rust/rayon-core/src/join/mod.rs:132:5
    #41 0x7fa627ae78f7 in rayon::iter::plumbing::bridge_producer_consumer::helper::hbdff87fa1f93f71e src/third_party/rust/rayon/src/iter/plumbing/mod.rs:416:47
    #42 0x7fa627a5f998 in rayon::iter::plumbing::bridge_producer_consumer::helper::_$u7b$$u7b$closure$u7d$$u7d$::hc3fb560020e50bd4 src/third_party/rust/rayon/src/iter/plumbing/mod.rs:427:21
    #43 0x7fa627a5f998 in rayon_core::join::join_context::call_b::_$u7b$$u7b$closure$u7d$$u7d$::hf9baeb8f142826d5 src/third_party/rust/rayon-core/src/join/mod.rs:129:25
    #44 0x7fa627a5f998 in rayon_core::job::StackJob$LT$L$C$F$C$R$GT$::run_inline::h903d51559bafd42e src/third_party/rust/rayon-core/src/job.rs:97:9
    #45 0x7fa627a5f998 in rayon_core::join::join_context::_$u7b$$u7b$closure$u7d$$u7d$::h770f75d7ee3fa002 src/third_party/rust/rayon-core/src/join/mod.rs:158:36
    #46 0x7fa627a604e2 in rayon_core::registry::in_worker::h4e106ab7d121351a src/third_party/rust/rayon-core/src/registry.rs:875:13
    #47 0x7fa627ae78f7 in rayon_core::join::join_context::hd698d4c3e8d29183 src/third_party/rust/rayon-core/src/join/mod.rs:132:5
    #48 0x7fa627ae78f7 in rayon::iter::plumbing::bridge_producer_consumer::helper::hbdff87fa1f93f71e src/third_party/rust/rayon/src/iter/plumbing/mod.rs:416:47
    #49 0x7fa627a5f998 in rayon::iter::plumbing::bridge_producer_consumer::helper::_$u7b$$u7b$closure$u7d$$u7d$::hc3fb560020e50bd4 src/third_party/rust/rayon/src/iter/plumbing/mod.rs:427:21
    #50 0x7fa627a5f998 in rayon_core::join::join_context::call_b::_$u7b$$u7b$closure$u7d$$u7d$::hf9baeb8f142826d5 src/third_party/rust/rayon-core/src/join/mod.rs:129:25
    #51 0x7fa627a5f998 in rayon_core::job::StackJob$LT$L$C$F$C$R$GT$::run_inline::h903d51559bafd42e src/third_party/rust/rayon-core/src/job.rs:97:9
    #52 0x7fa627a5f998 in rayon_core::join::join_context::_$u7b$$u7b$closure$u7d$$u7d$::h770f75d7ee3fa002 src/third_party/rust/rayon-core/src/join/mod.rs:158:36
    #53 0x7fa627a604e2 in rayon_core::registry::in_worker::h4e106ab7d121351a src/third_party/rust/rayon-core/src/registry.rs:875:13
    #54 0x7fa627ae78f7 in rayon_core::join::join_context::hd698d4c3e8d29183 src/third_party/rust/rayon-core/src/join/mod.rs:132:5
    #55 0x7fa627ae78f7 in rayon::iter::plumbing::bridge_producer_consumer::helper::hbdff87fa1f93f71e src/third_party/rust/rayon/src/iter/plumbing/mod.rs:416:47
    #56 0x7fa627a5f455 in rayon::iter::plumbing::bridge_producer_consumer::helper::_$u7b$$u7b$closure$u7d$$u7d$::h0bdff91ebad9d71a src/third_party/rust/rayon/src/iter/plumbing/mod.rs:418:21
    #57 0x7fa627a5f455 in rayon_core::join::join_context::call_a::_$u7b$$u7b$closure$u7d$$u7d$::h0a90fed1d90eb9f6 src/third_party/rust/rayon-core/src/join/mod.rs:124:17
    #58 0x7fa627a5f455 in _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h11bb5129af55f89e /builds/worker/fetches/rust/library/core/src/panic/unwind_safe.rs:271:9
    #59 0x7fa627a5f455 in std::panicking::try::do_call::h6080a873f99cc4e5 /builds/worker/fetches/rust/library/std/src/panicking.rs:403:40
    #60 0x7fa627a5f455 in std::panicking::try::h40c2f1c016c4dfb3 /builds/worker/fetches/rust/library/std/src/panicking.rs:367:19
    #61 0x7fa627a5f455 in std::panic::catch_unwind::h40c9990754be49bf /builds/worker/fetches/rust/library/std/src/panic.rs:129:14
    #62 0x7fa627a5f455 in rayon_core::unwind::halt_unwinding::h7471fbc0ba16f85d src/third_party/rust/rayon-core/src/unwind.rs:17:5
    #63 0x7fa627a5f455 in rayon_core::join::join_context::_$u7b$$u7b$closure$u7d$$u7d$::h770f75d7ee3fa002 src/third_party/rust/rayon-core/src/join/mod.rs:141:24
    #64 0x7fa627a604e2 in rayon_core::registry::in_worker::h4e106ab7d121351a src/third_party/rust/rayon-core/src/registry.rs:875:13
    #65 0x7fa627ae78f7 in rayon_core::join::join_context::hd698d4c3e8d29183 src/third_party/rust/rayon-core/src/join/mod.rs:132:5
    #66 0x7fa627ae78f7 in rayon::iter::plumbing::bridge_producer_consumer::helper::hbdff87fa1f93f71e src/third_party/rust/rayon/src/iter/plumbing/mod.rs:416:47
    #67 0x7fa627aee54f in rayon::iter::plumbing::bridge_producer_consumer::helper::_$u7b$$u7b$closure$u7d$$u7d$::hc3fb560020e50bd4 src/third_party/rust/rayon/src/iter/plumbing/mod.rs:427:21
    #68 0x7fa627aee54f in rayon_core::join::join_context::call_b::_$u7b$$u7b$closure$u7d$$u7d$::hf9baeb8f142826d5 src/third_party/rust/rayon-core/src/join/mod.rs:129:25
    #69 0x7fa627aee54f in _$LT$rayon_core..job..StackJob$LT$L$C$F$C$R$GT$$u20$as$u20$rayon_core..job..Job$GT$::execute::call::_$u7b$$u7b$closure$u7d$$u7d$::hd593021b281171fa src/third_party/rust/rayon-core/src/job.rs:113:21
    #70 0x7fa627aee54f in _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h1642f5dafb808850 /builds/worker/fetches/rust/library/core/src/panic/unwind_safe.rs:271:9
    #71 0x7fa627aee54f in std::panicking::try::do_call::h30eabcaae97f70d2 /builds/worker/fetches/rust/library/std/src/panicking.rs:403:40
    #72 0x7fa627aee54f in std::panicking::try::h0423b5c741d490f1 /builds/worker/fetches/rust/library/std/src/panicking.rs:367:19
    #73 0x7fa627aee54f in std::panic::catch_unwind::hcf5b7b41760afdd9 /builds/worker/fetches/rust/library/std/src/panic.rs:129:14
    #74 0x7fa627aee54f in rayon_core::unwind::halt_unwinding::h4697030baefde3a1 src/third_party/rust/rayon-core/src/unwind.rs:17:5
    #75 0x7fa627aee54f in _$LT$rayon_core..job..StackJob$LT$L$C$F$C$R$GT$$u20$as$u20$rayon_core..job..Job$GT$::execute::hf99af5423fc4a2df src/third_party/rust/rayon-core/src/job.rs:119:38
    #76 0x7fa61745032e in rayon_core::job::JobRef::execute::h74045f552c619ac1 src/third_party/rust/rayon-core/src/job.rs:59:9
    #77 0x7fa61745032e in rayon_core::registry::WorkerThread::execute::h37d6b3c282c8162e src/third_party/rust/rayon-core/src/registry.rs:749:9
    #78 0x7fa61745032e in rayon_core::registry::WorkerThread::wait_until_cold::h31d960cf9200795d src/third_party/rust/rayon-core/src/registry.rs:726:17
    #79 0x7fa62aaf651b in rayon_core::registry::WorkerThread::wait_until::h8e765de677ad8089 src/third_party/rust/rayon-core/src/registry.rs:700:13
    #80 0x7fa62aaf651b in rayon_core::registry::main_loop::h580ef685ddecc0c8 src/third_party/rust/rayon-core/src/registry.rs:833:5
    #81 0x7fa62aaf651b in rayon_core::registry::ThreadBuilder::run::haaeba247cf6e02cc src/third_party/rust/rayon-core/src/registry.rs:55:18
    #82 0x7fa62aaed26f in _$LT$rayon_core..registry..DefaultSpawn$u20$as$u20$rayon_core..registry..ThreadSpawn$GT$::spawn::_$u7b$$u7b$closure$u7d$$u7d$::h6761b6ccbae559d1 src/third_party/rust/rayon-core/src/registry.rs:100:20
    #83 0x7fa62aaed26f in std::sys_common::backtrace::__rust_begin_short_backtrace::h4f97be4da9afa348 /builds/worker/fetches/rust/library/std/src/sys_common/backtrace.rs:125:18
    #84 0x7fa62aaf01c6 in std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hed0d8f0c9e387cb6 /builds/worker/fetches/rust/library/std/src/thread/mod.rs:481:17
    #85 0x7fa62aaf01c6 in _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h89167dd2882d4624 /builds/worker/fetches/rust/library/core/src/panic/unwind_safe.rs:271:9
    #86 0x7fa62aaf01c6 in std::panicking::try::do_call::h1adeb507d8438ee2 /builds/worker/fetches/rust/library/std/src/panicking.rs:403:40
    #87 0x7fa62aaf01c6 in std::panicking::try::h90eaa657a7cba4f9 /builds/worker/fetches/rust/library/std/src/panicking.rs:367:19
    #88 0x7fa62aaf01c6 in std::panic::catch_unwind::h27c6bbf3f633e963 /builds/worker/fetches/rust/library/std/src/panic.rs:129:14
    #89 0x7fa62aaf01c6 in std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h90d1049625be4dd0 /builds/worker/fetches/rust/library/std/src/thread/mod.rs:480:30
    #90 0x7fa62aaf01c6 in core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h99fd14033cc91353 /builds/worker/fetches/rust/library/core/src/ops/function.rs:227:5
    #91 0x7fa62ac58942 in std::sys::unix::thread::Thread::new::thread_start::h3b1213720f18b702 std.ac3adaa7-cgu.2
    #92 0x7fa636fa8608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477:8
    #93 0x7fa636b70292 in __clone /build/glibc-eX1tMB/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:256:3 in MOZ_Crash
Thread T48 (WRWorkerLP#2) created by T0 (GeckoMain) here:
    #0 0x5598a63c8a8c in __interceptor_pthread_create /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cpp:207:3
    #1 0x7fa62ac587c4 in std::sys::unix::thread::Thread::new::h6fcfdf86716b7232 (/home/user/workspace/browsers/m-c-20211123215113-fuzzing-asan-opt/libxul.so+0x184b47c4)
    #2 0x7fa627afeb82 in rayon_core::registry::Registry::new::hd9460046afb5a48d src/third_party/rust/rayon-core/src/registry.rs:256:29
    #3 0x7fa627afeb82 in rayon_core::thread_pool::ThreadPool::build::h1e0f8cfb23d49845 src/third_party/rust/rayon-core/src/thread_pool/mod.rs:70:24
    #4 0x7fa627afeb82 in rayon_core::ThreadPoolBuilder$LT$S$GT$::build::h4319e46c3efafd4c src/third_party/rust/rayon-core/src/lib.rs:226:9
    #5 0x7fa627afeb82 in wr_thread_pool_new src/gfx/webrender_bindings/src/bindings.rs:1093:18
    #6 0x7fa61b249f9e in WebRenderThreadPool src/gfx/webrender_bindings/RenderThread.cpp:1078:17
    #7 0x7fa61b249f9e in mozilla::wr::RenderThread::RenderThread(RefPtr<nsIThread>) src/gfx/webrender_bindings/RenderThread.cpp:73:7
    #8 0x7fa61b24ab46 in mozilla::wr::RenderThread::Start() src/gfx/webrender_bindings/RenderThread.cpp:114:23
    #9 0x7fa61afebe3e in InitLayersIPC src/gfx/thebes/gfxPlatform.cpp:1291:7
    #10 0x7fa61afebe3e in gfxPlatform::Init() src/gfx/thebes/gfxPlatform.cpp:957:3
    #11 0x7fa61afeeeb0 in GetPlatform src/gfx/thebes/gfxPlatform.cpp:466:5
    #12 0x7fa61afeeeb0 in gfxPlatform::InitializeCMS() src/gfx/thebes/gfxPlatform.cpp:2084:9
    #13 0x7fa6200e05f9 in GetCMSMode /builds/worker/workspace/obj-build/dist/include/gfxPlatform.h:521:5
    #14 0x7fa6200e05f9 in nsXPLookAndFeel::GetColorValue(mozilla::StyleSystemColor, mozilla::ColorScheme, mozilla::LookAndFeel::UseStandins, unsigned int&) src/widget/nsXPLookAndFeel.cpp:866:9
    #15 0x7fa6200e420e in mozilla::LookAndFeel::GetColor(mozilla::StyleSystemColor, mozilla::ColorScheme, mozilla::LookAndFeel::UseStandins) src/widget/nsXPLookAndFeel.cpp:1211:47
    #16 0x7fa620057668 in Color /builds/worker/workspace/obj-build/dist/include/mozilla/LookAndFeel.h:449:12
    #17 0x7fa620057668 in ThemedAccentColor src/widget/ThemeColors.cpp:89:37
    #18 0x7fa620057668 in mozilla::widget::ThemeColors::RecomputeAccentColors() src/widget/ThemeColors.cpp:170:20
    #19 0x7fa62009a50a in nsNativeBasicTheme::LookAndFeelChanged() src/widget/nsNativeBasicTheme.cpp:123:3
    #20 0x7fa6200deae2 in nsXPLookAndFeel::GetInstance() src/widget/nsXPLookAndFeel.cpp:358:3
    #21 0x7fa6200e4c0d in mozilla::LookAndFeel::GetThemeInfo(nsTSubstring<char>&) src/widget/nsXPLookAndFeel.cpp:1328:3
    #22 0x7fa617f53947 in nsSystemInfo::Init() src/xpcom/base/nsSystemInfo.cpp:1041:5
    #23 0x7fa61805a4f4 in mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsISupports*, nsID const&, void**) /builds/worker/workspace/obj-build/xpcom/components/StaticComponents.cpp:9166:7
    #24 0x7fa6180a99e7 in CreateInstance src/xpcom/components/nsComponentManager.cpp:177:46
    #25 0x7fa6180a99e7 in nsComponentManagerImpl::GetServiceLocked(mozilla::Maybe<mozilla::MonitorAutoLock>&, (anonymous namespace)::EntryWrapper&, nsID const&, void**) src/xpcom/components/nsComponentManager.cpp:1276:17
    #26 0x7fa6180aa498 in nsComponentManagerImpl::GetService(mozilla::xpcom::ModuleID, nsID const&, void**) src/xpcom/components/nsComponentManager.cpp:1366:10
    #27 0x7fa61807e7bd in mozilla::xpcom::GetServiceHelper::operator()(nsID const&, void**) const /builds/worker/workspace/obj-build/xpcom/components/StaticComponents.cpp:12273:50
    #28 0x7fa617f10981 in nsCOMPtr_base::assign_from_helper(nsCOMPtr_helper const&, nsID const&) src/xpcom/base/nsCOMPtr.cpp:109:7
    #29 0x7fa61a47ab8c in nsCOMPtr /builds/worker/workspace/obj-build/dist/include/nsCOMPtr.h:999:5
    #30 0x7fa61a47ab8c in GetServiceImpl src/js/xpconnect/src/JSServices.cpp:84:32
    #31 0x7fa61a47ab8c in GetService src/js/xpconnect/src/JSServices.cpp:131:8
    #32 0x7fa61a47ab8c in xpc::Services_Resolve(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::PropertyKey>, bool*) src/js/xpconnect/src/JSServices.cpp:154:25
    #33 0x7fa625339177 in CallResolveOp src/js/src/vm/NativeObject-inl.h:634:8
    #34 0x7fa625339177 in NativeLookupOwnPropertyInline<js::CanGC, js::LookupResolveMode::CheckResolve> src/js/src/vm/NativeObject-inl.h:751:14
    #35 0x7fa625339177 in NativeGetPropertyInline<js::CanGC> src/js/src/vm/NativeObject.cpp:2099:10
    #36 0x7fa625339177 in js::NativeGetProperty(JSContext*, JS::Handle<js::NativeObject*>, JS::Handle<JS::Value>, JS::Handle<JS::PropertyKey>, JS::MutableHandle<JS::Value>) src/js/src/vm/NativeObject.cpp:2147:10
    #37 0x7fa624e30359 in GetProperty src/js/src/vm/ObjectOperations-inl.h:115:10
    #38 0x7fa624e30359 in js::GetProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, js::PropertyName*, JS::MutableHandle<JS::Value>) src/js/src/vm/ObjectOperations-inl.h:122:10
    #39 0x7fa624e2f9b4 in js::GetProperty(JSContext*, JS::Handle<JS::Value>, JS::Handle<js::PropertyName*>, JS::MutableHandle<JS::Value>) src/js/src/vm/Interpreter.cpp:4548:10
    #40 0x7fa624e007bd in GetPropertyOperation src/js/src/vm/Interpreter.cpp:204:10
    #41 0x7fa624e007bd in Interpret(JSContext*, js::RunState&) src/js/src/vm/Interpreter.cpp:2907:12
    #42 0x7fa624df8011 in js::RunScript(JSContext*, js::RunState&) src/js/src/vm/Interpreter.cpp:357:13
    #43 0x7fa624e26ddc in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) src/js/src/vm/Interpreter.cpp:507:13
    #44 0x7fa624e28f2b in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason) src/js/src/vm/Interpreter.cpp:552:8
    #45 0x7fa62509af3c in JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) src/js/src/vm/CallAndConstruct.cpp:53:10
    #46 0x7fa61a4c1908 in nsXPCWrappedJS::CallMethod(unsigned short, nsXPTMethodInfo const*, nsXPTCMiniVariant*) src/js/xpconnect/src/XPCWrappedJSClass.cpp:973:17
    #47 0x7fa61814ee02 in PrepareAndDispatch src/xpcom/reflect/xptcall/md/unix/xptcstubs_x86_64_linux.cpp:115:37
    #48 0x7fa61814db8a in SharedStub xptcstubs_x86_64_linux.cpp
    #49 0x7fa61809fdd2 in NS_CreateServicesFromCategory(char const*, nsISupports*, char const*, char16_t const*) src/xpcom/components/nsCategoryManager.cpp:687:19
    #50 0x7fa624b5f5c9 in nsXREDirProvider::DoStartup() src/toolkit/xre/nsXREDirProvider.cpp:976:11
    #51 0x7fa624b3a833 in XREMain::XRE_mainRun() src/toolkit/xre/nsAppRunner.cpp:5044:18
    #52 0x7fa624b3db39 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) src/toolkit/xre/nsAppRunner.cpp:5494:8
    #53 0x7fa624b3e873 in XRE_main(int, char**, mozilla::BootstrapConfig const&) src/toolkit/xre/nsAppRunner.cpp:5553:21
    #54 0x5598a64136d9 in do_main src/browser/app/nsBrowserApp.cpp:225:22
    #55 0x5598a64136d9 in main src/browser/app/nsBrowserApp.cpp:395:16
    #56 0x7fa636a750b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
Flags: in-testsuite?

Bugmon Analysis
Verified bug as reproducible on mozilla-central 20211128213906-34a7b5b9dbec.
Failed to bisect testcase (Testcase reproduces on start build!):

Start: b0865ea584621ce9e7f68833565e3d8ae117ce32 (20201130093031)
End: ba4d4963c38ba7a68e481d39b5b1a3698e5098d9 (20211123033957)
BuildFlags: BuildFlags(asan=True, tsan=False, debug=False, fuzzing=True, coverage=False, valgrind=False, no_opt=False, fuzzilli=False)

Whiteboard: [bugmon:bisected,confirmed]

The relevant line from the stack trace:

https://searchfox.org/mozilla-central/rev/21f0bb70ce5747d18625ad1c67f03ed2af7f9c56/gfx/2d/RecordedEventImpl.h#2423

I would guess it is because we are getting a ton of warnings, but we don't rate limit how much can be stored in mozilla::gfx::Log::mMessage:

https://searchfox.org/mozilla-central/rev/21f0bb70ce5747d18625ad1c67f03ed2af7f9c56/gfx/2d/Logging.h#761

Blocks: wr-oom
Severity: -- → S3

To help us direct our efforts accordingly can you please help categorize the impact of this fix. For example is this likely to benefit end users (improve performance, avoid OOM, etc) or only unblock testing/fuzzing? Thank you!

Flags: needinfo?(aosmond)

Reviewing the code, it should be releasing the buffer I am worried about during the calls. I don't exactly understand what happened here to cause such a large allocation, however reviewing the crash stats, there doesn't appear to be many OOMs related to the graphics logging, so I don't think this is a priority to investigate/fix from a user perspective.

Flags: needinfo?(aosmond)

The attached test case still reproduces the issue. The issue is not present on Chrome.

From a fuzzing perspective this is pretty easily triggered. Addressing this issue will help fuzzing efforts.

status-firefox103: --- → wontfix
status-firefox104: --- → affected
status-firefox105: --- → affected
status-firefox96: affected → wontfix
status-firefox-esr102: --- → affected
status-firefox-esr91: --- → affected
Keywords: csectype-oom
Whiteboard: [bugmon:bisected,confirmed] → [bugmon:bisected,confirmed][fuzzblocker]

This bug prevents fuzzing from making progress; however, it has low severity. It is important for fuzz blocker bugs to be addressed in a timely manner (see here why?).
:bhood, could you increase the severity?

For more information, please visit auto_nag documentation.

Flags: needinfo?(bhood)

Nical, please have a look and see if there's anything we might be able to do to move this forward.

Flags: needinfo?(bhood) → needinfo?(nical.bugzilla)

So I don't think the Logging code is to blame here. We get into a state where we run short on memory, short enough to fail a very small allocation at https://searchfox.org/mozilla-central/rev/21f0bb70ce5747d18625ad1c67f03ed2af7f9c56/gfx/2d/RecordedEventImpl.h#2423 (there's only 1 glyh here), and as we log our failure we get into a state where logging isn't even possible at all.

We could:

  • look into memory allocations in blob the record/replay code in general and see if we can lower them (to make things better in general)
  • mage the logging code allocations fallible or avoid using gfxCriticalNote to report failure to perform very small allocations (to avoid crashing in there but likely we'll crash elswhere, there's too many small infallible allocations everywhere.

The blob image recordings are in contiguous buffers and the recorded events we created from it aren't outliving the buffer. In a bunch of places we malloc and copy data from the buffer but we could have some of the recorded events point directly into the buffer itself instead.
glyph indices are one of these but paths would probably avoid more memory allocations.

Priority: -- → P3
Attached file heap_profile.txt —

Dropping this on Nical's plate just to make sure it's assigned.

Assignee: nobody → nical.bugzilla
Attached file testcase_2.html —

Adding a few more test cases, requested by nical.

Attached file testcase_3.html —
Attached file testcase_4.html —

Thanks for the new test cases.

Important takeaway from a discussion with Tyson on matrix: What constitutes the common element of the aggregation of OOMs that are associated to this bug is not the stack trace of the crash in comment 0, but rather is the amount of allocations happening within of Moz2dBlobRasterizer::rasterize. Typically these are the tile buffers we rasterize the content into, or allocations made in skia. They tend to grow with the number of pixels that need to be rendered.
In other words, forget about all of the discussion about optimizing out memory allocations in blob recordings in comments from 0 to 9 included, it was bad luck that the stack trace of the initial report landed there, but the real source of trouble is mostly the number of pixels that we need to rasterize.

Looking at the various test cases there appear to be several classes of issues which we can follow up on in separate bugs:

Blob layer bounds are too large

We request few layers but they are enormous. This probably means we failed to clip the bounds of whatever we are rasterizing against the display port.
This is the case of the attached test case 3. which requests 5204364 tiles. In that particular test case it is the combination of the scale and the rotation that for some reason causes us to fail to clip the bounds, and the enormous scale factors causes the huge surface area. This test typically fails when we enumerate the tile requests, even before we try to allocate space for them.

There are lots of rectangle parameters involved with blobs, the one that matters here is the blob image "visible area".

Too many blob layers

If hundreds of layers stack up, it will blow up memory even if the layers aren't unreasonably large. This is the case of the attached test case 2, with 300 layers. (each "A" adds a layer, the video element in the test case can be ignored).

Rasterizing the tiles require a lot of intermediate allocations within skia

That's test case 4 with few tiles (36 for me), but the large blur radius (282) causes each tile to preallocate a larger intermediate image to render and blur from. This class of issue is going to be hard to completely remove, because we have relatively low control over skia's allocations and no control over the shape of the content, but in the case of large blurs the smaller the blob tile size the more overhead, so right now with 256x256 tiles and a blur radius of 282 we need intermediate buffers that are at least 10x than the destination tile buffer. if we moved to 512x512 we would require "only" at least 4.5x the memory which is something.
I tried to increase the tile size to 512x512 globally in bug 1787706, but it got backed as it was regressing the performance of some content. The tile size is selected per blob, so we could track whether there are (large) blurs within a blob and increase the tile size if so.

Flags: needinfo?(nical.bugzilla)

We won't be able to put a strict limit on the memory allocated by blob images while still be able to render them, we can only improve on the pathological cases where we fail very badly. So if despite improving things the fuzzers are still hitting this type of issue too much, we could add a pref that skips blob images after a certain amount of tiles (for fuzzing only since we wouldn't render properly).

jrmuizel noted on matrix that some of these test cases should probably not use blobs at all so, there may also be an opportunity to reduce the volume of the issue by doign a better job at not falling back to blob images (to some extent).

Attached file oom_tests.zip —

As suggested by :nical on matrix, I've zipped a few more test cases that have been reduced. I'm not sure if they share a root cause with any others.

Once we start landing patches I will reduce what the fuzzers continue to report.

Depends on: 1815272
Depends on: 1815397
Attached file testcase_5.html —

Another silly test case that fuzzers can easily generate.

This triggers high memory usage with gfx.webrender.debug.restrict-blob-size=true.

Testcase crashes using the initial build (mozilla-central 20220507095414-1d8e24107902) but not with tip (mozilla-central 20230506092548-ca770a49d132.)

The bug appears to have been fixed in the following build range:

Start: fd7522f1263a0d073f9595b351a867516af44d5c (20230215213831)
End: 363d2f6f523a2dd3f4239a8b5a21b4a0a7b54152 (20230215222001)
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=fd7522f1263a0d073f9595b351a867516af44d5c&tochange=363d2f6f523a2dd3f4239a8b5a21b4a0a7b54152

nical, can you confirm that the above bisection range is responsible for fixing this issue?
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.

Flags: needinfo?(nical.bugzilla)
Keywords: bugmon

The range contains a pref that lets the fuzzers paper over the problem. The bug has not been fixed yet, we just put in a way to prevent it from tripping the fuzzers constantly.

Flags: needinfo?(nical.bugzilla)

I need to add some recent test cases.

Flags: needinfo?(twsmith)
Attached file testcase_6.html —
Flags: needinfo?(twsmith)

As a fuzzblocker, should this be prioritized higher? S2?

Flags: needinfo?(bhood)
Severity: S3 → S2
Flags: needinfo?(bhood)
Depends on: 1884959
Severity: S2 → S3
Priority: P3 → P1

fuzz blockers aren't really S2 so mark as P1 instead

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: