1716369 - high memory usage in [@ webrender::api_resources::ApiResources::create_blob_scene_builder_requests]

Status: NEW

(Core :: Graphics: WebRender, defect)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.html

Found while fuzzing m-c 20210614-e77eb14241b9 (--enable-address-sanitizer --enable-fuzzing)

To help catch this issue ASAN_OPTIONS=max_allocation_size_mb=512 was used. See Bug 1715316 for details about fuzzing triggered OOMs.

==43327==WARNING: AddressSanitizer failed to allocate 0x38000000 bytes
=================================================================
==43327==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x563e244fcb48 bp 0x7f1f36448e70 sp 0x7f1f36448e60 T40)
==43327==The signal is caused by a WRITE memory access.
==43327==Hint: address points to the zero page.
    #0 0x563e244fcb48 in mozalloc_abort src/memory/mozalloc/mozalloc_abort.cpp:33:3
    #1 0x563e244fccdd in mozalloc_handle_oom(unsigned long) src/memory/mozalloc/mozalloc_oom.cpp:51:3
    #2 0x7f1f6674ae46 in gkrust_shared::oom_hook::hook::h9889a70667e22002 src/toolkit/library/rust/shared/lib.rs:134:13
    #3 0x7f1f67f9a707 in rust_oom /rustc/74f7e32f43b5fb0f83896d124566d8242eb786b1/library/std/src/alloc.rs:330:5
    #4 0x7f1f655bd395 in __rg_oom /rustc/74f7e32f43b5fb0f83896d124566d8242eb786b1/library/alloc/src/alloc.rs:409:18
    #5 0x7f1f65579a05 in __rust_alloc_error_handler (/home/user/workspace/browsers/m-c-20210614095307-fuzzing-asan-opt/libxul.so+0x12b99a05)
    #6 0x7f1f655bd385 in alloc::alloc::handle_alloc_error::h1ec3a24ddd4da47f /rustc/74f7e32f43b5fb0f83896d124566d8242eb786b1/library/alloc/src/alloc.rs:363:9
    #7 0x7f1f69bb5ab9 in alloc::raw_vec::handle_reserve::hbed63b783594c68e /builds/worker/fetches/rustc/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:515:43
    #8 0x7f1f69bb5ab9 in alloc::raw_vec::RawVec$LT$T$C$A$GT$::reserve::h18567e64b2aa0f60 /builds/worker/fetches/rustc/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:311:9
    #9 0x7f1f69bb5ab9 in alloc::vec::Vec$LT$T$GT$::reserve::h13670b5d2399fcef /builds/worker/fetches/rustc/lib/rustlib/src/rust/library/alloc/src/vec.rs:505:9
    #10 0x7f1f69bb5ab9 in alloc::vec::Vec$LT$T$GT$::push::he9f06bf36c400742 /builds/worker/fetches/rustc/lib/rustlib/src/rust/library/alloc/src/vec.rs:1210:13
    #11 0x7f1f69bb5ab9 in webrender::api_resources::ApiResources::create_blob_scene_builder_requests::_$u7b$$u7b$closure$u7d$$u7d$::h1e5192af952ab1cc src/gfx/wr/webrender/src/api_resources.rs:282:17
    #12 0x7f1f69bb5ab9 in webrender::image_tiling::for_each_tile_in_range::h7e1fc5c33e1b0de5 src/gfx/wr/webrender/src/image_tiling.rs:587:13
    #13 0x7f1f69bb5ab9 in webrender::api_resources::ApiResources::create_blob_scene_builder_requests::h3ef80b11798c1a3f src/gfx/wr/webrender/src/api_resources.rs:263:13
    #14 0x7f1f69bb5ab9 in webrender::api_resources::ApiResources::update::h0301c4ad510ef4e2 src/gfx/wr/webrender/src/api_resources.rs:167:38
    #15 0x7f1f69bb5ab9 in webrender::render_api::RenderApi::send_transaction::hd0d620a4f6ff118c src/gfx/wr/webrender/src/render_api.rs:1242:9
    #16 0x7f1f69f2c016 in wr_api_send_transaction src/gfx/webrender_bindings/src/bindings.rs:2136:5
    #17 0x7f1f5a53e90c in mozilla::layers::WebRenderBridgeParent::SetDisplayList(mozilla::gfx::RectTyped<mozilla::LayoutDevicePixel, float> const&, mozilla::ipc::ByteBuf&&, mozilla::wr::BuiltDisplayListDescriptor const&, nsTArray<mozilla::layers::OpUpdateResource> const&, nsTArray<mozilla::layers::RefCountedShmem> const&, nsTArray<mozilla::ipc::Shmem> const&, mozilla::TimeStamp const&, mozilla::wr::TransactionBuilder&, mozilla::wr::Epoch, bool) src/gfx/layers/wr/WebRenderBridgeParent.cpp:1137:9
    #18 0x7f1f5a53f3ec in mozilla::layers::WebRenderBridgeParent::ProcessDisplayListData(mozilla::layers::DisplayListData&, mozilla::wr::Epoch, mozilla::TimeStamp const&, bool, bool) src/gfx/layers/wr/WebRenderBridgeParent.cpp:1170:8
    #19 0x7f1f5a5407f4 in mozilla::layers::WebRenderBridgeParent::RecvSetDisplayList(mozilla::layers::DisplayListData&&, nsTArray<mozilla::layers::OpDestroy>&&, unsigned long const&, mozilla::layers::BaseTransactionId<mozilla::layers::TransactionIdType> const&, bool const&, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, nsTString<char> const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::CompositionPayload>&&) src/gfx/layers/wr/WebRenderBridgeParent.cpp:1218:8
    #20 0x7f1f598807ee in mozilla::layers::PWebRenderBridgeParent::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PWebRenderBridgeParent.cpp:403:28
    #21 0x7f1f590d06d6 in mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PCompositorManagerParent.cpp:200:32
    #22 0x7f1f58e9509a in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) src/ipc/glue/MessageChannel.cpp:2155:25
    #23 0x7f1f58e917c8 in mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) src/ipc/glue/MessageChannel.cpp:2079:9
    #24 0x7f1f58e93125 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) src/ipc/glue/MessageChannel.cpp:1924:3
    #25 0x7f1f58e93c8b in mozilla::ipc::MessageChannel::MessageTask::Run() src/ipc/glue/MessageChannel.cpp:1955:13
    #26 0x7f1f57ca0e2a in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1153:16
    #27 0x7f1f57cab33c in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:548:10
    #28 0x7f1f58e9e120 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:330:5
    #29 0x7f1f58da3cb1 in RunInternal src/ipc/chromium/src/base/message_loop.cc:335:10
    #30 0x7f1f58da3cb1 in RunHandler src/ipc/chromium/src/base/message_loop.cc:328:3
    #31 0x7f1f58da3cb1 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:310:3
    #32 0x7f1f57c9a8e8 in nsThread::ThreadFunc(void*) src/xpcom/threads/nsThread.cpp:395:10
    #33 0x7f1f75acc3fe in _pt_root src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #34 0x7f1f79bea608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477:8
    #35 0x7f1f797b3292 in clone /build/glibc-eX1tMB/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV src/memory/mozalloc/mozalloc_abort.cpp:33:3 in mozalloc_abort
Thread T40 (Compositor) created by T0 here:
    #0 0x563e244ace6c in pthread_create /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cpp:205:3
    #1 0x7f1f75abc474 in _PR_CreateThread src/nsprpub/pr/src/pthreads/ptthread.c:458:14
    #2 0x7f1f75aad94e in PR_CreateThread src/nsprpub/pr/src/pthreads/ptthread.c:533:12
    #3 0x7f1f57c9d2ba in nsThread::Init(nsTSubstring<char> const&) src/xpcom/threads/nsThread.cpp:613:18
    #4 0x7f1f57ca8de6 in nsThreadManager::NewNamedThread(nsTSubstring<char> const&, unsigned int, nsIThread**) src/xpcom/threads/nsThreadManager.cpp:574:12
    #5 0x7f1f57cb3f41 in NS_NewNamedThread(nsTSubstring<char> const&, nsIThread**, already_AddRefed<nsIRunnable>, unsigned int) src/xpcom/threads/nsThreadUtils.cpp:169:57
    #6 0x7f1f5a8abba9 in NS_NewNamedThread<11> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:74:10
    #7 0x7f1f5a8abba9 in mozilla::layers::CompositorThreadHolder::CreateCompositorThread() src/gfx/layers/ipc/CompositorThread.cpp:55:17
    #8 0x7f1f5a8ac166 in CompositorThreadHolder src/gfx/layers/ipc/CompositorThread.cpp:39:25
    #9 0x7f1f5a8ac166 in mozilla::layers::CompositorThreadHolder::Start() src/gfx/layers/ipc/CompositorThread.cpp:94:33
    #10 0x7f1f5a941ee8 in gfxPlatform::Init() src/gfx/thebes/gfxPlatform.cpp:965:3
    #11 0x7f1f5a94094b in gfxPlatform::GetPlatform() src/gfx/thebes/gfxPlatform.cpp:481:5
    #12 0x7f1f5f61920c in mozilla::widget::GfxInfoBase::GetContentBackend(nsTSubstring<char16_t>&) src/widget/GfxInfoBase.cpp:1851:25
    #13 0x7f1f57ceb791 in NS_InvokeByIndex src/xpcom/reflect/xptcall/md/unix/xptcinvoke_asm_x86_64_unix.S:101
    #14 0x7f1f59ca6e59 in Invoke src/js/xpconnect/src/XPCWrappedNative.cpp:1644:10
    #15 0x7f1f59ca6e59 in Call src/js/xpconnect/src/XPCWrappedNative.cpp:1197:19
    #16 0x7f1f59ca6e59 in XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) src/js/xpconnect/src/XPCWrappedNative.cpp:1143:23
    #17 0x7f1f59cac43f in GetAttribute src/js/xpconnect/src/xpcprivate.h:1460:12
    #18 0x7f1f59cac43f in XPC_WN_GetterSetter(JSContext*, unsigned int, JS::Value*) src/js/xpconnect/src/XPCWrappedNativeJSOps.cpp:962:10
    #19 0x7f1f63b1af92 in CallJSNative src/js/src/vm/Interpreter.cpp:426:13
    #20 0x7f1f63b1af92 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) src/js/src/vm/Interpreter.cpp:511:12
    #21 0x7f1f63b1cccb in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason) src/js/src/vm/Interpreter.cpp:588:8
    #22 0x7f1f63b1e14b in js::CallGetter(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::MutableHandle<JS::Value>) src/js/src/vm/Interpreter.cpp:713:10
    #23 0x7f1f640343ed in CallGetter src/js/src/vm/NativeObject.cpp:2101:12
    #24 0x7f1f640343ed in GetExistingProperty<js::CanGC> src/js/src/vm/NativeObject.cpp:2129:12
    #25 0x7f1f640343ed in NativeGetPropertyInline<js::CanGC> src/js/src/vm/NativeObject.cpp:2274:14
    #26 0x7f1f640343ed in js::NativeGetProperty(JSContext*, JS::Handle<js::NativeObject*>, JS::Handle<JS::Value>, JS::Handle<JS::PropertyKey>, JS::MutableHandle<JS::Value>) src/js/src/vm/NativeObject.cpp:2305:10
    #27 0x7f1f63b0954c in GetProperty src/js/src/vm/ObjectOperations-inl.h:116:10
    #28 0x7f1f63b0954c in GetObjectElementOperation src/js/src/vm/Interpreter-inl.h:419:10
    #29 0x7f1f63b0954c in GetElementOperationWithStackIndex src/js/src/vm/Interpreter-inl.h:505:10
    #30 0x7f1f63b0954c in Interpret(JSContext*, js::RunState&) src/js/src/vm/Interpreter.cpp:3058:12
    #31 0x7f1f63aec196 in js::RunScript(JSContext*, js::RunState&) src/js/src/vm/Interpreter.cpp:395:13
    #32 0x7f1f63b1b0cb in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) src/js/src/vm/Interpreter.cpp:543:13
    #33 0x7f1f63b1cccb in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason) src/js/src/vm/Interpreter.cpp:588:8
    #34 0x7f1f64389fb0 in JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) src/js/src/jsapi.cpp:2722:10
    #35 0x7f1f59c989a9 in nsXPCWrappedJS::CallMethod(unsigned short, nsXPTMethodInfo const*, nsXPTCMiniVariant*) src/js/xpconnect/src/XPCWrappedJSClass.cpp:971:17
    #36 0x7f1f57ced122 in PrepareAndDispatch src/xpcom/reflect/xptcall/md/unix/xptcstubs_x86_64_linux.cpp:115:37
    #37 0x7f1f57cebeaa in SharedStub (/home/user/workspace/browsers/m-c-20210614095307-fuzzing-asan-opt/libxul.so+0x530beaa)
    #38 0x7f1f57c4d4a0 in NS_CreateServicesFromCategory(char const*, nsISupports*, char const*, char16_t const*) src/xpcom/components/nsCategoryManager.cpp:687:19
    #39 0x7f1f638e08e7 in nsXREDirProvider::DoStartup() src/toolkit/xre/nsXREDirProvider.cpp:982:11
    #40 0x7f1f638bd6a4 in XREMain::XRE_mainRun() src/toolkit/xre/nsAppRunner.cpp:4988:18
    #41 0x7f1f638c061e in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) src/toolkit/xre/nsAppRunner.cpp:5437:8
    #42 0x7f1f638c1373 in XRE_main(int, char**, mozilla::BootstrapConfig const&) src/toolkit/xre/nsAppRunner.cpp:5496:21
    #43 0x563e244f715a in do_main src/browser/app/nsBrowserApp.cpp:224:22
    #44 0x563e244f715a in main src/browser/app/nsBrowserApp.cpp:351:16
    #45 0x7f1f796b80b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16

Comment 1

[Jeff Muizelaar [:jrmuizel]]

I suspect we're getting a really big visible area and don't deal with that very gracefully.

Comment 2

[Tyson Smith [:tsmith]]

To help us direct our efforts accordingly can you please help categorize the impact of this fix. For example is this likely to benefit end users (improve performance, avoid OOM, etc) or only unblock testing/fuzzing? Thank you!

Comment 3

[Tyson Smith [:tsmith]]

Attachment: testcase.html

This testcase attempts a 14GB allocation.

Comment 4

[Tyson Smith [:tsmith]]

Attachment: heap_profile.txt

This test case also triggers an OOM with the allocation size limit removed.