1981021 - firefox: src/gl.cc:537: bool Texture::allocate(bool, int, int): Assertion `new_buf' failed.

Status: NEW

(Core :: Graphics: WebRender, defect)

Description

[Jason Kratzer [:jkratzer]]

Testcase found while fuzzing mozilla-central rev fbf32fb7861d (built with: --enable-debug --enable-fuzzing).

Testcase can be reproduced using the following commands:

$ pip install fuzzfetch grizzly-framework --upgrade
$ python -m fuzzfetch --build fbf32fb7861d --debug --fuzzing --cpu x86 -n firefox
$ python -m grizzly.replay.bugzilla ./firefox/firefox <bugid>

firefox: src/gl.cc:537: bool Texture::allocate(bool, int, int): Assertion `new_buf' failed.

    ==699975==ERROR: UndefinedBehaviorSanitizer: ABRT on unknown address 0x000aae47 (pc 0xf58b1579 bp 0xd6cffb40 sp 0xd6cfcd80 T700057)
        #0 0xf58b1579  (linux-gate.so.1+0x579) (BuildId: e834b606352cb3b6b18491f1b050513f917b912c)
        #1 0xf52faf86  (/lib/i386-linux-gnu/libc.so.6+0x8ef86) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
        #2 0xf52a80b4 in raise (/lib/i386-linux-gnu/libc.so.6+0x3c0b4) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
        #3 0xf528f2be in abort (/lib/i386-linux-gnu/libc.so.6+0x232be) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
        #4 0xf528f1c1  (/lib/i386-linux-gnu/libc.so.6+0x231c1) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
        #5 0xf529ff2e in __assert_fail (/lib/i386-linux-gnu/libc.so.6+0x33f2e) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
        #6 0xedbad9cc in Texture::allocate(bool, int, int) /gfx/wr/swgl/src/gl.cc
        #7 0xedb8d577 in set_tex_storage(Texture&, unsigned int, int, int, void*, int, int, int) /gfx/wr/swgl/src/gl.cc:1900:10
        #8 0xedb8fc79 in InitDefaultFramebuffer /gfx/wr/swgl/src/gl.cc:2568:3
        #9 0xedb83289 in swgl::swgl_fns::Context::init_default_framebuffer::hcd9a03577bdf6da3 /gfx/wr/swgl/src/swgl_fns.rs:364:13
        #10 0xed4634c2 in wr_swgl_init_default_framebuffer /gfx/webrender_bindings/src/swgl_bindings.rs:39:5
        #11 0xe56fb79f in mozilla::wr::RenderCompositorSWGL::AllocateMappedBuffer(mozilla::wr::Box2D<int, mozilla::wr::DevicePixel> const*, unsigned int) /gfx/webrender_bindings/RenderCompositorSWGL.cpp:143:3
        #12 0xe56fc3d0 in mozilla::wr::RenderCompositorSWGL::StartCompositing(mozilla::wr::ColorF, mozilla::wr::Box2D<int, mozilla::wr::DevicePixel> const*, unsigned int, mozilla::wr::Box2D<int, mozilla::wr::DevicePixel> const*, unsigned int) /gfx/webrender_bindings/RenderCompositorSWGL.cpp:191:8
        #13 0xe56f2d64 in wr_compositor_start_compositing /gfx/webrender_bindings/RenderCompositor.cpp:145:15
        #14 0xed44ac4b in _$LT$webrender_bindings..bindings..WrCompositor$u20$as$u20$webrender..composite..Compositor$GT$::start_compositing::h95b11c6ff83c0022 /gfx/webrender_bindings/src/bindings.rs:1482:13
        #15 0xed7d6541 in _$LT$webrender..compositor..sw_compositor..SwCompositor$u20$as$u20$webrender..composite..Compositor$GT$::start_compositing::he8ee1d11aea87716 /gfx/wr/webrender/src/compositor/sw_compositor.rs:1938:9
        #16 0xed91c8c1 in webrender::renderer::_$LT$impl$u20$webrender..composite..CompositeState$GT$::composite_native::h6b566b20f4df8506 /gfx/wr/webrender/src/renderer/mod.rs:6617:9
        #17 0xed91c8c1 in webrender::renderer::Renderer::draw_frame::h6691a16dcfadf263 /gfx/wr/webrender/src/renderer/mod.rs:5270:17
        #18 0xed8ed399 in webrender::renderer::Renderer::render_impl::h2d40b22845fa53cc /gfx/wr/webrender/src/renderer/mod.rs:1655:17
        #19 0xed8eabea in webrender::renderer::Renderer::render::habf76d630c7e58b9 /gfx/wr/webrender/src/renderer/mod.rs:1344:30
        #20 0xed44561c in wr_renderer_render /gfx/webrender_bindings/src/bindings.rs:650:11
        #21 0xe5706e6a in mozilla::wr::RendererOGL::UpdateAndRender(mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>> const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char>> const&, bool*, mozilla::wr::FrameReadyParams const&, mozilla::wr::RendererStats*) /gfx/webrender_bindings/RendererOGL.cpp:220:19
        #22 0xe5705cda in mozilla::wr::RenderThread::UpdateAndRender(mozilla::wr::WrWindowId, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, mozilla::wr::FrameReadyParams const&, mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>> const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char>> const&, mozilla::wr::RendererStats*, bool*) /gfx/webrender_bindings/RenderThread.cpp:849:31
        #23 0xe5705236 in mozilla::wr::RenderThread::HandleFrameOneDocInner(mozilla::wr::WrWindowId, mozilla::wr::FrameReadyParams const&, mozilla::Maybe<mozilla::wr::FramePublishId>) /gfx/webrender_bindings/RenderThread.cpp:663:3
        #24 0xe57048b6 in mozilla::wr::RenderThread::HandleFrameOneDoc(mozilla::wr::WrWindowId, mozilla::wr::FrameReadyParams const&, mozilla::Maybe<mozilla::wr::FramePublishId>) /gfx/webrender_bindings/RenderThread.cpp:611:3
        #25 0xe5704377 in WrNotifierEvent_HandleNewFrameReady /gfx/webrender_bindings/RenderThread.cpp:573:3
        #26 0xe5704377 in mozilla::wr::RenderThread::HandleWrNotifierEvents(mozilla::wr::WrWindowId) /gfx/webrender_bindings/RenderThread.cpp:536:9
        #27 0xe56be0bc in operator()<StoreCopyPassByConstLRef<mozilla::TimeStamp> &> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1085:18
        #28 0xe56be0bc in __invoke_impl<void, (lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1084:9), StoreCopyPassByConstLRef<mozilla::TimeStamp> &> /builds/worker/fetches/sysroot-i686-linux-gnu/usr/lib/gcc/i586-linux-gnu/8/../../../../include/c++/8/bits/invoke.h:60:14
        #29 0xe56be0bc in __invoke<(lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1084:9), StoreCopyPassByConstLRef<mozilla::TimeStamp> &> /builds/worker/fetches/sysroot-i686-linux-gnu/usr/lib/gcc/i586-linux-gnu/8/../../../../include/c++/8/bits/invoke.h:95:14
        #30 0xe56be0bc in __apply_impl<(lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1084:9), std::tuple<StoreCopyPassByConstLRef<mozilla::TimeStamp> > &, 0U> /builds/worker/fetches/sysroot-i686-linux-gnu/usr/lib/gcc/i586-linux-gnu/8/../../../../include/c++/8/tuple:1678:14
        #31 0xe56be0bc in apply<(lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1084:9), std::tuple<StoreCopyPassByConstLRef<mozilla::TimeStamp> > &> /builds/worker/fetches/sysroot-i686-linux-gnu/usr/lib/gcc/i586-linux-gnu/8/../../../../include/c++/8/tuple:1687:14
        #32 0xe56be0bc in apply<mozilla::gfx::VRThread, void (mozilla::gfx::VRThread::*)(mozilla::TimeStamp)> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1083:12
        #33 0xe56be0bc in mozilla::detail::RunnableMethodImpl<mozilla::gmp::ChromiumCDMChild*, void (mozilla::gmp::ChromiumCDMChild::*)(bool (mozilla::gmp::PChromiumCDMChild::*)()), true, (mozilla::RunnableKind)0, bool (mozilla::gmp::PChromiumCDMChild::*)()>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1134:13
        #34 0xe3ea7417 in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1153:16
        #35 0xe3eadaea in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:480:10
        #36 0xe4a7c485 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:329:5
        #37 0xe49d89be in MessageLoop::RunInternal() /ipc/chromium/src/base/message_loop.cc:369:10
        #38 0xe49d88ca in RunHandler /ipc/chromium/src/base/message_loop.cc:362:3
        #39 0xe49d88ca in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:344:3
        #40 0xe3ea2f16 in nsThread::ThreadFunc(void*) /xpcom/threads/nsThread.cpp:366:10
        #41 0xf4a504fe in _pt_root /nsprpub/pr/src/pthreads/ptthread.c:191:3
        #42 0xf52f8fe6  (/lib/i386-linux-gnu/libc.so.6+0x8cfe6) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
        #43 0xf53905a7  (/lib/i386-linux-gnu/libc.so.6+0x1245a7) (BuildId: 3b5c19cb483b5eb0b059104bbca1175bea99a609)
    
    ==699975==Register values:
    eax = 0x00000000  ebx = 0x000aae47  ecx = 0x000aae99  edx = 0x00000006  
    edi = 0x00000000  esi = 0x000aae99  ebp = 0xd6cffb40  esp = 0xd6cfcd80  
    UndefinedBehaviorSanitizer can not provide additional info.
    SUMMARY: UndefinedBehaviorSanitizer: ABRT (linux-gate.so.1+0x579) (BuildId: e834b606352cb3b6b18491f1b050513f917b912c) 
    ==699975==ABORTING

Comment 1

[Jason Kratzer [:jkratzer]]

Attachment: Detailed Crash Information

Comment 2

[Jason Kratzer [:jkratzer]]

Attachment: Testcase

Comment 3

[Bugmon [:jkratzer for issues]]

Verified bug as reproducible on mozilla-central 20250804213332-d15f84a81995.
Unable to bisect testcase (Testcase reproduces on start build!):

Start: aef973f394fddbb97537835d6b23bdc89ccad38e (20240806160017)
End: fbf32fb7861d75ca9d6a264e54b69b4a51698a24 (20250804155755)
BuildFlags: BuildFlags(asan=False, tsan=False, debug=True, fuzzing=True, coverage=False, valgrind=False, no_opt=False, fuzzilli=False, nyx=False, searchfox=False, afl=False)

Comment 4

[Lee Salzman [:lsalzman]]

This just looks like an excessively large allocation that is, as expected by the assert, failing. It doesn't really represent anything that bad that I can see from the testcase. In release builds, it will trigger a GL OUT_OF_MEMORY error which will be propagated up to WebRender to handle however it wants, which is the correct scenario.

I am marking this as an S4 unless there's really some burning need to remove this assert.