1876380 - Stack overflow on youtube.com

Status: NEW

(Core :: Graphics, defect)

Description

[mozillabugs]

When attempting to play "This cat has one of the best meows..." (https://www.youtube.com/shorts/HXiLA6k4rdM ), FF 121.0 Win32 crashes with a stack overflow. The crash appears to occur right on the call from linear_row_yuv<1>() into upscaleYUV42R8<BLEND>() (FIREFOX_121_0_RELEASE gfx/wr/swgl/src/composite.h line 1112), which is why upscaleYUV42R8<BLEND>() doesn't actually appear in the stack trace.

The crash happens every time I try to play the video. I have not yet diagnosed the root of the problem, but here is the stack trace:

 	xul.dll!_chkstk() Line 99	Unknown
>	xul.dll!linear_row_yuv<1>(unsigned int * dest, int span, glsl::sampler2DRect_impl * samplerY, const glsl::vec2_scalar & srcUV, float srcDU, glsl::sampler2DRect_impl * samplerU, glsl::sampler2DRect_impl * samplerV, const glsl::vec2_scalar & chromaUV, float chromaDU, int colorDepth, const YUVMatrix & colorSpace) Line 1115	C++
 	xul.dll!blendYUV<1>(unsigned int * buf, int span, glsl::sampler2DRect_impl * sampler0, glsl::vec2 uv0, const glsl::vec4_scalar & uv_rect0, glsl::sampler2DRect_impl * sampler1, glsl::vec2 uv1, const glsl::vec4_scalar & uv_rect1, glsl::sampler2DRect_impl * sampler2, glsl::vec2 uv2, const glsl::vec4_scalar & uv_rect2, const glsl::vec3_scalar & ycbcr_bias, const glsl::mat3_scalar & rgb_from_debiased_ycbcr, int rescaleFactor, NoColor noColor) Line 1269	C++
 	xul.dll!brush_yuv_image_ALPHA_PASS_TEXTURE_RECT_YUV_frag::swgl_drawSpanRGBA8() Line 961	C++
 	xul.dll!brush_yuv_image_ALPHA_PASS_TEXTURE_RECT_YUV_frag::draw_span_RGBA8(glsl::FragmentShaderImpl * impl) Line 1037	C++
 	[Inline Frame] xul.dll!glsl::FragmentShaderImpl::draw_span(unsigned int * buf, int len) Line 168	C++
 	[Inline Frame] xul.dll!draw_depth_span(unsigned int z, unsigned int * buf, DepthCursor & cursor) Line 628	C++
 	xul.dll!draw_quad_spans<unsigned int>(int nump, glsl::vec2_scalar * p, unsigned int z, glsl::vec3 * interp_outs, Texture & colortex, Texture & depthtex, const ClipRect & clipRect) Line 1025	C++
 	xul.dll!draw_quad(int nump, Texture & colortex, Texture & depthtex) Line 1627	C++
 	xul.dll!draw_elements<unsigned short>(int count, int instancecount, unsigned int offset, VertexArray & v, Texture & colortex, Texture & depthtex) Line 1656	C++
 	xul.dll!DrawElementsInstanced(unsigned int mode, int count, unsigned int type, int offset, int instancecount) Line 2750	C++
 	xul.dll!swgl::swgl_fns::impl$3::draw_elements_instanced(swgl::swgl_fns::Context * self, unsigned int mode, int count, unsigned int element_type, unsigned int indices_offset, int primcount) Line 1598	Rust
 	xul.dll!webrender::device::gl::Device::draw_indexed_triangles_instanced_u16(int self, int index_count) Line 3745	Rust
 	xul.dll!webrender::renderer::Renderer::draw_instanced_batch<webrender::gpu_types::PrimitiveInstanceData>(ref$<slice2$<webrender::gpu_types::PrimitiveInstanceData>> self, webrender::renderer::vertex::VertexArrayKind vertex_array_kind, webrender::batch::BatchTextures * textures, webrender::renderer::RendererStats * stats) Line 2030	Rust
 	xul.dll!webrender::renderer::Renderer::draw_alpha_batch_container(webrender::batch::AlphaBatchContainer * self, enum2$<webrender::device::gl::DrawTarget> alpha_batch_container, webrender::renderer::FramebufferKind draw_target, euclid::transform3d::Transform3D<f32,euclid::UnknownUnit,euclid::UnknownUnit> * framebuffer_kind, webrender::render_task_graph::RenderTaskGraph * projection, webrender::renderer::RendererStats * render_tasks) Line 2885	Rust
 	xul.dll!webrender::renderer::Renderer::draw_picture_cache_target(webrender::render_target::PictureCacheTarget * self, enum2$<webrender::device::gl::DrawTarget> target, euclid::transform3d::Transform3D<f32,euclid::UnknownUnit,euclid::UnknownUnit> * draw_target, webrender::render_task_graph::RenderTaskGraph * projection, webrender::renderer::RendererStats * render_tasks) Line 2683	Rust
 	xul.dll!webrender::renderer::Renderer::draw_frame(webrender::frame_builder::Frame * self, enum2$<core::option::Option<euclid::size::Size2D<i32,webrender_api::units::DevicePixel>>> frame, unsigned int device_size, webrender::renderer::RenderResults * buffer_age) Line 4645	Rust
 	xul.dll!webrender::renderer::Renderer::render_impl(webrender_api::DocumentId self, webrender::internal_types::RenderedDocument * doc_id, enum2$<core::option::Option<euclid::size::Size2D<i32,webrender_api::units::DevicePixel>>> active_doc, unsigned int device_size) Line 1527	Rust
 	xul.dll!webrender::renderer::Renderer::render(euclid::size::Size2D<i32,webrender_api::units::DevicePixel> self, unsigned int buffer_age) Line 1235	Rust
 	xul.dll!webrender_bindings::bindings::wr_renderer_render(webrender::renderer::Renderer * renderer, int width, int height, unsigned int buffer_age, webrender::renderer::RendererStats * out_stats, thin_vec::ThinVec<euclid::box2d::Box2D<i32,webrender_api::units::DevicePixel>> * out_dirty_rects) Line 619	Rust
 	xul.dll!mozilla::wr::RendererOGL::UpdateAndRender(const mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>> & aReadbackSize, const mozilla::Maybe<mozilla::wr::ImageFormat> & aReadbackFormat, const mozilla::Maybe<mozilla::Range<unsigned char>> & aReadbackBuffer, bool * aNeedsYFlip, mozilla::wr::RendererStats * aOutStats) Line 190	C++
 	xul.dll!mozilla::wr::RenderThread::UpdateAndRender(mozilla::wr::WrWindowId aWindowId, const mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> & aStartId, const mozilla::TimeStamp & aStartTime, bool aRender, const mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>> & aReadbackSize, const mozilla::Maybe<mozilla::wr::ImageFormat> & aReadbackFormat, const mozilla::Maybe<mozilla::Range<unsigned char>> & aReadbackBuffer, bool * aNeedsYFlip) Line 783	C++
 	xul.dll!mozilla::wr::RenderThread::HandleFrameOneDocInner(mozilla::wr::WrWindowId aWindowId, bool aRender, bool aTrackedFrame, mozilla::Maybe<mozilla::wr::FramePublishId> aPublishId) Line 624	C++
 	xul.dll!mozilla::wr::RenderThread::HandleFrameOneDoc(mozilla::wr::WrWindowId aWindowId, bool aRender, bool aTrackedFrame, mozilla::Maybe<mozilla::wr::FramePublishId> aPublishId) Line 573	C++
 	xul.dll!mozilla::wr::RenderThread::WrNotifierEvent_HandleNewFrameReady(mozilla::wr::WrWindowId aWindowId, bool aCompositeNeeded, mozilla::wr::FramePublishId aPublishId) Line 534	C++
 	xul.dll!mozilla::wr::RenderThread::HandleWrNotifierEvents(mozilla::wr::WrWindowId aWindowId) Line 498	C++
 	xul.dll!mozilla::detail::RunnableMethodArguments<mozilla::wr::WrWindowId>::apply<mozilla::wr::RenderThread,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall))>::void <lambda>(unsigned char *, unsigned char, int, unsigned char)<StoreCopyPassByConstLRef<mozilla::wr::WrWindowId> &>(StoreCopyPassByConstLRef<mozilla::wr::WrWindowId> & args) Line 1164	C++
 	xul.dll!std::invoke<`lambda at C:\mozilla-source\obj-i386-pc-mingw32\dist\include\nsThreadUtils.h:1163:9',StoreCopyPassByConstLRef<mozilla::wr::WrWindowId> &>(mozilla::detail::RunnableMethodArguments<mozilla::wr::WrWindowId>::apply<mozilla::wr::RenderThread,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall))>::void <lambda>(unsigned char *, unsigned char, int, unsigned char) && _Obj, StoreCopyPassByConstLRef<mozilla::wr::WrWindowId> & _Arg1) Line 1534	C++
 	xul.dll!std::_Apply_impl<`lambda at C:\mozilla-source\obj-i386-pc-mingw32\dist\include\nsThreadUtils.h:1163:9',std::tuple<StoreCopyPassByConstLRef<mozilla::wr::WrWindowId>> &,0>(mozilla::detail::RunnableMethodArguments<mozilla::wr::WrWindowId>::apply<mozilla::wr::RenderThread,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall))>::void <lambda>(unsigned char *, unsigned char, int, unsigned char) && _Obj, std::tuple<StoreCopyPassByConstLRef<mozilla::wr::WrWindowId>> & _Tpl, std::integer_sequence<unsigned int,0>) Line 974	C++
 	xul.dll!std::apply<`lambda at C:\mozilla-source\obj-i386-pc-mingw32\dist\include\nsThreadUtils.h:1163:9',std::tuple<StoreCopyPassByConstLRef<mozilla::wr::WrWindowId>> &>(mozilla::detail::RunnableMethodArguments<mozilla::wr::WrWindowId>::apply<mozilla::wr::RenderThread,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall))>::void <lambda>(unsigned char *, unsigned char, int, unsigned char) && _Obj, std::tuple<StoreCopyPassByConstLRef<mozilla::wr::WrWindowId>> & _Tpl) Line 979	C++
 	xul.dll!mozilla::detail::RunnableMethodArguments<mozilla::wr::WrWindowId>::apply<mozilla::wr::RenderThread,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall))>(mozilla::wr::RenderThread * o, void(mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) m) Line 1162	C++
 	xul.dll!mozilla::detail::RunnableMethodImpl<mozilla::wr::RenderThread *,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall)),1,0,mozilla::wr::WrWindowId>::Run() Line 1213	C++
 	xul.dll!nsThread::ProcessNextEvent(bool aMayWait, bool * aResult) Line 1192	C++
 	xul.dll!NS_ProcessNextEvent(nsIThread * aThread, bool aMayWait) Line 480	C++
 	xul.dll!mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate * aDelegate) Line 331	C++
 	xul.dll!MessageLoop::RunInternal() Line 371	C++
 	xul.dll!MessageLoop::RunHandler() Line 364	C++
 	xul.dll!MessageLoop::Run() Line 346	C++
 	xul.dll!nsThread::ThreadFunc(void * aArg) Line 372	C++
 	nss3.dll!_PR_NativeRunThread(void * arg) Line 421	C
 	nss3.dll!pr_root(void * arg) Line 140	C
 	ucrtbase.dll!75fd4f9f()	Unknown
 	[Frames below may be incorrect and/or missing, no symbols loaded for ucrtbase.dll]	
 	kernel32.dll!771bfcc9()	Unknown
 	mozglue.dll!mozilla::interceptor::FuncHook<mozilla::interceptor::WindowsDllInterceptor<mozilla::interceptor::VMSharingPolicyShared>,void (*)(int, void *, void *) __attribute__((fastcall))>::operator()<int &,void *&,void *&>(int & aArgs, void * & aArgs, void * & aArgs) Line 150	C++
 	mozglue.dll!patched_BaseThreadInitThunk(int aIsInitialThread, void * aStartAddress, void * aThreadParam) Line 565	C++
 	ntdll.dll!77837c6e()	Unknown
 	ntdll.dll!77837c3e()	Unknown

Comment 1

[mozillabugs]

So the crash occurs in _chkstk() called from upscaleYUV42R8<1>() , apparently because the amount of stack required (0x18f40 bytes in a Win32 debug build) exceeds the amount that can be allocated(?!).

Comment 2

[Nicolas Silva [:nical]]

That's happening in SWGL not sure if we can do something there or if we should just request more stack space on the renderer thread (if we can do that at all on win32 builds?).