1490991 - Mousewheel scrolling lags badly with webrender enabled

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P2)

Description

[Katelyn Gadd (:kael)]

Maybe the same root cause as bug #1481993. I am using 64.0a1 (2018-09-13) (64-bit) on a desktop w/a gtx 1080 on latest win10, with two 4K monitors running at 60hz. I have my firefox window maximized. If I scroll with the mousewheel it frequently lags/judders, it looks like it's compositing at 30hz at most. It doesn't matter what page I'm looking at (even bugzilla will do it). If I middle click scroll that is smooth on all the sites I've tried.

Is there a way for me to record a trace or some other information for troubleshooting?

For a sanity check since things feel mostly fine outside of mousewheel scrolling, I ran motionmark on both release channel chrome and ff nightly. nightly w/webrender scored 139.66 +- 27.82%, chrome 418.78 +- 22.68%. I didn't see any weird stuttering or hitching while it was running its tests so it doesn't seem like anything serious is going wrong with rendering or compositing for basic stuff here.

This may be related to gsync, because my main panel has gsync enabled. I think historically nvidia's drivers have blacklisted chrome and firefox from gsync manually, but maybe that's not working.

About:support graphics section follows:

----
Graphics
Features
Compositing	WebRender
Asynchronous Pan/Zoom	wheel input enabled; scrollbar drag enabled; keyboard enabled; autoscroll enabled
WebGL 1 Driver WSI Info	EGL_VENDOR: Google Inc. (adapter LUID: 0000000000008ded)
EGL_VERSION: 1.4 (ANGLE 2.1.0.ae3b5a6552ee)
EGL_EXTENSIONS: EGL_EXT_create_context_robustness EGL_ANGLE_d3d_share_handle_client_buffer EGL_ANGLE_d3d_texture_client_buffer EGL_ANGLE_surface_d3d_texture_2d_share_handle EGL_ANGLE_query_surface_pointer EGL_ANGLE_window_fixed_size EGL_ANGLE_keyed_mutex EGL_ANGLE_surface_orientation EGL_ANGLE_direct_composition EGL_NV_post_sub_buffer EGL_KHR_create_context EGL_EXT_device_query EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_get_all_proc_addresses EGL_KHR_stream EGL_KHR_stream_consumer_gltexture EGL_NV_stream_consumer_gltexture_yuv EGL_ANGLE_flexible_surface_compatibility EGL_ANGLE_stream_producer_d3d_texture EGL_ANGLE_create_context_webgl_compatibility EGL_CHROMIUM_create_context_bind_generates_resource EGL_CHROMIUM_sync_control EGL_EXT_pixel_format_float EGL_KHR_surfaceless_context EGL_ANGLE_display_texture_share_group EGL_ANGLE_create_context_client_arrays EGL_ANGLE_program_cache_control EGL_ANGLE_robust_resource_initialization EGL_ANGLE_create_context_extensions_enabled 
EGL_EXTENSIONS(nullptr): EGL_EXT_client_extensions EGL_EXT_platform_base EGL_EXT_platform_device EGL_ANGLE_platform_angle EGL_ANGLE_platform_angle_d3d EGL_ANGLE_device_creation EGL_ANGLE_device_creation_d3d11 EGL_ANGLE_experimental_present_path EGL_KHR_client_get_all_proc_addresses EGL_ANGLE_explicit_context 
WebGL 1 Driver Renderer	Google Inc. -- ANGLE (NVIDIA GeForce GTX 1080 Direct3D11 vs_5_0 ps_5_0)
WebGL 1 Driver Version	OpenGL ES 2.0 (ANGLE 2.1.0.ae3b5a6552ee)
WebGL 1 Driver Extensions	GL_ANGLE_client_arrays GL_ANGLE_depth_texture GL_ANGLE_explicit_context GL_ANGLE_explicit_context_gles1 GL_ANGLE_framebuffer_blit GL_ANGLE_framebuffer_multisample GL_ANGLE_instanced_arrays GL_ANGLE_lossy_etc_decode GL_ANGLE_pack_reverse_row_order GL_ANGLE_program_cache_control GL_ANGLE_request_extension GL_ANGLE_robust_client_memory GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_ANGLE_texture_usage GL_ANGLE_translated_shader_source GL_CHROMIUM_bind_generates_resource GL_CHROMIUM_bind_uniform_location GL_CHROMIUM_color_buffer_float_rgb GL_CHROMIUM_color_buffer_float_rgba GL_CHROMIUM_copy_compressed_texture GL_CHROMIUM_copy_texture GL_CHROMIUM_sync_query GL_EXT_blend_minmax GL_EXT_color_buffer_half_float GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_disjoint_timer_query GL_EXT_draw_buffers GL_EXT_frag_depth GL_EXT_map_buffer_range GL_EXT_occlusion_query_boolean GL_EXT_read_format_bgra GL_EXT_robustness GL_EXT_sRGB GL_EXT_shader_texture_lod GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc_srgb GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_storage GL_EXT_unpack_subimage GL_KHR_debug GL_KHR_robust_buffer_access_behavior GL_NV_EGL_stream_consumer_external GL_NV_fence GL_NV_pack_subimage GL_NV_pixel_buffer_object GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth32 GL_OES_element_index_uint GL_OES_get_program_binary GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_array_object OES_compressed_EAC_R11_signed_texture OES_compressed_EAC_R11_unsigned_texture OES_compressed_EAC_RG11_signed_texture OES_compressed_EAC_RG11_unsigned_texture OES_compressed_ETC2_RGB8_texture OES_compressed_ETC2_RGBA8_texture OES_compressed_ETC2_punchthroughA_RGBA8_texture OES_compressed_ETC2_punchthroughA_sRGB8_alpha_texture OES_compressed_ETC2_sRGB8_alpha8_texture OES_compressed_ETC2_sRGB8_texture 
WebGL 1 Extensions	ANGLE_instanced_arrays EXT_blend_minmax EXT_color_buffer_half_float EXT_frag_depth EXT_sRGB EXT_shader_texture_lod EXT_texture_filter_anisotropic EXT_disjoint_timer_query OES_element_index_uint OES_standard_derivatives OES_texture_float OES_texture_float_linear OES_texture_half_float OES_texture_half_float_linear OES_vertex_array_object WEBGL_color_buffer_float WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_depth_texture WEBGL_draw_buffers WEBGL_lose_context
WebGL 2 Driver WSI Info	EGL_VENDOR: Google Inc. (adapter LUID: 0000000000008ded)
EGL_VERSION: 1.4 (ANGLE 2.1.0.ae3b5a6552ee)
EGL_EXTENSIONS: EGL_EXT_create_context_robustness EGL_ANGLE_d3d_share_handle_client_buffer EGL_ANGLE_d3d_texture_client_buffer EGL_ANGLE_surface_d3d_texture_2d_share_handle EGL_ANGLE_query_surface_pointer EGL_ANGLE_window_fixed_size EGL_ANGLE_keyed_mutex EGL_ANGLE_surface_orientation EGL_ANGLE_direct_composition EGL_NV_post_sub_buffer EGL_KHR_create_context EGL_EXT_device_query EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_get_all_proc_addresses EGL_KHR_stream EGL_KHR_stream_consumer_gltexture EGL_NV_stream_consumer_gltexture_yuv EGL_ANGLE_flexible_surface_compatibility EGL_ANGLE_stream_producer_d3d_texture EGL_ANGLE_create_context_webgl_compatibility EGL_CHROMIUM_create_context_bind_generates_resource EGL_CHROMIUM_sync_control EGL_EXT_pixel_format_float EGL_KHR_surfaceless_context EGL_ANGLE_display_texture_share_group EGL_ANGLE_create_context_client_arrays EGL_ANGLE_program_cache_control EGL_ANGLE_robust_resource_initialization EGL_ANGLE_create_context_extensions_enabled 
EGL_EXTENSIONS(nullptr): EGL_EXT_client_extensions EGL_EXT_platform_base EGL_EXT_platform_device EGL_ANGLE_platform_angle EGL_ANGLE_platform_angle_d3d EGL_ANGLE_device_creation EGL_ANGLE_device_creation_d3d11 EGL_ANGLE_experimental_present_path EGL_KHR_client_get_all_proc_addresses EGL_ANGLE_explicit_context 
WebGL 2 Driver Renderer	Google Inc. -- ANGLE (NVIDIA GeForce GTX 1080 Direct3D11 vs_5_0 ps_5_0)
WebGL 2 Driver Version	OpenGL ES 3.0 (ANGLE 2.1.0.ae3b5a6552ee)
WebGL 2 Driver Extensions	GL_ANGLE_client_arrays GL_ANGLE_depth_texture GL_ANGLE_explicit_context GL_ANGLE_explicit_context_gles1 GL_ANGLE_framebuffer_blit GL_ANGLE_framebuffer_multisample GL_ANGLE_instanced_arrays GL_ANGLE_lossy_etc_decode GL_ANGLE_multiview GL_ANGLE_pack_reverse_row_order GL_ANGLE_program_cache_control GL_ANGLE_request_extension GL_ANGLE_robust_client_memory GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_ANGLE_texture_usage GL_ANGLE_translated_shader_source GL_CHROMIUM_bind_generates_resource GL_CHROMIUM_bind_uniform_location GL_CHROMIUM_color_buffer_float_rgb GL_CHROMIUM_color_buffer_float_rgba GL_CHROMIUM_copy_compressed_texture GL_CHROMIUM_copy_texture GL_CHROMIUM_sync_query GL_EXT_blend_minmax GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_disjoint_timer_query GL_EXT_draw_buffers GL_EXT_frag_depth GL_EXT_map_buffer_range GL_EXT_occlusion_query_boolean GL_EXT_read_format_bgra GL_EXT_robustness GL_EXT_sRGB GL_EXT_shader_texture_lod GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc_srgb GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_texture_norm16 GL_EXT_texture_rg GL_EXT_texture_storage GL_EXT_unpack_subimage GL_KHR_debug GL_KHR_robust_buffer_access_behavior GL_NV_EGL_stream_consumer_external GL_NV_fence GL_NV_pack_subimage GL_NV_pixel_buffer_object GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth32 GL_OES_element_index_uint GL_OES_get_program_binary GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_array_object OES_compressed_EAC_R11_signed_texture OES_compressed_EAC_R11_unsigned_texture OES_compressed_EAC_RG11_signed_texture OES_compressed_EAC_RG11_unsigned_texture OES_compressed_ETC2_RGB8_texture OES_compressed_ETC2_RGBA8_texture OES_compressed_ETC2_punchthroughA_RGBA8_texture OES_compressed_ETC2_punchthroughA_sRGB8_alpha_texture OES_compressed_ETC2_sRGB8_alpha8_texture OES_compressed_ETC2_sRGB8_texture 
WebGL 2 Extensions	EXT_color_buffer_float EXT_texture_filter_anisotropic EXT_disjoint_timer_query OES_texture_float_linear WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_context
Direct2D	true
Uses Tiling (Content)	true
Off Main Thread Painting Enabled	true
Off Main Thread Painting Worker Count	4
DirectWrite	true (10.0.17134.1)
GPU #1
Active	Yes
Description	NVIDIA GeForce GTX 1080
Vendor ID	0x10de
Device ID	0x1b80
Driver Version	24.21.13.9836
Driver Date	6-24-2018
Drivers	C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumdx.dll,C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumdx.dll,C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumdx.dll,C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumdx.dll C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumd.dll,C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumd.dll,C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumd.dll,C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_485c1c3102021986\nvldumd.dll
Subsys ID	61833842
RAM	8192
Diagnostics
AzureCanvasAccelerated	0
AzureCanvasBackend	direct2d 1.1
AzureCanvasBackend (UI Process)	skia
AzureContentBackend	skia
AzureContentBackend (UI Process)	skia
AzureFallbackCanvasBackend (UI Process)	cairo
GPUProcessPid	18116
GPUProcess	
Device Reset	
Decision Log
WEBRENDER	
opt-in by default: WebRender is an opt-in feature
available by user: Force enabled by pref
----

Hm, geforce experience hasn't been pushing me updates...

Comment 1

[Katelyn Gadd (:kael)]

So far, the scrolling does not hitch if I have shadowplay active to record it, which is interesting. IIRC shadowplay tends to mess with vsync timings and gpu clocks.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Can you turn on gfx.webrender.debug.profiler and compare scrolling with the mouse wheel and middle click scrolling?

Comment 3

[Katelyn Gadd (:kael)]

I just found it now and tried it out. Frame times for the backend and compositor seem to consistently hover in the 1.5-2.5ms range, draw call counts seem right. GPU is always reported as 0.00ms, which is weird.

Middle-click scrolling my twitter timeline delivers a near-perfect 60fps with backend/compositor times consistently in the 2.5 range. It's impossible for me to get a number close to 60 when scrolling with the mousewheel, but I suspect that's just because the page doesn't stay in constant motion due to the accelerate/decelerate curve you get when scrolling with the mousewheel.

I see brief flickers of sizable (~1000kb, maybe? it's a single frame so hard to tell) texture data uploads, but I assume that's due to twitter loading new jpegs and pngs as I scroll. Those happen during the middle click scroll too, but maybe mousewheel triggers more of them at once and hitches?

If I middle-click scroll at an extremely fast rate on twitter I can see a few dropped frames, but it's not remotely approaching a mousewheel scroll (and this is a speed exceeding what I get with the mousewheel). The dropped frames there appear to coincide with a big texture upload (probably from them adding a bunch of tweets + images into the dom at once).

Comment 4

[Katelyn Gadd (:kael)]

Here's a trace from the profiler that maybe captured some jank? It's hard to tell, I have no idea how to read the traces in this new UI. The flame chart seems to suggest that in the bad time window (I saw a couple hitches in the last 5-10 seconds) a lot of time was spent building display lists for twitter. There's also a really bad reflow earlier in the timeline but that's unsurprising.

https://perfht.ml/2NHjfPC

Comment 5

[Darkspirit]

> GPU is always reported as 0.00ms, which is weird.
That's behind gfx.webrender.debug.gpu-time-queries.

Comment 6

[Jeff Muizelaar [:jrmuizel]]

Does the problem improve if you reduce the window size, disconnect a monitor or lower the resolution?

Comment 7

[Jeff Muizelaar [:jrmuizel]]

(In reply to K. Gadd (:kael) from comment #4)

For the profile to be useful you'll need to add "RenderBackend,Renderer,WebRender,Wr" to the "Threads" setting in the profiler.

Comment 8

[Katelyn Gadd (:kael)]

I added those threads and captured a couple more traces.

I noticed a couple hitches near the end of this one (and maybe one early as well): https://perfht.ml/2NDcD57. I think it's probably the spot where two refresh driver ticks come almost back to back.

I think I got a couple hitches right at the beginning of this one: https://perfht.ml/2NC0WLZ

I updated my geforce drivers just to rule that out and it was fine for maybe an hour or so before I started seeing it again.

Comment 9

[Jeff Muizelaar [:jrmuizel]]

It looks this is some kind of weird scheduling problem where we end up with double composites and which ruins the smoothness.

Comment 10

[Markus Stange [:mstange]]

Here's an excerpt from the second profile: https://perfht.ml/2x7uPdI
There are four mousewheel ticks in the selected range, indicated by PAPZInputBridge::Msg_ReceiveScrollWheelInputEvent markers on the parent process main thread. These wheel events presumably kick off a smooth scroll animation, which we expect to composite evenly over the next 200ms or so. However, if you look at the composite markers on the Renderer thread in that range, you'll notice two gaps in the otherwise evenly-spaced markers. The first gap, around the 3.28 second mark, is preceded by two composite markers that are bunched up closely together: https://perfht.ml/2x4wkt1 . And the second gap, around the 3.52 second mark, is preceded by one regular composite marker which is preceded by two closely bunched up composite markers: https://perfht.ml/2xaGB6N .

In both cases of double-composites, the call stacks in the samples that were captured during those markers indicate that the composite was triggered by a call to mozilla::wr::RenderThread::NewFrameReady.

I think the problem is that we sometimes get NewFrameReady calls that are not evenly spaced.
Sotaro, Nical, any ideas?

Comment 11

[Aria Beingessner [:Gankra]]

Slightly offtopic, but since the reporter mentioned MotionMark, I just wanted to link this PSA about MotionMark usage in Firefox: https://gist.github.com/Gankro/013af66babdbf50428ac10450cb3ba8c

Comment 12

[Sotaro Ikeda [:sotaro]]

(In reply to Markus Stange [:mstange] from comment #10)
> I think the problem is that we sometimes get NewFrameReady calls that are
> not evenly spaced.
> Sotaro, Nical, any ideas?

WebRenderBridgeParent::CompositeToTarget() throttles frame generation by calling RenderThread::TooManyPendingFrames(). It seems to be related. At most 2 frame generation task is queued to WR(one for rendering, one for render backend).

Comment 13

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #12)
> 
> WebRenderBridgeParent::CompositeToTarget() throttles frame generation by
> calling RenderThread::TooManyPendingFrames(). It seems to be related. At
> most 2 frame generation task is queued to WR(one for rendering, one for
> render backend).

And the problem seemed to happen when fps did not reach 60 fps.

Comment 14

[Sotaro Ikeda [:sotaro]]

Bug 1461239 might improve the problem.

Comment 15

[Nicolas Silva [:nical]]

We discussed this a with Markus yesterday. Here are some of the relevant parts of the discussion:

When a scene is built we currently re-build the frame on the render backend side in order to have an up to date hit tester. When we re-build a frame we have to send it to the renderer and the latter has to do some off-screen rendering work, even though it will not render to the window.
At the same time, after having built the scene the compositor gets notified that it should schedule rendering at the next vsync. when that happens the another frame is built on the render backend thread and it is also sent to the renderer, this time with the instruction to render to the window.

This explains the fact that in the profile around the time of the stutter we see two render markers close to each other instead of being evenly spaced by vsync (we have the render marker also for the off-screen rendering work trigger by scene building).

The real problem is that after these two render markers we seem to skip a rendering. There's a hole where we would have expected the vsync to trigger another render.

One area to explore is that perhaps the extra render we do off-screen tiggered by scene building is counted as a normal render by some throttling mechanism which then cancels a later render (and causes the stutter).
I haven't found a way to verify whether that's true but it's worth closely inspecting the code in RenderThread.cpp that deals with mPendingCount and mRenderingCount. The NewFrameReady hook in gecko increments a counter of in-flight rendering work which is used to throttle scheduling frames, for normal renders mPendingCount is incremented while for off-screen ones it is decremented.
See: https://searchfox.org/mozilla-central/rev/6c82481caa506a240a626bb44a2b8cbe0eedb3a0/gfx/webrender_bindings/RenderThread.cpp#746 https://searchfox.org/mozilla-central/rev/6c82481caa506a240a626bb44a2b8cbe0eedb3a0/gfx/webrender_bindings/RenderThread.cpp#417 and the places where it is used.

I am working on removing the extra frames caused by scene building so this might go away as a byproduct, but we should still make it so that off-screen renders don't cause issues if this is the problem since since there may be other reasons to use them.

Comment 16

[Kartikaya Gupta [:kats]]

Some changes landed recently to eliminate the extra renders, so this might be fixed now. Can you retest and let us know if you still see the problem?

Comment 17

[Katelyn Gadd (:kael)]

I'm not sure if it's gone, but it is certainly less common now. I'm seeing it in more complex sites (though this could easily be due to JS or content changes/layout changes in the page) but not seeing it on sites like bugzilla.

Comment 18

[Kartikaya Gupta [:kats]]

Thanks! Can you grab a profile if you see it again? Be sure to have at least "GeckoMain,Compositor,WR,Renderer" in your profiler thread list.

Comment 19

[Sotaro Ikeda [:sotaro]]

Bug 1480964 and Bug 1493576 might be similar bug.

Comment 20

[Katelyn Gadd (:kael)]

FYI: I'm still seeing this periodically but have not caught it with the profiler. It seems to be restricted to more complex pages. I saw some comments on Hacker News also mention it. It is way less common than before the changes y'all made.

Comment 21

[Nicolas Silva [:nical]]

For complex pages it could be jank happening on frame with a lot of textures to upload or occasional high amount of GPU work to do. There's also bug 1487864 about occasional unfortunate frame scheduling/throttling in some cases. I'll close this since the issue that is very visible in the profiles has been fixed, the rest is covered by other bugs on file I think. Don't hesitate to open new bugs if you have any information that could help with tracking down any kind of frame drop.