1747999 - Brightness changes caused by subtitles with detachment on macOS

Status: RESOLVED WONTFIX

(Core :: Graphics, defect, P3)

Description

[Jeff Muizelaar [:jrmuizel]]

See https://www.reddit.com/r/firefox/comments/reshnn/firefox_95_on_macos_reduces_brightness_of_videos/hohb547/?utm_source=reddit&utm_medium=web2x&context=3

Comment 1

[Jim Mathies [:jimm]]

Colour LCD:
Display Type: Built-In Retina LCD
Resolution: 2880x1800 Retina
Framebuffer Depth: 24-Bit Colour (ARGB8888)
Main Display: Yes
Mirror: Off
Online: Yes
Automatically Adjust Brightness: No
Connection Type: Internal

Comment 2

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1731815

Comment 3

[Ryan VanderMeulen [:RyanVM]]

The regressing feature will be disabled in 96.0rc2 by way of bug 1748221.

Comment 4

[Brad Werth [:bradwerth]]

From the reporter via Reddit:

Features
Compositing: WebRender
Asynchronous Pan/Zoom: wheel input enabled; scrollbar drag enabled; keyboard enabled; autoscroll enabled; smooth pinch-zoom enabled
WebGL 1 Driver WSI Info: CGL IsWebglOutOfProcessEnabled: 1
WebGL 1 Driver Renderer: Intel Inc. -- Intel Iris Pro OpenGL Engine
WebGL 1 Driver Version: 4.1 INTEL-14.7.18
WebGL 1 Driver Extensions: GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier
WebGL 1 Extensions: ANGLE_instanced_arrays EXT_blend_minmax EXT_color_buffer_half_float EXT_float_blend EXT_frag_depth EXT_shader_texture_lod EXT_sRGB EXT_texture_compression_rgtc EXT_texture_filter_anisotropic MOZ_debug OES_element_index_uint OES_fbo_render_mipmap OES_standard_derivatives OES_texture_float OES_texture_float_linear OES_texture_half_float OES_texture_half_float_linear OES_vertex_array_object WEBGL_color_buffer_float WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_depth_texture WEBGL_draw_buffers WEBGL_lose_context
WebGL 2 Driver WSI Info: CGL IsWebglOutOfProcessEnabled: 1
WebGL 2 Driver Renderer: Intel Inc. -- Intel Iris Pro OpenGL Engine
WebGL 2 Driver Version: 4.1 INTEL-14.7.18
WebGL 2 Driver Extensions: GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier
WebGL 2 Extensions: EXT_color_buffer_float EXT_float_blend EXT_texture_compression_rgtc EXT_texture_filter_anisotropic MOZ_debug OES_texture_float_linear WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_context
Target Frame Rate: 60

GPU #1
Active: Yes
Vendor ID: 0x8086
Device ID: 0x0d26
RAM: 0

Display0: 1440x900 scale:2.000000
DisplayCount: 1

Decision Log
HW_COMPOSITING: available by default
OPENGL_COMPOSITING: available by default
WEBRENDER: available by default
WEBRENDER_QUALIFIED: available by default
WEBRENDER_COMPOSITOR: available by default
WEBRENDER_SHADER_CACHE: disabled by default: Disabled by default
WEBRENDER_OPTIMIZED_SHADERS: available by default
WEBRENDER_ANGLE: available by default unavailable by env: OS not supported
WEBRENDER_DCOMP_PRESENT: available by default disabled by user: User disabled via pref unavailable by env: Requires Windows 10 or later unavailable by runtime: Requires ANGLE
WEBRENDER_SOFTWARE: available by default WEBGPU: disabled by default: Disabled by default blocked by runtime:
WebGPU can only be enabled in nightly

Comment 5

[Marco Castelluccio [:marco]]

Is there a bug to track enabling specializing video by default, so we can mark this bug as blocking it?

Comment 6

[Brad Werth [:bradwerth]]

This Bug will likely be solved by the landing of Bug 1731821. I'll ask for a re-test when that lands.

Comment 7

[Brad Werth [:bradwerth]]

Jeff M can reproduce this. Let's re-open while I try to figure out the mechanism.

Comment 8

[Jeff Muizelaar [:jrmuizel]]

Attachment: A synthetic test case

Comment 9

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1747999: Use AVSampleBufferDisplayLayer for all video (if pref set).

This will break HDR display, but it's as simple as possible for testing
the relative behavior of CALayer versus AVSampleBufferDisplayLayer for
video surfaces.

Comment 10

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1747999: Set the kCVImageBufferCGColorSpaceKey on the AVSampleBufferDisplayLayer pixelBuffer.

Comment 11

[Brad Werth [:bradwerth]]

Further testing shows that this is broadly reproducible with attachment 9283769 [details] on macOS 10.15 on Intel hardware in both Firefox and Chrome. Safari does not reproduce it, but doesn't achieve low power mode either. We also have some folks who can reproduce it in Firefox on macOS 11. Not sure how to avoid this and get the benefits of low power mode.

Comment 12

[Jeff Muizelaar [:jrmuizel]]

I made an hdr version of the test case by using:

ffmpeg -i chip-chart-1080.mp4 -colorspace bt2020nc chip-chart-1080-hdr.mp4

The HDR version doesn't have a flash but it looks like colorspace conversion is done differently depending on whether it's in detached mode or not.

Comment 13

[Brad Werth [:bradwerth]]

This video test app shows that both SDR and HDR video can be played with an overlay on Intel macOS hardware and hit the detached mode when the overlay disappears -- with no flash. In Firefox, the primary difference is that we have a very different layer structure for the two cases of overlay vs no-overlay:

With overlay: A bunch of tiles showing black < video texture < one tile with the overlay
Without overlay: One large color layer < video texture

The large color layer is necessary to qualify for detached mode. When the overlay is present, the color layer is not. It's possible that the change in layer structure coinciding with the transition to the detached mode is creating the flash. I'll try to make the color layer appear in the overlay case and see if there's still a flash.

Comment 14

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1747999: Force a new slice before every promotable surface.

Comment 15

[Brad Werth [:bradwerth]]

(In reply to Brad Werth [:bradwerth] from comment #13)

The large color layer is necessary to qualify for detached mode. When the overlay is present, the color layer is not. It's possible that the change in layer structure coinciding with the transition to the detached mode is creating the flash. I'll try to make the color layer appear in the overlay case and see if there's still a flash.

This is not the root cause. Forcing the native color layer to appear (D152210) does not prevent the flashing.

Comment 16

[Brad Werth [:bradwerth]]

In the test app, forcibly setting the colorspace of the IOSurfaceRef to match the main display removes the flashing. In Firefox, I'm not sure how that should interact with the gfx.color_management.native_srgb pref. I'll make a test build that forces the colorspace when gfx.core-animation.specialize-video.check-color-space is set.

Comment 17

[Brad Werth [:bradwerth]]

Attachment: Bug 1747999 Part 1: Force macOS enqueued video buffer colorspace to match

Comment 18

[Brad Werth [:bradwerth]]

We probably can't force the video colorspace to match the display for color correctness reasons. There are definitely test failures: https://treeherder.mozilla.org/jobs?repo=try&revision=0617683ed3b6f0b2bfae23b3de78aaeb43a95308.

Comment 19

[Brad Werth [:bradwerth]]

Actually, the test failures are negotiable -- the change affects the results to have more pixels be wrong, but closer to the intended values in aggregate. I'll post a test expectations update patch and we can discuss it in the reviews.

Comment 20

[Brad Werth [:bradwerth]]

Attachment: Bug 1747999 Part 2: Update test expecations.

For TV range, this reduces the deviation from expected color values, but
increases the number of affected pixels.

For PC range, it only increases the number of affected pixels.

Depends on D152475

Comment 21

[Ryan VanderMeulen [:RyanVM]]

For posterity, bug 1772024 allowed this pref to ride the trains again for macOS ARM in Fx105, but this bug was filed against Intel Macs and therefore remains effectively in the disabled state.

Comment 22

[BugBot [:suhaib / :marco]]

The following patch is waiting for review from an inactive reviewer:

ID	Title	Author	Reviewer Status
D152475	Bug 1747999 Part 1: Force macOS enqueued video buffer colorspace to match	bradwerth	jrmuizel: Inactive

:bradwerth, could you please find another reviewer?

For more information, please visit auto_nag documentation.

Comment 23

[Marco Castelluccio [:marco]]

Sorry for the noise, there was a problem with a Bugzilla API that caused the bot to think more people were inactive than normal (bug 1789646).

Comment 24

[Brad Werth [:bradwerth]]

Until we get a response or help from Apple, this will remain unfixed.