1794438 - Quicktime video is cropped strangely when played in FF for Android

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P3)

Description

[Kevin Brosnan [Ex-Mozilla]]

From github: https://github.com/mozilla-mobile/fenix/issues/27327.

Open Quicktime video in FF for Android: https://www.mhohner.de/y/S.MOV

Video is cropped to a narrow strip, the rest of the video area is blank. See screenshot: https://www.mhohner.de/y/Screenshot_20221007-083011_Firefox.jpg

Happens with Galaxy S7 and Galaxy Tab S4.

Video plays just fine in FF for Windows.

┆Issue is synchronized with this Jira Task

Change performed by the Move to Bugzilla add-on.

Comment 1

[az [:az]]

Adding comment from the original thread:

amocirean commented 8 hours ago

Hello @grmblgrmbl, thank you for submitting the issue.
I was able to reproduce it on both Nightly 107.0a1 and Release 105.2.0.

Devices Used: Huawei P40 Lite (Android 10), Google Pixel 6 (Android 13).

The issue doesn't seem to crop up with Linux. I'd like to request access to the reporters' about:support text, but it doesn't look like either of the reporters have BZ accounts and the original report has been closed. Do we have a standard protocol for collecting information from the reporters once a report has been moved from another source to BZ?

Jolin, would you have any ideas here?

Comment 2

[Michael Hohner]

Attachment: about

I am the original reporter.

As the report says, this is a problem of FF for Android, not Linux.

Here's the about:support text:

Comment 3

[Michael Hohner]

(pasting as attachment does not work correctly, Bugzilla bug?)

Application Basics

NameFirefoxVersion105.0.2Build ID20221003135607Distribution IDUser AgentMozilla/5.0 (Android 8.0.0; Mobile; rv:105.0) Gecko/105.0 Firefox/105.0OSLinux 26 #1 SMP PREEMPT Tue Jul 26 15:29:15 KST 2022Multiprocess Windows 0/0 Fission Windows 0/0 Disabled by default Remote Processes 2 Google Location Service KeyFoundGoogle Safebrowsing KeyFoundMozilla Location Service KeyFoundSafe ModefalseMemory Size (RAM)3,5 GBDisk Space Available5,3 GB

Crash Reports for the Last 3 Days

Report IDSubmitted

Nightly Features

NameVersionID

Remote Processes

TypeCountWeb Content1 / 2GPU1

Add-ons

NameTypeVersionEnabledIDGerman dictionary (de_DE)[e-mail] Android Components - Ads [e-mail] Android Components - Browser [e-mail] Android Components - Firefox Accounts [e-mail] Android Components - [e-mail] Android Components - Search [e-mail] Android Components - Web Compatibility [e-mail] [e-mail]

Graphics

FeaturesCompositingWebRenderAsynchronous Pan/Zoomtouch input enabled; autoscroll enabled; smooth pinch-zoom enabledWebGL 1 Driver WSI InfoEGL_VENDOR: ARM EGL_VERSION: 1.4 Midgard-"r22p0-01rel0" EGL_EXTENSIONS: EGL_ANDROID_image_native_buffer EGL_ANDROID_recordable EGL_ANDROID_native_fence_sync EGL_ANDROID_framebuffer_target EGL_ANDROID_image_crop EGL_ANDROID_blob_cache EGL_KHR_partial_update EGL_KHR_mutable_render_buffer EGL_KHR_config_attribs EGL_KHR_image EGL_KHR_image_base EGL_KHR_fence_sync EGL_KHR_wait_sync EGL_KHR_gl_colorspace EGL_KHR_get_all_proc_addresses EGL_IMG_context_priority EGL_KHR_no_config_context EGL_EXT_protected_surface EGL_EXT_protected_content EGL_ARM_implicit_external_sync EGL_KHR_gl_texture_2D_image EGL_KHR_gl_renderbuffer_image EGL_KHR_create_context EGL_KHR_surfaceless_context EGL_KHR_gl_texture_cubemap_image EGL_EXT_create_context_robustness IsWebglOutOfProcessEnabled: 0WebGL 1 Driver RendererARM -- Mali-T880WebGL 1 Driver VersionOpenGL ES 3.2 v1.r22p0-01rel0.df0c07f7ff7915d24119c5461f72cc26WebGL 1 Driver ExtensionsGL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_to_texture GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_RG8 GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic GL_EXT_robustness GL_EXT_draw_buffers_indexed GL_OES_draw_buffers_indexed GL_EXT_texture_border_clamp GL_OES_texture_border_clamp GL_EXT_texture_cube_map_array GL_OES_texture_cube_map_array GL_OES_sample_variables GL_OES_sample_shading GL_OES_shader_multisample_interpolation GL_EXT_shader_io_blocks GL_OES_shader_io_blocks GL_EXT_tessellation_shader GL_OES_tessellation_shader GL_EXT_primitive_bounding_box GL_OES_primitive_bounding_box GL_EXT_geometry_shader GL_OES_geometry_shader GL_ANDROID_extension_pack_es31a GL_EXT_gpu_shader5 GL_OES_gpu_shader5 GL_EXT_texture_buffer GL_OES_texture_buffer GL_EXT_copy_image GL_OES_copy_image GL_EXT_shader_non_constant_global_initializers GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float GL_EXT_YUV_target GL_OVR_multiview GL_OVR_multiview2 GL_OVR_multiview_multisampled_render_to_texture GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_protected_textures GL_EXT_buffer_storage WebGL 1 ExtensionsANGLE_instanced_arrays EXT_blend_minmax EXT_color_buffer_half_float EXT_float_blend EXT_sRGB MOZ_debug OES_element_index_uint OES_fbo_render_mipmap OES_standard_derivatives OES_texture_float OES_texture_half_float OES_texture_half_float_linear OES_vertex_array_object WEBGL_color_buffer_float WEBGL_compressed_texture_astc WEBGL_compressed_texture_etc WEBGL_compressed_texture_etc1 WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_depth_texture WEBGL_lose_contextWebGL 2 Driver WSI InfoEGL_VENDOR: ARM EGL_VERSION: 1.4 Midgard-"r22p0-01rel0" EGL_EXTENSIONS: EGL_ANDROID_image_native_buffer EGL_ANDROID_recordable EGL_ANDROID_native_fence_sync EGL_ANDROID_framebuffer_target EGL_ANDROID_image_crop EGL_ANDROID_blob_cache EGL_KHR_partial_update EGL_KHR_mutable_render_buffer EGL_KHR_config_attribs EGL_KHR_image EGL_KHR_image_base EGL_KHR_fence_sync EGL_KHR_wait_sync EGL_KHR_gl_colorspace EGL_KHR_get_all_proc_addresses EGL_IMG_context_priority EGL_KHR_no_config_context EGL_EXT_protected_surface EGL_EXT_protected_content EGL_ARM_implicit_external_sync EGL_KHR_gl_texture_2D_image EGL_KHR_gl_renderbuffer_image EGL_KHR_create_context EGL_KHR_surfaceless_context EGL_KHR_gl_texture_cubemap_image EGL_EXT_create_context_robustness IsWebglOutOfProcessEnabled: 0WebGL 2 Driver RendererARM -- Mali-T880WebGL 2 Driver VersionOpenGL ES 3.2 v1.r22p0-01rel0.df0c07f7ff7915d24119c5461f72cc26WebGL 2 Driver ExtensionsGL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_to_texture GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_RG8 GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic GL_EXT_robustness GL_EXT_draw_buffers_indexed GL_OES_draw_buffers_indexed GL_EXT_texture_border_clamp GL_OES_texture_border_clamp GL_EXT_texture_cube_map_array GL_OES_texture_cube_map_array GL_OES_sample_variables GL_OES_sample_shading GL_OES_shader_multisample_interpolation GL_EXT_shader_io_blocks GL_OES_shader_io_blocks GL_EXT_tessellation_shader GL_OES_tessellation_shader GL_EXT_primitive_bounding_box GL_OES_primitive_bounding_box GL_EXT_geometry_shader GL_OES_geometry_shader GL_ANDROID_extension_pack_es31a GL_EXT_gpu_shader5 GL_OES_gpu_shader5 GL_EXT_texture_buffer GL_OES_texture_buffer GL_EXT_copy_image GL_OES_copy_image GL_EXT_shader_non_constant_global_initializers GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float GL_EXT_YUV_target GL_OVR_multiview GL_OVR_multiview2 GL_OVR_multiview_multisampled_render_to_texture GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_protected_textures GL_EXT_buffer_storage WebGL 2 ExtensionsEXT_color_buffer_float EXT_float_blend MOZ_debug OVR_multiview2 WEBGL_compressed_texture_astc WEBGL_compressed_texture_etc WEBGL_compressed_texture_etc1 WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_contextTarget Frame Rate59GPU #1ActiveYesDescriptionModel: SM-G930F, Product: heroltexx, Manufacturer: samsung, Hardware: samsungexynos8890, OpenGL: ARM -- Mali-T880 -- OpenGL ES 3.2 v1.r22p0-01rel0.df0c07f7ff7915d24119c5461f72cc26Vendor IDARMDevice IDMali-T880Driver VersionOpenGL ES 3.2 v1.r22p0-01rel0.df0c07f7ff7915d24119c5461f72cc26RAM0GPU #2DiagnosticsAzureCanvasBackendskiaAzureCanvasBackend (UI Process)skiaAzureContentBackendskiaAzureContentBackend (UI Process)skiaAzureFallbackCanvasBackend (UI Process)skiaCMSOutputProfileEmpty profile dataDisplay01440x2560@0Hz scales:4.000000|1.000000DisplayCount1GPUProcessPid12729Decision LogHW_COMPOSITINGavailable by defaultforce_enabled by user: Force-enabled by prefOPENGL_COMPOSITINGavailable by defaultforce_enabled by user: Force-enabled by prefGPU_PROCESSavailable by defaultWEBRENDERavailable by defaultWEBRENDER_QUALIFIEDavailable by defaultWEBRENDER_COMPOSITORdisabled by default: Disabled by defaultWEBRENDER_PARTIALavailable by defaultblocklisted by env: Blocklisted by gfxInfoWEBRENDER_SHADER_CACHEavailable by defaultWEBRENDER_OPTIMIZED_SHADERSavailable by defaultblocklisted by env: Blocklisted by gfxInfoWEBRENDER_ANGLEavailable by defaultunavailable by env: OS not supportedWEBRENDER_DCOMP_PRESENTavailable by defaultdisabled by user: User disabled via prefunavailable by env: Requires Windows 10 or laterunavailable by runtime: Requires ANGLEWEBRENDER_SOFTWAREavailable by defaultWEBGPUdisabled by default: Disabled by defaultblocked by runtime: WebGPU cannot be enabled in release or betaVP8_HW_DECODEavailable by defaultVP9_HW_DECODEavailable by defaultBACKDROP_FILTERavailable by defaultCrash Guard Disabled FeaturesWorkaroundsFailure Log

Media

Audio BackendopenslMax Channels2Preferred Sample Rate48000Roundtrip latency (standard deviation)...Output DevicesNameGroupVendorStatePreferredFormatChannelsRateLatencyDefault audio output deviceEnabledAlldefault: S16LE, support: S16LE S16BE F32LE F32BE2default: 44100, support: 44100 - 44100128 - 441Input DevicesNameGroupVendorStatePreferredFormatChannelsRateLatencyDefault audio input deviceEnabledAlldefault: S16LE, support: S16LE S16BE F32LE F32BE1default: 44100, support: 44100 - 44100128 - 441Media Capabilities Enumerate database

Environment Variables

Important Modified Preferences

browser.cache.disk.amount_written164820browser.cache.disk.capacity524288browser.cache.disk.filesystem_reported1browser.search.regionDEdom.presentation.device.nameSM-G930Fdom.w3c_pointer_events.enabledtrueextensions.lastAppVersion105.0.2idle.lastDailyNotification1665349908media.gmp.storage.version.observed1network.dns.disablePrefetchtruenetwork.http.speculative-parallel-limit0network.predictor.enabledfalsenetwork.prefetch-nextfalsenetwork.trr.blocklist_cleanup_donetrueprivacy.purge_trackers.date_in_cookie_database0privacy.purge_trackers.last_purge1665349962273security.remote_settings.crlite_filters.checked1655960865security.remote_settings.intermediates.checked1655960865storage.vacuum.last.index0

Important Locked Preferences

fission.autostart.sessionfalse

Accessibility

ActivatedfalsePrevent Accessibility0

Library Versions

Expected minimum versionVersion in useNSPR4.34.14.34.1NSS3.823.82NSSSMIME3.823.82NSSSSL3.823.82NSSUTIL3.823.82

Startup Cache

Disk Cache Path/data/user/0/org.mozilla.firefox/files/mozilla/1s5t5jqg.default/startupCache/startupCache.8.littleIgnore Disk CachefalseFound Disk Cache on InittrueWrote to Disk Cachefalse

Internationalization & Localization

Application SettingsRequested Locales["de-DE","en-US"]Available Locales["an","ar","ast","az","be","bg","bn","br","bs","ca","cak","cs","cy","da","de","dsb","el","en-CA","en-GB","en-US","eo","es-AR","es-CL","es-ES","es-MX","et","eu","fa","ff","fi","fr","fy-NL","ga-IE","gd","gl","gn","gu-IN","he","hi-IN","hr","hsb","hu","hy-AM","id","is","it","ja","ka","kab","kk","kn","ko","lij","lo","lt","lv","ml","mr","ms","my","nb-NO","ne-NP","nl","nn-NO","oc","pa-IN","pl","pt-BR","pt-PT","rm","ro","ru","sk","sl","son","sq","sr","sv-SE","ta","te","th","tr","trs","uk","ur","uz","vi","wo","xh","zam","zh-CN","zh-TW"]App Locales["de","en-US","en-CA","en-GB"]Regional Preferences["de-DE","en-US"]Default Locale"en-US"Operating SystemSystem Locales["de-DE","en-US"]Regional Preferences["de-DE","en-US"]

Remote Debugging (Chromium Protocol)

Accepting ConnectionsfalseURL

Michael Hohner

Comment 4

[Michael Hohner]

(oh it did work, but still gave me an error message...)

Comment 5

[Jim Mathies [:jimm]]

I can reproduce this on my Pixel 3.

Comment 6

[Jim Mathies [:jimm]]

bouncing this back to you Ashley, maybe you can poke at this when you have some free time.

Comment 7

[az [:az]]

Wanted to leave an update here -- will investigate, though the Android device I need to investigate with was delayed in the mail. It should come in tonight and I'll take a look at this tomorrow.

Comment 8

[az [:az]]

Update - I'm able to repro on a Pixel 6a running Android 12, though I'm not sure what's causing the issue. Going through the logs it looks like it's reading in the resolution/scaling parameters correctly, and nothing's really jumping out to me right now. Will need to investigate further.

Comment 9

[az [:az]]

Update: looks like this might be related to rotation metadata in the media file.

ffprobe reports for S.MOV:

    Side data:
      displaymatrix: rotation of -90.00 degrees

Removing/modifying the metadata to 0 or 180 degrees allows playback at a reasonable aspect ratio / eliminates the weird cropping. For example:

ffmpeg -i /srv/S.MOV -c copy -metadata:s:v:0 rotate=0 /srv/test.mov

This doesn't fix the root problem / provide the desired 90 degree rotation, but I'll investigate further.

Comment 10

[Jamie Nicol [:jnicol]]

Mozregression says I broke this in bug 1731980

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1731980

Comment 12

[Jamie Nicol [:jnicol]]

Okay, I have figured out the problem. This is an existing bug, which the patch from bug 1731980 uncovered by removing the RenderAndroidSurfaceTextureHost::NeedsYFlip() override. The removed overridden implementation always returned false, but the base class implementation always returns true on Android because SurfaceTextureClient sets TextureFlags::ORIGIN_BOTTOM_LEFT. This explains why we don't see this bug on other platforms: because other texture client types do not set that flag meaning TextureHost::NeedsYFlip() returns false.

The value returned from NeedsYFlip gets passed in to webrender until we get to here. With this video, we have a 90 degree rotation. And since 1731980 we also now have a y-flip. The y-flip is implemented by scaling the height by -1, and then translating by content_size.height. But since the video is rotated, content_size.height is actually the width of the video. So we translate by the wrong amount which effectively looks like the video is cropped, but I now notice it's also only displaying the right-most side of the video, which makes sense.

If we apply the y-flip after the rotation, then that solves the issue.

I have a patch ready, but I'm working on adding a test before landing.

Comment 13

[Jamie Nicol [:jnicol]]

Attachment: Bug 1794438 - Handle rotation with vertical flip transform correctly. r?#gfx-reviewers

On Android, videos with a hard-coded rotation were being incorrectly
cropped when rendering. This is due to a buggy transform calculation
in SceneBuilder.build_reference_frame().

To apply the y-flip, we scale the height by -1, then translate the y
offset by content_size.height. However, as this video contains an 90
degree rotation, this means we actually end up translating by the
video's width rather than its height. Because the width and height are
different sizes in this video, this means the video is rendered at an
incorrect offset and is cropped. To fix this, ensure we translate by
content_size.width instead for 90 or 270 degree rotations.

Note that this only occured on Android because SurfaceTextureClient is
the only texture client type that sets the ORIGIN_BOTTOM_LEFT texture
flag, thereby causing a y-flip in this location.

Comment 14

[Jamie Nicol [:jnicol]]

Attachment: Bug 1794438 - Add ability to test push_computed_frame() to wrench. r?#gfx-reviewers

Gecko uses the webrender API DisplayListBuilder.push_computed_frame()
to create a reference frame with a transform specified by a scale_from
size, an optional vertical_flip, and a rotation. This is used for
video.

This patch adds the ability to test this API from wrench yaml. An
example of a test using this capability can be found in the next patch
in this series.

Depends on D160816

Comment 15

[Jamie Nicol [:jnicol]]

Attachment: Bug 1794438 - Add test for computed reference frame transform with yflip and rotation. r?#gfx-reviewers

This test ensures that a y-flip works correctly along with a 90/270
degree transform and a non-square content size.

Depends on D160817

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/64d815871d70
Handle rotation with vertical flip transform correctly. r=gfx-reviewers,bradwerth
https://hg.mozilla.org/integration/autoland/rev/ae1bcfe2f7a1
Add ability to test push_computed_frame() to wrench. r=gfx-reviewers,bradwerth
https://hg.mozilla.org/integration/autoland/rev/16404ecbb5e3
Add test for computed reference frame transform with yflip and rotation. r=gfx-reviewers,bradwerth

Comment 17

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/64d815871d70
https://hg.mozilla.org/mozilla-central/rev/ae1bcfe2f7a1
https://hg.mozilla.org/mozilla-central/rev/16404ecbb5e3

Comment 18

[Chris Peterson [:cpeterson]]

Setting status-firefox107=wontfix. Since this regression has been in product for six months, I don't think we need to rush an uplift to late Beta 107. But if anyone disagrees, feel free to request uplift.