Closed Bug 1731980 Opened 2 years ago Closed 2 years ago

Youtube video playback only fills 1/4 of the expected display box

Categories

(Core :: Graphics: WebRender, defect)

Firefox 94
All
Android
defect

Tracking

()

RESOLVED FIXED
101 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox-esr91 --- unaffected
firefox92 --- wontfix
firefox93 --- wontfix
firefox94 --- wontfix
firefox95 --- wontfix
firefox96 --- wontfix
firefox99 --- wontfix
firefox100 --- wontfix
firefox101 --- fixed

People

(Reporter: vengatesh.alagarsamy, Assigned: jnicol)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: correctness, regression)

Attachments

(6 files, 1 obsolete file)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0

Steps to reproduce:

Build app Using geckoview in android TV(Redmi)
playing SD video content in app.

Actual results:

SD video content is playing on top left corner not in full screen(all other area are black screen).
But playing HD video its playing on full screen.

Expected results:

SD video content should play in full screen in TV by using geckoview 94.

Flags: needinfo?(vengatesh.alagarsamy)

Changing severity to S? because of <rationale>.

Mentor: vengatesh.alagarsamy
Flags: needinfo?(vengatesh.alagarsamy)
Priority: -- → P1
OS: All → Android
Hardware: Unspecified → ARM64
Mentor: vengatesh.alagarsamy
Mentor: vengatesh.alagarsamy

Priority and severity are set by the triage owner when they review the bugs. Mentor assumes that you know how to fix the bug and want to help a community member fix it at some point in the future.

Ideally a sample app or code snippets plus the exact video you are using for testing would be useful. Does the Geckoview Example browser reproduce the same problem?

Mentor: vengatesh.alagarsamy
Priority: P1 → --

Hi Vengatesh, could you attach a sample page/app to reproduce this?

Flags: needinfo?(vengatesh.alagarsamy)

(In reply to Kevin Brosnan [:kbrosnan] from comment #2)

Priority and severity are set by the triage owner when they review the bugs. Mentor assumes that you know how to fix the bug and want to help a community member fix it at some point in the future.

Ideally a sample app or code snippets plus the exact video you are using for testing would be useful. Does the Geckoview Example browser reproduce the same problem?

I have just install the geckoview_example.apk in android TV(Redmi) from the given link.
Playing youtube videos ,same issue is comming playing on top-left corner.

file:///home/arunjos/BK/device-2021-09-24-182353.png

Flags: needinfo?(vengatesh.alagarsamy)

I have just install the geckoview_example.apk in android TV(Redmi) from the given link.
Playing youtube videos ,same issue is comming playing on top-left corner.

I build app with implementation "org.mozilla.components:browser-engine-gecko:92.0.20210720182827 in android TV,its playing all SD and HD video in full screen.
But in this version i am getting video freeze in both mobile and TV(audio is playing but video stopped).
So, i moved to org.mozilla.components:browser-engine-gecko:94.0.20210907190136 version.
From this version i am not getting video freeze in both mobile and TV ,But in TV video is playing on top-left corner.(SD video with ABR).

I can reproduce this on Firefox 92 but not 91 using the x86 build on emulator.

Still reproduces on a 94 x86 nightly.
94.0a1 (Build #2015836685)
AC: 94.0.20210927143215, 3777ac76a7
GV: 94.0a1-20210927093458
AS: 84.0.0
Mozilla/5.0 (Android 11; Mobile; rv:94.0) Gecko/94.0 Firefox/94.0

Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: regression
Hardware: ARM64 → x86

2021-09-28T16:35:15.676000: INFO : Narrowed integration regression window from [2c6a5828, ccc6e271] (4 builds) to [2c6a5828, 19948f58] (2 builds) (~1 steps left)
2021-09-28T16:35:16.247000: DEBUG : Found commit message:
Bug 1721190 - Allow fallback from WR to SW-WR on Android. r=jrmuizel
Differential Revision: https://phabricator.services.mozilla.com/D120247
2021-09-28T16:35:16.247000: INFO : The bisection is done.

Has Regression Range: --- → yes
Has STR: --- → yes
Component: General → Graphics: WebRender
Product: GeckoView → Core
Regressed by: 1721190
Summary: I am using Geckoview in android tv, while playing SD video content it's playing on top left corner all other area in video is black screen. But HD video are playing good. → Youtube video playback only fills 1/4 of the expected display box - Intel Android

It sounds like we have 2 bugs - first something is going wrong causing us to fall back to SWGL from hardware webrender, and secondly the video only gets rendered in the top left corner in SWGL.

Could you please go to about:support, click "copy text to clipboard", and attach the contents to this bug? You mention both a mobile and android TV device, could you do that for both please?

Kevin, is there anything in the logcat when you reproduce this on the emulator?

Flags: needinfo?(vengatesh.alagarsamy)
Flags: needinfo?(kbrosnan)
Flags: needinfo?(kbrosnan)

(In reply to Kevin Brosnan [:kbrosnan] from comment #9)

2021-09-28T16:35:15.676000: INFO : Narrowed integration regression window from [2c6a5828, ccc6e271] (4 builds) to [2c6a5828, 19948f58] (2 builds) (~1 steps left)
2021-09-28T16:35:16.247000: DEBUG : Found commit message:
Bug 1721190 - Allow fallback from WR to SW-WR on Android. r=jrmuizel
Differential Revision: https://phabricator.services.mozilla.com/D120247
2021-09-28T16:35:16.247000: INFO : The bisection is done.

what is the solution for this issue?
Is there any possibilities get revert this changes and to avoid this issue when build geckoview from source?.

Flags: needinfo?(vengatesh.alagarsamy)

I'm afraid you cannot simply revert that change, as the code from the old fallback rendering path has been removed.

In Kevin's logcat I can see that we are using SWGL fallback because the emulator has no GLES 3 support. If he were to configure the emulator to allow GLES 3 (on linux: create the file ~/.android/advancedFeatures.ini and add the line GLESDynamicVersion = on) I suspect he would no longer be able to reproduce. However, setting gfx.webrender.software to true in about config would reproduce again (after a restart).

So it definitely seems like the video rendering in the top left corner is a SWGL bug. But I'm equally curious as to why your device is falling back to SWGL in the first place. From your description of it freezing, I suspect it occurs when attempting to play video rather than at startup. That's also an important bug which we should fix.

Vengatesh, could you please attach the about:support information from both your mobile and TV device to this bug. Could you please also capture a logcat when opening firefox and triggering this bug, then attach it here. That information will be invaluable for fixing this issue. Thanks!

Flags: needinfo?(vengatesh.alagarsamy)

Kevin, did you do anything in particular to reproduce on the emulator? What OS, emulator version, and AVD are you using?

Flags: needinfo?(kbrosnan)

(In reply to Jamie Nicol [:jnicol] from comment #14)

I'm afraid you cannot simply revert that change, as the code from the old fallback rendering path has been removed.

In Kevin's logcat I can see that we are using SWGL fallback because the emulator has no GLES 3 support. If he were to configure the emulator to allow GLES 3 (on linux: create the file ~/.android/advancedFeatures.ini and add the line GLESDynamicVersion = on) I suspect he would no longer be able to reproduce. However, setting gfx.webrender.software to true in about config would reproduce again (after a restart).

So it definitely seems like the video rendering in the top left corner is a SWGL bug. But I'm equally curious as to why your device is falling back to SWGL in the first place. From your description of it freezing, I suspect it occurs when attempting to play video rather than at startup. That's also an important bug which we should fix.

Vengatesh, could you please attach the about:support information from both your mobile and TV device to this bug. Could you please also capture a logcat when opening firefox and triggering this bug, then attach it here. That information will be invaluable for fixing this issue. Thanks!

From my point of view in new version ABR is working fine with web rendering thats why i am not getting any video freeze issue,the same thing for some devices(Redmi Tv) that web rendering is breaking.
can I please get in when the bug will get solved?.

Flags: needinfo?(vengatesh.alagarsamy)

I am trying to fix the bug as quickly as possible. In order to do so it would be very helpful if you could provide me with the about:support information from your device, as well as a logcat taken when the bug occurs

Flags: needinfo?(vengatesh.alagarsamy)
Attached file logcat for redmi TV
(In reply to Jamie Nicol [:jnicol] from comment #17)
> I am trying to fix the bug as quickly as possible. In order to do so it would be very helpful if you could provide me with the about:support information from your device, as well as a logcat taken when the bug occurs

(In reply to Jamie Nicol [:jnicol] from comment #17)

I am trying to fix the bug as quickly as possible. In order to do so it would be very helpful if you could provide me with the about:support information from your device, as well as a logcat taken when the bug occurs

I attached the logcat for Redmi TV (full screen issue)

Flags: needinfo?(vengatesh.alagarsamy)

Any update on this issue sir?

have you found any fix for this (In reply to Jamie Nicol [:jnicol] from comment #17)

I am trying to fix the bug as quickly as possible. In order to do so it would be very helpful if you could provide me with the about:support information from your device, as well as a logcat taken when the bug occurs

Have you found any fix for this issue?

I'm afraid I haven't been able to reproduce the issue on any of my devices. Are there any other websites where you see the issue? Or are you able to share the source code or apk of your geckoview app?

Flags: needinfo?(vengatesh.alagarsamy)

Its only reproducing in Mali-450 processor devices.
just build (In reply to Jamie Nicol [:jnicol] from comment #22)

I'm afraid I haven't been able to reproduce the issue on any of my devices. Are there any other websites where you see the issue? Or are you able to share the source code or apk of your geckoview app?

Its only reproducing in Mali-450 processor devices.
just https://nightly.maven.mozilla.org/?prefix=maven2/org/mozilla/components/browser-engine-gecko/94.0.20210930143139/ try this binary app in Mali-450 processor devices(Redmi TV) it will produce that issue.

Flags: needinfo?(vengatesh.alagarsamy)

Set release status flags based on info from the regressing bug 1721190

(In reply to Vengatesh Alagarsamy from comment #23)

Its only reproducing in Mali-450 processor devices.

I have Mali-400 android phone, but I could not reproduce the problem with GeckoView example app. And I tried to reproduce the problem with emulator, but I could not reproduce it.

I wonder if RenderCompositorOGLSWGL::HandleExternalImage() does not work as expected when the problem happened.

(In reply to Sotaro Ikeda [:sotaro] from comment #25)

(In reply to Vengatesh Alagarsamy from comment #23)

Its only reproducing in Mali-450 processor devices.

I have Mali-400 android phone, but I could not reproduce the problem with GeckoView example app. And I tried to reproduce the problem with emulator, but I could not reproduce it.

In all mobile its work fine.Only issue with Android TV.
I checked with two TV

  1. Redmi (Mali 450) ->its having that full screen issue
    2.TCL(Mali 470) -> its working fine.
    Thats why i am telling its having issue in Mali 450 only.

(In reply to Sotaro Ikeda [:sotaro] from comment #25)

(In reply to Vengatesh Alagarsamy from comment #23)

Its only reproducing in Mali-450 processor devices.

I have Mali-400 android phone, but I could not reproduce the problem with GeckoView example app. And I tried to reproduce the problem with emulator, but I could not reproduce it.

Are you able to reproduce this issue in Android TV (Mali 450) ?Is any ideo in which module its causing the issue?After the update on July 21 and 22 ,its causing this issue.

(In reply to Sotaro Ikeda [:sotaro] from comment #25)

(In reply to Vengatesh Alagarsamy from comment #23)

Its only reproducing in Mali-450 processor devices.

I have Mali-400 android phone, but I could not reproduce the problem with GeckoView example app. And I tried to reproduce the problem with emulator, but I could not reproduce it.

Have you found the issues?will you please tell any other way to solve this issue.

Unfortunately neither Soraro or I have been able to reproduce this issue so have been unable to find a solution.

Kevin, are you still able to reproduce in an emulator? If so, what AVD are you using? What android version, screen dimensions etc. What are the GPU acceleration settings? What host OS are you running on and what GPU does the host computer have?

Flags: needinfo?(kbrosnan)

Re-needinfoing for above question

Flags: needinfo?(kbrosnan)

I can't reproduce this with my current VMs. I did some cleanup a week or two ago and may have deleted the VM that reproduced this. Tried all the configs that I had downloaded (Android 10, 6 and 5.1) and could not reproduce. Not sure that the host info will be useful here but in case it is,

Host: Windows 10 21H1
Host GPU: Nvidia 980 driver v466.27 (27.21.14.6627)

Flags: needinfo?(kbrosnan)
Blocks: wr-android
Keywords: correctness
Hardware: x86 → All
Summary: Youtube video playback only fills 1/4 of the expected display box - Intel Android → Youtube video playback only fills 1/4 of the expected display box

Can confirm for Lenovo TB3-850M with Android 6. Videos works fine in official Youtube app, but not in Firefox. Youtube, Vimeo, Facebook videos reproduces this issue:

https://i.imgur.com/HQemo8v.png

Some more info:

Compositing: WebRender
WebGL 1 Driver Renderer: ARM -- Mali-T720
WebGL 1 Driver Version: OpenGL ES3.1

Severity: -- → S3
See Also: → 1757031

The severity field for this bug is relatively low, S3. However, the bug has 6 See Also bugs.
:gw, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Flags: needinfo?(gwatson)
Flags: needinfo?(gwatson)

I've finally been able to reproduce this, so am looking in to it

Assignee: nobody → jnicol

On Android we use SurfaceTextures to render content from sources such
as the video decoder. These may have a transform set which is supposed
to be applied to the texture coordinates used to sample the
texture. Webrender (and software webrender), however, do not handle
this correctly, meaning videos may be rendered at the incorrect size
on some devices.

SurfaceTextures should always be rendered with their bottom-left being
their origin, eg vertically flipped. Additionally, the texture
transform returned on most devices seems to be a simple y-flip
transform with no scaling. Webrender currently just ignores the y-flip
due to the texture origin, which cancels out us not handling the
y-flip from the transform, meaning video looks correct on most
devices. Some devices, however, do return a scaling transform which we
must handle.

This patch removes the override of WebRenderTextureHost::NeedsYFlip()
that was causing us to ignore the y-flip due to the texture origin -
since we will now apply the transform we must handle this correctly
too.

It adds a virtual method RenderTextureHost::GetUvCoords(), that
returns the texture coordinates that should be used by webrender to
sample from external textures. In most cases these are simply (0, 0)
and (size.x, size.y), but in RenderAndroidSurfaceTextureHost we
override this function to apply the transformation. This ensures we
use the correct coordinates whenever the texture is rendered by
webrender, eg in both software and hardware webrender when rendering
in the non-compositing-path, and by hardware webrender's draw
compositor. Additionally, the composite.glsl shader requires a fix to
calculate the UV bounds correctly, as the coordinates may now be
inverted.

Lastly, we fix software webrender with the OpenGL
compositor. CompositorOGL already has the required functionality to
apply the texture transformation as it was used back in the layers
days. We must simply ensure that we pass the value of the
mIgnoreTransform flag from the original SurfaceTextureHost, through to
the RenderAndroidSurfaceTextureHost, and finally to the
SurfaceTextureSource which we hand to CompositorOGL.

On Android we use SurfaceTextures to render content from sources such
as the video decoder. These may have a transform set which is supposed
to be applied to the texture coordinates used to sample the
texture. Webrender (and software webrender), however, do not handle
this correctly, meaning videos may be rendered at the incorrect size
on some devices.

SurfaceTextures should always be rendered with their bottom-left being
their origin, eg vertically flipped. Additionally, the texture
transform returned on most devices seems to be a simple y-flip
transform with no scaling. Webrender currently just ignores the y-flip
due to the texture origin, which cancels out us not handling the
y-flip from the transform, meaning video looks correct on most
devices. Some devices, however, do return a scaling transform which we
must handle.

This patch removes the override of WebRenderTextureHost::NeedsYFlip()
that was causing us to ignore the y-flip due to the texture origin -
since we will now apply the transform we must handle this correctly
too.

It adds a virtual method RenderTextureHost::GetUvCoords(), that
returns the texture coordinates that should be used by webrender to
sample from external textures. In most cases these are simply (0, 0)
and (size.x, size.y), but in RenderAndroidSurfaceTextureHost we
override this function to apply the transformation. This ensures we
use the correct coordinates whenever the texture is rendered by
webrender, eg in both software and hardware webrender when rendering
in the non-compositing-path, and by hardware webrender's draw
compositor. Additionally, the composite.glsl shader requires a fix to
calculate the UV bounds correctly, as the coordinates may now be
inverted.

Lastly, we fix software webrender with the OpenGL
compositor. CompositorOGL already has the required functionality to
apply the texture transformation as it was used back in the layers
days. We must simply ensure that we pass the value of the
mIgnoreTransform flag from the original SurfaceTextureHost, through to
the RenderAndroidSurfaceTextureHost, and finally to the
SurfaceTextureSource which we hand to CompositorOGL.

Attachment #9273617 - Attachment is obsolete: true
Pushed by jnicol@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/26f0e55c45e7
Ensure SurfaceTextures with transforms get rendered at correct size. r=gfx-reviewers,lsalzman
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 101 Branch
Regressions: 1778150
Regressions: 1784109
Regressions: 1794438
See Also: → 1814811
Regressions: 1825631
You need to log in before you can comment on or make changes to this bug.