1548116 - When playing video horizontally, the screen is black

Status: RESOLVED FIXED

(Core :: Graphics, defect, P3)

Description

[Mike Kaply [:mkaply]]

Attachment: Screenshot of problem

Description: When playing video horizontally, the screen is black.

Test Phones: Huawei device in research /Google Pixel with 1080x2340 resolution

Link to download the App: https://play.google.com/store/apps/details?id=org.mozilla.firefox

App Version: 64.0.1

Test Steps:

1,Auto-rotate screen is Open

2,Launch Firefox

3,Watch a video website and play a video

4, Place the phone horizontally

Actual result: the screen is black.

Occurrence Rate: 10/10 times

Analysis & Suggestion：

Switch pixel resolution to 1080x2340, the issue can also be reproduced.

Please adapted the resolution of 1080x2340.

Adb cut bas resolution command:

Adb shell wm size 1080x2340

Adb shell wm density 480

Additional information:

How do you think this issue? I test some video websites include www.abc.es / www.youku.com.

1, And If I use Firefox, these videos have something in common that if click “full-screen display”, this video would not rotated automatically, except that the area outside the screen video turns gray. Then Horizontally placing the device.

2, But if I use Chrome, there is no such issue.

This issue seems to have nothing to do with the resolution of the phone.

Sorry about that, it same this issue is still display. With Firefox 65.0.1.

Turn on the Auto-rotate. And use Firefox open this link:

https://www.abc.es/espana/abci-juicio-proces-defensa-junqueras-apela-constitucion-proces-pisoteo-201902122108_noticia.html

click “full-screen display”and Horizontally placing the device.

Comment 1

[Liz Henry (:lizzard) (relman/hg->git project)]

Sorina, can you or someone else in QA try to reproduce this in a current nightly? Thanks!

Comment 2

[Liz Henry (:lizzard) (relman/hg->git project)]

Mike are you still able to see this problem?

Comment 3

[Mike Kaply [:mkaply]]

Yes, still recreatable on latest Fennec. Screen is completely black when you rotate.

It also happens on Firefox Preview (Fenix) although at least the video controls are visible.

Comment 4

[Liz Henry (:lizzard) (relman/hg->git project)]

Stefan, I'm setting this to P1 since having video work seems fairly important and the issue also affects Fenix. Can you help find someone to work on this?

Comment 5

[Sorina Florean [:sflorean]]

Tested with Xiaomi Mi 8 Lite (Android 9) and I confirm that the issue is reproducible on Nightly build and Firefox 68.0.
Wasn't able to reproduce with Google Pixel (Android Q).

Comment 6

[Brad Arant]

I will take a stab at this. Was able to recreate from nightly on Samsung Note 8 running Android 9 from the www.abc.es site. Strangely enough the ad videos that interject into the video window before the main video starts seem to work appropriately. From USA the site is slow but I am able to test with it.

Comment 7

[Brad Arant]

After analysis is appears to be within the Gecko engine and @petru.lingurar also reports the same issue on Fenix (Fenix issue: https://github.com/mozilla-mobile/fenix/issues/4249). Please advise on setting status of this ticket.

Comment 8

[Chris Peterson [:cpeterson]]

(In reply to Brad Arant from comment #7)

After analysis is appears to be within the Gecko engine and @petru.lingurar also reports the same issue on Fenix (Fenix bug 4249). Please advise on setting status of this ticket.

Since this black video bug affects both Fennec and Fenix, then it's likely a Gecko video or graphics bug. I'll forward this bug to the Gecko Media Playback team.

Comment 9

[John Lin [:jhlin][:jolin]]

For the problem that abc.es and youku.com do not rotate automatically when entering/exiting fullscreen, that's because the feature is implemented in the built-in media playback controls but both sites use their own controls.

For the abc.es black screen issue, I can reproduce this issue in Fennec Beta but not in GeckoView Example. Also, whether it displays a black screen or the actual content, the log shows media playback pipeline works normally. This doesn't look like a video bug to me.

I am not sure which component would be a good candidate. Chris, could you please help?

Comment 10

[Chris Peterson [:cpeterson]]

(In reply to John Lin [:jhlin][:jolin] from comment #9)

For the abc.es black screen issue, I can reproduce this issue in Fennec Beta but not in GeckoView Example. Also, whether it displays a black screen or the actual content, the log shows media playback pipeline works normally. This doesn't look like a video bug to me.

I am not sure which component would be a good candidate. Chris, could you please help?

If you can reproduce the black screen in Fennec but not GeckoView Example, this might be an e10s issue. (Fennec doesn't use e10s.) But Mike Kaply said in comment 3 three months ago that he could reproduce the black screen in Fennec. I wonder why it works in your Fennec. (I can't test because I don't have my Android phone with me, atm.)

If the media playback pipeline is working, this might be a graphics issue. I'll send this bug to the Graphics team.

Comment 11

[Chris Peterson [:cpeterson]]

This is probably not an e10s bug if since it affects Fenix. I'm not sure why the bug would affect Fennec and Fenix, but not GeckoView Example.

Comment 12

[Jessie [:jbonisteel] pls NI]

Curious if you can reproduce this, Jamie.

Comment 13

[Jamie Nicol [:jnicol]]

I can reproduce this is fennec, and fenix with webrender disabled. But I cannot reproduce with webrender enabled.
It would be interesting to know whether this is a regression, and to find the regression range if so.

Comment 14

[Jessie [:jbonisteel] pls NI]

Can you repro this in the GV example app? I am trying to figure out if this is actually a Graphics issue

Comment 15

[Chris Peterson [:cpeterson]]

In comment 9, John Lin said:

For the abc.es black screen issue, I can reproduce this issue in Fennec Beta but not in GeckoView Example. Also, whether it displays a black screen or the actual content, the log shows media playback pipeline works normally. This doesn't look like a video bug to me.

I don't know if John tried to reproduce the issue on the youku.com website or not.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Comment 17

[Jamie Nicol [:jnicol]]

Mozregression gives me this regression range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=709f355a7a8c4ae426d1824841a71ffdb5ce0137&tochange=79d3e25106f8eb369dde6a47199547d131af8d3d

I'll try to bisect further by hand.

Comment 18

[Jessie [:jbonisteel] pls NI]

We haven't been able to narrow the regression range down further and trying to get older builds going has not been working. We cannot reproduce this on YouTube.

WebRender also fixes this.

Comment 19

[Pascal Chevrel:pascalc (PTO until April 26)]

Unassigned P3 and an old regression, not blocking 71.

Comment 20

[Emily Toop (:fluffyemily)]

This has been moved to Fenix's P1 list and GeckoView's 76 wish list.

Comment 21

[Jamie Nicol [:jnicol]]

I can take another look at this.

It would be useful to know if this affects any websites other than abc.es?

Comment 22

[Jamie Nicol [:jnicol]]

Hi Emily, Emily Toop said you might be able to answer whether this affects other websites than just abc.es? Is there a related Fenix bug with any other information?

Comment 23

[Jamie Nicol [:jnicol]]

I've figured this out, but it's not good news unfortunately.

The reason this affects landscape and not portrait is because the stylesheet has this in it:

@media (min-width:720px) {
 .vjs-tech {
  filter:blur(0) saturate(1) hue-rotate(0) brightness(1) contrast(1.1) invert(0) sepia(0);
  -webkit-filter:blur(0) saturate(1) hue-rotate(0) brightness(1) contrast(1.1) invert(0) sepia(0)
 }
}

So in landscape mode the width is greater than 720, so the page wraps the video element in a filter. Webrender handles this fine because it can apply the filter on the GPU. But in layers we need to do filters in software. Which means that we need to read back the video frame data in the content process, which we can't do on android because we use SurfaceTextures.

We've encountered similar issues before, usually in the form of trying to use a video along with canvas2d. It's the same underlying problem.

Fixing this would be a very large amount of work, for something that webrender will fix for us. Can we reach out to the website instead to see if they can remove that filter?

Comment 24

[Emily Kager [:ekager]]

The only related Fenix bug I'm seeing is the original opened by mkaply (and copied over to comment 0 here) :/
https://github.com/mozilla-mobile/fenix/issues/4249

Comment 25

[Jamie Nicol [:jnicol]]

Thanks! It'd be useful to know whether the priority is set so high on the basis that it might affect all fullscreen videos, rather than just this website (or other websites putting a filter on a video)

Comment 26

[Mike Kaply [:mkaply]]

The priority was set high originally because it was reported by a partner who was preloading Firefox. That's not the case anymore.

Comment 27

[Emily Toop (:fluffyemily)]

Fenix would like to see this issue resolved in Q2

Comment 28

[Jamie Nicol [:jnicol]]

Using ImageReader and AHardwareBuffer rather than SurfaceTexture for video data should allow us to fix this. It will only work on Android O or P onwards, however.

Comment 29

[Mike Kaply [:mkaply]]

(In reply to Jamie Nicol [:jnicol] from comment #28)

Using ImageReader and AHardwareBuffer rather than SurfaceTexture for video data should allow us to fix this. It will only work on Android O or P onwards, however.

Any plans to do that?

Comment 30

[Jamie Nicol [:jnicol]]

Not sure. This specific site should now work correctly on all devices, as webrender avoided the problem in this instance (by applying the filter on the GPU rather than having to read back the video frame on the CPU).

But there is still the underlying issue with SurfaceTextures. Sotaro, what is the status of the AHardwareBuffer support?

Comment 31

[Sotaro Ikeda [:sotaro]]

(In reply to Jamie Nicol [:jnicol] from comment #30)

Not sure. This specific site should now work correctly on all devices, as webrender avoided the problem in this instance (by applying the filter on the GPU rather than having to read back the video frame on the CPU).

But there is still the underlying issue with SurfaceTextures. Sotaro, what is the status of the AHardwareBuffer support?

Without AHardwareBuffer usage, content side readback(GetAsSourceSurface()) problem could be addressed. It is going to be done by Bug 1526207.

https://searchfox.org/mozilla-central/rev/daa5c195e5efe45a63bf79f97ae77419b16f62f1/gfx/layers/GLImages.h#53

Comment 32

[Chris Peterson [:cpeterson]]

(In reply to Sotaro Ikeda [:sotaro] from comment #31)

(In reply to Jamie Nicol [:jnicol] from comment #30)

Not sure. This specific site should now work correctly on all devices, as webrender avoided the problem in this instance (by applying the filter on the GPU rather than having to read back the video frame on the CPU).

But there is still the underlying issue with SurfaceTextures. Sotaro, what is the status of the AHardwareBuffer support?

Without AHardwareBuffer usage, content side readback(GetAsSourceSurface()) problem could be addressed. It is going to be done by Bug 1526207.

Sotaro, can this bug be closed as a duplicate of bug 1526207?

Or can we close this bug as "fixed" if all Android users are now getting WebRender (accelerated or software)?

Comment 33

[Sotaro Ikeda [:sotaro]]

We could set this bug as fixed, since all Android users are now getting WebRender (accelerated or software).