1155424 - White bar flashes at screen bottom while coming out of full-screen video

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::System::Status bar, Utility tray, Notification, defect, P3)

Description

[Diego Wilson [:diego]]

This happens on Flame v2.2

STR:

1.Open browser
2.Go to rtsp://203.35.158.135/TRL/trl-upload/Highbit/mpeg4_aac_114k.3gp
3.Click on fullscreen button
4.Click on window mode button to switch back to windowed mode

1/5 times you'll see a white bar flash at the bottom of the screen after coming out of full screen.

Comment 1

[Diego Wilson [:diego]]

Screen recording: https://www.dropbox.com/s/legsrp8flcqoupp/2015-04-16%2022.38.28.mp4?dl=0

Comment 2

[Vincent Chang[:vchang][changyihsin]]

Hi Jonathan, can you help to check this?

Comment 3

[Jonathan Hao (inactive) [:jhao]]

I can also reproduce this bug on an HTTP streaming video, so the problem is not in RTSP component, and I suspect it's Gaia::Browser's issue.

Comment 4

[Vincent Chang[:vchang][changyihsin]]

Thanks, Jonathan.

Comment 5

[Mike Lee [:mlee]]

(In reply to Jonathan Hao [:jhao] from comment #3)
> I can also reproduce this bug on an HTTP streaming video, so the problem is
> not in RTSP component, and I suspect it's Gaia::Browser's issue.

Gregor,

Can you have Ben, Dale, or someone else familiar with the Gaia Browser code look into this?

Thanks,
Mike

Comment 6

[Gregor Wagner [:gwagner]]

Thats not browser related code. Thats probably a gfx issue in our video playback widget.

Comment 7

[Gregor Wagner [:gwagner]]

Anthony, does your team handle this?

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

Given it happens with other videos and other protocols, I would guess either the player itself, just how we code in and out of full screen, or graphics.  Doubt it's video.

Comment 9

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

From my APZC experience a couple of years back, I'd start looking at APZC.

Comment 10

[Kartikaya Gupta (email:[e-mail])]

(In reply to Gregor Wagner [:gwagner] from comment #6)
> Thats not browser related code. Thats probably a gfx issue in our video
> playback widget.

What makes you say it's not browser related code? From the video it looks to me like the white space at the bottom is the same size as the dynamic toolbar so that would be my first guess.

Comment 11

[Gregor Wagner [:gwagner]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #10)
> (In reply to Gregor Wagner [:gwagner] from comment #6)
> > Thats not browser related code. Thats probably a gfx issue in our video
> > playback widget.
> 
> What makes you say it's not browser related code? From the video it looks to
> me like the white space at the bottom is the same size as the dynamic
> toolbar so that would be my first guess.

Hm good point. Ben, do we have any code in the browser that is related here?

Comment 12

[Ben Francis [:benfrancis]]

I'm not sure if this is related but I notice that when I overscroll the page at rtsp://203.35.158.135/TRL/trl-upload/Highbit/mpeg4_aac_114k.3gp I can see a white background showing through rather than the background scaling. Is there something special about the layers of these built-in HTML5 media player pages that makes the overscroll work differently?

It's also possible to get the .3gp page into a state where the Rocketbar is collapsed but there's a white background still showing where the expanded Rocketbar was, I think I remember seeing a separate bug filed for that.

Playing with YouTube videos it seems that exiting fullscreen is a bit janky in general, it's difficult to tell whether this is bad CSS or graphics struggling to keep up or both.

Comment 13

[Diego Wilson [:diego]]

Thanks for the analysis Ben! Is there anyone you recommend to take this bug forward?

Comment 14

[Peter Chang[:pchang]]

(In reply to Ben Francis [:benfrancis] from comment #12)
> I'm not sure if this is related but I notice that when I overscroll the page
> at rtsp://203.35.158.135/TRL/trl-upload/Highbit/mpeg4_aac_114k.3gp I can see
> a white background showing through rather than the background scaling. Is
> there something special about the layers of these built-in HTML5 media
> player pages that makes the overscroll work differently?
> 
> It's also possible to get the .3gp page into a state where the Rocketbar is
> collapsed but there's a white background still showing where the expanded
> Rocketbar was, I think I remember seeing a separate bug filed for that.
> 
> Playing with YouTube videos it seems that exiting fullscreen is a bit janky
> in general, it's difficult to tell whether this is bad CSS or graphics
> struggling to keep up or both.

The white background is the background color of browser's mozborwser iframe[1] in system app. If I changed to red, I could see the red background displayed when exiting fullscreen or overscroll during the page is loading.

For me, it's a kind of timing issue when the size of app and rocketbar are changing at the same time.

[1]https://github.com/mozilla-b2g/gaia/blob/master/apps/system/style/window.css#L61

Comment 15

[Keven Kuo [:kkuo]]

Hi! Tim,

Could your team help to take a look? Thanks

--
Keven

Comment 16

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Redirect to systemsfe for rocket bar issues.

Comment 17

[Gregor Wagner [:gwagner]]

Can we do a branch check?

Comment 18

[Ben Francis [:benfrancis]]

I'll try and investigate this further

Comment 19

[Yeojin Chung [:YeojinC]]

This bug reproduces on Flame Master, 2.2, and 2.1. When getting out of the fullscreen mode, the white section at the bottom of the screen flashes briefly. 

Environmental Variables:
Device: Flame Master
Build ID: 20150429164621
Gaia: db8ea705c0fd1b1684807f5a8e837bb9a36a6f96
Gecko: 4b9b12c248dc
Version: 40.0a1 (Master)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0

Environmental Variables:
Device: Flame 2.2
Build ID: 20150429203150
Gaia: aa1da5036f9425c25d515d14243d3473bfefb4fd
Gecko: 38b2838d43e1
Version: 37.0 (2.2)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

Environmental Variables:
Device: Flame 2.1
Build ID: 20150429180123
Gaia: 7cd76434b58be2cfef7d23451e668c76591f05c3
Gecko: b5d2472c2940
Version: 34.0 (2.1)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

=====================================
This issue does NOT reproduce on Flame 2.0. The white section at the bottom does not appear, most likely because the browser chrome is not a feature in 2.0. Therefore, this bug is not applicable in 2.0.

Environmental Variables:
Device: Flame 2.0
Build ID: 20150429180120
Gaia: 84898cadf28b1a1fcd03b726cff658de470282f0
Gecko: 9e3136b90c12
Version: 32.0 (2.0)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0

Comment 20

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Maybe this might help ? 
https://github.com/mozilla-b2g/gaia/compare/v2.0...v2.1

Comment 21

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

To note, we also have support for full screen with the software home button in v2.1.  So we should probably test the fix/issue with a software home button as well.

Comment 22

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

It turns out that we still get the white over the software home button.

Well, this is weird.  I tried on a test page : http://people.mozilla.org/~nhirata/fullscreen_test.html 
and I did not get any white flashing.  ( tap on the word a squirrel and then tap the screen to return from fullscreen)

In the logcat, it shows the same scroll and resizing as it does in the video.

Comment 23

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

I think it has to do with something odd in the full screen movie css layout and timing with the browser chrome scroll and resize.

Comment 24

[Michael Henretty [:mikehenrty][:mhenretty]]

Just as an update, we have a suspicion this might be a graphics issue. Ben is working on a reduced test case to suss out where the problem is.

Comment 25

[Peter Chang[:pchang]]

(In reply to Michael Henretty [:mhenretty] from comment #24)
> Just as an update, we have a suspicion this might be a graphics issue. Ben
> is working on a reduced test case to suss out where the problem is.

I still believe this is a timing issue between app and rocket bar which I mentioned in comment 14. The white background is the default background of mozbrowser iframe in system app for the launched app.

Comment 26

[Chris Lord [:cwiiis]]

Has anyone looked at the sequence of events that happens when a video is full-screened? Like what signals get fired and what styles get applied and when? It sounds premature to call this a gfx issue, there are likely several places it could be (apologies if there's been investigation that isn't cited on this bug).

Comment 27

[Ben Francis [:benfrancis]]

Chris, any chance we could find some time to work on debugging this together in London some time next week? I could do with your help.

Comment 28

[Chris Lord [:cwiiis]]

Sure, sounds like a plan!

Comment 29

[Dylan Oliver [:doliver]]

Hi Ben, can you provide an update on this one?

Comment 30

[Chris Lord [:cwiiis]]

Ben and I have been looking at this for a reasonable amount of time... We're still uncertain what this is, but I'm certainly leaning towards this being a Gecko/Gonk bug.

As far as we can tell, nothing actually changes in the system DOM when you fullscreen/unfullscreen, but there is clearly more than one frame being painted when you exit fullscreen (which implies that either something is changing in the DOM in response to the fullscreen change, or that there's some other issue).

Looking at the widget code, gonk/nsWindow.cpp::MakeFullScreen calls ::Resize, which sets its bounds and calls ::Invalidate, which just sets a 'please draw' flag, signals any listeners and spins the event loop. Then, at some later time, this flag is checked during the event loop and I suppose then that the redraw actually happens.

This is my flawed interpretation of events, but I don't know much about this code... It does seem though that this is a very asynchronous process, and that it's prone to these kind of drawing glitches.

n?alive to see if perhaps there's some fullscreen handling code in Gaia that we don't know about and haven't been able to find, and n?mwu to see if there's any comment about this from a Gonk/widget perspective. If either of you feel there's someone better to ask about answering this, please do pass it on :)

Comment 31

[(inactive after 6/18) Alive Kuo [:alive]]

gaia system has nothing to do when entering fullscreen mode but only showing the permission dialog if necessary. When the DOM element leaves fullscreen mode we don't do anything in JS.

But one thing to note is that we have CSS to control the style when there is fullscreen elements
example:
#screen.software-button-enabled:not(.fullscreen-layout-app):not(.locked):-moz-full-screen-ancestor #fullscreen-software-home-button,
#screen.software-button-enabled.locked.secure-app:not(.fullscreen-layout-app):-moz-full-screen-ancestor #fullscreen-software-home-button {
  visibility: visible;
}

Not sure if this is the problem; I agree with Chris that this is not gaia issue because we can't control the redraw of the system app. My 2$ would be we should wait the repaint of the system app when leaving fullscreen mode in gecko/gonk; but I am not sure how to achieve that.

Comment 32

[Ben Francis [:benfrancis]]

Note that you'll also see weird behaviour when exiting fullscreen on a YouTube video where it seems to show the fullscreen element in the non-fullscreen size before repainting with the correct content, it isn't limited to the built in media pages. May just be a general graphics/layers issue with fullscreen?

Comment 33

[Dylan Oliver [:doliver]]

Kats, is it worth taking another look at this one from the graphics perspective now?

Comment 34

[Kartikaya Gupta (email:[e-mail])]

As far as I can tell from the discussion above there is nothing actually point to graphics code as the culprit. So far it seems like we have concluded that there are multiple repaints happening when coming out of fullscreen, and I agree with that. We need to determine why there are multiple repaints happening, which is not a graphics issue (because graphics just processes the repaints, but doesn't trigger them). It would probably help to capture the frame tree and display list for each repaint as you exit fullscreen to see exactly what changes are happening and why repaints are being triggered.

Comment 35

[Ben Francis [:benfrancis]]

I don't think there's much more I can help with on this bug, we need support from the platform side. Waiting to hear from mwu on advice on who can help.

Comment 36

[Michael Wu [:mwu]]

Kat's advice sounds better than anything I could come up with. This stuff occurs on a higher level than I normally deal with.

Comment 37

[Daniel Holbert [:dholbert]]

I can reproduce, using a build from b2g trunk, and loading this video directly in the browser:
 http://media.tinyvid.tv/1pjr8ctdx0bpy.ogg
(going between full-screen & non-full-screen)

I'll follow up on comment 34 (poking at some frame trees and/or display lists) either over the weekend or early next week.

Comment 38

[Daniel Holbert [:dholbert]]

One data point: if I place a breakpoint in nsVideoFrame::BuildDisplayList, and (manually) continue in GDB each time that breakpoint is hit, then I can't reproduce this bug.  (I do see us changing between a few intermediate renderings as we exit full-screen mode, as the breakpoint is hit, but none of them show the white bar at the bottom.)

That makes me suspect there's a layers/compositing race condition of some sort going on here, though I don't know the layers code well enough to be sure.

Comment 39

[Daniel Holbert [:dholbert]]

So basically it looks like what's happening is: when we leave full-screen, we shrink browser's viewport to be 30px smaller, presumably to save some room for the status bar & maybe the URL bar.

BUT, we do this shrinking *before* the status bar has actually appeared. So our shortened viewport is still snapped to the top of the page, which means there's some extra unpainted space at the bottom.

Two simple hacks that work around this partially (not entirely):
 (1) Give the <body> an explicit tall height, so that it'll be taller than its viewport. This seems to work, but it means there'll be a vertical scrollbar, so unless we can make this tall-height a temporary thing, this isn't really a good solution.

 (2) Give the <body> a background-color that's close to the dark-gray background image.  This will be painted in visible areas beyond its background-image. (The "dark noise" background image fills up the shortened viewport, but the extra space gets painted with the background-color.)  According to one color-analysis site I used*, the average color of the "noise" image is rgb(33,33,33), so that's what we should use.

Option (2) is what I'd recommend at this point. The video still noticably snaps downwards after the status bar appears (since it pushes the viewport downward), but the white area at the bottom will end up basically the same color as the dark noise image, so that "flash" won't be noticeable.

* I uploaded imagedoc-darknoise.png to http://mkweb.bcgsc.ca/color_summarizer/?analyze, which says the min (darkest) color is rgb(31,31,31), and the max is rgb(35,35,35), and the average & median are both rgb(33,33,33).

Comment 40

[Daniel Holbert [:dholbert]]

Filed bug 1167999, with a patch, on doing option (2). I think this bug will be fixed by that bug, but I'm filing it separately in case there's anything else we need here.

(In reply to Daniel Holbert [:dholbert] from comment #38)
> That makes me suspect there's a layers/compositing race condition of some
> sort going on here, though I don't know the layers code well enough to be
> sure.

(Following up on this: I'm now pretty sure the real race condition here is between (a) the status bar appearing (and occupying space at the top), vs. (b) the video document repainting at its new, slightly shorter viewport-size. We're hitting this bug because (b) is beating (a).)

Comment 41

[Daniel Holbert [:dholbert]]

(Comment 40 is basically agreeing with comment 14, I guess.)

Comment 42

[Daniel Holbert [:dholbert]]

Another data point: I can reproduce this on Firefox for Android, too. And there, we actually stay stuck in a scenario with the white bar displayed, until the user actually tries to scroll (and then the url bar shows up & pushes the viewport down so that the white bar goes away).

Anyway, I've just landed a patch (for bug 1167999) which fixes this for me Firefox OS, on trunk, and I requested approval-mozilla-b2g37 so that we can get the fix into Firefox OS 2.2.

Comment 43

[Dylan Oliver [:doliver]]

ni? to Diego to call attention to the patch in bug 1167999. It hasn't landed on our branch yet but is there if you want to pick it up for your next build and re-test this issue.

Comment 44

[Josh Cheng [:josh]]

Hi Dylan, Should we dup this one to bug 1167999?
Thanks

Comment 45

[Ben Francis [:benfrancis]]

The platform patch in bug 1167999 provides a workaround and has now been uplifted to 2.2, let's dupe and get QA to verify that this fixes the bug.

Bug 1168022 is tracking the underlying problem to fix in a future release.

Comment 46

[Elie]

According Bug 1167999 comment 20, this bug have been verified as fail, and please leave it as "verifyme" to wait for a fix.

Comment 47

[Daniel Holbert [:dholbert]]

Un-duping, since it seems bug 1167999 didn't entirely fix this, per bug 1167999 comment 20.

I've now confirmed that I *can* still reproduce this in latest Firefox OS 2.2 nightly (build ID 20150603002503) on Flame, and latest Firefox OS 3.0 dogfood nightly on Xperia Z3C, *if* I use the RTSP url in comment 0.

I can't reproduce with an ogg video from tinyvid.tv, after repeated attempts. I originally could, per comment 34. So I'm pretty sure bug 1167999 fixed that ogg-video use case, at least.

I suspect the difference has something to do with the fact that the RTSP url has a continuously-animating progress bar, which may indicate that the page never finishes loading (and so never snaps to a final size, or something?).  If I tap the "x" in the URL bar to stop it loading, then I can't reproduce anymore.

So I think this may still be broken while a video is still streaming data (or thinks it is, according to that progress bar), but it's fixed once a video has finished loading all of its data.

Comment 48

[Gregor Wagner [:gwagner]]

I believe we are not blocking 2.2 on the rtsp use-case any more. Lets ask for uplift approval if we get a patch in time.

Comment 49

[Daniel Holbert [:dholbert]]

Ok. I think the right way to fix this is to dig in on bug 1168022, then, probably.

Comment 50

[Pi Wei Cheng [:piwei] (inactive)]

See comment 46 for failed verification confirmation. This bug is not fixed so removing verifyme keywords.

Comment 51

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Dan, this also causes an issue with the software home button; see bug 1171754.

Comment 52

[Daniel Holbert [:dholbert]]

(Let's keep that bug separate; the main issue there -- video controls overlapping software home button in fullscreen mode -- seems pretty distinct from the issue here)

Comment 53

[Diego Wilson [:diego]]

Removing ni? since it seems this issue has not been resolved

Comment 54

[Jerry Shih[:jerry] (UTC+8) (inactive)]

Hi Xidorn,

I remember that you have some resolved bugs related to the window resize problem. The video seems the resize event is not aligned, so we might see the white background during resizing.

Comment 55

[Xidorn Quan [:xidorn] UTC+11]

(In reply to Jerry Shih[:jerry] (UTC+8) from comment #54)
> I remember that you have some resolved bugs related to the window resize
> problem.

No, I don't, but I currently have some pending patches in bug 1177155 which may relate to this issue.

> The video seems the resize event is not aligned, so we might see
> the white background during resizing.

It looks wierd to me. How can the child viewport get resized before we reflow the outer UI? It seems to me the resize of an inner document is usually driven by a post-reflow callback of its parent document.

But anyway, it looks that we do an extra reflow which should be avoided.

Let's see whether fixing bug 1177155 could help.

Comment 56

[Xidorn Quan [:xidorn] UTC+11]

Bug 1177155 has been fixed several days ago. Could anyone test it again and see whether it still has this issue?

Comment 57

[Michael Henretty [:mikehenrty][:mhenretty]]

(In reply to Xidorn Quan [:xidorn] (UTC+10) from comment #56)
> Bug 1177155 has been fixed several days ago. Could anyone test it again and
> see whether it still has this issue?

Comment 58

[Sxean Lee-David [:sxean]]

No Repro per builds: 

Environmental Variables:
Device: Flame 2.5
BuildID: 20151005030238
Gaia: b994cedaa7ef9bfadcbe841601d9dc8d2e5379f9
Gecko: b56aeea0c4701677ffda6417183caa60d2a6a4a7
Gonk: c4779d6da0f85894b1f78f0351b43f2949e8decd
Version: 44.0a1 (2.5) 
Firmware Version: v18D
User Agent: Mozilla/5.0 (Mobile; rv:44.0) Gecko/44.0 Firefox/44.0

RESULT: 
When open a video of any format from Gallery, the video automatically defaults to FullScreen Mode in either Portrait or Landscape View. There is no option to minimize to non-fullscreen and therefore the white bar flash is non-existent.

Environmental Variables:
Device: Aries 2.5
BuildID: 20151005111147
Gaia: b994cedaa7ef9bfadcbe841601d9dc8d2e5379f9
Gecko: 45f01961ecd0ad9a45067f3e08bfb92539042eeb
Gonk: 2916e2368074b5383c80bf5a0fba3fc83ba310bd
Version: 44.0a1 (2.5) 
Firmware Version: D5803_23.1.A.1.28_NCB.ftf
User Agent: Mozilla/5.0 (Mobile; rv:44.0) Gecko/44.0 Firefox/44.0

RESULT: 
When open a video of any format from Gallery, the video automatically defaults to FullScreen Mode in either Portrait or Landscape View. There is no option to minimize to non-fullscreen and therefore the white bar flash is non-existent.

Comment 59

[Jayme Mercado [:JMercado] (Inactive 3/4/2016)]

We were able to reproduce this issue on earlier builds but no builds from today.  Resolving this as WFM but please reopen if it occurs again.