563377 - Fullscreen video stops responding to keyboard/mouse input, refuses to close itself when switched away

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Rob Arnold [:robarnold]]

STR:
1) Go to http://www.youtube.com/user/HDstarcraft#p/u/4/MBHhGx4jkdg
2) Select 1080p quality.
3) Select fullscreen
4) Wait 30-90 seconds
5) Try to move the mouse or press escape or alt-tab successfully to another window

The problem is intermittent and sometimes you can successfully alt-tab away. Sometimes alt-tabbing will cause the browser to hang. Killing the plugin-container process will fix the hang but may cause bug 545892 to manifest.

Comment 1

[Rob Arnold [:robarnold]]

I reproduced this with Gecko/20100503 Namoroka/3.6.5pre on
http://www.youtube.com/user/HDstarcraft#p/u/97/ccwWFxLOfV8

The window did gracefully disappear when I alt-tabbed but I still had the hang and the loss of glass.

Comment 2

[Marcia Knous [:marcia]]

I just filed Bug 563384 for similar behavior I have been seeing while testing 3.6.4 builds, although my issues were more with videos on fox.com. But I am able to easily reproduce this on Win 7 using 3.6.4 Build 1 and the site in Comment 1.

Comment 3

[Marcia Knous [:marcia]]

Does not happen for me using Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.5pre) Gecko/20100503 Namoroka/3.6.5pre, so this may be a Win only issue.

Comment 4

[Rob Arnold [:robarnold]]

I have not been able to reproduce this with Hulu which I am told is windowless.

Comment 5

[Jim Mathies [:jimm]]

Attachment: stack

Comment 6

[Jim Mathies [:jimm]]

Comment on attachment 443686 [details]
stack

wrong file

Comment 7

[Jim Mathies [:jimm]]

Attachment: stack

Comment 8

[Mike Beltzner [:beltzner, not reading bugmail]]

If we can reproduce this with Firefox 3.6.4 build 3, then we may need to block on the issue.

Comment 9

[Mike Beltzner [:beltzner, not reading bugmail]]

I can't reproduce the problem on latest trunk nightly, fwiw. Will try again with Firefox 3.6.4 build2 in a bit.

Comment 10

[Benjamin Smedberg]

Jim can with 1.9.2 nightlies in a VM, I think this may be non-aero only (the DWM is not active).

Comment 11

[Mike Beltzner [:beltzner, not reading bugmail]]

Yup, I'm getting the hang. The summary seems incorrect. Mouse input works and such, but switching away with alt-tab causes the browser to all-the-way lock up.

Comment 12

[Rob Arnold [:robarnold]]

(In reply to comment #10)
> Jim can with 1.9.2 nightlies in a VM, I think this may be non-aero only (the
> DWM is not active).

Is Namoroka/3.6.5pre not a 1.9.2 nightly? In comment #1 I had the DWM enabled and on trunk it most certainly occurs with the DWM enabled.

Comment 13

[Mike Beltzner [:beltzner, not reading bugmail]]

FWIW, when hung, two interesting things:

 - plugin-process grows continually in memory size (a leak in flash?)
 - killing plugin-process gets me the oopsie as expected

I suspect that this is actually a Flash problem, which is occasionally triggered. Rob/anyone: can we confirm that we *don't* see this behaviour in Firefox 3.6.3?

> (In reply to comment #10)
> Is Namoroka/3.6.5pre not a 1.9.2 nightly? In comment #1 I had the DWM enabled
> and on trunk it most certainly occurs with the DWM enabled.

It is, but we've taken new things on mozilla-1.9.2 already (for Firefox 3.6.5) so I wanted to test with the build that we're planning on shipping to users.(In reply to comment #12)

Comment 14

[Benjamin Smedberg]

->jmathies since he can reproduce and I can't

Comment 15

[Jim Mathies [:jimm]]

(In reply to comment #0)
> STR:
> 1) Go to http://www.youtube.com/user/HDstarcraft#p/u/4/MBHhGx4jkdg
> 2) Select 1080p quality.
> 3) Select fullscreen
> 4) Wait 30-90 seconds
> 5) Try to move the mouse or press escape or alt-tab successfully to another
> window


Not sure if I'm reproducing the same issue rob is seeing, as my STR are slightly different - 

1) Go to http://www.youtube.com/user/HDstarcraft#p/u/4/MBHhGx4jkdg
2) Select 1080p quality.
3) Select fullscreen
4) alt-tab out of the video
if you don't get a hang, goto 3

After maybe 5-10 tries I get a hang in the browser.

Comment 16

[Jim Mathies [:jimm]]

Also, I can only reproduce on an image. After about 50 tries, my desktop w/aero doesn't seem to have the problem.

Comment 17

[Jim Mathies [:jimm]]

Best guess based on the stacks - 

child: peekmessage: wm_activate -> parent 
parent: wm_activate -> reply message
parent: defwndprc(wm_activate) -> something to child
child: (still) peekmessage: wm_killfocus -> flash window -> evaluate -> waitfornotify

hang in parent, waiting on child to process win event
hang in child in waitfornotify waiting for an ipc response

Our waitfornotify deferred message handling in child does nothing to prevent this since it only hooks mozilla windows.

Comment 18

[Jim Mathies [:jimm]]

Attachment: better stacks

Comment 19

[juan becerra [:juanb]]

Marcia, what version of Flash were you using? I'm trying to recreate the problem in 3.6.3.

Comment 20

[Jim Mathies [:jimm]]

I just got an idea on this, pushing something to try for testing. will post back.

Comment 21

[Marcia Knous [:marcia]]

r45. I have installed 3.6.3 on the same machine I was seeing the issue during 3.6.4 testing so I will see if I can reproduce.

(In reply to comment #19)
> Marcia, what version of Flash were you using? I'm trying to recreate the
> problem in 3.6.3.

Comment 22

[Marcia Knous [:marcia]]

Using Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 with vr45 of flash and following the STR in Comment 15, I don't see any issues moving in and out of full screen mode. Using 3.6.4 Build 3 with r45 flash, I do see the issues with non responsiveness when trying to go in and out of FSM. One thing that does pop up is a slow script dialog - I have it on the other machine and can attach a screenshot. But there is a definite difference in how the two builds behave in terms of going in and out of FSM.

Comment 23

[Jim Mathies [:jimm]]

(In reply to comment #20)
> I just got an idea on this, pushing something to try for testing. will post
> back.

This didn't work out. Back to the drawing board..

Comment 24

[Jim Mathies [:jimm]]

Working on another test fix. This tends to get triggered when there's a lot of bandwidth being pumped to the screen. The starcraft video tends not to hang until you get to the fullscreen video play. Here's a hi-def surfing BBC video that can trigger it as soon as the 1080p video starts playing - 

http://www.youtube.com/watch?v=7BOhDaJH0m4

Comment 25

[Jim Mathies [:jimm]]

(In reply to comment #24)
> Working on another test fix. This tends to get triggered when there's a lot of
> bandwidth being pumped to the screen. The starcraft video tends not to hang
> until you get to the fullscreen video play. Here's a hi-def surfing BBC video
> that can trigger it as soon as the 1080p video starts playing - 
> 
> http://www.youtube.com/watch?v=7BOhDaJH0m4

No luck there either. One interesting note is that the wm_activate event on the parent side is not a sent message.

Comment 26

[Jim Mathies [:jimm]]

So this appears to be a hang in the child related to messages sent by parent. Basically:

1) user alt-tabs full screen video
2) parent's window gets an activate message and hands that to def wnd prco
3) windows sends wm_killfocus -> child
4) child calls peekmessage -> wm_killfocus -> flash window procedure
5) flash calls evaluate:
 try { __flash__toXML(checkCurrentVideo("7BOhDaJH0m4")) ; } catch (e) { "<undefined/>"; }

hang

based on vmware replay debugging, wm_killfocus isn't going to a window we have a subclass on, so most likely the remnants of the full screen window. To test this theory, I sent a generic "replymessage in eval" patch to try to see. The real fix would involve hooking whatever window this is being sent to and calling replymessage with the correct response.

Comment 27

[Jim Mathies [:jimm]]

..another possible fix: add the target window to deferred processing.

Comment 28

[Mike Beltzner [:beltzner, not reading bugmail]]

I think this needs to block. Fullscreen YouTube is a use-case we need to support. Looks like it's only in cases where the CPU is getting hammered (1080p on our machines, maybe less on others)

Comment 29

[Benjamin Smedberg]

Can we just hook *all* windows for deferred processing, instead of replacing targeted windowprocs?

Comment 30

[Jim Mathies [:jimm]]

(In reply to comment #29)
> Can we just hook *all* windows for deferred processing, instead of replacing
> targeted windowprocs?

Deferred processing is pretty invasive. We could try it for the child, but it's never been tested.

Comment 31

[Jim Mathies [:jimm]]

Huh, we already have full screen windows in our deferred processing.

http://mxr.mozilla.org/mozilla-central/source/ipc/glue/WindowsMessageLoop.cpp#379

added mystery...

Comment 32

[Jim Mathies [:jimm]]

I have a test fix that appears to address this. I need to translate it into a respectable patch though. If anyone would care to confirm, here's the try build - 

http://build.mozilla.org/tryserver-builds/[e-mail]-try-b708208d5809

Comment 33

[Marcia Knous [:marcia]]

Using the build in Comment 32 on Win 7, I do not see the issues noted in Comment 15. The one thing I do notice is that the video often pauses and has to be restarted when in FSM. But I am able to use keyboard controls, etc with no problem.

Comment 34

[Mike Beltzner [:beltzner, not reading bugmail]]

Please see also bug 563014 in which Blizzard has a debug stack for this sort of thing happening on hulu.com

Comment 35

[Christopher Blizzard (:blizzard)]

I don't know if this is the same bug or not.  But I can very easily reproduce bug 563014 very easily.  Just play something on hulu.com, go full screen and return a few times, eventually the browser hangs.

Comment 36

[Jim Mathies [:jimm]]

Attachment: patch v.1

Comment 37

[Jim Mathies [:jimm]]

We should re-verify Bug 541362 is fixed with this patch.

http://webmessenger.yahoo.com/

Comment 38

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Comment on attachment 444412 [details] [diff] [review]
patch v.1

>+            case WM_WINDOWPOSCHANGED:
>+            ReplyMessage(0);
>+            break;

Nit: let's indent the ReplyMessage/break and add newlines after so that we can see the differences between the classes of messages more easily. Below too.

>+            // Sync message that don't cause hangs or can't be handled:

Nit: Shouldn't that be s/can't/can/ ?

>+#ifdef DEBUG

On IRC you said you could clean this up a bit, let's do that.

Comment 39

[Jim Mathies [:jimm]]

try build:

http://build.mozilla.org/tryserver-builds/[e-mail]-try-c84b7305db61

Comment 41

[Christopher Blizzard (:blizzard)]

Note that try server build fixed my issues mentioned in bug 563014, at least with Minefield vs. patched-Minefield.

Comment 42

[LegNeato]

Comment on attachment 444412 [details] [diff] [review]
patch v.1

a=LegNeato for 1.9.2.4

Comment 43

[Jim Mathies [:jimm]]

1.9.2 try build:

http://build.mozilla.org/tryserver-builds/[e-mail]-try-f0a747250d7b

Comment 44

[Christopher Blizzard (:blizzard)]

I can verify that the 1.9.2 builds don't hang firefox on hulu.com with that tryserver build.

Comment 45

[Jim Mathies [:jimm]]

http://hg.mozilla.org/mozilla-central/rev/f884c59a5fbf

Comment 46

[Jim Mathies [:jimm]]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/22e71c6921b0
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/590aa1d475e0

Comment 47

[Al Billings [:abillings - ex-MoCo]]

(In reply to comment #15)
> (In reply to comment #0)
> > STR:
> > 1) Go to http://www.youtube.com/user/HDstarcraft#p/u/4/MBHhGx4jkdg
> > 2) Select 1080p quality.
> > 3) Select fullscreen
> > 4) Wait 30-90 seconds
> > 5) Try to move the mouse or press escape or alt-tab successfully to another
> > window
> 
> 
> Not sure if I'm reproducing the same issue rob is seeing, as my STR are
> slightly different - 
> 
> 1) Go to http://www.youtube.com/user/HDstarcraft#p/u/4/MBHhGx4jkdg
> 2) Select 1080p quality.
> 3) Select fullscreen
> 4) alt-tab out of the video
> if you don't get a hang, goto 3
> 
> After maybe 5-10 tries I get a hang in the browser.

Using Build 4 that was just made on XP in my virtual machine, if I go to http://www.youtube.com/user/HDstarcraft#p/u/4/MBHhGx4jkdg and play a video in 1080 and then start hitting escape or alt-tab, I get an OOPsie and Flash crashes.

I don't know if I'd call that a fix though it is better than a browser hang.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 (.NET CLR 3.5.30729)

Comment 48

[Al Billings [:abillings - ex-MoCo]]

If I just hit alt-tab over and over while doing fullscreen 1080 video on that channel, I crash Flash.

If I watch the video for 60 seconds on fullscreen and hit Esc repeatedly, nothing happens at all.

Comment 49

[Mike Beltzner [:beltzner, not reading bugmail]]

Jimm - expected?

Comment 50

[Al Billings [:abillings - ex-MoCo]]

(In reply to comment #48)
> If I watch the video for 60 seconds on fullscreen and hit Esc repeatedly,
> nothing happens at all.

I've found that flash often crashes when I hit Esc in the above scenario as well if I wait long enough.

Comment 51

[Jim Mathies [:jimm]]

Maybe we're hitting timeouts due to slow virtual machines and backed up networking?

I'm testing on a Win7 image and I'm not able to reproduce. Also check on my desktop. I'll take a look at XP here in a bit. 

Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.5pre) Gecko/20100513 Namoroka/3.6.5pre

Does anyone have any crash signatures?

Comment 52

[Jim Mathies [:jimm]]

Ok, on XP, I'm getting a strange alt-tab bug. When you alt-tab, the video window loses focus, but doesn't hide. Then you're sort of stuck until you click on the full screen window to bring back focus, at which point exit out with esc. Not reproducible on Win7 AFAICT.

We might try to get a regression range on this, could be this patch, but it might be something else as well.

Comment 53

[Al Billings [:abillings - ex-MoCo]]

I'm getting definite crashes. Maybe it is my slower network at home, maybe because it is XP.

This pair is from hitting alt-tab repeatedly as I play fullscreen 1080 video at the youtube channel above after letting it play for about 60 seconds:

http://crash-stats.mozilla.com/report/index/bp-a8085408-c2dd-41d9-a10a-481412100513

http://crash-stats.mozilla.com/report/index/bp-ed716947-369a-4f25-a890-2031d2100513

These go together. I notice that after these, I couldn't put focus into or edit the location/url bar either.

I couldn't get Esc to crash again though I do note that while I'm playing 1080 video fullscreen on youtube in my XP VM, the flash controls are completely unresponsive and I actually can't reliably exit fullscreen.

Comment 54

[Jim Mathies [:jimm]]

(In reply to comment #53)
> I'm getting definite crashes. Maybe it is my slower network at home, maybe
> because it is XP.
> 
> This pair is from hitting alt-tab repeatedly as I play fullscreen 1080 video at
> the youtube channel above after letting it play for about 60 seconds:
> 
> http://crash-stats.mozilla.com/report/index/bp-a8085408-c2dd-41d9-a10a-481412100513
> 
> http://crash-stats.mozilla.com/report/index/bp-ed716947-369a-4f25-a890-2031d2100513
> 
> These go together. I notice that after these, I couldn't put focus into or edit
> the location/url bar either.
> 
> I couldn't get Esc to crash again though I do note that while I'm playing 1080
> video fullscreen on youtube in my XP VM, the flash controls are completely
> unresponsive and I actually can't reliably exit fullscreen.

These stacks are a bit strange, we're sending windowing events via
nsNPAPIPluginInstance::HandleEvent to a windowed plugin which result in HandleEvent RPC calls. I've never seen that on youtube videos, but maybe it's expected.

Comment 55

[Jim Mathies [:jimm]]

Regardless, the over arching problem seems to be related to focus, and since I can reproduce at least part of of the problem on an image, I'll see about that first.

Comment 56

[Jim Mathies [:jimm]]

Hmm, so this kind of sucks. Parent deactivates the full screen flash window, sending wm_killfocus to it. When flash receives that, it makes an RPC call. (That was the hang.) The patch that landed (which for some reason fixes this on win7/vista, but not on XP) traps wm_killfocus, and replies to it before delivering the message to flash. Somehow, this interferes with the handling of the overall change of focus, causing flash to fail to close the window.

Comment 57

[mike]

this may be the wrong thread here but i think there has always been a problem with full screen rendering on older machines with hardware acceleration swithed on flash control panel, fact is adobe flash is still very unstable on some websites and i dont blame Firefox for wanting to control it but im not sure if plugin container is helping, still crashing on sites like facebook...

Comment 58

[Al Billings [:abillings - ex-MoCo]]

Mike, this is the wrong bug and thread for that. I'm not on an older machine, there is no hardware acceleration, and this is a specific issue in the new Out of Process Plugin code that we're testing since I'm causing an OOPP crash in the most recent iteration, as Jim notes.

Comment 59

[Jim Mathies [:jimm]]

I should have try builds in a bit with a potential fix.

Comment 60

[Jim Mathies [:jimm]]

try builds - 

http://build.mozilla.org/tryserver-builds/[e-mail]-try-951a8584cd07

Comment 61

[Al Billings [:abillings - ex-MoCo]]

All right, I used the try build on the same XP VM that I'd be testing on earlier. I'm not getting an OOPsie on 1080 full screen when I alt-tab repeatedly between the Firefox and Flash processes. I notice that when I alt-tab to Flash, it exits fullscreen mode every time. Is this an intentional side effect? It didn't do this before.

Comment 62

[Jim Mathies [:jimm]]

(In reply to comment #61)
> All right, I used the try build on the same XP VM that I'd be testing on
> earlier. I'm not getting an OOPsie on 1080 full screen when I alt-tab
> repeatedly between the Firefox and Flash processes.

> I notice that when I
> alt-tab to Flash, it exits fullscreen mode every time. Is this an intentional
> side effect? It didn't do this before.

Not sure I understand the problem. When you alt-tab in full screen video mode, the full screen video window should always close, just like hitting escape. Are you seeing different behaviors between 3.6.3 and this try build Al?

Comment 63

[Jim Mathies [:jimm]]

Attachment: flash full screen patch v.1

This is kind of a one-off for Flash's full screen window. What's happening:

1) user alt-tabs
2) parent sends wm_killfocus -> fs window
3) PluginModuleChild calls ReplyMessage
4) parent thread executes, removing focus
5) child hands killfocus to flash, flash apparently checks the foreground window and stops processing.

result: Fullscreen window hangs around.

This only seems to happen on single core systems.

Comment 64

[Al Billings [:abillings - ex-MoCo]]

(In reply to comment #62)
 
> Not sure I understand the problem. When you alt-tab in full screen video mode,
> the full screen video window should always close, just like hitting escape. Are
> you seeing different behaviors between 3.6.3 and this try build Al?

I don't watch fullscreen video normally so I didn't realize it was supposed to exit. That's fine.

Comment 65

[Jim Mathies [:jimm]]

Attachment: flash full screen patch v.1

This is a bit of a hack, but we need the default processing of the killfocus in other cases. This seemed like the simplest approach for 3.6.4.

Comment 66

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Comment on attachment 445787 [details] [diff] [review]
flash full screen patch v.1

r=me, though we now have three places with "ShockwaveFlashFullScreen" hardcoded. Any chance we could consolidate that?

Comment 67

[Jim Mathies [:jimm]]

http://hg.mozilla.org/mozilla-central/rev/6861c31f2ec6

also consolidated the wide character class names in dom.

Comment 68

[LegNeato]

Comment on attachment 445787 [details] [diff] [review]
flash full screen patch v.1

a=LegNeato for 1.9.2.4. Please land on both mozilla-1.9.2 default and
GECKO1924_20100413_RELBRANCH

Comment 69

[LegNeato]

Btw, typo: 
"// Used with fix for flash fullscreen window loosing focus."
should be
"// Used with fix for flash fullscreen window losing focus."

Comment 70

[IU]

(In reply to comment #63)
> Created an attachment (id=445742) [details]
> flash full screen patch v.1
> 
> This is kind of a one-off for Flash's full screen window. What's happening:
> 
> 1) user alt-tabs
> 2) parent sends wm_killfocus -> fs window
> 3) PluginModuleChild calls ReplyMessage
> 4) parent thread executes, removing focus
> 5) child hands killfocus to flash, flash apparently checks the foreground
> window and stops processing.
> 
> result: Fullscreen window hangs around.
> 
> This only seems to happen on single core systems.

This seems to be the same kind of thing I experience on MSNBC videos when the fly-out menus disappear but the video remains unresponsive to mouse input until the video ends.  However, this does not happen outside fullscreen mode.

Your patch, if I understand correctly, is for fullscreen mode only.  Is it possible you can make your patch work for all modes?

Comment 71

[Jim Mathies [:jimm]]

(In reply to comment #70)

> This seems to be the same kind of thing I experience on MSNBC videos when the
> fly-out menus disappear but the video remains unresponsive to mouse input until
> the video ends.  However, this does not happen outside fullscreen mode.

Is there an open bug on this related problem IU?

Comment 72

[IU]

I mentioned it in my report: bug 561818 comment 30; but no one has yet found comparable hardware with which to reproduce any of it.

Comment 73

[Jim Mathies [:jimm]]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/32d21e6d0b33
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/79600059813c