Closed Bug 434914 Opened 16 years ago Closed 16 years ago

Keyboard broken in Flash ActionScript 3 games/movies

Categories

(Core Graveyard :: Plug-ins, defect, P1)

Product:
Core Graveyard
Component:
Plug-ins
Platform:
PowerPC
macOS
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: jim, Unassigned)

References

(
URL
)

Details

(Keywords: regression, testcase, Whiteboard: [RC2+])

Attachments

(1 file)

Key Event Tester SWF/HTML
331.79 KB, application/zip
Details 
User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008051202 Firefox/3.0
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008051202 Firefox/3.0

Keyboard events appear to be totally broken for ActionScript 3 Flash movies on the mac. Movies/games simply don't receive them.

Reproducible: Always

Steps to Reproduce:
1.Load an ActionScript 3 movie that requires keyboard events.
2.It doesn't work.
Version: unspecified → 3.0 Branch
Please provide a publicly accessible example, and specific, detailed steps to reproduce.

The example provided is publicly accessible. 

http://www.kongregate.com/games/arawkins/dolphin-olympics-2

The only steps to reproduce are to attempt to play that game on a Mac and see that it is unresponsive to the arrow keys, as are all other ActionScript 3 Flash games. Works on all other browsers, including earlier builds of Firefox 3.

We've tried both OS X 10.5 and 10.4.
Please provide another publicly accessible testcase.  The results from this one are too ambiguous (the dolphin gets moved both by other fish and the arrow keys).

As best I can tell from this testcase, the arrow keys start out working fine, then stop working.  But I need a more reliable testcase to be sure of this.

I made a quick SWF that tests this behavior. It appears that in FF3 RC1 the Flash Movie is never receiving key-up events. You can load this SWF in the standalone Flash player and press some keys, and you'll see you get key-up events. If you load the HTML into FF3 RC1, you never get key-up events. Loading the HTML into Safari or FF3 on Windows yields the correct result.
Thanks, Ben.  Your test fits the bill exactly.

I've confirmed your report (the testcase reports receiving key-down events but not key-up ones).  And I've found that the problem starts happening with the 2008-04-17-04-trunk nightly.  This probably implicates the patch for bug 357670, but I need to do more checking to be sure.
Status: UNCONFIRMED → NEW
Ever confirmed: true
I've done some testing and found that the patch for bug 357670 did
indeed trigger this bug ... but it didn't cause it.

And I've also found something very strange:

Though the new code (added for bug 357670) does indeed send both an
NS_KEY_DOWN (actually an NS_KEY_PRESS) event and an NS_KEY_UP event to
whatever plugin is focused, only an NS_KEY_UP event ever actually gets
send to the plugin (by code in nsPluginInstanceOwner::ProcessEvent()
in layout/generic/nsObjectFrame.cpp).

So there are currently two problems:

1) As of the patch for bug 357670, Firefox 3 only sends key-UP events
   to plugins.

2) The Flash plugin somehow interprets these NS_KEY_UP events as
   key-down events.

At bug 357670 (whose patch I wrote), I tested in a number of Flash
"movies" that take text input, and didn't have problems in any of
them.  So these two problems somehow manage to be invisible, most of
the time.

http://www.playercore.com/bugFiles/ime/imekrjp.swf
http://examples.adobe.com/flex3/consulting/styleexplorer/Flex3StyleExplorer.html
http://www.playercore.com/bugfiles/146162/AddReturnHTML.html

Not here, though.

Michelle, can you shed light on all this?
Component: Keyboard Navigation → Plug-ins
Product: Firefox → Core
QA Contact: keyboard.navigation → plugins
Version: 3.0 Branch → Trunk
Flags: wanted1.9.0.x?
Priority: -- → P1
Flags: wanted1.9.0.x? → wanted1.9.0.x+
Thanks Steven. The reason for differing results with different Flash content is probably that they listen for different key events. Programs that work with text typically only care about a key press, but games need to know exactly when keys go up and down. Sounds like you've got this figured out though.
In the bug marked as a duplicate of this bug you have steps to reproduce it. If you need I can make an Actionscript 3 example that listen to events just to show this bug more easily but there are already some actionscript 3 example out there that does this.
> If you need I can make an Actionscript 3 example that listen to
> events just to show this bug more easily but there are already some
> actionscript 3 example out there that does this.

Take a look at the "Key Event Tester" from comment #4.  Is this what
you're referring to?

yes. is it trying to get key_up events also? because if it is then the bug is clearly visible by it. Now fixing the bug might take a lot of time, but this bug should be a blocker for FF 3.0 final right?
> but this bug should be a blocker for FF 3.0 final right?

No, it's too late for that.  I'm currently trying to get this fixed in
the first point-release after 3.0 (which might be called Firefox
3.0.1).

I want to add that this is not just an as3 issue. It also affects as2 swf files and seems to have potential to break a lot of existing content (especially flash games).

I found this bug when I noticed that our keyboard controls were no longer working on the new YouTube video player and the YouTube embedded video players.
Flags: blocking1.9.0.1?
I'm really amazed that FF3 is going to be released and break a huge part of the web.  IMO this should be a blocker, even if it delays FF3.
I also have to add my concern for FF3 being shipped with this bug. Shipping would lessen the quality of Firefox which I don't think anyone wants.
Adding my vote to escalate this as a blocker. Too many sites rely on Flash to work. I cannot use this app until this is fixed.
Adding my vote to escalate this as a blocker.
Adding my vote to escalate this as a blocker. You wouldn't want to ship an app that is going to affect such a large majority of your userbase.
Please simply vote for this bug if you want it, using the "vote" feature. Adding comments stating that you're voting only adds noise to the bug and keeps developers away from it. Thanks.

See also #1.1: https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
Flags: blocking1.9?
Keywords: regression, testcase
Sam, you were right to request blocking1.9 for this bug (in response
to popular demand).  But I don't think it should block the Firefox 3
release.

This bug involves (at least) two different programs (the Firefox
browser and the Flash plugin), will touch code code that has given us
a lot of trouble in the recent past (key-handling code), and will
likely require changes to more than one module (Cocoa widgets code and
(probably) OS-X-specific code in layout/generic/nsObjectFrame.cpp).

So the fix will probably not be simple, and will require a lot of
testing before we can be confident that it doesn't cause other
problems.

This is _not_ the sort of change that should be made just before a
major release.

As best I can tell, this bug will cause serious problems for Flash
games that use keyboard input.  I don't know how many such games
exist, though I suspect there are probably quite a few.  But how many
users do these games have compared to the number of people that would
be inconvenienced if (say) the patch for bug 357670 were simply backed
out, or if changes we make here broke other aspects of plugin keyboard
input?

If things are as bad as they seem, some Flash game providers will need
to tell users that they should continue using Firefox 2.0.0.X until
this problem is fixed (in two or three months, when Firefox 3.0.1 is
released).  But I think it's better to allow this to happen than to
rush into a risky change just before the Firefox 3 release.

If we do target this fix to the Firefox 3.0.1 release, there will be
time for lots of testing.  And people who really want/need this fix
(like those who have already commented here) will be able to
participate in the testing :-)

Steven, you make a valid point to the extent that it would be painful to hold off. However, this does not effect only Flash games, but any form of application. One of which, as Geoff Stearns mentioned, seems to be the You Tube video player and I am sure many other applications as well.

If the 2-3 months is a guess at how long a fix will take I imagine this is not going to sit well. Just look at the comments/awareness here alone, and this is probably only developers speaking up...
(In reply to comment #20)
> Sam, you were right to request blocking1.9 for this bug (in response
> to popular demand).  But I don't think it should block the Firefox 3
> release.

I concur, but it's something drivers should decide.
(In reply to comment #20)
> This is _not_ the sort of change that should be made just before a
> major release.

This is not the sort of bug that should be included in a major release. The Flash Player plug-in is installed on almost 99% of all internet-enabled computers. A large quantity of the Flash applications out there use keyboard input in some form or another. I definitely think that you guys need to think this through real carefully before releasing the first major version of FF3 with a bug that is going to cripple a lot of applications.
The thing is, a bugzilla ticket is not the place to do this discussions I think. Then the problem is, most people only have contact with the mozilla community through bugzilla (I for one used to read mozillazine but that seems to have died a year or so ago) so I think bugzilla should grow a discuss this, just like wikipedia.

My vote would be to either fix this or just not release FF3.0 for mac for 2 reasons. First on the mac you are running against webkit so people are not desperate for a half working browser like on windows (and on linux if you use gnome(is epiphany using webkit this days?)). The second thing is that releasing FF with this huge bug would make people afraid of installing the .0 releases on their machines as they are afraid of microsoft and apple .0 releases (here linux is usually much better, at least ubuntu is), wich could slow the whole process of taking back the web.

For me and probably all flash devs this would mean a very very bad thing, as there is no work around for this.

For what it's worth, this bug was introduced in RC1 - and the flash intergration in Beta5 felt rock solid. In my opinion, releasing before a fix is introduced would be a tragic mistake that could seriously hinder mac adoption of the 3.0 platform.
Steven,

I don't agree with your assumption that this bug is less important than bug 357670 whose fix introduced this new bug.  This bug is a serious regression in functionality that worked in beta 5 whereas the fix for bug 357670 is an already existing issue which if left in would not change the adoption or perception of FF3 over prior versions.

I am a software developer and I understand that it sucks when a new bug comes to light right before a major release, especially a regression.  That doesn't change the fact that the responsible thing to do is back out the change that caused the bug or fix the regression with adequate testing, even if that means delayed release.

This does not affect just games.  Our multi-million dollar product installations will be affected by this bug and we'd hate to have to tell our customers to switch to Safari when FF3 is released (downgrading is often not an option in institutional environments).

Sam

Another aspect that hasn't been discussed here is accessibility concerns.
Obviously people who don't or can't use their mouse will be largely affected by
this bug when encountering Flash content.

I am a flash developer and consultant; I have also developed many games. If this bug sits then developers will not say "downgrade to FF 2", they will say "switch to IE"

With all due respect, I think there is some disconnect with the core Mozilla development team and the Flash community. Flash is installed on more computers than ANY other software (including any other operating system, plugin, or browser). The world record Mozilla is attempting to set has already been set by Adobe (on AVERAGE 8 million Flash plug in downloads per day... Mozilla is hoping for 5 million in the first 24 hours).

Nevertheless, in order to fulfill the promise of attempting the world record, I don't think Mozilla will be willing to withhold the release of FF3 to Mac users. The right thing to do would be to solve this bug before the product is released. Don't underestimate how many Flash applications are on the web... A major percentage use the keyboard...

And yes, as a previous poster mentioned, these posters are only, like myself, developers who most likely read some blog post about Mozilla not fixing this bug. This is a microscopic percentage of how many developers there are and how much dissent, aggravation, and "help forum" posts there will be concerning this bug until it's fixed.

Lastly, I, like most likely everyone else who have posted, love FF3. The new search bar is brilliant and so are all of the other additions; the browser runs faster than ever! However, this is indeed a deal breaker, and the overwhelming web community will agree that this bug must be addressed before release.
Folks, thanks for all of your comments, but please read comment 19 and the link Sam posted. Advocacy comments and anecdotes have a solid history of muddying bugs and discouraging developers from jumping in and fixing them. Already it's hard to find the last comment with information that is useful for Steven (the developer most likely to fix this bug) - it's comment 13 (I think) and we're now up to 29.

This bug has been nominated to block the release of Firefox 3 and our drivers can be counted upon to weigh the pros and cons mentioned here thoroughly. They know that this is a serious problem.
Considering the title of the bug: "Keyboard broken in Flash ActionScript 3 games/movies"

We use the Flash Platform to develop enterprise and business intelligence applications, in a SaaS eco-system. We are not building "games or movies". And we are not the only ones doing this. I do not want to add "noise" to the discussion, but I want to remember that nowadays Flash is used in very serious business scenarios and letting Firefox 3 be released with a bug like this can be a nightmare for us, our clients, Adobe and Mozilla itself. Please, take this in consideration.

Thank you
(In reply to comment #20)
> This bug involves (at least) two different programs (the Firefox
> browser and the Flash plugin), will touch code code that has given us
> a lot of trouble in the recent past (key-handling code), and will
> likely require changes to more than one module (Cocoa widgets code and
> (probably) OS-X-specific code in layout/generic/nsObjectFrame.cpp).
> 
> So the fix will probably not be simple, and will require a lot of
> testing before we can be confident that it doesn't cause other
> problems.
> 
> This is _not_ the sort of change that should be made just before a
> major release.

Didn't we do exactly that with the IME change?  We took a change to Mac widget handling and plugin interaction after the last beta, didn't test it thoroughly with the set of events that used to be able to propagate through our widget code.  Are you arguing that we should back out the obviously-risky IME change, since it doesn't meet those criteria?
> exist, though I suspect there are probably quite a few.  But how many
> users do these games have compared to the number of people that would
> be inconvenienced if (say) the patch for bug 357670 were simply backed
> out, or if changes we make here broke other aspects of plugin keyboard
> input?
> 
> If things are as bad as they seem, some Flash game providers will need
> to tell users that they should continue using Firefox 2.0.0.X until
> this problem is fixed (in two or three months, when Firefox 3.0.1 is
> released).  But I think it's better to allow this to happen than to
> rush into a risky change just before the Firefox 3 release.
> 
> If we do target this fix to the Firefox 3.0.1 release, there will be
> time for lots of testing.  And people who really want/need this fix
> (like those who have already commented here) will be able to
> participate in the testing :-)
> 

Breaking keyboard handling for flash on the Mac for the entire world is significantly worse than breaking IME handling for flash on the Mac.   Given the timeframe we should immediately back out the fix for bug 357670, re-spin RC2 and take our time with a better fix for bug 357670.   Steven can we get your help with that?
> 
> Breaking keyboard handling for flash on the Mac for the entire world is
> significantly worse than breaking IME handling for flash on the Mac.   Given
> the timeframe we should immediately back out the fix for bug 357670, re-spin
> RC2 and take our time with a better fix for bug 357670.   Steven can we get

That should say fixing bug 357760 in a dot release.


Asa: apologies for my dismissive response yesterday; I should have dug deeper into this.

To the advocates and people who commented: I want to be clear that this bug does not "break flash"; mouse interaction, typing, and a lot of flash games actually still work fine. It breaks a certain type of keyboard interaction (where the app is looking for the KEY_UP event) which is common in games, and used sometimes in other apps (like, apparently, the YouTube Beta player).

Release drivers have decided to back out bug 357670 for Firefox 3, we're respinning the RC2 builds at this time.
Flags: blocking1.9? → blocking1.9+
Blocks: 357670
Flags: wanted1.9.0.x+
Flags: blocking1.9.0.1?
Whiteboard: [RC2+]
New RC2 OSX build with the backout:

ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0rc2-candidates/build2/firefox-3.0.en-US.mac.dmg

Also, nightlies were clobbered to include this change, so if you're on nightlies, check for updates.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Adding in-testsuite? ... perhaps the attachment here can be used?
Flags: in-testsuite?
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9) Gecko/2008053008 Firefox/3.0

No more keyboard probs with latest build.
Marking as verified fixed after own testing and per comment #37 from hasham
Status: RESOLVED → VERIFIED
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: