1312369 - dom.ipc.plugins.asyncdrawing.enabled changes mouse wheel behavior

Status: RESOLVED WONTFIX

(Core Graveyard :: Plug-ins, defect)

Description

[rewb0rn]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0
Build ID: 20161019084923

Steps to reproduce:

With dom.ipc.plugins.asyncdrawing.enabled set to true, the way the mouse wheel events are communicated to Flash changes. The majority of Flash games uses a so called MouseWheel trap that will disable the mouse wheel of the container while the mouse is over the Flash content, in order to prevent the browser page from scrolling while the user is scrolling in Flash, for example a game list.

Historically, Firefox would communicate the scrolling to the plugin, even if the mouse event was cancelled. With dom.ipc.plugins.asyncdrawing.enabled set to true, the mouse event will not be communicated to the plugin if it is cancelled. This means with for current games, the mouse wheel will no longer work. To make the mouse wheel work again, the mouse event has to be injected into Flash. This is possible, albeit not ideal because that means for older versions of Firefox, the mouse event will be communicated twice, once from the old way how Firefox handled the mouse event and once from the manual inject. As I am aware there is no way to detect in Javascript if dom.ipc.plugins.asyncdrawing.enabled is set to true, so the developer has to either accept that the mouse wheel will not work in newer versions or accept that the mouse wheel will be handled twice in older versions.


This is an idea of the code that Flash games usually use. Disclaimer: This code was not originally written by me and I do not claim to be a capable JS developer. I added indents and line breaks to improve readability:

var browserScrollAllow=true;
var isMac=false;

function registerEventListeners(inputIsMac)
{
	if(window.addEventListener)
	{
		window.addEventListener('mousewheel',wheelHandler,true);
		window.addEventListener('DOMMouseScroll',wheelHandler,true);
		window.addEventListener('scroll',wheelHandler,true);
		isMac=inputIsMac;
	}
	else
	{
		document.onmousewheel=wheelHandler;
	}
}

function wheelHandler(event)
{
	var delta=deltaFilter(event);
	if(delta==undefined)
	{
		delta=event.detail
	}
	if(!event)
	{
		event=window.event
	}
	if(!browserScrollAllow)
	{
		if(window.chrome||isMac)
		{
			document.getElementById('flash-object-id').scrollHappened(delta)
		}
		if(event.preventDefault)
		{
			event.preventDefault()
		}
		else
		{
			event.returnValue=false
		}
	}
}

function allowBrowserScroll(allow)
{
	browserScrollAllow=allow
}

function deltaFilter(event)
{
	var delta=0;
	if(event.wheelDelta)
	{
		delta=event.wheelDelta/40;
		if(window.opera)
			delta=-delta
	}else if(event.detail)
	{
		delta=-event.detail
	}
	return delta
}

The new version that works with dom.ipc.plugins.asyncdrawing.enabled has to be altered like this:

var browserScrollAllow=true;
var isMac=false;
var isFirefox = false; 

function registerEventListeners(inputIsMac)
{
	if(window.addEventListener)
	{
		window.addEventListener('mousewheel',wheelHandler,true);
		window.addEventListener('DOMMouseScroll',wheelHandler,true);
		window.addEventListener('scroll',wheelHandler,true);
		isMac=inputIsMac;
		isFirefox = navigator.userAgent.toLowerCase().indexOf('firefox') > -1;
	}
	else
	{
		document.onmousewheel=wheelHandler;
	}
}

function wheelHandler(event)
{
	var delta=deltaFilter(event);
	if(delta==undefined)
	{
		delta=event.detail
	}
	if(!event)
	{
		event=window.event
	}
	if(!browserScrollAllow)
	{
		if(window.chrome||isMac||isFirefox)
		{
			document.getElementById('flash-object-id').scrollHappened(delta)
		}
		if(event.preventDefault)
		{
			event.preventDefault()
		}
		else
		{
			event.returnValue=false
		}
	}
}

function allowBrowserScroll(allow)
{
	browserScrollAllow=allow
}

function deltaFilter(event)
{
	var delta=0;
	if(event.wheelDelta)
	{
		delta=event.wheelDelta/40;
		if(window.opera)
			delta=-delta
	}else if(event.detail)
	{
		delta=-event.detail
	}
	return delta
}



Actual results:

Mouse wheel does not propagate to Flash content if the mouse event is cancelled when dom.ipc.plugins.asyncdrawing.enabled is set to true.


Expected results:

Mouse wheel should propagate to Flash content event if the mouse event is cancelled when dom.ipc.plugins.asyncdrawing.enabled is set to true, like in classic versions of Firefox

Comment 1

[Alice0775 White]

Please attach a minimum testcase to reproduce the issue.

Comment 2

[Benjamin Smedberg]

Or link to a testcase, since it will be multi-file. I'm interested in the following things:

what wmode are you using, and does the behavior change depending on the wmode?
Do you get the expected behavior in Chrome and Edge?

Comment 3

[rewb0rn]

Hi,

for test purposes you can try the following games:

- Updated version that injects the mouse event into Flash for Firefox: https://apps.facebook.com/rummy-palace

- Original version that used to work with previous versions of Firefox: https://apps.facebook.com/doppelkopf-palast/

Navigate to the mainscreen in the lobby where you can see the open list of games with other players joining and leaving. Use the mouse wheel to scroll the gamelist.


We use wmode 'direct' to allow GPU accelerated content. I tried transparent and opaque now and they both make the mouse wheel event change back to the old behavior (i.e. changing the wmode fixes the problem) but it prevents us from using Stage3D for GPU accerelated content that is required for our games.


In Chrome and Edge it works as before. Please note, that with the latest version and wmode direct, Firefox now behaves exactly like Chrome (so the mouse wheel event has to be injected into Flash, see above Javascript code). The problem is not so much the actual behavior, the problem is that the behavior changed with the latest version of Firefox and is not compatible to how previous versions of Firefox handled the mouse event and that we can not detect if dom.ipc.plugins.asyncdrawing.enabled is set to true for the user.

Comment 4

[Jim Mathies [:jimm]]

(In reply to rewb0rn from comment #0)
> User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101
> Firefox/49.0
> Build ID: 20161019084923
> 
> Steps to reproduce:
> 
> With dom.ipc.plugins.asyncdrawing.enabled set to true, the way the mouse
> wheel events are communicated to Flash changes. The majority of Flash games
> uses a so called MouseWheel trap that will disable the mouse wheel of the
> container while the mouse is over the Flash content, in order to prevent the
> browser page from scrolling while the user is scrolling in Flash, for
> example a game list.
> 
> Historically, Firefox would communicate the scrolling to the plugin, even if
> the mouse event was cancelled. With dom.ipc.plugins.asyncdrawing.enabled set
> to true, the mouse event will not be communicated to the plugin if it is
> cancelled.

I think this is slightly off, when working with a direct or windowed plugin, mouse input would go directly to the plugin window without being delivered to the browser first. Once async painting is turned, on, we convert the plugin from windowed to windowless, and then you have to fall back on windowless type page input handling. (Mac is also windowless, so in your sample it's not surprising that they share the same functionality of forwarding events.)

> This means with for current games, the mouse wheel will no longer
> work. To make the mouse wheel work again, the mouse event has to be injected
> into Flash. This is possible, albeit not ideal because that means for older
> versions of Firefox, the mouse event will be communicated twice, once from
> the old way how Firefox handled the mouse event and once from the manual
> inject. As I am aware there is no way to detect in Javascript if
> dom.ipc.plugins.asyncdrawing.enabled is set to true, so the developer has to
> either accept that the mouse wheel will not work in newer versions or accept
> that the mouse wheel will be handled twice in older versions.

I'm not aware of a way to detect async painting. You can detect browser version which gets closer to what you want but it's not perfect.

Comment 5

[Jim Mathies [:jimm]]

I think we're going to have to rely on site authors updating these scripts as I don't see a way for us to fix this internally. One issue though is people can change 'dom.ipc.plugins.asyncdrawing.enabled', so unless we expose that pref in some way, site authors won't know when to use a the right wheel handling script. What we could do is pick a release version and force async drawing on and ignore the pref. That way site authors can detect the version of firefox, load the right script, and know that asyncdrawing is in use.

Comment 6

[Jim Mathies [:jimm]]

Mike, curious what your thoughts are.

Comment 7

[Jim Mathies [:jimm]]

(In reply to Jim Mathies [:jimm] from comment #5)
> I think we're going to have to rely on site authors updating these scripts
> as I don't see a way for us to fix this internally. One issue though is
> people can change 'dom.ipc.plugins.asyncdrawing.enabled', so unless we
> expose that pref in some way, site authors won't know when to use a the
> right wheel handling script. What we could do is pick a release version and
> force async drawing on and ignore the pref. That way site authors can detect
> the version of firefox, load the right script, and know that asyncdrawing is
> in use.

One issue this doesn't address, users who do not have the latest flash. So we still would end up with some users who would see mouse wheel issues on some sites.

Comment 8

[rewb0rn]

In addition there would be a lot of Flash game sites where the older Flash games will never receive any updates by the authors, this might cause recommendations to try another browser for those games.

Comment 9

[Mike Taylor [:miketaylr]]

(In reply to Jim Mathies [:jimm] from comment #6)
> Mike, curious what your thoughts are.

So this part is interesting:

> Please note, that with the latest version and wmode direct, Firefox now behaves exactly like Chrome (so the mouse wheel event has to be injected into Flash, see above Javascript code).

Do we have a list of top flash games (that might be affected?) that we can investigate how they handle mousewheel events? If everybody uses UA sniffing to handle Chrome differently than Firefox we might be in trouble, even if we align with Chrome now.

(In reply to rewb0rn from comment #8)
> In addition there would be a lot of Flash game sites where the older Flash
> games will never receive any updates by the authors, this might cause
> recommendations to try another browser for those games.

Yeah, that's a real concern. Advocating for UA sniffing a la `if (firefox >= 49) {injectMouseEvents()}` is an OK workaround if you a) know to do that (and can parse a UA string in a sane way) and b) actually have access to the codebase and can make updates.

Comment 10

[Jim Mathies [:jimm]]

I haven't found a good way to survey the web for use of mouse wheel intercept js code like this.

Comment 11

[Jim Mathies [:jimm]]

I managed to find a fb game that uses code similar to this - 

function mouseWheelHandler(e) {
  var e = e || window.event;
  if (window.chrome || app_vars.browserName == BROWSER_IE || app_vars.clientOS == MAC_OS) {
    var delta = e.wheelDeltaY || e.deltaY || e.wheelDelta;
    gameSWF.callSWFMethod("mouseWheelAction", delta);
  }
  if (e.stopPropagation) { e.stopPropagation(); }
  if (e.preventDefault) { e.preventDefault(); }
}

https://apps.facebook.com/slotomania/

Testing with async drawing:

1) scroll wheel over flash doesn't scroll the page
2) scrollable elements within the game might not respond to wheel events afaict but I don't see this
3) scrolling over page content scrolls the page


Testing without async drawing:
1) scroll wheel over flash scrolls the page (annoying)
2) flash applet flickers when scrolling page content
3) scrolling over flash doesn't seem to work at all thanks to captures

IMO the async drawing behavior is preferable. This is just one test case of course.

Comment 12

[Alice0775 White]

FYI
attachment 8707024 [details] of Bug 1239037 seems to be same behavior comment#11.
Adding wmode=opaque (attachment 8707076 [details] of Bug 1239037) seems to fix the without async drawing case.

Comment 13

[rewb0rn]

Hi Jim,

I would argue that the developers of Slotomania created a bad user experience by not cancelling the mouse wheel event in the browser in the first place. In this case it might seem that async drawing fixes this, but really it just covers up that the developers did not care. In games such as Farmville 2 or our own, where we explicitly prevent the mouse wheel scroll of the page while in the Flash content, the new behavior breaks existing content.

Comment 15

[Jim Mathies [:jimm]]

The current behavior (farmville 2 makes a good test case): outside the game the page scrolls, inside the game mouse wheel events are delivered to flash. This is acceptable behavior for us.

Comment 16

[Balazs Attila-Mihaly]

I think this is still a problem. I see it on Ubuntu 17.04 with the latest Firefox (54.0) and Flash (26.0.0.131). See for example this site: http://armorgames.com/play/15760/gemcraft-chasing-shadows?via-search=1

The main map should be zoomable but instead the entire document scrolls.