1312482 - Inject scrolling event for Android Firefox browser

Status: RESOLVED WORKSFORME

(Firefox for Android Graveyard :: General, defect)

Description

[manos]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36

Steps to reproduce:

I am creating an app that will inject touch events on the device.

The app will be available to limited Android device manufacturers that will sign the package, so the app will obtain relevant permissions.

When testing against FireFox browser I am able to inject a single touch event (click on a link, press a button) but I am not able to scroll on a large page. I first inject a Motion Event with action ACTION_DOWN, then a number of events with action ACTION_MOVE and a final event with action ACTION_UP.

I am using FireFox build 49.0.2 on an Android AOSP 6.0 device (Nexus 10 Manta).


Actual results:

No scrolling happens on page. I have tried with a number of pages (bbc.co.uk, amazon, google, etc).


Expected results:

The page should scroll.

Comment 1

[Ioana Chiorean]

Any input here Sebastian?

Comment 2

[Sebastian Kaspari (:sebastian; :pocmo)]

@Randall: You have been working on scrolling bugs.. do you know what could be the reason for this?

@manos: How exactly are you injecting the scroll events? Are you calling methods on the windows/views of the app?

Comment 3

[manos]

Hi,

Many thanks for looking into this.

I create first an EVENT_DOWN motion event, then a series of EVENT_MOVE motion events and then a final EVENT_UP motion event.

sample snippet (Android code, omitting try-catch blogs and other java boiler plate):

long now = SystemClock.uptimeMillis();
MotionEvent evt = MotionEvent.obtain(now, now, action, posx, posy, 0);
evt.setSource(InputDevice.SOURCE_ANY);
mInstrumentation.sendPointerSync(evt);

where 
- action is MotionEvent.ACTION_DOWN, MotionEvent.ACTION_MOVE and MotionEvent.ACTION_UP
- posx and posy are coordinates

The above snippet succeeds when using firefox in Android and pressing on a widget like reload or tabs button, but fails when trying to scroll down a page. It works on every other major mobile browser I tested it with.

Not sure whats causing this, any idea on how to modify my code would be great.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

Try using InputDevice.SOURCE_TOUCHSCREEN instead of InputDevice.SOURCE_ANY.

Comment 5

[manos]

That was the first thing I tried, but it had zero.

Comment 6

[manos]

*zero impact

Comment 7

[Kartikaya Gupta (email:[e-mail])]

Oh you probably also need to set MotionEvent.TOOL_TYPE_FINGER as the tool type for the pointer. We have injection code that we use internally for testing [1] and it works fine. It uses InputDevice.SOURCE_TOUCHSCREEN and MotionEvent.TOOL_TYPE_FINGER for simulating touch input.

[1] http://searchfox.org/mozilla-central/rev/e3e8571c5378ac92663d4f583ccc4ad0a3019716/mobile/android/geckoview/src/main/java/org/mozilla/gecko/gfx/GeckoLayerClient.java#525

Comment 8

[manos]

I guess org.mozilla.gecko.gfx.LayerView is the View used to host scrolling content.
I will try creating MotionEvents similar to

final MotionEvent event = MotionEvent.obtain(
            /*downTime*/ mPointerState.downTime,
            /*eventTime*/ SystemClock.uptimeMillis(),
            /*action*/ action,
            /*pointerCount*/ mPointerState.getPointerCount(source),
            /*pointerProperties*/ mPointerState.getPointerProperties(source),
            /*pointerCoords*/ mPointerState.getPointerCoords(source),
            /*metaState*/ 0,
            /*buttonState*/ (isButtonDown ? MotionEvent.BUTTON_PRIMARY : 0),
            /*xPrecision*/ 0,
            /*yPrecision*/ 0,
            /*deviceId*/ 0,
            /*edgeFlags*/ 0,
            /*source*/ source,
            /*flags*/ 0);

and I will update the post accordingly.

One thing I noticed is that downTime != eventTime while in my code.

You reckon this detail could affect scrolling?

Comment 9

[Kartikaya Gupta (email:[e-mail])]

No I don't think we use that timestamp for anything important. It might affect the fling velocity or something, but it shouldn't affect whether or not the scroll actually happens.

Comment 10

[manos]

I have updated my code as similar as I could to the example pointed out.
Unfortunately I still cannot get any scrolling in Firefox (still works fine on any other browser).

My latest update looks like this:

MotionEvent.PointerCoords[] pointerCoords = new MotionEvent.PointerCoords[1];
MotionEvent.PointerCoords coords = new MotionEvent.PointerCoords();
coords.x = posx;
coords.y = posy;
coords.pressure = 1.0f;
pointerCoords[0] = coords;

MotionEvent.PointerProperties[] pointerProperties = new MotionEvent.PointerProperties[1];
MotionEvent.PointerProperties properties = new MotionEvent.PointerProperties();
properties.toolType = MotionEvent.TOOL_TYPE_FINGER;
properties.id = 0;
pointerProperties[0] = properties;

long now = SystemClock.uptimeMillis();
MotionEvent evt = MotionEvent.obtain(
            /*downTime*/ now,
            /*eventTime*/ now,
            /*action*/ action,
            /*pointerCount*/ 1,
            /*pointerProperties*/ pointerProperties,
            /*pointerCoords*/ pointerCoords,
            /*metaState*/ 0,
            /*buttonState*/ 0,
            /*xPrecision*/ 0,
            /*yPrecision*/ 0,
            /*deviceId*/ 0,
            /*edgeFlags*/ 0,
            /*source*/ InputDevice.SOURCE_TOUCHSCREEN,
            /*flags*/ 0);
        Log.e(TAG, "evt: " + evt);
        return evt;

This produces the following events, as captured by Logcat:

evt: MotionEvent { action=ACTION_DOWN, id[0]=0, x[0]=183.0, y[0]=627.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828131, downTime=828131, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=186.0, y[0]=633.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828631, downTime=828631, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=186.0, y[0]=651.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828649, downTime=828649, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=189.0, y[0]=681.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828674, downTime=828674, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=189.0, y[0]=711.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828708, downTime=828708, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=192.0, y[0]=744.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828799, downTime=828799, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=192.0, y[0]=780.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828835, downTime=828835, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=801.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828849, downTime=828849, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=825.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828869, downTime=828869, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=852.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828880, downTime=828880, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=879.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828948, downTime=828948, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=900.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828967, downTime=828967, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=921.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=828997, downTime=828997, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=927.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829025, downTime=829025, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=942.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829046, downTime=829046, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=954.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829070, downTime=829070, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=195.0, y[0]=966.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829080, downTime=829080, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=198.0, y[0]=981.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829111, downTime=829111, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=201.0, y[0]=999.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829182, downTime=829182, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=201.0, y[0]=1014.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829200, downTime=829200, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=207.0, y[0]=1035.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829217, downTime=829217, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=207.0, y[0]=1053.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829264, downTime=829264, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=210.0, y[0]=1068.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829280, downTime=829280, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=213.0, y[0]=1080.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829299, downTime=829299, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=213.0, y[0]=1095.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829366, downTime=829366, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=216.0, y[0]=1104.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829398, downTime=829398, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1116.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829415, downTime=829415, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1125.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829437, downTime=829437, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1128.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829465, downTime=829465, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1131.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829515, downTime=829515, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1134.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829532, downTime=829532, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1137.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829551, downTime=829551, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1146.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829566, downTime=829566, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1161.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829600, downTime=829600, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1170.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829615, downTime=829615, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1173.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829632, downTime=829632, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1176.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829649, downTime=829649, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1179.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829680, downTime=829680, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1182.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829733, downTime=829733, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1185.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829766, downTime=829766, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1194.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829783, downTime=829783, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1209.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829807, downTime=829807, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=222.0, y[0]=1221.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829847, downTime=829847, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1227.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829901, downTime=829901, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1230.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829917, downTime=829917, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=219.0, y[0]=1233.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829934, downTime=829934, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=216.0, y[0]=1233.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829954, downTime=829954, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=216.0, y[0]=1236.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=829984, downTime=829984, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=213.0, y[0]=1236.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=830017, downTime=830017, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_MOVE, id[0]=0, x[0]=213.0, y[0]=1236.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=830034, downTime=830034, deviceId=0, source=0x1002 }
evt: MotionEvent { action=ACTION_UP, id[0]=0, x[0]=213.0, y[0]=1236.0, toolType[0]=TOOL_TYPE_FINGER, buttonState=0, metaState=0, flags=0x0, edgeFlags=0x0, pointerCount=1, historySize=0, eventTime=830051, downTime=830051, deviceId=0, source=0x1002 }

Any ideas?

Comment 11

[Kartikaya Gupta (email:[e-mail])]

Oh sorry I was wrong, we do use the downTime on the event to ensure that the right set of events are grouped together [1]. Please make sure that your MOVE events have the same downTime as the down event. i.e. the first argument to MotionEvent.obtain should be the same value for a given block of input events.

[1] http://searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/gfx/NativePanZoomController.java#66

Comment 12

[manos]

Yes, when EVENT_MOVE Motion events have the same downtime scrolling happens smoothly.
Thanks for pointing me to the right direction!

Comment 13

[Sebastian Kaspari (:sebastian; :pocmo)]

Okay, I'll close this because there's no bug to fix.

In the future you might be able to get faster answers or reach more people using our IRC channel (#mobile):
https://wiki.mozilla.org/IRC

..or using the mobile-firefox-dev mailing list:
https://mail.mozilla.org/listinfo/mobile-firefox-dev