863422 - Add a basic fake touch events support to the desktop-helper extension

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia, defect)

Description

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Attachment: poc

Here is a little poc about how to add fake touch events supports (only one touch) in the desktop helper extension.

It kind of works, which means you can scroll the settings, and pan a page with text fields without giving them the focus but... it does not works with inertial scrolling for some reasons I have not yet looked at.

Also it has a fun side effect. <select> element won't create a popup if they don't receive a mouse event but it seems like there is an issue with <select> that is visible in this case. Selecting something triggers an error in the change event. Not sure what is the root cause of this yet but I do wonder if this is a window target issue. For example it is possible that the change event is dispatched to the target but using a wrong window. that's just a theory.

Comment 1

[Myk Melez [:myk] [@mykmelez]]

Harald has been working on this for the Simulator and has a pull request for an implementation, although it has a few issues:

https://github.com/mozilla/r2d2b2g/pull/439

Comment 2

[Kevin Grandon :kgrandon]

Thanks for the link Myk! It would be ideal to reduce duplication and share code if possible. Perhaps we can focus on one solution and implement a git subproject?

Comment 3

[Myk Melez [:myk] [@mykmelez]]

The Simulator uses Git submodules to incorporate some third-party code, and it works pretty well, so we should be able to use them to share an implementation if there's a way to create one that works on both projects.

Comment 4

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Myk Melez [:myk] [@mykmelez] from comment #1)
> Harald has been working on this for the Simulator and has a pull request for
> an implementation, although it has a few issues:
> 
> https://github.com/mozilla/r2d2b2g/pull/439

The main difference between this code and the one from Harald is that mouse events are never sent to any apps like what it happens on the device.

The exception to that is the system app (this is a kind of hybrid beast, even on the device).


Also the thing you mentionned with scrolling very slowly could be that you hit the context menu path that is embedded in Harald's PR. (Which I removed here for testing).

Comment 5

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Myk Melez [:myk] [@mykmelez] from comment #3)
> The Simulator uses Git submodules to incorporate some third-party code, and
> it works pretty well, so we should be able to use them to share an
> implementation if there's a way to create one that works on both projects.

The simulator use Gaia in any cases right? If yes I would say the simulator will benefit from using desktop-helper extension.

desktop-helper is dedicated to device related APIs shims such as wifi, mobileconnection, sms, ... There are already some of them in Gaia that needs to be moved to the desktop-helper directory (or at least there is a wifi shim).

Also there are additional addons in gaia/tools/extensions that you probably don't want on the simulator such as keyboard and activities which are a part of the gecko/b2g/components folder. But the alarms extension will be useful for you as well I believe (it is a bit tied to desktop-helper for now).

I would also love to have a common place to develop the content of prosthesis/components. Fake geolocation and orientation supports would be lovely for Gaia developers.

Is there anything that prevent you from using the gaia tools/extensions folder?

Comment 6

[Myk Melez [:myk] [@mykmelez]]

(In reply to Vivien Nicolas (:vingtetun) (:21) from comment #5)
> (In reply to Myk Melez [:myk] [@mykmelez] from comment #3)
> > The Simulator uses Git submodules to incorporate some third-party code, and
> > it works pretty well, so we should be able to use them to share an
> > implementation if there's a way to create one that works on both projects.
> 
> The simulator use Gaia in any cases right? If yes I would say the simulator
> will benefit from using desktop-helper extension.

Yes, it bundles a Gaia profile.


> I would also love to have a common place to develop the content of
> prosthesis/components. Fake geolocation and orientation supports would be
> lovely for Gaia developers.
> 
> Is there anything that prevent you from using the gaia tools/extensions
> folder?

I'm not sure, as I'm unfamiliar with that folder.  It certainly seems possible in theory, and it'd be useful to share implementations; although there may be aspects of those implementations that are specific to the B2G shell the Simulator uses to load and run Gaia.  More investigation is needed here.

Comment 7

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Myk Melez [:myk] [@mykmelez] from comment #6)
> (In reply to Vivien Nicolas (:vingtetun) (:21) from comment #5)
> > 
> > Is there anything that prevent you from using the gaia tools/extensions
> > folder?
> 
> I'm not sure, as I'm unfamiliar with that folder.  It certainly seems
> possible in theory, and it'd be useful to share implementations; although
> there may be aspects of those implementations that are specific to the B2G
> shell the Simulator uses to load and run Gaia.  More investigation is needed
> here.

That would be awesome if it can be used by the simulator and so if we can share more code.

The specific things for touch events / scrolling should be very close when it comes to desktop runtimes.

Comment 8

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Attachment: Patch

I have tried to replicate as closely as possible what I observe on the device. The patch can land even without the 2 platforms patches that blocks it.

Comment 9

[Kevin Grandon :kgrandon]

Comment on attachment 740942 [details] [diff] [review]
Patch

Review of attachment 740942 [details] [diff] [review]:
-----------------------------------------------------------------

Everything seems to be working pretty well! A nice added benefit is that we can pull down the notification tray without selecting the text. Awesome!

::: tools/extensions/desktop-helper/bootstrap.js
@@ +34,4 @@
>      var mm = Cc['@mozilla.org/globalmessagemanager;1']
>                 .getService(Ci.nsIMessageBroadcaster);
>      mm.loadFrameScript('chrome://desktop-helper.js/content/content.js', true);
> +   mm.loadFrameScript("chrome://global/content/BrowserElementPanning.js", true);

nit: ahh, the eternal war of platform vs gaia quoting. Perhaps we should standardize.

Comment 10

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

https://github.com/vingtetun/gaia/commit/1c405e7a1c11b9a324ce23520244ca6c28f90bfb

Thanks!