516727 - Electrolysis+Fennec: forward mouse events from the chrome tiles to the content process

Status: RESOLVED FIXED

(Core :: IPC, defect)

Description

[Benjamin Smedberg]

In order to remote content in Fennec, we need to forward mouse and keyboard events from the parent (chrome) process tiles to the content process.

This is related to bug 505847, which covers the graphics portion of being able to do canvas.drawWindow from a OOP iframe into a tile.

Comment 1

[Olli Pettay [:smaug][[e-mail]]]

Forward? I thought Fennec currently sends events manually using windowutils.

Or is this bug about creating OOP handling for windowutils (or the event handling part of it)?

Comment 2

[Benjamin Smedberg]

It's about OOP handling of that entire chain of getting events from the canvas tiles to the underlying browser (which will be in a different process). I don't think anybody knows the exact details of how we plan to do that.

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

But is the idea to do something similar what Fennec does now?
Which, *I think*, is to capture events in chrome, filter out the relevant
events, convert coordinates etc and then use nsDOMWindowUtils to send new events
to content window. (No idea how or which mouse events are handled. How to handle
.relatedTarget etc.)

Comment 4

[Benjamin Smedberg]

Yes, I think so.

Comment 5

[Roy Frostig :froystig]

Yeah, Fennec currently captures MouseEvents and then forwards some of them
using DOMWindowUtils' sendMouseEvent.  Relevant files in mobile-browser repo
are:

* chrome/content/InputHandler.js   (MouseModule.prototype)
* chrome/content/browser.js        (ContentCustomClicker.prototype)

Some other events may happen to go to content via the <xul:browser> gaining
keyboard focus (tab key, typing in input fields, etc.).  These are not manually
forwarded.

Olli, I don't think .relatedTarget is handled explicitly anywhere in Fennec code that I've seen.

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

Actually, .relatedTarget shoulnd't be an issue. It is handled by ESM, if 
right mouse move events are sent.

Comment 7

[Olli Pettay [:smaug][[e-mail]]]

So as far as I know, Fennec creates only artificial mouse events.
Keyboard events are dispatched normally to the focused target.

Comment 8

[Olli Pettay [:smaug][[e-mail]]]

Attachment: wip

Comment 9

[Olli Pettay [:smaug][[e-mail]]]

I'm not sure if bug 516522 could fix this too. Basically get the wrapper for the
window, get windowutils from that and send mouse event.

Comment 10

[Olli Pettay [:smaug][[e-mail]]]

Attachment: patch

Comment 11

[Benjamin Smedberg]

JPW could fix this, but since JPW is synchronous, it's only meant to be a crutch, not a good long-term solution.

Comment 12

[Benjamin Smedberg]

Comment on attachment 409389 [details] [diff] [review]
patch

>diff --git a/dom/ipc/TabParent.h b/dom/ipc/TabParent.h
>--- a/dom/ipc/TabParent.h
>+++ b/dom/ipc/TabParent.h
>@@ -61,6 +61,9 @@
>     void LoadURL(nsIURI* aURI);
>     void Move(PRUint32 x, PRUint32 y, PRUint32 width, PRUint32 height);
>     void Activate();
>+    void SendMouseEvent(const nsAString& aType, float aX, float aY,
>+                        PRInt32 aButton, PRInt32 aClickCount,
>+                        PRInt32 aModifiers, PRBool aIgnoreRootScrollFrame);

How permanent is this? This at least needs a doccomment. Might it be better named SynthesizeMouseEvent? I don't particularly like that `aType` is a string (although I guess that's what you get from the DOM event code): strings are slightly more expensive to serialize/deserialize: do we know at compile-time the list of possible mouse events and could make this an enum instead?

r=me in any case, and file followups if appropriate.

Comment 13

[Olli Pettay [:smaug][[e-mail]]]

The message looks on purpose exactly like the method in nsIDOMWindowUtils.

Comment 14

[Benjamin Smedberg]

ok. At least reference the other method in the IPDL doccomment, then.

Comment 15

[Olli Pettay [:smaug][[e-mail]]]

Attachment: +comment

Comment 16

[Olli Pettay [:smaug][[e-mail]]]

http://hg.mozilla.org/projects/electrolysis/rev/2856c7a71f4f