Electrolysis+Fennec: forward mouse events from the chrome tiles to the content process

RESOLVED FIXED

Status

()

Core
IPC
RESOLVED FIXED
8 years ago
8 years ago

People

(Reporter: Benjamin Smedberg, Assigned: smaug)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 2 obsolete attachments)

(Reporter)

Description

8 years ago
In order to remote content in Fennec, we need to forward mouse and keyboard events from the parent (chrome) process tiles to the content process.

This is related to bug 505847, which covers the graphics portion of being able to do canvas.drawWindow from a OOP iframe into a tile.
Forward? I thought Fennec currently sends events manually using windowutils.

Or is this bug about creating OOP handling for windowutils (or the event handling part of it)?
(Reporter)

Comment 2

8 years ago
It's about OOP handling of that entire chain of getting events from the canvas tiles to the underlying browser (which will be in a different process). I don't think anybody knows the exact details of how we plan to do that.
But is the idea to do something similar what Fennec does now?
Which, *I think*, is to capture events in chrome, filter out the relevant
events, convert coordinates etc and then use nsDOMWindowUtils to send new events
to content window. (No idea how or which mouse events are handled. How to handle
.relatedTarget etc.)
(Reporter)

Comment 4

8 years ago
Yes, I think so.
Yeah, Fennec currently captures MouseEvents and then forwards some of them
using DOMWindowUtils' sendMouseEvent.  Relevant files in mobile-browser repo
are:

* chrome/content/InputHandler.js   (MouseModule.prototype)
* chrome/content/browser.js        (ContentCustomClicker.prototype)

Some other events may happen to go to content via the <xul:browser> gaining
keyboard focus (tab key, typing in input fields, etc.).  These are not manually
forwarded.

Olli, I don't think .relatedTarget is handled explicitly anywhere in Fennec code that I've seen.
Actually, .relatedTarget shoulnd't be an issue. It is handled by ESM, if 
right mouse move events are sent.
Depends on: 522533
So as far as I know, Fennec creates only artificial mouse events.
Keyboard events are dispatched normally to the focused target.
Created attachment 407047 [details] [diff] [review]
wip
I'm not sure if bug 516522 could fix this too. Basically get the wrapper for the
window, get windowutils from that and send mouse event.
Summary: Electrolysis+Fennec: forward mouse and keyboard events from the chrome tiles to the content process → Electrolysis+Fennec: forward mouse events from the chrome tiles to the content process
Created attachment 409389 [details] [diff] [review]
patch
Attachment #407047 - Attachment is obsolete: true
Attachment #409389 - Flags: review?(benjamin)
(Reporter)

Comment 11

8 years ago
JPW could fix this, but since JPW is synchronous, it's only meant to be a crutch, not a good long-term solution.
(Reporter)

Comment 12

8 years ago
Comment on attachment 409389 [details] [diff] [review]
patch

>diff --git a/dom/ipc/TabParent.h b/dom/ipc/TabParent.h
>--- a/dom/ipc/TabParent.h
>+++ b/dom/ipc/TabParent.h
>@@ -61,6 +61,9 @@
>     void LoadURL(nsIURI* aURI);
>     void Move(PRUint32 x, PRUint32 y, PRUint32 width, PRUint32 height);
>     void Activate();
>+    void SendMouseEvent(const nsAString& aType, float aX, float aY,
>+                        PRInt32 aButton, PRInt32 aClickCount,
>+                        PRInt32 aModifiers, PRBool aIgnoreRootScrollFrame);

How permanent is this? This at least needs a doccomment. Might it be better named SynthesizeMouseEvent? I don't particularly like that `aType` is a string (although I guess that's what you get from the DOM event code): strings are slightly more expensive to serialize/deserialize: do we know at compile-time the list of possible mouse events and could make this an enum instead?

r=me in any case, and file followups if appropriate.
Attachment #409389 - Flags: review?(benjamin) → review+
The message looks on purpose exactly like the method in nsIDOMWindowUtils.
(Reporter)

Comment 14

8 years ago
ok. At least reference the other method in the IPDL doccomment, then.
Created attachment 410547 [details] [diff] [review]
+comment
Assignee: nobody → Olli.Pettay
Attachment #409389 - Attachment is obsolete: true
http://hg.mozilla.org/projects/electrolysis/rev/2856c7a71f4f
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.