599001 - Support MozTouch events on Ubuntu

Status: RESOLVED INVALID

(Core :: Widget: Gtk, defect)

Description

[:Felipe Gomes (needinfo for replies!)]

Recently Canonical announced multitouch support to be first released with Ubuntu Maverick Meerkat (10.10). They've built drivers for various devices, raw APIs and a gesture library that applications can use.

The MozTouch events are raw multitouch information that are exposed to regular webpages via DOM Events.
What's needed to support this is to add code on the widget side that is able to retrieve the information from the system and dispatch the gecko events with typical event information (X/Y positions, time..)


Filing as support for Ubuntu only for now to keep the bug real. Support for other linux distros can come later when these drivers make into the upstream kernel and other distros provide libraries for it.

Comment 2

[Reed Loden [:reed]]

Felipe, Florian would love to get started getting this working on Linux, but he needs somebody to point him towards the applicable code. Think you could help him with this?

Comment 3

[:Felipe Gomes (needinfo for replies!)]

Of course. Florian, the code that deals with platform-specific events is located under the widget/ folders, and the nsWindow.cpp files in each subdirectory are usually the most relevant here.

For example, the gtk function responsible for receiving mouse events and converting them to DOM events is this one:
http://mxr.mozilla.org/mozilla-central/source/widget/src/gtk2/nsWindow.cpp#5758

The function that handles touch events will probably look very similar. When you retrieve the touch information from the system, you then create a nsMozTouchEvent object and feed the correct information to its properties, and finally call DispatchEvent.

Here's the function as implemented on Windows:
http://mxr.mozilla.org/mozilla-central/source/widget/src/windows/nsWindow.cpp#6181


That's basically it. There's another small part that handles registering each window to receive touch events, but that's mostly a quirk from Windows. I don't know if it's relevant on Linux, so I'll wait to see what you say. If it's needed for Linux too I'll point to the right direction.


I'll be happy to help with this, and feel free to find me on IRC too.

Comment 4

[:Felipe Gomes (needinfo for replies!)]

Also, the function that actually does the work on gtk is here:
http://mxr.mozilla.org/mozilla-central/source/widget/src/gtk2/nsWindow.cpp#2602

The one I pointed above just do some pre-checks and call it.

Comment 5

[Florian Echtler]

All right, I've had a look at the source and AFAICT there's no clean way to do this, it's going to be a hack. Reason: XInput2 support has been merged in GTK+ 2.90.1, but Lucid and Maverick are still packaged with the 2.2x series. So no joy wrt native XI2 support in GTK+, at least not for the next 6 months.

I have to admit I'm pretty rusty in GTK, therefore I have a few questions:
- Where is the relevant main window actually created?
- I need the X11 Window* object for the respective main window to register for XI2 events. I assume this is done in nsWindow.cpp;CreateGdkWindow, correct?
- I think something like plugin_window_filter_func will be also needed for the main window, XI2 events are delivered as GenericEvents with specific extensions. That's probably the easiest solution.

Florian

Comment 6

[Karl Tomlinson (:karlt)]

(In reply to comment #5)
> I have to admit I'm pretty rusty in GTK, therefore I have a few questions:
> - Where is the relevant main window actually created?

I'm not sure what you mean by the main window.  Gecko collects most X events on the event window rather than expecting them to bubble to the toplevel window.

> - I need the X11 Window* object for the respective main window to register for
> XI2 events. I assume this is done in nsWindow.cpp;CreateGdkWindow, correct?

Yes, that should usually be the window you want.  Some windows are also created in moz_container_realize.  Usually they will have a child window created from CreateGdkWindow, but popup and menu windows, for example, may not have a child window.

> - I think something like plugin_window_filter_func will be also needed for the
> main window, XI2 events are delivered as GenericEvents with specific
> extensions. That's probably the easiest solution.

Yes, probably.  gdk_x11_register_standard_event_type may be necessary.

Comment 7

[:Felipe Gomes (needinfo for replies!)]

If there is any cost involved in registering or filtering these events, you can implement nsWindow::RegisterTouchWindow and nsWindow::UnregisterTouchWindow. They're implemented as no-ops in nsBaseWidget but can be overriden.

RegisterTouchWindow will only be called when a webpage adds an event listener for MozTouchEvent (e.g. document.addEventListener("MozTouchDown"...)). If that page loses focus (say, switching to another tab), then UnregisterTouchWindow is called, and RegisterTouchWindow will be called again whenever that page is reactivated.

Comment 9

[Ernst Sjostrand]

In Ubuntu Precise and other linux distributions with Xserver 1.12 there's now support for "smooth scrolling" from touchpads for example, and it works out of the box in GTK3 applications. Would be nice if Firefox was hooked up to that. If GTK3 is a problem you could go directly for the XInput2 api I guess?