Open Bug 1581126 Opened 1 year ago Updated 9 days ago

Hook up pinch gestures on Linux touchpads for APZ zooming

Categories

(Core :: Panning and Zooming, enhancement, P3)

71 Branch
enhancement

Tracking

()

UNCONFIRMED

People

(Reporter: sawyerbergeron, Assigned: aemad)

References

(Depends on 2 open bugs, Blocks 2 open bugs)

Details

Attachments

(3 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

  1. Open a website on Firefox on Linux (either X11 or Wayland)
  2. Perform a pinch gesture on the touchpad

Actual results:

no-op

Expected results:

Currently: reflow zoom
Later: smooth APZ zoom as tracked by https://bugzilla.mozilla.org/show_bug.cgi?id=1461360 and described by https://bugzilla.mozilla.org/show_bug.cgi?id=789906

This is a sibling report to https://bugzilla.mozilla.org/show_bug.cgi?id=1568676, as I noticed there was no similar report for Linux. Expected result is similar/identical.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Panning and Zooming
Product: Firefox → Core

Thanks for filing.

Depends on: desktop-zoom-xp
Priority: -- → P3
Duplicate of this bug: 1607007

Fairly core functionality now for touchpads, which apparently no browser on linux has as of yet.

(In reply to Sury from comment #5)

Fairly core functionality now for touchpads, which apparently no browser on linux has as of yet.

There might be an opportunity to get ahead of Chrome here, and be the first browser to support this on Linux.

It was mentioned in bug 1607007 that Evince supports this. A good first step here would be investigate how Evince does this, i.e. what APIs it uses to listen for / detect pinch gestures.

(In reply to Botond Ballo [:botond] from comment #6)

(In reply to Sury from comment #5)

Fairly core functionality now for touchpads, which apparently no browser on linux has as of yet.

There might be an opportunity to get ahead of Chrome here, and be the first browser to support this on Linux.

It was mentioned in bug 1607007 that Evince supports this. A good first step here would be investigate how Evince does this, i.e. what APIs it uses to listen for / detect pinch gestures.

Yes, it really would be. Native Gnome apps have support, including Gnome-Web (ie. epiphany). However epiphany is far too choppy to be useful for web browsing.

There might be a clue here:

https://blogs.gnome.org/alexm/2019/09/13/gnome-and-gestures-part-1-webkitgtk/

The OP commented:

A large part of the reason is that all major browsers (Firefox, Chromium/Chrome) still run on X11, and there’s no way to have touchpad gestures (other than scroll-as-swipe) on X11.

For example, Chrome does support gestures on touchscreen, but not on touchpad precisely because of that.

I'm happy to try picking this up now that I'm a bit more familiar with APZ/APZC, a little less sure how to tie into the new granular zoom facility but I'm guessing some prior work should be visible in HandleGestureEvent for touchscreen pinch handling?

AFAIK it should be possible to get working on X, I could swear Xinput2 has support for smooth touchpad pinch gestures.

Evince does also support it on my end so that will also be helpful.

(In reply to Sawyer Bergeron from comment #8)

I'm happy to try picking this up now that I'm a bit more familiar with APZ/APZC,

Cool, thanks Sawyer!

a little less sure how to tie into the new granular zoom facility but I'm guessing some prior work should be visible in HandleGestureEvent for touchscreen pinch handling?

Here is the corresponding code that makes this work on macOS. Basically, it listens for OS events of the appropriate type, converts them into our PinchGestureInput events, and dispatches the converted events.

So, if we can figure out what to listen for in our GTK widget backend and convert that into PinchGestureInput events, that should basically just work. For an existing example of listening to a native GTK event, converting it into a Firefox event, and dispatching the event, see OnScrollEvent().

Looks like Evince uses GtkGestureZoom?

One issue I'm noticing is that although wayland itself supports pinch as a segmented [start][main gesture][gesture end] as detailed in https://github.com/wayland-project/wayland-protocols/blob/master/unstable/pointer-gestures/pointer-gestures-unstable-v1.xml, GTK itself only provides for deltas emitted during the middle of the pinch, and has no explicit way of saying a gesture will begin or the touch has ended. Not sure if that's going to end up being a problem later (especially if some sort of "zoom fling" or smooth animation out is planned)

(In reply to Sawyer Bergeron from comment #11)

Not sure if that's going to end up being a problem later (especially if some sort of "zoom fling" or smooth animation out is planned)

We don't currently do any sort of zoom fling or smooth animation on touchscreens (nor macOS touchpads), we just scale synchronously with the finger movement.

Actually correction to my earlier statement, it does broadcast the current phase of the gesture in the event: https://developer.gnome.org/gdk3/stable/gdk3-Event-Structures.html#GdkEventTouchpadPinch

Blocks: desktop-zoom

I was taking a look at GtkGestureZoom and GdkEventTouchpadPinch and they require GDK 3.14 and 3.18 respectively. Using GtkGestureZoom even with a GTK_CHECK_VERSION(3, 14, 0) guard results in a compilation error:

warning: 'gtk_gesture_zoom_new' is deprecated: Not available before 3.14 [-Wdeprecated-declarations]

This is because GDK_MAX_ALLOWED_VERSION is set to 3.4 currently. Unfortunately I've never contributed to Firefox before so I don't know if there is a specific procedure to up this to 3.14.

(In reply to Benjamin Cheng from comment #14)

Because Firefox' primary distribution method is a binary which has to run with the oldest GTK supported but wants to use the newest GTK features available, you cannot use compile-time feature detection. The only use of GTK_CHECK_VERSION in the code is to define macros and types that were introduced in later GTK versions.

You need to use dlsym to discover the functions at runtime. For an example, check out sGdkWindowMoveToRect in widget/gtk/nsWindow.cpp.

Depends on: 1640186
Summary: Hook up pinch gestures on Linux touchpads in preparation for APZ zooming → Hook up pinch gestures on Linux touchpads for APZ zooming

(In reply to Sawyer Bergeron from comment #8)

AFAIK it should be possible to get working on X, I could swear Xinput2 has support for smooth touchpad pinch gestures.

Evince does also support it on my end so that will also be helpful.

Hey Sawyer, just wanted to follow up on this: have you actually experienced touchpad pinch-zooming working under X11 in any application (such as Evince)?

I tried just now and it works for me in Evince under Wayland, but not under X11.

I just retested this under gnome on x11. Touchpad pinch zooming does not appear to work, but touchscreen zooming did which was probably part of my confusion earlier.

Testing gnome web and evince on x11 touchpad pinch zoom does not work, but works for both on wayland.

Thanks. I have meanwhile looked at the GTK source code as well, and confirmed that it only supports touchpad pinch gestures in its Wayland and Quartz (macOS) backends, not in its X11 backend.

So, at least in the short term, this will be fixed for Wayland only.

Could you please give us an update after you have released a fix for Wayland if it's not too troublesome for you? Currently, using Firefox on X11 with the Multi-Touch Zoom add-on, holding control + scrolling the webpage already does a great job at zooming the webpage, and it would be great to integrate with pinch-to-zoom! Also, after some web searching, it does seem like X11 has bad support for multi-touch events. Really looking forward to this feature! Thanks in advance!

Yep, we'll post updates to this bug as the Wayland implementation is being developed.

It has come to my attention that support for multi-touch gestures in X11 and GTK's X11 backend is being worked on: https://bill.harding.blog/2020/10/06/q3-linux-touchpad-like-macbook-update-multitouch-gesture-test-packages-are-ready/

Firefox will use GTK APIs, so while initially pinch gestures will only work with Wayland, once the work referenced above lands in X11 and GTK and users upgrade to versions of X11 and GTK that include this support, pinch gestures should start working in X11 as well.

I've done some initial work on this and am able to get scale events successfully from gtk, I still need to work on hooking things up with APZ

I'm getting somewhat lost trying to work through how to get events into APZ now, the directmanip prior work is helping somewhat but I think I may need a few pointers. Should I push what I have to phabricator or discuss here?

You can upload what you have (you don't need to use phabricator yet if it's an early patch), and then ask questions about it here. There is also the #apz channel on https://chat.mozilla.org/

I think nsBaseWidget::DispatchInputEvent should be used for getting the events into APZ.

Just as a FYI for people following along, this bug is the focus of an Outreachy project that Mozilla is sponsoring. So the intern that gets selected will be working on completing this bug. Although implementation advice is welcome, for now we'd like to leave enough scope on this bug for it to remain a worthwhile intern project. Thanks!

(In reply to Botond Ballo [:botond] from comment #6)

(In reply to Sury from comment #5)

Fairly core functionality now for touchpads, which apparently no browser on linux has as of yet.

There might be an opportunity to get ahead of Chrome here, and be the first browser to support this on Linux.

Just wanted to follow up on this and mention that I have confirmed that:

  • Newer versions of Chrome (87 and later) can run in native Wayland mode (using the command line flags --enable-features=UseOzonePlatform --ozone-platform=wayland).
  • However, even in native Wayland mode, touchpad pinch-zooming does not work in Chrome. (I've looked through their experimental feature flags as well and didn't find one that would enable this feature.)

So, it does seem likely that Firefox will be the first major browser to support this feature on Linux.

Assignee: nobody → eng.alaaemad.ae

I built Firefox in debug mode with the patch applied. I get a crash on startup, with the attached stack trace.

Blocks: 1685395
You need to log in before you can comment on or make changes to this bug.