Pinch action does not work in Browser app

RESOLVED INCOMPLETE

Status

RESOLVED INCOMPLETE
4 years ago
a year ago

People

(Reporter: njpark, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [affects=webqa])

Attachments

(6 attachments)

(Reporter)

Description

4 years ago
Following code zooms in/out images in Gallery app, but does not work in a Browser page:

    def test_UI_Actions(self):
        """https://moztrap.mozilla.org/manage/case/2462/"""
        gallery = Gallery(self.marionette)
        gallery.launch()
        gallery.wait_for_files_to_load(self.image_count)


         # Tap first image to open full screen view.
        image = gallery.tap_first_gallery_item()

        action = Actions(self.marionette)
        screen = self.marionette.find_element(*self._current_image_locator)

        time.sleep(5)

        pinch = MultiActions(self.marionette)
        current_image_mid_x = screen.size['width'] / 2
        current_image_mid_y = screen.size['height'] / 2

        index_finger = Actions(self.marionette)
        thumb = Actions(self.marionette)
        index_finger.press(screen,current_image_mid_x+10,current_image_mid_y-10).wait(0.15).move_by_offset(current_image_mid_x, -current_image_mid_y).release()
        thumb.press(screen,current_image_mid_x-10,current_image_mid_y+10).wait(0.15).move_by_offset(-current_image_mid_x, current_image_mid_y).wait().release()
        pinch.add(thumb).add(index_finger).perform()

Also tried a flick-like method, and while it does scroll the browser page, it still does not pinch zoom when combined together in a MultiActions object:

from gaiatest import GaiaTestCase
from marionette.marionette import Actions
from marionette.marionette import MultiActions
import sys,time
from marionette.by import By
from gaiatest.apps.browser.app import Browser

class TestUIActions(GaiaTestCase):

    _browser_frame_locator = (By.CSS_SELECTOR, 'iframe.browser-tab')
    _main_screen_locator = (By.ID, 'main-screen')

    def setUp(self):
        GaiaTestCase.setUp(self)

        self.data_layer.connect_to_wifi()

    def test_UI_Actions(self):
        browser = Browser(self.marionette)
        browser.launch()

        browser.go_to_url('http://mozilla.org/firefoxos')
        time.sleep(10)
        screen = self.marionette.find_element(*browser._browser_frame_locator)

        pinch = MultiActions(self.marionette)
        current_image_mid_x = screen.size['width'] / 2
        current_image_mid_y = screen.size['height'] / 2

        index_finger = Actions(self.marionette)
        thumb = Actions(self.marionette)
        #pdb.set_trace()
        index_finger.press(screen,current_image_mid_x-50,current_image_mid_y-50)
        x = 0
        while x < 100:
            index_finger.move_by_offset(-5, -5)
            index_finger.wait(0.001)
            x += 1
        index_finger.release()
      
        thumb.press(screen,current_image_mid_x+50,current_image_mid_y+50)
        y = 0
        while y < 100:
            thumb.move_by_offset(5, 5)
            thumb.wait(0.001)
            y += 1
        thumb.wait()
        thumb.release()
      
        pinch.add(index_finger).add(thumb).perform()
(Reporter)

Comment 1

4 years ago
It looks to me Browser's response to UI action is either slow, or requires something different from other apps.

Badly worded comment above: for the second code above, if I perform the actions individually, it does scroll up and down, but when combined as above, there was no UI action shown.
Whiteboard: [affects=webqa]
(Reporter)

Comment 2

4 years ago
mdas actually didn't get around fixing this issue, wondering whether there is any chance this can be fixed?  if so we can automate a lot of use case scenarios.  ni?ing :automatedtester for comments.
Flags: needinfo?(dburns)
Unfortunately the A-Team doesn't have bandwidth for this for some time. I think you would be better asking the FXOS Automation Team if they can do it. I am happy to do reviews.
Flags: needinfo?(dburns) → needinfo?(jlal)
This is unlikely to happen this quarter.... Are there any known workarounds?
Flags: needinfo?(jlal)
(Reporter)

Comment 5

4 years ago
CC'ing FXOS automation team
Created attachment 8663574 [details] [diff] [review]
pinchzoom.diff

I tried to get the current pinch action to work, which is currently used here:
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/graphics/orientation_zoom_base.py#59

But I couldn't get it working. I don't think what's in the tree currently, is working either.
Created attachment 8663608 [details]
test_gallery_zoom.py

I tried something like this based on comment 0, but that doesn't work either.
What I noticed with using print(datetime.datetime.now().time()) is that MultiActions takes only 0.015s, where it should take at least 1.15s, because of th wait() calls in the Actions().
Attachment #8663608 - Attachment mime type: text/x-python-script → text/plain
The tests that are supposed to test this behavior in Marionette unit tests are disabled because of bug 1060060.
Created attachment 8663686 [details]
test_gallery_zoom.py

This is something that works.
Attachment #8663686 - Attachment mime type: text/x-python-script → text/plain
Created attachment 8663687 [details]
test_browser_zoom.py

I tried the same in a browser zoom test, but there, it doesn't work for some reason.
Created attachment 8663727 [details]
test_browser_zoom.py

Using sendNativeTouchPoint like this works. This also works for the gallery app.

Thanks to :kats on irc who explained this to me.
In gaia browser, pinch zooming is handled by apz, in that case you need to use sendNativeTouchPoint. In the gallery app, pinch zooming is handled by the gallery app itself, so that's why sendTouchEvent works there as well:
https://groups.google.com/forum/#!msg/mozilla.dev.fxos/1Kh8RrK7QWI/lX74BeHJAQAJ
https://hg.mozilla.org/mozilla-central/raw-file/tip/gfx/doc/B2GInputFlow.svg
Created attachment 8664914 [details]
test_browser_zoom.py

This was my last attempt that points to a page that shows the various touch events. This is not zooming in the page on pinch zoom.
I noticed that when clientY is negative for the 2nd touch, then the page gets indeed zoomed in suddenly. This is what happened with the previous test that I attached.
Blocks: 1196432
closing as b2g related and new actions work has been done
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.