1001465 - [b2gperf] Scrolling tests fail to detect when scroll has finished

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

With bug 958036 it now appears we can scroll vertically in many apps that were previously failing, however we now fail to detect that the touch event has finished.

$ b2gperf --test-type=scrollfps --iterations=3 --settle-time=5 Contacts
mozversion.mozversion.RemoteB2GVersion INFO | Unable to find system/sources.xml
2014-04-25 16:39:03,985 B2GPerfRunner WARNING | Missing required DataZilla field: build revision
2014-04-25 16:39:03,985 B2GPerfRunner WARNING | Missing required DataZilla field: device name
2014-04-25 16:39:03,985 B2GPerfRunner WARNING | Missing required DataZilla field: oauth key
2014-04-25 16:39:03,985 B2GPerfRunner WARNING | Missing required DataZilla field: project
2014-04-25 16:39:03,986 B2GPerfRunner WARNING | Missing required DataZilla field: oauth secret
2014-04-25 16:39:03,986 B2GPerfRunner WARNING | Missing required DataZilla field: branch
2014-04-25 16:39:03,986 B2GPerfRunner INFO    | Reports will not be submitted to DataZilla
2014-04-25 16:39:04,607 B2GPerfRunner INFO    | Running B2GPerfScrollContactsTest
2014-04-25 16:39:04,858 B2GPopulate INFO | Populating 200 contacts
Traceback (most recent call last):
  File "/Users/dhunt/workspace/b2gperf/b2gperf/b2gperf.py", line 357, in run
    self.test()
  File "/Users/dhunt/workspace/b2gperf/b2gperf/b2gperf.py", line 491, in test
    lambda m: m.execute_script(
  File "/Users/dhunt/.virtualenvs/b2gperf/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/wait.py", line 143, in until
    cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.053303957 seconds

Traceback (most recent call last):
  File "/Users/dhunt/workspace/b2gperf/b2gperf/b2gperf.py", line 230, in measure_app_perf
    test.run()
  File "/Users/dhunt/workspace/b2gperf/b2gperf/b2gperf.py", line 375, in run
    raise ExceededThresholdError()
ExceededThresholdError: Exceeded failure threshold for gathering results

Note that this was using the latest gaiatest, which is pinned to marionette_client. I've also tried with the latest marionette_client version (0.7.6) with the same results.

The relevant line in b2gperf is: https://github.com/mozilla/b2gperf/blob/423fc744cea3bf040c1817f5db4d9feaa843f77a/b2gperf/b2gperf.py#L490 and pasted below:

Wait(self.marionette, timeout=30).until(
    lambda m: m.execute_script(
        'return window.wrappedJSObject.touchend;', new_sandbox=False))

I wonder if since fixing the APZ issue this touchend is somehow in a different window/context?

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Malini: Could you cast your eye over this? It might be something really simple that we need to change in b2gperf.

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

Yes, it's on my task list for the week, likely related to the issues we're seeing with Bug 995989 and Bug 997377. I'll assign it to myself so I can keep track of the potential fallout

Comment 3

[Malini Das [:mdas] - Away, not checking bugmail]

As an update, the patch for Bug 995989 fixes this problem, so I think we may be able to resolve this soon.

Comment 4

[Malini Das [:mdas] - Away, not checking bugmail]

Fixed after Bug 995989 landed