918572 - b2gperf fps browser test fail on inari v1train: timeout on return window.document.readyState

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Robert Wood [:rwood]]

Attachment: fps_browser_fail_logcat.log

As noticed by Stephen, the b2gperf fps browser test is failing on inari with the latest v1train (Sep19). Only some scroll_fps iterations fail, but it fails often enough to cross the failure threshold and fail the entire test.

I was able to reproduce the same failure on my local b2gperf setup. Sometimes takes awhile to reproduce i.e. the failure may happen on the 25th iteration, or sometimes on the very first iteration.

Watching the test run, there is no visible indication of the failure; looks like the scrolling is working just fine, so I couldn't determine exactly what is happening.

Command line output follows:

(thursday)rwood@ubuntu:~/b2gperf$ b2gperf --delay=5 --testvars=${TESTVARS_PATH} --dz-branch=v1-train --dz-project=b2g --dz-key=key --dz-secret=secret --test-type=scrollfps Browser
Missing required DataZilla field: build revision
Missing required DataZilla field: gecko revision
Missing required DataZilla field: gaia revision
Reports will not be submitted to DataZilla
running fps test
Traceback (most recent call last):                                    
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/b2gperf/b2gperf.py", line 298, in test_scrollfps
    self.scroll_app(app_name)
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/b2gperf/b2gperf.py", line 371, in scroll_app
    MarionetteWait(self.marionette, 30).until(lambda m: m.execute_script('return window.document.readyState;', new_sandbox=False) == 'complete')
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/b2gperf/wait.py", line 31, in until
    value = method(self._marionette)
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/b2gperf/b2gperf.py", line 371, in <lambda>
    MarionetteWait(self.marionette, 30).until(lambda m: m.execute_script('return window.document.readyState;', new_sandbox=False) == 'complete')
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/marionette/marionette.py", line 611, in execute_script
    scriptTimeout=script_timeout)
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/marionette/marionette.py", line 350, in _send_message
    self._handle_error(response)
  File "/home/rwood/.virtualenvs/thursday/local/lib/python2.7/site-packages/marionette/marionette.py", line 385, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: Error: Permission denied to access property 'document'
stacktrace:
	execute_script @undefined, line undefined
	inline javascript, line 1397
	src: "let __marionetteFunc = function(){return window.document.readyState;};__marionetteFunc.apply(null, __marionetteParams);"

Logcat attached. Hope this helps!

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

This has been happening since the FPS tests were first started. It was raised in https://github.com/mozilla/b2gperf/issues/16 and has already been resolved. I'll push a new release out today.