1036708 - [1.4] wpt tests fail to launch

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Certification Suite, defect)

Description

[Jonathan Griffin (:jgriffin)]

On 1.4, wpt fails to launch; the following error appears in the log:

{"source": "web-platform-tests", "thread": "Thread-TestrunnerManager-1", "time": 1404949769963, "action": "log", "message": "Traceback (most recent call last):\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/wptrunner-0.3.13-py2.7.egg/wptrunner/executors/executormarionette.py\", line 67, in setup\n    self.after_connect()\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/wptrunner-0.3.13-py2.7.egg/wptrunner/browsers/b2g.py\", line 312, in after_connect\n    self.browser.after_connect(self)\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/wptrunner-0.3.13-py2.7.egg/wptrunner/browsers/b2g.py\", line 225, in after_connect\n    self.use_cert_app()\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/wptrunner-0.3.13-py2.7.egg/wptrunner/browsers/b2g.py\", line 249, in use_cert_app\n    self.wait_for_homescreen(marionette)\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/wptrunner-0.3.13-py2.7.egg/wptrunner/browsers/b2g.py\", line 307, in wait_for_homescreen\n    }\"\"\", script_timeout=30000)\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette.py\", line 1166, in execute_async_script\n    filename=os.path.basename(frame[0]))\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/marionette_client-0.7.11-py2.7.egg/marionette/decorators.py\", line 35, in _\n    return func(*args, **kwargs)\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette.py\", line 613, in _send_message\n    self._handle_error(response)\n  File \"/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette.py\", line 675, in _handle_error\n    raise errors.ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)\nScriptTimeoutException: ScriptTimeoutException: timed out\n\n", "level": "ERROR", "pid": 18332}

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: web-platform-tests_structured.log

Comment 2

[James Graham [:jgraham]]

So I have no idea what's going on here, but it seems to be quite 1.4 specific.

With 1.3 and 2.0 things seem to work more or less as expected.

With 1.4 I get as far as waiting for the homescreen to load (with slightly rearranged code compared to the log output above) and after not very long I lose the marionette and adb connections to the device. After that it isn't possible to recover without power cycling the phone. I can't see anything obviously suspicious in logcat, and I don't see any crash logs in /data/b2g/mozilla/Crash Reports/

Comment 3

[Jonathan Griffin (:jgriffin)]

I keep getting this inaccurate error when the wpt tests try to launch:

Traceback (most recent call last):
  File "/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/fxos_certsuite-0.1-py2.7.egg/certsuite/harness.py", line 326, in run_tests
    device.restore()
  File "/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/fxos_certsuite-0.1-py2.7.egg/certsuite/harness.py", line 161, in restore
    self.device.pushFile(local_path, remote_path)
  File "/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/mozdevice-0.37-py2.7.egg/mozdevice/devicemanagerADB.py", line 204, in pushFile
    (localname, destname))
DMError: Attempted to push a file (/tmp/tmpZ6inxi/system/etc/hosts) to a directory (/system/etc/hosts)!

Comment 4

[Jonathan Griffin (:jgriffin)]

...and sometimes before that I get this error:

  File "/home/jgriffin/fxos-certsuite/certsuite_venv/local/lib/python2.7/site-packages/wptrunner-0.3.13-py2.7.egg/wptrunner/browsers/b2g.py", line 135, in setup_hosts
    f.seek(f.tell() - 1)
IOError: [Errno 22] Invalid argument

This implies that the adb pull command earlier failed.

Comment 5

[Jonathan Griffin (:jgriffin)]

Ah, there's a bug in b2g.py, it currently reads:

            self.device.getFile("/etc/hosts", hosts_path)

...should be "/system/etc/hosts".

Comment 6

[Jonathan Griffin (:jgriffin)]

So after working around those problems, I was able to get a logcat for when the phone dies, which is here:

W/Adreno-ES20(  766): <core_glBindTexture:496>: GL_INVALID_ENUM

One thing I notice is that the tests start running before the homescreen is actually visible (even though Marionette may report that it's there...the splash screen is still visible).  I wonder if we added some delay before starting the tests, if it would work better?

Comment 7

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #6)
> So after working around those problems, I was able to get a logcat for when
> the phone dies, which is here:
> 
> W/Adreno-ES20(  766): <core_glBindTexture:496>: GL_INVALID_ENUM
> 
> One thing I notice is that the tests start running before the homescreen is
> actually visible (even though Marionette may report that it's there...the
> splash screen is still visible).  I wonder if we added some delay before
> starting the tests, if it would work better?

We actually don't get as far as starting to run the tests; the phone dies while waiting for a network connection to appear.

Comment 8

[Jonathan Griffin (:jgriffin)]

I haven't been able to pin this down.  The hang occurs after we install a new profile and restart B2G, but if I reboot the phone after the hang with the new profile still active, it boots up OK.

I've also added tons of sleeps and logging in order to determine that it isn't an obvious race condition like we've seen in the past, in which we're restarting "too quickly".

It seems related to B2GDeviceRunner.start(), but I haven't nailed down where.

On eng builds, things seem to work fine, which is mysterious.

Comment 9

[Jonathan Griffin (:jgriffin)]

We didn't ship web-platform-tests with the MCTS for 1.4, and relevant devices have already passed through certification.