Closed Bug 948193 Opened 11 years ago Closed 11 years ago

Gaia UI Tests - Wi-Fi unittest is failing since Dec. 6th, 2:41 pm PDT

Categories

(Firefox OS Graveyard :: Wifi, defect)

Product:
Firefox OS Graveyard
Component:
Wifi
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: stephend, Assigned: hchang)

References

(
URL
)

Details

Attachments

(1 file)

Screen Shot 2013-12-16 at 4.48.36 PM.png
646.17 KB, image/png
Details 
Our Wi-Fi unittest is failing, like so:

Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.unittests.wifi/.env/local/lib/python2.7/site-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette_test.py", line 143, in run
testMethod()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.unittests.wifi/tests/python/gaia-ui-tests/gaiatest/tests/unit/settings/test_wifi_settings.py", line 11, in test_set_wifi
self.data_layer.enable_wifi()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.unittests.wifi/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 294, in enable_wifi
result = self.marionette.execute_async_script("return GaiaDataLayer.enableWiFi()", special_powers=True)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.unittests.wifi/.env/local/lib/python2.7/site-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script
filename=os.path.basename(frame[0]))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.unittests.wifi/.env/local/lib/python2.7/site-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette.py", line 577, in _send_message
self._handle_error(response)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.unittests.wifi/.env/local/lib/python2.7/site-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette.py", line 626, in _handle_error
raise ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)
ScriptTimeoutException: timed out

:vchang, any ideas here?  Thanks!

The function lives here: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/atoms/gaia_data_layer.js#L257

Regression range is:

Failed > Console Output  #1030 	Dec 6, 2013 2:41:53 PM	 
Success > Console Output  #102  Dec 6, 2013 11:55:01 AM

http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.unittests.wifi/1039/
https://pastebin.mozilla.org/3745078 is the logcat, Vincent/all; hope that helps!
Flags: needinfo?(vchang)
I am going to check this soon.
Flags: needinfo?(vchang)
I got "JavascriptException: Error: Expected IccHelper to be enabled." and can't run the test at all.
Did you encounter this?



TEST-START test_wifi_settings.py
test_set_wifi (test_wifi_settings.TestWiFiSettings) ... ERROR

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/chucklee/work/github/gaia/tests/python/gaia-ui-tests/env/local/lib/python2.7/site-packages/marionette_client-0.6.1-py2.7.egg/marionette/marionette_test.py", line 127, in run
    self.setUp()
  File "/home/chucklee/work/github/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 769, in setUp
    self.device.start_b2g()
  File "/home/chucklee/work/github/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 722, in start_b2g
    });""", script_timeout=60000)
  File "/home/chucklee/work/github/gaia/tests/python/gaia-ui-tests/env/local/lib/python2.7/site-packages/marionette_client-0.6.1-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/home/chucklee/work/github/gaia/tests/python/gaia-ui-tests/env/local/lib/python2.7/site-packages/marionette_client-0.6.1-py2.7.egg/marionette/marionette.py", line 577, in _send_message
    self._handle_error(response)
  File "/home/chucklee/work/github/gaia/tests/python/gaia-ui-tests/env/local/lib/python2.7/site-packages/marionette_client-0.6.1-py2.7.egg/marionette/marionette.py", line 612, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: Error: Expected IccHelper to be enabled.
	stacktrace:
	@app://system.gaiamobile.org/shared/js/operator_variant_helper.js, line 43
TEST-UNEXPECTED-FAIL | test_wifi_settings.py test_wifi_settings.TestWiFiSettings.test_set_wifi | 
----------------------------------------------------------------------
Ran 1 test in 25.181s

FAILED (errors=1)
The above stack isn't what we're seeing on:

  <!-- Mercurial-Information: <project name="mozilla-central" path="gecko" remote="hgmozillaorg" revision="9f12a9fab080"/> -->
  <project name="gecko.git" path="gecko" remote="mozillaorg" revision="a0bb585098cc89c454fac0297b5ef748d5cab82c"/>
Chuck are you running without a SIM in the device?
It happened with or without SIM in device.
But now I borrow another buri, and it doesn't happen on this one.
This is odd. This test fails for me in two different ways (both different than the reported failure).

When run with all the Settings application tests, it fails with the follow error (5 other tests fail as well):


TEST-START test_settings_wifi.py
test_connect_to_wifi_via_settings_app (test_settings_wifi.TestSettingsWifi)
Connect to a wifi network via the Settings app ... ERROR

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mike/.virtualenvs/gaia-ui/local/lib/python2.7/site-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/home/mike/projects/mozilla/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_wifi.py", line 26, in test_connect_to_wifi_via_settings_app
    wifi_settings.connect_to_network(self.testvars['wifi'])
TEST-UNEXPECTED-FAIL | test_settings_wifi.py test_settings_wifi.TestSettingsWifi.test_connect_to_wifi_via_settings_app | KeyError: 'wifi'
----------------------------------------------------------------------
Ran 1 test in 45.893s

FAILED (errors=1)
TEST-START test_settings_power_save_mode.py
test_power_save_mode (test_settings_power_save_mode.TestPowerSaveMode) ... ERROR



When run in isolation:



TEST-START test_settings_wifi.py
test_connect_to_wifi_via_settings_app (test_settings_wifi.TestSettingsWifi)
Connect to a wifi network via the Settings app ... ERROR

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mike/.virtualenvs/gaia-ui/local/lib/python2.7/site-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/home/mike/projects/mozilla/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_wifi.py", line 26, in test_connect_to_wifi_via_settings_app
    wifi_settings.connect_to_network(self.testvars['wifi'])
TEST-UNEXPECTED-FAIL | test_settings_wifi.py test_settings_wifi.TestSettingsWifi.test_connect_to_wifi_via_settings_app | KeyError: 'wifi'
----------------------------------------------------------------------
Ran 1 test in 45.843s

FAILED (errors=1)
Ignore Comment 8--I was not running the tests with the correct `--testvars`.

Unfortunately, my devices cannot find any wireless networks, even when manually navigating to the Settings application and enabling Wifi.
(In reply to Stephen Donner [:stephend] from comment #5)
> The above stack isn't what we're seeing on:
> 
>   <!-- Mercurial-Information: <project name="mozilla-central" path="gecko"
> remote="hgmozillaorg" revision="9f12a9fab080"/> -->
>   <project name="gecko.git" path="gecko" remote="mozillaorg"
> revision="a0bb585098cc89c454fac0297b5ef748d5cab82c"/>

Note: this is built on 2013-12-09-05-34-02.
I updated gaia and gecko to this version, and run gaia ui test by command:
gaiatest --address=localhost:2828 --testvars=testvars.json gaiatest/tests/unit/settings/test_wifi_settings.py --repeat=10 --restart

Nothing goes wrong.

The wifi settings in my testvars.json points to an AP with PSK encryption, while APN settings are left empty.
I don't think these settings affects the test result though.
(In reply to Chuck Lee [:chucklee] from comment #11)
> I updated gaia and gecko to this version, and run gaia ui test by command:
> gaiatest --address=localhost:2828 --testvars=testvars.json
> gaiatest/tests/unit/settings/test_wifi_settings.py --repeat=10 --restart
> 
> Nothing goes wrong.
> 
> The wifi settings in my testvars.json points to an AP with PSK encryption,
> while APN settings are left empty.
> I don't think these settings affects the test result though.

Are you on the Hamachi 1.2...11-05 base build (vendor version?)

Is there anything about our config or in our logcat that points to something we can try to address?
I can reproduce this bug and it is just because wifi took too much time to get disabled. 

Changing

https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/gaia_test.py#L323

to

result = self.marionette.execute_async_script("return GaiaDataLayer.disableWiFi()", special_powers=True, script_timeout=20000) 

solves this issue.
(In reply to Henry Chang [:henry] from comment #13)
> I can reproduce this bug and it is just because wifi took too much time to
> get disabled. 
> 
> Changing
> 
> https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/
> gaiatest/gaia_test.py#L323
> 
> to
> 
> result = self.marionette.execute_async_script("return
> GaiaDataLayer.disableWiFi()", special_powers=True, script_timeout=20000) 
> 
> solves this issue.

Would you mind please doing a GitHub pull request with those changes, so we can discuss/review/potentially merge?  Thanks!!
Flags: needinfo?(hchang)
Assignee: chulee → hchang
Flags: needinfo?(hchang)
https://github.com/mozilla-b2g/gaia/pull/14555

Are you able to try to reproduce this issue with this PR?
I personally couldn't reproduce it after applying this patch.

Thanks.
So, with Dave's switch to use the non-commercial-RIL builds in our Jenkins, as well as some tweaks such as sleep()'ing after "adb stop b2g", etc. we're no longer seeing this issue -- tests have been running fine all day today.  (See screenshot of Jenkins.)
Marking WFM, as, other than the Jenkins config-change (which /technically/ isn't a code commit), we don't have a corresponding Gaia/Gecko/Gonk commit, and it does seem to clearly have fixed the issue!

Thanks for your investigation and patience, all!
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: