Closed Bug 838236 Opened 12 years ago Closed 9 years ago

Most recent b2g panda Gaia UI test failures

Categories

(Testing :: General, defect)

Product:
Testing
Component:
General
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: armenzg, Unassigned)

References

Details

https://tbpl.mozilla.org/?tree=Cedar&jobname=b2g_&rev=ff4b5bb4f0df 07:42:34 INFO - TEST-UNEXPECTED-FAIL | test_edit_contact.py TestContacts.test_edit_contact | AssertionError: Unable to insert contact {'comment': 'Gaia automated test', 'city': 'London', 'tel': {'type': 'Mobile', 'value': '55554110454'}, 'name': 'gaia110454 test', 'zip': '90210', 'country': 'UK', 'familyName': 'test', 'street': '101 Testing street', 'givenName': 'gaia110454', 'email': 'gaia110454@restmail.net'} 07:46:15 INFO - TEST-UNEXPECTED-FAIL | test_cards_view_with_three_apps.py TestCardsViewThreeApps.test_cards_view | AssertionError: First opened app should not be visible in cards view 07:47:09 INFO - TEST-UNEXPECTED-FAIL | test_everythingme.py TestEverythingMe.test_launch_everything_me_app | TimeoutException: Element //div/b[text()='Facebook'] not visible before timeout 07:48:27 ERROR - Return code: 10
All tests are failing again: https://tbpl.mozilla.org/php/getParsedLog.php?id=19496624&tree=Cedar&full=1 All of them fail with this: > TypeError: object of type 'bool' has no len() This already started happening with rail's merge: https://hg.mozilla.org/projects/cedar/rev/41fa159f8bb4
(In reply to Armen Zambrano G. [:armenzg] from comment #1) > All tests are failing again: > https://tbpl.mozilla.org/php/getParsedLog.php?id=19496624&tree=Cedar&full=1 > > All of them fail with this: > > TypeError: object of type 'bool' has no len() > > This already started happening with rail's merge: > https://hg.mozilla.org/projects/cedar/rev/41fa159f8bb4 See https://github.com/mozilla/gaia-ui-tests/pull/372
(In reply to Jonathan Griffin (:jgriffin) from comment #2) > (In reply to Armen Zambrano G. [:armenzg] from comment #1) > > All tests are failing again: > > https://tbpl.mozilla.org/php/getParsedLog.php?id=19496624&tree=Cedar&full=1 > > > > All of them fail with this: > > > TypeError: object of type 'bool' has no len() > > > > This already started happening with rail's merge: > > https://hg.mozilla.org/projects/cedar/rev/41fa159f8bb4 > > See https://github.com/mozilla/gaia-ui-tests/pull/372 This is now fixed. The current errors are: 14:41:35 INFO - TEST-UNEXPECTED-FAIL | test_edit_contact.py TestContacts.test_edit_contact | AssertionError: Unable to insert contact {'comment': 'Gaia automated test', 'city': 'London', 'tel': {'type': 'Mobile', 'value': '55595380149'}, 'name': 'gaia380149 test', 'zip': '90210', 'country': 'UK', 'familyName': 'test', 'street': '101 Testing street', 'givenName': 'gaia380149', 'email': 'gaia380149@restmail.net'} 14:41:45 INFO - TEST-UNEXPECTED-FAIL | test_clock_all_items_present_new_alarm.py TestClockTestAllItemsPresentNewAlarm.test_all_items_present_new_alarm | NoSuchElementException: Unable to locate element: picker-container 14:45:00 INFO - TEST-UNEXPECTED-FAIL | test_cards_view_with_three_apps.py TestCardsViewThreeApps.test_cards_view | AssertionError: First opened app should not be visible in cards view 14:47:26 INFO - TEST-UNEXPECTED-FAIL | test_everythingme.py TestEverythingMe.test_launch_everything_me_app | TimeoutException: Element //div/b[text()='Facebook'] not visible before timeout 14:48:42 ERROR - Return code: 10 https://tbpl.mozilla.org/php/getParsedLog.php?id=19508280&tree=Mozilla-Inbound&full=1
Is anyone going to be able to look at this during this week? (In reply to Jonathan Griffin (:jgriffin) from comment #3) > (In reply to Jonathan Griffin (:jgriffin) from comment #2) > > (In reply to Armen Zambrano G. [:armenzg] from comment #1) > > > All tests are failing again: > > > https://tbpl.mozilla.org/php/getParsedLog.php?id=19496624&tree=Cedar&full=1 > > > > > > All of them fail with this: > > > > TypeError: object of type 'bool' has no len() > > > > > > This already started happening with rail's merge: > > > https://hg.mozilla.org/projects/cedar/rev/41fa159f8bb4 > > > > See https://github.com/mozilla/gaia-ui-tests/pull/372 > > This is now fixed. The current errors are: > > 14:41:35 INFO - TEST-UNEXPECTED-FAIL | test_edit_contact.py > TestContacts.test_edit_contact | AssertionError: Unable to insert contact > {'comment': 'Gaia automated test', 'city': 'London', 'tel': {'type': > 'Mobile', 'value': '55595380149'}, 'name': 'gaia380149 test', 'zip': > '90210', 'country': 'UK', 'familyName': 'test', 'street': '101 Testing > street', 'givenName': 'gaia380149', 'email': 'gaia380149@restmail.net'} > 14:41:45 INFO - TEST-UNEXPECTED-FAIL | > test_clock_all_items_present_new_alarm.py > TestClockTestAllItemsPresentNewAlarm.test_all_items_present_new_alarm | > NoSuchElementException: Unable to locate element: picker-container > 14:45:00 INFO - TEST-UNEXPECTED-FAIL | > test_cards_view_with_three_apps.py TestCardsViewThreeApps.test_cards_view | > AssertionError: First opened app should not be visible in cards view > 14:47:26 INFO - TEST-UNEXPECTED-FAIL | test_everythingme.py > TestEverythingMe.test_launch_everything_me_app | TimeoutException: Element > //div/b[text()='Facebook'] not visible before timeout > 14:48:42 ERROR - Return code: 10 > > https://tbpl.mozilla.org/php/getParsedLog.php?id=19508280&tree=Mozilla- > Inbound&full=1 This is still the same set of oranges as comment 3. https://tbpl.mozilla.org/php/getParsedLog.php?id=19674036&tree=Mozilla-Inbound&full=1
I should have the Gaia-Master tree unhidden tomorrow. Can we disable those 4 tests so we can unhide the Gaia UI tests on all trees?
We can't unhide them until we have a realistic sheriffing story, which we currently don't. Let's see how helpful the gaia-master work is in being able to identify the cause of future regressions; if that seems to work well, we can consider what we have to do to unhide the tests.
I've filed bug 843906 for the problem in test_edit_contacts.py.
The symbols issue got fixed yesterday (after a reconfiguration of our masters) but we had to wait for some mozpool issues that were happening to clear up. The most recent failures are these: https://tbpl-dev.allizom.org/php/getParsedLog.php?id=20559885&tree=Gaia-Master&full=1 I also see some logcat errors: 04:29:09 INFO - 01-16 18:49:21.430 E/GeckoConsole( 4346): Content JS ERROR at chrome://marionette/content/marionette-listener.js:1482 in GaiaDataLayer.getAllContacts/req.onerror: error finding contacts Not Allowed 04:29:09 INFO - 01-16 18:49:21.571 E/GeckoConsole( 4346): Content JS ERROR at chrome://marionette/content/marionette-listener.js:1482 in GaiaDataLayer.getAllContacts/req.onerror: error finding contacts Not Allowed 04:29:09 INFO - 01-16 18:49:21.571 E/GeckoConsole( 4346): Content JS LOG at chrome://marionette/content/marionette-listener.js:1503 in GaiaDataLayer.removeAllContacts/<: no contacts to remove 04:29:09 INFO - 01-16 18:49:27.071 E/GeckoConsole( 4346): [JavaScript Error: "[Exception... "Component returned failure code: 0x80470002 (NS_BASE_STREAM_CLOSED) [nsIOutputStream.write]" nsresult: "0x80470002 (NS_BASE_STREAM_CLOSED)" location: "JS frame :: chrome://global/content/devtools/dbg-transport.js :: DT_onOutputStreamReady :: line 94" data: no]" {file: "chrome://global/content/devtools/dbg-transport.js" line: 94}] and few more. Summary of test failures: ############################ 04:21:11 INFO - TEST-UNEXPECTED-FAIL | test_browser_lan.py TestBrowserLAN.test_browser_lan | TimeoutException: Element page-title not found before timeout 04:21:11 INFO - TEST-UNEXPECTED-FAIL | test_browser_lan.py TestBrowserLAN.test_browser_lan | JavascriptException: TypeError: window.navigator.mozSettings is null 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_settings_bluetooth.py TestBluetoothSettings.test_toggle_bluetooth_settings | InvalidResponseException: Could not successfully complete transport of message to Gecko, socket closed? 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_everythingme.py TestEverythingMe.test_launch_everything_me_app | InvalidResponseException: Could not successfully complete transport of message to Gecko, socket closed? 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_delete_app.py TestDeleteApp.test_delete_app | error: [Errno 32] Broken pipe 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_keyboard.py TestKeyboard.test_keyboard_basic | error: [Errno 32] Broken pipe 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_launch_app.py TestLaunchApp.test_launch_app | error: [Errno 32] Broken pipe Test failures: ################ 04:20:09 INFO - TEST-START test_browser_lan.py 04:21:10 INFO - test_browser_lan (test_browser_lan.TestBrowserLAN) ... ERROR 04:21:11 INFO - ERROR 04:21:11 INFO - ====================================================================== 04:21:11 INFO - ERROR: test_browser_lan (test_browser_lan.TestBrowserLAN) 04:21:11 INFO - ---------------------------------------------------------------------- 04:21:11 INFO - Traceback (most recent call last): 04:21:11 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/browser/test_browser_lan.py", line 46, in test_browser_lan 04:21:11 INFO - self.wait_for_element_present(*self._page_title_locator) 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 388, in wait_for_element_present 04:21:11 INFO - 'Element %s not found before timeout' % locator) 04:21:11 INFO - TEST-UNEXPECTED-FAIL | test_browser_lan.py TestBrowserLAN.test_browser_lan | TimeoutException: Element page-title not found before timeout 04:21:11 INFO - ====================================================================== 04:21:11 INFO - ERROR: test_browser_lan (test_browser_lan.TestBrowserLAN) 04:21:11 INFO - ---------------------------------------------------------------------- 04:21:11 INFO - Traceback (most recent call last): 04:21:11 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/browser/test_browser_lan.py", line 53, in tearDown 04:21:11 INFO - GaiaTestCase.tearDown(self) 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 481, in tearDown 04:21:11 INFO - f.write(json.dumps(self.data_layer.all_settings)) 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 155, in all_settings 04:21:11 INFO - return self.get_setting('*') 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 151, in get_setting 04:21:11 INFO - return self.marionette.execute_async_script('return GaiaDataLayer.getSetting("%s")' % name, special_powers=True) 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 539, in execute_async_script 04:21:11 INFO - scriptTimeout=script_timeout) 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 284, in _send_message 04:21:11 INFO - self._handle_error(response) 04:21:11 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 319, in _handle_error 04:21:11 INFO - raise JavascriptException(message=message, status=status, stacktrace=stacktrace) 04:21:11 INFO - TEST-UNEXPECTED-FAIL | test_browser_lan.py TestBrowserLAN.test_browser_lan | JavascriptException: TypeError: window.navigator.mozSettings is null 04:21:11 INFO - ---------------------------------------------------------------------- 04:21:11 INFO - Ran 1 test in 62.143s 04:21:11 INFO - FAILED (errors=2) 04:28:02 INFO - TEST-START test_settings_bluetooth.py 04:28:02 INFO - test_toggle_bluetooth_settings (test_settings_bluetooth.TestBluetoothSettings) 04:28:33 INFO - Toggle Bluetooth via Settings - Networks & Connectivity ... ERROR 04:28:33 INFO - ====================================================================== 04:28:33 INFO - ERROR: test_toggle_bluetooth_settings (test_settings_bluetooth.TestBluetoothSettings) 04:28:33 INFO - Toggle Bluetooth via Settings - Networks & Connectivity 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Traceback (most recent call last): 04:28:33 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/settings/test_settings_bluetooth.py", line 46, in tearDown 04:28:33 INFO - self.data_layer.set_setting('bluetooth.enabled', False) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 160, in set_setting 04:28:33 INFO - result = self.marionette.execute_async_script('return GaiaDataLayer.setSetting("%s", %s)' % (name, value), special_powers=True) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 539, in execute_async_script 04:28:33 INFO - scriptTimeout=script_timeout) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 269, in _send_message 04:28:33 INFO - response = self.client.send(message) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 95, in send 04:28:33 INFO - response = self.receive() 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 58, in receive 04:28:33 INFO - status=ErrorCodes.INVALID_RESPONSE) 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_settings_bluetooth.py TestBluetoothSettings.test_toggle_bluetooth_settings | InvalidResponseException: Could not successfully complete transport of message to Gecko, socket closed? 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Ran 1 test in 30.867s 04:28:33 INFO - FAILED (errors=1) 04:28:33 INFO - TEST-START test_everythingme.py 04:28:33 INFO - test_launch_everything_me_app (test_everythingme.TestEverythingMe) ... ERROR 04:28:33 INFO - ====================================================================== 04:28:33 INFO - ERROR: test_launch_everything_me_app (test_everythingme.TestEverythingMe) 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Traceback (most recent call last): 04:28:33 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/everythingme/test_everythingme.py", line 23, in setUp 04:28:33 INFO - GaiaTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 317, in setUp 04:28:33 INFO - MarionetteTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 131, in setUp 04:28:33 INFO - self.marionette.test_name = self.test_name 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 378, in test_name 04:28:33 INFO - if self._send_message('setTestName', 'ok', value=test_name): 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 269, in _send_message 04:28:33 INFO - response = self.client.send(message) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 95, in send 04:28:33 INFO - response = self.receive() 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 58, in receive 04:28:33 INFO - status=ErrorCodes.INVALID_RESPONSE) 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_everythingme.py TestEverythingMe.test_launch_everything_me_app | InvalidResponseException: Could not successfully complete transport of message to Gecko, socket closed? 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Ran 1 test in 0.001s 04:28:33 INFO - FAILED (errors=1) 04:28:33 INFO - TEST-START test_delete_app.py 04:28:33 INFO - test_delete_app (test_delete_app.TestDeleteApp) ... ERROR 04:28:33 INFO - ====================================================================== 04:28:33 INFO - ERROR: test_delete_app (test_delete_app.TestDeleteApp) 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Traceback (most recent call last): 04:28:33 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/test_delete_app.py", line 25, in setUp 04:28:33 INFO - GaiaTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 317, in setUp 04:28:33 INFO - MarionetteTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 131, in setUp 04:28:33 INFO - self.marionette.test_name = self.test_name 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 378, in test_name 04:28:33 INFO - if self._send_message('setTestName', 'ok', value=test_name): 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 269, in _send_message 04:28:33 INFO - response = self.client.send(message) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 93, in send 04:28:33 INFO - self.sock.send(packet) 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_delete_app.py TestDeleteApp.test_delete_app | error: [Errno 32] Broken pipe 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Ran 1 test in 0.001s 04:28:33 INFO - FAILED (errors=1) 04:28:33 INFO - TEST-START test_keyboard.py 04:28:33 INFO - test_keyboard_basic (test_keyboard.TestKeyboard) ... ERROR 04:28:33 INFO - ====================================================================== 04:28:33 INFO - ERROR: test_keyboard_basic (test_keyboard.TestKeyboard) 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Traceback (most recent call last): 04:28:33 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/test_keyboard.py", line 26, in setUp 04:28:33 INFO - GaiaTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 317, in setUp 04:28:33 INFO - MarionetteTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 131, in setUp 04:28:33 INFO - self.marionette.test_name = self.test_name 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 378, in test_name 04:28:33 INFO - if self._send_message('setTestName', 'ok', value=test_name): 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 269, in _send_message 04:28:33 INFO - response = self.client.send(message) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 93, in send 04:28:33 INFO - self.sock.send(packet) 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_keyboard.py TestKeyboard.test_keyboard_basic | error: [Errno 32] Broken pipe 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Ran 1 test in 0.000s 04:28:33 INFO - FAILED (errors=1) 04:28:33 INFO - TEST-START test_launch_app.py 04:28:33 INFO - test_launch_app (test_launch_app.TestLaunchApp) ... ERROR 04:28:33 INFO - ====================================================================== 04:28:33 INFO - ERROR: test_launch_app (test_launch_app.TestLaunchApp) 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Traceback (most recent call last): 04:28:33 INFO - File "/builds/panda-0094/test/build/gaia-ui-tests/gaiatest/tests/test_launch_app.py", line 16, in setUp 04:28:33 INFO - GaiaTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 317, in setUp 04:28:33 INFO - MarionetteTestCase.setUp(self) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 131, in setUp 04:28:33 INFO - self.marionette.test_name = self.test_name 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 378, in test_name 04:28:33 INFO - if self._send_message('setTestName', 'ok', value=test_name): 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/marionette.py", line 269, in _send_message 04:28:33 INFO - response = self.client.send(message) 04:28:33 INFO - File "/builds/panda-0094/test/build/venv/lib/python2.7/site-packages/marionette/client.py", line 93, in send 04:28:33 INFO - self.sock.send(packet) 04:28:33 INFO - TEST-UNEXPECTED-FAIL | test_launch_app.py TestLaunchApp.test_launch_app | error: [Errno 32] Broken pipe 04:28:33 INFO - ---------------------------------------------------------------------- 04:28:33 INFO - Ran 1 test in 0.000s 04:28:33 INFO - FAILED (errors=1)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.