806138 - [B2G] Marionette voicemail tests | test_mobile_networks.js | test_voicemail_statuschanged.py | sometimes fail

Status: RESOLVED WORKSFORME

(Core :: DOM: Device Interfaces, defect)

Description

[Jonathan Griffin (:jgriffin)]

Sometimes, the voicemail Marionette WebAPI tests fail:

test_mobile_networks.js fails with:

14:32:18     INFO -  FAIL: /home/cltbld/talos-slave/test/build/tests/marionette/tests/dom/network/tests/marionette/test_mobile_networks.js, runTest (marionette_test.MarionetteJSTestCase)
14:32:18     INFO -  ----------------------------------------------------------------------
14:32:18    ERROR -  Traceback (most recent call last):
14:32:18     INFO -    File "/home/cltbld/talos-slave/test/build/tests/marionette/marionette/marionette_test.py", line 211, in runTest
14:32:18     INFO -      '%d tests failed:\n%s' % (results['failed'], '\n'.join(fails)))
14:32:18     INFO -  AssertionError: 1 tests failed:
14:32:18     INFO -  TEST-UNEXPECTED-FAIL : TEST-UNEXPECTED-FAIL | got true, expected false | voiceNetwork changed while automatically selecting network? true
1

and test_voicemail_statuschanged.py:testStatusChanged fails with:

14:33:48     INFO - I/Gecko   (  385): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | undefined - false was false, expected true

Full log:  https://tbpl.mozilla.org/php/getParsedLog.php?id=16526063&tree=Cedar&full=1#error1

These errors are intermittent.

Comment 1

[Jonathan Griffin (:jgriffin)]

Disabled test_voicemail_statuschanged.py since it always fails, and we're trying to get a green set of WebAPI tests that we can turn on for m-c.  Note that there's a bug in Marionette (bug 807095) that prevents the testrunner from marking this as failed, even though you can see the test failures in the log, and TBPL detects the failure as well.

https://hg.mozilla.org/mozilla-central/rev/56d3db021482

Comment 2

[Jonathan Griffin (:jgriffin)]

Disabled test_voicemail_statuschanged.py on aurora as well:  https://hg.mozilla.org/releases/mozilla-aurora/rev/5b307b318efa

Comment 3

[Legacy TBPL/Treeherder Robot]

jgriffin
https://tbpl.mozilla.org/php/getParsedLog.php?id=16706812&tree=Cedar
b2g_ics_armv7a_gecko_emulator cedar opt test marionette-webapi on 2012-11-02 16:08:58
slave: talos-r3-fed-020

16:17:12     INFO -  TEST-UNEXPECTED-FAIL | test_mobile_networks.js | got true, expected false | voiceNetwork changed while automatically selecting network? true
16:17:47     INFO -  TEST-UNEXPECTED-FAIL | test_mobile_preferred_network_type.js | JavascriptException: undefined: undefined
16:18:55     INFO - I/Gecko   (  388): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | test_voicemail_statuschanged.py TestVoicemailStatusChanged.testStatusChanged | undefined - false was false, expected true
16:18:55     INFO - I/Gecko   (  388): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | test_voicemail_statuschanged.py TestVoicemailStatusChanged.testStatusChanged | undefined - false was false, expected true
16:18:55     INFO - I/Gecko   (  388): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | test_voicemail_statuschanged.py TestVoicemailStatusChanged.testStatusChanged | undefined - false was false, expected true
16:18:55     INFO - I/Gecko   (  388): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | test_voicemail_statuschanged.py TestVoicemailStatusChanged.testStatusChanged | undefined - false was false, expected true
16:18:55     INFO - I/Gecko   (  388): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | test_voicemail_statuschanged.py TestVoicemailStatusChanged.testStatusChanged | undefined - false was false, expected true
16:18:56     INFO - I/Gecko   (  388): MARIONETTE TEST RESULT:TEST-UNEXPECTED-FAIL | test_mobile_networks.js | voiceNetwork changed while automatically selecting network? true - got true, expected false

Comment 4

[Ed Morley [:emorley]]

Resolving WFM keyword:intermittent-failure bugs last modified >3 months ago, whose whiteboard contains none of:
{random,disabled,marked,fuzzy,todo,fails,failing,annotated,time-bomb,leave open}

There will inevitably be some false positives; for that (and the bugspam) I apologise. Filter on orangewfm.