Closed Bug 1170772 Opened 7 years ago Closed 7 years ago

Get password manager xpcshell tests running on Android again

Categories

(Toolkit :: Password Manager, defect)

defect
Not set
normal
Points:
2

Tracking

()

RESOLVED FIXED
mozilla41
Iteration:
41.2 - Jun 8
Tracking Status
firefox41 --- fixed

People

(Reporter: MattN, Assigned: MattN)

References

(Depends on 1 open bug)

Details

Attachments

(1 file)

For some reason `skip-if = toolkit == 'android'` was added to toolkit/components/passwordmgr/test/unit/xpcshell.ini in bug 1066735 so we don't have xpcshell test coverage on Android :( We should really fix this so we don't accidentally break things on Android with our changes.
Flags: qe-verify-
Flags: firefox-backlog+
I can fill in some history: 

When we first started running xpcshell tests on Android, only those tests that were specifically requested, or which happened to run without failure were run. At that time, Android xpcshell tests ignored the normal manifests and used https://hg.mozilla.org/mozilla-central/diff/3d1a89fbcaa1/testing/xpcshell/xpcshell_android.ini as a list of tests to run. 

In bug 1066735, xpcshell_android.ini was eliminated; xpcshell.ini files were updated with skip-if if those tests were not previously running via xpcshell_android.ini.
Yeah, I got that history from bug 1066735 but didn't go back further to understand if any of the tests in that directory ever ran on Android. It was definitely assumed that they were running since a test was set to only run on Android in 2014-01: https://hg.mozilla.org/mozilla-central/diff/a3f383647966/toolkit/components/passwordmgr/test/unit/xpcshell.ini

># Test SQLite database backup and migration, applicable to Android only.
>[test_storage_mozStorage.js]
>skip-if = os != "android"
test_storage_mozStorage.js and test_logins_decrypt_failure.js fail on Android:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4d1affc45dad

http://ftp.mozilla.org/pub/mozilla.org/mobile/try-builds/gbrown@mozilla.com-4d1affc45dad/try-android-api-11/try_ubuntu64_vm_armv7_mobile_test-xpcshell-3-bm122-tests1-linux64-build99.txt.gz

11:37:24     INFO -  TEST-START | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js
11:37:33  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | xpcshell return code: 0
11:37:33     INFO -  TEST-INFO took 8936ms
11:37:33     INFO -  >>>>>>>
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | xpcw: cd /sdcard/tests/xpcshell/toolkit/components/passwordmgr/test/unit
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | xpcw: xpcshell -r /sdcard/tests/xpcshell/c/httpd.manifest --greomni /data/local/xpcb/fennec-41.0a1.en-US.android-arm.apk -m -s -e const _HEAD_JS_PATH = "/sdcard/tests/xpcshell/head.js"; -e const _TESTING_MODULES_DIR = "/sdcard/tests/xpcshell/m"; -f /sdcard/tests/xpcshell/head.js -e const _SERVER_ADDR = "localhost" -e const _HEAD_FILES = ["/sdcard/tests/xpcshell/toolkit/components/passwordmgr/test/unit/head.js"]; -e const _TAIL_FILES = []; -e const _JSDEBUGGER_PORT = 0; -e const _TEST_FILE = ["test_storage_mozStorage.js"]; -e const _TEST_NAME = "toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js" -e _execute_test(); quit(0);
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | JavaScript strict warning: test_storage_mozStorage.js, line 76: SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | JavaScript strict warning: test_storage_mozStorage.js, line 84: SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function
11:37:33     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
11:37:33     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
11:37:33     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
11:37:33     INFO -  running event loop
11:37:33     INFO -  "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function" {file: "test_storage_mozStorage.js" line: 76 column: 9 source: "function getGUIDforID(conn, id) {
11:37:33     INFO -  "}]"
11:37:33     INFO -  "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function" {file: "test_storage_mozStorage.js" line: 84 column: 9 source: "function getEncTypeForID(conn, id) {
11:37:33     INFO -  "}]"
11:37:33     INFO -  toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | Starting test_common_initialize
11:37:33     INFO -  (xpcshell/head.js) | test test_common_initialize pending (2)
11:37:33     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
11:37:33     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
11:37:33     INFO -  (xpcshell/head.js) | test test_common_initialize finished (2)
11:37:33     INFO -  toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | Starting test_execute
11:37:33     INFO -  (xpcshell/head.js) | test test_execute pending (2)
11:37:33     INFO -  TEST-PASS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | test_execute - [test_execute : 99] true == true
11:37:33     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
11:37:33     INFO -  TEST-PASS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | test_execute - [test_execute : 128] 999 == 999
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | JavaScript error: jar:jar:file:///data/local/xpcb/fennec-41.0a1.en-US.android-arm.apk!/assets/omni.ja!/components/crypto-SDR.js, line 165: NS_ERROR_FAILURE: Couldn't decrypt string
11:37:33  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | test_execute - [test_execute : 70] 1 == 0
11:37:33     INFO -      resource://testing-common/LoginTestUtils.jsm:assertLoginListsEqual:70
11:37:33     INFO -      test_storage_mozStorage.js:checkStorageData:63
11:37:33     INFO -      test_storage_mozStorage.js:test_execute:132
11:37:33     INFO -      self-hosted:next:674
11:37:33     INFO -      _run_next_test@/sdcard/tests/xpcshell/head.js:1440:9
11:37:33     INFO -      do_execute_soon/<.run@/sdcard/tests/xpcshell/head.js:653:9
11:37:33     INFO -      _do_main@/sdcard/tests/xpcshell/head.js:207:5
11:37:33     INFO -      _execute_test@/sdcard/tests/xpcshell/head.js:513:5
11:37:33     INFO -      @-e:1:1
11:37:33     INFO -  exiting test
11:37:33     INFO -  Unexpected exception Error: FAILED in test #1 -- Test downgrade from v999 storage: 2147500036 at test_storage_mozStorage.js:427
11:37:33     INFO -  test_execute@test_storage_mozStorage.js:427:11
11:37:33     INFO -  _run_next_test@/sdcard/tests/xpcshell/head.js:1440:9
11:37:33     INFO -  do_execute_soon/<.run@/sdcard/tests/xpcshell/head.js:653:9
11:37:33     INFO -  _do_main@/sdcard/tests/xpcshell/head.js:207:5
11:37:33     INFO -  _execute_test@/sdcard/tests/xpcshell/head.js:513:5
11:37:33     INFO -  @-e:1:1
11:37:33     INFO -  exiting test
11:37:33     INFO -  "CONSOLE_MESSAGE: (error) [JavaScript Error: "NS_ERROR_FAILURE: Couldn't decrypt string" {file: "jar:jar:file:///data/local/xpcb/fennec-41.0a1.en-US.android-arm.apk!/assets/omni.ja!/components/crypto-SDR.js" line: 165}]"
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | JavaScript error: /sdcard/tests/xpcshell/head.js, line 484: TypeError: _testLogger is undefined
11:37:33     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_storage_mozStorage.js | JavaScript error: /sdcard/tests/xpcshell/head.js, line 130: TypeError: do_print is not a function
11:37:33     INFO -  <<<<<<<


11:39:24     INFO -  TEST-START | toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js
11:39:36  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | xpcshell return code: 0
11:39:36     INFO -  TEST-INFO took 12289ms
11:39:36     INFO -  >>>>>>>
11:39:36     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | xpcw: cd /sdcard/tests/xpcshell/toolkit/components/passwordmgr/test/unit
11:39:36     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | xpcw: xpcshell -r /sdcard/tests/xpcshell/c/httpd.manifest --greomni /data/local/xpcb/fennec-41.0a1.en-US.android-arm.apk -m -s -e const _HEAD_JS_PATH = "/sdcard/tests/xpcshell/head.js"; -e const _TESTING_MODULES_DIR = "/sdcard/tests/xpcshell/m"; -f /sdcard/tests/xpcshell/head.js -e const _SERVER_ADDR = "localhost" -e const _HEAD_FILES = ["/sdcard/tests/xpcshell/toolkit/components/passwordmgr/test/unit/head.js"]; -e const _TAIL_FILES = []; -e const _JSDEBUGGER_PORT = 0; -e const _TEST_FILE = ["test_logins_decrypt_failure.js"]; -e const _TEST_NAME = "toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js" -e _execute_test(); quit(0);
11:39:36     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
11:39:36     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
11:39:36     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
11:39:36     INFO -  running event loop
11:39:36     INFO -  toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | Starting test_common_initialize
11:39:36     INFO -  (xpcshell/head.js) | test test_common_initialize pending (2)
11:39:36     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
11:39:36     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
11:39:36     INFO -  (xpcshell/head.js) | test test_common_initialize finished (2)
11:39:36     INFO -  toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | Starting test_logins_decrypt_failure
11:39:36     INFO -  (xpcshell/head.js) | test test_logins_decrypt_failure pending (2)
11:39:36     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
11:39:36     INFO -  Unexpected exception NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIPK11Token.changePassword]
11:39:36     INFO -  resetMasterPassword@test_logins_decrypt_failure.js:24:3
11:39:36     INFO -  test_logins_decrypt_failure@test_logins_decrypt_failure.js:41:3
11:39:36     INFO -  createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:239:18
11:39:36     INFO -  Task_spawn@resource://gre/modules/Task.jsm:164:12
11:39:36     INFO -  _run_next_test@/sdcard/tests/xpcshell/head.js:1440:9
11:39:36     INFO -  do_execute_soon/<.run@/sdcard/tests/xpcshell/head.js:653:9
11:39:36     INFO -  _do_main@/sdcard/tests/xpcshell/head.js:207:5
11:39:36     INFO -  _execute_test@/sdcard/tests/xpcshell/head.js:513:5
11:39:36     INFO -  @-e:1:1
11:39:36     INFO -  exiting test
11:39:36     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | JavaScript error: /sdcard/tests/xpcshell/head.js, line 484: TypeError: _testLogger is undefined
11:39:36     INFO -  PROCESS | toolkit/components/passwordmgr/test/unit/test_logins_decrypt_failure.js | JavaScript error: /sdcard/tests/xpcshell/head.js, line 130: TypeError: do_print is not a function
11:39:36     INFO -  <<<<<<<
Thanks for looking into this Geoff.

I think we should skip just those 2 tests for now and file a new bug to fix those 2 when we have time. I'll throw together a patch and file the new bug.
Assignee: nobody → MattN+bmo
Status: NEW → ASSIGNED
Iteration: --- → 41.2 - Jun 8
Points: --- → 2
Depends on: 1171687
Bug 1170772 - Enable most password manager xpcshell tests on Android. r=gbrown
Attachment #8615575 - Flags: review?(gbrown)
Comment on attachment 8615575 [details]
MozReview Request: Bug 1170772 - Enable most password manager xpcshell tests on Android. r=gbrown

https://reviewboard.mozilla.org/r/10295/#review9063

Ship It!
Attachment #8615575 - Flags: review?(gbrown) → review+
https://hg.mozilla.org/mozilla-central/rev/fe9544eb533a
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla41
You need to log in before you can comment on or make changes to this bug.