1184637 - xpcshell tests shouldn't be able to call do_test_finished more than do_test_pending was called

Status: NEW

(Testing :: XPCShell Harness, defect)

Description

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

At the moment, running './mach xpcshell-test security/manager/ssl/tests/unit/test_sss_eviction.js' results in do_test_pending being called twice and do_test_finished about 11 times. This is a bug. The test harness should have caught this.

For reference, here's the output I'm seeing:

From _tests: Kept 40111 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:01.08 LOG: MainThread INFO Running tests sequentially.
 0:01.08 SUITE_START: MainThread 1
 0:01.08 TEST_START: Thread-1 security/manager/ssl/tests/unit/test_sss_eviction.js
 0:01.09 LOG: Thread-1 INFO security/manager/ssl/tests/unit/test_sss_eviction.js | full command: ['/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin/xpcshell', '-g', '/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin', '-a', '/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin', '-r', '/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin/components/httpd.manifest', '-m', '-s', '-e', 'const _HEAD_JS_PATH = "/home/keeler/mozilla-central/testing/xpcshell/head.js";', '-e', 'const _TESTING_MODULES_DIR = "/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/modules/";', '-f', '/home/keeler/mozilla-central/testing/xpcshell/head.js', '-p', '/tmp/xpc-plugins-gZFKYo', '-e', 'const _SERVER_ADDR = "localhost"', '-e', u'const _HEAD_FILES = ["/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/xpcshell/security/manager/ssl/tests/unit/head_psm.js"];', '-e', 'const _TAIL_FILES = [];', '-e', 'const _JSDEBUGGER_PORT = 0;', '-e', u'const _TEST_FILE = ["/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/xpcshell/security/manager/ssl/tests/unit/test_sss_eviction.js"];', '-e', u'const _TEST_NAME = "security/manager/ssl/tests/unit/test_sss_eviction.js"', '-e', '_execute_test(); quit(0);']
 0:01.09 LOG: Thread-1 INFO security/manager/ssl/tests/unit/test_sss_eviction.js | current directory: u'/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/xpcshell/security/manager/ssl/tests/unit'
 0:01.09 LOG: Thread-1 INFO security/manager/ssl/tests/unit/test_sss_eviction.js | environment: ['XPCSHELL_TEST_PROFILE_DIR=/tmp/xpc-profile-vV8Zn4', 'XPCOM_DEBUG_BREAK=stack-and-abort', 'MOZ_CRASHREPORTER=1', 'XPCSHELL_TEST_TEMP_DIR=/tmp/xpc-other-Iq6FSC', 'LD_LIBRARY_PATH=/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin::/home/keeler/local/lib', 'MOZ_DISABLE_NONLOCAL_CONNECTIONS=1', 'MOZ_CRASHREPORTER_NO_REPORT=1']
 0:01.12 PROCESS_OUTPUT: Thread-1 (pid:1795) Full command: ['/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin/xpcshell', '-g', '/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin', '-a', '/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin', '-r', '/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/dist/bin/components/httpd.manifest', '-m', '-s', '-e', 'const _HEAD_JS_PATH = "/home/keeler/mozilla-central/testing/xpcshell/head.js";', '-e', 'const _TESTING_MODULES_DIR = "/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/modules/";', '-f', '/home/keeler/mozilla-central/testing/xpcshell/head.js', '-p', '/tmp/xpc-plugins-gZFKYo', '-e', 'const _SERVER_ADDR = "localhost"', '-e', u'const _HEAD_FILES = ["/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/xpcshell/security/manager/ssl/tests/unit/head_psm.js"];', '-e', 'const _TAIL_FILES = [];', '-e', 'const _JSDEBUGGER_PORT = 0;', '-e', u'const _TEST_FILE = ["/home/keeler/mozilla-central/obj-x86_64-unknown-linux-gnu/_tests/xpcshell/security/manager/ssl/tests/unit/test_sss_eviction.js"];', '-e', u'const _TEST_NAME = "security/manager/ssl/tests/unit/test_sss_eviction.js"', '-e', '_execute_test(); quit(0);']
(pid:1795) "[1795] WARNING: Re-registering a CID?: file /home/keeler/mozilla-central/xpcom/components/nsComponentManager.cpp, line 551" 
 0:01.54 LOG: Thread-1 INFO (xpcshell/head.js) | test MAIN run_test pending (1)
 0:01.55 TEST_STATUS: Thread-1 run_test PASS [run_test : 65] true == true
 0:01.55 LOG: Thread-1 INFO (xpcshell/head.js) | test pending (2)
 0:01.56 TEST_STATUS: Thread-1 run_test PASS [run_test : 75] true == true
 0:01.56 LOG: Thread-1 INFO (xpcshell/head.js) | test MAIN run_test finished (2)
 0:01.56 LOG: Thread-1 INFO running event loop
 0:01.57 TEST_STATUS: Thread-1 do_state_read PASS [do_state_read : 43] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:01.57 TEST_STATUS: Thread-1 do_state_read PASS [do_state_read : 45] true == true
 0:02.39 LOG: Thread-1 INFO (xpcshell/head.js) | test pending (2)
 0:02.39 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (2)
 0:02.40 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.41 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.42 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.42 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (1)
 0:02.42 LOG: Thread-1 INFO exiting test
 0:02.42 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.43 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.44 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.44 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (0)
 0:02.44 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.45 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.46 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.46 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-1)
 0:02.46 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.46 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.47 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.47 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-2)
 0:02.48 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.48 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.49 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.49 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-3)
 0:02.49 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.50 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.50 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.51 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-4)
 0:02.51 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.51 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.52 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.52 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-5)
 0:02.52 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.53 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.53 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.53 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-6)
 0:02.54 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.54 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.55 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.55 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-7)
 0:02.56 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 12] "SiteSecurityServiceState.txt" == "SiteSecurityServiceState.txt"
 0:02.56 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 16] true == true
 0:02.57 TEST_STATUS: Thread-1 do_state_written PASS [do_state_written : 38] true == true
 0:02.57 LOG: Thread-1 INFO (xpcshell/head.js) | test finished (-8)
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) "[1795] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /home/keeler/mozilla-central/xpcom/base/nsTraceRefcnt.cpp, line 147"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) "[1795] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /home/keeler/mozilla-central/xpcom/base/nsTraceRefcnt.cpp, line 147"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) "nsStringStats"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => mAllocCount:          12604"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => mReallocCount:          586"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => mFreeCount:           12604"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => mShareCount:        1018216"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => mAdoptCount:            251"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => mAdoptFreeCount:        251"
 0:02.72 PROCESS_OUTPUT: Thread-1 (pid:1795) " => Process ID: 1795, Thread ID: 140582066903552"
 0:02.72 TEST_END: Thread-1 Harness PASS. Subtests passed 34/34. Unexpected 0
 0:02.72 LOG: MainThread INFO INFO | Result summary:
 0:02.72 LOG: MainThread INFO INFO | Passed: 1
 0:02.72 LOG: MainThread INFO INFO | Failed: 0
 0:02.72 LOG: MainThread INFO INFO | Todo: 0
 0:02.72 LOG: MainThread INFO INFO | Retried: 0
 0:02.72 SUITE_END: MainThread 
Summary
=======

Ran 35 tests (1 parents, 34 subtests)
Expected results: 35
Unexpected results: 0

OK