547818 - xpcshell-tests and check-one passes despite error (syntax error, exception, throw etc.) in test

Status: RESOLVED WONTFIX

(Testing :: XPCShell Harness, defect)

Description

[Ben Bucksch (:BenB)]

check-one passes the test, but check-interactive fails it.

$ make -s -C mailnews/imap/test/ SOLO_FILE="test_fakeserver_imap_acl.js" check-one
TEST-PASS | .../bin/mozilla/_tests/xpcshell/test_imap/unit/test_fakeserver_imap_acl.js | test passed
INFO | Result summary:
INFO | Passed: 1
INFO | Failed: 0
$ make -s -C mailnews/imap/test/ SOLO_FILE="test_fakeserver_imap_acl.js" check-interactive
To start the test, type |_execute_test();|.
js> _execute_test();
TEST-INFO | (xpcshell/head.js) | test 1 pending
[object Object]TEST-UNEXPECTED-FAIL | (xpcshell/head.js) | TypeError: foo.bar is not a function
before 1118208, after 1101824, break 09fe1000
js> ^C
$ make -s -C mailnews/imap/test/ SOLO_FILE="test_fakeserver_imap_acl.js" check-one
TEST-PASS | .../bin/mozilla/_tests/xpcshell/test_imap/unit/test_fakeserver_imap_acl.js | test passed
INFO | Result summary:
INFO | Passed: 1
INFO | Failed: 0

Comment 1

[Ben Bucksch (:BenB)]

make -s -C mailnews/imap/test/ xpcshell-tests also passes.
Same result after doing make tier_app.

Comment 2

[(not currently active) Ted Mielczarek]

Probably could be duped to bug 546756, although that's only fixed in the electrolysis repo. If that patch fixes your problem, we should get it landed in m-c.

Comment 3

[Ben Bucksch (:BenB)]

Same problem with
function run_test()
{
  throw "somebody home?";
}

Comment 4

[Ben Bucksch (:BenB)]

re comment 2:
That patch just does:
-              (stdout and re.search("^((parent|child): )?TEST-UNEXPECTED-FAIL", stdout, re.MULTILINE)):
+              (stdout and re.search("^((parent|child): )?TEST-UNEXPECTED-FAIL", stdout, re.MULTILINE)) or \
+              (stdout and re.search(": SyntaxError:", stdout, re.MULTILINE)):
             print

I better don't comment on that, but it's not going to fix this bug. All exceptions must be catched.
If you need to pass this via stdout (urgs), then whatever calls run_test() must do:
try {
  run_test()
} catch (e) { dump("TEST-FAIL-WITH-EXCEPTION: " + e); printStack(e); }
and then you can parse for "TEST-FAIL-WITH-EXCEPTION" in the above code, if you must.

Comment 5

[Geoff Brown [:gbrown]]

Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.