937370 - Errors loading xpcshell test file may be silently discarded.

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Mark Hammond [:markh] [:mhammond]]

Attachment: Catch and report all exceptions loading a test file.

If a test file fails to load due to an error other than a SyntaxError, that error will not be reported in the xpcshell logs.

STR:

* Verify that running modules/libpref/test/unit/test_warnings.js works as expected.

* Apply the following patch, which will cause the file to fail to load (as there is a 'return' statement in a generator function)

diff --git a/modules/libpref/test/unit/test_warnings.js b/modules/libpref/test/unit/test_warnings.js
index 9c0eed9..670b0c8 100644
--- a/modules/libpref/test/unit/test_warnings.js
+++ b/modules/libpref/test/unit/test_warnings.js
@@ -86,4 +86,5 @@ add_task(function() {
   buf = makeBuffer(32 * 1024);
   warned = yield checkWarning("string.warn", buf);
   do_check_true(warned);
+  return true;
 });

* Rerun the test.

Actual:
-------
Test output is:

 0:02.68 TEST-UNEXPECTED-FAIL | o:\src\mozilla-git\central\obj-release\_tests\xpcshell\modules\libpref\test\unit\test_warnings.js | test failed (with xpcshell return code: 3)
 0:02.68 >>>>>>>
 0:02.68 <<<<<<<

But note that no error detail is reported - the developer needs to deduce what the problem might be:

Expected:
---------
Test output is:

 0:02.67 TEST-UNEXPECTED-FAIL | (xpcshell/head.js) | Source file o:/src/mozilla-git/central/obj-release/_tests/xpcshell/modules/libpref/test/unit/test_warnings.js: TypeError: anonymous generator function returns a value at o:/src/mozilla-git/central/obj-release/_tests/xpcshell/modules/libpref/test/unit/test_warnings.js:89
JS frame :: o:\src\mozilla-git\central\testing\xpcshell\head.js :: loadTailFile :: line 420
JS frame :: _load_files@o:\src\mozilla-git\central\testing\xpcshell\head.js:436
JS frame :: o:\src\mozilla-git\central\testing\xpcshell\head.js :: _execute_test :: line 344
JS frame :: @-e:1

The above output is as seen when the attached patch is applied.

Comment 1

[Mark Hammond [:markh] [:mhammond]]

Comment on attachment 830476 [details] [diff] [review]
Catch and report all exceptions loading a test file.

Review of attachment 830476 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/xpcshell/head.js
@@ +425,5 @@
>              source_file: element,
>              stack: _format_exception_stack(e.stack)});
> +    } catch (e) {
> +      _log("javascript_error",
> +           {_message: "TEST-UNEXPECTED-FAIL | (xpcshell/head.js) | Source file " + element + ": " + _exception_message(e),

Note that the _exception_message(e) being appended to _message will not be necessary if bug 937412 is fixed.

Comment 2

[(not currently active) Ted Mielczarek]

Comment on attachment 830476 [details] [diff] [review]
Catch and report all exceptions loading a test file.

Review of attachment 830476 [details] [diff] [review]:
-----------------------------------------------------------------

A selftest for this would be nice.

Comment 3

[Mark Hammond [:markh] [:mhammond]]

thanks - pushed with a test - I hope it was OK that I didn't specifically request review for that part.

https://hg.mozilla.org/integration/mozilla-inbound/rev/4faa50ded571

Comment 4

[(not currently active) Ted Mielczarek]

That's fine, I wouldn't grant r+ unless I expected you to land.

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/4faa50ded571