Open Bug 1436654 Opened 3 years ago

SpecialPowers.loadChromeScript assertion failures don't fail tests


(Testing :: Mochitest, defect)

Not set


(Not tracked)


(Reporter: MattN, Unassigned)



I'm not sure if this is a regression since the assert.* ability landed in bug 965257 or whether it never worked properly but a simple test like the following does not cause a test failure (when run as a single file or the directory):

add_task(async function test_in_chrome() {
  let chromeScript = SpecialPowers.loadChromeScript(function test() {
    assert.ok(false, "This should fail the test");
  ok(true, "This is to avoid the 'No checks actually run'");

I do get the following in the log:
> 0:26.09 GECKO(71672) TEST-UNEXPECTED-FAIL | test | This should fail the test - false == true - got false, expected true (operator ==)

I think there are multiple issues going on but I don't really understand how it's all supposed to work in order to fix it.

I believe the logging is happening because the code is falling back to dump() if it can't find the TestRunner:
The assert.ok is not included in the pass count which I think is part of the problem. I'm not sure which API to use to report the pass/failure to have it count in the summary table and actually cause local failures.
You need to log in before you can comment on or make changes to this bug.