1480075 - Should Assert.rejects reject or report a test error when the exception is incorrect

Status: ASSIGNED

(Testing :: General, enhancement, P3)

Description

[Julian Descottes [:jdescottes]]

Currently, if the exception thrown by the code tested by Assert.rejects is not the expected one, the exception is just bubbled up and shows up in the test logs as 

FAIL Uncaught exception - ${Stringified version of the error}

If the error message is very different from the expected one, or worst if the error is an object (FAIL Uncaught exception [Object object]), it is difficult to realize that the code is already being checked in an Assert.rejects.

It would be nice if we could have an additional failure report, that logs the failure message provided by the caller.

The relevant code is at https://searchfox.org/mozilla-central/rev/196560b95f191b48ff7cba7c2ba9237bba6b5b6a/testing/modules/Assert.jsm#425-428

We could either report and reject, or only report.

Comment 1

[Julian Descottes [:jdescottes]]

Mark, any reason why we just reject the error at the moment?

Comment 2

[Julian Descottes [:jdescottes]]

Since we have unit tests checking that the original exception is still thrown, I suppose doing both is the only viable option.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=636c396124b4bf1a0e7b3bc09e05b5801dc3732b

Comment 3

[Mark Banner (:standard8)]

This definitely seems a reasonable thing to do. Mike is probably a bit more of an expert in this area though.

Comment 4

[Mike de Boer [:mikedeboer]]

I agree with Mark, I'd be happy with this change.

Comment 5

[Julian Descottes [:jdescottes]]

Attachment: Bug 1480075 - Avoid test timeout if no rejection gets caught by Assert.rejects

Comment 6

[Julian Descottes [:jdescottes]]

Attachment: Bug 1480075 - Report a meaningful failure when Assert.rejects catches an incorrect exception

Depends on D96772