370767 - detect if NS_ASSERTION happens during running a testcase and display that test as failed

Status: VERIFIED DUPLICATE of bug 368573

(Testing :: General, defect)

Description

[Nickolay_Ponomarev]

Assertions are bad and there was talk about treating them the same way as crashes. Right now it's a bit unclear what happens if something asserts. From what I could figure out on IRC, they are ignored now.

In any case, it would be nice if 
a) they were not ignored
b) just a particular test was marked as failed and the testsuite continued to run, since most assertions are not really -that- fatal.

This applies to all of our test harnesses ideally, in particular mochitest and reftest.

The relevant snippet of #developers log:

<asqueella> sayrer: is the idea that mochitests fail on assertion failures?
<asqueella> if not, how does one go about testing lack of assertion in certain cases?
<sayrer> asqueella: there's nothing set up for that
<asqueella> it's a pity
<asqueella> so it ignores assertions?
<sayrer> it doesn't operate at that level
<sayrer> tinderbox could watch for assertions if we wanted
<asqueella> well
<asqueella> yeah
<asqueella> that's what I'm talkign about I guess
<sayrer> talk to robcee then :)
<robcee> sayrer: I just run it from the command line on the test box and ran through perfectly
<shaver> it wouldn't be too hard to let mochitests detect assertions
<shaver> add a static int assertCount;
<shaver> have NS_ASSERT increment it
<shaver> add two little functions to a service or whatever to clear and sample the variable
<shaver> and robert is your mother's brother
<sayrer> patches welcome
<sayrer> asserts are the least of worries, really
<asqueella> why?
<sayrer> since the trunk is so horked 
<asqueella> they're supposed to be fatal
<asqueella> the state of trunk shouldn't affect the test harnesses :)
<sayrer> I'm not big on test harnesses that always fail
<sayrer> ymmv

Comment 1

[Alex Vincent [:WeirdAl]]

iirc xpcshell tests always treat assertions as fatal:
http://lxr.mozilla.org/seamonkey/source/tools/test-harness/xpcshell-simple/test_all.sh#47

Comment 2

[Nickolay_Ponomarev]

> <sayrer> asserts are the least of worries, really
> <asqueella> why?
> <sayrer> since the trunk is so horked 

Actually when I run mochitests (those checked out by default, dunno if there's more) I only get a single assertion failure. Sounds good enough to me.

Comment 4

[(not currently active) Ted Mielczarek]

Everything left in Core:Testing is going to Testing:General. Filter on CleanOutCoreTesting to ignore.