435376 - Start making leaks fatal when running tests in setups where we don't leak

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Jeff Walden [:Waldo]]

--chrome tests don't leak anywhere.
Normal Mochitests don't leak on OS X.
We should enforce this so that we don't backslide in these cases, because right now we could regress and nobody would notice.

I'm sure someone is going to complain that this will turn their tinderbox orange; if you're that person, just add --leak-threshold=N to your invocation script to manually override it.  The patch to be posted here only modifies the leak threshold if it wasn't set in the commandline arguments.

Comment 1

[Jeff Walden [:Waldo]]

Attachment: Patch

We could do this tinderbox side, but then people running the tests locally won't know they're running with a different setup from tinderbox, which is likely to result in more oranges.  Also, as we drive down leak numbers, we'll want to make changes to these numbers, and it's vastly easier to change a file in the repository than it is to update a buildbot config files on the tinderboxen themselves.  For the moment, until we hit 0 everywhere and can just make that the overall default, I think we're best off doing if-unset tweaks in the specific cases where we know we're doing well.

Comment 2

[Jeff Walden [:Waldo]]

Attachment: Patch if bug 435138 is WFM now

Hm, so if bug 435138 is WFM now and the Mochitest runs on the Windows tinderbox are now doing bloat logging correctly, it seems we're at 0 leaks on normal Mochitests on Windows, too.  This patch includes that restriction.

Comment 3

[Jeff Walden [:Waldo]]

Fixt in mozilla-central.

Comment 4

[Jeff Walden [:Waldo]]

Moving a bunch of Core :: Testing bugs to Testing :: Mochitest to clear out the former, which is obsolete now that we have more specialized categories for such bugs; filter on the string "MochitestMmMm" to delete all these notifications.