489985 - SeaMonkey mochitest leak threshold is leaking into (all) the other test suites now

Status: VERIFIED FIXED

(Release Engineering :: General, defect)

Description

[Serge Gautherie (:sgautherie)]

See bug 470710 comment 8 about MacOSX.

Windows has the same issue with its 200 bytes threshold:
{
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1240556666.1240561889.16656.gz&fulltext=1
WINNT 5.2 comm-central unit test on 2009/04/24 00:04:26

TEST-PASS | runtests-leaks | no leaks detected! (threshold set at 200 bytes)
}

Two things happened recently:
*Ben has been modifying steps.py leak handling code.
*KaiRo has been switching around the whole configuration code for SeaMonkey.

NB: We can't compare with Firefox as it's not using such leak thresholds.

Comment 1

[Serge Gautherie (:sgautherie)]

(Bug 469518 and bug 487496 |env| handlings have been fixed.)

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1241505831.1241512266.12945.gz&fulltext=1
WINNT 5.2 comm-central unit test on 2009/05/04 23:43:51

Bug still there.

Comment 2

[Chris Cooper [:coop] (he/him)]

Is SeaMonkey using the current buildbotcustom tip? unittest/steps.py doesn't exist any more on tip.

As Serge notes, in process/factory.py, the generic UnittestBuildFactory only
sets mochitest_leak_threshold:

http://hg.mozilla.org/build/buildbotcustom/file/7843f38a1687/process/factory.py#l1893

..although the newer CCUnittestBuildFactory does:

http://hg.mozilla.org/build/buildbotcustom/file/7843f38a1687/process/factory.py#l2162

KaiRo: are you running with local changes?

Comment 3

[Robert Kaiser]

I'm running with no local changes, but with buildbotcustom from about the time when CCUnittestBuildFactory landed.

Comment 4

[Serge Gautherie (:sgautherie)]

Could you update, to see if comment 1 fixes fixed this?

Comment 5

[Serge Gautherie (:sgautherie)]

major -> critical:
'xpcshell-tests' on our MacOSX and Windows tinderboxes are 'T-FAIL' now,
after landing bug 485736 patch Dv1a,
because of unsupported |EXTRA_TEST_ARGS=--leak-threshold=...| in the "env":
{
runxpcshelltests.py: error: no such option: --leak-threshold
}

Comment 6

[Robert Kaiser]

Looks fixed after I updated the master to current buildbotcustom.

Comment 7

[Serge Gautherie (:sgautherie)]

(In reply to comment #6)

I confirm comment 5 issue is fixed and Windows part of this bug is fully fixed too :-)

Strangely, MacOSX has a remaining part of this bug:
mochitests still report |EXTRA_TEST_ARGS=--leak-threshold=1112|, when the SeaMonkey config has '0' now...

Comment 8

[Robert Kaiser]

(In reply to comment #7)
> Strangely, MacOSX has a remaining part of this bug:
> mochitests still report |EXTRA_TEST_ARGS=--leak-threshold=1112|, when the
> SeaMonkey config has '0' now...

Nothing to do with this bug, I just forgot to update buildbot-configs and the master still has the old threshold for mac set. The bug here seems to be fixed, I'll fix the buildbot master when I come around to it.

Comment 9

[Serge Gautherie (:sgautherie)]

V.Fixed, then.