528133 - mochitest-plain leaks "the world", after bug 504862 landing

Status: VERIFIED FIXED

(Core :: General, defect)

Description

[Serge Gautherie (:sgautherie)]

This happens on both:
http://tinderbox.mozilla.org/Firefox3.5-Unittest/
http://tinderbox.mozilla.org/SeaMonkey2.0/

(16 hours) Regression timeframe:
http://hg.mozilla.org/releases/mozilla-1.9.1/pushloghtml?fromchange=a42d516d6d72&tochange=d89ca352b319
NB: It took all that time to eventually back out bug 524006, which had broken the test suite :-<

Comment 1

[Serge Gautherie (:sgautherie)]

Example:
{
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox3.5-Unittest/1257955679.1257957684.8258.gz&fulltext=1
Linux mozilla-1.9.1 test mochitests on 2009/11/11 08:07:59

TEST-UNEXPECTED-FAIL | automationutils.processLeakLog() | leaked 30711876 bytes during test execution
}

Comment 2

[Serge Gautherie (:sgautherie)]

Ah, luckily, on Firefox boxes, the leak was (already) visible in addition to bug 524006.

(2 changesets) Regression timeframe:
http://hg.mozilla.org/releases/mozilla-1.9.1/pushloghtml?fromchange=645622c570fb&tochange=7b5e903dffe1
This can be "You are not authorized to access bug #504862." only :-/

Please, back out.!.

PS: I still can't believe people care so less, especially on m-1.9.1 :-[

Comment 3

[Serge Gautherie (:sgautherie)]

9 days with permanent leak/orange!
What is the progress status?

Comment 4

[Samuel Sidler (old account; do not CC)]

Requesting status1.9.1 doesn't do anything to get an issue into drivers' eyes. If you think something is urgent, request blocking.

For more information on how to appropriately flag something, see:

  https://wiki.mozilla.org/Releases/Flags

Likewise, I don't see where jst actually "accepted" this bug or said he was going to work on it. Let's put this back to NEW until he says so.

Comment 5

[Samuel Sidler (old account; do not CC)]

Johnny backed out the offending patch.

Comment 6

[Serge Gautherie (:sgautherie)]

(In reply to comment #4)

[flames mode: on]

> Requesting status1.9.1 doesn't do anything to get an issue into drivers' eyes.
> If you think something is urgent, request blocking.

Afaict, noone but me cared about that perma-orange,
and drivers don't process many of my requests,
and many of them get rejected because there is no assignee working on the bug:
then why would I have bothered drivers with this bug?

> For more information on how to appropriately flag something, see:
>   https://wiki.mozilla.org/Releases/Flags

Oh, and count how many times I was told a (non-fully-diagnosed) test failure doesn't block a release!
No need to lecture me on flags, I think.

But then, that's just why I stopped caring about the FF tree: you typed a whole comment about how I did things "wrong" (when I'm the only one trying to actually do something) and not a single word about jst who didn't care about its changeset! :-[

> Likewise, I don't see where jst actually "accepted" this bug or said he was
> going to work on it. Let's put this back to NEW until he says so.

It seems I stupidly assumed that changesets which turn the tree (perma-)orange had to be backed out "immediately" ... not 3 weeks later ... especially on a stable branch!
And, afaict, jst was the author and pusher.

[flames mode: off]


(In reply to comment #5)
> Johnny backed out the offending patch.

V.Fixed, per FF3.5 and SM2.0 tinderboxes.

Comment 7

[Samuel Sidler (old account; do not CC)]

Several things happened here and we've talked about them all.

First, moving the unit test boxes to another tree caused this to get unnoticed by Johnny. I didn't notice it either for several days.

Second, when it was finally noticed, there was a second orange on the tree as well. Someone (I don't recall who at this moment) fixed the second orange and I (stupidly) assumed that this would make the tree green. It only made the second orange go away.

Third, drivers were unaware of the severity of the orange. Leaks of this severity do block releases when they occur and, in this case, it caused us to respin our builds of Firefox 3.0.16 and 3.5.6. That's why I suggested requesting blocking when you see these critical problems. When Reed finally saw this bug, it's immediately what he did.

Regardless, this is fixed in release builds now and we know how to ensure we can re-land bug 504862 without the leak and will do so for 1.9.1.7/1.9.0.17.

Comment 8

[Serge Gautherie (:sgautherie)]

(In reply to comment #7)

> First, moving the unit test boxes to another tree caused this to get unnoticed
> by Johnny.

Then this is an actual problem!
Also, didn't jst (or any of the cc'ed) get/read the emails from this bug?

> Second, when it was finally noticed, there was a second orange on the tree as
> well. Someone (I don't recall who at this moment) fixed the second orange and I
> (stupidly) assumed that this would make the tree green. It only made the second
> orange go away.

Yeah, it really makes me wonder who actually/correctly monitor the tree(s)!
(Eh, I know some people do, at least from times to times.)

> Third, drivers were unaware of the severity of the orange.

As I said, if (this kind of) perma-orange is not important enough to alert anyone, then why should I care (more than I still do)?

> When Reed finally saw this bug, it's immediately what he did.

Good for him.
Are you aware of how many times I asked to be able to access "security" bugs, instead of being forced to file new bugs (with no details!) that "nobody cares about"?

Comment 9

[Samuel Sidler (old account; do not CC)]

(In reply to comment #8)
> Then this is an actual problem!

It is. And I've complained about it.

> Also, didn't jst (or any of the cc'ed) get/read the emails from this bug?

I can't speak for Johnny, but I wasn't CCed until yesterday. Check the bug History. The people who were CCed weren't the right people. None of them was a driver, as I've said before.

> As I said, if (this kind of) perma-orange is not important enough to alert
> anyone, then why should I care (more than I still do)?

It is important enough, but, in this case, it was missed because of reasons I've already said.

> Are you aware of how many times I asked to be able to access "security" bugs,
> instead of being forced to file new bugs (with no details!) that "nobody cares
> about"?

To my knowledge, you have no been nominated to the security group. Regardless, if you were to ping me on IRC about a bug and explain why you wanted access (like, say, perma-orange), I'm sure I would CC you.

Comment 10

[Johnny Stenback (:jst)]

(In reply to comment #9)
[...]
> and not a single word about jst who didn't care about its changeset! :-[

This was clearly my fault, but not for lack of caring, but for lack of being aware. I didn't know there were two tinderboxes to look at (and yes, I think it's a mistake to have two), when this went green on the 1.9.1 and the 1.9.0 tinderboxes I thought I was in the clear, not knowing about the second 1.9.1 tinderbox.

And I've simply not able to keep up with all my bugmail (which is a problem on its own), and this I clearly missed.

Comment 11

[Serge Gautherie (:sgautherie)]

(In reply to comment #9)

> (In reply to comment #8)
> > Then this is an actual problem!
> 
> It is. And I've complained about it.

Good. Let's hope you'll have more success than I did.

> > Also, didn't jst (or any of the cc'ed) get/read the emails from this bug?
> 
> I wasn't CCed until yesterday. Check the bug History.

I wasn't implying that you were.

> The people who were CCed weren't the right people.

Then, maybe the automatic cc list should be updated to include at least one of the right people (= who care)...

> It is important enough, but, in this case, it was missed because of reasons
> I've already said.

Well, all you explained (yet thanks for that) is how a defined process (= backing out oranges) failed at every steps (for weeks): that should be worrying!

> To my knowledge, you have no been nominated to the security group.

I have not: that's why I said I *asked* "each" time cases like this happened before!

> Regardless,
> if you were to ping me on IRC about a bug and explain why you wanted access
> (like, say, perma-orange), I'm sure I would CC you.

Oh, I've spent a lots of time on irc and all I got was more blames ... that's why I stopped doing that too :-<


(In reply to comment #10)
> This was clearly my fault

Thanks for saying that!

> I didn't know there were two tinderboxes to look at

Hum. One should look at all tinderboxes, in other words "the tree(s)", shouldn't he? :-/
(Not really writing that SeaMonkey 2.0 has 1 'U' column per platform only, making it pretty obvious to monitor...)
Anyway.

> And I've simply not able to keep up with all my bugmail (which is a problem on
> its own), and this I clearly missed.

I can imagine that ... (though this became a real/lasting issue in this case.)

Just to be clear,
though I'm quite disappointed in this very case, I'm of course very pleased of all the work (and care) you're doing otherwise!
What did turn my flames on is Samuel coming out of nowhere and immediately/only starting to complain about what/how I had try to report...


Hopefully, we have all wrote what we thought: let's get back to work ... and possibly (even) better working processes.