SeaMonkey Linux leak test builds hanging in alive_test_2 since December 1

VERIFIED FIXED

Status

defect
VERIFIED FIXED
9 years ago
9 years ago

People

(Reporter: kairo, Assigned: Ehsan)

Tracking

({regression})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Since December 1, we're seeing Linux leak test builds on trunk hang in alive_test_2 and needing manual interruption to come back.

The revisions from the last good and first bad builds tells us that no c-c checkin has caused it, the m-c regression frame is http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e1983b9db75d&tochange=01bdda1252f0

Incidentally, I've seen Linux builds on my machines suddenly start up in offline mode in the last days, if there's a connection, then bug 614286 from ehsan sounds very likely to have affected this. Therefore, CCing him for a potential clue on what could be happening.
As a note, logs of the builds don't show anything strange, see e.g. http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1291343829.1291388374.29231.gz which I killed after hanging for a significant time in this test.

I've meanwhile clobbered all our builders for this, but the first clobbered run didn't show a difference and had to be manually interrupted again.
Does SeaMonkey ship the this service? "@mozilla.org/network/network-link-service;1"
(In reply to comment #2)
> Does SeaMonkey ship the this service?
> "@mozilla.org/network/network-link-service;1"

It seems to work locally to instantiate it and fetch .isLinkUp like the test added in attachment 493880 [details] [diff] [review] does. Also, If I get it right, that's an xpcshell-test and we have green xpcshell-tests on those builds. I checked an saw that this might go through the dbus service, and I see that one being built in our build logs.
BTW, I can confirm that my local Firefox builds start in online mode, while my SeaMonkey builds as well as my Thunderbird builds start in offline mode.
What does Firefox do differently than Thunderbird and SeaMonkey here?
Can someone point me to the mozconfig file used to build SM tinderbox builds?  I bet this is my fault, a regression from bug 614958.
Blocks: 614958
Assignee: nobody → ehsan
blocking2.0: --- → ?
So, when I start a locally built SeaMonkey, it starts in online mode without any problems...

Robert, how can I run the failing test locally?
(In reply to comment #5)
> Can someone point me to the mozconfig file used to build SM tinderbox builds? 

http://hg.mozilla.org/build/buildbot-configs/file/3125d448b587/seamonkey/linux/comm-central-trunk/debug/mozconfig is the one used by those builds (not that the PGO script is pointed to but we don't build with PGO, actually).

(In reply to comment #6)
> So, when I start a locally built SeaMonkey, it starts in online mode without
> any problems...

Fun, I wonder why mine don't (on two different machines).

> Robert, how can I run the failing test locally?

buildbot says that this is what is being called there:

python leaktest.py -- -CreateProfile default
 in dir /builds/slave/comm-central-trunk-linux-debug/build/objdir/mozilla/_leaktest

I've never actually tried that test itself locally.
Hmm, is this behavior Linux only?  I've been testing on Mac, but I could try Linux too, if needed.

I just ran the leaktest suite locally without any problems...
(In reply to comment #8)
> Hmm, is this behavior Linux only?

Yes, this is Linux only, sorry I just noticed that the summary missed that, even though the comments mentioned it. Mac (both 32bit and 64bit) and Windows work in this respect, it's somehow only Linux (only tested 32bit, but I'd suspect it happens for 64bit as well) that breaks.
Summary: SeaMonkey leak test builds hanging in alive_test_2 since December 1 → SeaMonkey Linux leak test builds hanging in alive_test_2 since December 1
(In reply to comment #4)
> BTW, I can confirm that my local Firefox builds start in online mode, while my
> SeaMonkey builds as well as my Thunderbird builds start in offline mode.
> What does Firefox do differently than Thunderbird and SeaMonkey here?

Thunderbird, and I think SeaMonkey, have some code for managing the online/offline state depending on user prefs. This is the Thunderbird version:

http://mxr.mozilla.org/comm-central/source/mail/base/content/mail-offline.js#41

offline.autoDetect is off for SeaMonkey, and off for Thunderbird only on Linux.

Maybe there's something about the status that's getting confused because ehsan's changed the default to offline, then it presumably comes online during startup. So could that be sending a "network:*" notification that we're not expecting?
The real issue here is that now IO Service defaults to being offline.  It's OK if the network link service is available (see comment 2), but for the cases which it's not (e.g., bug 617389), we start up in offline mode, which is not good.

I think we should fix this at the Necko level.  This is not really something which should be fixed in Thunderbird/SeaMonkey code.  I have a patch which fixes this.  I'm trying to write a test for it now...
Blocks: 617389
Attachment #496063 - Flags: review?(bzbarsky)
Attachment #496063 - Flags: approval2.0?
Comment on attachment 496063 [details] [diff] [review]
Patch (v1) - commited

s/if/whether/ and looks great.
Attachment #496063 - Flags: review?(bzbarsky)
Attachment #496063 - Flags: review+
Attachment #496063 - Flags: approval2.0?
Attachment #496063 - Flags: approval2.0+
Whiteboard: [needs landing]
(In reply to comment #10)
> Thunderbird, and I think SeaMonkey, have some code for managing the
> online/offline state depending on user prefs.

Wow, that's crufty. Shouldn't we clean this up now that there are generic things to handle online/offline status?

> offline.autoDetect is off for SeaMonkey, and off for Thunderbird only on Linux.

Shouldn't we turn it on on all platforms (or remove it and let ioService just do it on its own discretion) now that even Linux can detect the online/offline state (via DBus)?
blocking2.0: ? → ---
(In reply to comment #14)
> (In reply to comment #10)
> > Thunderbird, and I think SeaMonkey, have some code for managing the
> > online/offline state depending on user prefs.
> 
> Wow, that's crufty. Shouldn't we clean this up now that there are generic
> things to handle online/offline status?

AFAIK they extend the generic things for online/offline status. I'm pretty sure that core doesn't cover those preferences that we have.

> > offline.autoDetect is off for SeaMonkey, and off for Thunderbird only on Linux.
> 
> Shouldn't we turn it on on all platforms (or remove it and let ioService just
> do it on its own discretion) now that even Linux can detect the online/offline
> state (via DBus)?

Last I heard DBus wasn't very reliable at detecting online/offline state, hence the reason it was turned off.
(In reply to comment #15)
> (In reply to comment #14)
> AFAIK they extend the generic things for online/offline status. I'm pretty sure
> that core doesn't cover those preferences that we have.

There at least seems to be some equivalent of the .autoDetect one, if I saw things correctly in the other bug this one is blocking.
Comment on attachment 496063 [details] [diff] [review]
Patch (v1) - commited

Commited as <http://hg.mozilla.org/mozilla-central/rev/e68363d92389>
Attachment #496063 - Attachment description: Patch (v1) → Patch (v1) - commited
Whiteboard: [needs landing]
No longer blocks: 617389
Thanks for pushing the patch, Ben!
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
verified in terms of the hang on leaktest being gone.
Status: RESOLVED → VERIFIED
Duplicate of this bug: 616103
You need to log in before you can comment on or make changes to this bug.