Closed Bug 464141 Opened 16 years ago Closed 16 years ago

moz2-darwin8-slave02 has been orange with 14 browser test failures since 2008/11/06 17:58

Categories

(Release Engineering :: General, defect)

PowerPC
macOS
defect
Not set
major

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: dholbert, Assigned: mozilla)

References

Details

Attachments

(1 obsolete file)

The moz2-darwin8-slave02 tinderbox has been orange with 14 browser test failures starting with this build:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1226284518.1226294238.18031.gz

The other mac unittest tinderbox (and the other OS tinderboxen) have all been fine at various points during that period, so I'm assuming these failures are due to an OS issue -- the box probably needs a clobber or a reboot.
OS: Linux → Mac OS X
Hardware: PC → Macintosh
(In reply to comment #0)
> The moz2-darwin8-slave02 tinderbox has been orange with 14 browser test
> failures starting with this build:
> http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1226284518.1226294238.18031.gz

FWIW, that build started at 2008/11/09 21:18.

The most recent orange is:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1226356650.1226366384.17926.gz
which started at 2008/11/10 14:37
(In reply to comment #1)
> FWIW, that build started at 2008/11/09 21:18.

Sorry... s/started/ended/.  That build _started_ at 18:35, as reflected in this bug's summary.
It recently went green building rev:b5dc0e849959 a second time. Intervention still required ?
(In reply to comment #3)
> It recently went green building rev:b5dc0e849959 a second time. Intervention
> still required ?

I think you're looking at moz2-darwin8-slave01, which indeed did fail on its first build of that revision but succeeded on its second & third.

This bug's about moz2-darwin8-slave02, however, and it failed on all three builds of that revision (and on all builds since, and for a number of builds before)

Reference:
http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox&maxdate=1226324558&hours=24&legend=0&norules=1
I think Sheriff jst might already have contacted IT to have them kick this box -- CCing him on this bug.
Yeah, IT is rebooting all unit test boxes...
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → DUPLICATE
hrm... Those 14 tests are still failing, in the two cycles post-reboot / clobber:

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1226369633.1226378147.29428.gz
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1226378140.1226382834.10843.gz

Is there anything else that could be stuck on the box? If not, maybe the test is broken, or maybe a checkin actually regressed something... (though it seems odd that it'd *only* be showing up on one mac tinderbox)
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Actually, I was wrong in my initial judgement of how long this has been happening . (I saw one green cycle and thought that was how far back it went)

It turns out these tests have been failing since the build beginning at 2008/11/06 17:58, with an occasional green cycle interspersed.

First failure log:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1226023115.1226032863.29166.gz

According to the "rev:" annotations on the tinderbox logs, these tests started failing after rev:4b525208d2e3 but before/including rev:738aabad5f4b, which makes bug 459493, bug 456330, bug 463204, bug 418740, and bug 428096  possibly guilty. 
(though it's possible the guilty checkin happened earlier, and there was a sporadically-green cycle before it started causing test failures)
So the first orange cycle with these failures was at changeset 738aabad5f4b,
http://hg.mozilla.org/mozilla-central/rev/738aabad5f4b

Here are the 12 hours of checkins before that:  http://tinyurl.com/6la356
I just noticed that the first failure, a timeout in browser_customize.js, happens immediately after browser_ctrlTab.js finishes.  That test (browser_ctrlTab.js) was modified twice during the checkin-range linked in my previous comment.

I'm going to try disabling that test to see if it fixes these failures.
(In reply to comment #11)
> I'm going to try disabling that test to see if it fixes these failures.

Here's a patch to do that... I can't push it right now, though, because mozilla-central is barfing at me each time I try (see bug 462706 comment 2).
Ok, pushed attachment 347490 [details] [diff] [review].... Let's see if that fixes this issue.
moz2-darwin8-slave02 still shows the same failures after building changeset 7d7d075d58a3.
Summary: moz2-darwin8-slave02 has been orange with 14 browser test failures since 2008/11/09 18:35 → moz2-darwin8-slave02 has been orange with 14 browser test failures since 2008/11/06 17:58
Assignee: server-ops → release
Assignee: release → aki
QA Contact: mrz → release
Currently it appears as though the issue is that it's starting in Safe Mode and the prompt is timing things out.  Is there a way around this?  I've tried nuking the profile and recreating.
Assignee: aki → nobody
Component: Server Operations: Tinderbox Maintenance → Release Engineering: Maintenance
Assignee: nobody → aki
Attachment #347490 - Attachment is obsolete: true
Safe mode? It would normally only start in safe mode if the option key was held down or a command-line parameter was passed... is it possible there's a stuck key?
Just went green.  I'm guessing bhearsum fixed it in bug 464662.
Status: REOPENED → RESOLVED
Closed: 16 years ago16 years ago
Resolution: --- → FIXED
Component: Release Engineering: Maintenance → Release Engineering
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: