Closed Bug 411999 Opened 12 years ago Closed 12 years ago

New OS X 10.5 test box qm-xserve02 is permaorange

Categories

(Release Engineering :: General, defect, P2, major)

x86
macOS

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: Dolske, Unassigned)

References

()

Details

Bug 400313 added an OS X 10.5 test box, and it's reporting to Tinderbox. I just noticed it, so I think the reporting only recently started?

It's orange right now with reftest failures. Some pixel rounding, bidi, and canvas gradient tests. See URL for log.

Apparently the box went live on 1/7, has been failing to build for most of that time. This morning it seemed to successfully build but fails tests. I'm *assuming* the test failures are existing problems on 10.5, or something unique to the box, but I can't find anyone who knows what's going on with this.
The box was added to catch problems like this, so it's no surprise things are failing. Is it worth filing all those failures as individual bugs or can we tell exactly why it would fail for, say, a bunch of canvas tests?
I'd suggest that bugs be filed on the test failures, and the tests be disabled/fixed ASAP, so that the box is green. [Or have it not report to Tinderbox until it's green, but it looks like there are just a few groups of tests failing, so I think they should be worked around to get the benefit of having a 10.5 test box available.]
(In reply to comment #2)
> I'd suggest that bugs be filed on the test failures, and the tests be
> disabled/fixed ASAP, so that the box is green. [Or have it not report to
> Tinderbox until it's green

Incidentally, this should be standard procedure for new tinderboxen: they should be green when they show up on the main page, possibly with any failing tests disabled if they can't be quickly fixed beforehand.
Thanks to robcee for temporarily disabling this tinderbox to cut down on confusion.  Rob said he'd move it to MozillaTest when he gets a chance, after which we should troll for somebody to own debugging the orangeness so we can get it put on the main page for real.
Yeah, I expect to get this box moved to MozillaTest later today or tomorrow.

I'm not sure where the inconsistencies are coming from. There's also a significant performance hit running through these tests and this is on a newer box than qm-xserve01. Mrz contends that this box should be faster. I'll post-back when we have this running on the other tinderbox page.
Status: NEW → ASSIGNED
Priority: -- → P2
just for the record, cycle times have improved on this machine now but it's still constantly yellow. Someone should really take a look at the test failures and determine if there's a problem with the machine, with the build configs or with the tests. If none of the above, we might have some funky rendering and behavior issues on leopard.
Can you split out the separate types of failures into dependent bugs (making this a meta bug), so that people in those areas can attack the specific problems?
Depends on: 420909
Depends on: 420911
Depends on: 420913
first batch of reftest bugs filed. I'll take a look at the mochitest failures later.
Summary: WTF is up with qm-xserve02's orangeness? → New OS X 10.5 test box qm-xserve02's is permaorange
Summary: New OS X 10.5 test box qm-xserve02's is permaorange → New OS X 10.5 test box qm-xserve02 is permaorange
I grabbed a nightly build (built on 10.4), installed my reftest extension, ran on my 10.5 box, and I only failed one of the reftests that that box fails. However, I failed a bunch of other ones with what look like single-pixel antialiasing differences. So that's not awesome.
I'm pretty sure this is fixed now...
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Component: Testing → Release Engineering
Product: Core → mozilla.org
QA Contact: testing → release
Version: Trunk → other
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.