449408 - bm-xserve16,17,19 throw kernel panic starting browser during AliveTest

Status: RESOLVED WORKSFORME

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Here's a paste from the Apple Problem Report:
Wed Aug  6 08:38:33 2008
panic(cpu 0 caller 0x001A8C8A): Kernel trap at 0x00132c21, type 14=page fault, registers:
CR0: 0x80010033, CR2: 0x0a67c000, CR3: 0x00d7b000, CR4: 0x00000660
EAX: 0x00000014, EBX: 0x00000014, ECX: 0x00000078, EDX: 0x5c69fa80
CR2: 0x0a67c000, EBP: 0x5c69fa58, ESI: 0x00000000, EDI: 0x0a67bfec
EFL: 0x00010206, EIP: 0x00132c21, CS:  0x00000008, DS:  0x07250010
Error code: 0x00000000

Backtrace, Format - Frame : Return Address (4 potential args on stack) 
0x5c69f778 : 0x12b0e1 (0x457024 0x5c69f7ac 0x13321a 0x0) 
0x5c69f7c8 : 0x1a8c8a (0x460550 0x132c21 0xe 0x45fd00) 
0x5c69f8a8 : 0x19eb67 (0x5c69f8c0 0x246 0x5c69fa58 0x132c21) 
0x5c69f8b8 : 0x132c21 (0xe 0x48 0x5c690070 0x3e0010) 
0x5c69fa58 : 0x388ee2 (0x551b4974 0x5c69fa88 0x388e51 0x5c69fa80) 
0x5c69fa98 : 0x551aa762 (0xb7166a9 0x15 0x551b4974 0x14) 
0x5c69fb28 : 0x44cc52 (0x16afaee0 0x68bfb90 0x68e74c8 0xb7e3360) 
0x5c69fb78 : 0x44fb57 (0x686afa4 0x68bfb90 0xb7e3360 0x1108a2e0) 
0x5c69fbb8 : 0x1dab6a (0x68bfb90 0xb7e3360 0x1108a2e0 0x5c69fe64) 
0x5c69fc18 : 0x1ea2f7 (0xb7e3360 0x5c69fd50 0x5c69fe64 0x5c69fe8c) 
0x5c69fc78 : 0x1e0b5f (0x5c69fd38 0x5c69fcec 0x5c69fe8c 0x686afa4) 
0x5c69fd08 : 0x1e11de (0x686afa4 0x5c69fd38 0x20601 0x5c69fe8c) 
0x5c69ff78 : 0x3dbe77 (0x7216720 0x686aea0 0x686aee4 0x0) 
0x5c69ffc8 : 0x19f084 (0x89804c0 0x0 0x1a20b5 0x7248208) 
No mapping exists for frame pointer
Backtrace terminated-invalid frame pointer 0xbfffe888
      Kernel loadable modules in backtrace (with dependencies):
         com.apple.security.seatbelt(107.1)@0x551a5000->0x551bcfff

BSD process name corresponding to current thread: firefox-bin

Mac OS version:
9C31

Kernel version:
Darwin Kernel Version 9.2.0: Tue Feb  5 16:13:22 PST 2008; root:xnu-1228.3.13~1/RELEASE_I386
System model name: Xserve2,1 (Mac-F42289C8)


Interestingly, bm-xserve17 did a leak test build of the exact same revision (e68237c71074) previously and did not crash.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Can you run diagnostics on this to see what caused this kernel panic?

Comment 2

[Phong Tran [:phong]]

Can I take this offline?

Comment 3

[[e-mail] (:bhearsum)]

yep

Comment 4

[Phong Tran [:phong]]

CPU B Core Test 1 - Sensor test
-Verify that the sensor reads within the sensor limits
ERROR - Sensor is reading above the high limit
- TEST FAILED -

Comment 5

[[e-mail] (:bhearsum)]

What's this mean? CPU is in bad shape? Can we get whatever it is fixed?

Comment 6

[Phong Tran [:phong]]

I am calling Apple for support.

Comment 7

[Phong Tran [:phong]]

The previous error was because the server only had 1 physical CPU.  After excluding tests for CPU B, everything passed.

Comment 8

[[e-mail] (:bhearsum)]

Are there any more tests you can do?

Comment 9

[Phong Tran [:phong]]

There aren't any more tests we can run.  We'll keep an eye on it and see if happens again.

Comment 10

[[e-mail] (:bhearsum)]

Can you turn xserve16 back on? :) I can't VNC nor ssh to it.

Comment 11

[[e-mail] (:bhearsum)]

Same thing happened over the past few days to xserve17 and xserve19. I wonder if Firefox is tripping something or if it is related to bug 448616.

Comment 12

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Given its on multiple machines, and bm-xserve16 passed diagnostics, is this a code problem?

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Throwing this over to Firefox::General is a little optimistic, me thinks. 

Which branch are these builds from ? Which AliveTest are we talking about here ? Obviously this is buildbot, but Tinderbox uses its AliveTest function to do a bunch of different things. We only do AliveTest in the debug builds ?

Comment 14

[[e-mail] (:bhearsum)]

(In reply to comment #13)
> Throwing this over to Firefox::General is a little optimistic, me thinks. 
> 
> Which branch are these builds from ? Which AliveTest are we talking about here
> ? Obviously this is buildbot, but Tinderbox uses its AliveTest function to do a
> bunch of different things. We only do AliveTest in the debug builds ?

We're talking about AliveTest during debug builds. We spawn the process a few times in this process. IIRC it happened during the --trace-malloc test on a browser built from mozilla-central.

Comment 15

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #13)
> Throwing this over to Firefox::General is a little optimistic, me thinks.

I agree... Kernel panics being caused by a Firefox code change is pretty unlikely.

Comment 16

[[e-mail] (:bhearsum)]

bm-xserve sometime between sept. 18 PM and sept 19 AM

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

bm-xserve16 just rebooted, running "python leaktest.py -l bloat.log" on a leak test build from 
 http://hg.mozilla.org//releases/mozilla-1.9.1/index.cgi/rev/32c9710b2398

The log has only
  SSL tunnel pid: 49106
  Application pid: 49107
  ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to stdout
so probably something on Firefox startup.