This bug was filed from the Socorro interface and is
report bp-f1b89c71-b5ec-416a-82c8-ac70e2110821 .
Seen while looking at crash reports. https://crash-stats.mozilla.com/report/list?signature=__addAltHandler2 links to the crashes which happen across versions. High concentration of 10.5.x crashes.
Two comments mention "The browser had been acting extremely slow for several minutes for the crash"
Frame Module Signature [Expand] Source
0 CoreFoundation __addAltHandler2
1 CoreFoundation _CFDoExceptionOperation
2 Foundation _NSAddExceptionHandlerForLock
3 AppKit _NSAppKitLock
4 AppKit -[NSApplication _findWindowUsingCache:]
5 AppKit -[NSEvent window]
6 AppKit -[NSEvent _cgsEventRecord]
7 AppKit -[NSEvent _eventRefInternal]
8 AppKit -[NSEvent _postAtStart:]
9 XUL nsAppShell::ProcessGeckoEvents widget/src/cocoa/nsAppShell.mm:439
10 CoreFoundation __CFRunLoopDoSources0
11 CoreFoundation __CFRunLoopRun
12 CoreFoundation CFRunLoopRunSpecific
13 CoreFoundation CFRunLoopRunInMode
14 HIToolbox RunCurrentEventLoopInMode
15 HIToolbox ReceiveNextEventCommon
16 HIToolbox BlockUntilNextEventMatchingListInMode
17 AppKit _DPSNextEvent
18 AppKit -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
19 XUL nsAppShell::ProcessNextNativeEvent widget/src/cocoa/nsAppShell.mm:675
20 XUL nsBaseAppShell::OnProcessNextEvent widget/src/xpwidgets/nsBaseAppShell.cpp:171
21 XUL nsAppShell::OnProcessNextEvent widget/src/cocoa/nsAppShell.mm:856
22 XUL nsThread::ProcessNextEvent xpcom/threads/nsThread.cpp:582
23 XUL NS_ProcessPendingEvents_P obj-firefox/i386/xpcom/build/nsThreadUtils.cpp:195
24 XUL nsBaseAppShell::NativeEventCallback widget/src/xpwidgets/nsBaseAppShell.cpp:130
25 XUL nsAppShell::ProcessGeckoEvents widget/src/cocoa/nsAppShell.mm:422
26 CoreFoundation __CFRunLoopDoSources0
27 CoreFoundation __CFRunLoopRun
28 CoreFoundation CFRunLoopRunSpecific
29 CoreFoundation CFRunLoopRunInMode
30 HIToolbox RunCurrentEventLoopInMode
31 HIToolbox ReceiveNextEventCommon
32 HIToolbox BlockUntilNextEventMatchingListInMode
33 AppKit _DPSNextEvent
34 AppKit -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
35 AppKit -[NSApplication run]
36 XUL nsAppShell::Run widget/src/cocoa/nsAppShell.mm:769
37 XUL nsAppStartup::Run toolkit/components/startup/nsAppStartup.cpp:222
38 XUL XRE_main toolkit/xre/nsAppRunner.cpp:3686
39 firefox-bin main browser/app/nsBrowserApp.cpp:158
40 firefox-bin firefox-bin@0xa05
It's #3 top crasher in 7.0.1 on Mac OS X.
It happens almost only on Mac OS X 10.5 with Flash Player:
98% (53/54) vs. 27% (579/2128) FlashPlayer-10.4-10.5
2% (1/54) vs. 0% (7/2128) 48F84FE6AB8341F943264EE657D564330
4% (2/54) vs. 1% (15/2128) 618100E683D0F650E7AB834638DB5F6A0
20% (11/54) vs. 6% (123/2128) 9E0D5FDCE905F569CBE364E85AC98BFC0
4% (2/54) vs. 3% (71/2128) 9E952D5BE0C7A400FF8D0D280BE6E3990
37% (20/54) vs. 10% (212/2128) AB20F8CBD0050FDF33413B314A53F9150
4% (2/54) vs. 1% (18/2128) CCAF6CF191CA2C8EB8ED19375014F2130
4% (2/54) vs. 0% (4/2128) CDB00B650C699A1DBD9350A6EF4A36CA0
7% (4/54) vs. 2% (38/2128) ED71BBFA041EF60C49672DDA0CC6569C0
17% (9/54) vs. 3% (63/2128) FFCFA8DE6C45E84EB674EC324644884A0
This is still a valid top crash. Sits at #2 spot for all mac crashes on 8.0.1.
Adding Steven to the bug. This shows up in Firefox 9b4 but in pretty small volume - 38 crashes. Based on the correlation, perhaps something is happening when the browser is interacting with Flash on 10.5?
fyi, Flash version support for OSX 10.5 stopped at 10.3. the latest version is 10,3,183,10.
Stacks with this signature have been around for a long time, across
all versions of Firefox and of OS X. But it's only in the last few
months that we've started to see so many of them on OS X 10.5.8.
(I'll try to get a timeline for this in a later comment).
These crashes occur deep in system code. But if I'm interpreting
Breakpad's raw dump format correctly, they're all the result of the
failure of a call to malloc() (on OS X 10.5.8) or calloc() (on OS X
10.6.8) inside the __addAltHandler2() function. (More, too, on this
in a later comment.)
This is extremely puzzling. It's very unlikely that ordinary memory
pressure (i.e. running out of memory) is responsible for these
failures. Otherwise they'd be just one part of a whole cluster of
"out of memory" errors, which I'm pretty sure we haven't seen.
If we'd enabled jemalloc on OS X 10.5 in FF 8.0.1, that'd be my first
suspect. But we haven't.
Which leads me to wonder if this is some kind of OS bug. And also to
wonder if Flash has some part in it, perhaps by (somehow) intercepting
calls to malloc() and calloc(). (Once again, more on this in a later
In the raw dumps of this bug's Breakpad crash stacks on OS X 10.5.8,
the top line of the crash stack is represented as follows:
I take this to mean that these crashes (all of them NULL dereferences)
take place at offset 0xf6 (decimal 246) from the beginning of the
__addAltHandler2() function. From the assembly code at that offset,
you can see that a call to malloc() that returns NULL will cause a
NULL derefence at that offset:
0x934b13f8 <__addAltHandler2+232>: movl $0x9c,(%esp)
0x934b13ff <__addAltHandler2+239>: call 0xa0a46588 <dyld_stub_malloc>
0x934b1404 <__addAltHandler2+244>: mov %eax,%edx
0x934b1406 <__addAltHandler2+246>: movl $0xc,(%eax)
The few examples of this crash on OS X 10.6.8 happen at a different
offset from the beginning of __addAltHander2():
Hexadecimal 0x12a translates into decimal 298. But __addAltHandler2()
is slightly longer on OS X 10.6.8, and its assembly code shows that
these crashes take place as the result of a call to calloc() that
0x03c5ee54 <__addAltHandler2+276>: movl $0x9c,0x4(%esp)
0x03c5ee5c <__addAltHandler2+284>: movl $0x1,(%esp)
0x03c5ee63 <__addAltHandler2+291>: call 0x3d29758 <dyld_stub_calloc>
0x03c5ee68 <__addAltHandler2+296>: mov %eax,%ebx
0x03c5ee6a <__addAltHandler2+298>: movl $0xc,(%ebx)
On both OS X 10.5.8 and 10.6.8, the current version of the OS-specific
Flash Player "PlugIn" has the following string:
[mem] gc[%d] %s allocator: %d%% efficiency %d bytes (%d kb) in use out of %d bytes (%d kb)
This seems to imply that Flash has a way of measuring allocator
"efficiency", and some kind of garbage collection.
Another thing to note (though many of you will already have realized it):
Even on OS X 10.6.8 and 10.7.X, these crashes only happen in 32-bit mode.
Yet another thing to note: The allocations that are failing are very small -- 0x9c bytes (156 decimal).
There were 543 crashes at __addAltHandler2 in all FF versions in the
last week. The number slowly falls as you go further back. But the
most dramatic change that I can find is between the week ending
2011/08/28 (324) and the one ending 2011/08/21 (206).
This is just after FF 6.0 was released. I don't know how it
corresponds to the Flash release schedule.
(In reply to Steven Michaud from comment #10)
> This is just after FF 6.0 was released. I don't know how it
> corresponds to the Flash release schedule.
Flash 10.3 was released in May 2011 and Flash 11.0 in October 2011 (see http://en.wikipedia.org/wiki/Adobe_Flash_Player#Release_history).
There was a security fix in Flash 10.3.183.5 released on August 12th (see http://www.adobe.com/support/security/bulletins/apsb11-21.html).
(Following up comment #5)
> It's very unlikely that ordinary memory pressure (i.e. running out
> of memory) is responsible for these failures. Otherwise they'd be
> just one part of a whole cluster of "out of memory" errors, which
> I'm pretty sure we haven't seen.
And if these were ordinary out-of-memory symptoms, you'd think high
uptimes would be heavily overrepresented in the crash stats. But as
best I can tell (by eyeballing them) the uptimes are pretty evenly
Anyone know what the "uptime" unit is? Is it seconds?
(In reply to Steven Michaud from comment #12)
> Anyone know what the "uptime" unit is? Is it seconds?
It's currently #9 top crasher in 10.0 and #16 in 11.0b1 on Mac OS X.
This still sitting in the top 5 for Mac crashes on release. At #5 on 10.0.2 with 300 crashes a week.
99% of crashes happen on Mac OS X 10.5.
Let's close it as WFM now that Mac OS X 10.5 is no longer supported in Firefox 17.