Closed Bug 511758 Opened 13 years ago Closed 12 years ago

SkipList Out of memory crashes with "new" are shown as [@ RaiseException | _CxxThrowException] - should skip RaiseException, _CxxThrowException, append "operator new" to signature

Categories

(Socorro :: General, task)

x86
Windows XP
task
Not set
critical

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: cww, Assigned: griswolf)

References

Details

(Keywords: crash, topcrash, Whiteboard: [crashkill])

Crash Data

Attachments

(1 file)

Number 6 Firefox topcrash

http://crash-stats.mozilla.com/report/list?product=Firefox&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=RaiseException

(filing because no bugs were linked from crash-stats and I couldn't find any in a quick search)
0  	kernel32.dll  	RaiseException  	
1 	mozcrt19.dll 	_CxxThrowException 	throw.cpp:159
2 	mozcrt19.dll 	operator new 	new.cpp:57
3 	xul.dll 	LiteralImpl::Create 	mozilla/rdf/base/src/nsRDFService.cpp:484
4 	xul.dll 	RDFServiceImpl::GetLiteral 	mozilla/rdf/base/src/nsRDFService.cpp:1131
5 	xul.dll 	xul.dll@0x860ee3 	
6 	xul.dll 	RDFContentSinkImpl::OpenObject 	mozilla/rdf/base/src/nsRDFContentSink.cpp:1192
7 	xul.dll 	RDFContentSinkImpl::HandleStartElement 	mozilla/rdf/base/src/nsRDFContentSink.cpp:470
8 	xul.dll 	Driver_HandleStartElement 	mozilla/parser/htmlparser/src/nsExpatDriver.cpp:97
9 	xul.dll 	doContent 	mozilla/parser/expat/lib/xmlparse.c:2464
10 	xul.dll 	contentProcessor 	mozilla/parser/expat/lib/xmlparse.c:2095
11 	xul.dll 	MOZ_XML_ParseBuffer 	mozilla/parser/expat/lib/xmlparse.c:1618
12 	xul.dll 	MOZ_XML_Parse 	mozilla/parser/expat/lib/xmlparse.c:1589
13 	xul.dll 	nsExpatDriver::ConsumeToken 	mozilla/parser/htmlparser/src/nsExpatDriver.cpp:1125
14 	xul.dll 	nsParser::Tokenize 	mozilla/parser/htmlparser/src/nsParser.cpp:2435
15 	xul.dll 	nsExpatDriver::WillResumeParse 	mozilla/parser/htmlparser/src/CNavDTD.cpp:3177
16 	xul.dll 	nsParser::OnDataAvailable 	mozilla/parser/htmlparser/src/nsParser.cpp:2305
17 	nspr4.dll 	FileAvailable 	mozilla/nsprpub/pr/src/io/prfile.c:158
Group: core-security
#8 topcrash for thunderbird 3.0b3

example bp-4cba5244-a200-44e4-9388-8a4702090822
sending a message using Vista SP2 32-bit with network.auth-use-sspi=true while joined to Active Directory
Keywords: crash, topcrash
Summary: Crash at [RaiseException] → Crash at [@ RaiseException]
(In reply to comment #2)
> #8 topcrash for thunderbird 3.0b3
> 
> example bp-4cba5244-a200-44e4-9388-8a4702090822
> sending a message using Vista SP2 32-bit with network.auth-use-sspi=true while
> joined to Active Directory

Please compare more than just the top of stack - RaiseException looks to be a generic error in windows code (kernel32) and in this case it is probably more likely that something stupid is being done.

Also I've seen reports of a bug with sspi and vista, so may be worth looking there, I think bienvenu has been looking at it.
"RaiseException" is just a Windows API to...raise an exception. In the stack in comment 1, it looks like an OOM crash. We should get that function name (as well as CxxThrowException and probably "operator new") added to the list of "frames to skip or append to the signature" in order to get more unique signatures.
Getting more unique signatures for OOM is useless, even counterproductive, without a fix for bug 493779 (breakpad detecting OOM) or bug 441324 (xmalloc).
Summary: Crash at [@ RaiseException] → Crash at [@ RaiseException] indicating Out Of Memory
I don't actually believe that some of these are oom -- some could also just be bogus too-large allocations.  I just crashed at http://crash-stats.mozilla.com/report/index/8a037afd-678d-46c8-8b80-11ca02090830 and the browser/computer was fine just before the crash.
This is one case where I really wish we had argument values on the stack. :-/
Vlad has a good point.  Quite a few of these even happen at startup!

Since we don't use exceptions ourselves, let's skip "RaiseException" and "_CxxThrowException", and append "operator new".  

For example, bp-d238e1b5-12b5-47a9-b54d-d02242091013 should have the signature [operator new | nsFastLoadFileReader::ReadFooter].
Component: General → Socorro
Product: Firefox → Webtools
QA Contact: general → socorro
Summary: Crash at [@ RaiseException] indicating Out Of Memory → Out of memory crashes with "new" are shown as [@ RaiseException | _CxxThrowException]
Version: unspecified → other
Together, these crashes are topcrash #31.  Splitting them up would make them easier to tackle.  Many of the "too large allocation" crashes will be easy to fix, and getting correlations for the crashes we suspect are truly "out of memory" could tell us what extensions cause severe leaks.
Duplicate of this bug: 478995
re: breakdown of startup crashes and breakdown of other RaiseExpection signatures

report for the stack signature RaiseException 

2081 total crashes for RaiseException on 20091019-crashdata.csv
695 start up crashes inside 3 minutes
328.935 (days) total uptime for 1959 of these crashes where user crashed within the last year 
241.79 (minutes) avg time since last crash 
111  number of bogus time-since-last-crash reported for this signature 

RaiseException signature breakdown
signature distribution
     101
signature list
1313 RaiseException | _CxxThrowException
  90 RaiseException | mspbwkr.dll@0x25a22
  79 RaiseException | Shell32.dll@0x28b3
  62 RaiseException | wmp.dll@0x22933f
  40 RaiseException | AcroRd32.dll@0xf20df
  36 RaiseException | GoogleDesktopMozilla.dll@0x1555b
  35 RaiseException | npietab.dll@0x20192
  31 RaiseException | AcroRd32.dll@0xf2009
  27 RaiseException | _raise_exc_ex
  25 RaiseException | radhslib.dll@0x254f
  25 RaiseException | _hfirewall.dll@0x2a1b
  22 RaiseException | DeezRipDll.dll@0x3a56
  22 RaiseException | AcroRd32.dll@0xf1749
  21 RaiseException | MySpaceFFoxTB.dll@0x4354b
  17 RaiseException | AcroRd32.dll@0xf20d0
  16 RaiseException | wmp.dll@0x228c2b
  14 RaiseException | wmp.dll@0x1b138a
  14 RaiseException | AcroRd32.dll@0xf181f
  12 RaiseException | tmagent.dll@0x8d870
  11 RaiseException | wmp.dll@0x228c62
   9 RaiseException | AcroForm.api@0x4ce5ec
   8 RaiseException | nw.dll@0x153e1
   8 RaiseException | coreclr.dll@0x17c24d
   6 RaiseException | xwsg.dll@0x433c
   6 RaiseException | crescendoCatcher.dll@0x43ec9
   5 RaiseException | AcroRd32.dll@0xf1ffa
   5 RaiseException | AcroRd32.dll@0xef2be
   5 RaiseException | AcroRd32.dll@0xef1e8
   4 RaiseException | wmp.dll@0x228d3d
   4 RaiseException | mspbwkr.dll@0x295d2
   4 RaiseException | _hfirewall.dll@0x269f
   3 RaiseException | wmp.dll@0x2293fe
   3 RaiseException | wmp.dll@0x1b13ac
   3 RaiseException | sdata.dll@0x252f
   3 RaiseException | mscscc.dll@0x28ab
   3 RaiseException | gsl.dll@0x7bb0
   3 RaiseException | Shell32.dll@0x3f96
   3 RaiseException | SbieDll.dll@0x87ce
   3 RaiseException | GoogleDesktopMozilla.dll@0x154dd
   3 RaiseException | CoolType.dll@0x17200d
   2 RaiseException | wmp.dll@0x228c0b
   2 RaiseException | stormchaser.dll@0x494d7
   2 RaiseException | sis.dll@0x192c1
   2 RaiseException | radhslib.dll@0x252f
   2 RaiseException | nw.dll@0x6aa5
   2 RaiseException | npietab.dll@0x1fe89
   2 RaiseException | Shell32.dll@0x288b
   2 RaiseException | Scxpx86.dll@0x524cf
   2 RaiseException | SSS.dll@0x859c3
   2 RaiseException | GoogleDesktopMozilla.dll@0x16982
   2 RaiseException | CoolType.dll@0x1720fd
   2 RaiseException | CMediaViewHandler::Shutdown()
   2 RaiseException | Acrobat.dll@0x13cccc
   2 RaiseException | Acrobat.dll@0x13cbf6
   2 RaiseException | Acrobat.dll@0x13b519
   2 RaiseException | AcroRd32.dll@0xf0d3f
   2 RaiseException | AcroForm.api@0x4cde7c
   1 RaiseException | zgtips.dll@0xbfc88
   1 RaiseException | xwsg.dll@0x43d0
   1 RaiseException | xwsg.dll@0x4378
   1 RaiseException | xshared.dll@0x2e530
   1 RaiseException | wmp.dll@0x7c422c
   1 RaiseException | wmp.dll@0x284f1e
   1 RaiseException | wmp.dll@0x277a98
   1 RaiseException | wmp.dll@0x2774e8
   1 RaiseException | wmp.dll@0x2752f1
   1 RaiseException | wmp.dll@0x22aa7e
   1 RaiseException | wmp.dll@0x228cea
   1 RaiseException | wmp.dll@0x228c36
   1 RaiseException | wmp.dll@0x1b2e52
   1 RaiseException | sysspc.dll@0x28b3
   1 RaiseException | spxpclp32.dll@0x28b3
   1 RaiseException | spcffwl.dll@0x28b3
   1 RaiseException | sQusi20Blk.dll@0xa772
   1 RaiseException | nw.dll@0x265f
   1 RaiseException | npwinext.dll@0x10a82
   1 RaiseException | npctrl.1.0.30716.0.dll@0x987fa
   1 RaiseException | npGarmin.dll@0x200cb0
   1 RaiseException | npCCTVplayer.dll@0x11bc2
   1 RaiseException | nexFilter.dll@0x2a47
   1 RaiseException | mscorwks.dll@0x5e773
   1 RaiseException | mscoree.dll@0x2f99c
   1 RaiseException | lsp.dll@0x14bb6
   1 RaiseException | lsp.dll@0x135c5
   1 RaiseException | kikin.dll@0x1cb59
   1 RaiseException | hfirewall.dll@0x269f
   1 RaiseException | googletoolbar-ff3.dll@0x2b32f
   1 RaiseException | controlkids2.dll@0x1e134
   1 RaiseException | TMSender.XPCom.dll@0x4b8b1
   1 RaiseException | NPComponent.dll@0xa23b
   1 RaiseException | LsNdis.dll@0x273f
   1 RaiseException | IconixFF3_5.dll@0x4c3d8
   1 RaiseException | GoogleDesktopMozilla.dll@0x1639c
   1 RaiseException | GoogleDesktopMozilla.dll@0x161f4
   1 RaiseException | CoolType.dll@0x17213d
   1 RaiseException | CTabWindow::MarkTabAsNotHung(unsigned long)
   1 RaiseException | Acrobat.dll@0x13cbb5
   1 RaiseException | Acrobat.dll@0x13cadf
   1 RaiseException | Acrobat.dll@0x13b5ef
   1 RaiseException | AcroForm.api@0x4b6c2c
   1 RaiseException
Attached file RaiseException-crashes
something that might help organize the various crashes into buckets.   all the raiseException crashes sorted by "time since last crash".
In the set of close to start up crashes there seem to be a bunch of fast load related stacks that look something like this.

0  	kernel32.dll  	RaiseException  	
1 	mozcrt19.dll 	_CxxThrowException 	throw.cpp:159
2 	mozcrt19.dll 	operator new 	new.cpp:57
3 	xul.dll 	nsFastLoadFileReader::ReadFooter 	mozilla/xpcom/io/nsFastLoadFile.cpp:722
4 	xul.dll 	nsFastLoadFileReader::Open 	mozilla/xpcom/io/nsFastLoadFile.cpp:978
5 	xul.dll 	NS_NewFastLoadFileReader 	mozilla/xpcom/io/nsFastLoadFile.cpp:1225
6 	xul.dll 	nsFastLoadService::NewInputStream 	mozilla/xpcom/io/nsFastLoadService.cpp:172
7 	xul.dll 	mozJSComponentLoader::StartFastLoad 	mozilla/js/src/xpconnect/loader/mozJSComponentLoader.cpp:880
8 	xul.dll 	CallGetService 	nsComponentManagerUtils.cpp:94
9 	xul.dll 	nsSSLSocketProvider::AddRef 	mozilla/netwerk/base/src/nsFileStreams.cpp:364
10 	xul.dll 	mozJSComponentLoader::GlobalForLocation 	mozilla/js/src/xpconnect/loader/mozJSComponentLoader.cpp:1127
11 	xul.dll 	nsLocalFile::ResolveAndStat 	mozilla/xpcom/io/nsLocalFileWin.cpp:847
12 	xul.dll 	nsComponentManagerImpl::AutoRegisterComponent 	mozilla/xpcom/components/nsComponentManager.cpp:2988
13 	xul.dll 	nsLocalFile::IsDirectory 	mozilla/xpcom/io/nsLocalFileWin.cpp:2399
14 	xul.dll 	nsComponentManagerImpl::AutoRegisterImpl 	mozilla/xpcom/components/nsComponentManager.cpp:2862

Is this the first one we could spin out from this bug?
chofmann, comment 11 is offtopic.  PLEASE stop spamming bugs with long, offtopic comments.

Regarding comment 13, yes.
Depends on: 524313
Target Milestone: --- → 1.2
Lars: can you add RaiseException and _CxxThrowException to irrelevantSignatureRegEx, and "operator new" to prefixSignatureRegEx? (It looks like RaiseException is already in prefixSignatureRegEx, I suppose we should remove it from there.)
Summary: Out of memory crashes with "new" are shown as [@ RaiseException | _CxxThrowException] → Out of memory crashes with "new" are shown as [@ RaiseException | _CxxThrowException] - should skip RaiseException, _CxxThrowException, append "operator new" to signature
See also bug 514505: Fix at same time
Assignee: nobody → griswolf
Whiteboard: [crashkill]
Summary: Out of memory crashes with "new" are shown as [@ RaiseException | _CxxThrowException] - should skip RaiseException, _CxxThrowException, append "operator new" to signature → SkipList Out of memory crashes with "new" are shown as [@ RaiseException | _CxxThrowException] - should skip RaiseException, _CxxThrowException, append "operator new" to signature
Resolved per bug 529076 2009-11-18 16:07:48 PST
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
That didn't quite work.  

bp-fcb1828b-e1d2-419c-9424-4e9ae2091126
[@operator new(unsigned int) ]
should be:
[@operator new(unsigned int) | nsFastLoadFileReader::ReadFooter(nsFastLoadFileReader::nsFastLoadFooter*) ]
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
it just looks like "operator new" was not added to the prefixSignatureRegEx config parameter.  Griswolf, would you do the honors and submit another IT request for this change?
in bp-fcb1828b-e1d2-419c-9424-4e9ae2091126 we need operator new with an argument. In comment #1 and comment #13, we appear to need the no-argument version. Further code spelunking: We need to handle a single parameter. Do not want to match a placement new.
Fixed half an hour ago in bug 531895.  Seems to be working great for new crashes:

bp-fd9c67f9-f0bf-4cf9-b751-b31362091201
bp-7d15fbc0-b6fe-4c31-b56b-795882091201
Status: REOPENED → RESOLVED
Closed: 12 years ago12 years ago
Resolution: --- → FIXED
Crash Signature: [@ RaiseException | _CxxThrowException]
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.