Closed Bug 597260 Opened 14 years ago Closed 8 years ago

startup crash in nsFileOutputStream::Write

Categories

(Core :: Networking: File, defect)

x86
Windows XP
defect
Not set
critical

Tracking

()

RESOLVED WORKSFORME
Tracking Status
firefox10 - ---
firefox11 - ---

People

(Reporter: cbook, Assigned: michal)

References

()

Details

(Keywords: crash, Whiteboard: [crashkill][startupcrash][necko-active])

Crash Data

Firefox 3.6.10 Crash Report - Top #5 Crash so far - Firefox 3.6.10 Crash Report [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ] - http://crash-stats.mozilla.com/report/list?range_value=2&range_unit=weeks&date=2010-09-16%2009%3A00%3A00&signature=nsFileOutputStream%3A%3AWrite%28char%20const*%2C%20unsigned%20int%2C%20unsigned%20int*%29&version=Firefox%3A3.6.10 lots of very low downtime. Crashing Thread Frame Module Signature [Expand] Source 0 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:418 1 xul.dll nsStreamCopierIB::ConsumeInputBuffer xpcom/io/nsStreamUtils.cpp:523 2 xul.dll nsStringInputStream::ReadSegments xpcom/io/nsStringStream.cpp:276 3 xul.dll nsStreamCopierIB::DoCopy xpcom/io/nsStreamUtils.cpp:540 4 xul.dll nsAStreamCopier::Process xpcom/io/nsStreamUtils.cpp:323 5 xul.dll nsEventQueue::GetEvent xpcom/threads/nsEventQueue.cpp:100 6 @4dffeab 7 xul.dll nsAStreamCopier::Run xpcom/io/nsStreamUtils.cpp:439 8 xul.dll nsThreadPool::Run xpcom/threads/nsThreadPool.cpp:219 9 xul.dll nsThread::ProcessNextEvent xpcom/threads/nsThread.cpp:527 10 xul.dll NS_ProcessNextEvent_P obj-firefox/xpcom/build/nsThreadUtils.cpp:250 11 xul.dll nsThread::ThreadFunc xpcom/threads/nsThread.cpp:254 12 nspr4.dll _PR_NativeRunThread nsprpub/pr/src/threads/combined/pruthr.c:426 13 nspr4.dll pr_root nsprpub/pr/src/md/windows/w95thred.c:122 14 mozcrt19.dll _callthreadstartex obj-firefox/memory/jemalloc/crtsrc/threadex.c:348 15 mozcrt19.dll _threadstartex obj-firefox/memory/jemalloc/crtsrc/threadex.c:326 16 kernel32.dll BaseThreadStart
Version: Trunk → 1.9.2 Branch
start up crash Correlation to startup or time of session 2317 total crashes for nsFileOutputStream::Write.char.const...unsigned.int..unsigned.int.. on 20100916-crashdata.csv ~1990 inside 4 seconds of start up 2247 startup crashes inside 30 sec. 2282 startup crashes inside 3 min. 1710 repeated crashes inside 3 min. of last crash that's been around for awhile and/or is wide spread. Correlation to releases checking --- nsFileOutputStream::Write.char.const...unsigned.int..unsigned.int.. 20100916-crashdata.csv found in: 3.6.9 3.6.10 3.6.3 3.6 4.0b3 3.6.8 3.0.19 3.5.12 3.0.1 4.0b2 4.0b4 3.0.5 3.6b2 3.5.11 3.0.6 3.0.11 3.5.2 3.0.7 3.6.2 3.0.8 3.0 3.6.6 3.5.6 3.0b1 4.0b5pre 3.5.3 3.5.13 3.0.3 3.1b2 4.0b7pre 3.5.5 3.1b3 3.0.10 3.0.9 3.6b4 3.0.4 4.0b6pre 3.6.7 3.5.9 3.5 3.0.2 3.6.4 3.5.4 3.5.10 3.5.1 3.0.17 3.0.13 release total-crashes nsFileOutputStream::Write.char.const...unsigned.int..unsigned.int.. crashes pct. all 313788 2317 0.00738397 3.6.9 106069 568 0.005355 3.6.10 45610 351 0.00769568 3.6.3 11292 223 0.0197485 3.6 6444 211 0.0327436 4.0b3 1321 168 0.127176 3.6.8 40358 139 0.00344417 3.0.19 7876 115 0.0146013 3.5.12 8982 54 0.00601202 3.0.1 1241 57 0.0459307 4.0b2 1249 43 0.0344275 4.0b4 3192 41 0.0128446 3.0.5 811 39 0.0480888 3.6b2 455 30 0.0659341 3.5.11 4122 25 0.00606502 3.0.6 383 24 0.0626632 3.0.11 258 20 0.0775194 3.5.2 823 19 0.0230863 3.0.7 216 19 0.087963 3.6.2 1892 48 0.02537 3.0.8 316 16 0.0506329 100% windows XP not many urls due to nearness to start up, but when we do get them it seems connected to many russian and german sites. 2211 // 58 about:blank// 26 \N// 1 http://yandex.kz 1 http://www.wmraskrutim.ru 1 http://www.ultop.ru 1 http://www.ukr.net 1 http://www.playground.ru 1 http://www.neobroker.ru 1 http://www.isochi.ru 1 http://www.facebook.com 1 http://www.dogtied.com 1 http://www.bild.de 1 http://www.anekdotov.net 1 http://wrz.to 1 http://winx4u.ru 1 http://torrent.dml 1 http://talks.guns.ru 1 http://stopgame.ru 1 http://pornolab.net 1 http://pornokoldun.com 1 http://pornmylife.ru 1 http://maps.google.de
OS: Mac OS X → Windows XP
Summary: Firefox 3.6.10 Crash Report [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ] → Firefox Crash Report [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ]
this is probably the same problem as bug 599126.
In Thunderbird, this crash appears as [@ strstr | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] .. and only this signature. Bug 574996. only about half are startup crashes most recent thunderbird trunk crash is several from June 25, all the same user bp-c73daf92-4840-480f-bfcd-32a3e2100625 Last Crash 11360 seconds (3.2 hours) before submission Install Age 59482 seconds (16.5 hours) since version was first installed. Product Thunderbird Version 3.2a1pre Build ID 20100623035012
Blocks: 574996
Component: General → Networking: File
QA Contact: general → networking.file
Whiteboard: [crashkill] → [crashkill][tbird crash]
pretty much affecting all releases at his point and is now the #1 topcrash for 4.0b7pre http://crash-stats.mozilla.com/query/query?product=Firefox&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F01%2F2010+12%3A54%3A05&query_search=signature&query_type=exact&query=nsFileOutputStream%3A%3AWrite%28char+const*%2C+unsigned+int%2C+unsigned+int*%29+&build_id=&process_type=any&hang_type=any&do_query=1 6200 crashes per day and might also go by the these signatures signature list 6227 nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) 10 strstr | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) release total-crashes nsFileOutputStream::Write.char.const...unsigned.int..unsigned.int.. crashes pct. all 336929 6237 0.0185113 3.6.10 195929 3066 0.0156485 3.6 7135 487 0.0682551 3.0.19 9344 407 0.0435574 3.6.3 11144 409 0.0367014 3.5.13 16321 341 0.0208933 3.0.1 1606 173 0.107721 3.6.8 20011 141 0.00704612 4.0b3 1089 81 0.0743802 3.0.10 573 77 0.13438 4.0b6 23602 73 0.00309296 4.0b4 2075 70 0.0337349 4.0b7pre2278 69 0.0302897 its likely this is some kind of external thing (malware/another program) but we need to figure out if there are any defenses or protects that we do or tell the users about.
So, there are three versions of this crash. Signature nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) UUID d6813c58-98c0-44a8-85ef-8d8772101001 Time 2010-10-01 12:41:21.823170 Uptime 56 Last Crash 19402 seconds (5.4 hours) before submission Install Age 1341324 seconds (2.2 weeks) since version was first installed. Product Firefox Version 3.6.10 Build ID 20100914125854 Branch 1.9.2 OS Windows NT OS Version 5.1.2600 Service Pack 3 CPU x86 CPU Info AuthenticAMD family 6 model 10 stepping 0 Crash Reason EXCEPTION_ACCESS_VIOLATION_WRITE Crash Address 0x0 User Comments Processor Notes EMCheckCompatibility False Related Bugs Crashing Thread Frame Module Signature [Expand] Source 0 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:418 1 xul.dll nsStreamCopierIB::ConsumeInputBuffer xpcom/io/nsStreamUtils.cpp:523 2 xul.dll nsStringInputStream::ReadSegments xpcom/io/nsStringStream.cpp:276 3 xul.dll nsStreamCopierIB::DoCopy xpcom/io/nsStreamUtils.cpp:540 This is a null pointer crash. 532 PRUint32 DoCopy(nsresult *sourceCondition, nsresult *sinkCondition) 533 { 534 ReadSegmentsState state; 535 state.mSink = mSink; 536 state.mSinkCondition = NS_OK; 537 538 PRUint32 n; "n" is a stack variable 539 *sourceCondition = 540 mSource->ReadSegments(ConsumeInputBuffer, &state, mChunkSize, &n); here we get a reference to the stack variable and pass it as the last argument 260 nsStringInputStream::ReadSegments(nsWriteSegmentFun writer, void *closure, 261 PRUint32 aCount, PRUint32 *result) 262 { 263 NS_ASSERTION(result, "null ptr"); here we assert (debug only) that the last argument is not null 264 NS_ASSERTION(mLength >= mOffset, "bad stream state"); 265 266 // We may be at end-of-file 267 PRUint32 maxCount = LengthRemaining(); 268 if (maxCount == 0) { ... 271 } 272 NS_ASSERTION(mData, "must have data if maxCount != 0"); 273 274 if (aCount > maxCount) 275 aCount = maxCount; 276 nsresult rv = writer(this, closure, mData + mOffset, 0, aCount, result); writer was the first argument to this function, and it's ConsumeInputBuffer, 514 static NS_METHOD ConsumeInputBuffer(nsIInputStream *inStr, ... 519 PRUint32 *countWritten) 520 { 521 ReadSegmentsState *state = (ReadSegmentsState *) closure; 522 523 nsresult rv = state->mSink->Write(buffer, count, countWritten); here the last argument is passed again as the last argument 514 static NS_METHOD ConsumeInputBuffer(nsIInputStream *inStr, ... 519 PRUint32 *countWritten) 520 { 521 ReadSegmentsState *state = (ReadSegmentsState *) closure; 522 523 nsresult rv = state->mSink->Write(buffer, count, countWritten); here the last argument is passed again as the last argument 409 nsFileOutputStream::Write(const char *buf, PRUint32 count, PRUint32 *result) 410 { 411 if (mFD == nsnull) 412 return NS_BASE_STREAM_CLOSED; 413 414 PRInt32 cnt = PR_Write(mFD, buf, count); here NSPR is called, which can call into evil hooks or system drivers (more evil hooks) Whatever happened, none of the evil hooks claimed to fail, so we assume that everything's ok: 415 if (cnt == -1) { 416 return NS_ErrorAccordingToNSPR(); 417 } We did not pass "result" to PR_Write, so anyone hurting it inside this function is doing so by violating our stack integrity (they're being evil). 418 *result = cnt; Here we dereference null, but it was supposed to be a stack variable, it should not be null. This is the 3.6.10 version of the crash. The 4.0b7 version is a bit different. Actually, I can't find 4.0b7pre, but here's 4.0b6: Signature nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) UUID f7633235-0832-4e5f-a6f0-48b332101001 Time 2010-10-01 12:34:46.615482 Uptime 2 Last Crash 15 seconds before submission Install Age 1430512 seconds (2.4 weeks) since version was first installed. Product Firefox Version 4.0b6 Build ID 20100914083612 Branch 2.0 OS Windows NT OS Version 5.1.2600 Service Pack 3 CPU x86 CPU Info GenuineIntel family 15 model 4 stepping 3 Crash Reason EXCEPTION_ACCESS_VIOLATION_WRITE Crash Address 0x1e User Comments App Notes AdapterVendorID: 1002, AdapterDeviceID: 71c1 Processor Notes EMCheckCompatibility False Related Bugs Crashing Thread Frame Module Signature [Expand] Source 0 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:422 1 xul.dll nsBufferedOutputStream::Flush netwerk/base/src/nsBufferedStreams.cpp:558 2 xul.dll nsBufferedOutputStream::Write netwerk/base/src/nsBufferedStreams.cpp:541 3 xul.dll nsPrefService::WritePrefFile modules/libpref/src/nsPrefService.cpp:485 4 xul.dll nsPrefService::SavePrefFileInternal modules/libpref/src/nsPrefService.cpp:414 5 xul.dll NS_InvokeByIndex_P xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp:102 422 nsresult nsPrefService::WritePrefFile(nsIFile* aFile) 423 { ... 448 PRUint32 writeAmount; 449 nsresult rv; 450 451 if (!gHashTable.ops) 452 return NS_ERROR_NOT_INITIALIZED; 454 // execute a "safe" save by saving through a tempfile 455 rv = NS_NewSafeLocalFileOutputStream(getter_AddRefs(outStreamSink), ... 461 rv = NS_NewBufferedOutputStream(getter_AddRefs(outStream), outStreamSink, 4096); ... 479 // write out the file header 480 outStream->Write(outHeader, sizeof(outHeader) - 1, &writeAmount); the above line did not crash. 483 for (PRUint32 valueIdx = 0; valueIdx < gHashTable.entryCount; valueIdx++, walker++) { 484 if (*walker) { 485 outStream->Write(*walker, strlen(*walker), &writeAmount); hg@1 525 nsBufferedOutputStream::Write(const char *buf, PRUint32 count, PRUint32 *result) 526 { 527 nsresult rv = NS_OK; 528 PRUint32 written = 0; 529 while (count > 0) { 530 PRUint32 amt = PR_MIN(count, mBufferSize - mCursor); 531 if (amt > 0) { 532 memcpy(mBuffer + mCursor, buf + written, amt); 533 written += amt; 534 count -= amt; 535 mCursor += amt; 536 if (mFillPoint < mCursor) 537 mFillPoint = mCursor; 538 } 539 else { 540 NS_ASSERTION(mFillPoint, "iloop in nsBufferedOutputStream::Write!"); 541 rv = Flush(); here we call Flush 550 nsBufferedOutputStream::Flush() 551{ 552 nsresult rv; 553 PRUint32 amt; "amt" is a stack variable 554 if (!mStream) { 555 // Stream already cancelled/flushed; probably because of error. 556 return NS_OK; 557 } here we get a reference to the stack variable and pass it as the last argument 558 rv = Sink()->Write(mBuffer, mFillPoint, &amt); this function hasn't changed, 413 nsFileOutputStream::Write(const char *buf, PRUint32 count, PRUint32 *result) 414 { 415 if (mFD == nsnull) 416 return NS_BASE_STREAM_CLOSED; 417 418 PRInt32 cnt = PR_Write(mFD, buf, count); 419 if (cnt == -1) { 420 return NS_ErrorAccordingToNSPR(); 421 } 422 *result = cnt; we're crashing here, this time result or some mathematical operation on result is 0x1e (0+30). Again, we're talking about damage from something being evil inside file i/o. The last variant is a non null pointer. Only visible in the 4.0b series Signature nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) UUID d60644f5-bf55-4af1-80bd-e9a082101001 Time 2010-10-01 11:58:41.445602 Uptime 0 Last Crash 4300 seconds (1.2 hours) before submission Install Age 809487 seconds (1.3 weeks) since version was first installed. Product Firefox Version 4.0b6 Build ID 20100914083612 Branch 2.0 OS Windows NT OS Version 5.1.2600 Service Pack 3 CPU x86 CPU Info AuthenticAMD family 16 model 5 stepping 2 Crash Reason EXCEPTION_ACCESS_VIOLATION_WRITE Crash Address 0x10079027 User Comments App Notes AdapterVendorID: 10de, AdapterDeviceID: 0615 Processor Notes EMCheckCompatibility False Related Bugs Crashing Thread Frame Module Signature [Expand] Source 0 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:422 1 xul.dll nsStringInputStream::ReadSegments xpcom/io/nsStringStream.cpp:278 2 nspr4.dll nspr4.dll@0x1b3af 3 xul.dll nsAStreamCopier::Process xpcom/io/nsStreamUtils.cpp:323 nsAStreamCopier:: 301 void Process() 302 { ... 306 nsresult sourceCondition, sinkCondition; ... 323 PRUint32 n = DoCopy(&sourceCondition, &sinkCondition); this function hasn't changed, 532 PRUint32 DoCopy(nsresult *sourceCondition, nsresult *sinkCondition) ... 538 PRUint32 n; 539 *sourceCondition = 540 mSource->ReadSegments(ConsumeInputBuffer, &state, mChunkSize, &n); this function hasn't changed, 262 nsStringInputStream::ReadSegments(nsWriteSegmentFun writer, void *closure, 263 PRUint32 aCount, PRUint32 *result) 264{ 265 NS_ASSERTION(result, "null ptr"); 266 NS_ASSERTION(mLength >= mOffset, "bad stream state"); 267 268 // We may be at end-of-file 269 PRUint32 maxCount = LengthRemaining(); 270 if (maxCount == 0) { 271 *result = 0; 272 return NS_OK; 273 } 274 NS_ASSERTION(mData, "must have data if maxCount != 0"); 275 276 if (aCount > maxCount) 277 aCount = maxCount; 278 nsresult rv = writer(this, closure, mData + mOffset, 0, aCount, result); we don't see writer in the stack trace, but we know where it goes: this function hasn't changed, 413 nsFileOutputStream::Write(const char *buf, PRUint32 count, PRUint32 *result) 414 { 415 if (mFD == nsnull) 416 return NS_BASE_STREAM_CLOSED; 417 418 PRInt32 cnt = PR_Write(mFD, buf, count); 419 if (cnt == -1) { 420 return NS_ErrorAccordingToNSPR(); 421 } 422 *result = cnt; Again, we had a perfectly reasonable stack variable, and someone did serious damage to it. Oddly this time instead of making it 0, they seem to have made it something else: 0x10079027 (a browse through crash stats shows that there's generally one distinct value for each crashing version, which is a bit odd, i'll admit). What's interesting about this value is that it's odd. Integer pointers really shouldn't be odd. From memory 0x1...... is also not really a range where stack variables should be on Windows (for reference, the crash in question is XP, which means that ASLR shouldn't be confounding my understanding of address space layout) bp-d60644f5-bf55-4af1-80bd-e9a082101001 Firefox 4.0b6 20100914083612 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x10079027 0 bp-873f814b-7b72-40b2-9655-517302101001 Firefox 4.0b4 20100818132640 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x10011f67 0 bp-3a160d62-66e8-41f9-b64c-007052101001 Firefox 4.0b3 20100804193205 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x1005015e 8 bp-0b6be18e-f55b-4ced-aa05-78bf42101001 Firefox 4.0b2 20100720190347 Windows NT 5.2.3790 Service Pack 2 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x101303b0 0 bp-92bcb70d-bb38-4458-b5f0-6048a2101001 Firefox 3.6.10 20100914125854 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x10093afb 0 bp-8b8b4efc-bd3c-4788-ade0-13cea2101001 Firefox 3.6.9 20100824153629 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x10066b5b 1 bp-91180c73-629f-4442-8d9a-824842101001 Firefox 3.6.8 20100722155716 Windows NT 5.1.2600 Service Pack 2 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x10191cdb 0 bp-6834fba1-02f8-46ee-8f41-071ad2101001 Firefox 3.6.6 20100625231939 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x1009f9fb 1 bp-f3c69eb9-797b-45f5-a27d-5ec862101001 Firefox 3.6.3 20100401080539 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x1099b874 0 There's one outlier here: bp-30b72ec9-4677-4e0d-8c63-2f81d2101001 Firefox 3.6.3 20100401080539 Windows NT 5.1.2600 Service Pack 3 x86 EXCEPTION_ACCESS_VIOLATION_WRITE 0x100a5dea 2 почему не работает мазила! the crashing address isn't consistent with everyone else using 3.6.3 (the message seems to be "why does it not work" or something) -- the stack is a compressed version of the one examined above, nothing special. There's another bug somewhere else where this sort of io crash is blamed on a virus. I would definitely lump this into that same bucket as it fits. Note: Virus, not buggy LSP, we're doing disk io, not network io.
This stack is showing up as #22 top crash in FF 4 Beta 8. There are several addons which are correlated to that crash. 76% (131/172) vs. 0% (134/29957) {6f6e09f6-35f8-4f8a-8bd8-e61e073f8bce} 99% (171/172) vs. 26% (7689/29957) jqs@sun.com (Java Quick Starter, http://java.sun.com/javase/downloads/) 76% (131/172) vs. 16% (4869/29957) engine@conduit.com 22% (38/172) vs. 1% (173/29957) quickstores@quickstores.de 100% (172/172) vs. 89% (26676/29957) testpilot@labs.mozilla.com (Mozilla Labs - Test Pilot, https://addons.mozilla.org/addon/13661) 100% (172/172) vs. 92% (27700/29957) {972ce4c6-7e08-4474-a285-3208198ce6fd} (Default, https://addons.mozilla.org/addon/8150) Perhaps the crashes are related to people using the conduit apps?
Crash Signature: [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ]
No thunderbird crashes starting at v3.1.10/3.1.11
Still showing crashes in FF 9.0b4 but very low volume ie: 6 crashes. We have 150 on 8.0 in 4 weeks. Removing top crash keyword.
Keywords: topcrash
It's #8 top crasher in 10.0b5. 84% of crashes happen within one minute.
Whiteboard: [crashkill][tbird crash] → [crashkill][tbird crash], startupcrash
Is there no one who can utilize timeless' excellent comment 7? Are most of these indeed virus? (In reply to Wayne Mery (:wsmwk) from comment #7) > No thunderbird crashes starting at v3.1.10/3.1.11 hmm. apparently still happening in thunderbird, but rare. 1 crash in 1 month bp-21afeca9-27fc-4005-b325-a4c572120119 version 9.01
In the last two days this has been rising very significantly on all versions (well, not trunk, it seems, but Aurora, Beta, 9 Release as well as 7 and 8 Release). On Firefox 9.* it went from ~1 crash per million ADU to 137 crashes / M ADU in yesterday's data (~10k crashes on a single day after considering throttling), it's #18 there now and therefore I'm setting it as a topcrash. On 10.* it's #4 in yesterday's data with 242 crashes / M ADU and in 11.* it's #5 with 368 crashes / M ADU. Given where we are in the cycle, I'm requesting tracking for 10 and 11. I saw though that bug 671468 has been rising in the last two days as well and those signatures seems fairly similar, so could be connected.
Keywords: topcrash
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #11) > In the last two days this has been rising very significantly on all versions > (well, not trunk, it seems, but Aurora, Beta, 9 Release as well as 7 and 8 > Release). Before sending this to engineering for further investigation, can we get a correlation report that will hopefully shed some light on the cause? QA may then be able to reproduce.
Here are some correlations from 9.0.1 - will have to ask for custom report for 10.0 data. nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)|EXCEPTION_ACCESS_VIOLATION_READ (203 crashes) 14% (29/203) vs. 6% (8641/139017) yasearch@yandex.ru (Yandex.Bar, https://addons.mozilla.org/addon/3495) nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)|EXCEPTION_ACCESS_VIOLATION_READ (203 crashes) 95% (192/203) vs. 45% (62526/139017) iphlpapi.dll 95% (192/203) vs. 48% (66701/139017) hnetcfg.dll 95% (192/203) vs. 49% (67546/139017) wshtcpip.dll 94% (191/203) vs. 52% (72772/139017) comres.dll 95% (192/203) vs. 54% (74977/139017) ws2help.dll 68% (138/203) vs. 30% (41663/139017) MSCTF.dll 66% (133/203) vs. 40% (55877/139017) lz32.dll 67% (135/203) vs. 42% (58750/139017) xpsp2res.dll 100% (203/203) vs. 77% (106355/139017) softokn3.dll 100% (203/203) vs. 77% (107171/139017) firefox.exe 100% (203/203) vs. 77% (107339/139017) xpcom.dll 64% (129/203) vs. 41% (56999/139017) wldap32.dll 100% (203/203) vs. 78% (107901/139017) dbghelp.dll 98% (198/203) vs. 78% (108118/139017) wininet.dll 89% (180/203) vs. 72% (100222/139017) browsercomps.dll 34% (69/203) vs. 19% (26869/139017) MSCTFIME.IME 100% (204/203) vs. 91% (126454/139017) mswsock.dll 100% (203/203) vs. 91% (126525/139017) setupapi.dll 68% (138/203) vs. 60% (83530/139017) imagehlp.dll 67% (137/203) vs. 60% (83395/139017) t2embed.dll 8% (16/203) vs. 1% (1095/139017) imon.dll 6% (13/203) vs. 1% (780/139017) pr_imon.dll
Bug 721226 has been requested for the 10 correlations.
(In reply to Marcia Knous [:marcia] from comment #14) > Here are some correlations from 9.0.1 - will have to ask for custom report > for 10.0 data. Hmmm nothing jumps out at me in this correlation report. Can we pull out the URLs and test a few of them to see if we can reproduce?
Adding keyword to get an updated list of URLs for potential testing.
Keywords: needURLs
Here are some more URLs: 2384 771 jar:file:///C:/Program%20Files/Mozilla%20Firefox/omni.jar!/chrome/browser/content/browser/aboutSessionRestore.xhtml 583 \N 489 about:blank 284 jar:file:///C:/Program%20Files/Mozilla%20Firefox/omni.jar!/chrome/browser/content/browser/aboutHome.xhtml 227 about:sessionrestore 47 about:home 16 http://www.searchqu.com/406 14 http://id-id.facebook.com/ 12 http://www.google.co.id/ 12 http://www.facebook.com/media/set/?set=a.301034413269351.66863.100000883855701&type=3 10 https://www.facebook.com/login.php?login_attempt=1 10 https://www.facebook.com/ 9 http://www.ask.com/?l=dis&o=15187 9 http://en-gb.facebook.com/ 8 http://en-us.start3.mozilla.com/firefox?client=firefox-a&rls=org.mozilla:en-US:official 7 http://www.facebook.com/photo.php?fbid=149711168392988&set=t.100001627861491&type=3&theater 6 http://search.imesh.com/ 5 http://www.google.co.id/firefox?client=firefox-a&rls=org.mozilla:en-US:official 4 http://www.google.pl/ 4 http://www.google.co.id/search?q=fb&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a&source=hp&channel=np 4 http://search.livetool.ru/?from=5 4 http://search.conduit.com/?ctid=CT1572363&SearchSource=13 3 http://www.smadav.net/ 3 http://www.google.co.th/ 3 http://www.google.co.in/ 3 http://translate.google.co.id/?hl=id&tab=wT 3 https://www.facebook.com/profile.php 3 http://search.bearshare.com/ 3 http://search.babylon.com/?babsrc=HP_Prot 3 http://id.search.yahoo.com/firefox/?fr=yff80-sfp 3 http://home.sweetim.com/ 3 http://apps.facebook.com/chatrounds/?ref=bookmarks&count=0&fb_source=bookmarks_apps&fb_bmpos=1_0 2 http://www.ownskin.com/theme_search.oss?tm=324 2 http://www.mail.ru/cnt/ 2 http://www.google.com.my/firefox?client=firefox-a&rls=org.mozilla:en-US:official 2 http://www.google.co.id/search?q=facebook&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a&source=hp&channel=np 2 http://www.facebook.com/?sk=h_chr 2 http://www.dolancer.com/ads/visitad/785 2 http://www.ask.com/?l=dis&o=100000024&gct=hp 2 https://www.facebook.com/login.php 2 https://www.facebook.com/checkpoint/ 2 https://www.cimbclicks.co.id/ib-cimbniaga/Homepage.html 2 https://twitter.com/#!/
Keywords: needURLs
those urls probably confirm that this is still a start up/restart crash as mentioned in comment 1
Josh - this bug doesn't yet appear actionable from the engineering side (no STR, etc.), but we'd really appreciate help from someone on your team about how to further the investigation of this top startup crasher.
Assignee: nobody → joshmoz
We have over 3000 of these across all versions. About 74% of these happen < 1 min. It's not particularly high on any version in particular. Still worth investigating since it's a startup crash.
Assignee: joshmoz → michal.novotny
It's not very high volume. It's down below #200 on both 10.0.1 and 10.0.2. I don't see it on Thunderbird in the last week. Maybe there is some spike but appears to be gone. Removing the top crash keyword and removing tracking 10 flag.
(In reply to Sheila Mooney from comment #22) > It's not very high volume. It's down below #200 on both 10.0.1 and 10.0.2. That's the fate of startup crashes. Once the upgrade process almost done, users who hit that and didn't find a workaround switched to a previous Firefox version or another browser.
(In reply to Scoobidiver from comment #23) > (In reply to Sheila Mooney from comment #22) > > It's not very high volume. It's down below #200 on both 10.0.1 and 10.0.2. > That's the fate of startup crashes. Once the upgrade process almost done, > users who hit that and didn't find a workaround switched to a previous > Firefox version or another browser. You continue to state that in multiple places, but this doesn't entirely match what we are seeing. If this was the case, all startup crashes would need to fade out and come down to zero eventually, but the total volume of startup crashes stays relatively constant on all channels, even across version jumps. I'm pretty sure people actually do update to newer versions manually eventually.
The download is in the list of workarounds a normal user will try, but I don't think for this networking crash it will fix something as it's probably related to third party software. Bug 294260 that landed in Fx 13 will help users troubleshoot startup crashes.
I don't see any reason to track this anymore.
It's #30 top crasher in 11.0b4 and #16 in 12.0a2.
Keywords: topcrash
I have a very similar crashes in some of my code: https://bugzilla.mozilla.org/show_bug.cgi?id=721196 Startup crash, having an unexplainable access violation in a simple PR_Write.
This is spiking once again, #6 on both 11.* and 12.*, #10 on 13.* in yesterday's data. nsDiskCacheBlockFile::Write(int, void const*, int) and nsSocketOutputStream::Write(char const*, unsigned int, unsigned int*) seem to spike along with it, I suspect some connection.
This stays constantly at high ranks. #10 in yesterday's data for 12.* Any chance someone can find out what's going on there?
With combined signatures, it's #16 top browser crasher in 13.0 and #13 in 14.0b6. The first frames of the stack are various: Frame Module Signature Source 0 @0x242e760 1 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:647 2 xul.dll nsSafeFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:831 3 xul.dll nsBufferedOutputStream::Flush netwerk/base/src/nsBufferedStreams.cpp:638 4 xul.dll nsBufferedOutputStream::Write netwerk/base/src/nsBufferedStreams.cpp:621 5 xul.dll mozilla::Preferences::WritePrefFile modules/libpref/src/Preferences.cpp:769 6 xul.dll mozilla::Preferences::SavePrefFileInternal modules/libpref/src/Preferences.cpp:699 7 xul.dll mozilla::Preferences::SavePrefFile modules/libpref/src/Preferences.cpp:457 8 xul.dll nsAppStartup::TrackStartupCrashBegin toolkit/components/startup/nsAppStartup.cpp:927 Frame Module Signature Source 0 @0x9b72e600 1 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:647 2 xul.dll nsStreamCopierIB::ConsumeInputBuffer xpcom/io/nsStreamUtils.cpp:519 3 xul.dll nsStringInputStream::ReadSegments xpcom/io/nsStringStream.cpp:261 4 nspr4.dll nspr4.dll@0x2a3f 5 xul.dll nsAStreamCopier::Process xpcom/io/nsStreamUtils.cpp:319 6 xul.dll nsAStreamCopier::Run xpcom/io/nsStreamUtils.cpp:435 7 xul.dll nsThreadPool::Run xpcom/threads/nsThreadPool.cpp:217 8 xul.dll nsThread::ProcessNextEvent xpcom/threads/nsThread.cpp:657 9 xul.dll nsThread::ThreadFunc xpcom/threads/nsThread.cpp:289 Frame Module Signature Source 0 @0x16a637 1 @0x15cad9 2 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:647 3 xul.dll nsSafeFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:831 4 xul.dll nsBufferedOutputStream::Flush netwerk/base/src/nsBufferedStreams.cpp:638 5 xul.dll nsBufferedOutputStream::Write netwerk/base/src/nsBufferedStreams.cpp:621 6 xul.dll rdf_BlockingWrite rdf/base/src/nsRDFXMLSerializer.cpp:197 7 xul.dll nsRDFXMLSerializer::SerializeDescription rdf/base/src/nsRDFXMLSerializer.cpp:638 8 xul.dll nsRDFXMLSerializer::Serialize rdf/base/src/nsRDFXMLSerializer.cpp:1136 9 xul.dll RDFXMLDataSourceImpl::Serialize rdf/base/src/nsRDFXMLDataSource.cpp:1216 10 xul.dll RDFXMLDataSourceImpl::rdfXMLFlush rdf/base/src/nsRDFXMLDataSource.cpp:803 11 xul.dll RDFXMLDataSourceImpl::Flush rdf/base/src/nsRDFXMLDataSource.cpp:866 12 xul.dll LocalStoreImpl::Flush rdf/datasource/src/nsLocalStore.cpp:316 13 xul.dll nsXULDocument::~nsXULDocument content/xul/document/src/nsXULDocument.cpp:278 Frame Module Signature Source 0 @0xf348e600 1 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:647 2 xul.dll nsBufferedOutputStream::Flush netwerk/base/src/nsBufferedStreams.cpp:638 3 xul.dll nsBufferedStream::Seek netwerk/base/src/nsBufferedStreams.cpp:203 4 xul.dll nsZipWriter::Close modules/libjar/zipwriter/src/nsZipWriter.cpp:721 5 xul.dll mozilla::scache::StartupCache::WriteToDisk startupcache/StartupCache.cpp:495 6 xul.dll mozilla::scache::StartupCache::ThreadedWrite startupcache/StartupCache.cpp:533 Frame Module Signature Source 0 @0x5ef007b 1 @0x5ee5178 2 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:647 3 xul.dll nsInputStreamTee::TeeSegment xpcom/io/nsInputStreamTee.cpp:199 4 xul.dll nsInputStreamTee::Read xpcom/io/nsInputStreamTee.cpp:262 5 xul.dll nsNPAPIPluginStreamListener::OnDataAvailable dom/plugins/base/nsNPAPIPluginStreamListener.cpp:542 6 xul.dll nsPluginStreamListenerPeer::OnDataAvailable dom/plugins/base/nsPluginStreamListenerPeer.cpp:966 7 xul.dll nsHTTPCompressConv::do_OnDataAvailable netwerk/streamconv/converters/nsHTTPCompressConv.cpp:378 8 xul.dll nsHTTPCompressConv::OnDataAvailable netwerk/streamconv/converters/nsHTTPCompressConv.cpp:322 9 xul.dll nsStreamListenerTee::OnDataAvailable netwerk/base/src/nsStreamListenerTee.cpp:122 10 xul.dll nsHttpChannel::OnDataAvailable netwerk/protocol/http/nsHttpChannel.cpp:4478 11 xul.dll nsInputStreamPump::OnStateTransfer netwerk/base/src/nsInputStreamPump.cpp:514 12 xul.dll nsInputStreamPump::OnInputStreamReady netwerk/base/src/nsInputStreamPump.cpp:402 Frame Module Signature Source 0 @0x8de9e600 1 xul.dll nsFileOutputStream::Write netwerk/base/src/nsFileStreams.cpp:647 2 xul.dll LocalStoreImpl::CreateLocalStore rdf/datasource/src/nsLocalStore.cpp:383 3 xul.dll LocalStoreImpl::LoadData rdf/datasource/src/nsLocalStore.cpp:416 4 xul.dll LocalStoreImpl::Init rdf/datasource/src/nsLocalStore.cpp:342 5 xul.dll NS_NewLocalStore rdf/datasource/src/nsLocalStore.cpp:259 6 xul.dll mozilla::GenericFactory::CreateInstance obj-firefox/xpcom/build/GenericFactory.cpp:48 7 xul.dll nsComponentManagerImpl::CreateInstanceByContractID xpcom/components/nsComponentManager.cpp:1064 8 xul.dll nsComponentManagerImpl::GetServiceByContractID xpcom/components/nsComponentManager.cpp:1466 9 xul.dll nsCOMPtr_base::assign_from_gs_contractid obj-firefox/xpcom/build/nsCOMPtr.cpp:132 10 xul.dll nsXULDocument::Init content/xul/document/src/nsXULDocument.cpp:1963 11 xul.dll NS_NewXULDocument content/xul/document/src/nsXULDocument.cpp:315 12 xul.dll CreateXULDocument layout/build/nsLayoutModule.cpp:524 13 xul.dll mozilla::GenericFactory::CreateInstance obj-firefox/xpcom/build/GenericFactory.cpp:48 More reports at: https://crash-stats.mozilla.com/report/list?signature=nsFileOutputStream%3A%3AWrite%28char+const*%2C+unsigned+int%2C+unsigned+int*%29 https://crash-stats.mozilla.com/report/list?signature=%400x0+|+nsFileOutputStream%3A%3AWrite%28char+const*%2C+unsigned+int%2C+unsigned+int*%29 https://crash-stats.mozilla.com/report/list?signature=PR_Write+|+nsFileOutputStream%3A%3AWrite%28char+const*%2C+unsigned+int%2C+unsigned+int*%29
Crash Signature: [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ] → [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ] [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] [@ @0x0 | nsFileOutputStream::Write(char const*, unsigned int unsigned int*)] [@ PR_Write | nsFileOutputStr…
Summary: Firefox Crash Report [@ nsFileOutputStream::Write(char const*, unsigned int, unsigned int*) ] → crash in nsFileOutputStream::Write
Whiteboard: [crashkill][tbird crash], startupcrash → [crashkill][tbird crash][startupcrash]
Version: 1.9.2 Branch → Trunk
Correlations for 14.0.1 so far: (nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)|EXCEPTION_ACCESS_VIOLATION_READ (346 crashes) 83% (287/346) vs. 10% (6643/66763) {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} (Adblock Plus, https://addons.mozilla.org/addon/1865) 52% (179/346) vs. 1% (769/66763) elemhidehelper@adblockplus.org (Adblock Plus: Element Hiding Helper, https://addons.mozilla.org/addon/4364) 52% (181/346) vs. 2% (1320/66763) {D4DD63FA-01E4-46a7-B6B1-EDAB7D6AD389} (Download Statusbar, https://addons.mozilla.org/addon/26) 49% (169/346) vs. 2% (1081/66763) {19503e42-ca3c-4c27-b1e2-9cdb2170ee34} (FlashGot, https://addons.mozilla.org/addon/220) 44% (151/346) vs. 1% (547/66763) {3d7eb24f-2740-49df-8937-200b1cc08f8a} (Flashblock, https://addons.mozilla.org/addon/433) 43% (149/346) vs. 2% (1054/66763) {1018e4d6-728f-4b20-ad56-37578a4de76b} (Flagfox, https://addons.mozilla.org/addon/5791) 40% (140/346) vs. 1% (343/66763) {cd617375-6743-4ee8-bac4-fbf10f35729e} (RightToClick, https://addons.mozilla.org/addon/12572) 40% (138/346) vs. 1% (335/66763) {582195F5-92E7-40a0-A127-DB71295901D7} (Gmail Manager, https://addons.mozilla.org/addon/1320) 40% (139/346) vs. 1% (867/66763) {dc572301-7619-498c-a57d-39143191b318} (Tab Mix Plus, https://addons.mozilla.org/addon/1122) 38% (132/346) vs. 2% (1067/66763) {a0d7ccb3-214d-498b-b4aa-0e8fda9a7bf7} (WOT, https://addons.mozilla.org/addon/3456) 33% (113/346) vs. 1% (536/66763) {a7c6cf7f-112c-4500-a7ea-39801a327e5f} (FireFTP, https://addons.mozilla.org/addon/684) 32% (109/346) vs. 1% (643/66763) {1BC9BA34-1EED-42ca-A505-6D2F1A935BBB} 30% (103/346) vs. 0% (324/66763) {54BB9F3F-07E5-486c-9B39-C7398B99391C} (Text Link, https://addons.mozilla.org/addon/1939) 29% (100/346) vs. 3% (1941/66763) {37964A3C-4EE8-47b1-8321-34DE2C39BA4D} 23% (80/346) vs. 1% (341/66763) smarterwiki@wikiatic.com (FastestFox - Browse Faster, https://addons.mozilla.org/addon/9825) 23% (79/346) vs. 0% (270/66763) SkipScreen@SkipScreen (SkipScreen, https://addons.mozilla.org/addon/11243) 24% (82/346) vs. 3% (2296/66763) yasearch@yandex.ru (Yandex.Bar, https://addons.mozilla.org/addon/3495) 22% (76/346) vs. 5% (3067/66763) jqs@sun.com (Java Quick Starter, http://java.sun.com/javase/downloads/) 14% (49/346) vs. 0% (301/66763) {77b819fa-95ad-4f2c-ac7c-486b356188a9} (IE Tab, https://addons.mozilla.org/addon/1419) 16% (54/346) vs. 2% (1406/66763) personas@christopher.beard (Personas, https://addons.mozilla.org/addon/10900) 16% (56/346) vs. 3% (1950/66763) {e4a8a97b-f2ed-450b-b12d-ee082ba24781} (Greasemonkey, https://addons.mozilla.org/addon/748) 13% (44/346) vs. 0% (105/66763) imglikeopera@imfo.ru (ImgLikeOpera, https://addons.mozilla.org/addon/1672) 8% (28/346) vs. 1% (711/66763) {B100D0FF-0001-8CE4-2790-AACE49B8AE35} 10% (35/346) vs. 3% (2220/66763) {20a82645-c095-46ed-80e3-08825760534b} (Microsoft .NET Framework Assistant, http://www.windowsclient.net/)
Crash Signature: unsigned int*)] [@ PR_Write | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] → unsigned int*)] [@ PR_Write | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] [@ nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)]
Crash Signature: unsigned int*)] [@ PR_Write | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] [@ nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)] → unsigned int*)] [@ PR_Write | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] [@ nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)] [@ @0x0 | nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)]
Keywords: topcrash
At this time, this isn't a topcrash on our releases or betas any more.
There's a spike in crashes in 21.0a2 but seems to be from two users.
no tbird crashes in past month newer than version 14. crashes stop in version 15
Whiteboard: [crashkill][tbird crash][startupcrash] → [crashkill][startupcrash]
Hmm, looks to me like it's more than two installations on 21.0a2, actually.
Crash Signature: , unsigned int*)] [@ PR_Write | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] [@ nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)] [@ @0x0 | nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)] → , unsigned int*)] [@ PR_Write | nsFileOutputStream::Write(char const*, unsigned int, unsigned int*)] [@ nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)] [@ @0x0 | nsFileStreamBase::Write(char const*, unsigned int, unsigned int*)] [@ …
This signature does not occur in current versions
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
actually there still are a few. But rare. bp-0d18ee07-88c3-414e-b365-539bb2160912 0 @0x9cc31cf 1 xul.dll nsFileStreamBase::Write(char const*, unsigned int, unsigned int*) netwerk/base/nsFileStreams.cpp:275 2 xul.dll nsAtomicFileOutputStream::Write(char const*, unsigned int, unsigned int*) netwerk/base/nsFileStreams.cpp:1066 3 xul.dll nsCheckSummedOutputStream::Write(char const*, unsigned int, unsigned int*) toolkit/components/url-classifier/nsCheckSummedOutputStream.cpp:56 4 xul.dll mozilla::safebrowsing::HashStore::WriteFile() toolkit/components/url-classifier/HashStore.cpp:835 5 xul.dll mozilla::safebrowsing::Classifier::ApplyTableUpdates(nsTArray<mozilla::safebrowsing::TableUpdate*>*, nsACString_internal const&) toolkit/components/url-classifier/Classifier.cpp:663 6 xul.dll mozilla::safebrowsing::Classifier::ApplyUpdates(nsTArray<mozilla::safebrowsing::TableUpdate*>*) toolkit/components/url-classifier/Classifier.cpp:311 7 xul.dll nsUrlClassifierDBServiceWorker::ApplyUpdate() toolkit/components/url-classifier/nsUrlClassifierDBService.cpp:597
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Summary: crash in nsFileOutputStream::Write → startup crash in nsFileOutputStream::Write
Setting [necko-active] tag unless Michal decides otherwise.
Whiteboard: [crashkill][startupcrash] → [crashkill][startupcrash][necko-active]
With a initial crash reported 6 years ago, it is probably a new bug. I would rather open a new bug instead of reusing this one.
Yes, let's close this one and move it in the new one.
Status: REOPENED → RESOLVED
Closed: 9 years ago8 years ago
Resolution: --- → WORKSFORME
See Also: → 1302343
You need to log in before you can comment on or make changes to this bug.