601089 - spike in 4.0b7pre of crash under Windows XP mainly on start-up [@ nsStringInputStream::ReadSegments(unsigned int (*)(nsIInputStream*, void*, char const*, unsigned int, unsigned int, unsigned int*), void*, unsigned int, unsigned int*) ]

Status: RESOLVED DUPLICATE of bug 467167

(Core :: Networking: HTTP, defect)

Description

[Scoobidiver (away)]

Build : Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b7pre) Gecko/20100930 Firefox/4.0b7pre

This is a new crash signature that exists in trunk build at a low daily rate. In the today's build, there is a spike in crashes :
2010093000 	58
2010092900 	2 
It happens on Windows XP mainly on start-up.
It is #1 top crasher for this build.

Signature	nsStringInputStream::ReadSegments(unsigned int (*)(nsIInputStream*, void*, char const*, unsigned int, unsigned int, unsigned int*), void*, unsigned int, unsigned int*)
UUID	65e69256-f4f0-4d51-9e5c-cbe112101001
Time 	2010-10-01 01:05:26.539052
Uptime	7
Last Crash	8 seconds before submission
Install Age	41212 seconds (11.4 hours) since version was first installed.
Product	Firefox
Version	4.0b7pre
Build ID	20100930041305
Branch	2.0
OS	Windows NT
OS Version	5.1.2600 Service Pack 2
CPU	x86
CPU Info	GenuineIntel family 6 model 13 stepping 6
Crash Reason	EXCEPTION_ACCESS_VIOLATION_READ
Crash Address	0x1
App Notes 	AdapterVendorID: 0000, AdapterDeviceID: 0000

Frame 	Module 	Signature [Expand] 	Source
0 	xul.dll 	nsStringInputStream::ReadSegments 	xpcom/io/nsStringStream.cpp:282
1 	xul.dll 	nsHttpTransaction::ReadSegments 	netwerk/protocol/http/nsHttpTransaction.cpp:472
2 	xul.dll 	nsHttpConnection::OnSocketWritable 	netwerk/protocol/http/nsHttpConnection.cpp:578

More reports at :
http://crash-stats.mozilla.com/report/list?product=Firefox&query_search=signature&query_type=exact&query=&range_value=1&range_unit=weeks&hang_type=any&process_type=any&plugin_field=&plugin_query_type=&plugin_query=&do_query=1&admin=&signature=nsStringInputStream%3A%3AReadSegments%28unsigned%20int%20%28*%29%28nsIInputStream*%2C%20void*%2C%20char%20const*%2C%20unsigned%20int%2C%20unsigned%20int%2C%20unsigned%20int*%29%2C%20void*%2C%20unsigned%20int%2C%20unsigned%20int*%29

Comment 1

[Scoobidiver (away)]

The regression range for the spike is :
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=a60414d076b5&tochange=5a2012482a63
It could be bug 596476 that cause this spike.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Every single one of the crashes is at address 0x1 with ACCESS_VIOLATION_READ and the claimed crash location is line 282 here:

278 nsresult rv = writer(this, closure, mData + mOffset, 0, aCount, result);
279 if (NS_SUCCEEDED(rv)) {
280   NS_ASSERTION(*result <= aCount,
281                "writer should not write more than we asked it to write");
282   mOffset += *result;
283 } 

Which suggests that |result| is likely 0x1 at this point???  That looks really bad.

Comment 3

[Boris Zbarsky [:bzbarsky]]

In particular, it's clearly passed in, a few stack frames up, as a pointer to an int on the stack.  So someone must have clobbered that value somehow...

Comment 4

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Based on the crash times, I think the bulk, if not all, of the spike is a single user submitting a lot of crash reports.

Comment 5

[Jason Duell]

It would certainly be weird if this were related to bug 596476:  that code should really be unrelated.  But there's plenty of weirdness with necko crashes these days, so I can't rule it out.  I doubt the answer is backing it out.

Comment 6

[chris hofmann]

the step up in volume follow a strange pattern across releases and builds.

20100925 32  28 4.0b62010091408, 
	     3 4.0b52010083108, 1 3.6.102010091412, 
20100926 28  23 4.0b62010091408, 
	     3 4.0b52010083108, 2 3.0.182010020220, 
20100927 19  17 4.0b62010091408, 
	     2 4.0b52010083108, 
20100928 65  63 4.0b62010091408, 
	     2 4.0b52010083108, 
20100929 33  29 4.0b62010091408, 
	     4 4.0b52010083108, 
20100930 59  52 4.0b62010091408, 
	     3 4.0b7pre2010093004, 2 4.0b7pre2010092904, 
	     2 4.0b52010083108, 
20101001 82  59 4.0b7pre2010093004, 
	     16 4.0b62010091408, 6 4.0b52010083108, 
	     1 4.0b7pre2010092904,

Comment 7

[[e-mail]]

OK. So crash-stats has data back to around 7/18 (crash date, not build date). Which was 4.0b3pre.

We have one crash with the same stack (but different crash address -- not 0x1!) on b3; zero on b4; and 967 on b5. Most, if not all, of those on b5 have a crash address of 0x1. There are zero nightly crashes in b5pre or earlier.

So let's assume the b3 crash is something else; then we have a regression range of "somewhere in b5". That's big. :/

Comment 8

[[e-mail]]

b3 was 20100817, b4 was 20100831. I'm going to look at all changes in netwerk/ and xpcom/io during that timeframe...

Comment 9

[[e-mail]]

OK. So I'm basically positive that this shares the same cause as bug 467167. The fact that this crash occurs because of a stackstomp upon return from the socket write, rather than in the socket write itself, is incidental. The factor that this is also WinXP-only is also consistent with the other bug (see details there on why that's relevant).

The worrisome thing is that we have, for the other bug, 2400 crashes in _PR_MD_SEND in the past week on 2.0, and 8050 on 1.9.2. For this sig, we have 530 on 2.0 and 6 on 1.9.2. So either some form of malware is specifically targeting 2.0, or something's changed with our blocklisting. The only difference in nsWindowsDllBlocklist.h between 1.9.2 and 2.0 is the addition of rlxf.dll (http://hg.mozilla.org/mozilla-central/rev/dffacca467b7), which obviously shouldn't *increase* the number of crashes.

I'm going to look through and see a) if there's a dll common to most/all of the crashes here; b) if there have been any changes to our blocklist implementation recently.

Comment 10

[[e-mail]]

Bug 611930 might reveal a culprit LSP. My module analysis so far hasn't turned up any commonalities, though KeySwitcher.dll (a keylogger trojan) was present in one signature.

Comment 12

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Not a firefox issue it sounds like, so blocking doesn't make sense

Comment 13

[Jason Duell]

resolving as dupe per comment 9