Closed Bug 955900 Opened 11 years ago Closed 9 years ago

crash in nsDNSRecord::GetNextAddr(unsigned short, mozilla::net::NetAddr*)

Categories

(Core :: Networking, defect)

x86
Windows XP
defect
Not set
critical

Tracking

()

RESOLVED FIXED
Tracking Status
firefox36 --- fixed
firefox37 + fixed
firefox38 + fixed
firefox39 + fixed

People

(Reporter: baffclan, Assigned: dragana)

References

Details

(Keywords: crash)

Crash Data

This bug was filed from the Socorro interface and is 
report bp-59aae1fd-aa8e-40a4-842d-08c952140101.
=============================================================

---
User Agent : Mozilla/5.0 (Windows NT 5.1; rv:26.0) Gecko/20100101 Firefox/26.0
gecko.buildID;20131205075310

--- 
Signature 	nsDNSRecord::GetNextAddr(unsigned short, mozilla::net::NetAddr*) More Reports Search
UUID 	59aae1fd-aa8e-40a4-842d-08c952140101
Date Processed	2014-01-01 09:06:20.739433
Uptime	165203
Last Crash	8141674 seconds before submission
Install Age 	871458 since version was first installed.
Install Time 	2013-12-22 07:00:59
Product 	Firefox
Version 	26.0
Build ID 	20131205075310
Release Channel 	release
OS 	Windows NT
OS Version 	5.1.2600 Service Pack 3
Build Architecture 	x86
Build Architecture Info 	GenuineIntel family 6 model 8 stepping 10 | 1
Crash Reason 	EXCEPTION_ACCESS_VIOLATION_READ
Crash Address 	0x70617269
User Comments 	
App Notes 	

AdapterVendorID: 0x102b, AdapterDeviceID: 0x0525, AdapterSubsysID: 2179102b, AdapterDriverVersion: 5.1.2001.0
D3D10 Layers? D3D10 Layers- D3D9 Layers? D3D9 Layers- 

Processor Notes 	sp-processor07_phx1_mozilla_com.19186:2012; HybridCrashProcessor
EMCheckCompatibility 	

True

Winsock LSP 	

AVSDA over [MSAFD Tcpip [TCP/IP]] : 2 : 1 :  
 AVSDA over [MSAFD Tcpip [UDP/IP]] : 2 : 2 : C:\Program Files\Avira\AntiVir Desktop\avsda.dll 
 AVSDA over [MSAFD Tcpip [TCP/IPv6]] : 2 : 1 : C:\Program Files\Avira\AntiVir Desktop\avsda.dll 
 AVSDA over [MSAFD Tcpip [UDP/IPv6]] : 2 : 2 : C:\Program Files\Avira\AntiVir Desktop\avsda.dll 
 MSAFD Tcpip [TCP/IP] : 2 : 1 : %SystemRoot%\system32\mswsock.dll 
 MSAFD Tcpip [UDP/IP] : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD Tcpip [RAW/IP] : 2 : 3 : %SystemRoot%\system32\mswsock.dll 
 RSVP UDP Service Provider : 6 : 2 : %SystemRoot%\system32\rsvpsp.dll 
 RSVP TCP Service Provider : 6 : 1 : %SystemRoot%\system32\rsvpsp.dll 
 MSAFD Tcpip [TCP/IPv6] : 2 : 1 : %SystemRoot%\system32\mswsock.dll 
 MSAFD Tcpip [UDP/IPv6] : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD Tcpip [RAW/IPv6] : 2 : 3 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{8A633F2C-F1CB-44E3-8BB9-41A70E13FAE7}] SEQPACKET 3 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{8A633F2C-F1CB-44E3-8BB9-41A70E13FAE7}] DATAGRAM 3 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{D977ED64-B8F5-4257-9116-72294DD96AF5}] SEQPACKET 9 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{D977ED64-B8F5-4257-9116-72294DD96AF5}] DATAGRAM 9 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{3418077F-9CF4-4B8F-AF96-D52F6319547B}] SEQPACKET 5 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{3418077F-9CF4-4B8F-AF96-D52F6319547B}] DATAGRAM 5 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{3E458F48-0A06-4C09-850C-B935B7A809A0}] SEQPACKET 8 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip6_{3E458F48-0A06-4C09-850C-B935B7A809A0}] DATAGRAM 8 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{8A633F2C-F1CB-44E3-8BB9-41A70E13FAE7}] SEQPACKET 4 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{8A633F2C-F1CB-44E3-8BB9-41A70E13FAE7}] DATAGRAM 4 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{D977ED64-B8F5-4257-9116-72294DD96AF5}] SEQPACKET 10 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{D977ED64-B8F5-4257-9116-72294DD96AF5}] DATAGRAM 10 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{3418077F-9CF4-4B8F-AF96-D52F6319547B}] SEQPACKET 0 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{3418077F-9CF4-4B8F-AF96-D52F6319547B}] DATAGRAM 0 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{EE75D7AF-6323-463D-8B9E-7A54A3B7920F}] SEQPACKET 1 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{EE75D7AF-6323-463D-8B9E-7A54A3B7920F}] DATAGRAM 1 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{4EB01A8F-6A65-4AB1-8CCE-6320213DAD07}] SEQPACKET 2 : 2 : 5 : %SystemRoot%\system32\mswsock.dll 
 MSAFD NetBIOS [\Device\NetBT_Tcpip_{4EB01A8F-6A65-4AB1-8CCE-6320213DAD07}] DATAGRAM 2 : 2 : 2 : %SystemRoot%\system32\mswsock.dll 
 AVSDA : 2 : 1 : C:\Program Files\Avira\AntiVir Desktop\avsda.dll

Adapter Vendor ID 	

0x102b

Adapter Device ID 	

0x0525

Total Virtual Memory 	

2147352576

Available Virtual Memory 	

1556508672

System Memory Use Percentage 	

88

Available Page File 	

418131968

Available Physical Memory 	

63930368

-- 
Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	nsDNSRecord::GetNextAddr(unsigned short,mozilla::net::NetAddr *) 	netwerk/dns/nsDNSService2.cpp
1 	xul.dll 	nsDNSRecord::HasMore(bool *) 	netwerk/dns/nsDNSService2.cpp
2 	xul.dll 	NS_InvokeByIndex 	xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp
3 	xul.dll 	XPC_WN_CallMethod(JSContext *,unsigned int,JS::Value *) 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp
4 		@0x62e23b8 	
5 		@0x13f5fe10 	
6 		@0x6481f81 	
7 	mozjs.dll 	EnterBaseline 	js/src/jit/BaselineJIT.cpp
8 	mozjs.dll 	js::jit::EnterBaselineAtBranch(JSContext *,js::StackFrame *,unsigned char *) 	js/src/jit/BaselineJIT.cpp
9 	mozjs.dll 	Interpret 	js/src/vm/Interpreter.cpp
10 	mozjs.dll 	js::Invoke(JSContext *,JS::Value const &,JS::Value const &,unsigned int,JS::Value *,JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
11 	mozjs.dll 	JS_CallFunctionValue(JSContext *,JSObject *,JS::Value,unsigned int,JS::Value *,JS::Value *) 	js/src/jsapi.cpp
12 	xul.dll 	nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS *,unsigned short,XPTMethodDescriptor const *,nsXPTCMiniVariant *) 	js/xpconnect/src/XPCWrappedJSClass.cpp
13 	xul.dll 	nsXPCWrappedJS::CallMethod(unsigned short,XPTMethodDescriptor const *,nsXPTCMiniVariant *) 	js/xpconnect/src/XPCWrappedJS.cpp
14 	xul.dll 	PrepareAndDispatch 	xpcom/reflect/xptcall/src/md/win32/xptcstubs.cpp
15 	xul.dll 	SharedStub 	xpcom/reflect/xptcall/src/md/win32/xptcstubs.cpp
16 	xul.dll 	`anonymous namespace'::DNSListenerProxy::OnLookupCompleteRunnable::Run() 	netwerk/dns/nsDNSService2.cpp
17 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
18 	xul.dll 	NS_ProcessNextEvent(nsIThread *,bool) 	xpcom/glue/nsThreadUtils.cpp
19 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate *) 	ipc/glue/MessagePump.cpp
Here are some notes from my initial analysis (some of these are for my own records).

-- Crashing on the main thread, somewhere in JS, in a DNS listener.
-- There are two crash types:
   -- 1. READ violation: GetNextAddr called from either GetNextAddrAsString or HasMore.
   -- 2. WRITE violation: GetNextAddr called from OnSocketEvent. This crash occurs much more infrequently, but it is likely related.
Note: I will focus on the READ violation traces for now.

-- There is a plugin correlation, but only the signature shows, not the name: it starts with 972ce4c6. I see this for all crashes on version 29 and 28. I see refs to this signature in mozilla-central, but I'm not sure what the package name is.

-- In mozilla-central, only gonk's NetworkManager.js calls getNextAddrAsString and hasMore (for hasMore, I think this is the only call related to an nsIDNSRecord). These calls are on adjacent lines - I'm wondering if this is a B2G emulator crash ... I'm not sure how NetworkManager is packaged with the desktop build, if it all.

-- The final line number for the crash (READ violation) happens on different, but nearby lines in GetNextAddr - I've put the code snippet below instead of line numbers:
   -- mHostRecord->addr_info_lock.Unlock() for all Fx29 crashes
   -- mIter = mIter->GetNext() for all Fx28 crashes, and many samples from the other versions.

Back to the WRITE violation:
-- Crashes occur in GetNextAddr at:
   -- memcpy(addr, &mIter->mAddress...)
   -- or mHostRecord->addr_info_lock.Lock() for Fx27.0
   -- while (!mIter && mHostRecord->Blacklisted(&mIter->mAddress)) for Fx 25.0.1

So, maybe the nsIDNSRecords/nsDNSRecord is being corrupted, since mIter and mHostRecord are members.
-- Is it a corruption that starts in Resolve/AsyncResolve?
-- Is there something happening on another thread?


Not sure how to reproduce this one, and I don't have a speculative fix yet. Also, the numbers are pretty low so far, so I won't be attending to this very urgently. But it's on my radar :)
(In reply to Steve Workman [:sworkman] from comment #1)
thanks for a comment.

I found a "972ce4c6" in about:config.
> extensions.{972ce4c6-7e08-4474-a285-3208198ce6fd}.description;The default theme.
> extensions.{972ce4c6-7e08-4474-a285-3208198ce6fd}.name;Default
I might have some input on reproducing this issue. I encountered this issue twice within a few minutes while working in Gmail. Specifically, I was rapidly going through and forwarding a bunch of emails.

The workflow was simple:
 * Open Gmail web interface
 * Open an email -> Select forward -> Enter recipient & quick addendum to body -> Send
 * Repeat the latter step
And another one, again while fiddling around in the Gmail web interface. I'm increasingly confident that Gmail is doing something conducive to surfacing this bug.
I won't update this ticket anymore unless requested to but just had a fourth crash, again while using Gmail. I should note I'm browsing/using numerous other sites, but am only seeing this crash occur while interacting with Gmail.
I’m not an expert at all, but just wanted to add my 2¢: Firefox is crashing with increasing frequency again for me, and this was one of them. I don’t ever use Gmail. The program seems to shut down by itself without warning—not sure what is causing it. 

This was my crash that led to this page:

https://crash-stats.mozilla.com/report/index/365380da-9d3d-4699-b45c-ebea72140820
Sorry, one more—this one crashed while I was browsing on Vimeo. It also led to this page.

https://crash-stats.mozilla.com/report/index/00859b1f-332c-4ede-ab49-f8cfa2140820
steve - any ideas here? seems to be uaf. soccoro shows this active on nightly; though at low volume.

is it possible something is triggered when the dns service is reinitted? (something that is going to happen a lot more when daniel's patches land)
Flags: needinfo?(sworkman)
Unsure about the reinit question: I don't don't DNS Service should be getting re-initted very much at the moment. The 'network.manage-offline-status' pref should be disabled by default, and it's the only one I know that would re-init during runtime. Maybe a plugin/extension is re-initting it?

I poked around in the code again, and I'm wondering if DNSListenerProxy has something to do with it. I'll keep poking...
Flags: needinfo?(sworkman)
Just chiming in to advise I'm still seeing these crashes on the latest release (v32.0).

Most recent crash from a few moments ago:
https://crash-stats.mozilla.com/report/index/c70b426a-351f-4a41-b6b6-175ab2140911
This is definitely not scientific, but I'm fairly sure I witness this bug far more on slow connections. Every crash I've witnessed has been on a relatively slow connection. I've never seen this crash on my desktop at home, while of all the work places I have witnessed this, the one with the slowest connection has the vast majority of crashes witnessed. I wonder if slower connections result in circumstances more likely to reproduce this bug...

Just an observation that may be helpful.
I've found bug 1132358 which may be related to this issue. Please have a look.
(In reply to Steve Workman [:sworkman] (please use needinfo) from comment #10)
> I poked around in the code again, and I'm wondering if DNSListenerProxy has
> something to do with it. I'll keep poking...

Steve, are you still looking at this bug? I'm seeing this signature on the top-crash lists of various channels.
Flags: needinfo?(sworkman)
If Steve does mind I can take it over and look at it.
Bug 1132358 fixed some of this crashes.
Very happy for Dragana to take this one :) Thanks Dragana!
Flags: needinfo?(sworkman)
Assignee: nobody → dd.mozilla
Tracking this as it's a topcrash, happy to see it's assigned.
Bug 1132358 fixed this.
I do not see any crashes on 39, but we can wait some days. 
There are same crashes with build from 2015/02/23 but the patch from bug 1132358 is still not in. So probably it shipped with 24th Nightly.
(In reply to Dragana Damjanovic [:dragana] from comment #18)
> Bug 1132358 fixed this.
> I do not see any crashes on 39, but we can wait some days. 
> There are same crashes with build from 2015/02/23 but the patch from bug
> 1132358 is still not in. So probably it shipped with 24th Nightly.

If this is the case, 38 and 39 should both be fixed. Please do follow up to ensure that bug 1132358 has fixed this issue.
Depends on: 1132358
Looking at crash reports, there is non reports with version 39 and 38 after 23.2.2015.
So this is fixed with bug 1132358.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Bug 1132358 has been uplifted to 36 and 37 so both releases should be fixed as well.
You need to log in before you can comment on or make changes to this bug.