Closed Bug 480878 Opened 16 years ago Closed 13 years ago

Generic crash [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)], [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)], [@ nsSSLThread::requestRecvMsgPeek]

Categories

(Core Graveyard :: Security: UI, defect)

x86_64
All
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: pepper, Assigned: mayhemer)

References

Details

(Keywords: crash, Whiteboard: [CLOSEME 2012-03-20][psm-fatal][tbird topcrash])

Crash Data

Attachments

(1 file)

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1
Build Identifier: 2.0.0.19 (2.0.0.19)

No idea what caused this.

Reproducible: Didn't try

Steps to Reproduce:
No recipe, sorry.
Actual Results:  
Crash

Expected Results:  
No crash.

Log attached.
Attached file MOSX crashlog
Assignee: nobody → kaie
Severity: normal → critical
Component: General → Security: UI
Keywords: crash
Product: Thunderbird → Core
QA Contact: general → ui
Summary: Generic crash → Generic crash [@ __memcpy - nsSSLThread::requestRecvMsgPeek]
Version: unspecified → 1.8 Branch
Also seen with thunderbird trunk 3.0b2 20090223175111 and 3.0b3pre.  Haven't found any prior to 3.0b2.  Almost all crashes are after long periods, 2-10 hours

[@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int) ]
bp-f18e5ba9-19f1-45d7-9c65-2a7652090226

bp-a2845726-3398-4b88-9a16-747d62090321
0  	mozcrt19.dll  	memcpy  	memcpy.asm:348
1 	thunderbird.exe 	nsSSLThread::requestRecvMsgPeek 	security/manager/ssl/src/nsSSLThread.cpp:171
2 	thunderbird.exe 	PSMRecv 	security/manager/ssl/src/nsNSSIOLayer.cpp:1869
3 	thunderbird.exe 	nsSocketTransport::IsAlive 	netwerk/base/src/nsSocketTransport2.cpp:1808
4 	thunderbird.exe 	nsImapProtocol::CanHandleUrl 	nsImapProtocol.cpp:2008
5 	thunderbird.exe 	nsImapIncomingServer::GetImapConnection 	nsImapIncomingServer.cpp:705
This is a crash in PSM code.  Adding PSM folks to CC list.
Most likely the optimizer is optimizing 169/171 and 581/583
169           PRInt32 return_amount = NS_MIN(amount, si->mThreadData->mSSLResultRemainingBytes);
171           memcpy(buf, si->mThreadData->mSSLRemainingReadResultData, return_amount);

581         PRInt32 return_amount = NS_MIN(amount, si->mThreadData->mSSLResultRemainingBytes);
583         memcpy(buf, si->mThreadData->mSSLRemainingReadResultData, return_amount);

588           si->mThreadData->mSSLRemainingReadResultData = nsnull;

If I'm right, then 588 probably happened, and we're crashing reading a null pointer. Figuring out the legality of such a call sequence is up to someone else.
Timeless, 
Are you saying that you think this is a compiler optimization fault?

Is there anything about the lines at 
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/manager/ssl/src/nsSSLThread.cpp&rev=1.16&mark=169-173#169
that appears to be unsafe/improper code?

Likewise, I would ask the same questions for 
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/manager/ssl/src/nsSSLThread.cpp&rev=1.16&mark=581-588#580
no, i'm just explaining why it crashed at 171 instead of 169.

what's unsafe is that there isn't a null check. however i don't know the application logic (and don't want to figure it out), so i'm not certain if that null is supposed to be possible.
First, access to this property is not thread safe while it is accessed on multiple threads, check http://mxr.mozilla.org/mozilla1.8/ident?i=mSSLRemainingReadResultData for file nsSSLThread.cpp.

Second, we may probably experience call to memcpy before we assign  bstd.mSSLRemainingReadResultData = bstd.mSSLDataBuffer or before we call mThreadData->ensure_buffer_size. I have to walk all the code paths (lot of work) to try to figure out.

I never experienced this crash my self.
Honza, will you be able to fix those things?  If not, can you at least split them into separate bugs?
(In reply to comment #8)
> Honza, will you be able to fix those things?  If not, can you at least split
> them into separate bugs?

I have to analyze the code. I have some more important stuff at the moment, so I can get to this within few weeks. If this is more priority let me know please.
-> me

I'll take a look at this one.
Assignee: kaie → honzab.moz
Status: UNCONFIRMED → NEW
Ever confirmed: true
is #32 and climbing, so approaching topcrash status for thunderbird 3.0. Thunderbird signature is now memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)

in firefox it's quite rare, only 5 crashes in 2 weeks for all FF releases. eg bp-bc3514a3-0d55-428a-baf9-54c2c2091206 ... FF signature is still [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int) ]
Summary: Generic crash [@ __memcpy - nsSSLThread::requestRecvMsgPeek] → Generic crash [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)]
I had a core dump today in Thunderbid 2.0.0.23 with a very similar stack trace:

libc_hwcap1.so.1`_lwp_kill+0x15(1, b, 80461f8, fecaaffe)
libc_hwcap1.so.1`raise+0x22(b, 8046210, 0, 807117a)
void nsProfileLock::FatalSignalHandler(int)+0xf1(b, 0, 804630c, ffff, fdfb2a00, fed91000)
libc_hwcap1.so.1`__sighndlr+0x15(b, 0, 804630c, 807116c)
libc_hwcap1.so.1`call_user_handler+0x2af(b)
libc_hwcap1.so.1`sigacthandler+0xdf(b, 0, 804630c)
libc_hwcap1.so.1`memcpy+0x22(bce4400, 8046607, 1, 2, 0, 1)
libpipnss.so`int PSMRecv(PRFileDesc*,void*,int,int,unsigned)+0x8a(9e60508, 8046607, 1, 2, 0, feab6fdc)
libnspr4.so`PR_Recv+0x1d(9e60508, 8046607, 1, 2, 0, 804660c)
libnecko.so`unsigned nsSocketTransport::IsAlive(int*)+0xa1(b889a88, 8046664, 804673c, fc1717bd)
libmail.so`unsigned nsImapProtocol::CanHandleUrl(nsIImapUrl*,int*,int*)+0xc7(dfe75f0, bb87928, 8046794, 8046798)
libmail.so`unsigned nsImapIncomingServer::GetImapConnection(nsIEventQueue*,nsIImapUrl*,nsIImapProtocol**)+0x1fc(9554e70, 81109d0, bb87928, 804684c)
libmail.so`unsigned nsImapIncomingServer::GetImapConnectionAndLoadUrl(nsIEventQueue*,nsIImapUrl*,nsISupports*)+0x41(9554e70, 81109d0, bb87928, 0)
libmail.so`unsigned nsImapService::GetImapConnectionAndLoadUrl(nsIEventQueue*,nsIImapUrl*,nsISupports*,nsIURI**)+0x1b6(8b7d0a0, 81109d0, bb87928, 0, 8046a04, 8046938)
libmail.so`unsigned nsImapService::UpdateFolderStatus(nsIEventQueue*,nsIMsgFolder*,nsIUrlListener*,nsIURI**)+0x263(8b7d0a0, 81109d0, 8742d5c, 9554f0c, 8046a04, 8046a10)
libmail.so`unsigned nsImapMailFolder::UpdateStatus(nsIUrlListener*,nsIMsgWindow*)+0x90(8742d38, 9554f0c, 0, fc12b16a)
libmail.so`unsigned nsImapIncomingServer::OnStopRunningUrl(nsIURI*,unsigned)+0x170(9554e70, d405694, 0, fbf70e44)
libmail.so`unsigned nsUrlListenerManager::BroadcastChange(nsIURI*,nsUrlNotifyType,unsigned)+0x12b(ab1ac98, d405694, 1, 0)
libmail.so`unsigned nsUrlListenerManager::OnStopRunningUrl(nsIMsgMailNewsUrl*,unsigned)+0x25(ab1ac98, d405694, 0, fbf55ea6)
libmail.so`unsigned nsMsgMailNewsUrl::SetUrlState(int,unsigned)+0xc2(d405694, 0, 0, 0)
libmail.so`unsigned nsImapMailFolder::SetUrlState(nsIImapProtocol*,nsIMsgMailNewsUrl*,int,unsigned)+0xf1(8742d38, e00f558, d405694, 0, 0, b1f3178)
libxpcom_core.so`XPTC_InvokeByIndex+0x51(8742e3c, 1f, 4, bb8e200)
libxpcom_core.so`void*EventHandler(PLEvent*)+0x3c(d0511d0, 0, 8046c08, feba21a6)
libxpcom_core.so`PL_ProcessPendingEvents+0x17d(8255fc0, 81109d0, fec1a1a0, feba4be5)
libxpcom_core.so`unsigned nsEventQueueImpl::ProcessPendingEvents()+0xdd(81109d0, 0, 8046cb8, fe288310)
libwidget_gtk2.so`int event_processor_callback(_GIOChannel*,GIOCondition,void*)+0x18(8620158, 1, 81109d0, fe2caa82)
libglib-2.0.so.0.2200.0`g_io_unix_dispatch+0x3e(86201a0, fc836524, 81109d0, 8046d20)
libglib-2.0.so.0.2200.0`g_main_context_dispatch+0x262(80d0220, c8, a5a94e0, a)
libglib-2.0.so.0.2200.0`g_main_context_iterate+0x483(80d0220, 1, 1, 80ae678)
libglib-2.0.so.0.2200.0`g_main_loop_run+0x1dd(8620210, 8620210, 8046de0, fe57e726)
libgtk-x11-2.0.so.0.1800.0`gtk_main+0xb7(808c898, 84d3980, 8046e58, fc600c56, 84d5d48, fc7a5210)
libwidget_gtk2.so`unsigned nsAppShell::Run()+0x36(84d5d48, fc7a5210, 0, fc600c34)
libtoolkitcomps.so`unsigned nsAppStartup::Run()+0x2e(84d3980, 80746a4, 0, 8064c65)
XRE_main+0x381f(1, 804713c, 808c870, 805b028, 80470f8)
main+0x28(1, 804713c, 8047144, 8047130)
_start+0x7d(1, 8047314, 0, 807272c, 8072744, 80473b5)

This was on OpenSolaris (snv_126, x64). No obvious trigger but I think I've seen this before recently and it was associated with opening a new IMAP connection.
top 25 crash for TB3.0.0 combining windows and Mac crashes, so fixing this is very much wanted

bp-e328dce5-e002-4c98-8359-0a5d52091216
"Using a mailbox with an untrusted certificate. The according untrusted-certificate-message pops up everytime, TB tries to download E-Mails from this mailbox."

bp-6d1ef113-fc50-4adb-93f1-079312091213 (no extensions)

bp-4bbe1737-4b45-42a1-a0bc-daf7b2091222 every time sending a message
OS: Mac OS X → All
Version:         2.0.0.23 (2.0.0.23)
Code Type:       X86 (Native)
OS Version:      Mac OS X 10.6.2 (10C540)

I've hit this twice in the last couple of days. There was no obvious trigger (TB was in the background when it happened). I had tried TB3.0 recently but reverted back to TB2.0 due to stability issues which I couldn't track down (waiting for OS X 10.6.3 to try again with TB3.0.1). Previously I had been running TB2.0 for a long time without ever hitting it. The only extension I have is Nostalgy.
It would be interesting to know the parameter values of the function calls in the stack, it seems we don't have that?
I see "crash address" is being reported as 0x0.

Does this mean memcpy attempts to write to a destination buffer at address 0x0 ?

PSMRecv and nsSSLThread::requestRecvMsgPeek get "bug" as input, they don't change it.

Caller nsSocketTransport::IsAlive uses

   char c;
   PRInt32 rval = PR_Recv(fd, &c, 1, PR_MSG_PEEK, 0); 

How could &c ever become 0x0 ??
(In reply to comment #16)
> 
> PSMRecv and nsSSLThread::requestRecvMsgPeek get "bug" as input, they don't
> change it.

not "bug", I mean "buf"
si->mThreadData->mSSLRemainingReadResultData could be null, memcpy would crash.
Version: 1.8 Branch → 1.9.1 Branch
#30 crash for v3.0.3.  (counterintuitive but) it's bit unusual for us to see our top crashes on other branches, so citing...
bp-eace7a4e-a962-40e0-b06e-6d15a2100322 3.2a1pre
bp-64a3791a-aa3c-44a8-b6a3-5243d2100316 3.1b1

pmed 3.0.3 reporter bp-79baa7db-a87c-4413-91ef-951f82100317
a couple crashes mention thunderbird was idle (from their perspective)

bp-1abade11-b93a-4179-a28c-d3f522100207 (david)
bp-4b368ce3-3d7f-4f26-a2b1-3b4cd2100214

pmed also 
bp-28531f87-3806-4278-8b24-de9a72100226 (scott)
bp-0ee8e7e2-2ee3-48ac-8509-875bc2100405 (bjorn)
bp-46783f61-5dcc-4391-9086-8e2a12100406 (infos)
Summary: Generic crash [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] → Generic crash [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] and [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)]
Got an email asking for details about when the crash occurs.

The crash always occurs when I leave my computer locked over night. Thunderbird is running in the background along with a few other apps. I'm a software developer - for the first 10 hours after I leave each night the memory and CPU are maxed out building all my projects for the following days work. So there is likely lots of memory swapping if Thunderbird is trying to do anything.
Whiteboard: [psm-fatal]
Same here.  See https://bugzilla.redhat.com/show_bug.cgi?id=641422.  Seems simple - don't call memcpy if si->mThreadData->mSSLRemainingReadResultData is null (return_amount will be zero too).
from brc 641422 comment 2:
#3  nsSSLThread::requestRecvMsgPeek (si=0x7ff51c7ea700, buf=0x7fffd591c07f,
amount=1, flags=2, timeout=<value optimized out>) at
/usr/include/bits/string3.h:52
        threadLock = {<nsAutoLockBase> = {<No data fields>}, mLock =
0x7ff51c4a71c0, mLocked = 1}
        realSSLFD = <value optimized out>

Code:

          PRInt32 return_amount = NS_MIN(amount,
si->mThreadData->mSSLResultRemainingBytes);

          memcpy(buf, si->mThreadData->mSSLRemainingReadResultData,
return_amount);

(gdb) print amount
$2 = 1
(gdb) print si->mThreadData->mSSLResultRemainingBytes
$3 = 0
(gdb) print si->mThreadData->mSSLRemainingReadResultData
$1 = 0x0
topcrash for v3.1.7 so would be great to get this on branch.
semi frequent on trunk, eg bp-abe86663-f3aa-47a9-aafa-e38202110208 linux
linux signature nsSSLThread::requestRecvMsgPeek
Keywords: topcrash
Summary: Generic crash [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] and [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] → Generic crash [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)], [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)], [@ nsSSLThread::requestRecvMsgPeek]
Version: 1.9.1 Branch → Trunk
Honza , would you have more time to look into this now ?
Hopefully, if anyone wants to work on this immediately, feel free to take it and drop a note.
Crash Signature: [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek]
Crash Signature: [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek] → [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek]
Crashes involving nsSSLThread in it should be fixed by bug 674147, which removes that class (and thread) entirely.
Depends on: 674147
Whiteboard: [psm-fatal] → [psm-fatal][tbird topcrash]
On the desktop we have about 15 crashes with the signature = nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int) on all versions over the past 4 weeks. For Thunderbird, the volume is much higher. Leaving as a top crash for now.
Keywords: topcrash
indeed, for thunderbird this is ~top 15
recheck when version 11 hits the streets
Whiteboard: [psm-fatal][tbird topcrash] → [CLOSEME 2012-03-20][psm-fatal][tbird topcrash]
[@ libsystem_c.dylib@0x27cf1 ] looks like matching signature for v8 and 9.0.1

note, according to bug 674147 this is still at risk for staying in version 11 because Bug 542832 is not yet fixed, although it is making progress. perhaps it will land within a couple weeks?
Crash Signature: [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek] → [@ memcpy | nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek(nsNSSSocketInfo*, void*, int, int, unsigned int)] [@ nsSSLThread::requestRecvMsgPeek] [@ libsystem_c.dylib@0x27cf1 ]
This is impossible since nsSSLThread is gone with the patch for bug 674147. If we have to back out bug 674147 for any reason, we will have to reopen a bunch of bugs, including this one. We will know to do this because I am marking bug 674147 as blocking all of these bugs.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → WORKSFORME
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: