698093 - crash @ nsMsgDBView::FnSortIdKeyPtr and @ med3 sorting message list columns

Status: NEW

(MailNews Core :: Backend, defect)

Description

[Wayne Mery (:wsmwk)]

This bug was filed from the Socorro interface and is 
report bp-b3d9a73b-403f-48a1-b40a-d9c772111017 .
============================================================= 

In Bug 420257 comment 0 I reported the same stack/signature
nsMsgDBView::FnSortIdKeyPtr(void const*, void const*, void*)

no comments on crash-stats = no steps to reproduce

bp-b3d9a73b-403f-48a1-b40a-d9c772111017 version 7

0	xul.dll	nsMsgDBView::FnSortIdKeyPtr	mailnews/base/src/nsMsgDBView.cpp:3796
1	xul.dll	nsMsgDBView::GetIndexForThread	mailnews/base/src/nsMsgDBView.cpp:5157
2	xul.dll	nsMsgSearchDBView::AddHdrFromFolder	mailnews/base/src/nsMsgSearchDBView.cpp:476
3	xul.dll	nsMsgSearchDBView::InsertHdrFromFolder	mailnews/base/src/nsMsgSearchDBView.cpp:690
4	xul.dll	nsMsgXFVirtualFolderDBView::OnSearchHit	mailnews/base/src/nsMsgXFVirtualFolderDBView.cpp:312
5	xul.dll	nsMsgSearchSession::AddSearchHit	mailnews/base/search/src/nsMsgSearchSession.cpp:601
6	xul.dll	nsMsgSearchOfflineMail::AddResultElement	mailnews/base/search/src/nsMsgLocalSearch.cpp:818
7	xul.dll	nsMsgSearchOfflineMail::Search	mailnews/base/search/src/nsMsgLocalSearch.cpp:770
8	xul.dll	nsMsgSearchScopeTerm::TimeSlice	mailnews/base/search/src/nsMsgSearchTerm.cpp:1934
9	xul.dll	nsMsgSearchSession::TimeSliceSerial	mailnews/base/search/src/nsMsgSearchSession.cpp:691
10	xul.dll	nsMsgSearchSession::TimerCallback	mailnews/base/search/src/nsMsgSearchSession.cpp:544

Comment 1

[Wayne Mery (:wsmwk)]

onlly a couple per month.
eg. bp-c9331c2b-fded-4f5c-9254-7e7f02121223

Comment 2

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 3

[Wayne Mery (:wsmwk)]

https://crash-stats.mozilla.com/signature/?date=%3E2016-02-01&signature=nsMsgDBView%3A%3AFnSortIdKeyPtr&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_sort=-date&page=1
Not uncommon for version 10 and 38.  Then only a few crashes for 45.1.1 and 45.1.0.  And none for 45.2.0 - odd.
Have we changed substantially in nsMsgDBView?

Or did we morph to  nsMsgDBView::FnSortIdUint32 https://crash-stats.mozilla.com/signature/?signature=nsMsgDBView%3A%3AFnSortIdUint32&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_sort=-date&page=1#reports

Comment 4

[:aceman]

I picked 2 from the list (TB 45 and 38) and they crash on stack overflow in NS_QUICKSORT?

Maybe we pass some array with bogus values (uninitialized memory) to sort and it crashes on it.

Or, the FnSortIdUint32 function is used as the member comparison function in the quicksort. Maybe it compares incorrectly in some cases thus causing an infinite loop (until stack is depleted). E.g. I do not see when the function would return 0 (members equal). But I don't know if some members can ever be equal.

Comment 5

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #4)
> I picked 2 from the list (TB 45 and 38) and they crash on stack overflow in
> NS_QUICKSORT?
> 
> Maybe we pass some array with bogus values (uninitialized memory) to sort
> and it crashes on it.
> 
> Or, the FnSortIdUint32 function is used as the member comparison function in
> the quicksort. Maybe it compares incorrectly in some cases thus causing an
> infinite loop (until stack is depleted). E.g. I do not see when the function
> would return 0 (members equal). But I don't know if some members can ever be
> equal.

Let's assume this morphed to @ nsMsgDBView::FnSortIdUint32 bp-f8194569-0ed5-4fa3-b72b-748e82170202 and a friend  @ med3 bp-a894f465-c3b2-412c-8c87-53b922170126

see also http://forums.mozillazine.org/viewtopic.php?f=39&t=3027062

Comment 6

[:aceman]

Both the reports crash due to EXCEPTION_STACK_OVERFLOW ...

Derrick, you last made a logic change in the https://hg.mozilla.org/releases/mozilla-esr45/file/tip/xpcom/glue/nsQuickSort.cpp, can you please see if you can spot some problem in the algorithm, why the stack may be depleted?

Comment 7

[Wayne Mery (:wsmwk)]

https://support.mozilla.org/en-US/questions/1169242

Comment 8

[Wayne Mery (:wsmwk)]

I tried to PM Derrick but no response, so he seems to be gone. (And hasn't logged into bz since 2014)

Comment 9

[:aceman]

The report at bp-1c5cb55c-9318-4717-b727-617180170727 seems to have a recursive stack of xpcom/glue/nsQuickSort.cpp:171 calls. I think we either send some malformed data to the quicksort (but I'm not sure what that would be, the sort should handle any possible array of ints), or there is a bug in the algorithm so it runs endlessly on some specific type of input and exhausts the stack.

Comment 10

[Wayne Mery (:wsmwk)]

66% of crash locales are ja for  signature  nsMsgDatabase::GetCollationKeyGenerator bp-470d4b51-5b1c-4238-868f-0269d0171222

 0 	xul.dll	nsMsgDatabase::GetCollationKeyGenerator()	C:/builds/moz2_slave/tb-rel-c-esr52-w32_bld-0000000/build/mailnews/db/msgdb/src/nsMsgDatabase.cpp:3670
1 	xul.dll	nsMsgDatabase::CompareCollationKeys(unsigned int, unsigned char*, unsigned int, unsigned char*, int*)	C:/builds/moz2_slave/tb-rel-c-esr52-w32_bld-0000000/build/mailnews/db/msgdb/src/nsMsgDatabase.cpp:3732
2 	xul.dll	nsMsgDBView::FnSortIdKey(void const*, void const*, void*)	C:/builds/moz2_slave/tb-rel-c-esr52-w32_bld-0000000/build/mailnews/base/src/nsMsgDBView.cpp:3850
3 	xul.dll	med3	xpcom/glue/nsQuickSort.cpp:93
4 	xul.dll	NS_QuickSort	xpcom/glue/nsQuickSort.cpp:125
5 	xul.dll	NS_QuickSort	xpcom/glue/nsQuickSort.cpp:171

Comment 11

[Wayne Mery (:wsmwk)]

m_kato, can you spot the error?

nsMsgDBView::FnSortIdUint32 bp-3e92fc71-249b-4510-8e4e-691f90181116

med3 is the most common signature.  
bp-b3117ab5-5707-4bc4-ade7-213e10181123 ja locale
bp-e74b3f95-d67f-47bd-913d-364e60181121 ru locale
bp-522bd067-d970-4390-b3c7-bf6ca0180902 sorted junk% score (which can only be done with junquilla addon installed)
bp-e2f6b40d-721c-425e-9588-123980180711 sorted on junk status

Comment 12

[Jorg K (CEST = GMT+2)]

bp-b3117ab5-5707-4bc4-ade7-213e10181123, wow, stack overflow in Quicksort :-(

Comment 13

[Wayne Mery (:wsmwk)]

might bug 1498313 help?

Comment 14

[Jorg K (CEST = GMT+2)]

Maybe since that switches from NS_QuickSort() to C++'s standard qsort(). However, I don't think NS_QuickSort() is that buggy, so there might be an underlying issue.

Comment 15

[Matt]

User in https://support.mozilla.org/en-US/questions/1267021

Crashing with https://crash-stats.mozilla.com/report/index/dc508528-c545-4b53-a0b0-54d1c0190821

In this instance the crash was initiated by clicking on the sort by star. Thunderbird now crashes on each restart.

Comment 16

[Jorg K (CEST = GMT+2)]

nsMsgThreadedDBView.cpp:428 calls nsMsgDBView::Sort(sortType, sortOrder);, something goes wrong in there, and then it crashes in NS_QuickSort, or iterates until it gets a stack overflow. Bug 1498313 might help, or standard qsort will than also crash.

This needs careful code inspection. Maybe Ben wants to take a look.

Comment 17

[Ben Campbell]

That's an epic callstack :-)
I've had a look through the code and nothing obvious stands out, but there's a lot of code and a lot of potential for subtle issues there.
Is there a reliable way to trigger this crash? I reckon I could nail down it down pretty quickly with a debugger attached to it...

Comment 18

[Wayne Mery (:wsmwk)]

https://support.mozilla.org/questions/1267021 is a potential testcase that Ben has contacted

Comment 19

[alta88]

That poor reporter in https://support.mozilla.org/questions/1267021... The advice should have been to rename and save the folder's .msf file as well as a copy of the folder in some place outside the profile, and maybe also rename and save panacea.dat only if the .msf rename didn't work. Asking users to recreate profiles is very extreme and painful.

Comment 20

[Wayne Mery (:wsmwk)]

Also https://support.mozilla.org/en-US/questions/1270599

Comment 21

[Anje]

Perhaps this is another case:
Thunderbird 68.4.1
Crash Reason EXCEPTION_STACK_OVERFLOW
Signature nsMsgDBView::FnSortIdKey
https://crash-stats.mozilla.com/report/index/7f1ac55a-b890-410b-808f-040790200123
"function": "nsMsgDBView::FnSortIdKey(void const *,void const *,void *)"

Comment 22

[Anje]

Further to comment 21
Person uses Quick Filter Bar which also coincides with another person who had crash after using the star option on Quick filter Bar.
https://support.mozilla.org/en-US/questions/1277937

Comment 23

[Wayne Mery (:wsmwk)]

We'll see if Mark can provide us with a testcase wanted by Ben

Comment 24

[Wayne Mery (:wsmwk)]

still quite rare.

Comment 25

[Wayne Mery (:wsmwk)]

Perhaps this will become moot when new folder management/display comes to pass?

91.2.1 nsMsgDBView::FnSortIdUint32 bp-984d4918-59f7-44d0-85d0-11a9e0211027

Comment 26

[Wayne Mery (:wsmwk)]

Is nsMsgDBView::FnSortIdKey a continuation? bp-e598323c-d337-47ee-81d4-d0dc30240518

Comment 27

[Magnus Melin [:mkmelin]]

Might be.