531568 - crash [@ nsCOMPtr_base::~nsCOMPtr_base() | nsImapMailCopyState::~nsImapMailCopyState()], was [@ nsImapMailCopyState::`vector deleting destructor''(unsigned int)] in 3.0b4

Status: VERIFIED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Wayne Mery (:wsmwk)]

crash [@ nsImapMailCopyState::`vector deleting destructor''(unsigned int)]

crashes on the rise in v3.0. almost no crashes of 3.0b2 and 3.0b3 since january 2009 and every 3.0b2 and b3 is eudora, eg bp-beed88d3-654b-4e06-b1d2-a34f82091016, bp-56be5452-9520-48a8-9bdc-01dbf2090923

bp-241dd19d-bab3-43fa-b4f4-53ff62091128 v3.0
0		@0x1e9b857	
1	thunderbird.exe	nsImapMailCopyState::`vector deleting destructor'	
2	thunderbird.exe	nsImapMailCopyState::Release	mailnews/imap/src/nsImapMailFolder.cpp:7862
3	xpcom_core.dll	nsProxyReleaseEvent::Run	objdir-tb/mozilla/xpcom/build/nsProxyRelease.cpp:52
4	xpcom_core.dll	nsThread::ProcessNextEvent	xpcom/threads/nsThread.cpp:521 

bp-4600624e-0718-4dcf-9714-b631d2091128 v3.0
0		@0x18846d8	
1	thunderbird.exe	nsImapMailCopyState::`vector deleting destructor'	
2	thunderbird.exe	nsImapMailCopyState::Release	mailnews/imap/src/nsImapMailFolder.cpp:7862
3	xpcom_core.dll	nsCOMPtr_base::~nsCOMPtr_base	objdir-tb/mozilla/xpcom/build/nsCOMPtr.cpp:81
4	thunderbird.exe	nsImapUrl::~nsImapUrl	mailnews/imap/src/nsImapUrl.cpp:110
5	thunderbird.exe	nsImapUrl::`vector deleting destructor'	
6	thunderbird.exe	nsMsgMailNewsUrl::Release	mailnews/base/util/nsMsgMailNewsUrl.cpp:88
7	thunderbird.exe	nsMailboxUrl::Release	mailnews/compose/src/nsSmtpUrl.cpp:683
8	thunderbird.exe	XPCJSRuntime::GCCallback	js/src/xpconnect/src/xpcjsruntime.cpp:780
9	thunderbird.exe	DOMGCCallback	dom/src/base/nsJSEnvironment.cpp:3692
10	js3250.dll	js_GC	js/src/jsgc.cpp:3788
11	js3250.dll	JS_GC	js/src/jsapi.cpp:2458
12	thunderbird.exe	nsXREDirProvider::DoShutdown	toolkit/xre/nsXREDirProvider.cpp:872
13	thunderbird.exe	ScopedXPCOMStartup::~ScopedXPCOMStartup	toolkit/xre/nsAppRunner.cpp:952 


xref bug 156572 which I recently closed WFM

Comment 1

[Wayne Mery (:wsmwk)]

#12 in 3.0
#15 in 3.0b4

Comment 2

[Wayne Mery (:wsmwk)]

this has evolved to #26 for 3.0.1 with different signature 
[@ nsCOMPtr_base::~nsCOMPtr_base() | nsImapMailCopyState::~nsImapMailCopyState()]

bp-4215624d-0fcf-4520-b3f0-9a4332100120
0	xpcom_core.dll	nsCOMPtr_base::~nsCOMPtr_base	 objdir-tb/mozilla/xpcom/build/nsCOMPtr.cpp:81
1	thunderbird.exe	nsImapMailCopyState::~nsImapMailCopyState	mailnews/imap/src/nsImapMailFolder.cpp:7859
2	thunderbird.exe	nsImapMailCopyState::`vector deleting destructor'	
3	thunderbird.exe	nsDiskCacheStreamIO::Release	netwerk/cache/src/nsDiskCacheStreams.cpp:297
4	xpcom_core.dll	nsCOMPtr_base::~nsCOMPtr_base	objdir-tb/mozilla/xpcom/build/nsCOMPtr.cpp:81
5	thunderbird.exe	nsImapUrl::~nsImapUrl	mailnews/imap/src/nsImapUrl.cpp:110
6	thunderbird.exe	nsImapUrl::`scalar deleting destructor'	
7	thunderbird.exe	nsMsgMailNewsUrl::Release	mailnews/base/util/nsMsgMailNewsUrl.cpp:88
8	thunderbird.exe	nsImapUrl::Release	mailnews/compose/src/nsSmtpUrl.cpp:683
9	thunderbird.exe	XPCJSRuntime::GCCallback	js/src/xpconnect/src/xpcjsruntime.cpp:780
10	thunderbird.exe	DOMGCCallback	dom/src/base/nsJSEnvironment.cpp:3692
11	thunderbird.exe	XPCCycleCollectGCCallback	js/src/xpconnect/src/nsXPConnect.cpp:411
12	js3250.dll	js_GC	js/src/jsgc.cpp:3788 

perhaps same crashes (unknown rankings, not in top 300):
- nsImapMailCopyState::~nsImapMailCopyState() bp-7676bbd1-91ae-452b-9cc9-63a692100122
- [@ arena_dalloc_small | arena_dalloc | free | nsImapMailCopyState::~nsImapMailCopyState() ] bp-dd2a42fd-1c8f-4031-b353-aa0342100119

bp-4215624d-0fcf-4520-b3f0-9a4332100120 comments "Copy of local folder into IMAP postbox. Name of surce-folder includes a dot (".")"

Comment 3

[timeless]

Sorry, it's probably a ref counting issue. my guess is it's one of these:

http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapMailFolder.h?mark=90-99,110-110#87

but it's hard for me to guess.

Comment 4

[Wayne Mery (:wsmwk)]

steps - I encountered this copying a top level imap folder (no subfolders) to another account.  bp-0745cbb5-09a0-4833-97dc-bf6be2100702  trunk build 20100701034823
@0x0 | nsImapMailCopyState::`scalar deleting destructor''(unsigned int)  (which is not queryable)


If my tinkering is correct 3 crash sigs are the same issue, or closely related. 
Together [1] they are the #3 crash for v3.1 (behind the unidentifiable null sig at #2)
Suggest this be debugged and targeted for fixing in 3.1.2.  
Not suggesting blocking, but wanted.

bug 487288 might be the original bug for this crash - see bienvenu bug 487288 comment 1
bug 535125 is about copying multiple sub(folders) - so a variant?

[1]
#8 541 crashes nsCOMPtr_base::~nsCOMPtr_base() | nsImapMailCopyState::~nsImapMailCopyState()nsCOMPtr_base::~nsCOMPtr_base() | nsImapMailCopyState::~nsImapMailCopyState()
#31 152 crashes nsTextAddress::GetField(char const*, int, int, nsCString&, char)
nsImapMailCopyState::`vector deleting destructor''(unsigned int)
#43 122 crashes @0x0 | nsImapMailCopyState::`vector deleting destructor''(unsigned int)
#??  ?? crashes @0x0 | nsImapMailCopyState::`scalar deleting destructor''(unsigned int) (might be 3.2 only)

regarding GC is on the stacks ...

nsCOMPtr_base::~nsCOMPtr_base() | nsImapMailCopyState::~nsImapMailCopyState()
non-startup crashes like bp-f096008e-dff4-4e2d-99bd-8ffae2100628 have GC on stack
startup? crashes don't have GC, like bp-8024a6d2-cf22-4091-a138-73bd52100627

nsImapMailCopyState::`vector deleting destructor''(unsigned int)
non-startup bp-ebde97f6-b83e-4208-ae80-301272100628 (frank)
startup crash - don't see any

@0x0 | nsImapMailCopyState::`vector deleting destructor''(unsigned int)
(stack deviates)
non-startup bp-c223ae09-1c15-49fe-ab9c-489af2100630
non-startup? bp-1a35b48a-bcc8-4ee8-adc1-56b512100702

Comment 5

[Wayne Mery (:wsmwk)]

bienvenu, any need for a protocol log to go with the steps?

Comment 6

[David :Bienvenu]

Do you have reproducible steps? I've tried a bunch of different combinations of moving top level folders between accounts and haven't seen a crash

Comment 7

[Wayne Mery (:wsmwk)]

Attachment: screen shots of error console and activity manager after d&d

I can still consistently reproduce both 3.1.2 and trunk.

ctrl+click drag first level ab-bm3 folder from vseerror account to wsm0 account, dropped on the account folder. folder got created under #news folder, not root. the newly created ab-bm3 folder is empty.  crash doesn't happen for 5 seconds after the drop. no other actions were taken after the drop.

Comment 8

[Wayne Mery (:wsmwk)]

Attachment: imap:5 log

Comment 9

[Wayne Mery (:wsmwk)]

stack to go with the above bp-fb7e67c5-5695-4e5d-b356-d1a392100816

Comment 10

[Wayne Mery (:wsmwk)]

timeless, Is comment 2 aka bug 487288 the same refcount crash?  (see bug 487288 comment 1)

Comment 11

[David :Bienvenu]

Comment on attachment 466440 [details]
imap:5 log

this is a msgdb log, not an imap protocol log. But from the activity manager, it looks like you're getting a failure to create a folder, which might explain why you're seeing the crash, and I'm not.

Comment 12

[David :Bienvenu]

I was able to reproduce a similar looking crash by working backwards from some of the breakpad reports. In particular, the diskcachestreamio entries in the stack trace imply that one or more of the messages copied from the source folder to the new dest folder was copied using the disk cache (and not the imap server or the offline store). To get that to happen, I had turn off the offline store for the source folder, restart, repair the folder to clear the offline store, select one of the messages, and then copy the folder to a different server. Eventually I crashed because m_listener member of nsCopyRequest pointed to an already deleted object. I'm hoping this is reproducible, and related to the other crashes.

Comment 13

[Wayne Mery (:wsmwk)]

imap log and prefs sent to david

Comment 14

[David :Bienvenu]

OK, the reference counting for nsImapFolderCopyState is a bit screwy, but I should be able to fix it. It's doing a delete this, which is a bozo no no. I just need to find out why it thought it needed to do that.

The good news is that I think this bug also accounts for the crash on shutdown in nsCopyRequest::~nsCopyRequest that is a subject of a different bug.

Comment 15

[Wayne Mery (:wsmwk)]

(In reply to comment #14)
> The good news is that I think this bug also accounts for the crash on shutdown
> in nsCopyRequest::~nsCopyRequest that is a subject of a different bug.

I don't find a bug that has that in a comment.  Found only one crash in last 3 months with that signature bp-9fd5321e-e629-4998-b02e-c06a92100813

Comment 16

[Wayne Mery (:wsmwk)]

our current shutdown crashes from https://bugzilla.mozilla.org/buglist.cgi?bug_id=425710%2C394753%2C517690%2C475280%2C547954&bug_id_type=anyexact&query_format=advanced

- crash during shutdown [@ nsACString_internal::~nsACString_internal]
- crash during startup or shutdown [@ @0x7080149] [@ nsMsgAccountManager::`vector deleting destructor'(unsigned int)] // [@ arena_dalloc_small | arena_dalloc | free | nsMsgAccountManager::`vector deleting destructor'(unsigned int)]
- Thunderbird shutdown crash [@ nsScriptSecurityManager::IsCapabilityEnabled(char const*, int*) ]
- Shredder crashed on shutdown [@ nsTransactionItem::Release()]
- lanikai 3.1b1pre shuts down unattended - windows dialog "A problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available."

Comment 17

[David :Bienvenu]

Attachment: proposed fix

I'll generate a try server build with this patch

Comment 18

[David :Bienvenu]

Comment on attachment 467100 [details] [diff] [review]
proposed fix

I'm not sure how to make this fail reliably for unit test purposes (I had a test case copying imap folders across servers, but now I'm unclear why that was failing, and xpcshell tests don't support multiple imap servers), but this is definitely nicer than delete this.

Comment 19

[David :Bienvenu]

windows try server build with this patch.

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/tryserver-builds/[e-mail]-463aa2ed2bbb

trunk is a bit weird w.r.t. the disk cache; there's some weirdness going on with certs. Perhaps I need to do a clobber build.

Comment 20

[Wayne Mery (:wsmwk)]

tryserver build doesn't crash. yay!

I still need to file a bug on the underlying namespace/sub-folder creation placement issue.  the folders being copied aren't getting created under the account root

Comment 21

[David :Bienvenu]

Attachment: patch with unit test

this simulates the failure wsmwk was seeing, by adding the capability of making any particular type of command fail to the imap fake server, and having the imap append fail.

I don't think all the people seeing the crash are having an append fail, but this was the reproducible case I could write a test for.

Comment 22

[David :Bienvenu]

fixed on trunk and 1.9.2 branch

Comment 24

[Wayne Mery (:wsmwk)]

can't determine/verify that this is (fully) fixed in the v3.1.3 via crash-stats. trunk last and only 2 crashes is 20100816034424 build, and there are no crashes in v3.1.3pre

Comment 25

[Robert Montana]

Related?

POP account -> IMAP account folder copy hang. NO further copies allowed. 

Seems to be mostly with large folders, large emails and trees with four+ levels
deep.

Also subfolders that have a ".", Robert P. Montana, end up with an extra
subfolder, no msg copy and a hang:

> Folder
    > Robert
        > P Montana

Comment 26

[Wayne Mery (:wsmwk)]

bienvenu,

we've made progress. nsCOMPtr_base::~nsCOMPtr_base() | nsImapMailCopyState::~nsImapMailCopyState() is not in the top 300 of v3.1.6, so mostly gone. However, 
1. the same cannot be said for some other signatures mentioned in comment 4.
2. there are still some nsCOMPtr_base::~nsCOMPtr_base( .... crashes. for example
 bp-081178a0-9b3e-45b5-99e6-c134b2101105 v3.1.6
 bp-84dbe96e-a6dd-4094-977a-e54482101105 v3.1.6
 bp-b41c0422-ef29-46bf-a30b-d2e242101103 v3.1.6
File new bug(s)?

Robert, you aren't seeing a crash - need a new bug?

Comment 27

[David :Bienvenu]

yes, a new bug would be good - those three crashes all look to be the same, and seem to involve the disk cache and js garbage collection.

Comment 28

[Wayne Mery (:wsmwk)]

v.fixed then.
follow up is Bug 610995