313026 - crash when downloading many messages and "copy all" filter enabled [@ nsImapUrl::GetMsgFileSpec] [@ nsImapProtocol::ProcessAuthenticatedStateURL]

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[John Maline]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7

I get a crash on startup after downloading messages from my IMAP inbox.  It
crashes while I see it reading messages and marking spam, but before I otherwise
interacting with Thunderbird.

Requires the following conditions.  If either of these isn't met, I don't get
the crash.
- many (15-40) messages downloaded (e.g., first startup of the day)
- enabled mail filter that copies all messages (subject isn't zzzzz).  The
destination of the copy is a folder on another IMAP server.

Other potentially interesting config setups:
- spam filtering enabled
- 6 other message filters enabled (and 2 disabled).  All of these typically look
for specific subject strings and move the e-mail to a folder on the other IMAP
server.

Reproducible: Always

Steps to Reproduce:
1. Start thunderbird first thing in the morning when I'll have 15+ e-mails
waiting for me
2. Watch mail download, spam markers appear.  Don't otherwise touch TB.
3. crash
Actual Results:  
program crash

Expected Results:  
no program crash

I've seen this under 1.0.6 and 1.0.7.

Talkback ID's 1.0.7:
TB10849760K

Talkback ID's 1.0.6:
TB10824308E
TB10808403Z
TB10559873M
TB10553911Y
TB10515193E
(and many more)

Comment 1

[timeless]

Incident ID: 10849760
Stack Signature	nsImapUrl::GetMsgFileSpec d5b1d16c
Product ID	Thunderbird10
Build ID	2005092314
Trigger Time	2005-10-19 07:13:12.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	thunderbird.exe + (004a25ec)
URL visited	
User Comments	Same initial startup crash under 1.0.7 as I'd seen under 1.0.6. To
recap: I've got two IMAP servers, dlemail has inbox/trash/junk, all other
folders on raptor. I've got spam filtering enabled on dlemail. Also a couple of
filters that move a low
Since Last Crash	503 sec
Total Uptime	503 sec
Trigger Reason	Access violation
Source File, Line No.
e:/builds/tinderbox/Tb-Aviary1.0.1/WINNT_5.0_Depend/mozilla/mailnews/imap/src/nsImapUrl.cpp,
line 1237
Stack Trace 	
nsImapUrl::GetMsgFileSpec 
[e:/builds/tinderbox/Tb-Aviary1.0.1/WINNT_5.0_Depend/mozilla/mailnews/imap/src/nsImapUrl.cpp,
line 1237]
nsImapProtocol::ProcessAuthenticatedStateURL 
[e:/builds/tinderbox/Tb-Aviary1.0.1/WINNT_5.0_Depend/mozilla/mailnews/imap/src/nsImapProtocol.cpp,
line 7042]
nsImapProtocol::ProcessCurrentURL 
[e:/builds/tinderbox/Tb-Aviary1.0.1/WINNT_5.0_Depend/mozilla/mailnews/imap/src/nsImapProtocol.cpp,
line 1390]
nsImapProtocol::ImapThreadMainLoop 
[e:/builds/tinderbox/Tb-Aviary1.0.1/WINNT_5.0_Depend/mozilla/mailnews/imap/src/nsImapProtocol.cpp,
line 1179]

Comment 2

[John Maline]

I modified the "copy all" filter to copy to a local folder instead of second
IMAP server.  In two startups w/ 15+ e-mails, no crashes seen.  That's not a
desireable configuration, just a debugging experiment.

Comment 3

[Adam Guthrie]

Can you reproduce this in Thunderbird 1.5? http://www.mozilla.org/products/thunderbird/releases/1.5.html

Comment 4

[John Maline]

(In reply to comment #3)
> Can you reproduce this in Thunderbird 1.5?

Sorry for the delay.  Yes, I think it reproduces in TB 1.5.0.2.

I finally upgraded at work and reenabled the "copy all" filter directed to the other IMAP server (not the IMAP server holding my inbox).  Suddenly the crashes happen when getting new messages in my inbox.  If anything, the threshold for crashing is lower.  In 1.0.x, it took a big load of inbox messages to cause the crash.  Now it's happening with a download of 5-10 messages.

Talkback ID's
TB18536449M
TB18531995G
TB18529139E

Comment 5

[Adam Guthrie]

Incident ID: 18529139
Stack Signature	nsImapUrl::GetMsgFileSpec d5b1d16c
Product ID	Thunderbird15
Build ID	2006030803
Trigger Time	2006-05-10 09:40:34.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	thunderbird.exe + (004ba2ab)
URL visited	
User Comments	server. So I suspect the crash is from the startup and copying several messages. I'll try for a few more repetitions before getting convinced...
Since Last Crash	275490 sec
Total Uptime	275490 sec
Trigger Reason	Access violation
Source File, Line No.	c:/builds/tinderbox/Tb-Mozilla1.8.0/WINNT_5.2_Depend/mozilla/mailnews/imap/src/nsImapUrl.cpp, line 1197
Stack Trace 	
nsImapUrl::GetMsgFileSpec  [c:/builds/tinderbox/Tb-Mozilla1.8.0/WINNT_5.2_Depend/mozilla/mailnews/imap/src/nsImapUrl.cpp, line 1197]
nsImapProtocol::ProcessAuthenticatedStateURL  [c:/builds/tinderbox/Tb-Mozilla1.8.0/WINNT_5.2_Depend/mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 7128]
nsImapProtocol::ProcessCurrentURL  [c:/builds/tinderbox/Tb-Mozilla1.8.0/WINNT_5.2_Depend/mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 1377]
nsImapProtocol::ImapThreadMainLoop  [c:/builds/tinderbox/Tb-Mozilla1.8.0/WINNT_5.2_Depend/mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 1140]

Comment 6

[Adam Guthrie]

So it looks like it's crashing because m_filespec is null (it never gets set to something else after it's set to null in the constructor of nsImapUrl.

Comment 7

[Wayne Mery (:wsmwk)]

TB36123022 is one from TB v2.
buildid 2007072817
"Moving messages from one IMAP account / subfolder to another IMAP account / subfolder. The messages were quite large."

~1 crash/day for v2.

Comment 8

[Wayne Mery (:wsmwk)]

John, do you have any updates to your note of 2/14/2008 below?


"I just did a bit of testing with a cross-IMAP-server filter re-enabled.  A bunch of tests seemed successful, but I eventually crashed http://talkback-public.mozilla.org/search/start.jsp?search=2&type=iid&id=TB41465933M

The symptom of this crash wasn't the same.  It happened when I was deleting some of my test mails, not at startup when the filter ran.  And the thread dump is different.  So, I'd say there's still a bug, but it has moved.

My test consisted of generating a fair amount of example e-mail on a nearby Unix host.  Some of which matched a "subject contains filtertest" rule, some didn't.  The rule moved the e-mail to a folder on an IMAP server different from the one holding my inbox.

for i in 0 1 2 3 4 5 6 7 8 9
        do
        mailx -s filtertest [e-mail] <60-kb-text-file &
        mailx -s filtertest [e-mail] <5-kb-text-file &
        mailx -s lalala [e-mail] <60-kb-text-file &
        done


I'd run the above sequence a couple of times, then fire up thunderbird.  It'd churn away processing the e-mails.

I just recreated the new crash again, deleting the "lalala" e-mails from my inbox.  TalkBack TB41466261Z
Stack Signature	 0x02c2ad8b f2af3bea
Product ID	Thunderbird2
Build ID	2007103104
Trigger Time	2008-02-14 13:47:09.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600	
User Comments	Deleting some e-mails (non-filtered ones still in my inbox) while testing bugzilla 313026
Stack Trace 	
0x02c2ad8b
nsImapProtocol::ProcessAuthenticatedStateURL  [mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 7314]
nsImapProtocol::ProcessCurrentURL  [mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 1444]
nsImapProtocol::ImapThreadMainLoop  [mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 1164]

Comment 9

[John Maline]

Tried again w/ 2.0.0.14 and recreated the crash.  Similar to last time - it didn't crash during the filtering.  It crashed while deleting some unfiltered messages.

I've got a talkback incident, but it won't send right now, so no incident id.  Timestamp 5/20/2008 4:58 PM (assuming that helps find it once it finally sends).

Comment 10

[Wayne Mery (:wsmwk)]

John, does it crash using TB 3 alpha?
  ftp://ftp.mozilla.org/pub/thunderbird/releases/3.0a1/
It might succeed sending the crash where TB 2+talkback doesn't.


John, has your system successfully sent talkback ids before?   You might check http://kb.mozillazine.org/Talkback#No_incident_IDs_listed especially "Proxomitron"

Comment 11

[John Maline]

I've successfully submitted talkbacks before with this setup.  I confirmed that my corporate proxy settings were valid (using same in Firefox).  No authentication needed. I was wondering if the server was having troubles.  In any case, I'll keep letting the upload try and hope it happens sometime.

I haven't tried TB3 alpha and won't be able to do any work with pre-releases.  I can only recreate this in my work environment on my real e-mail.  I need to keep my setup conservative.  I know that's not an ideal answer...

Comment 12

[John Maline]

Talkback finally talked.

TB45360834G
TB45360779Y

Comment 13

[Wayne Mery (:wsmwk)]

A cousin to bug 341929? (I can't judge)

John - do you run any extensions?

TB45360779Y
nsEventQueueImpl::`vftable'
nsImapProtocol::ProcessAuthenticatedStateURL  [mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 7313]
nsImapProtocol::ProcessCurrentURL  [mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 1444]
nsImapProtocol::ImapThreadMainLoop  [mozilla/mailnews/imap/src/nsImapProtocol.cpp, line 1164]

Comment 14

[John Maline]

Tools / Add-ons shows only Talkback 2.0.0.9

Comment 15

[David :Bienvenu]

this crash hits me every once in a while - I have a filter that moves incoming pop3 mail to an imap folder.

Comment 16

[David :Bienvenu]

I should say copies incoming pop3 mail - the distinction is crucial since moves use the move coalescer, which batches the moves at the end, whereas copies happen while the download is going on, one at a time, relying on the copy service to serialize the requests. Which it seems to be doing imperfectly.

Comment 17

[David :Bienvenu]

one possible clue is that when I disabled downloading of pop3 messages larger than 50K, this crash doesn't seem to happen nearly as much, so it may be that large messages make this easier to reproduce.

Comment 18

[David :Bienvenu]

I'm very hopeful that the fix for bug 497622 fixes this as well.

Comment 19

[David :Bienvenu]

this is still happening in b3 - not sure if it's less common than before or not...

Comment 20

[David :Bienvenu]

Attachment: hold a reference to the file...

Not holding a reference is clearly not working out for us. This shouldn't create any cycles since nsIFile's don't hold onto anything interesting.

The file in question either comes from m_copyState->m_tmpFile, when copying local messages to an imap folder, or nsMsgComposeAndSend.mCopyFile when doing an fcc/save as draft, etc. Since I can't recreate this problem, I'm not sure what's happening to the file, but not crashing seems preferable to crashing.

Comment 21

[David :Bienvenu]

fix checked in.

Comment 22

[Wayne Mery (:wsmwk)]

John, if you are able, please test ftp://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-1.9.1/ to verify for us that your crash is gone with the patch. And also, ideally, that it occurs without the patch by using pre-patch build ftp://ftp.mozilla.org/pub/thunderbird/nightly/2009-07-29-05-comm-1.9.1/

Otherwise, we'll need to wait for 3.0b4 to verify this is gone, because there are no crashes reported on trunk.

(In reply to comment #19)
> this is still happening in b3 - not sure if it's less common than before or
> not...

we've only got one week of stats since automatic updates went into effect, but still very much visible in 3.0b3 ... not a common crash, but I estimate the crash rate to be the same as 3.0b2 
http://crash-stats.mozilla.com/report/list?product=Thunderbird&query_search=signature&query_type=exact&query=nsImapProtocol%3A%3AProcessAuthenticatedStateURL%28%29&date=&range_value=4&range_unit=weeks&do_query=1&signature=nsImapProtocol%3A%3AProcessAuthenticatedStateURL%28%29