filter after the fact: too much disk io

RESOLVED FIXED in mozilla1.2beta

Status

MailNews Core
Filters
RESOLVED FIXED
16 years ago
10 years ago

People

(Reporter: (not reading, please use seth@sspitzer.org instead), Assigned: (not reading, please use seth@sspitzer.org instead))

Tracking

Trunk
mozilla1.2beta
x86
Windows 2000

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 10 obsolete attachments)

20.20 KB, patch
Details | Diff | Splinter Review
filter after the fact:  too much disk io

saspitzer: runnning filter on selected folders just went crazy!
saspitzer: I heard the disk grind.
saspitzer: the UI hung, but then it came back.
saspitzer: I mean really grind!

saspitzer: I might do this:
saspitzer: LogRuleHit(hdr, PRBool flush);
saspitzer: for normal, yes, flush
saspitzer: for yours no,
saspitzer: and they give you a flush.

it's when logging is turned, working on a fix.

thanks to david for the info.
Created attachment 99091 [details] [diff] [review]
assertion fixes, other cleanup.
Attachment #99081 - Attachment is obsolete: true
Created attachment 99092 [details] [diff] [review]
update
Attachment #99091 - Attachment is obsolete: true
Created attachment 99096 [details] [diff] [review]
update, I put the flush call in the wrong spot.  damn these fat fingers!
Attachment #99092 - Attachment is obsolete: true
the last patch does what we want, but I found another scenario where we slam 
the disk.

file a large number of messages from one folder to the imap inbox.  then load 
the impa inbox.  we apply the filter rules to the moved headers.  (not sure if 
that is correct or not.)

so if I move a lot of msgs to inbox, we can slam the disk if we have logging 
enabled.

I'll look into this next.

but filter after the fact only flushes at the end now, and doesn't slam the 
disk.

nsMsgFilter::LogRuleHit(nsMsgFilter * const 0x04909690, nsIMsgDBHdr * 
0x01322b90, int 1) line 351
nsImapMailFolder::ApplyFilterHit(nsImapMailFolder * const 0x046d013c, 
nsIMsgFilter * 0x04909690, nsIMsgWindow * 0x03c3ed00, int * 0x0012fb54) line 
2993
nsMsgFilterList::ApplyFiltersToHdr(nsMsgFilterList * const 0x049084e0, int 1, 
nsIMsgDBHdr * 0x01322b90, nsIMsgFolder * 0x046d007c, nsIMsgDatabase * 
0x0483b778, const char * 0x04e1d028, unsigned int 282, nsIMsgFilterHitNotify * 
0x046d013c, nsIMsgWindow * 0x03c3ed00) line 285 + 29 bytes
nsImapMailFolder::NormalEndHeaderParseStream(nsIImapProtocol * 0x03aefe80) line 
2672
nsImapMailFolder::ParseMsgHdrs(nsImapMailFolder * const 0x046d0128, 
nsIImapProtocol * 0x03aefe80, nsIImapHeaderXferInfo * 0x03af0108) line 2537 + 
18 bytes
XPTC_InvokeByIndex(nsISupports * 0x046d0128, unsigned int 16, unsigned int 2, 
nsXPTCVariant * 0x079c0200) line 106
EventHandler(PLEvent * 0x078db778) line 567 + 41 bytes
PL_HandleEvent(PLEvent * 0x078db778) line 643 + 10 bytes
PL_ProcessPendingEvents(PLEventQueue * 0x01267f80) line 573 + 9 bytes
_md_EventReceiverProc(HWND__ * 0x0055027a, unsigned int 49447, unsigned int 0, 
long 19300224) line 1308 + 9 bytes
USER32! 77e11b60()
USER32! 77e11cca()
USER32! 77e183f1()
nsAppShellService::Run(nsAppShellService * const 0x012eee40) line 472
main1(int 2, char * * 0x00277ba0, nsISupports * 0x00000000) line 1508 + 32 bytes
main(int 2, char * * 0x00277ba0) line 1869 + 37 bytes
mainCRTStartup() line 338 + 17 bytes
KERNEL32! 77e8d326()


Status: NEW → ASSIGNED
I think there is a filter bug here.  because when I move messages back into the 
inbox, they get moved back.

shouldn't normal (not after the fact) filters only apply to "new" headers?  
Created attachment 99099 [details] [diff] [review]
patch for review.
Attachment #99096 - Attachment is obsolete: true
Created attachment 99102 [details] [diff] [review]
updated patch
Attachment #99099 - Attachment is obsolete: true
Target Milestone: --- → mozilla1.2beta
Created attachment 99109 [details] [diff] [review]
flush filter log to disk less often during imap folder parsing, header download, and pop download.
Attachment #99102 - Attachment is obsolete: true
Created attachment 99113 [details] [diff] [review]
patch
Attachment #99109 - Attachment is obsolete: true
Created attachment 99127 [details] [diff] [review]
fix something bienvenu pointed out privately.
Attachment #99113 - Attachment is obsolete: true
Created attachment 99132 [details] [diff] [review]
patch, again following tips from bienvenu.
Attachment #99127 - Attachment is obsolete: true

Comment 13

16 years ago
Comment on attachment 99132 [details] [diff] [review]
patch, again following tips from bienvenu.

sr=bienvenu - I had a couple nits over aim, but Seth has fixed those.
Attachment #99132 - Flags: superreview+
Created attachment 99137 [details] [diff] [review]
patch with last tips from dmb over aim...
Attachment #99132 - Attachment is obsolete: true
fixed.
Status: ASSIGNED → RESOLVED
Last Resolved: 16 years ago
Resolution: --- → FIXED
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.