Closed Bug 123427 Opened 23 years ago Closed 23 years ago

Offline message deletes take a very long time to do

Categories

(SeaMonkey :: MailNews: Backend, defect)

Product:
SeaMonkey
Component:
MailNews: Backend
Platform:
PowerPC
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: andrew.treloar, Assigned: Bienvenu)

References

Details

Attachments

(1 file)

proposed fix
3.13 KB, patch
naving
: review+
sspitzer
: superreview+
Details | Diff | Splinter Review 
In offline mode, if I either delete a message or move it to Trash (which should
be the same thing anyway), each delete takes over 30 seconds to do. If I select
a block of messages and choose delete, *each* message takes over 30 seconds.
Moves to other folders while offline are quite quick - I don't see why a move to
Trash should take so much longer. 

I have my settings so that Trash is on the server, which may be a factor.

I can reproduce this problem consistently.
two questions - do you  the trash configured for offline use? And do you have a
virus checker running?
1. Yes, Trash is marked for Offline use
2. No, I'm not running a virus scanner
When you move a message to an imap folder configured for offline use, we have to
open the db for that folder, and we have to open the local store for that
folder. Neither of those operations should take 30 seconds, unless there's a
virus checker running, which can cause writes to local files to take an
inordinate length of time, if not configured correctly. How big are the files
Trash.msf, and Trash?
Trash is 131.2 MB. Trash.msf is 444 K. Can't you open the folder db once and
then keep it open? That way you'd only take the performance hit once. I should
note that I don't see this slowdown when using Netscape 4.X under Classic.
I think the problem here is that the offline trash store has never been
compacted, not the size of the .msf file.  I wonder if we're not deleting the
offline store after the user empties the trash? 131MB is a very large file. I
would try deleting the offline store Trash file using the file system (which
will remove all the offline copies of messages in the trash). I would also
recommend not configuring the trash for offline use. I'll go make sure that
emptying the imap trash also deletes the offline store.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Using 2002020408 on mac 10.1.2
and 2002020103 on nt 4.0 
I can see this problem to David.


On mac:
I tried deleting 10 1kb mesgs (after downloading inbox/trash and going
offline) and it worked pretty fast. I deleted couple of more mesgs
(couple of 1kb mesgs, a 1 7kb & 1 8kb mesg) and it worked ok, teeny bit
slower. I then tried deleting 10 more 1kb mesgs and it took like 30
seconds to move it to trash folder. So after deleting a bigger sized
mesg, it permantly makes it slower?

On windows:
trash is 67kb and trash.msf is 3kb.

I go offline (after downloading inbox/trash) and delete 10 1kb mesgs:
took a little while and 
Now trash is almost 70,000kb and trash.msf is 7kb.


Gary, are the message sizes you quoted K or MB? You say you deleted a few 10 k
messages, but you said your Trash folder was 70,000 K, or 70MB. Is it really
70K, or really 70MB? I would see how a 70MB folder could take a while to open on
an archaic file system like the Mac's, but a 70K file would be surprising.
mesg sizes are in kb.
trash was 70mb. 


Gary, without knowing what's in your trash, I can't tell if you're saying that
the Trash folder is orders of magnitude larger than it should be, or if you
really have 70MB worth of data in your trash folder. Can we try the following
test instead? Delete your Trash file, and do the same test, without downloading
all the messages in your trash for offline use? Then, the only messages that
should be in Trash are the ones you delete in the test, and the size of Trash
should be the sum of the size of the messages you've deleted. Either that, or
you could tell me how large Trash was after you downloaded all the messages in
it, before you started deleting messages (keeping in mind I don't have a mac and
can't do any of those tests here). Thanks!
To answer your question, I believe what's making it slower on the mac is that
the mac is very slow opening large files.
If it's really to do with the Mac being slow in opening large files, then why
don't I see this behaviour using Netscape 4.79 under versions of the Mac OS
prior to OSX?
Reporter what build id is your build?

My nt build is 2-1 & Mac build is 2-4.

For NT 4.0 :
I deleted my trash file, deleted items from the trash can (so its empty)
and selected only my inbox to download. I then went offline.


  Using windows explorer:
    Inbox file is 7 mb. Inbox.msf is 157kb
    No Trash file. Trash.msf is 2kb.

  I now am going to delete 10 mesgs from my inbox (each mesg is 1 kb)
  The result: (deletion was very fast didn't take 30 secs):
    Inbox file is 7 mb. Inbox.msf is 162kb
    Trash file is 29kb. Trash.msf is 6kb.
        
  I am now going to delete 5 mesgs from inbox (each mesg size is between
  2kb and 10kb for a grand total of 27kb)
  Result (took about 10 to 15 secs to delete):
    Inbox file is 7 mb. Inbox.msf is 164kb
    Trash file is 34 MB !!!!. Trash.msf is 9kb.


On mac 10.1.2:
 No inbox or Trash files. Only Inbox.msf and Trash.msf.

 Initially Inbox.msf is 64kb and Trash.msf 64kb.

 I go offline only downloading my inbox folder (not trash folder).
 Result:
  Inbox file (appears) is 8mb. Inbox.msf is 64kb
  No Trash file. Trash.msf is 4kb.

 Note: I can't view mesg size in thread pane of messenger (not working
       in os x builds). So I am guessing on the size of each mesg
       (i have an idea which mesg is 1kb and which one is large say over 10kb).

 Now I delete 5 1kb mesgs.
 Result (took about 5 secs):
   Inbox file is 8mb. Inbox.msf is 64kb.
   Trash file (now appears) is 16.2 MB!!!! and Trash.msf is 24kb.

 Now I will delete a mesg I'm guessing not bigger than 18kb (at least
 that is size of the attachment).
 Result(about 5 secs):
   Inbox file is 8mb. Inbox.msf is 64kb.
   Trash file  is 24.1 MB and Trash.msf is 24kb.

Hope this helps. 
thanks, Gary, I'll try it on windows. Sounds like something very wrong is happening.
I was able to reproduce this - very strange. It's as if the entire contents of
the offline store for the folder the message was deleted from ended up in the
offline store for the Trash folder, not just the single message getting deleted.
In answer to your question, Andrew, 4.x, for one thing, didn't have this bug.
And for another, it stored the offline message contents in the same file as the
message db, and probably kept that db open across deletes. But if it had the
behaviour I described above, it would have slowed down dramatically as well.

My guess is that the code that copies the offline message contents to the trash
folder has been broken so that instead of seeking to the correct offset in the
offline store and just copying the number of bytes corresponding to the message
size, it's copying the whole contents of the source offline store.
Status: NEW → ASSIGNED
Attached patch proposed fix — — Splinter Review 
the problem was with deleting multiple messages; we ended up having an unsigned
variable become negative, which made us copy the whole offline store (or at
least from the current message to the end of the file) in some situations, when
deleting multiple messages. The fix is to make that variable signed, and only
write out the number of bytes we meant to read. Also, delete the offline store
when the imap/news folder is deleted. Can I get reviews, Seth and Navin?
cc'ing Navin for review.
OS: MacOS X → All
Comment on attachment 67939 [details] [diff] [review]
proposed fix

r=naving
Attachment #67939 - Flags: review+
Comment on attachment 67939 [details] [diff] [review]
proposed fix

sr=sspitzer

should the assertion string be:

"wrote out incorrect number of bytes"?
Attachment #67939 - Flags: superreview+
fix checked in.
Status: ASSIGNED → RESOLVED
Closed: 23 years ago
Resolution: --- → FIXED
*** Bug 123955 has been marked as a duplicate of this bug. ***
commercial trnk builds:
2002022103 nt 4.0
2002022108 linux 2.2, mac 9.2.2, mac 10.1.3

Tried deleting numerous downloaded mesgs while offline and
the size of the Trash store file ended up being the size
of sum of the total deleted messages. Emptying trash online results
in the online Trash store file being deleted also.

Deleting numerous email mesgs, while offline, didn't take
long time. 
Spoke with David Offline (pun not attended)..

one thing I noticed the offline store for user created folder
doesn't dissapear when the downloaded folder is deleted (whne online)
and emptied from trash can. See david's comment in comment 14.

David has filed a new bug 127140 on this.

marking this bug as verified.
Status: RESOLVED → VERIFIED
Product: Browser → Seamonkey
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: