485524 - Frequent freezing during composition with "Attaching..." on Status Bar (autosave, compacting, IMAP) at zero to low cpu - low value for preference Auto Save every N minutes

Status: RESOLVED DUPLICATE of bug 532395

(MailNews Core :: Composition, defect)

Description

[Nigel Green]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
Build Identifier: Shredder version 3.0b3pre

During email composition the Shredder application frequently freezes and becomes unresponsive for around 30 seconds.   This seems to happen about every minute, and so has become very annoying.

This has only just started happening since the automatic update which was automatically applied to my Shredder installation a couple of days ago.

The "Attaching..." seems to occur despite me not having any attachments to the email.   I believe it is referring to is the standard HTML signature applied to my email which calls in an image file from a URL.   Sometimes it does say "Attaching ..." with the name of the image called in by the signature.


Reproducible: Always

Steps to Reproduce:
1.Set up a signature to your email which pulls in an image via a URL
2.Start composing a new email
3.
Actual Results:  
Email composition screen freezes during typing.   If you operate the mouse to click on part of the entered text, the whole page gets greyed out by Windows with a "Not Responding" message at the top of the screen.


Expected Results:  
It should not have frozen at all.

Comment 1

[rsx11m]

This may be caused by the auto-save function, which should save your message every five minutes or so into the Drafts folder while composing as a backup.

Check your settings in the Tools > Options > Composition > General tab, is that option checked and a small number of minutes entered?

Since you linked it to the signature image, does this problem disappear when switching off the signature?

Also, are you using an IMAP account, and if yes, is your Drafts folder remote?

Comment 2

[Nigel Green]

My configuration is set to autosave every 1 minute, which explains why I am seeing it with annoying regularity.   The account concerned is an IMAP account.

I switched off the signature and the problem is still there, so it seems the problem is not related to attached signatures at all.

My drafts folder is set to "Local folders".   When I change this back to the account name (i.e. draft stored on the IMAP server) the problem goes away, as it allows you to continue typing whilst it is saving the message to the draft folder.   So this could be a valid workaround for those users that are experiencing this problem.

It seems therefore that the problem relates to the locking of the application whilst it is attempting to store a draft to a local folder.   

As mentioned in the initial report, I did not suffer this problem before, and I would definitely have noticed it, and I have not changed the configuration for months.   So it seems that this problem has been triggered by a change made recently to the application. 

**LATEST**
My copy of Shredder has just updated itself with the latest version and having tested it out, I can confirm the problem is still present with the latest version.   

In addition, I have noted another point that might be relevant.  While it is frozen on the composition page, if you keep typing it lengthens the period of the freeze, as if it has to take resource away from saving the data to a local folder to handle the key presses.   The keys pressed during this freeze period are not lost, but are retained for display in one go when the freeze period ends.

Comment 3

[rsx11m]

(In reply to comment #2)
> My configuration is set to autosave every 1 minute, which explains why I am
> seeing it with annoying regularity.   The account concerned is an IMAP account.

Ok, this confirms that it is related to the auto-save activity.

> My drafts folder is set to "Local folders".   When I change this back to the
> account name (i.e. draft stored on the IMAP server) the problem goes away, as
> it allows you to continue typing whilst it is saving the message to the draft
> folder.   So this could be a valid workaround for those users that are
> experiencing this problem.

This is interesting, I've moved my Drafts folder from IMAP to Local Folders
long time ago due to performance reasons (remote write takes more time), thus
it is surprising that you see such a delay with the local file. Other guesses coming to my mind are an anti-virus filter scanning and locking the Drafts file on every write, in which case you should see similar issues when just moving a message locally between folders. Maybe just reindexing and compacting the local Drafts folder resolves the issue. Neither of this would explain the extended duration of when you are typing though.

Comment 4

[Nigel Green]

(In reply to comment #3)
Thanks for your input to this issue.

> Other guesses coming to my mind are an anti-virus filter scanning and locking > the Drafts file on every write, in which case you should see similar issues  > when just moving a message locally between folders.
I have McAfee as my anti-virus tool.   It has always taken a long time to move messages between local folders and the IMAP server, but I have always put this down to the IMAP server.   It certainly has never "frozen" my application and said "Application Not Responding" when I have done this.

> Maybe just reindexing and compacting the local Drafts folder resolves the issue. 
Well, I compacted the local Drafts folder and it has improved the situation enormously.   The problem is still there, but it has reduced the lockout time from 30 seconds down to about 1 second, which is much more acceptable for normal use, as it now appears to be a hesitation rather than a complete freeze.

So I would say there is still a small question mark there as to why updates to the screen are not handled whilst saving a draft file.   If this were addressed, the problem could be eliminated completely and you would not notice when it was auto-saving a draft.

The other question that arises relates to the compacting of local folders.   I did not know that I needed to do this, and what a dramatic difference it would make.   An enhancement to the tool could either be to monitor the state of the local folders and warn when it is advisable to compact them, or even better to compact them automatically so I did not need to worry about doing this.

Comment 5

[rsx11m]

> (comment #4) The problem is still there, but it has reduced the lockout time
> from 30 seconds down to about 1 second, which is much more acceptable for
> normal use, as it now appears to be a hesitation rather than a complete freeze.

Good to hear that this improved after compacting.

> I did not know that I needed to do this, and what a dramatic difference it
> would make. [...] or even better to compact them automatically so I did not
> need to worry about doing this.

When you delete a message, they are only marked as deleted but not physically removed before the folder is compacted. It is possible to compact automatically based on a limit, but this is not activated by default (bug 341206).

See http://kb.mozillazine.org/Compacting_folders for more information.

> So I would say there is still a small question mark there as to why updates to
> the screen are not handled whilst saving a draft file.

There is bug 307042, the fix of which resolved part but not all of the problem.

Comment 6

[Nigel Green]

(In reply to comment #5)
> When you delete a message, they are only marked as deleted but not physically
> removed before the folder is compacted. It is possible to compact automatically
> based on a limit, but this is not activated by default (bug 341206).
The "Compact folders when it will save over 100kB" option was checked and I think it has always been checked.   So with this option on, why did my local folders get into such a state?   

Also, it does not make clear which folders this option applies to.   This is important as I note that you have to compact each email account separately.  Does the option check the compact condition of local folders as well as the state of the account folders (which may be remote IMAP folders)?

Anyway, thanks once again for your responses.

Comment 7

[rsx11m]

This is a global setting, thus it should apply to all folders equally.
I'll leave this unconfirmed so that somebody else can look into the issue,
but remove the "critical" as it is neither causing dataloss nor a crash.

Comment 8

[Wayne Mery (:wsmwk)]

I can cause a hang. 

dell d531 vista on wireless and cablemodem
turned on my favorite streaming music http://echoes.org/online2.html :)
set autosave to 1 minute
compose and attach 1mb file
give it a couple minutes
ms-windows tags Thunderbird  window "(Not Responding)" ... permanently

I would imagine under less extreme circumstances one might see only bad performance or occasional freezing rather than a hang. 

A stack is probably wanted but I'm outa time - perhaps someone could get one?
http://developer.mozilla.org/en/docs/How_to_get_a_stacktrace_with_WinDbg
 use http://symbols.mozilla.org/thunderbird instead of firefox.
!analyze -v -hang

Comment 9

[Wayne Mery (:wsmwk)]

an imap log might also be helpful http://wiki.mozilla.org/MailNews:Logging
unfortunately I don't have that enabled on the laptop 

compact would not have been factor in my testcase - drafts is a mere 500k

Comment 10

[Wayne Mery (:wsmwk)]

Mac also affected?? 
steps in:
* comment 1 + set autosave to 1 minute
* comment 8

Is there a Mac equivalent for MS's "Not Responding" ?

Comment 11

[Michael A. Pasek]

(In reply to comment #10)
> Is there a Mac equivalent for MS's "Not Responding" ?

Yes; the "spinning pizza wheel" (equivalent to MS hourglass).  In addition, choosing "Force Quit" from the Apple Menu will list "Not Responding" next to the application name (I think that's the actual text, but I don't have an application hung right now to be able to be 100% positive).

Comment 12

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #2)
> My drafts folder is set to "Local folders".
> When I change this back to the account name (i.e. draft stored on the IMAP server) the problem goes away, (snip)

Nigel Green(bug opener), is your profile directory located on network resource?
If yes, Bug 539389 occurs, and if big mail data, slowness is exposed.

Comment 13

[Wayne Mery (:wsmwk)]

Nigel seems to be gone. I emailed one month ago and no response. However, comment 8 lists a testcase for the hang.  

We can probably assume that people probably love to set low values for autosave and for checking new mail :)  .... so it would be great for _someone_ to create a hang stacktrace with protocol log imap:5,timestamp  on any OS.

Comment 14

[Nigel Green]

>> Nigel seems to be gone.
As Mark Twain once said, reports of my demise have been greatly exaggerated.
Sorry that I did not respond to your question, the email did not pop up on my radar for some reason.   At least I saw the most recent post.

In answer to your question in comment #12, my profile directory is located on my local C drive.

The problem still occurs for me.   When the freeze becomes unbearably long, compacting the draft folders reduces the freeze period to an acceptable level again.

I am still using Shredder (current version is 3.0.7pre)

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0 posted on 2009-03-27 07:51:10 PDT)
> Build Identifier: Shredder version 3.0b3pre

> During email composition the Shredder application frequently freezes and
> becomes unresponsive for around 30 seconds.
> This seems to happen about every minute, (snip)

> This has only just started happening since the automatic update which was
> automatically applied to my Shredder installation a couple of days ago.

(In reply to comment #14 posted at 2010-08-17 14:43:07 PDT)
> The problem still occurs for me.   When the freeze becomes unbearably long,
> compacting the draft folders reduces the freeze period to an acceptable level
> again.
> I am still using Shredder (current version is 3.0.7pre)

(Q1) Is your auto-save interval one minute?
> mail.compose.autosave = false
> mail.compose.autosaveinterval = 1
Enable dialog after draft save. You can see auto-save frequency.
> mail.identity.idX.showSaveMsgDlg = true

(Q2) You never executed "Compact" of Drafts folder or "Compact Folders" since 03-27 till 08-17?
What was file size of Drafts(not Drafts.msf) before you did compact?

(Q3) Do you set offline-use=on for Drafts folder?
     (Folder Properties, Synchronization)

If problem of bug 537498 happened on Drafts folder before you executed "Compact" of Drafts, re-synchronization may occur upon each draft save.
As "Compact" issues EXPUNGE, old draft mails flagged \Deleted is removed from server. It reduces time to take for "uid fetch N:* flags" for re-syncronization very much, in both offline-use=off case and offline-use=on case.

Comment 16

[WADA:World Anti-bad-Duping Agency]

Auto-compact may be affected by delete model. What is your choice of "IMAP delete model"?

Comment 17

[WADA:World Anti-bad-Duping Agency]

FYI.
Following are open bugs found by serach for "IMAP compact",
  Bug 439089 Comment #4 : Rough explanation of auto-compact.
  Bug 536447 : Slowness in "msf update" upon compact with "Mark as deleted".
Following is open bugs found by serach for "IMAP expunge".
  Bug 533696 : Bug opener says that manual compact was required to compact
               offline-store file.

Comment 18

[Nigel Green]

(In reply to comment #15)
> (Q1) Is your auto-save interval one minute?
Yes

> (Q2) You never executed "Compact" of Drafts folder or "Compact Folders" since
> 03-27 till 08-17?
Correct.   I don't do it as a matter of routine.

> What was file size of Drafts(not Drafts.msf) before you did compact?
Sorry, I did not record that information before the compact.
The file size now (after compact) is 2K

> (Q3) Do you set offline-use=on for Drafts folder?
>      (Folder Properties, Synchronization)
Yes

Comment 19

[WADA:World Anti-bad-Duping Agency]

(Q4) Which is your choice of "IMAP delete model"?
> Account Settings/Server Settings
>   When I delete message
>     [?] Move to thie folder: ...
>     [?] Just mark as deleted
>     [?] Remove it immesiately

Comment 20

[Nigel Green]

(In reply to comment #19)
> (Q4) Which is your choice of "IMAP delete model"?
> > Account Settings/Server Settings
> >   When I delete message
> >     [?] Move to thie folder: ...
> >     [?] Just mark as deleted
> >     [?] Remove it immesiately

Move to this folder: [Trash]

Comment 21

[WADA:World Anti-bad-Duping Agency]

Because auto-compact(EXPUNGE, and compaction of local offline-store file) doesn't seem invoked in your case, you can probably experience very slowness upon auto-save sooner or later.
Do next, please.
(1) Enable dialog after draft save. You can know that auto-save was done.
> mail.identity.idX.showSaveMsgDlg = true
(2) When you compose mail, change subject or mail body slighty and execute "Save As Draft" as frequently as possible, to force larger offline-store file size and larger not-expunged-yet mail number(mails of \Deleted falg).
(3) When you will experience your problem next time, do followings.
(3-1) Check file size of Drafts(not Drats.msf)
(3-2) Get IMAP log with timestamp for manual "Save As Draft" and/or auto-save.
> https://wiki.mozilla.org/MailNews:Logging
> Win's example : SET NSPR_LOG_MODULES=timestamp,imap:5
As imap:5 writes log for mail data, remove mail data part, sensitive data such as mail adrr, personal nam etc. from log file, please, before attach log file to this bug(==open to public). Never paste long data, please.

Comment 22

[WADA:World Anti-bad-Duping Agency]

(Q5) Do you use local mail folder(POP3 folder, folder of "Local Folders")?
     If yes, do you execute "mail moveor delete, by manually or by filtering"
     on local mail folder(POP3 folder, folder of "Local Folders")?

Comment 23

[Wayne Mery (:wsmwk)]

Bienvenu, is a stacktrace or protocol log of any use here?  Better to get a execution profile?


(In reply to comment #22)
> (Q5) Do you use local mail folder(POP3 folder, folder of "Local Folders")?
>      If yes, do you execute "mail moveor delete, by manually or by filtering"
>      on local mail folder(POP3 folder, folder of "Local Folders")?

wada, the account is imap per Nigel's earlier comments :)

Comment 24

[Rob Bednark]

I've been seeing this bug for maybe 5 years or more in various versions of Thunderbird.  It seemed to have went away for several years, but now I'm seeing it again.  Windows Vista 64-bit, Thunderbird 5.0, IMAP, gmail account.  I have an HTML signature that pulls in a 16kb photo.

I see the request for additional info in the above comments, and may do that if I have time and this bug starts driving me more mad.  I'm happy to see someone else already logged the bug and that developers have been looking at it!

Comment 25

[Rob Bednark]

I just saw this problem again:
- Ubuntu 11.10
- Thunderbird 9.0. 
- I hit "Send" and it hangs on "Status: Attaching...". 
- I'm able to Cancel it.
- I suspect it's due to a photo in my signature (which is shown to me in the message; it seems as if Thunderbird grabs it again when it goes to send the email)
- this was on an email reply
- it also hung on "Attaching..." while doing an autosave to the drafts folder
- while this was hung on "attaching...", I did another reply to the same email, using the same content, and that one successfully sent

Comment 26

[Thomas M.]

duplicate of bug 468159 ?

Comment 27

[WADA:World Anti-bad-Duping Agency]

Setting dependency to bug 532395 for ease of tracking.

Comment 28

[WADA:World Anti-bad-Duping Agency]

Closing as dup of bug 532395. If duping is wrong, please re-open with detailed explanation about difference of your case from that bug.