840000 - User Interface becomes unresponsive (beach balls) while processing new incoming imap mails (10 Mega bytes Subject: header was accidentally generated at IMAP server, then Tb freezed for a while)

Status: RESOLVED INVALID

(MailNews Core :: Database, defect)

Description

[christoph_wysseier]

Attachment: thunderbird_stack.txt

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:18.0) Gecko/20100101 Firefox/18.0
Build ID: 20130201065344

Steps to reproduce:

While reading or writing new mail Thunderbird 17.0.2 becomes suddenly unresponsive anomalously and repeatedly. OSX shows a spinning ball for almost a minute, the activity monitor lists Thunderbird as unresponsive.


Actual results:

Thunderbird becomes repeatedly unresponsive. I *suspect* this to be in context of loading new mails from my IMAP accounts (8 accounts with max. 10'000 messages in the inbox) because after some time (20 - 60s) the process recovers and the sound for incoming mails is being played. Besides, during the week-end with less mails the effect was not as regular as during work days with a lot of mails.

Running the process scan of OS X returns attached analysis which always involves "memmove" while being unresponsive.
- I am using the newest stable version of Thunderbird 17.0.2
- All add-ons were deactivated
- All filters were deactivated
- I deactivated global indexing
- I tried several values from 0 to 1000 MB for the cache size
None of this changed the behaviour of becoming unresponsive.

Comment 1

[christoph_wysseier]

Another remark: I also reinstalled Thunderbird, wiped the profile and the IMAP-Cache (msf files) completely without any effect. Conclusion: I am lost and out of ideas.

Comment 2

[:aceman]

Duplicate of bug 731910?

Comment 3

[Wayne Mery (:wsmwk)]

and you also see this in safe mode - https://support.mozillamessaging.com/en-US/kb/safe-mode ?

any gmail, and if so, how many accounts?

if you leave tools | activity manager open, what do you see there and in the status bar (bottom of thunderbird) just after slowness?

FYI cache, afaik, has never been a factor in these types of issues.

(In reply to :aceman from comment #2)
> Duplicate of bug 731910?

bug 731910 may be pop-only. but regardlless, not enough info to know if this is a duplicate.

Comment 4

[christoph_wysseier]

Attachment: Process Analysis right after being unresponsive

Comment 5

[christoph_wysseier]

Thanks for picking this up!

Yes, I also tried safe mode, without any effect. And there is also no AV software active.

No gmail account, all accounts are IMAP hosted on our own mail server running Dovecot.

After the "slowness": "The account [e-mail] is now up-to-date" (translated from German, [e-mail] being randomly one of my accounts). The activity manager shows during the unresponsive period always a similar stack with "memmove$VARIANT$sse3x" and "longcopy" involved (see thunderbird_stack.txt). After it recovers, the process analysis seems to be different every time I run it. I attached one to this bug report.

Besides, it is not slow, it is completely unresponsive for several minutes (02m 39s the last time) and this without any user interface action at the beginning from my side. It starts all of sudden, even when TB is in the background, probably during downloading new mail every 5 minutes for every account.

It seems to become more seldom if I exclude some accounts from the automatic inbox synchronisation but it does not disappear.

Comment 6

[woodengod]

I would not complain that during some activity Thunderbird becomes unresponsive. But I will strongly complain that my user actions (typing, drag and drop, etc) get interrupted by such activity. I consider this as a highest level of user unfriendliness - "we interrupt you at any time to do our stuff, and you will wait for that".

Comment 7

[christoph_wysseier]

You are right and I do complain strongly because this often happens during composing a new mail. I just wanted to accentuate that no user interaction causes the unresponsiveness.

Comment 8

[Wayne Mery (:wsmwk)]

mconley,irving would it be better to get profiler data?

this may be a duplicate, but too early in the morning for me to find bug#

Comment 9

[Mike Conley (:mconley) (:⚙️)]

Yes, let's get some profiler data here, if we can.

Please install the add-on from https://addons.mozilla.org/en-us/thunderbird/addon/gecko-profiler/

Then, in the status bar, click on the red "Disabled" so that it shows up as "Enabled".  Then, recreate the performance problem, and click "Dump Profile".

A new tab should show up with a profile in it. Find the "Share" button, submit the profile, and then post the link it provides in here.

Note that you'll definitely need Javascript enabled in order to do any of this.

Comment 10

[christoph_wysseier]

Attachment: Geckoprofiler raises an exception on startup

I would love to send you profiler data but the geckoprofiler Add-on raises an error on startup. My MacBook seems to be jinxed.

Comment 11

[Mike Conley (:mconley) (:⚙️)]

Christoph:

Is that with Daily? Or Release?

-Mike

Comment 12

[christoph_wysseier]

Mike:
The attachement was with Release... but at this moment I can also confirm the exception of the add-on with Daily:

(NS_ERROR_FILE_NOT_FOUND) [nsIXPCComponents_Utils.import]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/XPIProvider.jsm -> jar:file:///Users/chris/Library/Thunderbird/Profiles/k8fd6cjv.default/extensions/[e-mail]!/bootstrap.js -> resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/api-utils/lib/loader.js -> resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/geckoprofiler/lib/main.js :: <TOP_LEVEL> :: line 42"  data: no]
"]

Comment 13

[Mike Conley (:mconley) (:⚙️)]

Christoph:

Hm. For Daily, try v1.11.11 - available here: https://addons.mozilla.org/en-us/thunderbird/addon/gecko-profiler/versions/

-Mike

Comment 14

[christoph_wysseier]

Mike:

Thx a lot for your support! Profiler is running AND coincidentally the problem appeared twice on Daily too. I uploaded two data sets... one with more data after the unresponsiveness and another to show the length of the problem. The start of it seems not to be part of the data as far as I understand because it just lasts too long, right?
1. http://people.mozilla.com/~bgirard/cleopatra/?1361390920367#report=73adb10288a63089527fb924d38838c01eca8114
2. http://people.mozilla.com/~bgirard/cleopatra/?1361391215203#report=8430a5227bc453ea7d2c21108e1e738591faf27c

Hope this helps! - Chris

Comment 15

[Mike Conley (:mconley) (:⚙️)]

Hrm - our symbol server seems not to be mapping those address spaces to symbols. I'm talking to a member of the perf team about this right now. Once that's fixed, we'll need to take your profile again (sorry!)

-Mike

Comment 16

[woodengod]

I think I have a clue. I had a local folder containing about 200000 messages (search indexing disabled). The unresponsiveness would occur every time I try to access the folder, manually or when some filter tries to fill it. I archived most of the messages, and reduced the number of the messages in the folder about four times, and then most of the unresponsiveness went away.
But I'd repeat, one can solve the local reason for such a hanging, but I think in general the priority of the user interaction needs to go on top compared with the rest of the processes/threads that go in Thunderbird.

Comment 17

[christoph_wysseier]

My largest (IMAP) folder does not contain more than 10'000 messages. It may be that large folders impact the performance but I am surprised about the development of the occurrence. The unresponsiveness started suddenly not step by step therefore your explanation does not convince me. I would bet that it appeared with TB v17 but I am not sure enough. Let's hope that a newly process analysis reveals the real reason...

Comment 18

[WADA:World Anti-bad-Duping Agency]

Because IMAP API is currently synchronous(not asynchronous), UI of main window/main task waits for end of IMAP operations which is done at other thread(s). So, if IMAP operations takes long, UI freeze can be observed.
IMAP related operations roughly consists of(inaccurate, though);
(a) fetch flags 1:* or NextUID:* to know about new mails in SELECT'ed folder
    at a cached connection, or STAT Mbox if not SELECT'ed at any cached
    connection and select Mbox + uid m:n fetch flags if new mail exists,
    upon each new mail check interval.
(b) uid m:n fetch body[headers] header(Subject From ...) to get header data
(c) uid m:n fetch body[] if offline-store=On folder/auto-sync enabled
    or uid x fetch body[] if mail click at offline-use=On folder before auto-sync
(d) uid x fetch body[BODYSTRUCTURE] upon mail click if offline-use=Off,
    and uid x fetch body[a.b] for required sub parts
(e) uid y fetch fetch body[].2048 for previewText by Biff
(f) if Gloda is enabled, Indexer of Gloda starts indexing for mail of (c) 

To rule out suspects, do following, please.
(1) to rule out problem by add-on, start Tb with -safe-mode
(2) to rule out problem in (f), disable Global Search and Indexer, please.
(3) to rule out problem around (e), disable show preview text request.
  mail.biff.alert.show_preview = false
  See bug 823838 comment #12 for disabling from UI
(4) to rule out problem due to (c) for all mails in big IMAP folder, due to something wrong(incompatibility in .msf, panacea.dat, etc., corrupted/outdated msf, ...), disable auto-sync for problem analysis, please.
  mail.server.serverX.autosync_offline_stores = false
  (doesn't exist. create via Config Editor, then restart Tb)

(In reply to christoph_wysseier from comment #5)
> It seems to become more seldom if I exclude some accounts from the automatic
> inbox synchronisation but it does not disappear.

(5) To see affect by "auto-sync of an Mbox", do following please.
    After (1) to (4),
      mail.server.serverX.autosync_offline_stores = true for an account only
      Set all folders of the account to Offline-use=Off
      Set only Inbox of the account to Offline-Use=On
   What is workload increase by auto-sync upon each new mail check interval? 
(6) To see affect by simultaneous new mail checking.
   Default of max cached connections is 5.
   So, fetch of mail data is invoked at multiple cached connections at same time.
   Check difference from "max cached connections=1", please.
   (any IMAP accesses is serialized because only one connection is permitted.) 
   (max=1 may cause other problem due to resource contention. )
   (If so, compare with max=2, please.)

(In reply to woodengod from comment #16)
> but I think in general the priority of the user interaction needs to go on top
> compared with the rest of the processes/threads that go in Thunderbird.

Read bug 842371 comment #1 & #2, and open separate bug for it if you believe it's needed, please.

Comment 19

[woodengod]

(In reply to WADA from comment #18)
> Read bug 842371 comment #1 & #2, and open separate bug for it if you believe
> it's needed, please.
Thanks for the information. Bug 842371 looks like it covers my concern. As for this bug I don't have time to experiment very much but I will tell some details, because I think they already contain clues:
- The account that shows problems is Gmail IMAP account
- I have the search indexing turned off
- I have mail.biff.alert.show_preview = false
- I have a filter type "Checking mail (after classification) or manually run" which moves messages from the inbox to a "local" folder (a folder created in the section "Local Folders")
- The "local" folder contained already more than 200000 messages. When I archived 3/4 of them and compacted the "local" folder, I noticed that the unresponsiveness during the inbox checks substantially decreased.
- I don't open often the "local" folder in which the filter moves the messages into, but when I do, the user interface freezes again for a couple of seconds. After I have done the archiving, opening that folder became faster.

Hope this helps

Comment 20

[christoph_wysseier]

I found the reason for the unresponsiveness while following the instructions of WADA. It was, not surprisingly, a corrupted mail file on the mail server.

The result of the instruction was: 1,2,3: no change, 4: problem disappeared
5: To deactivate the Offline-Use beside the INBOX I clicked the Archive folder of one of my accounts that I do not really use but observe. I got the beach balls again and then analysed the content of the folder on our mail server. I found a corrupted mail file where all the content including header and 10 MB of attachment were on one single line (no line feed), starting with "Subject: ". I guess that TB tried to load that mega subject line and/or received this line from the mail server while auto-synching this folder which led to the unresponsiveness. Since I deleted the file, TB never became unresponsive and works now like a charm again.

I am sorry to have bothered you. It was really impossible to identify this single e-mail within 100 of folders and thousands of mails, just a coincidence helped. Thank you for your help and work around Thunderbird! - Chris

Comment 21

[Wayne Mery (:wsmwk)]

> found a corrupted mail file where all the content including header and 10 MB of
attachment were on one single line (no line feed), starting with "Subject: ". 

Nice detective work. I think we leave this open to find and fix the cause - why the folder got this problem.

/me flips the imap / database coin - it comes up database :)

Comment 22

[christoph_wysseier]

As you wish... if you need more information, I am in. I also archived the corrupt mail if you would like to analyse it yourself but the content (job application with CV) would be confidential.

Comment 23

[WADA:World Anti-bad-Duping Agency]

(In reply to woodengod from comment #19)
> - The "local" folder contained already more than 200000 messages. When I
> archived 3/4 of them and compacted (snip)

Bug 837530 is a report of performance problem in "mass archive"(=="mass mail move" to Archives), and some bugs are already opeed for performance proble in "mass delete" case(=="mass mail move" to Trash).
Please be carfull when you do "mass mail move"/"mass mail delete".

(In reply to woodengod from comment #19)
> Thanks for the information. Bug 842371 looks like it covers my concern.

As wrtten in bug 842371 comment #2, even though that bug is for similar concern, that bug doesn't include conern about "unresponsiveness due to IMAP access". Separate bug is needed for IMAP case.

Comment 24

[:aceman]

Wayne, the corruption was on the IMAP server, not in TB's database. However, do you think the bad file originated from TB and the server just stored any junk that TB sent it?

christoph, can you tell us if that email was a received one or if you can find out if it was moved, produced by Thunderbird in the past?

Comment 25

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #24)
> Wayne, the corruption was on the IMAP server, not in TB's database. However,
> do you think the bad file originated from TB and the server just stored any
> junk that TB sent it?

no idea. but it would be good for us to protect ourselves from the bad effect. Or the cause of course if we caused it.

Comment 26

[christoph_wysseier]

(In reply to :aceman from comment #24)
> christoph, can you tell us if that email was a received one or if you can
> find out if it was moved, produced by Thunderbird in the past?

I believe that the corruption originated from the generating application and not TB but impossible to prove. After deleting the mail on the servers file system I also needed to delete the MSF of this folder to resolve the problem completely so I assume the mail content was completely downloaded once.

Comment 27

[WADA:World Anti-bad-Duping Agency]

(In reply to christoph_wysseier from comment #20)
> I found a corrupted mail file where all the content including header
> and 10 MB of attachment were on one single line (no line feed), starting with "Subject: ".

If "10MB Subject: header", many problems can occur at same time, repeatedly.
Problem when "10 MB Subject: header" is fetched(it surely takes very long time), problem in too long subject header processing after fetch, problem in handling of "10 MB Subject: header data in .msf", problem in very long text display upon subject display at thread pane, problem in very long text display upon subject display at header pane pane, ...

Anyway, I think your case(bug opener's case, comment #0) is better closed as INVALID as you did once, because I believe "10 MB Subject: header" is very rare and I don't think Tb's torelance with "10 MB Subject: header" is mandatory, even though I think Tb should be able to bear "10 MB Subject: header" attack :-)

Comment 28

[christoph_wysseier]

(In reply to WADA from comment #27)
> Anyway, I think your case(bug opener's case, comment #0) is better closed as
> INVALID as you did once, because I believe "10 MB Subject: header" is very
> rare and I don't think Tb's torelance with "10 MB Subject: header" is
> mandatory, even though I think Tb should be able to bear "10 MB Subject:
> header" attack :-)

100% agreed... but you guys have to decide about the case status.

Comment 29

[:aceman]

Probably nobody expected TB to download and render a 10MB long field that usually is only several bytes long. So any operations on the field likely are synchronous and do not yield to the UI until finish. Rewriting all the operations to not block the UI would probably be hard.

christoph, maybe you would be able to put the file again into the server in some test folder and try other email applications how good they work on it? E.g. Outlook? Does the server have a web interface to check too?

Comment 30

[woodengod]

(In reply to WADA from comment #23)
> Bug 837530 is a report of performance problem in "mass archive"(=="mass mail
> move" to Archives), and some bugs are already opeed for performance proble
> in "mass delete" case(=="mass mail move" to Trash).
> Please be carfull when you do "mass mail move"/"mass mail delete".
> 
Archiving messages from a huge folder gave me one time hassle, possibly due to the number of messages. The smaller it got after I delete the messages (then compact of course) I have archived, the faster I can access it. As long as I don't use archiving frequently, I don't have big problems with it.

> As wrtten in bug 842371 comment #2, even though that bug is for similar
> concern, that bug doesn't include conern about "unresponsiveness due to IMAP
> access". Separate bug is needed for IMAP case.
Reducing the size of the size of the local folder gave me the speed up. I didn't observe problems opening IMAP folders in the mean time. As far as I can see bug 842371 covers unresponsiveness due to the disk I/O, and what happens to me looks quite lot like it.

Thanks for the clues.

Comment 31

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #29)
> christoph, maybe you would be able to put the file again into the server in
> some test folder and try other email applications how good they work on it?
> E.g. Outlook? Does the server have a web interface to check too?

:aceman, are you requesting christoph to reproduce problem he saw?

You can do such test very easily by yourself, if you actually want to test.
(1) By script, generate file which contains following lines;
    From - Mon Apr 01 01:02:03 2013[CRLF]
    Subject:[SP][10MB_data_without_space][CRLF]
    From:[SP][5MB_data]@[5MB_data_without_space_without_@_with_some_dots][CRLF]
    To:[SP][5MB_data]@[5MB_data_without_space_without_@_with_some_dots][CRLF]
    [CRLF]
    hello[CRLF]
(2) Copy the file under Tb's mail directory(call FolderX), restart Tb.
    (Above is malformed mail because of "no Date: header", but Tb can accept it)
(3) Click FolderX => you can see many phenomena by long header/text.
(4) Drag&Drop the mail to an IMAP folder(call FolderY)
(5) "Repair Folder" of FolderY
(6) Open FolderY, view the mail
(7) At Web Interface, view mail in FolderY, By Outlook, open FolderY, view mail.

:aceman, if you concern about Tb's issues in special case such as above, open separate bug for it, please. I can provide you script for step (1). I don't think 10MB or 5MB is required. Far shorter data is sufficient for your concern.

Anyway, closing as INVALID again, as bug opener did once.