784286 - Abnormal memory requirements viewing plain-text mail leading to crash

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[halfdog]

Attachment: test.eml

User Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/17.0 Firefox/17.0 SeaMonkey/2.14a1
Build ID: 20120816003006

Steps to reproduce:

The issue happens when a mail is shown in mail-folder view and occurs frequently with large (100-200kb) text-mails received from mailing lists, but it can also be reproduced by a synthetic mail. The crash occurs only within mail-folder view, not when opening the file directly.

Reproduce:
* Open mail view configured to show 3 regions: folder-list (left), folder-content (top), one message (bottom).
* Open file "test.eml" -- a normal mailing-list message, see bug report attachment
* Select from menu "Message -> Edit as new"
* Message edit opens, in this message select from menu -> save as -> Draft
* Close the edit message and view message window
* Navigate to drafts folder
* Select the newly created draft


Actual results:

* See allocation of at least 1GB of RAM, leading to crash on older machines


Expected results:

* Display of this 250kb e-mail

Comment 1

[halfdog]

Perhaps it might be efficient to see if issue is reproducible on other thunderbird or seamonkey, perhaps on 64-bit system (showing, that not i386-related).

The reproducer works 100% on my installation (seamonkey-2.14a1.en-US.linux-i686-20120816) and just takes 1 minute.

Comment 2

[Philip Chee]

We need a crash stack. When SeaMonkey crashed did you allow the Mozilla crash reporter to send a report to the Mozilla crash-stats server? If so go to:
about:crashes
Right click on the latest entry.
Select "Copy Link Location".
Paste that link in a comment here in this bug.

Comment 3

[halfdog]

After crash, mozilla asks about restoring of old session, but not about sending crash reports. "about:crashes" is empty.

If I understand linux Out-Of-Memory-Killer correctly, it kills (SIGKILL) Seamonkey without giving it a chance to execute only one more machine instruction, thus the memory-grabbing process itself will not be able to capture any stack trace.

Did the reproducer work for you?

Comment 4

[Philip Chee]

1. The crash reporter runs as a separate process.
Did you get SeaMonkey from our official download site or from your Linux repository? Distributions which supply their own version of SeaMonkey usually disable our crash reporter and use their OS crash report function.

2. I'm not using Linux.

Try this:
Edit->Preferences->Browser->Link Behaviour.
Set all settings to use new windows.

Comment 5

[Philip Chee]

> Edit->Preferences->Browser->Link Behaviour.
> Set all settings to use new windows.
Very sorry, I typed this into the wrong tab

Comment 6

[Philip Chee]

Try this:
Edit->Preferences->Advanced

In the Crash Reporter section make sure that the [X] checkbox for "Submit crash reports" is ticked.

Comment 7

[halfdog]

Crash reporter is ticked, another crash triggered it now, so I'm sure, that it is enabled.

The out-of-memory problem does not trigger the crash reporter.

Seamokey: Downloaded 20120816 from https://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk/seamonkey-2.14a1.en-US.linux-i686.tar.bz2
md5: 2c644b7542c11c8d1cf132d0c1941bf1  seamonkey-2.14a1.en-US.linux-i686.tar.bz2

Comment 8

[Phoenix]

Can't reproduce under Windows 7 with attached file.
Can you check, is problem appears in Safe Mode (Help - Restart with Add-ons Disabled)?

Comment 9

[halfdog]

It does not appear in safe mode. (I tried this already, but perhaps I did not do correctly or bug is not completely reproducible. Since good reproduce before, I assume first).

I have no custom addons/plugins except for enigmail. The test-mail does not contain any enigmail specific, but I try to start "normal" mode with only enigmail disabled, to see if bug is triggered with only standard plugins.

Comment 10

[halfdog]

For the records: There was another addon installed, I did not remember any more "Nightly test tools 3.3", but disabling that did not change anything.

Disabling engimail helped, although - or because - the test-mail does not contain any pgp signatures, the crash is then not reproducible any more.

I'll forward to enigmail tracking, to see if this is a generic enigmail problem or one related to my installation.

Comment 11

[Patrick Brunschwig]

I can reproduce the abnormal memory consumption on Mac OS X with Thunderbird Daily, but the process doesn't crash on my machine.

I have an idea why the memory consumption grows so dramatically. It may be triggered by Enigmail, however the reason is very likely in MailNews, as Enigmail only does some JavaScript operations to find out about the message structure. I'll do some investigation.

Comment 12

[halfdog]

For references: enigmail thread: Reply on enigmail mailing list https://admin.hostpoint.ch/pipermail/enigmail-users_enigmail.net/2012-August/000177.html

Comment 13

[Patrick Brunschwig]

This is a MailNews Code / Gloda bug. 

By disabling the following code I don't see the high memory and CPU consumption anymore:

messageDecrypt: function () {
  DEBUG_LOG("messageDecrypt()");
  var cbObj = { isAuto: true };
  MsgHdrToMimeMessage(gFolderDisplay.selectedMessage , cbObj, 
    Enigmail.msg.msgDecryptMimeCb, true, 
    {examineEncryptedParts: true, partsOnDemand: false});
}

Enigmail.msg.msgDecryptMimeCb = function() {
  DEBUG_LOG("msgDecryptMimeCb()");
  // do something ...
}

Looking at the log file that Enigmail generates, I find the following entries:

2012-08-25 16:35:41.968 [DEBUG] messageDecrypt
2012-08-25 16:36:13.863 [DEBUG] msgDecryptMimeCb()

I.e. MsgHdrToMimeMessage() requires 32 seconds with 100% CPU consumption to complete.

Comment 14

[Andrew Sutherland [:asuth] (he/him)]

This sounds like the same thing as:
https://bugzilla.mozilla.org/show_bug.cgi?id=782899

but implies that it's something like I/O coming in byte-by-byte or something pathologically bad.

Comment 15

[:Irving Reid (No longer working on Firefox)]

I agree this sounds like the same issue as bug 782899; unfortunately the local fix we have for that probably won't work here, since I expect Enigmail needs the entire body of the mime parts for further processing.

The I/O is coming in line-by-line, but also getting copied multiple times because of crossing JavaScript compartment boundaries; see https://bugzilla.mozilla.org/show_bug.cgi?id=782899#c26 and https://bugzilla.mozilla.org/show_bug.cgi?id=782899#c36.

I suggest we use 782899 as the bug for the mail summary fix, and keep this bug active to track fixing the underlying JS compartment / string copying issue.

Comment 16

[:Irving Reid (No longer working on Firefox)]

I didn't touch the tracking flag but somehow bugzilla changed it from + to ?, and won't let me change it back.

Comment 17

[:Irving Reid (No longer working on Firefox)]

Attachment: Create an appendBody method in mime part to avoid cross-compartment JS string append

This makes TB perform reasonably well on multiple message selection even if I back out bug 782899, so it's worth a try for some of the other perf issues that came out in TB 15. I'm not sure I want to call it a complete solution, but in early tests it seems to help a lot.

I'll push a try build of this in case any users are interested in checking it.

Comment 18

[Mozilla RelEng Bot]

Try run for 2d5513652000 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=2d5513652000
Results (out of 5 total builds):
    warnings: 4
    failure: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-2d5513652000

Comment 19

[Mark Banner (:standard8)]

The builds are actually available here:

https://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/[e-mail]-2d5513652000/

(ignore the warnings in the previous comments, that's something unrelated in to this patch).

Patrick/Halfdog, if you could give them a quick test to see if it improves the situation or not, that would be very useful.

Comment 20

[Mark Banner (:standard8)]

Comment on attachment 658156 [details] [diff] [review]
Create an appendBody method in mime part to avoid cross-compartment JS string append

Looks good r=me. 

[Triage Comment]
We'll also want to take this onto at least beta, so a=me for those also.

Comment 21

[Mark Banner (:standard8)]

Checked in:

https://hg.mozilla.org/comm-central/rev/e6a2c1c8b546
https://hg.mozilla.org/releases/comm-aurora/rev/339391a32ece
https://hg.mozilla.org/releases/comm-beta/rev/e273f6ba2c37

Comment 22

[Mark Banner (:standard8)]

Comment on attachment 658156 [details] [diff] [review]
Create an appendBody method in mime part to avoid cross-compartment JS string append

[Triage Comment]
The general feedback so far has been that this and bug 782899 are fixing issues, so taking onto the release branch.

Comment 23

[Mark Banner (:standard8)]

Checked in:

http://hg.mozilla.org/releases/comm-release/rev/f7ae9cfd4540