Abnormal memory requirements viewing plain-text mail leading to crash

RESOLVED FIXED in Thunderbird 18.0

Status

MailNews Core
Backend
--
critical
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: halfdog, Assigned: Irving)

Tracking

({perf})

Thunderbird 18.0
Dependency tree / graph

Thunderbird Tracking Flags

(thunderbird15+ fixed, thunderbird16+ fixed, thunderbird17+ fixed)

Details

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
Created attachment 653693 [details]
test.eml

User Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/17.0 Firefox/17.0 SeaMonkey/2.14a1
Build ID: 20120816003006

Steps to reproduce:

The issue happens when a mail is shown in mail-folder view and occurs frequently with large (100-200kb) text-mails received from mailing lists, but it can also be reproduced by a synthetic mail. The crash occurs only within mail-folder view, not when opening the file directly.

Reproduce:
* Open mail view configured to show 3 regions: folder-list (left), folder-content (top), one message (bottom).
* Open file "test.eml" -- a normal mailing-list message, see bug report attachment
* Select from menu "Message -> Edit as new"
* Message edit opens, in this message select from menu -> save as -> Draft
* Close the edit message and view message window
* Navigate to drafts folder
* Select the newly created draft


Actual results:

* See allocation of at least 1GB of RAM, leading to crash on older machines


Expected results:

* Display of this 250kb e-mail
(Reporter)

Comment 1

5 years ago
Perhaps it might be efficient to see if issue is reproducible on other thunderbird or seamonkey, perhaps on 64-bit system (showing, that not i386-related).

The reproducer works 100% on my installation (seamonkey-2.14a1.en-US.linux-i686-20120816) and just takes 1 minute.

Comment 2

5 years ago
We need a crash stack. When SeaMonkey crashed did you allow the Mozilla crash reporter to send a report to the Mozilla crash-stats server? If so go to:
about:crashes
Right click on the latest entry.
Select "Copy Link Location".
Paste that link in a comment here in this bug.
(Reporter)

Comment 3

5 years ago
After crash, mozilla asks about restoring of old session, but not about sending crash reports. "about:crashes" is empty.

If I understand linux Out-Of-Memory-Killer correctly, it kills (SIGKILL) Seamonkey without giving it a chance to execute only one more machine instruction, thus the memory-grabbing process itself will not be able to capture any stack trace.

Did the reproducer work for you?

Comment 4

5 years ago
1. The crash reporter runs as a separate process.
Did you get SeaMonkey from our official download site or from your Linux repository? Distributions which supply their own version of SeaMonkey usually disable our crash reporter and use their OS crash report function.

2. I'm not using Linux.

Try this:
Edit->Preferences->Browser->Link Behaviour.
Set all settings to use new windows.

Comment 5

5 years ago
> Edit->Preferences->Browser->Link Behaviour.
> Set all settings to use new windows.
Very sorry, I typed this into the wrong tab

Comment 6

5 years ago
Try this:
Edit->Preferences->Advanced

In the Crash Reporter section make sure that the [X] checkbox for "Submit crash reports" is ticked.
(Reporter)

Comment 7

5 years ago
Crash reporter is ticked, another crash triggered it now, so I'm sure, that it is enabled.

The out-of-memory problem does not trigger the crash reporter.

Seamokey: Downloaded 20120816 from https://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk/seamonkey-2.14a1.en-US.linux-i686.tar.bz2
md5: 2c644b7542c11c8d1cf132d0c1941bf1  seamonkey-2.14a1.en-US.linux-i686.tar.bz2

Comment 8

5 years ago
Can't reproduce under Windows 7 with attached file.
Can you check, is problem appears in Safe Mode (Help - Restart with Add-ons Disabled)?
(Reporter)

Comment 9

5 years ago
It does not appear in safe mode. (I tried this already, but perhaps I did not do correctly or bug is not completely reproducible. Since good reproduce before, I assume first).

I have no custom addons/plugins except for enigmail. The test-mail does not contain any enigmail specific, but I try to start "normal" mode with only enigmail disabled, to see if bug is triggered with only standard plugins.
(Reporter)

Comment 10

5 years ago
For the records: There was another addon installed, I did not remember any more "Nightly test tools 3.3", but disabling that did not change anything.

Disabling engimail helped, although - or because - the test-mail does not contain any pgp signatures, the crash is then not reproducible any more.

I'll forward to enigmail tracking, to see if this is a generic enigmail problem or one related to my installation.

Comment 11

5 years ago
I can reproduce the abnormal memory consumption on Mac OS X with Thunderbird Daily, but the process doesn't crash on my machine.

I have an idea why the memory consumption grows so dramatically. It may be triggered by Enigmail, however the reason is very likely in MailNews, as Enigmail only does some JavaScript operations to find out about the message structure. I'll do some investigation.
Status: UNCONFIRMED → NEW
Ever confirmed: true
(Reporter)

Comment 12

5 years ago
For references: enigmail thread: Reply on enigmail mailing list https://admin.hostpoint.ch/pipermail/enigmail-users_enigmail.net/2012-August/000177.html

Comment 13

5 years ago
This is a MailNews Code / Gloda bug. 

By disabling the following code I don't see the high memory and CPU consumption anymore:

messageDecrypt: function () {
  DEBUG_LOG("messageDecrypt()");
  var cbObj = { isAuto: true };
  MsgHdrToMimeMessage(gFolderDisplay.selectedMessage , cbObj, 
    Enigmail.msg.msgDecryptMimeCb, true, 
    {examineEncryptedParts: true, partsOnDemand: false});
}

Enigmail.msg.msgDecryptMimeCb = function() {
  DEBUG_LOG("msgDecryptMimeCb()");
  // do something ...
}

Looking at the log file that Enigmail generates, I find the following entries:

2012-08-25 16:35:41.968 [DEBUG] messageDecrypt
2012-08-25 16:36:13.863 [DEBUG] msgDecryptMimeCb()

I.e. MsgHdrToMimeMessage() requires 32 seconds with 100% CPU consumption to complete.
Severity: normal → critical
Component: MailNews: Message Display → Backend
OS: Linux → All
Product: SeaMonkey → MailNews Core
Hardware: x86 → All
Version: Trunk → 17
Keywords: perf
tracking-thunderbird17: --- → ?
This sounds like the same thing as:
https://bugzilla.mozilla.org/show_bug.cgi?id=782899

but implies that it's something like I/O coming in byte-by-byte or something pathologically bad.
tracking-thunderbird17: ? → +
I agree this sounds like the same issue as bug 782899; unfortunately the local fix we have for that probably won't work here, since I expect Enigmail needs the entire body of the mime parts for further processing.

The I/O is coming in line-by-line, but also getting copied multiple times because of crossing JavaScript compartment boundaries; see https://bugzilla.mozilla.org/show_bug.cgi?id=782899#c26 and https://bugzilla.mozilla.org/show_bug.cgi?id=782899#c36.

I suggest we use 782899 as the bug for the mail summary fix, and keep this bug active to track fixing the underlying JS compartment / string copying issue.
tracking-thunderbird17: + → ?
Depends on: 782899
I didn't touch the tracking flag but somehow bugzilla changed it from + to ?, and won't let me change it back.
tracking-thunderbird17: ? → +
Created attachment 658156 [details] [diff] [review]
Create an appendBody method in mime part to avoid cross-compartment JS string append

This makes TB perform reasonably well on multiple message selection even if I back out bug 782899, so it's worth a try for some of the other perf issues that came out in TB 15. I'm not sure I want to call it a complete solution, but in early tests it seems to help a lot.

I'll push a try build of this in case any users are interested in checking it.
Attachment #658156 - Flags: review?(mbanner)

Comment 18

5 years ago
Try run for 2d5513652000 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=2d5513652000
Results (out of 5 total builds):
    warnings: 4
    failure: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/ireid@mozilla.com-2d5513652000
The builds are actually available here:

https://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/ireid@mozilla.com-2d5513652000/

(ignore the warnings in the previous comments, that's something unrelated in to this patch).

Patrick/Halfdog, if you could give them a quick test to see if it improves the situation or not, that would be very useful.
Assignee: nobody → irving
Comment on attachment 658156 [details] [diff] [review]
Create an appendBody method in mime part to avoid cross-compartment JS string append

Looks good r=me. 

[Triage Comment]
We'll also want to take this onto at least beta, so a=me for those also.
Attachment #658156 - Flags: review?(mbanner)
Attachment #658156 - Flags: review+
Attachment #658156 - Flags: approval-comm-beta+
Attachment #658156 - Flags: approval-comm-aurora+
Checked in:

https://hg.mozilla.org/comm-central/rev/e6a2c1c8b546
https://hg.mozilla.org/releases/comm-aurora/rev/339391a32ece
https://hg.mozilla.org/releases/comm-beta/rev/e273f6ba2c37
Status: NEW → RESOLVED
Last Resolved: 5 years ago
status-thunderbird16: --- → fixed
status-thunderbird17: --- → fixed
tracking-thunderbird15: --- → +
tracking-thunderbird16: --- → +
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 18.0
Comment on attachment 658156 [details] [diff] [review]
Create an appendBody method in mime part to avoid cross-compartment JS string append

[Triage Comment]
The general feedback so far has been that this and bug 782899 are fixing issues, so taking onto the release branch.
Attachment #658156 - Flags: approval-comm-release+
Checked in:

http://hg.mozilla.org/releases/comm-release/rev/f7ae9cfd4540
status-thunderbird15: --- → fixed

Updated

5 years ago
Blocks: 789890
Duplicate of this bug: 789890
You need to log in before you can comment on or make changes to this bug.