182231 - crash on import mails (huge archive) from outlook 2000

Status: RESOLVED FIXED

(MailNews Core :: Import, defect)

Description

[Jan Henrik Sylvester]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.2) Gecko/20021126
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.2) Gecko/20021126

I tried to import my mails (outlook.pst file is 207MB) from MS Outlook 2000.
After a few seconds Mozilla crashed. On restart, only 18 mails in the trash
folder from Outlook have been immported.

Reproducible: Always

Steps to Reproduce:
1. Have MS Outlook 2000 installed with 200MB of mails.
2. Start import of mails from Mozilla.
Actual Results:  
crash

Expected Results:  
import mails

Comment 1

[Matthias Versen [:Matti]]

Can you please add a talkback ID from taht crash ?
(run Mozilla/components/talkback.exe to get the ID after Talkback submitted the
crash)

Comment 2

[Jan Henrik Sylvester]

Talkback: TB14447743X
(In Talkback, I reported Outlook was running at that time, but I have reproduced
that bug without Outlook running.)

Comment 3

[Matthias Versen [:Matti]]

The stack is without symbols and useless:-(


*Incident ID:  *   
Incident ID 14447743
Stack Signature ntdll.dll + 0x2a84 (0x77f42a84) 586c93ad
Product ID Mozilla1.2
Build ID 2002112611
Trigger Time 2002-11-27 06:54:40
Platform Win32
Operating System Windows NT 5.1 build 2600
Module ntdll.dll
URL visited
User Comments Imported Mails from Outlook. MS Outlook 2000 was running. There 
were many messages -- the outlook.pst file was 207MB.
Trigger Reason Access violation
Source File Name
Trigger Line No.
Stack Trace

ntdll.dll + 0x2a84 (0x77f42a84)
msvcrt.dll + 0x1ac14 (0x77bfac14)
msvcrt.dll + 0x1ac2a (0x77bfac2a)
gkview.dll + 0x7060 (0x61477060)
gkview.dll + 0x76c4 (0x614776c4)
gkview.dll + 0x77e9 (0x614777e9)
....

Reporter:
Can you please with a english 1.2.1 build ?
(you can use
http://ftp35.newaol.com/pub/mozilla/releases/mozilla1.2.1/mozilla-win32-1.2.1-talkback.zip
extract in in a temp folder, simply run mozilla.exe and after you got the ID
delete the whole folder)

Comment 4

[Jan Henrik Sylvester]

I'm not sure what is the reason for no symbols on the stack...
Can I do anything to provide the necessary information?

BTW: The bug is still there in Mozilla 1.2.1 -- TB14727452W.

Comment 5

[Jan Henrik Sylvester]

The bug happened in original English version with de-at language pack installed.
The last time (1.2.1), English was the language / content activated.

Comment 6

[Matthias Versen [:Matti]]

ntdll.dll + 0x2a84 (0x77f42a84)
msvcrt.dll + 0x1ac14 (0x77bfac14)
msvcrt.dll + 0x1ac2a (0x77bfac2a)
jsd_NewScriptHookProc [c:/builds/seamonkey/mozilla/js/jsd/jsd_scpt.c, line 544]
js_CallNewScriptHook [c:/builds/seamonkey/mozilla/js/src/jsscript.c, line 844]
js_NewScriptFromCG [c:/builds/seamonkey/mozilla/js/src/jsscript.c, line 814]
CompileTokenStream [c:/builds/seamonkey/mozilla/js/src/jsapi.c, line 2856]
JS_CompileUCScriptForPrincipals [c:/builds/seamonkey/mozilla/js/src/jsapi.c,
line 2931]
JS_EvaluateUCScriptForPrincipals [c:/builds/seamonkey/mozilla/js/src/jsapi.c,
line 3380]
nsJSContext::EvaluateString
[c:/builds/seamonkey/mozilla/dom/src/base/nsJSEnvironment.cpp, line 702]
GlobalWindowImpl::RunTimeout
[c:/builds/seamonkey/mozilla/dom/src/base/nsGlobalWindow.cpp, line 4618]
GlobalWindowImpl::TimerCallback
[c:/builds/seamonkey/mozilla/dom/src/base/nsGlobalWindow.cpp, line 4983]
nsTimerImpl::Fire [c:/builds/seamonkey/mozilla/xpcom/threads/nsTimerImpl.cpp,
line 368]
USER32.dll + 0x64a8 (0x77d164a8)
nsXULWindow::ShowModal
[c:/builds/seamonkey/mozilla/xpfe/appshell/src/nsXULWindow.cpp, line 296]
nsContentTreeOwner::ShowAsModal
[c:/builds/seamonkey/mozilla/xpfe/appshell/src/nsContentTreeOwner.cpp, line 449]
nsWindowWatcher::OpenWindowJS
[c:/builds/seamonkey/mozilla/embedding/components/windowwatcher/src/nsWindowWatcher.cpp,
line 783]
GlobalWindowImpl::OpenInternal
[c:/builds/seamonkey/mozilla/dom/src/base/nsGlobalWindow.cpp, line 4279]
GlobalWindowImpl::OpenDialog
[c:/builds/seamonkey/mozilla/dom/src/base/nsGlobalWindow.cpp, line 3039]
XPTC_InvokeByIndex
[c:/builds/seamonkey/mozilla/xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp,
line 106]
XPCWrappedNative::CallMethod
[c:/builds/seamonkey/mozilla/js/src/xpconnect/src/xpcwrappednative.cpp, line 2014]
XPC_WN_CallMethod
[c:/builds/seamonkey/mozilla/js/src/xpconnect/src/xpcwrappednativejsops.cpp,
line 1282]
js_Invoke [c:/builds/seamonkey/mozilla/js/src/jsinterp.c, line 841]
js_Interpret [c:/builds/seamonkey/mozilla/js/src/jsinterp.c, line 2804]
js_Invoke [c:/builds/seamonkey/mozilla/js/src/jsinterp.c, line 857]
js_InternalInvoke [c:/builds/seamonkey/mozilla/js/src/jsinterp.c, line 932]
JS_CallFunctionValue [c:/builds/seamonkey/mozilla/js/src/jsapi.c, line 3433]
nsJSContext::CallEventHandler
[c:/builds/seamonkey/mozilla/dom/src/base/nsJSEnvironment.cpp, line 1044]
nsJSEventListener::HandleEvent
[c:/builds/seamonkey/mozilla/dom/src/events/nsJSEventListener.cpp, line 184]
nsEventListenerManager::HandleEventSubType
[c:/builds/seamonkey/mozilla/content/events/src/nsEventListenerManager.cpp, line
1187]
nsEventListenerManager::HandleEvent
[c:/builds/seamonkey/mozilla/content/events/src/nsEventListenerManager.cpp, line
2183]
nsXULElement::HandleDOMEvent
[c:/builds/seamonkey/mozilla/content/xul/content/src/nsXULElement.cpp, line 3470]
PresShell::HandleDOMEventWithTarget
[c:/builds/seamonkey/mozilla/layout/html/base/src/nsPresShell.cpp, line 6294]
nsMenuFrame::Execute
[c:/builds/seamonkey/mozilla/layout/xul/base/src/nsMenuFrame.cpp, line 1699]
nsMenuFrame::HandleEvent
[c:/builds/seamonkey/mozilla/layout/xul/base/src/nsMenuFrame.cpp, line 463]
PresShell::HandleEventInternal
[c:/builds/seamonkey/mozilla/layout/html/base/src/nsPresShell.cpp, line 6263]
PresShell::HandleEvent
[c:/builds/seamonkey/mozilla/layout/html/base/src/nsPresShell.cpp, line 6169]
nsViewManager::HandleEvent
[c:/builds/seamonkey/mozilla/view/src/nsViewManager.cpp, line 2209]
nsView::HandleEvent [c:/builds/seamonkey/mozilla/view/src/nsView.cpp, line 304]
nsViewManager::DispatchEvent
[c:/builds/seamonkey/mozilla/view/src/nsViewManager.cpp, line 1949]
HandleEvent [c:/builds/seamonkey/mozilla/view/src/nsView.cpp, line 83]
nsWindow::DispatchEvent
[c:/builds/seamonkey/mozilla/widget/src/windows/nsWindow.cpp, line 1073]
nsWindow::DispatchWindowEvent
[c:/builds/seamonkey/mozilla/widget/src/windows/nsWindow.cpp, line 1090]
nsWindow::DispatchMouseEvent
[c:/builds/seamonkey/mozilla/widget/src/windows/nsWindow.cpp, line 5284]
ChildWindow::DispatchMouseEvent
[c:/builds/seamonkey/mozilla/widget/src/windows/nsWindow.cpp, line 5539]
nsWindow::ProcessMessage
[c:/builds/seamonkey/mozilla/widget/src/windows/nsWindow.cpp, line 4066]
nsWindow::WindowProc
[c:/builds/seamonkey/mozilla/widget/src/windows/nsWindow.cpp, line 1339]
USER32.dll + 0x3a68 (0x77d13a68)
USER32.dll + 0x3b37 (0x77d13b37)
USER32.dll + 0x3d91 (0x77d13d91)
USER32.dll + 0x438c (0x77d1438c)
nsAppShellService::Run
[c:/builds/seamonkey/mozilla/xpfe/appshell/src/nsAppShellService.cpp, line 472]
main1 [c:/builds/seamonkey/mozilla/xpfe/bootstrap/nsAppRunner.cpp, line 1557]
main [c:/builds/seamonkey/mozilla/xpfe/bootstrap/nsAppRunner.cpp, line 1905]
WinMain [c:/builds/seamonkey/mozilla/xpfe/bootstrap/nsAppRunner.cpp, line 1925]
WinMainCRTStartup()
kernel32.dll + 0x214c7 (0x77e614c7) 

[e-mail]:Is this a possible JS Engine Bug or I'm again to stupid?

Comment 7

[Phil Schwartau]

Attachment: Talkback stack traces (2)

Let me attach both Talkback stack traces as a text file. Even though
the first one did not have symbols, it still shows which source code
files were involved -

Comment 8

[Phil Schwartau]

The two stack traces were based on the same set of operations,
yet look completely different. The first one ends up in gkwidget
and gkview code, whereas the second on ends up in JS Engine code.

So this probably requires a more complete analysis using Purify
to see what's going wrong. I wouldn't think it's a JS Engine bug.

However, cc'ing rginda, rogerl to look at the 2nd stack trace,
which ends in these frames:

jsd_NewScriptHookProc [c:/builds/seamonkey/mozilla/js/jsd/jsd_scpt.c, line 544]
js_CallNewScriptHook  [c:/builds/seamonkey/mozilla/js/src/jsscript.c, line 844]
js_NewScriptFromCG    [c:/builds/seamonkey/mozilla/js/src/jsscript.c, line 814]
CompileTokenStream    [c:/builds/seamonkey/mozilla/js/src/jsapi.c, line 2856] 


The function js_NewScriptFromCG() calls:

/* Tell the debugger about this compiled script. */
js_CallNewScriptHook(cx, script, fun);


which is:

JS_FRIEND_API(void)
js_CallNewScriptHook(JSContext *cx, JSScript *script, JSFunction *fun)
{
    JSRuntime *rt;
    JSNewScriptHook hook;

    rt = cx->runtime;
    hook = rt->newScriptHook;
    if (hook) {
        /*
         * We use a dummy stack frame to protect the script from a GC caused
         * by debugger-hook execution.
         *
         * XXX We really need a way to manage local roots and such more
         * XXX automatically, at which point we can remove this one-off hack
         * XXX and others within the engine.  See bug 40757 for discussion.
         */
        JSStackFrame dummy;
        
        memset(&dummy, 0, sizeof dummy);
        dummy.down = cx->fp;
        dummy.script = script;
        cx->fp = &dummy;

        hook(cx, script->filename, script->lineno, script, fun,
             rt->newScriptHookData);

        cx->fp = dummy.down;
    }
}

Comment 9

[thecompjoe]

I am experiencing the same problem and I wanted to include my Trackback ID to
this bug: TB432557X.  Hope this helps.

Comment 10

[Unknown W. Brackets]

I recently rebuilt my computer, and in the process decided to move from Outlook
2003 to Mozilla Thunderbird 1.0... and, in doing so (after having to reinstall
Outlook 2003 so it would even convert) I came upon this crash - 217 mb data file
in my case.  My talkback incident ID was 2784286.

So, this is still happening with 1.0 (I'm sure this is no surprise) and is also
a problem with Outlook 2003 (probably no surprise either.)  Both the Import
wizard and the post-install import wizard crashed, but seemed to give different
stacks (of course... see incident #2784394.)

I did, however, find a workaround.  I created a second data file under "Data
File Management", named test.pst.  I think proceeded to move a portion (about
500) of my messages into that account's inbox folder.  Then, I changed my email
account settings to use the Test data file for incoming mail, and removed
Outlook.pst (the default) from the list of "active" pst files.

This took quite a number of steps, doing it 500 at a time, but had additional
benefits too, as I was able to notice that it did not handle marked read/unread
messages (is there a bug for that?) properly in the conversion either; so I used
separate folders for the unread ones and the read ones.

By this, it would seem very apparent that this is an issue with the size of the
converted pst file.  This was probably already known, though.  I might
additionally note that, when I was doing it step by step, my antivirus flagged a
file but this didn't seem to hurt the import at all; I don't think it was the cause.

I also noticed a few duplicates when looking for this, including: bug #245979,
bug #271833, bug #269299, bug #237569, and bug #217588 (Thunderbird, but I'm
quite sure it's the same underlying issue.)

Thanks,
-[Unknown]

Comment 11

[David :Bienvenu]

Attachment: fix for one crash

I don't know if this is the same crash, but this fixes a reproducible crash importing a .pst file that someone sent me - the problem is in the code that breaks up messages with long lines. The underlying issue is that the calling code strips off the spaces at the end of messages by reducing the bodyLen by the number of trailing spaces, but the fixed routine was basically ignoring the bodyLen passed in. I also changed the malloc to a calloc, to make sure the buffer is initialized to nulls.

Comment 12

[David :Bienvenu]

fixed on trunk and branch - the bug caused heap corruption so the crash could be anywhere...I don't think this is exploitable (e.g., tricking the user into replying to a message with a long line) because our quoting code wraps long lines but it's a nasty bug for importers.

Comment 13

[Wayne Mery (:wsmwk)]

*** Bug 277372 has been marked as a duplicate of this bug. ***

Comment 14

[Carsten Book [:Tomcat]]

verified fixed 1.8.1.3 with Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.3) Gecko/20070326 Thunderbird/2.0.0.0 Mnenhy/0.7.5.0 ID:2007032620 and using a 320MB pst File with Outlook 2000 SP3. The Import works as expected - no crash -> adding the verified keyword