415135 - Slow/Hang/lock-up on long From lines in header with high cpu

Status: RESOLVED FIXED

(MailNews Core :: MIME, defect)

Description

[Jon Watte]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
Build Identifier: 2.0.0.9 (20071031)

I received a message with a very long From line. Whenever this message moved into the message list view through scrolling, Mozilla hung/locked up. Removing the inbox index file, and/or rebuilding it did not fix the problem. Removing this particular From: line from the message using a text editor resolved the problem.

Reproducible: Always

Steps to Reproduce:
1. Receive this particular message.
2. Scroll your Inbox so it should include this message.

Actual Results:  
3. Observe hung mail client.

Expected Results:  
3. Message is displayed, with some truncation of From header

I will enclose the message itself in an attachment.

Comment 1

[Jon Watte]

Attachment: email message causing hang

I cut this message out of my Inbox. It was the last message in the Inbox; don't know if that matters to the reproducibility of the bug.

Comment 2

[Karsten Düsterloh]

Whew, 182644 characters in the From: line (RfC 2822 allows for 998), almost all of them illegal, because of the missing charset declaration. 
Anyway, we shouldn't hang on that...

Which version of which program are you using?
You're talking of Mozilla, your User-Agent says Firefox, but product is set to Thunderbird...

Comment 3

[Karsten Düsterloh]

Anyway, happens for SeaMonkey as well, and the problem is, that comi18n.cpp::utf8_nextchar is just grossly slow...
<http://mxr.mozilla.org/seamonkey/source/mailnews/mime/src/comi18n.cpp#170>

Comment 4

[Karsten Düsterloh]

Attachment: optimized UTF-8 recognition

The function utf8_nextchar calls strlen for each of the 180000 characters, which results in a 12s hang even on my machine _per_ usage of the header. On display, it's parsed several times (eg on each repaint), which makes the app very unresponsive...

Comment 5

[Jon Watte]

I'm using Thunderbird 2.0.whatever (as pasted in the "product release" field).

That old implementation of utf8_nextchar() was criminally bad! strlen() just isn't a useful function for parsing text.

Comment 6

[[e-mail]]

Comment on attachment 300752 [details] [diff] [review]
optimized UTF-8 recognition

Well, the code looks fine, but I have to admit it looks to me as if return str + 1; would work all the time...

Comment 7

[Karsten Düsterloh]

Comment on attachment 300752 [details] [diff] [review]
optimized UTF-8 recognition

It's worth having on branch as well, imo - after nine years. ;-)

Comment 8

[Karsten Düsterloh]

Landed on trunk, thus closing. Waiting for branch approval, though.

Jon, you might want to grab tomorrows nightly build and check with a *test* profile.

Comment 9

[Christopher Aillon (sabbatical, not receiving bugmail)]

Comment on attachment 300752 [details] [diff] [review]
optimized UTF-8 recognition

Is this still wanted for 1.8.0.15?  If so, please make sure this gets approval in the 1.8.1.x branch, too.

Comment 11

[Daniel Veditz [:dveditz]]

Comment on attachment 300752 [details] [diff] [review]
optimized UTF-8 recognition

approved for 1.8.1.13, a=dveditz for release-drivers

Comment 12

[Karsten Düsterloh]

Comment on attachment 300752 [details] [diff] [review]
optimized UTF-8 recognition

Landed on MOZILLA_1_8_BRANCH.
I didn't intend to ask for 1.8.0.x, thanks for correcting me. :)

Comment 14

[Al Billings [:abillings - ex-MoCo]]

I verified this in Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13pre) Gecko/2008030903 Thunderbird/2.0.0.13pre using the trash folder from bug 424349. It hung TB 2.0.0.12 and the above 2.0.0.13pre build was able to deal with this.

Marking this as verified for 1.8.1.13.

Comment 15

[Wayne Mery (:wsmwk)]

broaden summary