134053 - utf8 conversion problem in nsString.h

Status: RESOLVED FIXED

(MailNews Core :: Internationalization, defect)

Description

[ji]

With the current build, surrogate can be displayed correctly on Navigator
window, but on mail window, it's displayed as a pair of question mark and square
in the subject and displayed as two question marks in mail body.

Two testing mails to follow.

Comment 1

[ji]

Attachment: Two testing mails. Please note that surrogate are displayed alright on mail compose window.

Comment 2

[nhottanscp]

Shanjian, please help looking at this.

Comment 3

[Shanjian Li]

This bug expose 2 problems. One the conversion to utf-8, which is filed as bug
141621. After I applied that patch, imap folder could not be selected any more.
Need more investigation.

Comment 4

[Shanjian Li]

The 2nd problem is also a UTF8 conversion problem, but exist in nsString2.h. 
patch will follow. 

Comment 5

[Shanjian Li]

Attachment: patch

Comment 6

[Shanjian Li]

Attachment: patch with format change.

Comment 7

[Shanjian Li]

roy, could you r=?
scott, could you sr?

Comment 8

[Scott Collins]

Comment on attachment 99708 [details] [diff] [review]
patch with format change. 

sr=scc, but please add a comment pointing the reader to the part of the spec
that describes this math, so that one could verify it if they needed to

Comment 9

[Roy Yokoyama]

shanjian:  Looks good.

Few nits
can you define and replace the hex values?
+ #define PLANE1_BASE           0x00010000  
+ #define UCS2_REPLACEMENT_CHAR 0xfffd     
 ......
+ *mBuffer++ = UCS2_REPLACEMENT_CHAR;

I am not familiar with 'u'.  Does it compile on other compilers?
 ucs4 -= 0x10000u;

Comment 10

[Shanjian Li]

Attachment: patch take roy suggestion

Comment 11

[Shanjian Li]

roy, I modified the patch as you required. u is a standard C feature and it
should not be a problem. Please mark r=.

Comment 12

[Roy Yokoyama]

Comment on attachment 99748 [details] [diff] [review]
patch take roy suggestion

/r=yokoyama
thanks, shanjian

Comment 13

[Shanjian Li]

fix checked in.

Comment 14

[John G. Myers]

This fix introduced a buffer overflow.  Please see the fix attached to bug 172700.