288137 - Update transliterate.properties to Unicode 4.x

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Simon Montagu :smontagu]

As well as regenerating the properties file with a current version of the
Unicode Character Database, this requires making nsIEntityConverter surrogate-aware.

Comment 1

[Simon Montagu :smontagu]

Attachment: Patch

This also includes the fix for bug 264946. 
I'm asking for reviews now, but if Unicode 4.1.0 is released before the freeze
for 1.8b2 I'll do another cycle with the new UnicodeData.txt

Comment 2

[Jungshik Shin]

Comment on attachment 178934 [details] [diff] [review]
Patch

r=jshin

Comment 3

[rbs]

+          // never mind surrogates: none of these entities are outside the BMP
           else if (val > 127 && 

Actually, ȷ (just to cite an example) is a combined character in real life
-- but it was tricked into going to the PUA because not all platforms were
playing nice with surrogate pairs at the time. In the future, these PUA
re-assignmnents of entities might be removed.

Can you just use the surrogate-aware conversion there to bullet-proof the code
against future changes once for all?

Comment 4

[Simon Montagu :smontagu]

Attachment: Added surrogate treatment to serializer

I've tested this with |entity.120077=ȷ| hacked into mathml20.properties.

Transferring r=jshin

Comment 5

[rbs]

Comment on attachment 179052 [details] [diff] [review]
Added surrogate treatment to serializer

sr=rbs

Comment 6

[Simon Montagu :smontagu]

I forgot to mark this as fixed. It was checked in on 2005-04-02