1385724 - Page edit fails due to use of Unicode characters greater than U+FFFF

Status: RESOLVED DUPLICATE of bug 857438

(developer.mozilla.org Graveyard :: Editing, enhancement)

Description

[tangmu]

What did you do?
================
1. Edit the page https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Text_formatting
 
2. Include two non-BMP Unicode characters 

Comment 1

[tangmu]

I spent about an hour carefully editing the page. Now it appears that most of my work has been lost. Ironically, the error I was correcting in the documentation is closely related to the bug that evidently exists in the implementation of MDN. I'm willing to help fix the bug, as well as correct the documentation.

Comment 2

[tangmu]

This bug-report page itself has a bug in it. My "description" above did not in fact end with "2. Include two non-BMP Unicode characters"; after that I included the characters themselves and additional description. That part of my bug report itself has disappeared. From now on I'll refrain from including non-BMP characters in these comments; I'll stick with ASCII. The two characters in question are:

U+1F600 Emoji grinning face

U+20000 The first character in CJK Extension B

Comment 3

[tangmu]

Bugzilla is written in Perl, which supports Unicode up to U+10FFFF.

One possible cause of the bug is MySQL, if not configured correctly. "The character set named utf8 uses a maximum of three bytes per character and contains only BMP characters. As of MySQL 5.5.3, the utf8mb4 character set uses a maximum of four bytes per character and supports supplementary characters"

https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html

Comment 4

[John Whitlock [:jwhitlock]]

Yes, this is a limitation of our current MySQL encoding. Our plan is to fix this by moving to PostgreSQL.

Comment 5

[tangmu]

So this bug has been known for four years already!! I'll add further comment at https://bugzilla.mozilla.org/show_bug.cgi?id=857438