726888 - Move JS's Unicode character data tables and stuff into mfbt

Status: NEW

(Core :: MFBT, defect)

Description

[Jeff Walden [:Waldo]]

The JS engine uses various Unicode tables for stuff like uppercasing and lowercasing.  This is stuff the browser could and should be using as well, and not reimplementing itself as I believe it does now.  As a first step, the scripts, character data, headers, and so on should move into mfbt.  To do this, we need to resolve a couple little issues.

First, we need a library to stick the extern tables and stuff into.  Bug 717540 should fix this very shortly.  \o/

Second, we need to decide on a type to use for 16-bit characters/Unicode code units.  This stuff currently uses jschar, which is fine for the JS engine but uncool for everyone else.  char16_t from C11 and C++11 is the logical choice.  But nothing's ever that easy.  This will require some trickery to get working, seeing as in C++11 char16_t is a keyword, in C11 char16_t is a type defined by <uchar.h>, and all sorts of downrev compilers will support neither of these.  (Although, <uchar.h> in C++11 is specified to not define char16_t, so it's possible that header will work in C++11 and C11 compilers both.)  Funtimes.

Anyway.  Something to do sometime soon, although given everything got updated to Unicode 6.1 just now, it'll probably be a little while before this move (and subsequent consolidation of reimplemented details) is too helpful.

Comment 1

[Jeff Walden [:Waldo]]

Of course, we could just use uint16_t for now, too, and assume that we can rewrite our way to char16_t when it becomes feasible to do so.

Comment 2

[Tom S]

I am not sure how important that is, but at the moment the engine does not yet support special casing Bug 672042.

Oh and JavaScript unicode is kind of strange, I am not sure it really matches with what the browsers needs.

Comment 3

[Simon Giesecke [:sg] [he/him]]

I guess this is about the functions defined in https://searchfox.org/mozilla-central/source/js/src/util/Unicode.h?