23608 - Identifiers not ECMA-compliant

Status: VERIFIED FIXED

(Core :: JavaScript Engine, defect, P3)

Description

[Waldemar Horwat]

ECMA Edition 3 requires identifiers to allow Unicode letters and digits.  Our
current engine does not support these:

js> eval("var \u03B2 = 20")
10: SyntaxError: illegal character:
10: var * = 20
10: ....^

This is different from bug 23607 -- the escape sequence inside the string is
evaluated as part of the string constant, generating a Unicode character that is
passed to the evaluator.  Of course, after performing the above eval, one ought
to be able to type:

js> \u03b2

and get the result:

20

Comment 1

[Mike McCabe]

Reassigning to Roger.

Comment 2

[Waldemar Horwat]

One can find out whether a character c can start or continue an identifier by
decoding based on the value of (JS_CCODE(c) & 0x00070000).  The variants are
listed in the comment in jsstr.c:

 *  3 bits      0  may not be part of an identifier
 *              1  ignorable control; may continue a Unicode identifier or JS
 *                 identifier
 *              2  may continue a JS identifier but not a Unicode identifier
 *                 (unused)
 *              3  may continue a Unicode identifier or JS identifier
 *              4  is a JS whitespace character
 *              5  may start or continue a JS identifier;
 *                 may continue but not start a Unicode identifier (_)
 *              6  may start or continue a JS identifier but not a Unicode
 *                 identifier ($)
 *              7  may start or continue a Unicode identifier or JS identifier
 *              Thus:
 *                 5, 6, 7 may start a JS identifier
 *                 1, 2, 3, 5, 6, 7 may continue a JS identifier
 *                 7 may start a Unicode identifier
 *                 1, 3, 5, 7 may continue a Unicode identifier
 *                 1 is ignorable within an identifier
 *                 4 is JS whitespace

Don't use JS_ISALPHA, JS_ISALNUM, JS_ISWORD, JS_ISIDENT, or JS_ISIDENT2, none of
which accepts the proper set of characters.  These macros should be fixed or
removed.

Comment 3

[rogerl (gone)]

Fix checked in.

Comment 4

[Robert Ginda]

Covered by testcase ecma_3/Unicode/uc-003.js
Marking Verified.