352042 - Using Unicode escape sequences should not allow characters in identifiers which are disallowed by grammar

Status: RESOLVED INACTIVE

(Rhino Graveyard :: Core, defect)

Description

[Martin Honnen]

Rhino allows characters escaped with Unicode escape sequences in identifiers even if the characters are not allowed by the grammar for identifiers.


ECMAScript edition 3 section 6 says this about Unicode escape sequences:

"In string literals, regular expression literals and identifiers, any character (code point) may also be expressed as a
Unicode escape sequence consisting of six characters, namely \u plus four hexadecimal digits. [...] Within an identifier, the
escape sequence contributes one character to the identifier."

Section 7.6 additionally says:

"Unicode escape sequences are also permitted in identifiers, where they contribute a single character to the
identifier, as computed by the CV of the UnicodeEscapeSequence (see section 7.8.4). [...] A UnicodeEscapeSequence cannot be
used to put a character into an identifier that would otherwise be illegal. In other words, if a \
UnicodeEscapeSequence sequence were replaced by its UnicodeEscapeSequence's CV, the result must still be a
valid Identifier that has the exact same sequence of characters as the original Identifier."

Rhino however parses the following statement where the identifier has the character 'v' plus the Unicode escape sequence \u0020 (which represents the space ' ' character)
  var v\u0020 = 1;
without any error and creates a variable with name 'v '. As the space character is not allowed in an identifier using it escaped should not result in a valid indentifier and Rhino should report a syntax error.

Test case is

var v\u0020 = 1;
(function () {
  for (var propertyName in this) {
    print("propertyName: '" + propertyName + "'; length: " + propertyName.length);
  }
}());

Result when run with Rhino shell (Rhino 1.6 release 3 2006 09 10) is 

propertyName: 'v '; length: 2

In my understanding a syntax error should be given.

Comment 1

[Martin Honnen]

Attachment: test case to be run in script engine shell

Comment 2

[Miao Yu]

In ECMA262 3rd Edition, Non-Latin Unicode characters are allowed in identifiers, string literals, regular expression literals and comments.
Rhino conforms to this standard, so I dont think this is bug.

Comment 3

[Sylvestre Ledru [:Sylvestre]]

Closing. Bug management is now done here:
https://github.com/mozilla/rhino