451145 - Lexer still tries to produce tokens for non es3 keywords in es3 compatibility mode

Status: VERIFIED WONTFIX

(Tamarin Graveyard :: Self-hosting compiler (ESC), defect)

Description

[Matthieu Riou]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008072820 Firefox/3.0.1
Build Identifier: changeset 618

When starting the ESC interpreter with the -es3 parameter, for Ecmascript 3 compatibility, all definitions of properties starting with 'is' produce a SyntaxError. The REPL or the direct interpreter (passing a file) behave in the same way.

Reproducible: Always

Steps to Reproduce:
1. Start the ESC interpreter
2. Execute something like "var foo = {foo: 1}" or "var foo={}; foo.isFoo = 1;"

Actual Results:  
SyntaxError: (repl):1: String or identifier required, found is
SyntaxError: (repl):1: Expecting semicolon or newline, found Foo

Expected Results:  
Expression should get evaluated properly.

Comment 1

[Matthieu Riou]

I've debugged this a bit and I think I found the root of the problem. In lex-scan.es, anytime there's a keyword that's not in es3 that has to handled, there's a specific condition. For example for the 'is' keyword:

case 115: /* Char::s */
  if (!ESC::flags.es3_keywords && !(notPartOfIdent[src.charCodeAt(curIndex+1)]))
    break bigswitch;
  curIndex += 1;
  return Token::Is;

It seems to me that the test is wrong as if we're in es3 compatibility mode, we *don't* want this token to be produced. Additionally if the next token is not a blank, it shouldn't be produced either (if I understand correctly what notPartOfIdent does). So I think the test should really be:

  if (ESC::flags.es3_keywords || !(notPartOfIdent[src.charCodeAt(curIndex+1)]))
    break bigswitch;

Changing the code this way makes it work for me and doesn't break the ESC compilation. I've had to fix this for the 'is' and the 'type' keyword so I can provide a patch for these. But I think that all these conditions should be updated (I can provide a patch too if you agree).

Comment 2

[Matthieu Riou]

Attachment: Patch to fix es3 compatibility mode switch in lexer for 'if' and 'type' keywords

Comment 3

[Matthieu Riou]

Any chance of somebody looking at this?