Closed Bug 1302282 Opened 8 years ago Closed 2 years ago

Rewrite the frontend: lexer

Categories

(Core :: JavaScript Engine, defect, P3)

Product:
Core
Component:
JavaScript Engine
Type:
defect

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1424394

People

(Reporter: shu, Unassigned)

References

Details

(Keywords: triage-deferred) 

I wanted to experiment with templatizing TokenStream on char16_t and Latin1Char for performance reasons. It turns out I can't even prototype this easily because of existing template complexity. Much frontend code keep references to TokenStream for reasons unrelated to lexing:

1. For position information. ParseHandlers keep a tokenStream for this reason.
2. For error reporting.

This means if we were to templatizing TokenStream, basically everything in the frontend gets a 2nd template param. This explodes too quickly, since Parser is already templatized on ParseHandler.

To move forward, we probably need to extricate position handling in ParseHandlers and error reporting at large out of TokenStream. Once TokenStream is parsed down to *just* lex, it should be easier to prototype lexing Latin1 scripts.

Extricating position handling needs to be done for SyntaxParseHandler anyways, which currently just gets the position wrong.
See Also: → 1283712
Keywords: triage-deferred
Priority: -- → P3
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.