416399 - Massive reworking of the front end

Status: RESOLVED FIXED

(Tamarin Graveyard :: Self-hosting compiler (ESC), defect)

Description

[Lars T Hansen]

Massive reworking of the entire front end for performance and clarity:

  - switch-based lexer with optimizations for common cases
  - unthreading of the token stream in the parser
  - removal of debugging calls in parser (the compiler could insert this code)
  - lexer/parser interface has limited-length token queue, avoiding consing
  - concretization of token representation to avoid function calls on critical path
  - hash tables instead of linear lists to hash cons (!) tokens
  - allow esc.es to compile multiple files in one go to avoid JIT warmup
  - and more

This patch assumes that #416248 has been applied.  It subsumes #416249.

Comment 2

[Lars T Hansen]

Attachment: Big patch

Comment 3

[Lars T Hansen]

Changeset 370