1464311 - Can BytecodeEmitter emit subtrees separately and compose them later?

Status: RESOLVED INCOMPLETE

(Core :: JavaScript Engine, enhancement, P3)

Description

[Tooru Fujisawa [:arai]]

related to bug 1455547, bug 1455548, 1456006 comment #1, bug 1456404 comment #2, and https://github.com/binast/ecmascript-binary-ast/issues/35

If we perform streaming compilation from non-seekable .binjs file, we should emit
subtrees separately, and then compose them into single bytecode later,
because the field order in the file doesn't match to the bytecode order and compilation order.

What we should think about are:
  * composing bytecode
  * composing source note
  * linking/patching jump labels for local jump (branch)
  * linking/patching jump labels for non-local jump (break/continue/return)
  * composing
    * CGConstList
    * CGObjectList
    * CGScopeList
    * CGTryNoteList
    * CGScopeNoteList
    * CGYieldAndAwaitOffsetList
  * state management between subtrees for
    * EmitterScope
    * NestableControl
    * EmitterScope
    * TDZCheckCache

and maybe some more..

Comment 1

[Tooru Fujisawa [:arai]]

So far, JumpList should be re-implemented, given that the jump can happen between multiple bytecodelets

I'm going to reuse EmitSection struct for bytecodelet, which can store code and src note.

I wam thinking about switch statement, and in that case the stack depth for each subtree is known at the beginning,
so we could just emulate the situation for each subtree, and then just concatenate them in the expected order later,
with extra bytecode for jump etc.

then, we have to handle break statement there, and it's currently handled with JumpList (BreakableControl.breaks).
JumpList is implemented as linked list which uses bytecode's jump target (32-bit) as storage.

https://searchfox.org/mozilla-central/rev/5a744713370ec47969595e369fd5125f123e6d24/js/src/frontend/BytecodeEmitter.h#113-166

Now that the jump can happen between 2 EmitSections, and the link's elements can be inside multiple different EmitSections,
they cannot be linked with single 32-bit integer,
but the pair of EmitSection* and offset inside EmitSection's code.

I'll implement different BreakableControl structure which can handle it, to see if that's sufficient for handling inter-EmitSection jump.

Comment 2

[Tooru Fujisawa [:arai]]

here's WIP patch series for emitting switch statement with bytecode composition.
  https://hg.mozilla.org/try/pushloghtml?changeset=cb27b37b0369832596151a53119b00eef5c88508

the last patch ( https://hg.mozilla.org/try/rev/b324daf8d997 ) adds the following feature:
  * struct for inter-EmitSection jump
  * switching multiple EmitSection (previously there were only prelude and main), with saving/restoring stack depth
  * emitting and patching long jumps
  * NestableControl for long jump target
  * support long jump and EmitSection in SwitchEmitter

it implements only basic feature, but at least it's working.

I'll try experimenting with subtree-based optimization (TABLESWITCH optimization for this case) next.

Comment 3

[Tooru Fujisawa [:arai]]

(In reply to Tooru Fujisawa [:arai] from comment #2)
>   * switching multiple EmitSection (previously there were only prelude and
> main), with saving/restoring stack depth

*prologue and main


then, currently it's not handling the following:
  * CGConstList
  * CGObjectList
  * CGScopeList
  * CGTryNoteList
  * CGScopeNoteList
  * CGYieldAndAwaitOffsetList
and also some scope/control things

and I see there are some references to prologue and the length/offset etc,
so we should support composition for them.

Comment 4

[Tooru Fujisawa [:arai]]

just for note, here's the plan about streaming compilation (and where this bug is)

• decouple bytecode emitting feature from ParseNode (bug 1473796)
  • move part of BytecodeEmitter into helper classes (partially done)
  • make BytecodeEmitter a consumer of these classes (partially done)
• make it possible to compose multiple bytecodelet=BytecodeSection (this bug = bug 1464311)
  • move fields for bytecode and related data to BytecodeSection class (bug 1535994)
  • make JumpList possible to handle jump between BytecodeSections (bug 1536037)
  • make BytecodeEmitter to switch current BytecodeSection, just like previous EmitSection
  • make BytecodeEmitter/helper classes able to handle multiple bytecodelet
• generate bytecode for part of AST, and compose them later (bug 1455547)

Comment 5

[Tooru Fujisawa [:arai]]

very hacky patch, but passed try
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e0b98bd8f5ebfcd55b05b4b6a609309de4a5bee8

Comment 6

[Matthew Gaudet (he/him) [:mgaudet]]

Resolving BinAST bugs as Incomplete.