1692577 - Consider using CompilationState directly from CompilationStencil during delazification

Status: RESOLVED FIXED

(Core :: JavaScript Engine, task, P2)

Description

[Ted Campbell [:tcampbell]]

The CompilationState structure is stack-allocated and used directly by parser to accumulate stencil data. We currently then convert this to the final stencil structure. In the case of delazification (and probably eval), if we aren't going to cache the stencil then we should just use the CompilationState directly. Since the CompilationStencil uses Spans, we can just point into the CompilationState similar to we do for XDR decode.

Comment 1

[Tooru Fujisawa [:arai]]

I have another idea about optimizing away the copy inside CompilationState::finish, in bug 1692648.

Comment 2

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 1: Move Vector-variant of CompilationStencil out of CompilationState.

To return Vector-variant of CompilationStencil as the result of compilation,
it needs to be allocated separately from other CompilationState.

Depends on D105148

Comment 3

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 2: Support encoding ExtensibleCompilationStencil, and use ExtensibleCompilationStencil in self-hosting.

Depends on D105149

Comment 4

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 3: Use ExtensibleCompilationStencil in self-hosting.

Depends on D105150

Comment 5

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 4: Make CmpilationStencil and ExtensibleCompilationStencil convertible each other.

Depends on D105151

Comment 6

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 5: Add CompilationStencilMerger.

Depends on D105152

Comment 7

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 6: Use CompilationStencilMerger in XDRIncrementalStencilEncoder.

Also add XDRStencilEncoder for non-incremental case, and
cleanup XDRStencilDecoder.

StencilDelazificationSet and gcOutputForDelazification become unused,
and will be removed by later patches.

Depends on D105153

Comment 8

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 7: Remove StencilDelazificationSet.

Now all stencils don't have associated delazification.

Depends on D105154

Comment 9

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 8: Remove gcOutputForDelazification.

Depends on D105155

Comment 10

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 9: Merge AbstractBaseCompilationStencil and AbstractCompilationStencil.

Depends on D105156

Comment 11

[Tooru Fujisawa [:arai]]

uh, sorry, I wrote wrong bug number :P

Comment 12

[Phabricator Automation]

Comment on attachment 9203159 [details]
Bug 1692577 - Part 1: Move Vector-variant of CompilationStencil out of CompilationState.

Revision D105149 was moved to bug 1687095. Setting attachment 9203159 [details] to obsolete.

Comment 13

[Phabricator Automation]

Comment on attachment 9203160 [details]
Bug 1692577 - Part 2: Support encoding ExtensibleCompilationStencil, and use ExtensibleCompilationStencil in self-hosting.

Revision D105150 was moved to bug 1687095. Setting attachment 9203160 [details] to obsolete.

Comment 14

[Phabricator Automation]

Comment on attachment 9203161 [details]
Bug 1692577 - Part 3: Use ExtensibleCompilationStencil in self-hosting.

Revision D105151 was moved to bug 1687095. Setting attachment 9203161 [details] to obsolete.

Comment 15

[Phabricator Automation]

Comment on attachment 9203162 [details]
Bug 1692577 - Part 4: Make CmpilationStencil and ExtensibleCompilationStencil convertible each other.

Revision D105152 was moved to bug 1687095. Setting attachment 9203162 [details] to obsolete.

Comment 16

[Phabricator Automation]

Comment on attachment 9203163 [details]
Bug 1692577 - Part 5: Add CompilationStencilMerger.

Revision D105153 was moved to bug 1687095. Setting attachment 9203163 [details] to obsolete.

Comment 17

[Phabricator Automation]

Comment on attachment 9203164 [details]
Bug 1692577 - Part 6: Use CompilationStencilMerger in XDRIncrementalStencilEncoder.

Revision D105154 was moved to bug 1687095. Setting attachment 9203164 [details] to obsolete.

Comment 18

[Phabricator Automation]

Comment on attachment 9203165 [details]
Bug 1692577 - Part 7: Remove StencilDelazificationSet.

Revision D105155 was moved to bug 1687095. Setting attachment 9203165 [details] to obsolete.

Comment 19

[Phabricator Automation]

Comment on attachment 9203166 [details]
Bug 1692577 - Part 8: Remove gcOutputForDelazification.

Revision D105156 was moved to bug 1687095. Setting attachment 9203166 [details] to obsolete.

Comment 20

[Phabricator Automation]

Comment on attachment 9203167 [details]
Bug 1692577 - Part 9: Merge AbstractBaseCompilationStencil and AbstractCompilationStencil.

Revision D105157 was moved to bug 1687095. Setting attachment 9203167 [details] to obsolete.

Comment 21

[Tooru Fujisawa [:arai]]

bug 1687095 patch achieves it, as part of making "merge" operation work.
https://phabricator.services.mozilla.com/D105154

that patch stack, including bug 1692648, moves stencil-related fields of CompilationState into Vector-variant of CompilationStencil,
and makes the instantiation logic compatible with it.

Comment 22

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 1: Add BorrowingCompilationStencil. r=tcampbell!

BorrowingCompilationStencil provides a CompilationStencil instance that
borrows all ExtensibleCompilationStencil data,

Depends on D105892

Comment 23

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 2: Move finish to ExtensibleCompilationStencil. r=tcampbell!

Depends on D105893

Comment 24

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 3: Make more CompilationStencil& parameter const. r=tcampbell!

Depends on D105894

Comment 25

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 4: Unify Compiler class interface. r=tcampbell!

Make all classes to follow init() + compile() + stencil().
stencil() returns a reference to ExtensibleCompilationStencil, and consumer
can either use BorrowingCompilationStencil to create temporary
CompilationStencil from it, or use ExtensibleCompilationStencil::finish to
get non-temporary CompilationStencil.

Depends on D105895

Comment 26

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 5: Add CompileGlobalScriptToStencilAndMaybeInstantiate. r=tcampbell!

To support instantiating without heap-allocate CompilationStencil, modify
CompileGlobalScriptToStencil to receive either CompilationStencil pointer or
CompilationGCOutput pointer, and if CompilationGCOutput pointer is passed,
instantiate it internally.

Depends on D105896

Comment 27

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 6: Add frontend::CompileGlobalScriptToExtensibleStencil. r=tcampbell!

For the case the consumer needs heap-allocated ExtensibleCompilationStencil,
make CompileGlobalScriptToStencilAndMaybeInstantiate possible to also return
ExtensibleCompilationStencil.

Depends on D105897

Comment 28

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 7: Cleanup JSRuntime::initSelfHosting. r=tcampbell!

Also cleanup JSRuntime::initSelfHosting not to reuse already-initialized
CompilationInput from decode to compile.

Depends on D105898

Comment 29

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 8: Add ExtensibleCompilationStencil::{sizeOfExcludingThis,sizeOfIncludingThis}. r=tcampbell!

Depends on D105899

Comment 30

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 9: Use ExtensibleCompilationStencil in off-thread script compilation task. r=tcampbell!

Depends on D105900

Comment 31

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 10: Add ParseModuleToStencilAndMaybeInstantiate. r=tcampbell!

Depends on D105901

Comment 32

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 11: Use ExtensibleCompilationStencil in off-thread module compilation task r=tcampbell!

Depends on D105902

Comment 33

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 12: Add frontend::CompileLazyFunction. r=tcampbell!

Depends on D105903

Comment 34

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 13: Use frontend::CompileGlobalScriptToExtensibleStencil in CompileSourceBufferAndStartIncrementalEncoding. r=tcampbell!

Depends on D105904

Comment 35

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 14: Use frontend::CompileGlobalScriptToExtensibleStencil in CompileStencilXDR. r=tcampbell!

Depends on D105905

Comment 36

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 15: Use frontend::CompileGlobalScriptToExtensibleStencil in DumpStencil. r=tcampbell!

Depends on D105906

Comment 37

[Tooru Fujisawa [:arai]]

Attachment: Bug 1692577 - Part 16: Remove unnecessary and redundant ExtensibleCompilationStencil::finish from reflect_parse. r=tcampbell!

reflect_parse wasn't using CompilationStencil, but uses
ExtensibleCompilationStencil in CompilationState.

Depends on D105907

Comment 38

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a21a62d5bb75
Part 1: Add BorrowingCompilationStencil. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/386088c6a895
Part 2: Move finish to ExtensibleCompilationStencil. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/a0aa13363ec0
Part 3: Make more CompilationStencil& parameter const. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/24c81af7409a
Part 4: Unify Compiler class interface. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/3ef7987a19b9
Part 5: Add CompileGlobalScriptToStencilAndMaybeInstantiate. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/82c26c5615f5
Part 6: Add frontend::CompileGlobalScriptToExtensibleStencil. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/55d1272f20fd
Part 7: Cleanup JSRuntime::initSelfHosting. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/80291c93484e
Part 8: Add ExtensibleCompilationStencil::{sizeOfExcludingThis,sizeOfIncludingThis}. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/c18e9015fe3b
Part 9: Use ExtensibleCompilationStencil in off-thread script compilation task. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/2ff79c45b786
Part 10: Add ParseModuleToStencilAndMaybeInstantiate. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/271602dcf18d
Part 11: Use ExtensibleCompilationStencil in off-thread module compilation task r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/954ab0b62640
Part 12: Add frontend::CompileLazyFunction. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/3f4ceb6458d7
Part 13: Use frontend::CompileGlobalScriptToExtensibleStencil in CompileSourceBufferAndStartIncrementalEncoding. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/b7676660de86
Part 14: Use frontend::CompileGlobalScriptToExtensibleStencil in CompileStencilXDR. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/eb0886f3a78a
Part 15: Use frontend::CompileGlobalScriptToExtensibleStencil in DumpStencil. r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/6a1f9a908a97
Part 16: Remove unnecessary and redundant ExtensibleCompilationStencil::finish from reflect_parse. r=tcampbell

Comment 39

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/a21a62d5bb75
https://hg.mozilla.org/mozilla-central/rev/386088c6a895
https://hg.mozilla.org/mozilla-central/rev/a0aa13363ec0
https://hg.mozilla.org/mozilla-central/rev/24c81af7409a
https://hg.mozilla.org/mozilla-central/rev/3ef7987a19b9
https://hg.mozilla.org/mozilla-central/rev/82c26c5615f5
https://hg.mozilla.org/mozilla-central/rev/55d1272f20fd
https://hg.mozilla.org/mozilla-central/rev/80291c93484e
https://hg.mozilla.org/mozilla-central/rev/c18e9015fe3b
https://hg.mozilla.org/mozilla-central/rev/2ff79c45b786
https://hg.mozilla.org/mozilla-central/rev/271602dcf18d
https://hg.mozilla.org/mozilla-central/rev/954ab0b62640
https://hg.mozilla.org/mozilla-central/rev/3f4ceb6458d7
https://hg.mozilla.org/mozilla-central/rev/b7676660de86
https://hg.mozilla.org/mozilla-central/rev/eb0886f3a78a
https://hg.mozilla.org/mozilla-central/rev/6a1f9a908a97