1773319 - Add public Stencil APIs to compile without JSContext

Status: RESOLVED FIXED

(Core :: JavaScript Engine, task, P3)

Description

[Tooru Fujisawa [:arai]]

The idea is to move the thread handling from JS helper thread to embedding, and let the embedding trigger compilation itself off-thread, instead of calling the off-thread API from the main thread and waiting for the callback.

In order to move the off-thread parsing from JS helper thread to embedding, we need to expose internal functions called in helper threads, as public APIs.

This depends on removing JSContext-dependency from the parsing.

Comment 1

[Tooru Fujisawa [:arai]]

The target is internal functions that's called by ParseTasks

• CompileToStencilTask::parse
  • frontend::CompileGlobalScriptToStencil
  • frontend::PrepareForInstantiate
• CompileModuleToStencilTask::parse
  • frontend::ParseModuleToStencil
• DecodeStencilTask::parse
  • frontend::CompilationStencil::deserializeStencils

and maybe some small helpers for initialization etc.

Comment 2

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Make cx parameter to CompileGlobalScriptToStencilAndMaybeInstantiate optional r=arai!

Comment 3

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Intro API to create/destroy FrontendContext, ParseScript r=arai!

Need to fill out jsapi-test for more details and use cases.

AsyncParseGlobalScript() has a stackLimit parameter because I don't expect to
know what the limit for the TaskController task is, or even that the function
is being run by TaskController.

Comment 4

[Bryan Thrall [:bthrall]]

Bug 1759123 and bug 1799960 are cleaning up code that is no longer used, this API doesn't require that they are done.

Comment 5

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Rename maybeCx parameter of frontend::PrepareForInstantiate r=arai!

Depends on D167461

Comment 6

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Add CompilationInput parameter to ParseGlobalScript r=arai!

The CompilationInput will be needed for PrepareForInstantiate().

Depends on D168680

Comment 7

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Handle Global enclosingScope in CompilationInput::dump() r=arai!

Depends on D168681

Comment 8

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Add PrepareForInstantiate() to API r=arai!

The only failure cases for PrepareForInstantiate() are OOM, which will be
checked in a separate commit.

Depends on D168682

Comment 9

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Use ParseGlobalScript in compileToStencil testing function, add specific tests r=arai!

Depends on D168683

Comment 10

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Check failure propagation out of ParseGlobalScript() r=arai!

Depends on D169131

Comment 11

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Fix segfault when populating delazification cache r=arai!

Depends on D169132

Comment 12

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Rename CompileGlobalScriptToStencil r=arai!

Since CompileGlobalScriptToStencil(FrontendContext*...) is a thin wrapper
around CompileGlobalScriptToStencil(JSContext*...), it makes sense for the
names to match.

compileToStencil should match as well, since the script is not just parsed,
but actually compiled to a stencil.

jsapi-test testCompileNonsyntactic's test name conflicted with
testCompileScript's, so I renamed it to more precisely match what it is
testing.

Depends on D169133

Comment 13

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Pass LifoAlloc to ParseModuleToStencil() as parameter r=arai!

Heading toward making the JSContext parameter optional.

Depends on D169134

Comment 14

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Rename optional maybeCx parameters r=arai!

Depends on D169655

Comment 15

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Get supportedImportAssertions from FrontendContext r=arai!

By default, FrontendContext will not list any supportedImportAssertions, but
there is setSupportedImportAssumptions() to set that list once (just like
JS::SetSupportedImportAssumptions() for JSRuntime).

Depends on D169656

Comment 16

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Add CompileModuleScriptToStencil() and use it r=arai!

Depends on D169657

Comment 17

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Compile nonsyntactic scopes with FrontendContext only r=arai!

I didn't see a way to detect nonsyntactic scope in the Stencil, but there was
an easy way on the instantiated script.

Depends on D169658

Comment 18

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Add JS::SetSupportedImportAssertions(FrontendContext*) to API r=arai!

Depends on D169734

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ce294fa9953c
Make cx parameter to CompileGlobalScriptToStencilAndMaybeInstantiate optional r=arai
https://hg.mozilla.org/integration/autoland/rev/c10a596d7395
Intro API to create/destroy FrontendContext, ParseScript r=arai
https://hg.mozilla.org/integration/autoland/rev/1068cb71a7bf
Rename maybeCx parameter of frontend::PrepareForInstantiate r=arai
https://hg.mozilla.org/integration/autoland/rev/6c952cee28e2
Add CompilationInput parameter to ParseGlobalScript r=arai
https://hg.mozilla.org/integration/autoland/rev/8bdbbd322d34
Handle Global enclosingScope in CompilationInput::dump() r=arai
https://hg.mozilla.org/integration/autoland/rev/83d4b82795fe
Add PrepareForInstantiate() to API r=arai
https://hg.mozilla.org/integration/autoland/rev/4e7e1ec10ed2
Use ParseGlobalScript in compileToStencil testing function, add specific tests r=arai
https://hg.mozilla.org/integration/autoland/rev/26a8fc40437f
Check failure propagation out of ParseGlobalScript() r=arai
https://hg.mozilla.org/integration/autoland/rev/c3fd0107003b
Fix segfault when populating delazification cache r=arai
https://hg.mozilla.org/integration/autoland/rev/3005fd755990
Rename CompileGlobalScriptToStencil r=arai
https://hg.mozilla.org/integration/autoland/rev/7f773c7ee0fd
Pass LifoAlloc to ParseModuleToStencil() as parameter r=arai
https://hg.mozilla.org/integration/autoland/rev/50c3ae074be3
Rename optional maybeCx parameters r=arai
https://hg.mozilla.org/integration/autoland/rev/c4e12157c001
Get supportedImportAssertions from FrontendContext r=arai
https://hg.mozilla.org/integration/autoland/rev/d1f138fae2c4
Add CompileModuleScriptToStencil() and use it r=arai
https://hg.mozilla.org/integration/autoland/rev/55c0f82e8645
Compile nonsyntactic scopes with FrontendContext only r=arai
https://hg.mozilla.org/integration/autoland/rev/432d50b65ee6
Add JS::SetSupportedImportAssertions(FrontendContext*) to API r=arai

Comment 20

[Cristian Tuns]

Backed out for causing hazard bustages in TestingFunctions.cpp

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | hazards | unrooted 'stencilInput' of type 'mozilla::UniquePtr<js::frontend::CompilationInput, JS::DeletePolicy<js::frontend::CompilationInput> >' live across GC call at js/src/builtin/TestingFunctions.cpp:6373

Comment 21

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Rework FrontendContex::gcSafeCallback so rooting hazard analysis understands it r=arai!

Previously, rooting hazard analysis could report calls to gcSafeCallback as
triggering a GC because of the two calls to callback(). Now it understands that
the single call to callback() will not trigger a GC.

Depends on D170083

Comment 22

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Resolve rooting hazard in GeneralParser::mustMatchToken() r=arai!

The rooting hazard analysis assumes function pointer arguments might trigger
GC, but none of the functions used for mustMatchToken() will do so.

Depends on D172333

Comment 23

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Resolve root hazard in matchIntegerAfterFirstDigit() and matchInteger() r=arai!

The rooting hazard analysis assumes function pointer arguments might trigger
GC, but none of the functions used for matchInteger() or
matchIntegerAfterFirstDigit() will do so.

Depends on D172334

Comment 24

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Use wrapper around CompilationInput instead of UniquePtr r=arai!

Embedders would not be able to use UniquePtr<CompilationInput> since
CompilationInput is opaque to them.

This also resolves a rooting hazard with the UniquePtr because the JS script
might have a 'uses asm' directive, which the hazard analysis reports could be a
problem if CompilationInput has GC pointers. We know the CompilationInput has
no GC pointers, however because we are only compiling to Stencil.

Further, CompilationStorage::isValid() only returns true if its
CompilationInput contains no GC pointers.

Depends on D172335

Comment 25

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Assert if ScriptSource::setDisplayURL() is called more than once r=arai!

There is no code path currently which will call setDisplayURL() if it has
already been set. The previous code to report a warning could not execute, and
caused the rooting hazard analysis to report a hazard when setDisplayURL() was
called from the CompileScript API. Removing the call to WarnNumberLatin1()
avoids the hazard.

Depends on D172336

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e87b56a40133
Make cx parameter to CompileGlobalScriptToStencilAndMaybeInstantiate optional r=arai
https://hg.mozilla.org/integration/autoland/rev/4d9649652cad
Intro API to create/destroy FrontendContext, ParseScript r=arai
https://hg.mozilla.org/integration/autoland/rev/ac1327af1c8f
Rename maybeCx parameter of frontend::PrepareForInstantiate r=arai
https://hg.mozilla.org/integration/autoland/rev/feb45ca89d78
Add CompilationInput parameter to ParseGlobalScript r=arai
https://hg.mozilla.org/integration/autoland/rev/804f4087add7
Handle Global enclosingScope in CompilationInput::dump() r=arai
https://hg.mozilla.org/integration/autoland/rev/1ab9e2b34325
Add PrepareForInstantiate() to API r=arai
https://hg.mozilla.org/integration/autoland/rev/063afc991fd0
Use ParseGlobalScript in compileToStencil testing function, add specific tests r=arai
https://hg.mozilla.org/integration/autoland/rev/6c3d1aae58f7
Check failure propagation out of ParseGlobalScript() r=arai
https://hg.mozilla.org/integration/autoland/rev/3aa2a44717d2
Fix segfault when populating delazification cache r=arai
https://hg.mozilla.org/integration/autoland/rev/2ce2fb7d0ad2
Rename CompileGlobalScriptToStencil r=arai
https://hg.mozilla.org/integration/autoland/rev/5399b1528245
Pass LifoAlloc to ParseModuleToStencil() as parameter r=arai
https://hg.mozilla.org/integration/autoland/rev/3be347da0c96
Rename optional maybeCx parameters r=arai
https://hg.mozilla.org/integration/autoland/rev/2606ae2fe37c
Get supportedImportAssertions from FrontendContext r=arai
https://hg.mozilla.org/integration/autoland/rev/e7d0f790f097
Add CompileModuleScriptToStencil() and use it r=arai
https://hg.mozilla.org/integration/autoland/rev/dbe2b23d6e54
Compile nonsyntactic scopes with FrontendContext only r=arai
https://hg.mozilla.org/integration/autoland/rev/fa8d4e0d012b
Add JS::SetSupportedImportAssertions(FrontendContext*) to API r=arai
https://hg.mozilla.org/integration/autoland/rev/6b4bb6a9527e
Rework FrontendContex::gcSafeCallback so rooting hazard analysis understands it r=arai
https://hg.mozilla.org/integration/autoland/rev/4ef61ac2f4fa
Use wrapper around CompilationInput instead of UniquePtr r=arai
https://hg.mozilla.org/integration/autoland/rev/c62f9fb3170f
Assert if ScriptSource::setDisplayURL() is called more than once r=arai

Comment 27

[Bryan Thrall [:bthrall]]

Attachment: Bug 1773319 - Only define isGCSafe for DEBUG builds

It is only used in MOZ_ASSERT, so it was unused in non-DEBUG builds.

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/97ea5a6d852f
Only define isGCSafe for DEBUG builds

Comment 29

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/e87b56a40133
https://hg.mozilla.org/mozilla-central/rev/4d9649652cad
https://hg.mozilla.org/mozilla-central/rev/ac1327af1c8f
https://hg.mozilla.org/mozilla-central/rev/feb45ca89d78
https://hg.mozilla.org/mozilla-central/rev/804f4087add7
https://hg.mozilla.org/mozilla-central/rev/1ab9e2b34325
https://hg.mozilla.org/mozilla-central/rev/063afc991fd0
https://hg.mozilla.org/mozilla-central/rev/6c3d1aae58f7
https://hg.mozilla.org/mozilla-central/rev/3aa2a44717d2
https://hg.mozilla.org/mozilla-central/rev/2ce2fb7d0ad2
https://hg.mozilla.org/mozilla-central/rev/5399b1528245
https://hg.mozilla.org/mozilla-central/rev/3be347da0c96
https://hg.mozilla.org/mozilla-central/rev/2606ae2fe37c
https://hg.mozilla.org/mozilla-central/rev/e7d0f790f097
https://hg.mozilla.org/mozilla-central/rev/dbe2b23d6e54
https://hg.mozilla.org/mozilla-central/rev/fa8d4e0d012b
https://hg.mozilla.org/mozilla-central/rev/6b4bb6a9527e
https://hg.mozilla.org/mozilla-central/rev/4ef61ac2f4fa
https://hg.mozilla.org/mozilla-central/rev/c62f9fb3170f
https://hg.mozilla.org/mozilla-central/rev/97ea5a6d852f