1756003 - Fuzz off-thread concurrent-depth-first eager delazification strategy.

Status: RESOLVED FIXED

(Core :: JavaScript Engine, task, P1)

Description

[Nicolas B. Pierron [:nbp]]

While testing the eager delazification strategy, it seems that some mochitest were failing due to this toggle, while it is not supposed to have any observable behavior on the execution.

Currently fuzzing is already possible using OffThreadCompileToStencil with options to toggle it on. However, I suspect that the little number of test cases which not be enough to help fuzzers focus on this configuration.

We should improve fuzzing of off-thread eager delazification to figure out what might be going wrong.

Note: off-thread eager delazification is not enabled by default, and will soon have a preference to turn it on/off (off by default for now)

Comment 1

[Nicolas B. Pierron [:nbp]]

Attachment: Bug 1756003 - Let's always start off-thread delazification.

This patch is not meant for landing, but only for testing purposes, in order to
stress-test the delazification from stencils.

!!! THIS PATCH SHOULD NOT LAND !!!

Comment 2

[Nicolas B. Pierron [:nbp]]

Would it be possible to use this patch for finding test case which are failing with either:

• Stencil delazification,
• Concurrent off-thread delazification,
• Sweeping of delazification tasks.

The patch which is attached to this bug stress tests all these parts by creating delazification tasks for all non-module compiled code.

Comment 3

[Nicolas B. Pierron [:nbp]]

Attachment: Bug 1756003 part 1 - ScriptSource carry the delazification mode.

Previously, we only had a single mode of delazification, then we added the
concurrent delazification, which could be mixed with on-demand delazification,
by always checking for cached entries.

Now, with the addition of a third mode which attempt to compare the outputs of
on-demand delazification with concurrent delazification, we need to distinguish
and carry this difference to the function responsible for running the
delazification. However, this information is only available when we first run
the delazification.

This patch adds the JS::DelazificationOption (uint8_t) to the ScriptSource
class, such that we can communicate the selected mode to the functions in charge
of the delazification.

Comment 4

[Nicolas B. Pierron [:nbp]]

Attachment: Bug 1756003 part 2 - Compare concurrent and on-demand delazification.

Adds a delazification mode, which spawn a concurrent depth-first delazification
of all function to fill the stencil cache, to later compare cached results with
stencils produced by on-demand delazification.

The delazification mode also works when being used from a main-thread top-level
compilation, in which case the delazification tasks are created and potentially
blocking the main thread, while the delazification is happening.

This way of doing help comparing on-demand delazification, based on instantiated
stencils with concurrent delazification, based on stencils. Which is a good
mode for fuzzing purposes.

Comment 5

[Nicolas B. Pierron [:nbp]]

Attachment: Bug 1756003 part 3 - Add JS shell command line option to set default delazification.

This instrumentation is used to build test cases and help with fuzzing, by
selecting a custom delazification type as the default mode for the top-level.

Comment 6

[Nicolas B. Pierron [:nbp]]

Running the test suite with the instrumentation yielded multiple bugs which are now fixed, except that the tests are blocked on this instrumentation for landing.

Thus, I will open this bug, as the auditing part sound complete, and that this instrumentation is used to check a feature which is not yet enabled.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5b982beda860
part 1 - ScriptSource carry the delazification mode. r=arai
https://hg.mozilla.org/integration/autoland/rev/2dc65ba6edce
part 2 - Compare concurrent and on-demand delazification. r=arai
https://hg.mozilla.org/integration/autoland/rev/399a9a92ff78
part 3 - Add JS shell command line option to set default delazification. r=arai

Comment 8

[bszekely]

Backed out for causing build bustages on ScriptLoader.cpp.

Failure line: /builds/worker/checkouts/gecko/dom/script/ScriptLoader.cpp:1691:14: error: enumeration value 'CheckConcurrentWithOnDemand' not handled in switch [-Werror=switch]

Push with failures
Failure log 1
Failure log 2
Backout link

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3b6f96fb26fa
part 1 - ScriptSource carry the delazification mode. r=arai
https://hg.mozilla.org/integration/autoland/rev/672e8694ef7d
part 2 - Compare concurrent and on-demand delazification. r=arai
https://hg.mozilla.org/integration/autoland/rev/cf1af3ce0798
part 3 - Add JS shell command line option to set default delazification. r=arai

Comment 10

[Iulian Moraru]

Backed out for causing multiple thread sanitizer failures.

Push with failures

Failure log for when it fails on Span.h in in operator[]
Failure log for when it fails on stl_tree.h in _S_right
Failure log for when it fails on tsan_rtl_access.cpp in MemoryAccess

Backout link

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d108577eb21f
part 1 - ScriptSource carry the delazification mode. r=arai
https://hg.mozilla.org/integration/autoland/rev/fbb54a8ea4e4
part 2 - Compare concurrent and on-demand delazification. r=arai
https://hg.mozilla.org/integration/autoland/rev/b3459890368e
part 3 - Add JS shell command line option to set default delazification. r=arai

Comment 12

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/d108577eb21f
https://hg.mozilla.org/mozilla-central/rev/fbb54a8ea4e4
https://hg.mozilla.org/mozilla-central/rev/b3459890368e