1656588 - Create a disnative command for the JS Shell that provides the disassembly of a jitted function

Status: RESOLVED FIXED

(Core :: JavaScript Engine: JIT, enhancement, P1)

Description

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

This would be useful to observe the behavior of the JS Engine without needing to dig into its internals.

Comment 1

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Attachment: Bug 1656588 - Create a disnative command to print the assembly of the JITed function r?lth

Comment 2

[Lars T Hansen [:lth]]

Don't we already have this? We have a good, built-in disassembler on x86 (zydis) and reasonable disassemblers on ARM and ARM64; it is not necessary to import yet another one. See WasmDisassemble in TestingFunctions.cpp.

Comment 3

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

(In reply to Lars T Hansen [:lth] from comment #2)

Don't we already have this? We have a good, built-in disassembler on x86 (zydis) and reasonable disassemblers on ARM and ARM64; it is not necessary to import yet another one. See WasmDisassemble in TestingFunctions.cpp.

Yes - I intend for this patch to use Zydis and expose it to the shell for all JS functions; not just WASM. It's just... not working. I'm using capstone as a comparison to try to debug.

Comment 4

[Lars T Hansen [:lth]]

Ok :-) Good luck, and lmk if I can do anything, I'm sort of the keeper of the disassembler.

Comment 5

[Nicolas B. Pierron [:nbp]]

One of the reason why I think a disnative function might not be the answer that you are looking for is that JIT-code is transient, it is allocated and removed frequently. You want to store content which would persist over time.

I will suggest to revive the perf instrumentation that we had, as it dumps the generated code, which is later displayed when looking at functions within perf report. However this would not work for emulated code, unless you are going to use objdump on these (in which case you will have to compile it with all the architecture you are interested in).

Comment 6

[Nicolas B. Pierron [:nbp]]

(In reply to Nicolas B. Pierron [:nbp] from comment #5)

I will suggest to revive the perf instrumentation […]

Or simply check if it does not already work …
https://searchfox.org/mozilla-central/source/js/src/builtin/Profilers.cpp#439-458

Comment 7

[Lars T Hansen [:lth]]

Attachment: disasmNative.patch

Comment 8

[Lars T Hansen [:lth]]

Attachment: dis.js

Comment 9

[Lars T Hansen [:lth]]

Attachment: output.txt

Comment 10

[Lars T Hansen [:lth]]

Applying the patch, recompiling, and running the test case produces the output as shown here.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d393c620ee6d
Create a disnative command to print the assembly of the JITed function r=lth

Comment 12

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset d393c620ee6d (bug 1656588) for src related bustage.

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedTaskRun=JbKaZcOwT12VJTXPlXf0TQ.1&searchStr=windows%2C2012%2Cx64%2Casan%2Cbuild-win64-asan%2Fopt%2Cbo&fromchange=27a0738dc45b778aebed48242234aa30fa8001a3&tochange=a816580ea8a998b84f6dbddfe5683e2748455e5b

Backout link: https://hg.mozilla.org/integration/autoland/rev/593af26c5b7b7b462ed1c6d2612a77f18fe62757

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=315115195&repo=autoland&lineNumber=22369

[task 2020-09-08T18:50:14.244Z] 18:50:14     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/workspace/obj-build/js/src'
[task 2020-09-08T18:50:14.245Z] 18:50:14     INFO -  z:/build/fetches/sccache/sccache.exe z:/build/fetches/clang/bin/clang.exe --driver-mode=cl -Xclang -std=c++17 -FoUnified_cpp_js_src3.obj -c  -guard:cf -U_FORTIFY_SOURCE -DNDEBUG=1 -DTRIMMED=1 -DWASM_SUPPORTS_HUGE_MEMORY -DJS_CACHEIR_SPEW -DJS_STRUCTURED_SPEW -DJS_HAS_CTYPES -DFFI_BUILDING -DEXPORT_JS_API -DMOZ_HAS_MOZGLUE -Iz:/build/build/src/js/src -Iz:/build/workspace/obj-build/js/src -Iz:/build/workspace/obj-build/js/src/ctypes/libffi/include -Iz:/build/build/src/js/src/ctypes/libffi/src/x86 -Iz:/build/workspace/obj-build/dist/include -Iz:/build/workspace/obj-build/dist/include/nspr -MD -FI z:/build/workspace/obj-build/js/src/js-confdefs.h -DMOZILLA_CLIENT -Qunused-arguments -Qunused-arguments -fsanitize=address -fsanitize-blacklist=z:/build/build/src/build/sanitizers/asan_blacklist_win.txt -fcrash-diagnostics-dir=z:/build/public/build -fcrash-diagnostics-dir=/z/build/public/build -fcrash-diagnostics-dir=/z/build/public/build -TP -D_CRT_SECURE_NO_WARNINGS -Zc:sizedDealloc- -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -Wno-inline-new-delete -Wno-invalid-offsetof -Wno-microsoft-enum-value -Wno-microsoft-include -Wno-unknown-pragmas -Wno-ignored-pragmas -Wno-deprecated-declarations -Wno-invalid-noreturn -Wno-inconsistent-missing-override -Wno-implicit-exception-spec-mismatch -Wno-microsoft-exception-spec -Wno-unused-local-typedef -Wno-ignored-attributes -Wno-used-but-marked-unused -GR- -Z7 -Xclang -load -Xclang z:/build/workspace/obj-build/build/clang-plugin/clang-plugin.dll -Xclang -add-plugin -Xclang moz-check -O2 -gline-tables-only -Oy- -Werror -Xclang -fexperimental-new-pass-manager  -Xclang -MP -Xclang -dependency-file -Xclang .deps/Unified_cpp_js_src3.obj.pp -Xclang -MT -Xclang Unified_cpp_js_src3.obj   Unified_cpp_js_src3.cpp
[task 2020-09-08T18:50:14.245Z] 18:50:14     INFO -  In file included from Unified_cpp_js_src3.cpp:2:
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -  z:/build/build/src/js/src/builtin/TestingFunctions.cpp(1182,37): error: cast to smaller integer type 'unsigned long' from 'unsigned char *' [-Werror,-Wpointer-to-int-cast]
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -      unsigned long expected_length = reinterpret_cast<unsigned long>(jit_end) -
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -  z:/build/build/src/js/src/builtin/TestingFunctions.cpp(1183,37): error: cast to smaller integer type 'unsigned long' from 'unsigned char *' [-Werror,-Wpointer-to-int-cast]
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -                                      reinterpret_cast<unsigned long>(jit_begin);
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -  z:/build/build/src/js/src/builtin/TestingFunctions.cpp(1176,15): warning: Usage of ASCII file functions (here fopen) is forbidden on Windows.
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -      FILE* f = fopen(fileName, "w");
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -                ^
[task 2020-09-08T18:50:14.246Z] 18:50:14     INFO -  z:/build/build/src/js/src/builtin/TestingFunctions.cpp(1176,15): note: On Windows executed functions: fopen, fopen_s, open, _open, _sopen, _sopen_s, OpenFile, CreateFileA should never be used due to lossy conversion from UTF8 to ANSI.
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  z:/build/build/src/js/src/builtin/TestingFunctions.cpp(2936,18): warning: Usage of ASCII file functions (here fopen) is forbidden on Windows.
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -        dumpFile = fopen(fileNameBytes.get(), "w");
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -                   ^
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  z:/build/build/src/js/src/builtin/TestingFunctions.cpp(2936,18): note: On Windows executed functions: fopen, fopen_s, open, _open, _sopen, _sopen_s, OpenFile, CreateFileA should never be used due to lossy conversion from UTF8 to ANSI.
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  2 warnings and 2 errors generated.
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  z:/build/build/src/config/rules.mk:723: recipe for target 'Unified_cpp_js_src3.obj' failed
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  mozmake.EXE[4]: *** [Unified_cpp_js_src3.obj] Error 1
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/workspace/obj-build/js/src'
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  z:/build/build/src/config/recurse.mk:72: recipe for target 'js/src/target-objects' failed
[task 2020-09-08T18:50:14.247Z] 18:50:14     INFO -  mozmake.EXE[3]: *** [js/src/target-objects] Error 2
[task 2020-09-08T18:50:14.248Z] 18:50:14     INFO -  mozmake.EXE[3]: *** Waiting for unfinished jobs....

Comment 13

[Bogdan Tara[:bogdan_tara | bogdant]]

Also seeing spidermonkey failures starting with the backed out changes:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&searchStr=spider&fromchange=27a0738dc45b778aebed48242234aa30fa8001a3&tochange=a816580ea8a998b84f6dbddfe5683e2748455e5b&selectedTaskRun=QIISkZMuReaSjyquDXB4Ow.1

Also seeing jit failures starting with the backed out changes:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&searchStr=jit&fromchange=27a0738dc45b778aebed48242234aa30fa8001a3&tochange=a816580ea8a998b84f6dbddfe5683e2748455e5b&selectedTaskRun=SwLknwJvTK2w_DNetEMSlw.0

Comment 14

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Okay, these should be fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c42777b04daf1f113bac2aaa7a964c77f01ea6d3

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a267f4e6ba1d
Create a disnative command to print the assembly of the JITed function r=lth

Comment 16

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/a267f4e6ba1d