1494674 - [BinAST] Instrument with Telemetry

Status: RESOLVED INVALID

(Core :: JavaScript Engine, enhancement, P3)

Description

[David Teller [:Yoric] - still alive but not very active]

We want to instrument BinAST with Telemetry for our first release.

We haven't discussed exactly which probes we want, so opening this bug to both come up with the list of probes and actually track implementation.

Comment 1

[David Teller [:Yoric] - still alive but not very active]

Note: when npb returns from PTO, he may have some ideas, after his work on the Bytecode Cache.

Comment 2

[Nicolas B. Pierron [:nbp | PTO until 11/11]]

Most of the instrumentation I added was to later tune the threshold for encoding.

Part of the instrumentation I added at the time was in the ScriptLoader::EvaluateScript function, now removed.  This instrumentation was in charge of measuring the (parse + first-run) time. This was good enough for the JSBC, as I could expect wide ranges of requests to be evenly distributed against the normal parser and the JSBC.

For BinAST, I would suggest to measure the throughput of both compilers, in terms of bytes per milli-seconds by adding code either to the ScriptLoader::EvaluateScript function. Or potentially to the JS engine it-self, in which case I have not yet practice that my-self.

Comment 3

[David Teller [:Yoric] - still alive but not very active]

Ok, that sounds mostly simple enough. We can measure the throughput in ms/byte for both text source and binary source, probably discarding files that are too small (<10kb?). This has the drawback that it will accidentally make worse compression look better, though.

Perhaps we could somehow use the number of nodes in the AST as a measure? It's a little bit longer to compute, but it shouldn't be too hard to do.

Comment 4

[David Teller [:Yoric] - still alive but not very active]

Ah, we should be able to quickly compute the delta between `lifoAlloc.used()` before and after parsing. This should be a reasonable measure of size that works across formats. Eric, Arai, do you see any obvious problem with this approach?

Comment 5

[Tooru Fujisawa [:arai]]

(assuming you're going to measure "ms / number of nodes")

there are some case that parsers (both normal one and binast) allocate temporary nodes and throw them away instantly without putting it into the final tree.
iiuc, they're also going to be counted if you use lifoAlloc.used(), so we might need to think about those cases (like, how frequent it is and how much it affects, and how different the effect is between those 2 parsers).
I'll check the actual cases shortly.

also, we might need to think about how lazy parsing affects it.

Comment 6

[Eric Faust [:efaust]]

the good thing about ms/# nodes is that the skipping isn't a huge deal. We handle fewer nodes, and we're faster.

We should also make sure we track the metadata object size in this, since it's extra memory overhead that the standard parser doesn't have.

Comment 7

[David Teller [:Yoric] - still alive but not very active]

Arai was suggesting that using bytecode length (of the toplevel?) as a measure of size might be less sensitive to implementation details than other stuff.

Comment 8

[David Teller [:Yoric] - still alive but not very active]

For the moment, the bytecode length is the safest option, and we should be able to get started with that.

Currently discussing alternatives with chutten.

Comment 9

[Tooru Fujisawa [:arai]]

checked relation between number of parse nodes and LifoAlloc.used(),

looks like there's not so much case that ParseNode is thrown away in ES5 case.
(it happens in arrow function).
tested with js/src/jsapi-tests/binast/parser/multipart/spidermonkey/ecma_2/String/split-002.js with modification to make all functions IIFE, and encoded without --lazify.
there's only 1% difference (regular: 63104 vs binast: 64224).

so at least if there's no lazy parsing, LifoAlloc.used() (which reflects eager functions) could be used as well.

Comment 10

[David Teller [:Yoric] - still alive but not very active]

A few notes, as I explore the code.

* Finding out that a document uses BinAST.

We can get the information from WebExtension land through https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onCompleted#details, by looking at the http headers.

* Time To Interactive

We can access Time To Interactive from WebExtension land through https://developer.mozilla.org/en-US/docs/Web/API/PerformanceNavigationTiming .


So, it looks like we could do all of the work in WebExtension land.