1528861 - Parser worker consumes a lot of resources on pages with large generated scripts

Status: NEW

(DevTools :: Debugger, defect, P3)

Description

[Brian Hackett [Laid off!]]

While looking at performance characteristics on a large, artificial webpack app, I noticed that the parser worker seems to be able to hold onto a lot of resources. I mainly noticed this through the effect on GC times. On this app --- 1000 modules, 14 MB uncompressed generated JS file (about the same size as the app in bug 1521539, and 3x the size of debugger.html's generated JS file) --- parser worker GCs take 20 seconds on my machine. These GCs happen frequently and aren't collecting anything, so the worker ends up not spending much time doing work and it takes a long time for frame scopes to be mapped.

It would be nice if the parser worker didn't hold onto so much stuff. There are a couple options I see:

• Get the necessary information from spidermonkey, which has already parsed the generated source (it is plain JS after all). This would be the most efficient but is probably not easy to do with the current architecture.

• Don't parse the entire generated file at once. JS files generally don't have much logic at the top level: most of it is in inner functions. Webpack-generated files follow this pattern, with functions at the top level for each module in the project. Spidermonkey takes advantage of this to lazily parse source files. Parse trees for a function are not generated until the first time it is called. The parser worker could use a similar strategy, where it parses any code at the top level but skips over all top level functions except for the one containing the point it is interested in parsing. Getting the boundaries of top level functions from spidermonkey would make it possible to skip over these functions without having to look at the source text at all.

Comment 1

[Brian Hackett [Laid off!]]

Attachment: generate.js

This JS file can be used to populate a directory with auto-generated modules for feeding into webpack. The settings used here generate a 14MB unminified generated file.

Comment 2

[KOLANICH]

Get the necessary information from spidermonkey

This is definitely possible. If the stuff uses ESPrima, it takes no effort to replace it with FF native parser.