849087 - Source compression degrades js shell performance

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Katelyn Gadd (:kael)]

(Didn't see a component/product for js shell so I picked JS engine)

I use the compiled js shell in my automated test suite to run ~600 individual JS tests. Each JS test loads all my runtime libraries then runs a simple test case that consumes the libraries.

The previous build of the js shell that I was using dates back to April of 2012, so it was missing out on a lot of SpiderMonkey improvements since then; this meant that for performance-sensitive tests they were completing a lot slower than they do in the browser.

Unfortunately, when I upgraded to a trunk build of the js shell, I discovered that every single test run had gotten dramatically slower.

Some investigation showed that while the actual test cases are generally faster (sometimes much faster), the cost of starting up the js shell and loading my runtime libraries has increased by something like 10x. 

I frontload some of the startup work by spawning js.exe and then having it load the libraries well in advance of running the test case, in order to minimize the amount of time I need to wait. With the old build of js.exe from last year, this meant that simple test cases would complete end-to-end in 20-30 milliseconds on Win32.

Now, with a modern trunk build, the same test case takes 200-300 milliseconds, a 10 performance hit.

I instrumented the test runner a little bit and plugged in an empty test case so that I could measure the cost of loading my libraries and then running the test fixture (sans test), and the two of those stages have indeed regressed considerably in performance:

Old SpiderMonkey
Loading JSIL.js took 50.135986328125 ms
shellStartup -> runMain took 12.02392578125 ms
New SpiderMonkey
Loading JSIL.js took 167.669189453125 ms
shellStartup -> runMain took 42.324951171875 ms

I've tried disabling Ion, JM and TI in the new shell to see if it would improve these timings, but it made no significant difference. Using 'time' to measure the overall cost to start js.exe and then let it exit shows that it's still pretty quick, on the order of 30ms, so actual startup doesn't seem to have degraded.

Given that this is a js shell problem, I don't know if it actually matters in the real world unless it affects in-browser performance. But it seems like it is potentially indicative of some deeper problem for something to take so much longer compared to how it used to perform.

Comment 1

[Katelyn Gadd (:kael)]

Attachment: Simple startup time test with old/new spidermonkey builds

Comment 2

[Katelyn Gadd (:kael)]

benjamin and mjrosenb came up with a good theory for this and helped me test it:

I was building the full browser (in release mode) using a normal release build mozconfig, and then grabbing the js.exe from that build folder. Apparently this produces a js.exe with source compression turned on, and for some reason js.exe is synchronously compressing loaded javascript when in this mode, resulting in the increased startup time.

If I instead build js.exe by itself following the steps at https://developer.mozilla.org/en-US/docs/SpiderMonkey/Build_Documentation , I get a js.exe that is only ever so slightly (like 5ms) slower than the really old spidermonkey build I had lying around. This gets my automated test suite's performance back into the acceptable (~30s for a group of 190 tests) instead of the 5+ minutes it was with the browser build.

So, this probably isn't necessarily a regression, but it might be worth changing the js shell to make it obvious whether or not source compression is active, since it's synchronous.

Comment 3

[Matthew Gaudet (he/him) [:mgaudet]]

We do off-thread script compression these days.