1245279 - [meta] Various tests in Speedometer seem to spend quite a bit time in jS

Status: NEW

(Core :: JavaScript Engine, defect)

Description

[Olli Pettay [:smaug][[e-mail]]]

While looking at some benchmarks I'm evaluating Speedometer again.
(not sure it is a very good benchmark, seems to focus on one particular case and testing how different frameworks deal with that, but we're quite slow with it anyhow).

http://mozilla.pettay.fi/moztests/Speedometer.tar.bz2
is from https://github.com/WebKit/webkit/tree/master/PerformanceTests/Speedometer
but I modified the InteractiveRunner.html to make profiling a bit easier.
So, using InteractiveRunner.html, check EmberJS-TodoMVC and click run.
In Zoom I see most of the time spent in JS, and that is also that Gecko profiler tells.

Also BackboneJS-TodoMVC seems to spend most of the time in JS, and AngularJS-TodoMVC, and
AngularJS-TodoMVC. And React-TodoMVC is based on Gecko profiler also largely JS. I see quite a bit IonBuilder calls there.

Could someone more familiar with JS engine and JIT look at the test.

Comment 1

[Olli Pettay [:smaug][[e-mail]]]

s/and AngularJS-TodoMVC, and AngularJS-TodoMVC/and AngularJS-TodoMVC/

(jQuery test has at least some parsing stuff showing up in the profile, and VanillaJS-TodoMVC, and also FlightJS-TodoMVC, although it is mostly JS, )

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

Could someone from JS team look at this.
Recently I've seen JS to take lots of time in benchmarks which try to be realistic, like Speedometer.
(Note, the test just tries that, but totally is not realistic.)

Comment 3

[Jan de Mooij [:jandem]]

In profiles of these benchmarks I see the usual suspects: CrossCompartmentWrapper stuff, IC misses, and some non-trivial mprotect overhead when attaching Ion IC stubs.

I've some serious IC redesign work planned for the coming weeks/months (starting with bug 1255352), hopefully that will make our ICs more robust and allow us to optimize more cases.

Added this to my perf TODO list, clearing the needinfo for now.

Comment 4

[Olli Pettay [:smaug][[e-mail]]]

jandem, any news on this?

Comment 5

[Jan de Mooij [:jandem]]

(In reply to Olli Pettay [:smaug] from comment #4)
> jandem, any news on this?

Not yet. Right now I'm working on modernizing our exception handling and the JSContext/JSRuntime unification. I hope to continue working on CacheIR and other IC work soon though.

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

(Just CCing some folks from bug 1241091, there might be something in common here, given that Speedometer tests various JS frameworks)

Comment 7

[David Ross]

Using Nightly 53 (20161217) on desktop this site stays in a loop. In Android Nightly it stalls and eventually pulls up "Warning: Unresponsive script.. Script: https://browserbench.org/Speedo...ower_components/react/react.js:7174"

Comment 8

[Tom S [:evilpie]]

At least EmberJS seems to spend some time in the interpreter and fun_apply.

Comment 9

[David Ross]

Been regularly testing speed on 2008 model laptop running Arch Linux. 
  * January on Browserbench 'Speedometer' I was maxing out at 10 runs per minute.
  * Test [on Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0 ID:20170515100238 CSet: e66dedabe582ba7b394aee4f89ed70fe389b3c46] is now running at 23.1 runs per minute.

131% speed improvement on the January test.

Keep up the amazing work folks!

Comment 10

[David Ross]

Further update since Fx55:

Same machine - 2008 model laptop running Arch Linux. 
  * January on Browserbench 'Speedometer' I was maxing out at 10 runs per minute.
  * Test 2 months ago [on Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0 ID:20170515100238 CSet: e66dedabe582ba7b394aee4f89ed70fe389b3c46] is now running at 23.1 runs per minute.
  * Test [on Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0 ID:20170720100139 CSet: 0985725c848ec0cfc6f2f3c3a5aa3d71321e7620] is running at 27.9 runs per minute.

Firefox Nightly is 279% faster on my machine than it was in January of this year.

Comment 11

[Gary [:streetwolf52]]

Attachment: Very low score on benchmark - 34.

I don't understand why I'm getting a 34 on the benchmark.  My specs are high as seen below:

i7 Quad 3770K @ 5Ghz 
ASUS P8Z77-V Deluxe 
Corsair 1050W PSU 
Corsair H100iV2 CPU Cooler 
Corsair 16GB RAM 
Sapphire Nitro R9 390 8GB 
DUAL ASUS PA249Q IPS 24" LCDs 
Samsung SSD 830, 840 256GB, 2TB Seagate 
Windows 10 Pro x64 (v1703) 
AMD Crimson 17.7.2 
FIOS 1Gb/1Gb

Comment 12

[Gary [:streetwolf52]]

After enabling my add-ons one by one and running the benchmark I came up with these number. No add-ons enabled resulted in a score of 97.1.

1. AutoplayStopper 0.9.8. This is the Webextensions version of Legacy FlashStopper. Lost ~ 12%

2. Zoom Page WE 5.1. Lost ~ 50%

3. UBlock Origin/webext 1.13.9b3. Lost ~ 28%. If I disable uBO only for the benchmark site I don't lose any points, I get a 97.1.

I've contacted the developers of these three add-ons letting them know of these results.

Comment 13

[(no longer active)]

(In reply to Gary [:streetwolf] from comment #12)
> 3. UBlock Origin/webext 1.13.9b3. Lost ~ 28%. If I disable uBO only for the
> benchmark site I don't lose any points, I get a 97.1.

See bug 1339572 for example.

Running benchmarks with extensions installed is usually not super useful unless if you're trying to study the details of the performance impacts of those extensions.  We have some efforts ongoing about the improving the performance impact that having extensions like this installed can have over what happens inside web pages (which is part of the overhead you are seeing here) for example under bug 613498 but it's a lot of work and we need still more focus there.