1304984 - (ares-6) Optimize WebKit's ARES-6 benchmark

Status: NEW

(Core :: JavaScript Engine, defect, P3)

Description

[Tetsuharu OHZEKI [:tetsuharu] (UTC+9)]

The result of /PerformanceTests/ES6SampleBench/index.html in WebKit's tree is very slower (~5x) than the result of WebKit.

## Benchmark codes

- https://github.com/WebKit/webkit/tree/d2679d89a5417abd1f4b89501f88dba11bf08d25/PerformanceTests/ES6SampleBench

## Measurement envionment

  - MacBook Pro (Retina, 15-inch, Mid 2014), 2.8 GHz Intel Core i7, 16 GB 1600 MHz DDR3
  - OSX 10.11.6
  - x64

## Results

### Firefox Nightly 52.0a1, 20160920030429


Geometric Mean Result: 579.33 ms +- 10.15 ms

| Benchmark |First Iteration       | Worst 2%              | Steady State             |
|-----------|----------------------|---------------------- |--------------------------|
| Air       | 195.29 ms +- 1.93 ms | 123.89 ms +- 14.82 ms | 11416.11 ms +- 237.49 ms |
| Basic     | 105.67 ms +- 3.53 ms | 98.28 ms +- 9.08 ms   | 13452.75 ms +- 520.96 ms |




### Chrome Canary 55.0.2866.0 canary (64-bit)


Geometric Mean Result: 398.87 ms +- 9.03 ms

| Benchmark |First Iteration     | Worst 2%          | Steady State           |
|-----------|--------------------|-------------------|------------------------|
|Air        |152.76 ms +- 5.49 ms|59.35 ms +- 3.36 ms|6956.44 ms +- 33.39 ms  |
|Basic      |78.34 ms +- 1.95 ms |71.23 ms +- 5.66 ms|11525.40 ms +- 169.84 ms|


### WebKit Nightly r206163


Geometric Mean Result: 113.36 ms +- 3.36 ms

| Benchmark |First Iteration    | Worst 2%          | Steady State         |
|-----------|-------------------|-------------------|----------------------|
|Air        |53.88 ms +- 6.67 ms|28.98 ms +- 2.53 ms|2364.88 ms +- 61.77 ms|
|Basic      |26.78 ms +- 0.85 ms|14.56 ms +- 0.79 ms|1500.21 ms +- 21.12 ms|

Comment 1

[Tom S [:evilpie]]

Attachment: ES6SampleBench.zip

Thanks for pointing this out to us, I am always interested in ES6 benchmarks

I tried modifying cli.js to work with our shell, but I think using newGlobal() + evaluate() probably causes us to unnecessarily recompile stuff, which would skew our results.
Tracelogger suggest most time is spend in Baseline, but again that might be because of evaluate.
IONFLAGS=aborts shows various aborts for spreadcallarray, pushlexicalenv and generators.

Comment 2

[Tom S [:evilpie]]

I think the newest version of this https://github.com/WebKit/webkit/tree/master/PerformanceTests/ARES-6. Almost looks like something they want to release publicly.

Comment 3

[Jan de Mooij [:jandem]]

Clearing NI. Tom and Ted have been working on this.

Comment 4

[Julien Cristau [:jcristau]]

Too late for firefox 52, mass-wontfix.

Comment 5

[Alexandre Folle de Menezes]

This is now a public benchmark: http://browserbench.org/ARES-6/

And there's a blog post where Firefox compares very unfavorably against Chrome and Safari:
https://webkit.org/blog/7536/jsc-loves-es6/